Case study
Why your AI assistant invents an upload button that does not exist
A client asked our own chat assistant how to send us two page designs. It sent them to an upload button and an email address that did not exist. We changed the portal, the team's view and the assistant's instructions, and added a check before replies leave.
Key takeaways
- An assistant with no fact for a task will describe a plausible process, so the real one has to be written down for it.
- Every address or number an assistant may give is worth testing from the customer's side.
- Files should arrive attached to the work they belong to, not through a chat or as a link to someone's computer.
- A reply check catches the invented contact detail that instructions alone let through.
- A failed conversation is a map of the gaps: read it from the first question to the last bounce.
When an AI assistant making up answers sounds confident, a customer follows the answer and gets nowhere. One of our clients asked our own chat assistant how to send us two page designs. It sent them to an upload button in our client portal, then to a support email address. Neither existed.
A confident answer that sends the customer nowhere
The first answer pointed to a button that was not there, and the second pointed to an address that did not work. The client could not find the button, so the assistant gave a support email address and said to send the files there. It also promised to check that the email had arrived. The address did not exist either; the client's email came back undelivered.
In the end the work was logged with links to the files. The links pointed to the client's own computer, so nobody on our side could open them. The person doing the work could not get the designs, and the job waited. For any business, the second cost is worse than the first: a customer who is sent nowhere decides the service does not know its own tools.
Why an AI assistant making up answers sounds so sure
An assistant asked something its instructions do not cover will usually describe what a service like yours ought to have, not say it does not know. Files are the start of much of the work a customer asks for: a design, a document, a screenshot of what is wrong. If the assistant has no fact about how files reach you, it reaches for the usual picture: a button on a portal and an address for support.
A promise to check something the assistant cannot see is a promise nobody keeps, and the customer waits for an answer that never comes. Building a file upload into the app but not into the website leaves a customer on a computer with no way in. The assistant then fills that gap with a guess, because nothing in its instructions gives it the real route.
What our engineer found
Reading the chat from the first question to the bounced email showed two invented answers in a row, and no real way to send a file through the portal. Our two mobile apps could already attach photos to a request, the job we log for the work, and the files were kept with it. The web portal could attach nothing, and it did not even show files that had already been kept for a request. The screen our team works from showed only a file's name, not a way to open it.
The assistant's instructions said nothing about files. Asked a question it had no facts for, it described what a portal like ours would usually have: an upload button and a support mailbox. Both sounded right; neither was real. The links to the client's own computer were the last piece. They looked like links, but they open only on the machine that made them.
What we changed
We gave files one real way in, told the assistant that way in plain words, and put a check on its replies. In the portal, every request now has a Your files panel. A client can attach several files at once, on a new request or an existing one, and sees each one listed with a small picture and a link to open it. The panel takes pictures, PDFs, design files, Word, Excel and PowerPoint documents, text, spreadsheet exports and zip files. A file larger than 12 MB is named and turned down before anything is sent, and a file that fails never cancels the request; the client is told which one to attach again. When a request description holds a link to a file on the client's own computer, the portal says so and asks for the file instead.
On our side, the team opens what the client sent straight from the request, rather than seeing only its name. The assistant's instructions now say how a file reaches the work: on its request, in the portal or as photos in the app, and never through the chat, messaging apps or email. They also say never to give an address, number or account of ours that did not come from a tool, and never to promise to check something it cannot see.
Instructions alone are not a promise, so there is also a safety net. Before a reply leaves, any email address that looks like one of ours but is not on our real list is replaced with our real office address. A client's own address, or anybody else's, is left as it is. The new portal words are in all six languages.
Where it stands now
The wrong answers went out over two days; the fix in the portal, the team's screen and the assistant went live on the third. A client can now attach files to any request from the portal, the apps keep sending photos as before, and the team opens every file from the request. The assistant now points to the real way. An invented address of ours is replaced before a reply leaves, so it cannot reach a customer even if the assistant writes one.
What you can check yourself
- Ask your own AI assistant how a customer sends you a file, and follow its answer step by step. If a step ends at a button or page you cannot find, you have found the gap.
- Ask it for your support email and phone number, then check each one by sending a message or making a call. An address that bounces or a number that rings nowhere needs fixing.
- Send a test file the way a customer would, then ask the person who does the work to open it. If they cannot see it on their screen, it did not arrive.
- Look through recent chats for promises such as “I will check” or “I will confirm”. Ask who keeps them; if nobody can, the assistant should not make them.
- If a customer sends a link to a file on their own computer, open it on a different device. If it does not open, ask them to attach the file instead.
What to do next
Start with the question a customer asks when they need to send you something. Follow your assistant's answer yourself, fix the facts it is missing, and add a check before its replies leave. This is the kind of work our engineers do: showing an assistant the real way a business works, and checking what it says before a customer sees it. If you cannot test safely, ask whoever looks after your website or chat to do it with you.
Questions people ask
Why does my AI assistant give answers that sound right but are wrong?
Because it fills gaps in its facts with plausible detail. If it has not been told how a customer sends you a file, it describes a button or address that a business like yours might have. The answer is fluent because the assistant is good at language, not because it has checked your tools. The fix is to give it the real steps and test them.
How do I stop a chatbot from giving out an email address that does not exist?
Write down the addresses and numbers it may give, and tell it never to give one that no tool supplied. Then check replies on the way out: an address at your own web name that is not on your list can be replaced with a real one. Instructions help, but a check before the reply leaves is what catches the rest.
What is the simplest way for customers to send me files?
Give them one place, attached to the job it belongs to, rather than a choice of chat, messaging app and email. They should see the file listed after they attach it, and the person doing the work should be able to open it from there. Photos from an app can use the same job record.
Why do links to files on a customer's computer not open for me?
A link can point to a place on the customer's own machine. It looks like a normal link, but it only works on that computer. Ask the customer to attach the file to the request instead; that also gives the person doing the work a copy they can open.
How often should I read through my assistant's chats?
There is no set interval that fits every business. Read a few conversations after you change your services, your prices or the way customers send you things, and read any conversation where a customer says they cannot find something. Look for promises the assistant made and check that someone keeps them.