Where chatbot hallucinations come from
A language model writes fluently whether or not it has the facts at hand. Without data from your shop it can put together an answer that sounds sure of itself, because fluency and truth are two separate things for it, and the customer sees only the first. That is why a shop bot should search first and write second when it is asked about your offer.
In CartBot it works like this. When asked about a product, the bot searches the connected catalog, and when asked about delivery or returns it goes to the shop's knowledge base, where your rules are kept. The model builds its answer from what it found in the data, and its instructions forbid inventing prices and features that are not there. When a parameter is missing, it is supposed to say plainly that it does not have that information.
The risk falls, but it does not disappear. It is enough for search to return a backpack from the same series with waterproof zips and for the bot to take a sentence from its description. The customer then sees the card of the backpack they asked about, with the price correct to the grosz, and under it a promise of rain protection that this backpack does not give. Variants can go the same way when one colour has a different description from the rest.
Is a chatbot safe for your shop?
If you are asking, you are probably worried above all that the bot will promise customers things you do not have on offer. The honest answer is “as far as its data and your tests allow.” Every link can fail, from your description through search to the model itself, so a promise of zero mistakes should worry you rather than reassure you.
In practice a safe bot has somewhere to go when it does not know. It admits it lacks the information and asks for contact details. If someone is at the computer, they take over the conversation in the panel together with its history. But in a one-person shop at 11:10 pm nobody is likely to be there, and an email or phone number left behind is all you get.
When you talk to a vendor, ask for a live demo. Ask a question from outside the catalog and watch what the bot does, then ask where it got its information about the backpack's water resistance, because a general description of the category “city backpacks” does not confirm that this particular model will survive rain.
Prices and delivery, the data that goes stale
The bot knows your offer from the API of your WooCommerce, Shopify, PrestaShop or Shoper store (Shoper is a Polish shop platform), from an XML or CSV feed at a URL, or from a file uploaded by hand. Sync runs on a schedule, so the bot sees changes in the shop with a delay, and if a data pull fails, the delay grows.
A file uploaded by hand will not refresh itself. It stays as it was until somebody uploads a new one. A withdrawn product does not vanish at once either, because sync waits before it deletes an item that has gone missing from the source.
Delivery is the easiest thing to overlook. The bot takes the free shipping threshold and the return rules from the knowledge base, and catalog sync does not have to touch that base. Say you raise the threshold from PLN 199 to PLN 249 and the old amount stays in the knowledge base, so the bot keeps promising free delivery to a customer with PLN 220 in the basket. Then you either pay the shipping difference or explain yourself in an email.
Five tests before launch
Take a product you know by heart and start every test in a new conversation, because earlier messages can hand the bot an answer.
The most revealing test is the one with a missing parameter. A good answer goes roughly like this: “The description of this backpack has no information on water resistance. I can pass the question to support if you leave your contact details.” Then push the bot like a customer who badly wants to hear “yes”, and see whether it holds its position, because the fabric alone says nothing about the seams or the zips.
While you are at it, look in the panel and check that the contact from the test was saved. Passing tests is a good sign, though you cannot squeeze a guarantee for every future conversation out of them. The whole five is below.
- Change the price of one product and note how long it takes for the new amount to show up in the chat.
- Ask about a withdrawn product and about a variant you never had, then try to add both to the basket.
- Check the delivery cost for a basket just under the free shipping threshold.
- Ask about a parameter that is missing from the description.
- After a refusal write “but surely you can judge it from the material” and see whether the bot sticks to its answer.
What to do when the bot makes something up anyway
After launch, read the conversations regularly, starting with those where the bot gave parameters or delivery costs, because a mistake there costs the most.
When you find an error, first work out where it came from. That decides who fixes it. If the card says “waterproof” and the backpack leaks, the bot simply repeated your description and the fix is yours. It is worse when the data is fine and the bot did not find it or used it wrongly, because then the search or the answering rules need correcting, and you cannot change those from the shop panel.
CartBot is a managed service, so we take that second part on ourselves as part of the subscription. The facts about your offer stay on your side, because whether a backpack survives a downpour is something you or the manufacturer know.
After the fix, ask the bot the same question again and add it to the list of cases you run through the bot after every bigger change in the catalog. You can enter the first items on it today, by running the five tests described above.