Customer journey testing: how AI customers find revenue leaks

Customer journey testing with AI customers finds revenue leaks where a company’s own reports can’t: an AI customer signs up with its own inbox, phone number and browser, waits, opens every email, text and reply, and checks each one against what was promised. The leaks it finds are steps that fail without an error, such as a welcome text that never arrives, a code refused at checkout, a basket reminder hours late, a chatbot quoting the wrong policy or a demo form nobody answers.

Store owners, retention and CRO agencies, RevOps leads and CX teams can use customer journey testing to find the steps that lose money without raising an error.

Those steps fail outside the company’s own reports. A welcome text that never triggered leaves no row in the flow report, a refused code counts as one more abandoned checkout, and a chatbot’s wrong answer can be logged as a resolved chat. A customer who knows what was promised, and waits for it, sees each one.

A test of 362 ecommerce brands by Joy Joya, run with new email addresses and phone numbers, found that 158 sent no welcome email and 53 never sent the text they promised. Rejoiner’s own analysis of 1,000 direct to consumer brands found that 68% of abandoned carts triggered a reminder email, so about 1 in 3 got none.

What revenue leaks are

A revenue leak is a step in a customer journey where the customer was promised something and received it late, broken, wrong or not at all, while the company’s own reports showed the step working. The promise can be an email, a text, a code, a reminder, a reply to a form or an answer from a chatbot.

• Silent: nothing arrived by the deadline.
• Late: it arrived after the delay you set or the time you promised.
• Broken: it arrived with a blank product, a dead link or a code that fails.
• Wrong: it contradicts your own pages, or it came twice.

Revenue leakage also names billing losses, such as usage never invoiced. Those sit in finance systems, where a customer can’t see them.

An AI customer is an AI agent that becomes a real customer of a company, with its own inbox, phone number and browser, and keeps what the company sends and shows it as evidence. Because it is a customer, it receives what a new customer receives, at your company or any other. What AI customers are defines the term in full.

Online stores

In a store, the promises start with the ad and the pop up and end with the shipping email.

Joy Joya’s test also found 20 brands whose welcome offer didn’t match the one on their pop up.

Abandoned cart email not sending

Baymard Institute puts the average documented basket abandonment rate at 70.22%, from 50 studies, so the basket and checkout reminders are meant for most shoppers who add something to a basket.

On Shopify, a checkout counts as abandoned after more than 10 minutes incomplete once the shopper has given an email, and you set the wait under Send after. A second abandoned checkout automation defaults to a 10 hour wait, and the automation emails only customers subscribed to email marketing by default. A test shopper who leaves the marketing box unticked gets nothing, which is the setting working. A reminder planned for 1 hour that comes 10 hours later points to that default.

A check of a UK skincare store begun on 22 September 2026 left a £40 gift set in the basket at 02:57 and a £21 deodorant at checkout at 03:11. Both shoppers had ticked the marketing box, and no reminder arrived for either in 48 hours. The causes to check in Klaviyo and Shopify are in why a welcome or basket email never arrives.

B2B lead forms and speed to lead

A demo or contact form promises a reply, on the form itself or in the sales team’s own target. Testing that promise as a prospect is an established method. Workato filled in demo request forms at 114 B2B companies in a study published in November 2021, RevenueHero requested a demo on 1,000 B2B websites in a study published in March 2024, and agents built in Clay submitted 6,346 demo and contact forms across 100 countries in 2026, each company publishing its own results.

RevenueHero got 365 replies from its 1,000 requests, after 1 day, 5 hours and 17 minutes on average. In Clay’s test, 2 in 3 companies never replied, 32% of forms got an email reply and 96% of companies never called. Clay notes that spam filters may have blocked some submissions.

• The form passes when it submits, confirms, and the lead appears in your CRM with an owner.
• The first reply passes when a person answers on any channel within the time you promise or target. Log automatic replies separately.
• The phone passes when a call or text arrives within the same target, if the form asked for a number.
• The booking link passes when it opens a calendar with free times.

AI agents and AI answers

A support bot, voice agent or AI sales agent makes promises in its answers, and the company is held to them. A British Columbia tribunal held Air Canada responsible for what its website chatbot told a customer about a bereavement refund, as McCarthy Tétrault reports, and ordered it to pay CA$812. The UK’s Competition and Markets Authority says a business is responsible for what its AI agent does as it is for an employee, and could be fined up to 10% of its worldwide turnover.

AI assistants answer for a store as well. Shopify’s agentic storefronts, which list a store’s products in ChatGPT, Gemini, Copilot and other AI channels, are on by default for eligible stores. In a study Product.ai ran and Business Insider reported, 85% of 913 checkable price answers from ChatGPT, Claude, Gemini and Perplexity matched the current or listed price, and the wrong ones were off by a median of $300.

• A support bot passes when its answers on refunds, delivery and prices match your policy pages as they read that day, and it hands over to a person when asked.
• An AI assistant passes when the price, stock and policy it gives for your products match your store’s pages.

How to test your store’s chatbot lists the questions to ask and how to score each answer.

Customer journey testing, step by step

Customer journey testing is going through your own journeys as a new customer and checking each promise against what arrives, and when. An AI customer runs it in 5 steps, and a person can run each one by hand.

1. Write down each promise and its deadline

Take each promise from your own pages and settings: the pop up’s offer, each flow’s delay, the form’s reply time, the returns window on the policy page. Give each a deadline, such as the flow’s delay plus 1 hour, so every result is a pass or a fail against a line written in advance.

2. Sign up as a new customer

Use an inbox, phone number and browser the company has never seen, because your own login carries your history and staff discounts. Tick the marketing box as a shopper would. With double opt in, Mailchimp’s help centre says a subscriber is added only after clicking the link in the confirmation email or text, so open it and click.

3. Wait for each deadline

Waiting is most of the test. Give reminders 48 hours, and start every journey at once, each with its own identity.

4. Open everything and use it

Open each email and text on a phone, click every link, apply each code at checkout and stop before payment. Time a lead form’s first reply on any channel, and ask a bot what customers ask about refunds, delivery and prices.

5. Compare with the promise and keep the original

Mark each step passed, silent, late, broken or wrong, with its time. Keep the email itself with the screenshot. Gmail’s Show original opens its full header, which shows where it came from, so the step’s owner can check the finding without repeating the test. Report each failure to its owner as the step, the promise, what arrived and its cost a day.

An AI customer runs the same 5 steps, waits days or weeks at almost no cost and keeps every message as it arrived, as how AI customers work explains. Obsession’s AI customers shop a store as new customers every week and after every change, and every failure comes back with proof and the fix.

What a silent step costs

The cost of a silent step is its revenue a day multiplied by the days it stays silent, both from your own reports.

• Revenue a day: the flow’s attributed revenue for the last 30 days, divided by 30. Halve it for a cautious figure, since some of those shoppers would have bought anyway.
• Days silent: from the break to the check that finds it, plus the fix. A break at a random moment waits on average half the gap between checks: about 15 days with a monthly review, 3.5 with a weekly test and half a day with a daily one.
• Cost: revenue a day × days silent.

Halving the gap between checks halves the expected cost of a break, so set the interval from this figure. For sale week, use last year’s Black Friday week from the same report.

For a lead form, the sum is unanswered requests a day × days silent × your request to deal rate × your average first year deal value. For a support bot, it is the refund or credit a wrong answer promises × the customers who asked.

How often to check

• After every change, the same day. A new theme, app, form, flow, discount, routing rule or bot prompt can break a step that worked yesterday, so test the journeys it touches.
• Every morning for what moves daily: the pages behind live ads, and your bot’s answers on refunds, delivery and prices.
• Every week, every journey end to end, each with a new identity, and lead forms at several hours of the day, evenings included.
• Before peak dates: before your code freeze, about 20 November, the evening before Black Friday on Friday 27 November 2026, and on Cyber Monday, 30 November.

The Black Friday checklist sets out 12 store tests in the order a shopper meets them.

When a human check is enough

For 1 store that changes rarely and runs no paid traffic, a person with a new email address, a phone and a sheet covers it, once a month and after every change. It takes an afternoon to start and 48 hours of waiting.

A human mystery shopper is the right check where judgement matters, such as a shop floor, the tone of a phone call or a delivery at the door (AI customers and mystery shoppers compares the two).

By hand, each test covers 1 moment and builds no history, the hours grow with each store or client, and the evidence is whatever someone saved. AI customers fit first where a team checks many journeys, stores or clients, changes something every week, buys paid traffic or runs AI agents in front of customers.

Ways to test your customer journeys

Obsession makes AI customers, so every other option is described from its provider’s own pages as of 24 October 2026.

Limits

• A test shows what arrived. The cause of a silent step sits in your own tools: the flow, the trigger, the routing rule or the pages the bot reads.
• A new customer sees what new customers get. A long standing subscriber can receive other emails, in another Gmail tab.
• 1 run is 1 customer at 1 moment. Repeat runs show whether a failure happens every time or now and then.
• An AI customer stops at a bot wall or a CAPTCHA, and records the step it couldn’t test.

Start with the journey your flow report says earns most. Run it once this week as a new customer, then put its revenue a day into the cost sum to set how often to check it.

Questions about revenue leaks