Early access
AI agents that test AI vendors on your cases before you sign.
Vendor agent check compares AI agents on your own cases before you sign: with each vendor’s OK, declared test customers run your hardest real cases on every shortlisted agent and time every handoff, so you choose on signed evidence, not a demo.
We keep your email and your AI agent’s address to run your check and tell you about Obsession. Privacy notice
Vendor agent check
Obsession agent for Your companyCases with a fail9 of 20
Vendor B
/20passed
Vendor C
/20passed
2 week run·
H-01Ask for a person
Policy 6.1 · a person within 3 min
Test customer 2 · AI · for Your company
Can I speak to a person, please?
Vendor B
Connecting you.
Person in 1 min 50 s
Vendor C
How can I help?×3
No person in 6 min
Signeded25519 · 7f3a 91c2 … c91e
With both vendors’ OK · no jailbreaks
Report readyfor procurement
OpenExample
Pick it, and everything it needs is already set up.
Nothing to build or buy, and nothing to run yourself.
- Declared AI test customers, working for your company
- Their own inboxes and numbers on every agent
- Your hardest cases, each with its policy
- Each vendor’s written OK, on file
- Every test tagged on every vendor
- Every answer and handoff signed and dated
It runs on your schedule, and keeps the proof.
Every shortlisted agent on your own cases, side by side, with every answer and handoff signed.
Example: 2 vendors, your 20 cases
Vendor agent check
With both vendors’ OK, run 20 cases from our tickets on Vendor B’s and Vendor C’s agents, set up on our policies, and compare them side by side.
- Targets
- Vendor B and Vendor C, both with their OK
- Journey
- Set up the same 20 cases
- Say it’s AI, then ask
- Time every handoff
- Score against your policy
- How often
- 2 weeks before you sign, then monthly
- Report
- A signed report for procurement
- Set up
- Agent ID, declared as AI
- An inbox and number per vendor
- 20 cases from your tickets
- Tagged as tests on both
Run log
- Vendor B and Vendor C agree to the test in writing. Both agents are set up on your policies, and the same 20 cases go to each.19 Sep
- Case H-01. Test customer 2 says it’s AI and asks Vendor B for a person. “Connecting you.” A person answers in 1 minute 50 seconds.22 Sep 10:40
- The same case on Vendor C: “How can I help?” 3 times, and no person in 6 minutes. Your policy says 3.22 Sep 10:52
- Both fail a refund on day 35 and an expired code. Vendor B also fails an order edit. Vendor C fails 6 more, from an address change to a data deletion request.29 Sep
- Every case run on both. Vendor B passes 17 of 20, Vendor C 12.3 Oct
Finding
Vendor B passed 17 of your 20 cases and Vendor C 12. Asked for a person, Vendor C kept a customer waiting 6 minutes where your policy says 3.
A side by side report for procurement, with both transcripts on every case, signed. The choice stays yours.
Example run. 20 cases on 2 vendors, 40 runs in 2 weeks, every answer signed and dated.
The agents do the legwork. You make the calls.
Pick the cases that matter
Take them from your own tickets: a late refund, an address change, an expired code, a customer who wants a person. Each gets the policy that decides it.
Every vendor agrees, then sets up
Each vendor on your shortlist agrees to the test in writing and sets its agent up on your policies, on the channels you’ll run. No agreement, no test.
Test customers run every case on every agent
Each says it’s AI and who it works for, asks the same way on every agent, and times every answer and handoff. Every test is tagged.
You get a side by side report
Every case passed or failed for each vendor, with both transcripts and the policy that decided it, signed for procurement. Once you go live, the same cases run monthly.
It covers the whole job, and stops where it should.
Every case, on every agent
Right answers
Refunds, returns, order changes and cancellations, against your policy.
A person when asked
Whether a person is offered, how long it takes, and whether the handoff lands in your helpdesk.
Says it’s AI
In its first message, and truthfully when a customer asks “Am I talking to a person?”
The same answer every time
The same case asked again on another day, compared.
Before you sign
The deck against the agent
The resolution rate, handoff time and languages in the sales deck, against what the agent did on your cases.
Your hardest cases
A refund on day 35, an expired code, a customer asking you to delete their data: the cases a demo skips.
The channels you’ll run
Chat, email, phone or your portal, wherever the agent will meet your customers.
Your current agent
The agent you run today, or one you build in house, on the same cases, so every vendor has a bar to clear.
Where it stops
Each vendor agrees first
In writing, before a single case runs.
No tricks
It asks what an ordinary customer asks. No jailbreaks, prompt tricks or load tests.
Your customers’ details
Never shared. Each test customer asks a case as its own question.
The results
Yours alone. No agent is ever published or ranked in public, and no vendor sees another’s results.
Every vendor runs the same cases, side by side, before you sign.
See a real store checkA verdict per case
Passed or failed for each vendor, with the policy that decided it.
Both transcripts
Every answer from every vendor on the same case, timed, beside your policy.
Every handoff, timed
How long each agent took to reach a person, and whether the case arrived in your helpdesk.
The deck, checked
Each vendor’s claims beside what its agent did on your cases.
A report for procurement
Side by side and signed, for your team, legal and finance, as a page and a PDF, or an email, Slack, a sheet or a webhook.
A monthly check
Once you go live, the same cases every month on the agent you chose, with what broke or recovered.
Start from the recipe. Change anything.
- Vendors
- Every vendor on your shortlist, each with its written OK
- Cases
- For example: 20 from your own tickets, your hardest included
- Policies
- Your refund, returns, cancellation and data pages, set up on every agent
- Channels
- The ones you’ll run: chat, email, phone or your portal
- What counts
- For example: a person within 3 minutes, the right refund window
- Your bar
- The agent you run today, or your own build, on the same cases
- Tagging
- Every test tagged on every vendor, agreed with each first
- How long
- 2 weeks before you sign, then monthly once you go live
The same recipe, different jobs.
- FoundersChoose the AI agent your customers will meet on your own cases, not a demo, and keep checking it once it’s live.
- AgenciesPick the chat or voice platform you build client agents on, with signed evidence to show each client why.
- SalesBefore an AI agent answers your inbound buyers, run your real buyer questions on every one you’re weighing.
- DevelopersRun the same cases on your own build and every vendor’s, from your code, and decide on the results.
Decide in 2 weeks, before a customer meets an agent you’d turn down.
Up to 76 days sooner to a decision: 2 weeks of side by side checks on your own cases, in place of a 90 day pilot on live customers.
| Today | With a vendor agent check | |
|---|---|---|
| Evidence | A demo, a deck and a live pilot | Your own cases, run on every agent |
| Time to decide | A 90 day pilot | 2 weeks |
| Who it’s tested on | Your customers, during the pilot | Declared test customers |
| A fair comparison | Each vendor on different weeks and customers | The same cases, customers and channels on each |
| A person when asked | Found when a customer complains | Timed on every case |
| Proof for procurement | Notes and screenshots | Every answer and handoff signed and dated |
Every test customer says it’s AI. Every vendor agrees first.
Run your own cases on it: with each vendor’s OK, Obsession’s Vendor agent check puts your hardest real cases to every shortlisted agent, set up on your policies, and scores every answer side by side.
Yes. Vendor agent check needs each vendor’s agreement in writing before a single case runs. No agreement, no test.
Vendor agent check takes them from your own tickets: the questions your customers really ask, your hardest included, each with the policy that decides it. Each test customer asks a case as its own question, so no real customer’s details reach a vendor.
Yes. Every Vendor agent check test customer says it’s an AI test customer working for your company, so every vendor can see it’s a test.
No. Vendor agent check asks what an ordinary customer asks. No jailbreaks, prompt tricks or load tests.
Every Vendor agent check test is tagged on every vendor, and we agree that with each one before the first case runs.
No. The Vendor agent check report is for your team alone, and we never publish or rank anyone’s agent in public.
Yes. In Vendor agent check, the agent you run today, or one you build in house, runs the same cases, so every vendor has a bar to clear.
Vendor agent check runs the same cases monthly on the agent you chose, and you hear which ones broke or recovered.
Vendor agent check gets you to a decision up to 76 days sooner: 2 weeks of side by side checks on your own cases, in place of a 90 day pilot on live customers. And none of your customers meets an agent you then turn down.
No. A passed Vendor agent check is dated evidence of what each agent did on each case.
The free check is a Vendor agent check on the AI agent you run today, or a vendor’s you’re trialling with their OK: 3 test customers use it on 1 channel, and your report lands within 4 days.
Choose your AI agent on your own cases, before you sign.
Your first check is free: 3 test customers use the AI agent you run today on 1 channel, and your report lands within 4 days. It sets the bar every vendor on your shortlist has to clear.