Web search API: a search engine your code or AI agent can call
A person reads a page of results. An agent needs the same results as data, with the text of each page attached, so it can answer from them.
A web search API is a service that takes a query from a program and returns matching web pages, each with its address, title and text. Search APIs built for AI agents, such as Exa, Tavily and Parallel, add page text or excerpts that a language model can read and cite.
On this page
What a web search API returns
A program sends a query and gets back a ranked list of results. What comes with each result is what sets one API apart from another. Three built for AI agents describe their results this way in their own docs, read on 11 November 2026:
- Exa: each result’s title, URL, an estimated published date, the full page text, highlights from it and a summary.
- Tavily: each result’s title, URL, a short description and a relevance score. The parsed page text and a generated answer come on request.
- Parallel: takes an objective in plain words and returns each page’s URL, title, publish date and excerpts written for a language model.
Each is set beside Obsession on its own page: Obsession vs Exa, Obsession vs Tavily and Obsession vs Parallel.
Why an agent needs a web search API
A language model answers from what it learned in training, which stops at a date. A search API hands it pages as they stand today, so an agent can answer a question about last week’s price change and cite the page it read. Handing a model sources to answer from is called LLM grounding.
A support agent checks a supplier’s returns policy before it replies. A research agent reads 30 pages on a market before it writes a brief. Both call search first, then read in full the pages that matter.
What a public crawler never sees
A search index holds what its crawler could fetch, and a public crawler doesn’t create accounts or wait for the next email. Behind that login wall sit the welcome email a company sends a new subscriber, its signed-in price and the limits inside its free trial. The offer in its cancel flow is there too, and none of it appears on a page a crawler can open.
Obsession collects what companies show only to customers, step by step: their screens, emails and texts, member prices, trial limits, quotes and cancel offers, each dated with a screenshot. Teams and AI agents look up any company through the API or MCP, and a company not yet covered is added on request. An agent calls it beside a search API, to set what a company shows the customers who signed up next to what it publishes.
A search API finds pages for a query. A scraping API fetches a page whose address you already have and returns its content. Many agents use both: search to find the pages, then scrape to read them in full.
Web scraping, definedIt quotes the URL the API returned with the result. Many search APIs also return the passage that matched, so the answer can point at the exact text it relied on.
It depends on what the agent needs back: full page text, short excerpts, a ready-made answer or a relevance score. Our comparisons set each one’s results side by side, from its own docs.
Every comparisonSources
In the order the page cites them. Every page was read on 11 Nov 2026.