A Siteline study examined a Claude agent’s performance with 100 leading B2B software products, revealing that access issues and undisclosed pricing led the agent to seek information on external websites when it was unavailable on official sites.
Siteline founder David Kaufman conducted a study using a simulated Claude agent to make 534 attempts in order to find monthly prices for all plans and emphasize key features.
The information enters the funnel stage that most agent-visibility coverage overlooks, following a buyer’s familiarity with your product and the dispatch of an agent to assess pricing and features.
Siteline’s business involves providing agent analytics and an AI tool to assess agent readiness, so it is financially motivated by the discovery that numerous websites are unprepared for agents.
What the Siteline agent was requested to do
Siteline examined 20 items across five sectors, including productivity, developer tools, marketing and sales, customer support, and analytics. It verified the agent’s ability to enter the website, retrieve information on plans and prices, and calculate expenses in terms of tokens and tool usage.
On average, it took around 32 seconds and $0.24, with three search-or-fetch tool calls, to complete a task on Sonnet 4.6 at the median. There is a considerable time and cost disparity between the fastest and slowest runs on Siteline, primarily attributed to web-search calls. Linear proved to be more effective, processing four plans in one fetch for approximately $0.11 on the company’s website.
The agent was driven towards third parties due to issues with access errors.
Roughly 30% of attempts to access a website encounter an error, with about 25% of these errors stemming from bot blocking or pages that cannot be read. While most of these attempts are eventually successful upon retry, 5% of users opt to visit third-party websites instead, which may provide outdated or inaccurate information.
Errors led to a significant disparity in content, as runs with access errors extracted 58% of content from third-party sources, whereas error-free runs only pulled 12%. Anthropic and OpenAI do not utilize JavaScript, unlike Google. Vercel data, as reported by SEJ, revealed that AI crawlers account for 28% of Googlebot volume. Siteline’s findings indicate that 13% of runs included internal references to JavaScript or rendering issues, which were not classified as errors. SEJ also noted that one-third of top fintech homepages provided minimal content, exposing a blind spot in JavaScript handling.
The instances demonstrate that Zendesk’s pricing page utilized JavaScript rendering for the plan table, causing readability issues for the agent. As a result, the agent resorted to third-party blogs, incurring five times the Linear cost. Coda’s pricing fetches were unsuccessful, prompting the agent to refer to third-party sources. Braze’s agent encountered difficulty accessing the pricing page and sought information from G2 and Vendr.
The report suggests maintaining an active pricing page, even if it does not display prices, to avoid relying on third parties.
Pricing Pages Fell Short
Across all the plans, 65% displayed clear prices, whereas 14% did not show any prices and instead directed to a sales representative. In contrast, approximately 30% of marketing, sales, and customer support products lacked pricing information, in contrast to none in productivity and developer tools.
A button labeled ‘Contact Sales’ can hinder buyer’s agents who are trying to compare prices. Siteline proposes that this button may allow agents to suggest competitors who have public rates. Tests have shown that FullStory’s page did not show prices, and Databricks was the priciest at $0.95 per run, with pay-as-you-go rates being concealed behind a difficult-to-access calculator that redirects to third-party sites.
Why This Is Important
A plan table loaded on the client side may appear empty to agents, despite appearing complete on the browser. Siteline’s solutions, discussed in SEJ, involve generating pricing and features server-side and emphasizing important information upfront, as agents usually access only the initial 15,000 to 20,000 tokens. Siteline also suggests using llms.txt, although Google’s advice differs, and its effectiveness is uncertain without unbiased data.
Siteline is one of the companies providing agent-readiness scoring, which was a topic explored by SEJ when Cloudflare introduced its own scanning tool. This metric evaluates individual agents on a specific task related to accessing particular websites, rather than assessing all agents on their handling of every product.
Looking to the future
Sites that present plan details clearly and prominently on the first view facilitate agents in representing confidently to potential buyers who compare plans before making a purchase. The key issue is whether measures of agent readiness will be based on common metrics or differ across vendor assessments, with each emphasizing distinct factors.
Vasilyev Alexandr/Shutterstock is the image that is highlighted.
