Testing competitor AI claims

Competitors claimed AI features. We separated fact from fiction.

A B2B SaaS company was developing its own AI features while competitors were already promoting theirs. Product marketing needed to understand what those features did, how they were packaged, and what customers paid for them. The company also needed evidence on whether the products really matched the claims, so that salespeople could respond.

The challenge

Competitor AI messaging changed quickly. Vendors announced copilots, agents, automated workflows, generated insights, and other capabilities using similar language. The wording made direct comparison difficult.

Some features were included in existing plans. Others required an add-on, a premium package, usage credits, or a separate contract. Public pricing rarely showed the full commercial structure. Customers could face limits based on users, tasks, tokens, data volume, or access to specific models.

The client also knew that a polished demonstration did not prove that a feature worked consistently in a customer environment. An AI tool could perform well on a prepared example and struggle with incomplete data, complex workflows, permissions, or unusual requests. Product marketing needed to know where competitor claims were supported and where they depended on ideal conditions.

Timing made the project more important. Competitors were selling an AI story while the client was still building. The company needed a credible response for sales conversations and launch planning without overstating unfinished functionality.

The research had to answer four practical questions. What could each competitor’s AI features do? How mature were they? How were they packaged and priced? Where did the customer experience fall short of the marketing?

What we did

We created a comparison framework based on customer tasks instead of competitor labels. This allowed the client to compare products that used different names for similar functions.

The framework covered the main AI workflows relevant to the market, including how users initiated a task, what data the system used, how much control the user retained, and what happened when the output was incomplete or wrong. We also tracked setup requirements, integrations, permissions, review steps, and other factors that affected real use.

We reviewed product documentation, release notes, demonstrations, webinars, help content, and pricing material. We separated generally available features from beta releases, limited previews, and future announcements. Claims without sufficient proof remained qualified.

Commercial research examined where each feature sat within the competitor’s packaging. We looked at plan eligibility, add-ons, usage allowances, limits, and the conditions attached to access. Where public information was incomplete, the comparison recorded the gap instead of treating the feature as free or universally available.

We also examined evidence from users and buying situations. This helped test whether the promoted workflows worked outside controlled demonstrations. The research looked for recurring problems such as unreliable output, limited context, weak integrations, slow setup, unclear usage limits, or the need for substantial human review.

Each competitor received a concise assessment. It explained what the AI feature did well, where the evidence was weaker, which customers were most likely to value it, and how the vendor charged for access.

We then translated the findings into launch and positioning implications. These covered areas where the client could claim a meaningful difference, features where parity would be expected, and competitor weaknesses that sales could address while the client’s own development continued.

Results

The client gained a more accurate view of the AI market. Leadership could distinguish between established capability, limited functionality, and marketing that ran ahead of the product.

The packaging analysis informed decisions on whether the client should include AI in existing plans, reserve parts for a premium tier, or charge according to usage. It also showed where competitor pricing created friction or uncertainty for customers.

Product marketing used the findings to sharpen the launch narrative. Messaging focused on customer outcomes and the practical strengths the company could support. The team avoided broad claims that would be difficult to prove and concentrated on areas where the planned product had a credible advantage.

The research also gave the company time. Sales teams could respond to competitor AI claims with specific evidence about limits, packaging, and real-world use. Current messaging stayed within the capabilities the company could support.

Product and engineering gained a clearer view of which competitor capabilities deserved attention. Some visible features were genuine requirements. Others had shallow adoption, narrow use cases, or significant operational limits.

By the time the client prepared its launch, it understood the products it would be compared against and the commercial models surrounding them. The company could position its AI features with greater precision, make better packaging decisions, and manage the period before full release with a credible competitive response.