Questions to Ask Vendors About Agentic Delivery Claims

Every vendor evaluation eventually comes down to a conversation, and that conversation is where agent washing either survives or falls apart. A capabilities deck can be engineered to look identical across genuine and rebranded vendors. A direct question, asked well and listened to carefully, cannot be answered the same way by both.

Gartner estimates that only about 130 of the thousands of self-described agentic AI vendors are real, with the rest rebranding assistants, robotic process automation, and chatbots as agentic. The difference between the two is rarely visible in marketing. It is almost always visible in how a vendor answers a specific, evidence-seeking question. The questions below are chosen because a genuine AI-native partner can answer each one concretely, and agent washing has to deflect.

Every useful question shares a structure. It asks for something specific and verifiable, a name, a number, a mechanism, a commitment, rather than a description of intent. The tell is not usually a wrong answer. It is a vague one. When a question that should have a concrete answer produces a shift to roadmap language, adjectives, or a change of subject, that deflection is the signal.

Read the questions in that spirit. What matters is not just what a vendor claims, but whether the claim resolves to something you could check.

Can you name a production deployment, with the agent count and how long it has run? This is the single most revealing question. A genuine partner names an environment, describes what the agents do, and offers a metric a reference will confirm. Agent washing retreats to pilots, proofs of concept, and hypotheticals, because it has no production to point to.

What in your platform is proprietary, and what is resold? A real AI-native partner can draw the line clearly and explain what it built and controls. If the “platform” dissolves under questioning into a third-party model with an integration layer, you are evaluating a reseller, and the capabilities you are being sold belong to someone else.

How are agent actions governed, logged, and reviewed? The strong answer describes specific mechanisms: human-in-the-loop checkpoints on consequential actions, audit trails, controls that operate inside your compliance rules. The weak answer describes good intentions and a governance feature that is coming soon.

Will you commit to a measurable outcome and put fee at risk against it? Willingness to price to a result is the hardest claim to fake, because it puts money behind the confidence. A partner that believes its delivery will engage with outcome-based structures. One selling repackaged effort will insist on time-and-materials and explain why outcomes are impractical.

How much of your own delivery runs on this platform? A firm that has genuinely re-engineered its engineering around agents can answer concretely and often proudly. A firm that sells agentic transformation while delivering the traditional way underneath cannot, and the hesitation is informative.

What happens when an agent fails or produces a wrong output? Maturity lives in the failure modes. A mature partner describes detection, escalation, human review, and recovery as designed behavior. An immature one has not thought past the happy path, which means you will discover the failure modes in production.

The same question produces recognizably different answers depending on which side of the line a vendor sits on. This is worth internalizing before the meeting, because in the moment a confident delivery can make a weak answer sound adequate.

The left column is not a set of lies. It is a set of answers that sound reasonable and commit to nothing. Recognizing the difference in real time is most of the skill in vendor evaluation.

Whatever a vendor claims, the highest-value follow-up is the same: can you show me, or can I speak to a client who has seen it. Genuine capability survives that request. Agent washing tends to explain why a demonstration or a reference is difficult to arrange right now. The willingness to be verified is itself the answer, and it costs a real partner nothing to offer.

This is also where regulated-industry experience separates cleanly. A partner that has delivered agentic systems in banking or healthcare can describe how it passed an audit, satisfied an examiner, or held up under compliance review. Those are not claims a vendor can improvise, which is why they are worth asking for.