The one call our probe sometimes holds back

One of our checks, error_taxonomy, needs our probe to call a tool. It calls exactly one, by a name we reserve for a tool that should not exist: __mcpcheckup_probe_nonexistent_tool__. The observation is what the server does with a call to a tool it does not have. We have written about what servers answer.

We also say, on the claim page and in another post, that we never call one of your tools. Those two statements fit together only if the reserved name is not one of your tools. Nothing stops a server from defining a tool with that name. So the probe does not assume. It checks.

When the call is held back

Before the call, the probe asks the server for its tool list. It holds the call back when any of these is true:

• the first page of the list contains a tool with exactly that name (an exact string comparison: no case folding, no trimming);
• the probe did not read the whole list: a page it read carries a nextCursor for a page it did not read, so it cannot see whether that page has the name;
• the list could not be read;
• a credential gate stopped the handshake or the tool list.

So the call is sent only when the probe, without credentials, has read the tool list to its last page (the one with no nextCursor) and no page it read contains the name. Apart from a credential gate, the handshake does not decide it: a run whose handshake failed for some other reason still sends the call if its tool list came back clean.

When the call is held back, error_taxonomy is recorded as skipped and unverified, with the reason, and the report page says which reason. It is never shown as a failure.

The fourth rule came later

The first three rules reached the probe on 2026-09-23. The fourth reached it on 2026-09-26.

Before that, when a credential gate stopped us, the probe still sent the call, because our auth_metadata check read the 401 that the call came back with. The reasoning was that a server behind a credential gate would not run a tool for a caller without a key. That holds only if the gate covers the tool call as well. A server that gated its tool list but not its tools, and defined the reserved name, would have run that tool. Since 2026-09-26, auth_metadata reads the challenge the probe has already received from the gate, and the call is not sent.

We have no evidence that this ever happened. We also cannot see the tool lists of those servers, and that is the point: a promise that holds only when we are lucky is not the promise we made.

How often

The latest run of each of the 2,282 public endpoints we were observing on 2026-09-30:

the reserved-name call	endpoints
sent, and the answer judged	1,424
held back: a credential gate	704
held back: the tool list could not be read	133
held back: the list had more pages	1
held back: the list named the reserved tool	0
not reached: our time or size budget ran out, or the server asked us to slow down	20

In all the tool lists we have been able to read since 2026-09-23, the reserved name has not appeared once.

What it costs

A call held back is one observation fewer. For those endpoints the report cannot say how they refuse an unknown tool, and where the list could not be read, auth_metadata has nothing to read either. We would rather record “not observed”, with the reason, than get the observation by running something a server might treat as real.

If you maintain a server

Please do not name a tool __mcpcheckup_probe_nonexistent_tool__. If your tool list comes in pages, the probe will not send the call. That is by design, and your report says why.

Where the numbers come from

A read-only query against the production database at 20:05 UTC on 2026-09-30: for each public target’s latest finished run, the error_taxonomy result, and, where the call was held back because the list could not be verified, the tools_list result of the same run. The all-time figure for the name appearing is a count over every stored error_taxonomy row. The dates are the days each version of the check suite reached the probe.