Tagged: methodology
2026-10-09The words our blog will not store, and why this post cannot name themSince 2026-10-08 new posts on our blog live in a database whose trigger refuses to publish any post holding a verdict word, this one included.2026-10-04How to test your MCP server before you ship itPaste your endpoint into the MCP Checkup homepage and see what an agent would see: handshake, tools/list, error shape, redirects, tool text. What each check can conclude, why the 24-hour result is unsigned, and why baseline checks show UNVERIFIED.2026-10-01Your agent added an MCP server by itself. What do you actually know about it?Personal agents now add MCP servers on their own. No one can see from the outside what a server does with what your agent sends it, but some things can be checked: whether it answers, which tools it offers, and whether they changed. 236 of 1,470 servers in our directory did in 30 days.2026-09-30Which MCP protocol revision do servers answer with?Every probe starts by agreeing on a protocol revision, and every signed result records the one it got. Across the 1,004 servers whose latest result is signed, 580 ended on 2025-06-18, the revision our fallback offers, and 394 on 2026-07-28. What those two numbers do and do not say.2026-09-30The one call our probe sometimes holds backTo see how an MCP server refuses an unknown tool, our probe calls a name reserved for a tool that should not exist. A server could define that name, so the probe checks the tool list first and holds the call back when it cannot rule that out: 838 of 2,282 public endpoints on 2026-09-30.2026-09-30A rate that could only ever say 100 percentOn 2026-09-23, all 833 servers in our directory showed the same reachability figure: 100 percent. Not because they had all answered us every day, but because of how we counted. We found it while fact-checking a blog post. What was wrong, what we changed, and what the figure shows now.2026-09-24The null byte that would not storeOne tracked MCP server published a legal regular expression containing \u0000 in a tool schema. PostgreSQL cannot hold that character, so for four days every snapshot of that server failed after its header row was written. How we fixed it without rewriting the evidence.2026-09-24Which code checked you: suite_commit and suite_digestEvery signed result names the check-suite commit that produced it and a digest of that commit's check-suite tree, which anyone with the public repository can recompute with one command. How the value is made, how the probe is stopped from deploying a stale one, and what older results cannot offer.2026-09-24We asked 1,144 MCP servers for a made-up toolThe one tool call our probe makes names a made-up tool, to see how the server says no. Until this week we recorded 58% of signed servers as a risk for that check and could not say what they had answered. Now we record seven kinds of answer, and nine in ten of the flagged servers gave the same one.2026-09-23Why we don't score MCP serversMCP Checkup shows what it observed about an MCP server and refuses to roll it into a number or a grade. The reasons are structural, not modesty.2026-09-23Why we probe unclaimed servers every 12 hoursMCP Checkup checks unclaimed servers twice a day, claimed servers on the Free plan every 8 hours, and Indie servers every 15 minutes. The reasoning behind each number, what the cadence does and does not tell you, and what money does and does not change.2026-09-14What a signed attestation proves, and what it does notA probe run on MCP Checkup produces a signed DSSE envelope unless it is disqualified, and the page says so. Here is exactly what the signature covers, how the code that ran is identified, how a third party can check it, and the honest list of things the signature says nothing about.2026-09-14The homepage check leaves no trace in the directoryPaste an MCP endpoint on the MCP Checkup homepage and you get a one-off result page that expires. What that check does, what it deliberately does not do, and why a trial is never allowed to become a public page on its own.2026-09-14Why MCP Checkup doesn't run your code (and what that means for stdio servers)A remote MCP server is an address. A stdio server is a package, and checking it means executing it on somebody's machine. Where the line sits today, and the three-tier design we are building toward.2026-09-14The several ways an MCP server says "no", and which ones we count against itA refusal is not a failure. MCP Checkup distinguishes a well-formed JSON-RPC error, a credential gate with a real challenge, a modern protocol rejection, and a malformed refusal, and records each differently. Here is the taxonomy the probe uses.