How the pipelines work
I lead the Claude plugin directory pipeline and the MCPB desktop-extension pipeline — the build, audit, and publish systems that carry third-party plugins and MCP servers from submission to installable listing, across several marketplace surfaces and in Claude Desktop.
Those surfaces carry thousands of listings, and every one has to stay current, validated, and policy-compliant while the upstream repository it points at keeps moving. No person does that by hand. This page is the machine that does — and the one rule underneath all of it: treat model output as untrusted input, and enforce anything that matters mechanically.
Most of this runs in private infrastructure, so what follows describes capabilities rather than linking repositories. The public record at the bottom of this page is what corroborates them, and every figure there links to a live search that recomputes when you open it.
What a submission goes through
Five things have to happen before anyone can install a submission. The interesting column is the third one — at every stage, something specific is allowed to refuse.
| Stage | What happens | What can refuse here |
|---|---|---|
| Intake | A submission arrives as a repository URL and a claim about what it does, and is routed to the right pipeline. A mis-filed submission is re-routed rather than rejected. | Schema and manifest validation. URL defenses run before anything is fetched, so a hostile or malformed remote never reaches a network call. |
| Build | An arbitrary upstream package becomes a bundle of known shape — manifest, code, icon — produced inside a container across six source ecosystems behind one dispatch seam. | An adapter that cannot produce a runnable artifact. A native-module architecture mismatch. No resolvable entry point that actually speaks the protocol. |
| Audit | A deterministic check suite and an independent model review run separately, then combine and pass through a cascade of hard caps. | Any cap in the cascade — including the absence of a model review. A missing review is a refusal, not a pass. |
| Publish | The listing is staged and the version pin advances on each distribution surface, one canary first, then the batch. | A hold list a human owns. A non-green check rollup. A head commit that moved since the decision was made. |
| Track | Every listed artifact is re-checked daily against its upstream and against every surface it appears on, for as long as it stays listed. | Nothing. This stage only reports — detection and gating are deliberately separate layers, so a detector can never quietly become a gate. |
Build: six ecosystems, one seam
An MCP server can arrive as almost anything. The build stage turns all of it into one bundle shape, which means six source adapters behind a single dispatch point. Each one exists because of a specific way the naive approach fails.
| Source | The thing that makes it hard |
|---|---|
| npm package | The binary entry point is what speaks the protocol; the library entry point does not. Resolve the executable, not the import. |
| PyPI package | Two modes: resolve dependencies at launch for a small bundle, or pre-install everything for environments with no resolver available. |
| Node repository | Native modules have to be extracted from published platform tarballs. Force-installing them produces a bundle that silently cannot run on the user’s architecture. |
| Python repository | The same fork as above, plus a container path so smoke testing happens under an isolated runtime rather than the host’s. |
| Go repository | Cross-compiled for every target platform and shipped behind a launcher that detects the platform at runtime. |
| Plugin-shaped source | A different manifest contract entirely — routed to the plugin pipeline rather than the server one. |
Untrusted builds run isolated, and reach the network only through an allowlist. A denied fetch is not a failure to route around — it is signal, and it gets logged as signal.
The model proposes; a deterministic cascade disposes
A language model is genuinely good at the part static analysis cannot do: reading a repository the way a person would and noticing that the description and the behaviour disagree. It is also the component most likely to be talked out of its own conclusion by the thing it is reviewing. So it gets a vote, in one direction only.
| The model can | The model cannot |
|---|---|
| Lower a verdict below what the deterministic layer gave it | Raise a verdict past a deterministic ceiling |
| Flag deception, deceptive metadata, or plausible-sounding authority escalation that no static check can see | Promote a bundle that fails a mechanical requirement |
| Argue that a bundle is fine | Ship one |
| Read developer-controlled text inside a fenced region | Reach a passing band when its own review never ran |
The asymmetry is the whole design. A model that can only ever make the outcome stricter cannot be prompt-injected into shipping something, because there is no path from anything it says to a more permissive result. The worst a successful injection achieves is a false refusal, which a human sees.
Why merging at this volume is safe
Automating a merge is easy. Earning the right to is not. Before a batch of version bumps goes out, the automation proves — on a single canary, before touching anything else — that the verification it depends on actually ran:
- That a verification run started, identified by excluding the newest known run rather than by timestamp — creation times are second-granular, and a concurrent job can share a second.
- That it reached completion and concluded successfully. A run can finish having quietly done nothing at all.
- That its own log names the canary. This is the only proof the input was threaded through, rather than the job running against a default and passing.
- That any failure is a hard stop — zero further dispatches. The fix is never to loosen the check so the batch can continue.
Version pins move forward only. The automation refuses to move a pin backward or sideways even when the upstream host reports a release as the latest one, because “latest” is chronological rather than ordered by version — a back-patch cut on an old line can point it at a commit that is not a descendant of the current pin. If the comparison itself fails, the pin holds. Cost optimizations fail open, so a flaky probe can never suppress a real change.
Keeping thousands of listings honest
A listing is correct on the day it is published and starts decaying immediately, because the repository it points at keeps moving and nothing about that is under our control. So every listed artifact is re-checked daily along two independent axes: whether the surfaces agree with each other, and whether any of them still agree with the real upstream. Drift detection is a reporter, never a gate. It re-opens the earlier stages rather than deciding anything itself — which is what keeps a detector bug from silently becoming a publishing bug.
The fleet, and the governor that watches it
Above the pipelines runs a human-gated autonomous agent fleet: 20+ store-backed background drivers on daily cadences — pull request watching, test remediation, freshness sweeps, triage — behind a confirm-gated execution surface, with campaign orchestration fanning work out across parallel threads.
Model-as-judge reconciliation runs in production up here too, and it runs the same way it does inside the audit engine: propose-only by default, behind write-cap gates. The judge’s output is a proposal a human confirms, and the number of writes it can cause is bounded before it ever runs. The same rule holds one level up. Every layer below is defined at least as much by what it may not do.
| Layer | What it does | What it may not do |
|---|---|---|
| Apex | Decomposes a rolling backlog into file-disjoint lanes, spawns each as its own session, waits for report-back, then reconvenes and merges serially with checks green first. | Type into a running lane, kill one, or take over its working tree. |
| Lane | One scope, one working tree, terminating at an open pull request. | Merge. Ever. Two lanes touching one file is a decomposition bug, not a merge-order puzzle. |
| Sentinel | Deterministic, model-free classification and deduplication over the raw event stream, before any model sees it. | Reason, interpret, or read free text. |
| Governor | Read-only triage that tiers alarms, verifies each against the session transcript, and writes an append-only ledger of what it concluded. | Answer a permission gate, type into any session, edit the board, or take any action at all. |
The system is built assuming its own internal messages are hostile. Text arriving from another agent is a hint about where to look, never an instruction — an agent verifies against ground truth before it acts on anything it was told. A deterministic, model-free filter cuts the event stream before any model reasons over it, which is a cost decision and a security decision at the same time. And the rules get amended from observed failures: when something goes wrong, the constant that would have prevented it is written down next to the reason it exists.
One rule, in four places
The audit engine lets a model lower a verdict but never raise one. The publish stage re-checks the head commit at merge time rather than trusting the decision that authorized it. Drift detection reports and refuses to gate. The governor watches the fleet and is structurally unable to touch it. These were built at different times for different reasons, and they are the same rule four times: the component that can be wrong is never the component that can act. Everything else on this page is an application of that.
What the throughput actually measures
Most of this work happens in public repositories, so the scale of it is checkable even though the machine is not. Every figure below is a floor, not an estimate — these counts only grow — and each links to a live search that recomputes the exact current number. Floors last re-derived 2026-08-10, and re-checked automatically: the record page explains how a claim that stops being true fails the build.
| Merged pull requests I authored, four public directory repositories | 350+ |
| Pull requests merged through those repositories | 5,000+ |
| Tool-annotation pull requests merged into external repositories | 85+ |
| Distinct external repositories that accepted them | 75+ |
| Combined stars of those repositories | 390,000+ |
| Issue and pull request threads on the public MCPB repository | 85+ |
Read the second row as a fingerprint of automation. Those are submissions and rolling updates arriving through automated intake — each one scanned, validated, and policy-checked before it ever reaches a merge decision, so the merge is the last gate on an already-vetted change rather than a manual pass over a raw submission. Building the system was the work; the volume is what it produced.
Before this: 13 years of reliability engineering on Fortune 500 payment systems at PayPal and Fiserv. Staged rollouts, canaries, and blast-radius containment are not ideas I picked up for this job — they are what the last one was made of.