Article

journal / 11-release-vs-test-extension

Why the release extension and the test extension had to be different

A11YWeb Apps
Carla March 25, 2026 3 mins read

This is one of those engineering details that sounds boring until you realise it is protecting the truth of the whole release story.

Release extension toolbar launch

One of the smarter moves in this repo is that the extension test package and the extension release package are not the same thing.

At first that might sound like overengineering. Why not just test the package you plan to ship? The answer is that browser automation has its own practical constraints. Playwright still cannot exercise the real toolbar action in exactly the same way a human clicks it, so the repo creates a test-only bridge in dist-extension-test/ that can call the same handler function without pretending that bridge belongs in the shipping artifact.

That boundary shows up everywhere once you notice it. The build script adds localhost-only permissions and externally_connectable only in test mode. The release manifest must stay free of those fields. The release validator fails if test-only hooks leak into the shipping package. The docs keep repeating that dist-extension-test/ is for Playwright only and must not be shipped.

I really like this because it avoids a common trap: letting automation convenience quietly redefine the product boundary. The repo is basically saying, β€œyes, I need a test seam, but no, that seam does not get to become release reality.”

The same pattern shows up in the release checklist. The automated extension boundary is spelled out: the MV3 suite uses a localhost-only bridge, it hits the same handleActionClick(tab) code path, but manual release review must still click the real toolbar action. That is a subtle but important piece of honesty. The automation is good, but it is not allowed to claim more than it proves.

This also helped the project talk more clearly about permissions. Once you have to maintain separate release and test packages, you are forced to say exactly what the release package does and does not include. That turns into better docs in extension/PERMISSIONS.md, better manifest hygiene, and a much clearer store-review posture.

I think this is a good example of the project getting more mature without getting more self-important. There is no fancy abstraction here. It is just a careful response to a messy practical problem: automation needs hooks, but shipping the hooks would be dishonest.

Visual evidence

The release-vs-test distinction is mostly in manifests, scripts, and automation plumbing, so there is no reliable screenshot that isolates it. The closest visible release-package evidence is the store-style toolbar launch and hostile-site screenshots:

Release extension hostile-page proof image

What I was really learning here

I was learning that test infrastructure can quietly distort the product if I let it. The clean answer was to give automation its own package and then defend the release package from that convenience.

Evidence

  • Commits:
    • a5bba65 – release hardening and extension AI UI
    • dff2606 – deterministic canonical packaging artifacts and docs
  • Files:
    • ../../scripts/build-extension.js
    • ../../scripts/check-extension.js
    • ../../extension/PERMISSIONS.md
    • ../../extension/README.md
    • ../../EXTENSION_RELEASE_CHECKLIST.md
    • ../../tests/playwright/extension-mv3.spec.js