Changelog

The pull-request report has a real run behind it now

The action could post its findings on a pull request and send them to code scanning since 0.2.0. Neither path had ever run, because this repository never turned them on. It does now, so a demonstration request carries the comment, the job summary and both findings in the Security tab, off two faults planted on purpose: a test that calls the scanner and asserts nothing, plus a script no workflow invokes. The suite stays green with both in place, which is the claim this tool exists to make.

That run also caught something in our own writing. The finding heading joined the rule and the summary with an em dash, in a comment we post on other people's repositories. It is a colon now. Shipped in this release because the action installs the published package rather than the checkout, so a fix that is not released is a fix nobody gets.

The suggestion box and the hardened intake forms from 2026-08-15 are in this release too. They changed the site rather than the CLI, so they were left unpublished at the time.

The mutation numbers on the proof page were counted again

The suggestion box added two source files. A published count that no longer matches the repository is the exact failure this tool was built to report, so the run was repeated over everything now in scope: 40 files tracked as source, 27 carrying a mutation, 27 mutated, 27 caught, no survivors, across 108 tests. The proof page carries the run.

The duration was wrong too. It was wrong in our favour, so it is worth naming. The page said fourteen seconds. Measured end to end today the run takes 208 seconds, because every mutation runs the whole suite once. Twenty seven mutations is twenty seven test runs. That is the reason the default cap is four rather than every file.

A suggestion box, for the finding you think is wrong

A box for feedback and feature requests. Six checks is a deliberate number rather than a finished one. The most useful message this project gets is a finding somebody disagrees with. So there is a form for it now: pick feedback or a feature, say the thing, leave a contact if you want a reply.

It stores nothing, for the same reason the fix intake stores nothing. What you type is validated, given a deterministic reference and turned into a prefilled issue on the public repository, which you submit under your own account. That makes the request yours to watch and ours to answer in public. It also means there is no inbox on this side to fill up and no database of other people's ideas to guard. The security page carries the curl that shows what the endpoint returns. The privacy page now says what both forms do with what you type.

One detail worth writing down for anyone building the same thing. The prepared link carries a title and a body and no labels parameter, because GitHub only honours that parameter for someone who has permission to label an issue, then treats a label the repository does not have as an invalid URL. Either one answers with a 404, which would have made the single click this design rests on a dead end for exactly the people it is for. The kind is in the title and the label is applied at triage.

The same review found three ways to reach a 500 rather than an error message, all of them in the shape the fix intake had been using since it shipped: a slice that cut an emoji in half so the encoder threw, a finding field that was not a string, then both calls sitting outside the try that catches the body reader. Fixed in one shared module both forms now use, with the crash cases as tests.

The CLI did not change, so there is no new npm release. The version numbers on this page are the package's.

A sixth check, for tests that assert something that cannot be false

cannot-fail. no-assertion reports an empty test body. This one reports the harder case, a body full of assertions that hold whatever the code does: a literal answered inside a catch, an assertion swallowed by a catch that cannot fail the test, a deliberate fail marker whose catch is satisfied by the marker's own error, a status list that accepts both the success and the failure. Every shape was measured against real repositories before it shipped, and two more were measured and dropped. A top level || in an assertion was right twice in 110 real sites. An empty catch on its own was wrong fourteen times out of fourteen. Neither is a knob left off, both are rules that would have cried wolf.

The first draft reported 41 findings on fastify and 39 of them were wrong, all under t.plan(n), where the declared count normally catches the swallowed assertion itself. That gate is in the check and in the tests. What is left on fastify is two, and both are real.

A test title is not an assertion. The loose assertion matcher read the word "should" wherever it appeared, so every test whose title began "should ..." was treated as asserting. Comments, string contents and regular expression bodies are now blanked before anything is matched, and the title with them. That found a seventh hollow test in fastify and stopped this repository's own CLI test from reporting itself, since it writes a test into a fixture as a string. The proof command carried the same fault and disagreed with the check that emitted it, which is how it was caught: the finding appeared and its own proof withdrew it.

Every mutation this tool can make is now caught by its own suite. 63 tests became 93, and the eight survivors it reported on itself are gone. The run is on the proof page, with what each test pins.

Twenty two false positives, found by pointing Margyn at other people's repositories

Build output is no longer reported as missing source. Scanned five real projects and every ignored-source finding on four of them was wrong. A Vite app's committed assets were matched against the copies the build left in dist. A Foundry project's artefacts under contracts/out were reported although its own foundry.toml declares that directory. A bundler's output was reported because the build script that writes it also names it. All 22 passed proof mode too, which is the part worth saying out loud: the proof asked whether that path was committed, when the question is whether anything in the commit answers the path the reader asks for.

So a reference is now resolved against everything git has. A finding is only reported for a path that no committed file answers. The proof asks that question, so a finding that slips through retracts itself rather than failing a build on a copy of a committed file. On top of that, output a tool in the repository declares it writes is skipped: out from foundry.toml, a vite, next or cargo build from the script that runs it, an outdir in a build script, an outDir in a tsconfig. Read from declarations and resolved to real paths, never from a directory being called dist, because the defect this check was written from was vendored source in vendor/dist and that one still fires.

A help link that pointed at nothing. Every SARIF rule carries a helpUri into these docs, and the mutation rule's has always missed: it links /docs#mutation while the heading carried id="mutation-check", so a reader in the Security tab landed at the top of a long page. Fixed, with a test that resolves the help link of every check the scanner can emit. The existing links test could not catch it, because the site never links that fragment itself.

Margyn runs its own reproductions, posts them on your PR, and gains a cheaper way in

Proof mode, --prove. Every finding already ships a reproduction. Now Margyn runs it. For each finding it executes the read-only proof its check emitted, checks the output carries the markers it predicted, and marks it reproduced. A finding it cannot reproduce is retracted and dropped, so a gate never fails a build on a claim the tool could not show on your tree. The mutation proof reports as observed, because running your suite is how it was established. It is free, because a finding you can watch reproduce is the whole product.

Findings on the pull request. The action can now write a job summary every run, keep one pull-request comment updated in place, and upload SARIF to the Security tab. It uses the job's own token, so nothing is hosted and no secret leaves the repository. New CLI outputs --sarif-out and --comment-out back it, and both are one scan, so the mutation proof never runs twice.

Fix flow has an intake, and a cheaper tier. A page takes a finding, works from it rather than your repository, and prepares a tracked request without ever carrying the code snippet a finding sometimes holds. Solo Fix, 19 USD a month, is one fixed finding a month, the cheap way in below Fix flow's three. A real fix is a person writing a patch, so one a month is the honest floor rather than a number chosen to look small.

Two more ways to buy, one of which is a person rather than a binary

Team, 29 USD a month, is Watch for every repository an organisation owns, with issues triaged first and invoices instead of a card on request. It expands into the same capability Watch grants, because a customer on the larger plan finding the paid check locked would be the worst bug to ship in a billing path. That expansion now lives in one module both hosts import, with a test naming each rule, rather than as a copy in each of them.

Fix flow, 79 USD a month, is up to three findings a month fixed for you, each returned as a patch carrying a test that fails before it and passes after. It works from the finding rather than from your repository, so it needs no token, no clone and no repository access, which keeps the promise on the security page intact. Three is the ceiling because three is what one person can do properly in a month, so that is the number on the page instead of the word unlimited.

The pricing page carries all three, checkout is wired per product rather than to one hard coded id, then the terms say exactly what the service delivers plus what happens when it is late.

Seven false positives found on a real repository, then fixed

The scan was run over five public repositories to replace a precision claim nobody could reproduce. It reported 17 findings on fastify at 39e87e8. Seven of them were wrong: the tests in test/trust-proxy.test.js declare t.plan(11) then assert through a helper that takes the test context. A planned count fails the test when it comes up short, so a body carrying one cannot be hollow.

no-assertion now counts a declared assertion count as an assertion, plus any helper handed the test context. Both directions are tested. fastify reports 10 rather than 17. The six tests that remain each name a file and a line you can open.

The finding itself was overclaiming too. It said a test with no assertion "cannot fail", which is wrong for a test whose failure mode is an exception. It now says nothing but a thrown error can fail it, which is what is actually true.

The whole run, with every commit, is on the proof page. It replaces the older 132 to 0 precision table, which was true but named none of its repositories, so nobody could repeat it.

First published release, plus a site instead of a page

On npm as margyn-scan, so npx margyn-scan /path/to/repo resolves for anyone. npm refuses the name margyn as too close to an existing package called morgan, so the package carries a suffix while the command it installs stays margyn. 0.1.0 went out first then was replaced within the hour, because its README still printed the name we turned out not to be allowed. The tarball carries src, the README and the licence, nothing else.

The CLI grew --help, --version and --max. The last one is what makes "raise the cap" a sentence you can act on rather than an option only reachable by importing the checker.

Two pages scrolled sideways on a phone. Measured in a headless browser at 360 and 390 pixels: the docs grid took its width from the 680 pixel prose measure, then the pricing card took its width from the nowrap install command. Both are fixed, both were re-measured, so all eight pages now fit a 360 pixel screen.

The pricing page now says which Tiun environment its buttons are wired to. While the live account finishes onboarding, checkout runs in the sandbox, so the page says that where someone is about to press a buy button rather than leaving them to find out.

The site went from one page to eight, all built through one shell: docs, pricing, changelog, security, terms, privacy and a real 404. Paths are served without the .html. Both sitemap.xml and robots.txt are generated from the same page list, so a new page cannot be shipped and left out of the sitemap.

Corrected our own self audit. The home page had named bin/bundle-static.mjs as a surviving mutant, from a run against an older tree. That file no longer carries a line this checker knows how to invert, because the line it inverted went away when the bundler was rewritten. At the default cap of four, four of four now survive. The suite is 41 tests, not the 26 the older copy claimed.

Live on margyn.xyz, with the scanner deliberately left out

The hosted half runs on Cloudflare Workers: the page, sign in, checkout and the licence endpoint. There is no /api/scan in production. On a public host that route would take a filesystem path from a stranger and run git against it.

The mutation proof became a paid check behind an Ed25519 licence that the CLI verifies offline. A refusal prints the reason then runs the free scan in full, so billing can never fail a build. The Worker signs with WebCrypto, the local server signs with node:crypto. A test asserts the two produce identical bytes rather than merely both being valid.

The product was renamed from Placebo to Margyn. The palette became a gate: every text pair is measured against WCAG in both colour schemes and the build fails on a miss.

Five checks, written from a pull request that went red

Eight vendored modules sat under a path containing dist/, which the root ignore file excludes at any depth, so git dropped all eight while they sat on disk. The first two checks came straight out of that failure. Then three more, from the same question: what else in this repository reads as verification and performs none?

Then the precision pass, because a scanner that cries wolf is hollow itself. The first run produced 132 findings on one repository and nearly all were wrong. Four fixes later the same five repositories gave 0, 2, 2, 2 and 0 findings. The eight that remain are true.