Results of our measurements
Every antivirus vendor claims that it detects. Almost none publish the number that really matters to a merchant: how many times has it been wrong about a perfectly healthy site? Here are ours, along with the protocol and the commands to reproduce them.
0,0 %
false positives
Staying quiet when there is nothing to say. It's the metric no vendor publishes, and it's the one that decides whether you'll still be reading our alerts six months from now.
19/19
implants detected
Speaking up when there is something to say. The implants are placed inside a real CMS core, not in some sample file.
These two numbers do not measure the same thing, and neither replaces the other. A tool that accuses everyone shows perfect detection — and no one can use it. A tool that never says anything has no false positives. It is from holding both at once that a product derives its value.
27 real sites, chosen to be difficult
A panel of accommodating sites would prove nothing. This one is chosen against us: from ad-saturated media outlets to security sites that publish attack code in the clear — it's precisely this last family that tripped us up, and you'll read how below.
Press and media
the most ad networks and trackers per page on the web
lemonde.fr · theguardian.com · bbc.com · edition.cnn.com · nytimes.com · liberation.fr
Commerce
the product's real-world terrain: funnels, marketing tags, consent banners
fnac.com · cdiscount.com · leboncoin.fr · laredoute.fr · decathlon.fr · shopify.com · prestashop.com · woocommerce.com
Public services
the other extreme, austere, where a false positive would stand out glaringly
service-public.fr · gouvernement.fr · economie.gouv.fr · cnil.fr
Documentation
sites that publish code on every page
developer.mozilla.org · stackoverflow.com · php.net · wordpress.org · github.com
Security
they QUOTE attack code, display webshells, recount incidents
wordfence.com · blog.sucuri.net · cert.ssi.gouv.fr · krebsonsecurity.com
Three rules of honesty underpin this figure, without which it would be worthless. A site unreachable is not a healthy site: it drops out of the calculation and it is named. The rate is calculated on the sites actually analysed, never on the starting list. And hardening recommendations — a missing security header, for example — don't count: they advise, they accuse no one.
28,972 genuine PHP files, zero accusations
The second real-world condition no longer looks at pages, but at files — the business of the paid tier. It runs on code that nobody wrote for us: the official WordPress archive, and the core of a real PrestaShop store in production, with its modules and its legacy code.
| Corpus | PHP files read | Wrongly accused |
|---|---|---|
| WordPress 6.5.2, official archive | 1 149 | 0 |
| A PrestaShop 9 store in production | 27 823 | 0 |
| Total | 28 972 | 0 |
The second corpus is the hardest one in existence here: a complete PrestaShop core, that is to say precisely the place where perfectly legitimate code spends all day doing what a malicious program would do.
We look for our own mistakes before a customer finds them
A real-world false-positive condition is only worth anything if it catches us out. This one did: it exposed two ways in which we were accusing perfectly honest code, ways that our automated tests of the time failed to see. Here they are, and they are closed.
A public key is recognised as such
Looking for "a long unreadable string" and "something to decrypt it" in the same file proves nothing: on a modern site, both are true at all times. A public key, a browser trial token, a consent banner's configuration all look like it. From now on, what accuses is no longer the meeting of the two — it's that the result of the decryption be executed.
A cookie sent to an ad network is recognised as legitimate
Sending a cookie to a third party is the daily business of advertising and audience measurement. Extending a list of trusted providers would have changed nothing: a list is always one player behind. It is now what is carried away that decides — a piece of data entered by a customer remains an accusation, a cookie no longer is one.
Both fixes were measured in both directions: silence returned on the 27 sites, and detection hasn't budged by a single implant — 19 out of 19. That's the condition we impose on ourselves: a softening that would cost us a detection wouldn't be a fix, it would be a cover-up.
The protocol is in the product
Both real-world conditions are software commands. The site panel, the corpora, the counting rules: everything is readable, and everything can be replayed.
npm run conditions-reelles:faux-positifs
Replays the 27 real sites and returns the rate, naming each unreachable site.
npm run conditions-reelles
Plants the backdoors in an official tree and measures both numbers on the same tree.
Added to these are 1,535 automated tests, one entire class of which exists for a single reason: to verify that a harmless addition to a site carrying implants never makes a finding disappear.
The scope of a measurement is part of the measurement
A measurement without its scope always ends up being misread — and it's whoever published it who gets blamed for letting it happen.
- Those 27 sites are a panel, not the web. A site doing something we have never seen could produce a false positive: the measured rate says what we observed, not what cannot happen.
- The external analysis, with no server access, sees only what the page shows. It does not replace the server-side analysis — and it's the latter that the 19/19 and the 28,972 files measure.
- The page executed in an isolated browser was measured separately, on six major ad-funded sites: no accusation, an inventory only. What these sites contact without announcing it is named; only a destination's reputation, or a password page, accuses.
- "Justified" does not mean "safe". A site where every file justifies its presence can still have a known vulnerability, a stolen password, or a malicious extension signed by its own vendor.
- Restoration is proven on official trees, not on client sites: it is the publisher’s version that is rewritten, fingerprint checked before writing and read back after. A file with no publisher — your own code — is never rewritten.
- An implant is written by us. A real-world condition only measures what its corpus contains — which is why we publish the corpus and the command, so that yours can prove us wrong.
Eleven systems returned to the publisher’s state, proven end to end
Knowing a file has been altered is useless if you can’t set it right again. On each of these eleven systems, we graft a backdoor into a core file of an official installation, then we measure the whole cycle: the file is recognised as modified, reinstated to the version published by its editor, and read back identical — byte for byte.
From the lightest to the heaviest, from 4,943 to 40,199 files depending on the system, each proof replays the same chain, and every time the file returns exactly to what the publisher ships. No other tool restores across so many systems, and none proves it.
What the analysis can recognize, in numbers
These three numbers are counted in the code, and recounted with every release. The first two are the ones that matter least: detection rests on file provenance, never on a catalogue.
- 19
- named families
- WSO, c99, China Chopper, WP-VCD, AnonymousFox… named AFTER being found without them
- 25
- behaviour signatures
- each describes a behaviour, never a file — with its severity, its pattern and its effect
- 1 535
- automated tests
- the complete test suite of the analysis — green, and automatically recounted before every release
How many files on your site can justify their presence?
You'll get the exact number, every file named, and the means to prove it to someone else.