Security reviews: development testing is not an audit

Two different questions

Tests are written from the specification by the people who built the thing. They encode the author's model of how the system works, and they are excellent at catching regressions against that model.

An audit attacks the model itself. It asks what happens under conditions nobody specified: reentrancy, unexpected call ordering, an oracle returning something implausible, a dependency behaving differently than documented, an economic incentive nobody priced.

The blind spot is structural rather than a matter of diligence. A team cannot test for the case it did not conceive of, and the cases that matter are precisely the ones outside the author's model. That is why independence is the point, not thoroughness.

So a supplier saying their contracts are thoroughly tested has said something true and answered a different question.

• Tests check the system against its author's model
• An audit attacks the model
• Independence is the mechanism, not extra effort

What an audit actually involves

Scoping and a threat model: what is in scope, what the assets are, who the adversaries are, and what assumptions the system makes about its dependencies.

Manual review, which is where the findings that matter come from. Automated tooling catches known classes and misses the logic errors and economic flaws that cause the large losses.

A findings report with severities, and then the part teams forget to budget for: remediation and re-review. A finding is not resolved when it is acknowledged, and a fix introduced under deadline pressure is exactly the kind of change that deserves another look.

Plan calendar time for all of it. An audit that returns findings a week before launch produces either a delayed launch or an unremediated finding, and both are decisions somebody should make deliberately rather than by accident.

Planning it as a line item

It is a separate scope with its own cost, performed by reviewers qualified for the chain and the contract in question. It should be named in the proposal as a distinct item rather than implied as part of development.

Freeze the code before the audit starts. Auditing a moving target produces a report about something that no longer exists, and re-auditing the changes costs more than waiting would have.

Book it early. Good reviewers have lead times measured in weeks, and the availability question should be settled during planning rather than discovered at the end.

And decide in advance what severity of finding blocks the launch. Making that decision under launch pressure, with a finding in front of you, is how a critical becomes a known issue.

• A named line item, not an implication
• Freeze the code before review starts
• Book reviewers during planning, not at the end
• Agree the blocking severity before you have a finding

What an audit does not give you

It is not a guarantee, and a report is not insurance. It is a point-in-time review of a specific version by specific people, and audited protocols have been exploited.

It does not cover what changed afterwards. A contract deployed with a modification made after the report is unaudited code with an audit badge, which is worse than no badge because it borrows confidence it has not earned.

It rarely covers the off-chain surface: the keys, the deployment pipeline, the admin functions, the backend that holds the interface. Those need their own review, and they are where a great many real incidents actually start.

And it says nothing about economic design. A contract that is correct and a mechanism that is exploitable are entirely compatible.

How we scope this

We do not present development testing as an audit, and we will say so in writing. Our work covers architecture, the on-chain and off-chain boundary, contract integration, transaction states and the testing that belongs to a build.

Independent auditing, threat analysis, remediation and re-review are defined separately in the project scope and performed by appropriately qualified reviewers. Where a project warrants one, we plan for the calendar time it needs including remediation.

Much of what we build reduces the contract surface deliberately. Wen runs on embedded wallets the backend never holds. Axon Wallet lets a model propose a typed intent that passes schema validation, policy limits and a two-factor gate before anything reaches a chain. Trade Rewards puts an admin approval between a points ledger and the hot wallet. Less on-chain logic and tighter off-chain limits is a smaller thing to get wrong.