Credentum
"Truth, remembered."
Telemetry and evals for AI agents, where gates catch claims that don't match reality.
Why We Exist
AI agents report on their own work. "Tests pass." "Fixed." "Deployed."
Sometimes the report and the record disagree.
Our approach: telemetry that records what an agent actually did, and evals that check what it says against that record. The check is a gate the agent can't reach, so a claim that doesn't match reality gets caught instead of believed.
Not because we don't trust agents.
Because trust requires evidence.
Get research findings first
New studies and findings before they go public. No spam.
Research updates only. Unsubscribe anytime.
Our Approach
Where our work is headed. Two pieces that only work together: a record of what happened, and a check that holds the agent's claims to it.
Telemetry
A record of what the agent did: what it said, what it ran, what came back. Written somewhere the agent can't edit, so it can be checked later.
Evals with gates
Checks that compare an agent's claim with the record. "All tests pass" only counts if the run log says so. The gate sits outside the agent's reach, and its pass bar is set before the results come in.
Open-Source Tools
Tools we've published for AO/Arweave developers and their AI assistants.
ao-lens
Static analysis and security auditing for AO processes. Tree-sitter parser, 25+ built-in checks, 20 community detection rules. Catches nil guard bypasses, determinism violations, missing auth, unsafe JSON, and state-wipe-on-replay bugs.
Vivarium Lab
Research on AI agents, evals and reliability. Real experiments. Honest results.
The Verifiability Seam
What happens to an eval when the judges have a stake in the verdict. Output-only signals, thresholds fixed before any data. The first pilot was inconclusive on its main question; on clear items, staked panels flipped to the wrong answer 9 times out of 12. In progress.
Governance as Computation
In a multi-agent commons dilemma, giving agents the pre-computed answer — not more information, not moral prompts — is what prevents collapse. 80% vs 0% survival. Preliminary (N=5), full data included.
Agent Bootstrap
160 words of flat navigational anchors beat 1,000 words of memory. Four Laws of Agentic Context from 174 trials.
Movable Feast
LLMs know lunar holidays but can't find them on a calendar. We found a 0% to 100% accuracy gap across frontier models.
Persona Skills
What we learned breaking Claude's skill system. 81% routing reliability, and where it fails completely.
We share research findings, tool updates, and honest takes on AI. No hype.