SimpleAudit achieves status as Digital Public Good
Published:
Simula's open AI safety auditing framework, SimpleAudit, has been officially approved as a Digital Public Good by the Digital Public Goods Alliance (DPGA). This approval confirms that the solution meets international standards for transparency, accountability, and its contribution to sustainable development.
The tool becomes the seventh Norwegian digital solution to receive DPG approval.
“We believe safe and secure AI adoption requires sovereign oversight and scalable, real-world testing. SimpleAudit is our contribution, helping society understand AI systems’ limits in practice, so we can build better safeguards and better systems over time,” says Professor Michael A. Riegler, Head of AI at Simula and founder of SimpleAudit.
Developed by researchers Michael Riegler, Klas Pettersen and Sushant Gautam, SimpleAudit is available as open-source software. The framework evaluates critical safety issues that are often overlooked by standard AI benchmarks, such as how models handle harmful requests, maintain necessary boundaries, and provide accurate institutional information.
This allows organizations to test and document model behavior in specific, context-dependent scenarios, such as healthcare, law, and public services.
The tool requires minimal setup and can be run entirely locally (no APIs required), ensuring that no data leaves the user's machine. It supports both open-source and API-based models.
"We live in a time where AI systems are becoming basic infrastructure. Ensuring the safety of these tools is a national and global security imperative," says Klas Pettersen, SimulaMet Director.
"Norway has ambitious policies regarding AI implementation in the public sector, and Simula’s goal is to build a foundation of trust for this next generation of digital infrastructure. SimpleAudit achieving Digital Public Good (DPG) approval is a vital step in that direction."
SimpleAudit is currently being used in research evaluating the safety of several large language models in Norwegian social and administrative contexts. The study revealed that models with high performance on standard language benchmarks are not necessarily the safest in practical use, a finding the researchers refer to as a "paradox of understanding."
Digital Public Goods (DPG)
The Digital Public Goods Alliance (DPGA) is a UN-backed global initiative promoting digital public goods to reach the UN Sustainable Development Goals. To be approved and included in the global registry, a solution must meet nine criteria. These ensure the technology is open-source, protects privacy, and follows safety and security best practices.
Read more about the DPGA here: https://www.digitalpublicgoods.net/
References
SimpleAudit listed at Digitalpublicgoods.net
Safety Evaluation of Language Models for Norwegian Deployment