SimpleAudit

Simula's open AI safety auditing framework that allows you to test your AI systems and find their weak spots.

Simpleaudit is recognised as a Digital Public Good. This confirms that the solution meets international standards for transparency, accountability, and its contribution to sustainable development. SimpleAudit is also one of the tools selected for the DPGs for AI (DPG4AI) Collection—a curated portfolio of DPGs designed to drive responsible, equitable, and localised AI development.

The framework was among the three finalists to "Fyrlyktsprisen", presented during the NOKIOS conference for ICT in public sector in 2026:

Today’s large language models are powerful, but they also behave unpredictably. They may follow instructions in one moment and ignore them the next, or provide confident explanations even when their answers are wrong. For organisations that rely on accuracy, accountability and audit trails this is a serious challenge.

The systems are guided through “prompts”. They are flexible but fragile, difficult to control, hard to document, and almost impossible to verify.

A research-driven instruction schema has been validated and integrated into an operational environment. In this environment, structured AI governance moves from a theoretical concept to something that works in practice. 

Instead of describing system behaviour in plain text, this framework defines instructions as structured, versioned configuration. Making the behaviour more predictable, testable, and easier to audit.

This is done through a Large Language Model (LLM) Instruction Schema Standard (LISS) paired with a Per-session Instruction Schema (PSIS).

It can be compared to how web development became more disciplined when structured standards like CSS replaced ad-hoc formatting.

Available on GitHub: https://github.com/kelkalot/simpleaudit

Contributors

• Michael A. Riegler (Simula)
• Sushant Gautam (SimulaMet)
• Klas H. Pettersen (SimulaMet)