I've been doing training on ISO 42001 and some of the other AI-related standards, and building a public app with AI functionality at the same time. So, I thought I'd share some of my approach so far to building governance in from the start of this tech project, at ground level, rather than (more typically) the top down. This approach is a work-in-progress and I’ve summarized it as a checklist. The various AI-governance standards are all relatively new, so I'm working a lot of it out as I go.
Two things before the list. First, some of this will look familiar to any software team. Decision records, for example, are really just a tidied-up version of something good engineering teams already do. Other bits, like writing down the societal impacts, are newer. Second, nearly everything here can be automated for creation – but some items require manual approval. These don't have to be documents you write up by hand at the end of a long day. Most of them can be produced as a side effect of your work (which is the point really).
Note: I'm one person, this is a small non-profit tool, and I've tried hard not to do anything I don't need to i.e. “governance theater”. Some of this might be overkill for what you're building. But, as always, read the standards!
(These are not in any order. )
☐ Write down decisions as you make them.
Your dev team may already half-do this. Architecture decision records - short notes on why you picked one tool or approach over the other - are a normal habit in good engineering teams. I keep the same kind of record for the governance calls: who decided, what we decided, and the evidence it rested on. The part I've added is a section listing the specific things that would make me revisit it later (what would trigger a revisit of this decision?), things like: a supplier changing its terms, a new model version, fresh guidance from a regulator. Plus, a date to review the record again regardless. And a lot of this can be automated: a commit or a release can open the empty record and prompt you to fill in the why.
☐ Review what you're shipping before it goes out.
Before a release I go through a single sign-off sheet: a checklist of everything that happened that release: intended use, data provenance, citation accuracy, refusals, the red-team results, drift results, accessibility, how names and personal details are handled, the roll-back plan. And, for each one, I mark it as met, deferred to a later version, or shipping as a known risk I've decided to accept. Then it's a clear go, no-go, or go-with-risks accepted, and I sign it. That's what the record is really for: shipping becomes a deliberate decision with a written trail of what went out, what I held back, and which gaps I launched with on purpose and why. A lot of the sheet fills itself in from the test runs, release tags and so on; add the judgment calls.
☐ Work out who this affects in terms of individuals, groups, and society.
This is new, and the least like normal software work. In the past, reviews for privacy, security, etc. mostly ask whether you could harm an individual user, an organization. AI governance adds a wider question: what does this do to individuals, groups of people, and to society? So before launch I wrote down the foreseeable effects for my app at all three levels. For individuals: the tool surfaces the names of people already in the public domain, someone might act on a wrong answer, the chat interface isn't fully accessible, it's English-only, so it leaves non-English speakers out. The societal ones were harder. My tool answers civic questions so a confidently wrong summary of a vote could shape how residents understand their local decision-making entities. And if you ask it about “public opinion,” the answer only reflects what is in the public arena already. Note that naming these doesn't make them disappear, but you can't mitigate or honestly disclose an impact you never acknowledged.
☐ Write down the risks you're living with.
This is one of the newer ones. Not just the risks you've fixed or necessarily mitigated against. Here we're also talking about the risks you're shipping with on purpose, knowing they're there. Writing it down turns “we never considered that” into “we considered it, and here's why we were OK with it.” Explainability is the key term to remember here. Document what you shipped with and what you didn’t ship with, and why.
☐ Make the settings you didn't choose visible.
When you build with AI coding tools, vibe-coding, these tools set defaults you never picked. Make all these settings visible on an admin screen, including the system prompt. Once the settings are visible you can decide whether they're right. If you don’t do this, you're basically trying to govern a system you can't see!
☐ Check the claims on your app or website.
An overclaim on your website is a governance problem even if the code behind it is perfect, and it's the part an ordinary person actually reads. Make sure your marketing material matches your governance claims.
☐ Try to break your app every month.
Run it through a big batch of hostile prompts (this is called red teaming; I use an open-source tool called promptfoo for redteaming) against a test copy. “I tested it against a hundred-plus attacks and it held” is a much better thing to be able to say than “the instructions or prompts tell it to behave.”
☐ Export your evidence as you go.
This is a big one, and something that dev teams mightn't realize. My eval results live in Braintrust, an observability tool. It's great for running things, but it isn't a vault — it's a live product owned by someone else, and data in it can move or disappear. Data sitting in these tools is not evidence. I set up a GitHub Actions job that copies each release's evidence out into version control (plain git), with a checksum on every file so I can show nothing's been changed, all tagged with the date. And “evidence” means more than the scores. It's three things: the test sets (the exact questions I asked it), the results of each run, and the graders/judges that did the scoring. My graders are plain code, so the commit that produced the numbers already pins them; if yours only live inside the tool, export those too. It runs on a schedule, so if anyone ever asks what the tool did on a given day, I can show them the whole set, not just tell them.
☐ Check what your vendors do with your data.
Your tool almost certainly sends data to other companies: the model provider, whatever you use for logging or analytics, your host. Before I trusted any of them I checked three things: how long they keep the data, whether they train on it, and whether there's an agreement that says so in writing. For the model provider that meant confirming the inputs aren't used for training and are only held briefly; for the observability tool it meant reading the data-processing terms before piping real user questions into it. You're still on the hook for what a supplier does with your users' data, so “I assumed it was fine” isn't a position you want to be in.
☐ Pin your model version.
The model behind your app can change without you doing anything. I pin the exact model snapshot in code, and a scheduled check confirms the live model is still the one I pinned. It's a one-line change that closes off a whole category of “why did it suddenly start answering differently?” with no answer.
☐ Check for prompt drift.
Run a small, automated test of “golden experiments” against live on a schedule that checks the responses are still right, to catch drift that only shows up in production. Add an alert for this so you know if something goes wrong.
☐ Failure handling: remember the basics and fail safe where you can.
Where you can, move the check to the moment someone actually uses your thing.
· Retries with backoff. Transient errors (the AI API is briefly overloaded) get retried a few times, spaced out, before giving up — but only for errors that are safe and likely to succeed on retry. Don't retry a request that was rejected for a real reason.
· Timeouts. Cap how long you wait, so one stuck request can't hang everything.
· Graceful degradation, and “fail closed.” When it genuinely can't answer, show a friendly “try again” message, not a crash. And the key principle for a citation tool: if it can't reach the source records, it must say so rather than invent an answer.
More often than not, top-down governance asks you to describe the system you wish you had, and then tries to remold the product into that shape. Ground-up governance writes down the one you're actually building, as you build it.
I don't think I've got all of this right. The standards themselves are still young, and this list will likely keep growing as I build out the tool, and I'll probably change my mind on some of it. I expect I’ll publish a revised version of this list later on.