AI governance
AI model audit
The model decides. Someone has to be able to explain why.
What this work consists of
We examine the model on two fronts that people often confuse: whether it treats different groups fairly, and whether anyone can explain a specific decision when the person affected asks. A model can be fair and incomprehensible, or perfectly explainable and biased. They are different examinations, and you choose one or both.
The work starts by putting the scope in writing: which model, which decisions, which groups to compare. We then test with evaluation data: performance by group, behavior on the hard cases, and the ability to reconstruct an individual decision. The result comes as a report with an action plan, and the full option adds the executive report and the presentation to leadership.
We need access to the model or its outputs, whatever development documentation exists, and your data team available to answer questions. We audit the model as it arrives: fixing what we find stays with whoever built it, and our action plan says where to start.
The backdrop is legal, not just technical. When AI makes decisions about people, the LGPD gives them the right to request a review, and answering takes more than saying the system is good. The model card and the reports are the material that supports that answer before a client, a regulator, or a data subject.
How we conduct it, stage by stage
The stages and deliverables below describe the Both modality. The other modalities appear when you request the proposal.
Scope definition
We agree in writing what is in and what is out, and why. A badly defined scope is the most common cause of a project running over.
Evidence gathering
We collect what already exists: documents, configurations and logs. We start from what the company has, rather than from a blank form.
Model assessment
We test the model against evidence data: performance by group, behaviour in the hard cases and the ability to explain a specific decision.
Report
We consolidate the findings into a report where every item comes with severity, evidence and the path to fix it. We write to be read by the people who will act, not to fatten pages.
Presentation
A meeting with leadership translating the technical result into business risk and investment decisions. We arrive with the answers to the questions the board always asks: what to attack first, how much effort it takes and what happens if nothing is done.
What is not included
- Fixing or retraining the model, which stays with your data team
- Labeling or collecting test data, which only enters if agreed separately
- Any guarantee of results: we audit the model as it is, not as it could be
- Continuous model monitoring after the audit: we examine a snapshot in time, and tracking what changes is your team's routine
- Adversarial attacks on the system, trying to make it act outside its intended behavior, which is work of a different nature
- Legal opinions on the automated decision, which stay with the law firm you hire
- The direct response to a data subject who requested a review, which is the controller's role: we deliver the material that supports that response
Usually comes together with
Not a bundle, and it changes nothing you have already chosen. It is what tends to come up next, in the experience of companies that have been through this.