Evidence of AI training: what to measure and what to keep
Evidence of AI training: tell participation from knowledge, grade question by question and per skill, and keep a record that helps with Article 4.
General information, not legal advice. This page helps you organise the record of a training programme. It does not say what an authority will expect from your organisation: check that with your lawyer or adviser.
Training your teams on AI is one thing; being able to show what was done, and what people know afterwards, is another. This page explains what a useful training record contains, and how Maketools produces one.
Why keep a record?
Article 4 of the AI Act asks organisations that provide or use AI systems to take measures to develop their staff's AI literacy. The European Commission says no certificate is required and that an organisation can keep an internal record of the training and other actions it carries out. Beyond Article 4, a record mostly helps you steer: knowing who has been trained, on what, and where teams still get it wrong.
Participation and knowledge: two different measures
Many programmes only measure participation: enrolments, time spent, modules completed. That is useful for coverage, but it says nothing about what people know. A solid training record tells apart:
| Measure | What it tells you | What it does not tell you |
|---|---|---|
| Participation, completion | Who took the training, and when | What the person retained |
| Questionnaire grade | What the person can recognise, question by question | How they behave day to day |
| Game score (speed, streaks, lives) | Skill at the game | Knowledge |
The classic trap of a serious game is to mix the last two: a fast but sloppy player can beat an accurate but slow one. A training grade should only measure accuracy.
How Maketools grades a game
Every catalogue game is a quiz in disguise: whatever the storyline, players move forward by answering questions written in the content, each with one expected right answer and one skill.
- The grade is the questionnaire: the share of questions answered correctly at the first attempt, each question weighing 1 to 5. A question left unanswered (timer ran out) counts as wrong; a play quit before the end gets no grade. Later attempts are for learning, not for the grade.
- It is recalculated by the server from the recorded answers and the play's draw: a score claimed by the browser is ignored, and a play whose questions do not match the draw is refused.
- Game points are kept apart: speed, streaks, lives or altitude earn points shown in the game and on the leaderboard, never in the grade, the pass mark or the LMS.
- Five question types, all graded right or wrong with no partial credit: one answer, several answers, an order, pairs, a value in a range.
What the reports show
The report of each deployed game gives participation, completion, average score, pass rate and the share of right answers per skill, with a group filter. For a quiz-format game, it adds a per-question table:
- the first-attempt success rate of each question;
- the number of answers (a question left to the timer counts as wrong);
- the jokers and hints used;
- the three most chosen wrong answers, which reveal your teams' misconceptions.
That is the most useful part for improving a programme: a question almost everyone misses points to a reflex to work on again, or to a badly written question. See read a game's statistics.
By default, reports are pseudonymised and a group of fewer than 5 players is never broken down: they are enough to show a programme's coverage without naming anyone. The named mode can be turned on if your organisation needs it (see player privacy). Reports can be exported as CSV and PDF.
In your LMS or LRS
If your training record lives in an LMS, a SCORM 1.2 or 2004 package writes the grade, the status ("incomplete" as soon as it opens, then passed or failed), one interaction per question and one objective per skill, always from the answers graded by the server. The xAPI connector sends the same information to your LRS. See LMS integration.
Keeping the record comparable over time
- A fork publishes frozen releases: players always play a precise version of the content.
- Before you publish a version that changes the questions of a game shared in an LMS, the studio warns you: the next learners will get other questions.
- The per-question report groups questions by major version of the game. During a campaign, keep the questions stable so you can compare groups.
What a quiz does not prove
A grade measures what a person can recognise in situations close to their work, not what they will do alone in front of a real case. Add clear rules (a short charter), team discussions and, for exposed roles, more in-depth support. Maketools issues no certificate and does not replace your compliance analysis.
Where to start?
Play a session of AI Summit or AI or Not AI?, no account needed, then see what the 30-day plan suggests for a first programme.