A participant gives a training course 9 out of 10. Can we conclude that they now master the skill? No. Can we say they will use it correctly at work? Not either. That score says something useful — their immediate experience — but it answers neither the question of learning, nor that of transfer, nor that of performance.
This confusion is common in training dashboards. The same questionnaire is used to justify teaching quality, operational usefulness and sometimes even return on investment. Yet a measure is only relevant if it matches the decision to be made. Should the facilitation be improved? An acquired skill validated? An obstacle on the job removed? A person authorised to carry out a sensitive task? Each question calls for different evidence.
For an HR director or L&D manager, the point is therefore not to choose between hot evaluation, cold training evaluation and on-the-job evaluation. It is to combine them without duplication, with an effort proportionate to the risk and the value of the skill.
Why Excellent Satisfaction Does Not Prove a Training Course Is Effective
A course can be enjoyable without being demanding. It can also be very useful while generating average satisfaction, because it confronts participants with mistakes, a change of practice or a real cognitive load. Satisfaction and learning are linked to experience, but they are not interchangeable.
Before designing a questionnaire, four objects of measurement must be separated:
- Experience: perceived quality, clarity, pace, accessibility, anticipated usefulness and material conditions.
- Learning outcomes: what the person knows, understands or can do at the end of the programme.
- Transfer: what the person reuses after returning to work, under what conditions and with what obstacles.
- Performance: what is actually observable in a professional task, possibly supplemented by an operational indicator.
A fifth question may concern the impact on the business — lead times, quality, customer satisfaction, revenue, incidents, productivity. It calls for extra caution, however: a business change rarely depends on training alone. A plausible contribution must be distinguished from demonstrated causality.
What Hot Evaluation Really Measures
Hot evaluation is administered immediately after a session or within 24 to 48 hours. It captures a still-precise memory of the experience. That is its main advantage and its main limit.
What it measures well
- Understanding of the stated objectives and the perceived coherence of the programme.
- The quality of facilitation, materials, pace and interactions.
- The perceived relevance of examples and the intention to use what was learned.
- Concrete irritants: access, duration, instructions, a level that is too easy or too hard.
- A warning signal allowing a quick correction before the next session.
What it does not prove
A statement such as “I feel capable” does not replace a successful task. Likewise, recommending the course demonstrates neither retention after three weeks nor use on the job. Participants may overestimate or underestimate their mastery, want to please the trainer, or answer quickly to close the session.
The associated decision
Use hot evaluation to improve the experience, diagnose a design or facilitation problem and spot an immediate need for support. Do not use it alone to decide that a critical skill is mastered.
What Learning Assessment Really Measures
Learning assessment is not always cited in the “hot, cold, on-the-job” trio, but it is essential to avoid a blind spot. During or at the end of the programme, it checks whether the participant reaches the learning objective under the planned conditions.
A multiple-choice quiz suits checking a rule, a reference point or a procedure. It becomes insufficient if the objective is to conduct an interview, handle a complaint in English, produce a reliable analysis with AI or arbitrate a management situation. In those cases the evidence must resemble the expected action.
Three conditions make this measure credible:
- A task aligned with the objective: explaining is not enough if execution is expected.
- Observable criteria announced in advance: accuracy, autonomy, safety, quality of argumentation, compliance with a process.
- A defined pass threshold: what is acceptable, what requires a retake and what still rules out autonomy.
The associated decision is to validate a skill, prescribe remediation or authorise progression through the programme. This measure remains evidence produced in a training setting; it does not yet guarantee use in real work.
What Cold Training Evaluation Really Measures
Cold evaluation takes place after a period back at work, often between four and twelve weeks. Its value is to shift the question: no longer “did I enjoy it?” but “what did I really use, in which situation and with what perceived result?”
What it measures well
- The declared frequency of use of a practice or tool.
- Perceived usefulness in specific professional situations.
- Whether confidence or ease is maintained after several weeks.
- Obstacles to transfer: lack of opportunity, lack of time, unavailable tool, incompatible process, manager not involved.
- The need for a refresher, extra practice, mentoring or coaching.
A solid cold questionnaire covers in particular perceived progress, ease with the targeted task, fit with real activity, development of new skills, whether expectations were met, relevance to the job, recommendation and actual use of what was learned. These themes link the learning experience to declared use.
What it does not prove
Cold evaluation generally remains self-reported. Memory reconstructs experience; respondents may credit the training with progress due to other factors or, conversely, overlook a change that has become routine. A positive answer is therefore not enough to certify the quality of execution.
It measures return on investment even less on its own. To link training to a business indicator, you need at least a starting point, a comparison period and an examination of the other factors that changed.
The associated decision
Use cold evaluation to decide on a refresher, a community of practice, managerial support, a change to the programme or the removal of an organisational obstacle. It is particularly useful when use depends on the opportunities the job offers.
Which timing should you choose?
There is no universal timing. The right moment is the first point at which use should reasonably be observable: two to four weeks for a frequent practice, six to eight weeks for a skill used monthly, longer for a rare task. Set this timing at design stage, based on the real rhythm of the work.
What On-the-Job Evaluation Really Measures
On-the-job evaluation observes a person carrying out an authentic task or a sufficiently realistic simulation. It produces more direct evidence than a questionnaire: behaviour, deliverable, decision, compliance with a procedure, quality of an exchange or ability to correct an error.
What it measures well
- The simultaneous use of knowledge, actions, judgement and communication.
- The quality of execution in a context close to real work.
- The level of autonomy and the need for assistance.
- Compliance with sensitive criteria: compliance, safety, confidentiality, customer quality.
- The ability to adapt to the unexpected or to explain one’s reasoning.
Its conditions of validity
Observing is not enough. Two managers may judge the same performance differently if criteria are vague. An effective grid describes three to five visible behaviours or results, sets an expected level and states the acceptable evidence. For a critical skill, evaluator calibration is necessary.
The situation must also be fair. Do participants have access to the same resources? Is the difficulty comparable? Does the manager’s presence change behaviour? Is the data collected strictly necessary? These questions prevent a good intention from turning into a contestable system.
The associated decision
Use on-the-job evaluation to confirm operational readiness, target coaching, grant an internal authorisation or check that a quality standard is applied. It is the most relevant method when an error is costly or risky.
Decision Table: Which Method for Which Question?
| Question to address | Suitable method |
|---|---|
| — | — |
| Improve pace, materials or facilitation | Hot evaluation |
| Check that a rule, method or skill has been acquired | Learning assessment aligned with the objective |
| Find out whether the practice is reused and what blocks it | Cold training evaluation |
| Confirm the quality of real execution | Criteria-based observation or simulation |
| Estimate an effect on the business | Contextualised operational data compared with a starting situation |
| Decide on individual support | Combine self-assessment, task result and observation |
This logic avoids two symmetrical mistakes: asking a questionnaire for what it cannot prove, or deploying a costly observation when a simple experience signal is enough for the decision.
How to Make Your Evaluation Tools More Reliable
If you already have a cold evaluation grid, you have a working base. To turn it into a steering system, five improvements are a priority.
1. Separate satisfaction, use and results
Group questions by object. An “experience” block must not be aggregated with a “use” block. Display distinct indicators: satisfaction, declared achievement of objectives, frequency of use, obstacles and need for support.
2. Stabilise the scales
Choose a single scale, define each end and keep it from one campaign to the next. Mixing a 1-to-4 scale and a 1-to-5 scale makes comparisons ambiguous.
3. Ask for an example, not just an opinion
After “did you use what you learned?”, add: “Describe a recent situation, the action you took and the result you obtained.” A factual answer gives more material for follow-up than general agreement.
4. Involve the manager without creating a bureaucracy
The manager does not need a twenty-question form. A short check can cover the opportunity given to practise, an observable behaviour and the next challenge. Managerial involvement is particularly useful for removing obstacles the participant does not control.
5. Link every question to a decision
For each item, complete the sentence: “If the result is low, we will…”. If no action is possible, the question is probably decorative. This rule shortens questionnaires and increases how much of them gets used.
Hot and Cold Evaluation: A Minimal Five-Step Protocol
A robust system does not need to be heavy. Here is a minimal architecture adaptable to training in professional English, management, sales, digital skills or artificial intelligence.
Before the training: define a reference task
Describe the expected professional situation, the observable result and two to five criteria. When possible, have participants perform a comparable initial task. This baseline makes it possible to talk about progress rather than mere impression.
During and at the end: check learning outcomes
Collect light evidence: a reasoned decision, a production, a demonstration, a role play, a case resolution or a targeted test. The format must match the verb of the objective: identify, apply, produce, negotiate, secure, diagnose.
Hot: correct quickly
Limit the questionnaire to six or eight useful questions plus an open field. Analyse experience, anticipated usefulness and irritants separately. Provide an action rule, for example a systematic review if an item falls below a defined threshold.
Cold: document use and obstacles
Send a short survey at the moment the skill should have been used. Ask for a frequency, a concrete example, a perceived result, an obstacle and the support wanted. A single reminder is often more effective than an overly long questionnaire.
On the job: confirm performance
Select the skills for which direct evidence really changes a decision. Organise a real observation, a simulation or a review of a deliverable. Keep only what is necessary, share the criteria and give usable feedback.
Three Operational Examples
Example 1 — Handling a customer complaint in English
Hot, the participant rates the clarity of the scenarios and their perceived ease. At the end of the programme, they handle a simulation with four criteria: listening, rephrasing, explaining the solution and confirming the agreement. Six weeks later, cold evaluation asks whether they handled a real complaint, what helped and where they still hesitated. If the stakes justify it, supervised listening or an analysis of a report confirms the quality of the practice.
Example 2 — Using generative AI without exposing sensitive data
A quiz checks the basic rules, but the skill is demonstrated in a task: rephrasing a need, preparing the data, applying a confidentiality rule, checking the output and documenting the decision. Cold, you measure frequency of use and blockages. On the job, a case deliberately containing a sensitive piece of data checks that the participant can spot it and choose an alternative. To frame these uses, see our analysis of what the AI Act really requires of companies.
Example 3 — Holding a difficult feedback conversation
Satisfaction does not allow us to conclude that a manager will be able to conduct the conversation. A simulation observes how facts are stated, listening, the explicit request and agreement on next steps. Cold, the manager describes a conversation they actually held and the obstacle they met. A brief follow-up with their own manager can confirm a change of practice without recording unnecessary personal data.
Building a Dashboard That Does Not Mix Types of Evidence
A useful L&D dashboard preserves the nature of each measure. Avoid an overall score that adds satisfaction, test success and declared use: it would be easy to read but impossible to interpret.
- Experience: response rate, satisfaction by dimension, recurring irritants.
- Learning outcomes: achievement rate per objective, distribution of levels, remediation needed.
- Declared transfer: frequency of use, examples provided, obstacles and support requested.
- Observed performance: criteria achievement rate, level of autonomy, critical errors.
- Activity: one or two operational indicators linked to the use case, with context and comparison period.
Add a “decision” column and an owner. An indicator with no owner or action threshold almost always ends up in reporting with no effect. This approach fits naturally into a results-driven skills development plan.
The Biases to Anticipate
- Desirability bias: the participant gives the answer they think is expected. Reduce it with neutral wording and by explaining how answers will be used.
- Memory bias: several weeks later, the memory is incomplete. Ask for a recent example and a precise period.
- Opportunity bias: a skill not used is not necessarily forgotten; the situation may not have arisen. Measure the opportunity to practise.
- Observer bias: the manager assesses according to their preferences. Use visible criteria and calibrate evaluators.
- Attribution bias: an operational improvement may come from a new tool, seasonality or a change of team. Document concurrent factors.
- Selection bias: only the most engaged participants answer cold. Track the response rate and avoid absolute conclusions.
Qualiopi: What Evaluation Must Demonstrate
The Qualiopi national framework (the French quality certification for training providers) distinguishes evaluating the achievement of objectives from simply collecting opinions. In the historical numbering of the reading guide, indicator 11 requires the provider to evaluate how far beneficiaries achieve the objectives of the service. In other words, a satisfaction questionnaire is not enough when you want to produce evidence of acquisition.
Decree no. 2026-728 of 1 August 2026 updates the national quality framework for actions contributing to skills development, with application planned from 1 November 2026. When preparing an audit or changing a quality procedure, you should therefore check the version of the framework and reading guide in force, rather than reproduce an old questionnaire template.
For the client company, the aim goes beyond documentary compliance. The system must make it possible to decide: should the programme be changed, practice reinforced, the manager mobilised, more opportunities given to use the skill, or autonomy confirmed?
Express Workshop: Audit Your System in 30 Minutes
Choose a priority training course and gather the L&D manager, a manager and, if possible, a former participant. Answer five questions.
- Which decision must be made thanks to the evaluation?
- Which professional behaviour or result must become observable?
- What minimal evidence would be sufficient and credible?
- At what moment can this evidence appear?
- Who can act if the result is insufficient?
Then assign each existing question to one of the four objects: experience, learning, transfer or performance. Remove duplicates, isolate indicators, then add the missing evidence. The result should fit on one page: objective, method, timing, owner, threshold and action.
FAQ — Training Evaluation
When should a cold evaluation be sent?
When the skill should have been used. For a weekly practice, four to six weeks is often enough. For a monthly or rare task, wait longer. The timing must follow the reality of the job, not a uniform administrative rule.
How many questions should it include?
Eight questions tied to decisions are better than thirty generic questions. A cold survey can combine three or four scales, two factual questions and an open field. The announced length must remain credible.
Should cold evaluation be anonymous?
It depends on the objective. Anonymity encourages honest feedback on the programme but prevents individual support. If answers are named, clearly inform participants of the purpose, the recipients and the retention period, then limit data to what is necessary.
What is the difference between cold evaluation and impact measurement?
Cold evaluation mainly documents use, perceived usefulness and obstacles after the training. Impact measurement examines a change in the business and tries to understand the training’s contribution among other factors. The two can complement each other but do not answer to the same level of evidence.
Who should carry out on-the-job evaluation?
Someone who knows the task and the criteria: a trainer, a tutor, a qualified peer or a manager. For a sensitive decision, provide a standardised grid, calibration and, if necessary, a second assessment.
Can the same questionnaire be used for all training courses?
A common core makes steering easier, but questions on transfer and performance must be adapted to the skill. The best compromise combines a few stable indicators with two or three items specific to the use case.
Is a good hot score useless?
No. It tells you about learning conditions and allows a programme to be corrected quickly. It becomes misleading only when it is made to say that learning or performance is demonstrated.
How can the manager be involved in follow-up without overloading their schedule?
Plan a ten-minute exchange on three points: an opportunity to practise, a behaviour to observe and a next challenge. Managerial follow-up is more useful when it facilitates use than when it repeats the participant’s questionnaire.
What is the Kirkpatrick model?
The Kirkpatrick model distinguishes four levels of evaluation: the trainee’s reaction, learning, behaviour and results. Hot and cold evaluation mainly cover the first two levels; to measure the impact on the business you need a longer follow-up, for example three to six months after the training, to check that employees can put into practice what they learned.
How do you evaluate a training course?
To evaluate a training course, combine a hot evaluation, a learning assessment, a cold evaluation and, if needed, an on-the-job observation. A simple form or training software is enough to collect trainees’ answers; good practice is to optimise the system according to the decision to be made.
Conclusion: Measure Less, Decide Better
Hot evaluation, cold training evaluation and on-the-job evaluation are not three variants of the same questionnaire. They observe different realities. The first sheds light on experience; the second, delayed, reports declared use and obstacles; the third provides direct evidence of performance. Between them, learning assessment checks that the learning objective was achieved within the programme.
A mature system does not seek the perfect score. It organises a coherent chain of evidence, from the reference task to behaviour on the job, and ties each data point to a decision. That is how evaluation stops being an end-of-training formality and becomes a tool for steering skills.
Do you want to distinguish satisfaction, learning, transfer and performance, harmonise your grids or build a protocol adapted to your jobs? Marmignon Brothers can audit your evaluation system, define the evidence expected and design light evaluations usable by L&D teams and managers. Also discover our approach to AI training for companies, where on-the-job evaluation is particularly decisive.
Sources and References
- French Ministry of Labour, reading guide to the Qualiopi national framework, in particular indicator 11 on achievement of objectives (in French).
- Légifrance, Decree no. 2026-728 of 1 August 2026 on the national framework for the quality of actions contributing to skills development (in French).
- Cedefop, Defining, writing and applying learning outcomes: alignment between expected outcomes, criteria and assessment methods.
- Cedefop, The impact of learning-outcomes-based curricula on work and practice-based IVET: the role of follow-up, feedback and managerial support in transfer.
