The short answer
Interview calibration is the process of getting interviewers to apply the same standard, so a given answer earns the same rating no matter who scores it. Teams calibrate by scoring the same sample interviews independently, comparing their ratings against the rubric, and agreeing on what evidence earns each score. It matters because uncalibrated interviewers turn a fair process into a lottery: the same candidate passes with one interviewer and fails with another.
How interview calibration works
Recruiters also use "calibration" for the intake meeting where a recruiter and hiring manager agree on the profile they are looking for. That is useful, but this page covers rating calibration among interviewers.
- Pick a sample. Use two or three recorded or mock interviews for the role, ideally one strong, one middling and one weak.
- Score independently. Every interviewer uses the real scorecard and rubric, without talking.
- Reveal and compare. Put the ratings side by side, competency by competency.
- Discuss the evidence. For each gap, each person points to what the candidate said and the anchor they matched it to.
- Fix the cause. If the anchor was vague, rewrite it. If one person misread it, note the agreed reading.
- Repeat. Score a new sample to check the change worked.
This mirrors OPM's guidance for panels: each member rates independently, then the panel explores the basis for any differences and reaches a consensus. OPM's sample interviewer training plan also includes critiqued practice with a videotaped interview.
Example: one session, before and after
Four interviewers scored a recorded answer on stakeholder management, then a second recording after rewriting the anchors.
On sample 1, Interviewer 2 gave credit for a confident tone, while Interviewer 1 noticed the candidate never named the stakeholder they had disagreed with. The team rewrote the 3 anchor to require a named stakeholder, the disagreement and how it was settled. The spread on sample 2 fell to one point. To track this over time, measure inter-rater reliability.
Common mistakes
- Calibrating on live candidates in the debrief. The most senior voice sets the score. Use samples.
- Doing it once. Interviewers drift. Recheck when a role changes or new interviewers join.
- Ignoring individual patterns. OPM names leniency (high ratings for everyone) and strictness (low ratings for everyone) as common errors. Look at each interviewer's average.
Our interview scorecard template has more calibration tips.
Related terms
- Inter-rater reliability: the number that shows whether calibration worked.
- Interview rubric: the anchors interviewers calibrate against.
- Interview scorecard: the form used in each calibration round.
- Structured interview: set questions and scales that make calibration possible.
- Behavioral interview: past-behavior questions, the most common thing teams calibrate.
Questions
How often should hiring teams calibrate?
Before a new role opens, whenever new interviewers join the loop, and when the role changes. Many teams also run a short check each quarter by scoring one shared recording and comparing results.
What is the difference between a calibration session and a debrief?
A calibration session uses sample interviews to align interviewers before or between hiring cycles. A debrief discusses real candidates to make a decision. Calibrating in the debrief tends to pull everyone toward the most senior opinion.
Does calibration apply to AI interview scores?
Yes. Have your interviewers score a few AI-run interviews from the recordings and compare their ratings with the automated scores. Large, consistent gaps tell you to adjust the scoring criteria or how much weight you give the score.
Want to calibrate your team against a recorded AI interview? Book a 20-minute demo.
