The short answer
How are AI interviews scored? The good ones work like a well-run structured interview. Every question maps to a skill the job needs, every skill has a rubric that says what a weak, solid and strong answer sounds like, and the AI rates the candidate's actual words against that rubric. The scores roll up into a breakdown by competency, with the evidence (quotes, clips) attached so a person can check it. A score you can't trace back to something the candidate said is a score you shouldn't use. And a person, not the model, should make the hiring decision.
The foundation: structured interviews
AI scoring borrows almost everything from decades of research on structured interviews. The US Office of Personnel Management (OPM) describes a structured interview as one that uses rules for eliciting, observing and evaluating responses. At the highest level of structure, every candidate gets the same pre-defined lead and probe questions, and answers are scored against benchmarks of proficiency.
OPM's summary of the research says more structure brings higher validity, better agreement between raters, and less adverse impact than unstructured interviews. It describes two main question types:
- Behavioral: "Tell me about a time you..." questions, based on the idea that past behavior predicts future behavior. OPM notes these show higher validity for highly complex jobs. The STAR method (Situation, Task, Action, Result) is the usual way to structure and score them.
- Situational: "What would you do if..." questions that put a candidate in a job-like scenario.
Technical interviews add a third type: problem-solving questions with a correct or better answer, such as explaining a bug, designing a system, or choosing a data structure.
What a scoring rubric looks like
A rubric turns "that was a good answer" into something two people (or a person and a model) can agree on. Here's an example for one competency, stakeholder communication, scored on a 1 to 4 scale:
The anchors describe content, not style. Nothing in this rubric rewards confidence, accent, vocabulary or eye contact. That matters for fairness, and in the EU, inferring emotions in the workplace with AI has been prohibited since February 2, 2025 (with medical and safety exceptions). See our EU AI Act hiring guide.
How are AI interviews scored? Step by step
The details differ by vendor, but a defensible pipeline has these stages:
- Define the competencies. From the job description, pick 3 to 6 skills (for example: Python fundamentals, debugging, system design, communication). Each gets a rubric and a weight.
- Ask and follow up. The interviewer asks questions mapped to each competency. In a live, adaptive interview, the AI also asks follow-ups when an answer is vague, and raises or lowers difficulty based on how the candidate is doing. Our post on adaptive questioning in AI interviews explains why that matters.
- Transcribe. Speech becomes text. Transcription quality sets a ceiling on scoring quality, so check how the tool handles accents, noise and technical terms.
- Find the evidence. For each competency, the model pulls the parts of the transcript that bear on it: the example the candidate gave, the actions they named, the trade-off they explained.
- Rate against the rubric. The model compares that evidence with the rubric anchors and assigns a level, ideally with a short reason ("gave a specific example and result; did not discuss trade-offs").
- Roll up. Competency scores combine into an overall score using the weights you set. Good reports show the breakdown, not just the total.
- Attach proof. Quotes, timestamps or video clips sit next to each score so a reviewer can check the model's work in minutes.
Adaptive difficulty changes how you read the result. If two candidates both "got the question right," but one was answering at a harder level, a fair report shows that context. Ask vendors how difficulty is reflected in the score.
What a good score report contains
Whatever tool you use, the report should answer three questions quickly: how did the candidate do on each skill, why did they get that score, and where can I see it for myself. A checklist:
- a score per competency, on a scale that's explained;
- a one-line reason per score that cites what the candidate said;
- links to the moments in the recording or transcript behind each score;
- flags for anything unusual (for example, a very short interview or a dropped connection);
- a format you can share with the hiring manager.
As an example of outputs, NTRVSTA's reports include a score breakdown (technical skills, English proficiency and communication), video snippets of key moments, and shareable candidate reports with scores, highlights and notes.
How to check that the scores are any good
Don't take a vendor's word for it, and don't take ours. Run a quick calibration before you rely on scores:
- Pick 20 completed interviews across a range of outcomes.
- Have two experienced interviewers score them with the same rubric, without seeing the AI score.
- Compare. Where the two humans agree with each other but not with the AI, read the evidence. Is the rubric unclear, or is the model missing something?
- Check outcomes by group. If you collect voluntary demographic data, compare scoring rates across groups. NYC's Local Law 144 rules define a "scoring rate" as the share of a group scoring above the sample's median. Our AI bias audit guide walks through the math.
- Repeat after changes. New questions, a new rubric or a new model version means a new calibration.
If you hire in New York City and use the score to decide who advances, you may also need an independent bias audit and candidate notice. See our Local Law 144 guide. This is not legal advice.
Red flags in AI interview scoring
- One number, no breakdown. You can't coach, compare or defend a single opaque score.
- "Personality" or "enthusiasm" from video. If a tool scores facial expressions or tone of voice, ask what job skill that measures and how it was validated.
- Different questions, same scale. If every candidate gets unrelated questions, scores aren't comparable unless the rubric is tied to competencies, not questions.
- No human in the loop. Auto-rejecting below a score cutoff, with no review, puts all the risk on the model.
For scored question banks you can use right away, see our STAR interview questions, which include a 1 to 4 rubric, or compare structured interview software. New to the format? Start with what an AI interview is.
Questions
Do AI interviews score you on your face or body language?
Some tools may use video signals, but a well-built AI interview scores what you say against a rubric of job skills. In the EU, AI that infers emotions in the workplace has been banned since February 2, 2025. Ask any vendor exactly which inputs feed the score.
What is a good score on an AI interview?
It depends on the rubric and the role, so compare candidates within the same interview rather than against a universal number. On a 1 to 4 rubric, a 3 usually means a clear, specific, complete answer, and a 4 adds trade-offs and measurable results.
Can a recruiter override an AI interview score?
They should be able to. The score is evidence for a human decision. Look for reports that show the reasoning and the clips behind each score, so a recruiter can agree, disagree and note why.
How accurate is AI interview scoring compared with human interviewers?
It depends on the rubric and the tool. The fair test is to have experienced interviewers score a sample of the same interviews blind and compare. Structured scoring tends to be more consistent than unstructured human judgment, which is the main reason to use it.
See a scored report from a real adaptive interview: book a 20-minute demo, or take a sample interview and see what the AI asks.
