Skip to main content

Free template

Interview question and evidence matrix

This is the bridge between the scorecard and the interviews. Each must-have gets a row: where it is tested, what is asked, and what a convincing answer contains. Written before the interviews, it stops the panel from grading personality by accident.

Who fills it in

Whoever owns the technical bar for the role.

How long it takes

Thirty to forty minutes, reused across every candidate for the role.

Reviewed by

Joseph Edney, 29 October 2026

When to use it

Use it when

  • The scorecard is written and you now need questions that actually test it.
  • Panel debriefs keep producing impressions rather than evidence.
  • Several interviewers are involved and you need them to assess the same things.

What good looks like

  • Every must-have from the scorecard appears exactly once as a row, and every row names the stage that owns it.
  • Questions ask about work the candidate has actually done, not hypotheticals they can rehearse.
  • The evidence column describes what a strong answer contains, so two interviewers reach the same conclusion.
  • Anything not on the matrix is explicitly a conversation, not an assessment.

Common mistakes

  • Writing questions with one correct answer. You learn more from how someone reasons than from whether they recall a definition.
  • Testing the same capability three times because it is the easiest one to ask about.
  • Leaving the evidence column blank and deciding after the fact what would have counted.
  • Confusing enthusiasm with evidence, particularly in the final stage.

The template

Anything you type stays in this browser tab, so reloading the page keeps it. It is not sent to us or added to the link, so it is safe to put real salary numbers and colleagues’ names in. Closing the tab clears it — copy or print before you go.

Scope

If the scorecard changes, this matrix has to change with it.

Capability matrix

One row per must-have capability. Add rows for the teachable areas only if you intend to assess them.

RowTested at which stageQuestion askedEvidence of a strong answerEvidence of a weak answer
Must-have 1
Must-have 2
Must-have 3
Must-have 4
Must-have 5
Must-have 6 (optional)

Question quality check

Run down this list once the matrix is drafted.

A worked example

Illustrative example. The company, role and answers are fictional, written to show a finished template. It is not a client, a quote or a result. Our fictional clinic-scheduling startup again, a few days after its interview plan was settled. The CTO has turned each of the four scorecard must-haves into one question and placed it at the stage of the plan that already tests it: operating experience at stage 3 with the CTO, and pipeline and writing at stage 2 with a product engineer.

Scope

Role
Platform engineer (first infrastructure hire)
Scorecard version this matrix is built from
The signed-off role scorecard, first version. Any change to its must-haves reopens this matrix.

Capability matrix

RowTested at which stageQuestion askedEvidence of a strong answerEvidence of a weak answer
Must-have 1Stage 3, CTO. Must-have: on-call ownership of live cloud infrastructure.Tell me about a production system you were on call for. What did a bad week look like, and what did you change because of it?Names the system, their part in the rota and a specific change they made afterwards, such as an alert removed, a limit raised or a runbook rewritten.Describes the platform in general terms, or what the team did, but cannot say what they personally were paged for or changed.
Must-have 2Stage 2, product engineer, around the real pipeline change from our backlog. Must-have: a release pipeline that colleagues depended on.Walk me through a pipeline you built or rebuilt. Who used it, what was wrong before, and how did you know it was better?Explains the problem the users had, the trade-offs they chose, how they moved people onto it and what they measured or watched afterwards.Lists tools and steps without saying who it was for or what problem it solved. Cannot say what they would do differently.
Must-have 3Stage 3, CTO. Must-have: debugging an outage from first symptom to fix.Pick one outage you worked on. Take me through it in order: what you saw first, what you tried, and what finally fixed it.Keeps to the order things happened, including the wrong turns, and separates what they knew at the time from what they learned later.Jumps straight to the root cause, or tells the story as if every step was obvious. Follow-up questions about the dead ends get vague answers.
Must-have 4Stage 2, product engineer, with a short written follow-up after the call. Must-have: instructions a colleague can use unaided.Show or describe a document you wrote for other engineers to act on. Who used it, and what did they ask you afterwards?The document has a clear reader, steps in order and a way to tell each step worked. They mention changing it after someone got stuck.Writing is described as something they would do given time. Any example is notes for themselves rather than for another engineer.
Must-have 5Not used. The scorecard has four must-haves, so the matrix has four rows.None. Teachable areas are not assessed at interview for this role.Not assessed on this matrix.Not assessed on this matrix.
Must-have 6 (optional)Not used. The stage 1 call screens motivation and fit before stages 2 and 3 test the must-haves in depth, and the stage 4 conversation with the CEO is not an assessment.None here. The stage 1 call is a go or no-go against the scorecard, and the CEO conversation is not scored.Not assessed on this matrix.Not assessed on this matrix.

Question quality check

  • Done: Every question is about work the candidate has really done
  • Done: No question can be answered yes or no
  • Done: Each row has at least one planned follow-up probe
  • Done: Questions allow for different company sizes and contexts
  • Done: No question depends on a specific school, employer or accent of experience
  • Not yet: The stage has enough time for the rows assigned to it