A good data scientist turns a vague business question into a clear answer you can act on, and knows when the data can't support one. These data scientist interview questions test that range: statistics and experiments, modeling judgment, hands-on data work, and the stories that show how a candidate works with people. Hiring managers can pick 6 to 10 for a first round. Candidates can use the "what good looks like" notes to check their own answers. The best follow-up for almost every question here is "how would you check that?"
What a strong data scientist interview checks
- Statistical reasoning: correct use of tests, uncertainty and sample size, explained in plain words.
- Experiment design: clean A/B tests and honest reading of results, including when not to trust them.
- Modeling judgment: choosing a sensible model, the right metric, and spotting leakage.
- Data handling: SQL and Python fluent enough to get and clean their own data.
- Causal thinking: knowing correlation from cause, and the tools for when you can't run an experiment.
- Communication: turning findings into a recommendation a non-technical leader can use.
Data scientist interview questions: statistics and experiments
1. Explain a p-value to a product manager.
What good looks like: "If the change truly did nothing, the p-value is how often we'd see a difference at least this big just from random noise." They add that it is not the probability the change works, and that a small p-value says nothing about whether the effect is large enough to matter.
Red flag: "A p-value of 0.03 means there's a 97% chance we're right."
2. A team checked their A/B test every day and stopped it the first day it hit p < 0.05. What's wrong?
What good looks like: repeated peeking inflates the false positive rate well above 5%, because random swings will cross the line at some point. The fix is to set the sample size in advance with a power analysis, or to use a sequential testing method built for continuous monitoring.
3. How do you decide how long an experiment should run?
What good looks like: a power analysis using the baseline rate, the smallest effect worth detecting, the significance level and the desired power (often 80%). Halving the detectable effect needs roughly four times the sample. They also run for full weeks to cover day-of-week patterns.
4. How can you get a trustworthy answer from an experiment faster without peeking?
What good looks like: variance reduction. CUPED uses each user's pre-experiment behavior as a covariate to remove noise, and stratification or a more sensitive metric also help. They note the pre-period data must not be affected by the treatment.
5. Give an example of Simpson's paradox and why it matters.
What good looks like: a trend that appears in every subgroup reverses when groups are combined, because the groups differ in size or mix. For example, a new checkout converts better on both mobile and desktop but looks worse overall because more of its traffic came from mobile, which converts lower. Lesson: check segment mix before trusting a blended number.
6. Leadership wants to know if a loyalty program increased spending, but there was no experiment. How do you estimate the effect?
What good looks like: they first warn that members likely spent more before joining (selection bias). Then they offer methods with their assumptions: difference-in-differences with a comparable group (parallel trends), matching on pre-join behavior, or a regression discontinuity if eligibility had a cutoff. They say how confident the answer can be.
Red flag: comparing members and non-members directly and calling it the effect.
Modeling questions
7. How do you tell whether a model is overfitting or underfitting?
What good looks like: compare training and validation error. A large gap means high variance (overfitting): add data, regularize or simplify. Both errors high means high bias (underfitting): add features or capacity. Learning curves show which one more data would fix.
8. Fraud is 1% of transactions, and a model scores 99% accuracy. Is it good?
What good looks like: not necessarily, because predicting "not fraud" every time also scores 99%. They'd look at precision, recall and the precision-recall curve, then pick a threshold based on the cost of a missed fraud versus a blocked good customer.
9. What is data leakage? Give two examples.
What good looks like: information in training that won't exist at prediction time. Examples: a feature recorded after the outcome (such as a "refund issued" flag when predicting refunds), fitting a scaler on all data before splitting, random splits on time-based data, or the same customer in both train and test. Fixes include pipelines fit on training data only, time-based splits and group-aware splits.
10. Compare L1 and L2 regularization.
What good looks like: L1 adds the absolute value of weights to the loss and can push some weights to exactly zero, which selects features. L2 adds squared weights and shrinks them smoothly without zeroing them. Elastic net mixes both, which helps with correlated features.
11. For a tabular churn problem, would you start with gradient-boosted trees or a neural network?
What good looks like: usually gradient-boosted trees (XGBoost, LightGBM or CatBoost), which are strong on tabular data, need less tuning, and handle mixed feature types well. A simple baseline like logistic regression comes first. Neural networks earn their place with text, images or very large datasets.
12. A customer asks why the model flagged them. How do you explain it?
What good looks like: local explanations such as SHAP values for that one prediction, turned into plain reasons ("short account history, unusual location"). They warn that explanations describe the model, not real-world cause, and that regulated decisions may need specific reason codes and review by the right team.
Practical data scenarios
13. How would you compute 7-day retention by weekly signup cohort?
What good looks like: group users by signup week, define "retained" precisely (active on day 7, or any day from 1 to 7), join to activity, and divide retained users by cohort size. They exclude cohorts too recent to have seven days of data and handle time zones consistently.
14. Write the logic for each user's 7-day rolling average of daily spend.
What good looks like: first build one row per user per day, filling days without purchases with zero. Then use a window function, such as AVG(spend) OVER (PARTITION BY user_id ORDER BY day ROWS BETWEEN 6 PRECEDING AND CURRENT ROW). Strong candidates point out that without the filled-in days, a row-based window spans more than seven calendar days.
15. A key column is 30% missing. What do you do?
What good looks like: find out why before fixing anything. Missing completely at random, missing based on other columns, and missing because of the value itself each call for different handling. Options include imputing from training data only, adding a "was missing" flag, or using models that handle gaps. Dropping rows can bias the result.
16. A model that worked well has slowly gotten worse over six months. How do you investigate?
What good looks like: rule out pipeline bugs first (a changed upstream field, a broken join). Then compare input distributions now and at training time (data drift), and check whether the relationship between inputs and outcome changed (concept drift). They mention label delays and a retraining schedule with monitoring.
Behavioral (STAR)
17. Tell me about an analysis that changed a business decision.
What good looks like: the question, the data, the method, the recommendation, and what the business did with it, with a number in the Result. Strong candidates also mention what they left out and why.
18. Describe a time a stakeholder disliked your result.
What good looks like: they double-checked the work, explained it in the stakeholder's terms, stayed firm on what the data showed, and were open about its limits. Bonus for offering a follow-up test.
Red flag: changing the analysis until it said what the stakeholder wanted.
19. Tell me about a time your analysis or model was wrong.
What good looks like: how they found out, how fast they told people, what the impact was, and what check they now run every time.
A simple scoring rubric
Running this interview with AI
For a data science pipeline with many applicants, an adaptive AI interviewer can cover the statistics and modeling basics before your team invests time. NTRVSTA's exam library includes Python and AI/ML topics, and difficulty moves up or down with each answer, so someone who explains peeking well gets a question on variance reduction next. Behavioral questions use STAR, and a hint system shows whether a candidate can use a nudge, which is useful for early-career roles.
Your data leads get a score breakdown and video snippets of key moments, then run the deeper case themselves. Our guide to how AI interviews are scored explains what those scores mean, and the best AI interview software compares tools. Related sets: machine learning engineer, data engineer, SQL and Python.
Questions
How much coding should a data scientist interview include?
Enough to confirm they can pull and shape their own data. One SQL problem and one short Python or pandas task usually do it. Hard algorithm puzzles add little for most analytics and modeling roles, and they screen out people who would do the job well.
Is a take-home assignment better than a live case?
Each has a cost. A take-home shows real work quality but takes candidates hours, so cap it at about three and pay attention to it. A live case is shorter and shows reasoning out loud. Many teams use a live case first and save take-homes for finalists.
How do product analytics and machine learning data scientist interviews differ?
Product analytics roles weigh experiments, metrics and SQL most, so use the statistics and practical sections here. Modeling-heavy roles weigh the modeling section more and add deployment questions. For roles that mostly ship models into production, see our machine learning engineer questions.
Want to hear an AI interviewer adjust a statistics question in real time? Book a 20-minute demo, or start a free sample interview to see the candidate side.
