How to Hire a Data Scientist in 2026: Statistics, Business Judgment, and the ML Engineer Difference

The confusion between data scientist and machine learning engineer is one of the most expensive hiring mistakes I see. Teams write a job description for one role, screen for the skills of the other, and end up surprised when the hire spends all their time building a Jupyter notebook nobody acts on, or, conversely, gets asked to put a model in production without any ML infrastructure experience.

The roles are genuinely different. Get clear on which one you need before you write the job description.

The core distinction

A data scientist’s job is to answer questions. They take a business problem (“why is retention dropping?”, “which user segments respond to this promotion?”, “is our change actually working?”), find and shape the data that can answer it, apply appropriate statistical methods, and communicate the result to someone who will make a decision.

An ML engineer’s job is to build systems. They take a trained model and make it work in production: serving latency, monitoring drift, retraining pipelines, feature stores, A/B testing frameworks. They work in Python and Terraform and Kubernetes. Their output is running software, not a slide deck.

There is an overlap zone called applied ML or research ML, where data scientists train and evaluate models as part of their analysis. But that is different from owning production model deployment, which is the ML engineer’s territory.

Ask yourself: what decisions do I want this person to inform? If the answer is product strategy, user behavior, business performance, and experiment design, you probably want a data scientist. If the answer is a model that runs in your app and needs to be reliable at scale, you want an ML engineer. Many teams need both.

Statistical thinking as the filter

The thing that most clearly separates good data scientists from people who learned to call Python libraries is whether they understand the statistics underneath the functions they are running.

The most revealing interview prompt I know is simple: “Walk me through how you would design and run an A/B test to evaluate a new feature.”

A weak answer describes the mechanics: split users into two groups, measure some metric, compare. It does not discuss the choices that determine whether the result is trustworthy.

A strong answer covers:

The hypothesis. What exactly are we measuring and in what direction? “We expect the new checkout button color to increase conversion rate” is a hypothesis. “We want to see if the change helps” is not.

The primary metric. What is the one number that determines success? Secondary metrics matter but having too many increases the chance of false positives. The candidate should pick one and defend it.

Sample size and power. Before running the test, how many users per group do you need to detect a meaningful effect at reasonable confidence? This requires knowing the baseline conversion rate, the minimum effect size worth caring about, and the target statistical power (typically 0.8). Running a test without planning sample size is the most common A/B testing mistake.

The run duration. Running a test until it looks significant is called p-hacking, even unintentionally. The candidate should mention running the test for a pre-specified duration (often at least one business cycle) rather than peeking and stopping early.

The decision. A p-value below 0.05 means the observed effect is unlikely under the null hypothesis; it does not mean the effect is large or practically important. A strong candidate distinguishes statistical significance from practical significance and mentions the effect size alongside the p-value.

If a candidate cannot walk through that structure, they have not run a real A/B test under someone who knew what they were doing. That is a real gap for a role that will be expected to inform product decisions with experiments.

What a p-value actually is

Ask this directly: “Without using the word ‘significance’, what is a p-value?”

The correct answer: a p-value is the probability of observing results at least as extreme as the ones you got, assuming the null hypothesis is true. It is not the probability that the null hypothesis is true. It is not the probability that the result is due to chance. Those are common misstatements even among people with data experience.

This question sounds pedantic. It is not. A team acting on a p-value because someone told them it means “the chance you got this by luck” is acting on a misunderstanding. The data scientist’s job is to be the person in the room who is not confused about this.

SQL and data access

Most business data lives in relational databases. A data scientist who cannot write SQL is dependent on an engineer for every data question, which is a bottleneck. Expect SQL fluency as a baseline, not a bonus.

What SQL fluency means in practice:

• Window functions: ROW_NUMBER(), LAG(), LEAD(), SUM() OVER (PARTITION BY...). Ask a candidate to write a query that finds the second-highest order value per customer. If they cannot use a window function for this, they will write SQL that is brittle or correct but slow.
• CTEs (Common Table Expressions) for readable, composable queries. A candidate who builds all their analysis in nested subqueries is technically correct but writing code that nobody else can maintain.
• Aggregation: GROUP BY, HAVING, the difference between WHERE (filters before aggregation) and HAVING (filters after). This is basic enough that a gap here is a real signal.
• Date and time manipulation. Most retention and cohort analysis requires working with timestamps. Ask them how they would write a query to find the 30-day retention rate for users who signed up in a given month.

The Python screen

The core Python stack for data science work is pandas (data manipulation), NumPy (numerical arrays), matplotlib and seaborn (visualization), and scipy (statistical tests). Scikit-learn covers ML algorithms and preprocessing.

Ask them to describe how they would handle a dataset with 30% null values in a key column. A strong answer does not just say “fill with the mean.” It says: first understand why the data is missing (is it missing at random, missing not at random, missing completely at random?). The mechanism matters: if the nulls are concentrated in a specific cohort or time period, imputing the mean would introduce bias. If the nulls are random, median or mean imputation may be appropriate. Sometimes the correct answer is to exclude the rows or flag the nulls as a feature themselves.

Ask them how they would visualize a distribution to identify whether it is suitable for parametric statistical tests. The answer: a histogram, a KDE plot, or a Q-Q plot against a normal distribution. Parametric tests like the t-test assume normality. If the distribution is heavily skewed, a non-parametric alternative like the Mann-Whitney U test may be more appropriate.

Communication as part of the job

Data scientists who cannot communicate findings might as well not have found them. The analysis ends with a decision, and decisions are made by people who were not in the Jupyter notebook.

Ask them to describe a time they presented a finding that was counterintuitive or that the stakeholder did not want to hear, and what happened. This surfaces whether they have real experience communicating under pressure, whether they can explain trade-offs without jargon, and whether they understand that the goal is the decision, not the analysis.

Ask them to describe an analysis where they were wrong and what made them realize it. Data scientists who have never been wrong have not done enough analysis to be good at it. The right answer describes the mistake concretely, explains what they missed, and says what they check for now as a result.

Avoiding the skills mismatch

Before posting the role, decide whether you need deep learning and neural networks or classical statistical methods and business analysis. Most companies that say “we need a data scientist” actually need someone who can run experiments well, analyze user behavior in SQL, build readable dashboards, and communicate findings. That person does not need to know how to fine-tune a large language model.

If you post a job description with requirements for TensorFlow, PyTorch, and transformer architecture knowledge, you will attract candidates who want to do ML research. If what you actually need is someone to help the product team understand user behavior, those candidates will be bored within six months and leave.

Get specific about the work. The data science hiring cluster in our blog covers adjacent roles if you are building a data team: the ML engineer hiring guide covers the production ML side in depth.