Workspace
Bookmarks

Prepare

Success Path
Modules
Webinars

Practice

Interview Questions
Mock Interviews

Prepare

Modules

Data Analyst

Data Engineer

Product Data Scientist

AI Engineer

Machine Learning Engineer

Prompt Engineer

Dataford - Ace your Data Interview

In this course, you’ll master how to design, run, and evaluate experiments that drive real product decisions.

You’ll cover:

Foundations of Experimentation
Experimentation Across the Product Lifecycle
The Experiment Lifecycle Deep Dive
Measuring Long-Term Impact
Interview Case Studies
Alternative methods when A/B testing is not possible

Module Overview

Related Roles

Product Data Scientist

Data Analyst

Module Tags

Experimentation

A/B Test

Propensity Score

PSM

Causal Inference

Time Estimate

5 hrs 3 min

Difficulty

Advanced

Related Roles

Product Data Scientist

Data Analyst

Module Tags

Experimentation

A/B Test

Propensity Score

PSM

Causal Inference

Time Estimate

Syllabus

8 chapters49 sections

1. What is Experimentation?

What is Experimentation?

2. Why run experiments?

5 hrs 3 min

Difficulty

Advanced

Why run experiments?

3. When to run experiments?

When to run experiments?

1. Steps of the Product Lifecycle

Steps of the Product Lifecycle

2. Embedding Experimentation into Product

Embedding Experimentation into Product

3. Stakeholders in Experimentation

Stakeholders in Experimentation

1. Phases in Experimentation

Phases in Experimentation

2. Defining the Problem Statement

Defining the Problem Statement

3. Hypothesis Generation

Hypothesis Generation

4. Experimentation Details

Experimentation Details

5. Defining Test Metrics

Defining Test Metrics

6. Test Metrics Decision Flow

Test Metrics Decision Flow

7. Decide on number of variations

Decide on number of variations

8. Choosing test unit and assignment method

Choosing test unit and assignment method

9. Qualified Target Base

Qualified Target Base

10. Control Group Size & Structure

Control Group Size & Structure

11. Exposure duration

Exposure duration

12. Measurement horizon (Power Analysis)

Measurement horizon (Power Analysis)

13. Power Analysis Formula & Parameter Choices

Power Analysis Formula & Parameter Choices

14. Measurement horizon

Measurement horizon

15. Risk Tolerance and Ramp plan

Risk Tolerance and Ramp plan

1. Data Validation & Logging Checks

Data Validation & Logging Checks

2. Data Contamination Checks

Data Contamination Checks

3. Sample Ratio Mismatch

Sample Ratio Mismatch

4. Intermediate Reads & Monitoring

Intermediate Reads & Monitoring

1. Interpreting Results - P-Value

Interpreting Results - P-Value

2. Misconceptions of P-Value

Misconceptions of P-Value

3. Interpreting Results - Confidence Intervals

Interpreting Results- Confidence Intervals

4. Misconceptions of Confidence Interval

Misconceptions of Confidence Interval

5. Statistical vs Practical Significance

Statistical vs Practical Significance

6. Experiment Outcome Guideline

Experiment Outcome Guideline

7. Stakeholder Communication Framework

Stakeholder Communication Framework

8. Stakeholder Communication Expectations

Stakeholder Communication Expectations

1. Importance of Long-Term Measurement

Importance of Long-Term Measurement

2. Long-Term Detect behavioral & experimental Effects

Long-Term Detect behavioral & experimental Effects

3. Long-Term Detect behavioral & experimental Effects Visualization

Long-Term Detect behavioral & experimental Effects Visualization

4. Determine holdout strategy Introduction

Determine holdout strategy Introduction

5. Determine holdout strategy Design options

Determine holdout strategy Design options

6. Determine holdout strategy - Challenges and Mitigation

Determine holdout strategy - Challenges and Mitigation

7. Challenges of Holdout Frameworks

Challenges of Holdout Frameworks

1. Youtube Interview Case Study

Youtube Interview Case Study

2. Uber Interview Case Study

Uber Interview Case Study

1. Introduction to Casual Inference

Introduction to Casual Inference

2. Casual Inference Methods

Casual Inference Methods

3. Propensity Score Matching (PSM) Introduction

Propensity Score Matching (PSM) Introduction

4. Propensity Score Matching (PSM) Core Idea

Propensity Score Matching (PSM) Core Idea

5. Propensity Score Matching (PSM) Methodology

Propensity Score Matching (PSM) Methodology

6. Propensity Score Matching (PSM) Design

Propensity Score Matching (PSM) Design

7. Propensity Score Marching (PSM) Real World Applications

Propensity Score Marching (PSM) Real World Applications

399

399