Learning Experiment & Statistical Analysis
Research design · randomized evaluation · effect size · subgroup analysis
A synthetic randomized A/B learning experiment demonstrating how I structure research questions, pre-specify outcomes, check randomization balance, analyze knowledge gains, quantify effect size and uncertainty, and connect statistical evidence to program decisions.
The comparison is between standard instruction and an applied/scaffolded learning design. All participants, scores, outcomes, and findings are synthetic.
Results at a glance
| Measure | Synthetic result |
|---|---|
| Included participants | 234 |
| Standard mean knowledge gain | 4.0 points |
| Applied/scaffolded mean gain | 9.0 points |
| Difference in mean gain | 5.01 points |
| 95% confidence interval | 3.66 to 6.35 points |
| Cohen’s d | 0.95 |
| Completion-rate difference | +11.6 percentage points |
These are synthetic demonstration results, not evidence about a real program or population.
Research design
- Blocked randomized assignment across job family and experience bands.
- Pre-specified primary outcome: pre-to-post knowledge gain.
- Transparent record-level inclusion and exclusion logic.
- Baseline balance checks before outcome interpretation.
- Independent-group comparison with standard error, Welch-style test statistic, confidence interval, and Cohen’s d.
- Secondary outcomes for confidence, completion, satisfaction, and time.
- Exploratory subgroup analysis by experience and job family.
Workbook architecture
- Executive Dashboard
- Analysis Plan
- Participant Data
- Summary Statistics
- Balance Check
- Outcome Analysis
- Subgroup Analysis
- Research Integrity
- Data Dictionary
- Portfolio Guide
Aggregate statistics are formula-driven from participant-level data. The workbook also documents why effect size, uncertainty, operational outcomes, and design quality matter alongside statistical significance.
Research integrity and interpretation
The artifact keeps excluded records visible, documents balance checks, separates the primary outcome from secondary outcomes, and treats subgroup results as exploratory. It also states explicitly that this is not human-subjects research. A real study would require the appropriate institutional review and consent determination for its context.
The statistical conclusion is not based on a p-value alone: the workbook displays the estimated difference, uncertainty interval, standardized effect size, completion outcome, and design assumptions together.