CS 222 - Advanced Statistical Methods for Data Analysis

1. Review basic probability theory including law of total probability, conditional probabilities, probability distribution functions and cumulative distributive functions.
2. Practice designing experiments including determining number of variants, holdout strategy, sample sizes.
3. Practice evaluating experiments and testing for significance in cases of large versus small samples or skewed data.
4. Discover pitfalls of A/B testing. Examine Type I and Type II errors, and how to assess trade-offs between the errors when evaluating experiments.
5. Identify when to use causal inference, and evaluate data for causality when experimentation at scale is not possible.
6. Explore how to design objective functions when examining errors between predicted and modeled outputs for numerical data.
7. Tabulate confusion matrices for categorical data, and compare various performance metrics to assess performance of models.
8. Describe bias-variance trade-off, and its importance in analyzing data results.
9. Recognize standard approaches for survey design and analysis.