We show that group sequential designs (which allow to perform interim analyses while data collection is ongoing) can unlock numerous benefits for researchers engaged in confirmatory hypothesis testing: They facilitate sample size decisions, allow researchers to achieve a desired level of statistical power with a smaller number of observations, and help conduct more efficient pilot studies. We validate this cost-saving potential through a comprehensive re-analysis of 212 studies published in the Journal of Consumer Research, which shows that using these designs would have reduced costs by 20% to 29%. We conclude with a discussion of limitations and possible alternatives.
Creating studies that are powered to detect the smallest effect of interest... without collecting more data than you need to detect bigger effects
A two-part blog post on outlier exclusion procedures, and their impact on false-positive rates.
How should researchers exclude outliers: Across the data, or within conditions? In this paper, I show that when outlier exclusions are performed in a way that is not 'blind' to the researchers' hypothesis (e.g., within conditions), they increase Type I error to unacceptable levels.