Meta-analytic findings of the self-controlled motor learning literature: Underpowered, biased, and lacking evidential value

Citation

APA 7th

McKay, B., Yantha, Z., Hussien, J., Carter, M., & Ste-Marie, D. (2022). Meta-analytic findings of the self-controlled motor learning literature: Underpowered, biased, and lacking evidential value. Meta-Psychology, 6. https://doi.org/10.15626/MP.2021.2803

Bibtex

@article{,
  title = {Meta-Analytic Findings of the Self-Controlled Motor Learning Literature: {{Underpowered}}, Biased, and Lacking Evidential Value},
  shorttitle = {Meta-{{Analytic Findings}} of the {{Self-Controlled Motor Learning Literature}}},
  author = {McKay, Brad and Yantha, Zachary and Hussien, Julia and Carter, Michael and Ste-Marie, Diane},
  date = {2022-11-08},
  journaltitle = {Meta-Psychology},
  volume = {6},
  issn = {2003-2714},
  doi = {10.15626/MP.2021.2803},
  url = {https://open.lnu.se/index.php/metapsychology/article/view/2803},
  urldate = {2023-07-13},
  langid = {english},
  keywords = {Choice,Meta-Analysis,Motor Learning,OPTIMAL Theory,p-curve,Publication Bias,Retention}
}

Abstract

The self-controlled motor learning literature consists of experiments that compare a group of learners who are provided with a choice over an aspect of their practice environment to a group who are yoked to those choices. A qualitative review of the literature suggests an unambiguous benefit from self-controlled practice. A meta-analysis was conducted on the effects of self-controlled practice on retention test performance measures with a focus on assessing and potentially correcting for selection bias in the literature, such as publication bias and p-hacking. First, a naïve random effects model was fit to the data and a moderate benefit of self-controlled practice, g = .44 (k = 52, N = 2061, 95% CI [.31, .56]), was found. Second, publication status was added to the model as a potential moderator, revealing a significant difference between published and unpublished findings, with only the former reporting a benefit of self-controlled practice. Third, to investigate and adjust for the impact of selectively reporting statistically significant results, a weight-function model was fit to the data with a one-tailed p-value cutpoint of .025. The weight-function model revealed substantial selection bias and estimated the true average effect of self- controlled practice as g = .107 (95% CI [.047, .18]). P-curve analyses were conducted on the statistically significant results published in the literature and the outcome suggested a lack of evidential value. Fourth, a suite of sensitivity analyses were conducted to evaluate the robustness of these results, all of which converged on trivially small effect estimates. Overall, our results suggest the benefit of self-controlled practice on motor learning is small and not currently distinguishable from zero.