82. Geoff Cumming: p-values, estimation, and meta-analytic thinking

11/24/23 • 72 min

Geoff Cumming is an Emeritus Professor at La Trobe University. In this conversation, we discuss his work on New Statistics: estimation instead of hypothesis testing, meta-analytic thinking, and many related topics.
Support the show: https://geni.us/bjks-patreon
Timestamps
0:00:00: A brief history of statistics, p-values, and confidence intervals
0:32:02: Meta-analytic thinking
0:42:56: Why do p-values seem so random?
0:45:59: Are p-values and estimation complementary?
0:47:09: How do I know how many participants I need (without a power calculation)?
0:50:27: Problems of the estimation approach (big data)
1:00:08: A book or paper more people should read
1:02:50: Something Geoff wishes he'd learnt sooner
1:04:52: Advice for PhD students and postdocs
Podcast links

Website: https://geni.us/bjks-pod
Twitter: https://geni.us/bjks-pod-twt

Geoff's links

Website: https://geni.us/cumming-web
Google Scholar: https://geni.us/cumming-scholar
Mastodon: https://nerdculture.de/@thenewstats

Ben's links

Website: https://geni.us/bjks-web
Google Scholar: https://geni.us/bjks-scholar
Twitter: https://geni.us/bjks-twt

References/links
Dance of the p-values: https://www.youtube.com/watch?v=5OL1RqHrZQ8
Significance roulette: https://www.youtube.com/watch?v=OcJImS16jR4
Episode with Simine Vazire (SIPS): https://geni.us/bjks-vazire
Coulson, ...(2010). Confidence intervals permit, but don't guarantee, better inference than statistical significance testing. Front in Psychol.
Cumming & Calin-Jageman (2016/2024). Introduction to the new statistics: Estimation, open science, and beyond.
Cumming (2014). The new statistics: Why and how. Psychol Sci.
Cumming & Finch (2005). Inference by eye: confidence intervals and how to read pictures of data. American Psychol.
Errington, ... (2021) Reproducibility in Cancer Biology: Challpenges for assessing replicability in preclinical cancer biology. eLife.
Errington, ... (2021) Investigating the replicability of preclinical cancer biology. eLife.
Finch & Cumming (2009). Putting research in context: Understanding confidence intervals from one or more studies. J of Pediatric Psychol.
Hedges (1987). How hard is hard science, how soft is soft science? The empirical cumulativeness of research. American Psychologist.
Hunt (1997). How science takes stock: The story of meta-analysis.
Ioannidis (2005). Why most published research findings are false. PLoS Medicine.
Loftus (1996). Psychology will be a much better science when we change the way we analyze data. Curr direct psychol sci.
Maxwell, ... (2008). Sample size planning for statistical power and accuracy in parameter estimation. Annu Rev Psychol.
Oakes (1986). Statistical inference: A commentary for the social and behavioural sciences.
Pennington (2023). A Student's Guide to Open Science: Using the Replication Crisis Reform Psychology.
Rothman (1986). Significance questing. Annals of Int Med.
Schmidt (1996). Statistical significance testing and cumulative knowledge in psychology: Implications for training of researchers. Psychol Methods.

Website: https://geni.us/bjks-pod
Twitter: https://geni.us/bjks-pod-twt

Geoff's links

Website: https://geni.us/cumming-web
Google Scholar: https://geni.us/cumming-scholar
Mastodon: https://nerdculture.de/@thenewstats

Ben's links

Website: https://geni.us/bjks-web
Google Scholar: https://geni.us/bjks-scholar
Twitter: https://geni.us/bjks-twt

Previous Episode

81. Brooke Macnamara: Growth mindset, deliberate practice, and the benefits of diverse experiences

Brooke Macnamara is an associate professor at Case Western Reserve University. In this conversation, we talk about her research on growth mindset and deliberate practice, whether deliberate practice is falsifiable, the benefits of diverse experiences, and much more.
BJKS Podcast is a podcast about neuroscience, psychology, and anything vaguely related, hosted by Benjamin James Kuper-Smith.
Support the show: https://geni.us/bjks-patreon
Timestamps
0:00:00: How Brooke started working on mindset and deliberate practice
0:02:10: (Growth) mindset: does it matter?
0:21:10: Mindset interventions
0:36:48: Deliberate practice
0:47:06: Benefits of diverse experiences
0:56:20: Is the theory of deliberate practice unfalsifiable?
0:59:36: What can we take practically from the growth mindset and deliberate pratice research?
1:01:06: A book or paper more people should read
1:02:10: Something Brooke wishes she'd learnt sooner
1:04:32: Advice for PhD students and postdocs
Podcast links

Website: https://geni.us/bjks-pod
Twitter: https://geni.us/bjks-pod-twt

Brooke's links

Website: https://geni.us/macnamara-web
Google Scholar: https://geni.us/macnamara-scholar
Twitter: https://geni.us/macnamara-twt

Ben's links

Website: https://geni.us/bjks-web
Google Scholar: https://geni.us/bjks-scholar
Twitter: https://geni.us/bjks-twt

References/links
Brainology mindset intervention: https://www.mindsetworks.com/programs/brainology-for-schools
Trello: https://trello.com
Burgoyne, Hambrick, & Macnamara (2020). How firm are the foundations of mind-set theory? The claims appear stronger than the evidence. Psychol Science.
Dweck (2006). Mindset-Changing the way you think to fulfil your potential.
Epstein (2021). Range: Why generalists triumph in a specialized world.
Ericsson & Harwell (2019). Deliberate practice and proposed limits on the effects of practice on the acquisition of expert performance. Frontiers in Psychol.
Ericsson, Krampe & Tesch-Römer (1993). The role of deliberate practice in the acquisition of expert performance. Psychol Rev.
Gladwell (2008). Outliers: The story of success.
Macnamara & Burgoyne (2023). Do growth mindset interventions impact students’ academic achievement? A systematic review and meta-analysis with recommendations for best practices. Psychol Bull.
Macnamara, Hambrick & Oswald (2014). Deliberate practice and performance in music, games, sports, education, and professions: A meta-analysis. Psychol Science.
Macnamara & Maitra (2019). The role of deliberate practice in expert performance: Revisiting Ericsson, Krampe & Tesch-Römer (1993). Royal Society Open Science.
Macnamara, Moreau & Hambrick (2016). The relationship between deliberate practice and performance in sports: A meta-analysis. Perspec Psychol Science.
Macnamara, Prather & Burgoyne (2023). Beliefs about success are prone to cognitive fallacies. Nat Rev Psychol.
Sisk, Burgoyne, Sun, Butler & Macnamara (2018). To what extent and under which circumstances are growth mind-sets important to academic achievement? Two meta-analyses. Psychol Science.

Next Episode

83. Rachel Bedder: Rumination, teaching without grades, and managing yourself as a PhD student

Rachel Bedder is a postdoc with Yael Niv at Princeton. In this conversation, we talk about her research on rumination and repetitive negative thinking (in the context of a partially observable Markov decision process), her work as a curator, why she enjoys teaching without grades, how to manage yourself as a PhD student, and much more.
BJKS Podcast is a podcast about neuroscience, psychology, and anything vaguely related, hosted by Benjamin James Kuper-Smith.
Support the show: https://geni.us/bjks-patreon
Timestamps
0:00:00: Teaching maths in prison
0:06:40: Teaching without grades
0:15:42: Working as a full-time research assistant (after BSc) and dealing with lots of rejections
0:25:51: How Rachel ended up doing a postdoc with Yael Niv
0:32:08: Discussing Rachel's conference proceedings 'Modelling Rumination as a State-Inference Process' (featuring partially observable Markov decision processes)
0:56:49: Rachel's background in art and curation
1:10:58: How to not turn hobbies into a stressful thing you need to get done
1:14:46: A book or paper more people should read
1:16:47: Something Rachel wishes she'd learnt sooner
1:19:05: Advice for PhD students/postdocs, with a twist: 5 tips for managing yourself during a PhD
Podcast links

Website: https://geni.us/bjks-pod
Twitter: https://geni.us/bjks-pod-twt

Rachel's links

Website: https://geni.us/bedder-web
Google Scholar: https://geni.us/bedder-scholar
Twitter: https://geni.us/bedder-twt

Ben's links

Website: https://geni.us/bjks-web
Google Scholar: https://geni.us/bjks-scholar
Twitter: https://geni.us/bjks-twt

References and links
Episodes with Matthias Stangl and Toby Wise about postdoc jobs & fellowships:
https://geni.us/bjks-wise-postdoc
https://geni.us/bjks-postdoc-stangl
Episode with Paul Smaldino on modelling social behaviour, and with Eiko Fried on theories in psychology
https://geni.us/bjks-smaldino_2
https://geni.us/bjks-fried
POMDPs: https://en.wikipedia.org/wiki/Partially_observable_Markov_decision_process
Dear World Project: https://engagement.fil.ion.ucl.ac.uk/projects/dear-world-project/
5 tips for managing yourself during a PhD: https://www.rachelbedder.com/phdtips
Scientific virtues (including stupidity): https://slimemoldtimemold.com/2022/02/10/the-scientific-virtues/
Bedder, Pisupati & Niv (2023) Modelling Rumination as a State-Inference Process. https://doi.org/10.31234/osf.io/tfjqn
Burkeman (2021). Four thousand weeks: Time management for mortals.
McCullers (1940). The Heart is a Lonely Hunter.
Montague, Dolan, Friston & Dayan (2012). Computational psychiatry. Trends in cognitive sciences.

BJKS Podcast - 82. Geoff Cumming: p-values, estimation, and meta-analytic thinking

Transcript

[This is an automated transcript with many errors]
Benjamin James Kuper-Smith: [00:00:00] Anyway, we'll be discussing statistics, new statistics, confidence intervals, all that kind of stuff. And I wanted to maybe start kind of quite broad and, uh, maybe... a little vague and that's kind of the question of what's the purpose of statistics and in particularly what I was wondering is that when I had a, you know, a little bit of a look about the history of statistics, it's, yo