
Episode 32: Measurement Schmeasurement (with Jessica Flake)
Explicit content warning
10/16/19 • 80 min
Jessica Flake joins Mickey and Yoel to talk measurement. What is it, how do you do it well, and do social psychologists care about it? What does measurement theory tell us about the validity of standardized tests like the GRE? Jessica also talks about how she went from high-school dropout to professor at McGill.
Bonus: what the hell is Kentucky Gentleman?
Special Guest: Jessica Flake.
Links:
- Brasserie Harricana, Montréal, Quebec, Canada | RateBeer
- JK Flake 📈📏 on Twitter: "What? The GRE is not a face valid measure of quantitative + verbal reasoning? These discussions are off the rails."
- Construct Validation in Social and Personality Research: Current Practice and Recommendations - Jessica K. Flake, Jolynn Pek, Eric Hehman, 2017
- PsyArXiv Preprints | Measurement Schmeasurement: Questionable Measurement Practices and How to Avoid Them
- Psychological Science Accelerator – a distributed laboratory network
- Kentucky Gentleman - Wikipedia
- Warpaint - 'Billie Holiday (Rough Trade Sessions)' - YouTube
Jessica Flake joins Mickey and Yoel to talk measurement. What is it, how do you do it well, and do social psychologists care about it? What does measurement theory tell us about the validity of standardized tests like the GRE? Jessica also talks about how she went from high-school dropout to professor at McGill.
Bonus: what the hell is Kentucky Gentleman?
Special Guest: Jessica Flake.
Links:
- Brasserie Harricana, Montréal, Quebec, Canada | RateBeer
- JK Flake 📈📏 on Twitter: "What? The GRE is not a face valid measure of quantitative + verbal reasoning? These discussions are off the rails."
- Construct Validation in Social and Personality Research: Current Practice and Recommendations - Jessica K. Flake, Jolynn Pek, Eric Hehman, 2017
- PsyArXiv Preprints | Measurement Schmeasurement: Questionable Measurement Practices and How to Avoid Them
- Psychological Science Accelerator – a distributed laboratory network
- Kentucky Gentleman - Wikipedia
- Warpaint - 'Billie Holiday (Rough Trade Sessions)' - YouTube
Previous Episode

Episode 31: Is Ego Depletion Real?
By listener request, Yoel quizzes Mickey about ego depletion. How did we start studying it? How has the replication crisis changed how we think about it? After more than a decade studying ego depletion, does Mickey still have any faith in the phenomenon?
Bonus: what does it mean to say, "don't @ me"?
Links:
- De Hemel Brewery
- Don't @ me
- The Strength Model of Self-Control — Self-control is a central function of the self and an important key to success in life. The exertion of self-control appears to depend on a limited resource.
- Is Ego Depletion Real? An Analysis of Arguments — An influential line of research suggests that initial bouts of self-control increase the susceptibility to self-control failure (ego depletion effect). Despite seemingly abundant evidence, some researchers have suggested that evidence for ego depletion was the sole result of publication bias and p-hacking, with the true effect being indistinguishable from zero.
- Self-Control, Ego Depletion, and Social Psychology’s Replication Crisis — Provides Baumeister's perspective on ego depletion and its status in the context of psychology's replication crisis. Reviews history, controversy, evidence.
- A series of meta-analytic tests of the depletion effect: Self-control does not seem to rely on a limited resource. - PsycNET — We find very little evidence that the depletion effect is a real phenomenon, at least when assessed with the methods most frequently used in the laboratory. Our results strongly challenge the idea that self-control functions as if it relies on a limited psychological or physical resource.
- Bias-Correction Techniques Alone Cannot Determine Whether Ego Depletion is Different from Zero: Commentary on Carter, Kofler, Forster, & McCullough, 2015 by Michael Inzlicht, Will Gervais, Elliot Berkman :: SSRN — Despite our admiration for this program of meta-research, we suggest that bias-corrected meta-analyses cannot yet resolve whether the overall ego depletion is different from zero or not.
- RRR – The Ego-Depletion Paradigm — A Multi-Lab Pre-Registered Replication of the Ego-Depletion Paradigm Meta-analysis of the studies revealed that the size of the ego-depletion effect was small with 95% confidence intervals (CIs) that encompassed zero (d = 0.04, 95% CI [−0.07, 0.15].
Next Episode

Episode 33: What is Heterodox Academy? (with Debra Mashek)
Mickey and Yoel talk with Debra Mashek, the executive director of Heterodox Academy, an organization working to increase open inquiry, viewpoint diversity and constructive disagreement in academia. But what does that mean exactly? Aren't these just codewords for "more conservatives"? We talk to Debra to find out.
We also ask Debra about her decision to leave her tenured faculty position and move across the country to take this job.
Special Guest: Debra Mashek.
Links:
If you like this episode you’ll love
Episode Comments
Generate a badge
Get a badge for your website that links back to this episode
<a href="https://goodpods.com/podcasts/two-psychologists-four-beers-40450/episode-32-measurement-schmeasurement-with-jessica-flake-1828258"> <img src="https://storage.googleapis.com/goodpods-images-bucket/badges/generic-badge-1.svg" alt="listen to episode 32: measurement schmeasurement (with jessica flake) on goodpods" style="width: 225px" /> </a>
Copy