Changing How We Do Science with Brian Nosek [Idea Machines #3]

12/08/18 • 58 min

My guest this week is Brian Nosek, co-Founder and the Executive Director of the Center for Open Science. Brian is also a professor in the Department of Psychology at the University of Virginia doing research on the gap between values and practices, such as when behavior is influenced by factors other than one's intentions and goals.

The topic of this conversation is how incentives in academia lead to problems with how we do science, how we can fix those problems, the center for open science, and how to bring about systemic change in general.

Show Notes

Brian’s Website

Brian on Twitter (@BrianNosek)

Center for Open Science

The Replication Crisis

Preregistration

Article in Nature about preregistration results

The Scientific Method

If you want more, check out Brian on Econtalk

Transcript

Intro

[00:00:00] This podcast I talked to Brian nosek about innovating on the very beginning of the Innovation by one research. I met Brian at the Dartmouth 60th anniversary conference and loved his enthusiasm for changing the way we do science. Here's his official biography. Brian nozik is a co-founder and the executive director for the center for open science cos is a nonprofit dedicated to enabling open and reproducible research practices worldwide.

Brian is also a professor in the department of psychology at the University of Virginia. He's received his PhD from Yale University in 2002 in 2015. He was on Nature's 10 list and the chronicle for higher education influence. Some quick context about Brian's work and the center for open science.

There's a general consensus in academic circles that there are glaring problems in how we do research today. The way research works is generally like this researchers usually based at a university do experiments then when they have a [00:01:00] result they write it up in a paper that paper goes through the peer-review process and then a journal publishes.

The number of Journal papers you've published and their popularity make or break your career. They're the primary consideration for getting a position receiving tenure getting grants and procedure in general that system evolved in the 19th century. When many fewer people did research and grants didn't even exist we get into how things have changed in the podcast.

You may also have heard of what's known as the replication crisis. This is the Fairly alarming name for a recent phenomena in which people have tried and failed to replicate many well-known studies. For example, you may have heard that power posing will make you act Boulder where that self-control is a limited resource.

Both of the studies that originated those ideas failed to replicate. Since replicating findings a core part of the scientific method unreplicated results becoming part of Cannon is a big deal. Brian has been heavily involved in the [00:02:00] crisis and several of the center for open science is initiatives Target replication.

So with that I invite you to join my conversation with Brian idzik.

How does open science accelerate innovation and what got you excited about it?

Ben: So the theme that I'm really interested in is how do we accelerate Innovations? And so just to start off with I love to ask you sort of a really broad question of in your mind. How does having a more open science framework help us accelerate Innovations? And I guess parallel to that. Why what got you excited about it first place.

Brian: Yeah, yeah, so that this is really a core of why we started the center for open science is to figure out how can we maximize the progress of science given that we see a number of different barriers to or number of different friction points to the PACE and progress of [00:03:00] Science.

And so there are a few things. I think that how. Openness accelerates Innovation, and I guess you can think of it as sort of multiple stages at the opening stage openness in terms of planning pre-registering what your study is about why you're doing this study that the study exists in the first place has a mechanism of helping to improve Innovation by increasing The credibility of the outputs.

Particularly in making a clear distinction between the things that we planned in advance that we're testing hypotheses of ideas that we have and we're acquiring data in order to test those ideas from the exploratory results the things that we learn once we've observed the data and we get insights bu...

Show Notes

Brian’s Website

Brian on Twitter (@BrianNosek)

Center for Open Science

The Replication Crisis

Preregistration

Article in Nature about preregistration results

The Scientific Method

If you want more, check out Brian on Econtalk

Transcript

Intro

So with that I invite you to join my conversation with Brian idzik.

How does open science accelerate innovation and what got you excited about it?

Previous Episode

Venture Capital Meets Fusion Power with Malcolm Handley [Idea Machines #2]

My Guest this week is Malcolm Handley, General Partner and Founder of Strong Atomics.

The topic of this conversation is Fusion power - how it’s funded now, why we don’t have it yet, and how he’s working on making it a reality. We touch on funding long-term bets in general, incentives inside of venture capital, and more.

Show Notes

Strong Atomics

Malcolm on Twitter (@malcolmredheron)

Fusion Never Plot

Fusion Z-Pinch Experiment.

ARPA-e Alpha Program

ITER - International Thermonuclear Experimental Reactor.

NIF - National Ignition Facility

ARPA-e

Office of Fusion Energy Science

Sustainable Energy without the Hot Air

Transcript

[00:00:00] This podcast I talk to Malcolm Hanley about Fusion funding long-term bets incentives inside of venture capital and more Malcolm is the managing partner of strong atomics. Strong atomics is a venture capital firm that exists solely in a portfolio of fusion projects that have been selected based on their potential to create net positive energy and lead to plausible reactors before starting strong atomics.

Malcolm was the first employee at the software company aside. I love talking to Malcolm because he's somewhat of a fanatic about making Fusion Energy reality. But at the same time he remains an intense pragmatist in some ways. He's even more pragmatic than I am. So here in the podcast. He thinks deeply about everything he does.

So we go very deep on some topics. I hope you enjoy the conversation as much as I did.

Intro

Ben: Malcolm would you would you introduce yourself?

Malcolm: Sure. So I'm Malcolm heavily. I found in strong [00:01:00] atomics after 17 years is software engineer because I. I was looking for the most important thing that I could work on and concluded that that was kind of change that was before democracy fell off the rails.

And so it was the obvious most important thing. So

my thesis is that climate change is a real problem and the. Typical ways that we are addressing it or insufficient, for example, even if you ignore the climate deniers most people seem to be of the opinion that we're on track that Renewables and storage for renewable energy are going to save the day and my fear as I looked into this more deeply is that this is not sufficient that we are in fact not on track and that we need to be looking at more possible ways of responding to [00:02:00] climate change.

So I found an area nuclear fusion that is that it has the potential to help us solve climate change and that in my opinion is underinvested. So I started strong atomics to invest in those companies and to support them in other ways. And that's what I'm doing these days

What did founding strong atomics entail?

Ben: and he did a little bit more into what founding strong atomics and Tails. You can just snap your fingers and bring it into being

Malcolm: I almost did because it was extremely lucky but in general Silicon Valley has a pretty well worn model for how people start startups and I think even the people getting out of college actually no a surprising amount about how to start a company and when you look at Fusion companies getting started you realize just how much knowledge we take for granted in Silicon Valley.

On the other hand as far as I can tell the way [00:03:00] that every VC fund get started in the way that everyone becomes a VC is unique. It was really one story for how you start a company and there are n stories for how funds get started. So in my case, I wasn't sure that I wanted to start a fund more precisely.

It hadn't even occurred to me that I would start a fund. I was a software engineer and looking for what I could do about climate change. I'm just assuming that I was looking for a technical way to be involved with that. I was worried because my only technical skill is software engineering but I figured hey, but software you can do many things.

There must be a way that a software engineer can help. So I made my way to The arpa-e Summit in DC at the beginning of 2016 and went around and talked to a whole lot of people if they're different boots about what they were doing and. My questions for myself was does what you're doing matter. My question for them was how might a software engineer help [00:04:00] and to a first approximation even at a wonderful conference like the arpa-e summit.

...

Next Episode

Rethinking R&D with Adam Wiggins [Idea Machines #4]

My Guest this week is Adam Wiggins, the cofounder of Ink & Switch — an independent industrial research lab working on digital tools for creativity and productivity.

The topic of the conversation is the future of product-focused R&D, the Hollywood Model of work in tech, Ink & Switch’s unique organizational structure, and whether it can be extended to other areas of research.

Links

Adam Wiggins’ Home Page

Adam on Twitter

Ink & Switch's Home Page

A presentation on Ink & Switch's Structure

Sloan Review Article on Applying Hollywood Model to R&D (Paywalled)

Transcript

How the idea came about

Ben: How did you come up with this idea? Like wait what what originated that I'm just really interested in the thought process behind there

Adam: sure, you know, I think me and my partner's we come out of the sort of the startup kind of school of thought on Innovation, I think.

There's a lot of way to think about there's the more academic research minded approach to Innovation. There's made which get a bigger companies. So yeah, we come out of very much from the yeah. I don't know what you want to call it ad Jolene startup y combinator or whatever that you know mix of elements is which is really about build a thing really quickly get it in front of customers minimal viable product innovate, but at least my thinking is that the startup model has been so successful in the last let's say decade.

Particularly with the kind of mass production of the startup that you get through groups like y combinator such that I feel like the problems the space of problems that can be solved with that kind of, you know group of 25 25 year old Founders spending three months to build a thing not say it's let's say saturated.

Yeah to some degree in that maybe the more interesting problems are like bigger or longer in scope. And so then we thought about okay. Well, what's a what's a model that is more possible for going after bigger things. And that's when I kind of fell down the rabbit hole of researching these Industrial Research Labs.

I know that you spent a lot of time on as well, you know, these big famous examples like Bell labs and Xerox Parc and arpa and so forth. And of course many other examples when we thought okay, well, You know, we're not at the we're not in a position to you know, be setting up a multimillion-dollar research arm of a government or commercial institution.

But what can we do on a smaller scale with a small Grant and it's kind of a scrappy band and people and that's kind of what led us to the Incan switch approach.

The Thought Process Behind the Model

Ben: can you go one step further where it's you have the constraint that you can't do a straight-up corporate research lab, but I think there are a lot of unique ideas in terms of a model that are sort of just unique and.

In that like how did you cope that Lee idea that like, okay, we're going to like have our principles. We're going to pull in people temporarily. We're going to build this network that that seems sort of to come out of the blue. So what was what was the thought process behind that?

Adam: Well, maybe it came out of the constraint of do it with very little money. And so part of that is we're trying to work on a big problem. Hopefully and I can talk about that if you want, but the in terms of the the model that we're using we came at it from do it with very little money and that in turn leads to okay.

Your big costs are usually sort of like office space and then the people right, but if we can do these really short term projects, we called the Hollywood model and I can explain about that if you want the basically we have like a four or six or eight week project. You can bring in some experts on a freelance basis and you don't necessarily need to commit to paying salary is over the longer term and you couple that with no office.

We have an all distributed team. We're not asking people they don't need to pick up. Move somewhere to even temporarily to work on a project. Right? And so we what we can offer them as a lot of flexibility. And so the I think there's certain there's benefits for the people to participate in these projects join, but from the lab point of view again, it was we were embracing this constraint of do it really really cheap.

Yeah and that basically boiled down to very short projects people on a freelance basis only no office and that that's kind of what what led us there, but I think there actually is a lot. Benefits to doing things that way there's some big do...