Open Science and how to do better research with better statistics

by Geoff Cumming
|
8 books
For a decade, Open Science practices have been sweeping across many scientific disciplines--perhaps the most important recent advance in how science is done. They simply make sense, are easy to grasp and a joy to teach. Open Science aims to make research more efficient, and published research more trustworthy. It arose partly in response to the terrible realisation that many ‘established’ research findings could not be replicated—they were wrong ☹ Open Science aims to make science more open by requiring very full descriptions of what was done and the complete data, all freely available online. It also aims to encourage replication: Studies need to be repeated by others. Open Science requires integration of results from related studies, which is ‘meta-analysis’. Meta-analysis requires estimation information—often a mean and confidence interval—from each included study. Estimation and meta-analysis (“the new statistics”) need to be at the heart of statistics practice and statistics education. Happily, they are a joy to teach, way better than traditional significance testing, which has caused so much damage to research. Even beginning students easily grasp the essentials of estimation, meta-analysis, and Open Science. The way of the future, here now. Enjoy!
Read more... Read less...

Introduction to the New Statistics

Estimation, Open Science, and Beyond
Geoff Cumming, Robert Calin-Jageman, 2024
The second edition of my second book. Even beginning students grasp the basics of Open Science and the statistical approach it needs—estimation and meta-analysis, aka ‘the new statistics’. They are a joy to teach, especially compared with statistical significance testing, the traditional approach. This edition introduces the wonderful esci software, open-source and very easy to use. esci is ideal for this book and also for more advanced courses, and by researchers. More at https://thenewstatistics.com

Understanding The New Statistics

Effect Sizes, Confidence Intervals, and Meta-Analysis
Geoff Cumming, 2013
I ‘retired’ from La Trobe University to write this, my first book. In it, I explain the severe limitations of traditional statistical significance testing and why estimation—effects sizes and confidence intervals—is so much better. The book assumes very little and is intended for students at every level and for researchers. Estimation extends to meta-analysis, which combines results from a number of related studies. I use the term ‘the new statistics’ for estimation and meta-analysis—which are not new, but their use as the main approach to data analysis would for many disciplines be new, and a great step forward. There are three chapters on meta-analysis that back then was little known by students and many researchers. The book has helped push along the shift in emphasis from significance testing to better methods. More at https://thenewstatistics.com

A Student's Guide to Open Science: Using the Replication Crisis to Reform Psychology

Charlotte Pennington, 2023
An ideal book to accompany our intro book! Charlotte Pennington eagerly started her PhD but became anxious and depressed when her experiments ‘didn’t work’ and were refused publication. She read about the replication crisis—many published studies won’t replicate—and Open Science. Soon it was her passion to teach about Open Science, and what’s needed to achieve efficient research and a trustworthy research literature. The result is A Student’s Guide to Open Science. It’s short, clear, and highly readable. She’s well-informed and covers the story in engaging detail. There are practical guides to using Open Science practices and numerous links to follow as you wish. Statistical issues are mentioned but are not her focus, so her book fits very well alongside ours.

The Seven Deadly Sins of Psychology

A Manifesto for Reforming the Culture of Scientific Practice
Chris Chambers, 2019
This book diagnoses the problems and explains why we need Open Science. Chris Chambers tells of his disillusionment with so many aspects of what researchers were doing, in psychology, but also in medicine and many other fields. That rang true to me—I traveled that same road. He goes on from explaining the problems to describing solutions. Many of these, including openness, better statistics, replication, and increased scrutiny, are now being advocated or required by funders and journal editors and adopted by researchers. That’s Open Science, hooray!

Science Fictions

Exposing Fraud, Bias, Negligence and Hype in Science
Stuart Ritchie, 2021
I love how Ritchie starts with a rather wry account, addressed directly to you as a would-be researcher, of the challenges and weirdness you’ll encounter as you launch into your own research. His own research helped sparked recognition of the ‘replication crisis’, so he’s well placed to tell us about misguided practices and the sometimes wicked deeds of researchers. More happily, he describes how we can return to the sound foundations of good scientific practice. A current term for that is ‘Open Science’.

Beyond Significance Testing

Statistics Reform in the Behavioral Sciences
Rex B. Kline, 2015
You may have heard of ‘significance testing’, and the magical ‘p < .05’, which somehow makes a research result ‘significant’, which is often taken as (almost) ‘true’. Even if you haven’t heard of all that, Kline explains clearly why significance testing has been disastrous for science, leading to misleading conclusions and much valuable research not even being reported. He draws on my work to explain how ‘the new statistics’ (estimation) is a much better way to understand results. The first chapter is fairly easy reading. Later chapters are also terrific but get more technical as Kline explains lots of ways to do things better. As I’m quoted on the back cover “Read this book and see the future!” Happily, the future is increasingly looking as Kline recommended.

Research Methods in Psychology

Evaluating a World of Information
Beth Morling, 2021
Yes, this is a textbook, but if you are seeking a research design and methods text for psychology or a related discipline, this is easily my top choice. There are lots of references to topical stories to keep everything relevant for students. There’s a truckload of valuable stuff online to support both teachers and learners. This fourth edition is right up-to-the-moment, Chapter 3 especially so, as it explains three types of scientific claims, and four types of validity that researchers should aim to achieve. That may sound forbidding, but Morling’s examples and explanations are pleasingly accessible.

The Design of Experiments in Neuroscience

Mary Harrington, 2011
Another research design textbook, this one more specifically about neuroscience. My co-author, neuroscientist Robert Calin-Jageman, highly recommends it. This third edition has clear and up-to-date discussions of issues such as p hacking and publication bias that emphasise the need for Open Science. There’s a focus on effect sizes and confidence intervals, as in the new statistics. The book also describes strategies needed to enhance the rigor and reproducibility of neuroscience research.