Version License Platform Required Rocker

Description

This is a seminar on Open Science offered at the University of Tuebingen for students of the Hector Research Institute of Education Sciences and Psychology (and beyond).

Motivation

This lecture series does NOT aim to undermine science / research / academic psychology. On contrary, it aims to show the struggles / difficulties the community faces due to various factors (some are in our control and some are not), and to show how the community aims to overcome them to produce better science / research / academic psychology!

Regardless whether a student, a practitioner, or a tax-payer, you are a recipient of science (and it may affect you in one or the other way). Hence, YOU are also part of the process! You have an impact on the credibility revolution that is happening NOW!

Content

In recent years, the scientific community has faced a significant challenge where many key research findings failed to replicate, undermining the trustworthiness of scientific research (so called replication crisis). In response, a transformative shift has emerged (credibility revolution) focused on promoting transparency, reproducibility, and robustness through Open Science practices. This course will explore how these practices are restoring trust and enhancing the quality of research. Through a range of activities (presentations, journal-club discussions, and hands-on exercises) students gain insight into how Open Science is reshaping scientific practices. In this course, students develop practical skills for navigating, contributing, and assessing the evolving academic standards. This seminar is designed to benfit everyone regardless whether you plan to pursue a career in academia or beyond – as recipients of science, we are all impacted by its quality and advancements.

Target audiance

  • Students from BSc / MSc level (with a background in psychology / social sciences)
  • Background in statistical concepts and analysis software (e.g., R) is helful but not necessary
  • Interest in research quality, transparency, and communication

Course objectives

The primary goal of the seminar is to familiarize students with the tools and practices of Open Science, equipping them with the knowledge and skills to integrate these principles into their own academic activity but also to critically evaluate the work of others, fostering scientific rigor. To name a few specific goals:

  • Students can identify several key examples that contributed to the so-called replication crisis, as well as the practices that have since emerged to mitigate such issues in the future.
  • Students understand the rigor and current trends in academic standards that make research trustworthy. For example, students recognize the importance of transparency in research and can differentiate between high-quality, reproducible research and studies with potential biases or methodological flaws.
  • Students can evaluate the benefits and challenges of Open Science practicies in their field of study. For example, students can assess how open access policies improve the dissemination, accessibility, and quality of research with their limitations with respect to legal and ethical considerations.
  • Students have practical skills in using Open Science tools, for example, how to write reproducible analysis code and openly share their study materials in appropriate repositories.
  • Students have a good overview of what makes scientific research credible.

The course material encourages students to develop a mid-set for seeing their studies (and beyond) as an ongoing epistemic activity rather than a fixed body of knowledge that needs to be consumed.

Topics

  1. Intro: from replication crisis to credibility revolution
    • Reproducibility, Replicability, Roboustness
    • Replication crisis
    • Questionable Research Practices
  2. What can be justified? From data to conclusions
    • Hypothetico Deductive Method
    • Schools of statistical inference
  3. Unleash the power: chasing absence or certainty
    • An estimand and an estimate
    • Effect size
    • Power (bayesian and frequentist)
  4. Dealing with data like a boss: robust statistics and other opportunities
    • Quntiles
    • Bootstraping
    • Contamination
  5. Beneath the surface: data hiding and story telling with plots
    • Good plots and bad plots
    • Safe color palettes
    • Custom and high-level plot functions
  6. Contain the chaos: project and data management
    • Standardized file naming and directory structure
    • Data managment plan
    • Research compendia
  7. Version control: tiny victories with one line of code at a time 💻
    • Gentle intro to Git
  8. Reproducible environments: one setup to rule them all 💻
    • Gentle intro to Docker / Podman
  9. Reproducible workflows: your analysis code on its best behaviour 💻
    • Gentle intro to GNU Make – the ultimate tool for reproducible research
  10. Nobody is perfect but still… Open materials, code, and data
    • FAIR Data
    • Copyright and licenses
    • Ethical aspects (GDPR)
  11. Innovative publishing
    • Pre-prints
    • Pre-registration
    • Registered Reports
  12. Responsible Open Science in Europe (ROSiE project)
  13. Your mini projects (1)
  14. Your mini projects (2)

Acknowledgment

Whenever possible, I tried to acknolwedge the sources of the included information. But I am not perfect. If there is some material that is yours and you were not properly acknolwedged, or wish to remove your content, shoot me an email so that I can correct myself. 🙏

Funding

The funding to develop this course was provided by the University of Tuebingen in the framework of the prestigious Humboldt Professorship awarded to Prof. Kou Murayama. 👋

License

This work is licensed under CC BY 4.0.

(Main) References

Schönbrodt, F. D., & Ihle, M. (2024, October 1). Open Science Workshop Materials of the LMU Open Science Center. Retrieved from osf.io/zjrhu

Wilson G, Bryan J, Cranston K, Kitzes J, Nederbragt L, Teal TK (2017) Good enough practices in scientific computing. PLoS Comput Biol 13(6): e1005510. https://doi.org/10.1371/journal.pcbi.1005510

The Turing Way Community. (2022). The Turing Way: A handbook for reproducible, ethical and collaborative research (1.0.2). Zenodo. https://doi.org/10.5281/zenodo.7625728

The Turing Way Community, & Scriberia. (2021, May 29). Illustrations from the Turing Way book dashes. Zenodo. https://doi.org/10.5281/zenodo.4906004

This image was created by Scriberia for The Turing Way community and is used under a CC-BY licence.

Ioannidis, J. P. A. (2019). Why Most Published Research Findings Are False. CHANCE, 32(1), 4–13. https://doi.org/10.1080/09332480.2019.1579573

Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., Fidler, F., Hilgard, J., Kline Struhl, M., Nuijten, M. B., Rohrer, J. M., Romero, F., Scheel, A. M., Scherer, L. D., Schönbrodt, F. D., & Vazire, S. (2022). Replicability, Robustness, and Reproducibility in Psychological Science. In Annual Review of Psychology (Vol. 73, Issue 1, pp. 719–748). Annual Reviews. https://doi.org/10.1146/annurev-psych-020821-114157

Peikert, A., & Brandmaier, A. M. (2021). A Reproducible Data Analysis Workflow. Quantitative and Computational Methods in Behavioral Sciences, 1, Article e3763. https://doi.org/10.5964/qcmb.3763

Wiebels K, Moreau D. Leveraging Containers for Reproducible Psychological Research. Advances in Methods and Practices in Psychological Science. 2021;4(2). https://doi:10.1177/25152459211017853

Yarkoni T. (2020). The generalizability crisis. The Behavioral and brain sciences, 45, e1. https://doi.org/10.1017/S0140525X20001685

Oberauer, K., Lewandowsky, S. Addressing the theory crisis in psychology. Psychon Bull Rev 26, 1596–1618 (2019). https://doi.org/10.3758/s13423-019-01645-2

Vazire, S. (2018). Implications of the Credibility Revolution for Productivity, Creativity, and Progress. Perspectives on Psychological Science, 13(4), 411-417. https://doi.org/10.1177/1745691617751884

The psyTeachR team. (2024, October 16). Teaching Reproducible Psychology. psyteachr. https://psyteachr.github.io/

Frank, M. C., Braginsky, M., Cachia, J., Coles, N. A., Hardwicke, T. E., Hawkins, R. D., Mathur, M. B., & Williams, R. 2025. Experimentology: An Open Science Approach to Experimental Psychology Methods. Stanford University. https://doi.org/10.25936/3JP6-5M50.

Leipzig, J (2020). Awesome Reproducible Research: A curated list of reproducible research case studies, projects, tutorials, and media. leipzig/awesome-reproducible-research