Method Workshops

Instructor:  Timo Gnambs, Leibniz Institute for Educational Trajectories, Bamberg, Germany

Meta-analyses allow summarizing results from multiple primary studies to derive an estimate of a pooled effect and its variability across contexts. Experimental designs with more than two conditions require pooling dependent effect sizes because each experimental condition is compared to a common control condition. The workshop will highlight the problems of ignoring these dependencies and present several meta-analytic approaches that appropriately account for dependent effect sizes (e.g., three-level and multivariate meta-analysis). The implementation of these models will be demonstrated in R using the metafor package. Participants are expected to have a basic understanding of univariate meta-analysis as a statistical method for aggregating results across primary studies and should be familiar with the basics of R.

Instructors: Claudia von Aufschnaiter, University of Gießen & Verena Petermann, University of Gießen, Germany

In a first phase, the workshop will introduce participants to the idea of validity as an argument in education research. Current concepts (APA, Kane, Messick) will be briefly presented, with a special focus on six central aspects of validity, which are illustrated with an example. In a second phase, participants will work in groups, using a grid, on the six central aspects and discuss the validity of an exemplary psychological study. The main goal of the workshop is to prepare students to construct an argument about the validity of conclusions drawn from their own study.

Instructor: Johannes Naumann, University of Wuppertal, Germany

Scenarios in which an independent variable exerts its influence on a dependent variable through a third variable (mediation), or where its effect on the dependent variable are conditional on a third variable (moderation) are ubiquitous in psychological and educational research. This workshop will cover the basics of estimating models that include moderation, mediation, or both in combination (moderated mediation/mediated moderation), and testing their respective effects for significance. We will cover models with continuous and categorical independent variables, as well as models with both manifest and latent variables. All analyses will be conducted using the R environment. Participants will be briefed in advance on the required software and packages required. 

Instructor: Christian Maximilian Thurn, ETH Zürich, Switzerland

This workshop provides a comprehensive overview of network analysis in learning sciences and education. You will learn how to construct, analyze, and visualize networks. This includes the most common data formats, global and local network indices, and the limitations of the method. Through a combination of lecture, discussion, and hands-on exercises, the goal is to develop a solid foundation in network analysis to enable you to apply network analysis to your own research questions. We will use R and the igraph package. Moderate R skills are required (such as data import, ggplot, basics of tidyverse, debugging). You should bring a laptop with R and RStudio, and at least the ggplot2 and igraph packages installed. You can bring your own datasets, but sample datasets will also be provided. If you would like to read a teaser on network analysis already: https://de.in-mind.org/blog/post/mehr-als-connecting-the-dots-was-netzwerkanalysen-abbilden-koennen

Instructor: Simon Tiffin-Richards, University of Kiel, Germany

Eye movements are a valuable measure of cognitive processes involved  in a wide range of tasks such as reading, scene perception, and visual search (Rayner, 2009). In this workshop, we will look at the basic  characteristics of eye movement patterns and how these may be interpreted in terms of cognitive processes. We will concentrate on how eye movement measurements are used in reading research to analyse higher-level processes of comprehension building and lower-level word recognition processes. In the course of the workshop, we will discuss a range of different experimental designs to provide a broad insight into eye movement research.

Instructor: Marina Klimovich, University of Würzburg, Germany

R is an open-source software for performing statistical analyses. This workshop is intended as an introduction to R and its basic functionality. The objectives of this course are to learn how to (1) import and export data from different sources, (2) deal with large, messy data (organizing and transforming data sets), and (3) perform descriptive and basic inferential statistics. You will acquire these skills by learning R syntax. The R-scripts developed in this course can be saved, commented on, and adapted for your own use. No programming background is required, but please make sure you have the latest version of R and RStudio installed before the workshop.

Instructors: Mirjam Ebersbach, University of Kassel, Germany & Tobias Richter, University of Würzburg, Germany

Doctoral students are expected to publish their work in international journals with peer review but getting started in the process of publishing can be a challenge. This workshop will cover the basic steps of publishing in peer-reviewed journals in psychology and the learning sciences, from selecting the right journal, planning and writing the paper, up to mastering the review and revision process. We aim to provide background knowledge about the writing and publication process that will help doctoral students with little or no publication experience to make the right decisions, to present their research effectively, and to get their manuscripts published in a good journal.

Instructor: Wolfgang Lenhard, University of Würzburg, Germany

The free statistical environment R provides flexible and advanced methods for applying psychometric analyses to tests. This workshop offers a hands-on-tutorial for young empirical scientists and explains psychometrics using the packages foreign, psych, TAM, eRM and cNORM. No prior knowledge of using R is required, but already having “played” with RStudio and R will reduce the cognitive load for you. We will go through the program step by step, and you will be able to practice each analysis on your own computer. The workshop includes visualization, basic analyses such as assessing item discrimination and scale homogeneity, and progresses to advanced methods such as IRT modeling, analysis of differential item function, assessing local independence, and testing model assumptions.

Instructor: Andreas Wertgen, University of Würzburg, Germany

Data in experimental research typically consist of multiple levels (e.g., persons nested in test items and test items nested in persons, students nested in classes, classes nested in schools). However, analyses based on ANOVA or multiple regression cannot adequately account for multiple levels, leading to biased results and flawed hypothesis testing. In contrast, mixed-effects models are advantageous for efficiently analyzing multi-level data. In two sessions, this workshop will provide a brief introduction to mixed-effects models for experimental researchers and hands-on exercises using the R-package lme4. No R experience is required.

Instructor: Veit Kubik, University of Würzburg, Germany

This workshop will provide an overview of the principles and practices of Open Science (OS), such as open data, open code, open materials, and preregistration/registered reports. These OS practices can help to prevent questionable research practices such as HARKing (i.e., formulating the hypotheses after the results are known) and p-hacking (i.e., analyzing data in way that produces statistically significant results when there is no effect or association), which have contributed to the replication crisis and publication bias in psychology and many other scientific disciplines. In this workshop, I will introduce specific Open Science tools (e.g., osf.io, as.predicted.com), demonstrate how to implement OS practices with these tools, and provide recommendations on how to efficiently integrate them into participants’ research workflow.