Ben Domingue is a statistics researcher in the Graduate School of Education at Stanford University. Ben studies the tools that researchers use to measure children’s outcomes. He is developing online platforms for storing and sharing complex data. Annie Brookman-Byrne talks with Ben about the explosion of psychological data and why making such data accessible is so important for improving children’s lives.

Annie Brookman-Byrne: What is psychometrics, and what are you trying to find out through your research?

Ben Domingue: Suppose you want to understand whether some new educational technology improves learner abilities, or whether a school “belongingness” program actually improves student wellbeing. How do you measure something intangible like ability or belongingness? Psychometrics is the scientific field that designs tools to measure constructs like these.

Psychometric tools are used to measure outcomes in education, psychology, social science, and medicine. I work on techniques for analyzing data collected from psychometric measures.

“How do you measure something intangible like ability or belongingness?”

ABB: How will your research help children?

BD: Psychometrics can improve the systems that help children develop. For example, I am building a website that will provide researchers with large amounts of psychometric data in a standardized format. The goal is to make it easier for researchers to test their theories and models about children’s learning and development. This may sound a little boring, but it’s really important! Improving research infrastructure like this may lead to innovations in psychometrics. For example, it might enable scientists to develop and test better methods for dealing with complex data that have been collected from the same learners over a long period of time. Advances in such approaches may help educators make better sense of student learning over time.

I am also working with a team to develop an online platform that can be used in a broad array of settings and countries to measure skills of interest in child development. It will be exciting to see how the platform works and then see it embedded in a large number of new studies in the near future. 

ABB: What changes have you seen in psychometrics?

BD: Psychometrics goes back at least a century, so having worked in the field for just 15-20 years, I’ve only seen a limited period of its history! When I entered the field, few computational tools were available to allow researchers to measure complex constructs. Now there are many more such tools, which makes it much easier for researchers to apply advanced methods.

We are also seeing an explosion of data from psychological measures, as many can now be administered electronically – tests of oral reading fluency, for example. These data are useful, but there is a problem: They are often hard to find, and difficult to use with different software or to address new research questions. This greatly limits the degree to which such data can help other researchers develop new ideas. I spent some years working with genetic data that were easily shared and used across different research teams. I was impressed by that experience, and I think the field of psychometrics could learn from that approach!

“The goal is to make it easier for researchers to test their theories and models about children’s learning and development.”

ABB: What is the biggest challenge for psychometrics?

BD: The field will need to reckon with how to measure dynamic abilities. For the last century, we’ve taken a snapshot of abilities by having people run through a bunch of tasks in a single sitting lasting a few hours. We have an enormous variety of tools for understanding such scenarios. In the future, technology will allow us to continuously collect small bits of information about learners’ capacities that we’ll convert to a “video” showing their changing capacities over time.

Imagine a child in a math classroom, for example. Each week they might answer ten items, thus providing a small window into their current abilities in math. Over the course of the year, hundreds of bits of such information will capture their evolving capacities, and psychometrics will need to stitch that information into a coherent view of what the child knows and can do. Approaches are emerging for tackling this kind of scenario, but I think it will be an area of massive growth in the future.

More from Ben Domingue
Has the COVID-19 crisis affected reading development?

ABB: What are your hopes for the future in this field?

BD: In the near-term future I’d love to see psychometrics become a more data-rich discipline. If psychometrics researchers use more data, they will be able to further advance psychometric methods and develop better theories. More kinds of data can now be collected during psychological measurement than ever before. Response time is a great example: In many psychological experiments, researchers want to know how long it takes participants to respond when carrying out a task, as that gives insight into how information is processed in the brain. Collecting such data was challenging before computers were the default for administering many assessments. Response time is now easily collected when a key is pressed.

But collecting the data is just the first step. We now need to focus on making the data available in ways that both align with privacy mandates and restrictions on data sharing and allow a broad community of scientists to use large volumes of data to advance their ideas. This will ultimately help them improve children’s lives.

Leave a Reply

Your email address will not be published. Required fields are marked *