Hidden bias in standardized tests
Low-SES students tend to score lower than expected on math items that refer to money, food, and social interactions
Interviewee:
Interview by:
Partner
npj Science of Learning
Marjolein Muskens has found that educational systems and teaching methods can inadvertently limit the opportunities of certain students. She is interested in students’ hidden talents and shedding light on the unintended obstacles that hinder students from disadvantaged backgrounds. Marjolein tells Annie Brookman-Byrne about her research into the bias found in standardized math tests.
Annie Brookman-Byrne: What inspired you to research educational inequality?
Marjolein Muskens: Education should promote empowerment and equality rather than perpetuating disadvantage. It is disturbing to think that systemic barriers that are not immediately visible or intentional might prevent talented individuals from achieving their full potential. Such barriers can unfairly hinder these individuals’ progress and opportunities, which is both a waste of their talents and an injustice to them as individuals.
“Education should promote empowerment and equality rather than perpetuating disadvantage.”
Conversations with researcher Willem Frankenhuis inspired me to explore the concept of “hidden talents.” We discussed how certain abilities may be overlooked or undervalued because of biases in how we recognize and assess talent, particularly in the case of students from disadvantaged backgrounds. These hidden talents can play a significant role in such students’ future success, yet they often go unnoticed because of a failure to recognize that talent can manifest itself in different ways.
I seek to identify and address these overlooked talents. My goal is to help create educational environments that recognize and nurture the potential of all students, regardless of their backgrounds or circumstances. By focusing on such hidden opportunities, we can help create a more equitable educational system in which every student has the chance to succeed and thrive.
ABB: What did you find in your recent research with Willem?
MM: Math tests are used around the world to determine whether students graduate or are admitted to certain educational programs. These tests are seen as an important measure of academic achievement and are used to select students for tertiary education. However, the tests unintentionally disadvantage certain students, such as those from low socioeconomic status (SES) backgrounds. We examined whether and how the content of these tests may cause bias.
“The tests unintentionally disadvantage certain students, such as those from low socioeconomic status (SES) backgrounds.”
Surprisingly, low-SES students were less likely to answer correctly math questions that have content related to money, food, or social interactions compared to similar questions that didn’t mention that content. We had assumed that these students would perform better when content was linked to the real world, since they are likely to have hidden talents in these areas. It may be that the content distracted students by reminding them of financial stress or social challenges, making it more difficult for them to focus. It is also possible that they found it difficult to apply their informal knowledge to formal mathematical tasks.
Content related to the real world may therefore make it more difficult for low-SES students to solve math problems, particularly when they are required to draw analogies. We need to learn more about bias in standardized tests that may perpetuate educational inequality, with a particular focus on items that reference money, food, and social interactions. It is important to understand what kinds of content trigger a negative effect and what kinds do not. We need to ensure fair and equal opportunities for all students, regardless of their backgrounds.
“We need to ensure fair and equal opportunities for all students, regardless of their backgrounds.”
ABB: How can your findings be used to improve children’s outcomes?
MM: When constructing assessments that have significant consequences for evaluations, graduation, or admissions, test designers should be cautious about using money, food, and social interactions as examples. These topics appear to have emotional or practical associations that can hinder performance. It would be better to choose subjects that are less likely to carry such burdens.
However, we do not suggest eliminating real-world examples from math instruction altogether. Indeed, examples that relate to money and food can be highly relevant and helpful when learning mathematics. Our caution applies to using them in summative assessments, where the stakes are higher, and when bias may negatively affect certain students. Policymakers and teachers should be mindful of these issues when designing and administering tests, to avoid placing a further burden on students from disadvantaged backgrounds. The aim is to promote fairness in educational assessments.
ABB: What has surprised you while working on this topic?
MM: Our study has sparked significant interest and discussion among scholars and policymakers, and on social media as well. However, I have been surprised to discover that our results are not always fully understood. Some experts who focus on developing effective math instruction have concluded that realistic examples should be avoided, or reserved for a later stage. They believe that using examples from real life risks lowering the quality of math education, and have embraced our findings as support for their viewpoint. But our research does not look at math instruction in general, but specifically at summative assessments.
In fact, because a goal of math education is to equip students with skills they can apply in real-world situations, we think that more attention should be given to the real-world context during the learning process. Students who are unable to apply their math skills effectively in these contexts might benefit from more instruction on these topics. Learning to handle practical situations—such as managing finances and avoiding debt—would benefit all students.
Footnotes
Marjolein Muskens, a developmental psychologist with a PhD in educational economics, specializes in educational equity. As a senior researcher at KBA Nijmegen, she conducts scientific research and leads complex policy studies on equal opportunity and talent development. She is the lead researcher for a multi-year study on effective strategies to improve educational equity in primary and secondary education in the Netherlands, funded by the Dutch Research Council (NWO). She also serves as a member of the Educational Inequality Research Group in Radboud University’s Faculty of Social Sciences, contributing to scientific research on systemic barriers and interventions that promote fair access to education.
Marjolein on LinkedIn
Marjolein Muskens contributed to a special collection on understanding and addressing inequality in education in the journal npj Science of Learning. This interview is part of a series dedicated to sharing practical takeaways and personal insights from authors.
The interview has been edited for clarity.
Interviewee:
Interview by:
Partner
npj Science of Learning