Assessment Bias in Exam Questions: A Practical Review
Identify avoidable assessment bias with a before-and-after question example and a practical review checklist.
Assessment bias occurs when a question gives some learners an advantage or disadvantage for reasons outside the skill it is meant to measure. A complex story, unfamiliar cultural reference or inaccessible diagram can change a student’s chance of answering even when they know the target content. That does not mean every contextual detail is biased. It means each detail should earn its place in the task.
For schools, the practical question is: Could a learner who has mastered the intended outcome still miss this because of an unnecessary barrier? Ofqual’s accessible-assessment guidance emphasises appropriate language, layout and context, and the UK’s national test development framework describes expert reviews for fairness, accessibility and bias. These are principles to adapt, not a substitute for local policy or required accommodations.
Bias is not the same as difficulty
A physics problem may be difficult because it requires multi-step reasoning. That is intended difficulty. A physics problem may be unnecessarily difficult because it includes an unexplained idiom that a learner must decode before reaching the physics. That extra reading demand may be irrelevant. The target skill determines the difference.
Before writing a question, finish this sentence: “A correct answer will show that the student can ___.” Any required knowledge not in that sentence deserves review.
Four common sources of avoidable bias
- Context: a question assumes familiarity with a sport, product, local custom or household resource that has not been taught.
- Language: idioms, unusually dense syntax or ambiguous pronouns make a non-language assessment harder to interpret.
- Presentation: a diagram relies on colour alone, a table is hard to scan or the relevant text is separated from the prompt.
- Scoring: an answer key rewards a particular phrasing even when another response demonstrates the same knowledge.
A before-and-after review example
Original maths item: “During the seventh inning stretch, three friends buy two hot dogs apiece at $4 each. What is the total cost?”
Issue: baseball language is unnecessary to the multiplication task. The cost and quantity are enough. A learner unfamiliar with the event can still solve it, but may waste time interpreting the setting.
Revision: “Three people each buy two snacks. Each snack costs $4. What is the total cost? Show your calculation.” The answer is 3 × 2 × 4 = $24. This is not a claim that baseball is an inappropriate topic. If the learning goal were reading a baseball passage, the context might be relevant. The review is always relative to the construct.
A practical five-question bias review
| Question for the reviewer | Action if the answer raises concern |
|---|---|
| What learning outcome does this item measure? | Rewrite the objective or remove unrelated demands. |
| Does the context require untaught cultural or background knowledge? | Explain the context or use a more neutral one. |
| Is reading complexity necessary to assess this skill? | Simplify syntax without simplifying the target concept. |
| Can the information be perceived without colour or a tiny image? | Add labels, text alternatives or a clearer layout. |
| Could another valid response be marked wrong? | Expand the scoring guidance and examples. |
Review with people and evidence
A single writer cannot anticipate every barrier. Ask a second subject expert to answer the question and a reviewer with accessibility knowledge to inspect the wording and layout. If the assessment is important, pilot questions where practical and examine unexpected response patterns after use. A statistical difference between groups is a signal to investigate, not automatic proof of bias; content and context review still matter. Cambridge International describes multiple expert review stages in its assessment quality standards.
Build the review into the bank
Keep a record of the original item, the concern, the revision, the reviewer and approval date. This prevents an old version from quietly reappearing in a future paper. Pair the review with a test blueprint so an accessible wording change does not accidentally alter the intended skill or topic coverage. For the broader distinction between fairness, validity and reliability, read our exam quality guide.
Bottom line: remove barriers that do not belong to the learning outcome, then check the revised item with reviewers and students’ actual responses where possible.
Do not confuse review signals with a verdict
Suppose one group answers an item correctly less often than another. That pattern can arise from many causes, including differences in teaching opportunity, prior knowledge, sampling noise or the item itself. It deserves investigation, but the number alone cannot identify the cause. Preserve the item version and context, inspect the content with appropriate expertise and avoid blaming learners for a poorly specified question.
The same caution applies to reader feedback. One student finding a passage confusing is useful evidence to inspect the wording, not proof that the assessment is biased. In a small classroom, qualitative review may be more informative than a formal statistic. The goal is to improve the item and the decision it supports, not to attach a label without evidence.