Writing Achievement and Aptitude Test Items

Cognitive Test Development

Rizqy Amelia Zein

Departemen Psikologi, Universitas Airlangga

2026-09-16

Agenda

  1. Anatomy of a multiple-choice item
  2. Writing the stem
  3. Writing the answer options
  4. Writing achievement test items vs. aptitude test items
  5. Example aptitude test items based on narrow ability
  6. Example achievement test item from a TIK (learning objective)
  7. Activity: spotting errors in an item

What you’ll submit this week

  • Assignment 2 is done individually: write 20 items — 2 items for each achievement test indicator (5 indicators) and 2 items for each aptitude test indicator (5 indicators).
  • Each item should follow the blueprint you developed yourself (Meetings 2 and 4).
  • Today’s material is a practical guide for the item-writing stage, which is Stage 5 of the eleven stages covered in Meeting 2.

Anatomy of a Multiple-Choice Item

The parts of an item

  • Stem — the part that contains the core question or statement.
  • Options — the answer choices provided, consisting of:
    • Key — the correct answer.
    • Distractors — incorrect choices, but designed to still seem plausible.

Why these parts need to be clearly separated

Note

The stem and options serve different functions: the stem conveys one clear core idea, while the options provide alternative answers that are equivalent in form. A good item doesn’t mix the two — for example, by putting part of the essential information in only one of the options.

Writing the Stem

Basic principles

  • State the core idea clearly and concisely in the stem — don’t tuck it away in the options.
  • Use positive phrasing; avoid negative forms such as “EXCEPT” or “NOT”, which are easy to miss when reading and increase the risk of misunderstanding.
    • So if a test-taker answers incorrectly, you’ll have a hard time telling whether they genuinely couldn’t answer or simply misread the instruction.
  • Limit each item to a single type of content and a single level of cognitive process (Meeting 3).
  • Match the complexity of the language to the group who will take the test; avoid needlessly long sentences.

Writing the Answer Options

Principles for the key and distractors

  • Make sure there is only one correct answer, with no ambiguity.
  • Make the distractors plausible. Ideally, base them on common misconceptions about the topic, rather than choices that are obviously wrong or impossible.
  • Keep the length and grammatical structure of all options consistent; an option that is noticeably longer or more detailed often gives away the correct answer.
  • Avoid absolute words like “always” or “never” in a distractor, since they tend to be easy to spot as the wrong answer.

Further principles

  • Avoid options like “all of the above” or “none of the above”. These can often be answered correctly without truly understanding the other options.
  • Randomize the position of the correct answer across items; don’t always place it in the same spot.
  • Order the options logically — alphabetically or by numerical value, for example — whenever possible.

Writing Achievement Test Items vs. Aptitude Test Items

Different sources of content

  • Achievement test items are derived from the TIK (learning objectives) already formulated (Meeting 4). Their content is tied to the material or curriculum being taught.
  • Aptitude test items are derived from the definition of a more general ability construct (Meetings 5 and 6). Their content is deliberately kept independent of any one subject area, so that what’s measured is the reasoning process itself, not specific knowledge.

Practical implication

When writing an achievement test item, you start from the TIK. When writing an aptitude test item, you start from the narrow ability you want to measure — for example, induction or quantitative reasoning from the broad ability Gf discussed in Meeting 6.

Example Aptitude Test Items

Based on narrow abilities from Gf

Three narrow abilities commonly measured on aptitude tests (Meeting 6):

  • Induction — discovering the rule underlying a pattern.
  • General sequential reasoning — drawing a logical conclusion from given premises.
  • Quantitative reasoning — reasoning using numerical relationships and operations.

Example: induction

“SHOE : FOOT, as … : …”

  1. Earring : Ear　B) Finger : Ring　C) Head : Hat　D) Brush : Paint

Key

A — the relationship between a body part and the item worn/used on it, just like a shoe on a foot.

Example: general sequential reasoning

“All flowers have a pistil. All jasmines are flowers.”

  1. All jasmines have a pistil　B) Some jasmines do not have a pistil　C) Some flowers are not jasmines　D) All flowers are jasmines

Key

A — a direct logical conclusion drawn from the two premises (categorical syllogism).

Example: quantitative reasoning

“3, 6, 12, 24, …”

  1. 30　B) 36　C) 48　D) 60

Key

C (48) — pattern: each term is multiplied by 2.

Example Achievement Test Item

Derived from a TIK

Using the TIK from Meeting 4: “Students can match each principle of achievement test construction with an example of it.”

Example item

“The principle of ‘measuring a representative sample of content’ in an achievement test is best illustrated by:”

  1. A final exam that contains questions only from the last meeting

  2. A final exam whose questions are spread evenly across all meetings

  3. A final exam with a uniform level of difficulty

  4. A final exam with a large number of questions

Activity: Spotting Errors in an Item

The item below has several problems. Discuss with the classmate next to you what needs to be fixed.

Item to review

“Which of the following statements is NOT a characteristic of an aptitude test?”

  1. Always measures an innate ability present from birth that can never change at all over a person’s lifetime

  2. Is predictive of future performance

  3. Is not tied to any one specific subject area

  4. All of the above are correct

  • The stem uses a negative word (“NOT”), which should be avoided.
  • Option A uses absolute words (“always”, “can never”, “at all”), which gives away that this option is wrong.
  • Option D (“all of the above are correct”) should be avoided, per the principle discussed today.

Summary

  1. A multiple-choice item consists of a stem, key, and distractors — each with its own function and writing principles.
  2. The stem is written clearly, concisely, and positively; the core idea belongs in the stem, not the options.
  3. The options are made equal in length and structure, with plausible distractors, free of absolute words or “all/none of the above” choices.
  4. Achievement test items are derived from the TIK; aptitude test items are derived from the narrow ability being measured.
  5. These principles are directly relevant to Assignment 2, due this week.

Looking Ahead to Meeting 8

Next up: the Midterm Exam

Meeting 8 is the Midterm Exam, covering the material from Meetings 1–7: types of cognitive tests, stages of test development, taxonomy and instructional objectives, achievement tests, aptitude tests, intelligence tests, and item-writing principles.

Thank you!😊

Any questions?

These slides were prepared using and Quarto with a template from UNAIR Theme.

References

Azwar, S. (1987). Tes prestasi: Fungsi dan pengembangan pengukuran prestasi belajar. Liberty.

Haladyna, T. M., Downing, S. M., & Rodriguez, M. C. (2002). A review of multiple-choice item-writing guidelines for classroom assessment. Applied Measurement in Education, 15(3), 309–334.

Lane, S., Raymond, M. R., & Haladyna, T. M. (Eds.). (2016). Handbook of test development (2nd ed.). Routledge.

Osterlind, S. J. (2002). Constructing test items: Multiple-choice, constructed-response, performance, and other formats (2nd ed.). Kluwer Academic Publishers.

Schneider, W. J., & McGrew, K. S. (2018). The Cattell–Horn–Carroll theory of cognitive abilities. In D. P. Flanagan & E. M. McDonough (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (4th ed., pp. 73–163). Guilford Press.