Chapter Five: Item Formats
Central to the development of the NAEP assessment in mathematics is the careful selection of items/tasks. Since the 1992 assessment, items have consisted of three formats: multiple choice, short constructed response, and extended constructed response. Testing time on NAEP is divided evenly between multiple-choice items and both types of constructed-response items, as shown below:
Multiple-choice items require students to read, reflect, or compute, and then to select the alternative that best expresses what they believe the answer to be. This format is appropriate for quickly determining whether students have achieved certain knowledge and skills. A carefully constructed multiple-choice item can assess any of the levels of mathematical complexity (described in chapter 4), from simple procedures to more sophisticated concepts. Such items are limited in the extent to which they can provide evidence of the depth of students’ thinking.
Example
In the graph above, each dot shows the number of sit-ups and the corresponding age for 1 of 13 people. According to this graph, what is the median number of sit-ups for these 13 people?
- 15
- 20
- 45
- 50
- 55
This item assesses students’ understanding of two concepts: median, and how data are displayed on a graph. For students who are able to combine these concepts, little or no computation is required to determine the correct answer (D).
Short Constructed-Response Items
To provide more reliable and valid opportunities for extrapolating about students’ approaches to problems, NAEP assessments have included items that are often referred to as short constructed- response items. These are short-answer items that require students to give either a numerical result or the correct name or classification for a group of mathematical objects, draw an example of a given concept, or perhaps write a brief explanation for a given result.
Example
Product
-
2 x 2 = 4
2 x 2 x 2 = 8
2 x 2 x 2 x 2 = 16
2 x 2 x 2 x 2 x 2 = 32
-
If the pattern shown continues, could 375 be one of the products in this pattern?
Explain why or why not.
Scoring Guide
|
Score and Description |
|---|
| 1 - Correct Correct reason: “Because 375 is not divisible by 2 (or is not even)” OR “Because 375 is between two of the numbers in the pattern.” Acceptable: “Because 375 is uneven.” Not acceptable: “Because you’re counting by 2s.” OR “Because you’re multiplying by 2.” |
| 0 - Incorrect Any incorrect or incomplete reason. |
This item assesses students’ understanding of number patterns and of odd and even numbers. Students are given the opportunity to explain why 375 could not be one of the products in the pattern. Scoring for this item was done on a correct/incorrect basis. Items such as this are valuable in providing greater evidence of the depth of students’ understanding than might be accomplished in other formats. Scoring for some short constructed-response items uses more than two points in the scale to award partial credit for work that is partially correct. Full credit is reserved for work that is mathematically correct and complete.
Extended Constructed-Response Items
Extended constructed-response items require students to consider a situation that demands more than a numerical response or a short verbal communication. If it is a problem to solve, the student is asked to carefully consider a situation within or across content areas, understand what is required to “solve” the situation, choose a plan of attack, carry out the attack, and interpret the solution derived in terms of the original situation. In the example that follows, the student is asked to analyze a situation and provide a mathematical explanation.
Example
In a game, Carla and Maria are making subtraction problems using tiles numbered 1 to 5. The player whose subtraction problem gives the largest answer wins the game.
Look at where each girl placed two of her tiles.
|
|
 |
|
Who will win the game? __________________________
Explain how you know this person will win.
In this item, students are given the opportunity to display their understanding of place value and subtraction. They must examine the numbers left on each player’s tiles, consider all the possible placements of those tiles in the remaining spaces, then compare the results of the subtraction and construct an explanation of the situation.
Scoring Extended Constructed-Response Items
Extended constructed-response items in mathematics should be evaluated according to an established grading scale developed from a sample of actual student responses. The scale used should follow a multiple-point format similar to the following (this is the scoring guide for the previous example item):
Scoring Guide
|
Solution: Maria will win the game. The following reasons may be given:
|
|
Score and Description |
|---|
| 4 - Extended Student answers Maria and gives explanation such as (a) or (b), or an appropriate combination of the other explanations. |
| 3 - Satisfactory Student answers Maria and gives explanation such as (c), (d), or (e). |
| 2 - Partial Student answers Maria with partially correct, or incomplete but relevant, explanation. |
| 1 - Minimal Student answers Maria and gives example such as in (f) but no explanation OR answers Maria with an incorrect explanation. 0 - Incorrect Incorrect response |
A scoring scale adapted to a particular problem and applied by experienced scorers provides rich information about students’ understanding of concepts and procedures, and about their ability to solve a problem and communicate understanding of the process.
| Previous | Contents | Next |
| Mathematics Framework for the 2005 National Assessment of Educational Progress |