Although NAEP was designed to monitor, assess, and report student achievement nationally, an inevitable effect of this monitoring and reporting is improvement in mathematics learning. If real change in the mathematics curriculum is to take place, the manner in which assessment is conducted will also have to change. Assessment activities often are the primary sources from which students discern what teachers really value and what teachers really want them to know.

Mathematical Power

Mathematical power is characterized as a student’s overall ability to gather and use mathematical knowledge through exploring, conjecturing, and reasoning logically; solving nonroutine problems; communicating about and through mathematics; and connecting mathematical ideas in one context with mathematical ideas in another context or with ideas from another discipline in the same or related contexts.

Assessing a student’s mathematical power requires many different indicators over time. As power develops beyond the general mathematical abilities of conceptual understanding, procedural knowledge, and problem solving, it is important to ensure that students are assessed on their ability to reason in mathematical situations, communicate perceptions and conclusions drawn from a mathematical context, and connect the mathematical nature of a situation with related mathematical knowledge and information gained from other disciplines or through observation.

It is the total interaction of all of these abilities that defines a student’s overall mathematical power at a given time. The mental skills of reasoning, communicating, and connecting are the foundation of each content strand and each mathematical ability featured in previous NAEP assessments. These relationships, discussed in chapter two, indicate the multidimensional nature of mathematical power.

Mathematical power can be viewed from various perspectives. Students may encounter a new problem in an old context or an old problem in a new context. When first attempts to solve a problem fail, the student may reexamine the information, rework it, and then reapply it to the situation in a more productive fashion. The process of revising an approach to a problem based on reasoning, gathering new information, and making connections with other ideas is a dynamic ability. This feature of mathematical power can be viewed through student performance within a particular content strand at the conceptual, procedural, and problem-solving levels of ability. Similarly, a particular concept, procedure, or problem context might be viewed across strands. In the latter case, families of items are particularly helpful in assessment. The use of calculators enables students to quickly pursue alternative paths and determine whether they provide fruitful new information or reconfirm judgments made through other approaches.

Students demonstrate their mathematical power by formulating problem-solving and reasoning strategies in situations involving a multitude of possibilities. It is here that the recommendation that students experience a number of extended open-ended items requiring construction of responses is important. Through a student’s report of his or her thinking, questions of the relevance of the approach, the nature of reasoning, and the ability to solve problems become less inferential and more conclusively based on evidence. This is especially true when the collected evidence includes the communication of a student’s approach and when partial credit for student efforts is awarded in the scoring of an item.

Finally, mathematical power is a function of students’ prior knowledge and experience and the ability to connect that knowledge in productive ways to new contexts. This aspect of power can be measured with multiple-choice items and through analysis of the ways in which students develop their responses to the constructed- response items on the assessment.

Information related to these features of students’ development is as difficult to isolate and statistically extract from the data as the mathematical abilities featured in the past NAEP assessments in mathematics. However, they are important aspects of the mathematical development of students. As such, the three features of mathematical power (reasoning, communication, and connections) will be used as underlying threads for item construction and overall test design. For the mathematics assessment, these threads may not be specifically reported, although they will be represented in the overall way the assessment is conceived and developed.

Mathematical Abilities

As previously discussed, the general mental abilities associated with mathematics and targeted in past NAEP assessments are conceptual understanding, procedural knowledge, and problem solving. These three areas are specifically identified as primary foci for assessment, and they received focal attention in the design of the 1990 and 1992 assessments. Conceptual understanding can be viewed simply as a measure of a student’s “knowing that” or “knowing about,” whereas procedural knowledge can be viewed as a student’s “knowing how.” These two abilities are the foundation for recognizing and understanding a problem, formulating a plan to solve the problem, arriving at a solution to the problem, and reflecting on the solution. The later stages can be thought of as facets of problem solving.

However, as recommended in chapter one, the role of these dimensions of students’ mathematical power in the new assessment should change from one of a direct matrix feature to one of a design characteristic that assists in providing balance to the overall assessment. The NAEP design for the mathematics assessment should certainly continue to focus on conceptual understanding, procedural knowledge, and problem solving in bringing some balance to the assessments for grades 4, 8, and 12. In particular, it is recommended that at least one-third of the items for each grade level measure conceptual understanding, procedural knowledge, and problem solving.

As with the mathematical content strands, mathematical abilities are not separate and distinct factors of an individual’s ways of thinking about a mathematical situation. These abilities are, rather, descriptions of the ways in which information is structured for instruction and the ways in which students manipulate, reason with, or communicate their mathematical ideas. Consequently, no unanimous agreement exists among educators about what constitutes a conceptual, a procedural, or a problem-solving item. What can be classified are the actions a student is likely to undertake in processing information and providing a satisfactory response. Thus, within the content strands, assessment tasks are classified according to the ability categories they most closely represent in terms of the type of processing they are expected to require. Furthermore, the mathematical power features of reasoning, communication, and connections are woven through the specifications to provide an added level of richness to the assessment tasks.

The following discussions of conceptual understanding, procedural knowledge, and problem solving illustrate the primary features the NAEP assessment should employ to capture features of cognitive activities that combine to empower a student in mathematical situations.

Conceptual Understanding

Students demonstrate conceptual understanding in mathematics when they provide evidence that they can recognize, label, and generate examples and nonexamples of concepts; use and interrelate models, diagrams, manipulatives, and varied representations of concepts; identify and apply principles (that is, valid statements generalizing relationships among concepts in conditional form); know and apply facts and definitions; compare, contrast, and integrate related concepts and principles to extend the nature of concepts and principles; recognize, interpret, and apply the signs, symbols, and terms used to represent concepts; or interpret the assumptions and relations involving concepts in mathematical settings.

Conceptual understanding reflects a student’s ability to reason in settings involving the careful application of concept definitions, relations, or representations of either. Students demonstrate conceptual understanding when they produce examples or common or unique representations, or when they manipulate central ideas about a concept in various ways.

Procedural Knowledge

Students demonstrate procedural knowledge in mathematics when they select and apply appropriate procedures correctly; verify or justify the correctness of a procedure using concrete models or symbolic methods; or extend or modify procedures to deal with factors inherent in problem settings.

Procedural knowledge includes the various numerical algorithms in mathematics that have been created as tools to meet specific needs efficiently. Procedural knowledge also encompasses the abilities to read and produce graphs and tables, execute geometric constructions, and perform noncomputational skills such as rounding and ordering. These latter activities can be differentiated from conceptual understanding by the task context or presumed student background—that is, an assumption that the student has the conceptual understanding of a representation and can apply it as a tool to create a product or to achieve a numerical result. In these settings, the assessment question is how well the student executed a procedure or selected the appropriate procedure to perform a given task.

Procedural knowledge is often reflected in a student’s ability to connect an algorithmic process with a given problem situation, employ that algorithm correctly, and communicate the results of the algorithm in the context of the problem setting. Procedural understanding also encompasses a student’s ability to reason through a situation, describing why a particular procedure will solve a problem in the context described.

Problem Solving

In problem solving, students are required to use their accumulated knowledge of mathematics in new situations. Problem solving requires students to recognize and formulate problems; determine the sufficiency and consistency of data; use strategies, data, models, and relevant mathematics; generate, extend, and modify procedures; use reasoning (spatial, inductive, deductive, statistical, or proportional) in new settings; and judge the reasonableness and correctness of solutions. Problem-solving situations require students to connect all of their mathematical knowledge of concepts, procedures, reasoning, and communication/representational skills in confronting new situations. As such, these situations are perhaps the most accurate measures of students’ proficiency in mathematics.