Chapter One
This document, Reading Framework for the 2005 National Assessment of Educational Progress, describes the content and format of the 4th, 8th, and 12th grade assessments. The 2005 Reading Framework continues the NAEP reading trend begun in 1992.
What Is the NAEP Reading Assessment?
The NAEP reading assessment measures the achievement of the nation's students in reading. Authorized by Congress and administered by the U.S. Department of Education, NAEP regularly reports to the public on the educational progress of students in various subject areas. NAEP collects achievement data from nationally representative samples of students in grades 4, 8, and 12 and from state-level samples of fourth and eighth graders.
As the "Nation's Report Card," the NAEP reading assessment reports how well students perform in reading various texts and responding to those texts in multiple-choice and constructed-response formats. It provides information about student achievement that is needed to help the public, decisionmakers, and education professionals understand the strengths and weaknesses in student performance and make informed decisions about education.
The NAEP reading assessment measures comprehension by asking students to read passages and answer questions about what they have read. As such, it represents a measure of reading achievement and provides a broad picture of what our nation's students should be able to read and understand at specific grade levels. Students use various skills and strategies in the reading assessment. However, NAEP does not report on strategies such as finding a detail or summarizing a plot. This is in keeping with NAEP's role as an assessment of overall achievement rather than a diagnostic test for individual students.
Although broad implications for instruction can be inferred from the assessment, NAEP does not specify how reading should be taught, nor does it prescribe a particular curricular approach to teaching reading.
The NAEP Reading Framework provides the guidelines and theoretical basis for developing the 1992–2005 reading assessments. It reflects the ideas of many diverse individuals and organizations involved in reading education. In developing the framework for the national assessment of reading, researchers, policymakers, teachers, business representatives, and other experts have specified behaviors of proficient readers who are active, strategic, knowledgeable, and motivated to read.
The Report of the National Reading Panel summarizes research describing such a reader: "In the cognitive research, reading is purposeful and active. According to this view, a reader reads a text to understand what is read, to construct memory representations of what is understood, and to put this understanding to use" (National Institute for Child Health and Human Development (NICHD), 2000b, p. 4–39). These processes are the basis for establishing broad goals for reading that are addressed by NAEP.
Goals for Reading Literacy
The goals for reading literacy are to develop good readers who:
- read with enough fluency to focus on the meaning of what they read;
- form an understanding of what they read and extend, elaborate, and critically judge its meaning;
- use various strategies to aid their understanding and plan, manage, and check the meaning of what they read;
- apply what they already know to understand what they read;
- read various texts for different purposes;
- possess positive reading habits and attitudes.
These characteristics of good readers are appropriate for students as early as third grade, as this is the grade level by which it is expected that children will learn to read. National programs such as the Reading Excellence Act and No Child Left Behind focus on improving reading achievement. Policy and the commitment of resources at the national and state levels continue to focus on this goal. Therefore, it makes sense that the NAEP Reading Framework should also reflect this aim.
The National Research Council's research-based report Preventing Reading Difficulties in Young Children (Snow et al., 1998) corroborates the characteristics listed above when it describes third-grade accomplished readers as being able to:
- summarize major points from fiction and nonfiction texts;
- read longer fictional selections and chapter books independently;
- discuss underlying themes or messages when interpreting fiction;
- distinguish cause and effect, fact and opinion, main idea, and supporting details when interpreting nonfiction.
These characteristics are important for describing the performance of good readers in the NAEP reading assessment.
Definitions of Reading Literacy
National
Reading literacy is a fundamental right of every person. It gives people access to information and the ability to function in life. Reading enriches through the power of language and the beauty of poetry. It extends the human experience through the exploration of events in literary works. It is the key to knowledge and information.
Today, reading literacy means more than a functional ability to read simple directions and books. In The Keys to Literacy, Reid Lyon (1998) describes the multifaceted nature of the reading process:
In general, if children can read the words on a page accurately and fluently, they will be able to construct meaning at two levels. At the first level, literal understanding is achieved. However, constructing meaning requires far more than literal comprehension. Children must eventually guide themselves through text by asking questions such as: "Why am I reading this, and how does this information relate to my reasons for doing so?" "What is the author's point of view?" "Do I understand what the author is saying and why?" "Is the text internally consistent?" It is this second level of comprehension that leads readers to reflective, purposeful understanding. (p. 4)
In a world driven by information technology, the complexity of reading literacy is increasing as the format of texts becomes more diverse. Varied texts such as CD-ROMs,Web pages, newspapers, and magazines place different demands on the reader. As information technology grows, people will encounter even more varied texts and will be called on to use information in new ways.
International
When the 1992 NAEP reading assessment was developed, no one could have predicted the effect the NAEP Reading Framework would have on the design of international reading assessments. The frameworks of two international assessments provide pertinent evidence of this trend:
- The Progress in International Reading Literacy Study (PIRLS) is being conducted under the auspices of the International Association for the Evaluation of Educational Achievement and focuses on the achievement of young children (ages 9 and 10).
- The Program for International Student Assessment (PISA), conducted under the auspices of the Organisation for Economic Co-operation and Development, is a regular survey of 15-year-olds and assesses aspects of their preparedness for adult life. Reading literacy, mathematical literacy, and scientific literacy are the primary domains assessed.
PIRLS most closely aligns with the fourth-grade NAEP assessment. PISA samples students in grade 10, in contrast to NAEP's sampling of grades 8 and 12. Strong similarities exist among the three assessments' definitions of reading literacy.
PIRLS defines reading literacy as "the ability to understand and use those written language forms required by society and/or valued by the individual. Young readers can construct meaning from a variety of texts. They read to learn, to participate in communities of readers, and for enjoyment" (International Association for the Evaluation of Educational Achievement, 2000, p. 3).
PISA defines reading literacy as "understanding, using, and reflecting on written texts in order to achieve one's goals, to develop one's knowledge and potential, and to participate in society" (Organisation for Economic Co-operation and Development, 2000, p. 18).
All three definitions of reading literacy (NAEP, PIRLS, and PISA) convey the notion that reading involves developing an understanding of various texts, thinking about them, and using various texts for many different purposes. For example, readers may use maps to gain information, stories to enjoy the human experience, and science texts to form hypotheses.
The congruence in framework definitions clearly represents a growing international agreement on the important dimensions of reading literacy. This agreement has played the same unifying role in the international assessment of reading as that of the Third International Mathematics and Science Study in the international assessment of mathematics.
One way NAEP reflects current definitions of literacy is by differentiating among three contexts for reading and four aspects of reading. Contexts for reading and aspects of reading are the foundation of the NAEP reading assessment.
Contexts for Reading
|
|
Many commonalities exist among different reading contexts, including developing understanding, reflecting critically on the text, and analyzing the author's perspective. The contexts are not mutually exclusive. For example, readers may read historical novels for literary experience and information. However, distinctions exist because various texts and tasks can place differing demands on the reader. Readers often have greater facility with one type of reading material than another. For example, some people can read novels with greater ease than they can read technical manuals.
Different contexts for reading lead to real differences in what readers do. When reading for literary experience, readers make complex, abstract summaries and identify major themes. They describe the interactions of various literary elements (for example, setting, plot, characters, and theme). When reading for information, readers may critically judge the form and content of the text and explain their judgments. They may also look for specific pieces of information. When reading to perform a task, readers may search quickly for specific pieces of information. The contexts for reading and the reader's expectations may influence the comprehension process, determine what strategies and skills are used to develop meaning, and influence the extent to which content is integrated with prior knowledge.
The contexts for reading are addressed by the specific tasks, or items, on the NAEP reading assessment. An item can be easy even if the text is complex. For example, given a difficult text and a question with wording directly from the text, a reader might scan the text, find the relevant information, and answer the question with ease. Because this type of item requires little more than matching words or phrases, it can be accomplished without having an understanding of the passage as a whole or even the specific part of the text addressed by the item. Once readers have a certain level of automaticity, they can handle simple tasks such as this. However, to respond to a text within the Reading Framework, readers must be able to understand the text in its entirety as well as focus on specific parts and think beyond what was directly stated.
Reading for literary experience involves the reader in exploring themes, events, characters, settings, problems, and the language of literary works. The reader brings his or her experiences and knowledge to the text in activities such as anticipating events, picturing settings, predicting consequences, analyzing actions, and considering the language of literary works. He or she thinks about the authors' and characters' perspectives and considers the language and story structure. Various types of texts are associated with reading for literary experience, including novels, short stories, poems, plays, legends, biographies, myths, and folktales.
Reading for information engages the reader with aspects of the real world. Reading for information is most commonly associated with textbooks, primary and secondary sources, newspaper and magazine articles, essays, and speeches. Two features that distinguish informational text from literary text are organization and presentation of information. Informational text is organized by topic and supporting details, whereas literary text is organized by the structure of a story, poem, or drama. Informational texts may have boldfaced headings, graphics, illustrations, and captions that signal important text. However, some commonalities exist between literary and informational text and the skills and strategies required for reading each: Both require people to critically analyze the text, reflect on it, and draw conclusions.
When reading for information, readers need to know the specific text patterns, or forms of organization (for example, cause and effect, sequential order, comparison/contrast, opinion and supporting arguments), to develop understanding. People frequently have different purposes for reading text of this nature; for example, to find specific pieces of information, answer a question, or get some general information when glancing through a magazine article. Reading informational text requires orientations to the text that differ from those used in reading for literary experience because readers are specifically focused on acquiring information. When people read for information, they may select parts of the text they need rather than reading from beginning to end.
Reading to perform a task involves reading to accomplish something. When people read to perform tasks, they use their expectations of the purpose and structure of practical text to guide how they select, understand, and apply information. Practical text may include charts, bus or train schedules, directions for games or repairs, classroom or library procedures, tax or insurance forms, recipes, voter registration materials, maps, referenda, consumer warranties, or office memos. The reader's orientation involves looking for specific information to do something. Readers need to apply information, not simply understand it. For this type of reading, readers are not likely to savor the style or thought in the texts as they might in reading for literary experience.
NAEP assesses reading for literary experience and reading for information in grades 4, 8, and 12. Reading to perform a task is assessed only in grades 8 and 12. Contexts for reading are shown in exhibit 1.
Exhibit 1. Contexts for Reading Specified in the NAEP Reading Framework
| Context for Reading | Description |
|---|---|
| Reading for literary experience | Readers explore events, characters, themes, settings, plots, actions, and the language of literary works by reading novels, short stories, poems, plays, legends, biographies, myths, and folktales. |
| Reading for information | Readers gain information to understand the world by reading materials such as magazines, newspapers, textbooks, essays, and speeches. |
| Reading to perform a task | Readers apply what they learn from reading materials such as bus or train schedules, directions for repairs or games, classroom procedures, tax forms (grade 12), maps, and so on. |
The proportion of items related to each context for reading changes from grade to grade to reflect the changing demands made of students as they mature. The proportion of items in each grade is shown in exhibit 2.
Exhibit 2. Percentage of NAEP Reading Items, by Grade and Context for Reading
| Grade | Context for Reading | ||
|---|---|---|---|
| For Literary Experience (%) |
For Information (%) |
To Perform a Task (%) |
|
| 4 | 55 | 45 | No scale |
| 8 | 40 | 40 | 20 |
| 12 | 35 | 45 | 20 |
Aspects of Reading
|
|
Readers develop understanding in different ways. They focus on general topics or themes, interpret and integrate ideas within and across texts, make connections to background knowledge and experiences, and examine the content and structure of the text. NAEP's questions and tasks are based on these four aspects of reading and require the selection and integration of various reading strategies rather than the application of a specific strategy or skill. Inherent in these aspects are the strategies that readers use to build and examine their understanding and adjust their approach. According to the Report of the National Reading Panel (NICHD, 2000b), readers use their knowledge of the world, including language and print, to make meaning.
Forming a General Understanding
To form a general understanding, the reader must consider the text as a whole and have a global knowledge of it. Students may be asked, for example, to demonstrate a general understanding by providing the topic of a passage, explaining the purpose of an article, or reflecting on the theme of a story. Tasks and questions that measure this aspect of reading include:
- Write a paragraph telling what the story/poem is about.
- Which of the following is the best statement of the theme of the story?
- Write a paragraph telling what this article generally tells you.
- What is this text supposed to help you do?
- What would you tell someone about the main character?
Developing Interpretation
To develop an interpretation, the reader must extend initial impressions to develop a more complete understanding of what was read. This process involves linking information across parts of a text as well as focusing on specific information. Questions that assess this aspect of reading include drawing inferences about the relationship of two pieces of information and providing evidence to determine the reason for an action. Questions that assess this aspect of reading include:
- What event marked a change in the plot or situation?
- What caused the character to __________?
- What caused this event?
- What is the meaning of __________?
- What type of person is this character? Explain.
- What does this idea imply?
- In what ways are these ideas important to the topic or theme?
- What will be the result of this step in the directions?
- What does this character think about __________?
Making Reader/Text Connections
To make reader/text connections, the reader must link information in the text with knowledge and experience. This might include applying ideas in the text to the real world. All student responses must be text based to receive full credit. NAEP does not ask students about their personal feelings. Tasks and questions that assess this aspect of reading include:
- Why do __________ (bullfrogs eat dragonflies)? Is there anything else you think __________ (they might eat)? Explain your answer using information from the text and what you know.
- Colonists lived in different ways than we live today. Tell about two of these differences.
- Would you have liked to live in colonial times? Use information from the text to support your answer.
- In this story, two characters chose different ways to solve a problem. Which solution was most effective in solving the problem? Use information from the text to support your answer.
- How would you change the directions to build this __________ if you did not have a __________?
Examining Content and Structure
Examining text content and structure requires critically evaluating, comparing and contrasting, and understanding the effect of features such as irony, humor, and organization. Questions used to assess this aspect of reading require readers to stand apart from the text, consider it objectively, and evaluate its quality and appropriateness. Knowledge of text content and structure is important. Questions ask readers to determine the usefulness of a text for a specific purpose, evaluate the language and textual elements, and think about the author's purpose and style. Some questions also require readers to make connections across parts of a text or between texts. For example, students might be asked to compare a poem and a story with the same theme or relate information from a first-person account to a textbook description of an event. Questions that assess this aspect of reading include:
- Compare the structure of this magazine article to that one.
- How useful would this be for __________? Why?
- Does the author use (irony, personification, humor) effectively? Explain.
- What is the author's point of view? Using the text, provide a sentence or two to support your response.
- Is this information needed for __________? Explain your reasoning.
- What other information would you need to find out about __________? Support your answer with information from the text.
Exhibit 3 portrays the four aspects of reading. Although the strategies and skills required to master these aspects overlap to some extent, successfully mastering one aspect may not depend on successfully mastering any other aspect, and the aspects are not mutually exclusive. For example, comparing two authors' points of view would require forming a general understanding before making the comparison. The reader would use several aspects of reading to provide a wide range of responses to reading.
Exhibit 3. Aspects of Reading and Reader Responses
| Aspects of Reading | |||
|---|---|---|---|
| Forming a General Understanding | Developing Interpretation | Making Reader/Text Connections | Examining Content and Structure |
| Consider text in
its entirety
Understanding in a broad way |
Focus on
specific parts
Linking information across parts of the text |
Think beyond
the text
Applying the text to real-world situations |
Consider why
and how the
text was
developed
|
Exhibit 4 shows the percentage of time students in a given grade would spend on NAEP items that measure each aspect of reading (general understanding and developing interpretation are combined) if they responded to all items in the NAEP reading assessment. Originally, the 1989 Reading Committees proposed a distribution of one-third of the items for each reading aspect. Please refer to the NAEP Web site (nces.ed.gov/nationsreportcard) for sample student responses, performance data, and a glossary of NAEP terms.
Exhibit 4. Percentage Distribution of Student Time, by Grade and Aspect of Reading
| Grade | Aspect of Reading | ||
|---|---|---|---|
| Forming a General Understanding and Developing Interpretation (%) |
Making Reader/Text Connections (%) |
Examining Content and Structure (%) |
|
| 4 | 60 | 15 | 25 |
| 8 | 55 | 15 | 30 |
| 12 | 50 | 15 | 35 |
Exhibit 5 shows sample NAEP questions by context for reading and aspect of reading. The bulleted items are the skills involved in each aspect of reading. Readers are expected to demonstrate a level of competency in each aspect.
Exhibit 5. Sample NAEP Questions and Reading Strategies, by Context for Reading and
Aspect of Reading
| Context for Reading | Aspect of Reading | |||
|---|---|---|---|---|
| Forming a General Understanding | Developing Interpretation | Making Reader/Text Connections | Examining Content and Structure | |
| Reading for literary experience | What is the story/ plot about?
|
How did this character change from the beginning to the end of the story?
|
What other character that you have read about had a similar problem?
information and ideas |
What is the mood of this story, and how does the author use language to achieve it?
|
| Reading for information | What point is the author making about this topic?
|
What caused this change?
effect |
What other event in history or recent news is similar to this one?
information and ideas |
Is this author biased? Support your answer with information about this article.
fact and opinion |
| Reading to perform a task | What time can you get a nonstop flight to X?
|
What must you do before step 3?
|
Describe a situation in which you would omit step 5.
information and ideas |
Is the information in this brochure easy to use?
|
The following fourth-grade informational passage and questions were released from the NAEP 2000 reading assessment. The bracketed text below each question presents the reading aspect assessed and the percentage of students who answered the question successfully. Additional sample passages and questions can be found in appendix A and on the NAEP Web site (nces.ed.gov/nationsreportcard). Please refer to the Web site for sample student responses and performance data.
Grade 4 Sample Reading Passage and Questions: Informational
I magine shivering on a cold winter's night. The tip of your nose tingles in the frosty air. Finally,
you climb into bed and find the toasty treat you have been waiting for—your very own hot
brick. If you had lived in colonial days, that would not sound as strange as it does today.
Winters were hard in this New World, and the colonists had to think of clever ways to fight the
cold. At bedtime, they heated soapstones, or bricks, in the fireplaces. They wrapped the bricks
in cloths and tucked them into their beds. The brick kept them warm at night, at least for as long
as its heat lasted.
Before the colonists slipped into bed, they rubbed their icy sheets with a bed warmer. This was a metal pan with a long wooden handle. The pan held hot embers from the fireplace. It warmed the bedding so well that sleepy bodies had to wait until the sheets cooled before climbing in.
Staying warm wasn't just a bedtime problem. On winter rides, colonial travelers covered themselves with animal skins and warm blankets. Tucked under the blankets, near their feet, were small tin boxes called foot stoves. A foot stove held burning coals. Hot smoke puffed from small holes in the stove's lid, soothing freezing feet and legs. When the colonists went to Sunday services, their foot stoves, furs, and blankets went with them. The meeting houses had no heat of their own until the 1800s.
At home, colonial families huddled close to the fireplace, or hearth. The fireplace was wide and high enough to hold a large fire, but its chimney was large, too. That caused a problem: Gusts of cold air blew into the house. The area near the fire was warm, but in the rest of the room it might still be cold enough to see your breath.
Reading or needlework was done by candlelight or by the light of the fire. During the winter, animal skins sealed the drafty windows of some cabins and blocked out the daylight. The living area inside was gloomy, except in the circle of light at the hearth.
Early Americans did not bathe as often as we do. When they did, their "bathroom" was the kitchen, in that toasty space by the hearth. They partially filled a tub of cold water, then warmed it up with water heated in the fireplace. A blanket draped from chairs for privacy also let the fire's warmth surround the bather.
The household cooks spent hours at the hearth. They stirred the kettle of corn pudding or checked the baking bread while the rest of the family carried on their own fireside activities. So you can see why the fireplace was the center of a colonial home. The only time the fire was allowed to die down was at bedtime. Ashes would be piled over the fire, reducing it to embers that might glow until morning.
By sunrise, the hot brick had become a cold stone once more. An early riser might get dressed under the covers, then hurry to the hearth to warm up.
Maybe you'd enjoy hearing someone who kept warm in these ways tell you what it was like. You wouldn't need to look for someone who has been living for two hundred years. In many parts of the country, the modern ways didn't take over from the old ones until recently. Your own grandparents or other older people might remember the warmth of a hearthside and the joy of having a brick to cuddle up to.
Used by permission of Highlights for Children, Inc., Columbus, OH
Copyright © 1991. Illustration by Katherine Dodge.
Questions for "A Brick to Cuddle Up To"
(See appendix A for scoring criteria or rubrics.)
1. You would probably read this article if you wanted to know how the colonists
a. cooked their food
b. traveled in the winter
c. washed their clothes
d. kept warm in cold weather
[Aspect, General understanding; Key, D; Percent correct, 85]
2. After reading this article, would you like to have lived during colonial times? What information in the article makes you think this?
(Responses to this question were scored according to a three-level rubric.)
[Aspect, Reader/text connections; Percent full comprehension, 20]
3. Some of the ways that colonists kept warm during the winter were different from the ways that people keep warm today. Tell about two of these differences.
(Responses to this question were scored according to a three-level rubric.)
[Aspect, Reader/text connections; Percent full comprehension, 17]
4. Do you think "A Brick to Cuddle Up To" is a good title for this article? Using information from the article, tell why or why not.
(Responses to this question were scored according to a three-level rubric.)
[Aspect, Developing interpretation; Percent full comprehension, 37]
5. Give two reasons stated in the article why the hearth was the center of the home in colonial times.
(Responses to this question were scored according to a three-level rubric.)
[Aspect, Developing interpretation; Percent full comprehension, 20]
6. A colonist would probably have used a foot stove when
a. going on a trip
b. sleeping in bed
c. sitting by the fireplace
d. working around the house
[Aspect, Developing interpretation; Key, A; Percent correct, 36]
7. Pretend that you are an early American colonist. Describe at least three activities you might do during a cold winter evening. Be specific. Use details from the article to help you write your description.
(Responses to this question were scored according to a four-level rubric.)
[Aspect, Developing interpretation; Percent extensive, 12]
8. In writing this article, the author mostly made use of
a. broad ideas
b. specific details
c. important questions
d. interesting characters
[Aspect, Examining content and structure; Key, B; Percent correct, 66]
9. Does the author help you understand what colonial life was like? Use examples from the article to explain why or why not.
(Responses to this question were scored according to a three-level rubric.)
[Aspect, Examining content and structure; Percent full comprehension, 20]
| Previous | Contents | Next |
| Reading Framework for the 2005 National Assessment of Educational Progress |