Request for Public Comment on Design Document for Setting Achievement Levels on NAEP TEL

Feedback Due November 28

WASHINGTON – (October 30, 2014) — The National Assessment Governing Board is soliciting public comment for guidance in finalizing the design document for the National Assessment of Educational Progress (NAEP) in Technology and Engineering Literacy (TEL). The NAEP TEL assessment was administered for the first time in 2014 to a nationally representative sample of over 22,000 grade 8 students. Comments are due by November 28.

Achievement levels have become a powerful way to communicate student achievement on an assessment like the NAEP TEL because they interpret test performance with reference to cut scores that quantitatively define ordered categories of achievement such as Basic, Proficient, and Advanced. An important source of evidence used by policymakers to establish achievement levels is the cut score recommendations. Cut scores are the outcome of a facilitated process, called an achievement levels-setting or standard setting meeting, eliciting judgments from experts related to the test content and the knowledge, skills and abilities of the test takers.

The design document for developing achievement levels on the NAEP TEL at grade 8 is intended to provide the foundation for all achievement levels-setting activities. The design document for the TEL achievement levels-setting process includes discussion of the methodology, procedures, and documentation of the entire project.

Under P.L. 107-279, the Governing Board is authorized to set policy for NAEP. The legislation specifies that the Governing Board is to develop appropriate student achievement levels for each subject and grade tested as provided in section 303(e). Such levels are determined by identifying the knowledge that can be measured and verified objectively using widely accepted professional assessment standards; and developing achievement levels that are consistent with relevant widely accepted professional assessment standards and based on the appropriate level of subject matter knowledge for grade levels to be assessed.

To finalize plans for the TEL achievement levels-setting activities, and in preparation for reporting the results of the new TEL assessment at grade 8, the Governing Board has issued a contract to Pearson to implement a process to produce a set of cut score recommendations to assist the Governing Board in developing achievement levels for the 2014 NAEP TEL at grade 8.

Pearson has developed a design document that describes in detail the NAEP TEL achievement levels-setting activities. The Governing Board now seeks comment on the document to provide recommendations for improvements. All responses will be taken into consideration before finalizing the document, which will guide the achievement levels-setting activities to produce a set of cut score recommendations for reporting achievement levels for the 2014 administration of the NAEP TEL at grade 8.