banner Technology and Engineering Literacy Framework for the 2014 NAEP
Examples of Practices Applied in Each of the Assessment Areas
Practices Applied in Design and Systems

Practices Applied in Design and Systems

Assessment targets in the area of Design and Systems relate to the nature of technology, the engineering design process, maintenance and troubleshooting, and systems thinking. The sections below describe how the three crosscutting practices apply to targets in the area of Design and Systems.

Understanding Technological Principles

Technological principles for Design and Systems in chapter two specify for this assessment the core understandings that students should have about the different types of technologies, processes for designing technologies, approaches to preventing failures, and how components of technological systems interrelate. The practices for Understanding Technological Principles in these areas could, for example, ask students to draw on their knowledge to identify examples of technologies, components of the design process, components of a system, or maintenance and troubleshooting methods. Students could be asked to explain the relationship among technologies in a system, analyze the components of a system, recognize design constraints, or evaluate alternative representations of a system.

Developing Solutions and Achieving Goals

Problem solving is a major part of the engineering design process. Thus there are many opportunities for students to demonstrate their problem solving skills in assessment tasks for this area. Such tasks could require them to develop designs, to propose or critique solutions to problems after being given criteria and constraints, to select appropriate resources by considering trade-offs, to construct and test a model or prototype, to troubleshoot systems and applications, or to determine the consequences of making a change in a system. Students must use their understandings about the technological principles specified for Design and Systems in chapter two as they plan, try out, critique, and revise attempted solutions.

Communicating and Collaborating

Communication and collaboration practices are integral to achieving the goals of technological design and systems. Students can demonstrate teamwork in tasks where design assignments are distributed among team members, progress and results are integrated and shared, and products are presented jointly. Designs and the design process can be represented in visual and verbal forms. Students can create instructions for system assembly and prepare documentation of a procedure for maintaining a system. Students must use their understandings of the technological principles specified for both Design and Systems and ICT in chapter two to use tools and strategies to communicate and collaborate. Because students are completing the assessment individually on the computer, collaborators will be virtual, that is—presented by the computer.

Table 3.3 offers examples of how the 3 practices can be applied to assessment targets for Design and Systems to generate tasks and items for the middle school level. The key principles and grade 8 targets were selected from tables 2.6-2.9 in chapter two. The chapter two tables present simpler versions of the targets for grade 4 and more challenging versions of the targets for grade 12. Again, these are sample ideas for items and tasks and will not be used in the actual assessment.

Table 3.3 Examples of grade 8 tasks representing practices in each subarea of Design and Systems

Practices

A. Nature of Technology

B. Engineering Design

C. Systems Thinking

D. Maintenance and Troubleshooting

Selected Principles

Technologies developed for one purpose are sometimes adapted to serve other purposes.

Technological development involves creative thinking.

Requirements for a design challenge include the criteria for success and the constraints or limits that cannot be violated in a solution.

All technological systems have parts that work together to accomplish a goal.

Tools and machines must undergo regular maintenance to ensure their proper functioning.

Understanding Technological Principles

Describe the properties of a spring that inspired the invention of the Slinky. (animation)

List three important criteria for a device that will toast bread, and justify each one.

How do the physical and human components of the school cafeteria food service system work together?

(animation)

Why do Bill and Sally oil their bike chains and axles and check the brakes each month? What may happen if they do not?

Developing Solutions and Achieving Goals

Given a collection of objects, design a new toy (for example, for a baby, young child). What are the criteria for a toy, and how does your design meet them?

Design a process to serve 50 slices of warm toast in 5 minutes, given specific equipment and resources.

Design modifications to the cafeteria for people with a particular physical disability (for example, vision, mobility).

(animation)

Bill's new bike gears are not working correctly. What should he do?

Communicating and Collaborating

Select a team of people who could design and build a new toy for a 5-year-old, and justify the choices. Work individually, or collaborate with a virtual person to make your selections.

How would an industrial toaster salesperson develop talking points for selling a particular toaster to a given restaurant?

Collaborate with a virtual group of people to redesign the tasks of the cafeteria staff, given the new machines.

Bill, Sally, and many other students would like to ride their bikes to school. Present a design for a bike parking lot at the school.

Illustrative Tasks and Items

These items from existing tests were selected as examples that could be adapted for assessing targets in Design and Systems. Each item is analyzed for the ways in which it would call on the practices.

Scenario-Based Item Set

In this example, students are asked a series of questions related to a simulation of a nuclear reactor. In the NAEP Technology and Engineering Literacy Assessment, the questions might relate to all three practices. Students may be asked to demonstrate the practice of Understanding Technological Principles by identifying the inputs and outputs of the system or analyzing the potential hazards. Students might employ the practice of Developing Solutions and Achieving Goals in items asking for results of investigations based on manipulating the simulation to find safe levels of temperature and power. The practice of Communicating and Collaborating could be elicited in tasks involving virtual (that is, computer-generated) experts in preparing a report of findings.

Context: Energy
Topic: Nuclear Reactor
Target Level: Grade 8
Watch a video clip of this item
Sample item showing a diagram of a generator with power and temperature readings. Text reads, "In the nuclear power plant simulation below, the number of nuclear interactions produced by the reactor is regulated by the Control Rods. The power output is regulated by the Generator Valve. The Control Rods and the Generator Valve can be controlled by dragging the sliders up and down. The generator shown has a Temperature reading of 237°C, Control Rods showing 5%, Generator Valve showing 0%, a Power reading of 0 MW, and a safety valve that is closed. Question: "Set the generator valve at 80%. How far (%) should the control rods be lifted for the power plant to supply a continuous output of 800 Megawatts (MW) without the safety valve opening?" Answers: "55%, 70%, 85%, 100%"
click for larger version
[Source: Organisation for Economic Co-operation and Development, 2005]
Conventional Items

These items will ask for students to select or construct their answers. The examples below are of the familiar constructed and selected response formats.

Constructed Response

Similar to the previous example, this item represents a sample task in which students must use their knowledge about the engineering design process to answer a constructed response item. The practice of Understanding Technological Principles is measured by this computer-based item. In this item, students use various tools to explore the factors that affect plant growth in a greenhouse. In the NAEP Technology and Engineering Literacy Assessment, a similar item might ask students to evaluate different greenhouse designs in terms of their effects on plant growth.

Context: Plant Growth
Topic: Designing a Greenhouse
Target Level: Grade 12
Watch a video clip of this item
Sample item labeled Plant Growth. Text reads, "Sarita and Tom want to check out how much difference the light and warmth can make. They decide to make one end of the greenhouse much hotter than the other, and one side with much more light than the other. They put two plants in pots. One plant is to be grown as fast as possible, the other as slowly as possible. Drag each pot to the correct part of the greenhouse." Graph shows temperature: 10°C, 20°C, 30°C, & 40°C; vs. light: Very Dark, Quite Dark, Quite Bright, & Very Bright. Pot labeled as Fast is dragged to 30°C / Quite Bright. Pot labeled as Slow is dragged to 10°C / Very Dark.
 
Below, text reads, "The bar chart shows how much the plants grew each day at different temperatures. Click on the slider to see how the amount of light affects the bar chart." Graph shows Growth in one day: from 0mm to 6mm; vs. Temperature: 10°C, 20°C, 30°C, & 40°C; with a slider below marked Very Dark, Quite Dark, Quite Bright, & Very Bright. Current graph shows 10°C: 1mm, 20°C: 3mm, 30°C: 4mm, 40°C: 3mm, and slider on Quite Bright.
 
Below it reads, "Tom says that the plants will grow faster if they keep them as hot as possible." Question: "Is Tom right?" Answers: "Yes, No" Then, "Explain your answer in your workbook."
click for larger version
[Source: Adapted from Ripley, 2009]

Selected Response – Multiple Choice

The following item is another conventional item type—in this case, multiple choice. Students would employ the practice of Understanding Technological Principles by demonstrating their knowledge about the engineering design process.

Context: Product Design
Topic: The Design Process
Target Level: Grade 12
Sample item. Text reads, "An assignment for the design of an emergency light is described in the box below." Box reads, "Design and build an emergency light. It must have its own battery, provide a bright light, and have a built-in charger. It must be capable of operating while being charged. It must have a watertight enclosure with a handle on the top." Question: "After receiving this assignment, which of the following most likely would be the next step in the engineering design process?" Answers: "A. testing and evaluation, B. selecting the best solution, C. investigation and research, D. construction of a prototype" Option C, investigation and research, has a check mark.
click for larger version
[Source: Massachusetts Department of Elementary and Secondary Education, 2009]