banner Technology and Engineering Literacy Framework for the 2014 NAEP
Examples of Practices Applied in Each of the Assessment Areas

Examples of Practices Applied in Each of the Assessment Areas

The following sections describe how the three practices of Understanding Technological Principles, Developing Solutions and Achieving Goals, and Communicating and Collaborating can be used to classify the general types of thinking and reasoning intended by the assessment targets in the three major assessment areas of Technology and Society, Design and Systems, and Information and Communication Technology (ICT).

Practices Applied in Technology and Society

Assessment targets in the area of Technology and Society are concerned with the effects of technology on human society, the natural world, and the world of information and knowledge as well as with issues of ethics, equity, and responsibility.

Understanding Technological Principles

To provide evidence that they understand principles in these three subareas, students could be asked to perform a variety of tasks, such as recognizing examples of the effects of technologies; identifying examples of ethical and equity issues; describing local and global effects of technologies; explaining the effects of rapidly changing technologies on knowledge creation, access, and management; analyzing beneficial and negative impacts; recognizing examples of responsible, ethical uses of technologies; comparing costs and benefits of technologies; predicting potential impacts on society and the environment; and explaining the relationships among technologies.

Developing Solutions and Achieving Goals

Students must use their understanding of the technological principles for Technology and Society specified in chapter two to apply that knowledge as they address novel issues and problems. To demonstrate their capacity to address issues and problems in the assessment area of Technology and Society, students could be asked, for example, to develop alternative proposals for a new technology based on an analysis of potential positive and negative impacts. Problem solving practices could be demonstrated in a series of tasks and items involved in analyzing the uses of the new technology, gathering data and information on its impacts, analyzing the data, interpreting results, and evaluating alternatives.

Communicating and Collaborating

To communicate and collaborate with others (virtual others in the assessment) in the course to respond to issues, students must draw on their understanding of the technological principles specified in chapter two and apply their knowledge as they work through given problems and issues. For example, to address issues in Technology and Society, students can use a variety of modalities to represent and exchange data, ideas, and arguments about the advantages and disadvantages of technologies. Students can collaborate (virtually) to form teams that will gather and integrate information about the potential impacts of a technology on human society or the natural world. Students can evaluate the qualifications, credibility, and objectivity of virtual experts. Tasks can require students to demonstrate their capability to interact, collaborate, and contribute to work as a team. Students can use various media and representations to share their analyses and recommendations.

Table 3.2 provides some examples of how the 3 practices can be applied to assessment targets for Technology and Society in order to generate tasks and items at the middle school level. The key principles and targets were selected from tables 2.2-2.5 in chapter two. These are sample ideas for items and tasks and will not be used in the actual assessment. Simpler tasks could be developed for grade 4 targets and more complex tasks could be developed for grade 12 targets.

Table 3.2 Examples of grade 8 tasks representing practices in each subarea of Technology and Society

A. Interaction of Technology and Humans

B. Effects of Technology on the Natural World

C. Impacts on the World of Information and Knowledge

D. Ethics, Equity, and Responsibility

Selected Principles & Practices

The relationship between technology and society is reciprocal. Society drives technological change, while changing technologies in turn shape society.

Reusing, recycling, and using fewer resources can reduce environmental impacts.

Information technology is evolving rapidly, enabling ever-increasing amounts of information and data to be stored, managed, enhanced, analyzed, and accessed through a wide array of devices in various media formats.

Technology by itself is neither good nor bad, but its use may affect others.

Understanding Technological Principles

Explain what factors need to go into a decision to change the use of a river and identify possible consequences of doing so.

Identify and provide a rationale for appropriate and inappropriate procedures for disposing of electronic devices.

Compare the impact of geographical information systems and 14th century maps on people's capability to explore new territory.

What are the positive and negative consequences of the predicted change from print to digital news?

Developing Solutions and Achieving Goals

The community has decided to implement a new wind turbine system. Design an investigation into the impact on the community.

Given a specific consumer electronics product such as a cellular telephone, design a new way to increase its appropriate disposal.

Use a simulation to test the adequacy of exit routes for evacuating residents of a mountain town during a wildfire.

What processes and digital tools might the city council put into place in order to make sure all citizens have a say?

Communicating and Collaborating

Collaborate with engineers and urban planners (virtual) through a website to collect and communicate data about the effects of a wind turbine system on the community.

Organize a campaign with a virtual team to inform the public of the dangers of improper disposal of consumer electronic products.

Present a set of images from two artists of the period that represent different perspectives on a major event.

Debate with a virtual team member the privacy and safety issues involved in establishing international projects.

Illustrative Tasks and Items

The following examples represent three types of tasks and items that could be used to assess targets related to the Technology and Society assessment area. The first example is a scenario-based set of items, and the next two are familiar, conventional constructed and selected response items.

Scenario-Based Item Sets

Items of this type present students with a problem or goal set within a broader context. The example below calls on the student to employ the practice of Developing Solutions and Achieving Goals. In this task, the student must use ICT tools to analyze the impacts of technology on human society. This example employs a computer-based interactive format in which students search for information concerning video games and violence and then use word processing, spreadsheet, and Web-based tools to develop a PowerPoint presentation. The PowerPoint presentation calls on the crosscutting practice of Communicating and Collaborating

Context: Human Society
Topic: Video Games and Violence
Target Level: Grade 12
Watch a video clip of Australia MCEEDYA presentation video
Screenshot of a program that features a sample item. Screen reads, "Assignment Details: You and your partner Amanda are organising a class forum on the topic 'Video Games and Violence.' You will need to: • find some information; • analyse some information; and • create a presentation."
click for larger version
Screenshot of a program that features a sample item. Screen reads, "Task Details: You are going to create a slide show about the relationship between Video Games and Violence to present to your class. You should use no more than five slides. Your slide show will be assessed on: • the ideas and information you include; • the way you organise the information; • the design of the presentation; and • your use of the software features. This is the big task for this module. After this big task you will have one small task to complete but this should only take one minute. You should spend most of your remaining time on this big task. Before you begin this task you will watch a demonstration of how to do it."
click for larger version
Screenshot of a program that features a sample item. Screen shows a blank PowerPoint screen.
click for larger version
[Source: Australian Ministerial Council for Education, Early Childhood Development and Youth Affairs [MCEECDYA], 2008]

Conventional Items

These items will ask for students to select or construct their answers. The examples below are of the familiar constructed and selected response formats.

Constructed Response

This task illustrates students' skill in using simulations in a problem solving activity. The student constructs a response by manipulating force arrows in a simulation-based scenario in which an emergency rescue truck must deal with various problems along a fire road in a forest. The student constructs text-based explanations of the balanced and unbalanced forces acting on the truck. While designed to be used as a middle school science item about force and motion, such a simulation could be adapted for the NAEP Technology and Engineering Literacy Assessment to study how the design of the technological system (transportation) affects the environment positively by making it possible to contain forest fires and rescue people and also negatively because of the cutting of trees and the disruption of wildlife habitat. This task also illustrates an interactive item in which students can manipulate different factors in a simulated environment and then provide responses to questions related to the scenario. For the NAEP Technology and Engineering Literacy Assessment, a series of items could call on the practices of Understanding Technological Principles (for example, describing positive and negative effects of roads in the forest) and Developing Solutions and Achieving Goals (for example, designing a fire road that would not cross wildlife migration corridors).

Context: Transportation
Topic: Emergency Rescue
Target Level: Grade 8
Watch a video clip of WestEd Force and Motion Simulation video
Screenshot of a program that features a sample item. Screen shows a firetruck with a arrow pointing to the left, labeled "45 kN," and an arrow pointing to the right, labeled "40 kN." There is a "RUN" button in the middle of the screen. Below, the text reads, "In this challenge you were asked to draw the force that keeps the speed of the truck constant." The first question reads, "How do the balanced forces affect the speed of an object?" The second question reads, "How do unbalanced forces affect the speed of an object?"
click for larger version
[Source: Quellmalz, Timms, and Buckley, 2009]

Selected Response – Multiple Choice

In this conventional multiple choice item, the student selects an answer that describes the effect of fossil fuel on the natural environment, employing the practice of Understanding Technological Principles to analyze positive and negative effects. This item assesses student understanding of the interaction of a technology (energy production) and climate change. The combination of the two can be thought of as a system, with each affecting the other, and the item below also assesses a student's system thinking, albeit on a relatively simple level. Similar items in the NAEP Technology and Engineering Literacy Assessment could focus on the effects of technology on the environment.

Context: Natural World
Topic: Climate Change
Target Level: Grade 8
Sample item. Question: "The burning of fossil fuels has increased the carbon dioxide content of the atmosphere. What is a possible effect that the increased amount of carbon dioxide is likely to have on our planet? Answers: "A. A warmer climate, B. A cooler climate, C. Lower relative humidity, D. More ozone in the atmosphere"
click for larger version
[Source: Trends in International Mathematics and Science Study, 2003]