In October 2008, the Governing Board awarded a contract to WestEd to develop a framework and specifications for assessing technological literacy. In carrying out its work, WestEd collaborated with the Council of Chief State School Officers (CCSSO), ISTE, ITEEA, the Partnership for 21st Century Skills, and the State Educational Technology Directors Association (SETDA). In working with these groups, WestEd used a process for developing the framework and related products that was inclusive, deliberate, and designed to achieve as much broad-based input as possible.
A two-tiered committee structure with a steering committee and a planning committee provided the expertise to develop the framework as specified by the Governing Board. (See NAEP Technology and Engineering Literacy Project Committees and Staff section for lists of committee members.) The two committees were composed of members who were diverse in terms of role, gender, race or ethnicity, region of the country, perspective, and expertise regarding the content of the assessment to be developed.
The steering committee members included leaders from a variety of fields and subject areas, including schools, engineering, education, 21st-century skills, the Internet, business, science education, general education, and assessment. The co-chairs were balanced, with one representing technology in schools, and the other, STEM and assessment. Functioning as a policy and oversight body, this group outlined the planning committee’s responsibilities in developing the framework. The committee reviewed and provided feedback on drafts of the framework and related materials. The interaction between the two committees was iterative over the course of the project.
The planning committee, as supported by the project staff, was the development and production group responsible for drafting the framework, specifications, recommendations for background variables, and preliminary technology and engineering literacy achievement level definitions. This committee was composed of business leaders, researchers, state and district technology coordinators, teachers, and representatives from educational organizations as well as experts in research, assessment, and evaluation. As with the steering committee, the planning committee co-chairs were balanced, with one being an expert in ICT-based learning and assessment and the other being an expert in K-12 science and engineering education.
The planning committee's work was guided by policies, goals, and principles identified by the steering committee (see summary box on the previous page;for full text, see Appendix B ). In addition to the sources cited previously, the planning committee relied on guidance provided by NAEP Technological Literacy Framework and Specifications Development: Issues and Recommendations, a paper prepared by Sharif M. Shakrani and Greg Pearson for the Governing Board.
The work was carried out in a series of meetings, with numerous telephone calls and email exchanges between meetings. From December 2008 through September 2009, the steering committee met four times and the planning committee met six times. Three steering committee and planning committee meetings overlapped so that the two committees could share understandings and discuss critical issues. Governing Board staff supported and participated in all of the meetings. In addition, between formal work sessions, Governing Board members and staff provided feedback and guidance on project documents and processes.
After the development of initial drafts of the framework, WestEd led a series of outreach efforts to solicit feedback. Formal activities included, but were not limited to, presentations and sessions with industry representatives (for example, IBM and Cisco), ITEEA, the Organisation for Economic Co-Operation and Development (OECD) International ICT Research Workshop, CCSSO, SETDA, ISTE, and the Partnership for 21st Century Skills Webinar. (See NAEP Technology and Engineering Literacy Project Committees and Staff section for more complete lists of individuals and organizations that contributed to the development of this framework.) The planning committee reviewed feedback from these groups as well as from the steering committee and made revisions, as it deemed appropriate. Governing Board members and staff closely monitored the project and provided comments on draft documents. After final approval from the steering committee, the framework was submitted to the Governing Board for action.