Introduction

This article is a presentation of an ongoing project and initiative that most likely will have huge implications in the years to come. This is partly due to the number of well-known researchers involved, the agencies and countries involved, and the collaboration with both the Organisation for Economic Co-operation and Development (OECD) and the International Association for the Evaluation of Educational Achievement (IEA), both of which plan to include the results of this project in their future developments of the Program for International Student Assessment (PISA) study and other similar studies. The project’s starting point is a forecast of future skills and competencies, where media and technologies are an embedded element, and an awareness of technological developments plays a role on different levels.

The term “skills” was used in the first initiatives of the project. However, many argued for the term “competencies” instead, since that term includes broader understandings in many languages of what is needed in the future. Still, it was decided to keep “skills” as a term, but clearly state that “skills” in this sense incorporates broader cultural “competencies”.

I was invited to be a member of one of the five working groups (Working group 1 led by Senta Raizen (US), core members: Marilyn Binkley (US), Ola Erstad (Norway), Joan Herman (US), and Martin Ripley (UK)), which imply working on writing a White Paper, which will be the foundation for further developments of the project. I took part in two international meetings of all researchers involved in the project, and then collaborated with the others in Working group 1 on writing the paper. The draft versions of the five White Papers are now finished and available at http://www.atc21s.org/home/. In addition, many researchers around the world have been invited to give comments and input on the papers during the process of writing them. So this part of the project, presented here, is now finalized.

This presentation is compiled based on official documents from the launch of the project and available texts produced as part of the project. I have adjusted and re-written these texts for this presentation.

Background

The Assessment and Teaching of 21st Century Skills (ATC21S) project was created by Cisco, Intel and Microsoft. The companies wanted to reorient their educational commitments and the ways technologies might have impacts on learning. In looking ahead at challenges for the 21st century, the companies wanted to sponsor a project to research and develop new approaches, methods and technologies for measuring the success of 21st -century teaching and learning in classrooms around the world. Robert Kozma was contacted to pull together these interests, organize and define the outline of the project. Researchers from around the world were contacted and organized into different working groups, and several additional researchers commented on drafts of texts.

The focus of the project was set on defining 21st century skills and developing ways to measure them. Based on the status of research and use of ICT in education, assessment became the focal point and the key area for defining future developments. Assessment plays a critical role in setting standards and influencing curricula at the local, regional, national and global level, so it is expected that these new assessments will motivate schools to do more to instill 21st century skills.

The initiative was launched at the Learning and Technology World Forum 2009 in London. During 2009, the project operated with five working groups, each of which produced a White Paper. In announcing the project, the three companies stated, “What is learned, how it is taught and how schools are organized must be transformed to respond to the social and economic needs of students and society as we face the challenges of the 21st century.”

School officials and global assessment organizations such as the OECD and the IEA will be able to use these methods of assessment to evaluate how well schools are teaching 21st century skills.

Defining the field

How we live, work, play and learn has been dramatically transformed by technology over the past twenty years. Today we need different skills than were required in the 20th century, and educational institutions have a critical role to play in developing those skills. But by and large, primary and secondary schools have not kept pace with the changing skill sets that students need to succeed. In fact, there are no large-scale initiatives in place right now to determine whether our schools are doing well at teaching these skills. -Governments as well as schools need to know what works and what does not.

The economy of leading countries is now based more on the manufacture and delivery of information products and services than on the manufacture of material goods, and even manufacturing depends on innovative uses of information and communication technologies. And people access, use, and create information very differently from the way they did in previous decades.

In the 21st century economy and society, the ability to respond flexibly to complex problems, to communicate effectively, to manage information, to work in teams, to use technology, and to produce new knowledge is crucial. These economic and social trends have significant implications for education.

Yet most educational systems operate much as they did at the beginning of the 20th century. While contemporary business and social practices engage people in collaborative efforts to solve complex problems and create and share new ideas, traditional instructional and assessment practices require students to work individually as they recall facts or perform simple procedures in response to pre-formulated problems within the narrow boundaries of school subjects and they do so without the aid of books, computers, social networks, or other resources. Schoolwork is shared with and judged by only the teacher and there is little feedback to the student or opportunity for revision. This project points out that significant reform is needed in education worldwide: what is learned, how it is learned and how schools are organized. However, reform is particularly needed in educational assessment, related to how education and society, more generally, measure the skills needed by productive and creative workers and citizens.

The need for new ways of assessing educational achievement

Assessment in education is one of the most powerful determinants of practice in the classroom, made more so by the standards and accountability reforms of the past decade. Many previous, well meaning and well resourced attempts to reform education have stumbled through an inability to demonstrate improvement on standardized tests designed for last century’s education. More often than not, such efforts have assessed what was easiest to measure rather than what was most important to measure. Consequently, along with changes in other areas of the educational system, educational assessment must be transformed to be more responsive to the social and economic needs of students and society as we face the challenges of the 21st century.

Existing models of assessment are typically at odds with the high-level skills, knowledge, attitudes and characteristics of self-directed and collaborative learning that are increasingly important for our global economy and fast-changing world. New assessments are needed that engage students in the use of technology and digital resources and the application of a deep understanding of subject knowledge to solve complex, real world tasks and create new ideas, content, and knowledge.

The goals of the project

The goals of the project are to:

  • Mobilize the international educational, political, and business communities around the need and opportunity to transform educational assessment, and hence, instructional practice, and make doing so a global priority.

  • Specify high-priority skills, competencies, and types of understanding that individuals needs to be productive and creative workers and citizens of the 21st century and turn these specifications into measurable standards and an assessment framework.

  • Examine innovative ICT-enabled, classroom-based learning environments and formative assessments that address 21st century skills and draw implications for ICT-based international and national summative assessment and for reformed classroom practices aligned with assessment reform.

  • Identify methodological and technological barriers to ICT-based assessment and support the specification of breakthrough solutions that are needed to measure 21st century skills.

  • Support the implementation of the newly developed standards and breakthrough methodologies, pilot test them in selected countries, and make recommendations for broader educational assessment reform.

Working together with existing assessment efforts

There are many international and national assessment programmes, assessment organizations, nongovernmental organizations, businesses, research centres, and individual researchers working on the specification of 21st century skills and development of ICT-based formative and summative assessments. The project does not aim to duplicate these efforts or to develop its own assessment. Rather, the project will add value to efforts of OECD, IEA, and national assessment efforts by catalysing the international community to identify the problems, issues, and barriers that:

  • are common to all

  • are of the highest priority

  • cannot be addressed by any individual project alone.

Furthermore, the project will provide a structure by which this international community can draw on and share existing knowledge and create effective solutions to address the problems, issues, and barriers associated with the identified skills and foster wide-scale adoption of assessment reforms.

Five working groups

The project is led by the Executive Director, Dr Barry McGaw of the University of Melbourne, a former education director at OECD, and a Management Team. The ATC21S project is operating through five working groups, each of which will review the current state of development and propose research and development activities to address current deficiencies. Together, the working groups comprise individuals from more than 60 research institutions. Each working group has produced a White Paper available on the project website; http://www.atc21s.org.

Five “founder countries” have agreed to take part in the research and will deploy ATC21S pilot projects in schools as early as February. The five are Australia, Finland, Portugal, Singapore and the United Kingdom.

Summaries of the five White Papers

White Paper 1 (Led by Senta Raizen, US)

This paper synthesizes research on the role of standards and assessment in promoting learning, describes the nature of assessment systems that can support changes in practice, illustrates the use of technology to transform assessment systems and learning, and proposes a model for assessing 21st century skills. Large-scale assessments should be only a part of any system to support student learning. Assessments at each level represent a significant opportunity to indicate the important learning goals of the broader system as well as to provide valuable, actionable data for policy and practice. Moreover, they can model next generation assessments that can support learning. To do so assessments should:

  • be aligned with the development of significant 21st century goals

  • be adaptable and responsive to new developments

  • be largely performance-based

  • add value for teaching and learning by providing information that can be acted on by students, teachers, and administrators

  • meet the general criteria for good assessments (i.e. be fair, technically sound; valid for purpose, and part of a comprehensive and well-aligned system of assessments at all levels of education).

The model for assessments of 21st century skills, based on an analysis of curriculum and assessment frameworks for 21st century skills developed around the world, identifies ten important skills in four broad categories. The paper provides measurable descriptions of the skills, considering knowledge, skills, attitudes, values and ethics (advanced as the KSAVE framework). The paper concludes with a discussion of challenges to be addressed in developing an assessment system that supports learning using, for example, research-based models of skill development and assessments that make students’ thinking visible to establish their strengths and weaknesses and help shape future learning choices.

White Paper 2 (Led by Mark Wilson, US)

This paper has surveyed the methodological perspectives that are seen as being important for assessing 21st century skills. Some of these issues are specific to 21st century skills, but the majority of them would apply more generally to the assessment of other psychological and educational variables. The narrative of the paper initially follows the logic of assessment development, commencing with the definition of constructs to be assessed, the design of tasks that can be used to generate informative student responses, the coding/valuing of those responses, the delivery of the tasks and the gathering of the responses, and the modelling of the responses with respect to the constructs. The paper continues with a survey of the strands of validity evidence that need to be established, and a discussion of specific issues that we see as being prominent in this context, such as: the need to resolve issues of generality versus contextual specificity, the relationships of classroom to large-scale assessments, and the possible roles for technological advances in assessing these skills. The paper also adds a brief segment discussing some issues that arise with respect to specific types of variables involved in the assessment of 21st century skills. It concludes the main text of the paper with a listing of particular challenges considered prominent at this time. The paper has an annex that describes specific approaches to assessment design that are seen as useful in the development of new assessments.

White Paper 3 (Led by Beno Csapó, Hungary)

This paper reviews the contribution of new information-communication technologies to the advancement of educational assessment. Improvements can be described in terms of precision in detecting the actual values of the observed variables, efficiency in collecting and processing information, and speed and frequency of feedback given for the participants and stakeholders. The paper reviews previous research and development in two ways, describing the main tendencies in four continents (Asia, Australia, Europe and North America) and summarizing research on how technology advances assessment in some crucial dimensions (assessment of established constructs, extension of assessment domains, assessment of new constructs and in dynamic situations). As there is a great variety of applications of assessment in education, each one requiring different technological solutions, the paper classifies assessment domains, purposes and contexts and identifies the technological needs and solutions for each. The paper reviews the contribution of technology to the advancement of the entire educational evaluation process from authoring and automatic generation and storing items through delivery methods (Internet-based, local server, removable media, mini-computer labs) and forms of task presentation made possible with technology to response capture, scoring and automated feedback and reporting. The paper also reviews some special cases for which new technologies have enabled significant advances (e.g. assessments of students with special educational needs, assessment of collaborative skills and group achievement) and discusses the validity issues raised by the application of the new technologies (e.g. factors influencing achievements when working with technological tools, the question of transferability of skills measured in a virtual environment). Finally, the paper identifies areas where further research and development is needed (migration strategies, security, availability, accessibility, comparability, framework and instrument compliance) and lists themes for research projects feasible in the Assessment and Teaching of 21st Century Skills project.

White Paper 4 (Led by Marlene Scardamalia, Canada, and John Bransford, US)

The paper reviews literature on knowledge-creating organizations to identify sequences leading from entry-level 21st century skills to mature levels of the skills defined by the Assessment and Teaching of 21st Century Skills project. It suggests a developmental scheme to allow students and teachers in any classroom to find a starting point and advance along dimensions identified. In a fully developed knowledge building environment, the ways people and ideas interact are critical to the integration of deep understanding, knowledge creation, and practical action. After proposing a framework for moving toward high-end knowledge environments the paper considers basic principles of learning and developmental trajectories relevant to them. The paper distinguishes two approaches: “working backward from goals” and “emergence of new skills”. It discusses how modern technologies can help integrate and enhance these different approaches, how formative assessments can be used to increase the pace of innovation, and how a broader systems perspective might inform large-scale summative assessments.

An analytic framework, with developmental trajectories defined by 21st century skills, is provided for analysing environments in light of the extent to which they support knowledge creation. The goal is to provide a scheme comprehensive enough to identify starting points, as well as pathways to higher-order achievements for all, from elementary through to tertiary education, and applicable to out-of-school contexts, so as to support an inclusive model of 21st century knowledge building. The paper also aims to distinguish efforts that prepare students for work in knowledge-creating organizations after they leave school from those that aim to transform schools to operate as knowledge-creating organizations in their own right. It ends with suggestions for new initiatives to help advance education for a knowledge-building society.

White Paper 5 (Led by Linda Darling-Hammond, US)

Many nations around the world have undertaken wide-ranging reforms of curriculum, instruction, and assessments with the intention of better preparing all children for the higher educational demands of life and work in the 21st century. While large-scale testing systems in some countries emphasize multiple-choice items that evaluate recall and recognition of discrete facts, there is growing use of more sophisticated approaches in many countries. These approaches include not only more analytical selected response items but also open-ended items and curriculum-embedded tasks that require students to analyse, apply knowledge, and communicate more extensively, both orally and in writing. A growing emphasis on project-based, inquiry-oriented learning has led to increasing prominence for school-based tasks in state and national systems, including research projects, science investigations, use of technology to access information and solve authentic problems, development of products, and presentations about these efforts.

This paper briefly describes the policy frameworks for assessment systems in Australia, Finland, Singapore and the United Kingdom, with special attention to identifying where assessment of 21st century skills has been or may be developed in assessment systems that report information at the national or state, as well as local, levels. Identifying the role of 21st century skills within these assessment systems serves two purposes. First, this process furthers knowledge about distinct approaches to the integration of 21st century skills in countries with different educational governance systems. Second, it provides information about how assessment systems work within the broader policy landscape of each country that determines student learning opportunities through the construction of policies governing teacher education and development, as well as curriculum, instruction, and assessment.

Overall key challenges

  • Developing new kinds of psychometrics (educational and psychological assessments)

  • Making students’ thinking “visible” (being able to see students’ mental processes, including how they draw on their own experiences)

  • Accounting for new modes of communication

  • Ensuring the validity of standards on which assessments are based

Next steps

In 2010, the project will set up new working groups to develop and pilot fresh approaches to assessing individual and collaborative problem solving and digital literacy in primary and secondary schools. This work will be done in collaboration with OECD and IEA, the leading global assessment agencies. Both intend to use the research findings in their next major rounds of assessments: OECD in PISA 2012 and IEA in 2013.

More information is available at: http://www.atc21s.org