Teachers’ perceptions of data management as educational resource
A comparative case study from China and Norway
- Side: 178-189
- DOI: https://doi.org/10.18261/issn.1891-943x-2020-03-04
- Publisert på Idunn: 2020-10-09
- Publisert: 2020-10-09
- Creative Commons (CC BY-NC 4.0)
Introduction of data-driven tools and practices in education may improve learning but also change education in ways not yet fully understood. Globally, ministries are now beginning to define big data policies for schools. How will this influence the work of teachers, and how do they want to engage with the new sources of information? This study compares two cases from Norway and China, two countries with very different educational systems and cultures, and different implementation policies. In a survey, a hundred primary and secondary school teachers from each country were asked what they wanted to know about their students, and what data they wanted educational big data analysis to be based upon. Qualitative analysis of learning technology infrastructure projects in Tongzhou district in Beijing and in Oslo and national policies provided a backdrop for interpreting the teachers’ answers. Focus group interviews with teachers gave additional insights into the different approaches to educational big data. This study shows that Chinese teachers have a much more open mind and interest for all information data may provide, while Norwegian teachers are more focussed on knowledge acquisition and would like to limit the scope to learning and teaching the curriculum, being concerned about questions related to privacy and data surveillance. Some of these differences could be explained by the differences in context of introducing big data in schools. In China, economic innovation is the driver, while in Norway the pretext for the introduction of learning analytics is adapted education. This study contributes to the knowledge of how data policies will impact on design of learning analytics tools and solutions. Approaches and technologies in this field have a global scope. Therefore, understanding how personal digital data is perceived in different cultural contexts is vital for both development and implementation.Keywords: Big data, educational big data, learning analytics, comparative case study, Chinese education, Norwegian education
This study asks how educational big data will have the potential to influence primary and secondary school development by comparing two very different countries, Norway and China. This is the first comparative case study exploring how the unprecedented access to learning activity data and other school data plays a role in shaping different levels of the educational systems in Europe and Asia. Today, big data is about to transform many sectors of society, education included. Cities are becoming ‘smart’ and data-driven; we are seeing radical changes in transport (self-driving cars), in retail (online shopping), in energy (smart grids and smart homes), in banking (mobile apps), and in other sectors. It is fair to assume that artificial intelligence, machine learning, deep learning and other big data-driven technologies will also influence the educational sector due to ‘cross-field effects’ (Yu, 2017; Rawolle, 2005). Norway and China start from contrasting positions in developing policies for how big data could be used in education. The Norwegian national digitalisation strategy for schools highlights new opportunities for adaptive learning materials and learning analytics (LA) targeting the individual learner, but cautions on quality, ethics, privacy and data protection issues related to big data (Kunnskapsdepartementet, 2017). When asked if ‘digital learning analytics’ should be used in Norwegian schools and thus part of a new Education Act, a recent Official Report hits the brakes: “The situation today appears to be unclear and more characterised by what is on offer in the market than what will serve schools and pupils. (…) The Commission proposes that the Government puts together a [new] public commission to consider use of digital learning analytics in schools” (NOU 2019:23, p. 408, authors’ translation). The Chinese education development plan, on the other hand, claims support for nation-building goals, while using big data and cloud computing to promote the sharing of educational resources (China State Council, 2016). In China, the concept of educational big data (EBD) is linked to the Internet Plus action plan that was launched in 2015, “aiming to integrate Internet with traditional industries, and fuel economic growth” (Xinhua, 2015). Both policy rationales for the use of educational big data, the Norwegian learner-centred motivation and the Chinese utilitarian one, will receive responses from the educational system itself. Teachers will respond to suggestions to use different kinds of data; institutions will adopt different kind of tools; and other stakeholders such as parents, local schools authorities, institutional managers, and others will develop new practices and policies.
This paper explores the responses to calls for the use of educational big data in the two countries: What is the response from the ministries of education in terms of policy development? And what are the teachers’ responses?
Data handling in Chinese and Norwegian education
As a consequence of continuous testing and comparison with local, regional, national and international peers, numbers play an important role in most educational systems. ‘Centers of calculation’ (Ruppert, 2012) are established with the task of monitoring performance levels of schools, allowing ministries of education and school authorities to conduct governance at a distance and by numbers (Hartong, 2017) by applying the power of measurements (Caspersen, Smeby, & Aamodt, 2017) and numbers (Feniger, Israeli, & Yehuda, 2016; de Freitas & Dixon-Román, 2017). Literature search in both western and Chinese databases reveals that comparative research on educational development and reform in China and the West is scarce. Internationally, big data and data science have recently become an important issue for educational research (Daniel, 2017). However, searching in the Chinese national full-text research database (cnki.net) using keywords identifying EBD gives very limited results, mainly papers focussed on clarification of aims and affordances of EBD. In addition, western insights into Chinese advancements in educational technologies are limited. Investment in educational technology in China should be seen in light of the rapid expansion that higher education, especially, has undergone since 1998. Enrolment of undergraduate students grew on average about 27% annually in 1998–2004 (Wan, 2006). Economic growth has been a central theme in China since the 1990s. “Educational policies, without exceptions, were made to serve this need” (Wan, 2006, p. 27). The changing practices in Chinese education need to be understood against the backdrop of a culture of learning that is based on Confucian heritage: careful study of a canon of texts combined with the practice of moral self-cultivation (Jin & Cortazzi, 2006). After junior middle school, students are at a crossroads, either progressing to senior middle schools that may lead to university, or engaging in vocational training. Exam pressure is strong in Chinese education (Zhao, Selman, & Haste, 2015). Since the 17th national congress, ‘special education’ has been recognised for the first time in China’s policy papers as a special need addressed in its own right (Han & Ye, 2017, p. 403).
In Norway, schools fell behind in the PISA ranking at the turn of the century, and this ‘PISA shock’ stimulated reform of the entire primary and secondary education system. “[N]ew outcome-based curricula with cross-disciplinary basic skills were accompanied by major revision of assessment regulations, comprehensive government projects promoting formative assessment, national tests as a main component in a new national quality assessment system and new regulations for examinations and teacher reporting of overall achievement marks” (Tveit, 2014, p. 221). While Norwegian primary and secondary education is “riddled with unresolved tensions as to the role of assessment criteria and nations test” (Tveit, 2014, p. 221) these tensions are eased by another major discourse on adaptive education, “which is an important political measure meeting challenges in the school sector related to social equality, learning outcome, and inclusion” (Jenssen, 2011).
Over the last decade, data use in education has been linked to learning analytics (LA), most commonly defined in 2011 as “the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs” (Siemens & Gasevic, 2012). Data is part and parcel of LA; the challenge, however, is to understand what comes together with the data analytics practices when embedded into the educational sector. We build our analysis on three premises, shortly set out below:
Firstly, LA is always introduced in a political context. This aspect is explored more in detail in the next section.
Secondly, the fact that data drives LA brings a risk of LA contributing to the datafication of education. One may claim – at least rhetorically – that the definition of LA most often espoused by the Society for LA Research, SoLAR, (Siemens & Gasevic, 2012) has already taken a stand regarding the role of the learner in analytics. It says the LA is about data about the learners, not with the learners, suggesting an objectification of the learners. The consequences of the datafication of education have been a concern even before LA came to attention (Breiter and Hepp, 2016; Hartong, 2017; Williamson, 2018a, 2018b, 2017a, 2017b, 2016, 2015; Selwyn, 2015, 2014). In terms of ethical pitfalls related to LA, this concern is about how the data traces left by the learner represent the true learner. Are the data describing the real learner or just a skewed view of the learner provided by the limited set of data available? And does the data practice give the learners agency?
Thirdly, LA may be part of a ‘perfect storm’ that involves trending technologies known as artificial intelligence (AI), all with a voracious appetite for data and a promise to provide personalised precision learning. Even if LA research and LA practices are framed by educational values, LA will be strongly influenced by what happenings in society at large. In countries like Norway, schools and universities distinguish clearly between data produced and managed by institutional systems, and data the users generate outside of education using sports apps, social media, and interacting with the myriads of systems that leave data traces. This is not necessarily the case in other countries, and it may be changed by pressures that build outside of education and potentially without too much pushback from the data subjects – the students – themselves. This needs to be explored through empirical studies recognising the complexity of understanding data and data use underscored in the papers included in the NordSTEP’s special issue on data use in education (2017), which emphasized “the difficult balance that must be struck between data use for development and control purposes” (Prøitz, Mausethagen, & Skedsmo, 2017a, p. 2).
The educational policy context of LA in Norway and China
In Norwegian educational policy development, LA was first introduced in 2013 in a report by the Norwegian Agency for Quality Assurance in Education (NOKUT, 2013), and the following year in an Official Report on “MOOC to Norway” (NOU 2014:5). Norway was one of the first countries in the world to have a policy document highlighting MOOCs and LA and its implications for higher education and lifelong learning. For schools, however, the interpretive context of LA was more restrictive. In the Ministry of Education strategy for digitalisation of primary and secondary education, the term learning analytics was used four times, each time in conjunction with the concept of adaptivity: “Learning resources based on learning analytics contribute to an education adapted to the students’ needs and preconditions” (Kunnskapsdepartementet, 2017, p. 12); teachers should have “knowledge about pros and cons regarding the use of learning analytics and adaptive learning resources” (p. 13); “new technologies and use of big data open up possibilities for adaptive learning researches and learning analytics” (p. 19); and “ICT may lead to better assessment practice, and possibilities to use learning analytics and adaptive learning” (p. 22, authors’ translation).
Adapted education is described by Fasting (2013) as the Norwegian pathway to inclusive and efficient education. The national law regulating primary and secondary education prescribes that “education shall be adapted to the abilities and preconditions of the student” (§1-3). Taking care of and respecting the variety of pupils are important values in Norway, and the education system is regarded as inclusive. However, there is an ongoing debate on how to increase learning outcomes, as promoting inclusiveness and effectiveness in the political discourse are often depicted as contradictory values. Adapted education is thought to solve this conundrum, as highlighting inclusion, variation, experience, relevance, values, context and participation are seen as the key to achieving both inclusive and effective education. When educational big data establishes itself as a viable technology, the yearlong debate on adaptive education may have prepared the ground for Norwegian schools to welcome approaches based on learning analytics. However, this may take some time, as mentioned in the Introduction where we referenced the recent Official Report (NOU 2019:23) on the need for a new Education Act.
For the commission behind the Official Report 2019:23, it is mainly issues related to privacy and the sharing of personal information that raises a red flag for LA. It refers to the Norwegian Data Protection Authority, which wants a new education act limiting for “how far into the students’ private sphere the digital tools may go” (p. 403). According to the Data Protection Authority, if a new education act does not take a stand on this issue, the commission leaves the decisions in the hands of the technology companies.
In China, on the other hand, LA is framed in a Big Data context (as in the preferred term ‘educational big data’). This framing invokes national strategies, planning, resources and projects that will eventually trickle down the layers of government to be felt by the individual school and teacher. President Xi Jingping, in his report to the 19th Party Congress in October 2017, promoted the “profound convergence of the Internet, big data, artificial intelligence and the real economy” (Creemer, 2017). The national information technology development strategy has an ‘Internet Power’ strategy and a ‘National Big Data’ strategy, both with transformative potentials. When the Beijing Municipal Education Commission developed their reform and development planning for the period, two of 20 projects to be supported are related to ‘Internet + education’ and ‘Big Data’. Data management will be strengthened, a large data system for education built, and digital schools and digital resources sharing exchange platforms promoted (Beijing Municipal Education Commission, n.d.). The educational big data perspective is a system perspective where a “new model for future education” is explored by supporting “all levels of schools to build smart campus with comprehensive use of the Internet, big data, artificial intelligence and virtual reality technology” (Creemer, 2017).
When comparing official policy narratives in Norway and China, we see that Norwegian school authorities are guided towards a narrower scope for the application of LA than their Chinese counterparts. There seems to be more carte blanche or even encouragement for the latter to test out the most advanced AI technology in the market in order to see if there is any educational benefit to its use.
So far in this paper we have applied desk research with review of literature from both western and Chinese sources, focussing on how educational data can be used and understood. This study applies a qualitative interpretative or hermeneutic multiple-case approach (Bos and Tarnai 1999; Yin, 2009), as well as a quantitative survey. In studying teachers’ views on student data, two ‘municipalities’ in Beijing and Norway responsible for primary and secondary education were chosen as cases. In Norway, 108 respondents from Oslo, in China 104 respondents from Tongzhou, answered a questionnaire. In addition, a focus group of 20 secondary level teachers in Tongzhou were interviewed. The Oslo survey was also supplemented with individual and unstructured interviews with educational administrators in the municipality. A convenient sampling methodology was followed for the survey and interviews.
The variety of data sources and methods of analysis is due to the challenges of collecting data in cross-cultural settings. In China in particular, access to data may be a problem. This is primarily not an issue of language, but more about research culture and sometimes sensitivity due to political events, e.g., a party congress.
A number of open questions allowed the respondents to explain their views. In addition there were questions soliciting the teachers’ views on LA and EBD as a tool and practice.
The Norwegian questionnaire was sent to official e-mail addresses of all 186 Oslo schools in late 2017. In Beijing, we recruited respondents in early 2018 by using a social media list targeting Tongzhou teachers taking part in an Internet+ project focussing on educational reform.
The teachers of Oslo and Tongzhou serve approximately the same population in size, around 700,000. In national tests, Oslo on average performs better than other primary and secondary pupils in Norway. Tongzhou is singled out as new administrative district of Beijing that will be turned into a ‘smart city’. In September 2016 Beijing Municipal Committee of Education launched an Internet+ project to reform the junior high school in Tongzhou to achieve better in senior high school examination. The Oslo teachers covered both primary and secondary level, while the Tongzhou teachers mostly covered lower secondary level. Educational level, however, does not seem to influence responses to the survey significantly. Both teacher groups, with on average almost the same years of service (Oslo 15 years and Tongzhou 14 years), are closely monitored for educational results. However, among Oslo teachers LA is not widely known as a phenomenon. Less than 20% knew the term, while in Tongzhou 50% of our sample knew about EBD. Both groups are focussed on improving results; what information do they want to get from educational data?
What do teachers want data to tell them?
We asked the teachers what they wanted to learn about their students; then we asked them what data source they wanted to use to get the answers. Ranking the importance on a five-point Likert scale, the two teacher samples vary significantly on a number of variables according to independent variable T-test.
Figure 1 shows the answers to 20 questions about teachers’ interests grouped in four dimensions, comparing the two ‘most important’ and the two ‘least important’ measures, leaving out the neutral category. The levels of interest in learning activities and learning results among Oslo and Tongzhou are about the same according to Figure 1. This may indicate that when it comes to pedagogical interest in a narrow sense – activities directly related to learning activities and its results – there is no difference between Oslo and Tongzhou. A closer look at the individual questions (Table 1) shows that interest in test scores is the same between the two groups; however, there might be differences in pedagogical focus. The Oslo teachers score significantly higher on interest in Knowledge gaps (t=4.5, p<.001), in Subject mastery (t=5.02, p<.001), and in Recommendations for learning materials (t=3.85, p<.001). Tongzhou teachers, on the other hand, score significantly higher on interest in Details on activities in class (t=4.99, p<.001), and activities in Discussion fora (t=3.89, p<.001). The observation may indicate that the Oslo focus is more on the knowledge acquisition of the individual – they score higher on questions related to the learning task, while the Tongzhou focus is more on contextual activity factors in class, like whom the students communicate with, or how the students score on personality tests.
This finding is strengthened by looking at what data sources the teachers found useful (Figure 2. Tongzhou teachers scored significantly higher on Click on screen (t=4.11, p<.001), Gaze (t=5.26, p<.001), Log-ons online platforms (t=4.10, p<.001), and Reading of relevant books (t=5.48, p<.001).
Figure 1 shows that Oslo teachers are slightly less interested in issues related to personality and psychological state. However, the survey found no significant difference between Oslo and Tongzhou teachers regarding Students’ personal interests, and Feeling of well-being (Table 1). On the other hand, regarding the specific data that may be used to quantify this dimension we found significant differences between the two groups. Tongzhou teachers scored significantly higher on Physical fitness (t=8.58, p<.001), Nutrition (t=6.03, p<.001), and Score personality tests (t=10.44, p<.001).
Figure 1 shows that Oslo teachers are less interested in students’ social activities or activities outside of class. For all variables (Table 1) Tongzhou teachers score significantly higher (p<.001) than their Oslo colleagues.
Overall, our data indicate that the Tongzhou teachers have a stronger appetite for a more varied set of information than their Oslo colleagues. This finding is strengthened by the answers given to an open question allowing the respondents to reflect upon other things they wanted to know about their students. Some Oslo teachers express their uncertainty as to what data can answer. “Some of this may be important to me, however, maybe I already have enough information about these matters.” Other responses are: “As a contact teacher I have access to most of the information that concerns the school”. “Most of these things I know a lot about today; I don't need to know more.” “It is impossible to validate data (the truth) from students interacting with electronic tools. This is human relations, which are key to this type of information.”
Some Oslo respondents, however, were more open to the affordances of LA and wanted to know more about learning strategies, who the students are, social and family relations, etc. These respondents all reported that they had heard about LA.
When expressing their views in their own words, the Tongzhou teachers focus on two main categories of interest: family background of their students, and mental health/psychological status. They want to know “what factors of family background impact students’ learning”; “family living environment, home learning environment, partners’ attitudes towards education,” etc. What are the students’ psychological development; are they motivated to learn, enthusiastic for completing homework, etc. Nearly half of the sample wrote their opinion; only a handful of them mentioned issues related directly to the learning activities, e.g., study method, use of information technology, weak knowledge spots, etc.
In this study we have been interested in what data sources should be made available to improve education. We began this part of the questionnaire with an open question about sources of data the teachers found most appropriate to focus on. Nearly half of both the Norwegian and Chinese teachers answered, strengthening the differences in focus noted above.
“We want to focus on things that are directly related to learners’ work on subject matters and mastery of subjects, and the further development of the students to address knowledge gaps. More peripheral information on things outside the classroom should be gathered through talking to students.” This answer captures the sentiment of the Oslo teachers. Tongzhou teachers, on the other hand, are more likely to ask, “how does the family perform?”, and “what is the students’ mental health?”. Both groups are interested in data that shows learning impact; however, in the Oslo teachers’ own words there is a clear opposition to capturing this information by digital means (reflected in 13 of 49 responses). This scepticism against big data was not present in the Tongzhou responses.
When given a range of 17 data sources that could be used for analytics, the Oslo teachers view most of them, with the exception of test results, as not useful (Figure 2). Our sample of Tongzhou teachers have a nearly opposing position to their Oslo colleagues; they view most sources of data as useful. The least useful data sources for the Oslo teachers are the ones tracking students’ locations, likes on social media, fitness, family income, and spare time activities. In Tongzhou, data on test results and whom the students are communicating with are seen as most useful; and least useful are data on 24/7 location, family income, and where students live. The answers to our questions provide a picture of Oslo teachers with a gaze into the classroom, and of the Tongzhou teachers with a gaze both into the classroom and into the family situation outside of class. This observation does not mean that there are clear differences in the two teacher groups when it comes to care for and understanding of the student as a whole person. However, the survey results give a clear difference in the attitudes towards using digital data to support their teaching practice.
When asked to reflect on other relevant data sources, the Oslo teachers who respond are critical; three respondents use the word ‘surveillance’: “Some of the questions have a flavour of surveillance!” “Half of what you describe here is most likely not allowed according to the Norwegian Data Protection Authority”. The few Tongzhou teachers giving their response, on the other hand, are looking for ways to expand the access to data, mentioning reading speed, calculation speed, reading time, facial emotions, etc.
Will big data in education be useful?
The Chinese teachers in our sample find learning analytics much more useful for the school system and for themselves as teachers than their Norwegian colleagues. They also expect the technology to be easier to implement in Tongzhou than the Oslo teachers found (Figure 3).
With less than 20% of the Oslo respondents having prior knowledge about LA, one could expect that for them, threats are more prominent than benefits. Around one-third of them offered their opinions to an open question, pointing to problems related to time to implement, loss of time that could be used to communicate with the students, and to problems of surveillance and data protection. However, some benefits were also mentioned: “I wonder if LA may contribute to coming up with bespoke learning designs for the students. I therefore think about this as a form of adaptive education.” Half of the Tongzhou sample had prior knowledge of LA; for the three-quarters who answered, the focus was on the benefits of the new technology. More than half of the answers pointed to improved understanding of students and their learning. A quarter highlighted that the effectiveness of teaching might be improved, while only one-tenth touched upon possibilities for adaptive learning. In terms of problems with LA, almost half of the respondents addressed problems with access to and reliability of data. Other concerns were time for LA, application of LA, and the theoretical understanding of how LA could be developed and used.
Solving the bigger problem – inclusion vs selection
A focus group discussion with biology teachers in Tongzhou provides context to the survey results, showing an almost omnivorous interest in data about students. Current practice for this group of teachers is very much data-driven. Once every week students fill out self-report forms detailing how they are doing in class, and what kind of activities they are involved in after class. Then the teachers report on their observations. After that the two forms are brought to the parents who fill in their evaluation. In addition, teachers do exam data analysis, perform learning style tests, and ‘big 5’ psychological testing. The teachers are also tested, information going up to the administration, not down again, it was said.
One teacher worked in a school set up to prepare students for studies abroad. She explained that the main use of the data was for filtering those students who were allowed to go. Selection of talents was also the focus of the other teachers of the focus group. They wanted to know who had the potential to qualify for university studies, so that they knew whom to give special attention. In discussing what aspects of LA system might provide value to this group the selection theme came up again. If a system could help them learn what contributed towards bringing their students to a good university, the administration would be pleased and they would benefit as teachers.
When asked in the survey what role adaptive teaching played, the Tongzhou teachers were not sure. The Oslo teachers, on the other hand, clearly expressed that adaptive education plays a central role in their work, ‘in principle’; however, ‘in practice’ it is harder to achieve (19.2% finds it difficult; 37.6% easy; and 43.1% in-between). In Norway, the Education Act (1998, no 61) states “education shall be adapted to the abilities and aptitudes of the individual pupil, apprentice and training candidate”. One does not find an equivalent statement in Chinese educational statutes.
At school level, the findings of this study indicate that EBD may play distinctly different roles in Chinese and Norwegian education. In China, the unit of analysis for EBD is more students in a group with the challenge to rank students and administer educational support to those selected, while in Norway the focus is on the individual pupil, and the challenge is to help individuals to reach their learning potential. If this observation is valid we should see that implementations and policies on local school authority level and national level point in the same direction.
Conclusions and further work
The presented case studies from China and Norway draw up two contrasting trajectories for the future of big data in education. The narratives unfolded at school, municipal, and national levels in Oslo and Beijing are consistent. In Norway, the priority is inward-looking, adaptive education for the individual, and data used to improve cognitive skills on subjects. In China, on the other hand, the heart of the matter is outward-looking. It is about improving the economy, providing the right skill sets, and selecting whom to support with targeted resources. What is most interesting in our findings is the degree to which teachers are prepared to act according to these narratives. Only time constraints would prevent a Tongzhou teacher from including data from students’ most private out-of-school moments. Oslo teachers would strongly object to the use of such data, resisting ‘surveillance’ and infringement of privacy.
In this study we have focussed on the willingness of the teachers to require student data to improve teaching. Indirectly, this has given us a glimpse into different learner models with varying degrees of sovereignty over their student data. The Tongzhou learner leaves data trails that are there to be exploited if the teachers and the system so wish. The Oslo learner may be surrounded by more legal constraints following the European General Data Protection Regulations, which make it more difficult to simply grab data without individual consent. However, we would suggest that teacher expectations and teacher culture would be most influential in shaping LA and EBD. Tongzhou and Oslo are currently at opposing ends of a data access continuum.
It is obvious that the research presented in this paper has limitations. One may think that language would pose a serious problem doing comparative studies between Norway and China. In fact, language is the easiest stumbling block to negotiate. Different political contexts need to be handled when engaging with the topic of data use in education, both in China and Norway. In addition, there are different research traditions with different emphases on empirical versus theoretical research. In sum, this comes down to theoretical foundation and choice of methods. Prøitz, Mausethagen, and Skedsmo (2017a) called for the development of finer-grained concepts for what we are studying in relation to data use in education. This study has provided some contributions, but there are more to be added in further work.