Visualisation of Trajectories of Participation in a Wiki: A Basis for Feedback and Assessment?
Learning from digital resources requires different competences compared to learning from textbooks. Such competences cannot be fostered without support; social and material. However, assessment of such practices is not well accounted for. We analyse data from a four year intervention study where teachers and researchers collaborated in developing a wiki to afford relevant tasks and assessment of students’ activities. Our findings show that material tools may afford and make visible ideas for future assessment practices, but that such practices have not materialized beyond ‘what might be’.Keywords: Wiki, feedback, assessment, tasks, trajectories of participation.
Teacher: “…. in general there is no extensive space for a teacher in ICT [applications] in class, it is difficult to know what is the function of the teacher because everything is automatically moving, it becomes a separate world, it becomes so expansive. (…) When we have used wikis in projects they [learners] tend to disappear into their separate worlds and it becomes difficult for me to guide them and maintain my job as a knowledge provider (…) I don’t know what is the end product, what I am supposed to assess at the end, what I should assess, what a grade is applied to.” (9th November 2007)
This statement from a teacher serves as a vignette for the topic of this paper; the challenge of providing productive feedback when students are working on school tasks in networked learning environments, in this case a wiki environment. The teacher states that there is no space in ICT applications where she can enact her role as a knowledge provider. This lack of a defined space and the forever changing nature of the environment seem to give way to the feeling that the learners disappear into a separate world. This results in a challenge for the teacher; what becomes her function and how shall she perform this new role? We also see that questions concerning how and what to assess are closely connected to challenges that computer-based learning activities pose. Traditionally, assessment seeks to determine how well students respond to aims and targets in a given curriculum. The model underlying assessment is often referred to as the assessment triangle, where the three key elements are: observation, interpretation and cognition. Assessment can be described as the process of reasoning from evidence (Pellegrino, Chudowsky, Glaser, 2001). But the teachers’ monitoring, overview and follow-up of individual as well as collective contributions are disrupted when online computers become integrated in the learning environment. This implies disruption of the connection between feedback and learning. How can a teacher assess students’ learning and give feedback when she has no access to and cannot participate in the students’ activities?
These challenges have been a point of departure for the TWEAK research project (Tweaking Wikis for Education and Advancement of Knowledge)1 For more information, see: http://www.uv.uio.no/intermedia/english/research/projects/tweak/. The project had two interrelated goals; to address the role of the teacher as a most vital force in designing, supporting and assessing learning activities (Lund & Smørdal, 2006), and to match collectively oriented tools with tasks that require collaborative efforts (Lund & Rasmussen, 2008). In this paper we let the two topics converge, focusing on how the teacher’s concerns gave rise to the design of a wiki assessment tool that provides an overview of the students’ individual and collective activities and which included a space for the teacher for giving feedback to the students. We address the following empirical questions:
- How can we facilitate the possibilities for teachers to participate in students’ on-going, networked individual and collective knowledge production, through technological design?
- Did the design of the wiki assessment tool impact current practices?
We start by approaching challenges that digital and networked technologies pose for valid measurement and traditional school assessment and what computer-based feedback offer in terms of learner guidance and possibilities for teachers to participate in students’ ongoing networked individual and collective knowledge production. Next, we analyse the wiki tool (an activity map) that was designed, data generated by wiki use, and the teachers’ reflections upon computer use, assessment and feedback practises. In light of the long assessment tradition in schools, assessing skills and competences involving digital literacy and collaborative activities in Web 2.0 environments is still largely underdeveloped, it involves going beyond what we currently understand and practice. The design of the wiki assessment features in this study can therefore be described as a process in which the participants are trying to create something that ‘might be’ (Schofield, 1993), something which has yet to be created or, in activity theoretical terms, expanding the object by constructing “something that is not yet there” (Engeström, 1996:165).
Assessment, computer-based feedback and the teacher role
Digital technologies have challenged the historically stable relationship between teachers, textbooks, tasks and tests (Clarke-Midura & Dede, 2010; Lund & Rasmussen, 2008, 2010). Students today have access to multiple information resources and they often retrieve content from the Internet, which is not necessarily produced for schooling. While information in school textbooks is selected, controlled and “authorized”, information on the Internet is often unfinished, sometimes messy, produced by peers or persons unknown, and not always verified and proofread. With networked technologies and Web 2.0 applications we encounter a universe that is open, invites exploration, negotiation and large-scale collaboration. It appears fragmented and even unreliable. Therefore, using the Internet productively for learning requires additional skills and competences than those needed within the more traditional textbook dominated schooling. It requires a more complex selection and evaluation of sources, validation, relevance judgement and successful help seeking. For educational purposes, such competences cannot be fostered without systematic guidance, nor is it easy to assess the development of these kinds of skills and competences through traditionally individually oriented tests and summative assessment.
As a consequence, various types of formative assessment have emerged. Formative assessment can be explained as feedback for learning. It is assessment conducted to create feedback on performance to improve and accelerate learning (Sadler, 1998). An alternative distinction can be found between Static Assessment (SA) and Dynamic Assessment (DA) (Lantolf, 2006; Poehner, 2005). The concept of DA is closely linked to Vygotsky’s notion of the zone of proximal development; what learners can do today with support from peers and cultural tools they can later do on their own (Vygotsky, 1978; 1986). Consequently, DA will embrace development and potential instead of measuring a person’s competence at some point in time. This can be done in several ways. For example, a test followed by instruction targeting problems and potential revealed by the test, and which is again followed by a new test, can reveal to what extent learning has occurred. In a less instrumental and formalized setting we often use verbal cues, prompting and eliciting response beyond the answer (e.g. “How do you know this?”, “But what would happen if…”). At the heart of such an extension we find the principle of mediation; i.e. the teacher acts in a way or provides support that is conducive to learner development. Consequently, learners can concentrate on improvement strategies that take them beyond their current level of competence. In this way DA can also function as a predictor of learners’ needs. Of course, many teachers practice variations of DA (often without using the concept). However, such practices have not been sufficiently “institutionalized” in national assessment criteria or guidelines.
Several scholars have noted that it is difficult for teachers to provide formative assessment feedback because they face large numbers of students (Buchanan, 2000; Clarke, 2011). The feedback from teachers to a large number of learners might be eased by a division of labour between the teacher and the computer, i.e. the computer can adopt some aspects of the role of the teacher in providing feedback through, for example, drill and practice programs. Even though the computer may assist the teachers in providing subject feedback (Hattie & Timperley, 2007), what the students are doing on their computers – their ongoing activities – is still privatized and often hidden from the teacher. When learning with computers and in computer-based environments, activities are typically a hybrid of individual and collaborative activities (Stahl, 2006). Hence, giving a large number of students appropriate and precise feedback is challenging for the teacher (Clarke, 2011; Hattie & Timperley, 2007).
Often, teachers resort to their traditional offline way of working, leaving the online activities for learners to complete on their own (Lund & Smørdal, 2006; Trenting, 2008). Teaching and learning are then separated. Lack of teachers’ monitoring, overview and follow-up of individual as well as collective contributions when the students are online may be the consequence (Lund, Rasmussen and Smørdal, 2009; Lund & Smørdal, 2006). This is problematic since the teachers’ presence and participation is vital for learners to progress (Darling-Hammond, 1999, Krange, 2007). Simplified we might say that it is important that feedback is timely and that it contains information for the student so that she or he can improve performance, change strategies or reconsider her/his thinking about the problem to be encountered or solved.
In sum, the studies referred to in this section demonstrate that access to the students’ ongoing activities as well as awareness towards time constraints are of vital importance in the development and institutionalization of new assessment practices. We also find a range of design studies of web-based tools that provide automated feedback or that visualise participation (e.g. Trier, 2005; Templovs, 2008; Sins, Bauters & Damsa, 2008; de Jong & Jules, 2005). However, how teachers put them to use in their practices has not yet received much attention. The web-based formative assessment tools that exist do not seem to hold a space for the teacher to provide feedback. One way of addressing this issue is to re-examine the nature of feedback, and who provides it (e.g. teacher, peer, self, automated). We argue that there is a need for further technological development, but that such studies should also include analysis of how teachers cope with assessment of collaborative, online learning when the technological conditions are tailored for dynamic feedback and also afford a space for the teacher. In the following, we use participation as a metaphor and a principle for learning, combined with the notion of trajectories to describe the processes and results of having taken part in activities over time (Lave and Wenger, 1991; Rasmussen, 2012). The reason for using trajectories of participation as a metaphor in designing a new tool is that traces of different kinds of participation and collaboration can be displayed by the technology as paths or activity maps. These bring to attention the consequences and the traces of our actions.
Data and method
Our empirical study reports from research at two senior high schools in Norway from 2005 to 2009. Starting with wikis in the school subject English as a Second Language (ESL) at Western Senior High from 2005 till 2009 (Lund & Smørdal, 2006; Lund, 2008; Lund and Rasmussen, 2008), the experience and findings from this wiki research were brought into the TWEAK research project and modern history at Eastern Senior High (Lund, et.al., 2009; Lund and Rasmussen, 2010). The research concerns that transcend these two projects are in particular the effort and collaboration between teachers and researchers to better configure the inner consistency between educational tasks, collectively oriented tools such as wikis, and teacher support. A rich set of data was collected throughout the four-year period, including field notes and recordings from classroom activities, computer generated logs and recordings of our collaboration with the teachers (design meetings and semi-structured conversations). However, for the purpose of the present investigation most of the collected material functions as background information. The data that we base our analysis on is from November 2007 to May 2009, and consist of:
- Field work. One researcher observed modern history lessons from 22nd January to 3rd June 2008, one class and their teacher N=30. Type of data: field notes. Place: Eastern Senior High.
- Wiki use. Five of the observed history lessons during spring 2008 included the wiki. 2 researchers observed these lessons. Type of data: field notes. Place: Eastern Senior High.
- Talks with teachers. 9th November 2007: first TWEAK meeting. Transition from Western to Eastern Senior High. Participants: four teachers, two technology designers and three researchers (the authors of this article). Place: University of Oslo. 6th Mach 2008: starting with the wiki. Participants: three teachers from Eastern, researchers and technology developers. Place: University of Oslo. 23rd May 2008: end of first semester. Participants: two teachers and two researchers. Place: Eastern Senior High. 27th May 2009: project closure. Participants: two teachers and two researchers. Place: Eastern Senior High. All talks were audio-recorded.
- Wiki logs. Type of data: automatically generated logs of wiki activities. Duration: September 2008 - May 2009. Place: Eastern Senior High.
We started the TWEAK project in January 2008, building on our earlier investigations, design workshops, technology developments and field observations. In March the teachers started using the wiki in their everyday lesson activities. In the beginning the focus was mainly centred on developing tasks that matched the collective wiki tool (for analysis see: Lund, 2008; Lund and Rasmussen, 2008). Still, throughout the whole period and especially in conversations and meetings with the teachers the topic of evaluations surfaced. As a consequence, the wiki assessment tool was developed. This tool was in regular use from September 2008 until May 2009. Hence the log data from this period also include the use of the wiki assessment tool. It consists of logs documenting two teachers and their students’ activities from the use of the wiki during one school year (N= 62). We used the user IDs to distinguish between the teachers’ and the pupils’ contributions. The wiki stores the type of contributions, e.g. page creation, page edit, new comment and new labels (key words), and we could count the occurrences. During this period the researchers (authors) were not present. These data are presented in Table 1 and display the frequency of wiki activities.
|Table 1: Log data from wiki activities.|
Pupils N= 60
Teachers N= 2
Create new document
Write comment (i.e. discuss the content of the document)
618 total comments.
Create and apply label
312 unique labels created. 458 labels applied to 124 documents.
64 labels applied.
By combining observations and log data we can base our description on wiki use during nearly one and a half years. The patterns of use that we point to were quite stable over time and are also consistent with findings from other studies of wiki use (e.g. Trentin, 2008; Knain, 2009; Forte & Bruchman, 2006).
Throughout the TWEAK project we have employed exploratory and descriptive methods. The study rests on principles of design-based research where interventions are iterative, theory-informed and aim to capture the ecology of the learning situation (Brown, 1992). The study might be described as an empirical case of what Schofield calls ‘what might be’ and as an alternative to ‘what is’ (Schofield, 1993: 214). That is, one part of our analysis describes ‘what is’ while the other indicates ‘what may be’ or what has yet to be created. The ‘what is’ analysis is the design result, i.e. the wiki assessment tool, the everyday practices and the wiki use, based on the data described above. The analysis that indicates ‘what might be’, consists of audio recordings of reflections from the teachers in semi-structured conversations. In these conversations the teachers talk about their role and the practices that have yet to be created. We conducted four such semi-structured conversations (Kvale, 2010).
The present study is therefore based on data and methods for interpretations, which range from participant observation, logs of use and accounts of ‘what might be’. The latter type of data will be analysed after the presentation of the design result. We have selected utterances where the teachers explain their challenges and dilemmas concerning the use of networked computers, assessment and feedback. Seen from a sociocultural perspective, meanings and functions of tools and signs are constituted in social practices (Linell, 2009; Säljö, 2004; Wertsch, 1998). This implies seeing students’ encounters with networked computers as a particular practice embedded in an institutional setting with certain traditions of organizing teaching and learning. Thus, we are also interested in identifying institutional concerns in the teachers’ talk since such concerns are crucial for something new to take hold in schooling beyond the project period. All the excerpts have been translated from Norwegian into English by the authors.
Results #1: everyday practices (‘what is’)
We start with a broad report of how the lesson activities were organised at Eastern Senior High and how feedback and assessment practices were situated, the ‘what is’. We draw on our fieldwork, the log data and our conversations with the teachers. Our analytical attention is first directed towards portraying everyday practices in order to use these as points of reference when analysing the design and the use of the wiki assessment tool. First, one of the teachers describes how they usually organize their lessons:
Teacher 1: In my school we have always started with classroom instruction and then handed out tasks and guidelines to the pupils, and these have usually been produced by using Word. Quite traditional, I would think (March 6th 2008).
This teacher’s account of classroom instruction, followed by individual or peer group work, is confirmed by observations in our field work. Usually the teachers posted the tasks in the school’s learning management systems (LMS) and the pupils wrote their papers by using Word and returned them to dedicated folders in the LMS. With regard to feedback and assessment, the teachers’ whole class instructions were characterised by lectures with question and answer interactions. And typically during seat work, the teachers walked around class assisting pupils individually. As our teacher notes: quite traditional.
On several occasions we talked about the practice of providing individual feedback and the fact that it was very time consuming for the teachers, but that they still prioritized it. In the following quote one of the teachers explains some of the reasons for prioritizing individual student feedback.
Teacher 2: It is a way of keeping the day to day contact with the students ‒ how things are going – not only feedback on subject and task progress (6th Mach, 2008).
We see that the teacher emphasises the social aspect of feedback, “how things are going”. This way of scaffolding and following up students characterised all the teachers we observed in the TWEAK project.
However, and also according to the teachers, the challenges with this way of organisation were several. The teachers considered the quality of the students’ production as rather poor. Reproduction without reflection and copy-paste of content from the Internet dominated the student products. Students’ copying of texts is often framed as an evaluation problem, because access to Internet published texts and content makes it difficult to detect and judge the students’ own contribution. Research on students’ use of networked computers confirms that copying is widespread (Rasmussen, 2005; Hakkarainen et al., 2002). The teachers both at Western and at Eastern Senior High felt that they were challenged by the reproduction possibilities provided by networked computers. Consequently in the TWEAK project we focused on how tasks and feedback could prompt reflection and subject engagement. Vital to the design of the wiki assessment tool was therefore the work that was conducted to develop new tasks that involve digital literacy and collaborative activities suited for web 2.0 environments (Lund & Rasmussen, 2008, Lund & Rasmussen, 2009).
In the next utterances selected from a conversation about halfway into the project period (23rd May 2008), one teacher explains this shift in his task design and how this prompted collaboration and reflection between students.
Teacher 1: I have often divided my class into three groups and given three different tasks and then I have told them [learners] to answer one task assigned to them and then to comment on the answers that others have given (…). So, in that way the work is not only to answer a question handed out by me, but also to evaluate how other pupils have replied and to add critical input or complementary information to what others have written. And that is ehmm…actually a very new way of working compared to what I have done previously.
To divide the class, find answers to different questions and then to judge and consider the responses may not seem like a radical shift. Still, this shift implies a change in epistemological position; from individual response within a universe with “authorized” or “given” versions to a position of collective negotiation and relevance judgement. The teacher also states that it is “a very new way of working”. Both tasks and evaluation procedures are new. Students take part in evaluation by commenting on each other’s contributions; the task involves extending responses beyond individual “solutions” to a fundamentally collaborative activity, a culture of collective commentary. A teacher describes the gain of this shift in the following section:
Teacher 1: …, they problematize each other’s contributions in a way that I find to be of great value and in a way that they seldom do in relation to, for example, a textbook because they just accept that as an authority. But when it comes to commenting on each other, then there is more of an opening for them to correct something, or it is possible to be critical of what is written. In that way, their [learners’] comments seem to be more thought provoking than, say, for example, a textbook or even what I say…. because; far too often they just take that as given.
Here we see that the teacher acknowledges that it is difficult to get the students engaged in the problematization of content. Students are not necessarily used to tasks that require problematization. This can be interpreted in relation to schooling’s long historical tradition of matching tasks and assessment to measure specific content reproduction. However, according to the teacher, peer commentary seemed to open up for deeper and more independent reflections. Even though the students still reproduced content and still often copied, their activities expanded. As our log data display, labels and comments were frequently used. The wiki’s peer contribution feature seems to open a space where students can practice being critical; not just taking information on the Internet as given, but asking each other questions, discussing meaning and significance. As pointed out earlier in this paper information on the Internet that appears as topically relevant often requires problematization and a critical approach. Hence, to problematize text development and discuss peer contributions is one way to develop competencies that prepare learners for the challenges they meet when they use the Internet.
Alongside the development of new tasks and the experience of trying them in practice, the wiki assessment tool was designed. The teachers’ experiences were important for prioritizing features that provided an overview of the students’ individual and collective activities and the construction of a space for the teacher for giving feedback to the students.
Results #2: visualisation of trajectories of participation in a wiki
Turning now from existing practices to the design of the material tool, we asked how we through technological design can facilitate possibilities for teachers to participate in students’ ongoing networked individual and collective knowledge production. In the following we present the wiki assessment tool, the activity map, that was developed. The activity map is a representation of learners’ diverse contributions over time as well as a visualization of their collaborative activities. This tool was integrated in the wiki and provided teachers with an overview of the students’ on-going technology-based activities; the characteristics of the emerging work, who is engaged and in what ways, and means for providing feedback. In order to design a relevant visualization for the teachers, we had to balance the complexity of the students’ work relations and the essential information in each student’s individual trajectory. Three important characteristics will be highlighted and discussed: individual trajectories, participation trajectories and intervention trajectories. These trajectories correspond to requests from the teachers who wanted to be able to follow both individual and collective knowledge production. The visualisation represents the material relations between pages, comments and labels in the wiki (Hey, Linsey, Agogino, & Wood, 2008)
Individual trajectories visualize contributions from each individual student. The contributions correspond to events in the wiki, such as page creation, edits and labelling (i.e. adding key words to a page). Each student’s contribution has an icon representing the activity. In Figure 1, Johaina has 10 contributions; a page creation, an edit, another page creation, an edit, a label applied to a page, two edits, and finally three comments.
When moving the mouse cursor over the icons, the content is shown as well as how it relates to other contributions. This is a quick way to get an idea of the overall work of the student, without having to navigate to the various pages and comments in the wiki. The visualization corresponds to the teachers’ requests for an overview of each individual’s work.
Participation trajectories display several students’ trajectories and the relations between them: who initiated a page, who edited it, who commented on it, etc. In the wiki assessment tool these relations are drawn with lines between contributions and with different colours. Following the lines, the teacher can see who made edits to a page, who commented on it, and whether there were subsequent edits. The participation trajectories are complex and aim to address the need of the teachers to know what is going on in the class, such as who is working with who, the nature of their collaboration, the distribution of work in class and the uptake of the resources provided for them. Also issues about timing and self-regulation, such as ability to set deadlines, can be monitored. As such, the displayed participation trajectories highlight patterns of collaboration in the class, such as co-writing and peer commentary.
Intervention trajectories hold spaces for the teacher to guide the students in their work as it unfolds, that is: a tool for dynamic and formative interventions. The teacher can move the mouse cursor over the icons and inspect the content. In the case of page creation, the whole text is shown, in the case of edits, the increment of the text from one version to the next is displayed, and in the case of comments the full comment is shown. The content of these different contributions is shown in a floating window where there is also a space for the teacher to comment on the revision. The students will see the teacher’s comments along with other comments in the wiki.
We designed this floating window to ease the teacher’s work, as the comment or feedback can be made from one place without having to navigate to the various pages in the wiki, and provide her/his comments there.
The wiki assessment tool described above can be seen as a materialization of the theoretical concept of trajectories of participation that hold dynamic and formative qualities. Modern interactive media make visible what students are doing and, thus, afford documentation in the form of snapshots and even capturing longitudinal processes. This, in turn, can be used to understand what and how students are learning. The theoretical suggestion is that analysing such multi-dimensional data from different trajectories can provide new insights on learning.
Results #3: emerging practices (‘what might be’)
Students display considerable individual differences, and their way of taking part in the activities matters, whether they add a text, or whether they systematically revise, add comments and discuss with peer students. As displayed in Figure 3, the teacher can provide students with immediate feedback on how they are doing and how they can improve their work. Here the old assessment triangle is replaced by collaborative, mediated, scaffolded and data-generated event logs that visualize the students’ participation trajectories. Consequently, the concept of trajectories of participation is used both as a principle for learning and as a design metaphor for assessing collaborative and networked learning activities in the wiki.
So, did the design of the wiki assessment tool impact on current practices? As mentioned in the method section, we observed the everyday practices one semester (spring 2008) while at the same time we collaborated with the teachers in developing task designs and the technological design of the wiki. From the school year 2008 and 2009 our data consist of log files and audio recordings from two conversations with two teachers. In this last conversation the question of change in assessment and feedback practices was central. The excerpts in this section are from that meeting, which took place on 27th May 2009. We have selected three excerpts that illustrate and summarize the teachers’ experience and where they tell how they solved challenges. We start with an account where the challenge is articulated:
Teacher 2: ...a lot of computer use boils down to evaluation - the question: good mmmm - bad computer use - or banal computer use like behaviourist - with tick off questions and, like, multiple choice. We do that too, but it is very plain. Good computer use is often about prompting reflection and discussion and such. But then we face the challenge of the evaluation regime that is quite old fashioned, where we must document every single pupil’s performance. It is not easy to work with a wiki and evaluate one single pupil’s performance.
We see here that although the wiki activity map provided the teachers with an overview of the students’ activities and a space where they could provide feedback, their concern and frustration regarding the traditional evaluation regime, where they are required to mostly account for learners’ unassisted and individual learning efforts, still remained. According to teacher 2 (above) the teachers’ goal is to engage students in active participation and reflection upon the content and topics in question. Tasks and evaluation of responses are not separated, but seen as intimately connected. So, how did they solve this dilemma of being stuck in ‘what is’ while simultaneously articulating ‘what might be’? The following quote reflects a pragmatic approach.
Teacher 2: But back to what I just said - if you think of the wiki, not as something that needs to be assessed with grades, but more as a place where you practice, the wiki as a place where you practice getting better, then one may evaluate with regular written assignments or tests so then there will not be a conflict.
The teachers did not challenge nor did they change the existing assessment practices, but opted for a pragmatic solution, which nonetheless also included shifts in task design. What the teacher describes here is a modern version of traditional computer-based seat work. This practice is made more interactive by developing tasks that afford interactive and collectively oriented tools such as the wiki. Tasks and seat work were transformed, but the teachers stayed with the established assessment practices. The log data show that the production in the wiki was voluminous and that commenting and labelling became part of the learners’ educational activities. However, our data indicate that there were few examples of true co-writing.
If we look at how teachers participated in the wiki, the log data show that the teachers too used comments and labels. Both teachers’ and students’ comments were attached to texts posted by individual students. Labels were used to define documents.
Our last excerpt points to challenges in changing practices on an institutional level; new practices do not surface as merely stunts, but as potentially sustainable practices.
Teacher 1: ...it has been difficult to engage other teachers in a new way of thinking, but I do not want to go back to the traditional organisation where I have to correct the students’ answers.
Teacher1: /Yes then you can - you don’t need to respond to all and you can participate in and comment on (...)
Despite difficulties in engaging colleagues in changing their practices both teachers rule out the option of going back to the traditional organisation, i.e. correcting individual student answers. These student answers – produced in Word with extensive use of the Internet – are often voluminous and the texts are often samplings based on a range of copied texts (Lund & Rasmussen, 2008). Hence, the teachers did not rely on these types of student deliveries, they: “rarely corrected them”. Both teachers convey that it is much more productive to be selective – not responding in the same way to all students, but to take part and contribute with comments. This shift is connected to the creation of new tasks that match the tools in use, and to trying out new ways of participating, i.e. feedback through comments. The teachers are in a way facing the same challenge as their students; they are overwhelmed by the amount of content and feel the need to create strategies and competence in selection and relevance judgement.
Conclusion: Creating something that has yet to be created
The development of the Internet, Web 2.0 and digital infrastructures has been radical and rapid from a technological and societal perspective. In our review we point out that although there are some studies where new computer-based tools are developed to assist the teacher and to facilitate possibilities to participate and support students’ ongoing activities, the practices attached to the use of such tools have received little attention. Our study shows that when we include both technology design and the enacted practices we identity two different types of development, and although they are mutually constitutive they develop at a very different pace; the development of the material tool/product happens at a faster pace than situated human practices and especially the institutionalisation of these. New educational practices – with or without computer technology – do not change or develop overnight - but are created from circumstances that have previously been established, such as assessment and task practices. In our case we see complex relationships between specific subject and institutional practices and how these are challenged by web-based tools. The teachers’ repertoires evolved with their experiences of how to organise and take part in lesson activities that involved the wiki. In other words, the new practices arise as these relationships evolve. The tasks the teachers created were new in the sense that they demanded new types of participation and feedback. However, what counted as knowledge in the school subject of modern history was not challenged in our case. It was the ways in which students and teachers take part in, and engage with the subject content that was transformed.
The existing assessment regime remained in the midst of emerging collaborative and collective knowledge production. The teachers’ approach can be characterized as pragmatic, which nonetheless results in a germ cell that could possibly transform existing practices. New types of tasks for the individual students and for students collectively represent a crucial effort in the creation of new conditions for evaluation. As the teachers themselves point out, the challenge was to bring this to the institutional level. Hence, the changes that we identify from our longitudinal work can be characterized as incremental. Such findings are very much in line with research that has investigated the impact of implementing digital technologies in classrooms (e.g. Cuban, 2001; Kozma & McGhee, 2003). Processes often occur at the same time: traditions continue while simultaneously new elements are added. Traditional tasks and assessment will probably continue, while incrementally teachers’ experiences with new types of tasks that takes on the possibilities afforded by collective tools will develop a broader repertoire of feedback practices. We believe that this in turn will evolve our understanding of the connection between assessment and learning. It takes time for new teaching and learning practices to be adopted, and even longer before new institutional practices are formed and made robust and sustainable.
By the same token this study demonstrates how the development of new practices depends on both the appropriation of new tools, such as the wiki, and expansion of existing repertoires, i.e. new tasks and new ways of participating in educational activities. Or in other words: the how and why of teaching and learning activities are necessarily intertwined with the use of material tools. Simplified our analysis can be described as creating something (the material tool), which has yet to be created (the practices attached to the tool). The material incorporates and displays the ideas for future practice, and as such it contains the embryonic starting point of what ‘might be’. Since ideas of what ‘might be’ are often volatile and hence hard to hold on to, the material inscription of ideas (in our case in the wiki) might be valuable for designing future assessments. Reflecting on this four year long research and technology development project we might form the proposition that there is no change without materialization and no sustainability without institutionalization. The relationship between learning and tool use therefore needs to be considered and reframed continuously.
Acknowledgement: The National Network for IT-Research and Competence in Education (ITU)2Since 1 January 2010 ITU is a part of the National Centre for ICT in Education for funding the TWEAK project (2008-2009). Professor Per Hetland for inviting and organising this special issue, and the reviewers for helpful feedback. We also thank Professor Jaan Valsiner for an insightful discussion and valuable comments on our presentation of this work at ISCAR 2011. And most important we thank the teachers for taking part in this research.
Brown, A. L. (1992). Design Experiments: Theoretical and Methodological Challenges in Creating Complex Interventions in Classroom Settings. The Journal of the Learning Sciences, 2(2), 141-178.
Buchanan, T. (2000) The efficacy of a World Wide Web for formative assessment. Journal of Computer Assisted Learning, 16, 193-200.
Clarke-Midura, J., & Dede, C. (2010). Assessment, Technology, and Change. Journal of Research on Technology in Education, 42(3), 309-328.
Cuban, L. (2001). Oversold and underused. Computers in the classroom. Cambridge, Massachusetts: Harvard University Press.
Clarke, M. (2011). Promoting a culture of reflection in teacher education: the challenge of large lecture settings. Teacher Development, 15(4), 517–531.
Clarke-Midura, J., & Dede, C. (2010). Assessment, Technology, and Change. Journal of Research on Technology in Education, 42(3), 309-328.
Darling Hammond, L. (1999). Teacher Quality and the student achievement: A review of state policy evidence. Center for the Study of Teaching and Policy, University of Washington.
de Jong, T., & Jules, P. (2005). The design of powerful learning environments. In P. A. Alexander & P. H. Winne (Eds.), Handbook of Educational Psychology (second edition) London: LEA.
Engeström, Y. (1996). Non scolaesed vitae discimus. Toward overcoming the encapsulation of school learning. In H. Daniels (Ed.), An Introduction to Vygotsky. London and New York: Routledge.
Forte, A., & Bruckman, A. (2006). From Wikipedia to the Classroom: Exploring Online Publication and Learning. Paper presented at the International Conference of the Learning Sciences, Bloomington, IN.
Hattie, J. & Timperley, H. (2007). The Power of Feedback. Review of Educational Research. Vol. 77, No. 1, pp. 81-112.
Hey, J., Linsey, J., Agogino, A. M., & Wood, K. L. (2008). Analogies and Metaphors in Creative Design. International Journal of Engineering Education, 283-294.
Kozma, R.B., & McGhee, R. (2003). ICT and innovative classroom practice. In R.B. Kozma (Ed.), Technology, innovation, and educational change. A global perspective (pp. 43–80). Eugene, OR: International Society for Technology in Education
Knain, E. (2009) Et praksisbasert kategorisystem for vurdering av tekstutvikling i Wiki. Nordic Journal of Digital Literacy. 02/2009. ISSN Online: 1891-943X
Kvale, S. & Brinkmann, S. (2009). Det kvalitative forskningsintervju. Gyldendal Norsk Forlag AS.
Krange, I. (2007). Students' conceptual practices in science education - Productive disciplinary interactions in a participation trajectory. Cultural Studies of Science Education. ISSN 1871-1502. 2, s 171- 203 . doi: 10.1007/s11422-006-9040-y
Lave, J. & E. Wenger (1991). Situated learning: Legitimate peripheral participation. NY: Cambridge University Press.
Lantolf, J. P., & Thorne, S. L. (2006). Sociocultural Theory and the Genesis of Second Language Development. Oxford: Oxford University Press.
Linell, P. (2009). Rethinking Language, Mind and World Dialogically: Interactional and contextual theories of human sense-making. Charlotte, NC: Information Age Publishing.
Lund, A., & Rasmussen, I. (2008). The right tool for the wrong task? Match and mismatch between first and second stimulus in double stimulation. International Journal of Computer-Supported Collaborative Learning, 3(4), 25-51.
Lund, Andreas (2008). Wikis: a collective approach to language production. ReCALL. ISSN 0958-3440. 20(1), s 35-54
Lund, A., & Rasmussen, I. (2010). Tasks 2.0: Education Meets Social Computing and Mass Collaboration. In C. Crawford, D. A. Willis, R. Carlsen, I. Gibson, K. McFerrin, J. Price & R. Weber (Eds.), Proceedings of Society for Information Technology & Teacher Education International Conference 2010 (pp. 4058-4065). Chesapeake, VA: AACE.
Lund, A., Rasmussen, I., & Smørdal, O. (2009). Joint designs for working in wikis: a case of practicing across settings and modes of work. In H. Daniels, A. Edwards, Y. Engeström, T. Gallagher & S. Ludvigsen (Eds.), Activity theory in practice: promoting learning across boundaries and agencies (pp. 207-230). Oxon, UK and New York: Routledge.
Lund, A., & Smørdal, O. (2006). Is There a Space for the Teacher in a Wiki? Proceedings of the 2006 International Symposium on Wikis (WikiSym '06) (pp. 37-46). Odense, Denmark: ACM Press.
Pellegrino, J. W., Chudowsky, N., & Glaser, R. (2001). Knowing what students know: The science and design of educational assessment. Washington, DC: National Academy Press.
Poehner, M. E., & Lantolf, J. P. (2005). Dynamic assessment in the classroom. Language Teaching Research, 9(3), 233-265.
Rasmussen, I. (2012) Trajectories of participation: temporality and learning. In Norbert Steel (Ed.), Encyclopedia of the Sciences of Learning. Berlin Heidelberg: Springer Verlag.
Teplovs, C (2008). The Knowledge Space Visualizer: A tool for visualizing online discourse. Retrieved January 2012 from: http://chris.ikit.org/ksv2.pdf
Trentin, G. (2008) Using a wiki to evaluate individual contribution to a collaborative learning project. Journal of Computer Assisted Learning, 25(1), pp. 43-55.
Trier, M (2005). IT-Supported Visualization of Knowledge Community Structures. Proceedings of the 38th Hawaii International Conference on System Sciences. Retrieved January 2012: http://www.hswbasel.ch/iwi/publications.nsf/acf25f471c55d783c12572180036e1ab/9df5a2cb00e5062ac125722e002923f4/$FILE/HICCS38_Trier.pdf
Sadler, D. R. (1998). Formative assessment: revisiting the territory. Assessment in Education, 5(1), 77-84.
Schofield, J. W. (1993). Increasing the Generalizability of Qualitative Research. In M. Hammersley (Ed.), Social Research: Philosophy, Politics and Practice (pp. 200-225). London: SAGE.
Sins, P.H.M., Bauters, M. , Damsa, C.I. (2008) Knowledge Practices Environment: Design and Application of Technology for Trialogical Learning. In P. Dillenbourg & M. Specht (Eds.), Times of Convergence: Technologies Across Learning Contexts. Proceedings of the third European Conference on Technology-Enhanced Learning (ECTEL). Lecture Notes in Computer Science. Berlin Heidelberg: Springer Verlag.
Stahl, G. (2006). Group Cognition: Computer Support for Building Collaborative Knowledge. MIT Press.
Säljö, R. (2004). Lärande & kulturella redskap. Om lärprocesser och det kollektive minnet. Falun: Norstedts Akademiske Förlag.
Vygotsky, L. S. (1978). Mind in Society. London: Harvard University Press.
Vygotsky, L. S. (1986). Thought and Language. Cambridge: Massachusetts Institute of Technology. The MIT Press.
Wertsch, J. V. (1998). Mind as Action. New York: Oxford University Press.