Anna De Liddo in January 2020. (Photo: Lars Nyre).

Scholars should act on the presumption that there are big problems in the public sphere, and that they require long-term research and innovation to be addressed properly. The dying out of local newspapers, the monolithic dominance of big technology companies, and the fake news permeating social media are just a few of the problems that undermine public debate. In particular, media scholars and information scientists must take centre stage in shaping the qualities of emerging media. We must do our best to improve the methods and technologies of public debate each time a new possibility arises. But how can we possibly achieve such an ambitious goal?

The following text is a dialogue between Anna De Liddo, Senior Research Fellow at the Knowledge Media Institute since 2008, and Lars Nyre, Professor of Media Design, Journalism and Technology at the University of Bergen, where he is affiliated with the Media City Bergen section. The dialogue centres around a design approach called collective intelligence and associates it with the ideal that John Dewey formulated for the public sphere in the 1920s. We believe this combination of influences points toward a good direction for technologically oriented media scholarship in the future.

Anna De Liddo is a Senior Research Fellow at the Knowledge Media Institute at the Open University, Milton Keynes, UK. She has worked there since 2008, and originally comes from Italy. De Liddo is educated as an urban planner and designer and has a master in Transport Planning from the Polytechnic University in Bari, and she gained a PhD in Urban and Environmental Planning from the same university in 2008, investigating ICT for Participatory Planning and Deliberation. De Liddo’s research focuses on the role of technology in scaffolding dialogue and argumentation in contested domains in which there is more than one point of view. She has a particular interest in the design and uptake of collective intelligence infrastructures. These are online public spheres and environments which seek to improve the collective awareness of and capacity to solve societal problems.

Lars Nyre is a Professor of Media Design, Journalism and Technology at the University of Bergen, where he is affiliated with the Media City Bergen section. He has worked there since 2008, and has a licentiate thesis (1997) and doctoral thesis (2003) from the University of Bergen, investigating the technologies of radio and music recording. Nyre’s research focuses on the role of new technology in stimulating sustainable local public spheres in Norway. He is particularly concerned with ways of engaging media students in collaboration with the local media industry to boost their innovation competence and ability to contribute to the quality of news and information in the public sphere.

Discovering a common concern

In January 2020, while it was still commonplace to travel across national borders, I visited the Knowledge Media Institute at the Open University in Milton Keynes, UK. By a lucky coincidence I was introduced to senior researcher Anna De Liddo, and we quickly realized we had overlapping interests. We share a concern with the problems of the democratic public, and we both want to design better solutions that might have an impact in the future. Both of us work with value-driven design and technology development, De Liddo focusing on democratic discussion (De Liddo et al., 2012; Schuler, De Liddo et al., 2018) and Nyre on media design for journalism (Nyre 2014; 2020). What follows is a written dialogue based on our initial oral conversation.

Wicked problems of the public

De Liddo: “Technologies based on collective intelligence are superior to those currently being used for public discussion, because they go beyond enabling conversations at scale and focus on improving the quality and means of public deliberation. The unfolding COVID-19 crisis has demonstrated more than ever the important role of technology in facilitating and shaping public behaviour. There is a lack of established practices to mobilize civic engagement around the pandemic, and the means to engage citizens into a proactive and democratic discussion on ongoing crisis responses at the policy level are scarce or non-existent.”

Nyre: “How come you are so concerned with issues related to design, discussion and democracy? These are not topics that are commonly associated with information science.”

De Liddo: “I started out as a civil engineer and worked with transport planning and urban planning. I was concerned with introducing participatory design into the decision-making process to allow citizens to influence the planning of new suburbs, roads and public transport systems. Urban planning inspired one of the most important presumptions in my work.”

Nyre: “I guess there was a bridge towards your interest in the democratic public here?”

De Liddo: “Yes, indeed. There are ‘tame problems’ where one particular solution is actually better than all the others. The belief that most planning challenges can be solved by one superior solution can be called first-generation problem-solving. Technocratic, hierarchical and bureaucratic decision-making traditionally presumes that problems are ‘tame’. Increasingly, however, researchers acknowledge that there are also ‘wicked problems’ where there is no best solution. Any solution will actually only solve part of the challenge and leave other parts of it unsolved. The realization that there is a wide variety of challenges, knowledge elements and interests that can only be partially satisfied has led to what can be called second-generation problem-solving methods. These originate from the seminal work by Rittel and Webber in the 1970s on wicked problems. They realized that wicked problems are complex societal challenges that need processes of dialogue and argumentation between a variety of stakeholders to be tackled, and the quality of these discussions is strongly related to the effectiveness, fairness and accountability of the decisions on how to address these problems. Designing technology-mediated solutions to support multiparty, multipurpose, multistake discussions is therefore as much a technical issue as it is a democratic issue. Complex problems need collective engagement in co-constructing solutions, or they are doomed to stay unsolved.”

Nyre: “Your argument reminds me of John Dewey’s democracy theory as formulated in The Public and its Problems (1927). Dewey is a pragmatist who considers it important for society to have a grassroots-based democracy where ordinary citizens inform the political system and its representatives about what to do. Furthermore, he is concerned that the public sphere consists of checks and balances that can keep politicians, capitalists and technocrats in check. Local journalism as well as public debate are important for him. It seems that your notion of collective intelligence design attempts to realize Dewey’s ideal in a digital and networked way.”

De Liddo: “This is an interesting association. I realized that collective intelligence technologies are needed to approach the best solutions for wicked problems. Collective intelligence suggests that you can make better decisions together than in isolation. Several people working together can solve more complex problems than individuals working alone and can also find solutions that are more satisfying to more people than individuals can on their own. This approach is highly consonant with what, in technology design research, is called co-creation or participatory design.”

Collective intelligence instead of crowdsourcing

Nyre: “What about the current ways of designing participatory technologies? There are social media with their different designs, and there are crowdsourcing technologies that are used effectively in a range of contexts.”

De Liddo: “Crowdsourcing is a fascinating phenomenon for good and bad. It works better than any other method to predict solutions to problems. Like swarm intelligence among bees or ants, it allows humans to make very good decisions by giving input to a computer system. The more people who contribute, the better the predictions and decisions become. This is empirically proven in a range of scientific studies. However, this technology receives input from individuals who are independent of each other and lack awareness of the reflections and decisions made by the other contributors. While the technology is advanced, it presumes that the problems and challenges to deal with are ‘tame’, and that they can be solved with first-generation hierarchical problem-solving. It is presumed that by stacking up the numbers, the leadership will find the best solution.”

Nyre: “In the 1920s there was a debate between Walter Lippman and John Dewey that has become famous among public-sphere theorists, and it comes to mind in your contrast between crowdsourcing and collective intelligence. Lippman and Dewey both had responsible and well-meaning arguments about how the public should be organized. As I have already suggested, Dewey promoted a grass-roots intelligence based on the voices of as many citizens as possible, while Lippman represented an expert regime based on a few well-educated representatives like journalists, academics and intellectuals. While Lippman was just as anti-authoritarian as Dewey, he wanted public debate to be characterized by a type of information-gathering and interpretation that was centralized. It seems to me that crowdsourcing is conductive to expert regimes, and that your position somehow resembles Dewey’s position on what an informed public should be.”

De Liddo: “Yes, while I acknowledge that crowdsourcing has its merits, it does indeed have serious shortcomings regarding its suitability for democratic purposes. Crowdsourcing at its worst can be exemplified with the Amazon Mechanical Turk where workers are exploited by not knowing exactly what they are contributing to. Even the workers in a big industrial plant can relate to their fellow workers, but the crowdsourcing structure of Amazon Mechanical Turk means every contributor is completely isolated from all the others. Crowdsourcing lacks transparency, it lacks accountability, and it lacks a socially sensitive contextualization during the decision-making process. Crowdsourcing is typically used in commercial contexts where this feature may be legitimate, but when it comes to decision-making in democratic contexts like the municipality, the local school, charities and other institutions that work for the common good, crowdsourcing is not the best technology. A majority decision isn’t necessarily fair or reasonable just because it got the most votes. Crowdsourcing allows for participation in decisions without any of the decision-makers having been exposed to the opinions of others, and so nobody has learnt something new or adjusted their opinion in the process. In democratic contexts it is absolutely vital to make sure that new technologies are transparent, accountable and allow for a collective, mutual awareness of the others among everybody involved. Democratic decision-making is a wicked problem that needs second-generation problem-solving.”

Designing improvements

Nyre: “It is always easier to criticize things that don’t work well than to formulate viable alternatives to them, and it is easier to make hypotheses about how things should be than to construct technological designs that actually work. What exactly are the principles and technologies that you are working with?!”

De Liddo: “In 2020 we possess very advanced technologies based on big data, artificial intelligence and complex network structures among people, but nevertheless crucial decision-making processes are conducted in ways that are hundreds of years old. Social media like Facebook organize all discussions so that opinions are presented in lists organized according to timelines. They are unstructured, confusing and often lead to atomized filter bubbles, hard frontlines, and trolling. So how can information scientists design conversational structures that allow for real diversity among people and also stimulate deliberation and better decision-making? This is the main research question for my work at the Knowledge Media Institute, and it is a big challenge. All my software is network-centric, with visualization of connections, that shows arguments in an intuitive way, so that it triggers reflection and assessments before people express their opinions. Collective intelligence designs have to be transparent, fair, accountable, and they must be able to handle wicked problems that may turn out to be unsolvable despite all efforts to solve them. Collective intelligence works much slower than crowdsourcing because there has to be deliberations between a large number of people for a range of separate or interconnected issues.”

Nyre: “The relevance of John Dewey’s public sphere theory gets clearer as you speak. He was disillusioned with the workings of the mass media of his time and stated that ‘the essential need is the improvement of the methods and conditions of debate, discussion and persuasion. That is the problem of the public’ (Dewey 1991: 208). It sounds like you are trying to make a technological design to achieve an improvement of the methods of public discussion, and that you are making what could be called ‘Dewey Devices’.”

De Liddo: “Indeed, Lars! I am designing technologies for transparent communication and dialogue. There is an inner goal to improve critical thinking and sensemaking among people who are atomized and detached from each other. They must learn how to challenge their own assumptions.”

The “Democratic Reflection” tool

Nyre: “Can you describe one of your designs for improved public debate?”

De Liddo: “Yes, with pleasure. The Democratic Reflection tool ( is an app for users to engage with video replay or live events, for instance televised election debates (see figure 1).

Figure 1:

The “Democratic Reflection” tool was used during the 2017 General Elections Debate hosted by ITV on 31 May.

“It consists of a grid of coloured flashcards each representing a statement that the viewer would like to express during the debate. These cards can capture a variety of psychometrics such as emotions, trust, empowerment, open questions etc. For instance, a card can say: ‘This is really boring, I would leave the room if I could’, or ‘I love this!’, or it can capture more subtle trust statements such as ‘Even if what he says is true I do not trust him’. Viewers can click instantly on the statements while watching the debate and target specific speakers. As figure 1 shows, cards are organized by a positive to negative polarity (left to right), and clustered around four target dimensions presented to citizens: how politicians speak to you (red), your capability to make sense of the debate (blue), felt personal relevance (green), and felt empowerment (purple). The app is able to record thousands of feedback cards live and automatically aggregates and analyses them to produce personal learning analytics that the users can then explore at the end of the debate (see figure 2).

Figure 2:

Example of personal analytics shown to individual users at the end of the debate.

The analytics provided by the app help unveil ‘the long tail’ of the viewing experience and are designed to spark critical thinking and surprise. Whereas normal social media would show a summary of the most frequent responses, we’d rather present the users with automatic reports such as ‘Despite you mostly reacted emotionally in a positive way toward this speaker there were five instances in which you disagreed with what he said. Would you like to re-play these moments?’

The tool promotes critical thinking by supporting the re-examining, questioning and reflection on our own assumptions, of which we may not always be aware. We tested Democratic Reflections with thousands of users in the 2015, 2017 and 2019 general election debates (De Liddo et al., 2020), and we systematically found that the tool promoted people’s capability to challenge assumptions they had before the televised debate. It helped them to better understand and even change their political position, and last but not least, it improved people’s willingness to engage with political debates in the future. In the current climate of political distrust and disengagement, these findings give us hope. People can, in the right circumstances and with the right technological support, think critically, change their mind, proactively question their assumptions, and re-engage with politics in a healthier way.”

Thinking better with the help of technology

Lars: “Are you sure this type of design can influence people to actually become more responsible in their day-to-day deliberations? Research in cognitive psychology suggests that people are instinct-driven and will follow a set path for their political behaviour based on deep-seated emotions. Are you naive in your attempt to change them?”

De Liddo: “Ha-ha! Colleagues sometimes say I’m naive. I should realize that people don’t change their minds very easily, they say. But people are much cleverer than what they presume. They can choose differently if they are stimulated differently, for example if they are not exposed to balkanization, a rhetoric of fear, or reminded of safety concerns instead of being stimulated in more constructive ways. Citizens need a ‘safe space’ where they can be in a position to change their minds according to the ideas of others. Many political situations in recent years show that people tend to have opinions in the moment that might show their authentic position on the topic, but later on they inauthentically change their opinion. The televised debate between Boris Johnson and Jeremy Corbyn showed that a majority of undecided voters found Corbyn to be more rational and sensible (Della Giusta et al., 2019), but Boris Johnson nevertheless won at the polling station. If we look at “Democratic Reflection” with Daniel Kahneman’s concepts of thinking fast and slow (Kahneman, 2011), the tool engages both system 1 (thinking fast, intuitively and in the moment) and system 2 (thinking slow by focusing, deliberating and reasoning) at the same time. The cards enable users to think intuitively in the moment, while the reflective analytics enhance focus and analysis thus promoting thinking ‘slow’.

We argue that it is important to provide such internal space for deliberation, and not just a public arena. The individual citizen needs to be able to move back and forth between ‘deliberation between’ and ‘deliberation within’ (Goodin 2000), from thinking fast and slow, to be sure that their opinion is actually authentic. There is often a mismatch between the feeling in the moment and the reflection afterwards, and often the immediate feeling is your genuine opinion. The reflection afterwards can be influenced by social pressure and logical thinking that alters your position, and this leads to decision-making that fails to be true to yourself and your inner bias. This is in most cases a desirable outcome that can be achieved only if we are consciously confronted with those biases and assumptions. To achieve this we must challenge ourselves to slow down and improve our critical thinking. Designs for collective intelligence are a blessing and a curse. Clearly, slowing down requires plenty of time, while at the same time people typically are very busy and feel the need to speed up. Quality decisions require thorough consideration, at both individual and collective level. This requires time to think individually and talk collectively.

Ultimately, the question we are asking is: Given the right socio-political contexts and processes for public deliberation, how can we act more intelligently, democratically and fairly as a society? And what technologies can help design and improve, rather than hinder these processes?”

Nyre: “This is indeed a wicked problem! And John Dewey formulated it in a quite similar way in 1927: ‘The essential need is the improvement of the methods and conditions of debate, discussion and persuasion. That is the problem of the public’ (Dewey 1991: 208). The fact that you and your colleagues are working on it in the 2020s certainly suggests how difficult it is to solve.”


De Liddo, A., Sándor, Á. and Shum, S.B. (2012). Contested collective intelligence: Rationale, technologies, and a human-machine annotation study. Computer Supported Cooperative Work (CSCW), 21(4–5), pp. 417–448.

De Liddo, A., Plüss, B. and Ardito, A., 2020, October. Democratic Reflection: Nudging Citizens? Democratic Engagement with Political Election Debates. In Conference Companion Publication of the 2020 on Computer Supported Cooperative Work and Social Computing (pp. 25–29).

Della Giusta, M., Jaworska, S., Vukadinović, D. and De Liddo, A. (2019) What is going on with economic expertise? LSE Blog, December 12th, 2019.

Dewey, J. [1927] Kahneman, Daniel (2011) Thinking, Fast and Slow. Macmillan.

Goodin, R.E., 2000. Democratic deliberation within. Philosophy & Public Affairs, 29(1), pp.81–109.

Lippman, W. [1922] (1997) Public Opinion. Free Press.

Nyre, Lars (2020) Tre spørsmål om metodeutvikling i medievitskapen. Norsk Medietidsskrift nr. 1/2020.

Nyre, Lars (2014) “Media design method. Combining media studies with design science to make new media”. Inaugural issue of Journal of Media Innovations. Vol 1(1).

Rittel, H.W. and Webber, M.M. (1973) Dilemmas in a general theory of planning. Policy Sciences, 4(2), pp. 155–169.

Schuler, D., De Liddo, A., Smith, J. and De Cindio, F. (2018) Collective intelligence for the common good: cultivating the seeds for an international collaborative enterprise”. AI & Society 33:1–13.