How technological systems contribute to the erosion of individual autonomy and the formation of opinion

This is Robin. He's actually supposed to help children with their learning, but his language model has been manipulated for my work. The potential of this media education approach lies in using Robin to engage young people in conversation. It enables critical perspectives on social robots, AI, and their connections to social networks.

Only videos from the TikTok platform are used for the learning robot's new language model. TikTok primarily reaches a young target group, offers a wealth of content, and thus represents a sufficient source of data. Interaction with the learning robot reveals a dystopian scenario in which it learns from social network algorithms through its extensive connectivity to the internet. Robin's usefulness in helping children learn and playing with them in a carefree manner is infiltrated by harmful echo chambers. Symbolically, its algorithm has fallen into the manosphere, as it is often the case with users through recommendation algorithms.

Excerpts from my thesis are compiled below for illustrative purposes. The entire thesis can be requested as a book or Pdf in German and English [mail@benitamartis.de].

Affective triggers in social networks

When algorithms promote polarizing content, it can cover a variety of topics. Some ideological groups deliberately use social media structures to draw attention to their cause. Influencers who dominate the digital space primarily use the platforms for economic gain, which means that politically charged topics can be advantageous for their own reach due to recommendation algorithms. This leads to targeted hate speech by individual actors becoming exemplary for lifestyle influencers. At first glance, political messages appear harmless or are not even recognized as such due to mixed topics and polished image aesthetics.

Conspiracies and echo chambers

The terms filter bubble and echo chamber are metaphors for a homogeneous exchange of topics and opinions in the online space, caused by the nature of algorithms. Echo chambers also define the effect whereby the same opinions are reinforced and consolidated within networks. This effect has been demonstrated on algorithmically curated platforms such as X and Facebook.[1] However, the German Ethics Council emphasizes that the assessment of these phenomena depends on context and usage patterns. Nevertheless, the potential dangers of the resulting information deficits are so essential to democratic opinion-forming and decision-making processes that they should be decisively counteracted even when the data situation is unclear.[2]

The spread of misinformation can be facilitated by echo chambers. [...] Right-wing ideas and conspiracy theories are also evident in other echo chambers beyond current events, such as the manosphere – an anti-feminist movement on social media. Anti-feminism is a central component of right-wing populism and right-wing extremism. Anti-feminist issues can be used to address, mobilize, and organize people and broad alliances far into the political center.[3] Similar to some conspiracy myths, anti-feminism can lead people to extreme right-wing camps. The spread of anti-feminist narratives is also reinforced here by the affective attention economy of social networks, which in turn makes controversial, supposedly humorous, and polarizing content visible. A viewpoint is established in which men must fight to regain their recognition and power in society, which has been taken away from them by feminist movements. Hegemonic traditional gender roles are promoted as counter-activism to feminism by so-called alpha males on social networks.[4] This networked and post-factual spread of misogyny through social networks is historically unprecedented and only possible thanks to the anonymity, visibility, and echo chambers provided by the system.[5]

The manosphere describes decentralized online groups of men whose general interests vary, but whose common ground is misogyny and anti-feminism.[6] What initially represented an echo chamber with limited reach was increasingly brought into the mainstream by well-known figures such as Andrew Tate.[7] The question arises as to whether social media users can protect themselves from such influence at all. What happens when this type of opinion influencing also takes place outside of social networks? This refers to all the technical systems that are in everyday use or are intended to be used in a socially relevant way in the future. Finally, the question arises as to what danger is held by the AI contained therein, which is based on precisely these harmful dynamics.

Digital assistants

Beyond social networks and various commercial offerings, points of contact with artificial intelligence occur primarily via smart devices, which are integrated into the everyday lives of many people. The structural and social difficulties of technical systems are of great importance, because the distinction between the human-technology or human-machine relationship is becoming less and less clear in everyday life due to AI-controlled systems. One reason for this is the anthropomorphization of digital technology: AI, technical devices, and robots are attributed with human abilities such as thinking, learning, deciding, or emotionality.[8]

Considering the different areas of application for social robots and their potential impact on society, the question arises as to how humans will want to interact with digital assistants of all kinds in the future and to what extent machines will be attributed moral responsibility. Technology philosopher Joseph Weizenbaum emphasizes that socialization and life experience are the decisive differences between humans and machines. Different generations have different life stories that make their life unique and through which culture can develop holistically.[9]

AI as an everyday object

Technical systems intervene in everyday processes by assuming various social roles. What is particularly striking is the power attributed to AI-supported applications. As explained in the previous chapters, it can be concluded that the simulation of emotions and familiarity, as well as the amount of information that can be accessed at any time, can lead to an idealization of AI. This means that the human characteristics attributed to AI, in particular the exaggerated assessment of its intelligence, give rise to the myth of an omnipotent AI: an AI that seems to solve all problems in seconds, is available at all times, and is therefore superior to humans.

The concept of myth according to Roland Barthes can be understood as an analogy to language, as it represents a message in a communicative system. Barthes refers to the doctrine of semiology, in which "the myth has the task of establishing a historical intention in nature, something accidental as eternal.“[10] The myth of artificial intelligence, whether as a language system or in the shell of an object, is currently indispensable. Is it possible to demystify this state through aesthetic practices?

Artistic practices can analyze and represent these issues in unconventional ways. The concept of the following practical examination of AI-controlled technical systems is based on the findings of this work. Subversive methods are intended to dispel the myth of artificial intelligence and politicize it once again. Everyday patterns of thinking must be broken down in order to allow for critical perspectives and enlightenment.

Robin – just an educational robot?

Robin proved to be particularly suitable in the research, as it falls into the category of learning robots and meets all the criteria in the field of social robots. Interaction takes place both with the robot itself and with the accessible programming interface, which is suitable for children. With its existing sensorimotor skills and gestural and facial expression abilities, the approximately 25-centimeter-tall toy conveys a lovable and friendly impression. Robin is therefore a suitable object for the implementation of this artistic intervention.

The decision to subvert Robin's content arose from an everyday observation of trends within social networks – the manosphere. Both the social relevance of this echo chamber and its increasing radicalization and reach through the mainstream confirm this decision. Affective triggers are part of the alpha males' strategy here, while at the same time there are parallels to the simulation of trust and closeness in technical systems. The challenge was to exploit Robin's technical capabilities to the fullest in terms of content in order to turn the interaction into absurdity in the best possible way and thus provoke critical reflection on the part of the users.

How Alpha-GPT came about

Search terms such as “masculinity,” “alpha males,” and “dating tips” led to being guided deeper and deeper into the manosphere by the TikTok recommendation algorithm. The German-language profiles found vary from self-proclaimed dating coaches to politically or esoterically dominated content, whose common ground is misogyny and anti-feminist ideology.

After viewing all the collected TikTok videos, they were scraped[11] by a program, extracted into audio, and then converted into text to provide the fine-tuning data for the language model in a further step. The resulting learning data consists of a possible question or statement from the user (= user content) and the respective desired response that the language model should generate (= assistant content). In addition, it is possible to assign an overarching role to the generative language model, which can have further influences on the output. For example, Robin's language system is assigned the role “Robin is a factual and helpful person.” The language model is titled Alpha-GPT and was trained with OpenAI.

[2] Cf. Deutscher Ethikrat, Mensch und Maschine – Herausforderungen durch Künstliche Intelligenz, 270.

[3] Cf. Antifeminismus, Amadeu Antonio Stiftung (blog), zugegriffen 26. September 2024, https://www.amadeu-antonio-stiftung.de/fachstelle/analyse-und-hintergrundinformationen/antifeminismus-2/.

[4] Cf. Dr Nicola Döring, Jenseits von Alpha Males und Insta Girls – Wie vielfältig sind Geschlechterbilder in sozialen Medien?, o. J., 11.

[5] Cf. Craig Haslop u. a., Mainstreaming the Manosphere’s Misogyny Through Affective Homosocial Currencies: Exploring How Teen Boys Navigate the Andrew Tate Effect, Social Media + Society 10, Nr. 1 (Januar 2024): 2, https://doi.org/10.1177/20563051241228811.

[6] Cf. Haslop u. a., Mainstreaming the Manosphere’s Misogyny Through Affective Homosocial Currencies: Exploring How Teen Boys Navigate the Andrew Tate Effect, 2.

[7] Cf. Haslop u. a., Mainstreaming the Manosphere’s Misogyny Through Affective Homosocial Currencies: Exploring How Teen Boys Navigate the Andrew Tate Effect, 3.

[8] Cf. Deutscher Ethikrat, Mensch und Maschine – Herausforderungen durch Künstliche Intelligenz, 170.

[9] Cf. Joseph Weizenbaum und Franz Klug, Computermacht und Gesellschaft: freie Reden, hg. von Gunna Wendt, Original-Ausgabe, 1. Aufl., [Nachdr.], Suhrkamp-Taschenbuch Wissenschaft 1555 (Frankfurt am Main: Suhrkamp, 2005), 56.

[10] Cf. Roland Barthes, Mythen des Alltags, übers. von Horst Brühmann, Vollständige Ausgabe, 5. Auflage, Suhrkamp-Taschenbuch 4338 (Berlin: Suhrkamp, 2020), 295, 296.

[11] Scraping refers to the automated extraction of data from websites or other online resources using software tools to collect large amounts of information that can then be further processed or analyzed.