Multimodality in Epistemological Foreign Language Research? Two Case Studies in the Field of German as a Foreign and Second Language

: This article aims to elaborate on the epistemological interest of cultural studies for German as a foreign and second language and, simultaneously, to point out the limitations of the current focus on certain modes. We intend to broaden this focus by highlighting the connectivity of the epistemological interest to multimodal research using two relevant case studies. The ﬁ rst case considers music videos and their associated comments on YouTube, examining how patterns of freedom are negotiated. The second considers textbooks for the so-called cultural orientation of immigrants to Germany and examines one of the ﬁ rst textbooks developed for this purpose through the lens of meaning-making of the Holocaust. Both case studies were selected on the basis of existing doctoral projects of the authors. The analyses follow Bateman, Wildfeuer, and Hiippala, but take into account the extensive history of multimodal studies. The case studies exemplify that restricting our study to only one mode would have undermined the results.


Introduction
Although multimodality is a familiar term in the fi eld of German as a foreign and second language (GFSL), interest has been limited largely to questions of didactics, revolving around the use of multimodal materials for teaching or aiming to enhance learners' multimodal literacy or multiliteracy. This also applies to the related fi eld of Anglo-American German studies, since multimodal research is among others borrowed from linguistics. However, in the case of GFSL, a further reason for this limitation is that the fi eld is still struggling to promote its epistemological research profi le, and GFSL continues to be considered a "child of practice" (Weinrich 1 qtd. in Altmayer, "Kind der Praxis" 921, own translation) to this day. Thus, aspects of multimodal learning aside, multimodality itself comprises a valid subject for research. Epistemological research should do more than simply applying scientifi c results of a problem posed out of thin air-it should be driven by real-life problems to then consolidate practical action in classrooms (Altmayer,"Wissenschaft und Praxis" 82).
This paper introduces some major cultural studies concepts in GFSL and links them with the promising multimodal approach recommended by Bateman, Wildfeuer, and Hiippala. 1 Two case studies are then briefl y presented to exemplify the possible diverging methodological implications.

Concepts in GFSL cultural studies
One of the dominant topics in cultural studies for GFSL in the last 20 years has been the dichotomous distinction between supposed own and foreign cultures and the associated homogenizing and essentializing understanding of culture. Criticism of this understanding of culture has been a driving force for the move from an intercultural to a broader cultural studies approach, also known as the discursive approach. Despite the diff erent concepts of culture challenging intercultural approaches, culture is often still understood as a delimitable system. This is especially the case in textbooks, which persistently promote a complexity-reduced understanding of culture. Like Altmayer (e.g., "Konzepte von Kultur" 1407-09, "Von der Landeskunde zur Kulturwissenschaft" 17-20, "Landeskunde im Globalisierungskontext" 11-12), we place a meaning-and knowledge-oriented concept of culture at the centre of modern cultural studies for GSFL. Following Altmayer,Kasper,and Wolbergs (34), who also refer to Reckwitz (84) and Altmayer ("Konzepte von Kultur" 1407-09, "Landeskunde im Globalisierungskontext" 12, "Von der Landeskunde zur , we conclude that culture should be understood as follows: Not as common behavioural habits or patterns of thinking and perception of people on a national-ethnic level, but as symbolic orders or repertoires of meaning-making on a level of language and discourses that we access for (interactive) meaning-making to enable us to perceive "reality"-things, situations, or actions-as meaningful. (own translation) The term patterns of meaning was deemed appropriate for understanding culture in the fi eld of language acquisition-a term that refers to the individual parts of the (assumed) shared knowledge that we apply to every action in discourse when making meaning of a situation. The term presumes this knowledge to be "taken for granted", but provides a means to analyze already given propositions of meaning in certain situations and, in turn, derive orientations for action ( Altmayer,Mitreden 9,own translation). When acquiring objective discourse literacy, learners can activate, refl ect on, and revise their existing patterns of meaning but also confi rm, reject, or modify them. Learners must expand their knowledge and ability to function in a foreign language to facilitate their participation and engagement in their communities of interest. This requires them to understand and actively participate in meaning-making processes. Besides concrete possibilities for their implementation in foreign language teaching, patterns of meaning must be researched to provide the basis for an appropriate discussion in the discipline. A method for researching patterns of meaning was presented by Altmayer ("Kulturwissenschaftliche Diskursanalyse" 578-83) and, in a revised form, by Altmayer,Kasper,. A patterns-of-meaning analysis fi rst identifi es a pattern and then analyzes its representation and use in a specifi ed corpus, but such an analysis continues to be primarily text-based or image-based, which is inadequate, in our opinion. The two case studies discussed in this paper exemplify multimodal artefacts-textbooks and music videos on social media-and we consider their inherent properties. Hence, we present the core aspects of multimodal studies in the next section and combine them with the premises of GFSL in the subsequent section.

Interest of GFSL cultural studies in multimodality
As many multimodal studies and discourse analyses have theorized and practically shown, multimodal research is-due to its inherent nature-interdisciplinary and vast in scope (e.g., Wildfeuer, Bateman, and Hiippala; Ledin and Machin, "Doing Critical Discourse Studies"; Jewitt; Jewitt, Bezemer, and O'Halloran; Klug and Stöckl; van Leeuwen; Djonov and Zhao; O'Halloran; Machin; Machin and Mayr; Kress; Kress and van Leeuwen, Multimodal Discourse, Reading Images). Hereafter, the question of why multimodality should gain more consideration in cultural studies GFSL is addressed.
Multimodality's core concept of meaningfully combining diff erent modes of expression is already familiar, as Bateman,Wildfeuer,and Hiippala (71) stated, but it has only recently become an object of interest in GFSL, as indicated by contributions at the most recent 2022 IDT (Internationale Tagung der Deutschlehrerinnen und Deutschlehrer) conference. Research relating to this subject has thus far been limited to visual artefacts and visual literacy in classroom teaching (e.g., Hallet, "Visual Culture"; Kasper, "Manga"), only hinting at "multimodal literacy" (Hallet, "Viewing Cultures" 46, own translation).
Ever since the manifesto A Pedagogy of Multiliteracies: Designing Social Futures was published by the New London Group, new literacies-including multimodal literacy-have become the subject of academic discussion regarding didactics in foreign language teaching (e.g., Rowsell and Pahl; Gerlach; Warner and Dupuy; Ryshina-Pankova; Unsworth), but only relating to curricula ( Wildfeuer,Bateman,and Hiippala 16). Due to the academic background of the New London Group-many of them are linguists and educators-and groundbreaking papers from individual members, their work unsurprisingly draws heavily on linguistics, particularly Halliday's systemic-functional linguistics.
Many multimodal researchers, such as O'Halloran and Mayr, have adopted this approach, but it poses problems for the integration in GFSL cultural studies, as will be discussed later in this article. Another reason for the absence of multimodal research on cultural studies in GFSL is that it still relies heavily on German-speaking multimodality scholarship, which remains rather scarce, according to Wildfeuer, Bateman,and Hiippala (14). Moreover, epistemological research on GFSL cultural studies primarily deals with plain language in written texts, although the move from solely visual or textual to multimodal artefacts is a logical development since the technological aff ordances of the internet have facilitated new means of communication (e.g., Fraas, Meier, and Pentzold).
Discourse increasingly depends on the use and consumption of audio and video clips on social media rather than on plain language in texts (e.g., ARD/ ZDF-Forschungskommission). Thus, it is hardly surprising that research has taken a multimodal turn recently (e.g., Bateman,Wildfeuer,and Hiippala 9;Jewitt). This communicative change has driven interest in addressing the lack of multimodal research in GFSL cultural studies. Since new digital forms of communication have arisen on an almost daily basis, fuelled by the COVID-19 pandemic, it seems natural to incorporate multimodal approaches into research on communication in GFSL.
Many introductions, defi nitions, approaches, concepts, and methods in multimodal studies have dealt with diff erent kinds of multimodal artefacts, but space limitations prevent us from discussing and comparing them in this paper. Thus, we have chosen to favour the approach recommended by Bateman, Wildfeuer, and Hiipala. Our objective is to selectively point out essential epistemological links and conceptual overlaps between the model by Bateman, Wildfeuer, and Hiippala and the previously mentioned positions in GFSL cultural studies.

Epistemological common ground
Cultural research in GFSL has discussed multimodality largely in relation to linguistics. Arising from a diff erent epistemological interest, classic ideas of multimodality descending from de Saussure diff er greatly; therefore, clear distinctions must be made, but this does not mean that researchers must discard the tools developed for those ideas. However, Bateman, Wildfeuer, and Hiippala (among others 63-64) refer to Peirce instead of de Saussure to lay their foundation of semiotics, taking a diff erent turn on the subject of multimodality, which seems more adaptable for multimodal research in GFSL cultural studies. 2 Furthermore, besides the great need for a truly interdisciplinary approach (see Bateman,Wildfeuer,and Hiippala 72), three aspects of their work are particularly interesting regarding GFSL cultural studies.
First, there is common ground between material semiotic regularities and patterns of meaning. The compatibility of the concepts of Bateman, Wildfeuer, and Hiippala and GFSL cultural studies is evidenced by the understanding of knowledge as (assumed) shared knowledge that helps us ascribe meanings to things, situations, and actions. Bateman, Wildfeuer, and Hiippala claim that "knowledge of distinguishable material regularities is shared across some collections of individuals. That is, a community of receivers knows how to interpret the material regularities found" (84). This is compatible with the previously described understanding of patterns of meaning as (assumed) shared knowledge, which underpins discursive adherence and orientation for action in the process of meaning-making (Altmayer,Mitreden 9). In other words, patterns of meaning and material regularities require a community to share knowledge for meaningful-and, in turn, meaning-making-communication to occur. Obviously, there may be imbalances in the allocations of knowledge, as is always the case with social knowledge, and these relate to the allocation and exercise of power within society, as Kress emphasized. This shared understanding of knowledge provides a unifying foundation for the following two aspects.
Second, as a subject that deals with language and language teaching, GFSL discusses texts as research objects. Regarding the term text, GFSL mostly falls back on broad concepts. For example, Hallet ("Umgang mit Texten" 137-38) stated that it is no longer only the closed written material that is considered text, but rather text is a concept that also includes open, multimodal forms of multimedia hyper-text. In this sense, text is understood as a full range of modal occurrences. Altmayer (Kultur als Hypertext 165) clearly showed that in GFSL cultural studies, it is not crucial to distinguish text from non-text according to fi xed criteria. He stated: Following various linguistic, communication, and media studies concepts, we assume in the following that we are essentially dealing with "texts" as symbolic actions that are embedded in a larger social-communicative framework of action and that can also only be meaningfully analyzed against the background of this framework. (Altmayer,Kultur als Hypertext 171,own translation) Although GFSL cultural studies generally conceptualize text broadly, the focus of discourse analysis is mostly on written and spoken language, sometimes including images Altmayer,Kasper, or video. This diff ers from Bateman, Wildfeuer, and Hiippala's (101-02) holistic understanding of objects as subjects of research. However, this should not prevent us from introducing the concept in a viable way into GFSL cultural studies. We see possibilities for connection in the broad concept of text. Instead of analyzing individual modes-for example, written text, images-separately, as has been the case thus far, we consider the modes as an integrated whole. Although we are interested in language and discourse resources manifested in language, discourse literacy can only be achieved if all meaning-bearing modes are taken into account.
Third, part of the shift in GFSL cultural studies has been a critical refl ection on the intercultural paradigm. Previously, the idea of culture was tied solely to nations, which has never been accurate, but certainly not in a globalized world. Globalization has had an impact on interpretative communities, showing that patterns of meaning are closely tied to language as a carrier of culture. While some of these patterns and their inherent variants of meaning exist in other languages, the aim of GFSL cultural studies is to provide foreign language learners with the discourse literacy necessary for decoding these patterns in the target language. The idea of community can be meaningfully linked to Kress and van Leeuwen's (Multimodal Discourse 8) assertion that "communication depends on some 'interpretative community' having decided that some aspect of the world has been articulated in order to be interpreted". Bateman, Wildfeuer, and Hiippala stated: The fact that we need to have certain knowledge about what material regularities are relevant to begin talking of "communicative signs" at all leads us on immediately to a further component that is critical for actually having a communicative situation. (84) The knowledge for decoding these communicative situations-as Bateman, Wildfeuer, and Hiippala (84) saw it-is probably unequally distributed. This can be usefully linked to possibilities for discourse participation, and to general questions of power.
Overall, it is, therefore, possible to use multimodal analyses to answer questions regarding research on cultural studies for GFSL. We will share two case studies in the following sections.

Practical application to research: Two case studies
In the fi rst section, we discussed why multimodality is (or should be) of interest to GFSL cultural studies. We then linked our research interests to the epistemological approach of Bateman,Wildfeuer,, revealing common ground between concepts. Now, we shall apply the concept of canvas to two case studies. Basically, a canvas should be understood as anything where we can inscribe material regularities that may then be perceived and taken up in interpretation, regardless of whether actual, virtual (digital), simply produced, performed physically in time, or the result of a complex technological process. This places minimal demands on the materialities of their adopted sign vehicles-almost anything that can carry some material regularities that suggest intentionality and order might serve. ( Bateman,Wildfeuer, In other words, any artefact that carries various regular semiotic modes provides meaning, which, in turn, is subject to potential interpretation and negotiation within a community that shares common knowledge about sign-making, on the one hand, and (intended) meaning-making, on the other. This is exemplifi ed by the following two case studies.

Case study I: Freiheit on YouTube
The fi rst case study considers the meaning-making of Freiheit (freedom/liberty) in the music video "Freiheit" by rap artist Curse on YouTube. This extends an earlier study that verbally analyzed the well-known namesake song "Freiheit" by Marius Müller-Westernhagen, originally released in 1987, which later became a so-called Wiedervereinigungssong (reunifi cation song) about the 1990 German Reunifi cation, and its associated YouTube comments ( Kasper,. Since the 2008 version by Curse features the original artist Marius Müller-Westernhagen, intertextuality is a critical research consideration. Following the canvas approach, the YouTube music video is seen as a multimodal artefact that constitutes a canvas, meaning that the video, music, lyrics, comments, usernames, dates, and other features of the website are subjected to a single analysis. However, a canvas may contain several sub-canvases (see Bateman,Wildfeuer,and Hiippala 214), and in each, diff erent semiotic modes require diff erent (interdisciplinary) methods of analysis. Ultimately, it depends on the research question whether research focuses on sub-canvases or certain aspects of the artefact. In this case, the research concerns the multimodal meaning-making of the patterns of meaning in the "Freiheit" music video by Curse and the associated comments.
Visual representation of notions of Freiheit was identifi ed by analyzing the video, following the basic principles of fi lmmaking introduced by Bateman, Wildfeuer, and Hiippala (see also Bateman, "Critical Discourse Analysis"; Bateman and Schmidt) to analyze the fi lm Lola rennt. These were combined with a MAXQDA-software for qualitive data and text analysis-coding analysis of the language used in the song's lyrics and the audience's comments. In the multimodal research literature, videos are often analyzed using transcription and/or annotation of stills with timestamps in spreadsheets (e.g., Baldry and Thibault) or with annotation software such as ELAN or ANVIL (e.g., Bateman, "Multimodal Corpora", or O'Halloran et al.). However, MAXQDA facilitated more effi cient handling with the research material through its integrated YouTube tool, which allowed us to cross-annotate lyrics (in written form), comments, and video clips simultaneously. Figures 1-8 show some screenshots from the music video and selected YouTube comments.        We analyzed movements, gestures, outfi ts, colours, camera angles, sound effects, and other modes, which revealed how Freiheit was constructed by multimodal means.
The comments showed that the video stimulated discourse over time. Long after the release date, comments on contemporary discourse (e.g., regarding the COVID-19 pandemic and the so-called Querdenker movement, a group including pandemic sceptics, anti-vaxxers and anti-lockdown protestors) were added. Also, the audience took the artist's attempts to concretely defi ne the meaning of Freiheit as an invitation to add their own defi nitions. Furthermore, historical references regarding the propagandistic instrumentalization of the 1936 Olympics under Nazi rule (exemplifi ed by shots of Curse behind bars in the empty Berliner Olympiastadion) or the Fall of the Berlin Wall were included in the video. These (re)constructions of knowledge through the representation of personal experiences and emotions showed that Freiheit carries commonly shared emotional patterns of meaning.
It is striking that the visual features, such as security cameras or animal allegories, are not refl ected in the lyrics but make other references to public security issues as a threat to freedom or to linguistic idioms (being trapped like a rat, free as a bird, etc.). Colour codes emphasize the importance of freedom through the word Freiheit in red (or sometimes white) and in more or less contrastive three-dimensional letters in the room, while everything else is black and white. This simple colouring, besides the empty stage in most scenes, may represent the abstract nature of freedom, literally leaving room for interpretation. Stereotypical rap gestures accompanied by a heavy beat support the notion of freedom as a fi ght, and intimate, humble postures accompanied by smooth tunes highlight the notion of personal freedom refl ected in the lyrics.
The featuring of the artist Westernhagen clearly links the video to his wellknown classic song from 20 years earlier. Note that Curse does not wear his sunglasses as a gesture of respect when facing Westernhagen (whose artistic contributions marked an important turning point in history 3 ) in a seated dialogue (maybe symbolizing a heart-to-heart intergenerational exchange about a collective past), whereas wearing them might symbolize freedom as the last refuge of privacy (although note that Westernhagen wears sunglasses in his Christ pose). The Christ pose is also a reference to the earlier version of the song, reminding the audience of the actual, more popular 1989 live version. In fact, this pose commonly represents freedom, as simple image queries on Google or Getty Images confi rm. Westernhagen also sings the chorus (or hook) in the song, which is exactly the same as the 1987 chorus but is presented in a telephonic voice, suggesting a direct phone call with the past.
In terms of intertextuality, the lyrics include even more references (e.g., to the well-known folk song "Über den Wolken" by Reinhard Mey), highlighting the human assumption that freedom is without boundaries, as symbolized by fl ight (see LyricsTranslate for the full text). Other notions of freedom portrayed in the lyrics include global, political, and economic confl icts, as well as questions of human rights, all of which were mentioned in the YouTube comments to some extent.
A deeper look at the comments reveals notions of Freiheit as feeling or emotion-the felt absence of freedom-in the patterns of meaning. This corresponds well to the overall message of the song that "freedom is to be exhaled" (Curse, own translation), symbolizing both its essentialness to human life and the feeling of freedom embodied in one of the most natural physical functions of the human body. This message was the main reason for Westernhagen's decision to collaborate with Curse (pielo84). Interestingly, another 14 years later, Westernhagen also commented on the absence of freedom, claiming that freedom is an "illusion" (Rüth) in an interview about his song "Zeitgeist", in which he refers to several global crises, including the war in Ukraine and social disarray in general. 4 The application of an open, holistic, and interdisciplinary multimodal canvas approach revealed many aspects of the patterns of meaning of Freiheit that would probably have remained hidden if we had focused only on selected modes. Also, the entanglement of two sub-canvases as diff erent spaces of social and multimodal interaction provided insights not only into meaning-making itself, but also into how meaning-making as a social process takes place in a multimodal scenario. This led to the following observations: -Visual input not only supports, but also enhances meaning-making in lyrics.
-Comments elaborate on and add meaning, triggering discussions.
-The international audience interacts, contributes, and occasionally explicitly expresses interest in learning German! -The research revealed valuable, authentic cultural insights that are accessible to all language learners around the world, providing a foundation for research on cultural learning.
-A holistic multimodal perspective was vital since partial analysis would have limited the meaning-making process through which discourse unfolds.
Further research could include a more detailed critical analysis of, for example, historical intertextuality and argumentative strategies in discourse (e.g., Reisigl and Wodak) given the dialogic nature of YouTube comments, or multimodal metaphors (e.g., Forceville; Spieß) given the poetic nature of music (Falkenhagen and Volkmann).

Case study II: Holocaust in textbooks
In the second case study, we examined meaning-making regarding the Holocaust as a pattern of meaning in the textbook Pluspunkt Deutsch. Der Orientierungs-kurs [Plus Point German: The Orientation Course] for immigrants to Germany. Although textbooks are frequently used as examples of multimodality (e.g., Bateman,Wildfeuer,Kress and van Leeuwen,, this is not yet the case in GFSL. This case study also rested on Bateman, Wildfeuer, and Hiippala's canvas approach; hence, we understood a textbook as a multimodal artefact constituting an entire canvas. However, as mentioned previously, sub-canvases may require diff erent methods of analysis (see Bateman,Wildfeuer,and Hiippala 214). This second case study considered the texts, images, and layout of the pages within the textbook Pluspunkt Deutsch. The textbook considered here is one of the fi rst approved after the introduction of the courses in 2005. However, the textbooks changed considerably after the course was established; the course gradually increased in length, growing from 30 to 100 teaching units. In describing the results, comparisons are also made to the currently approved textbooks, as these were examined parallel to this article in a dissertation project. Figure 9 shows two pages from Pluspunkt Deutsch. These pages clearly diff er from the other pages in the textbook (e.g., Rights and Duties of Citizens, own translation, 16) in terms of the otherwise colourful pictures and the structure. On the other pages, pictures are displayed diagonally and overlapping, whereas here, they are more ordered. The other pages appear dy-namic, or, as Ledin and Machin said in Doing Visual Analysis, convey "creativity" (83), whereas the images on this page contain empty space. Regarding this point, Ledin and Machin (83) stated: The elements may be part of the same domain but have some similarity since they are separated by space. … Such a design feature can have the eff ect of communicating that things are of the same order which in fact have very little in common or are highly contradictory.
This should be stressed because, here, images of German soldiers-perpetrators-are shown alongside forced labourers and Hungarian Jews-victims of the Nazi regime-arriving in Auschwitz. The suff ering of beaten soldiers is visually placed in the same context as forced labour and the systematic murder of European Jews. The context of origin of the images is not visible. The pictures of the forced labourers and the Hungarian Jews in Auschwitz are not presuppositionless documents-they are part of the crime. The picture "Arrival in Auschwitz" is a picture from the so-called Lili Jacob album, named after its fi nder and the only visually documented arrival of Jews to Auschwitz. Other pictures from this photo album were used in later textbooks. This aspect of order is underscored by the second part of the page, which includes the work assignment "Read the Chronicle" and contains only this chronicle. Such timelines or other chronological sequences are already highly evident in multimodal research. To speak further to this, Ledin and Machin stated, "In other words these fi xed time units are a classifi cation imposed, necessarily concealing how events unfold in real life" (Doing Visual Analysis 186). Individual possibilities for action remain invisible, and both the war and the Holocaust are portrayed as variables that could not have been infl uenced. Stalingrad is mentioned in the picture and the timeline, but it no longer appears in any of the later textbooks. However, one feature that is completely omitted in this textbook but is prominent in later textbooks is German resistance. While there are cloze texts or true/false tasks for other topics within Pluspunkt Deutsch, here-also with regard to reading the chronicle-the questions "What do you know about the time of National Socialism and the Second World War?" and "What do you learn from the pictures?" must be answered. Learners are instructed to form groups, collect keywords, and report back to the class. The second page features the task "Look at the posters and read the text. Why was Hitler able to be so successful?" These questions, in turn, open up more space for discussion than questions on the other pages (e.g., "There are equal, confi dential, and free elections to parliament-true/false?").
Based on the same open, holistic, and interdisciplinary multimodal canvas approach that underpinned the fi rst case study, this case study provided insights into the pattern of meaning Holocaust. In summary, it can be said that images in textbooks play a special role in meaning-making through their selection and positioning on the page. In this and later textbooks, National Socialism and, thus, the crimes of the Holocaust are portrayed as inevitable through timelines and particular sequences of images. The selection of an image from the Lili Jakob album refers to the German visual memory tradition. Following the fi ndings of the fi rst case study, we again believe that a holistic multimodal perspective was essential because partial analysis would have limited the examination of the meaning-making process. Likewise, the point already made in the fi rst case study should be reinforced: Further research must include a more detailed critical analysis of historical intertextuality and argumentative strategies in discourse.

Conclusion
The two case studies make visible the epistemological connections previously identifi ed between GFSL cultural studies and multimodal studies. The research allowed us to elaborate on how GFSL can benefi t from assuming multimodal reality. This fi nding calls for new avenues of research in this area. Although this paper is highly epistemological, it is grounded in teaching and learning practice and remains committed to promoting discourse competence. The ability and competence to participate in discourse-as one of the main goals of cultural learning in GFSL-requires multimodal competence. These two case studies represent the fi rst attempt to approach the exploration of patterns of meaning in a multimodal way. Further studies should apply patterns of meaning approaches to systematize and support their explorations. They should also consider how the tools of multimodal studies can be adapted to the general context of GFSL and patterns of meaning, in particular.