Assassin’s Creed Shadows and the Rise of Emotional Subtitles in Accessible Game Design

29 Mar

Written By JC Pass

Enhancing Narrative Accessibility in Video Games through Non-Verbal Subtext.

Video games have evolved into a dominant form of media and entertainment, captivating a global audience with intricate narratives and immersive worlds. This widespread appeal underscores the critical need for accessibility features that ensure these experiences are inclusive and enjoyable for everyone, regardless of their abilities. A key element in narrative-driven video games is communication, which extends beyond spoken dialogue to encompass a rich tapestry of non-verbal cues. These subtle signals, including facial expressions, body language, and tone of voice, play a vital role in conveying emotions, intentions, and the intricate social dynamics that drive compelling storylines. However, interpreting these non-verbal cues can present significant challenges for individuals with certain disabilities, such as Autism Spectrum Disorder (ASD) and prosopagnosia (face blindness).

In recognizing these challenges, game developers are increasingly exploring innovative accessibility solutions. A notable example of this progress is the recent implementation of non-verbal subtext in the subtitles of Assassin's Creed Shadows. This feature, which has garnered positive initial attention from both the gaming community and accessibility advocates , represents a significant step forward in enhancing narrative accessibility. By explicitly providing textual descriptions of non-verbal cues, Assassin's Creed Shadows offers a potentially transformative tool for players who may struggle to interpret these subtleties. This article aims to explore the benefits of this feature, with a particular focus on its impact on individuals with ASD and prosopagnosia, drawing upon existing research and analysis to support its findings.

The inclusion of non-verbal subtext in subtitles marks a pivotal moment in inclusive game design. It acknowledges the diverse ways in which players interpret and engage with game narratives, moving beyond a one-size-fits-all approach to accessibility. By providing an alternative means of understanding crucial narrative elements, this feature has the potential to unlock richer and more meaningful gaming experiences for a wider range of players.

The Significance of Non-Verbal Communication in Gaming and Challenges for Certain Individuals.

Non-verbal communication is an intrinsic part of the human experience, and its presence in video games significantly contributes to the depth and realism of their narratives. In the virtual worlds of games, players encounter a multitude of non-verbal cues that enrich their understanding of the story and its characters. Facial expressions, ranging from subtle twitches to overt displays of emotion, provide immediate insights into a character's internal state. Body language, including posture, gestures, and movement, further conveys their attitudes and intentions. Even the implied tone of voice, though not directly transcribed in standard subtitles, can often be inferred from the context and delivery of spoken lines. Beyond character interactions, contextual cues and environmental storytelling also contribute to the non-verbal narrative, hinting at past events, character traits, and the overall emotional atmosphere of the game.

These non-verbal elements work in concert with spoken dialogue to create a holistic understanding of the game's narrative, revealing character motivations that might not be explicitly stated and establishing the emotional tone of specific scenes. However, for individuals with Autism Spectrum Disorder (ASD), interpreting these often-nuanced social signals can be a considerable hurdle. Research indicates that many individuals with ASD experience difficulties in processing facial expressions, body language, and subtle emotional cues. This can lead to misunderstandings of character intentions, misinterpretations of emotional states, and a general struggle to fully grasp the social intricacies of the game's world.

Similarly, individuals with prosopagnosia, commonly known as face blindness, face unique challenges in navigating game narratives that heavily rely on visual identification and facial expressions. Prosopagnosia is characterized by a significant difficulty in recognizing faces, and in some cases, this can extend to interpreting emotions conveyed through facial expressions. This inability to readily identify characters and understand their emotional states based on visual cues can severely hinder their ability to follow character interactions and the emotional arcs that drive the story. While the primary deficit in prosopagnosia lies in facial identity recognition, research suggests that emotion recognition through faces can also be affected in some individuals , further complicating their engagement with narrative-rich games.

The reliance on the implicit understanding of non-verbal cues in video games creates an uneven playing field for players with these conditions. While some players naturally and effortlessly absorb these nuances, others require explicit information to fully engage with and comprehend the narrative. This highlights the need for accessibility features that can bridge this gap, providing alternative pathways to understanding the crucial social and emotional information conveyed through non-verbal communication. The challenges are multifaceted; for individuals with ASD, it often revolves around the interpretation of the cues themselves, while for those with prosopagnosia, it can be about the initial recognition of the person and their expressions. This underscores the necessity for diverse, yet potentially overlapping, accessibility solutions that cater to these distinct needs.

Benefits for Individuals with Autism Spectrum Disorder (ASD):

Understanding Social and Emotional Nuances through Explicit Subtext.

For individuals with Autism Spectrum Disorder (ASD) who often face challenges in interpreting non-verbal social cues, the inclusion of explicit non-verbal subtext in subtitles can be profoundly beneficial. This feature provides direct and unambiguous information about character emotions and intentions that might otherwise be missed or misinterpreted. Consider the example from Marvel's Avengers highlighted by an accessibility advocate and autistic gamer Laura Kate Dale (Laura K Buzz), where subtitles explicitly clarified that Kamala Khan's loud slurping of her drink was intended to annoy Bruce Banner, and his subsequent eye roll was an exasperated response. This textual explanation of non-verbal communication proved immensely helpful in understanding the intended interpretation of the scene for the player. The potential of this feature lies in its ability to improve comprehension of social interactions and the often-subtle narrative nuances that rely on these cues for autistic players.

The suggestion of offering this information as a separate, optional subtitle track is particularly noteworthy. This approach would provide greater clarity by distinguishing between spoken dialogue and non-verbal cues, potentially reducing cognitive clutter and allowing players to focus on the information most relevant to their needs. This explicit textual explanation can function as a "social translator" for individuals with ASD, effectively bridging the gap in understanding implicit social communication within the game world. Many individuals with ASD learn social rules and cues through explicit instruction rather than intuitive understanding. This feature provides that explicit instruction within the context of the game, making the social dynamics more accessible. Furthermore, the optional nature of a separate subtitle track is crucial for respecting player preferences. Some autistic players might find this additional layer of information highly beneficial, while others may not need or want it, preventing potential overstimulation or information overload. Accessibility is not one-size-fits-all, and providing options allows players to customize their experience based on their individual needs and preferences, promoting autonomy and comfort.

Research on the Role of Visual Cues in Comprehension for Individuals with ASD.

Research indicates that individuals with ASD often process visual information in unique ways and may rely more heavily on explicit visual cues for understanding social and emotional information. This makes the provision of non-verbal subtext in subtitles, a form of explicit visual cue, particularly relevant. Studies in Applied Behaviour Analysis (ABA) therapy have consistently demonstrated the effectiveness of visual cues, such as choice boards and emotion thermometers, in enhancing communication and emotional understanding for children with ASD. These tools provide structured and clear visual signals that aid comprehension and reduce anxiety, especially during transitions and when expressing needs or emotions. The concept of non-verbal subtext in subtitles aligns with this broader understanding of the importance of visual and textual cues for comprehension in ASD.

While atypical visual perception in ASD might complicate the processing of subtle social stimuli presented visually , providing explicit text can bypass this difficulty by offering a direct and unambiguous representation of the intended meaning. Research in neuroscience suggests that visual encoding plays a significant role in social processes, and alterations in how individuals with ASD visually encode faces, for example, have been linked to social and communication impairments. In this context, explicit subtitles can provide an alternative pathway for understanding social cues, circumventing potential difficulties in visual encoding. Furthermore, findings from educational research indicate that picture cues can increase engagement and comprehension in reading for students with ASD. This suggests a parallel benefit for textual cues in video games, where explicit descriptions of non-verbal elements can enhance narrative comprehension. Even considering research on sensory integration in ASD, which suggests potential differences in how individuals with ASD integrate multisensory information like audio and visual cues , the provision of clear, unimodal textual information about non-verbal cues can still be highly beneficial by offering a less ambiguous form of information that can be readily understood and integrated.

The thoughtful integration of visual cues, including textual ones like non-verbal subtext, into media can promote independent learning and facilitate meaningful growth in comprehension for individuals with autism, harnessing their potential to thrive.

Benefits for Individuals with Prosopagnosia (Face Blindness):

Interpreting Emotions and Character Interactions Beyond Facial Recognition.

Individuals with prosopagnosia often struggle with recognizing faces, which can significantly impair their ability to interpret emotions conveyed through facial expressions. In such cases, non-verbal subtext in subtitles offers a crucial alternative pathway to understanding the emotional tone and intent behind a character's dialogue and actions, without the need to rely on facial recognition. This is particularly important considering the different forms of prosopagnosia, including apperceptive prosopagnosia, where the ability to recognize facial expressions can also be impaired. For these individuals, non-verbal subtext can provide the necessary emotional context that would otherwise be inaccessible due to difficulties in processing facial cues. This feature acts as a vital supplementary tool, allowing individuals with prosopagnosia to still grasp the emotional landscape of the game and follow the nuances of character interactions.

Exploring the Impact of Visual Cues on Emotion Recognition in Prosopagnosia.

Research on prosopagnosia has revealed a complex relationship between face recognition and emotion recognition. While some studies suggest a dissociation, where individuals with prosopagnosia retain the ability to recognize facial expressions , this is not universally true, and other research indicates potential difficulties in emotion recognition for some individuals with prosopagnosia. Even when emotion recognition through faces is possible, it might be less efficient or require more cognitive effort for individuals with prosopagnosia. In such instances, non-verbal subtext can significantly reduce this cognitive load by providing immediate and explicit information about the character's emotional state.

Interestingly, some findings suggest that the presence of facial expressions can sometimes improve face identification in prosopagnosic patients , indicating a complex interplay between these processes within the brain. Regardless of this interaction, non-verbal subtext provides consistent emotional information, offering a reliable cue even when facial expressions might be ambiguous or difficult to process. Furthermore, research has shown that some individuals with acquired prosopagnosia tend to rely more on the mouth area rather than the eyes for facial expression recognition , highlighting the variability in how prosopagnosia affects emotion processing. Non-verbal subtext offers a consistent, alternative cue that does not depend on specific facial features. Notably, studies on developmental prosopagnosia have indicated that some individuals experience difficulties in classifying facial expressions , further emphasizing the need for alternative cues to aid in understanding emotional nuances.

The potential benefits of audio descriptions for individuals with prosopagnosia , where narrators describe visual cues and character emotions, also parallel the benefits of textual descriptions in subtitles. Both approaches aim to provide information that might be missed or misinterpreted due to visual processing differences associated with prosopagnosia. The varied research findings on emotion recognition in prosopagnosia underscore the importance of providing multiple accessibility options. Non-verbal subtext in subtitles offers a valuable alternative when facial cues are unreliable or difficult to interpret, ensuring that individuals with face blindness can still engage with the emotional depth of the game's narrative.

Assassin's Creed Shadows: An Innovative Implementation of Non-Verbal Subtext.

Examining the "Emotional Indicators" Feature and its Functionality.

Assassin's Creed Shadows has taken a significant step towards enhanced narrative accessibility by including an innovative feature that adds non-verbal subtext to dialogue within the game's subtitles. This accessibility option, referred to as "Emotion" in the subtitle settings , functions by displaying cues such as "comforting" alongside the speaker's name. This indicates the emotional tone or manner in which the character is delivering their line of dialogue, with the spoken words appearing underneath this contextual information. Players have the flexibility to toggle this feature on or off within the game's audio and caption settings, allowing them to customize their experience according to their individual needs.

The primary aim of this feature is to provide valuable context for individuals who are hard of hearing, offering insights beyond the literal words being spoken. Additionally, it is specifically designed to assist neurodiverse players, particularly those with ASD, and individuals who generally experience difficulties in picking up on subtle social cues. The implementation of this feature has been met with considerable excitement and positive initial reactions from the accessibility community. Notably, accessibility advocates like Laura Kate Dale (Laura K Buzz), who has long championed the inclusion of such features in video games, have expressed their enthusiasm and appreciation for this development. Assassin's Creed Shadows' implementation of "Emotional Indicators" directly addresses the needs of both individuals who are hard of hearing and neurodiverse players, demonstrating a thoughtful approach to inclusive design. By recognizing the overlapping needs of different disability groups, the developers have created a feature that has broad accessibility benefits. Furthermore, the explicit naming of the feature as "Emotion" within the game's settings is clear and user-friendly, making it easy for players to understand its purpose and toggle it according to their specific requirements.

Discussions and Initial User Experiences of this Accessibility Option.

Early discussions and initial user experiences with the "Emotional Indicators" feature in Assassin's Creed Shadows have been largely positive. Players have taken to social media to share their experiences, with several noting the significant usefulness of this feature in better understanding the emotional tone conveyed during dialogue. Some players have specifically highlighted how these emotional tags have aided them in interpreting dialogue options and gaining a clearer understanding of how non-player characters (NPCs) feel towards the player character. This enhanced understanding can contribute to a more immersive and satisfying role-playing experience.

However, it is important to acknowledge that initial feedback also includes some areas for potential refinement. Some players have reported instances where the emotional tag displayed did not seem to accurately align with the spoken dialogue in their perception. This suggests that while the feature is a valuable addition, ongoing monitoring and potential adjustments may be necessary to ensure its accuracy and consistency across all in-game interactions. Despite this feedback, there has been widespread positive sentiment regarding Ubisoft's general commitment to accessibility in their video games. This broader recognition provides a positive context for the "Emotional Indicators" feature, suggesting that it is part of a larger, company-wide effort to prioritize inclusivity in game design. Initial user feedback indicates that the "Emotional Indicators" feature is largely well-received and effectively enhances understanding of emotional nuances, although some fine-tuning might be necessary to ensure accuracy in all contexts. The recognition of Ubisoft's broader accessibility efforts provides a positive context for this specific feature, suggesting a company-wide commitment to inclusivity in their game design.

General Best Practices for Accessible Subtitle Design in Video Games.

Creating accessible subtitles in video games is crucial for ensuring that a wide range of players, including those with disabilities, can fully engage with the game's narrative. Several established guidelines and best practices exist to enhance subtitle accessibility. These recommendations encompass various aspects of subtitle design, aiming to improve readability, clarity, and overall user experience.

Customization options are paramount for accessibility. Players should have the ability to adjust the font size, colour, and background opacity of subtitles to suit their individual visual needs and preferences. Assassin's Creed Shadows notably offers these customization options. The use of clear, sans-serif fonts is generally recommended for optimal readability. Subtitles should also be timed appropriately to appear on screen in sync with the spoken dialogue and remain visible for a sufficient duration to allow players to read them comfortably. Including speaker identification in subtitles is essential, especially in scenes with multiple characters, to help players track who is speaking. Assassin's Creed Shadows also incorporates speaker names in its subtitles. For players who are deaf or hard of hearing, the inclusion of sound effects and other important non-dialogue audio cues through closed captions or SDH (Subtitles for the Deaf and Hard of Hearing) is vital for a complete understanding of the audio landscape. Assassin's Creed Shadows provides closed captioning as an option. Directional cues for off-screen speakers can further enhance accessibility by indicating the source of sounds , a feature also present in Assassin's Creed Shadows. To prevent visual clutter and improve readability, long lines of text in subtitles should be avoided. Offering players the option to toggle different types of subtitle information, such as dialogue, sound effects, and speaker names, provides greater control over their viewing experience. Assassin's Creed Shadows allows players to toggle speaker names, emotion indicators, closed captions, and gameplay captions. Considering the needs of players with dyslexia, providing dyslexia-friendly font options can also significantly improve accessibility. Finally, providing a preview of subtitle settings within the menu allows players to see how their chosen configurations will appear in the game before applying them.

Assassin's Creed Shadows appears to align with many of these established best practices for subtitle design, moving beyond basic dialogue transcription to offer a more comprehensive and customizable experience for players with various accessibility needs. The inclusion of non-verbal subtext as an additional toggleable option within the subtitle settings reflects a positive trend towards offering more granular control over the information presented to the player, which is a key principle of accessible design. Allowing players to choose what information is conveyed in subtitles empowers them to tailor the experience to their specific needs and preferences, reducing cognitive load and improving comprehension.

Potential Benefits for Other Players.

While the inclusion of non-verbal subtext in subtitles offers significant benefits for individuals with ASD and prosopagnosia, its utility extends to a broader range of players, potentially enhancing their understanding and engagement with the game's narrative as well.

Individuals who are hard of hearing can greatly benefit from the added context provided by non-verbal subtext. While standard subtitles transcribe spoken dialogue, they often miss the crucial emotional nuances conveyed through tone of voice and other auditory cues. Non-verbal subtext can bridge this gap by explicitly stating the intended emotion or tone, ensuring a more complete understanding of the communication. Similarly, non-native speakers playing a game in a language they are still learning can find non-verbal subtext helpful in interpreting nuanced language and implied tones that might be challenging to grasp from the literal translation alone.

Players who find themselves in noisy environments or those who prefer to play with the sound turned off can also benefit significantly from this feature. In these situations, they rely solely on visual information, and the inclusion of non-verbal subtext provides essential contextual cues that would otherwise be lost without the accompanying audio. Furthermore, individuals with other cognitive processing challenges might find the more explicit understanding of social interactions offered by non-verbal subtext beneficial.

Even neurotypical players can experience enhanced understanding and engagement with the game's narrative through this feature. Subtle character emotions that might be missed during fast-paced gameplay or in visually busy scenes can be easily conveyed through the subtitles. This can lead to a richer and more immersive narrative experience, allowing players to pick up on nuances they might have otherwise overlooked. Additionally, non-verbal subtext can be particularly helpful in understanding the subtle differences in tone and intention behind various dialogue options, aiding players in making choices that align with their desired roleplaying style. The benefits of non-verbal subtext in subtitles extend beyond individuals with ASD and prosopagnosia, potentially improving the gaming experience for a wide range of players by providing clearer and more explicit information about character emotions and intentions. This feature can be particularly valuable in situations where audio cues are missed or misinterpreted, such as in noisy environments or when playing in a language that is not the player's first. By providing a visual representation of emotional tone, the game becomes more accessible in various playing conditions and for a more diverse linguistic audience.

Simply put

The inclusion of non-verbal subtext in video game subtitles represents a significant advancement in fostering inclusive gaming experiences. For individuals with Autism Spectrum Disorder (ASD), this feature offers a valuable tool for understanding social and emotional nuances by providing explicit textual explanations of non-verbal cues, aligning with research highlighting the benefits of visual supports for this population. Similarly, for individuals with prosopagnosia, non-verbal subtext provides an alternative means of interpreting emotions and character interactions when facial recognition is challenging or unreliable, addressing the varied impacts of this condition on emotion processing. The implementation of "Emotional Indicators" in Assassin's Creed Shadows serves as a compelling example of how this feature can be effectively integrated into a mainstream title, garnering positive feedback from the accessibility community and initial players.

Beyond the specific benefits for individuals with ASD and prosopagnosia, non-verbal subtext in subtitles holds the potential to enhance the gaming experience for a wider audience, including those who are hard of hearing, non-native speakers, players in noisy environments, and even neurotypical players seeking a deeper understanding of the narrative. This highlights the "curb-cut effect" often associated with accessibility features, where solutions designed for specific needs ultimately benefit a broader range of players.

It is crucial for game developers to continue to innovate and implement such accessibility features, recognizing the diverse needs of their player base and striving to create more inclusive and enjoyable experiences for all. Future research could further investigate the effectiveness of non-verbal subtext for different conditions and player demographics, explore various methods for presenting this information within subtitles, and gather more comprehensive user feedback on its implementation in games like Assassin's Creed Shadows. The positive impact of accessibility features on the gaming community is undeniable, and the continued development and refinement of features like non-verbal subtext have the potential to make video games a truly inclusive and universally enjoyed form of entertainment. The implementation of non-verbal subtext in Assassin's Creed Shadows represents a positive trend in the video game industry towards recognizing and addressing the diverse needs of its player base, paving the way for more inclusive and accessible gaming experiences. Continued research and user feedback will be crucial in refining and expanding the use of non-verbal subtext and other innovative accessibility features in future video game development.