1 Introduction↩︎

Image memes are humorous and culturally referential visual artifacts that have become a dominant form of online communication across social media, forums, and messaging platforms [1]–[5]. Users rely on memes to express opinions, share experiences, and participate in collective cultural discourse [6], [7]. Ensuring equitable access to memes is therefore critical, particularly for blind users who interact with content through synthesized speech via screen readers (e.g., JAWS, VoiceOver, NVDA).

However, meme accessibility remains limited. Screen readers primarily depend on alt text, which is often missing, incomplete, or insufficient for conveying meaning of images to the blind users [8]–[10]. To compensate, blind users increasingly rely on AI-based tools (e.g., ChatGPT, Seeing AI, Be My AI) that generate image descriptions on demand [11], [12]. While these systems improve comprehensibility, they typically produce static, caption-style outputs that fail to capture the humor, timing, and contextual nuance central to memes [13], [14]. As a result, the users often receive functionally correct but experientially diminished representations.

Recent work has explored enhancing engagement through emotional narration [15]. However, the approach largely retains a single-speaker, descriptive paradigm. We argue that the limitation lies not only in what is conveyed, but in how it is structured. Memes are inherently social and interpretative; their meaning often unfolds through reactions, timing, and shared understanding. This motivates modeling memes as dialogs rather than static descriptions.

Towards this, we present MemeBuddy, a system that generates dialog-style audio representations of memes using role-based speakers. Specifically, MemeBuddy reimagines a meme as a structured, multi-turn conversation between two speakers, enabling the gradual unfolding of humor, intent, and cultural context. As illustrated in Table 1, MemeBuddy supports two dialog variants: (i) commentator–commentator (CO), where two speakers collaboratively interpret a given meme, and (ii) commentator–character (CH), where a speaker interacts with a character from the meme. A caption-style baseline (BL) is also supported for comparison.

We evaluate MemeBuddy through a user study with \(14\) blind participants. Results show that dialog-style representations consistently improve engagement, immersion, and enjoyment compared to caption-style descriptions, while maintaining comparable comprehension. Our findings further reveal trade-offs between conversational coherence and expressiveness, offering insights into how non-visual content representations can be made both accessible and engaging for blind users.

In summary, we contribute: (i) Novel dialog-based audio representations using role-based speakers for engaging meme consumption; and (ii) An empirical evaluation demonstrating that dialog structure consistently improves meme engagement while preserving comprehension.

2 Related Work↩︎

Our work closely relates to prior efforts in making visual media accessible, as well as research on meme understanding and accessibility.

2.1 Accessibility of Visual Web Content↩︎

Prior work has explored improving blind and low vision users’ access to visual web content, including images, memes, GIFs, and videos [9], [16], [17]. Early systems, such as WebInSight, generated alt text using web context and OCR, highlighting both the prevalence of missing descriptions and the potential of automation [18]. Later approaches introduced automatic image descriptions (e.g., Automatic Alt-text [19]), yet studies show that both user-provided and AI-generated descriptions are often sparse or misaligned with users’ needs [8], [9], [20]. Large-scale audits confirm that many images still lack meaningful alt text [10].

Beyond basic descriptions, research has examined richer representations of visual content. Previous works highlight the need for flexible, context-sensitive descriptions and interactive access to visual structure [21]–[23]. Accessibility efforts have also extended to dynamic media, including memes, GIFs, and videos, through approaches such as emotion-aware captions, expressive narration, and hierarchical summaries [15], [24]–[27]. However, AI-based tools, while increasingly used in practice, still exhibit limitations in usability and experiential quality [11], [12], [28].

Overall, prior work primarily focuses on semantic access, i.e., helping users understand what is depicted, while retaining caption-style paradigms. Less attention has been given to preserving social and experiential qualities of content. Our work addresses this gap by modeling memes as dialog, enabling engagement as well as comprehension.

2.2 Memes and their Accessibility↩︎

Internet memes are a multimodal and culturally-grounded form of communication that combine visual and textual elements to convey humor, opinions, and social meaning [1]–[7]. Understanding memes requires reasoning over visual metaphor, inter-textual references (e.g., celebrities, popular culture), and contextual knowledge [14], [29], [30], making them more complex than standard images. Prior works have addressed meme understanding through captioning, explanation, and multimodal reasoning [29], [31], [32], as well as emotion and metaphor analysis [33], [34]. These efforts highlight the inherently inferential nature of memes, where meaning extends beyond literal content.

Prior accessibility work has focused on enriching meme descriptions. Gleason et al. [24] proposed structured descriptions beyond OCR text, while Prajwal et al. [25] incorporated facial-emotion cues. More recently, MemeSpeak [15] demonstrated that emotional speech can enhance meme narration. However, existing approaches largely remain within a descriptive paradigm, typically relying on single-speaker narration. This overlooks the inherently social and temporal nature of memes, where meaning unfolds through timing, interaction, and interpretation. As a result, prior work improves understandability but does not fully address engagement. Our work addresses this gap by modeling memes as dialogs through structural re-representation. MemeBuddy generates multi-turn, role-based conversational representations that enable meaning to gradually unfold through interaction, providing a more engaging non-visual meme experience while complementing prior approaches in this area.

3 MemeBuddy Design↩︎

The design of MemeBuddy was informed by both prior research on dialog as an effective medium for human-information interaction [35]–[39] and insights from an IRB-approved focus group study. The study involved seven participants with complementary expertise: two authors, a psycholinguist, a media researcher, a social computing researcher, and two blind university students with extensive experience in meme consumption and creation on social media platforms.

The focus group was conducted as a structured design elicitation session. The participants were guided using seed themes derived from prior work on accessible content representation and conversational interfaces, and were asked to collaboratively explore how visual memes could be translated into engaging non-visual audio formats. Discussions centered on key design dimensions, including number of speakers, distribution and timing of information across turns, mechanisms for conveying humor and sarcasm, and desirable prosodic characteristics of voices. All sessions were audio-recorded and transcribed for subsequent analysis.

We analyzed the transcripts using a qualitative coding process following open and axial coding procedures [40]. In the open coding phase, recurring concepts (e.g., “turn-taking improves clarity”, “characters increase immersion but may confuse roles”) were identified. These codes were then grouped during axial coding into higher-level design principles. Two dominant dialog structures emerged: (i) a commentator–commentator format emphasizing clarity and conversational coherence, and (ii) a commentator–character format emphasizing immersion and expressiveness. Additionally, participants consistently requested a caption-style fallback to ensure reliability when dialog representations are insufficient. We operationalized these insights as prompt-level constraints in MemeBuddy (Appendix 7).

At a system level, MemeBuddy uses controlled prompts to a multimodal LLM to transform an input meme image into structured outputs containing both dialog and corresponding text-to-speech (TTS) instructions. These outputs are rendered as audio using expressive TTS engines. The strategies are described next.

3.1 Commentator-Commentator (CO) Design↩︎

The group strongly preferred two-speaker dialogs over multi-speaker alternatives, citing reduced cognitive load, improved coherence, and avoidance of unnecessary complexity. We encoded these preferences directly into the LLM prompt used for CO dialog generation (Table ¿tbl:table:CO-dialog-prompt? in Appendix 7).

In this design, the model produces a structured conversation between two speakers: Host1, who provides explanatory content, and Host2, who offers reactions, questions, or informal interpretations. The dialog is intentionally concise, with meaning distributed across turns to support gradual understanding and anticipation, and concludes with a punchline-style summary that conveys the meme’s intent. The prompt also specifies TTS constraints (e.g., distinct voice styles and genders) to ensure clear speaker separation. Outputs are generated in a structured JSON format containing both dialog content and TTS specifications, enabling consistent downstream rendering.

3.2 Commentator-Character (CH) Design↩︎

To enhance immersion, participants proposed incorporating a character from the meme as an active speaker in the conversation. We operationalized this idea through the CH design, implemented via prompt constraints (Table ¿tbl:table:CH-dialog-prompt? in Appendix 7).

In this configuration, the dialog occurs between a Host (commentator) and a Guest (a character from the meme). The Guest may adopt a first-person perspective or respond in a manner consistent with the character’s role, identity, or persona. The prompt enforces explicit self-introduction of the Guest to maintain role clarity, and guides the interaction such that the character meaningfully contributes to the narrative and punchline. Similar to CO, the dialog remains concise and structured, and includes TTS instructions specifying distinct and contextually appropriate voice characteristics. Compared to CO, which prioritizes interpretability through neutral commentary, CH emphasizes expressiveness and narrative immersion.

3.3 Caption-style Design (Baseline)↩︎

We also include a caption-style baseline (BL) in MemeBuddy to reflect current practices and provide a reliable fallback. This design was strongly recommended by participants for scenarios requiring maximal clarity and explicitness.

The BL prompt (Table ¿tbl:table:BL-description-prompt? in Appendix 7) generates a single-speaker narration delivered by a Narrator. The narration follows a structured progression: identifying the meme template (when applicable), describing visual elements in detail (e.g., layout, expressions, embedded text), and concluding with an explicit explanation of the meme’s intent. The prompt encourages a natural, conversational tone while maintaining completeness and self-containment. It also specifies a consistent TTS configuration (warm, descriptive voice) to ensure intelligibility. In contrast to dialog-based designs, BL prioritizes semantic completeness over interaction and interpretive engagement.

3.4 Offline MemeBuddy Evaluation↩︎

Following standard practice in dialog and LLM-output evaluation, we conducted an offline human assessment of MemeBuddy-generated meme representations. The evaluation was designed to address the following research question: RQ1: How well do the audio representations capture the intent, cultural nuances, and contextual details of a meme?

3.4.1 Dataset Collection and Annotation↩︎

We constructed a dataset of \(100\) meme images to support the offline evaluation, ensuring diversity in structure, content, and style. The dataset was compiled from two sources: the Kaggle Meme Images Dataset [41] and the Imgflip platform [42].

The Kaggle dataset, originally curated for sentiment analysis, contains a mixture of general images and memes. To ensure relevance, we filtered this dataset to retain only images that conform to common meme characteristics (e.g., presence of overlaid text, recognizable templates, or multimodal humor). From this filtered subset, we randomly sampled \(62\) memes. To complement this, we additionally collected \(38\) popular meme instances from Imgflip, prioritizing templates with high usage frequency and cultural familiarity.

We ensured that all selected memes contained both visual and textual components, as such multimodal interplay is central to meme interpretation and poses known challenges for blind users. We excluded memes containing hateful, abusive, or politically sensitive content to avoid confounds.

Each meme was manually annotated using a structured schema (Table 4 in Appendix 8) designed to capture both low-level visual details and high-level semantic intent. Specifically, annotations included template identity, panel structure, key visual elements, verbatim textual content, and a concise intent summary describing the humor mechanism. Template names were validated against popular meme repositories (e.g., Imgflip, Kapwing) to ensure consistency and correctness.

To improve annotation reliability, two annotators independently labeled each meme, followed by a reconciliation process to resolve disagreements through discussion. This process ensured consistency in interpreting subjective fields, such as intent summary, which are critical for evaluating generated outputs. The resulting dataset provides a grounded reference for assessing the quality and fidelity of MemeBuddy representations.

3.4.2 Models and Metrics↩︎

We evaluated MemeBuddy representations generated using multiple LLMs (with temperature set to 0 to minimize random results). Specifically, we experimented with: (i) Gemini 3 Flash, (ii) Gemini 3 Pro, (iii) GPT-5.1, and (iv) Gemma 4. For each meme, all models were prompted using the same instructions (Tables ¿tbl:table:BL-description-prompt?, ¿tbl:table:CO-dialog-prompt?, and ¿tbl:table:CH-dialog-prompt?) for each representation type (BL, CO, CH) to ensure consistency.

Evaluating dialog-like and multimodal outputs remains challenging, as standard n-gram metrics (e.g., BLEU) fail to capture semantic and experiential quality [43]. Recent work, particularly the FED framework, has advocated for fine-grained human evaluation along multiple dimensions, such as coherence, engagement, grounding, and correctness [44]. Guided by this framework, we define the following metrics tailored to meme understanding: (i) Intent Clarity: How easily a listener can understand the meme’s setup, punchline, and tone; represented as a 5-point Likert item (1 = very difficult, 5 = very easy); (ii) Representation Fidelity: Whether humor-critical visual elements (e.g., expressions, relationships, scene structure) are correctly captured; binary representation (0 = incorrect/missing, 1 = correct); (iii) Text Accuracy: Whether on-image text is reproduced or paraphrased accurately; binary representation (0 = incorrect, 1 = correct); and (iv) Speculation: Presence of unsupported, irrelevant, or hallucinated content; binary representation (0 = absent, 1 = present).

Each <meme, model, representation> triplet was independently evaluated by two annotators who were not part of the focus group, improving external validity. The annotators were provided with the original meme, generated outputs, and the dataset annotations (Table 4) as grounding reference. Items were presented in randomized order via a custom web interface to mitigate ordering effects.

We computed inter-annotator agreement for each metric (Cohen’s \(\kappa\) for binary measures and weighted \(\kappa\) for Likert ratings). Rather than resolving disagreements, we retained annotator variability by aggregating ratings across annotators, following prior work [44], [45]. Specifically, Likert ratings were averaged and binary metrics were reported as proportions. All subsequent analyses were conducted on these aggregated scores.

3.4.3 Results↩︎

Table 2 summarizes results across 100 memes and two annotators. Inter-annotator agreement was strong (Intent Clarity: \(\kappa=0.78\); Representation Fidelity: \(0.71\); Text Accuracy: \(0.86\); Speculation: \(0.82\)), supporting reliability (variance statistics in Appendix 9). Gemini Pro and GPT performed the best, followed by Gemini Flash, with Gemma trailing. BL and CO exhibited high Intent Clarity, while CH was lower across all models. In contrast, Fidelity and Text Accuracy remained high across all conditions.

Friedman tests showed a significant effect of condition on Intent Clarity for all models (Gemini Flash: \(\chi^2(2)=28.34\), \(p<.001\), \(W=0.142\); Gemini Pro: \(\chi^2(2)=9.64\), \(p=.008\), \(W=0.048\); GPT: \(\chi^2(2)=19.18\), \(p<.001\), \(W=0.096\); Gemma: \(\chi^2(2)=29.49\), \(p<.001\), \(W=0.147\)), with small effect sizes. Post-hoc Wilcoxon signed-rank tests (with Holm’s correction) showed CH was significantly worse than BL and CO for Gemini Pro, Gemini Flash, and GPT; for Gemma, all pairwise differences were significant (BL \(>\) CO \(>\) CH). No significant effects were observed for Fidelity and Speculation; for Text Accuracy, marginal omnibus effects were observed for GPT and Gemma, but these did not yield significant pairwise differences.

Absolute differences in Intent Clarity were small (typically \(\leq 0.3\)), indicating limited practical degradation. CO closely matched BL, especially for stronger models, showing that dialog structuring preserves semantic quality when coherence is maintained. In contrast, CH exhibited lower clarity and higher speculation, suggesting added complexity from character role modeling and turn alignment.

Overall, results indicate that dialog representations largely preserve comprehensibility, with modest degradation mainly in CH. We next examine if these structural differences impact engagement and enjoyment via a user study with blind users.

4 User Study with Blind Users↩︎

We conducted an IRB-approved user study with blind participants to evaluate MemeBuddy and address the following research questions: RQ2: What are the strengths and limitations of dialog-style meme representations compared to caption-style descriptions? and RQ3: How can dialog-style representations be improved to enhance engagement and enjoyment?

4.1 Study Design↩︎

We recruited 14 blind participants through email lists and word-of-mouth. All participants were familiar with AI-based image description tools and reported regular exposure to memes. Table ¿tbl:table:participants? (Appendix 10) shows the demographics of participants.

We used a within-subject design with three conditions:

• Baseline (BL): caption-style audio description by a single speaker.

• Commentator–Commentator (CO): dialog between two speakers.

• Commentator–Character (CH): dialog between a speaker and a meme character.

The BL condition reflects the dominant paradigm used by current AI-based accessibility tools, where images are described through single-speaker, caption-style narration. As such, it provides an ecologically valid and deployment-relevant baseline for comparison. Each participant completed 6 tasks (2 per condition), given the session time limit of 2 hours. In each task, the participant listened to a meme’s audio representation and completed a questionnaire measuring comprehension, emotional response, immersion, entertainment, funniness, and willingness to engage further (Table 3). Q1 was open-ended; Q2–Q6 used 5-point Likert scales adapted from prior work [15], [46]–[50]. These measures capture dialog effectiveness from a user perspective, reflecting perceived coherence, pacing, and conversational engagement rather than purely structural dialog metrics. We used Gemini 3 Pro to generate the representations, given its best performance in the offline evaluation.

To control for ordering and learning effects, we used six distinct memes (Appendix 11) and counterbalanced study conditions using a Latin square design [51]. The selected memes were self-contained and required only common background knowledge, minimizing the influence of participants’ prior familiarity with specific topics and enabling a fair comparison across conditions. The study was conducted in person using a Windows laptop with JAWS and NVDA installed. Following a brief practice session, participants completed all six tasks and then participated in a semi-structured exit interview. Each session lasted approximately two hours, and participants received an $80 gift card as compensation.

4.2 Measures and Analysis↩︎

With participants’ written consent, we collected questionnaire responses, think-aloud observations, and interview data. For Q1, responses were manually coded for correctness against the intended meme meaning. For Q2–Q6 (ordinal repeated measures), we used Friedman tests followed by Wilcoxon signed-rank post-hoc tests with Holm correction. Qualitative data were analyzed using open and axial coding [40]. Two authors independently conducted initial coding, resolved discrepancies through discussion, and collaboratively derived higher-level themes.

4.3 Results↩︎

Comprehension (Q1). We coded open-ended responses against the intended meme meaning (binary correct/incorrect). Two annotators achieved high agreement (Cohen’s \(\kappa=0.87\)), with disagreements resolved by a third annotator.

All conditions achieved high comprehension: BL (92.85%), CO (89.28%), and CH (82.14%). BL performed best, likely due to its explicit and coverage-oriented narration. CO remained close to BL, suggesting that distributing information across dialog turns does not substantially affect interpretability. CH showed a modest decrease, which may reflect increased ambiguity in speaker roles and turn coherence. Importantly, comprehension remained above 80% across all conditions.

Engagement, Affect, and Preference (Q2–Q6). Results are shown in Figure 1. We analyzed Likert responses using Friedman tests with Wilcoxon signed-rank post-hoc tests (Holm-corrected); full statistics are reported in Appendix 12. Friedman tests indicated a significant effect of condition for all measures (\(p<.001\)). However, post-hoc comparisons revealed more nuanced patterns.

Key patterns: (i) Emotional response (Q2) increased significantly under CO and CH compared to BL; (ii) Immersion (Q3) improved under CO and CH, with CO outperforming BL and CH; (iii) Entertainment and funniness (Q4–Q5) showed selective improvements: CO significantly outperformed CH (Q4) and BL (Q5), while other pairwise differences were not consistently significant; (iv) Preference (Q6) showed the strongest effect, with CO significantly outperforming both BL and CH, while CH did not significantly differ from BL.

Overall, CO consistently yielded the highest ratings and most robust improvements across measures, whereas CH showed more variable gains. These results suggest that engagement benefits arise not only from multi-speaker dialog, but also from maintaining conversational coherence. Collectively, the findings indicate that structural re-representation plays a central role in shaping affective and experiential aspects of meme consumption.

Repeated Listening and Interaction Effort. Participants replayed memes under the CO condition more frequently for enjoyment, whereas replays under BL were primarily for clarification, based on think-aloud observations. Informally, CO appeared to reduce perceived effort by segmenting information into conversational turns. CH exhibited mixed patterns, with replays occurring both for enjoyment and for disambiguation when speaker roles were unclear. These patterns align with Q3–Q6 outcomes and suggest that turn-level segmentation contributes to engagement.

Qualitative Insights. Six themes emerged from interview data analysis.

(i) Two speakers improve engagement.  Participants consistently preferred dual-speaker dialog over single-speaker narration, citing reduced monotony, improved pacing, and clearer segmentation of information. Turn-taking helped distribute content into manageable units, making it easier to follow and less cognitively demanding. Participants also noted that dialog introduced anticipation, as they expected reactions or follow-ups from the second speaker. As one participant remarked, “It feels like when two people are talking, it’s not boring anymore.” These benefits were most consistently observed in the CO condition, where conversational flow was clear and predictable; CH also improved engagement over BL, but less reliably.

(ii) Emotional expressiveness and coherence jointly shape experience.  Participants emphasized that engagement depends not only on having multiple voices, but also on how expressively and coherently those voices interact. While CH introduced character-driven dialog, its effectiveness was sometimes limited by flat prosody or unclear conversational roles. Participants suggested richer emotional cues (e.g., sarcasm, emphasis, pacing) and more consistent voice behavior to make interactions feel natural. For example, one participant noted, “If the character sounded more expressive, it would feel like a real clip.” These observations help explain why CH condition improved engagement over BL condition but did not match the consistency of CO condition, where speaker roles were clearer and interactions more coherent.

(iii) Dialog encourages an interactive mental model.  Participants desired to engage more actively with dialog-style representations, such as replaying specific turns, asking follow-up questions, or probing unclear parts. This suggests that dialog naturally shifts users from passive consumption toward a more interactive interpretation process. As one participant put it, “I wish I could jump in and ask what they meant.” This expectation of interaction was less evident in the BL condition, which was perceived as more static and explanatory.

(iv) Applicability beyond memes.  Participants noted that dialog-style representations could extend beyond memes to other forms of visual content, particularly personal photos. They felt that dialog could better convey emotional context and lived experience compared to static descriptions. One participant explained, “This would be great for my photos ... it would bring back memories better than just descriptions.” This suggests broader applicability of dialog-based representations for experiential accessibility.

(v) Avoid over-explanation.  Participants preferred representations that preserve some level of inference rather than fully explaining the meme. BL descriptions were often perceived as overly explicit, reducing enjoyment by eliminating the need to interpret the humor. As one participant noted, “It tells me everything, so there’s nothing left to figure out.” In contrast, dialog, particularly in CO, allowed meaning to unfold across turns, supporting humor through timing and partial revelation.

(vi) Need for customization.  Participants highlighted the importance of flexibility in representation. They expressed interest in switching between BL, CO, and CH depending on context, as well as customizing voice styles and navigating content at finer granularity (e.g., replaying individual turns). One participant suggested, “I’d like to go back to just one part if I miss something.” These preferences indicate that no single representation is universally optimal, and that adaptable interfaces may better support diverse user needs.

5 Discussion↩︎

Our results show that dialog-style representations can improve the non-visual experience of memes for blind users, while maintaining high comprehension (Q1). However, these improvements are not uniform across conditions or metrics, with the commentator–commentator (CO) design consistently yielding the most robust gains.

RQ2: Strengths and limitations. Dialog-style representations shift meme access from an informational toward a more experiential paradigm. While caption-style descriptions (BL) ensure semantic clarity [24], [28], they often fail to capture humor, timing, and cultural nuance. In contrast, dialog representation distributes meaning across turns, enabling users to infer intent through temporal unfolding. This is reflected in significant improvements in emotional response (Q2), immersion (Q3), and user preference (Q6), with more selective gains for entertainment and funniness (Q4–Q5).

These findings align with prior work on narrative engagement and experiential design, which emphasize temporal structure and anticipation [37], [50]. Dialog introduces rhythm and contrast, allowing humor to emerge through interaction rather than explicit explanation.

At the same time, effectiveness depends critically on conversational coherence. The consistent advantage of CO over CH suggests that simply introducing additional expressive elements (e.g., meme characters) is insufficient. In CH, role ambiguity and breakdowns in turn coherence occasionally increased cognitive effort and reduced immersion. This highlights an important design principle: engagement arises not from expressiveness alone, but from structured and coherent interaction; expressive elements that disrupt role clarity can reduce both interpretability and immersion.

RQ3: Improving engagement. Our findings suggest three directions for improvement. First, emotional expressiveness remains critical. Participants noted that richer prosody (e.g., sarcasm, emphasis) would improve realism and engagement. While prior work demonstrates benefits of emotional narration [15], our results suggest these gains may be amplified when applied at the level of dialog. Second, dialog representations naturally invite interaction. Participants expressed interest in asking follow-up questions and revisiting specific turns, suggesting that dialog-based systems should evolve toward interactive conversational interfaces. This aligns with emerging reliance on conversational visual assistants [52], [53]. Third, customization is essential. Participants requested flexibility in representation style, voice characteristics, and navigation granularity. This indicates that future systems should support adaptive, user-controlled configurations rather than a single fixed representation.

Implications for Accessible Content Design. Our work extends prior research on accessible image descriptions [15], [46], which has largely focused on semantic comprehension. We show that structural re-representation, specifically dialog-style rendering, plays a central role in enhancing engagement.

This perspective aligns with prior work emphasizing that user experience depends on both content and presentation [15]. By modeling memes as conversations, MemeBuddy better reflects their inherently social nature. More broadly, our findings suggest a shift from equivalence (same information) to experiential parity (comparable experience), particularly for socially and culturally rich media such as memes.

Limitations. Our work has several limitations. First, static voice configurations likely constrained the effectiveness of CH; future work should explore context-aware expressive speech. Second, we only considered English memes, limiting generalizability. Third, we focus on static images, excluding GIFs and videos where dialog representations may be even more effective. Fourth, each study session was limited to six meme tasks to fit within the two-hour session duration. Future longitudinal in-the-wild studies should evaluate dialog-style representations using substantially larger and more diverse meme collections spanning a broader range of topics, formats, and cultural contexts. Fifth, our user study sample size is modest, however, it is consistent with prior user studies involving blind participants, where recruitment is challenging and within-subject designs provide sufficient sensitivity for comparative evaluation. Sixth, we did not isolate the individual contributions of MemeBuddy’s two key design elements, i.e., multiple voices and temporal unfolding, to user engagement. Although participants consistently identified both as important during the exit interviews, future work should disentangle their effects through controlled experiments to quantify their respective contributions to engagement, immersion, and enjoyment. Lastly, we note that contextual interpretation (e.g., cultural references) is inferred implicitly by the LLM and not externally grounded, which may introduce errors for niche memes.

6 Conclusion↩︎

We presented MemeBuddy, a system that rethinks meme accessibility through dialog-style audio representations that model memes as structured two-speaker conversations. Across an offline evaluation and a user study with blind participants, we show that dialog-based representations maintain high semantic fidelity while improving engagement, immersion, and perceived humor over caption-style descriptions. These findings highlight a key limitation of current approaches: optimizing for comprehension alone is insufficient for socially and culturally rich media such as memes. Instead, conversational structure and temporal unfolding play a central role in shaping user experience. By moving toward experiential accessibility, MemeBuddy points to a promising direction for assistive systems that support both understanding and engagement.

Acknowledgments↩︎

We thank anonymous reviewers for their insightful feedback. We also thank the participants and Dr. William Seiple for their support with the user study.

7 Meme Representation Generation Prompts↩︎

[breaklines=true, breakanywhere=true, breaksymbol=]
You are tasked with explaining memes to a blind person in an engaging monologue as a single narrator, and also generating TTS voice style instructions.
## Rules for Output
- Output strictly in JSON with two top-level keys:
  - "dialogues" → array of objects with "speaker": "Narrator" and "text".
  - "tts_instructions" → array with one object describing how the narrator’s voice should sound.
- No extra narration, no text outside the JSON.
## Rules for Dialogue
- Always use "speaker": "Narrator".
- Start by recognizing or naming the meme template (if known).
- Describe what’s happening visually so a blind listener can clearly imagine it (panels, labels, expressions, text placement).
- Keep the tone casual, cohesive, and natural — like a friend explaining something funny. Add light filler words or small reactions so it doesn’t sound robotic.
- End with a short summary punchline that explains the joke in plain language.
- Keep total length around 25-75 seconds depending on meme complexity. Don’t pad unnecessarily.
## Rules for TTS Instructions
- Provide "tts_instructions" with one object:
  - "speaker": "Narrator"
  - "style": Warm, conversational, descriptive voice. Clear pacing, friendly tone, with a hint of humor.
---
# Example Input → Output
Input: Meme: Distracted Boyfriend with labels “me,” “responsibilities,” and “new hobby.”
Output:
{
  "dialogues": [
    {"speaker": "Narrator", "text": "Oh hey, this one’s the distracted boyfriend meme."},
    {"speaker": "Narrator", "text": "You’ve got the guy walking with his girlfriend, but his head’s turned to check out another woman passing by."},
    {"speaker": "Narrator", "text": "In this version, the girlfriend is labeled ‘responsibilities,’ the guy is ‘me,’ and the other woman is ‘new hobby.’"},
    {"speaker": "Narrator", "text": "So yeah, the joke is basically about ignoring your duties just because something fun comes along."}
  ],
  "tts_instructions": [
    {"speaker": "Narrator", "style": "Warm, conversational, descriptive voice. Clear pacing, friendly tone, with a hint of humor."}
  ]
}

[breaklines=true, breakanywhere=true, breaksymbol=]
Your task is to describe a meme to a blind person in an engaging dialog format between two commentators, and also generate text-to-speech (TTS) voice style instructions. While creating the dialog and TTS instructions, you should strictly follow these rules:
Rules for constructing the dialog:
- The two commentators must be labeled "Host1" and "Host2".
- Host1 should start the conversation.
- Host1’s utterances should be more explanatory and Host2’s utterances should be more casual/reactive.
- The conversation should not be overly detailed. There should be room for self-interpretation for the listener.
- The conversation should be natural and concise: 25-75 seconds if narrated with a TTS. Don't pad content unnecessarily.
- The conversation should end with a plain punchline-style summary from Host2 to convey the meme intent such as humor or sarcasm.  
Rules for generating TTS instructions:
- Host1’s voice should have a clear, explanatory style.
- Host2’s voice should be more casual and reactive, complementing Host1’s voice. 
- Host 1 and Host 2 should not have the same gender.
- The voices of Host1 and Host2 should be distinct so the listener can easily tell them apart.
- Gender assignment should be either "male" or "female".
Rules for Output format:
- Output must be strictly a JSON object with two keys: "dialogs" and "tts_instructions".
- "dialogs" must be an array of objects, with each object having “speaker” and “text” fields along with their values.
- "tts_instructions" must be an array of objects, with each object having “speaker”, “style”, and “gender” fields along with their values.
- An illustrative example showing an input-output pair is given next. 
Example Input-Output pair:

Input: [Drake "Yes/No" meme image with labels "Working overtime" and "Taking a nap."]

Output:
{
  "dialogs": [
    {"speaker": "Host1", "text": "This is the classic Drake meme—two stacked panels showing preference."},
    {"speaker": "Host2", "text": "Oh yeah, the top panel has Drake waving his hand dismissively at 'working overtime.'"},
    {"speaker": "Host1", "text": "And the bottom shows Drake pointing approvingly at 'taking a nap.'"},
    {"speaker": "Host2", "text": "So basically: work bad, sleep good. The eternal struggle!"}
  ],
  "tts_instructions": [
    {"speaker": "Host1", "style": "Clear, explanatory tone. Professional but friendly, like a radio host.", "gender": "female"},
    {"speaker": "Host2", "style": "Casual, laid-back voice with good energy. More conversational and reactive.", "gender": "male"}
  ]
}

[breaklines=true, breakanywhere=true, breaksymbol=]
Your task is to describe a meme to a blind person in an engaging dialog format between a commentator (i.e., Host) and a character (i.e., Guest) from the meme, and also generate text-to-speech (TTS) voice style instructions. While creating the dialog and TTS instructions, you should strictly follow these rules:
Rules for Dialog:
- The speakers must be labeled "Host" and "Guest".
- The Host should start the conversation.
- For Guest, if there are more than one characters in the meme, select the character that is associated with the punch-line that conveys intent like humor or sarcasm.
- The Guest must introduce themselves in their first utterance in the dialog. If the selected character/person from the meme has a known name, use it (e.g., "Hi, I'm Drake"). If the name is unknown, have the Guest describe themselves by relating to how the Host refers to them (e.g., if Host says "the person in the meme," Guest says "Yeah, I'm the person in the meme"). This helps blind listeners distinguish between speakers.
- If the meme suits first-person narration (e.g., character/person style meme), then the Guest should speak as if they are the character. If not, the guest should speak like a commentator.
- The conversation should not be overly detailed. There should be room for self-interpretation for the listener.
- The conversation should be natural and concise: 25-75 seconds if narrated with a TTS. Don't pad content unnecessarily.
- The conversation should end with a plain punchline-style summary conveying the meme intent such as humor or sarcasm. 
Rules for TTS Instructions:
- First decide the Guest's gender based on the meme character or context. If the gender of the character is not clear (e.g., a bird), choose the gender that creates better contrast between the two speakers.
- The gender of the Host should be the opposite of Guest to maximize voice distinction.
- The gender should be either “male” or “female”.
- The voice style for the Host should be warm and clear with an explanatory tone. The pacing should be moderate like a professional podcast narrator.
- The voice style for the Guest will depend on the meme character. If the selected meme character for the Guest is well known (e.g., a celebrity), then configure the style to match their publicly-known speech attributes (tone, personality, pacing). If the selected character is not a public figure, then use a friendly and conversational style that best complements the Host’s style.
- Try your best to make the voice profiles of Host and Guest distinct so the listener can easily tell them apart.
Rules for Output format:
- Output must be strictly a JSON object with two keys: "dialogs" and "tts_instructions".
- "dialogs" must be an array of objects, with each object having “speaker” and “text” fields along with their values.
- "tts_instructions" must be an array of objects, with each object having “speaker”, “style”, and “gender” fields along with their values.
- An illustrative example showing an input-output pair is given next. 
Example Input-Output pair:

Input: [Meme: Drake “Yes/No” with labels “Working overtime” and “Taking a nap.”]

Output: {
  "dialogs": [
    {"speaker": "Host", "text": "This is the Drake meme—the two stacked panels."},
    {"speaker": "Guest", "text": "Yeah, in the top I’m waving my hand like, ‘nope,’ at the words ‘working overtime.’"},
    {"speaker": "Host", "text": "And then in the bottom, you’re pointing happily at ‘taking a nap.’"},
    {"speaker": "Guest", "text": "Exactly—work is out, sleep is in."}
  ],
  "tts_instructions": [
    {"speaker": "Host", "style": "Warm, clear, explanatory tone. Moderate pace, like a friendly podcast narrator.", "gender": "female"},
    {"speaker": "Guest", "style": "Casual, smug, playful voice—try to sound like Drake, with a relaxed pace and confident energy.", "gender": "male"}
  ]
}

8 Meme Annotation Schema↩︎

9 Additional Offline Evaluation Details↩︎

9.1 Variance Statistics↩︎

Table 5 reports the standard deviations (SD) for Intent Clarity and binomial standard errors (SE) for the binary metrics, computed across 100 memes and two annotators. Variance is moderate overall, with slightly higher variability in CH across models, indicating less consistent performance for character-based dialog generation.

9.2 Effect Sizes↩︎

Table 6 shows Kendall’s \(W\) for Friedman tests on Intent Clarity.

All effect sizes are small (\(W \leq 0.15\)), indicating that although condition differences are statistically significant, their practical impact on comprehensibility is limited.

9.3 Pairwise Effect Sizes↩︎

Table 7 reports representative Wilcoxon signed-rank effect sizes (\(r\)).

Pairwise comparisons show larger (but still modest) effects involving CH, consistent with reduced clarity in character-driven dialog.

10 User Study Participant Demographics↩︎

11 Memes in User Study Evaluation↩︎

The six memes shown in Figure 2 were used in the user study.

12 Full Inferential Statistical Results (Q2–Q6)↩︎

We analyzed Likert-scale responses (Q2–Q6) using Friedman tests followed by pairwise Wilcoxon signed-rank tests with Holm correction for multiple comparisons. Effect sizes are reported as Kendall’s \(W\) for omnibus tests and \(r\) for pairwise comparisons. The inferential statistical results for Q2–Q6 are shown in Table 8.

References↩︎