DiscoSign: Discourse-Aware Text to Sign Language Gloss Translation

Summary

DiscoSign addresses a limitation in sign language processing: systems have traditionally translated at the sentence level, overlooking discourse phenomena important to sign language comprehension. It presents a modular Large Language Model (LLM)-based framework for text-to-sign language gloss translation that handles spatial coreference resolution, Question-Answer Clauses (QACs), and concept-gloss consistency, preserving entity locations and stable English-to-ASL sign mappings across discourse. Because traditional translation metrics do not measure discourse-level quality, the authors introduce metrics for each coherence dimension targeted by the framework. Experiments on sentence-level and discourse-level datasets report improved spatial consistency and entity tracking over sentence-only translation, while maintaining competitive single-sentence gloss quality; the work describes this as the first systematic framework with a corresponding evaluation methodology.

Takeaways

1. 1

   The framework addresses spatial coreference, QACs, and concept-gloss consistency in text-to-ASL gloss translation.

2. 2

   The evaluation suite is designed to measure discourse coherence dimensions that traditional translation metrics do not capture.

3. 3

   Experiments report improved spatial consistency and entity tracking over sentence-only translation while retaining competitive single-sentence gloss translation quality.