Which statement accurately describes long-range dependencies in language models?

Which statement accurately describes long-range dependencies in language models?

• A. Dependencies across entire documents
• B. Local dependencies between neighboring words
• C. Short-term dependencies within one clause
• D. Long-range dependencies, analogies, and nuanced meaning

Answer: (D)

Long-range dependencies are about connections between words or ideas that are far apart in the text, not just immediate neighbors. To understand an analogy or a nuanced meaning, you often need to link distant context—what was said earlier with what comes later—and maintain coherence across larger spans. That’s why describing long-range dependencies as involving broader, distant connections and deeper meaning best captures what language models must handle. The other statements focus on local or short-range patterns (neighboring words or within a single clause) or push the idea to the entire document, which isn’t the standard scope of long-range dependencies.

Long-range dependencies are about connections between words or ideas that are far apart in the text, not just immediate neighbors. To understand an analogy or a nuanced meaning, you often need to link distant context—what was said earlier with what comes later—and maintain coherence across larger spans. That’s why describing long-range dependencies as involving broader, distant connections and deeper meaning best captures what language models must handle. The other statements focus on local or short-range patterns (neighboring words or within a single clause) or push the idea to the entire document, which isn’t the standard scope of long-range dependencies.