How do context-dependent embeddings differ from static embeddings?

How do context-dependent embeddings differ from static embeddings?

• A. They are dynamic and adapt to context; static remain fixed
• B. They are always random vectors
• C. They ignore semantics and only encode syntax
• D. They require less computational resources

Answer: (A)

Context-dependent embeddings produce representations that adapt to surrounding text, changing the vector for a word based on its sentence or neighboring words. This lets the model distinguish different senses of a word and capture both meaning and grammatical role in context. Static embeddings, by contrast, map each word to a single fixed vector regardless of where it appears, so they can’t disambiguate meaning or adjust to syntax in a particular sentence. Because they’re generated by a model that reads the whole context, contextualized embeddings usually reflect semantics and syntax as used in that specific instance. They’re more powerful for understanding language where meaning shifts with context (like “bank” as a financial institution vs. the riverbank). However, this comes with higher computational cost, since you must run a neural network to obtain each word’s representation, rather than simply looking up a fixed vector. The other statements don’t fit: context-dependent embeddings aren’t random vectors, they do capture nuanced meaning from context rather than ignoring semantics, and they generally require more resources rather than less.

Context-dependent embeddings produce representations that adapt to surrounding text, changing the vector for a word based on its sentence or neighboring words. This lets the model distinguish different senses of a word and capture both meaning and grammatical role in context. Static embeddings, by contrast, map each word to a single fixed vector regardless of where it appears, so they can’t disambiguate meaning or adjust to syntax in a particular sentence.

Because they’re generated by a model that reads the whole context, contextualized embeddings usually reflect semantics and syntax as used in that specific instance. They’re more powerful for understanding language where meaning shifts with context (like “bank” as a financial institution vs. the riverbank). However, this comes with higher computational cost, since you must run a neural network to obtain each word’s representation, rather than simply looking up a fixed vector.

The other statements don’t fit: context-dependent embeddings aren’t random vectors, they do capture nuanced meaning from context rather than ignoring semantics, and they generally require more resources rather than less.