In learning the classifier for word embeddings, what is the general objective?

In learning the classifier for word embeddings, what is the general objective?

• A. Minimize the similarity of all word pairs.
• B. Increase similarity for positive (target, context) pairs and decrease similarity for negative pairs.
• C. Only maximize the frequency of words in the vocabulary.
• D. Predict the next document in a sequence.

Answer: (B)

The main idea is to train the embedding system so that words that truly relate to each other—in their usual contexts—end up with higher similarity, while word pairs that don’t share a real relationship end up with lower similarity. In practice, this means the model emphasizes positive pairs (a target word and its actual context) by increasing their similarity, and it emphasizes negative pairs (random or non-contextual words) by decreasing their similarity. This contrastive objective lets the vector representations capture meaningful relationships: the geometry of the embeddings reflects which words tend to appear together. Concretely, the model learns to assign a higher likelihood to positive pairs than to negative ones, often by using a loss that rewards rising similarity for true pairs and rewards falling similarity for non-contextual pairs (via mechanisms like negative sampling). This is why increasing similarity for positive pairs and decreasing similarity for negative pairs is the appropriate objective. The other ideas don’t fit as well. Minimizing similarity for all word pairs would erase the useful structure that comes from actual language use. Simply maximizing word frequency ignores how words relate to one another. Predicting the next document is a different task that operates at the document level rather than focusing on word-by-word context relationships.

The main idea is to train the embedding system so that words that truly relate to each other—in their usual contexts—end up with higher similarity, while word pairs that don’t share a real relationship end up with lower similarity. In practice, this means the model emphasizes positive pairs (a target word and its actual context) by increasing their similarity, and it emphasizes negative pairs (random or non-contextual words) by decreasing their similarity. This contrastive objective lets the vector representations capture meaningful relationships: the geometry of the embeddings reflects which words tend to appear together.

Concretely, the model learns to assign a higher likelihood to positive pairs than to negative ones, often by using a loss that rewards rising similarity for true pairs and rewards falling similarity for non-contextual pairs (via mechanisms like negative sampling). This is why increasing similarity for positive pairs and decreasing similarity for negative pairs is the appropriate objective.

The other ideas don’t fit as well. Minimizing similarity for all word pairs would erase the useful structure that comes from actual language use. Simply maximizing word frequency ignores how words relate to one another. Predicting the next document is a different task that operates at the document level rather than focusing on word-by-word context relationships.