I think the best way to start talking about embeddings is to propose a Lab about it. In the next articles about Embedding I will explain in detail what are, but in this I will explain an introduction of the meaning with a Lab that you can do with the properly code.
1. What are word embeddings?
- Models cannot use raw text; they need numerical representations.
- A word embedding is a dense vector that captures semantic meaning.
- Embeddings learn relationships such as:
- “happy” is close to “joy”
- “king – man + woman ≈ queen”
Embeddings can be:
- Trained from scratch inside your model
- Loaded from pretrained sources (like GloVe or Word2Vec)Before using Embeddings you should:
2. Data preparation using the IMDB dataset
Example. Using IMDB movie reviews to perform sentiment classification.
- Tokenize the text
Lowercase all words
Lemmatize
Remove stopwords
Build a vocabulary
Convert sentences into sequences of indices
Create DataLoaders for PyTorch
Understand that before embeddings, raw text must be cleaned and turned into tokens.
Next step:
3. Training a neural network with an embedding layer
The model architecture is could be simple:
- Embedding layer : turns word indices into vectors
- Global Max Pooling : extracts the most important features
- Linear layer : predicts positive or negative sentiment
PyTorch Lightning could be used to simplify training and evaluation loops.
Main idea:
- The embedding layer is the first step of the model.
The model learns the meaning of words as it learns to classify sentiment.
Next step:
4. Visualizing the learned embeddings
After training, you:
- Extract the embedding matrix
- Reduce dimensionality using PCA
- Plot the top frequent words in 2D
Goal:
To visually inspect whether semantically similar words form clusters.
You might expect:
- “good”, “great”, “amazing” close together
- “bad”, “awful”, “terrible” forming another cluster
Next step:
5. Measuring similarity between words
Using Gensim, you can:
- Compute cosine similarity
- Find the most similar words
- Test analogies like:
- king − man + woman → queen
The idea:
Good embeddings capture real semantic structures.
Next step:
6. Comparing your embeddings with pretrained ones
Once you test your own embeddings (which are limited), you can:
- Load GloVe embeddings
Convert them to Word2Vec format
Repeat similarity and analogy tests
Purpose:
Highlight the difference in quality between self‑trained and pretrained embeddings.
Usually:
- Your embeddings : work for basic sentiment, but weak overall
- GloVe : much better semantic reasoning
Next step:
7. Re‑training the model using pretrained embeddings
You could:
- Replace the embedding layer with GloVe vectors
- Freeze the embeddings
- Train the classifier again
Key lesson:
Using pretrained embeddings often yields better accuracy, especially with limited data.
Main ideas the lab wants you to take away
- Embeddings turn words into vectors with semantic meaning.
- Good text preprocessing is essential.
- Training embeddings from scratch works but needs lots of data.
- Pretrained embeddings (GloVe, Word2Vec) are usually far superior.
- Embeddings support similarity and analogy reasoning.
- Visualization reveals semantic clusters.
- Integrating embeddings into neural networks boosts NLP performance.
Bonus 1
You can use AI to transform your handwritten words into digital words to use on the digital environment.
@Yolanda Muriel 
yolandamuriel.com 