Access link for the visualizer: https://word-embeddings.wbkolleg.unibe.ch/
Repository: https://gitlab.com/JChataigne/embeddings-visualiser
As an engineering student learning about neural networks in 2017, before Transformers were introduced, one of the most intriguing and funny things I discovered was that one could apply arithmetic to word embeddings and it actually seemed to make sense. To quote the paper that introduced this notion:
Somewhat surprisingly, it was found that similarity of word representations goes beyond simple syntactic regularities. Using a word offset technique where simple algebraic operations are performed on the word vectors, it was shown for example that vector(”King”) – vector(”Man”) + vector(”Woman”) results in a vector that is closest to the vector representation of the word Queen.
In this paper, we try to maximize accuracy of these vector operations by developing new model architectures that preserve the linear regularities among words.
This is a nice, intuitive way to show how vectors can capture semantics. This particular example (“King” – “Man” + “Woman” ≈ “Queen”) is still very often used in introductory courses.
But when taking, and later teaching such classes, I always found it disappointing that the examples were fixed, because (1) they could have been cherry-picked to make the phenomenon seem more consistent and (2) I couldn’t explore more examples. The only way to see more examples was to run the code yourself, which requires either more time or more skill than people typically have.
So I took a few days to build a word embedding visualizer, and made it public. You can now go to this link to try it.
I have also made a version to visualize document embeddings, but I don’t have a server to host that one at the moment. You can still run it locally since the code is public.