How to win a beer with high-dimensional statistics
A friendly bet with a colleague leads to a surprising discovery: by searching through 25,000 word2vec embeddings, one can find ten seemingly unrelated words that form a perfect circle in PCA space and a circulant Gram matrix, mimicking the famous month embeddings. This spurious pattern, found via iterative search, highlights the danger of overinterpreting geometric structures in high-dimensional data and has implications for interpretability research.
That’s circular. You can just find other sets of random-looking words that form circles!
- hhjinks
Does the mapping depend on the chose set, or could you just map all 25k words and pick 10 points that form a circle? From the process, they seemingly replaced single words in the set that didn't fit, which makes it sound like the position of the remaining words didn't change much from changing the set.
- mph1027
The best bar bets are the ones where you're technically not cheating, just abusing linear algebra.
- ViscountPenguin
A fun step-up would be to see if you can find an infinity shape by repeating the process with a set of points which are maximally far away from all points but 1 on the existing circle.