Guava's whitespace check hides a perfect hash function
How hard is it to check for whitespace?
A look at the surprisingly complex whitespace matcher inside Google's Guava library. Unicode defines 25 whitespace characters, but Java, Python, and JavaScript each disagree on which ones count — Java excludes NO-BREAK SPACE, Python adds FILE SEPARATOR, and JavaScript drops NEXT LINE while accepting ZERO WIDTH NO-BREAK SPACE. Guava skips a linear scan and uses a perfect hash function with a mysterious multiplier to test membership in constant time.
Despite the name, it's a format character that can indicate a word or line break opportunity without adding a visible gap.
- Whitespace
I was here before the first character was carved into stone and I will be here when the last photon is emitted.
I'm also available via: @gmail.com
(please don't write code with `@whitespace` as I get a lot of GitHub notifications)
- pixelbeat__
I think horizontal white spacing should be distinguished, as that's what most care about with splitting etc.
Have a look at how GNU coreutils classifies each type of spacing (in a cross platform consistent way) at https://github.com/coreutils/coreutils/blob/master/src%2Fsys...
- smallstepforman
With bitmasks, if they just assumed bit 5 (whatever) as zero=whitespace, they would have saved billions of cpu cycles per device per day. With 32 bit unicode, that gives you 32 unique whitespace chars.
Should have got guys that grew up with the 8-bit micros to write up a spec.
- exacube
If you want to learn how that magic multiplier value was derived, look up how to derive Perfect Hash Functions for a known set of values.
I'm not sure which method was used for this specific set of values, but from googling it's seen advances in the past decade.
- efilife
This code is very simple but I still don't really get it. Why is the hashing needed and how is it faster than a regex? Can someone smarter than me explain?