Programmers Love map and filter, but Can't Stand reduce
Anecdotally, programmers dislike "reduce"
I use map and filter constantly and rarely get pushback in code review. But whenever I submit a patch containing reduce, someone says it's hard to read. I see reduce far less often than map, filter, or some. I don't know why, though I have theories: it's harder to read, less familiar, sometimes slower, and less elegant in JavaScript, Python, and Swift. Curiously, I never heard this complaint as a Clojure developer.
Often, when I've submitted a patch with reduce inside, I get a comment like, "this part is hard to read."
- rspeele
Map and filter usually have only one arg and if they have 2, the 2nd is almost always a 0-based index. They look identical in most languages, even when Microsoft chooses to call them Select and Where.
Reduce has an accumulator and a 2-arg function and languages are not very consistent amongst each other as to whether it's reduce(initial_acc, callback(acc, elem)) or reduce(callback(acc, elem), initial_acc) or reduce(callback(elem, acc), initial_acc) or what.
Hard to remember. Also some languages have a version of reduce that doesn't take an initial accumulator at all, which is just a footgun waiting for you to hit an empty collection. Also ALSO, the accumulator can easily become awkward in languages that don't support anonymous types or don't support easy mutation of an anonymous type record. Which is most of them!
- chubot
Related to the point about worse performance, I'm pretty sure I was there when reduce was "banished" from Python 3 -- demoted to functools.reduce(), instead of the builtin reduce() in Python 2
The story is that sometime in 2006 or 2007, Guido van Rossum was debugging why a web page in Google's internal code review tool (which he wrote) was taking 30+ seconds to render.
This is basically a "production" incident, since thousands of Google engineers relied on the tool. Requests like this were probably tying up threads and exhausting thread pools, perhaps
Eventually it was tracked down to a line wrapping algorithm written with reduce(). I don't think he wrote it -- it may have come in through a dependency. As many know, reduce() is basically:
s1 + s2
s1 + s2 + s3
s1 + s2 + s3 + s4
...
And that's O(n^2) when s_i are strings. And I think it showed up if you viewed a 5000+ line diff, or a 5000+ line file. (Newer programs like Github also suffer here)
I believe, in Python at that time, += was already optimized to avoid this (just like essentially all JS VMs are). Or you can use the idiom of append() to list and join() after.
But reduce() basically forces the inefficient implementation, and I'm sure this is still true in Python 3.
---
So basically Guido spent a long time debugging a performance problem related to reduce(), and made the decision to eject it, to help users avoid "footguns". I was his officemate at the time, so I recall this, but I wasn't i […]
- el_oni
Ive only used reduce at work half a dozen times and it does raise an eyebrow each time.
But for unioning a bunch of spark dataframes together i think
df = reduce(DataFrame.union, list_of_dfs)
is much nicer than
df, *rest = list_of_dfs
for other in rest:
df = df.union(other)
People just get a bit funny, especially now you have to import it from functools
- dochne
I like it, but it is by far the most ungainly of the three with the most footguns in it's usage.
While not as functionally pure, I always appreciate the Ruby each_with_object https://ruby-doc.org/3.4.1/Enumerable.html#method-i-each_wit... as a more pleasant interface for it.
- adverbly
Speaking as someone who often tries to reduce my use of reduce by replacing it with map and filter where possible, for me, falling back to reduce is analogous to falling back to a while loop or a for loop: I avoid it if I can.
The problem with reduce is that it can do so much, and therefore it is less clear when reading it quickly what it might be doing.