This is (or was) deliberate, with the rationale being that it makes O(n) blocks of code trivial to identify in review.
Whether or not you buy that argument is a separate thing, of course.
How is such identification made easier by manually writing out a `for` loop applying a function foo to each element rather than writing `map foo myCollection`?
Would anyone expect such a developer to read and parse several more lines of code more reliably, in order to understand the algorithmic complexity? Seems unwarranted to me...
In programming, if you miss an important detail the repercussions can be high. In a million line codebase every idiom that's slightly more complex than it needs to be is going to result in dozens of additional bugs, sheerly because of the increased surface area for making mistakes with that idiom.
I think the clue to your thinking, for me, is in your description of the `map` HOF as "slightly more complex". Having years of experience with both paradigms, I've found that grokking a call to one of these fundamental building blocks (map, filter, reduce, fold, etc) is nearly instantaneous. We've all experienced reading prose where the author was excessively verbose when the same point could have been made succinctly. It feels the same way reading for loops once you get over the learning curve of these very basic functional constructs. You have to keep repeating that boilerplate endlessly and it's very tedious to keep writing and reading it.
Yeah, and it's much easier to make mistake with a loop than with a map or filter.
Memory allocation is so expensive that making that copy is often more expensive that calling `foo` on each element.
Sometimes making a copy is exactly what you need and there's no way around that cost (but hold that thought).
But I've also seen `sum map foo myCollection` so many times (especially in JavaScript).
Here you have a short, neat but also extremely wasteful way of doing things. I see it so frequently that I assume that many people are unaware of this cost (or completely disregard performance concerns).
If you were to write this imperatively, it would be obvious that you're making a copy and maybe you would stop and re-think your approach.
But there's more.
If you're paying attention to performance, an easy way to improve performance of `map foo myCollection` in e.g. Go is to pre-allocate the resulting array.
For large arrays, this avoid re-allocating of underlying memory over and over again, which is the best a `map` can do.
In imperative code those costs are more visible. When you have to type that code that does memory allocation, you suddenly realize that the equivalent of `map` is extremely wasteful.
Language support for Transducers can fix this, you can compose functions like map / filter / reduce over a collection and only hit each item once.
numbers.iter().map(|n| { ... }).sum();
Your points about efficiency are a separate topic entirely from the original claim that manually writing out an imperative solution makes it easier to see the algorithmic complexity. That was a surprising claim to me because, in my experience, if I understand what some HOF is doing, reading the code is even easier because there is less of it to wade through (and mental exhaustion doesn't make one easier to read vs the other).
> In every mainstream language I can think of `map foo myCollection` creates an intermediary map
You need to build up the final result, sure. Not an intermediary map but whatever structure (or functor) you're mapping over. That is the whole point of immutable data structures. Also when you're using persistent data structures, which all modern FP languages do, the cost of constructing the result can be far less than what you expect, especially if the result is lazy and you need only some of the results. There is a cost to immutability and if it's unbearable in some situation, fall back to in-place mutation but the semantics of these two approaches are definitely not the same.
> But I've also seen `sum map foo myCollection` so many times
Yeah... that should be a fold (reduce, whatever). :)
Whereas, greping for .map(), .filter(), .reduce() wouldn't?
Like how
for (int i = 0; i < strlen(str); i++) { ... }
for (int i = 0; i < strlen(str); i++) { ... }
for (int i = 0; i < str_length; i++) { ... }
After all, if you call functions inside the body, all bets are off as to what complexity the loop means anyway...