Maybe "we" should not be writing our server code using a terrible ecosystem? Just an idea.
Can you provide any examples where Node does not provide a good ecosystem for backend programming? Genuinely curious.
Node is missing very basic features, leading to a million tiny modules like leftpad. That's annoying by itself because there's overhead in finding the best package for everything.
The other consequences of that issue are much worse. You have lots of easy opportunities for misleadingly-named packages with malware in them. Package hijacking is easier to miss. Security updates, if packages even get them, have to be applied seemingly every hour.
It's just a major problem to have a simple project with 500 dependencies.
And that doesn't even get me started on issues with the mess that module importing was/is.
I just find it hard (however biased I may be) that we can just say "Node has all these problems, let's throw it out" when it's helped lower the barrier to entry for new people programming at-large.
I'd argue that JS raises the barrier because lots of beginners now think it's normal to have a bizarre type system where null is an object, to have a complex Webpack/Babel setup just to use new-ish language features, and to have to constantly hit "run" to debug code.
Most previous-gen web languages also have lots of gotchas, but something like Go, Kotlin, or Dart would be a much more sane beginner language.
This seems to assume that it would be impossible to design a better package manager which also provides a low barrier to entry.
Bear in mind, however, that all you actually need to write javascript and learn programming with it at a basic level is a text editor. The low barrier that NPM provides is for publishing to NPM, which needn't be synonymous with "javascript development."
Your comment also seems to assume that the barrier to entry everywhere else is prohibitively high, but plenty of people still learn with Ruby or Python or other languages. There are whole industries for teaching new programmers.
By choosing node you are choosing single-threaded async IO as your primitive. You have to be ok without a threading model that supports shared memory (v8 objects don't pass between isolates, only certain types pass between web workers). And what's more, you have to be certain that your problem will fit these choices in the future.
Is it ever a good idea to hamstring a backend service with one narrow concurrency model? By comparison, golang offers you all of those things. If you need to write a backend service that performs well and can respond well to changing requirements, you have a much better chance of doing it with golang than node. I just can't promise any individual developer that they will enjoy writing the golang service more than the node one.
I don't want you to think I'm saying golang is better than node, we haven't even started to talk about the module system. I'm just saying fundamentally it is more flexible and can do everything that node does with at least the same level of performance, and generally it is "faster". Most things out of the box are faster. Benchmarking and performance is in the DNA of the go community and this isn't true for the node community.
In node is it very easy to fork processes and have them communicate. Scripting languages for serving HTML and JSON aren't the place for threads.
Node now offers sync versions of most IO functions as well.
JS has nice fast runtimes, and crucially the ability to share code between client and server. I think there’s potential (even without threads) to have a JS server architecture that does not require the entirety of a request/response to be processed in a single thread but could break it up so it could be processed by a pool of workers at IO event boundaries. Either as one isolate per request or explicitly passing state for IO event callbacks.
If your JSON payloads are large, you can't have a forked process do the deserialization of it (because you would need to re-serialize across the communication boundary, which defeats the point).
I think it would be reasonable to have immutable data structures shared with web workers. I also think it would be reasonable to make it possible to pass complex objects (all primitive types supported by JSON) between workers without a serialization step.
So many problems have spawned out of the desire to thread everywhere that the safety of single-threadedness is a feature not a bug.
I write a lot of stuff in Rust these days, where (unless you use unsafe) it doesn't even let you do anything that would break under multi-threading, even if you don't use it.
As a result, it was incredibly easy to write a light-weight crawler that delivery results into a main worker queue and the data is processed by <cpu_cores> threads. The crawler largely uses async for IO advantages but the worker threads rely heavily on data crunching where your throughput is limited by CPU not by IO and async suffers a bit.
And since crawler and worker are on the same process, the latency between a page being grabbed and starting to be processed is incredibly low, any kind of networking would likely cut throughput by a factor of 10 or more. Not even mentioning having to serialize the data instead of relying on references to get read-only copies and only making minimal copies on the stack.
My point is Node isn't as bad as a server-side language as people claim. There are certain use cases where you can leverage Node's strengths and use another language somewhere else where its shines more.
I use Node heavily in my backend stack and have been extremely pleased with it's performance and library ecosystem.
Futures and friends are in nightly and soon in Stable, which enable you to mix the best of both world; Multithreaded and Async.
Once you get used to Rust, you fight the borrow checker a lot less.
I use Rust all the time yet I'll still use Node for most HTTP-based applications and tools I start, and not because I'm an idiot or beginner, as many comments here will suggest. They are completely different languages and ecosystems and workflows that satisfy different goals.
The "spinning up multiple workers" is what's wrong with that. Specifically a language/runtime that does support multi-processing handles it for you; ie gomaxprocs. Another example would be servers like apache forking off several workers during start up (depending on the mpm module). With node this has to be managed manually, as you describe, with Docker or whatever.
With a node async server, you're generally not cpu bound, so there's really no reason to try to utilize all the cores on the machine. Except if you are cpu bound, then you'd want to do that.
1) you can use a launcher like pm2 to auto-fork your node app
2) you can horizontally scale (put your servers behind a load-balancer)
if you do need multithreading for computations, you are right that node probably isn't a very good choice, but you have some options:
1) use worker threads
2) offload your intensive computations to a separate process written in a different language.
Do you know about copy on write?
Right, so why wouldn't you? Node has a "fork" where you can give it a path to a script to run. Yeah it has to JIT it, but the V8 and Node part isn't duplicated.
Plus you can more easily do shmem in those languages than JS which gives you another performance advantage over pipes.
If your counter-suggestion is Rust, have you considered.unwrap() that.unwrap() thinking.unwrap() about.unwrap() managing.unwrap() memory.unwrap() all the time might not be the wisest choice and that a GC is a useful abstraction to have?
Have you considered that avoiding race conditions isn't hard and you can't avoid having to avoid them even in JS?
Regarding whether its hard or not, yes, I have some first-hand (non-Rust) experience.
On second thought I agree with your original point. Node isn't great for CPU-efficient + multithreaded. But
* most platforms aren't (either not great because unsafe, non-cpu-efficient or both) * most of the time its not what you really need.
So a good question would be, what would you use shared memory parallelism for in typical back-end programming?
But all shared memory problems can, without much difficulty, be solved by using a mutex. It might not be best performance but you'll likely not need that much performance in the first place (after all, you're coming from the JS ecosystem).
Memory barriers aren't particularly hard to understand either, I've written some systems using them that ran very stable. Same for lockless or atomic or reentrant algorithms. It's all very fun and doesn't take that much skill if you're willing to read into it.
So anyway... what would you use shared memory parallelism for in typical back-end programming?
There is plenty of reasons for shared memory in back-ends, workers queues with zero-copy messaging would be one example that a lot of applications can benefit from.
NPM's module selection is so wide that it's sometimes hard to find the best one. Too many choices! I would rather have that problem than lack of any choice.
Overall I'd guess to get a functional product, NPM's modules saved me about 50% of my time.
Personally I love C#, but can't use that for backend because of the lack of module ecosystem.
If you're talking about async I/O, TCL (among others) has offered that for more than 20 years (from AOLServer on), and in a much better-designed language than JS.
While Tcl is running as a server in an event loop, if an error is thrown while processing an event it is reported via a background error API, and the server keeps going.
You can go a step further and fully isolate "crash-assuming" code from "non-crash-assuming" (promise based code): https://gist.github.com/spion/ed8deb7a3b4add0a6d727dc78fe635...
Prototypical inheritance isn't a "deep" design mistake in so far that its not actually used as such (its mostly used as classical inheritance). Classical inheritance could be argued a design mistake, but one that most mainstream languages make, so I wouldn't go as far as to call it a deep one.
`this` but also hoisting - variables get hoisted out of blocks where they are declared, so their scope is not what it looks like lexically.
> Prototypical inheritance isn't a "deep" design mistake in so far that its not actually used as such (its mostly used as classical inheritance).
I'd argue that even if it's not being used it still adds complexity when reading or debugging.
`this` is not non-lexical scope, its argument passing. Its the most confusing wart in JS but its actually not a fundamental design flaw, just a syntactic wart: https://gist.github.com/spion/7180482
> I'd argue that even if it's not being used it still adds complexity when reading or debugging.
I disagree, it barely adds a couple of bits of noise here and there. Definitely not a deep flaw.
The worst JS flaws are in its anemic standard library and the pre-promises backend standard library included with node. They are overdue for a refresh.
A decent URL parsing/matching lib not tied to a full web framework ? Disclosure - haven't done serious node work in a while but ~2 years ago when I was building something in with it all the published libs for this were complete garbage - looking at the code for the packages made me realize what sort of crap gets published to NPM and then actually used/referenced by community.
I think it's better than PHP (took on a PHP integration job to help out a friend in trouble recently - that was going back to the dark ages with package management and module system). But it's the same tier in my eyes - filled with clueless newbies trying to lead other newbies - a community of blind leading the blind.
In my eyes a sane/mature language like Python beats Node any day of the week quality/productivity wise and so does stuff built on top of JVM/CLR. As much as C#/Java devs like to obfuscate code with useless patterns I'd take a random C#/Java package over a random Node package any time - at least the developer managed to convince the type system to compile his code - wouldn't say that the authors could do the same for some of the NPM packages I saw.
https://nodejs.org/api/url.html#url_url_parse_urlstring_pars...
My impression is that it's soo easy to publish to NPM and the ecosystem is newbie friendly - this is good but it also creates a lot of noise and popularity/usage is not a valid indicator of reliability/quality (compared to other platforms). And because the core was so barebones you need to use third party solutions a lot more often so it compounds the issues.