Hacker News Reader

Comment by raverbashing | Hacker News Reader

The more I read about the js ecosystem, the more it seems the inmates are running the asylum

It's a long time since js needed a good stdlib.

Everybody was using this package but apparently no one but an adversarial player stepped up to actually maintain it.

(And don't get me started on "let's always get the latest version of the package")

> You want to download thousands of lines of useful, but random, code from the internet, for free, run it in a production web server, or worse, your user’s machine, trust it with your paying users’ data and reap that sweet dough. We all do. But then you can’t be bothered to check the license, understand the software you are running and still want to blame the people who make your business a possibility when mistakes happen, while giving them nothing for it? This is both incompetence and entitlement.

Not surprising. Not surprising in the least. "Oh wow somebody 0wned the package I needed". Maybe because js projects have an order of magnitude more dependencies than a Python/Java/Go, etc project. Maybe because in the extreme opposite of NIH, people feel the need to import a module for every small thing they want to do? "Stack overflow programming", "how do I add 2 numbers using React, is there a module for that!?!?!"

View on HN

The cynic in me is saying that JS is the new PHP. Ironically, both are very capable and suitable languages, but the sheer freedom they offer attracts crowds of wannabe developers who think they are gods because they know how to shave off 10% of execution time in some small part of their overengineered and unmaintainable mess. And give talks about it, write blogs and post clever twits.

However developing robust software is possible in JS, the same way it was possible in PHP. You just need to take some time to vet dependencies a bit better and not use every fad when it surfaces. Common sense and experience help a lot. But it's incredible what can be built with modern toolchains and how maintainable it can be. Don't let the anti-JS feeling scare you from trying React / Vue /..., just don't forget that state management, separation of concerns and similar concepts are still valid.

> the sheer freedom they offer attracts crowds of wannabe developers who think they are gods because they know how to shave off 10% of execution time in some small part of their overengineered and unmaintainable mess. And give talks about it, write blogs and post clever twits

Honestly, I think JavaScript just has a lot of new developers who have never been exposed to concepts of software development such as dependency management, cultivating trust, and how (free and) open source works. I think the kind of developer you're describing is actually a minority.

This is my thought as well. "I have creative experience and/or student loans but I don't know what to do for work so I'll take a Javascript course and become a web dev."

It's wonderful that programming is becoming a kind of "equalizer" on the job market. But without supervision from software engineers (i.e. people with the experience and aptitude to design software properly) you end up with a mess.

The myriad of small snippets of JS encapsulated as npm modules is best compared to the alternative of copy-pasting solutions from websites like stack overflow. Instead of lots of copies of solutions to common problems, now everyone shares the same implementation. This obviously has the centralization weaknesses but also has the centralization strengths.

The fact that there's no panacea is a fact of life. There's a tradeoff, but it is not reflective of the people in the JS community.

I'm not talking specifically about having dependencies instead of copying and pasting utility functions.

In both cases you are expected to at least sort of understand the code you are copying/importing.

In Java world those snippets tend to combine into bigger libraries. Apache commons and Google guava are the most known ones and their reputation is established. I don't understand why JS world does not do something similar. Some essential-utils from AiRbNb or whatever JavaScript analog of Google. And if there's still need for some tiny function, just implement it and move on! I can understand that JavaScript favors small download size, but modern tools should be able to remove unused code, so it shouldn't be an issue either.

That's exactly what happened to PHP, with the difference that PHP core developers were also not exposed to good community and development practices before starting their project, while there exist JS core projects in good shape.

There exist PHP projects in good shape as well... and remember JS existed before Node and NPM, and a lot of javascript is still written entirely outside of that ecosystem.

If anything, taking for granted that the JS ecosystem has a higher caliber of developers in it than PHP only serves to make the JS ecosystem look worse in contrast.

There exist PHP projects in good shape now. Even the core developers learned their lessons and are seeking a good language now. PHP is maturing, and may become a good language at some point... if the world hasn't moved on that this point so it become an old, brittle, powerless one before it has a chance to put its flexibility to good use.

Javascript started from a much better position. You simply couldn't choose a sane toolset for working in PHP on the begining, but while there is no great choice for JS, there exist some that don't suck.

Javascript started in the same position as PHP - a wild west community of amateurs writing their own code by hand... then JQuery and other frameworks came along and more amateurs were writing their own code mostly by hand around those frameworks. Neither language or community started off on a good footing.

PHP has many built in functionality and modules that are packaged in Linux distros,in comparison JS is very bare bones and JS ecosystem encourages the reuse of 5 lines helper function from npm.

Edit, a culture difference example:

a dev wanted to get the video length in a mp4, so he installed a npm package but it did not worked, the issue was that ffmpeg was not setup in the PATH so he asked me to solve it, I asked to use the absolute path of the ffmpeg binary but that was not an option n the package.

In the end I checked the package he found and used and I showed that it just called ffmpeg from the command line and if we just do that we can pass the exact options we want and not have to install a third party package.

As a PHP dev when I have to perform something is to find if there is a Linux CLI app that does this, install the app on the server and call it (like ffpmeg, wkhtml2pdf, an epub to mobi app, etc) I can trust a CLI app that is packaged in Linux then a random package.

PHP has composer and we use that for third party integration like Dropbox,Facebook,Amazon have official packages that we can grab and use.

>As a PHP dev when I have to perform something is to find if there is a Linux CLI app that does this, install the app on the server and call it (like ffpmeg, wkhtml2pdf, an epub to mobi app, etc) I can trust a CLI app that is packaged in Linux then a random package.

This is only better because your distribution has a curated set of packages with trusted maintainers that (in theory) read through the source code to make sure it's not doing anything malicious.

also because most of those libraries are just thin wrappers of other C libraries, which have already been vetted and most of the time are already part of the OS. Javascript reinvents everything .

Is not JS itself, but the NPM culture where you instead of writing a few lines of code you include someone else package.

So in my example the job was to get the video length, the dev tried to use https://github.com/eugeneware/ffprobe if it all would have worked would have been faster but at the cost of not knowing what is happening under the hood and depending on this package that also seems to depend on 3 others(not sure if those also depend on others)

I believe the culture started from people who didnt trust themselves to write good code. Also, the lack of preexisting libraries compatible with the async model led to de novo implementations of many things that are covered by wrappers in other languages.

I agree, but I think you can call a CLI app using node without any need for at least 4 extra packages.

IMO browser vendors should check some statistics , see what helper libraries are most popular and put those functions in the browser, similar for node, see what is popular and install that. Also the culture needs to change, when someone wants to explain how to create a thumbnail using image magic or other task that is just a wrapper then they should show the code on how to do it and avoid creating a new NPM package.

I assume but I could be wrong that some developers think that having published npm packages, github repos, blog posts are helpful n getting better jobs so the culture of "CV programming" appeared where a lot of packages,side projects and blogs are created but the quality suffers.

I was hoping Gist would become this, a place for sharing patterns and implementations so that devs would learn what they're doing rather than just blindly playing npm lego. But it doesn't seem to have happened.

I think the biggest problem with the JS community is that the mindset is always "implement first, understand second", rather than the other way around. It's ok to use libraries but you should at least have a think about how you would implement it yourself first, and understand at a basic level what's happening under the hood.

Whenever I've brought up this idea the pushback in the real world is absolutely insane. Here on HN people are more receptive to the idea, but in most companies the mindset is just "shortest path to implementation at all costs".

The ironic thing is that bigger companies, rather than giving their devs the time to learn, will force them to spend 80% of their effort on processes and unit testing to try and account for the shit code being produced in the other 20% of their time.

Which is, as you say, better.

Not only that but I use the CLI app directly then using a package that wraps it and hides what is happening from you. Plus you will not have a surprise that an update (on an LTS server distro) removes your dependency, replaces it with malware or drops a feature.

I noticed that my colleagues that never used Linux are not familiar with the CLI tools and the power this tools have so they are not trained to think "hey we can use this CLI tools and we can solve our problem" but they think let's search SO, GitHub, or NPM for a solution on how to do X with node/PHP etc.

I think JS has a better standing than PHP, because it was standardised and got world class devs early on.

But in all other aspects I think JS is the new PHP.

javascript isn't even usable out-of-the-box without a few (hundreds) libraries to make it comfortable to use. I disagree, PHP is nothing like it, its standard modules library is chock-full of stuff that is fast, useful and based on existing C/C++ libraries or standards.

but sure if one wants want to classify incompetent developers in the PHP bucket, then everything is "the new PHP"

Fast and useful stuff like mysql_escape_string?

That's been deprecated for years, removed from current PHP and was just a wrapper around an existing C library to begin with. If you want to blame the current language for its former vulnerabilities, fine, but comparing apples to apples would mean looking at all of the terrible ways in which javascript has been integrated into the browser and OS over the years, particularly with Internet Explorer.

And I can guarantee you, from personal experience, that PHP is not the only language in which string concatenation with variables is the most common means of writing SQL queries.

I’m not defending js. On the contrary the two languages are similar in that early on a lot of people had a negative first impression because of some mind bogglingly stupid decisions made by the initial implementators. This lead to feedback loops where those that were repulsed by those negative impressions moved on to other languages while those that didn’t care stuck around. In both cases that lead to cowboy cultures developing.

It is true that in both cases, eventually top notch engineers got involved in the projects’ core and cleaned things up to the extent possible. But it’s a heck of a lot easier to improve a spec than a culture.

Yet, this ancient library did not have serious issues. It was not deprecated because someone hacked it or it was even bad, but because it had not been maintained for many years. The reason was probably that it did not need maintainance, it just worked, but programmers often used it in the wrong way. In fact, the removal from PHP has caused some very small and old projects to break, and IMHO php should not have removed it, for the sake of compatibility.

Also, it is much faster than the suggested alternative (pdo).

It did have serious security issues. That’s why not long after there was another function added named mysql_real_escape_string.

No, that's not true. That function name was copied directly from the MySQL C API. It was not added by PHP.

https://dev.mysql.com/doc/refman/5.5/en/mysql-real-escape-st...

“You can use this function safely with your MySQL database queries if and only if you are sure that your database connection is using ASCII, UTF-8, or ISO-8859-* and that the backslash is your database's escape character. If you're not sure, then use mysqli_real_escape_string instead. This function is not safe to use on databases with multi-byte character sets.”

you're focusing on a function with a known limitation, that was superceded 15 years ago and practically everyone used mysql_real since then. That's not the reason why the module was removed though. The reason is that the mysql_* functions are "too low level for noobs".

https://stackoverflow.com/questions/16859477/why-are-phps-my...

That is a function made by MySQL developers [1]. Why don't you blame Sun or Oracle or a Swedish company whose name I don't remember for this?

[1] https://dev.mysql.com/doc/refman/8.0/en/mysql-escape-string....

You did remember it; "MySQL". It was an joint stock company so it is actually MySQL AB, but that postfix is quite similar to LLC, S.A. or A/S. AB literally translates to stock company, I think the english term is joint stock company

> javascript isn't even usable out-of-the-box without a few (hundreds) libraries to make it comfortable to use

What? Start small and vanilla:

   <script type="text/javascript">...
   <?php ...

You always have a chance of drowning in complexity with whatever you use: Webpack, Compose, Laravel, Make, JSX, SSR, apt-get...

This "small and vanilla" lacks 1000s of helpful functions PHP has built-in in its standard library, that's what the parent says.

Right, but PHP alone is also not "usable out-of-the-box" if it needs hundreds of third-party dependencies, whatever as static compiled bundle or linked distro, libc, libmysql, libgd, so on... Comparing generic language with language bundle is unfair - that is my point.

> both are very capable and suitable languages

No, both are terrible, garbage languages, languages that have been whipped into reasonable shape over a decade or two by good programmers forced to use them due to awful monocultures that arose through successions of largely arbitrary events. This reasonable shape means that the most recent versions can be used with relative pleasure if you ignore half of their syntax, clamp massive libraries to them to replace the other half, and assume that your end product will be as fragile as glass.

edit: sorry about the trolly comment (which I make in response to a good comment), but if PHP and javascript aren't bad languages, I wouldn't know how to make a case that any language is a bad language.

> I wouldn't know how to make a case that any language is a bad language

I have a checklist I've been building which is purely based on my experience and is subjective. Appearing on the checklist makes teaching someone else more difficult and looks bad for the language in general.

I believe the ecosystem is part of the language. You can't do much without running into npm if you use node but you can avoid it if you just use JS - are they separate? I treat them so. If you aren't a general purpose language being used as a general purpose language, that's partially the language's (including ecosystem) fault. eg don't make a UI with erlang.

* Language has a toxic ecosystem - node

* Language makes it easy to do the wrong thing - node, js, PHP, lua, Perl (notice no Python)

* Language makes it hard to do the right thing - node, js, PHP, Perl, erlang, Python, Haskell, Java

* Language is based on an esoteric design principle - erlang, Haskell, lua (meta-things)

* Language which is internally inconsistent - node, js (floats, time, etc), PHP (bifs), Java (type system)

etc. I think there's plenty of languages which have problems and few seem to be shoring them up because we still don't have a consensus on how dynamic typing should be implemented, so we build upon the sand of flawed languages and argue about triviality.

No programming language design survives first contact with the programmers. This is also holds on project level if you are 100 persons working on the same project it's very easy to mess it up.

I've seen lots of bad python code, I don't think it's hard to do the wrong thing.

JS is haunted by its past, today we are almost there where we have been in 2011: on the one hand it has this perfect learning curve, being accessible for everybody who mastered basic computer usage but still offering a lot of advanced but optional features. At the same time it has this bad reputation of being unreliable and having bad documentation/code.

I think in 2012 a frameworks got popular that simplified the task of building JS apps that are as complex as apps in C++ or Java. Actually using these frameworks wasn't trivial when they came out. Now that's pretty easy but the deps are crazy. I wonder what happens next, maybe some super advanced package manager will come...

> I wonder what happens next, maybe some super advanced package manager will come...

This is what needs to happen. Javascript isn't a proprietary language, yet the entire ecosystem is currently inextricably linked to the single point of failure in a single company's proprietary database, language extension (which Node.js is) and package manager.

I would hold out hope for Webassembly but I'm certain it's just going to be swallowed up by the Node monster along with everything else.

Javascript is inherently broken and unfixable. With Python, if I find a bug or something that I think should be in the standard library then I can just post on the mailing list and make my case. The core devs might not accept my proposed change, but at least I'm guaranteed my day in court. And if you get ruled against then you can go back and gather more evidence and appeal at least once and probably even twice before you get told to fuck off.

Whereas with Javascript if you find a bug or something you want added to the core library then you need to pay millions of dollars to join ECMA. How can you find competent developers to work in an ecosystem with that kind of governance? You can't, which is why we are where we are now.

Even if with respect to the implementations, if you file a bug with any of the major browser vendors then you'll probably die of old age before the ticket is even triaged, let alone fixed.

I haven't done it, but if you want to propose a change to JavaScript, it looks like there is a process to follow, and it doesn't necessarily involve spending money:

https://github.com/tc39/ecma262/blob/master/CONTRIBUTING.md

> It's a long time since js needed a good stdlib.

To me this illustrates a fundamental problem that JS has to deal with that is virtually unique in the programming language space: multiple browsers across multiple browser versions. It is incredibly difficult to get all browsers to adopt a "standard library" and even then it takes years for all users to adopt those browsers that support the standard library. Even on top of that, not all browsers implement the standard library properly. It really is a nightmare grown out of competing browsers with users that do not update them enough.

> Maybe because js projects have an order of magnitude more dependencies than a Python/Java/Go, etc project.

This is because JS file sizes matter a lot. We have huge libraries like `lodash` which are like a standard library, but nobody wants to use them because they dramatically increase the filesize of the JS bundle. I would rarely want to bring in lodash for a couple utilities, even with treeshaking and the like because it still dramatically increases bundle size. We have pretty excellent datetime libraries that most people hesitate to use -- like moment.js -- because they are huge. So what's the result? A ton of dependencies with very limited scopes because developers do not want to bring in massive libraries that do everything.

Let's flip to Python. Let's say magically you can run python inside a browser starting tomorrow. The second you bring in a library like `numpy` you're looking at a bundle size of 40 MB, and that's just one dependency. In the JS world that is utterly unacceptable. All the languages you mentioned take advantage of the fact that they can download those libraries to the filesystem and forget about it. JS has to download libraries over the wire, it's a completely different game.

What I'm trying to say is that the JS ecosystem didn't invent a bunch of problems to solve or that the people running the ecosystem are script kiddies. There are very unique problems that need to be solved in this ecosystem that make it different, especially when referencing the three languages you mentioned in your post.

With all of these "JS sucks" arguments I see a severe lack in empathy or even remotely trying to understand why JS has the problems it does.

I agree that download size is a big deal in JavaScript, but it still seems rather unfortunate to use download size as a reason for ignoring the good work of well-established teams with good reputations and relying on random individuals instead.

If tree-shaking isn't good enough, it needs to get better. Instead of having lots of individuals creating tiny packages with one function in them, there need to be fewer, broader libraries that are closely watched by larger teams.

>To me this illustrates a fundamental problem that JS has to deal with that is virtually unique in the programming language space: multiple browsers across multiple browser versions. It is incredibly difficult to get all browsers to adopt a "standard library"

You don't need to. The standard library could just be a community curated project (with the help of major browser vendors) that ships as extra code. It could even be on npm.

If the browser has it included, even better, if not, it's referenced the usual way third party dependencies are.

The problem is having a package.json like:

  random lib 1
  ...
  random lib 500

Not having a package.json like:

  function 1 of well curated lib X
  function 2 of well curated lib X
  function 22 of well curated lib Y
  ...
  function 35 of well curated lib Y

where e.g. Y is lodash, or react, and such.

>This is because JS file sizes matter a lot. We have huge libraries like `lodash` which are like a standard library, but nobody wants to use them because they dramatically increase the filesize of the JS bundle. I would rarely want to bring in lodash for a couple utilities, even with treeshaking and the like because it still dramatically increases bundle size. We have pretty excellent datetime libraries that most people hesitate to use -- like moment.js -- because they are huge. So what's the result? A ton of dependencies with very limited scopes because developers do not want to bring in massive libraries that do everything.

That doesn't seem the case either. In major web pages, even from big companies, there are multiple versions of dependencies, even full deps like lodash and co. And people use all kinds of gigantic (web wise) frameworks and third party libs like moment.js with wild abandon.

Besides, even if that was the problem, there's nothing stopping you from having a modular set of libraries (like lodash), that you can cherry pick from the functionality you need and only load that.

The problem the parent mentions is not "JS needs to include big libraries and stop using small dependencies" but JS needs to stop using random small dependencies from here and there.

Using 100 dependencies from all kinds of crappy upstream places (e.g. some crappy leftpad implementation), is different than having a curated set of libraries and loading 100 small dependencies from that.

I had same idea few years ago. Basically, we need a new URL scheme for this, which will be able to uniquely address files from well known sets and be backward compatible with old browsers.

In supported browser, resource will be downloaded once and kept in cache for long period, maybe forever.

For example, `https://well-known.js/common-1.1/lib-1.2.js` . Browser can download just `lib.1.2.32.js` (redirected from lib-1.2.js) every time, or it can download whole `common-1.1` bundle once and keep it in separate long-time cache until it will be deprecated And when it will be deprecated, browser will just revert to old behavior.

That's a good implementation idea!

> The more I read about the js ecosystem, the more it seems the inmates are running the asylum

the issue is NPM. NPM is a bad package manager. A package manager that allows duplicates version of the same dependency is broken to begin with. People complain that fetching react-native results in hundreds of packages installed on their computers. How many freaking dupes? off course nobody is going to audit all that crap. Conflicts should be resolved upstream, which would lead to more stable packages to begin with.

NPM was developed by people who were clueless about package management and now are profiting directly from that shit-show, and even Node.js creator went on recording saying tying it to NPM was a mistake.

Nah. Package manager that does not allow multiple versions of same library is the bad one.

Npm allowing it is a good thing about npm.

This thread and the complains about Node.js broken ecosystem proves you wrong.

It's not the dependencies that cause me concern. It's the dependencies of the dependencies.

This. The direct dependencies are usually quite manageable. As a project developer, I know what they do, why they were chosen, roughly how it works, and have decided that the risk is worth it. This is not the case for sub, subsub... dependencies, and there are soo many. If the tree was flatter due-dilligence would be much easier.

Library authors. Please be extremely conservative in when to pull in dependencies. I will prioritize this higher for projects that I maintain.

This reasoning has a problem. We should always be concerned with dependencies and always keep them to the minimum you can correctly get by with.

The problem with thinking these x packages are OK for this library is how we got where we are. Yes we could just avoid using trivial libraries, but even for the other ones the decision to add it into your lib has to be made in the context of the app or lib, or lib's lib using your library. If you only imagine your lib being directly used, your scales a d judgements could be way off the actual cost/bebefit.

As someone else alluded to, I think the problem stems more from the fact that major official libraries are glued together using a lot of dependencies that aren’t necessarily trustworthy. This doesn’t happen nearly as much in the other ecosystems (at least none that I have seen). In Android dev, for instance, you often will see the official Google support binaries as dependencies but nothing else. In .NET you’ll often see Newtonsoft’s JSON serializer but very few non-Microsoft third party dependencies.

It does to a lesser, but still worrisome, extent in Java.

I think it's precisely because it lacks a good committed (commercial) owner like Android and .NET have. I could see it going from bad to worse in the future, as Oracle continues to jettison responsibility for bits of the ecosystem.

I think it’s healthy to have an ecosystem of large, well vetted, third party libraries maintained by long-standing organizations like Google (guava), Apache (commons, hadoop), or Pivotal/Dell (Spring).

The early decade of Java open source was mostly driven by the Apache Foundation

Anecdotal, but I find the Node std lib + the language primitives to be good enough for me. I never find myself reaching for external dependencies outside of React or an equivalent on the front end, a testing framework, etc. maybe 5 dependencies total for a moderately complex app. I’m honestly not sure why the JS ecosystem is so dependency happy. It’s not necessary, and the downsides are massive and obvious.

>It's a long time since js needed a good stdlib.

Yes. Huge package ecosystems are hurtful. For the basics, language communities should come with a nice, batteries improved, standard library.

For the rest, people should only trust community projects with big following, processes, etc (like Apache stuff, Django, moment.js, postgres, etc), or open source projects supported by companies (e.g. nginx, mongo, React, etc).

The rest, sorry, but you got to write them yourself.

Downloading and using 1-person libs for trivial stuff like lefpad and such is madness.

> The more I read about the js ecosystem, the more it seems the inmates are running the asylum

Let's safely call it immature. The principle objectives here:

    * obtain employment immediately
    * prevent unemployment forever

Part of the problem is that people are taught one thing in school, expected to immediately find work, and the then perform something immediately different at work. That, or they are simply Java (or whatever fill in the blank) and get retasked to write JavaScript.

What ends up happening is that people want to do their previous job in the context of their current job, such as write something that looks like Java in JavaScript. When that becomes the failure you would expect they blame the technology for not being their previous thing.

Compounding that training failure is that JavaScript has a low barrier of entry. Some of the people writing in this language are less than brilliant. The end result is a cluster fuck as evidenced by frequently used terms like imposter syndrome, snowflake, Dunning Kruger, and so forth. The insecurity, fear, and delusion clearly evident to me as a corporate JavaScript developer.

How do you solve for the obvious emotional failure that comes from the described lack of preparation? You hide under layers and layers of abstractions. You make life easier to the point of hoping code exists that does your job for you. Unfortunately that easiness is the opposite of slim or simple. It's great until it isn't if you aren't in a hurry.

Npm should delete the top useless libraries and force the package maintainers to rewrite the code to be less dependant on them.

Or they should do what they appear to be incapable of which is to provide certification and verification of certain packages. If people really need a lib for inarray then npm should maintain and certify it.

It should not be difficult for npm to raise the funds needed to do that.

Reply on news.ycombinator.com