SomeCallMeTim

You can't always, though.

I described a situation where TDD really, really, really wouldn't have worked. The whole structure needed to be developed, or at least 80% of it, before it would have made sense to write any tests--and the actual TDD philosophy would be to write "one small test" and only write exactly as much code as required to satisfy the test.

The sane approach was to create the entire structure based on the design, and then test it after it was complete as an entire system. Some of the micro-functionality that TDD would have had you test would have become technical debt as a change-detector later when the client changed their specific requirements.

As I said above, there is no evolutionary path from tiny pieces to the full structure, and TDD requires that you follow such an evolutionary path. If you're writing a bunch of tests and then creating a nontrivial amount of code, then you're following test-first, but not really following TDD. And I question even how valuable that is when you don't necessarily understand what would need to be tested before you've finished implementing the system.

Good to know. I use that; didn't know it was Delphi. It's...not terrible looking. It's all Windows controls, as I mentioned above, which makes it look, well, simple. And it's a utility that really doesn't need to look fancy, so it's fine for what it is. It doesn't seem to have a dark theme, which is a bummer, but otherwise it does the job.

But that doesn't mean I want to work on it.

Depends on what "look great" means.

And I've seen Delphi apps; I recently worked on one. In my experience they don't look great. Maybe you've done a lot of work to theme the controls; if so, wonderful. But what I'm familiar with just brought up Windows dialogs, and ... well, that's functional but I wouldn't say it looks great.

But here are a few examples of ways you could create 150 fields in a day on a web app and have them look great:

Arrays of control types: https://formik.org/docs/examples/field-arrays for simplicity of creating the controls plus something like https://chakra-ui.com/docs/components/input to make them pretty.

Spreadsheet look (with spreadsheet functionality!) in one control: https://www.npmjs.com/package/react-spreadsheet

Want to build the form by hand and customize it? https://www.form.io/

Form should follow a database schema and automatically display database records? https://marmelab.com/react-admin/ (I used this approach to bring up an admin panel for more than 150 fields, though spread over a couple dozen pages, because of course that looks better than trying to bring up 150 fields on one page.)

The standard way to create a large form (or any data table on a page) today is to iterate over the fields you want and add them to a page. There are frameworks like Chakra that can help you make everything look attractive--and you can theme the result in whatever color scheme or styles you like.

I really don't want to have to touch Delphi ever again; it felt like I'd fallen back into the 1990s in code development terms, and I missed all of the modern features I'd grown accustomed to. If it's what you like, and it pays the bills, then enjoy. But know that people who have used more recent tools like myself consider it to be anachronistic.

That's one issue with TDD. I agree 100% in that respect.

Another partly orthogonal issue is that design is important for some problems, and you don't usually reach a good design by chipping away at a problem in tiny pieces.

TDD fanatics insist that it works for everything. Do I believe them that it improved the quality of their code? Absolutely; I've seen tons of crap code that would have benefited from any improvement to the design, and forcing it to be testable is one way to coerce better design decisions.

But it really only forces the first-order design at the lowest level to be decent. It doesn't help at all, or at least not much, with the data architecture or the overall data flow through the application.

And sometimes the only sane way to achieve a solid result is to sit down and design a clean architecture for the problem you're trying to solve.

I'm thinking of one solution I came up with for a problem that really wasn't amenable to the "write one test and get a positive result" approach of TDD. I built up a full tree data structure that was linked horizontally to "past" trees in the same hierarchy (each node was linked to its historical equivalent node). This data structure was really, really needed to handle the complex data constraints the client was requesting. As yes, we pushed the client to try to simplify those constraints, but they insisted.

The absolute spaghetti mess that would have resulted from TDD wouldn't have been possible to refactor into what I came up with. There's just no evolutionary path between points A and B. And after it was implemented and it functioned correctly--they changed the constraints. About a hundred times. I'm not even exaggerating.

Each new constraint required about 15 minutes of tweaking to the structure I'd created. And yes, I piled on tests to ensure it was working correctly--but the tests were all after the fact, and they weren't micro-unit tests but more of a broad system test that covered far more functionality than you'd normally put in a unit test. Some of the tests even needed to be serialized so that earlier tests could set up complex data and states for the later tests to exercise, which I understand is also a huge No No in TDD, but short of creating 10x as much testing code, much of it being completely redundant, I didn't really have a choice.

So your point about the design changing as you go is important, but sometimes even the initial design is complex enough that you don't want to just sit down and start coding without thinking about how the whole design should work. And no methodology will magically grant good design sense; that's just something that needs to be learned. There Is No Silver Bullet, after all.

Needing tests != TDD.

Needing tests != Unit Tests.

Adding larger system tests after the fact is perfectly reasonable. TDD wants you to write tiny tests for every square millimeter of functionality. It's just not worth it, and 99% of the value is to make up for shortcomings in dynamic languages.

I don't care whether it's all in one file or in a dozen files, but I want all of that information to be available programmatically in a text file (unlike in a readme or on Github) in a standardized location in a project.

In that respect, package.json is a strict win. Your lack of willingness to use `git blame` to see why you added a line, or lack of reasonable git comments, is not to be blamed on the file.

Complexity is unavoidable. How could you write a tool like license-checker [1] for a Go-based project without having license information in a standardized location? Without the scripts section, how can you create a tool like husky [2] that automatically installs git hooks for a project? Every single part of package.json is there for a good reason; at best you could argue that putting some of it in other files would be aesthetically superior, but that's just bikeshedding.

Complexity isn't de facto bad. Some complexity is required if you want a certain level of functionality to become available. Deno (and Go) are slowly accumulating that "cruft" as people realize that those functions are actually useful or even critical to a mature ecosystem.

[1] https://www.npmjs.com/package/license-checker

[2] https://www.npmjs.com/package/husky

The dependency inspector is showing each TypeScript file, which strikes me as a newb mistake if I've ever seen one. That's just tons of unnecessary noise distracting you from what you need to know. Heck, it's also showing library files, which are going to be mostly irrelevant. If I include a framework that has 50 source files, I really just want to see what framework version I've included, not the place where every one of those fifty files are included every time they're included. Heck, if one indirect dependency had a bad version tag it would be nearly impossible to see.

The lock file format is also TypeScript-file based, which ... well, since dependencies can point at arbitrary git repos, you could easily end up in a situation where the version tag on the repo is changed and then you can't find the file you need to download. It's a pretty worthless lock file if the file can just disappear from the internet; a lock file should exist to guarantee that a package can be rebuilt exactly, not just prevent it from being rebuilt if it can't be rebuilt exactly, which is all the Deno lock does.

npm on the other hand will prohibit anyone from deleting an older version of a package if anyone else is using that package. "leftpad" can never happen again now in Node. Having a centralized repository is actually a Good Thing; being able to search through all the packages in one central repo and determine relative popularity of each is also useful.

Also: The lockfile is a generated shrinkwrap file. That's only a tiny piece of what I'm talking about. What I want to be able to do is set the exact version of an indirect dependency without needing to fork and rebuild every dependency. See yarn resolutions [1] for an example in the npm ecosystem.

No, I'm not mad because it's different. I habitually chase the bleeding edge up to and including jumping between frameworks and build systems and even languages and editors in constant search of ways to improve my development workflow. I thrive on learning new ways of doing things. I was enthusiastically digging into Deno shortly after they very first 0.x announcement, but quickly saw a number of showstopper flaws (including lack of npm compatibility) that caused me to write it off.

No, I'm complaining because Deno is missing crucial features. "Lack of features" isn't a feature on its own, especially when people use those features. (See also the Go language.) I also would miss the "scripts" section of package.json, since it's a centralized location for the various commands that a project might need; things like "generate a new migration file" or "run both the client and both servers" vs "run only server A".

This is all complexity that was swept under the rug by Deno and that will need to be replaced by community standards that won't be standard.

[1] https://classic.yarnpkg.com/lang/en/docs/selective-version-r...

This chips away at one of the showstoppers for Deno for me, which is good.

But while "the vast majority" of npm packages don't require a gyp build step for native addons, some of those modules are pretty important, and I see no indication in the announcement that they're also going to be implementing the Node C API or the gyp build process.

Right now I'm working with a machine learning project, and XGBoost [0] is a direct Node.js extension [1] through the binary interface.

So this does bring things a step closer to being generally usable, but there are still significant roadblocks.

A WebAssembly build of XGBoost could work with Deno, but aside from some guy's unsupported side project/proof-of-concept for use in a browser, I'm not seeing an XGBoost WebAssembly build. And generally when deploying something like a machine learning model I'd rather use well-supported tools than to need to dive into the rabbit hole of maintaining my own.

And yes, XGBoost will likely eventually have that kind of support for Deno, but then the next bleeding-edge project will come along and only support Node.

Even assuming Deno eventually hits a tipping point in popularity where everyone wants to release Node _and_ Deno support in their bleeding-edge projects, there are still things that I miss from package.json that don't seem to exist in the Deno ecosystem.

Things like the "scripts" block: A nice centralized place to find all of the things that need to be done to a project, plus auto-run script entries that can trigger when a project is installed. And inheritable, overridable dependency maps (see the yarn "resolutions" block).

I'd love to jump into Deno, but I think there has been far too much "baby thrown out with the bathwater" to its design. It's the classic development problem of looking at a system and seeing a ton of complexity, but not really understanding that all of that complexity was there for a reason. Maybe when it re-evolves 80% of Node's and npm's features I'll be convinced to make the jump. I'm a huge TypeScript fan after all. But it still strikes me as a violation of "As simple as possible, but no simpler."

[0] XGBoost is a _very_ promising approach to machine learning, training models much faster and with much more accuracy than traditional approaches.

[1] https://github.com/nuanio/xgboost-node

Except that, as the ecosystem grows, you eventually have packages that are complex enough that they have imports, and those packages have imports.

And now because you've hard-coded imports at every level, you can end up with a dozen different versions of the same import, causing potential conflicts between the various components of your app.

Package managers evolved to be the way they are _because they're useful._ It's not like people said "let's add all of this complexity for fun!"

Direct imports failed as a solution for Go, and they already have been acknowledged as an incomplete solution for Deno (import maps are already a larval form of package.json, but without the ability to inherit child import maps, which means it's still broken, but presumably it will eventually be fixed).

Not having a standardized place where you can see the dependencies of a project, and not being able to distinguish which are _development_ vs _runtime_ dependencies, is also rather broken, and would require a package analysis tool that's far more complex than a package manager to use to trace the dependency tree reliably.

The default should be shrinkwrap, like `yarn`, with the ability to upgrade packages manually when you decide that it's a good time to upgrade packages. I don't like things changes at all without explicit interaction. Services like Snyk will let me know if there's a specific security concern with a package version I'm relying on.

Which is why I use `yarn`, I guess.

What good is Bitcoin without the ability to trade it then?

Your assertions are the ones that are extraordinary, so yours is the side that requires proof. You're effectively claiming that a distributed peer database with a somewhat secure history has some kind of intrinsic value. I'm calling that BS.

Where is the money? Where is the _value_?

The fact that a few people were trading Bitcoins doesn't mean it can ever go mainstream. Something that's a niche hobby isn't going to change the world. If you want to use Bitcoin as a hobby barter medium, then have fun, but that doesn't imply value outside of a tiny community.

Prove me wrong. A "proof of concept" is in the technology domain. For crypto to work outside of a community of crypto-libertarians, it needs to be usable by regular people. Libertarians are famous for not actually being able to design social systems that work in the real world; last time they tried they were chased away by bears. [1]

And as the article pointed out (did you read it?), most apps that support crypto rely on a centralized server since the chain can't be downloaded to mobile devices. How useful is a medium of exchange that requires you to bring a laptop with you everywhere?

[1] https://www.vox.com/policy-and-politics/21534416/free-state-...

Open source crypto protocols are completely irrelevant without connections to crypto companies.

There's this fantasy that crypto can replace money, and that you would therefore never need to trade it for real dollars. That's not going to happen, for reasons outlined in the article and because no medium of exchange that's deflationary can ever be a currency.

If it goes up in value over time, it's an investment and people won't spend it.

Will the open source be free forever? Sure. Whatever. Can people run nodes that burn electricity? Yup. Does that matter? In no way that's relevant to the general public.

Worse, even the crypto networks themselves are subject to issues with their authority: Enough bad actors who join can steal the value of a large percentage of the network.

When we're looking at potential theft of billions of dollars, you better bet that someone will invest the hundreds of millions required to rewrite the chain and take control of a large number of coins.

Then what? Then everyone will need to get together and act like a central authority to either rewrite the chain back to its state before the attack, in which case you do need to rely on a central authority, or you need to admit that _allowing untrusted nodes into a financial network is a Bad Idea._

"Censorship resistance" is irrelevant too, by the way.

With a bug that's trivial to exploit, you've got hundreds of people exploiting it; with a bug that's hard to exploit, you've got maybe only yourself or your agent exploiting it.

Which of the two do you think will more likely come back to haunt you? The one where any authorities investigating will need to dig up the backgrounds and connections of hundreds of blockchain addresses exploiting the transaction, or the one where only a single address exploits the transaction? Won't their attention be primarily on those idiots who try to withdraw the money in the US, say, vs that one transaction out of hundreds where someone in Barbados had their proceeds deposited into a bank and withdrawn as cash before closing the bank account (that they opened with a false identity, maybe?).

When there's a single account performing the exploit, all of the investigative resources will be applied immediately to that account, making it far more likely that the account in question will be tracked up to the point of withdrawal, and potentially flagged in time to prevent such a withdrawal. With hundreds of others distracting any authorities, it becomes hundreds of times harder track down the original perp.

Think of those heist movies where someone throws a handful of cash up in the air to help avoid pursuit. Same idea.

The problem is that crypto doesn't really solve those problems either. As the article says far more eloquently.

I understand that you are saying you were burned by a central authority. But the truth is that the only way to get everything to work is with a central authority. The article gives examples of crypto being fully reliant on central authorities, for instance.

Everything being done by crypto today could be done better by trusted companies putting together a semi-open peering arrangement that both doesn't waste pointless electricity and that doesn't allow just any potentially hostile actor join the group of authorized peers.

Software engineering is complex. Some of the challenges are simply irreducible.

I believe this is one of them.

True enough, though Snyk will sometimes offer a patch rather than a PR to a new version of the dependency.

The Snyk tools apply to the patch directly to (e.g.) `node_modules` after the latest code has been downloaded.

99% of the time it's a minor update patch that you really should just apply, or Snyk will add a patch to the build to fix the security hole directly.

Honestly it's just not worth worrying about whether a security hole is exploitable when the decision is whether to accept a minor update patch. Just accept the patch PR and move on.

The times you need to do a major update to get a patch fix it's equally important to just make that upgrade, since that means the ancient version you're using is likely out of maintenance and likely is throwing warnings that you're using a deprecated version.

Yes, it's more work to perform a major upgrade. But that doesn't mean it isn't good engineering to keep your tools up to date.

You can't use a technical solution to fix a management problem, no. :)

I worked for one company that enforced on-call for the entire team. For a reason that's out of scope for this comment, it didn't apply to me, though it should have by the normal rules.

I loved working for the company, but at the same time I would have totally despised the on-call system as implemented.

The main problem was that I was working on client apps. My focus was an Android app, to be specific. There was a completely separate web team that developed the backend; we not only didn't work on the backend, we were prohibited from working on the backend. I never even saw the backend code. We even asked to develop part of the backend we needed at one point, and they refused to let us, telling us that we didn't understand their security requirements and therefore couldn't contribute.

And from what I heard from people who were in the on-call rotation, every single 2AM call was from some badly designed alarm. Designed by their team.

Yes, every one of those alarms was fixed. Some may have been only "fixed" the first time, but I got the idea that each did eventually get adjusted to not be completely spurious.

But what really offended my sensibilities was that the backend team was pushing for the client team to be on-call to cover for what seemed to me to be profoundly poor alarm definition. They were eager to get others on board to cover the on-call rotation because being on-call was such a nightmare.

Another comment [1] points out that the median number of alerts in a week should have been zero. It wasn't close, from what I could tell. And the whole "make them eat their own dog food" approach of putting the people on-call who are actually responsible for writing the code was broken by the practice of including people only loosely associated with the code (as in, we used it) in suffering the consequences of writing the code (or designing the alarms) badly.

In general, if I've broken something that's affected a site or product, I'm happy to fix it, even if it's after hours, though I prefer it to be on a "best efforts" basis rather than a "drop everything and work on it now" basis. What I don't want to sign up for is being roused out of bed at 2AM to fix problems caused by someone who isn't even on my team, where I wouldn't have even seen the PR that caused the problem or any of the related code, and there's absolutely no way I could have prevented it.

/rant

[1] https://news.ycombinator.com/item?id=32163155

The advantage of being able to build to a binary is that you can then distribute the binary directly instead of distributing source and forcing people to rebuild it.

When 98% of your users only have the binary and not the source code it means 98% of your users won't even be aware that they need a new version that isn't vulnerable.

Remember the vulnerability disclosure will be something like "Log4Go has a vulnerability!!", not that the binary apps you've downloaded have a vulnerability. So if it affects 200 projects, every one of those projects will also have to publish vulnerability notifications independently.

And frankly 80% of those projects will likely be abandoned or at least neglected after a couple of years, leaving many people with vulnerable binaries they don't even know to check for vulnerabilities.

In other words, they very advantage of Go (easy to distribute binaries directly!) is a security disadvantage. If you instead installed the app using a package manager, the package manager could at least notify you that you have a package with a security issue. But that's nothing new; security and ease-of-use are often at odds.

>"Use a build tool to do dependency checks!" > >And then what? > >"Only update libraries if they have vulnerabilities." > >So, should this be a manual process where I have to dig through obscure warnings every time I build something? > >"No, you can automate it!" > > Congrats, you've just outsourced the decision making process about which versions of libraries to use to some 3d party. You've also made your build process reliant on yet another online service.

Sorry, this is a total strawman. Automation doesn't need to be integrated with the build in such a way that it creates noise, and it doesn't mean you need to let changes be applied without human intervention.

I'm hooked up with Snyk which scans all of my dependencies and sends me a summary email if it discovers any vulnerabilities. It creates PRs that can patch dependency vulnerabilities, and I can apply them if I approve of the change. If I don't think the change is worth it or the vulnerability is relevant, I can go to the Snyk site and mark a vulnerability to be ignored, or even tell it to ignore it for a month or whatever. I'm sure Snyk isn't unique, either.

Yes, a human should be involved in decisions. No, it doesn't need to be noisy.

I'm on Node.js and working on multiple projects with crazy numbers of dependencies, and the number of "new vulnerabilities" I see is typically a few per month at most. Some months there are no new notifications at all. It simply doesn't change that quickly, and it's not constant noise unless you ignore the notifications, and that's just bad software engineering.

Article tl;dr: Design is hard and can't be boiled down into applying pithy mindlessly.

For what it's worth, I agree with your points and disagree with the various counterpoints that were posted; "optimization" can mean a lot of things, and I for one understand what you mean.

Some of us lived through that time and despise pixel-art. It didn't even look like pixel art back then because the monitors were blurry. [1]

You're welcome to like pixel art if you want. I don't actually like the new aesthetic either, but I respect the choice--and I do like similar approaches where it's not high-res painting or 3d rendering and instead is abstract art.

[1] https://weareludicrous.com/blog/2018/pixel-sharp-graphics-at...

Yeah...some of us don't drink.

Buy me food, however, and barring a conflict, I'll go. ;)

> The big thing America really lacks is universal healthcare

Throw in employee protections and reasonable amounts of vacation time, and I'm with you 100%.

Object pools are not hard, and I certainly used them a lot in game development. That's just a cost of doing business in low-latency programming.

There's no "tuning of the GC" beyond that, though, unless you count linking in a faster multithreaded malloc library, which is another thing that I've tried.

What's problematic about Java in particular in relation to latency is that Java's libraries can be pointlessly wasteful with regard to allocating garbage. I found that a function that was supposed to call through to a system function that got the system time in milliseconds was somehow allocating an object and discarding it as part of the call. The function returned an integer.

I found this when my Android JNI (C++) app was getting janky because of GC pauses, and I traced the allocation to that function. I wrote a quick JNI function in C++ to call the system and return the integer directly, and the jank went away.

The best GC in the world won't make up for wasteful allocations, and some languages (or their ecosystems?) seem to encourage rampant allocations.

> When the last reference to an object that is the sole owner of a large graph of other objects gets decremented, the entire object graph must be immediately deallocated. It's possible for a single dereference to trigger an unbounded amount of deallocation.

Two things:

1. Unbounded may be technically true, but it's really not a hard problem to ensure that time-critical runtime deallocations either rarely happen or are simply never huge chains of objects that release each other. Source: Used C++ & boost::shared_ptr/boost::weak_ptr extensively in game development and never had unexpected jank due to deallocations.

2. There's no actual reason that the full deallocation needs to happen immediately in all reference counting implementations. If you have hard time or latency constraints you could queue up the actual dereferencing and ensure that you only dereference a certain number of objects per time period.

Yes, #2 changes some assumptions about when objects will be dereferenced; in C++ one might rely on guarantees that objects are immediately released along with associated resources. But if you go into a project knowing that it needed to support deferred reference counting, manually releasing critical resources is hardly the worst thing you'd need to do.

A bootcamp doesn't guarantee anything; without the right aptitude and temperament the tech industry is just as miserable, with a side of failure if you can't keep your jobs.

And no, "excellent work ethic" isn't enough.

It was a brain fart: Meant to say CockroachDB instead of CouchBase. Oops.

So yeah, it's on my short list. :) I've used triggers in one project recently, but that was literally the first time in five years so it's not as much of a must-have for me.

I said Couchbase above but meant CockroachDB. Brain fart, but can't edit the message now. :|