mightybyte

My point about software systems being less constrained than physical systems means that even ho-hum CRUD apps will likely have more unknown unknowns than a ho-hum dirt moving project. Also, the task of stating the requirements for a software system is much more involved than stating the requirements for a bridge or tunnel.

In short, I think there are very clear and tangible things that give software estimates a meaningfully higher uncertainty than other engineering disciplines.

Yes, I get that all engineering (not just software) has inherent uncertainty in estimates. And there is always the chance for unanticipated events causing major overruns. But I still think that the uncertainty in software is significantly higher than in other engineering disciplines.

Of course I don't have any scientific evidence for this claim. But there are lots of phenomena that exist in the world that don't yet have scientific support. I think that software his higher uncertainty because it is much less constrained than physical engineering. There are a relatively limited number of ways that one part of a physical system can impact other parts, and those are constrained by the laws of physics. Software systems have no such laws. It's often possible for any component to write to arbitrary locations in memory controlled by any other component. This is kind of like a rusty hinge on a 34th floor bathroom door causing a problem with a doorbell on the third floor. Those kinds of actions at afar just don't happen in the physical world and if they do, they happen via pretty well understood physical mechanisms. To make matters worse, the discipline of software engineering is less than a hundred years old as opposed to thousands of years for physical engineering.

It's not about feeling special. I think there is something fundamentally different about software.

I sympathize with your view (although with s/5 year old/me/). However, over the years I have come to realize that many of the things I used to think I could have done, I actually could not have...to say nothing of your 5 year old. It might look like a few apparently random splotches of paint could have been put there by you, but in many cases looks are deceiving.

You might watch a grandmaster chess game and think that you could make those moves. Or perhaps you see a solution to a traveling salesman decision problem and think "it's easy to see there exists a path that short". But in both cases you're mistaken. The moves you would have played wouldn't be as good as the grandmaster's. The path you would find wouldn't be as short. And the paint splotches you would throw on the canvas wouldn't look as compelling.

Yes, there does exist shitty "art"--things that you or your 5 year old could have done. But I suspect it's less common than you think.

Yep, thanks for the correction.

The same author has another post that describes in more detail his views on sheep and how the modern popular connotation is not at all the way sheep actually behave.

http://epsilontheory.com/sheep-logic/

Completely agree. In particular I think the word "hash" is easy to explain. Just tell them it's kind of like the process of creating corn beef hash. Something goes into the process, and when it comes it it's unrecognizable and and you can reverse the process to get a cow back.

My username at the google mail service..

Kadena (http://kadena.io) | NYC REMOTE | Full-time

Kadena is looking to fill a variety of roles to build next generation blockchain technologies. The software is primarily Haskell, but we are also looking for project managers, frontend developers, designers, etc.

The "No One Believes Anyone Can Actually Code" point is right on the money by my anecdotal experience. Just a few weeks ago I was asked to implement FizzBuzz in an interview for the first time in my 15 years of engineering! This was despite having a large amount of verifiable open source Haskell projects.

NixOS is a Linux distro built around the Nix package system. Nix is built around the idea of immutability. It makes all packages immutable by giving them their own directory identified by a hash that is derived from ALL of that package's dependencies. This has a number of desirable properties:

* It makes it trivial to have multiple versions of the same package installed at the same time and allows you to switch between them at will.

* It is trivial to roll back your system after a failed upgrade. Difficult system recovers after you upgrade to a new unstable version are a thing of the past.

* Non-privileged users can install software completely securely.

* Projects packaged with nix have the best possible build reproducibility because nix accounts for ALL of your dependencies all the way down to the lowest level system libraries, compilers, etc.

This strikes me as an area where legislation could have a significant impact. It's an area where there is absolutely no economic incentive to do the right thing. It seems similar to the way email lists and spam were back in the early days of the internet. Based on my observations, the CAN-SPAM act requiring automated mailings to have an unsubscribe link actually seems to have been fairly effective at improving that situation. Granted, there is still spam, but the legit mailing lists are a lot easier to opt out of than they used to be. Perhaps some legislation relating to terms of service agreements could help with this problem?

This is great news. Now maybe they'll have the resources to ditch your phone number as your identity. Traveling overseas is really frustrating when you get a new sim card and your Signal identity changes.

One of my comments got a job for someone else who I had never met. And it got me a nice referral bonus. :)

Awhile back I left my first software job (they did have systematic hiring of new grads) at a very large corporation because I was moving to another city. As it happened, that company also had an office in the city I was moving to. I applied for a position at the new location and they gave me an offer. But their offer was something like 15-20k lower than just about all the other offers I had. Apparently there was some limit on how much you could increase someone's compensation that was preventing them from offering me more. I heard a rumor that my previous boss (one of only two level 6 software engineers in the company's whole eastern region) made some noise about how ridiculous it was given that hiring me was way less risky than hiring someone else since my track record at the company was well established. But it was all to no avail. He couldn't move the bureaucracy. So of course I took the highest paying offer with another company.

Completely agree. I used to use ag, but its handling of .gitignore files is buggy and often returns a lot more results than it should. Switching to ripgrep made a pretty significant difference for me.

In college I had a wind symphony conductor who was phenomenal. Playing for him was so much fun. Someone once told me that the better we played the more he danced, and the more he danced the better we played. I think this concisely captures one of the big functions of the conductor.

I agree with your assessment here. But the first thing I thought of when I saw this headline was fixing the fact that SSNs are massively bad as authentication. Instead of giving people a number to authenticate, we should be using public key crypto and digital signatures. If the Equifax hack is the catalyst that pushes us to public key crypto instead of SSNs, then I'm all for it.

Takt | DevOps Engineer | San Francisco, CA | Full-time, ONSITE preferred, but REMOTE is an option for senior candidates.

Check out http://takt.com/careers for more information. Here is the DevOps Engineer job description:

Takt is seeking DevOps Engineers to support the development of our flagship product. Our platform learns and adapts to people's preferences, habits, and feedback—orchestrating highly relevant experiences that are truly unique to each person. Our vision will change the way people engage with their favorite brands across multiple industries, be it retail, finance, or healthcare.

As one of the first DevOps Engineers at Takt, you will be able to contribute to a large scale environment and an early stage infrastructure in addition to influencing our culture and processes which we operate. You will capture the entire stack as code and provide tools and solutions that facilitate the release, maintenance, and monitoring of the platform. You will be working on a cross-functional team and collaborating closely with Engineers, Data Scientists, and Product Managers. As an early member of the DevOps team, you will help us build a culture and practice that leads improved product quality, cohesive collaboration, and integration between Operation and Development.

> Yes, you can work around the limitations of a static type system, either by finding another solution (lenses) or by making functions more specific (assume all keys are strings), but that doesn't mean the limitations disappear. It just means you're working around them.

I'm less concerned about the theoretical limitations of types that you seem to be talking about and more interested in looking at concrete real-world applications where you think you need dynamic types and seeing exactly what we can do to address them with a type system. I know there exist things that are difficult or maybe impossible to define types for (the Y combinator for instance, or possibly this assoc-in). My argument is that you don't need this power. I don't care whether difficult to type things exist, I care about whether real-world problems require them. I have yet to encounter one that I felt requires the same amount of dynamic typing power that you seem to be arguing for.

See this thread for some concrete examples:

https://www.reddit.com/r/haskell/comments/792nl4/clojure_vs_...

You do have a point that they're getting real work experience. But that's where the similarities end. I've never heard stories about health problems, stress, and burnout from side projects. I'm sure it probably happens to some people, but it's almost certainly nowhere near the same rate that it sounds like it happens in the game industry.

> Are you proposing that a distinct `assoc-in` function should be written for every combination of types you'd need in your program?

No. I'm proposing something like what tel described elsewhere in the thread. If that's not dynamic enough for you, then you can go with something like JSON's Value type. In both cases lenses give you very convenient and composable access and manipulation.

> I've always thought the function was trivial. I mean, it's just four lines of code:

But you have to read the code. The type signature / code boundary is very useful for allowing you to chunk things and abstract over implementation details. This particular case may not be much code to read, but that is often not true in the general case.

> Sure, but you have to write the type signatures, too. Type inference certainly cuts down on a lot of work, but it doesn't entirely eliminate the need for explicit typing.

> I also tend to prefer adding explicit type signatures, even for functions that could be inferred. It makes type errors easier to catch.

That's exactly my point. Type signatures can be inferred, specs cannot. Choosing to add them is irrelevant. If you want the add them that can be done automatically.

> The latter is trivial to type; the former is not.

That's why I'm not trying to make it typed. Instead of

    Map a (Map b c)

I'm dropping down to something like

    Map Int (Map Text a)
    Map Int (Map Text Text)
    [Map Text Text]
    Vector (Map Text Text)

...which is more like what's really going on in Clojure.

Come to think about it, assoc-in is a great example of why I want static types. It's very hard to figure out what that function does from reading the documentation. There's no type signature to help me, the description doesn't give me much more to go on. Reading the examples leaves me wondering whether I actually understand what it's doing or if I'm missing some corner case.

> it will generate and run tests for all functions that have specs

But you have to write the specs. And that's the code I'm talking about. Fundamentally the generator can't just know what behavior you want and what you don't want. You have to tell it.

> it requires some rather esoteric type system extensions

No, it's just a simple [Map Text Text]. Or if you want a little more safety, then [Map Text Value].

> you can get an good approximation of correctness with runtime specs coupled with test generation.

But this requires more code. More code is error-prone and takes time to write and maintain.

> There are functions that are trivial to write in a dynamically typed language, but are hard to statically type. For instance, the `assoc-in` function in Clojure.

This is not a hard function to write in Haskell. You make it operate on a list of maps, which is what the Clojure one is doing too.

> You don't necessarily need static typing to give functions some form of type signature or specification.

But you do have to have static typing to make sure those signatures are automatically verified to be coherent.

It seems like you're thinking about my statement from a turing-complete computable point of view. But I meant it from the point of view of what the compiler can do for you. It's not what things you can compute, but what things will be automatically checked.

This is an outstanding point. When watching this talk, I got the distinct impression that Rich has simply worked on a very different class of problems than the ones I have worked on. As we all are wont to do, we extrapolate our experience to the whole universe. But that's not always the right thing to do.

I would say that for the most part, a statically typed language can do the same things a dynamic language can do, but not the other way around. Another commenter mentions the Any type, but I don't think that's it. It's maps. Rich wants to be able to combine maps with a union operation, take subsets of keys, etc. And that is exactly what maps afford. I'd rather work in a language like Haskell where I can use strong types when I need them and drop down to untyped maps when I need them than a language like Clojure where all I ever have is the maps.

Another thing static types do for you is that they make things in your application more discoverable. Instead of tracing through to figure out which data is available and operations something supports, the compiler tells you. I'd rather work in a language where my compiler can help me this way than in one where it can't.

Takt | DevOps Engineer | San Francisco, CA | Full-time, ONSITE preferred, but REMOTE is an option for senior candidates.

Check out http://takt.com/careers for more information. Here is the DevOps Engineer job description:

Takt is seeking DevOps Engineers to support the development of our flagship product. Our platform learns and adapts to people's preferences, habits, and feedback—orchestrating highly relevant experiences that are truly unique to each person. Our vision will change the way people engage with their favorite brands across multiple industries, be it retail, finance, or healthcare.

As one of the first DevOps Engineers at Takt, you will be able to contribute to a large scale environment and an early stage infrastructure in addition to influencing our culture and processes which we operate. You will capture the entire stack as code and provide tools and solutions that facilitate the release, maintenance, and monitoring of the platform. You will be working on a cross-functional team and collaborating closely with Engineers, Data Scientists, and Product Managers. As an early member of the DevOps team, you will help us build a culture and practice that leads improved product quality, cohesive collaboration, and integration between Operation and Development.

Yep, it sounds like you pretty much have it. In addition to the two examples you mention, here are some other examples of monoids:

Lists under concatenation Booleans under AND Booleans under OR Numbers under max Numbers under min

I left out associativity because I was focusing on the ELI5 part.

A monoid is something that allows you to take two of the same kind of thing and smash them together to be one of that same kind of thing. It also requires you to have an "empty" thing that can be smashed together with any other thing without changing the result.

In code:

    smash :: Monoid a => a -> a -> a
    empty :: Monoid a => a

A monoid is a property of things, not languages, so the idea of "a language without a monoid" doesn't really make sense.

Yes, databases and filesystems are global mutable state (at least if you do anything other than append to them). And they also require vastly more care. For instance, if you store a JSON blob in your database, you have to be VERY careful with how you change it over time. If you make a backwards-incompatible change to the schema without realizing it, you break. A likely response is to advocate better testing, but you need a special kind of testing to detect this problem. Plain old unit tests won't cut it because it's not about whether your parsing and rendering code at any given point in time is correct. It's about whether the parsing code from today's commit is compatible with the rendering code from the previously deployed commit!

Furthermore, if you deploy incorrect code, it could add JSON blobs to the database with the new schema before it crashes on the old schema. Now you've got two incompatible versions of the JSON blob in the DB at the same time, so rolling back to the old code will still be broken!

Software engineers deploying production apps have been dealing with these issues since we started using databases, but that doesn't mean that it's any easier or less error prone. It just means that we have to be more careful. And that care has a cost.

This cost isn't unquestioned for larger systems. It's a big part of the motivation for append-only databases, for example.

http://thesoftwarelife.blogspot.com/2010/12/append-only-data...