DanielBMarkham

Fascinating. More examples of large organizations encountering unknown and non-reproducible phenomena.

Like everybody else, I have no idea what this is. I have, however, consumed scads of stories about military organizations encountering things they can't deal with. Some of these are quite prosaic for their situation, such as UN forces figuring out that the Chinese were entering the Koran War.

I _think_ I have enough of these examples to reach a tentative conclusion: for many organizations, the information that they've had incursions and are stumped is extremely important. If there's a known weakness, other competing powers might take advantage of it. What actually happened is besides the point. The mission is to control information, access to information, and the larger narrative.

You layer on half-remembered (and perhaps exaggerated) stories from fifty years ago and you get articles like this. My best guess is that this is very similar to what happened to the US Navy recently, only in this case the military has managed to confuse the issue enough that investigators will be debating details for decades to come.

Reality is not required to conform to our standards of scientific proof. If we can't observe or manipulate something, there's no way for us to scientifically reason about it. That's tough for a lot of folks to deal with.

Healthy systems exist in multiple-layered robust feedback systems. Short answer: we're the same people just as afraid as we've always been, but we've dismantled or disabled several different feedback systems. We kept mucking with a complex system until we broke it.

This news is a horrible thing, and I'm outraged. It needs to be fixed. Right now. Having said that, it's good to have a little bit of historical context. Every governmental structure we've set up, from the military to a national police force, has pushed the limits to see how far they can go.

You think you'd set up these agencies with clear missions and they would stay inside the coloring book and mind their knitting, but that's not how it actually plays out. Instead, egotistical, ambitious people get in charge and want to do something even more impressive than the last guy. That means pushing things. IIRC, one of the first uses of a telephone tap was made by law enforcement. The intelligence community got so far out of whack that by the 70s it took a major reform to try to un-screw them. A lot of that work we dismantled after 9-11.

Here's another way of saying the same thing: it used to be that when extreme things happened the people we elect and the government we've created took drastic, emergency and probably extra-constitutional action to react. You can go back as far as Jefferson (or before) and the Louisiana Purchase and see people we put in charge kind of making it up as they went along.

This actually worked well because if there was a mistake? Well, we were doing it the wrong way to begin with. Elect new people, clean up the agency, say a bunch of apologies and write checks, and don't do that again --- at least in the same way.

Now, however, we want to create systems of things. Whereas it used to be that if they thought there was a bomb on a plane they might forceably unload it and strip-search everybody there, now we just hired tens of thousands of people and treat everybody that flies like a potential terrorist. Problem solved. No more worrying about those one-of incidents. No more people getting fired, agencies looking bad, or any of those huge cleanups.

After the 93 World Trade Center bombings, the United States Secret Service wanted to put anti-aircraft missiles on the White House. They were laughed out of the room. Who knows, they might have put some up anyway and nobody ever found out. After 9-11 that changed. Now it's a system. They wanted to shut down Pa Avenue, also laughed out of the room. They got it. They regularly ruin cell service, and are monitoring all calls in the area. Now they're bulk-buying privacy data and asking for even more permissions. These aren't "Hey, we're worried that something might happen, so let's try this for a while even if it's wrong" These are "We want permission to do this from this point on out and you leave us the hell alone" I don't mean to pick on them. Everybody's getting in on the action.

I could continue the rant, but as you see, this isn't a case of people changing or agencies changing in the last 20 years or so. This is a case of continuing to tinker with a system over 230+ years until you broke it. I wish it were just as easy as blaming the Border Patrol. It is not.

Hey driesdep, I love the look of this!

Meta question: are there any tips, pointers, tutorials, or classes you can recommend to make that beautiful metal enclosure you've built? It's a thing of beauty.

We live in an increasingly-authoritarian society where most people are unable to describe or reason with simple concepts put forth dozens of decades ago.

These two things are not accidentally correlated.

"My neighbour’s freedom to play music at 4am limits my freedom to have a good night’s rest. Does this increase or decrease freedom?"

Freedom != Liberty

This concept is easily over 200 years old. I appreciate the author's interest in the subject. It might be good to start much nearer to where others left off.

I think the thing a lot of devs don't know is that there's a limit to refactoring. At some point, you're done.

It's like a mountain. The team is dropped half-way up the mountain. The dev that refactors all of the time continues to head up the mountain. It can be slow work. It's a tough climb. Most of the time, devs on the team don't know any better, so they just do the work in front of them, slowly sliding down the mountain. Eventually, they look at the mountain and say "There's no way anybody could conquer that! We're lucky just to get our work done without too many bugs!"

Meanwhile, the refactoring dev reaches the top and slides down the other side. That dev starts looking for bigger mountains. The rest of the team he left behind sink into despair and cynicism.

Eventually, when organizations grow large enough, most everybody lives at the bottom of the first mountain. Devs who leave and try to summit are ridiculed at best, shunned at worst. They're hated if they only go halfway and then fail.

I am of the opinion that most big-organization developers don't actually know how to code. Sure, they know how to type in programs. Hopefully they know enough to test what they're writing. But a real understanding of what programming is? No. Until you've been over a few mountains you don't know what you don't know. Sadly these are many times the folks that get promoted.

Thanks for engaging. I'm perfectly happy switching to something new. I've only been on this place for six weeks or so. But I'm a bit conflicted about options.

Looking at the front page of HN on any given day, a significant portion of the top comments are not about the material, but about the site itself: the fonts are too big, the layout is hard-to-read, it's a paywall, Medium has gone to crap, the site shouldn't be trusted, the load times are too long, the page is too heavy, and so on. (You have to know what I'm talking about here)

I'm a coder, so I can (and have) put content in all kinds of places. I could give you a detailed list of a dozen or so that I've tried, from plain html/js on my own server to Git Pages and this current experiment. The problem is that I have _yet_ to use any of these where I didn't get some sort of meta comment like yours.

I believe, and I hope I'm wrong, that the _control_ issue you mention is never going to go away. It's just going to be an issue of what trade-offs people are willing to make for ease-of-use. Yeah, I could drop web pages and multi-media content on AWS and set up my own membership and streaming services. Then I could get into multi-format streaming, SAML, CDN configuration, and a hundred other technical issues that I'd love to play with, but have nothing at all with trying to create some kind of useful content out of my career experiences so far.

So thanks, and I agree, but I'm still kind of at a loss at what to do. If more people complain I'll send the admins an email. I could leave, but to where? Meh.

EDIT: If I wasn't clear enough, I don't like the popup either.

I've tried several platforms for blogging over the years. This latest one promises to let me own the content, no ads, and I can set up a subscription program for folks who want to participate more. I thought about 2 bucks a month. It would be an experiment to see if when people pay they tend to argue less and post less low-effort stuff. I suspect this is the case, but it'll be interesting playing around with the idea.

Many thanks to both of you guys for mentioning and linking to these other sites. I should have called them out explicitly. There's really very cool work here.

Having said that, this is not a chess app; it's an app that uses chess to talk/teach about programming and architecture choices. I will not be adding the holographic smell-o-vision to the feature list anytime soon. (grin)

If you guys want to comment or ping/email me with any other chess apps for folks, I'll come back in a few days put them in a prominent place under the title of the post. Sorry about the omission.

Right now there are people sitting in their doctor's office, hearing that they have a terminal disease. They only have a small amount of time to live.

This is about as negative as news can get, and it's been happening ever since there have been people and doctor's offices.

This is a horrible thing to have to bear, and many commit suicide or have a terrible time with the news. By and large, however, people go on with their lives, finding hope, arranging their affairs, and getting ready to both fight and enjoy the time they have left.

What they don't do, by and large, is have a continuous public meltdown, even though their life might look as bleak as is possible.

Negative news is great. What's needed and not provided is context. In a world of billions of people, with millions of people able to imagine and prove to various degrees that some horrible thing is about to happen, the amount of negatively-tainted news is for all intents and purposes infinite. It always has been. We've just had shitty communication systems.

People love to consume and share negative news. They love to slow down at the car wrecks to see if they can see bodies. They love to socially-gossip about how bad something might be and how so-and-so might be taking it or not. That's wired into our systems, probably as a survival behavior. Hominids who worry about lions tend not to get eaten by them, whether or not they live near any lions or not.

In this environment and evolutionary configuration, then, some sort of filtering mechanism is required so that the things we're hard-wired to do don't take over most of our life. To admit that isn't to want to live in a pollyannish world, it's simply to grow up and start consuming a media diet instead of a shitstorm firehose. There are real problems, yes. There always have been. To solve them requires precision, emotion-free discernment and slow progress. Most folks aren't getting any of that today. They're just getting the lizard brain stimulus over and over again. Because money.

Looks like the bullet list of things we might complain about, almost randomly presented.

Since there's no reasoning presented, I guess there's no point engaging?

Fascinating idea.

I've noticed it for a couple of years or more now. I figured most readers of HN have also noticed it, so I thought I would throw it out there to see if my observation was shared or not.

I know that people cycle through moods in their lives. I also know that people require some amount of drama in order to feel as if they have meaning (Even/especially people who say they don't like drama). Doomscrolling and the associated content promotion, curation, and aggregation probably tap into that primal need.

The internet at scale and the associated Markov-chain-like content selection and promotion we're seeing is hacking the species in some very significant and unexplored ways. Between the folks completely in denial and those claiming that the end is near, there needs to be some real philosophy and science going on (I don't think either one by itself is going to help much since this is a completely new area of intellectual exploration)

*(Reference to phrase "Sunday Scaries" for those who haven't seen it before: www.sunday-scaries.com/what-are-sunday-scaries )

Interesting, and (so far) well-sourced. You picked a good day to post here. For some reason, Sundays is always it's-the-end-of-the-world-as-we-know-it on HN. Every week I see variations around the same theme. It's cool to have the topic covered with reduced scope (societies, not the planet at large), and in-depth.

Which brings me to my questions. One, what is "The West"? Would the French Revolution count as a civilization-wide failure? The Bronze Age collapse? Sea Peoples? The Long March? The Hellenic Era ending? Perhaps this is explained elsewhere, but some clarification would be helpful.

Two, it might be good to cover all of the "false positives" when it comes to predicting civilization collapse. It's been quite the cottage industry over the centuries. Romans were quite concerned with the lack of morals and corruption the Greeks brought them. Many believed that signaled the end of Rome. Why, some of the "new" Romans weren't even sleeping on the ground any more! They were using beds!

Three, and I apologize for so many queries, but "collapse" or "failure", or whatever you're using could also use a bit of definition. The Western Roman Empire collapsed and failed, that much is obvious. But just when that occurred is much less clear. People in Europe thought of themselves as Romans long after modern historians view Rome as being gone. I believe there are a couple of other examples of unclear failures, where the people involved in the failure couldn't see what was happening right in front of them, but none come to mind right away.

Good work. Look forward to seeing more of it!

As a writer, I think you also need to decide: are you writing to be read, or to help people? They're not the same thing. If you're writing to be read, you'll go for low-hanging fruit, say things most people would agree with, and show a passion that matches whatever the popular passion already is. If you're writing to help people, you may have various messages that are unpopular or don't appeal to the average coder.

Unstated here, but also important, is that fame brings fame. There are a lot of folks writing schlock but are quite popular because they're the people that everybody thinks are supposed to be popular.

I found it fascinating that there was such a divergence in perceived risk, inversely to actual risk.

This study indicates that there should be a follow-on study about "pandemic shock", the process a population goes through when a new and serious disease vector is introduced. It could have important public health implications for future pandemics.

There's an odd theory in history that you become your enemy.

Over time, whatever group or cause you focus on, you demonize it so much, justify fighting them the same way they fight you (or you perceive them to be fighting you), that eventually you're just like your enemy, only with different slogans.

That's why political differences should never be represented as wars, and groups of people with other opinions as enemies. Every effort should be made to personalize people who disagree with you, humanize them, show them kindness they do not deserve (at least in your own mind). It's not for them; it's for you.

Wars are bad not because there's some worthy cause that we confront and eventually defeat it, making the world a better place. Wars are bad because they harm the people fighting them, many times forever.

In the free world, we've realized that peaceful, loving, kind civil disobedience is the only way to change ourselves for the better. This is why. It has nothing to do with how worthy our cause is or what an awful thing we're defeating. It's about the stage we leave for the next generation with their struggles.

I code both ways simultaneously.

I'm interested in what types I need to solve my business problem. The rest of it, I don't care. The reason I care about the types I need for my business problem is that I don't want my code doing something that it shouldn't: DDD-style coding allows me to code in such a way that it's impossible for the program to exist in a state that's invalid. That's a great time-saver. Other than that, polymorphic coding FTW.

1. It's a program to translate LeanPub files to html

2. Because F# is fun? An hour or two of work, get something useful done, go do other things. Would you rather spend month-after-month hip-deep in some library or framework or simply solve a problem and move on? I'll take the quick, useful, fun version, please.

Mostly agree.

> I don't get involved because these platforms simply encourage the worst in us.

In-person, I've found I can meet and talk with folks who believe and say most anything. In fact, I enjoy it. As long as we're trying to get along, we can have powerful and dynamic conversations. I can learn things and we can challenge one another.

Online, there's none of this "trying to get along". Some person types something wrong. Other people pile on. Pointless argument ensues.

People agree on most things! That's the crazy thing about it. But if we only interact online through text, we're raising an entire generation that doesn't grok that fact.

Good luck!

I love writing. I used to freelance and string. I wrote a lot of speculative sci-fi that didn't go anywhere. I was an early blogger. I was an early HN'er as well. So I'm creating content constantly no matter what the current tech.

I've been watching what works. Random thoughts: every conversation should have a moderator who is responsible for the subject and tone of the conversation. The start of every conversation should include what the goal is: analysis, venting, compassion, meme, etc. Conversation participants should be rated over time based on their ability to adjust their tone based on what the moderator requests. Each person votes up and down various moderators depending on the type of material they present and how well they manage their conversations. Over time the ranking system should match up moderators and participants, but not subject areas (important). There should be some escalating trust level based on this, the ability for people to make mistakes, be forgiven, delete/apologize for bad-day remarks, etc. I think with the right system this would happen organically, though. Not sure.

Note that a popular/well-used system will by necessity not be a hugely-successful commercial one. You might make it work for a bit, but these are separate and conflicting goals.

Personally, I'm dialing back FB and moving all of my content over to locals. I want to own whatever I write and I want the option of putting things I create behind a paywall easily. I also don't want to ever worry about having a bad day and getting my life destroyed by the mob.

"...two party system continues to devolve..."

Negative. Humans form into clans. Washington's cabinet split into two groups of bitter enemies almost immediately.

Clanning and being bitter and ugly is nothing new, sadly. What's new is everybody stuck in one big room yelling at one another.

To attempt to use an innocuous example, I may be a rabid Cardinals fan and you may be a rabid Yankees fan. We could be best of friends. We meet once a week, perhaps argue some viciously about the other team sucking, and then move on to families, hobbies, and the rest of our friendship. Our friendship is bigger than our passions.

But if we were stuck in the same room together 24/7, with hundreds or maybe thousands of other Cardinals and Yankees fans, we'd probably shoot one another with bazookas. Same people, same passions and opinions, different crowd size simultaneously communicating.

The interesting question is this: at what point does it fall apart? If I brought another Cardinals fan to our weekly meeting and you brought another Yankees fan, would it still be a workable/fun lunch? Perhaps, but I bet by the time we get to six or more people all with the same passions but different opinions it would be impossible to do anything but bitterly fight. (Note: larger crowds happen all of the time, of course! But those are crowds gathered for a specific purpose under clear rules. They're not stuck together 24/7 memeing one another. Large crowds become boring crowds; once you take out any passions and passionate opinions, there's nothing left but cardboard people using pre-programmed responses in order not to get called out.)

Now multiply that thought experiment by 500 million and add in advertising money chasing emotionally-driven conversations, liking, RTing, sharing, etc. That's why some companies should not exist. They don't have a business model that is compatible with the current evolution of our species.

Great essay. This guy nails it, although I think his scholarly approach misses some key ramifications. (I'm also not sure I like "content collapse", since there's nothing unusual happening here, just everything all at once, but that's a fight for another day)

I've been kicking around this problem since 2009 and I keep getting back to different physical devices for different social contexts. It's not just that the contexts are different; it's that we need to use our bodies to show ourselves and others that we are moving into some different context. Otherwise the "programming" doesn't work.

The fundamental error here is thinking that all human communication can collapse to data. That's wrong in a ton of ways, but the idea has a helluva powerful following among us nerd.

Cool stuff. Thanks.

It's a really odd social phenomenon. It's one thing to say "We don't like X and find it in poor taste. Please don't do that"

I don't like it when folks say such things, but I understand it. Instead, what many folks seem to be saying is that based on one thing they can tell what the person is like inside. It's an amazingly-broad generalization and people seem to be enormously confident of themselves when they make it. I can look at your shoe or color of your shirt and conclude that you are a bad person inside.

I don't find that to be especially rational, but what is one to do?

You're certainly not using SCOTUS arguments for Enlightenment ideals. I sure hope not.

My argument was mean in the generic and at an abstract level. I understand that for some people this is not compelling. Conversely, if were to dive down into specifics, we would need a somewhat lengthy conversation about where the guardrails were and where the (hopefully unmovable) goalposts were.

Of course at some level of analysis free speech can cause harm, otherwise there would be no point in defending it. My admittedly-oversimplified point, which I have yet to see refuted, is that in the main, we really suck at predicting the difference between harm and progress. Many times it takes decades or centuries to sort it out. From there we can end up with the law being an ass[1] or some finer definition of legitimate public policy choices. But unless we can all admit that we suck at determining who should speak or not, we really don't have a basis to continue the conversation, legally or philosophically.[2]

It's fine to say "Let's start here, and given the current tech, governmental structures, and laws we live under, where do we go?" It's also fine to say "What is the purpose of letting people say things that can hurt others, anyway?" Pick one. I chose the second one. If you'd like to talk about the first one, that's a completely different conversation. However if you don't grok the conversation on root principles, it's unlikely any sort of more complex or nuanced conversation is going to lead you anywhere useful. That has to be where we all start.

Also, insisting man is a rational animal is yet another can of worms which I didn't open up. I don't think man is a rational animal at all, and that doesn't change my opinion or where I think the conversation has to begin.

[1] https://www.phrases.org.uk/meanings/the-law-is-an-ass.html

[2] When I say that we suck at determining who should speak or not, I mean in a moral, ethical, and public policy sense. I do not mean in a legal sense. Obviously there's a ton of case law around most all of our amendments. I'm surveying a landscape, not preparing a legal brief.

Excellent point.

We have a tech problem that is orthogonal to the free speech issue. It's making things worse, yet the principles of free speech don't go anywhere simply because other problems have arisen.

Up until a decade or two ago, everybody spoke and everybody spent most of their time not listening to one another. Instead we did other things. Today, however, we carry these wondrous little devices in our pockets which connect us to every angry person and demagogue all over the world. We can find and associate with people who believe some really eccentric things we also believe. We can spend our free time encouraging one another to become more and more radical.

I didn't comment in order to go down this path, but your point is valid. Unfortunately, speech is not the only thing under attack. Pervasive tech has changed much of the definition of what it means to be human. That's a much more involved subject (!) that seems a bit OT for now. But I agree.

I completely agree with what you're saying. I find that one of my greatest frustrations with attacks on utilitarianism is how, well dumb they make the attacks. "Would you kill one person in a scientific experiment to save a dozen?"

No, because any world in which such a trade was acceptable is not a just or fair world that I would want to live in. You have to reason at multiple levels.

I usually make online comments for two reasons. One, to see if there's some neutral way of explaining things that might slightly advance understanding of my views without being offensive, perhaps start a friendly conversation. Two, to try to put these higher-level ideas into simpler and simpler terms in an effort to find out if there's some common-language leverage point to be found.

The first goal has to avoid any kind of trigger phrase people are programming themselves to emotionally respond to. That's getting more and more difficult. More and more people just don't want to hear anything that's from a suspect source. People are treating online discourse as a game: identify friend-or-foe, engage (or support). The second goal is more fun, but I run the risk of oversimplifying. Apologies.

There has been a long list of assholes that people would have gladly shut up at the time that turned out to change us all for the better. Galileo, for instance, wasn't known as the nicest guy. He basically wrote a book calling the pope an idiot. (The guy probably wasn't a genius, but Galileo could have made his point without going that far)

Offhand I could come up with a list of dozen people who were widely unliked by the folks in charge that were critical to our advancement. Oddly, many times the more unpleasant people found their personality, the more important the change eventually brought. They killed Socrates.

ADD: Related fun read for those interested: https://www.amazon.com/Galileos-Middle-Finger-Heretics-Activ...

From a meta standpoint, it's interesting to me that the most vociferous deniers of free speech throughout history have made some version of the case "But they're wrong!" (You can add stupid, evil, whatnot. The point is that whatever they're saying, it should not be said because it is not correct)

Whereas folks supporting free speech have made the argument: maybe I'm wrong. I don't know everything, and I've been wrong before. More concisely, looking back through history a lot of those times we got so upset about people saying various things, we were wrong. Those people changed all of us for the better. Unless we continue to humbly think we could continue to be wrong, we stop evolving. Cue curtain. There's no way to have difficult societal discussions and fight for the moral and right thing for all of us if we're constantly going to use current group consensus as a measure of what can be said or not. That's not progress. That's a popularity contest.

Thank you for taking my comment in the spirit it was intended. I'm happy to oblige you.

If your goal is to play and learn about stuff, whatever you're doing is fine.

If your goal is to get basic weather information, buy a used weather station with usb access, plug it up to whatever hobby computer you already have (I have a pi, so I'd use that). Then write a cron to read from the station and write it out to some publicly-available spreadsheet or database. There are a bunch: Google, Airtable, and so on. Most all of these can be written by one POST.

Then you can figure out how you'd like to see the data. Add in a station ID for more fun and a distributed system. I'm seeing Ebay new weather stations as low as 24 bucks or so. That'd be the only cost I know of, and I think with some searching for used gear you could get it under $10.

There's a _huge_ difference between focusing on the problem and focusing on the tech. If you're preparing yourself for a world of corporate coding? Please keep learning about message queues and so forth! You'll need those skills. If, however, you just want to know the weather in various places with your own setup? Do that. Just don't do one of those things while pretending you're doing both. Each project has vastly-different parameters.

To drive this home, if I've decided I want a coder to come to my office and make a cool little way of telling the weather outside? I want the project that just does that. Code for an hour or two, then leave. If you come and start deploying a kubernetes cloud-based auto-scaling megasytem, you're coding for yourself instead of my problem. My only point was that since there is such a huge gap, when you teach people you should be clear about the hidden lessons of what you're teaching.

* Now traditionally programmers throw out a lot of "what if"s here. What if we needed split-second reports? What if we're dealing with an offline situation or a saturated net? All of those are great considerations. Also, all of those were not included in the spec I read back to you. Don't go inventing what-ifs where none existed. It's a good way to build unmaintable monsters. Hopefully you got the point. It was a minor point and not directed at you, just a comment on cool-tech articles in general. (Also, I'd love to demo actually building it, but it would probably be quite less dramatic and interesting than your version, for the reasons I explain above)

Keep up the good writing! Looking forward to reading you again.

This is a really cool project that I would perhaps like to try myself one day! A home weather station has been on my list for several years.

Now I will say something negative. It's a nit, but I think it needs to be said.

I've seen a lot of these "In which I write a (perhaps series of) essays on building X using all kinds of cool tech you'd probably like to play with when you could get the same result with less than ten bucks, maybe an hour or two, and just a handful of code, if any"

I'm okay with and completely love "Let's do something cool and learn!" I'm not okay with "I have this problem, so let me by implication teach you how you should architect problems when you run across them" We need to clearly differentiate. You should not, and you should never, start with an important project and begin speculating on what cool tech you could play with.

Playing and learning are great. Yay! Adding structure and complexity where none is needed is an antipattern if you're doing anything of value for yourself or somebody else. This stack might end up being necessary. Absolutely. Make the problem "prove" that it's necessary. You should have a clear and unambiguous process for choosing a stack; do it by necessity. Never even accidentally teach that you do it by "hey, that's cool!" People end up doing that, and somebody ends up having to clean it up.

tl;dr Love fun projects like this. Keep 'em coming. Just please be sure you're clear to the reader that they're learning/fun projects.

* Perfectly fine to add a tech or two to a new project in order to learn-as-you-work, it should just be done with the upmost care and a close eye to how much team bandwidth it's using versus the value being provided and how you'd back it out if you had to.