HNHacker News
TopNewBestAskShowJobs

rented_mule

1,755 karma · joined July 4, 2021

submissionscomments
rented_mule··on PawSense: Catproof Your Computer
Late one evening, about 15 years ago, we were wondering why we were seeing several hits per second from a single IP address on our company's search page. It didn't follow any of the patterns of the (admittedly simple) bot detection we had in place. It wasn't bad enough to be a problem, but we were trying to watch some real-time metrics and it was a distraction.

We had a monitor on the wall showing the most popular search terms over the past hour. A few minutes into the event, we saw a search term steadily move to the top of the list. It was something like `'[]`. After thinking about it for a few minutes, we concluded a user left their browser on our search page, a cat stepped on their keyboard in just the right pattern, and then sat down on the F5-key (i.e., refresh key).

No way to know if we got it right, but it was the best we came up with in the 20-ish minutes before it stopped. Oh the things you'll diagnose...

rented_mule··on Pyinfra: Turns Python code into shell commands and runs them on your servers
pyinfra is for running things across a number of machines and/or containers. For example to provision them, much like Ansible. So Python's startup time isn't terribly relevant.

Also, have you tried Python's startup time in the last decade or so? On my 2-year old, $150 computer with an Intel N100 (a very low-end CPU) running Ubuntu, `python3 -c "print('hello')"` takes 15ms. On my 8-year old Mac mini, it's 35ms. Both are significantly faster than what humans perceive as instantaneous, so Python's startup time is more than fast enough for many things.

rented_mule··on Ask HN: Is it time for HN to implement a form of captcha?
My favorite code editor is where I write anything longer than a sentence or two, including this comment. Then I paste the result into its final destination. Given that I tend to heavily edit my writing, the muscle memory I've built up in that editor helps me focus more deeply on my writing. So I guess I'll look pretty suspicious? Let me add an em-dash — everyone knows that real humans don't use those. ;-)
rented_mule··on Ask HN: How to go back to listening to MP3s?
I've been using https://www.beatunes.com/en/beatunes-matchlist.html (35€ for Mac or Windows) to create static playlists like this for years from my library of ~13K songs. I've created a few dozen such playlists with light curation (just kicking a song out here and there). I find myself listening to these playlists far more than the ones I created completely by hand.

I suppose you could do it often, or maybe even script it (e.g., with Keyboard Maestro on Mac) to get something a little more dynamic. But it's not something that just matches songs on the fly server side.

rented_mule··on Python numbers every programmer should know
What many startups need to succeed is to be able to pivot/develop/repeat very quickly to find a product+market that makes money. If they don't find that, and most don't, the millions you talk about never come due. They also rarely have enough developers, so developer productivity in the short term is vital to that iteration speed. If that startup turns into Dropbox or Instagram, the millions you mention are round-off error on many billions. Easy business decision, and startups are first and foremost businesses.

Some startups end up in between the two extremes above. I was at one of the Python-based ones that ended up in the middle. At $30M in annual revenue, Python was handling 100M unique monthly visitors on 15 cheap, circa-2010 servers. By the time we hit $1B in annual revenue, we had Spark for both heavy batch computation and streaming computation tasks, and Java for heavy online computational workloads (e.g., online ML inference). There were little bits of Scala, Clojure, Haskell, C++, and Rust here and there (with well over 1K developers, things creep in over the years). 90% of the company's code was still in Python and it worked well. Of course there were pain points, but there always are. At $1B in annual revenue, there was budget for investments to make things better (cleaning up architectural choices that hadn't kept up, adding static types to core things, scaling up tooling around package management and CI, etc.).

But a key to all this... the product that got to $30M (and eventually $1B+) looked nothing like what was pitched to initial investors. It was unlikely that enough things could have been tried to land on the thing that worked without excellent developer productivity early on. Engineering decisions are not only about technical concerns, they are also about the business itself.

rented_mule··on Starlink satellites being lowered from 550 km to 480 km
It seems plausible that it would help with latency, but not by a game changing amount. This is 70/550 or 13% closer. Not all of the latency is caused by distance to the the satellites (e.g., terrestrial latency is not 0), so my guess would be that the latency experienced by users would improve by less than 13%.
rented_mule··on Go away Python
I just tried the one you are replying to and it worked great on macOS. I frequently use a variant of this on my Mac.
rented_mule··on What makes you senior
It's subtle, but I think the use of "senior" rather than "Senior" in the article is an attempt to distinguish the concept of being a senior engineer from the title of Senior Engineer. The article is focused on actually being senior, not playing title games. I'd take it further and use the term "leader" instead of "senior engineer".

Leaders reduce ambiguity, so others can operate with more clarity. The ambiguity involved can be in many different domains. It can be focused on product and tech, as in the article. Another example is ambiguity around people and organizational structure, which is more common in management roles, where some in management are more effective leaders than others. It can be around finding ways for people to understand why they might want a product, which is more common in sales and marketing roles. And so on.

rented_mule··on Over 40% of deceased drivers in vehicle crashes test positive for THC: Study
A few years ago, I moved from San Francisco to a rural area. Smelling weed in SF was not at all unusual. One summer night in the rural area, I smelled it coming through open windows for the first time. I wondered which house it was coming from and how it still smelled so strong after traveling a hundred feet or more. Then I spotted the actual skunk in our yard.
rented_mule··on AWS CEO says replacing junior devs with AI is 'one of the dumbest ideas'
> Software has been unreliable for over a decade

The "over" deserves a lot of emphasis. To this day, I save my code at least once per line that I type because of the daily (sometimes hourly) full machine crashes I experienced in the 80s and 90s.

rented_mule··on A “frozen” dictionary for Python
It's a little early to say that "Python is shipping" anything related to this. The PEP is still in draft status and it may be modified (as it was three hours ago) or even rejected. The article says it is likely to be accepted, but that has not happened yet. That also means there is time to comment on the PEP in the discussion it links to if you'd like.

https://peps.python.org/pep-0814/

rented_mule··on The programmers who live in Flatland
> The idea of a linear hierarchy in languages is the true flatlander mindset.

100% this. I think you can replace "languages" in that sentence with many things (employee levels is another big one that is relevant to this forum - employee value comes in many, many shapes). Reducing complicated things to one dimension can be a useful shortcut in a pinch, but it's rarely the best way to make complicated choices among things.

rented_mule··on How good engineers write bad code at big companies
I've seen too much of the same. It strikes me that the pattern you describe also matches a lot of AI generated code I see, especially when it's big chunks of generated code. Are we automating this problem and going all-in on the long term costs?
rented_mule··on Stop Telling Us XMPP Should Use JSON
XML and JSON only started about 5 years apart (1996 vs. 2001). I think XML's adoption was more broad more quickly, so the gap felt a bit longer than that (XML was a core part of my jobs from 1998-2007 and JSON ever since). I think more of what makes XML look so much older is the fact that so many new projects have been picking JSON over XML for 10-20 years now, and that's why younger folks have seen much more JSON than XML.

My software engineering career started long before either, and I have used both extensively. While I don't think that it makes sense for most well established projects to switch, just as rewrites often don't make sense, I would almost always pick JSON for new projects. Your points are valid, but one part of the ergonomics that you didn't mention is that it's so much easier for humans to read and write JSON, thereby speeding up development, debugging, testing, etc.

rented_mule··on Tesla's ‘Robotaxis' Keep Crashing—Even With Human ‘Safety Monitors' Onboard
Self-driving cars also require us human drivers to learn new defensive-driving skills.

A month or two back, I was driving down a steep one-way, three-lane street in SF, late on a rainy night. I saw a Waymo stopped at the left curb and I moved to the from the left lane to the center lane in case it started to pull out into the left lane. There were no cars in front of it or behind it, so I was shocked to see it quickly leave the curb at about a 45 degree angle, as if it were pulling out of a tight spot with a car parked just in front of it, but much faster. If I saw a human driver doing that, it would almost certainly mean they are trying to get all the way across the street immediately. If it was doing that, there is no way I could stop in time on the wet downhill. I tried, but that just made steering difficult as my anti-lock brakes struggled to find any traction at all. Then, just as quickly, it straightened out in the left lane. I'm glad I was the only other car around.

One element of defensive driving is thinking about how to avoid surprising other drivers. When will self-driving cars' defensive driving rise to that level? Waymo certainly wasn't there in that situation on that night.

rented_mule··on Creating an all-weather driver
All the school buses near where I live (Sierra Nevada mountains in California) have these - it's cool to watch them lower and start spinning.

But chains aren't enough in some common situations around here that locals, including school bus drivers, know well. When we get a good size snow storm (multiple feet) and the sun comes out a day or two later, thick ice forms on the sections of road that the sun hits - snow melt runs across the road during the day and freezes at night, getting thicker and smoother each day. When that happens on our steeper inclines, chains on AWD/4WD vehicles are not enough to get up those inclines or to stop on the way down them. Locals know where those spots are and take other routes in those situations. It's hard for me to imagine autonomous vehicles having such local information in remote areas like this anytime soon.

rented_mule··on The Privacy Theater of Hashed PII
Not calling it anonymous is an improvement. Before I retired, I read many "anonymous" surveys taken by my reports. Any free-form text in the survey that goes beyond a sentence fragment usually made it obvious who wrote it. At least in the case of my teams, writing styles tended to be pretty distinct, as were the things each person cared about enough to write at any length. I tried to ignore the clues, but it was usually so obvious that it jumped out at me. The people administering such things insisted that anonymous meant their name wasn't on it, so it was fair to call it that.
rented_mule··on Don't Be a Sucker (1943) [video]
> This video, to me, seems to deride it.

I don't see any derision of the first amendment or of the public square (not sure which you were referring to as "it" in your last sentence). When we exercise our freedom of expression, we have zero guarantee that we will be listened to, believed, or respected.

The derision I see in this video is directed at visceral belief in whoever is shouting in the public square, especially when their message is so clearly divisive. The discussion between the Freemason and the naturalized citizen is itself a fine example of free expression in the public square.

rented_mule··on Empathy for Dummies
Isn't it possible for someone to spot a growth opportunity for someone else without they themselves being perfect, or maybe even good, at that thing?

I've certainly benefited from useful ideas in my core area of expertise from people who were nowhere close to being experts in that same topic.

rented_mule··on Magic Wormhole: Get things from one computer to another, safely
> A modern version should, of course, work in a browser, so that there'd be no Python packages to install

Now, only uv needs to be installed (two self-contained binaries). Then you can do `uvx magic-wormhole --help` and it will quickly create a transient environment with the required packages. You don't even need to install Python (uv will do a transient install of Python as well).

rented_mule··on Binary Formats Gallery
In addition to languages, there's a Python library called "construct" that's been around for a long time. It uses a declarative style to make it surprisingly easy to make binary parsers and emitters.

https://construct.readthedocs.io/en/latest/intro.html#exampl...

rented_mule··on Introduction to Multi-Armed Bandits (2019)
I was just mentioning that any feedback loops have the potential downside of coupling things together in ways that make experiments (and analysis, improvements, etc.) difficult.

I used control theory as an example of another type of feedback loop because we were using controllers in other parts of the system. One was roughly the proportional term of a PID controller. Another was both the proportional and integral terms, but with a lot of domain specific heuristics added on (e.g., ML models to set baselines that help the controllers understand how quickly to move values). This was all incredibly powerful, but also required experiment design and analysis that was far more complicated than we would have liked.

This was not a ride-sharing company, but you can imagine dynamic pricing for Uber or Lyft working this way. Use a controller to try to keep the number of available riders and drivers in balance. If riders exceed drivers, increase payment to drivers (and therefore cost to riders), thereby attracting more riders. If drivers are sitting idle, decrease payment. Ideally, this is done constantly to respond to time of day, weather, traffic, etc. (with some predictive baseline expectations to keep it from being too slow to respond). When you get it right, riders get the cheapest price that ensures a driver is available. In economic terminology, this is finding the market clearing price. A controller can get you a long way in a hurry on such a system. But it's certainly not the only approach that can work, and it takes a lot to get it right (I have no idea how Uber and Lyft actually do any of this).

rented_mule··on Introduction to Multi-Armed Bandits (2019)
Agreed. In the case I was describing above, new arms were constantly being introduced (often several times a day for each of hundreds of thousands of scenarios). Manual experiments weren't an option. This also meant we were in a constant state of partial convergence for most scenarios, but the same would be true with experiments.

How to cull arms, so that there are enough samples for any kind of convergence, is another problem in this setup. We eventually built an ML model to select arms and used bandits to pick between them. This proved "too effective". The arms were all user-generated content. The bandits on top of the model, both setup to maximize clicks was stupidly good at surfacing inappropriate content because it got a lot of clicks. We ended up having to put more safeties on arm selection for certain categories of our content where we had the most inappropriate submissions.

rented_mule··on Introduction to Multi-Armed Bandits (2019)
Yes, this is exactly the kind of scenario I was alluding to.

For example, cohorts with very light traffic are likely to get undue benefit as a lot of exploration might be done before the smaller cohort needs to select an arm, so things are closer to convergence.

Another example is if there are wildly different outcomes between cohorts. More of the exploration will be done in cohorts with more traffic, leading bandit optimizations to fit large cohorts better than lower traffic cohorts.

Even if you do manage to make things independent, you have to wait for bandit convergence before you know what converged results will look like. That requires being able to measure convergence, which isn't always easy, especially if you don't know to do that before designing the system.

With all of these problems, we kept bandits, and even expanded their application. At least for the 10 years I was still around. They are incredibly powerful. But there was a lot of "I wish those damned bandits didn't work so well!"

For anyone who is not aware, A/B tests assume cohorts behave independently of each other. The less true that is, the less reliable the results are. This was even worse for us in parts of our system where there were no bandits, but there was direct interactions between individuals.

rented_mule··on Introduction to Multi-Armed Bandits (2019)
We employed bandits in a product I worked on. It was selecting which piece of content to show in a certain context, optimizing for clicks. It did a great job, but there were implications that I wish we understood from the start.

There was a constant stream of new content (i.e., arms for the bandits) to choose from. Instead of running manual experiments (e.g., A/B tests or other designs), the bandits would sample the new set of options and arrive at a new optimal mix much more quickly.

But we did want to run experiments with other things around the content that was managed by the bandits (e.g., UI flow, overall layout, other algorithmic things, etc.). It turns out bandits complicate these experiments significantly. Any changes to the context in which the bandits operate lead them to shift things more towards exploration to find a new optimal mix, hurting performance for some period of time.

We had a choice we could make here... treat all traffic, regardless of cohort, as a single universe that the bandits are managing (so they would optimize for the mix of cohorts as a whole). Or we could setup bandit stats for each cohort. If things are combined, then we can't use an experiment design that assumes independence between cohorts (e.g., A/B testing) because the bandits break independence. But the optimal mix will likely look different for one cohort vs. another vs. all of them combined. So it's better for experiment validity to isolate the bandits for each cohort. Now small cohorts can take quite a while to converge before we can measure how well things work. All of this puts a real limit on iteration speed.

Things also become very difficult to reason about because their is state in the bandit stats that are being used to optimize things. You can often think of that as a black box, but sometimes you need to look inside and it can be very difficult.

Much (all?) of this comes from bandits being feedback loops - these same problems are present in other approaches where feedback loops are used (e.g., control theory based approaches). Feedback mechanisms are incredibly powerful, but they couple things together in ways that can be difficult to tease apart.

rented_mule··on Total porn ban proposed by Michigan lawmakers
So, for example, Viola / Cesario in Shakespeare's Twelfth Night. Who needs great literature? https://en.wikipedia.org/wiki/Viola_(Twelfth_Night)

And regarding "the other biological sex"... It is not at all simple to define biological gender in a such a way that there are only two of them. Genitalia, hormones, and chromosome patterns are far from sufficient. In many cases, the gender assigned at birth is arbitrary. https://en.wikipedia.org/wiki/Intersex

rented_mule··on Charlie Kirk killed at event in Utah
> What change was the person who shot him hoping to elicit?

I think a difficulty in searching for such answers is assuming that it was a well reasoned decision. I'm not sure how often attempting to take a life is a purely rational decision, devoid of intense emotional motivations (hatred, self-preservation, fear, revenge, etc.). And that's all assuming the assailant was of somewhat sound mind.

I think one of the dangers of more and more extreme divisions in society is that those divisions cloud our mental processes, threaten our emotional health, and take away opportunities for meaningful civil discourse. All of which can lead to more heinous acts that we struggle to make sense of. One of the scariest parts for me is that this can all be too self reinforcing ("Their side did this bad thing to our side, let's get them back!!!" repeat/escalate...). How do we break the cycle?

rented_mule··on A high schooler writes about AI tools in the classroom
An older analog is calculators. My college intro to stats course didn't allow them. We did simple arithmetic by hand and looked up things like roots and logs in tables. I still have my copy of this: https://www.amazon.com/dp/0849306922

I just tutored my nephew through his college intro to stats course. Not only are calculators allowed, but they had a course web app so that all they did was select a dataset, select columns from those datasets, and enter some parameters. They were expected to be able to pick the right technique in the app, select the right things, and interpret the results. Because of the time savings, they covered far more techniques than we did in my day because they weren't spending so much time doing arithmetic.

Despite lots of cries about "who will know how to make calculators?", this transition to calculators (and computers) being allowed was unavoidable because that's how these subjects would be applied later on in students' careers. The same is true of AI, so students need to learn to use it effectively (e.g., not blindly accepting AI answers as truth). It will be difficult for the teachers to make their lesson plans deeper, but I think that's where we're headed.

Another lesson we can draw from the adoption of calculators is that not all kids could afford calculators, so schools sometimes needed to provide them. Schools might need to provide access to AI as well. Maybe you are required to use the school's version and it logs every student's usage as the modern version of "show your work"? And it could intentionally spit out bad answers occasionally to test that students are examining the output. There's a lot to figure out, but we can find inspiration in past transitions.

rented_mule··on Why Dishwashers Are Quietly Disappearing from American Homes
Many show up on Google:

https://www.google.com/search?q=leak+sensor

and on Amazon:

https://www.amazon.com/s?k=leak+sensor

rented_mule··on Anything can be a message queue if you use it wrongly enough (2023)
In the 1990s, I was at a startup that had a need for a message queue. The only thing we found at the time was a product from TIBCO that was priced way-way-way out of our reach. IIRC, it didn't even run on PCs, only mainframes and minis. Microsoft Exchange Server (Microsoft's email server) had just been released at the time, and we decided to use it as a message queue.

Message-submitting clients used SMTP libraries. Message-consuming clients used Exchange APIs. Consumers would only look at unread messages, they would mark messages as read when they started processing, and move them to a folder other than the Inbox if they succeeded. Many of the queues were multi-producer, but all queues were single-consumer (CPUs were pricey at the time - our servers were all Pentiums and Pentium Pros), which simplified things a lot.

Need a new queue / topic? Add an email address. Need to inspect a queue? Load up an email client. An unexpected benefit was that we could easily put humans in the loop for handling certain queues (using HTML in the messages).

It worked surprisingly well for the 5 years that the company was around. Latency was okay, but not great. Throughput was much better than we would have hoped for - Exchange was almost never the bottleneck.

← PreviousPage 4 of 12Next →