347 karma · joined August 30, 2013
- am a principal software engineer at PointCard! - worked at Square for 7.5 years. - studied enough applied mathematics to earn a PhD.
More about me: http://traviscj.com/ My blog: http://traviscj.com/blog Get in touch: traviscj@traviscj.com or https://twitter.com/traviscj
[ my public key: https://keybase.io/traviscj; my proof: https://keybase.io/traviscj/sigs/JevjENzfsY9OoqMKH7gl9y_hQu6o2Mji-ERRb18kXpE ]
I think I still have the “wrong” preference, but I appreciate learning some new stuff!
``` alias h3='howdoi -n 3' ```
_really_ frequently, because often the top answer wasn't what I needed.
[1]: screaming picked up by mic; detected maniacal CTRL-C'ing as immediate as it is futile; permanent loss of peer relationship to the project's cloud (indicating subsequent reformatting and/or booting-off-tall-building)
> Characters saved with "<"? 3. > Hours lost with accidental ">"? unbounded.
Preface: I'm confident on the shell & with my keyboard: I've been using shell redirects for a couple decades, learned touch-typing up to around 80WPM, and used to note-take math lectures live in latex. Yes, I have noclobber enabled. But...
I can't promise myself I'm on a machine configured that way OR that I won't typo this one on a file that really matters.
I kinda use of `cat` as a `immutability-please` command: it _creates_ a barrier/seam between this file and whatever I'm doing with it, such that I'm _guaranteed_ nothing downstream will mutate it (as long as everything only works on stdio).
In a kinda similar way: by starting with `cat x |` (or even just `cat x`, frequently!), the rest of that compound command is _input source independent_ (and even _ignorant_!) -- if I want to hook it up later to a `curl example.com/xyz.json`, I don't need to split it into separate commands/introduce temp files/etc. This is especially because I find myself doing a _really_ quick scan of the entire file ("did it return the actual object this time or just `{}`?"), then piling on transforms from there.
Something about doing `cat x | head` and then kill-wording to fill in `next-thing` feels faster and more consistent than `head x` and then `next-thing x`, maybe because the filenames are often much longer than the commands? Maybe I need a "go to start of line, forward-kill one word" hotkey?
tl;dr: i'm playing psychological games with myself and TBH not sure I'm coming out ahead.
impossible to understate the value in: not frequently fucking up your ability to run the critical/mundane/everyday/musclememory commands you've come to rely on after maybe 100+ invocations/week as far back as you can remember just because someone you've never met wanted to bump the .ruby-version in your java monorepo
I don’t have a proof, but For things like state transitions, I’ve often solved this with a <new_state>_at column. Seeing that field populated would indicate a second event need be emitted. (Assumes no cycles in the state machine.)
Often that table is not the one you have, even if (perhaps especially if) you emitted events for every update — seems like some entropy-like force pushes the universe toward emitting details in events that aren’t reflected in the table records, over a long enough timeline.
Because it runs contrary to maintainer’s interests?
I have neighbors with mechanicals that bother me so bad I’ve had to pack up and go home for the day. This is over music on noise canceling headphones.
Different people actually have different amounts of trouble with this, with some having lots of problems. Just lucky I guess?
I’ve been similarly untrusting of a lot of “high tech” approaches to various things, and I derive a lot of joy from products/services/etc that take a “back to basics” or at least minimally- or non-digital philosophy. In particular: I have an affinity for automatic watches and carbureted motorcycles.
If nothing else, it’s a bit of a break from what feels like a constant struggle to keep all the gears turning at work.
BUT... I’ll confess I’m also a sucker for innovations / the occasional new hotness. I recently upgraded a Kawasaki KLR650 (a competent but... “well tested”, shall we say? motorcycle) in favor of KTM’s top-of-the-line adventure bike. The technology difference between the two (despite only 3 model years between them) is incredible: the latter adds 5x the power, ride-by-wire, cruise control, lean-sensitive ABS/traction control, an up/down-capable quickshifter, probably a thousand other improvements.
One day, about 1100 miles into owning the new bike, the dash pops up a low tire pressure warning from the tire pressure monitoring system. It showed the rear pressure was fairly low, and sure enough, I’d picked up a small screw between the treads.
Certainly a TPMS is nothing compared to anything self-driving, but honestly it was a bit of a wake-up call — I WANT systems on my bike to increase my safety level.
I’m not really sure what the lesson is here. Maybe “Look for the middle grounds (the ABS/TPMS-maturity systems) between ancient technology (anything on my beloved KLR) and bleeding edge (non-replicable papers on self-driving cars)”? Seems like this holds up ok, especially as a consumer of those techs... But maybe not for the innovators?
WORST—-and I mean WORST—case, could crank out a clone during a hack week.
I've been building systems on some pretty similar ideas for the last 5+ years, and have a similarly optimistic outlook on it. Just to add another datapoint here, we've been able to scale them up to 20-40x higher than your figure (a couple thousand jobs/sec) on a single (albeit beefy) MySQL instance. It definitely takes a fair bit of care & we certainly have some time invested in it, but it can be done.
I guess my services haven't had (gotten to? :-) ) to share databases with other services for quite some time -- they did when I started my job! -- but even with that concession, the benefits of DB guarantees have been amazing. There's nothing like a transaction for preserving your sanity.
The best tactic I've learned from these systems is to think really hard about the batching & partitioning behavior to keep msgs/sec high enough & transactions/sec low enough to stay afloat.
In seriousness, though, how do you build up this level of comfort/familiarity with the tax code? Did you do it professionally?
[1]: https://www.consumerismcommentary.com/verify-funds-on-that-c...
Fun story: A coworker and I are in the process of tracking down exactly violations of this in a misbehaving app, due to an accumulation of RPC within transactions across the entire app. (I used to be responsible for it as a greenhorn ~5years ago, but walked away for the last several years to work in a different part of the org.)
The approach is actually kinda interesting: we add an interceptor in the RPC layer for every outgoing call that asks the DB machinery “am I currently in a transaction?”, and emits a warning (containing at least the RPC being made, perhaps a (partial) stack trace) if so. We had so many occurrences of those warnings that our logging framework proactively started dropping log lines! So the next step is to only emit the warnings once per $duration per message.
Another day in the life...
"give up connection after every query" is, IMO, not _less_ of a footgun, but a _more subtle_ footgun. That is to say, sure, for the most part, it will work ok.
_But when it doesn't_... like when your database goes down for a couple hours, or your rack loses network, or your proxy to whatever cloud provider is running your database, or maybe some other dependency had an outage between the queries... you end up not really knowing what executed and what didn't, with some fraction of your 10k req/sec leaving behind some dangling state -- a partially processed $whatsit -- and no clear/easy way to fix it or even necessarily identify how many $whatsits were affected. At least a few times, I've needed to either wait until records make their way into some OLAP system for offline analysis, or deploy some fixup code, or emulate either/both to fix stuff up after something weird happens.
At least that's what I am constantly afraid of. Do you also worry about this kind of stuff? If not, what prevents it from being a problem in your environment?
Signed,
The guy constantly badgering seemingly everyone about minding their transaction boundaries.
I’m still on the fence about whether that justifies their protocols. I think I actually lean toward “no”, but I’ve also lately become keenly aware of the difficulty of even simple things like keeping everything up to date, and my lack of real insight into what those updates include. If I’m effectively trusting them anyway, might as well trust them to get it to me ASAP, right?
I’m also enough of a realist to assume there’s a Fight Club style “A times B times C > X” reputational/financial risk logic going on here. If there’s few enough of the devices out there, it’s probably cheaper to apologize (legally, as in settle).
I’m also kinda reminded of aws lambda URLs, or I guess URLs in general.
I’ve been really distracted lately thinking about what things need named handles in my data models/representations, and what they should be handles for, and how to do so, and how to change them in sane ways. So far all my thinking has just convinced me that I am a mediocre architect.
1. It was probably a tax write off.
2. It was probably also both a community service project AND a reflection of lower expected insurance payout.
3. Maybe the straight-A students are correlated with insurance payers that would rather pay the repair known 1-off payment (or skip doing anything at all) vs the unknown recurring increase in insurance payments?
(Source: drove a dented-up pickup for over a year after a short romp along/into an orchard)
[1]: https://www.amazon.com/Sony-ICDPX370-Digital-Recorder-Built/...