If it helps, this was discussed just yesterday: https://news.ycombinator.com/item?id=27882652
778 karma · joined April 4, 2017
Which makes me curious...outside of teaching and PL research, is anyone else using Racket? Has it been used in other CS research fields perhaps? Some of the items in this changeset makes me think it might be used in numeric computation/stats modelling. Is it seeing some use in the industry as well? It seems to have a pretty extensive ecosystem but are the libraries mature?
Would love to hear what/how people have been using Racket.
A correct solution, even when not clean, isn't going to earn money. No one (except perhaps textbook publishers) will pay for a solution that is clean alone.
You can clean up a correct solution at a later date, all the while maintaining correctness; a clean (but incorrect) solution might need to be completely redesigned to make it correct.
I've come to accept that the best engineers are those who ship (you don't get to 10x productivity if you don't ever ship) and that often means being able to call the correct compromises to make. Heck, I'd even say engineering in general is finding a working compromise given a set of constraints.
I would have let this slide but I really don't like it when people put words in my mouth. Check your previous post, read carefully, you are the first person to use those words :).
> Isn't this basically a variation of https://pbs.twimg.com/media/B_QAQumXIAAZ-re?format=jpg
No it is not. You're telling us to use car tires when we are on bikes. Car tires are nice but it's useless for me when I am riding a bike.
> That I'm either already logged in or that I can login in "2 taps" (very dubious, specially for old.reddit.com)
This is a reasonable assumption because, I quote myself:
> Of course I made the assumption that you are already logged in to Reddit, because that's the type of person the usage of this bot is for anyway. If you browse without an account, this is not the bot (or service) you are looking for.
I grant you I did overlook old.reddit.com does not have OAuth. But browsers do save your login credentials, which is even faster.
> That I can search whether there is an existing link with no effort or taps (easier with old.reddit.com, at the cost of swiping, unlikely with the new one), or that I don't otherwise care about telling everyone that I have downloaded the video.
I'll grant searching for an existing link isn't the easiest thing in the world but the upvote system kinda compensates for this.
You can delete comments if you don't mind announcing to everyone you DLed a video. Or use an alt. Or if you do it rarely enough, yeah why not a downloader site.
Again, this bot is for people invested in Reddit's ecosystem.
> * That the bot is going to reply immediately and not make me wait an indefinite amount of time, and that I will not need to manually tap refresh N times to see when it is finished (which is the case in old.reddit.com).
That's why you have notifs. I mean if you need to download this video right now then, maybe this bot isn't for you. But most people don't need right now.
> Does that really change the argument? Would that make it "useful" instead of "shameless self promotion"? > > Of course not.
Except it does. If you add the DL link to each of your comment that also happens to link your product, you become _that guy_ who's always provides a link to the video download without being asked to. It's useful precisely because you anticipated what the user would need your services for. I'm baffled I need to spell that out.
If I stuff every bulletin board with a flyer for my computer repair services, I'm just another voice in that messy pile. But if I repair people's computers and then give them my calling card/flyer, they are likely to not find that annoying because I had a good pretext for advertising my services.
Well, that's you. It might surprise you to know, not everyone is you.
> Basically, the "unconvenient" method you are describing is literally fewer taps away than the "convenient" method of messaging SaveVideo, which has no autocomplete at all, and _requires me to login to Reddit_
Of course I made the assumption that you are already logged in to Reddit, because that's the type of person the usage of this bot is for anyway. If you browse without an account, this is not the bot (or service) you are looking for.
But I'm gonna humor you and count taps, tell me where I'm wrong. I'll go for the shortest actions possible. Unconvenient method:
1. Find the URL (+1 swipe to reveal address bar), copy and paste URL (+1 long tap and +1 short tap).
2. Tap on bookmarks (+1). Search for your DL service ((+1) swipe, depending how much bookmarks you use), tap on actual bookmark entry (+1).
3. Paste URL from Reddit (+1 tap minimum. If it did not autofocus to textbox, that's another (+1)). Tap action button to DL video (+1).
4. Download video finally (+1).
Total: 8 definite taps + 2 possible swipes + 1 possible additional tap if DL website does not autofocus on textbox) + the cognitive load of opening a new browser tab.
Convenient method:
1. Type command. Let's make things equal and assume autocomplete (this is a string you type frequently), score it +2 taps. Hit comment button (+1).
2. Open notifications (+1), open comment (+1), click link to download (+1).
Total: 6 definite taps.
I'll humor you further, if you had to login (and let's assume OAuth, Google login to make it equal to having a bookmark on hand) you tap on the login link (+1) and tap on the Google option (+1).
Total: 8 definite taps without the inconvenience of going out of Reddit.
Where's the "extreme flaw" in my reasoning, please?
> the only reason SaveVideo is popular is because their viral spamming strategy actually works
I really find it disingenuous of you to keep harping about a "viral spamming strategy" without mentioning that video download sites are guilty of excessive ads and dark patterns more often than not. "Use adblock!" of course, but even that is not 100% foolproof or they would annoy you by making you wait before your DL link is available. You were supposed to be seeing an ad while the wait countdown is on by the way.
I did not even count the cognitive load of these dark patterns in my accounting above (is the download link the green "GET VIDEOS" button or the blue "Save" button?) because it's too varied to settle a proper score.
SaveVideo might be a viral ad strategy but yet another difference with the ad strategy of these DL sites it is that SaveVideo is a useful trick to know for Reddit users; i.e., that ad is far more relevant. The people who learn about it might actually find use for it one day.
That's not the only reason they are popular too. It's also because...
> If on every thread of Reddit I was posting a message about my magnificent browser extension/bookmarklet/website for downloading videos, how popular do you think it would get, convenient or not?
Another aspect of these bots is that one person invokes it, the rest of the community can benefit from it. Plus, though my opinion is unchanged on the subsequent thread of "good bot" comments, the community finds that kind of behavior charming. This behavior you describe is just shameless self promotion, I'd wager you'd get more detractors than users before your account is moderated out of existence.
Man, classic HN short-sightedness. Not everyone is comfortable with these tools. Not everyone has a Python interpreter installed by default. Not everyone is in a device/platform that can even run Python.
These bots are just damn more convenient. Assuming you are accessing old.reddit.com in mobile (the One True Way), to download a video you need to:
1. Find the URL. With mobile browser's penchant to autohide UI chrome, there's an additional cognitive load in doing it.
2. Open a new browser tab.
3. Fumble on your mobile keyboard to type the address of your preferred save video service. (Alternatively, you Google search the name, scroll past the first "result" because it's an ad, then click.)
4. Paste the URL from Reddit.
5. Wait and download video.
In contrast with bots:
1. Fumble on your mobile keyboard to type "!SaveVideo" (or whatever command it is).
2. Wait for the notif and download video.
> you don't spam people who is discussing in the same thread
Is that honestly spammy? I rarely see more than one subthread for the bot call, it gets upvoted so people can see it easily, sometimes people do search for it first. The worst I can say is that it quickly devolves into a stream of "good bot" comments but at that point, it's just another conversation to collapse.
WRT presenting information, there are more problems with Reddit's design decisions than these bots.
What's nice to watch even in bird's-eye wide-field view are great dribblers (like Messi) and great team tactics (if you know what to watch out for).
> Is it possible that football clubs are optimizing for the wrong things?
Man, tactical philosophy is another religous war to get into. Mourinho vs. Guardiola could be an ideological debate to rival our beloved tabs vs. spaces.
If you're trying to get into this, I'd say find a team whose narrative you like and start with them.
> Would there be a way to change the game so that artistry is stimulated
I think this depends on team tactics. The tactic dictates the behavior of individual players.
I haven't followed football as closely in the last two years but I think your best bet for teams like this is either Paris St-Germain or try to watch games in the South American leagues. After all it's the South American (Brazil, in particular) players who are well known for this.
(As for PSG, they paid a king's ransom to get Neymar in their team, and Neymar loves to play like he's still in the streets of Brazil. So I assume they built a team and tactics that allows him to do what he loves.)
Another factor is the difference of team dynamics in Club vs International football. The first three you mentioned are all international tournaments (nations play each other) while UCL is club football.
Consider that national teams only play a handful of times over a season (generally 2x per international break), and then on to a big event like the Euros. And all the other times they play outside of a tournament (friendlies, tournament quals) their manager is trying to win and tweak the team at the same time. Not to mention, the selection pool for a national team is very limited compared to what even a mid-table club could assemble (Arsenal, looking at you :P ).
So, national teams playing this tournament are not only stressed, they are also less-acquainted with each other.
I guess you could argue for comparing Euros and Copa America but another thing to keep in mind is the variance of skill level per player in this tournament. I find it reasonable to claim the variance is less in Euros than in Copa (quick reason: Messi, Neymar, Aguero, Coutinho, Firmino, etc. all play in European clubs but are eligible for Copa).
This gap in skill in Copa makes OGs less likely. South American defenders don't always deal with Messi or Neymar but the European defenders and strikers could've played (with or against) each other in club competition. This makes close situations leading to OGs more likely.
I think football has made this transition a long time ago. I wouldn't argue exactly when but consider how extremely talented players with lots of showmanship rarely perform well at an elite level on guile and "artistry" alone. Ronaldinho comes to mind vs. the athleticism and discipline of CR7 and Messi's games respectively.
Personally, I like the hypothesis that tactics are now so data-driven [1] it's catching defenders unaware more often than not, a prime situation for getting an own-goal counted in your account. I think "more intricate play in the final third" (Burton in the article) is a result of this. Max the xGs in your striker positioning [2]!
I'm also certain fatigue plays a part here (so, similar to Thorsvedt in the article). Playing schedule of an elite athlete is already brutal but the pandemic shifting and compressing timetables definitely did not help. I'm inclined to think defenders react ever so slowly in this contest.
In short: It's a weird time to be playing elite international football.
[1] Can't be bothered to look up a proper link but Liverpool's PL-winning team was touted to be data-driven, with analysts informing Klopp's selection and strategies.
[2] This is just my loose claim but someone with access to the games' data could verify this.
However, I (an armchair linguist, I should note) am also a big adherent to the Sapir-Whorf Hypothesis: your language affects your world view. And in terms of gender, there's at least research[1] that shows linguistic gender affects our perception of things. Of course, as with all research, it's open to critique, but I don't have the training for evaluating research in this field. Maybe someone else better equipped could chime in.
Interesting quotes:
> In one study, we asked German and Spanish speakers to describe objects having opposite gender assignment in those two languages. The descriptions they gave differed in a way predicted by grammatical gender. For example, when asked to describe a “key” — a word that is masculine in German and feminine in Spanish — the German speakers were more likely to use words like “hard,” “heavy,” “jagged,” “metal,” “serrated,” and “useful,” whereas Spanish speakers were more likely to say “golden,” “intricate,” “little,” “lovely,” “shiny,” and “tiny.” ... This was true even though all testing was done in English, a language without grammatical gender.
> Look at some famous examples of personification in art — the ways in which abstract entities such as death, sin, victory, or time are given human form. How does an artist decide whether death, say, or time should be painted as a man or a woman? It turns out that in 85 percent of such personifications, whether a male or female figure is chosen is predicted by the grammatical gender of the word in the artist's native language.
The second quote struck me in particular because when I first came across this, I just had a vacation in Paris where I visited the Pantheon. In one of the Pantheon's pillars is a depiction of Death (French: La Mort, feminine) and it struck me as very curious that in this depiction, Death is female. The only other work I knew of to do that was Neil Gaiman's Sandman.
(This is already long enough, but going back to the topic of "Latinx" terms...I don't intend my point above as an endorsement of this practice. In fact I find them quite virtue signalling. I have more thoughts on this but I think the majority of gender (and racial) activism is distracted on bike shedding issues such as using this "x-form". But enough said for now.)
[1] https://www.edge.org/conversation/lera_boroditsky-how-does-o...
Which makes it problematic when we make HR people do the initial screening. They would filter out a lot of good candidates just because they used Python instead of Java, or had been working on a C++ project one year less than the person leaving the team.
Could it be then that this is a communication problem between the Engineering team and the HRD? After all, the Eng'g team writes the requirements for the job post, HRD just checks it and makes it look attractive.
(Though honestly, I don't think this general comment applies to the job post in Carmack's tweet. Honestly I can't fault the job post for the way it was worded. I say the higher you are in an SE-org chain, the less this is a problem.)
> Sure it does: moving gold and commodities between countries in non-mutually-friendly regimes, very much resembles crypto.
Parent is talking about a legal framework and yet your counter-example is a transaction between two jurisdictions/parties/legal entities that are in less-than-legal agreement with each other. Apples to oranges.
Also, I'm quite certain you and parent interpret "the world" differently. You seem to interpret it in the sense of a pre-formalized economic framework society. Whereas parent (with whom I side with), use the term "the world" to refer to that sector of society with a codified economic framework.
> Many crypto platforms also have the mechanisms within them to build fiat finance systems atop them. (The keyword to search here is "security tokens".)
That might very well be the case but parent's argument is that our legal economic system is not purely algorithmic. Crypto might have features that are analogous to our current fiat systems but those features do not describe the whole fiat system; it misses some features (or "bugs", depending on how you look at it).
To paraphrase parent's statement, having someone able to override transactions is a feature of our existing systems, for better and for worse. Conversely, a system without this arbitrator has pros and cons too. You pick your poison.
It could be as simple as a `sleep` in the right circumstances. Maybe your resulting patch is a boring handful like:
bool cond1 = ...;
bool cond2 = ...;
if (cond1 || cond2) {
sleep(1000);
}
How this can be useful: Long time ago, we were observing odd behavior from Redis under extremely high load. Of course it did not replicate consistently and when it did, it lasted too short for us to properly observe and make hypotheses. So I had the brilliant idea of installing a custom Redis binary in our test environment that tweaked the odds so that the behavior happened almost consistently, no need to thrash the server. I had to read through Redis source code to make it happen (and added a hell lot of logs too). Plus in the end, it's not quite compiling Linux but, boy, I did compile Redis from scratch!
- Simplest game ever is a number guessing game. Obviously solvable by binary search on the player side. It's like an exciting counterpart for Fibonacci for learning PL flow-control structures. Additional features: limit number of guesses, make players guess floats instead of just ints. Latter will obviously teach you about FP handling in your language of choice.
- Next up, make that two dimensional, so the player is now guessing coordinates instead of numbers. Needs more math, you will definitely implement the distance formula. You can add a backstory if you like (are you bombing enemies in stealth perhaps?) and add mechanics based on that backstory.
- For something more complex---and I think this is your level now---people might suggest a constrained DF/Rogue-like. But me, that always grows more complex than a pedagogical device when I do it. So I stick to boring text games, sometimes inspired by game shows. Quizmaster-types (ala Who Wants to be a Millionaire), hangman-types (ala Wheel of Fortune), would force you to create structures for abstraction, all staying within complexity budget.
Happy hacking!
If you find the right mechanical pencil (i.e., not a cheap plastic one), it wins in both aspects. Plus a mechanical pencil never becomes too short that it does not rest stably on your grip.
But still, there is a certain charm to a drawing produced with the variable stroke widths of a wooden pencil. So I guess it depends what you're going for.
There was a time I was asked to work with a C code base that was last touched three years ago. Now, C in any standard isn't even my forte. It was doing some string manipulation whose behavior I can't exactly understand. Thinking I need a basics refresher, I Googled "c strings".
Woops. Got to be among my quickest tab closes. Cue look around if anyone happened to be looking at my monitor too.
(I will save people the mystery and possible blushes: apparently there's a type of lingerie called a "C string", in the same fashion as a "G string". The more you know.)
I feel like the understanding we can come to here is that we have differing definitions of necessary complexity.
You did mention you work in data analytics and I have worked and am working in more traditional product-oriented gigs. Everything you mentioned are nice, impressive even, but to me they are toys. I don't need a queue with a WAL stream, or PITR backups. Queues, to me, are inter-process communication mechanisms, or a means to concurrency. In fact, worst case, you can delete the queues or restart Redis even without triggering a disk write (though I note this is another feature that comes free with Redis); it would inconvenience our customers but they can always just retry.
Of all the benefits you mentioned, leader-election is the only one I could make a good case for.
> then you may as well understand it deeply in order to wring the most productive use you can out of each OpEx dollar/man-hour spent.
Understanding it is one thing but using that understanding to create/develop (CapEx) and then maintain (OpEx) an in-DB queue feels like unnecessary costs. CapEx to install Redis is practically nil, and you seem to agree that running it is far simpler OpEx than PG too ("Well, yeah, but...").
Of course, I keep in mind your point about libraries doing the dirty work. But if it all boils down to a third party in the end, the point I'm replying to above is rather moot no? And if it comes to a third dependency anyway, I've already reasoned out my preference for an actual queue solution.
> explained in a vendor-neutral manner: SQL DDL
> You can if those features are in the SQL standard. I’ve never heard of a DBMS regressing on its level of SQL standard support.
While this is an ideal case, my experience so far shows that every major DB package relies on their own tweaks to the SQL standard, and so "If those features are in the SQL standard" turns out to be a pretty big "if" to ask. I don't worry about vendors regressing on SQL standard support but rather that the libraries for DB queues are relying on the non-SQL-standard features/behavior to mock out a full queue functionality---non standard behavior that has no guarantee of consistency across versions.
I mean, if there's a DB queue middleware/library that works across DB vendors, be it Postgres, Oracle, Maria, MySQL (with a reasonable storage engine, perhaps) then that's a convincing argument that the SQL standard can support queuing sufficiently. But otherwise, this counterpoint is a hard sell for me.
Emphasis added because I feel like I addressed this in the paragraph immediately after the one you quoted:
> Plus, as me and others keep saying, Redis (and other proper queues) isn't a complex addition to your infra
I'm speaking out of experience and, as I already pointed out in another subthread, Postgres is far more complex than Redis. Consider the presence of "DB Admins/Specialists" and the lack of counterpart thereof for Redis and other queuing solutions.
Of course, if queues are not central to how your platform operates, you might be able to get away with Postgres. I still advise using Redis as a reasonable hedge against someone famous tweeting organically about your service because in this case, you don't want your DB to go down because some queue table had a surplus of transactions (or vice versa).
Not to mention, at an early stage, your tech decisions set precedents for the team. Maybe you have 10K users with a low qps but soon you are sending marketing emails to them and your system has periodic bursts of queue activity for all 10K users at once. When discussing this marketing "feature" rarely anyone thinks, "Hey we can't do that with our Postgres queue", rather "Yeah I saw functions in our codebase for queuing---this is doable". This is a small effort but a huge technical investment for later on.
Yes service accounts are fine and dandy but [a] isn't that more hoops borne by choosing the wrong tool for the job (gotta stay mindful of those permissions, you can have an Ansible typo, etc) and [b] as I said, proper queue services would not even allow tampering of enqueued data. For no extra cost other than installing the service. In contrast, your service account falling into a malicious actor is still a threat vector.
> For a more complex — but perhaps useful? — option ...
Now this is just a queue service with extra steps running in a relational DB instead of natively as an OS process. You did cite it as just an option but I don't see why this is an attractive option.
> I didn't see any mention of anything I'd consider an OpEx cost.
It seems to me we have differing definitions of OpEx and, judging by your previous comment, you value this almost exclusively in terms of "the number of ops salaries you have to pay". Even if I play by that definition, I could tell from experience (and also corroborated by other commenters here, in other subthreads of this discussion), that operating a Redis cluster does not need extra warm bodies. The people who monitor that your app responds, your servers aren't running out of memory, could take on the task with little to no additional training.
The experience I want to cite, bearing in mind of course that everyone's mileage varies: in my previous job, customer base of 2M and growing across two countries, we operated and monitored a redundant Redis cluster with an ops team of 3. In my first job I talked about in another subthread here, we managed three high-traffic Redis queues (and a few other smaller-traffic special cases) with a team of 7. In both jobs, we also did active across-the-stack development; we weren't dedicated to monitoring, nor to the components related to the queues.
In fact I would argue running a DB (queue or no queue) is just more complex than running a queue service. Rather telling is the need for "DB Administrators/Specialists" but no such equivalent for Redis or other proper queueing solutions.
> Which is usually also a cost already paid for you in advance when you use an abstract job-queue library. They all know how to expose Prometheus metrics endpoints...
Honestly, this sounds fine but I'm rather wary of the caveats that might be present. How do they do it efficiently? What do I need to keep in mind to keep things efficient? As such, unless you can point me to your preferred job-queue library which does all these wonders, I have to reserve judgment.
Edit: I just remembered mentioning this in another subthread. Even if you have a fantastic library/middleware abstracting all those queue concerns, that ties you in with that library. If someone wants to build integrations into your queue, they have to play by that library. If majority of your app is in Java (and so is the library), and Data Science (who, in this example, uses Python, not improbable to happen) wants to share your data pipeline, if that library isn't available for them, tough luck I guess? More dev time for someone.
And also, whatever underlying DB features or semantics your library might rely on in order to enforce a queue structure, you can't be assured that future versions of your DB would support those features because, you know, your DB isn't really in the business of queues. It opens you up to running an outdated DB version just to keep that queue functionality.
Using an in-DB queue doesn't give you zero OpEx does it? Maybe you can make the argument that it's cheaper but it's cheaper for a reason: the job wasn't done properly, so to speak.
Both options introduce new failure modes and op costs into your system. Might as well do it properly if (and this is a huge IF in my opinion) for slightly more cost. When you run into a failure it's standard, maybe the solution is even one Google search away as opposed to realizing, one hour into a late-night debugging session that, shit, that table should be treated as a queue!
Repeating my other comment: do you mean to say using an in-DB queue will not need special monitoring other than what's already standard metrics for DBs?
Since a queue is a different structure, I would want to know metrics that might be unfamiliar for standard DB monitoring. Queue length, queue throughput, etc. Can I get that from my DB queue with negligible cost?
Another thing, there might be standard practices and patterns for in-DB queues, often encapsulated in a nice library/middleware. But as databases are not queues there is no assurance from the vendor that those patterns will remain applicable in future versions.
> Paying attention to the tips/tricks/gotchas is something you'd pay once
Actually, I disagree with this. Everytime someone connects to that database, they have to remember that one special table which is used as a queue, and remember to tread differently where it is concerned.
Setting up a queue? You do that once and everyone who interacts with it will not forget it is a queue. Heck you can't even do your usual DB operations there. How can you not remember it is a queue?
> use it blindly without fully understanding our requirements and how redis helps scale
I'm sorry I don't get how I could come across as advocating the use of Redis blindly. My point is if your data flow looks like a queue, then use a queue, don't hack a relational DB to become a queue. I think that's reasonable rule of the thumb, not going in blind.
And, you mean, a PGQ will not need any special monitoring other than the common DB metrics?
For instance, if I ever run a queue, it's just due ops diligence to know the length of the queue, average time an item spends in queue, throughput over time, among others. Are there standard monitoring modules that would check this for a PGQ? Because in exchange for setting up a proper queue cluster, compatibility and documentation for common stuff like this is also what you get.
The first one is particularly a sticking point for me. If you don't do it right, you will end up issuing a COUNT on a table periodically. You might say it's acceptable for most companies, because they don't need high-performance queues, but I specifically need my monitoring to be reliable during times of unexpected high load. Also, ideally, there is close to zero chance that my telemetry is the one that ends up causing me trouble.
Also, why does no one consider the added "surface for human error" that using a DB for a queue introduces. For instance anyone can drop in the DB and perform a query you were not supposed to. A malicious actor can update all outgoing emails in the queue to another recipient. If these are sensitive emails like password reset or OTP, good luck. A dedicated queue process does not allow such operations.
I have to say, it's very perplexing to me that people (especially in HN) would avoid using the right structure just because they have to invest a bit more work.
Absolutely and that reason is, you are still a small team, with a small user base to boot. That's fantastic opportunity to learn a new technology and build on it properly! Remember everything is easier in software engineering if you assume you have no users[1] and your situation is as close as it gets to this ideal. Leverage it.
Plus, as me and others keep saying, Redis (and other proper queues) isn't a complex addition to your infra. This isn't Hadoop, or Kafka, which is a very special type of queue (one way to put it, at least).
> one request per second, and postgres for a queue is more than sufficient for that
Yes I agree but...
> Or would we do just as well to abstract away the queue in the code so that we can switch to redis when we do run into scaling problems?
What I read when I see such statements is this mythical software engineering ideal that with enough abstraction, a migration is just a matter of writing a new class that implements some interface and then changing a config. For a sufficiently complex app infra, that happens almost never because you could never keep the abstraction leaks to an acceptable level.
Another thing, abstraction does not solve all your problems if the underlying implementation is poor fit to begin with. Let me paint you a plausible scenario:
Once you are large enough, you might find your PGQ acting weird and you realize it's because someone in the team wrote code that accesses your queue table like it's an actual table of records, not a queue. So you think, okay let's prevent that from happening. Maybe you add users and permissions to distinguish connections that need to access between tables proper and queue. Maybe you start writing stored procs to check and enforce queue invariants periodically.
Well, guess what, all those problems would've been solved for free if you invested maybe one work day getting a Redis server running when you were a two-person op serving one request per second.
Lastly, scaling a relational DB is an entirely different beast from scaling a queue. Scaling anything does not ever come painless but you can reduce the suffering when it comes. Would you rather scale PG so it can keep acting as a queue or scale a queue that's, you know, really a queue in the first place? Heck the latter might even be solvable by throwing money at the problem (i.e., give it more compute).
[1] Except for the part where you need to make money, of course.
In fairness, Redis has only been available for 12 years so someone who has been in the industry longer has probably encountered systems using DBs as queues for no better reason than an alternative was not available when the system was made. (Rabbit just a couple years older I think.)
But in this day and age, you have better options to start with c'mon.
> they spent too much time fiddling with RabbitMQ, when Postgres would have been fine.
Rather ironic claim in a thread full of tips, tricks, and gotchas just to make PG behave like a queue no?
I can't speak for RabbitMQ personally but Redis has got to be among the simplest drop-in dependencies I've ever encountered. For a simple queue, the defaults do out-of-the-box.
Gut feel also says, Redis defaults for queuing would also serve more than PG defaults for queuing so you could leave it untouched for far longer to focus on your product, get to profitability, etc. Feel free to debunk with data.
Whatever happened to use the proper data structures for the job? PostgreSQL and MySQL are, at the end of the day, b-trees with indices. Throw in relational properties and/or ACID too. Those aren't properties you need or want in a queue structure.
I know I don't have a solid argument against not doing it; it's just experience (and dare I say, common sense) telling me not to. Not quite like parent but I spent the first two years of my professional career in a team that had the brilliant idea to use DBs as queues. The big task I partook in for that stint is moving them off that v2 into a v3 which used---wait for it---Redis. Everyone's quality of life improved with every migration, proportional to the size of the v2 cluster we retired.
I think I first saw the usage of this term either from Yahoo (wrt either GeoCities or Flickr) or Google (wrt Reader). Definitely around that era. IIRC it was on some company roadmap presentation that basically say "These services are scheduled to be sunset by <date>". Heck I even think those slides were leaked and not officially presented.
I like to think the first audience to encounter this usage of the word was similarly perplexed. "When you say 'sunset' you mean 'termination', right? What about the teams assigned to them?"
"'Termination'? Oh goodness, no. When we say 'sunset' we mean we leave them alone to naturally drive into a beautiful sunset."
No less perplexed, they put emphasis on their second question. "What about the teams then?"
With the cadence of a well-rehearsed actor, the facilitator responds, "Hey, sunsets. Very peaceful, very relaxing---who doesn't want those? You won't get that from staring at your computer screen all day!"
You see, this is the wonder of due process. In a mostly-reasonable jurisdiction, police officers are required to, first and foremost and at the very least, identify themselves clearly as law-enforcement, and with that carries the implicit suggestion that "you need to cooperate and not fight back". No identification and they might as well have arrested you without reading you your Miranda rights.
> Police officers in America, and most other countries, need to adhere to a completely different and much more lax set of rules than most people
I wouldn't call it lax but I'd settle for slightly different. Of course, a lot can be said about abuses of this "privilege to be different", which is maybe why observers would say law-enforcement is held to a relaxed standard.
It is different but not lax because:
- ordinary citizens can't break down doors into private property but law-enforcement can if they have probable cause (Have you watched Breaking Bad?) or a search warrant. Good luck obtaining a search warrant as an ordinary citizen, if that's even possible.
- ordinary citizens are not ever expected to result to fisticuffs and if they do so, they would face fees or even jail. But law-enforcement can do so except there's gonna be a fuck ton of paper work afterwards and, should their reason for resorting to force be successfully challenged, face suspension or dismissal but hardly ever fees or jail time.
Of course the MOST IMPORTANT caveats here are (a) law varies from jurisdiction to jurisdiction, even for reasonably-democratic territories and (b) this is HN playing r/legaladvice, which is fun but not as productive IMO.
In one part of the contest, the teams had to discuss their design. One of the judges asked the Carl Hayden team why they had no Powerpoint slides.
"Powerpoint is what you use when you don't know what to say," answered one of them without skipping a beat.
Ballsy, of course, but the subtext of the story was that the team really had their hands full as they tried to punch above their weight class. They knew what they were doing but there was no shortage of opportunities for them to improvise and the presentation was one such instance.
I was in high school when I read the piece and that's the line that stuck with me. Since then I always made sure I know my material, slideshow or no slideshow.
Bug report: Me and another player happened to spawn on the same spot, locking us in place and producing a satanic screeching sound.
Other than that, great mechanics!