jvans

A lot of it happens in closed discussions with pros who are running various simulations. Some of it is posted on training sites but a lot is not.

The game these days is all about reverse engineering strategy principles from GTO simulation output. E.g. what general principle causes the solver output to say raise 30% in this spot vs raise 80%? If you come up with a coherent story, you can apply that thought process to new situations. It's a totally different game than it was a decade ago

Here is one of the solvers if you are interested: https://piosolver.myshopify.com/

Fwiw there is a significant amount of skill involved in choosing how to run simulations. I think they have some intro videos to help you

I'm always a little skeptical of general purpose search solutions. The hard part about search is ranking and relevance. BM25 and ANN similarity are great algorithms for sourcing documents, but without a ranking step the user is left to scroll through a lot of potentially relevant results that aren't sorted very well.

It doesn't matter what your queue semantics are. If it's not part of the transaction, delivery and push cannot be guaranteed to be atomic.

dralley is saying your queue push and database write aren't transactional. You have to be ok suffering some small % of message loss or messages sent without a database commit

Definitely. The good version of small PRs are small but useful changes that incrementally produce value on their own

The bad version is people adding random unused classes and methods that don't make sense until the final PR comes in and uses them

> but when you need to go back and re-open 4, 5 merged PRs in different tabs to get the full context again, the time to review increases tenfold with tiny PRs

You can't really say much about the first 4 PRs because if you don't know how that code is going to be used, it's kind of hard to giving meaningful feedback. By PR 5, it's too late to give feedback on PRs 1-4.

I also have worked with people that prefer that but I never understood it. Code without context isn't reviewable imo

Machine learning systems are hard because your system can be badly under performing but still doing objectively great. I've seen systems that produce amazing improvements in core business metrics and then years later some subtle bug in the labeling process is accidentally uncovered and fixing it boosts performance by an additional 20%.

A ton of time in ML is spent on minimizing the surface area for bugs because it can be difficult to even know they exist.

Advances in knowledge and technology will always come from people who understand the nitty gritty details very well and recognize patterns between abstract ideas.

I don't think current generation AIs are very good at this. A nearest neighbor search returns similar concepts but misses the very highest level abstractions. A NN search on a piece of text won't return text that talks about completely different ideas but uses similar sentence structure and argument patterns

What are you trying to build? I would personally avoid JS if i was in startup/scrappy mode and stick to server side rendering. It can take you very far and reduces a lot of complexity. Most apps don't need to be single page applications

I would say this is the fate of all products where the decision makers are focused on short term metric gains instead of long term quality. Data driven decision making encourages myopic product vision in a lot of scenarios.

Good data organizations acknowledge that all metrics can be manipulated and take changes in metrics with a grain of salt. They combine a/b testing with first principles thinking and make imperfect decisions based on imperfect information. They don't always get it right, but they embrace uncertainty and critically evaluate outcomes both "good" and "bad", because they know a/b test metrics don't tell the full story.

Bad data organizations focus on the a/b test metrics and ignore common sense logic, because short term metric wins are rewarded. Rinse and repeat for years on end and the product evolves to puzzling place like you mention. It was all the result of someone winning an a/b test over and over, and not critically thinking about the long term.

Also see: expedia, quora, facebook, instagram, etc.

There has to be some accountability, otherwise capitalism just turns into a game of socializing externalities and personalizing profits.

Calhoun makes the decision, reaps the profits regardless of outcome. If things go wrong boeing pays (maybe) and hundreds of people die

Calhoun should conditionally agree to the following for FAA approval:

If this event occurs within 10 years of launch he:

* forfeits 10x the compensation he received from boeing including capital gains to the families

* spends 3-5 years in prison for wrongful death

Let's see how confident he is when he has skin in the game

Exercises like this really make you a better programmer. Someone should collect these types of examples in a git repo somewhere.

From an engineering perspective don't you have to assume the worst case scenario when you use materials that might be inhaled? Is there any reasonable argument for using formaldehyde here or is it just because of cost?

It really seems like some decision makers should be going to prison here

In many cases yes. A simple example where this is not true is if your populations have different variances. T tests are no longer valid but heterogenous variances can be modeled with a Bayesian approach

The controversy is decades old and largely resolved. Both methods work well on large enough data sets, Bayesian methods are far superior on small datasets

This. Bayesian assumptions are explicit and you'll be forced to defend your priors. Frequentist assumptions like homoscedasticity are implicit and rarely challenged.

I wouldn't be surprised if many people using frequentist methods have no idea what assumptions need to be true for the methods to be valid. In a Bayesian framework you can't avoid the assumptions

Statistical rethinking by McElreath is an exceptional resource for Bayesian modeling. It's very accessible and gives you some good examples to work through.

Some very basic Bayesian models can go a long way towards making informed decisions for a/b tests

Nassim Taleb would agree with this. I forget which of his books he talks about this in but he gave the general recommendation to only read books more than 20 years old, because by then it's stood as a valuable book for so long that it's worth it.

both posts are very much appreciated, thank you

This is very helpful thank you. I tried looking at a few job pages on those links but can't seem find any open SWE positions. Couple of follow up questions:

1. What am I looking for in a job description that would indicate i'd be working on CFD simulations?

2. What would make me a compelling applicant? This seems very different than the distributed systems/recommendation system work that I have a lot of experience in.

Thanks again!

Does anyone have any recommendations for a former math major turned SWE to get into CFD simulations? I find this material fascinating but it's been a while since I've done any vector calculus or PDEs so my math is very rusty.

the treadmill of perpetual tech migrations only to find out about the downsides after months of work

Less is more work posts are so trendy now. The comments invariably point out unexplored confounding variables and the posts lack sufficient substance to be interesting. At the end of the day you can reason that hours != productivity, which everyone knows by now

> There's valuable learning (and empathy too) in having to see your own decisions and creations through their whole lifecycle.

This is so true. It's extremely enlightening to watch a design go from design docs to implementation and then finally the maintenance phase. A lot of problems that could happen never do and some unexpected ones pop up along the way.

I've spent so much time chasing down performance problems and bugs where the problem was due to an outdated dependency. Simply spending a few hours a month upgrading dependencies is a big win to avoid those situations

Wire fraud and prison sentences would be a good start. Besides the absolute dollar amount, the amount of damage done in the form of second order affects is enormous.

I've been advocating for this approach for a long time. At some level of size it is so brutally difficult to maintain an environment that mirrors production that the effort isn't worth it. With enough tooling in place you can mitigate the risk to customers significantly

> This may eliminate or simplify some engineering processes but IME massive migrations like this simply replace them with a different set of processes

Yes

> because they’re different

Now I have to learn an entire new set of tools/processes etc that are more useful to someone else but not helpful for me. The old one had its quirks but I knew it inside out and now the whole org has to re-learn how to do everything we did before.

The problem is not with estimates themselves, but on giving point estimates. A better way of saying something will take 2 weeks, is that there's a 80% chance this takes less than weeks, 15% chance it takes 2-4 weeks and 5% chance 4+. But people have a hard time wrestling with uncertainty so they'd rather a point estimate that's wrong but makes them feel like they understand the situation