drej

That’s not the point I’m making. http://wipmoore.info/posts/2017/09/17/security-simple-vs-eas...

> Devii is powered by Next.js. We chose Next.js because it's the simplest, most elegant way to generate a static version of a React-based website.

There's nothing simple about this crapton of dependencies. Sure, it may be easy to build things this way, but let's not conflate the two terms.

They recently bought Citus and they host pg on Azure.

There's this system, which has become extremely popular for processing data (Apache Spark). And its default configuration cannot parse CSVs properly and can lead to pretty severe data loss. I wrote about this a while ago and it's still an issue. (Yes, we did lose data his way, it's not a theoretical exercise.)

https://kokes.github.io/blog/2019/07/09/losing-data-apache-s...

Many of the stories here are from large companies, where the costs are quite a different beast. I want to offer an opposite view - from a small company (20-30 people), which is usually the kind of company best suited for the cloud.

We ran a number of modelling jobs, basically CPU intensive tasks that would run for minutes to hours. Investing in on-prem computers (mostly workstations, some servers), we got very solid performance, very predictable costs and no ops issues. Renting beefy machines in the cloud is very expensive and unless you get crafty (spot and/or intelligent deployment), it will be prohibitive for many. Looking at AMD's offering these days, you can get sustained on-prem perf for a few dollars.

Three details of note: 1) We didn't need bursty perf (very infrequently) - had this been a need, the cloud would make a lot more sense, at least in a hybrid deployment. 2) we didn't do much networking (I'm in a different company now and we work with a lot of storage on S3 and on-prem wouldn't be feasible for us), 3) we didn't need to work remotely much, it was all at the office.

Obviously, it was a very specific scenario, but given how small the company was, we couldn't afford people to manage the whole cloud deployment/security/scaling etc. and beefy workstations was a much simpler and more affordable endeavour.

That's not at all a point I'm making. OP praised Microsoft for embracing Rust and making it work well on Windows and thanked them for not doing their own thing and instead supporting an existing project. So I'm just pointing out to a very recent endeavour that goes against all of that. I just enjoyed the irony of it all, that's it.

Not saying anything about anything else.

As much as I appreciate what Microsoft is doing here, your NIH praise is not entirely well targeted - look at this research project from Microsoft, which aims to build a slightly different Rust. https://github.com/microsoft/verona

I've used a fair share of (painful) APIs over the years and I have one simple plea: can we please stop being a-holes? Can we stop scraping websites that have APIs? They are already offering machine readable data and maybe they have a reason for not providing everything. Scraping their sites circumvents this API and not only abuses their systems, but also makes your code super brittle - any change to their site breaks your code - if you code wasn't broken already as you were banned by the provider.

I recommend the author's talk about this project (and his blog is a good resource). https://www.youtube.com/watch?v=wlvKAT7SZIQ

Code is excellent, Innovators are immersive, the Code Book is brilliant, albeit less about computer history and more about cryptography in general (all books by simon Singh are super interesting).

True, but it's also quite a bit more discontinued :(

The Surface Go is an amazing device for work on the... go, but the ergonomics of the display and cramped keyboard make it suboptimal for any longer period of time (be it a coffee shop, your bed or anything like that).

If you plan to work only at your desk, where you connect it to an external display (USB-C, ideally, to get both power and video) and external keyboard and only use it as a tablet otherwise, then sure, it would work well. But using it as a standalone machine would be rather inconvenient.

MacOS: check out the latest MacBook Air 13", the one just released - it has a new keyboard, better base disk configuration and it's cheaper than before.

Windows: Dell XPS 13 has been the go-to laptop for years, they keep refreshing it, so make sure to get at least the 2019 model (which has a camera on the top of the screen, not the bottom), or ideally the 2020 model, which features a new keyboard and even smaller bezels (and a new, 16:10 screen).

Windows runner up: look at Surface Pro 7 (x86) and Pro X (ARM), if the form factor, performance and ergonomics fit your needs, they might be preferable to the XPS 13. The Pro X is quite a wild card, since the compatibility is not quite there yet, but it's closer to a paper notebook than laptops or even the Pro 7.

I would advise against anything smaller than 13", because it will be fairly suboptimal to type on.

A lot of people mistake S3 for a filesystem. S3 and other object stores have very different characteristics, both in terms of performance (latency, troughput) and access patterns (seeks, atomicity). All abstractions on top of objects stores have been really painful for me at work, because people would just ignore the differences between a local filesystem and an object store and were wondering why their perf was terrible.

Unless there's a mobile site, I usually do this: I use my Logitech MX Master 2s, hit the scroller really hard to let it spin for a while thanks to inertia, triggering as many "page" events as possible :-)

There is a great talk about this testing, it goes all the way to deterministic seeds and other basics. The level of determinism is admirable. https://youtu.be/4fFDFbi3toc

I really like the pg protocol (like e.g. Cockroach), it let me use my usual tools. There are a few things I noticed:

1. It has a fairly rich support for types - these new-ish SQL engines often lack quite a lot of things, but this seems pretty decent. 2. I don't see any comparisons to KSQL, which seems to be the primary competitor. 3. Read the license. Read it carefully. It has a weird "will become open source in four years" clause, so keep that in mind. It also disallows it being hosted for clients to use (esentially as a DBaaS).

Can we drop the "in Go" in links like these? I don't really see how that adds any value to the end user.

My favourite performance advice, regardless of context: just do less stuff.

In this case - load fewer and/or smaller resources.

This is not Rust specific, this is a compiler thing, both LLVM and GCC can detect sums and generate closed form formulas instead. There are other fun algorithm detections - e.g. if you try to do bitcounts yourself, LLVM will use popcnt instead.

Compilers are awesome, check out Matt Godbolt's talk on this very topic: https://www.youtube.com/watch?v=nAbCKa0FzjQ

I have an experience related to this. We used to run a [scalable SQL engine, PaaS] for ~$1000/mo, but it wasn't cool enough, so people started migrating stuff to Spark, Databricks, S3 etc. Now the infra costs are ~$8000/mo + several people to manage it build tooling around it (~$6000 per person).

People just want to run their selects, man.

I've been waiting for this release for years and I hoped for one thing and one thing only - for pandas to have a proper way of dealing with NULLs. And it does have it... OPTIONALLY.

It's great that the whole thing with extension arrays, custom types etc. has lead to this, but when the devs have, after 10+ years, the biggest chance for a backward incompatible change, this is the one to make. By making it optional, they are fixing it for the very few that know of its existence.

I love pandas and a sizeable part of my career depended on it - and while I don't use it anymore (partly because of the NULLs), I wish it the best and I hope there will be a future release with this breaking change.

I think there are bits of false dichotomy in here. Writing optimised code from the get go and thinking about it at a later stage are not your only two options.

You can always think about performance implications of your implementation and keep them in mind. This is typically the runtime, your infrastructure, your chosen level of abstraction etc. You know your implementation is not the fastest, but you know what your limits are and where the low hanging fruit is.

To make it more specific - I implemented some economic model thingy and all I wanted was for it to compile and run correctly, so I wrote it in a way that I could swap the data storage for concurrency-safe structures without impacting the results - but since that added tons of complexity, I just designed it with a pluggable storage in mind. Then I developed it as a single-threaded program and wrote tons of tests. Once that and the external API were done and validated, I plugged in the concurrent storage layer and enjoyed great speedups.

So no, my first prototype was not performant at all and I was okay with that.

There are two aspects of this:

1. If all superbowl-related articles used an extraction mechanism that links wikipedia with wikidata, then only the wikidata entry needs updating and all these articles get updated automatically. 2. That wikidata update I mentioned was manual, but a lot of data in Wikidata is updated automatically (e.g. births and deaths from some databases, city population counts etc.), so some things get updated across the board automatically. That's a source of pushback against this - articles being updated automatically, so bad actors can change a lot of things at once.

It's only shame that Wikidata doesn't get as much attention. It's a knowledge graph that runs _some_ of Wikipedia's content. I became involved recently and what I'd like to do is to get Wikipedians to use Wikidata more (e.g. automatically loading births and deaths on people's profiles), because once these two services are more interlinked, we're gonna get more knowledge graph info for free since people editing Wikipedia will keep improving this structured dataset.

I'd encourage people to get involved in either of these two projects - there's always a niche you know about and could improve its presence in the Wiki world.

Yeah, sorry, I mean VM as a bit of compute, regardless of its provisioning.

A fun story! Packet support contacted me one day last fall to ask about my business. I said I had none, I used Packet years before then for personal projects and only for a few days. Then they asked what my plans were for the 13 VMs I had running. I knew of no such VMs.

I guess my account got pwned.

They asked how I'd pay for these - $900 worth I think - since I didn't have a valid card in their system (the previous one expired since I last used Packet). I told them to stop the VMs and allow me to reset my credentials. I also asked them to investigate their logs and geolocate all the users that logged into my account. I was also curious they actually let someone log into the account from a new IP (without notifying me) and also allowed them to run instances without a valid card.

They blocked my account, which also blocked this support thread. So I started a new thread from an alternative e-mail address to resolve the issues.

I never heard from them again.

The problem and solution are broadly fine, but having done a bit of performance critical work, I usually dislike when a function allocates without a) me asking, b) me having the option to tell it not to.

I like passing in pre-allocated containers to house the results of my computation. That way I have not only predictable memory consumption, but I can also avoid costly allocation (and de-allocation) in tight loops. Having worked with Go a bit, there were times when I used this (like passing in byte slices to readers), but oftentimes you couldn't just tell a function not to allocate (which it did for safety), because you and only you knew it'd be safe to modify the structure in place.

The bottom line is that people should not be "always copy/always use pure functions" or "always use mutable structures, because performance!!!", but be aware of what the upsides and downsides of each are.

I can super highly recommend talks by Adrien Grand on Lucene and Elasticsearch. They are super instructive yet easy to follow.

https://www.youtube.com/watch?v=T5RmMNDR5XI

https://www.youtube.com/watch?v=eQ-rXP-D80U

A somewhat related discussion: https://news.ycombinator.com/item?id=21289827