HNHacker News
TopNewBestAskShowJobs

syastrov

185 karma · joined June 1, 2016

submissionscomments
syastrov··on Mypyc: Compile type-annotated Python to C
Not having worked with cython, the difference seems to be that cython requires using special types in its annotations as well as not supporting specializing the standard types like ‘list’.

Mypy aims to be compatible with the standard Python type annotations and still be able to optimize them. So in theory, you don’t need to modify your existing type-annotated program. In practice I believe there are limitations currently.

syastrov··on Mypyc: Compile type-annotated Python to C
As a bit of background info, mypyc is “not really” ready for broader use yet. The devs are planning a soft-launch: https://github.com/mypyc/mypyc/issues/780

It is quite promising though, if it becomes more robust and compatible. I also believe they have still only scratched the surface of possible optimizations.

syastrov··on Choose Boring Technology (2015)
Reading this sounds like dejavu.

A company I’ve worked with also had a PHP monolith which was supposed to be split up into microservices to improve maintainability. The choice of language was free, and the first ones were, kid you not, written in PHP using some alpha-quality library which was supposed to make PHP asynchronous, to improve “scalability”. Comically, this stack was used in an image-serving service which had to perform blocking image resizing using ImageMagick. Due to this misunderstanding of the technology, the only way to keep it afloat was to run 90 containers in a home-managed Kubernetes cluster just to keep requests from queuing up. Another comic misunderstanding of this paradigm was when I traced down a bug where the process seemed to hang due to the fact that the developer had intentionally added a call to sleep in a loop. Coming from a Java/.Net background, they believed it would cause the “current thread” to sleep in order to not hog resources. This was problematic since the application (like nodejs) was single-threaded.

I didn’t want to work with that stack, so when it was my turn to work on a new microservice, I chose Scala + MongoDB (same as Etsy) because I wanted to learn about functional programming. Ironically, the microservice was basically a “checkbox as a service”, but I and the tech lead on the team were too brainwashed/high on learning new things to realize the overcomplexity.

The entire thing was an expensive lesson that I and some others got to learn from. The ones who didn’t learn kept their microservices ideology and found other jobs.

Fast-forward and the entire thing was scrapped and rewritten as a Django + Postgres monolith. Not to say that it wasn’t costly to do a rewrite, but infinitely less costly than continuing down the microservices road. Long live boring technologies.

syastrov··on MuseScore 3.6
I’m glad to have contributed to MuseScore 13 (time flies!) years ago. Haven’t tried it in awhile, but glad to see it still going and from what it seems, thriving. The authors are very dedicated.
syastrov··on PaSh: Light-Touch Data-Parallel Shell Processing
Very cool idea. Trying to parallelize bash scripts can be annoying to do manually, so is not often done.
syastrov··on No Cookie for You
Sorry, but why is GitHub’s blog on HN 3 times today?
syastrov··on How can we, as web professionals, help to make the web more energy efficient?
Use an efficient language like Rust. Work on improving performance/memory usage of slower languages, e.g. Python.
syastrov··on IBM looking for 12 years’ experience in Kubernetes administration
Us programmers tend to be very literal-minded, but my understanding is that in some parts of the Middle East, it is common to exaggerate numbers to provide emphasis, where it’s understood that it’s not to be taken literally. So maybe they just wanted someone with “a lot” of experience in Kubernetes. (Strange idea: Maybe writing it this way even acts as a filter for literal-minded people). It probably works better in-person though :)
syastrov··on Json-Base – Database built as JSON files
Back in the day, we used to use CSV for this :)
syastrov··on Ask HN: What projects are you working on now?
This is actually something I thought of doing, but you actually did it. It would be amazing if you can find a way to open-source at least part of it.

But my application was to be able to use raw SQL queries instead of an ORM in an application. By statically analyzing the SQL queries, then you can, in a statically typed language, automatically type-check the parameters to and results of the queries. Combining that and an IDE like the Jetbrains ones, which provide intelligent SQL autocompletion/refactoring, then tbh., I think it would beat an ORM in terms of ergonomics. Some people like LINQ or various Haskell/Scala ORMs because they are type-safe. This would be totally natural to those who know SQL without the downsides of being totally unanalyzed and type-checked.

It would be like the clojure library called hugs, but with static type-checking.

syastrov··on Ken Thompson's Unix Password
That cracked me up
syastrov··on On the Impact of Programming Languages on Code Quality
I think it really depends on how you use the “weird parts”. Some of those let you write code much more concisely and effectively. Used in the wrong context, it’s a unnecessary distraction. It also depends on if you want to write “typical code” or code that the competition can’t match.

See Paul Graham’s essay on the Blub Paradox [0], which discusses why some languages seem to have “weird parts” from the perspective of someone who knows a less-powerful language.

[0] http://www.paulgraham.com/avg.html

syastrov··on Pyright: Static type checker for Python
Django has quite a lot of "magic" as well. There is work being done on a mypy plugin to support this magic in addition to type stubs. It is already quite usable and supports type-checking many common patterns: https://github.com/mkurnikov/django-stubs

See also mypy's recent blog post on plugins (which are also used for supporting Python 3.7's dataclasses, which are quite dynamic as well): http://mypy-lang.blogspot.com/2019/03/extending-mypy-with-pl...

syastrov··on Pyright: Static type checker for Python
Mypy performance is quite good in daemon mode for incremental updates. The recently released mypy-mypyc (AOC compiled version) gave me a 6x performance improvement on typechecking a large project.
syastrov··on Lisp Macros, Delayed Evaluation and the Evolution of Smalltalk
Scala has implemented support for async-await syntax using its experimental macros: https://docs.scala-lang.org/sips/async.html

Just an example of how powerful macros can be.

They also have an example in their docs about implementing printf using a macro so that formatting parameters are typechecked.

syastrov··on Open-Sourcing ClusterFuzz
Makes you think about choosing to write software in C / C++ / other non-memory-safe languages when you need 25000 cores churning away to ensure you don’t make mistakes that could cause serious security issues.

It makes me wonder why Google wouldn’t put their efforts into using Rust, for example.

Of course, server power is cheap, but not for our planet.

syastrov··on Tech Choices I Regret at Spectrum
I think boring could also be described as easily understandable. Kludgy could be interpreted to mean that all possible theoretical scenarios are not handled. Working could mean that the product works for the use cases it was intended for. In that case, I see this as a very positive pragmatic approach.

You don’t waste time implementing features you “know” you will need in the future (you don’t know). You don’t waste time handling scenarios that make your feature theoretically complete and elegant but that have no significance to the users of the product. You go quicker to market and can more quickly iterate on your product.

I think a much better approach is to implement these kludgy solutions that apply to the specific problem being solved. Then, you can revisit the code when it makes sense (e.g. the scope of the feature is being expanded, or just do it regularly to keep things healthy). Generally holding off on implementing things you don’t need will put you in a better position to implement the things you will need when you reach that point in the future.

syastrov··on Using Elasticsearch as the Primary Data Store in an ETL Pipeline
Is elasticsearch really the primary data store if they can replay events to reconstruct the data in elasticsearch?

It sounds like the primary data store would be the stream of events.

syastrov··on A lost language encoded in Inca cords
This data is begging to be analyzed by machine learning algorithms.
syastrov··on How Streak built a graph database on Cloud Spanner to wrangle billions of emails
Indeed. They admit that it could be done with an traditional relational database but argue that it wouldn’t be able to scale in the same way as with Cloud Spanner. But they are making it sound as though Cloud Spanner has some graph data model.

My interpretation is that they are arguing that they would have to sacrifice the ability to fully model all connections in the system or answer any kind of query (they talk about doing some things “per-user”) because they would have scalability issues or would have to do manual sharding of data. They are considering the full model with all connections as a “graph database”. This terminology is confusing and seems to ascribe extra capabilities to Cloud Spanner.

syastrov··on How Streak built a graph database on Cloud Spanner to wrangle billions of emails
The same applies for any (well-) managed platform (e.g. Heroku).
syastrov··on The unreasonable effectiveness of Soccermatics? (2017)
Newton developed the mathematics of calculus to solve physics problems. That goes against Wigner’s hypothesis. I believe that it is difficult to pinpoint where inspiration comes from. However, mathematical rigor can often help. You could also argue reductionalistically that, since we all live in a physical universe, we are always inspired by something physical.
syastrov··on Why not program right?
It’s a really important distinction that the invariant has to be an invariant in the real world (e.g. in the problem domain). Programming “right” is context-dependent. Some programmers try to program to some theoretically correct ideal, even if it doesn’t apply to their particular problem. Or, they may try to make invariants for things that they think they will need to hold true (possibly because of some future requirement). All of these can lead people to shy away from having any form of invariants. Agreeing on what invariants exist in the problem domain at a given time is really important for avoiding confusion. That kind of discussion happens all the time with teams who collaborate well and across disciplines. “Should it possible for that menu and that modal dialog to be open at the same time?” (Invariant)

“A post can’t be published if it hasn’t been reviewed.” (Precondition)

“After submitting a payment, an audit log entry must be created” (post condition)

syastrov··on Why not program right?
I have heard of property based testing (e.g. QuickCheck). Is the main innovation of Eiffel that the author is referring to that these invariants are written alongside the actual code AND are configurably checked at runtime? Because ordinarily, you might sprinkle assertions throughout your code, but these are checked only at the point you add them, rather than between statements as I assume Eiffel does. In that case, I see some benefits in the form of documenting logical assumptions which are actually validation when code is run (also during manual testing).

I could imagine this being implemented in other languages by hooking into statement execution much like a profiler or debugger and executing the checks. Does anyone know of such tools for other languages, e.g Python or JS?

Does anyone have real-world experience with this development technique on an actual project in production?

syastrov··on How not to structure database-backed web apps: performance bugs in the wild
I’ve experienced a lot of n+1 queries problems as causes for bad performance. Often times this was a result of wrapping the ORM in abstraction layers (for business logic and fears of being “locked in” to the ORM). We rewrote that part of the application using a different ORM (in Python). Making use of a tool that could help find these problems automatically helped greatly and we didn’t have a single performance problem when we went into production the rewritten service.

This module can detect the n+1 queries problem automatically in Python ORMs: https://github.com/jmcarp/nplusone

Looking forward to seeing more automated tools like this in the Python/Django world.

syastrov··on Monitoring 9600 banks at scale
Nice write up. I love reading these kinds of postmortems.

Unlike a lot of those I read, it sounds like you actually set out with a good set of requirements and really understood the problem.

I had a good experience using Prometheus as well for a smaller project (server monitoring). It’s interesting to know that it can handle so many metrics and scale so well to more complex problem areas.

syastrov··on Apple will attempt to jam Facebook's web-tracking tools
I have experienced the same thing myself, but I would explain it differently.

Even in science where the focus is actually on proving things, it is difficult to prove causation. I see this more as a question of whether you act on a correlation that you’ve discovered or whether you simply talk about it. While it’s fine to discuss things, it sounds like at the small company you worked at, that that was more important than taking action (or perhaps the company was too insecure to make a decision about what to do). At the big company, the focus was on making experiments and seeing what the results are, and then iterating on that. In the end, the business doesn’t care whether something causes something else, it is only interested in adjusting things, measuring results and if they increase some important metrics, then the business is happy. That said, it can be taken to an extreme where only the metrics are thought of as important, rather than other qualitative or hard-to-measure things. It is much easier with web companies who can quickly roll out changes and use AB testing to run experiments. Also, those companies with larger user bases will have better sets of data on which to base their decisions. Companies that are well established will also have historical data that will make it easier to interpret results.

It also sounds like at the first company, the tech team was interested in being theoretically correct, while at the second one, the engineers were interested in trying things.

It is an understandable problem in some ways: good programmers are lazy and won’t want to make unnecessary changes, so they tend to argue against changes that haven’t been proven ahead of time to yield results. However, for the business to thrive, knobs must be adjusted and experiments made. And those programmers have to turn the knobs (or make them adjustable by someone else). So there is a strong counter-motive at play there.

However, good programmers who are also engineers with a focus on the product / business will understand the importance of doing this. It’s important for the engineering team to be aligned with the business and it helps to have cross functional teams. Otherwise you may end up with departments protecting their own interests. See also Conway’s law and its derivatives.

syastrov··on Hijack of Amazon’s domain service used to reroute web traffic for two hours
To be accurate, that’s a very precise time. We don’t know if it’s accurate or not.
syastrov··on H.264 is Magic
An enjoyable, short and to the point article with many examples and analogies. But my favorite part was this:

"Okay, but what the freq are freqX and freqY?"

syastrov··on Emacs 25.1 released
Congrats on the release.

But I'm wondering, why not provide the PGP signatures for the code over HTTPS instead of HTTP?

← PreviousPage 2 of 3Next →