Mypy aims to be compatible with the standard Python type annotations and still be able to optimize them. So in theory, you don’t need to modify your existing type-annotated program. In practice I believe there are limitations currently.
185 karma · joined June 1, 2016
Mypy aims to be compatible with the standard Python type annotations and still be able to optimize them. So in theory, you don’t need to modify your existing type-annotated program. In practice I believe there are limitations currently.
It is quite promising though, if it becomes more robust and compatible. I also believe they have still only scratched the surface of possible optimizations.
A company I’ve worked with also had a PHP monolith which was supposed to be split up into microservices to improve maintainability. The choice of language was free, and the first ones were, kid you not, written in PHP using some alpha-quality library which was supposed to make PHP asynchronous, to improve “scalability”. Comically, this stack was used in an image-serving service which had to perform blocking image resizing using ImageMagick. Due to this misunderstanding of the technology, the only way to keep it afloat was to run 90 containers in a home-managed Kubernetes cluster just to keep requests from queuing up. Another comic misunderstanding of this paradigm was when I traced down a bug where the process seemed to hang due to the fact that the developer had intentionally added a call to sleep in a loop. Coming from a Java/.Net background, they believed it would cause the “current thread” to sleep in order to not hog resources. This was problematic since the application (like nodejs) was single-threaded.
I didn’t want to work with that stack, so when it was my turn to work on a new microservice, I chose Scala + MongoDB (same as Etsy) because I wanted to learn about functional programming. Ironically, the microservice was basically a “checkbox as a service”, but I and the tech lead on the team were too brainwashed/high on learning new things to realize the overcomplexity.
The entire thing was an expensive lesson that I and some others got to learn from. The ones who didn’t learn kept their microservices ideology and found other jobs.
Fast-forward and the entire thing was scrapped and rewritten as a Django + Postgres monolith. Not to say that it wasn’t costly to do a rewrite, but infinitely less costly than continuing down the microservices road. Long live boring technologies.
But my application was to be able to use raw SQL queries instead of an ORM in an application. By statically analyzing the SQL queries, then you can, in a statically typed language, automatically type-check the parameters to and results of the queries. Combining that and an IDE like the Jetbrains ones, which provide intelligent SQL autocompletion/refactoring, then tbh., I think it would beat an ORM in terms of ergonomics. Some people like LINQ or various Haskell/Scala ORMs because they are type-safe. This would be totally natural to those who know SQL without the downsides of being totally unanalyzed and type-checked.
It would be like the clojure library called hugs, but with static type-checking.
See Paul Graham’s essay on the Blub Paradox [0], which discusses why some languages seem to have “weird parts” from the perspective of someone who knows a less-powerful language.
See also mypy's recent blog post on plugins (which are also used for supporting Python 3.7's dataclasses, which are quite dynamic as well): http://mypy-lang.blogspot.com/2019/03/extending-mypy-with-pl...
Just an example of how powerful macros can be.
They also have an example in their docs about implementing printf using a macro so that formatting parameters are typechecked.
It makes me wonder why Google wouldn’t put their efforts into using Rust, for example.
Of course, server power is cheap, but not for our planet.
You don’t waste time implementing features you “know” you will need in the future (you don’t know). You don’t waste time handling scenarios that make your feature theoretically complete and elegant but that have no significance to the users of the product. You go quicker to market and can more quickly iterate on your product.
I think a much better approach is to implement these kludgy solutions that apply to the specific problem being solved. Then, you can revisit the code when it makes sense (e.g. the scope of the feature is being expanded, or just do it regularly to keep things healthy). Generally holding off on implementing things you don’t need will put you in a better position to implement the things you will need when you reach that point in the future.
It sounds like the primary data store would be the stream of events.
My interpretation is that they are arguing that they would have to sacrifice the ability to fully model all connections in the system or answer any kind of query (they talk about doing some things “per-user”) because they would have scalability issues or would have to do manual sharding of data. They are considering the full model with all connections as a “graph database”. This terminology is confusing and seems to ascribe extra capabilities to Cloud Spanner.
“A post can’t be published if it hasn’t been reviewed.” (Precondition)
“After submitting a payment, an audit log entry must be created” (post condition)
I could imagine this being implemented in other languages by hooking into statement execution much like a profiler or debugger and executing the checks. Does anyone know of such tools for other languages, e.g Python or JS?
Does anyone have real-world experience with this development technique on an actual project in production?
This module can detect the n+1 queries problem automatically in Python ORMs: https://github.com/jmcarp/nplusone
Looking forward to seeing more automated tools like this in the Python/Django world.
Unlike a lot of those I read, it sounds like you actually set out with a good set of requirements and really understood the problem.
I had a good experience using Prometheus as well for a smaller project (server monitoring). It’s interesting to know that it can handle so many metrics and scale so well to more complex problem areas.
Even in science where the focus is actually on proving things, it is difficult to prove causation. I see this more as a question of whether you act on a correlation that you’ve discovered or whether you simply talk about it. While it’s fine to discuss things, it sounds like at the small company you worked at, that that was more important than taking action (or perhaps the company was too insecure to make a decision about what to do). At the big company, the focus was on making experiments and seeing what the results are, and then iterating on that. In the end, the business doesn’t care whether something causes something else, it is only interested in adjusting things, measuring results and if they increase some important metrics, then the business is happy. That said, it can be taken to an extreme where only the metrics are thought of as important, rather than other qualitative or hard-to-measure things. It is much easier with web companies who can quickly roll out changes and use AB testing to run experiments. Also, those companies with larger user bases will have better sets of data on which to base their decisions. Companies that are well established will also have historical data that will make it easier to interpret results.
It also sounds like at the first company, the tech team was interested in being theoretically correct, while at the second one, the engineers were interested in trying things.
It is an understandable problem in some ways: good programmers are lazy and won’t want to make unnecessary changes, so they tend to argue against changes that haven’t been proven ahead of time to yield results. However, for the business to thrive, knobs must be adjusted and experiments made. And those programmers have to turn the knobs (or make them adjustable by someone else). So there is a strong counter-motive at play there.
However, good programmers who are also engineers with a focus on the product / business will understand the importance of doing this. It’s important for the engineering team to be aligned with the business and it helps to have cross functional teams. Otherwise you may end up with departments protecting their own interests. See also Conway’s law and its derivatives.
"Okay, but what the freq are freqX and freqY?"
But I'm wondering, why not provide the PGP signatures for the code over HTTPS instead of HTTP?