https://docs.julialang.org/en/v1/manual/calling-c-and-fortra...
739 karma · joined February 23, 2015
https://docs.julialang.org/en/v1/manual/calling-c-and-fortra...
Julia is both dynamic and fast. It doesn’t solve all issues but uniquely solves the problem of needing 2 languages if you want flexibility and performance.
Virtualenvs, and requirements are a thing in Python for ages.
I’ve used tons of languages and while not the best, Python dependency management and project isolation is decent. IMO certainly better than JavaScript.
However you can get to a very good spot with heuristics. This particular issue with SW looks like bad data collection(crew has to phone their location!) and a combination of lack of reserves, bad weather and holidays surge.
My best effort in finding replacements of those tools that don't leverage the JVM:
HDFS: Any cloud object store like S3/AzBlob, really. In some workloads data locality provided by HDFS may be important. Alluxio can help here (but I cheat, it's a JVM product)
Spark: Different approach but you could use Dask, Ray, or dbt plus any SQL Analytical DB like Clickhouse. If you're in the cloud, and are not processing 10s TB at a time, spinning an ephemeral HUGE VM and using something in-memory like DuckDB, Polars or DataFrame.jl is much faster.
Yarn: Kubernetes Jobs. Period. At this point I don't see any advantage of Yarn, including running Spark workloads.
Hive: Maybe Clickhouse for some SQL-like experience. Faster but likely not at the same scale.
Storm/Flink/Cassandra: no clue.
My preferred "modern" FOSS stack (for many reasons) is Python based, with the occasional Julia/Rust thrown in. For a medium scale (ie. few TB daily ingestion), I would go with:
Kubernetes + Airflow + ad-hoc Python jobs + Polars + Huge ephemeral VMs.
There are statistical controls, processes are transparent and follow standards. Parties officials and technicians are involved in all steps.
But back to tech, since this is HN, parties can ask for the actual source code, tool chain (regular GCC) and binaries used; and verify the compilation themselves against published hashes. The individual machines do the tally and print public results at the end of the day before any kind of central connection. All the central office does is sum those results.
Thus any successful attack must happen at machine level, prior to Election Day. There are hundreds of thousands of machines from many different HW generations being used in a given election; all of them air gaped.
Loading rogue software without cooperation requires tampering undetected with trusted execution modules that validate binaries twice, before and after election to avoid scrutiny.
If you assume cooperation, you have to assume the collusion of way too many civil servants, from high court judges that oversees the processes appointed by parties of different colors, to low level tech folks who operates and validate much of the process.
All in all, pulling an undetected fraud here would be a massive, Hollywood-level hacking feat. And in almost 30y no major election came close to being suspect of fraud.
https://pkgdocs.julialang.org/v1/managing-packages/#developi...
It’s packaging tool is top notch. Maybe because it’s developed as a core part of the language.
K8s provides a consistent way to package, deploy with HA and monitor those apps. Ideally it replaced tons of ad-hoc scripts and confluence pages that are very poorly maintained.
Same with k8s. People conflated running k8s from scratch, with administrating. The former is very hard, you should buy it from a cloud provider. The latter is IMO not much harder than VMs at small scale and much simpler at you grow bigger.
For comparison, you can just as easily expose/hide your VM as your managed k8s cluster.
Julia is not not only fine, but my go to language for performance problems.
We got the option to work 9 out of 12m so I may eventually take that.
that's what I call an understatement.
Like the author said, actively improving the working environment when possible earns you "currency" and trust into making unpopular decisions easier to swallow.
Also, I don't know the grip with productivity tools.
A message in Netflix/Google/Facebook home telling “we see your network is blocking X, that may result in a not optimal experience”
Whatever MPA alternative you bring will need to address other shortcomings. There will always be something quirky due to building applications in a technology designed for hypertext documents.
Also forget the legal course. There's no upside.
But my comment was that people have such a "need for being ruled" that we repurpose ideas with clear autocratic root for our democracy.
We cherish technocrats and "philosopher-king" to rule us in the Supreme Court and high ranking agencies and military. In turn they're not expected to have relevant property to avoid temptation.
Truth is people like being ruled and being told where they fit, if this leads to some sort of purpose and stability. That's why such millennia old ideas prosper
There are ways to create roundabouts so that the solution looks “elegant” but only after “hard thought”.
There’s also the risk that the simple solution has some real limitation (eg. some regulatory issue) then you’ll will indeed look stupid