HNHacker News
TopNewBestAskShowJobs

cjalmeida

739 karma · joined February 23, 2015

Software developer. Find me at cjalmeida(at)gmail.com
submissionscomments
cjalmeida··on Why Python keeps growing, explained
Calling C from Julia is pretty straightforward

https://docs.julialang.org/en/v1/manual/calling-c-and-fortra...

cjalmeida··on Why Python keeps growing, explained
You beat me to it!

Julia is both dynamic and fast. It doesn’t solve all issues but uniquely solves the problem of needing 2 languages if you want flexibility and performance.

cjalmeida··on Why Python keeps growing, explained
I don’t get it. Go is as imperative as a language can be.
cjalmeida··on Why Python keeps growing, explained
No.

Virtualenvs, and requirements are a thing in Python for ages.

I’ve used tons of languages and while not the best, Python dependency management and project isolation is decent. IMO certainly better than JavaScript.

cjalmeida··on Korean startup rival to Nvidia seeks $400M valuation and plans new A.I. chip
TSMC will fab for them
cjalmeida··on Ask HN: What's Happening at Southwest?
I’ve worked on airline scheduling system and solving to optionally is indeed NP-hard and scales badly.

However you can get to a very good spot with heuristics. This particular issue with SW looks like bad data collection(crew has to phone their location!) and a combination of lack of reserves, bad weather and holidays surge.

cjalmeida··on Sorting with SIMD
Though about that as well. Few years ago I implemented a SIMD sort using those and mergesort.
cjalmeida··on Reducing logging cost by two orders of magnitude using CLP
Of note, Java !== JVM. Spark and Flink, for instance, are written in Scala which is alive and well :).

My best effort in finding replacements of those tools that don't leverage the JVM:

HDFS: Any cloud object store like S3/AzBlob, really. In some workloads data locality provided by HDFS may be important. Alluxio can help here (but I cheat, it's a JVM product)

Spark: Different approach but you could use Dask, Ray, or dbt plus any SQL Analytical DB like Clickhouse. If you're in the cloud, and are not processing 10s TB at a time, spinning an ephemeral HUGE VM and using something in-memory like DuckDB, Polars or DataFrame.jl is much faster.

Yarn: Kubernetes Jobs. Period. At this point I don't see any advantage of Yarn, including running Spark workloads.

Hive: Maybe Clickhouse for some SQL-like experience. Faster but likely not at the same scale.

Storm/Flink/Cassandra: no clue.

My preferred "modern" FOSS stack (for many reasons) is Python based, with the occasional Julia/Rust thrown in. For a medium scale (ie. few TB daily ingestion), I would go with:

Kubernetes + Airflow + ad-hoc Python jobs + Polars + Huge ephemeral VMs.

cjalmeida··on Ask HN: What is your opinion on electronic voting?
No electronic system is deemed impossible to hack. That said, the system we have in Brazil relies on way more than just technical measures to ensure the security.

There are statistical controls, processes are transparent and follow standards. Parties officials and technicians are involved in all steps.

But back to tech, since this is HN, parties can ask for the actual source code, tool chain (regular GCC) and binaries used; and verify the compilation themselves against published hashes. The individual machines do the tally and print public results at the end of the day before any kind of central connection. All the central office does is sum those results.

Thus any successful attack must happen at machine level, prior to Election Day. There are hundreds of thousands of machines from many different HW generations being used in a given election; all of them air gaped.

Loading rogue software without cooperation requires tampering undetected with trusted execution modules that validate binaries twice, before and after election to avoid scrutiny.

If you assume cooperation, you have to assume the collusion of way too many civil servants, from high court judges that oversees the processes appointed by parties of different colors, to low level tech folks who operates and validate much of the process.

All in all, pulling an undetected fraud here would be a massive, Hollywood-level hacking feat. And in almost 30y no major election came close to being suspect of fraud.

cjalmeida··on How to create a Python package in 2022
Julia had a dev mode as well

https://pkgdocs.julialang.org/v1/managing-packages/#developi...

It’s packaging tool is top notch. Maybe because it’s developed as a core part of the language.

cjalmeida··on Docker, Kubernetes, Terraform, and AWS crash course series
On enterprises, k8s solves a different kind of scaling problem. You tend to have hundreds of apps that are business critical and need to be managed.

K8s provides a consistent way to package, deploy with HA and monitor those apps. Ideally it replaced tons of ad-hoc scripts and confluence pages that are very poorly maintained.

cjalmeida··on Efficient dehumidifier makes air conditioning a breeze (2014)
An even simpler method is painting your rooftop white/light gray.
cjalmeida··on Kubernetes is a red flag signalling premature optimisation
Modern CPUs are giant, complicated pieces of hardware. But they’re useful, and have a well defined interface that you can interact with it at a higher level.

Same with k8s. People conflated running k8s from scratch, with administrating. The former is very hard, you should buy it from a cloud provider. The latter is IMO not much harder than VMs at small scale and much simpler at you grow bigger.

cjalmeida··on Kubernetes is a red flag signalling premature optimisation
Friends don’t let friends run their own k8s instances. You should use cloud managed deployment and rely on their security services.

For comparison, you can just as easily expose/hide your VM as your managed k8s cluster.

cjalmeida··on Why I still recommend Julia
We recently used Julia to solve (correctly) a super complex scheduling problem for a major airline. Our budget was 5min, Python would get not even close to a solution in that time. The tool is being actively used in production.

Julia is not not only fine, but my go to language for performance problems.

cjalmeida··on Python 3.11 Performance Benchmarks Are Looking Fantastic
Like Julia :)
cjalmeida··on Ask HN: What Is Your Number?
I’d slow down a bit, but I do like what I do at $dayjob.

We got the option to work 9 out of 12m so I may eventually take that.

cjalmeida··on The Machine Learning Job Market
"Crypto community has weird vibes"

that's what I call an understatement.

cjalmeida··on Principles of Engineering Management
You would be amazed at the number of managers that largely ignore the "feel-good" part of the job, or even contribute to the workplace being a crappy place to work.

Like the author said, actively improving the working environment when possible earns you "currency" and trust into making unpopular decisions easier to swallow.

cjalmeida··on Python’s “type hints” are a bit of a disappointment to me
We use Spark extensively. We also use @pandas_udf to run some part of our Spark pipeline through some Python-only ML libraries.
cjalmeida··on Python’s “type hints” are a bit of a disappointment to me
You know you can use type aliases, right?
cjalmeida··on Python’s “type hints” are a bit of a disappointment to me
As many already said, 99% is just a modern IDE with a one-click-install plugin.

Also, I don't know the grip with productivity tools.

cjalmeida··on Ask HN: What are the next internet infra problems?
Ossification is an interesting problem. We dealt with it in the past with IE6 basically via “shaming” users/corps into switching.

A message in Netflix/Google/Facebook home telling “we see your network is blocking X, that may result in a not optimal experience”

cjalmeida··on The Future of Kubernetes
Cloud “value added services” ie, all but elastic VM, networking and storage, are typically very poor compared to whatever you can find in k8s just one helm install away.
cjalmeida··on SPAs Were a Mistake
This ship has sailed. SPAs shortcomings are widely known and addressed by frameworks.

Whatever MPA alternative you bring will need to address other shortcomings. There will always be something quirky due to building applications in a technology designed for hypertext documents.

cjalmeida··on How to write better scientific code in Python?
Came here to say the same thing. With multiple dispatch and JIT compilation, you’ll go a long way in Julia with the naive version of the code.
cjalmeida··on Upwork asking me for a $12.5k refund as the client was using someone else’s card
That's the only possible alternative. You're an expert at a hot topic living in Zurich. There are plenty of alternatives to get work at even better rate.

Also forget the legal course. There's no upside.

cjalmeida··on Swedes were fooled by one of the biggest scientific bluffs of our time (2020)
That was not my argument. And the organization of state and Plato's politics are way more connected.

But my comment was that people have such a "need for being ruled" that we repurpose ideas with clear autocratic root for our democracy.

cjalmeida··on Swedes were fooled by one of the biggest scientific bluffs of our time (2020)
Interesting that Popper denounce Plato's political theory as totalitarian. But if you look at it, that's basically how we divide society and government.

We cherish technocrats and "philosopher-king" to rule us in the Supreme Court and high ranking agencies and military. In turn they're not expected to have relevant property to avoid temptation.

Truth is people like being ruled and being told where they fit, if this leads to some sort of purpose and stability. That's why such millennia old ideas prosper

cjalmeida··on Willingness to look stupid
The problem was throwing the solution at their faces like that. The key to looking stupid is not making others look stupid.

There are ways to create roundabouts so that the solution looks “elegant” but only after “hard thought”.

There’s also the risk that the simple solution has some real limitation (eg. some regulatory issue) then you’ll will indeed look stupid

← PreviousPage 4 of 10Next →