agent281

Elixir is such a delight, but are there many data engineering jobs for Elixir? Could you describe how you came into your current position?

Well, Python came out in 1991 so comparatively OCaml is the new hotness! ;)

They are doing it for the same reason people keep ripping JS engines out of browsers: there has been so much work done to make them fast, harden the sandbox, and port them to different platforms. It's free real estate.

You have to remember that WASM comes from web browsers. It is meant to be transferred across the web and run in a the browser on the target computer. It isn't very practical to compile everything to every architecture/os combination and serve the correct implementation.

This seems backwards to me. Docker is built on Linux process controls and requires the Linux kernel. I believe Docker on MacOS/Windows requires a Linux VM.

> Unfortunately, one of the challenges of running Docker on macOS or Windows is that these Linux primitives are unavailable. Docker Desktop goes to great lengths to emulate them without modifying the user experience of running containers. It runs a (light) Linux VM to host the Docker daemon with additional "glue" helpers to connect the Docker client, running on the host, to that VM.

https://mirage.io/blog/2022-04-06.vpnkit

WASM and WASI would actually be a cross compatibility story because you could compile to an architecture and syscall interface that is platform agnostic.

I don't want to hop on the "just do this" train, but I'll say that sitting on a ski lift and talking with a random stranger about their life is a simple joy. You probably won't see that person ever again on a big mountain, but you can learn a lot about them. Underrated social activity.

For a while AWS Glue had an issue where the running job counter would permanently increment. This was a problem if you only wanted one copy of a job running. The advice support gave us was to increase the allowed count by one. I saw references to this issue that were years old. I think it is fixed now because I haven't seen it happen in months.

Same with data engineering. Move data from point A to point B. Do these roll-ups.

And the name just rolls off the tongue!

I think that's a bit of a strawman. I don't think anyone says tests prevent all bugs. They can help fix some parts of the contract down and give you a place to check against regressions.

This looks very intriguing, but I'm at a loss for what it is and how it works. Can someone provide an explanation at a level between a tweet and an academic paper?

They actually sit right next to each other on programming language benchmarks game. Scroll down to CPU seconds, median and percentiles.

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

I often look at C codebases and think "what an unreadable mess!" Most of that is probably because of my own shortcomings, but occasionally I see a codebase like this and realize it doesn't have to be that bad!

Thank you for elaborating!

I think that Erlang/Elixir could make amazing TUI apps if they were easier to distribute. The ability to push work to the edges of the system while keeping the UI responsive would be awesome. It sounds like static binaries provided by Firefly would make this a viable option.

Why do you say the last 15 years? We had the Enron blackouts 20 years ago that were so bad they effectively forced the recall of the governor. The grid has had its issues for a while.

You can design the backend with the big ass and small ass tags.

Wouldn't these be good examples? Boston is a pre-car city that weathered population decline well. Detroit was a city that rebuilt itself around the car and experienced decline.

Honestly, if we are talking pie in the sky, I'd say pile people up along the coast. Specifically in LA, if you had fewer people in Riverside with lawns + AC and more people in Santa Monica with year round good weather then the quality of life would be better and environmental impact would be lower.

Do I think it will happen? Not quite.

This is why I think California needs more high density urban housing, but not more single family home housing. Densify the urban core; don’t sprawl into the wild periphery.

Glad to hear that they are lasting longer. I probably should have prefaced my comment by saying that this was around the first year that these companies really blew up in LA. (Can't remember when that was.) Vandalism was super high and they were just figuring things out.

A few years back, I talked to an engineering manager at one of the scooter companies and he said that their longest lived scooter at the time was 11 months old. I can't remember what the average was at the time, but it was shockingly low. Something like 3 months. No idea what it is now.

I agree. I think a zero defect mentality keeps things moving along smoothly. Unfortunately, people think I'm just being a perfectionist. I'm just trying to not trip on the treadmill. Every bug is another stumbling block.

Definitely subjective. I don't find some parts very readable. E.g., this line took my a minute to parse:

  append34 := func() func([]int) []int {

If I was going to rank the readability I would say:

1. Rust function bodies 2. Go code 3. Rust function signatures 4. C++ code

Which you could argue is me shifting the boundaries a bit, but sufficiently statically typed languages seem to develop two (or more) sublanguages. Global complexity definitely pushes Rust down peg.

This is my current struggle. I've enjoyed mentoring in the past, but right now I'm getting very exasperated feedback. It's hard because I don't have control of the environment to alleviate the deadline pressure, but I still want to help people learn. The end result seems to be a pile of tech debt for now. C'est la vie.

They are replacing an open source extension with a closed source extension. Makes people nervous given the embrace, extend, extinguish paradigm that they previously employed.

Oh, I see: including a header file injects forward declarations into the calling file. Each c file gets compiled separately into object code unless you directly import them like the code above is doing. In that case it essentially dumps the full text of the file into the calling file, which causes serial compilation of a single object file.

Unless I'm mistaken that clears some stuff up for me. Thank you!

I never really had a good grasp of C. Why is this code novel? It looks like importing a function in any language.

Before I start, let me say that I'm on mobile and I don't know how to post code on this site. I'm assuming backticks work.

---

One benefit is that functions are more reusable. Functions with more arguments can be partially applied and used where applicable.

For instance, if you wanted to increment the values in a list you might do something like this in python:

  map(lambda x: x + 1, [1,2,3])

But in an ML like language you can do something like:

  map((+ 1), [1,2,3])

Because plus is actually a function that takes two arguments and can be curried down to one argument.

Additionally, you can think of currying as a form of dependency injection.

For example, if you wanted to inject a database client and a logger before running a query you might do something like this:

  def queryer(db, logger):
    def _q(query):
      logger.info("before query")
      results = db.fetch(query)
      logger.info("after query")
      return results

    return _q

  rows = queryer(db, logger)(query)

You could write it a bit more simply in this hypothetical ML like language:

  queryer db logger query =
      logger.info("before query")
      results = db.fetch(query)
      logger.info("after query")
      results

MLs don't require parens/commas for function calls, but you can use parens to force a particular execution order. The lines below have the same result, but some produce and execute intermediate curried functions

  rows = queryer db logger query
  rows = (queryer db) logger query
  rows = ((queryer db) logger) query

It's a bit contrived, but you have the flexibility to provide some arguments based on the call site rather than the definition site. I.e., the first example was broken up into 2 function calls at the definition site, but the second example could have 1, 2, or 3 function calls.

---

I hope this makes sense and I wish I could preview my comment to see if it was formatted reasonably!

It does get stale and changes per jurisdiction. Just look at how GDPR and CCPA changed the privacy landscape. At least programmers don’t have to deal with linked lists having different time complexities in California.