grantjpowell

This looks neat. I'm the author of a similar project in typescript we use at Cotera called Era [0]. Y'all might be implement something similar to our caching layer [1] which we think is super useful. Once you have a decent cross warehouse representation it's pretty easy to "split" queries across the real warehouse and something like duckdb. The other thing that we find useful in Era that y'all might like are "Invariants"[2]. Invariants work by compiling lazily evaluated invalid casts into the query that only trigger under failing conditions. We use "invariants" to fail a query at _runtime_, which eliminates TOUTOC problems that come from a DBT tests style solution.

    [0] https://newera.dev/
    [1] https://cotera.co/blog/how-era-brings-last-mile-analytics-to-any-data-warehouse-via-duckdb
    [2] https://newera.dev/docs/invariants

Love the article.

In my mind I see the problem of dynamic linking in rust to have a bunch of overlap with the "I want this rust library to be exposed in my higher level GC'd language with minimal safety issues/tedious handmaintained bindings" problem.

My hunch is that the lack of expressiveness of the C ABI is holding back both. the thing I'd love to see some sort of "higher level than the C ABI" come out. And something like `wasm-bindgen`[0] to exist for more languages.

Here's a link to the rust "interopable_api" proposal! I don't understand all the implications, but it seems to be in the right direction https://github.com/rust-lang/rust/pull/105586

[0]https://rustwasm.github.io/docs/wasm-bindgen/

I enjoy existential comics, which like this site also aims to be a more approachable introduction to philosophy https://www.existentialcomics.com/

> now you need a Monad

"need a Monad" sounds scary but in practice it looks like this

    impl<T> Secret<T> {
        pub fn map<U>(&self, func: impl FnOnce(&T) -> U) -> Secret<U> {
            Secret(func(&self.0))
        }

        pub fn flat_map<U>(&self, func: impl FnOnce(&T) -> Secret<U>) -> Secret<U> {
            func(&self.0)
        }
    }

If you need an escape hatch for something more complicated, you could provide an api to that

    impl<T> Secret<T> {
        pub unsafe fn reveal(&self) -> &T {
            &self.0
        }
    }

Great callout, I haven't had my coffee yet. Here is a version that better shows what I intended

    pub struct Secret<T>(T);

    impl<T> Secret<T> {
        pub fn map<U>(&self, func: impl FnOnce(&T) -> U) -> Secret<U> {
            Secret(func(&self.0))
        }
    }
    
    impl<T: AsRef<[u8]>> PartialEq<&[u8]> for Secret<T> {
        fn eq(&self, other: &&[u8]) -> bool {
            constant_time_eq(self.0.as_ref(), other)
        }
    }

    /* Some other file */

    use secret::Secret;
    
    // Translated from the example
    fn check_mac<T: AsRef<[u8]>>(mac_secret: Secret<T>, message: &[u8], mac: &[u8]) -> bool {
        // This returns a new Secret<[u8; 32]>
        let computed_mac = mac_secret.map(|secret| hmac_sha_256(secret.as_ref(), message));

        // This uses the `constant_time_eq` impl from above
        computed_mac == mac
    }

I think the interesting part of the example is what you _can't_ do in the other file. It's pretty hard to misuse because the return type of `Secret::map` is a new `Secret`, the only way to do `==` on a `Secret<T>` uses a constant time compare.

I guess my main point is that when you have a instead of having to add new things at the language _level_, if I have something as powerful as the rust type system I can implement the same functionality in not much of code.

~I love when I see programming languages who's first advertised features are implementable in 8 lines of rust~

Edit: ^ the above had the wrong tone. Thanks to dang for pointing it out. What I meant to express was that it's possible to accomplish a similar safety/ergonomics at the library level in rust in not too many SLOC. My personal preference is towards Rust's approach because the type system gives really powerful composable primitives which makes it possible to have the compiler check a wide range of invariants, instead of just the ones that are common/special enough to go into the language itself

(Example edited after comments from mumblemumble)

    // The struct is public, but the contents are private, meaning you can't directly access the secret once it's inside the struct
    pub struct Secret<T>(T);

    impl<T> Secret<T> {
        // The only public way to access the secret, returns a new secret
        pub fn map<U>(&self, func: impl FnOnce(&T) -> U) -> Secret<U> {
            Secret(func(&self.0))
        }
    }
    
    impl<T: AsRef<[u8]>> PartialEq<&[u8]> for Secret<T> {
        // == does the correct thing (and only works for types that would make sense (`AsRef<[u8]>`)
        fn eq(&self, other: &&[u8]) -> bool {
            constant_time_eq(self.0.as_ref(), other)
        }
    }

    /* Some other file */

    use secret::Secret;
    
    // Translated from the example
    fn check_mac<T: AsRef<[u8]>>(mac_secret: Secret<T>, message: &[u8], mac: &[u8]) -> bool {
        // This returns a new Secret<[u8; 32]>
        let computed_mac = mac_secret.map(|secret| hmac_sha_256(secret.as_ref(), message));

        // This uses the `constant_time_eq` impl from above
        computed_mac == mac
    }

Edit: It looks like you can implment SOA as a macro too https://github.com/lumol-org/soa-derive

Edit: mumblemumble helpfully points out I demonstrated this poorly, so I tried to better demostrate what I was going for in this comment https://news.ycombinator.com/item?id=33764037

If you use the vim fugitive plugin[1], The `:Gbrowse` command [2] will open your browser to github on the correct file/commit. It also works on visually selected ranges, automatically linking to the range in github

[1] https://github.com/tpope/vim-fugitive

[2] https://github.com/tpope/vim-fugitive/blob/master/doc/fugiti...

Both of my tech jobs specifically have had non-solicitation clauses to prevent me from doing this type of thing. They both have 1 year lock outs on convincing my friends to quit their jobs

Wow, the stack in the article feels like a ton of innovation tokens[1].

I hate to be an armchair expert, but I'll do my best to give the _counter_ opinion to "this is a model of a good startup stack".

If you're looking to build a web app for a business on a small team, some guiding values I've found to be successful (that feel counter to the type of values that lead to the stack in the article)

1.) Write as much "Plain Ol'" $LANGUAGE as possible[2]. Where you do have to integrate with the web framework, Understand your seams well enough that it's hard for your app _not_ to work when it receives a well formed ${HTTP/GQL/Carry Pigeon/Whatever} request

2.) Learn the existing "boring" tools for $LANGUAGE, and idioms that made _small_ shops in similar tech stacks successful.

3.) Learn $LANGUAGE's unit test/integration test framework like the back of your hand. Learn how to write code that can be tested, focus on making the _easy_ way to write code in your codebase to be to write tests _then_ implement the functionality

4.) Have a strong aversion from adding new technologies to the stack. Read this [1], then read it again. Always be asking "how can I solve this with my existing tools". Try to have so few dependencies that it would be hard to "mess up" the difference between local and prod (you can go a LONG way with just Node and PostgreSQL).

Some heuristics to tell if you're doing a good job at the above points,

1.) You don't have to prescribe Dev tool choices (shells, editors, graphical apps, git flows, etc)

2.) You can recreate much of your app on any random dev machine, and feel confident it acts similar to production.

3.) Changing lines of code in your code base at random will generate compiler errors/unit test failures/etc

Most every real world software project I've worked on in the SaaS world ended up with "complexity" as the limiting factor towards meeting the business's goals. When cpu/network/disk etc was the culprit, usually the hard part of fixing it was trying to _understand_ what was going on.

Plain may be very successful in their flow, but I'd say most everything in this article runs counter from the ideas that I've seen be successful in the past.

[1] https://boringtechnology.club/

[2] At our shop we'd say "You're a ruby programming, not a rails programmer, your business logic is likely well factored/designed if it could be straight-forwardly reworked into a rails free command line app"

I constantly recommend this at work. The specific content isn't super helpful to Saas day to day development, but for me it built an intuition about postgres that has been invaluable. I think once I understood the "heart and soul" of postgres, the heap and the mvcc, many other properties about the database just "clicked" in my head.

I learned basic command line skills from the "Bandit" game[0]. Huge fan of the genre and it's one of the things I always recommend new developers.

[0] https://overthewire.org/wargames/bandit/

I've worked in Erlang/Elixir for the past few years and I haven't had the opportunity to work "in anger"[0] with languages with traditional threads/mutexes/sempahores. This game was a blast and made me appreciate the Erlang's approach to concurrency. The best beginner resouce I've read on Erlang's concurrency model is here https://learnyousomeerlang.com/the-hitchhikers-guide-to-conc...

[0] https://erlang-in-anger.com/

Yeah! I answer how I got my trail name and a few of my other favorite trail names below

Yep! I went by my real name "Grant" for a long time because I knew I would have to tell the story of how I got my trail name 1000 times and I wanted it to be a good story. Then one night I told people that's why I was waiting to take a trail name and a woman started calling me "Bear Bait" and there was no story...

Edit:

Couple of my other favorite trail names I heard

- Pissbag -- on his first night on trail "ol' Pissbag" peed in a ziplock bag while on the second floor of a A.T. Shelter[0], and then "bragged" about his "brilliant" idea to not wake people up in the middle of the night the next morning and _instantly_ earned the name "Pissbag". I heard stories about "ol' Pissbag" for the first 700 hundred miles of the trail and I was super excited to meet (what I assumed to be) the degenerate who was _proudly_ going by that name. I was shocked when I finally ran into him and he turned out to be a super put together former SF enterprise software salesman. He ended up getting a pretty serious case of lyme disease and had to take several weeks off from the Trail.

- Ballsack -- A former ballerina who carried a leather bag with two massage balls on her pack. Her favorite was when she would meet 70 year old women from church groups who came out to feed hikers and when they would ask her her trail name and she got to see "the light drain from their eyes" when she proudly said 'ballsack'. The woman who went by "D*ck Nipples" told a similar story about how she enjoyed seeing the reaction of little old ladies when she told them

- Chingona -- A woman who didn't speak Spanish got called "Chingona" by a man who did speak spanish, who told her that it meant "Badass Woman". She was insanely proud of it for 500 miles until someone told her that word had some vulgar connotations in spanish and she switched to "Chin"

- High Five -- When someone would ask his trail name, he'd raise his hand up and say "High Five". It usually took people several minutes before they figured it out...

-- 5 & 6 -- A man who's this was his 5th attempt to finish and a woman who would sneak up "on people's 6". Who got their trail names totally separately and ended up hiking the last 800 miles together. Everyone assumed they were a couple because of the names...

-- Pack Professor -- Former traveling musician who was very serious about gear (especially his pack). When we would start drinking he'd turn into "Party Professor".

-- China -- She'd tell people her name was "China" and people would instantly try to guess how she got that trail name. Then she'd let them know that "China" was her real name and she went by it because it was funny to watch people guess and "her parents already gave her a trail name"

-- Schrodinger -- I asked him if he was called "Schrodinger" was because he was really into physics and he replied "No, it's because I'm like schrodinger's cat but with whether I've ** my pants". Later we started calling him "Tool of war" because we found out he had that phrase tattooed on his genitals and he was honestly kinda a tool

[0] https://appalachiantrail.org/explore/hike-the-a-t/thru-hikin...

I finished the A.T. in October after 174 days, by far the hardest thing I've ever done in my life. I can't imagine trying to do a whole triple crown in a single year. Hats off to these two!

- Bear Bait

As silly as it sounds, I use the technique of using of using _really_ informal language to avoid spookiness

"Wanna see somthing cool?" "Lets shoot the shit after this" "Yo, you wanna hang for a minute at 4"

I see this as evidence that the privacy features of these coins may be effective in their goals. If I'm in a position where I need the privacy features of these coins for my own safety, I'm going to lean towards the one with all the regulatory hoopla stemming from it being so hard to trace.

I really liked this video series by some of the monero core team that goes through some of the privacy features and limitations of the monero tech https://www.youtube.com/watch?v=WOyC6OB6ezA&list=PLsSYUeVwrH...

Ring signatures are cool tech, but it's important to point out ring signature plausible deniability promises only work _in the absence of outside (off chain) information_.

When coupled with outside information (Cooperation with exchanges and other large holders, timing attacks of IP addresses, etc). The plausible deniability you get from ring signatures can be much lower in practice

Edit: Here's a really good video with some of the core monero contributors on EAE attacks https://www.youtube.com/watch?v=iABIcsDJKyM

This is satire right? The author's point is that this stack trace isn't very helpful because it's repeating the same thing over and over?

Elixir has a few forms of updating a struct (or map)

    %{existing_map_or_struct | some_key_1: :some_val, some_key_2: :some_val}

Then there is also `put_in/2`, `put_in/3`, `update_in/2`, `update_in/3`, these are nice because they work in pipelines

   some_val
   |> put_in([:foo, :bar, :baz], 3)


   some_val
   |> update_in([:foo, :bar, :baz], fn x -> x + 1 end)

The macro forms are cool too

   put_in(foo.bar.baz, 2)

   update_in(foo[:some]["nested"][:path], & &1 + 1)

The cool thing about the `/2` forms of `put_in` and `update_in` is that they are macros they rewrite the ast to return the entire object, and not just the changed value.

    iex(1)> foo = %{bar: %{baz: 2}}
    %{bar: %{baz: 2}}

    iex(2)> update_in(foo.bar.baz, & &1 + 1)
    %{bar: %{baz: 3}}

There's even more powerful macros like `get_and_update_in` which can use "selectors" to extend their functionality (https://hexdocs.pm/elixir/master/Kernel.html#get_and_update_...)

https://hexdocs.pm/elixir/master/Kernel.html#put_in/2

https://hexdocs.pm/elixir/master/Kernel.html#put_in/3

https://hexdocs.pm/elixir/master/Kernel.html#update_in/2

https://hexdocs.pm/elixir/master/Kernel.html#update_in/3

This is one of my all time favorite books. If you work with Postgres, an awesome follow up is this article series (much more advanced) http://www.interdb.jp/pg/

> How do people deal with things like deeply nested JSON in Elixir?

Have you gotten to play with the `put_in/2` [0] and `update_in/2` [1] macros? They're designed to manipulate deeply nested structures like JSON

    iex(2)> my_parsed_json = %{"some" => %{"deeply" => %{"nested" => %{"key" => 1}}}}
    %{"some" => %{"deeply" => %{"nested" => %{"key" => 1}}}}
    
    iex(3)> update_in(my_parsed_json["some"]["deeply"]["nested"]["key"], & &1 + 1)
    %{"some" => %{"deeply" => %{"nested" => %{"key" => 2}}}}

As a side note, a common misconception is that updating a large data structure on the BEAM is inefficient because values are immutable, so you have to copy the whole structure to make a tiny change. This is untrue, as Elixir (and Erlang) "maps" are implemented as HAMTs[2], which support very memory efficient updates by "sharing" the unchanged parts between the the old and updated maps.

[0] https://hexdocs.pm/elixir/master/Kernel.html#put_in/2

[1] https://hexdocs.pm/elixir/master/Kernel.html#update_in/2

[2] https://en.wikipedia.org/wiki/Hash_array_mapped_trie

I'll build on this.

One thing I like about phoenix/elixir is that its as easy as a rails app to get started. The really cool thing is when you need some power/polish you can reach under the hood and do very powerful things with the BEAM.

My experience running rails apps at scale has been nothing but headaches when I do something concurrent or slightly weird in ruby. I always feel like I end up rewriting parts of the BEAM with redis/postgres

My startup is build on Elixir for two reasons

1.) The nature of the problem (Distributed Multiplayer Game with complex logic but not computation heavy/number crunchy)

2.) I just like the language and have used it professionally for years

The value of #2 can't be overstated. In my experience, for some (most?) applications, the tech stack doesn't really matter. What tends to be more important is being comfortable with the language enough to iterate quickly and deliver value to customers

> Elixir sucks at:

> number-crunchy, like a shoot em up, or HPC.

> something which requires mutable bitmaps (someone this past weekend brought up "minecraft server")

One thing I'd like to see for the BEAM communities long term are well maintained libraries of NIFs[0] for high performance and possibly mutable data structures. Projects like rusterl[1] and the advances made on dirty schedulers make this more feasible than it used to be.

It would be cool to write all the high level components of a minecraft-esque game in Elixir, and drop down to rust when you need raw performance. Similar to the relationship between lua/c++ in some modern game engines

[0] http://erlang.org/doc/man/erl_nif.html

[1] https://github.com/rusterlium/rustler

> [ruby] is almost as semantically flexible as Lisp and ultimately friendlier

Elixir for the most part _is_ a Lisp, and inherits almost all of Lisp's semantic flexibility also

I write Elixir full time now after writing Ruby for several years. At first I struggled getting out of the ruby meta programming mindset. After reading some advanced lisp books, the concepts of quote/unquote began to click and now I feel like my ability to meta-program in Elixir is much stronger than in Ruby.

I really like Fred (ferd) Hebert's take on erlang and concurrency and distribution problems (Earlier in the book he points out how concurrency and distribution are a similar type of problem).

https://learnyousomeerlang.com/distribunomicon

> distributed programming is like being left alone in the dark, with monsters everywhere. It's scary, you don't know what to do or what's coming at you. Bad news: distributed Erlang is still leaving you alone in the dark to fight the scary monsters. It won't do any of that kind of hard work for you. Good news: instead of being alone with nothing but pocket change and a poor sense of aim to kill the monsters, Erlang gives you a flashlight, a machete, and a pretty kick-ass mustache to feel more confident

> This is the standard 'tools, not solutions' approach seen before in OTP; you rarely get full-blown software and applications, but you get many components to build systems with. You'll have tools that tell you when parts of the system go up or down, tools to do a bunch of stuff over the network, but hardly any silver bullet that takes care of fixing things for you.

As for the weird syntax, I use Elixir (https://elixir-lang.org/) on the daily and its pretty great. See here for a different write up I did https://news.ycombinator.com/item?id=24173635

> They actually said that to me: it makes the code messy so they don't want to do it.

I'm going to give one possible interpretation (my own) of the sentiment I believe the engineers were trying to get across with the statement "it makes the code messy".

Functionality is an asset, and code is a liability. Minimizing the amount of code and making the code "less messy" is a way of reducing that liability. That doesn't excuse not checking for certain error conditions, but I think it's important to point out that non "messy code" has merit on its own.

On the topic of effective error handling, the number of possible invalid states in a program (especially dealing with the messy real world) tends to be combinatoric. Error checking is vital to building a successful application, but my experience is that in practice it quickly becomes impractical to have _total_ potential error coverage. At some point tradeoffs need to be made, and effective engineering is the art of evaluating those tradeoffs.

> "And yet, here we are, because it DID happen."

I'm not trying to imply the author may be suffering from hindsight bias, I don't have any information that could tell me that. I do know that in similar situations I've been in in the past I feel like I've suffered from hindsight bias significantly. Here's a great article on hindsight bias that I think about often. https://www.lesswrong.com/posts/fkM9XsNvXdYH6PPAx/hindsight-...

The pivotal section from the lesswrong 2.0 article for me

> Viewing history through the lens of hindsight, we vastly underestimate the cost of effective safety precautions. In 1986, the Challenger exploded for reasons traced to an O-ring losing flexibility at low temperature. There were warning signs of a problem with the O-rings. But preventing the Challenger disaster would have required, not attending to the problem with the O-rings, but attending to every warning sign which seemed as severe as the O-ring problem, without benefit of hindsight. It could have been done, but it would have required a general policy much more expensive than just fixing the O-Rings.

I've never really explored the native grep functionality, so its possible I'm missing something.

The two features I really like about fzf are

1.) the preview window (visible here https://github.com/junegunn/fzf)

2.) Fuzzy finding/multi-selection/gradual refinement of search terms

My observation is that I tend to like languages that I don't need powerful tools to work with. That seems to be a mark of a well designed language to me