Hacker News Reader

I don't think Elixir aims to be a general purpose scripting language. I only use it for hobby stuff, but lately I've been writing my scripts in a few languages to compare their characteristics: performance, lines of code, and delta code change when making a single threaded script into multi threaded.

I find that Elixir code for these tasks tends to be a bit longer than standard scripting languages, a bit shorter than Go/Java, with performance around Go and slightly better than Java. One of the interesting things I find though is that with the Flow library, bringing it from single to multiple threads tends to require fewer changes - generally I just have to change some uses of Stream to Flow and I'm up and running with a solution that's using all my cores.

For others not familiar with "Flow" I'm guessing it is this (which looks awesome): https://hexdocs.pm/flow/Flow.html

Also of note is Broadway (https://github.com/dashbitco/broadway)

"Comparison to Flow You may also be interested in Flow by Dashbit. Both Broadway and Flow are built on top of GenStage. Flow is a more general abstraction than Broadway that focuses on data as a whole, providing features like aggregation, joins, windows, etc. Broadway focuses on events and on operational features, such as metrics, automatic acknowledgements, failure handling, and so on." - (https://github.com/dashbitco/broadway#comparison-to-flow)

That'd be about my assessment as well, though it's been a while since I programmed Java/C++ etc. I ported Vault's Shamir secret algorithm from Go to Elixir, and it was about 25% less lines of code. Though Python scripts tend to be shorter for small projects, bigger ones get more verbose when dealing with classes and data sharing, IMHO. Python scripting is also more prone to difficult debugging issues due to immutability and inconsistent/dated libraries. For example, a Python SPI library I tried for an IoT sensor deleted all of the items from the list of SPI bytes given to it... which took an hour to figure out as that was completely unexpected and I kept thinking my part was bad (though perhaps I'm too used to Elixir's immutability now).

> However, I feel like I can throw my daily driver programming languages including Go, Ruby, Rust... even Haskell at any problem and give or take performance, come out the other end with high quality software at scale.

You're probably right. With a very high-level language like Ruby and an extremely efficient systems language like Rust, you've pretty much got the entire spectrum covered. With libraries like Helix, you've even got an escape hatch if you start with Rails and hit unexpected bottlenecks.

IMO Elixir is nearly ideal for web startups due to productivity, enough performance, easy scaling and a very gentle learning curve. If you've already got all those other tools under your belt, then those pluses are probably less of a draw.

Elixir has been my daily driver for four years, I came from a brief stint in Ruby.

It's true that most problems are largely manageable by most/all languages.

What I've come to appreciate is how often Elixir has made me a better programmer with its conventions.

Their choices have steered me away from a lot of foot guns and forced me to learn better ways to do things.

It really depends on what you mean by general purpose.

At work we use Elixir for web and data ingestion, and I could have used Ruby for web but Ruby is considerably behind for data stuff.

Elixir is fine for one-off scripts, Ruby and Python may be a bit more convenient, but overall they are all too heavy to distribute compared to languages like Go and Rust.

Elixir is also used for high-end embedded and there is the whole distributed programming bits, which is quite specific to its platform, and more complicated to setup in almost all other places.

So overall it does provide some nice coverage across a couple areas although it lacks on other ones (GUIs, ML, etc).

I've written scripts in elixir that did performance testing of parallel downloads over cloud storage services. Also I have a script that I run once a year year to scrape Google maps for driving mileage.

The deployment story for elixir scripts sucks because it is less portable than say ruby or python (the person running the script needs to have elixir and you can't really use library deps in a sane way), but I don't mind it, because it's totally worth having less hassle for the cases where you want a well-defined deployment.

At work, I am using elixir to: orchestrate VMs (this is more traditional elixir-ish), but definitely not really a website or embedded, as a PXE boot provisioning system for metal systems, as a substitute for ansible for laying out software on systems, which are more script-ish, but on the side of "scripts for which you are really going to want to have a consistent deployment story".

I'd be curious to hear about your experience writing a parser vs Rust. NimbleParsec or just plain binary pattern matching make parsers suer easy to write in Elixir. I wrote a pretty complicated one for an old telecom data exchange format a few years ago and it was a breeze.

I'm in the same place as you. I love using Elixir, but like you it doesn't feel very general purpose to me. I mostly just write Phoenix apps with it though. I went through Dave Thomas' book and it did feel like a mostly general purpose language when going through that, but of course those examples were chosen intentionally. It makes me think I just need to try more often. I'm so productive and fast with Ruby however that I always reach for that for scripts. I really need to buckle down and make myself write a non-trivial CLI tool in Elixir.

Part of what I suspect might make it feel less general purpose is that unless you spend time really learning the internals then it seems somewhat mysterious and constraining. The immutability for example (which most of the time I love) made it a nightmare when I was trying to process and transform deeply nested JSON. I tried a few times in Elixir and just bailed to Ruby where I could directly mutate stuff 5 levels down with ease and efficiency. The code seemed way more straight forward and readable.

How do people deal with things like deeply nested JSON in Elixir? Is there a way to make use of actors to avoid some of the pain points?

Hard to say without knowing more about your usecase but normally I reach for lenses [1]

[1] https://github.com/obrok/lens

Thanks, I haven't heard of lens before.

For others there's a blog post that introduces it: https://yapee.svbtle.com/nested-data-structures-with-lens

> How do people deal with things like deeply nested JSON in Elixir?

Have you gotten to play with the `put_in/2` [0] and `update_in/2` [1] macros? They're designed to manipulate deeply nested structures like JSON

    iex(2)> my_parsed_json = %{"some" => %{"deeply" => %{"nested" => %{"key" => 1}}}}
    %{"some" => %{"deeply" => %{"nested" => %{"key" => 1}}}}
    
    iex(3)> update_in(my_parsed_json["some"]["deeply"]["nested"]["key"], & &1 + 1)
    %{"some" => %{"deeply" => %{"nested" => %{"key" => 2}}}}

As a side note, a common misconception is that updating a large data structure on the BEAM is inefficient because values are immutable, so you have to copy the whole structure to make a tiny change. This is untrue, as Elixir (and Erlang) "maps" are implemented as HAMTs[2], which support very memory efficient updates by "sharing" the unchanged parts between the the old and updated maps.

[0] https://hexdocs.pm/elixir/master/Kernel.html#put_in/2

[1] https://hexdocs.pm/elixir/master/Kernel.html#update_in/2

[2] https://en.wikipedia.org/wiki/Hash_array_mapped_trie

I think it would be hard for Elixir to replace Python or Go for general purpose CLI tools.

Part of what makes Python nice is it's available everywhere. If you have a zero dependency CLI tool, using Python is nice because you can just run the script straight up. The standard library also has good support for argument parsing. I've written a bunch of little tools to help my daily workflow on the command line. I always reach for Python or Bash depending on what I'm doing. I haven't really seen a compelling reason to switch.

New Elixir thread -> nickjj pointing out the superiority of Python or Ruby. Lol!

> New Elixir thread -> nickjj pointing out the superiority of Python or Ruby. Lol!

Not really. He asked a question about using Elixir as a general purpose programming language so I gave my honest advice and opinion on using it as a scripting language based on prior experience.

Instead of cherry picking my replies out of context, why not go through my entire comment history in all Elixir and Phoenix HN threads? I'll guarantee you'll find tons of positive comments.

Here's a couple to get you started after spending 60 seconds searching:

- https://news.ycombinator.com/item?id=20267184

- https://news.ycombinator.com/item?id=20357572

- https://news.ycombinator.com/item?id=17616692

- https://news.ycombinator.com/item?id=18838449

The first two have over 100 up votes combined for reference.

I'm not looking to argue on the internet, but I do find it strange that so many folks in the Elixir community cannot take any constructive criticism of the language in any way. You realize we're all on the same team right? To build kick ass applications that we enjoy building while users enjoy using it.

Just gonna say, I'm pretty upfront about when not to use elixir, and I have found your attitude to be very pessimistic across several platforms.

You just stick out as somebody who's always complaining, mostly on the official forums.

I see it more like asking questions, offering suggestions and potentially trying to improve the language and eco-system by offering feedback and sometimes even pointing out things that hang me up.

For example, the other day someone asked if Phoenix could ever support similar features of Rails such as having something similar to Rails Application Templates. I thought it was a great idea and it was something I wanted too, so I gave as much feedback as I could around why it might be useful to have.

In the end, it looks like most people don't want such generators, or having an ability to build a community around templates like Rails, Laravel and Python. That's based on only a few people commenting to say it's not a good idea and pretty no one else replying saying they wanted it.

You might see that as complaining but I see that as trying to fight for a really useful feature that will help grow the community and make it easier for folks to build applications in the end.

If I didn't care I would have left the Elixir community long ago. I have way better things to do than troll forums. The project I was gung-ho about developing in Phoenix is no longer using Phoenix because after all of those forum posts and struggling endlessly with Elixir I came to the conclusion it isn't the right technology for me to build web apps in. I continue to sometimes post because I hope one day things will change and I'll try to pick it back up again when the community and eco-system is more mature (from a library selection POV, not personal maturity).

The issue is that there is a gap and if you won’t learn the language nor use it, the gap won’t ever be closed.

For example, you said elsewhere a solution was complicated because it used io lists and a binary constructor but both concepts are explained in Elixir’s official getting started guide. So sometimes it feels you are saying things are complex while you don’t seem to be actively learning the language and some will definitely find that frustrating.

It would be the same if I went to Rust forums, a language I haven’t yet studied in any depth but read a lot about, and constantly said the ownership mechanism is complicated.

Apologies if this comes up as rude. You do seem to be a person open to feedback, so I thought I would drop my $.02.

Thanks.

It's not so much an unwillingness to learn.

It's not being able to write what I think is intuitive code and let's be real now. Programming is all about using tools that make you happy and productive. That's why there's so many different viable choices.

For me, personally (opinion of course), I don't want to have to worry about using binary constructors or io lists instead of strings every time I want to work with a string. I just want to use strings. Fortunately other languages can work with strings efficiently so I don't need to concern myself about such optimizations. I can just focus on the business problem at hand.

When stuff like this happens all the time, yeah, it gets frustrating and eventually you decide a tool isn't right for you. I think I gave it a fair shake too. I wrote a pretty big app twice in Phoenix, once with and once without Live View. I even delayed shipping my product for over a year waiting for Live View to get more fleshed out. And even then it felt like I was pioneering things because I kept running into severe bugs (which I reported).

The lack of libraries is kind of the straw the breaks the camel's back. I can maybe get beyond some of the language issues if it lets me be super productive but I found myself having to implement so many libraries myself because there's no official libraries available for Stripe, AWS, etc.. It just lead to feeling like I was spending more time writing libraries than developing my own app.

In time I hope the Elixir compiler could be smart enough to make optimizations for you behind the scenes and maybe there will be more libraries in the future too. In time Live View will get more fleshed out and polished as well, which is always welcome.

I am well aware you know this but Elixir is a different paradigm than Ruby and Python. Approaching it like you could do everything as in Ruby or Python means you are unwilling to learn the functional aspects of the language.

IO lists have useful properties that are not readily available in Ruby and Python, so they come with benefits, and Phoenix uses them to great extent for fast templates.

Binary constructors aren’t optimizations either but the way to construct binaries and strings in Elixir. They’re as fundamental as pattern matching.

The lack of libraries is something I can identify with but please understand it is frustrating to hear over and over again something is hard in Elixir while refusing to learn the language basic concepts because you would rather do it as in Python or in Ruby.

>You might see that as complaining but I see that as trying to fight for a really useful feature that will help grow the community and make it easier for folks to build applications in the end.

The change in mindset you're demanding from the devs is impossible to accomplish through comments I think. You'd need to develop personal real world relationships with the devs to be able to have such an impact, or create pathways for people with this mindset into the community.

It's the mindset of DHH or Taylor Ottwell what makes their product oriented frameworks more product oriented. Core devs and core parts of the community having this mindset is necessary as a fruitful foundation for more product oriented feature requests I think.

> It's the mindset of DHH or Taylor Ottwell what makes their product oriented frameworks more product oriented.

Yeah I totally 100% agree with this. I think it's very much so why Rails and Laravel won't be going anywhere anytime soon. As it turns out most people who are into web development are trying to build products and services.

But I will say that every time I've talked to Jose or Chris (I don't know them personally, I just mean in chat), they always say the best way to offer suggestions or feedback about the language or framework is to post on the forums. It's just unfortunate the community is so against feature requests or open to talking about things.

It just feels like the forums have the same regulars shooting down ideas from anyone new, or feature request threads get little to no replies.

>It's just unfortunate the community is so against feature requests or open to talking about things.

I don't have that impression, but I also got into Elixir knowing that it's not as product focused and more hands on.

For me it was more important that I don't hate the language and programming paradigm I'm mainly using. For purely business oriented stuff or quick MVPs I can still use no-code tools, which in 2020 is really the way to go if you're solo or a very small team.

>It just feels like the forums have the same regulars shooting down ideas from anyone new, or feature request threads get little to no replies.

Most on the Elixir forums seem to be employees and hardcore programmers, not founders and business people. I think even the main devs are employed, this changes how you think about prototyping features or tiem savings if you're never really in that situation and work fixed hours.

I think that the forums are a great learning resource and people seem nice, I mean you can't say that they aren't pacient with you.

> using Python is nice because you can just run the script straight up.

except when it doesn't work

which happens more often than not

Elixir is not probably best suited for CLI tools, given the BEAM has slow startup times, but people wrote scripts in Ruby, so it might be acceptable for someone

Elixir scripts can be distributed as releases which include the compiled modules bytecode and the VM to run them or as escripts, which are binary executable that can be run on any system with Erlang installed

No, Elixir does not aim to be a general-purpose language. It is built for a virtual machine that greatly excels at distributed, fault-tolerant systems.

Are you sure about that? I heard Jose say that they did aim for it to be general purpose during a talk at ElixirConf last week.

That may be the case, but I don't know how it can beat Go's ability to build executables for any platform, or Python's ubiquity.

I don't think it'a trying to do that. It has its own merits and a number of great use cases. Being general purpose doesn't mean that it needs to compete head to head on cross platform scripting tasks.

there are degrees of general purpose. I think the point is "more general purpose than just web servers". That it excels at high-end embedded is part of the point. Should you sue it for low-end embedded? Probably not.