Elixir, Phoenix, Absinthe, GraphQL, React, and Apollo
schneider.dev
schneider.dev
Main reason is the absurd amount of complexity with costs heavily outhweighting benefits gained from the solution.
For example, simple task of adding new entity consists off: On backend: creating migration, creating data entity (Ecto), writing structure module (sanitization, validation, basic logic, if needed), mounting queries, writing input mutation(s), writing output query/queries, unit testing On frontend: creating component, creating form, writing graphql queries, writing mutation, wrapping components/queries in components, connecting query to components, providing action handlers for inputs, unit testing, integration testing
Now I have authors list. And even though I am full stack I haven’t yet spent even single minute on having Proper UX Design set in place. Oh, do we need to add author’s birthdate? Dang, let me adjust all of that.
In my opinion technical debt accumulates faster than in other solutions. GraphQL is complex. React (done right) is complex. Apollo is complex (Elixir is simple, yet it’s only one cog). Deciding on doing file upload in GraphQL led me to a rabbit hole, which took at least a week to dug out from.
When trying to find the source of all development issues my thoughts go towards GraphQL. Maybe it is too complex or we didn’t had enough experience in it? Yet it was really nice to work with when all the backend and frontend boilerplate was already written. It makes sense, even though requires some heavy thought behind it. Maybe it’s Apollo, which locks in one of two specific trains of thought, or Absinthe, which requires side-hacks in order to get some bit more advanced features working with Apollo, like file uploads or query batching.
From a perspective I’d say this is just too much. Every single part of this stack adds overhead to the other parts. Maybe if I had more specialized team members it would get easier, but being only 3 of us, with me, as a full stack, constantly switching between all of that was a tiresome, low-productive effort. Right now we’re disassembling app to a classic REST-app and we’re seeing development speed increase on week-to-week basis, even though we had to scrap most of the frontend.
I guess there would be some benefit on having the write up of all of it, since it doesn’t even scratch the surface of the year of development issues with this stack, but even in this very "short" form it may serve for a word of warning that this stack is not necessarily something you want to care for.
For front end I wanted to move back to Ember since I had a lot of positive experience with it but that was impossible due to the business requirements.
As for Redux I try to stay away from it utilizing philosophy “you don’t need it unless you know you need it”. Cargo cult made Redux default piece of stack and it’s another very complex and sensitive to mis-design piece.
Instead of using Redux we rolled our own adapter since we had to work with some data outside Apollo and it’s we are rather happy with it. Not sure how it will scale to RN though.
For a simple UI I'd generate HTML server side with eex and spare us the cost of front end development. It's also a productivity nightmare. The amount of work needed to add a single form field with React/Redux is insane.
I have forms either having their own state or (preferred) just use Formik for all of this. In my stack, this then allows to just add a field in the GraphQL schema (backend), add it in the query, add the formik field + yup validation and done.
That would mean that if you use Redux, a form also requires actions for form update/submit/success/error and the form data should be stored in the redux store.
That is one of the main issues I have with Redux, which I feel adds automatic complexity for simple things, but at the same time I'm not sure if it's very good to have a mix of tings happening from store/actions/reducers and others from local state/ajax.
I won't disagree that this is a popular opinion, but there's little practical benefit to storing state that's truly local to a single component (or a very small tree) in Redux just because it's there.
Even the maintainers of Redux maintain that it's perfectly acceptable to use local state - https://redux.js.org/faq/organizing-state#do-i-have-to-put-a...
In fact, it's most of the times not desired to update your store before you know the data has been validated anyway. The store should always be the source of truth, but that also means that it should be valid.
That's the approach I am going with in any case when working with some kind of global state.
Bonus: it's almost certainly more accessible than custom solutions.
We had a pool of 4 Node instances which is the default of the solution we were using (a patched version of https://github.com/hassox/std_json_io)
Each Node instance has a 1 GB memory limit (the heap? It seems Java like). We failed to find a way to raise it but, again, we didn't invest too much into it except some googling. It seems there used to be a command line option for that but it doesn't work anymore.
Each hit to Node raises the memory usage until it gets to 1 GB and throws an error and gets recycled, which unfortunately translates to a 502/503 to the client. We can intercept those errors in Elixir and try again but it's far from ideal.
To have less errors we naively decided to increase the number of workers but we also had to increase the RAM of the server. The first hit for each client gets served by Node so eventually Node's resource usage dwarfed Elixir's. We felt like we were doing it wrong (I'm sure there is a way to get a saner setup) and decided to turn off server side rendering. Nobody complained and we're saving some $40/50 per month on that single server, plus our time which is worth more than that.
I think that projects with little load should run on low tech uncomplicated solutions: a reverse proxy and an application server were enough in the 90's for the same scenario and are still OK now.
In my experience, SSR should not introduce more complexity than you already have.
still with elixir/phoenix or did you scrap even that part?
We also took a step back and removed GraphQL-stack and use simple clean REST-Api only, it has increased our productivity and we don't have to divide our time for an other module, which must be maintained too. Before we used REST and GraphQL, becuase it make no sense to put everything into GraphQL.
It really would be easier. We are using this stack, and have separate backend and frontend devs. Each love their side of the stack. Being backend myself, I don't find the it too time consuming to get new entities going. However, I imagine if you were doing the whole stack and repeating ecto schema, absinthe schema, apollo queries, then it might get more tedious. I particularly enjoy how easy it is to modify the schemas once it's set up too. If we ever need to expose more data, it is usually done within minutes. There is a massive benefit in the forced standardization of GraphQL too. Being rigorous about standardizing APIs and how you do filtering, sorting, embedding, nesting and so on is tiring and a waste of time - you end up writing mini frameworks. Absinthe and GraphQL reduce this pain for us considerably.
If I would be to working on only 1 part it would be great, but instead of parallelizing effort it was sequenced which kind of sucked.
On top of all you said, the GraphQL stack is horrible for caching. You will not have this problem until you reach really high traffic, but once you do, it will eat you alive.
Unlike a rest endpoint, you can't cache a URL. You can't use HTTP headers. You don't know beforehand what GQL query will come and even with batch queries for different types it's super hard to optimize. It's very easy to have an N+1 query hell.
The lesson I learned is to use the boring stuff until it really needs to scale up. A REST API with static HTML and some sprinkles of JS will get you to the phase where you actually need to start using React, GQL, etc.
GQL trades a lot of things for flexibility, but 99% of the apps don't need that in the first place.
But hey, on the upside everyone can put in their resumes that they used all the new hot shit :)
Both Postgraphile and Hasura deal with this. I have no idea about Absinthe.
I will remember that one "Right now we’re disassembling app to a classic REST-app and we’re seeing development speed increase on week-to-week basis" . thx
Its technique is not new, it's just server-side rendering via websocket and DOM patching (Morphdom is also very old)
> not appropriate at all
"At some" would be appropriate. Pretty much typical CRUD (real world) that needs to fire requests to server anyway (e.g. form, business logic validation).
The only reason not to use it _right now today_ is it's not 1.0 yet.
We're in a similar boat these days so in fact our latest projects are back to simple server side rendering, but we're still making the data retrieval calls with GraphQL. It ensures that that the mobile app and reporting tools we're also developing will have parity, and we don't need to write ad hoc query logic for each use case. The built in docs and validations have simply proven too useful to pass up, and you really don't need a heavy weight client to make requests.
Any alternatives that you would like to suggest?
Although complex, GraphQL (done right) is much easier than REST (done right).
As for file uploads, it seems to me that the best way to do that is to have the front-end upload directly to your cloud storage service. Both S3 and Google Cloud Storage have a feature called signed URLs, where your back-end can create an object, grant the necessary permission, then give the client a temporary URL to upload to that object. Then just store the URL in the database.
Newer technology usually adds a ton of different patterns and abstractions that can hide a lot of things away from you, so it becomes hard to understand, unless this information is presented as a 101, or you invest all your time to read documentation about each individual part of everything. Which, to be frank, nobody has time for.
I have used GraphQL with Apollo and React for the last couple of years for different kinds of projects and what I have noticed is that the tools themselves, while quite abstracted, also try to support a lot of edge cases which can make it hard to apply to your project. I had to either use other libraries or build my own that helped me abstract some of the things that were most commonplace but required a lot of boilerplate otherwise.
I have found tremendous value by following the philosophy of not to pre-optimize for everything, until it becomes a burden to work with. Although, it can be hard to determine when something is going to become so big that it will be difficult if not impossible to refactor, but you learn that as you go.
```
def call(conn, _) do
context = build_context(conn)
Absinthe.Plug.put_options(conn, context: context)
end
def before_send(conn, %Absinthe.Blueprint{} = blueprint) do
if blueprint.execution.context[:logout?] do
Auth.drop_current_user(conn)
else
conn
end
end
defp build_context(conn) do
with ["Bearer " <> token] <- get_req_header(conn, "authorization"),
{:ok, data} <- MyApp.Token.verify(token),
%{} = user <- get_user(data) do
%{current_user: user}
else
_ -> %{}
end
end
# rest of the fileThen made a Logout middleware that sets logout? to true in the resolution context.
Is digging into the Blueprint as in the code above necessary? Is there a simpler way of solving this?
The last app I did was a mobile app written in Elm with Phoenix. I can't recommend Elm enough, it makes JS much easier to work with. For communication I used a simple REST oriented API.
While I understand why GraphQL exists, I think it is a major addition in complexity for little to no benefits. I tried it, wrote half a project with it, but they removed it. When things gets complex (for example, let's say the user.email field can be redacted under some conditions) your endpoint becomes really hard to manage. GraphQL certainly have a use for large API with graph oriented datasource (well, like facebook), but it is a specific use case.
Basically, if you need "API-driven" front-end, the only thing that might make harder than REST for a given language is lack of libraries. If you don't need API-driven, then just do things the traditional way, or try Phoenix LiveView.
The one caveat is that GraphQL (especially with Relay) maps exceptionally well do the component-based way of building things. So if you've chosen that paradigm, GraphQL might still be a good fit, even if you're just building a website.
Same goes for react.
Definitely don't let this hold you back. As long as you're up front about where you are, and are willing to learn, lack of professional experience with a particular language is by no means a deal breaker.
I've hired Rails developers for Django roles and seen them get productive within weeks.
If you've used Elixir on-and-off for four years you have more experience than the majority of engineers who might be considered for an Elixir role.
What did you like about the Rails developer that you chose the person for the role? Basically, I want to know irrespective of the language, how do I show my "quality as an engineer"?
We’re building a business management and financial services platform for music creators and rightsholders. Our stack is GCP, Elixir, Typescript and React Native.
Feel free to email me at the address in my profile. Would be happy to chat.
If it's just plain old request/response styles website (I think most of the project falls into this bucket) if you can handle other frameworks like Rails or Python, you can handle the same thing really well after some usage. Some basic knowledge of the language/MVC/Ecto would do.
Though there are a lot of React developers already using React but don't know what they are doing. Using React actually requires a good sense of the ecosystem - which set of libraries to use, which pattern to follow etc. Plus some basic concept like state/props/immutability/HOC would do.
It almost feels like these are two completely distinct worlds, but because they're part of the same language and community, it's difficult to feel proficient or even 'adequate' at times.
I never quite felt this way as a PHP/Ruby/JavaScript developer, as for the most part the challenge seemed to be knowing the right frameworks, new language features, or bundler/build tools to be 'good'.
In the most recent episode of Elixir Outlaws, the topic of overusing OTP/processes/Agents is discussed (https://elixiroutlaws.com/37), and it made me feel a bit better about my lack of experience in this area (and much as the podcast can be on the 'whiny' side of things, it's probably my favorite Elixir-focused one and I highly recommend it! If you're reading this: Hi Chris, Amos and Anna. being a friend of the show one day is on my bucket list :).)
The more important thing is if you understand how the web works. I'd much rather work with an senior Rails or Laravel dev who had just been playing with Elixir for a few weeks than someone who had studied the language deeply but had little web development experience.
Gatekeepers are annoying but you can usually side step a few qualifications as long as you can do the work.
At least that's what's holding me back.
This.
I use Elixir everyday. While it feels 'refreshing' to use pattern matching, sometimes it gets over-used, as a simple swtich-case statement is easier to read.
Same with pipe operator. I found newbie programmers tend to pipe for the pipe's sake, and write functions just to form the 'shape' of pipe, but the data shape changes are not easy to see and inspect.
Pattern matching allows you to express the code in a more declarative way, and usually would result in a more succinct code.
The majority of engineering articles I see today are along the lines of "X with Y, using Z", where X, Y, Z are specific products, frameworks or libraries, often trademarks.
I rarely see more generic engineering/architecture topics such as: [virtual DOM based / string based client side templating] with [JSON/Protobuf] over [REST/Websocket] with a [compiled / interpreted / JIT compiled / virtual machine based] backend runtime, built on [RDBMS/NoSQL/hybrid] etc.
Am I alone in feeling this way?
I think it's a great way to learn concepts of software engineering instead of just memorizing conventions.
please do post the link.
For many web developers, front-end, full-stack or otherwise, perhaps it's usually clear enough what the underlying approach of a particular framework is. But I imagine people who don't read up on the latest developments especially it might be helpful to get an idea of the underlying fundamentals of particular stacks/frameworks.
I've since had the Fortune to spend time learning about cloud native apps, distributed service patterns, and supporting infrastructure (spring cloud, pcf, vanilla k8s, gcp) and now returning to elixir having at least better understanding of what erlang and OTP offers.
I'm super excited to see I'm not alone in finding this sort of stack is worth fiddling with(although, I tend to pick Vue when not using angular for work).
Thanks for posting this!!
For those interested in what resources I'm leaning on: The Manning 'Elixir in Action' and the Pragmatic Programmer's graphql texts along with exercism.
Anybody else have any preferred resources for these technologies?
It's written by the creator of Phoenix (Chris McCord) and the creator of Elixir (José Valim), and it's a fantastic intro to Phoenix.
https://codestool.coding-gnome.com/courses/elixir-for-progra...
Major kudos to them for open sourcing their platform. It covers like 50+ common web dev features.
I pretty much skimmed the docs to get the ultra basics, looked at that source while I was building my own app and then asked questions when I got really stuck. Even managed to sneak in some refactoring on their code base a few days into learning Elixir. It's super approachable without much more if you have some type of programming background beforehand.
Even if you've worked at it a bit already, you'll probably get something out of doing the first dozen or so challenges on my channel as well: https://youtu.be/G3JRv2dHU9A?t=595
Apologies to others for the “me too” reply, but I wanted to let you know you aren’t alone, I also feel this way. It’s intellectual masturbation (without any reward/benefit), IMHO.
Haskell is a pretty decent general purpose programming language, and if used well you can produce nice programs, but the problem is that the library you need will use some clever type system stuff and suddenly you are spending hours trying to understand how it all works.
For example I'd be happy doing a lot of stuff in the IO monad with some pure bits to the side where needed. Avoid the free monads, monad transformers, type classes and all that jazz. So use it like an imperative language for the most part.
I've had the same experience with Yesod and Scotty too, although not recently - every time I decide it's been long enough for the ecosystem to mature a bit more, I bounce off issues like this which I just don't get elsewhere, and decide to give it another couple of years.
That's what one of the things that's kind of making me give up on OCaml as well although there things aren't that bad.
Phoenix LiveView looks super interesting for simplifying this further: https://dockyard.com/blog/2018/12/12/phoenix-liveview-intera...
> I’d prefer a simple React, Postgres, Elixir stack (REP).
That's not even a web stack. Web stack for example LAMP include a web server (Apache) or something that is able to response to client request. Elixir is a language it can't serve anything. You need cowboy or something and then build around it. Or you can just use Phoenix.
Incidentally, that's what I did for the project in the blog post. If you bootstrap with `mix phx.new --no-html --no-webpack` then you get a leaner version of Phoenix that's good for an API, without the HTML and frontend asset parts.
Phoenix also is pretty minimal layer over Plug, especially if you don't include the html library.
Generate a new Phoenix app. Open up the endpoint file. Don't want routing? Comment it out. Don't want logging? Comment it out. Keep what you want, discard the rest.
The fact that you'll have 4-20 small(ish) boilerplate files on a brand new project isn't the end of the world and it gives you a lot of flexibility later.
fwiw, I work at a telecommunications company and I've had no success convincing anybody to use either Go or Erlang/Elixir over Java (& Clojure).
But I think statefulness is the right context between the two.
Why do you think Go would be more maintainable?
But in any case, Go is a very straightforward language if you're already familiar with popular languages like C/C+, C#, Java, or JavaScript. You basically just need to learn the concurrency features (goroutines/csp/select keyword). Elixir is a much larger conceptual leap in that it is functional and the currency toolset is a lot more to grok. Deploying code with Go is also easier than Erlang/Elixir. Static typing is also pretty huge (dialyxir seems more like a bolt on).
Sadly you are correct. But what I was getting at is that I'm interested in how do you choose a language. Us the techies should strive to make technical decisions not based on politics, wouldn't you agree?
> But in any case, Go is a very straightforward language if you're already familiar with popular languages like C/C+, C#, Java, or JavaScript.
In every single area of my life I found out that what is cheaper at the start always ends up hugely expensive later. But I am not here to argue life philosophy. Just a quick tidbit.
> Elixir is a much larger conceptual leap in that it is functional
True. I still found the investment very worth it though, even after I spent 14-15 years with OOP / imperative languages before that, mind you. But that's a matter of personal career choice.
> Static typing is also pretty huge (dialyxir seems more like a bolt on).
This is undeniably one of the big weaknesses of Erlang/Elixir (and the BEAM languages in general). It made me try and reach for languages like Go and OCaml.
Go is pretty easy but extremely verbosive. Also the imperative languages' tendency to always show you how you are doing things instead of the what you are doing (namely the way of the functional languages) is something that is poking my eyes out lately, for the better or worse. Additionally, people use Go's typing escape hatch (`interface{}`) way too often, to the point that the typing system feels completely optional.
OCaml I like quite a bit, and its typing system is world-class. Lightning-fast compilation times are a huge productivity boost, one I didn't expect. But lack of even basic parallelism wrappers is pretty off-putting (I am not counting convenient pthreads-like DSL, they are just that and nothing else).
All in all, I'd still go for Elixir for 90% of what I ever do due to its preemptive scheduler and very friendly parallelism and concurrency story. And higher-level tools -- like Broadway -- keep getting added.
The BEAM languages are not a panacea. That's a fact. They seem to fit excellently in the web programming though.
I kinda know the theoretical differences but I’m curious what hn thinks. Has someone actually deployed things in some of these.
I do have preference towards rust but maybe there’s something better.
If you are specifically making a graphql backend, idk why you would anything but node, since it has the most mature ecosystem.
I've been building software using the exact stack described in the linked article for the last three years. I don't think about my backend very much, it mostly just works. Most of the work is on the front end. Elixir/Phoenix/Absinthe have been almost completely frustration-free. I'm sure Rust and Kotlin are great too.
https://stressgrid.com/blog/100k_cps_with_elixir/
What it's not good at is numerical computation.
As for the others I have no clue but I am sure jvm won't be as low latency as Elixir's VM since it's not a preemptive scheduler.
There's a JVM vs BEAM VM comparison paper (http://ds.cs.ut.ee/courses/course-files/To303nis%20Pool%20.p...).
I believe section 2.2. ERTS of the paper talk about BEAM VM advantage for low latency.
> The per process heap architecture has manybenefits, one of which is that because each pro-cess has it’s own heap and there are presum-ably numerous process’s,...
update:
Ah... I think the relevant part is in 3.3 Lightweight threads.
> 3.3. Lightweight threads
> The effect is that Erlang is one of a fewlanguages that actually does preemptive mul-titasking. The reduction count of 2000, whichis sub 1ms, is quite low and forces many smallcontext switches between the Erlang processes.
I've been using it lately, coming from Node.js for the last few years, and it's been a really nice change.
The main difference is I've been using TypeScript and the Absinthe resolvers are using Dataloader.
I don't like the stack. It's way too heavy. If I were building everything again now, I'd go with a standard Phoenix app and ditch Redux for sure and hold off on fullblown front-end frameworks and Graphql.
I'd ship v1 with Phoenix, UJS and Turbolinks, and it would take 1/4 the time. Then, if the project grew to where it needed and could support a larger team, I'd bring in Absinthe, Apollo and then finally Vue.
My main recommendation would be to keep things as simple as possible for as long as possible. Use something like Phoenix or Rails or Laravel and get stuff shipped.
In Elixir, as in Lisp, you can transform your code as a data structure into new code. This is importantly used in Ecto, the de-facto object data mapper in Elixir. It’s also used in Absinthe, a library from the posted link for making GraphQL APIs.
* efficient utf8 binary strings (which is actually conforms with utf8 standard), no mixup with charlists
* mix etc standard tooling
* macros
* enum stuff
* pipeline operator "|>", which by its mere existence enforces input/output param's consistency in user-written functions
etc
A big one is that Elixir allows rebinding of a variable. Erland does not, which tends to lead to ugly names since in erlang you can't do common reassignment patterns like x = x * someConversionFactor.
The two really big features are macros and protocols.
Elixir's whole reason for existence is to make the Erlang programming model easier to use. Try it and see if it does.
* Tests inside documentation.
* Macros.
* Protocols.
* `mix xref` has a family of sub-commands that gives you good code analysis (callers, callees etc.)
* First-class Unicode support. All strings are UTF-8 by default (you can fallback to ASCII if you need it; there are also good transcoding libraries in both Elixir and Erlang).
This drives me batty. More familiar? Sure. “Friendlier” is very subjective.