HNHacker News
TopNewBestAskShowJobs

mononcqc

2,658 karma · joined March 17, 2009

Blog: https://ferd.ca books: https://learnyousomeerlang.com, https://erlang-in-anger.com, https://propertesting.com
submissionscomments
mononcqc··on Usefulness of mnesia, the Erlang built-in database
I'm the Fred mentioned in the quoted text. As mentioned by FLGMwt, Mnesia being a 90s database was an offhand remark done at a workshop in a discussion doing a break.

The reasons I considered it a DB of the 90s is that back then, it could have been state of the art, but by today's standards, under its current form, it makes sense to be used mostly on fixed cluster sizes with reliable networks and a fairly stable topology.

Any fancier cases and you start requiring to dive into the internals when it comes to coming back up from failures, partitions, requiring repairs, and so on. The DB has 3 standard backends: all in RAM, all on Disk (with a 2GB limit), or as a log-copy (infinite disk size, but also bound by memory size).

That ends up leaving you with a DB that has a need for its whole dataset to fit in memory, supports distributed transactions but can't deal with network failures well out of the box (you need something like https://github.com/uwiger/unsplit)

Mnesia gaining new backends (Klarna is currently open-sourcing code for an experimental postgres backend and are using a leveldb one) would fix a lot of issues as a single-node DB, but another overhaul would be required for the rest.

The problem I see is that it was a very cool database back then, but it started lagging behind for a long while and now it has to play a catch-up game. Its model and Erlang interface is still extremely nice and I wish it made more sense to use in production without committing to learning its internals in case of troubles.

mononcqc··on The Little Printf
I would say it's a cautionary tale against making it so people get used to you accomplishing small miracles, or making sure everyone is aware you wouldn't tolerate only doing the tasks you hate forever.
mononcqc··on The Little Printf
There's a PDF version on S3, if that helps: https://s3.amazonaws.com/ferd-ca/printf.pdf
mononcqc··on Time in Go
Erlang is an example of a language correctly handling these aspects, regarding time correction, drift, and so on: http://learnyousomeerlang.com/time
mononcqc··on RethinkDB 2.1 beta: automatic failover with Raft
Well the problem is obvious in mobile devices, but any asymmetric netsplit will result in a single node taking down your cluster for as long as the netsplit lasts.
mononcqc··on RethinkDB 2.1 beta: automatic failover with Raft
Have you dealt with mitigating Asymmetric partitions in Raft? http://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-857.pdf shows that a single defective node has the potential to entirely render a Raft cluster useless, specifically in the case of asymmetric partitions:

> The mobile devices will often be disconnected from the majority of other devices, thus running election after election without being able to gain enough votes. When these devices rejoin the network, they will force a new election as their term will be much higher than the leader. This leads to frequently disconnected devices becoming the leader when reattached to the network, thus causing regular unavailability of the system when they leave again.

> [...]

> It is also interesting to consider the impact of a single node being behind an asymmetric partition [...] This node will repeatedly re-run elections as it will not be able to receive incoming messages, each RequestVotes will have a higher term than the last, forcing the leader to step down.

Specifically, they required to add client request caches, exposing serial transaction IDs in every single record and log entry, specific timeouts and measures for livelock bugs this introduced, which include replacing an entry by a noop value to be replicated on some read-only operation done at a later time. On top of this, these modifications may end up reducing the serializability constructs to bring it closer to what a Master-Follower replication may give in a DB -- i.e. it doesn't guarantee one true serializable sequence of commits anymore, but only that the state replication will be consistent across all nodes

mononcqc··on Mnesia and LevelDB: Liberating Mnesia from the limitations of DETS
Distributed Erlang on its own is fine to run in production (particularly as a control plane for metadata between nodes) because few assumptions about reliability of the network are baked in; most of them can be made by the application designer.

Distributed OTP applications (with the automated takeover/failover mechanism) see very little use in the real world because of their set of assumptions that network failures are rarer than software or hardware failure, which result in perceiving all netsplits as nodes going down (a great way to get split brains!)

mononcqc··on Setting Go variables from the outside
What's the advantage of doing that vs. just using a macro defined at compile-time?
mononcqc··on Elixir Streams
The closest thing I can think of outside of using funs or macros to define the usual `delay` and `force` operators, and that currently exists, would be something like the QLC interface, which allows you to turn whatever you want into a type that can be used in list comprehensions and functional queries around it: http://www.erlang.org/doc/man/qlc.html
mononcqc··on Elixir vs. Erlang: a question of momentum
Yes, pretty much. I think we went short on the rebar3 descriptions just because of how known 'rebar' is as a household name. Go to almost any Erlang project, and you'll find 'rebar' or some variant.

Rebar3 is the next version of it.

But you're entirely right, the page and documentation is lacking to people unfamiliar with the Erlang/OTP ecosystem.

mononcqc··on Parallel Map in Elixir
This implementation does rely on a lot of memory copying going on there. Rather than copying the original element to the new process, then the new mapped element back to the original process (giving you the original list's worth of memory copied twice), you'll be copying each list element once to every worker you spawn, and then you'll copy the parts of the list worked on to the next worker, on and on, until the full list is finally copied back to the caller.

So instead of copying 2n elements over the processes, you'll be copying roughly 2nlogn (or is it n(n+1)/2 ?) elements around.

mononcqc··on Vimb – Vim-Like Web Browser
Their own nightlies started working again recently: http://5digits.org/nightlies
mononcqc··on Queues Don't Fix Overload
The longest queue is going to be in front of the tightest bottleneck, but not the central bottleneck critical to your application.

If you find you have a bottleneck halfway through the pipe and you make that bottleneck go away, your true real bottleneck down in the wall still remains there, and all your optimizations done above that one will end up just opening the floodgates wider.

Once you've protected that kind of more at-risk core, then it's easier and safer to measure everything that sits above it in the stack, and it puts an upper bound on how much you need to optimize (you stop when you hit your rate limits on the core component).

mononcqc··on Queues Don't Fix Overload
Yeah, the text that goes behind that post in chapter 3 of http://www.erlang-in-anger.com focuses a lot more on finding the underlying true bottleneck, but I didn't feel like re-writing the same thing in many places, differently.

So I headed for a queue rant, because at least that way I only have to link to the blog post when I end up discussing these issues in the context of queues.

mononcqc··on Queues Don't Fix Overload
Yes, that's accurate. The loss of end-to-end principles when people use queues in a fire-and-forget manner is something I also decry.
mononcqc··on Queues Don't Fix Overload
It sounds like common sense, but a lot of people use the queue as a means to widen the neck, or rather the current perceived (and measured!) bottleneck they have. Usually there is another bottleneck hidden deeper in the system, and that one is the one that will cause all kinds of terrible issues.
mononcqc··on Queues Don't Fix Overload
Yep, that's a good point I did not mention. Thanks for providing the links.
mononcqc··on Queues Don't Fix Overload
Author here. I use queues all the time. I program in Erlang and there is one unbounded queue in every single process' mailbox in every node.

The problem is not queues themselves, it's when queues are used to 'optimize' an application (usually the front-end) by removing the synchronous aspect of things. In some cases it's reasonable, say long running tasks like video processing or whatever could be handled in many other ways than blocking up the front-end, for friendlier results.

I've seen queues applied in plenty of times where their use case will be to handle temporary overload (by raising latency and pushing some work to be done later, when the system slows down). The problem is that as time goes and that temporary overload becomes more and more permanent (or has such a magnitude as you can't catch up on it in your slow hours), it suddenly starts exploding very violently.

How much time the queue buys you is unknown, because the people running in these issues have likely not been able to characterize the load on their system, or the peak load they'd have (because the system dies before then). It's a challenging problem. Ideally you fix it by putting boundaries on your system (even on your queue! that's good!) and expect to lose data eventually or block it.

The really really dangerous thing is using a queue and never expecting you will lose data. That's when things get to be very wrong because these assumptions lead to bad system design (end-to-end principle and idempotence are often ignored).

And then people go on to blame queues as the problem, when the real issue was that they used a queue to solve overload without applying proper resource management to them.

Most of what I rant about in that blog post is stuff I've seen happen and even participated in. I don't want that anymore.

mononcqc··on How I Start: Haskell
I'm the auhtor of the Erlang article on How I Start. I've felt the same in some ways because it would offer a nice comparison point, but the problem with this idea is that it doesn't let the authors pick a problem for which they think their tool shines.

In a way, forcing the same problem on all authors is more or less risking telling them "you may have to use the wrong tool for this job, but please tell us how you'd do it."

Instead, I think the current format is, well first of all nicer for the author (they can pick what inspires them), and also gives you a richer level of content where you may end up learning more than you thought you would.

For example, the Erlang text focuses on Releases and shipping Erlang systems, something I felt was still blurry in a lot of people's minds. the Elixir project is done with a concurrent approach that wouldn't necessarily carry over the same way in Ruby, and Ruby's text is showing how to build a gem.

Letting the authors pick their subject means they can choose to focus as much as possible on what they feel is trickier with their tool of choice, rather than just showing a given program, for which a site like rosettacode.org might be more interesting.

mononcqc··on What can I only do in Erlang?
Erlang has Dialyzer, which is an optional type checker and discrepancy analyzer.
mononcqc··on What can I only do in Erlang?
I've personally used Erlang in Real Time Bidding scenarios, where soft-real time constraints exist and is on the order of milliseconds.

It's a fine tool for this, too.

mononcqc··on What can I only do in Erlang?
A few points:

- A large system never runs on a single machine if you want any level of realistic fault tolerance. Two nodes is a bare minimum, 3 an acceptable one.

- The number of nodes will always depend on where your bottlenecks lie. Erlang developers would be rare and few to write CPU-bottlenecked code directly in Erlang. The usual approach would be to write your system in whatever is appropriate, and then too coordinate things with an Erlang layer.

- I would a hundred times more willingly maintain and debug a running Erlang system than a C++ one. I'm kind of sold on the idea though, and wrote http://erlang-in-anger.com to share my experience there.

mononcqc··on Meteor hits 1.0
If you can swap out the drivers to solve the problem, chances are this is not "especially large scale."
mononcqc··on Trouble at the Koolaid Point
That something is legal does not force it to be acceptable by any business whatsoever.
mononcqc··on Stuff Goes Bad: Erlang in Anger
I'll possibly do it. I have to think.

The issues I can see with putting the source itself open is receiving pull reqs, which quickly turn into an issue of copyrights and form-waiving and all kind of stuff I'm not comfortable running around with.

I have a friend who's currently playing with making the conversion work with far more determination than I have, though.

mononcqc··on Elixir v1.0.0 released
Elixir currently compiles to the Erlang abstract parse tree. This then gets to be BEAM byte code later on.
mononcqc··on Stuff Goes Bad: Erlang in Anger
Pandoc exploded on the first code sample with an underscore it met. Too bad.
mononcqc··on Stuff Goes Bad: Erlang in Anger
I tried tools like pandoc and they promptly choked on the latex. Every code sample that had an _ was problematic, and that issue likely didn't come alone.
mononcqc··on Stuff Goes Bad: Erlang in Anger
Nope. Not for sale. Always for free.

I will be looking into whether it's possible to do decent conversions from LaTeX -> [epub, mobi] later this week, and then figure out how to make that available, if it goes well.

mononcqc··on Stuff Goes Bad: Erlang in Anger
This should have been updated to text.v1.0.2.pdf roughly 8 hours ago. I fixed issues with the running header, page numbering, and a few content issues pointed out to me.
← PreviousPage 4 of 11Next →