markpapadakis

Related, and an excellent write up on Shalin Monks: Demythologizing Shaolin Monks( https://skeptoid.com/episodes/4516 )

I moved to C++ in 2000, from C, because it made it easier for me to do more with less code, and as consequence of that, require less time to build what was needed and less time to debug and maintain it. I have looked at various other contenders over the years for a replacement, not because I felt I needed one, but because they all had something interesting to offer, and because the buzz have gotten to me. I have concluded that C++, especially with the recent ‘modernization’ initiatives (C++11/14/17) is as good as it gets for me, considering my needs and expectations, and obviously factoring in my familiarity with the language.

Most popular programming languages nowadays are providing fancy runtime features (GC, introspection and reflection, dynamic dispatch) at the expense of performance and unpredictable operation. You may get some control over the execution semantics, but not much. For most use cases, I suppose this is fine, but when you go deeper or higher(in the Y performance and scalability axis), you start to question your choice and realize you don’t know what is really going on, especially if the profilers tell you that the culprit is the language runtime, and so you end up spending your time working around the language limitations instead of doing ‘real’ work.

The prevailing opinion nowadays is that C++ is complex, and use of templates, operation overloading, and even hierarchical objects structure, makes it hard to understand what is going on. It certainly mystifies many ‘pure C’ developers who are eager to come out and speak against the language, as if they are on some sort of mission to bash it, or just because they are not familiar with the language themselves and, well, if other like minded people are speaking against it, they may as well join them.

C++ is is a superset of C, and everything availed by C++ is optional — you absolutely don’t need to use anything you don’t want to use, or don’t feel yet comfortable using. I don’t understand why this just escapes more people who speak against it. Maybe because it’s not helping their cause, whatever it is.

There are brilliant people who will work around the limitations of a language and deliver incredible products, frameworks, libraries (Aeron - https://github.com/real-logic/Aeron comes to mind). There are languages that are better suited to many tasks than C++ - where performance is a big concern, but managed runtime environments and writing high level code makes more sense (or maybe you just care about compilation times, a very valid concern). There is no perfect programming language.

There is only the language that works best for you and your needs. For me, that is C++, and I can’t wait for the C++17 updates.

Apple definitely hoped for an iPhone like leap forward for smart watches, and certainly tried hard to accomplish just that - but they probably realised soon after it was made available for purchase that this was no repeat, maybe even long before that.

The fact of the matter is, the iPhone was a truly revolutionary product(they often quoted that in their marketing material, but that was no hyperbole), whereas the Watch has been an evolutionary advancement compared to what was already at the market, at best. That is to say, it didn’t do anything particularly different, or did what alternative products did differently. It integrates well with the Apple ecosystem, and that digital crown is interesting( from what I am hearing, it’s not that great in practice ), and..well, that’s about it, really.

I believe Apple can eventually transform this into a great product, one step at a time. I am not personally drawn to the Watch, or any ‘smart watch’ for that matter, but maybe it’s because I haven’t experienced yet what Apple hasn’t invented yet that would make me want to buy and wear it. Maybe next year when the Home Kit initiative enables ‘smart devices’ and the watch to play nice together, or when you can control your Apple Car using the watch and Siri (if you can name your Apple car, I ‘ll name mine KITT), then I ‘ll really want it.

Same here. I read this every year -- it motivates me like no other work or fiction or reality. I bought a few copies for friends, and it seems to have similar effects on them. I can't recommend it enough.

I hope f.lux gets their chance with Apple. Their application is great and helps so many people. Btw, Michael Herf is the guy who built Picassa. He is a brilliant programmer and be sure knows a lot about graphics and colors.

I ported GCD to Linux when it was made available/open-sourced some many months ago. Effectively, GCD uses a bunch of (named) FIFO queues, protected by semaphores, and manages some OS threads, which dequeue tasks(or blocks) from the queues and execute them. It provides some nice higher level abstractions and functionality (e.g groups ) on top of that. It also supports both synchronous and asynchronous execution semantics. The OS threads are of course managed by the OS scheduler. (unfortunately, pthread workqueues are not supported on Linux yet -- the original implemntation creates workqueues for each defined priority and that's how priorities are supported in GCD. On Linux you can use setpriority(), ioprio_set() and other such APIs that don't really offer much in practice. But I digress).

Contrast that to coroutines/fibers where tasks(i.e functions), which run in the same I/O thread(though not an absolute requirement). Cooperative multitasking is used, where a running function can yield control to other runnable functions by explicitly calling a yield like function -- that is, there is no scheduler that preemptively stops a function and runs another. You need to do that yourself.

There is some overlap between the benefits and properties of tasks that run on OS threads and fibers, but for the most part, they have different pros and cons. You can't really use in practice fibers to get the benefits you get from a GCD like framework (which utilizes multiple H/W cores to run functions/tasks concurrently), but also, you can't use GCD to improve throughput on a per-thread basis and deal with long-running and/or blocking lightweight tasks, or implement a fair-scheduling execution scheme).

libmill offers a channels(FIFO queues) abstraction. This is a fibers/coroutines framework. They solve different problems. See go-lang channels for what libmill is providing.

For what it's worth, the network I/O model design+implementation is rarely what makes the difference between high performance, or not, 'servers' -- and that is a mostly solved problem, at least in terms of patterns and practices that are adopted by 'modern' servers (few threads, no more than actual hardware CPU cores, async.multiplexing, etc).

High performance is about everything else, including processing actual incoming requests efficiently, reducing block/busy time in threads, caring a lot about memory access/caches. They key IMO is concurrency and migrating blocking operations to other dedicated threads, and this is where an efficient coroutines implementation would help both with accomplishing that task, but, perhaps more importantly, keeping the programming model simple.

C++17 will introduce support for reusmable functions ( http://blogs.msdn.com/b/vcblog/archive/2014/11/12/resumable-... ). This will be a game changer. Now, you can approximate this by e.g creating task abstractions and use lower-level(but slow) setjmp/longjmp functions, or just design tasks/functors that hold state that can schedule other tasks in turn, and so on. It works, but the overhead, mental, but also n terms of processing/executing/scheduling can be too great.

I am not using AeroSpike nor am I affiliated with them. I just studied their code base (I study codebases as a hobby ). There is a _lot_ of room for improvement there IMO but that's true for the majority of codebases I went through :)

ClangFormat is your friend ( http://clang.llvm.org/docs/ClangFormat.html )

There is nothing fundamentally new described in the blog post, although pinning threads to core is almost never the default operation mode (though many popular servers expose an option for turning this one).

As someone else stated in another post, SO_REUSEPORT and accept4() is best, all things considered, way to accept connections across multiple threads. Again, most modern servers support this by default, if supported by the underlying operating system (e.g nginx).

By the way, Aerospike accepts new connections in a single thread and then directly adds the socket FD to one of the i/o threads (round-robbing selection scheme) directly using e.g epoll_ctl(fd, EPOLL_CTL_ADD, ..).

See http://blog.tsunanet.net/2010/11/how-long-does-it-take-to-ma... for costs of context switching, and performance improvement when pinning threads to cores (it's quite impressive). Also, note that according to the author, on average, a context switch is 2.5-3x more expensive when using virtualization.

You may also want to read https://medium.com/software-development-2/high-performance-s... -- it's been a long time since I wrote this, but it describes how one can deal with asynchronous I/O and other operations that may block a thread.

Obviously, I thought I didn't need to mention that, by everything ourselves I meant our core infrastructure. We didn't write an OS, a compiler tools chain, an editor, a standard library etc. Though we did write our own javascript syntax/semantics based language/compiler/runtime for server side needs. Like I said, if we were to start over, we d use OSS, but, all things considered, that decision didn't hurt us and gave us leverage, knowhow, flexibiliry and that helped is be where we are now.

This is really about closed source, proprietary infrastructure technology (not all kinds of software in general ), offered by startups, not established companies.

That is to say, people use Oracle, SQL server, Teradata, etc and they are not worried they will go out of business or be sold or otherwise that they will be left in the dark by a sudden shift in business practices and product availability.

The problem is almost entirely with startups, which are an easy target for bigger companies interested in their technology and team skills set. This is even more so an issue because the majority of startups are VC funded and are under pressure to sell or comply with VCs interests.

So in practice you have three ‘safe’ ways to build your infrastructure, which are not mutually exclusive. Choose OSS software, buy from companies such as Oracle, IBM, HANA, etc, and build it yourself. Depending on the expertise of your team and funds available for purchases, as well as qualifies of the available solutions, OSS is probably the safest way, followed by purchased from large corps. Rolling your own infrastructure is expensive, time consuming, requires committing your developers to building infrastructure instead of well, building a product, and may not work in the end at all.

Even the companies that can afford to do it everything in house (Facebook, Yahoo, Twitter, etc) choose OSS for the majority of their needs and build on top of that. Google is Google. However, if that infrastructure is your selling point and what differentiates you from the rest, and/or you have very specific needs and makes more sense to do it this way, doing it yourself can be a great alternative.

We rarely use OSS here, and we don’t use any proprietary infrastructure technology. We have built everything ourselves, and it has worked out great so far, but we have put a lot more effort and resources into that, whereas we could have instead invested on the Product. If we had to make a choice again, in retrospect, we ‘d most likely have gone the OSS way. It’s all about tradeoffs.

In practice, you indeed yield often enough — there is no perfect formula I believe — so in loops or elsewhere where you are spending too much time doing one thing. You don’t need to do it perfectly, you just need to yield often enough to give other coros a fair chance at completion. Relying on the kernel is a bad idea; that’s where you lose on performance. Also, to do what you are suggestions would mean that you use many, many OS threads and the kernel somewhat manages them all in a pre-emptive fashion, and you don’t need to do it yourself; but that means you need to pay the context switching, memory footprint etc costs. It won’t scale and will be slow.

The fewer the trips to the kernel, the more work you can do in user-space, the fewer the sys-calls, the fewer the context switches, the better the throughput and performance.

Even if a request is completely CPU-bound, it’s a good idea to be fair to other requests accepted in the thread; that is, if a request should take 2 seconds of processing time, while others would take a few microseconds, they shouldn’t all wait until that one request is processed.

Instead, yielding back to the scheduler where appropriate, will give them a chance to complete in time proportional to the effort required to process them, not to the time required to handle long-running requests that stall processing of other coros.

This is pretty cool. Thanks for sharing.

I am familiar with Little's Law. The key concept is optimistic sequential execution. The overwheleming majority of requests do not need to page in cold-data or otherwise stall for too long. Also, in the case where they can stall to e.g process a lot of data in a loop, they can yield back to the scheduler, thereby being fair to all other accepted coroutines/requests and giving a chance to others to complete earlier. In practice, at least based on my prototype and tests, this works extremely well. If about 10-15% of the requests may end up blocking the thread, while the read can read kernel-cache resident data and proceed, that's really huge in terms of performance gains. The background threads that either become owners and responsible for running slow/blocking coros can rely on a work-strealing scheme to keep them all busy, and if you don't rely on them, thereby removing contention/stall from the 'fast' threads, you are going to eventually suffer stalls and latency spikes.

This is a C++ implementation. Erlang and other high level languages use Processes(in the case of Erlang) or other such constructs to encapsulate logic and those are indeed scheduled based on either cooperative/preemptive fashion.

The main goals of this system is to minimize potential for blocking or otherwise slow processing steps which can stall processing of other requests, especially if this happens for many such requests/second.

Specifically, the 'optimistic sequential execution' idea is that you run the request on the request it was accepted in, use mincore() to determine if will _likely_ stall if you read from it, and if so, either migrate the coro to a 'slow' coros threads list, or just keep on going (which should be the fast-path here).

This is a solution to a specific class of problems. The similarities with Erlang's Process scheduling is that a process is a lot like a coroutine (it encapsulates logic and data and it's scheduled by a user-space scheduler, and that there can be thousands of them in runnable state, and each has its own stack). But that's really the definition of stackless coroutines.

Wired Tiger's storage engine implementation quality is _very_ high. The core developers are experts in the field and really know what they are doing; check the codebase on Github.

Integrating that technology in MongoDB will make a big difference in terms of performance and scalability, and I imagine the folks who built it will be able to put use their skills in good use there, further improving MongoDB left and right. A great deal, of great value, for MongoDB.