leef

That is kind of the point of this repo. If your API implementation is too expensive for lambda move the same container to run on EKS/ECS fargate or ec2. The opposite holds as well - if you can save money by moving it from fargate to lambda then you can use the same container.

Ion has a binary format but is not specifically about real-time streaming. It is a JSON replacement.

Ion originated 10+ years ago from the Amazon catalog team - the team that kept data about the hundreds of millions of items available on Amazon. Nearly every team in the company called the catalog to get information about items all the time - scanning the entire catalog, parts of the catalog, millions of individual item lookups every second, etc.

They did the math and some very large percentage of network traffic in Amazon Retails data centers was catalog data. If that data, currently in XML or JSON format, was sent in a more compact format it would save some ridiculous millions of dollars every year. So Ion was born and eventually open sourced.

I have my Calibre library store to Dropbox. Remarkable integrates with Dropbox so my library is always in sync!

Works pretty well.

The original pdf - http://www.cypherpunks.ca/%7Eiang/pubs/shattersecrets-spw18....

I finally got into emacs using the prelude distribution - https://github.com/bbatsov/prelude

It has sane defaults and contextual tips that help when you're getting started (and can turn off later).

Yes - http://amznlabs.github.io/ion-docs/spec.html

The intent of the stored schema isn't really for self-description. A typical use case for Avro is data storage over long periods of time. It is expected that the schema will evolve at some point during this time. Therefore you still need to specify a target schema to read the data into which is allowed to be different than the stored schema. Avro then maps the stored data into the target schema by using the stored schema. Most avro libraries expect you to get the target schema from a separate source before reading data.

Ion has functions to turn Ion into JSON which will, of course, lose information. Annotations are dropped, decimals turn into a JSON number type which may lose precision, etc.

That's not true. Avro at least existed at the time. However they wanted something self-describing to replace JSON/XML usage. Avro is better suited as a data storage format rather than a transit oriented format. Of course, both Ion and Avro can be used for either, but Avro will give you better compression on disk, but Ion is less cumbersome since it doesn't require a schema

Finally! I've had to live the JSON nightmare since I left Amazon.

Some of the benefits over JSON:

* Real date type

* Real binary type - no need to base64 encode

* Real decimal type - invaluable when working with currency

* Annotations - You can tag an Ion field in a map with an annotation that says, e.g. its compression ("csv", "snappy") or its serialized type ('com.example.Foo').

* Text and binary format

* Symbol tables - this is like automated jsonpack.

* It's self-describing - meaning, unlike Avro, you don't need the schema ahead of time to read or write the data.

Why the severe restriction on users? $300/mo to go beyond five users eliminates any advantage on log volume.

I realize your previous comment was more in reference to transient network partitions so my comment is out of place. But whether the mechanism is manual or automatic once a network partition is discovered the reshuffle begins.

Really it doesn't matter and your point only goes to show one of Cassandra's flaws. When either a network partition or a server failure happens Cassandra starts reshuffling data amongst multiple hosts and filling network pipes. Contrast this with a setup where you have a static partitioning of hosts to partition and a leader per partition. Then you only need to (possibly) elect a new leader and carry on.

This is especially relevant when you need to do these things because of unexpected load increases or the loss of hosts in your cluster.

What's interesting to note is that Dynamo/Cassandra usage was killed at both Amazon and Facebook (DynamoDB from AWS is actually not based on Dynamo tech except in terms of what not to do).

[1] - https://www.facebook.com/notes/facebook-engineering/the-unde...

edit: Added link to facebook post on hbase vs cassandra

It doesn't. As an average developer it's my public/private cloud providers job to take my images/dockers and schedule them correctly on whatever underlying fleet of hardware they run.

I can only assume Tectonic and related tech are aimed at those developers who run hybrid or private clouds for the rest of their company.

Don't get too excited. This proposal is more a codification of the "easy" part of managing dependencies for top level/binary packages that much of the community already does. This proposal does not (and isn't intended) to cover how dependencies work when vending libraries and does not address shared package and diamond shaped dependencies.

That's not to say having a standard for vendoring isn't a good thing.

I've been using an (ubuntu) system 76 laptop [1]for the last two years. Great machine, no issues.

[1] - https://system76.com/laptops/gazelle

Nothing is magic but in this case moving from a language with a GIL to a language built for (single machine) concurrency probably scales a lot.

The OP says as much in the comments: "For us, one of the biggest latency wins comes from the fact that go can truly execute sql statements in parallel (whereas python's GIL serialized these parallelizable operations). In general, single-threaded go is at least 5x faster than pure python (without c-module)."

"If you want a quiet oncall rotation, it's imperative to have a system for dealing with things that need timely response, but are not imminently critical."

This is an excellent point that is missed in most monitoring setups I've seen. A classic example is some request that kills your service process. You get paged for that so you wrap the service in a supervisor like daemon. The immediate issue is fixed and, typically, any future causes of the service process dying are hidden unless someone happens to be looking at the logs one day.

I would love to see smart ways to surface "this will be a problem soon" on alerting systems.

Back then? We haven't gotten all that much better according to John Oliver (http://www.youtube.com/watch?v=1Y1ya-yF35g)

Because apparently Apple is too busy for redirects.

For when you want to supervise goroutines instead of processes (Presumably because processes are consuming too many resources).

Seems a bit difficult to be the anti-Facebook when the very first screen presents a login with Facebook button.

No need to vendor. Use Godep without copying (godep save -copy=false) to create the equivalent of a bundler.lock file and check that into source.

Sounds like something to use Kinesis[1] for. Read the kinesis stream, process the updates and write/rewrite the S3 file.

1 - https://aws.amazon.com/kinesis/

Pure FUD. Oracle has owned Innodb since 2005 and now that they own the whole of MySQL impressive releases are coming at a steady clip. To date, Oracle has been a better steward of MySQL than Sun was.

It's mostly a moot point since it's overseas, but I doubt the secret court would be willing to pass through over 6 million FISC requests a day (181,280,466 requests every 30 days the NSA reportedly gets through this wire tapping). At the very least the FISC courts provide a bottleneck at whatever government 'efficiency' throughput they can provide.

No it's not. The court system, even if it is rumber stamp-y, provides way more visibility and accountibility than secret and total access to Google's data via tapping data links.

The intent is for add-ons to ship in a box with the main Prime order. This doesn't always happen due to the combination of what you order and the locations of the items across Amazon's fulfillment network. When this happens Amazon is eating the cost of those extra shipments and is considered a 'miss' by the inventory folks.

Everything in the warehouse is not prime. If fact, an item can flip from prime to not prime in near real time if the most accessible inventory is used up and all that is left is the stuff in the 'reserve' areas of the warehouse. This is the area where the giant boxes of inventory stored. Grabbing inventory from there involves pulling it down, opening, pulling apart eaches, and then stowing in fast access areas.

Add-On items are items that are not economical enough to be in the Prime program by themselves but can be thrown in a box already on it's way to you. Without the Add-On program they would not be Prime shippable at all.

Therefore Add-On items are just an improvement to Prime.