mariusae

For myself, it's been super helpful in at least two ways:

1) Staying on top of my kid's school obligations, homework, etc. School Learning Management Systems are a hot mess, and I use agents to consolidate and collate, to keep track of the homework that has been done, what needs to be done, and even pre-grade homework to give feedback (most of the work is done on digital platforms too). It's been immensely time saving for these kinds of things. 2) Multiple instances of watching things passively: hotel room prices, availability of tickets, finding the best place to sell things online. In general things that require a lot of browsing, collating, and watching. These are the kinds of things that specialized software products could do as well, but with an agent they are just a prompt away, and you get exactly what you want. (More or less.)

> Page and Brin […] and are often at each other's throat.

Citation needed.

If you want to work with Go, check out bigslice [1] (and bigmachine [2]), which is built on a similar architecture.

[1] https://github.com/grailbio/bigslice/ [2] https://github.com/grailbio/bigmachine

Reflow [1] is a similar attempt at a slightly different domain: bioinformatics and ETL pipelines. Reflow exposes a data model and programming model that reclaims programmability in these systems, and, by leaning on these abstractions, gives the runtime much more leeway to do interesting things. It unties the hands of the implementer.

[1] https://github.com/grailbio/reflow

See also bigslice (https://bigslice.io) for another take on this.

> I want an Airflow that runs and scales in the cloud

I'd encourage you to look at Reflow [1] which takes a different approach: it's entirely self-managing: you run Reflow like you would a normal programming language interpreter ("reflow run myjob.rf") and Reflow creates ephemeral nodes that scale elastically and that tear themselves down, only for the purpose of running the program.

> has extensive observability (monitoring, tracing)

Reflow includes a good amount of observability tools out of the box; we're also working on integrating tracing facilities (e.g., reporting progress to Amazon x-ray).

> has a full API, and maybe some clear way to test workflows.

Reflow's approach to testing is exactly like any other programming language: you write modules that can either be used in a "main" program, or else be used in tests.

[1] https://github.com/grailbio/reflow

Reflow [1] is also well-suited for ETL workloads. It takes a different tack: it presents a DSL with data-flow semantics and first-class integration with Docker. The result is that you don't write graphs, instead you just write programs that, due to their semantics, can be automatically parallelized and distributed widely, all intermediate evaluations are memoized, and programs are evaluated in a fully incremental fashion:

[1] https://github.com/grailbio/reflow

That's false. With anonymous closures you can implement select. See concurrent ML, or https://github.com/twitter/util/blob/develop/util-core/src/m... (which also happens to be lock free)

yes, they were recorded, and will be posted.

You are right that this fails when you move outside of the model. However, we don't.

You may be surprised (amazed?) to learn that, internally, 100% of composition happens in this manner. We have a massive code base, and we've not seen this be an issue.

Further, we've worked with the Scala community to standardize the idea of an "execution context" which helps make these ideas portable, the particular of the implementation transparent to arbitrary producers and consumers of futures, so long as they comply to the standard Scala future API. (Twitter futures will when we migrate to Scala 2.10.)

(For fun, I just searched through our entire codebase. The only manipulation of locals anywhere is in the util library.)

More or less 100% of asynchronous composition is done via Futures[1] whose default implementation[2] does this for you.

[1] https://github.com/twitter/util/blob/master/util-core/src/ma... [2] https://github.com/twitter/util/blob/master/util-core/src/ma...

Also make sure you read Eddie Kohler's LaTeX usage notes:

  http://www.read.seas.harvard.edu/~kohler/latex.html

Many modern text editors (can) use proportional fonts. It's quite nice. You should try it out sometime.

hi, i'm one of the authors.

the scala school is mostly intended to be used in conjunction with a course we teach.

we're working on moving it towards being more self-contained, independent from the accompanying lectures.

not at all -- we're aggressively adding datasets, and those are among the ones that are on the top of our list :-)