stochastician

Hi! Pywren project lead here, we're working on more and more fun HPC-style projects! We're also interested in expanding to more and more backends, so please feel free to reach out if there's something you'd like to see.

Could you point to where this number comes from more precisely? I'm incredibly curious, given that it is an order of magnitude different from what I've seen. For example, there's the Tufts study:

http://csdd.tufts.edu/news/complete_story/pr_tufts_csdd_2014...

which arrives at $2.6B. And if you are interested in keeping up with some of these studies from someone in the industry, I highly recommend Derek Lowe's blog -- he's a medicinal chemist. For example, http://blogs.sciencemag.org/pipeline/archives/2017/09/12/it-...

Can you explain by what you mean w.r.t. "NSF and DOE funded projects, you are increasingly being told to use supercomputing centers"?

You can actually just place a random diffuser directly on a sensor -- this is work from some colleagues at UC Berkeley which I (surprisingly) didn't see mentioned in this article https://www.medgadget.com/2018/01/diffusercam-lensless-3d-im...

DiffuserCam was previously discussed on HN but I couldn't find the link.

Author of that PlosCB paper here, I love these plots! This is such great work, and I've been telling everyone I know to check it out.

See https://arxiv.org/abs/1705.08292 from our group

Original paper author here. I think we are in vehement agreement!

1a. we don't run big systems. Well, I don't, at least -- my coauthors wrote spark and similar systems, so they're certainly aware of some of the challenges, but for us the major appeal here is someone else (cloud provider) is running a tremendous amount of the system!

1b. I think you're totally right that these sorts of systems tend to be generalized beyond their sweet spot. We have had quite passionate debates internally as to the amount of complexity we should add to the underlying system to support richer BSP efficiently. I just want to make sure the "obvious" stuff works as easily and quickly as possible (that is, basic map)

2. Segregating compute from storage _is_ untenable for a lot of workloads! But for a lot of the compute-heavy work we do, it's not. And we were really, truly shocked at how big the bandwidth to S3 really is -- for our compute applications, this degree of data disaggregation

3. "Model what the dev is using for dev" is _exactly_ what we're trying to accomplish. We even try and replicate the dev environment (thanks to anaconda, cloudpickle, etc.) as transparently as possible. That's exactly what we're trying to achieve, at the end of the day -- get back to where condor (another bird-named job system that we love) had us at 20 years ago.

Yes please elaborate, this is fascinating! Can you recommend any good undergraduate textbooks on the subject, especially for someone with a quantitative background (PhD in ML-adjacent field)?

I would recommend taking the "0.16 times per second" number with a real grain of salt. That neuron is evaluating inputs from (oh, say) 1k-10k sources simultaneously, computing some analog function that we don't really understand, which involves membrane dynamics, past spiking history, the amount of post-synaptic intracellular calcium, etc. These little units are absolutely incredible, and we don't quite know yet what parts of this behavior are actually the "secret sauce" that makes it work.

For me the most authoritative answer is what the Human Brain Project in europe is able to achieve: http://www.nature.com/news/fragment-of-rat-brain-simulated-i...

31k cells with 37M synapses. That's a start, but pretty far from a mouse brain (100M cells) or a human brain (80B? ).

Also if Moore's law really is dead, this all gets much more depressing.

I think that's totally correct! but I'm skeptical that a lot of the non-sensory neurons work on the basis of average activity -- rate codes versus timing codes and all of that. I spent a fair amount of time in the hippocampus where individual spikes. We were always going to run into challenges with how to interpret the voltage levels of the transistor, so we tried to be "fair" and not cheat on either side (we could have decoded the values on the memory bus or the adder numerically, for example).

For the next paper, though, we're taking a more careful look at a lot of these encoding questions, as well as trying to use processors with a greater degree of functional specialization so we can more accurately capture what's going on.

Awesome, do you have a good cite for that? The last time I paid attention in this space was a year+ ago, when the vision people were trumpeting these results, so I'd love to know more about the current thinking. It's been my go-to example for "connectomics will help with some things", but I'm not a sensory physiologist.

I totally agree with this. In spite of a lot of the recent hype, we still have very limited understanding of intelligence as a science. I agree "never" seems like a strong claim given that we're walking around with 7B existence proofs that it's possible, but for most of us "never" and 300 years are functionally equivalent.

Expansion microscopy is new and exciting, but also still in progress, and I think struggles to identify some of the traditional areas where you have really fine neuropil. I'm actually most excited about Tony Zador and George Church's DNA-barcode approaches. They will potentially give us different answers, though -- DNA barcodes will give us a graph and detailed information about synapse sites, but not a lot about morphology, for example.

I think many in the neuroscience community would agree that there's a lot we can learn from connectomes, and there's a lot of value from just having a canonical, authoritative map of the underlying neuroanatomy. Every year there are a few new neuroanatomy papers that say "surprise, area X projects to area Y!" and that's sort of embarrassing given that many people have been studying an area for 30 years and the sudden appearance of dopaminergic projections upsets all their previous models.

Also, there exist brain areas and regions where we do in fact have a few good good models, and connectomics has the potential to help us resolve them -- see http://www.nature.com/nature/journal/v500/n7461/full/nature1...

Also in the time-honored tradition of HN, the title isn't quite right. 1. we didn't build the connectome, the visual 6502 team did. 2. It's such a good connectome that we can simulate everything perfectly -- a Hansonian "Em", if you will. 3. It's still hard with total visibility to use contemporary techniques to achieve understanding.

Original author of the paper here, happy to answer questions. It's a little strange refreshing HN and discovering new popular press about your paper.

Author of the figures used in the blog post here. We wrote https://github.com/ericmjonas/pywren somewhat on a lark, because it seemed to fit well with our research goals and it's fun to push systems to their limit. I'm now a total serverless convert! I'd love more collaborators and feedback, the goal is to make these sorts of computations as easy as possible for python developers, especially on the scientific computing side of things.

I didn't realize this, and it's absolutely incredible. You guys are great!

Author here -- so actually we've done a fair bit of anatomical work recovering motifs, even while looking at the processor [1] -- we took that out of this paper based on several readers' recommendation that the content was "too new" and that people wouldn't understand it. That said, without the ability to then directly probe those specific circuits, it's quite hard to figure out what the motifs are doing, and motif finding in general is an incredibly challenging problem, especially at scale.

[1] http://ericjonas.com/pages/connectomics.html

It totally is! I read the original Lazebnik paper when I was young and it had quite an impact.

(Totally self-promoting here) I did a lot of work along this line in my PhD -- stochastic architectures for probabilstic computation. http://ericjonas.com/pages/circuits.html There's an increasing amount of interest in this space.

Note that SF already spends a large amount on the homeless http://www.sfchronicle.com/bayarea/article/S-F-spends-record... and that at least some politicians are starting to take notice http://ww2.kqed.org/news/2016/01/25/s-f-supervisor-scott-wie...

It's not clear what the solution is to this complex problem.

Things I love about academia:

1. incredible freedom to work on whatever I want 2. a real focus on prototypes and curiosity, with less pressure to "ship working artifacts" 3. explicit recognition that "learning new skills" is part of the job. When I was in industry, if I needed an antenna designed, I would just go hire an RF engineering firm. Now I can spend time hacking on it myself, which I really enjoy 4. Absence of corporate bullshit, which has traditionally included tremendous IT freedom, no centralized control of messaging / official PR line, etc. 5. Generally no IP bullshit unless you want it -- everyone loves open-source, etc. AMP is a real leader here.

Things that are a challenge:

1. project management is often lacking -- it can be hard to focus, and the freedom to explore is a double-edged sword of "it's been 18 months, what papers have we actually finished?" 2. The pay isn't great -- CS postdocs make $60-70k/year. Fortunately I can consult on the side to pad this out. 3. A lot of your inputs are 23-year-old graduate students, who are smarter than you, enthusiastic, and inexperienced. This can be a fantastic source of new creative ideas, but sometimes they also decide to switch from julia to rust 80% of the way through a project. Sometimes you wish you could just hire that RF engineer and get shit done (see above above)

tl;dr : I get to build the coolest shit with the most amazing people helping me and learn a ton of new things, in an environment which generally shares my values.

Where do you go?

I left industry to be a postdoc in the AMPLab and it's really depressing to see this sort of intrusive monitoring coming to the home of BSD and sockets.

One of the benefits of working in academia in CS has been the absence of top-down corporate IT control -- no MDM on you mobile devices, no third-parties having root on your devices, the ability to have a static IP, etc.

Those numbers are amazing, do you perhaps have a citation ? I'm a machine learning researcher and would love to include them in talks about data-aware algorithms.

I want to take issue with "it would take little effort to find 100s of projects at universities across the world that are better versions of these projects", both as a trained biologist (neurobiology), current postdoc, and former entrepreneur.

I share your intuition, I really do. I even said similar things in grad school (and still do, sometimes). But the reality is that:

1. These are not supposed to be projects, they are supposed to be companies creating products. The gulf between "Nature paper" and "something someone will buy" is incredibly long and arduous. Plus, these are VC-backed, meaning there's an expectation of timeline (short) and return (high). These aren't "take $20m from darpa and engage in another four years of R&D" type endeavors. So while they might look trivial scientifically (although I do not think that they do), they have to be working sooner rather than later.

2. "Better versions" is also a tough thing to assess. The people involved matter a lot -- thus even technically superior solutions from very smart people might still not be "better versions". I know many grad students who are much more knowledgable than some of the top data scientists, who nonetheless would fall on their faces if they tried to do a data science startup. Similar for life sciences and biotech.

I think the people who poo-poo academia are largely setting up strawmen -- even Peter Thiel (whose first VC firm, Founders Fund, backed both of my startups) admits that there's a substantial role for government and academic activity in enabling long-term technical innovation.

As biologists, we know just how hard it is to make _anything_ work in this space. Most new drugs fail, most clinical trials fail, most pathways end up being hard to target, etc. etc. Often building a company in the biotech space is like trying to build a microprocessor company where we're still not entirely sure "how the silicon works".

The challenge for YC and other VCs is to identify technologies that are "almost there", and help turn them into companies and products. That's very hard, and I'm not really even sure that we (as academics) are any good at it either.

Will some of them fail? Sure. Will some of them leave the startup scene and maybe go back and do a postdoc or work for an industrial research lab or do another startup, much wiser? Of course. But that hardly seems like a horrible outcome, especially if the scientists learn a lot along the way.

The Way Things Work by David Macaulay. Especially if we still had mammoths. http://www.amazon.com/The-Things-Work-David-Macaulay/dp/1405...

They don't really need a lot of domain knowledge, although having some technical background would be nice. Thanks for the pointer, I'll check them out!

In reading The Idea Factory[1] it became incredibly clear that Bell Labs only released and licensed a great deal of this technology as the results of various antitrust settlements that plagued the company throughout its entire existence. Also, part of the role of the labs appears to have been to give the company something to "show off" whenever congress or the DOJ complained about the extraction of monopoly rents. I highly recommend the book, it was really fascinating to see the degree to which many of our assumptions about the functioning of the labs and its relationship with the corporation are in fact historically inaccurate.

[1] http://www.amazon.com/The-Idea-Factory-American-Innovation/d...