"Analysis".
There is a difference between meaningful analysis which answers long standing questions of biology, and running a program which calls SNPs and spits out genome stats.
The hard part is knowing what questions to ask, and how to interpret the answers when you get them. That takes years of training to develop a critical mind.
There are tools with GUI's but they don't help that much if you don't understand the underlying systems.
EDIT: I want to clarify my mindset. In the environment that I work in there are three stages of analysis: primary, secondary, and tertiary. We mostly ignore the tertiary stage (I'm not actually doing science). Primary analysis is on instrument (base calling), secondary analysis takes you through variant calls and annotation, and tertiary analysis is where the real science happens. When I'm talking about canned analyses I'm talking about what I refer to as secondary analysis. Tertiary analysis is a very hard problem and will often require custom solutions. In my mind I'm separating bioinformatic analysis (secondary analysis) with interpretation (tertiary analysis).
Genomics England is brilliant because it has good access to excellent NHS data which it will use to do a lot of analysis. Which its not getting off the shelf but a lot of talented clincal+bio+informaticians are working hard on that.
Or in other words getting the variants to the human genome reference is just the start of the analysis not the end.
i.e. I am more worried about the end of the pipeline than the contents of them.
I suspected as much on the 1,2,3 level. But even 2 is not really fixed yet. There are great off the shelf tools but even the concept of variation calling against a reference linear genome is not perfect yet especially with the alt_loci business. Which driving the variation graph work in the GA4GH projects.
If you could message me at hngenometemp@forward.cat that would be terrific!
It takes a 100mbit stream/$1M of sequencing capital, so network connectivity to transfer to a data center is a tiny tiny cost of the whole ordeal.
However, paying for AWS storage is pretty prohibitive, unless you're at a small scale. So big centers will build their own storage facilities.
The small data producers like the ones that the thread author talks about can use often use AWS more cost efficiently than building a compute cluster. However, they need to budget for that, which is not always thought of. They may also need to fight their institute's core center so that they can use DNANexus.
For academic centers though there is often an incentive to move things in house due to different treatment for capital expenditures and the opportunity to externalize some of your costs from your grant onto central services.
Keeping this data for less than 5-10 years is pretty questionable, since it's so expensive to generate. Eventually it may be cheaper to store the DNA and resequence when if it needs to be looked at again. However, if you're doing petabytes of storage, it's going to me much more economical to have your own storage and compute than to use AWS. Particularly at the rate that academic centers pay for sysadmins.
Remember to account for future reductions in storage costs. S3 has come down from $0.1500/GB month in 2010 to $0.0300/GB month today. And the recently introduced infrequent access storage tier is under half that again at $0.0125/GB month. It's now significantly cheaper to use S3/Azure/Google than running the storage ourselves.