One of my first memorable PhD experiences was getting invited to an impromptu meeting in Lithuania, where we met a Finish and Lithuanian scientist both working on astrocyte models and wondering how that could be applicable to neuromorphic hardware. Needless to say this didn't go anywhere, but at least we got to sample some really nice Lithuanian food.
In academia there always is a difference between the way results are advertised and what conclusions are drawn internally. This is more true in some fields than others, I'm most familiar with it ML, Physics. Part of your skill as a researcher is to understand based on omissions, the datasets etc. the quiet part that isn't said out loud. Depending how you sell things you can get a Nature / Science paper with confusing inconsistent terminology, hand rolled C++ implementation, provided you are the first and a another method which might be 1000x times faster will only make it into PRL (yes I'm thinking of two specific papers, but won't say which).
I think there is an expert vacuum that was filled by opportunists and enabled politicians to decide non-sensical measures. Compare that to the situation in Taiwan, where the vice-prime Minister is an epidemiologist.
It tells you a lot of the level of expertise in a country, if a physicist like Viola Priesemann previously not involved in any kind of disease modelling is able to be a significant voice in the national conversation. The 'experts' that are listened to are either numerically illiterate like Drosten or non-experts in disease control like Priesemann. Moreover the actual tools and quality of code that is used to model the situation are so far away from what would be ideal it is laughable.
As a working physicist I would agree with you for the most part. Science fiction doesn't help because it doesn't contain ideas, which work beyond a surface plausibility level. However you can still get inspired by them in indirect ways. I find Paul Feyerabend's work interesting in that regard. In "Against Method" he argues that most breakthrough scientific discoveries can't be cleanly attributed to the scientific method.
It seems obvious that USAID is an intelligence front (I've encountered a few instances where it was mentioned that someone worked for USAID at the time, while it was simultaneously obvious that it would make way more sense if they were Intelligence), but is there any concrete evidence for that?
At least in Germany (where I am from), but I believe also other places in Europe, it is pretty common to do a PhD that is partially or fully funded by some company. For example Apple, Bosch was looking for ML PhD students at my university. Basically many larger companies, which might not come immediately to mind like Roche, Bosch, Merck, BMW, obscure Sawmill manufacturers, have some interest in Machine Learning. Only for a subset of them I would really call this research and as I said in most cases you either need to be exceptionally talented with connections or you are probably better off having a PhD.
Realistically I think really high profile research labs will want you to have some published work or for whatever reason know that you are a really talented programmer and will hire you as a scientific programmer. I know at least one person that went to an industry research lab after a masters degree, but he had the necessary connections to there and the appropriate exceptional programming skills. Alternatively he could also consider doing an industry PhD, there should be lots of potential opportunities for that.
Yeah it also lead to trillions in wealth increase of the ultra rich. There also is a difference between deliberately investing in vaccine companies and then publicly advocation for patent protections to remain in place and buying a vanguard etf. Just because someone is the target of Crazy conspiracies doesn’t make their behavior less reprehensible.
The (western) COVID vaccine manufacturers still make ~1000$/s in profit from them. Gates / BMGF are invested in literally almost all of them, even the obscure Indian ones, not just Moderna and BionTech
most of that processing power is unfortunately special purpose and can't be easily repurposed for something else. We have very good vision processing, but it is not like you can entirely high-jack that for something else.
During my PhD I burned through ~10^5 of compute hours on 4-8 V100 GPU, 128 Core, 1TB+ HPC nodes. Your 10k AWS credits run out really quickly compared to that. And as a startup you are far better of spending some money on a few beefy machines before renting them.
I would say I'm relatively pragmatic. My main use case would involve DifferentialEquations.jl modelling either certain quantum processes or model equations of biological neurons. I have a rough idea how that would work and what kind of abstractions I would like to implement. I mainly have not been able to get a good developer experience with it, so far. Things tend to either work or fail rather drastically (spectacular long back traces). But it clearly is very useful, so I will probably bite the bullet and try to use it again.
Ha, yeah I knew that analysis of how to do second order adjoints fast :), in what is basically documentation no less. I figured someone would probably do a ML paper on that eventually. I have had a hard time judging, what is non-trivial in the past, aswell. The next thing that they will probably discover is Jet spaces and Hopf algebras.
I'm also slightly salty because a recent paper (https://arxiv.org/pdf/2011.03902.pdf) claimed "Differentiable event handling generalizes many numerical methods that often have specialized methods for gradient computation,...", citing one of our results. We instead went through the trouble to find a semi-complete list of prior work, which would make it abundantly clear that what they claim as new is in fact traceable to a paper by Rozenvasser in ~1965 (we translated it from Russian to make sure), with a long history of more recent work. In fact I think DifferentialEquations.jl just supports this out of the box and has non-broken event-handling (their implementation tests against a product of jump conditions).
I am aware of those benchmarks :). I will probably adopt your library for this very reason, although I have to say that I really don't like Julia as a language. It is pretty clear that besides Sundials, PETSC and other C++ based libraries, you are rapidly becoming the only game in town. As much as I am tempted I really shouldn't be handwriting integration routines :). I don't think you need a random internet stranger to tell you that, but it is abundantly clear that DifferentialEquations.jl provides far more long-term value than these types of machine learning papers.
The pytorch lightning people have come up with grid.ai, I personally have obtained good results by using pytorch lightning plus slurm on HPC machines. If I were a startup, I would probably try to build my own small HPC cluster, since that is far more cost effective than renting.
I agree that it is a good name and evaluating this on "ML tasks" was novel. However this and subsequent papers did a really poor job in delineating what is novel, from what is well known. Moreover this is a pattern in almost all subsequent papers, where people literally pretend that they were the first to consider parameter gradient computation of controlled differential equations, hybrid dynamical systems, etc., while in fact this has been worked out in full generality since essentially the 60s.
Neural ODEs are essentially a rebranding of adjoint sensitivity analysis, which has been around in various forms in established solver suites, such as Sundials, PETSC, etc. The machine learning community got a hold of it, cited one book and otherwise happily reinvented everything.
I think it would have helped if the people writing that paper had not confused the issue by introducing a new name for something that is a well known thing in optimal control and had been invented even before neural networks, namely adjoint sensitivity analysis. There even appear multilayer networks of switching components in Pontryagin's book on the subject.
I have a PhD in physics and studied graduate level mathematics. Naive set theory is sufficient for basically everything except a few instances, where you have to bite the bullet and invoke the axiom of choice. But this happens rarely and typically is only relevant for instances where there are more constructive but less general approaches.
For the most part sets are used as a notational device, you almost never use any axioms about sets explicitly. If you asked most professional mathematicians to give you the ZFC axioms (or whatever) they would need to look them up.
I think shader specialisation is handled pretty well in circle. Since you can essentially run arbitrary C++ code at compile time, selection and specialisation of a shader can even depend on hardware specific benchmarks. There is an extensive repo with examples here: https://github.com/seanbaxter/shaders. One example decodes a sprite sheet stored as a png at compile time and creates a specialised compute shader for it. You can also easily implement a control UI based on reflection of uniform shader parameters.
Anyways I understand that under different constraints (adherence to C++ standard, can't easily implement a whole C++ frontend and interpreter) this becomes much harder.
I had this thought that it might be possible to define some of the social media interfaces in terms of CapnProto services. Then it wouldn’t matter where the service was hosted. Especially for things like LinkedIn the major draw is that CV data is made available in a convenient and canonical form.
I wasn't surprised at all as a ten year old who got a recommendation for gymnasium and who didn't. It is a recommendation by the way, parents can chose to enter their kids in whatever they wish to. This leads to kids with high income / social economic status entering gymnasium and struggling mightily and kids from worker families who might make it not going to gymnasium.