The first work, I linked, arguably gets around this issue completely. There we train a neural network to approximate Bayesian prediction directly. It accepts the dataset as input. Thus, there is no optimisation procedure per dataset, just a forward pass. So far, this seems to be pretty stable.
Our lab works on changing this. I think it might still take some years for a full solution, but so far we are successful with NNs for small datasets with meta-learning (https://arxiv.org/abs/2207.01848) and large datasets with regularisation (https://arxiv.org/abs/2106.11189). The first is very new, but the second is also cited in this paper, but they didn’t run it as a baseline.
From working in a company providing neural network inference as a service, I can attest you, that we did this. We did it especially since we are scared people distill on our results. If the other service makes the same weird mistake, they distilled from us.
The real comparison would have been between Cifar-10 (or Imagenet) and a downsampled version there of, not upsampled, and I actually know for sure that this harms performance.. So this ideal world and real world training are definitely not the same!
I really like this critique of environmentalism, as an environmentalist myself. While I think it should have ended with an argument in favour of some sustainability law, like a carbon dioxide tax, it clearly remembers us that still we are so much stuck in ideologies when it comes to saving co2 and that this could potentially be, because it makes us feel better than really saving co2.
Interesting idea! Future Me: A computer became better at my job, but I might still claim that it is not generally intelligent because it does not have the same kind of relationships to humans I have.
I would really like a thorough analysis on how expensive it is to multiply large matrices, which is the most expensive part of a transformer training for example according to the profiler. Is there some Moore’s law or similar trend?
I wonder if Zeit, people with machine learning background, could do to help given open data. Is there some open source project or so to contribute to that does something helpful e.g. predicting cases or areas at risk?
I guess the difference here is that a decentralised order book on the block chain would be run by every node, which is why compute in the blockchain (gas) is quite expensive.
It changes the value fluctuations. Right now the biggest problem with using crypto as your day to day money is that you cannot trust the money that you put into crypto to stay at the same value over time + transactions are slow.
What these attacks are about is stealing money, right? Is that your plan? Why do you want donations to steal something or if you don’t steal anything how will you change the chain?
Why isn’t it actually possible to just encrypt saved emails on server? So that government does not have access. Couldn’t one use a hash of the password as key for the data for the data and not save that hash to check password but another one. This way (practically), at least if the password is not eavesdropped and saved by the mail provider, it would be much harder to give away emails.
You don’t need a reconnect for every invocation actually, only per instance. I know you don’t want to think about instances with lambda, but actually they just run on some VM and you can just open one connection per VM.
(i) I think as pointed out in the last paragraph, the state is as guilty as Amazon in letting something like this happen. If Amazon would not bring workers to the legal/market limit. There would be another company to pick at here.
(ii) No corporation cares about being ethical correct. Companies only purpose is to generate (more) profits and this can be even more easily (without moral burdens) achieved if (like in big corporations) there are a lot of layers of abstraction between the decision makers view and the workers view.
(iii) I think if Amazon would be a nice company something like this would not happen.