Do you have an estimate of energy used / total value secured ? If I am understanding correctly, an upper bound on "total value secured" would just be the total transaction volume moved in a block?
2,242 karma · joined February 8, 2015
I think it would be good for Fred or Coinbase comp team to provide the expected payoff calculations for various kinds of employees under this new scheme - the hard numbers and probabilities (even optimistic ones) would be what convince me.
Drawing samples from "simple" distributions like normal distributions is computationally easy. You can do it in microseconds. Sampling an arbitrary 1-D distribution is a bit harder - you have to invert its cumulative density function to recover probability under an "equivalent" uniform random variable, and potentially use a rejection sampling approach to sample 1-D values under this distribution.
Sampling a high-D distribution (such as an image) is even harder - you need to learn a mapping from this high-D image back to "tractable" densities. This imposes some pretty harsh "optimization bottlenecks" when trying to contort the manifold of images to normal distributions. The whole point of this exercise is that the transformation respects a valid probability distribution, so you can start from the normal distribution and apply this mapping to get a valid sample from the image distribution. This in practice is pretty hard, and the quality of samples seems to be lower than other forms of deep generative models which use less parameters.
Now instead of learning such a complex, hard-to-optimize transformation to valid densities, what if we instead learn a function E(x) that outputs a scalar "energy". The energy is low for "realistic" images, and high for unrealistic images. Kind of like inverse probability, except its not normalized - the energy value tells you nothing about likelihood unless you know the energy for all other images possible. This tends to actually be easier than learning densities, because the functional form of this energy function is unconstrained.
Furthermore, not knowing likelihoods doesn't stop you from getting "realistic" image, as all you need to do is descend the gradient x -= d/dx E(x), which takes you to an image with "lower" energy (i.e. more realistic). Under certain procedures (e.g. adding some noise to the gradient), this can be thought of as actually equivalent to sampling from a valid probability distribution, even though you can't compute its likelihood analytically.
The diffusion probabilistic model you refer to can be thought of as such a model - the more steps you take (i.e. the more compute you spend), the better the quality of the model.
GANs can be thought of as a one-pass neural network amortization of the end result of the diffusion process, but unlike MCMC methods, they cannot be "iteratively refined" with additional compute. Their sample quality is limited to whatever the generator spits out, even if you had additional compute.
def f(x):
for _ in range(3):
x = g(Wx + b)
return x
Transformers are a modification to that architecture - using different multiplications, additions, and non-linearities. both of these are general in the sense that if you have enough of them, they can approximate any function. The ones used for transformers empirically do well on a lot of machine learning problems, particularly where data has a sequential nature.
And vice versa, wolves also eat plants sometimes https://www.dogsnaturallymagazine.com/feed-your-dog-like-a-w....
I agree with Shkreli: if you want to "amass a fortune in a short amount of time" AND do so without taking on an unreasonable level of risk, then doing so with cheap cost-of-capital (e.g. investing friends&family wealth, using insurance float, gift cards) is the way to go.
But "making 10s of millions quickly" is a different statement than "active trading is not worth the effort".
I'm surprised the author focused singularly on the "funding the open web" stance, and did not discuss the alleged negative externalities created by advertising-driven business incentives. The top comments of the blog comments (e.g. https://www.jefftk.com/p/why-i-work-on-ads#lw-Nvd9ZHJLxZpNZD...) also point this out.
For instance, The Netflix documentary "The Social Dilemma" discusses these at length, and it forms a lot of the political zeitgeist around regulation of adtech companies (e.g. driving polarization, radicalization, depression and such).
> The question is, what is the alternative? I see two main funding models:
It's possible to simultaneously believe that ads are currently the effective way to lower barrier of entry to accessing content, and also believe that ads are a detriment to people's mental health.
One alternative would be an ad-supported system that performs less-personalized ad-targeting, and in doing so influences people's behavior less but generates enough revenue to keep the lights on with a reasonable profit margin. The government has laws for regulating natural monopolies like this, where they prevent utility companies from price gouging but grant them a certain amount of margin.
I think it's worth working out the math for yourself, rather than taking anyone's words for granted. Let's say the average annualized return of S&P500 is 10% (obviously, past returns are no guarantees of future returns). If you buy and hold, then you essentially pay no taxes on your capital gains.
Let's suppose the alternative is to actively trade, and your strategy involves holding securities < 1 year, incurring the maximum short term capital gains of 37%. Let's say your strategy generates an annualized average return of 20%, pre-tax.
So the annualized net worth (after capital gains taxes) starting from $100 would be:
passive: 100, 110, 121, 133
active: 100, 112, 126, 141
So you can generate 10% excess returns, after taxes you compound approximately 2% additional take-home money each year. Then you should ask yourself, is it worth the hassle? Maybe it is. Maybe it isn't. Some people enjoy researching companies, making predictions, and allocating capital to their beliefs.
obviously, there are ways to try and mitigate capital gains taxes - incorporating offshore LLCs, tax loss harvesting, etc. those are important considerations when designing an active trading strategy.
Separately from whether active investing is worth it or not, I think it's valuable to learn how to read financial reports and understand businesses as a projected set of cash flows, along with understanding things like balance sheets and incomes. Learning to read 10-Qs has helped me think of my own budget / personal finance planning much like a business. Financial literacy is super important, even if you are a passive investor.
here's the first one in the series: https://www.youtube.com/watch?v=VI_riscmviI
Just explaining why, I'm not making a normative claim here.
Let me try to poke some holes in the argument.
1. Market return (M) is a weighted combination of passive (P) and actively managed portfolio (A) returns:
M = (1-w)P+(w)A.
2. Passive investor achieves market return M by holding the whole market, so P = M.
3. To satisfy the equation, A must also be M, regardless of w.
The logical error is in the assumption that P = M. To make this a concrete programming problem: let's say at time t=1, immediately prior to the start of a trading period, a passive investor decides to construct a portfolio. The passive investor makes investment allocations based on the current set of market prices.
Subsequently, how can the passive investor possibly match market returns without w being 0, or the passive investor knowing exactly how the active investors will allocate their holdings? That information lies in the future - one only needs to go though the exercise of simulating market returns on a discrete time basis to realize that the passive investor cannot possibly allocate to achieve exactly market returns.
M[1] = (1-w[1])P[1]+w[1]A[1]
Some of the math surrounding the derivation of the weakest forms of EMT also relies on the assumption that everyone has access to the same information, which is patently false in the world we live in. Even retail traders sometimes have an information edge (e.g. working at a biotech company or a specialized industry like semiconductors)
In this sense, YC acts as an index fund on startups. Traditionally the IPO process weeds out the fraudulent businesses, and YC's application process is supposed to function the same way (with obviously less diligence) so that the efficiency of passive index returns aren't exploited.
In the limit, YC would be essentially investing in the equity of the totality of the younger technology workforce (who have majority of their earnings and talent and capital lying in the future). This is a good investment.
Where it differs slightly from a "indexing" model is that YC is allowed to encourage portfolio companies to buy and sell from each other, much like how Berkshire Hathaway does the same for its portfolio companies.
Training a GAN can be thought about as a proof-of-work type job, where you need to recover the best Generator from the SOTA Discriminator, or vice versa.
So the proof of work chain looks like D-G-D-G-D-G. As long as the good actors continue to dedicate more compute to tackling deepfakes than bad actors, I think it should help mitigate the issue.
As to whether biophysical learning has anything to teach us is an entirely different question which I don't discuss in the post.
STDP falls under biophysical models of neuron simulation, where we try to faithfully reproduce biophysics of brain simulation (trivia: I started my undergrad in computational neuroscience and implemented STDP several times [1, 2, 3]). STDP is a learning mechanism, but it has not demonstrated the ability to learn as powerful models as DNN.
[1] https://github.com/ericjang/pyN
Skilled engineers turn to cybercrime when white-hat bounties are insufficiently rewarding, so it is in everyone's interest to pay competitive rates for finding security vulnerabilities.
Thanks for enumerating the use cases. They are fun to think about. I wanted to dive into the first application you mentioned. As mentioned in my post, even the most basic manipulation tasks on human-centric objects and spaces are actually full of little details.
I hope people think more on that low-level dextrous manipulation when designing robot hardware rather than the fairly high level "open door, pull package inside, close the door", which might make robotics people receptive to the idea that a humanoid is truly the only viable solution.
What if your door has a step down to the porch where the package is delivered? The robot can't "just" pull a large box inside, it would get stuck. You'd need to lift it up, and at this point you'd require two arms or a gantry that can extend outside of the home above the porch. Obviously the home can be re-designed around this, but my point is that there are really two kinds of robotics - ones that try to solve a human problem, and ones that try to do everything a human can.