mattb314

Wait, Fauna uses the Calvin protocol? Would you mind linking a white paper? I didn't realize it was in use outside Calvin/CalvinFS.

It looks correct to me, but keep in mind that deleting elements from the middle of an array is O(n), so overall your algorithm is O(n^2). You could fix this by always swapping your chosen element with the last element of the sorted_deck before removing it because removing from the end of an array is O(1).

"As such, R^2 answers the question: 'does my model perform better than a constant model?' But we often would like to answer a very different question: 'does my model perform worse than the true model?' "

Maybe I'm over-generalizing here, but I think this fundamental assumption is untrue for most people who use statistical models to solve actual problems. All of the "undesirable" behavior of the R^2 metric makes complete sense when you view it as a comparison with the most naive model (the constant model), and while R^2 certainly doesn't measure how close to "true" a model is, it very well captures the utility of using a more sophisticated model over an extremely simple one, which (I believe) is a critical question.

For example, if you had predict a process where measurement noise overwhelms variation in the process itself, as in his first example of log(x) from .99 to 1.0, then the constant model is pretty much the best you can do, and both log(x) itself and the linear model offer little additional benefit. Getting low R^2 values for those two models makes total sense--they offer no marginal benefit. In fact, if you had to make the decision "should I use log(x) or the constant model?", you're often better off going with the constant model for simplicity and predictability (unless you have domain knowledge motivating a different choice).

I like well thought-out articles like this one on statistical concepts because they make me think about things I often take for granted, and while treating R^2 like a better version of RMSE or MAD is clearly wrong, it often better captures things people actually care about by taking the difficulty of the problem into account. If you're doing advanced statistical modeling, it's easy to underestimate how common it is for a beginner to celebrate getting (say) 96% classification accuracy on a 2-class problem where one class makes up more than 95% of the samples--an issue that using R^2 can quickly reveal.

tldr: Awesome article but (imo) R^2 is more useful than most other metrics, not less

Your linked blog post was a really good read, but I'm not totally sure if the correct takeaway was that brute force outperforms "clever" algorithms. Unless I misinterpreted the results, FLANN and Annoy (which implement hierarchical k-means and a binary space partitioning algorithm, respectively) both ran about 100x faster than brute force.

Were you referring to the number of libraries the author couldn't get to work? This certainly might be a failure of over complicated algos, but it seemed more like an engineering failure than a performance failure to me.

To be fair, as Lxr points out below, big O notation is an upper bound on the growth rate of a function, and since 2^N grows faster (asymptotically) than (1.618...)^N, O(Fib(N)) <= O(1.618...^N) <= O(2^N) (with some abuse of notation), and the article is technically correct. You may be thinking of big theta notation, which requires both an upper bound and lower bound.

This example is helpful: https://en.wikipedia.org/wiki/Big_O_notation#Use_in_computer...

Worth noting that (a) this is an analysis of scientific consensus, so environmental effects (like the "arctic methane positive feedback loop") should already be included in the numbers if such effects are significant, and (b) GP was referring 1-3 feet in the next century, which matches your quoted numbers of 70-120 cm reasonably closely.

Still, interesting source.

The accompanying paper[1] claims the phone lasts four hours running the current version of fingerIO, but also that improvements could be made to preserve power (such as reducing the sampling rate).

[1] http://fingerio.cs.washington.edu/fingerio.pdf

Probably the second example here [1] for those curious (I think the integral of sin(x)*e^x dx is the only place I've seen this used, would love to know if there are other examples).

[1] https://en.m.wikipedia.org/wiki/Integration_by_parts#Tabular...

The linked article is a little over-hyped, but the paper authors themselves acknowledge that their system is simply a really cool example of parallel computing and doesn't solve or avoid the P?=NP problem. From the paper:

"...it is inherent to combinatorial and NP-complete problems (assuming P! = NP) that the exploration of the entire solution space requires the use of exponentially increasing amounts of some resource, such as time, space, or material. In the present case this fundamental requirement manifests itself in the number of agents needed, which grows exponentially with 2N. Effectively we are trading the need of time for the need of molecular mass."

They could have implemented many different paralellizable algorithms, so the choice to solve a case of subset sum was probably for PR/the awesome factor (not to be underestimated as an academic motivation).

If you are referring to the law invoked in the Apple case (the All Writs Act), a company unable to comply does not face legal consequences. From Wikipedia: "In the case U.S. v. New York Telephone Co. 434 U.S. 159 (1977), the Supreme Court established a three-factor test for the admissible application of the All Writs Act: the party ordered to perform an action cannot be too far removed from the case, the government's request cannot impose an undue burden on that party, and the party's assistance is necessary."[1]

I'm not a lawyer (and law can be tricky), but forcing a company to do something impossible (breaking into a truly secure device) seems very clearly to be a "undue burden". The real issue here is that Apple has created a device that is possible for them to backdoor themselves, and the clear solution is to remove that capability from future devices.

[1] https://en.wikipedia.org/wiki/All_Writs_Act#Conditions_for_u...

I think the argument here is that stealing a private key isn't fundamentally more difficult than stealing this iOS backdoor (were Apple to create it). Under this model, Apple's refusal to create a backdoor is in some sense an appeal to security through obscurity because it implies that engineering knowledge is the main obstacle to adversaries, whereas the true obstacle is the ability to steal protected information from Apple (the key or the backdoor). If the key and backdoor were equivalently useful, the existence of a backdoor wouldn't impact the safety of customers because the key already exists.

Of course, whether stealing a backdoor is actually as hard as stealing a key is a legitimate question, but (I thought) Apple had the option to unlock the phone in-house, which would at least keep the backdoor out of FBI hands (the legal precedent, however, could still pose a real threat to user security).