obastani

I think the market prerequisites are a key point overlooked by the article. There has to be some application where a low-performing version of the invention is already useful for it to be worth it to develop better-performing ones. For example, in the case of Watt's steam engine, versions were already used for the application of pumping water out of mines [1]. This application made it economically worthwhile for Watt to work on his improved steam engine. Of course, once the improved version was available, it found many new applications beyond the original one.

[1] https://en.wikipedia.org/wiki/Watt_steam_engine

The first three can reasonably be thought of as defining linear transformations. For linear systems of equations A x = b, x is an unknown vector in the input space that is mapped by A to b.

Both covariance matrices and Hessians are more naturally thought of as tensors, not matrices (and therefore not linear transformations). That is, they take in two vectors as input and produce a single real number as output.

As for graph adjacency matrix, this can actually be thought of as a linear transformation on the vector space where the basis vectors correspond to nodes in the graph. Linear combinations of these basis vectors correspond to probability distributions over the graph (if properly normalized).

2D images... Yes, these cannot really be interpreted as linear transformations. But I'd say these aren't really matrices in the mathematical sense.

Part of the challenge with linear algebra is that a lot of the basics are somewhat dry, since they serve mostly as a way to organize computation -- e.g., a system of linear equations can be expressed as Ax <= b, where A is a matrix, b is a vector, and x is a vector of variables.

Much of what makes linear algebra interesting and powerful comes from more advanced topics, especially eigenvalues. This power comes when we are not looking at a single matrix in isolation, but when we repeatedly apply a matrix. For instance, consider the equation x_t = A^t x_0. It turns out we can rewrite this an equation by diagonalizing A -- i.e., A = P^{-1} Sigma P, where Sigma is diagonal; most, but not all, matrices can be written in this form. We call s_i the "eigenvalues" of A.

Then, the equation simplifies to x_t = P^{-1} Sigma^t P x_0, or equivalently (P x_t) = Sigma^t (P x_0). This equation is dramatically simpler, since Sigma is diagonal, so if Sigma = diag(s_1, ..., s_n), then Sigma^t = diag(s_1^t, ..., s_n^t). In other words, this transformation "disentangles" the different components of A into ones that act independently. Here, the transformation x -> P x is what is called a "change of basis".

These repeated matrix applications are common in physics, where they represent how a dynamical system evolves over time. The main difference is that in physics, the system evolves continuously, but similar transformations can be applied to solve these problems.

That might just be because didn't schedule risky operations on those days. It's hard to establish causation without a randomized trial.

That is not actually the way to define e^x for complex numbers. There are a few ways you can do so. The most natural way is as the unique solution to the differential equation

df/dx = f

The solutions are e^x + const; requiring f(0) = 1 gives you the choice const = 0.

Another way to define it is as the infinite series

e^x = \sum_{i=0}^inf x^i/i!

It is pretty easy to show this definition is equivalent to the previous one. A third way is as

lim_{n -> inf} (1 + x/n)^n

This definition comes from the intuition that e^x represents the limit of continually compounding interest. As before, it is pretty easy to show that it is equivalent to the previous two.

In any case, all these definitions extend directly to complex values of x. The fact that

e^{ix} = cos(x) + i sin(x)

holds (and hence Euler's identity holds) is a consequence of a more natural definition, not the typical definition.

That's not actually true. There exist functions of a single real variable that have derivatives at the origin that are all zero, yet are nonzero. For an example, see the following:

https://en.wikipedia.org/wiki/Taylor_series#Analytic_functio...

Nevertheless, it is actually true that all complex differentiable functions satisfy this property, which is miraculous.

This is a naive question, but would it also fix the problem if there was a video codec that could incrementally compress each frame? And if so, do you think such a codec is feasible to design? I don't know much about video codecs so this is just out of curiosity.

Based on my understanding, I don't think there is any particularly compelling evidence that there is an entirely new layer beneath what we know. But it's always a possibility. For example, it wouldn't be too surprising if modern physics breaks down beneath the Planck scale (equivalently, at very high energies) [1], or inside black holes.

In other words, I think both the possibility that we are nearing a complete understanding of the rules and the possibility that we are nowhere close are both quite plausible.

[1] https://en.m.wikipedia.org/wiki/Planck_length

I don't have real knowledge of gravity, but I can speculate a bit. First, I don't think the blackbox argument necessarily makes sense. If the black box contains a massive point particle, and this particle moves from the left side of the box to the right, then the gravitational properties certainly change. For example, if there is a particle on top of the box, it will initially be pulled to the left, and subsequently be pulled to the right.

Having said that, your intuition that photons should have gravitational influence makes sense, since according to special relativity, mass and energy are equivalent. Since total energy (mass + energy) is conserved, the gravitational influence of the photon should equal that of the electron-positron pair (assuming they are all located at the same position). I think the article is agreeing with this point of view.

The more common usage is 2a/b, as you point out. I believe "success" is specifically derived from this meaning of "succeed", not 1a. Typically, securing a position in a company is still in the sense of definition 2a/b.

As far as I know, the definition 1a is unrelated and specifically meant to describe sequence relations, usually for positions of power. For example, "Trump succeeded Obama as president" means Trump became president after Obama. Related words are "succession" (e.g., "succession of the throne" refers to the process of transferring power in a monarchy), "successor" (e.g., the successor function in mathematics takes an integer n and returns n+1), and "precede" (the antonym of "succeed").

One idea would be to let {0, 1} be the integers mod 2. Then, AND is multiplication (i.e., a AND b = a b, since a b = 1 iff a = b = 1), NOT is inversion (i.e., NOT a = 1 - a), and OR follows from the previous two definitions (i.e., a OR b = 1 - (1 - a) (1 - b) = a + b - a b).

To get complex values, we can consider the Gaussian integers [1] mod 2. Then, we have (1 + i) (1 + i) = (1 + i + i - 1) = 0 + 2 i = 0. So these integers are a ring where there are nonzero elements that multiply to 0.

[1] https://en.wikipedia.org/wiki/Gaussian_integer

The misconception is that p_1 × p_2 × ... × p_n + 1 has to be prime at all. It can very well be the case that p_1 × p_2 × ... × p_n + 1 is composite.

I'm not sure, and this is a lot of speculation, but I feel like it has to do with the fact that mass/energy is conserved over time (but not in space) [1]. The mass now corresponds to the mass in the past/future, which, for example, allows memories to form. In any case, time is still a qualitatively different dimension than space.

[1] https://en.m.wikipedia.org/wiki/Conservation_law#Exact_laws

That's what I meant -- thanks for the catch!

I'm not an expert, but I don't think that quote is quite right. There is still a notion of time, there is just no notion of a direction of time. For example, if you go deep into empty space, then one patch of space looks just like the next one. That doesn't mean there is no longer a physical concept of space. Similarly, just because one point in time doesn't look very different from the next, doesn't mean that there isn't a notion of time.

The main point of thermodynamic time is to describe the idea that time appears to be moving forward. According to fundamental physics, as far as we can tell, time is symmetric (with a few caveats). So going forward in time should look similar to going backwards in time, just like going to the left in deep space is the same as going to the right. But the fact that we are in a pocket of high entropy means that time has an apparent direction.

Waymo is almost certainly using a ton of computer vision in its cars. Vision is needed to classify objects on the road, predict the behaviors of pedestrians, bikers and other drivers, read road signs and traffic lights, etc.

As far as I can tell, LIDAR makes a lot of prediction tasks much easier (since you have both images and LIDAR point clouds), and is also a backup for obstacle detection to ensure that your self-driving car doesn't crash.

Entropy is causal of the arrow of time. In deep space, going forward on time doesn't look that different than going backward in time. Going faster forward in time also doesn't look that different than going slowly forward. The difference is in our perception of time, not in time itself.

Any chance you could you share some references on this kind of type inference? It sounds very interesting!

The GP is saying that Walmart avoided providing these benefits, instead paying it's employees so little money that they qualified for welfare, which is paid by th taxpayer, not Walmart. Since you are anyway subsidizing Walmart, might as well shop there.

Considering that Dragon is based on machine learning (hidden Markov models, or HMMs, to be exact) [1], I'm not sure what point you're trying to make. In any case, I used Dragon in the late 90's, and it was terrible, whereas Google's voice recognition on Pixel devices works great for me. One benefit of using deep learning is that they are much less brittle, e.g., working much better for people with accents, in noisy environments, etc.

[1] https://en.wikipedia.org/wiki/Dragon_NaturallySpeaking

Machine learning has been used for more than two decades in ATMs to perform optical character recognitions for automatic processing of checks [1]. I believe it is used by the post office as well. It is used by PayPal and other financial services companies for the purposes of fraud detection [2]. It is widely used to filter spam emails as well [3]. It is also widely used in recommender systems, starting with Netflix movie recommendations [4].

While currently more academic, it also has tremendous promise in healthcare diagnostic tasks [5,6]. It has also been used in economics applications such as measuring the global impact of fishing [7] and in poverty prediction in third world countries [8].

[1] http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.26....

[2] https://www.infoworld.com/article/2907877/how-paypal-reduces...

[3] https://en.wikipedia.org/wiki/Naive_Bayes_spam_filtering

[4] https://en.wikipedia.org/wiki/Netflix_Prize

[5] https://jamanetwork.com/journals/jama/fullarticle/2588763

[6] https://www.nature.com/articles/nature21056

[7] http://science.sciencemag.org/content/359/6378/904

[8] http://science.sciencemag.org/content/353/6301/790

I don't know myself, but here are some possible answers:

https://mathoverflow.net/questions/114034/learning-path-for-...

Edit: Here is another overview that seems good:

https://www.math.ucdavis.edu/~osserman/math/pcm.pdf

The "Background and history" section of the following article gives a very high-level idea of Grothendieck's contribution:

https://en.wikipedia.org/wiki/Weil_conjectures

It's not at all obvious that "if you don't train for it, it won't learn it". Humans do not at all learn in this way: we are very good at adapting existing knowledge to solve new tasks, and relatedly at learning new tasks from very little data. This challenge is a major issue with deep reinforcement learning (and maybe deep learning more broadly). It's unclear how we might surmount this problem, but I believe it'll involve some combination of model-based approaches and deep learning models that internally use more symbolic structures.

You can absolutely visualize why [1, 0] and [-1, 0] is not a basis for R^2! The problem is if you plot them in R^2, then they and the origin all lie on a single line. No matter how you sum copies of these vectors, you can never "escape" this line. Thus, you cannot construct vectors in R^2 that do not lie on this line.

So did a lot of rocket companies, until SpaceX came along and revolutionized the market. As far as I see it, Tesla is building a state-of-the-art driver assist product, but the approach they are taking is unlikely to reach full autonomy for a very, very long time. Waymo, while in a much higher risk position, has the chance (however large or small) to completely revolutionize the way we commute.

In principle I agree with what you are saying. We have to be a bit careful, however, since past financial history may have itself been biased by prejudice, e.g., if you were unfairly denied a job based on your race. Of course, we sometimes have to draw the line somewhere, and personally I agree that using previous financial behavior probably is fair.

This is what makes fairness so challenging. There isn't a clear place to draw the line, where certain covariates are permitted and others are not. Intuitively, what you are saying makes sense---your financial situation is a consequence of your choices, whereas things like zip codes are correlated both with your choices (i.e., I chose to live here) but also your race (e.g., due to lasting consequences of racial bias). Thus, we might decide that it's fine to use financial history, but not zip code.

Of course, things are not so clear cut. Your financial situation can also be a consequence of discrimination, e.g., if you were denied a job or unfairly arrested. At the end of the day, some human has to sit down and decide what is OK and what is not. Personally, I believe the goal of research on algorithmic fairness should be to give the people who will ultimately make these decisions (e.g., judges, politicians, etc.) the tools to understand both broadly, the kinds of things that can go wrong when using algorithms, and also to understand what might have gone wrong in a specific situation.

The problem you mentioned is not the one that is being tackled. The real challenge is that being blind to just race isn't enough. For example, zip code is often highly correlated with race, so using zip code to predict whether an applicant would repay a loan may implicitly be discriminatory. We need to know race to determine whether a given covariate is okay to use. But we would usually not use race to predict outcomes (except possibly in a limited number of affirmative action settings, such as college admissions).

In this case I don't think what the article says is exactly right. Naively, tensors are just n-dimensional arrays, which TensorFlow supports. The paper linked in the article appears more to be talking about how derivatives of tensors are represented in TensorFlow. The difference doesn't seem to matter unless you are taking higher-order derivatives. This makes sense, since TensorFlow is focused on first-order derivatives needed for gradient descent, but traditional machine learning algorithms also rely on second-order derivatives to make use of more powerful optimization algorithms based on Newton's method. I'm not sure exactly where the difference comes from, but it comes from a convenient notation for tensors used in physics, known as Einstein notation (Einstein invented this notation to make his life easier when deriving general relativity). In this notation, tensors are represented by a single scalar variable. For example, matrix multiplication y = A x is expressed as

y_i = A_ij x_j.

If I understand correctly, the paper points out that an algorithm for computing derivatives based on this notation is faster for taking higher-order derivatives compared to using TensorFlow.

Mathematically, tensors are more complicated objects. Basically, they are what you get when you take higher-order derivatives of a function. In particular, the first-order derivative of a function f: R^n -> R^m at a point x \in R^n is the best linear function A_x \in R^{m X n} that approximates the original function, i.e.,

f(x + dx) ~= f(x) + A_x dx.

A linear function is represented by a matrix, so a first-order derivative is a matrix. If I take the second-order derivative, I get a more complicated object B_x, which represents the quadratic term in the Taylor expansion:

f(x + dx) ~= f(x) + A_x dx + B_x(dx, dx)

where B_x(a, b) is a linear function (or more precisely, a "multilinear" function) of two vectors a, b (which are the same in the above formula). That is, whereas A_x is a (linear) function R^n -> R^m, B_x is a (multilinear) function R^n X R^n -> R^m. This mathematical object B_x is an example of a tensor. In R^n and R^m, tensors are pretty boring, but they become more interesting when dealing with functions on manifolds.