civility

Generics and operator traits ought to be orthogonal and composable. Would you still be comforted if the compiler didn't let you mix for loops and arithmetic in one function? I mean, even if it gave you a detailed error message...

I think he's saying one way to control misinformation is by applying censorship, with the obvious implication that censorship is undesirable. You take the good with the bad, whether that means freedom of speech and some misinformation, or "correct" information and censoring any other news or view points.

I know which I'm more afraid of...

I assumed you were contradicting this quote:

> This just means that the Rust language does not have the feature of being simple.

I really don't know what it means for a language to be sane, so it didn't make any sense to me that you were replying to that part. My bad.

I've used Rust. It's not simple. For instance, lifetime annotations are ugly and confusing.

What a snarky reply from someone who already knows that Rust is not simple. The person you replied to was right, and yet you cast doubt on what he guessed to be true.

Rust also has features which interact in subtle and surprising ways. Generics and operator traits bumping into the "coherence" rules is my personal pet peave. I doubt any user would have predicted that interaction, and even though the compiler gives a detailed pointer to why, it's cold comfort.

Honestly, after hating C++ for years, trying to use Rust made me appreciate more about C++. I wish someone would make a language which took the best parts of both.

They aren't my audience... they're my coworkers. Sometimes I get to pick how we do a project and sometimes I'm there to help them with their project. If they chose to use threads, I generally try to escape quickly.

No experience or comment on Python/Tornado. We don't really do a lot of web stuff.

But sure, people can screw things up in lots of ways. However, once a threaded program is screwed up, you really only fix it by starting a new version - it's near impossible to incrementally fix race conditions and dead locks - you can't reliably repeat the bug to debug it. Bugs in non-threaded code can at least be tracked down one by one.

Yes, small careful software teams can make threads work. However, if you start to work with physicists, mathematicians, electrical engineers, and so on who are incredibly smart in their own areas, but who don't have or even value a skill in software, you'll discover they make a real mess out of threaded programs in a way that doesn't happen with separate single-threaded processes.

I really wish XKCD would've gone back 400,000 years instead of only 20,000 for that strip. CO2 and temperature have been higher during the longer time frame, and it would be valuable for someone to explain the bigger picture adequately.

I do not have to recompile my C program every time I run it. Suggesting I'd want to include that is just silly.

I have the same complaint. I couldn't think of a polite way to put it, so this won't be polite, but whenever someone compares their algorithm with a poor version of a competing technique, it makes me question their integrity and results.

> Not all python programs are compatible with PyPy though.

I'm not sure how this is relevant to my reply... I didn't say PyPy was a replacement for Python. I said PyPy, Lua, and JavaScript are existence proofs that dynamic "hash table" languages don't have to be slow. Therefore, CPython must be slow for some other reason.

To each their own :-)

You might like Tcl - associative arrays (hash tables), arrays (lists), strings, numbers, and code are all the one type (at least conceptually).

> The reason Python is slow is not incompetent programming or lack of magic. We know why that's happening. Because every variable in Python interpreter is a hash map and pretty much every operation is a hash map lookup.

This statement is easily refuted by PyPy. Here's a simple program which runs 70 times faster in PyPy than Python on my machine - including startup time:

     https://pastebin.com/2n0PL9hY

> How do you optimize this?

Semantically, everything in JavaScript and Lua is also hash map lookups. Yet very smart people have made those languages very fast. CPython is not slow for the reasons you stated.

Other folks complain about the lack of libraries. Personally, the two things which turn me off on Lua (luajit) are 1-based arrays and the conflation of hash-tables and array-lists into a single thing.

> Your statement is a [sic] opinion.

Yeah, that's why put a smiley there. :-)

> Generally math formulae are one based, the fft and the Taylor expansions being the major exception.

What formulas do you regularly use which benefit from being 1-based? Note, I'm not asking for instances where the index doesn't really matter and the author of a paper simply chose base-1.

There are clearly cases where zero based is more natural. But I don't regularly use any cases where one based is an improvement.

I will admit that zero-based adds a lot of confusion when communicating to other people.

A bunch of wrongs don't make a right :-)

And honestly, it's because I do numerical programming that I value zero-based offsets. In addition to subscripting arrays (which I could do in any base), I use those subscripts in the math itself. For instance, the zeroth bin of an FFT indicates the zero frequency. I also choose the zeroth array element to represent the constant term (zeroth power) of a polynomial, and so on.

The common places where math notation uses 1-based subscripts (matrix notation) have more to do with people saying "first", "second", etc... With a few exceptions (the Hilbert matrix comes to mind), the base of the subscript isn't actually relevant to the math itself.

Heh, because 1-based arrays are gross. I'm poking fun of course, but you really shouldn't underestimate how many people are turned off of Julia by this.

> compiler optimization will rarely make last year's program faster, but it will make next year's program faster.

This doesn't follow. From your argument, last years and next years programs will run the same speed (about as fast as they can). It's just that next year's programs can be cleaner in some sense...

Which is interesting, because Dr. Proebsting's page also says his current interests include improving programmer productivity by removing syntactic baggage from statically typed languages.

Heh, I think we stepped in the wrong direction. I didn't mean to offend you.

Take care.

> It's not. You want to know the most probable value of A.

Nah, it's really not that simple. When I've done this in the past, I've needed both the mean (which is the mode for Normal distributions) and the variance, so I can make confidence ellipses. I don't just care about the most probable location.

I already have a set of techniques for working with Kalman filters. The only reason I would want to understand applying the Bayes' theorem in this context is if it offers insight into a wider class of problems (non-Gaussian PDFs) or if it helps me communicate with others. In both of those cases, I'd like to understand the thing first before I hand-waive the denominator away.

Yeah, the difference between lowercase p and capital P seems pretty important. Most places show capital P, so that's part of the confusion.

There's more though. Lowercase p(B|A) seems to really mean p_B(h(a)), and that's not obvious. Hell, I might still have it wrong.

And most everyone says to ignore p(B) in the denominator, but that's really sloppy hand waiving. The notation means something, and there should be a well defined set of substitutions, but in each of the four terms, they do something radically different. I can't see a pattern to follow.

Thank you for the reply. I spent most of the day talking to a coworker about this stuff, but I didn't see your message until just now.

I agree about the abuse of notation, but even with your improvements, there is still plenty to be confused about. In particular, the denominator has P_[B](b), which looks like a function from R^2->R (the PDF of B in measurement space). I guess it could also mean that PDF evaluated at the specific value of b. However, I think it's actual value is supposed to be "the integral from negative to positive infinity in all directions of whatever is in the numerator", which isn't at all obvious from the notation. However, maybe I've got that wrong.

Let's go one step further. On the left side we've got a function R^3->R with parameter 'a'. On the right, we've got two functions of 'b' (R^2->R), one in the numerator, and another in the denominator, presumably evaluated at the value 'b'. Both of those evaluations give scalars, and so it looks like we just get a scaled version of P_[A](a), which is definitely not the right answer.

An alternative is that we're not really evaluating those two functions at 'b', and instead leaving them as functions. If that's so, where did the free variable come from? It's not in the argument list on the left hand side. Although it is in the subscript on the left... Yeah, I'm still pretty confused :-)

> I agree that probability notation isn't the best [...]

There is so much hidden context in mapping Bayes' theorem to Kalman filters that I still have to cringe. First we've got the sub-expression "p(x_k)" in the numerator, and you know that is a multivariate PDF for the state estimate, or possibly the density value of that PDF at x_k. However, we've got a very similar looking expression "p(z_k)" in the denominator, and in this case it means "an integral of the product of two PDFs". Those are wildly different substitutions for nearly the same syntax.

That same confusion applies to p(x_k|z_k) and p(z_k|x_k). The first seems to indicate "the PDF of my estimate given this measurement", but the second really says something like "evaluate the PDF in the measurement space at the location which corresponds to my estimate". Essentially, p_Z(h(x_k)).

If you don't already know the solution, this notation is not prescriptive for getting there.

Thank you for your reply here, and the one below. I wish I had seen it before posting my sibling message to this one. I'm a bit too tired to go any further with this tonight, but I plan to look at your links tomorrow.

Cheers.

In term 1, you say P(z_k | x_k) is a probability (which should be a scalar value between 0.0 and 1.0 inclusive). How did we get from the vector pdfs of measurement and estimate to a scalar probability? Maybe you meant evaluating a pdf at an arbitrary point z_k, getting its density (scalar non-negative value, between 0.0 and infinity) at that point? I have the same confusion on your other enumerated terms. If they are probabilities, I'm only familiar with getting those by integrating the pdfs over some set of bounds or range.

I've seen the k subscripts plenty before, and some people use + and - to indicate pre and post parts of an operation, others put conditional vertical bars in the subscripts, but your definition for the pdf of z_k is a bit different than I'm used to. If we focus on the current instant of time (dropping the subscripts), ignore the motion model part, and only deal with a single measurement, I would say the estimate and measurement are defined like this:

    estimate    ~ Normal(mean=x, covar=P)
    measurement ~ Normal(mean=z, covar=R)

Then there is an elegant formula looking at it as an information filter, using Matlab-ish notation:

    Y = inv(P)    +  H'*inv(R)*H
    y = inv(P)*x  +  H'*inv(R)*z

    update ~ Normal(mean=inv(Y)*y, covar=inv(Y))

Ignoring the information filter stuff, and getting back to your list, Term 1 looks like a pdf with the dimensionality of z, but the result in Term 4 is a pdf with the dimensionality of x. In otherwords, the resultant Term 4 is a pdf (function) which takes a vector argument in the x space, and I don't see where to pick any given z for the pdf in Term 1.

It's getting late, and I'm getting spacey, but it's tempting to try and write the type signatures of these functions in some strongly typed pseudocode so I can show you my confusion.

> I'm not able to understand why any of that is bothersome.

Ok. It's bothersome to me because Bayes gets trotted out in the context of Kalman filters as though it is shedding light on the topic or providing rigor to the approach, but I haven't found anyone who can really describe what the elements of that expression are without hand waiving.

You've switched from capital 'P' to lowercase 'p' in your message (which I take to mean Probability and pdf respectively). If that was intentional, it opens a new batch of questions for what the notation means.

Anyways, maybe I'm just a slow learner, too pedantic, or something else. If you're interested, read my other reply where I go into it more.

I appreciate the kind reply, but I don't think you understood the gist of my complaint. I have a degree (just undergrad) in math, and I've implemented Kalman filters, Kalman smoothers, information filters, particle filters and so on at least a dozen times. I know what operations to perform, and I even have an intuition about why they work.

When I complain about the Bayes theorem way of describing or thinking about Kalman filters, I really mean that I don't understand what almost anything in that expression means. The notation is intuitive if read as English, but opaque to what operations are being performed on what mathematical objects. Capital P means probability of an event or predicate, and A is some abstraction of the thing I'm trying to estimate, and B is the abstraction of a new observation or measurement.

Think of it this way: I have some knowledge of my prior estimate, and I'm able to characterize it as a Normally distributed random variable. It has a mean and a covariance in 3D space (let's call those x and P). There is a multivariate function for that pdf (lowercase p); it's an exponential of the negative square of the Mahalanobis distance normalized to have a unit volume. The same thing applies for my measurement, except it's only a 2D mean and covariance (let's call those z and R). I already know that I can use my H matrix, some matrix inverses and multiplications to combine this information. It's really just a case of multiplying the pdf for the state estimate and the measurement and re-normalizing to a unit volume. In other words, I believe both things are true, and they are independent, so I can multiply their pdfs and re-normalize to get a combined result. The rest is just linear algebra.

Now let's get to Bayes. P(A) is supposed to be the probability of an "event" or predicate. I can squint sideways and translate that as integrate my pdf for the prior estimate (Normal with mean x and covariance P) over a some unspecified bounds and convert that to a probability of my estimate being in those bounds, but I'm not sure that's what is intended. Again a similar thing applies for B (Normal with mean z and covariance R). It's frustrating that the bounds are never stated, because they could be radically different spaces in the numerator and denominator. I guess I'll give that a pass because maybe the notation would be too cumbersome if it was included, but this simplification seems to never be stated in any of the books or papers I've read on it.

Next, P(B|A) has to be probability of another "event", and if I squint again, the best I'm able to come up with is that it means a new Normal pdf with mean = H'z and covariance = H'RH. However, I haven't seen that spelled out any where, and so really that's just assuming the conclusion I want, which is super questionable. It also doesn't help me understand the left side of the Bayes equation - that vertical bar there seems to mean something different. When I look to the definition of conditional probability, I don't see anything about the vertical bar applied to multivariate Normal distributions.

If you translate it all to English, it reads as a coherent sentence, and that's fine. This is the "prior state estimate", that's the "observation", and this other thing is that's the "a posteriori" etc... However, if Bayes really helps with the understanding the math, the vertical bar has to mean something specific as an operator, and one would hope it meant the same thing on the left and right sides of the equation.

Similarly, in order for all of those P( ... ) to become scalars so multiplication and division are well defined, you need some integration bounds to turn the event into a probability. But at this point, I'm not even sure if that's what is intended by the notation - maybe they aren't scalars, and multiplication and division mean something radically different here. I honestly don't know.

> It's just Bayesian inference to update the posterior.

I've had this discussion with a number of smart people (math PhDs and so on) over the years, and I've never gotten a good explanation. I understand in the very abstract sense how Kalman filters (and particle filters and so on) can be seen as Bayesian, but when it gets to actually applying Bayes' theorem it really gets complicated very fast. For instance, if you're estimating a 3D position (that will be A) and your measurements are some 2D subspace (these will be B), the least squares approach works nicely (normalized products of Normal PDFs).

However, I'm not sure I understand a single component of the expression P(A|B) = P(B|A) P(A) / P(B) in this context. What does A|B or B|A even mean when A is a 3D Gaussian and B is 2D? What are the dimensionalities of these operations? What does it mean to divide a product of pdfs?

I don't expect an answer, but I always wince a bit when I see Bayes in this context. For Normal distributions, I prefer thinking of it as least squares.

On top of downvotes, you can say very toxic / abusive / condescending things and get away with it if you share the "correct" viewpoint, but unpopular views have to be exceptionally polite to avoid biased moderation. You can't bluntly refute or critique a questionable (but popular) argument without being accused of lacking civility...