11,905 karma · joined May 21, 2009
So, people already knew how many things they had gotten right and wrong when making their judgement. They just judged that most other people had also gotten a similar number wrong.
Developing high-quality software, which runs on as many devices and is compatible with as much content as Firefox is is a hell of a lot of work. And Mozilla isn't just working on Firefox; there's Thunderbird, though that's in the process of being spun off, Firefox OS (also in the process of pivoting), Mozilla Research (Rust, Servo, Emscripten, asm.js, and more). There are continuous integration systems to test all of this against dozens of combinations of platforms and versions, and huge numbers of tests on each (https://treeherder.mozilla.org/, http://buildbot.rust-lang.org/waterfall, http://build.servo.org/console), all of which needs to be maintained and triaged. There's standards work, to negotiate new standards in standards bodies with users, other browser vendors, and the like. There's advocacy work, of times when you find bugs that are simply due to broken websites that happen to be relying on other browser's bugs, to reach out to developers and try to get them to change in a more compatible or standards compliant manner.
$385,000 is, what, the yearly salary, benefits, and overhead of maybe two full time engineers in SFO? Complaining about $385,000 given to deserving projects, given the amount of investment and energy Mozilla is actually putting into Firefox and other substantial, valuable projects, is a bit of a stretch.
I personally eat a low-meat diet (no more than 3 servings of meat a week), but don't generally feel the need to eat any kind of meat-substitutes; there are plenty of delicious, already vegetarian or already vegan foods. It does help eating only vegetarian and not fully vegan; cheese and eggs are good sources of both flavor and protein, and make a good "center" of a dish or "main course" of a meal that is sometimes hard to do with purely vegan food without substitutes, but I've also had plenty of delicious fully vegan meals without meat substitutes.
And even when getting closer to meat base dishes, I'm personally more likely to order a "black bean burger" or "chickpea burger" than a generic "veggie burger"; it helps to know what I'm getting, while a lot of the meat substitutes are complicated and processed. And for those processed items that do more often act as a stand in for meat, I generally prefer the more minimally processed ones like seitan, tempeh, or tofu, rather than a "tofu burger" that has a whole bunch of other things in it to make it more similar to a hamburger.
About
Space Mono is an original fixed-width type family designed by Colophon
Foundry for Google Design. It supports a Latin Extended glyph set,
enabling typesetting for English and other Western European languages.
Developed for editorial use in headline and display typography, the
letterforms infuse a geometric foundation and grotesque details with
qualities often found in headline typefaces of the 1960s (See:
Microgramma[1], Eurostile[2]), many of which have since been co-opted by
science fiction films, television, and literature.
Typographic features include old-style figures, superscript and subscript
numerals, fractions, center-height and cap-height currency symbols,
directional arrows, and multiple stylistic alternates.
[1]: https://en.wikipedia.org/wiki/Microgramma_(typeface)
[2]: https://en.wikipedia.org/wiki/EurostileHowever, I have been discouraged from using the site in recent years due to too many disagreements over closing and downvoting questions when actually just getting a little bit more clarification from the person asking the question would have helped a lot more.
I've answered over 1,000 questions, ranging from very simple beginner level to some things that took a good deal of research or debugging to answer. I've seen my fair share of poor questions. If a question is completely off topic, then sure, I click the "close" button immediately.
However, if it's on topic, and about some identifiable issue, but not specific enough, too poorly worded, or doesn't include a code sample, or the like, then I post a comment requesting clarification, and listing how they can improve their question. In some cases, they actually work with me, improve the question, and then I or someone else can answer it. In some cases, they just ignore my comments or continue with requests to just give them the code or the like, in which at that point I find it worth a downvote or close vote.
But a lot of the times, I've seen perfectly good questions that were just a little confusingly worded, however I was able to tease out what they were getting at, I've done some research to find the answer, and come back, only to find the questions closed. Now I have an answer for the question, but can't actually post it. If I vote to reopen, sometimes even after editing it to clarify or getting the OP to edit it, it will generally languish with one or two reopen votes. Sometimes I'll then post on meta to complain about this behavior; people being way to quick to close. This will frequently lead to enough people clicking through to reopen it, but also a torrent of complaints about how the question really is bad and how we shouldn't let this kind of stuff on the site because it's destroying the community.
The thing is, bad questions don't destroy the community. If they're bad questions, they'll generally fall off the front page and get forgotten about. They don't really cause much in the way of problems; they're just a few extra bits. But this hostile behavior does destroy the community. It pushes beginners away, who may ask better questions later once they get a bit more of a handle on what they're doing. And it pushed people like me away; people who are there to help, and willing to do so even for beginners.
I think that Stack Overflow is a very different thing than MathOverflow. Professional programmers sometimes run into thing that they need to do that they are complete beginners at; you may have years of experience writing Java, but then you wind up needing to integrate with and write a plugin for something written in Ruby, and you really have very little experience with it and so don't know where to look. Due to the huge numbers of different languages, APIs, platforms, and so on, rapid pace of change, and shortage of qualified people with the exact relevant experience, almost everyone is a beginner at something they are doing at some point in their career.
Furthermore, some people just don't know how to ask good questions online. It's a skill that takes some time to learn. If you're asking a question to someone in person, they may already have more context about what you're doing, or will be able to more easily ask you questions to clarify and sit down with you to work through it, while asking good questions online can involve a certain amount of preparation in advance to narrow the issue down to a self contained, reproducible test case, to phrase the question appropriately, and so on.
So while I agree that complete junk should be closed and delete, and people who just post "plz gimmeh teh codez" should be discouraged, I am really frustrated by the way people extend that to things that are just beginner questions, poorly worded questions from people who aren't good at asking questions online or aren't good at English, or the like.
https://github.com/mw8/white_keyboard_layout
He defined a somewhat different set of scoring, then ran a simulated annealing algorithm to find a layout that optimized that score, then did a few manual tweaks for consistency and learnability. The code for the optimization is available, so you can define your own scoring and run it yourself.
The improvement in quality compared to the previous
status update is quite obvious. Daala is now much better
than both WebP and JPEG (libjpeg-turbo is the most
commonly used JPEG encoder). As for Daala vs BPG/HEVC,
the artifacts are obviously different and hard to judge
from just four images. Opinions are likely to vary based
on the viewer and the input image. At this point, what
we'd really need is a full subjective test. Fortunately,
this is exactly what is going to take place shortly, as
Daala has been submitted as a candidate for the Image
Compression Grand Challenge at ICIP 2016. The results
should be available in September. In the mean time, you
can read the paper we will be presenting.
As such, it's really hard to say what kind of abuse they are talking about; slut shaming and rape threats, while both awful, can be quite different. Also, I am curious about how they accurately determine gender of Twitter users; there are many fake accounts on Twitter, and I don't believe Twitter even reports gender identity of users.
My comment was not meant to point out that libsodium doesn't make a general OpenSSL replacement. It was meant to point out that if you restrict the problem domain enough, of course it's possible for people to write secure C. I can write a secure "Hello, World!" program; even a secure networked "Hello, World!". But being able to write a secure "Hello, World" does not mean that I am capable of writing secure C in general.
Pointing out one small piece of code that is apparently secure isn't what a claim like "nobody can write secure C" is really about. "Nobody can write secure C" means that when programming in the large, implementing standardized network protocols with all of their warts, having codebases that evolve over time, that no one can consistently and reliably write secure C.
So sure, libsodium may, as of now, have no known CVEs. But it's based on NaCL, which is written by djb. He's also written qmail and djbdns, both claimed to be secure. qmail has had exploitable problems on 64 bit platforms with large amounts of virtual memory: http://www.guninski.com/where_do_you_want_billg_to_go_today_... and djbdns has had security problems exploitable on any platform: http://article.gmane.org/gmane.network.djbdns/13864.
OpenBSD is likewise a minimalist system with a heavy emphasis on security, but even with that approach, they have had to change their slogan to "Only two remote holes in the default install, in a heck of a long time."
Now, of course, this does point out a few things. Minimalism is important for security; and more minimalist approaches and care in writing code can help reduce the frequency and severity of critical vulnerabilities. But even given some of the most careful, security conscious approaches, people still make mistakes.
That's why, when practicing responsible security, you should use defense in depth. In addition to all of the care, review, minimalism, principal of least authority, isolation, etc, you should also use tools that can prevent whole classes of bugs at compile time.
Yes, NaCL is a better design for cryptographic primitives, being much more limited in scope and reducing the number of primitives supported. It's great, if you can control both sides of the protocol so you can use something like this, and if you can also write your own safe, secure protocol code on top of it that doesn't introduce any errors of its own.
The problem being solved by OpenSSL is much harder; for interoperability, you need to support a much wider range of standards, and they are always changing. It also supplies the full protocol stack to implement TLS, including multiple versions of TLS, rather than just the crypto primitives.
It's when you have code that is trying to do all of this, and evolve over time, with contributions from multiple people, that it's easy for such mistakes to creep in. Writing a single piece of code in C with no vulnerabilities is hard, but not impossible; maintaining such a piece of code, implementing complex and changing standards, over time, over a variety of platforms, and so on, without introducing vulnerabilities, does seem to be impossible in C.
https://github.com/briansmith/webpki
Which is designed to validate TLS certificates. This is doing the ASN.1 parsing and signature verification in a zero-copy, memory safe way, built on top of ring for the core crypto primitives.
Now, this isn't a full implementation of what you would need for replacing OpenSSL for certificate handling (and also isn't yet complete); in particular, this is extremely limited in scope as only being for client-side certificate verification. This particular OpenSSL issue was when parsing and re-encoding certificates, which is out of scope for webpki. But it is a good starting point for demonstrating memory-safe and efficient handling of complex tasks like parsing and verifying of TLS certificates.
If by "tech", you mean "high tech", well, that encompasses an awful lot of different things, and of course the goalposts are constantly moving as today's high-tech becomes tomorrow's tech becomes the next day's obsolete.
If by "tech" you mean computer hardware, software, networking, and networked services, then sure, that's a pretty important driver of some other advances, but likewise advances in other fields help to drive this as well. Globalization, shipping, mining, energy, RF engineering, photonics, aerospace, and so on all play into this, and all both benefit from and contribute to the success of computer hardware, software, networking, and networked services. But just because this form of high tech can help with other fields, doesn't mean it's driving the economy. It may be increasing productivity to a certain degree, but there is an awful lot of the economy which is only minimally influenced by these kinds of things.
One of the many great features of the Rust regex crate (https://doc.rust-lang.org/regex/regex/index.html) is that there's actually a regex_syntax crate that provides this parsing support (https://doc.rust-lang.org/regex/regex_syntax/index.html), which can be used for purposes like this, to do alternative kinds of matching engines.
Rust's regex library also has pretty good DFA and NFA matching engines, and an Aho-Corasick engine for parts of the regex that can be decomposed into just a set of simple strings to match, as well as some fairly well optimized exact substring search to speed up search for expressions with some substring that must match exactly, so it can be fairly fast for the kinds of matches like your "TAG(A|C|T|G)GG" example.
Another, even more radical change, would be to build a finite state transducer of your data set, which you can then do regex matching on directly by taking a union of an automaton representing the regex and the FST representing your data (which is itself an automaton). I learned about FSTs from the author of the Rust regex crate: http://blog.burntsushi.net/transducers/. An FST is actually used internally for certain types of searches in Elasticsearch/Lucene (https://www.elastic.co/guide/en/elasticsearch/reference/curr...). I don't know if these would be suited to this problem as is by just building an FST from your set of sequences, or maybe using the overlapping subsequences idea to make FSTs from a larger number of shorter strings, but it would be an interesting project to explore.
And yes, the "home" button should go to the project home, not the blog home (or there should be a separate button for that).
Can you check to see if you have the font installed locally, and if uninstalling it (or updating it) changes the behavior?
What this might point to is room for a new high-level
language that maps better to the reality of current
hardware.
There's more to the world than end-user applications
though. I think your mental model is that there are two
kinds of Rust user: OS kernel writers and people who
create applications with menu bars and save buttons.
What about network services that would rather begin
failing requests on overload than shut down entirely and
restart, incurring potentially big delays in the
process?
However, even for this case, you can handle OOM more gracefully if you change allocation failure to panic rather than abort (either by changing the default in Rust's standard library, or using a custom allocator). At that point, you can define a proper task boundary on which you catch unwinding, make sure that everything shared across that task boundary is exception safe, and recover gracefully.
What about scientific computing projects that
are happy delaying work once they've hit pre-defined
limits? I think you're suffering from a failure of
imagination.
If Rust's goal is to supplant C, it needs to be capable
of everything C is capable of doing. Arguing that
applications in general need X or Y is a canard, because
most of those applications have no specific need of the
kind of direct memory control that Rust affords.
The goal is to provide a reasonable, safe alternative, that offers better abstractions and greater safety, and can be used in situations that other high-level safe languages are unsuitable for.
As far as replacing C, Rust absolutely is capable of replacing C; just use #![no_core] and handle allocation failure however you want. C++'s standard library is more comparable to Rust's standard library.
To put it another way: who are you trying to satisfy? Are
you trying to compete with Go, Nim, Python, and Java and
provide high-level facilities that work most of the time,
at the cost of control, or are you trying to compete with
C and C++, which still fill an essential niche?
By appealing to arguments about the requirements of
applications in general instead of requirements of
systems programming languages specifically, you're
suggesting that the former audience is the better bet.
Rust is a general purpose programming language, that is designed to appeal to a wide audience, but fill needs that other high-level languages cannot, and provide safety and abstraction that C or C++ cannot.
The first audience is likely a much larger audience, and so it is worth keeping their needs in mind, while the second audience can take the most advantage of Rust's safety and performance guarantees.
That kind of targeting is sad, since one of the promises
of Rust is that its memory safety would save us from the
plague of security holes in low-level software. The
decisions the Rust project is making right now make it
less likely that Rust will be able to fully fill C and
C++'s niche.
However, even for applications that do not need to handle allocation failure, they would be able to take advantage of type safety, memory safety, and easy, safe concurrency. You are focusing on one, small issue, and ignoring the huge swath of other issues that you run into when writing C or C++ code that can go away by using Rust.
One of the purposes of having a standard library for a
project is to be a universal resource for all users of a
language. If Rust's standard library isn't suitable for
all environments where Rust might be used (like C++'s
standard library is), then maybe it should be packaged as
a separate project, like Qt.
Something like C++ or Rust's standard library cannot be used in all situations, and even in places where it could run, no general purpose standard library is ever going to satisfy all users. What Rust aims to provide is one that works best, and most naturally, for a wide variety of use cases, which includes GUI applications, web apps, network daemons, and scientific application.
Since handling allocation failure as anything but an abort is so uncommon, it chooses to avoid either of the other two options: requiring everything that allocates to return a Result, making the interfaces to every collection type much more painful to use, or having pervasive exceptions and exception handling, meaning you need to think about exception safety everywhere.
The approach that Rust takes is a moderate approach; it uses return values for those errors that pretty much any user will have to handle, and panics for truly exceptional situations that normally should lead to an abort but which you can add special handling for at task boundaries if you need to provide higher availability, which means that you limit the number of places in which exception-safety needs to be considered to just those boundaries.
At the moment, it uses aborts for allocation failure, but there's nothing inherent to the language about that, just the current implementation.
I think the main point where our opinions diverge is that I see handling memory allocation failure with anything other than an abort as much, much more rare than the extremely common cases of exceptional situations leading to much worse results in C or C++. The sheer amount of undefined behavior, the mysterious bugs caused by buffer overruns overwriting random bits of the stack, the security vulnerabilities, the bugs caused by some undefined behavior you didn't realize was there causing the optimizer do do something strange to your code, and so on.
If allocation failure causing an abort when pushing to a Vec, unless you supply a custom allocater that panics instead and implement proper panic handling, is something that you think is fatal in terms of choosing a language, why is it not fatal that one single missed buffer length check buried in one library somewhere can cause completely unrelated parts of your application to fail mysteriously? As far as appropriateness for the kinds of projects you describe, other than the greater library and tool support due to being much more mature ecosystems, I can think of very few cases in which C or C++ would be preferable to Rust; so much of their behavior on unexpected situations is so much worse than an abort.
Also, I know that for some things that I write about, I'm not actually an expert on it beforehand. Instead, I see an interesting question, say to myself "hmm, that's interesting, and I don't actually know the answer", go out to Google, books I own, or just try some experimentation out myself, and then comment on what I found. I usually do try to provide sources and some details about what I did when doing this, but I realize that it may sometimes make me look like more of an expert on topics than I actually am. For example, I have a silver badge in JavaScript on StackOverflow, even though I've only ever written a few dozen lines of JavaSript in my life (in fact, I've probably written more lines of it on StackOverflow than off), since I tend to be good at quickly researching and experimenting with a question and providing a good explanation, not because I'm an expert on JavaScript.
So yeah, it's great the Julia writes down her thought and search process while doing this, as it helps people realize that they too can go through the same process and learn new things.
Besides, if you make exception-safe code difficult to
write, nobody in practice will write it, so you'll end up
with a system that's tantamount to one that just aborts.
Saying that "Rust the language handled OOM just fine
without stdlib!" and "we can convert OOM to panic!" is
useless when these measures don't help real world code.
As I said, work is ongoing to determine if this AssertUnwindSafe approach is actually workable in practice. The initial implementation had some usability issues, but it looks like it may be more workable now that you can use it on the entire closure if you need to. It's still a speedbump, but a very minor one.
That kind of claim sounds reasonable, sober, and
measured, but it leads to bad outcomes in every system
I've seen, because the "exceptional" case in practice
becomes a hard abort.
For people who are not trying to write extremely fault-tolerant code like SQLite, and going to great lengths to do so, that is a good thing; adding some half-assed normal error handling around these truly exceptional cases is more likely to lead to mistakes and problems down the line than just aborting is.
For people who are trying to write extremely robust, fault tolerant code, you can either handle panics, or avoid the standard library and do error handling via results. Both should approaches should be viable, depending on your requirements; the standard library does take exception safety into account, so it shouldn't on its own cause issues if you handle errors via panics.
I'm not comfortable to casual changes in core runtime
semantics.
Now, Rust not having a formal specification or multiple implementations is not a good thing; it's just a fact of life for a language that is not yet very mature. But I think that this particular behavior is something that should be considered similar to unspecified behavior at the moment. Just like out of memory situations or stack overflow behave differently on different platforms in C and C++ at the moment, how the Rust runtime behaves on out of memory could also be subject to change or different implementations. Given the standard library API, you couldn't return a result, but either aborting or panicking would both be consistent with the language as currently defined.
People (especially from the GNU/Linux world) constantly
assert that allocation failure is rare
SQLite [1] and NTFS [2] come to mind, as well as lots of
tools I've discovered.
SQLite is written in C, which doesn't have exceptions, nor a standard library similar to the C++ or Rust standard library. SQLite has had to implement all of their data structures by hand. You can do exactly the same in Rust by using #![no_std] and just using the core library, which only defines basic data types and never allocates.
NTFS is written in the NT kernel, which doesn't have support for exceptions either, nor does it use the C++ standard library.
So yes, you can actually write code that handles allocation failure properly. The examples you've given both eschew a high-level standard library, and instead implement all of their data structures and memory handling themselves, reporting errors by passing error values back. All of which you can do in Rust using #![no_std].
Meanwhile, there are lots of user-space applications that people use all the time which have no special handling for OOM situations; they rely on the OS to provide them with sufficient amounts of virtual memory, and either be killed by not handling an exception, aborting explicitly on getting NULL from malloc, or being killed by an OOM killer if they exceed the capacity of the machine and try to access an overcommitted page.
I'm sure there are some examples out there, somewhere, of user-space applications that actually do catch such issues, and attempt to do graceful cleanup. On the other hand, I don't know how successful they will be, especially if they have to be cross-platform; since any kind of cleanup you may do, such as writing state out to disk before dying, will hit the kernel's page cache, which may involve allocating memory, which may fail in such a situation, even if you do try to handle the issue gracefully in user-space you may not have anything you can do.
I'm just wondering if this argument is all hypothetical, or if there are any teams of C++ programmers who are actually disciplined enough to be able to handle this case in practice, in large scale software. I know that in most code that I've seen, the only use of std::bad_alloc has been to log the error and abort.
In Rust, exceptions (panic) are used for truly exceptional situations, like programmer error (indexing beyond the end of an array, division by zero) or things that practically are not expected to happen in a recoverable way in the course of ordinary use, like malloc failure. On modern virtual memory operating systems, malloc failure is so unlikely, and in application code there's so little you could reasonably do if it happened, that it is considered be a truly exceptional case.
On the other hand, Result is used for those kinds of errors that are expected to happen in practice even with working code on reasonable systems. IO errors, errors decoding UTF-8, etc.
Right now, catching exceptions (panics) using recover() is still considered unstable. There is some work ongoing to try and work out the API to help ensure safety, by marking types based on whether they are exception-safe or not; so you can use recover() with types that are built in an exception-safe way, or you can wrap types in AssertRecoverSafe to assert that you are providing exception-safety guarantees yourself, but you can't just arbitrarily recover from panics in code that has access to arbitrary data without someone having added an annotation somewhere that they believe that the code is exception-safe. https://github.com/rust-lang/rust/issues/27719 Note that based on the latest discussion, recover() will likely be named something else involving "unwind" to be more explicit about what it's doing.
And exception safety is quite important to the Rust authors. Note that Mutex has a built-in exception safety mechanism, poisoning the mutex on panic so that other users can't accidentally access the protected resource without being aware that another thread panicked while holding it.
Now, there are times when handling memory allocation failures properly is more important, such as in embedded systems or in operating system kernels, where you don't have a virtual memory abstraction with over-provisioning. However, in those cases you couldn't use the standard library anyhow, as the standard library depends on OS support; so you might as well use alternate data types that do return Result on allocating operations.
I'm just not sure about the utility of providing a convenient way to recover from malloc failure in applications running on virtual-memory operating systems. Can you show me an example in C++ (or any other language) where this is handled properly in application code in any way that doesn't simply log and abort, in which all unwinding code in the same application also avoids allocation as it may occur while unwinding from an allocation failure, and in which these code paths are actually tested in the test suite to ensure they behave properly?
Rust tutorials always spend a lot of time interpreting
Rust's notoriously poor error messages
I have always found Rust's error messages to be quite good. Of course, they can be hard to interpret occasionally if you don't already grasp the concepts they are referring to. But I don't see any way to solve that in error messages; at some point, you have to learn enough about the language for the error messages to make sense. And occasionally, the suggestion they provide for how to fix the issue isn't the right suggestion, but I don't know if there's a way to always do that without strong AI that understands what you mean. The best you could hope to do is to provide suggestions for all of the possible fixes, though that could lead to some messages that are too verbose to the point of being useless.
I think it would help to provide some examples of error messages you have had trouble with, to help figure out ways to make them better.