lambda

It was a bit frustrating viewing this site on Firefox and not having any indication of why the settings weren't changing anything.

Of course, maybe that's because there hasn't been enough demand for it, and now that there is a font that's widely available which can use this, Firefox might finish enabling it (looks like there is a flag for it, but it's not enabled by default).

Languages that were developed before the advent of Unicode, like C and C++, frequently leave the encoding of strings up to the system; but this can be complicated, because there are a number of different legacy 8 bit encodings, along with legacy encodings that don't fit into 8 bits. So they have two character types, char and wchar_t; char is a type that can always encode at least 8 bits, and wchar_t is an implementation defined wide character type, which is 16 bits on some platforms and 32 bits on others.

When Unicode was first designed, and released as Unicode 1.0, it was envisioned as a 16-bit universal encoding, with the expectation that you would use it just like 8 bit character encodings but with 16 bit character types instead. This encoding is known as UCS-2.

However, experience and unification of the Unicode standard with ISO 10646 showed that 16 bits would not be sufficient for all of the characters in all of the world's writing systems; in order to fit CJK characters into that 16 bit encoding, a large effort was made to unify characters from different languages that represented basically the same thing but might be written slightly differently in Traditional Chinese, Simplified Chinese, Korean, Japanese or the historical Vietnamese writing system. Additionally, a lot of decisions had to be made about inclusion of historical or obscure characters, and so there were many characters which couldn't be encoded properly in early Unicode.

This led to Unicode 2.0 introducing a surrogate character mechanism, now known as UTF-16, in which certain unused ranges of code points were used in pairs to represent a wider range of code points, in order to expand Unicode from a single 16 bit plane of characters to 17 16-bit planes.

Meanwhile, Ken Thompson and Rob Pike adapted an earlier 8-bit encoding of ISO-10646 called UTF-1 into UTF-8, which had a number of desirable properties; it was a superset of ASCII, the interpretation of the ASCII subset of bytes never depended on state, and it was self-synchronizing, so if you started reading in the middle of a multi-byte character, you could always find the start of the next character.

This meant it was possible to use UTF-8 in all of the traditional APIs and file formats which were compatible with 8-bit extensions of ASCII; which meant that you could migrate to Unicode support without significant changes to APIs and file formats. Also, for text which consists mostly of the ASCII subset, which includes many ASCII-based markup languages, it was much more efficient than UTF-16.

In the early days, however, many languages and systems which adopted Unicode started doing so via UCS-2, and then with the advent of Unicode 2.0 and surrogate pairs, needed to adopt to UTF-16. This was done somewhat inconsistently, and it can cause a lot of pain and confusion.

I've already probably written more than is necessary, so I won't go into the history of each system, but a quick survey of a few popular platforms and languages and their native string encoding:

* C and C++: Support 8-bit unspecified encoding strings (char * , std::string), unspecified width wide characters (wchar_t * , std::wstring, which are 16-bit UTF-16 on Windows and 32-bit UCS-4 on most other platforms), and explicit widths (char8_t * , std::u8string, char16_t * , etc...) which are explicitly UTF-8, UTF-16, and UCS-4.

* Windows uses 16-bit wchar_t * , nominally in UTF-16 but not enforced, as its native string type, and also provides 8 bit APIs with character encoding which can vary at runtime

* Linux and other Unix systems generally use 8-bit char * in an unspecified encoding (which can vary at runtime) as their native string type; most systems these days are using UTF-8 as that 8-bit encoding, but it's not guaranteed

* Java, JavaScript, and C# all use UTF-16 as their native encoding, though in many cases there are still APIs which make most sense for UCS-2 rather than UTF-16.

* Python 2 used unspecified 8-bit encodings for the str type, and a custom encoding that supports all of Unicode for the unicode type. Python 3 changed so that the bytes type was 8-bit with unspecified encoding, and str type was Unicode in a custom encoding; a bytes type alias for str was added to Python 2 to ease the transition, as was a unicode type alias for str in Python 3

* Go and Rust use UTF-8 as their native string types

* Most text-based file formats, such as HTML, have standardized on UTF-8 as the backwards compatibility with ASCII is a big benefit for such formats.

There are a few references on why UTF-8 is generally a better choice for languages and APIs than UTF-16, despite some of the legacy use of UTF-16 in languages and APIs:

* http://utf8everywhere.org/

* https://www.cl.cam.ac.uk/~mgk25/unicode.html

I was working on some component and needed to add a new table to our database. I was fairly new to SQL at the time, so I wanted to do some experimentation and prototyping on my own, before submitting the schema changes to our DBA in order to get them into the shared dev database.

Oracle did have a free trial version you could download for dev purposes or learning or whatever, but like this mentions, it was slow and cumbersome to set up that I soon decided it would be faster to spin up a PostgresQL database, make any necessary changes to our schema and code to be compatible with Postgres, and do my prototyping there, then submit the schema modification request to our DBA. So that's what I did; it took a single evening to do all of that in Postgres, even for someone who was pretty much brand new to SQL.

After I submitted my schema chages for review, rather than emailing review comments back, my DBA made me come down for a meeting to explain the issues. Of course this was intimidating to me as a young itern; had I done something so wrong that it deserved a talking to?

After all that, it turns out the issue was that I had ordered a VARCHAR column before some other column of fixed width in the schema of the new table; and apparently, it's preferable to order all fixed width columns before all variable width columns in order to speed up the column accesses. I agreed with the DBA that I could change the order of the columns, though I did have to point out that this particular table was a table of worldwide regions like "North America", "South America", etc, and that there would never be more than 5-10 of these, so any optimization of this particular table was likely premature.

After all that experience; the complexity of just getting a dev environment up, the fact that production instances cost somewhere around ~$50,000 per CPU per year, the fact that we had a full time DBA who was spending a substantial fraction of her job letting interns know that they needed to apply some trivial optimization that you would expect such an expensive database to just do for you automatically, I resolved to never touch Oracle again if I could avoid it, and have had good luck in that I've never had to deal with Oracle in any jobs since.

Funny thing was that the project I was working on was named "ASAP" which officially had no expansion but unofficially stood for "Another Siebel Avoidance Project", because we were actually using our Oracle database as a place to dump information for which the primary store was Siebel, due to how much more of a pain it was to interact directly with Siebel so doing a periodic dump into Oracle and then building our API on top of Oracle was a better choice.

I have to object to that. I work for a company which pays him to build branded versions of FUSE for macOS, and has been doing so for several years.

He does reliably ship us new builds, but it has sometimes been a little difficult to get him to respond to feature requests or even patches that we send him.

I don't believe that he is scaling much better with his commercial licensing than he was as a FLOSS maintainer.

As it is, we're fairly happy with his work to continue to support new versions of macOS, which is fairly hard to ship systems software on these days (the code signing and notarization requirements keep getting more and more onerous, system integrity protection keeps getting in the way of debugging, and so on), but it would be nice if we weren't limited to a single person working on the project.

I'd really rather he kept the project open source, and continued to provide custom builds for companies on a contract basis. Due to the difficulties of building and signing the kernel extension across all of the versions of macOS that it supports, just having the source available but keeping the binaries under a proprietary license would probably be sufficient to make it worth the while of most companies to license with him to provide custom builds like we do.

Given that most code in Rust is safe code and includes bounds checks before access, you should be able to have the verifier rely on those when they exist, and add in bounds checks in cases in which the access is not protected by a bounds check.

Maybe that would be intractable, or to inefficient to be worth it with all of the extra bounds checks. I'm not sure. I'm asking because it's something that I feel should be possible, but I haven't been involved in the research or development, so I'm wondering if those who have been more involved have references to discussion about the topic.

Forging a pointer is UB in all of these languages as far as I know.

It seems like you should be able to have opaque types that represent these unforgeable pointers which you can't do arithmetic on or cast to raw pointers, but can access values in type safe ways, or provide a view to a byte slice which does bounds check on access.

Is there a good place for discussion of this design? I seem to be having this conversation with you and Josh both here and on Reddit, and it seems like a lot of the discussion is spread out in a lot of places.

I was wondering why the focus on copying memory between the processes. It should be possible to make capabilities which represent a pointer and a length (and maybe an access mode), which could be used to give a process direct access to shared memory safely. I don't know if that could be done with low enough overhead for small objects, it's possible that there would need to be a threshold below which you would just copy values to be efficient, but it seems like exploring fat pointer capabilities would be worthwhile.

Memory mapping files or inter-process shared memory is the very coarse-grained version of this, using hardware protection. I feel like it should be possible to do something more efficient and finer grained with pointer capabilities and the static type checking that is done to verify WASM during compilation, but it may be a significant research project of its own.

I've been buying Google phones (Nexus/Pixel) because they do get the longest support, but I still get frustrated by the fact that if I buy one late in the cycle, I only really get 2 years of support for it.

There are lots of apps that make lots of assumptions about how filesystems behave, generally based on the local filesystem and maybe on one popular networked filesystem for the platform (NFS, SMB, AFP).

If one of those assumptions is violated, applications can crash or refuse to interact with you. Some just refuse to write to any networked filesystem. Some run only on whitelisted filesystems. Some will hit an error due to an unsupported operation on your filesystem, fall back to some ancient code path using long-since deprecated Carbon APIs that only work properly on 32 bit systems, and so truncate all of your data to 2 GB.

Problems like the latter are really helped by being able to do some reverse engineering of the application to figure out why the heck it just writes out the first 2 GB of the file.

Because this isn't our bread and butter but only an occasional tool in our toolbox, the licensing on IDA Pro can be rather frustrating. We use it only once every couple of years to debug some kind of compatibility issue like this, and so we usually have to dig around to figure out if we still have valid licenses, deactivate systems that we're no longer using, and so on.

https://github.com/rust-lang/regex/commit/203c509df9e1dcba61...

Both libraries also rely on a number of techniques from existing published research.

Hyperscan is ruthlessly optimized for x86 platforms, but has dropped support for non-x86 platforms. Rust's regex crate is designed more as a general purpose, portable regular expression implementation, suitable for wide scale use.

The way I read the quote, you use TF-Lite to produce a quantized TF-Lite model, and then use a cloud based compiler to compile it for the actual chip.

This is why I asked "am I missing something." Do you have a reference for where the compiler exists in the open source TensorFlow project?

Mostly, what I'm interested in is learning what capabilities their TPU provides, to see if it would be useful for other similar kinds of kernels like DSP (which, like machine learning kernels, also involves a lot of convolution).

So I'm interested in looking at what the capabilities of the chip are, seeing what could be compiled to it. But I haven't found those docs, or found a compiler that could be studied. But maybe I'm not looking in the right place.

Here's an overview of the architecture of their Cloud TPUs, which has some good architectural details but doesn't documet the instruction set:

https://cloud.google.com/blog/products/gcp/an-in-depth-look-...

> You need to create a quantized TensorFlow Lite model and then compile the model for compatibility with the Edge TPU. We will provide a cloud-based compiler tool that accepts your .tflite file and returns a version that's compatible with the Edge TPU.

This seems like a new low in software freedom, and pretty risky to depend on as Google is known to shutter services pretty often and could just decide to turn off their cloud-based compiler at any time they feel like.

SDR, digital audio processing, digital image processing (especially compression and filtering), all of them have roots in DSP.

That looks like an interesting resource, but I really prefer written word and text-based programming languages to video lessons and flow-graph based configuration.

I don't know of what other resources are available, but I do know I've seen some that use Matlab. I've never played with Octave much, so I don't know how compatible it is; can you just directly copy Matlab code, or do you need to do translations?

Anyhow, if there's a book that uses Matlab rather than BASIC, and could be used in Octave, that might be more readable. I would find C, C++, Python, or Rust more familiar, though.

While I suppose it would be a good exercise to translate the examples into the language of my choice, are there any good resources on DSP that use a more reasonable language for the examples? And I'd prefer to avoid proprietary languages like Matlab.

You can work in Courier Prime any time you want, and then switch to Courier for submitting your screenplay without worrying about it changing the pagination.

So it can help out any time you are working on your screenplay or working on it with anyone you know won't care, both before and after submission.

Additionally, if driverless cars are being tested on public roads, you need laws and practices that are robust enough to prevent the kind of accident that killed someone. They had only one operator in the car, who was responsible for monitoring the car and logging information about its performance. They had specifically disabled safety systems which would have caused it to brake sooner.

Even if its expected that driverless cars will reduce fatalities due to drunk or distracted driving, you need to have sufficient evidence of the improvement, safety systems in place for test systems, and public discussion of the tradeoffs and what situations driverless cars might perform worse in than human drivers, because lots of people have behavioral expectations based on human drivers; things like making eye contact with a driver before crossing the street.

So yes, it is good to be working on and testing driverless cars, but you need to have a serious sense of humility and safety about it, and not push it, because even if you are incredibly safe you can expect some backlash, but if you miss the mark at all and oversell the safety, you'll get a lot more backlash, with justification, much sooner, which could set the whole program back significantly.

  use std::time::SystemTime;

  fn main() -> Result<(), Box<dyn std::error::Error>> {
      let t0 = SystemTime::now();

      let triples = (1..).flat_map(|z|
          (1..=z).flat_map(move |x| 
              (x..=z).filter_map(move |y|
                  if x * x + y * y == z * z {
                      Some((x, y, z))
                  } else {
                      None
                  }
              )
          )
      );
    
      for triple in triples.take(100) {
          println!("{:?}", triple);
      }
    
      let elapsed = t0.elapsed()?;
    
      println!(
          "{}", 
          elapsed.as_secs() as f64 + elapsed.subsec_nanos() as f64 * 1e-9
      );
    
      Ok(())
  }

In debug mode, it runs in 0.047586s, in release mode, 0.002304.

I was pleasantly surprised to see this is actually better on the build times and debug build performance, given that those are two often cited sore points in Rust.

I would also say that while this is a little bit noisy, it is more readable than the proposed C++20 version. "1.." looks more like a range starting at 1 than "iota(1)" does. There's an actual "if" statement in there.

In fact, while Rust doesn't officially have generator/coroutine support yet, it is available in nightly builds for the implementation of async/await. If you use the nightly compiler, enable the unstable features, and wrap an adapter type around Generator that implements Iterator, you can write:

    let triples = IterGen(||
        for z in 1.. {
            for x in 1..=z {
                for y in x..=z {
                    if x*x + y*y == z*z {
                        yield (x, y, z)
                    }
                }
            }
        }
    );

Rust doesn't have generators yet, so this is only really fair to compare to the generator/co_yield example in C++ (which is also proposed but not yet standardized). I'd say Rust's syntax compares favorably to that example; the ranges are more clear than the traditional C-style "for" loops, there's native tuple syntax, and it's just be "yield" instead of "co_yield".

It is actually fairly comparable to the Rust Reference (https://doc.rust-lang.org/reference/introduction.html), though the way the Rust Reference is divided into chapters makes it a bit hard to see everything at once or search.

They both do contain some examples, as well as things like the syntax in a BNF like format.

Perhaps what I'm looking for is just to have the Rust Reference made a bit more complete and a little more accessible in format (the division into chapters doesn't really seem to help much), as well as improving the SEO since it doesn't show up in Google searches often.

  lrwxr-xr-x  1 root  wheel     9 Jan  2  2018 /usr/pkg/bin/python -> python2.7
  -rwx---r-x  1 root  users  5952 Nov 12 17:13 /usr/pkg/bin/python2.7
  -rwx---r-x  1 root  users  1685 Nov 12 17:14 /usr/pkg/bin/python2.7-config
  lrwxr-xr-x  1 root  wheel     9 Jan 18  2018 /usr/pkg/bin/python3 -> python3.6
  -rwx---r-x  1 root  users  7728 Feb  9  2018 /usr/pkg/bin/python3.6
  -rwx---r-x  1 root  users  3132 Feb  9  2018 /usr/pkg/bin/python3.6-config
  -rwxr-xr-x  1 root  wheel  5984 Dec 24 01:27 /usr/pkg/bin/python3.7
  -rwxr-xr-x  1 root  wheel  3132 Dec 24 01:27 /usr/pkg/bin/python3.7-config

There is the additional membership level MetaARPA, at $9/month which you need to have to have access to tools like git:

  $ ls -l /usr/pkg/bin/git
  -rwxr-x---  1 root  MetaARPA  2481424 Sep 11 13:50 /usr/pkg/bin/git

But anyhow, this kind of restriction of use of basic development tools for the free tier makes it kind of useless for what I wanted to use it for, even if the restriction isn't enforced all that well. I could understand if they restricted usage, like disk or CPU, or things like TLS for your home page, and I can understand restricting some outbound networking to prevent abuse, but not providing access to Python and Git in the free tier?

There are the "instructor" and "student" levels, but since this wasn't an official course I was running but just "I want to show someone the ropes of Bash scripting," I found it a lot easier to just spin up a free-tier VM on Google than try to find someone who could give me the appropriate membership level.

> Fridge0 does not need a big battery bank, since it stores cold in its thermal mass and runs only when solar power is available.

> But, at least a mimimal battery is needed to run the computer control. Probably on the order of 5 watts for a computer like a raspberry pi; this could be reduced more with a more embedded computer like an arduino.

> Enough battery to run the fridge for a minute or two is also needed. Consider what happens when the fridge is running, and the sun goes behind a rain cloud. Suddenly, there's not enough power to run it, but it's still turned on. The inverter will try to run with whatever small solar power is still coming in, but it's not enough. Some inverters might manage a clean power off, probably accompanied with loud beeping. Other inverters might fail in more interesting ways.

> To deal with this situation, the computer needs to check the incoming solar power frequently, and power off the fridge if there's not enough. The battery is needed to keep the inverter and fridge running until that happens.

Basically, it sounds like the answer is because with a solar panel, there are states that are between "off" and "on," you rather just get an amount of power proportional to the amount of sunlight, and there's a concern that those states could cause problems if you don't provide a more constant power supply and active switching to fully shut off the power when your solar panel is not producing enough power for the "on" state.