eadan

Relevant section:

3.1.3(d) Person-to-Person Experiences: If your app enables the purchase of realtime person-to-person experiences between two individuals (for example tutoring students, medical consultations, real estate tours, or fitness training), you may use purchase methods other than in-app purchase to collect those payments. One-to-few and one-to-many realtime experiences must use in-app purchase.

This is huge news. Being able to use third-party payments methods to bypass Apple's 30% charge is essential for service driven marketplace apps. Classpass and AirBnB bumped into this issue [0], but, I wonder if this exception will apply for them?

[0] https://www.nytimes.com/2020/07/28/technology/apple-app-stor...

Edit: In excitement, I missed the last sentence; group services aren't covered by the exception :(

This is such a cynical outlook on the world. Of course there are for-profit companies -- most I would argue -- that have a positive impact on the world, and Tesla is one such example.

R + the tidyverse (dplyr, ggplot2 etc.) is easily the most elegant data analysis combo I've used. However, I tend to use Python for this usecase nowadays, primarily to reduce friction when working with others where Python is the lingua franca. Also, RMarkdown is superior to Jupyter notebooks in my opinion.

Yes, and it's a good thing. For once lately, this is the market doing what markets should be doing -- fair price discovery

Is something like dolthub [0] what you have in mind?

[0] https://www.dolthub.com/

This is very cool. Relatedly, as a data scientist, I wish companies would expose their APIs through SQL. I've spent a lot of time pulling data into ETL jobs from things like mixpanel, adwords etc., and having a unified interface would make things much simpler.

I'm trying to understand the architecture of Splitgraph. Are all foreign data wrappers controlled directly by you, or can third parties host a database and connect it to Splitgraph in a federation?

Agreed, with modern reactor designs nuclear "waste" is a misnomer. The Dutch keep all of their nuclear waste in buildings open to the public for tours, buildings which also happen to house priceless artwork:

https://www.covra.nl/en/radioactive-waste/the-art-of-preserv...

I think the fact that binary size has become smaller rather than larger is notable in a world where most other software seems to be increasingly more resource hungry.

The problem with vertical farming is where does the energy come from? An acre of solar panels is not capable of supplying enough energy to grow an acre of crops, even with the efficiency gains we've had in photovoltaics and lighting with wavelengths tuned to crop growth. What's the point in having fields of solar panels when we could have smaller greenhouses filled with crops? If only nuclear energy was politically viable in the west we could have vertical farming and so much more.

It seems you misunderstood the article. When a minority has an intolerant stance about which the majority has no preference towards, the minority wins. For example, the majority doesn't care if all food is kosher. But, the majority would care if all food was vegan.

I agree, the lack of stack traces can be annoying when debugging Go programs. Instead of returning raw errors everywhere, the problem can be alleviated by adding some context to returned errors:

  if err := foo(x); err != nil {
      return fmt.Errorf("foo: bad argument %s", x)
  }

Dropbox have a good real-world comparison [1] between BBRv1 and BBRv2. They found the retransmission rate to be 4x lower for v2. It's on their edge network, so would be interesting to see how v2 performs on connections with a high bandwidth delay product.

[1] https://dropbox.tech/infrastructure/evaluating-bbrv2-on-the-...

Thanks for your response! This technology is very exciting. Since you're targeting WASM there's a whole host of possibilities. Python is already supported through Iodide [1], but I imagine other languages could also be ported, and even DSLs specifically tailored for agent based modelling. That's not the mention all the other things happening in the WASM space like webgpu [2] and WASI [3].

The potential for ABMs are huge now that we have access to cheap and massively parallel compute. Imagine arbitrarily complex models of each individual in an economy interacting with each other over a distributed network of thousands of machines -- that's possible now. Instead of trying to predict the future, we can compute it.

[1] https://github.com/iodide-project/pyodide

[2] https://github.com/gpuweb/gpuweb/wiki/Implementation-Status

[3] https://wasi.dev/

Congratulations on the launch! I've been excitedly waiting for the public release since hearing about HASH on Joel's blog a few months ago. I've been playing with some models over the past few hours and I have a few questions / comments.

First, would it be possible to perform sensitivity analysis on parameters of the model? For example, with the city infection model, run a number of simulations in parallel with different infection rate values, and see how it affects the model output. Relatedly, I think it would be useful to specify a probability distribution over a parameter rather than setting a fixed value. The state space of the model would become exponential in the number of parameters which can vary, so I understand if it's infeasible, at least in the public version.

The real-time spatial and plot views are great. I would like to try some network based simulations. Any plans to add visualizations for these?

This is amazing! The first thing that popped to my mind seeing the life sized "portal" was the farcaster portals from the sci-fi novel Hyperion

https://hyperioncantos.fandom.com/wiki/Farcaster

This is interesting. I haven't seen much in regard to comparing TCP+BBR to FASP other than a masters thesis which suggests FASP outperforms on transferring large files over long distances (which is exactly its intended purpose). But, I wonder if splitting a file over multiple QUIC connections and re-assembling at the other side would come closer to the performance of FASP? Could be a fun experiment, I might try it!

My understanding is that the protocol underlying Aspera (FASP) uses UDP for the main data transfer and a TCP connection for coordination. By using UDP for data transfer it's not restricted to the ACK ranges imposed by TCP which can hinder throughput on networks with relatively high packet loss. Its throughput is not necessarily achieved by being a "bad citizen" but by having full control over how and when it communicates lost packets.

Since QUIC is also over UDP, perhaps we now have more flexibility on ACK windows etc.

Btw. there are open protocols like Tsunami UDP (https://en.wikipedia.org/wiki/Tsunami_UDP_Protocol) that try to fill the same niche

How well does QUIC perform over networks with relatively high packet loss? It seems like Aspera (https://www.ibm.com/products/aspera) is the industry standard for high performance WAN transfers, but it's a proprietary protocol. I'm wondering if QUIC performs better than HTTP1/2 in this respect?

Yes, S3 uses the content-md5 header to verify data integrity for puts.

https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjec...

You want to make sure PyCharm is only indexing files within your project. Depending on your setup, it can try to index all files within virtual environments etc.

Also in the super fast, but not designed to be cryptographically secure category:

xxhash (https://github.com/Cyan4973/xxHash) with 32/64 bit output. The latest version, xxh3, supports up to 128 bit output.

meow hash (https://github.com/cmuratori/meow_hash)

The recently released Blake3 which is designed to be cryptographically secure is very fast also (https://github.com/BLAKE3-team/BLAKE3)

Not only sub-optimal, but unsuitable. Use algorithms specifically designed for password hashing: argon2 / scrypt etc.

SHA256, SHA512 and Blake* algorithms are suitable for secure checksums and HMACs, but not password hashing

I actually really appreciate companies that produce research and write articles on technologies adjacent to their business. It's a great way of building trust and a community around the business' core product. Digital Ocean are a good example. Reminds me of Patrick Collison mentioning on a podcast [0] that one of the biggest drivers of new customers to Stripe in their early days was a blog they wrote on using the Python debugger.

[0] https://tim.blog/2018/12/20/patrick-collison/

I have experience implementing similar models for analyzing and predicting customer spending behavior. The posted paper may be somewhat abtruse for the uninitiated, so I thought I'd provide some background knowledge.

As the title states these are probability models for describing the purchasing behavior of customers. The paper outlines outlines several models depending on if customers may purchase at discrete times or continuously, and whether the customer relationship is contractual or not. As an example let's take the "BG/NBD" model for continuous time, non-contractual customer relations.

We assume each customer, `i`:

  1. Makes purchases according to an exponential distribution with mean rate `lambda[i]` [0].
  2. After a purchase becomes inactive (never makes a purchase again) with a fixed probability `p[i]`.

And, across all customers:

  3. The purchasing rate, `lambda[i]`, follows a Gamma distribution with parameters `alpha` and `beta` [1].
  4. The inactivity probability, `p[i]`, follows a Beta distribution with parameters `a` and `b` [2].

This is a generative model of the purchasing behavior of each customer. The goal is to find values of the parameters `lambda[i]`, `p[i]`, `alpha`, `beta`, `a` and `b` that "fit" your data the "best". Emphasis here because there different notions on what "best" means, but the two main approaches are optimization (typically maximizing the likelihood) and Bayesian approaches which rather than finding the "best" value for a parameter, find a distribution over all possible values called the posterior.

There are implementations of these models for python [3] and R [4].

[0] https://en.wikipedia.org/wiki/Exponential_distribution

[1] https://en.wikipedia.org/wiki/Gamma_distribution

[2] https://en.wikipedia.org/wiki/Beta_distribution

[3] https://github.com/CamDavidsonPilon/lifetimes

[4] https://github.com/cran/BTYD

It's far from comprehensive, but here are a handful of operating system projects that have caught my eye:

[1] MezzanoOS, an OS written in Common Lisp (https://github.com/froggey/Mezzano)

[2] SerenityOS, a Unix-like OS (https://github.com/SerenityOS/serenity/). The primary author has a youtube channel (https://www.youtube.com/c/AndreasKling/)

[3] RedoxOS, a microkernel OS written in Rust (https://www.redox-os.org/)

[4] Nebulet, a microkernel based on webassembly (https://github.com/nebulet/nebulet)

[5] It was never released by Microsoft, but Midori OS had some fascinating ideas like software isolated processes, asynchronous message passing and object capabilities. Nice write-up about the project by Joe Duffy here http://joeduffyblog.com/2015/11/03/blogging-about-midori/

The Secure Enclave processor on Apple devices runs a version of the L4 microkernel

[0] https://support.apple.com/en-us/HT209632

Nebulet [0] was an interesting take on this -- a Rust microkernel compiled to WebAssembly. Unfortunately, the project seems to have run out of steam.

[0] https://github.com/nebulet/nebulet

The experimental Singularity OS [0] from Microsoft implemented processes in the same address space using so-called "Software Isolated Processes" or SIPs [1]. Singularity was a microkernel design with the filesystem, networking, drivers implemented outside the kernel in a variant of C# called Sing#. It seems Sing# had ownership semantics similar to Rust which allowed SIPs to share memory by transferring ownership through lightweight message passing.

[0] https://en.wikipedia.org/wiki/Singularity_%28operating_syste...

[1] https://courses.cs.washington.edu/courses/cse551/15sp/papers...

A recent discussion here on Latacora's "How (not) to sign a JSON object" [0], had me thinking of CBOR. Unlike JSON, MsgPack, protobufs, BSON, or any other commonly used data interchange format that I'm aware of; CBOR has a canonical representation (although, with seeming ambiguity in float representation) [1].

Anyone have any thoughts on using canonical CBOR for object signing? Currently, I'm building a system with a content-addressable data store, and I'm particularly interested in data formats with a canonical form for this use-case.

[0] https://news.ycombinator.com/item?id=20516489

[1] https://tools.ietf.org/html/rfc7049#section-3.9

Previous HN discussion [0], and associated lectures [1].

[0] https://news.ycombinator.com/item?id=19701767 [1] https://ocw.mit.edu/courses/mathematics/18-s097-applied-cate...