pradn

I like IPFS, and want it to continue improving, but it is just too slow, uses too many resources, and is often unreliable.

https://annas-archive.org/blog/putting-5,998,794-books-on-ip...

You’re talking about possible sibyl attacks. Yes that’s a concern but there’s a large literature on how to reduce their threat.

This might be a place where “proof of x” concepts from crypto land work.

But we don’t need to go all the way to the maximal solution for it to be practical.

The critical thing here is that regular people can help. The archive is already split into many O(GB)-sized chunks. We need to ensure each chunk has many, many copies.

Torrents are a widely-understood, robust way to mirror large files. But there's no "meta-coordination" built in to the protocol. It's not possible, using just torrents, to have the swarm cooperatively assign who stores which chunks. The optimization function here is maximal chunk availability, subject to individual storage limits and reliability (ie: how often they're online).

It should be easy to just press a button to join a shadow library, allocated 100GB, and be part of the mission.

The best effort I've seen in this space is some guy running a script that crawls the number of seeders for a list of SciHub torrents. Users manually pick the ones with the lowest seeds. All very cumbersome, and prone to staleness.

Of course, this is all a technical problem, separate from infringing copyright or whatever. In the same way as torrents being a technical solution for sharing files, in a general way.

The use-case for having isolated objects with parameters, much like classes in Java, is to be able to a) share code, b) hide internal details, and c) have object behavior be governed solely by a constrained set of inputs.

So the point isn't to have your web component be different from the rest of the page. The point is that you can pass in parameters to make an off-the-shelf component look how you want. However, exactly how much freedom you want to give users is up to the component author. It is possible for there to be too little freedom, true.

See here [1] for a concrete example of someone writing a reusable web component, and figuring out how to let users customize the styling.

[1]: https://nolanlawson.com/2021/01/03/options-for-styling-web-c...

How has Systems Theory changed how you think? Is there a good book you recommend on the topic? Thank you!

Graphs are inherently "chatty" because there are more shapes in which you could store them. The same goes for querying. Similar to "degrees of freedom".

Even storing a graph in memory, you're going to have load a lot more cache lines to traverse/query its structure. For remote graphs, this translates into more network calls.

The smart thing Netflix did here is finding the minimal abstraction that supports their online querying needs. Turns out that they need a few things above a bare KV store:

1) Idempotency keys allow multiple reads/writes without reordering issues. You can use them to do request hedging, which greatly helps w/ tail latency, at the cost of higher resource usage.

2) KV, with the value being a map. A little more structure, which can use the backing store's native structure.

3) Passing client/server parameters back and forth in a handshake. This allows clear request policy propagation, so the whole path behaves the way the client op wants it to.

4) Filtering/selection - to reduce the set of items returned, on the server side. So the network + client don't have to bear the extra burden.

The summary is: "minimal viable structure", "maximal chances to hedge requests / reduce data movement".

This isn't all so unusual if its written into the job description. SREs in tech companies are expected to respond within a few minutes if they're paged in the middle of the night. They are usually compensated for their oncall time, however.

Expecting a worker to come to the factory out of fear or good will is not the way. Just write it into the contract/expectations/evaluations.

If image sensor / lens system quality are orthogonal to the final megapixel output, why bother with 48MP output JPEGs? They take up more space, so the practical benefit to smaller files is there.

The optical zoom on the Pro is so good! I often zoom in 3-5x and still get great results. I've gotten comments from people saying they thought photos of far off building features, etc, were taken with a DSLR.

It's a no-brainer to get the Pro for this reason, if you care about photos.

Thank you, that's useful!

Pretty cool you got to see this stuff in the real world!

Questions:

1) Was the problem able to be solved with one machine?

2) Was the problem reliably able to be solved within a time-bound, or was it "spiky"?

3) Was the solution all or nothing? Aka would you get a partial/suboptimal solution after a time-bound?

> Distributed systems are different because they fail often

The key here is not just the rate of failure, but the rate of failure in a system of multiple nodes.

And - "distributed systems problems" don't only arise with several servers connected by a network. Any set of nodes with relations between them - files on disk linked logically, buffers on different IO devices - these are also going to face similar problems.

> Google here uses SD 1.4, as the core of the diffusion model, which is a nice reminder that open models are useful to even giant cloud monopolies.

A mistake people make all the time is that massive companies will put all their resources toward every project. This paper was written by four co-authors. They probably got a good amount of resources, but they still had to share in the pool allocated to their research department.

Even Google only has one Gemini (in a few versions).

You're right that the roles are somewhat flexible: somewhere between raw technical output and technical/design leadership and "program/project management".

These ceremonies are meaningful if you invest meaning into them.

For my college graduation, the dean did something small to make the ceremony more meaningful. He asked us all to stand up, look back to our family, and applaud them for supporting us all these years. For me, looking back at my parents and thanking them for all they've done for me was a beautiful moment. And it was a full-circle moment for our family, the culmination of a long journey of immigrating to the US, moving around in search of stability. We had moved to be able to buy a house and to get us good, affordable educations. Both of those dreams were fulfilled at that time.

I don't recall much else from that ceremony. Not the speakers, but a few of the interactions afterward with my fellow students and their families.

Google changed their terminal level to L4, which is a level where you can be do designs with guidance, and implement big chunks of projects on your own. The previous terminal level was L5 ("senior"), where you're expected to own multi-quarter long projects and be a tech-lead of ~5-10 people. Oddly it does end up being structured as a pyramid - lots of L3/L4 folks than L5 folks.

I'm totally with this idea - use the language you know best for mundane tasks.

I used to switch to shell or Python to do one-off scripts. But there's not a super great reason why I can't do the same in C++, which is what I know best. All the build stuff is easy to do in my company's repo, so that's not a big blocker.

It's really nice that Go and Rust have one standard package management system, and a built-in standard formatter. Both of these things seem obvious now, but they were innovative when they were introduced. They add a new set of "batteries" to the "batteries-included" mindset. And they put community and ease-of-use first, which are crucial for adoption.

The AI boom is, in theory, a godsend for Intel and AMD. You can focus on creating good tensor computation hardware, without having to worry about getting gamers on board. No need for "Game ready drivers" or compatibility with complex legacy graphics APIs.

Of course, there's the elephant the room for general purpose tensor machines, which is CUDA - famously closely guarded Nvidia. But with the new wave of "above-CUDA" APIs like TensorFlow, PyTorch, and Keras, there's an opportunity to skip the CUDA layer altogether.

Yes, there's humans in the loop for the knowledge graph. I'm unsure if snippets are reviewed in the same way.

You are correct - if a workflow/agent company starts bundling data and API access - they'll multiplicatively increase the capability of their agents.

LLMs themselves are becoming a commodity, plus or minus prompt-following/format-following growing pains. In a year or two, we'll have pretty decent general LLMs that can make use of databases and tools/APIs.

It's a race to see who can integrate all these things in a good way. It really is an execution problem, not an idea problem - it's so obvious.

One sad part of the GenAI wave happening right now is that we're past the golden age of open APIs.

It's hard to read data with widespread anti-abuse checks (CAPTCHAs), lack of open-format data (RSS support being spotty), and restricted APIs (ex: Twitter API). Companies have all the incentives to prevent bot use, and select for human eyeballs.

If we had a Yahoo Pipes sort of golden age, GenAI agents would have a vaster playground to play in, and would be more useful for us.

Consider building an agent for choosing what to do on weekends for a group of friends. The agent would need to keep state for past activities (X, Y, and Z went upstate to Storm King last week) and users' preferences (ex: liking dosas or Calder, dietary restrictions). This part is easy enough - you could just keep a notebook that's passed as context. Older context gets simply deleted or condensed into high level points.

But would it be easy for the agent to:

1) Look up nearby restaurants and events? (Perhaps Resy/OpenTable allow listing restaurants, but it's likely they have tons of anti-abuse tech. Is there even a place where you could see a list of public events - Google pays a third-party for this feed.)

2) Actuate on behalf of the user? (Do Resy and OpenTable allow authority delegation so the agent could book restaurants for users? There's no standard way to do this across venue types - concerts, museums, cooking classes. Is it realistic for agents to click through these sites on their own?)

Friction is signal.

I recall I used to individually send snaps to like 30 friends each time, back before Snapchat invented Stories. I had to think a tiny bit, and put a tiny bit of effort, for each person. We lost that when people started posting to stories only.

If brands can be put on the balance sheet as "goodwill", how much money was burnt by obliterating this brand name - one almost as generic as Kleenex, for its product category (small streaming-first HDMI dongles)?

Yes, totally. Automated, fallible systems.

Ultimately it's a cost issue. If they wanted to, they could fact check the top million question queries - but they don't want to spend on that.

It's certainly wrong, but its not out of place w/ the sheer number of recipes and blogs saying the same thing. It's parroting "common wisdom" which is wrong.

It's not as egregious as some of the generative AI results spewed out by the lowest-cost-inference LLMs they hooked up to Google search a few months back.

The words "friend" (everyone knows how shallow a "Facebook friend" is), "share" ("ride-sharing" instead of calling a taxi), and "community" (is the entire customer base of Facebook really a community?) have been shorn of their sociable, human meanings. It's as if a corporation were mining the good will humans have accreted to those words over millenia.

Sometimes there are communities in these spaces - NUMTOTs or small Discord servers. Other times its just marketing foo foo.

That makes sense. The only reason to separate the metadata out is to avoid having to parse the main content files for front-matter. Plus the metadata files themselves could be in several formats, if necessary. I don't see a great need for that, however.

I was thinking of how to setup a simple static blog without the baggage of a full-blow framework. Of course, I'd be re-inventing the wheel, but at least it'll make sense to me, personally.

What I came up with:

* Each page gets its own folder.

* An optional METADATA file in each folder has stuff like the type of the page (blog post, photo gallery, code listing).

* The index file can be in Markdown or rST or something. I'd just branch to a different parser based on the file name suffix.

This is simple and lets me evolve the blog over time.

For a personal blog, an ideal mix might be just branching to a different HTML converter based on the

Quite helpful, thanks!