Comment by robotstxtwasbad | Hacker News Reader

Comment by robotstxtwasbad | Hacker News Reader

Parent Full thread

robotstxtwasbad

RFC 8820 section 2.3, also known as BCP 190, is not a theoretical argument. That's why we call it a "best current practice".

You, nor any other standard like you, are not entitled to declare what a root URI means in my Web namespace, and you are up against an IETF MUST NOT by fighting for this. This is a _very_ philosophical argument, not a practical one, and it's why I'm firmly against your standard out of the gate (and would work to reject it as, say, an RFC).

The same paragraph takes you to RFC 8615, which is the .well-known you are being told to use. That is not your "secondary location" for v1.1. That is the only path you are permitted to consider as someone with intent to standardize a portion of the HTTP URI namespace. The decades-old precedent you are citing here, and leaning on as foundational, was rejected at the philosophical level by the IETF, and it is completely rejected as appropriate precedent for the writing of standards going forward.

You are being told how the Web works. It's not about you, the magical universe of agentic, or your community. You are attempting to standardize a part of the technical commons. If you want the public to obey your standard, this isn't the way to engage while selling it -- despite it being CC0, you're phasing in and out of "my" standard and "our" standard a little oddly, and you're a little standoffish to (correct) feedback, feedback that in this case is existential to your project making it to a dozen stars and a discussion.

Wix and Shopify have zero bearing on the standardization of the Web. Companies in general shouldn't, in fact (har har), which is useful background for an aspiring standards writer.

I appreciate the detailed feedback (and the edits).

You are technically correct regarding IETF norms.

But you say: "Wix and Shopify have zero bearing on the standardization of the Web."

I fundamentally disagree. The Web is not just a namespace for engineers; it is an economy for millions of small businesses. If a standard is technically "pure" but unusable by 80% of merchants on hosted platforms, it fails the Web.

However, to respect the namespace: We will mandate checking /.well-known/commerce.txt first.

But we will keep the root location as a fallback. We prioritize accessibility for the "aspiring" shop owner over strict purity for the standards writer.

If you fundamentally disagree with that, you are simply never going to deliver a workable standard via the IETF process. Yeah, yeah, SPDY, QUIC, elephants in rooms, I realize what I'm saying, but that doesn't mean I'm wrong about this. Commerce is a _subset_ of what happens on the _technical_ Web, standards for which must consider all users (and the arena you're now playing in). We haven't even gotten to the merits yet, or how you collide with Open Graph philosophically, etc. This is just one piece of technical feedback, and I'm discouraged by your approach to it.

Thankfully, you've licensed your work CC0, so someone who wants to see this standardized could simply fork your work, fix the offending parts, and move for successful standardization without you.

You really gotta stop saying "we," too, like, it's a nit, but it speaks to your long-term intentions. You're here to build a community around an effort you've singlehandedly spearheaded over the last few weeks (I can read GitHub). Claiming you have one already, and there's Big Discussion on these points, is pretty transparent. You and I both know where you're at in the lifecycle, and that you definitely have room to consider the feedback being offered.

The CC0 license is not a bug. It is a feature. If you fork this and build a standard that helps merchants better, the mission succeeds. I will be the first to applaud. As for "We": It is an invitation, not a pretension. A standard cannot be a solo act. I am bootstrapping the working group. You are welcome to join it, disagreements and all.

Thank you for the invitation. One of the things you realize reading and writing a lot of standards -- and I really don't mean that to be condescending towards you, promise -- is that there's a certain orthodoxy to the whole thing regarding keeping an arm's length from commerce.

Consider C#. Yeah, yeah, we all know the provenance of the language, that what ECMA has standardized is basically a Microsoft specification, but once it's an ECMA standard it's Something Else. Competitors can work on it together, and we're all fine with that. Carrying on C# development in the open is harder for Microsoft in some ways, and easier for them in others. This opinion is about ten years old, mind you, and speaks more to the origin of C# (I'm not a practitioner), so I'm sure the Core stuff has changed all of this and made me look silly saying this, but that speaks to my point -- work evolves in public. But they work on it, their competitors work on it, randoms like you and me work on it, and everybody benefits.

Say I work at Apple. I tell my boss I had lunch with a Samsung guy, I might get a side eye. I tell my boss I had lunch with a Samsung guy because we're collaborating on some revision to SSD TRIM or something, it's oh, cool. That's the orthodoxy. Look at, like, WebKit threads before the schism (itself very relevant to this point, in fact). It's extremely important to even _attain_ public standards and collaboration that we all suspend the rules of commerce and competition and conflict and all that. You're arguing the opposite in saying the words "Wix" or "Shopify" should be anywhere near influencing the effort you're proposing. Step back practically, even, and ask yourself: "why should every Web operator deal with some standards crap due to a Shopify product decision? Why is /llms.txt or /products.txt or /yourthing.txt a new land mine for an unsuspecting nginx admin to find?"

There's a collaborating on the common good that should be inherent to the production of shared standards of humanity. Much like science, and their centuries of wrestling with this very point in colorful ways. The Internet is one of humanity's most important inventions, and getting trillion-dollar caps to agree on how to operate it is so incredibly fragile.

If you try to argue with me that because Wix and Shopify both have stupid designs that remove control over a URI from a Web author, I should relax my belief that standardization efforts are fundamentally an activity agnostic of commerce itself, I'd rather gnaw off my left leg than collaborate with a group you lead. We're just going to fight too much. I don't mean this to be disrespectful, for the record, I'm only trying to vividly illustrate how far apart philosophically that seemingly minor opinion places us.

And sure, you're addressing commerce as a subject matter, but one of the ways to lift this from idea to standard is realize the generality behind your effort ("things" available here, not items available for purchase, i.e., philosophically Open Graph's approach, one of the few ways I see your work succeeding).

I respect that orthodoxy. It is the bedrock that allows the Internet to function. But we are optimizing for different variables. You optimize for architectural purity on a timeline of decades. You protect the namespace from temporary corporate flaws. I optimize for utility on a timeline of now. I want the flower shop owner to be visible to AI today, even if their platform is rigid. We have different North Stars. That is okay. You guard the temple. I will help the merchants outside. No leg-gnawing required. Thank you for the perspective.

Reply on news.ycombinator.com