Dave Winer: An open Twitter-like ecosystem | Hacker News Reader

Dave Winer: An open Twitter-like ecosystem

scripting.com

175 pointsaaron-lebo80 comments

I wish more people would embrace what Dave Winer is saying here and run with the ball. The current web as we know it has benefited from an open ecosystem: everything from the servers running Apache to web pages serving up HTML. I see a real long term danger in closed walled gardens like Facebook and Twitter, they're just not healthy.

But there are celebrities and that hot girl from high-school within that walled garden.It sucks and i wish the products were better and more open but the bar for entry is mostly user draw, and T&A is what draws them in..

*edited for spelling >_>

You just made me think of something. The common folk may as well continue using Twitter, and you're right about the bar to entry. We few, the vanguard of openness and reliability, can use approaches like Dave Winer's to provide an open, durable parallel "Twitter-like ecosystem" that integrates with Twitter—and once Twitter goes away (and Facebook and Google+ and even SMS) we will integrate with its successor—or hopefully, ideally, help create it's successor on top of the open web. It could be a durable base for future work, and a safety net for now.

Also: any time I consider using identi.ca again, it's with this view. Even if it doesn't gain popularity, I can at least provide my own guarantees that it will be around (so long as some few others use it) even when Facebook or Twitter go away.

I can at least provide my own guarantees that it will be around (so long as some few others use it)

Isn't this exactly the problem? Keeping a service alive in perpetuity is missing the point because it's the communication that's important, not the service upon which the communication flows... Human communication is inherently temporally limited. If you ignore that, you're not gonna be heard now, which makes you even less likely to be heard in the future.

EDIT: I guess I'm just saying: nobody reads old messages on this sort of service. If you're not targeting "now", you're starting off with an immense disability.

Aha, I didn't understand at all before your edit (no offense :P), and I think you have a good point, but it is actually on the "pro" side for approaches like Winer's: the hypothetical "Twitter-like ecosystem" is targeting now, just like Twitter (less the realtime-related problems, sadly), but is set up to continue targeting "now" once nobody uses Twitter, even during the transition period!

You got it baby!

That's exactly the idea. It's a bootstrap. You use the systems that are in place now to boot up the successor.

It's never either/or. You use everything that works, that has people on it that you need to reach, as long as they welcome you.

This has been the problem with Google-Plus. They don't have an API that lets you post to it. But Twitter does. To everyone who follows me on Twitter, they don't have any idea that I'm not really on their network. In every sense that matters I am.

But when Twitter goes down, I keep posting, and people who are hooked in my feed still get the new stuff.

Good to know I got the point! :P Can I ask about your thoughts on StatusNet?

That's exactly why I work on http://rstat.us [1] -- we need to be here, ready, for the next decision that Twitter makes that pisses more people off, for the next time that Twitter is down, etc.

[1] - a microblogging site that uses ostatus; we integrate with identi.ca. code here: https://github.com/hotsh/rstat.us

It's OK, for the time being, APIs allow you to pump content into the walled gardens from the new open Twootbook. Users can be bled off over time.

OK,

I don't use Twitter but I see celebrities and T&A a lot more when I log out of my yahoo webmail than when I'm on Facebook. Real, honest to god porn, for that matter, has stayed on the "open web". There are still, uh, forces pushing people that way.

Also, as far as I know the masses have never been on Twitter.

Read item #8 in the doc that you're discussing.

"8. BTW, it has to hook into Twitter. Key point. The thing that's kept the other networks from working is that they don't peer with Twitter. Luckily this is in keeping with the new Twitter mandate of putting stuff in but not taking stuff out. Great. If you want to read what someone says on Twitter you have to use Twitter. Not a big deal it turns out."

http://scripting.com/stories/2012/07/25/anOpenTwitterlikeEco...

RSS? DNS? camelCased JSON?

I don't know if you're going to entice many developers with that combination. What we need is a simple protocol (not an API), maybe JSON/MessagePack based with UDP signaling, that makes it easy to build distributed Twitter-like services, while also reachable by HTTP. The developer experience needs to be easy enough so that a distributed "hello world" service can be built in less than 5 minutes. It needs to come with a cross-platform P2P server component, and client libraries for a few popular languages. Make the barrier to entry so low that any dev can do "apt-get install <fancy-distributed-system>" to get the server/client bits.

The average user doesn't know DNS (username.twitter.com) as well as they do email addresses (username@twitter.com) and URIs (twitter.com/username). If this is going to gain adoption, it needs to prioritize UX familiarity over technical superiority. Everyone has an email address, so use that for identification, but don't clutter people's inboxes by using them to transport or store app data.

Make it super easy to federate with existing walled gardens by providing open-source implementations of server components so Twitter, Facebook, Google, etc. can get up and running quickly.

This has always been Dave's problem. He's a Big Idea Guy, but his implementations often end up being somewhat less than fully thought through. I say this as someone who has been following him for way more years than I like to think about (I wrote a bunch of Frontier code WAAAY back in the Classic MacOS days :) As always though, he's thought provoking and gets other people discussing better ways to do it, which I think he entirely approves of.

Why reinvent the wheel when XEP-0277 (http://xmpp.org/extensions/xep-0277.html) and OStatus (http://ostatus.org/sites/default/files/ostatus-1.0-draft-2-s...) have already accreted quite a bit of thinking about what open microblogging could be ?

https://identi.ca/doc/source

You read my mind. identi.ca is a really good idea, it's a pity that it hasn't managed to catch up with Twitter in terms of user base.

..and why is that ? Wouldn't what Dave Winer is speaking of have the same fate as identi.ca ?

in my mind, identi.ca is a red herring. What really matters is the platform, StatusNet. This is a software package that, kind of like Wordpress, can be installed on your host allowing you to run your own microblog service.

This package, and the underlying OStatus protocol, is where organizations that want to retain control over their own reliability and namespace should be looking.

I agree entirely. But identi.ca is the most popular implementation of status.net and it hasn't taken off. identi.ca, status.net, etc, need to offer something else that twitter.com, other then Freedom.

I think that's a marketing problem, rather than technical.

It's a product problem, rather than technical.

"3. To identify users -- please use DNS."

Using DNS to identify users is unwise, in my opinion, because it means that people won't own their own on-line identities -- they'll have to rent them and for real money, too. And if some users are assigned a sub-domain on a shared domain, their identity won't be portable.

I think it is worth doing a little extra work to make a user name system that doesn't have those problems.

Inventing a new identity system that features 1, portability and 2, end-user price of $0 is surely possible, but surely more than "a little extra work."

DNS has the virtue of being here now, being tested and refined over multiple decades, and offering a choice between subdomains for free or portability for a nominal annual cost. It's not perfect but it's good.

To avoid building in a dependency on DNS does not require that the identity system problem be fully solved, first. It only requires ensuring a good abstraction barrier before too much code propagates that depends directly on DNS. For example, perhaps, URIs.

Why can't the usernames be like email addresses?

  send email to username@example.net
    => lookup example.net
    => pull MX record
    => route email to example.com

Looks like a SRV record[1] could be used for this.

[1] http://en.wikipedia.org/wiki/SRV_record

"Why can't the usernames be like email addresses?" [in the sense of querying DNS to find them].

In this context we're talking about what it takes to avoid relying on DNS (because DNS is a centralized, highly politicized system). Your solution would still rely on DNS.

A system which relies on DNS, but in which anyone can setup a node seems better than a system that relies on DNS but only has one centralized node (Twitter), no?

In a truly decentralized system, you're not going to be able to have readable unique names without collisions. Why? What happens when the network splits, then people on either side of the split setup the same username. How do you rectify this when the network rejoins? How do you know that the network has split vs. a node going offline (if you wanted to do something like shut down new usernames until the network was whole again)?

"A system which relies on DNS, but in which anyone can setup a node seems better than a system that relies on DNS but only has one centralized node (Twitter), no?"

The concern here is that whoever is currently leasing the domain name has authority over users' identities. A better system would let users own their identities outright.

"In a truly decentralized system, you're not going to be able to have readable unique names without collisions. Why? [....]"

This is a well explored topic. A good place to start might be to look up "Zooko's Triangle" and then go forward from there towards various ways people have figured out to deal with such problems. (Zooko's wasn't the last word.)

Is the webfinger protocol [1] like email addresses in the way you're thinking of?

[1] - https://code.google.com/p/webfinger/wiki/WebFingerProtocol

That's true of email as well, though.

Uh, DNS is too hard for most people. So is maintaining a web server. If you want any kind of reliability you're going to have to spend 10/year on domain name, 15/year on good DNS server, and 10/month on web hosting. That's $145/yr. Users can use twitter and facebook for free.

Besides, the general public can't even use HTML well, so what chance do they have with xml?

The other compelling thing about twitter is that 140 characters thing. Blogs enable people to train-of-thought-rant for pages before making their point (if at all). Tweets, on the other hand, force people to think and condense before writing. That's an awesome feature for readers. Also, twitter makes it very easy to follow and unfollow.

It doesn't seem like Dave is advocating wrapping this up into something friendlier to end users. However, you certainly could have something powerful if you did.

I agree that expecting people to muck around with DNS and even RSS wouldn't gain wide-scale adoption.

I think his suggestions are for the people who are planning on writing a Twitter-like open service: https://join.app.net/

Except app.net isn't open in any of the ways Winer describes.

App.net isn't open nor closed, because it doesn't exist yet.

It will be if its paying customers demand it.

The stuff that is hard for most people is moderately easy for those who actually have a message to transmit. I'm thinking of government agencies, or businesses.

Have you seen the websites of most local businesses, like restaurants? They look like they are from 1995. Web dev not easy for them.

Many of the same businesses are using Twitter to get their message out far more effectively.

I know what you mean. Not all businesses are going to have the same technical competency. In fact if they lack the technical competency, perhaps Twitter is their best choice.

"4. A user is a feed. So the name points to a feed."

What if the user wants to have more than one feed? Or wants, sometime down the road, to have routable resources that are not feeds?

Wouldn't it be better to say that a user name is a user name and that a default feed name can be automatically built given just the user name?

This makes the "Twitter-like" system less "Twitter-like". Don't get me wrong, I see your point, and I can think of a technical solution (point to OPML feed collections instead of feeds), but the further this gets from the original popular thing (Twitter) the less support and momentum it's going to have, imho. On Twitter it's one stream per user, and that keeps things simple. And simplicity is a huge part of the value add of twitter.

I agree that "emulate the user==feed simplicity of twitter" is the best counter-argument to "make the user name distinct from the feed name for greater flexibility" position.

But here:

In twitter APIs, can't you get something like, say, a user's avatar image by keying off the user name?

So, even on twitter, a user name maps to multiple different things -- not just a single feed.

Over and over I emphasize that Twitter is not a public utility. If you're an organization that has to get your micromessage out there, you're better off hosting your own services.

I don't get what an "open" twitter system will have over the current one. Unless I'm missing something, this is just RSS feeds.

What goals are you trying to accomplish using the service, be it Twitter.com or an open alternative?

Are you just trying to post/read feeds as an individual? If that is the case the open alternative does not provide you a benefit, and you will probably find it less convenient.

On the other hand, what if you have many users under one org? How about you're just a member of a division of another org, and they have many division and subdivisions. And they really want be sure that your message gets out there, without relying on a third party. The public relies on those messages, for example a fire department posting about wildfires. Those are the use cases that will probably see benefits.

But in the second case, what's the difference between that and email?

If you have the infrastructure to handle a setup like that, you should already have an internal email server.

EDIT: After your edit, I think I've found where the disconnect is from my perspective. The problem with this method is that discovery(arguably the most important part of what Twitter provides) is still reliant on a third party's index.

From a user's perspective, Twitter provides 3 key services from one URL:

1. A unified feed for everyone you follow(This proposal also does this).

2. An easy way to post/host content(This proposal does not deal with this).

3. An easy way to discover new people to follow(This proposal also does not deal with this).

Out of those three, I would argue that the second and third are the most important. The problem isn't getting the message out to people that are already subscribed with Twitter, email, or a hosted website. The problem is discovery, and giving people an easy way to actually find the information that they're looking for.

The only way to handle discovery on this way is to have some hosted, third party method of searching through the users to find the ones you want to follow.

The difference is, email isn't (usually) public. A status message is there, for anyone to see it, like a tweet.

If a message is posted in public and no one sees it, does it matter?

People won't be using this new decentralized service. They'll be using Twitter. I guess you can feed into Twitter, but the last mile is still Twitter.

No, you won't. You will be dramatically worse off. You have to go where your users are. And they are on Twitter, not your own hosted thing.

Twitter will not make or break most organizations. If it were to disappear tomorrow most any organization will have plenty of time to look for alternatives. In the meantime Twitter wins. Always. By a mile.

Do you trust Twitter? Do you believe that they will have the same motivations that you do, as an organization? I believe that is the fundamental question you should be asking.

I'm not really sure it matters where the users are. Ideally in a well designed system/protocol, it shouldn't matter. Email solved that problem 30 years ago.

No, I don't. But it doesn't matter. As an organization you have to go where your users are. And they are on Twitter.

This is something no single organization can solve. It's impossible. And email is about the absolute worst example. No, it absolutely didn't solve the problem. First the users had to come. Before that it was worthless.

I think many here are delusional about how this works.

Hacking together a microblogging system out of RSS and Atom with a little network glue is fine for a prototype.

But in the long run you won't beat Facebook and Twitter by merely replicating their functionality.

hm, don't be so sure. I was of your opinion and recently changed that when learnt about Instagram success.

It seems like internet is still a very young thing and people jump the ship or move on to a new thing very often, without much sentiment, like a child on a playground going from one toy to another.

Twitter introduced photos sharing long time ago, but it didn't stop guys from Insta to literally rip it off (adding "follows" and similar features). Where one thought "well I cannot build photo sharing service because the giant in the room (twitter) just introduced it" -- the others just continued pushing the code and eventually exit for $1B.

I think at some point users will move on from Facebook like they did from MySpace. MySpace biggest problem was javascript, audio and graphical clutter that rape you fresh out of the screen once the page finished loading. So others (hello Facebook) took advantage of it and come up with similar page but this time nice fresh and cleancut. So let's move further and ask ourselves a question: what is the biggest Facebook problem? I believe right now it is overclutter with irrelevant info served from your network. Having more than 100 "friends" makes your wall so clutter than no information is of value anymore.

So you want to come up with Facebook killer? no problem -- just build a system (whether mechanical or manual) where users can take real advantage out of their network, not miss important update from important friends and miss the unnecessary garbage they are not interested in. Don't be fooled by "oh they wont come because they did all this work building up their facebook profiles". Not only there is FB Api that you can connect to and download all FB user data in minutes, but if FB would close that gate, you can still program a info scrapper that would do so.

Bottom line: Facebook won't be here forever, and next stage of social is million connected friends but only relevant info rendered.

Beating them is not in the cards.

If you look at tech industry cycles the leaders don't get beat, they run out of room to grow, or evolve into something less monolithic.

Hegemony is always short lived. Once you get on top of the heap it's usually a short time before there's a new generation rising up.

You HAVE been reading Orson Scott Card! ;) But you're right and for a very good reason. The barrier to entry has dropped to the point where a single guy in NYC (or elsewhere) can challenge the status quo.

This article is a bit confusing, I'm not sure it is stating its view in the simplest way. Trying to explain that view to someone else I'd do like this:

Twitter, Facebook and the like should be like emails: if you send me an email from hotmail I can read it on gmail or any other mall client. I should be able to subscribe to friends, interesting people or other social content provider and consume this comment from the client of my choice. Something that Google reader was not very far to provide.

Wrote something about this view sometime ago: http://www.douban.com/note/174513094/

We need to treat our data like we treat our email. Define the common attributes comprising data within a particular application and define how to access that data (through an API). Come up with a common vocabulary for all data (crowd-sourced based on the current stewards of that type of application) and tie those calls into user identity providers (again, built around the common attributes of a user identity). Every interaction between apps and users goes through the user to collect permissions. Permissions are based on signals gained from all other interactions that have passed through the user identity provider. Signals like how often you interact with the app or user requesting the data, which topics you've interacted on previously, etc.. Data is still stored on separate app providers, but we now have simple access. The app provider uses the signals to build permissions specific to their application. Users can transfer their data from one provider to the next easily since all of the data definitions have been translated (assuming the apps are similar in nature).

More here: http://GoPalmetto.com/

Great article. Loaded with links to things I need to learn and many good points.

If someone can put this together in a package that consumers can wrap their head around, I think the open twitter movement would have a shot.

Probably the best candidates would be all the Twitter client apps that are getting burned by their API lockdown

"If you want to read what someone says on Twitter you have to use Twitter. Not a big deal it turns out."

I have to go to all the trouble to maintain my webserver and setup DNS, then go to twitter to read others? Why is this not a big deal?

You could support existing RSS readers by putting the entire microblog post in the title field. And if you're worried about it being truncated, then duplicate it in the body.

Was confused myself as to why he doesn't do this. He mentions not having titles being an issue in Google Reader, but the same issue is apparent if you try to add the rss feed of his linkblog in Firefox; you just get a bunch of links with no details.

Do RSS titles have character limits?

No, neither RSS nor Atom specify a maximum number of characters for TITLE elements. Which isn't to say that particular readers/clients don't truncate them, of course, just that the specs don't say they should.

Interesting. So does anyone have any suggestions, technical or otherwise, other than "twitter posts don't have titles" that he does it this way?

Here's a link to his linkblog.

http://static.scripting.com/myReallySimple/linkblog.html

It's great, except it's completely unusable due to it being a huge download of a page full of disparate links with no organization other than by date.

The trick is not in writing something like this, it's in making the information aggregated and easily organized.

He mentions the lack of a notifications system. Isn't there room in this open system for a ping type notification service or group of services? The benefit for such a service to be free would be that they could aggregate the data and mine it as twitter does. Then clients could fail back to polling should this service not be available.

pubsubhubbub

A simple, open, server-to-server web-hook-based pubsub (publish/subscribe) protocol as an extension to Atom and RSS.

Parties (servers) speaking the PubSubHubbub protocol can get near-instant notifications (via webhook callbacks) when a topic (feed URL) they're interested in is updated.

The protocol in a nutshell is as follows:

- An feed URL (a "topic") declares its Hub server(s) in its Atom or RSS XML file, via <link rel="hub" ...>. The hub(s) can be run by the publisher of the feed, or can be a community hub that anybody can use. (Atom and RssFeeds are supported)

- A subscriber (a server that's interested in a topic), initially fetches the Atom URL as normal. If the Atom file declares its hubs, the subscriber can then avoid lame, repeated polling of the URL and can instead register with the feed's hub(s) and subscribe to updates.

- The subscriber subscribes to the Topic URL from the Topic URL's declared Hub(s).

- When the Publisher next updates the Topic URL, the publisher software pings the Hub(s) saying that there's an update.

- The hub efficiently fetches the published feed and multicasts the new/changed content out to all registered subscribers.

The protocol is decentralized and free. No company is at the center of this controlling it. Anybody can run a hub, or anybody can ping (publish) or subscribe using open hubs.

To bootstrap this, we've provided an open source reference implementation of the hub (the hard part of the protocol) that runs on Google App Engine, and is open for anybody to use.

https://code.google.com/p/pubsubhubbub/

I've run across pubsubhubhub before (geez that's hard to write), and it seems like a really useful implementation.

What I wasn't sure of is when he mentions polling is that instead of pshh or would that work with it?

Polling always works, but if the feed and the client are PSHB-enabled, then they can use it.

Re the "hard to write" thing, PubSubHubbub's developers sometimes referred to it as "PuSH" for short. Which is easier.

Dave also has a proposed notification standard of his own, called RSSCloud:

http://rsscloud.org/

There's been some drama between the developers of the two systems in the past:

* http://scripting.com/stories/2009/07/22/snappyRetardedAnswer...

* http://brad.livejournal.com/2405147.html

* https://twitter.com/davewiner/status/23404535816912896

(Drama being nothing new to the RSS world, alas. Sigh)

I use this with my atom feed, and updates show up within seconds on Google Reader. Therefore I don't know why he says an open system can't be as fast as Twitter. Unless the technology he's committed too, RSScloud, doesn't work as well.

There's more than one person named Alice, and there's more than one person named Bob.

Like others, I'm allergic to DNS for identity. Yes, the UX is atrocious (see: OpenID).

But, what about getting away from a global namespace?

Let people refer to their friends however they want! Use marked up links to reference the underlying feeds.

There are also a great number of similar projects.

https://en.wikipedia.org/wiki/Distributed_social_network#Com...

(Not all, but some of them follow a similar principle)

"6. I think the part that's hard to scale is the notification."

Why not email (SMTP/IMAP)? Deployed, standardized, widely supported.

I can't help but think that Google dropped the ball when deciding to be a closed centralised platform. Being open IS the marketing.

let's call it Wintter

It's fairly obvious to me that an open system, that incorporates Twitter-like functionality as a subset of a greater whole, could take over this space in less than 12 months, leaving Twitter/Facebook/* as AOL-like also rans. Winer appears to have functional solutions for many of the parts needed to deploy this ecosystem. What is missing is a spark of genius to fire people's imagination, such that they flock to the new tool(s) with an eagerness not seen since early days of the www when Mosaic hinted at what could be done. I can't believe that all that is missing is a healthy dose of Mad Men marketing. Maybe that is the missing ingredient?

Big statement. How do you get critical mass when practically every journalist, celebrity, and person you went to school with uses Twitter or Facebook and not your new open system?

While I definitely think an open solution could eventually "take over the space" and leave Twitter and Facebook as "AOL-like also rans," it's far from obvious to me how one would do it in a year.

It is a big statement, isn't it? I do believe it to be true, however. What I'm positing is that the technological pieces can all be deployed relatively easily. What is required is the 'spark of genius' that triggers mass adoption. The hook into the new system has to be easy; it has to be cool; it has to smart; it has to be desirable. Possibly even irrational.

Reply on news.ycombinator.com