Bringing Craigslist Back | Hacker News Reader

Bringing Craigslist Back

blog.padmapper.com

309 pointsericd362 comments

I think it's perfectly fine if you want to play games dancing around what's legal and forge ahead against Craigslist's wishes. There's a long history of that kind of behavior in information services and they may or may not be able to stop you.

But making a big deal out of being some kind of ux hero is pretty tacky. At the end of the day, no matter how small you are you're still a commercial enterprise gaining benefit at another commercial enterprise's expense. You're bidding on eyeballs with an offer of time savings.

A vast majority of the web deliberately degrades their user experience in order to achieve business goals. Ads are almost always at least somewhat distracting, retailers stretch out checkout flows for up selling, sites "recommend" their high margin items ebay requires paypal and facebook sabotages their privacy management so you're less likely to bother.

If CL was to build a bunch of sweet vertical specific apps for their sections they'd need a ton more developers, designers, testers, project managers and so on. To compensate for the greatly increased burn rate they'd need to monetize a lot better which almost always means features, content or policies that negatively impact the user. Or maybe they'd have just taken too much VC to pay for it and gone under in '02 when the bottom fell out.

So by all means jockey away it's healthy for the industry. But crowing about it is a bit like taking a shit on the guy whose shoulders you're standing on while bragging about how tall you are.

I'm not making a big deal about this, and I have a lot of respect for Craigslist, don't put words in my mouth. I think Craigslist is nearly perfect for the vast majority of things, I'm not on a crusade to "disrupt" them. It's just that for a long, drawn out, location-based search, it's objectively terrible, and I'm trying to patch that bug.

http://news.ycombinator.com/item?id=4220106

It would seem like the definition of respect mostly conflicts with your behavior.

   re·spect
     1. To feel or show deferential regard for; esteem.
     2. to pay proper attention to; not violate "to respect Swiss neutrality"
     3. to show consideration for; treat courteously or kindly

His behavior is not incompatible with respect, and you're being pedantic. I really wish "beating an argument to death with Merriam-Webster" wasn't as common on HN as it is.

One can respect Craigslist for their history, and what they have done for the online community in the past, without conceding the right to say that they're wrong now, and that you'll be proceeding without their blessing.

One can also respect and understand Craigslist's objection to Padmapper scraping their site, without conceding that the good vastly outweighs whatever valid concerns may exist.

Respect is a complicated social concept, the colloquial usage of the word even more so. Using dictionary definitions to try and corner complex social phenomena is at best a poor idea, at worst an attempt at being disingenuous.

back in the usenet days, it was generally[0] held that the first person to reach for the dictionary had implicitly conceded the argument.

[0] at least on the newsgroups i used to frequent

Clearly my post came across poorly and not as I intended, as at the time I imagined it to be a softer way of suggesting that his words don't represent his actions. Other ways of saying that directly sound much more personal and less civil in my head, but perhaps not to others.

With respect to your definition, I submit that you misunderstand the element that differentiates respect from admiration, appreciation or value. Further debate seems unhelpful, whatever you call it it's the same behavior.

>I’ve been wrestling with whether to bring back Craigslist listings in the search results. I’ve found a way to include them that I’m told is legally kosher since it doesn’t touch their servers at all, but it still seems somewhat dickish to go against their wishes in this, and I’ve always had a lot of respect for what they’ve done for the world. Also, who wants to waste their time in court?

>But then I did some back of the envelope estimates of how much of people’s time and effort it would waste if I didn’t, and it became clear how much less nice it is to waste the time of millions of apartment hunters out of stubbornness or some clearly inaccurate assumption about the will of the community.

Christ, what an asshole.

Translation: "We found a loophole that lets us get around the spirit of what was communicated to us so we could continue to build out our product. I'm going to conveniently step over the moral grey lines of using someone else's data without their consent by claiming my service is better."

Ouch, sorry, I thought it was the right thing to do, on the balance. Like you said, it's a really morally gray decision, either way, but I think it's actually worse to do nothing.

Dude, I have no doubt your service IS way more useful than craiglist. I look forward to using it when I next move.

Just… don't dance around your motivations here. You want to build a better service, and you need CL's data for that. CL doesn't want you to use their data. You said that's too bad.

That's all there is to say.

The why is just as important as the what in these sorts of decisions. So, this post was my attempt to explain the thought process I went through that finally tipped the scales. Some people make life better via charity, I try to do it via software. So that's what's tipping the scales on my moral compass. Money can be useful for that too, but mostly as a tool, it's not my main motivator.

I would respect you more if you just came out and said "fuck Craigslist, we're using their data" than go through a bunch of pie in the sky philosophical reasoning to justify your decision after the fact.

Let's be frank and direct: you're being a dick to Craigslist, and you're doing it so you can build a better product.

I even agree with that decision--Craigslist has brought it upon itself by creating what is (IMO) a mediocre and clearly substandard UI. But your choice is what it is. Embrace it.

The idea that he's "being a dick to Craigslist" is the weirdest part of this whole thing. The PadMapper service consists of advertising that certain content is available on Craigslist. I mean, it literally consists of saying, "oh, you're looking for a 3-bedroom house under $3,000 in this neighborhood? Here's a page on Craigslist you might want to check out." That's the extent of the appropriation.

I mean, imagine I was reading the Craigslist apartment listings for my own housing search, and I knew you were looking for 3-bedroom Craigslist listings in a given neighborhood, so every time I saw one I sent you the link. Heck, imagine I even charged money for that service. In what universe am I being a dick to Craigslist? How on earth does that follow?

I understand that it's a little different here because Craigslist has asked PadMapper not to help people out that way. But the information PadMapper is passing on, it's not getting from Craigslist anymore. It's getting it from a third party. So it's the equivalent of me saying, "hey, you know you were looking for a 3-bedroom in that neighborhood? I'm not allowed to visit Craigslist myself, but a dude told me about these pages on Craigslist you might want to check out." There is purely no way that this constitutes dickish behavior toward Craigslist, and Craigslist has no moral right to forbid it.

> I would respect you more if you just came out and said "fuck Craigslist, we're using their data"

The material facts of a listing on Craigslist don't belong to Craigslist. You can't copyright the fact that a given listing is for $2400 a month. You can't copyright that it's at a given street and cross-street and the number of bedrooms and bathrooms on the premises. I don't even think anyone should copyright the aggregate of the above data. To do so would be evil. (In that there would be widespread implications.)

It's not their data. It's their listing. Those are two different things.

Are you saying that Craigslist doesn't own the data, but it still owns the aggregation of the data?

> I even agree with that decision--Craigslist has brought it upon itself by creating what is (IMO) a mediocre and clearly substandard UI. But your choice is what it is. Embrace it.

So are you saying that if CL had a more elegant UI you would be against the OP continuing agasinst CL's wishes? Seems like very arbitrary ethics.

I wanted to explain what I was doing. In a lot of ways I still admire Craigslist, so I wouldn't say fuck them. I feel some remorse for being a dick, but that's strongly outweighed by the other side of the argument for me.

YOU might respect him more, because that's what YOU would say.

But Eric isn't you. He isn't a dick. He doesn't believe he's "fucking" Craigslist. He found a solution that fit with his own moral framework. If this doesn't fit within your framework, that's fine.

The world doesn't revolve around you. So please deal with it and stop calling him names.

>Some people make life better via charity, I try to do it via software.

...wow.

Are you seriously saying that building padmapper is a charitable act? Or am I totally misunderstanding what you said there, because...

...wow.

The central concept of the "invisible hand" of the market is that it leaves both creator and consumer better off.

There's a long line of argument that free entreprise is actually more helpful than charity in many circumstances.

In a sense, building any company is a charitable act, because you leave consumers better off than they were before .

Let's say I invent a hyper-efficient, clean, alternative energy source that averts climate change, makes cheap energy universally available and stops a lot of the causes of war.

Would me also becoming the richest man on earth as a result matter to you? Would it diminish the value of what I would contribute? And if so, ask yourself why that should matter at all?

This isn't some Rand Objectivist thing. I just don't understand why we use words like charity and philanthropy that tie the outcome/value of something in the world to the self-sacrifice of the outcome's author. It doesn't seem all that relevant to most discussions about the value of a thing.

Heh corny, I know, but I think that's the majority of my motivation. Otherwise I would be making a lot more money off of it and it would be a very different site.

"lifetimes saved" is a rhetorically powerful metric. Shitty UI kills - one hour at a time.

Yes, you knee-jerked on a contrasting of what he does with charity. He does claim that his software improves life, and this is his stated motivation in creating it. You may object to the latter?

I don't get what the fuss is about here.

I think that the statement you quoted is explicitly saying that for-profit software isn't a charitable act. Instead, in an implicit comparison to charitable acts, it's saying that for-profit software can also make life better for users.

That was just my immediate reaction. You sound like you're alright.

Thanks. Sorry if Andrey gave you any grief over this, he sort of alluded to that when we were chatting on IM.

Self-interest isn't incompatible with passion and personal moral indignation. You're essentially accusing him, if I read you right, of being disingenuous: claiming he's restoring that data for one set of reasons, when really, he's doing it for another. I don't think it's fair of you to call his integrity into question, and I don't think he's done anything to merit it. I disagree that your reasons, not his, are "all there [are] to say."

Is it really Craiglist's data? When I post to Craigslist, I think of the listing as my listing, not Craigslist's.

Here’s a comparison. You walk into a used instrument store looking for an eight string bass. Another fellow walks in at the same time looking to sell the bass. You wink at each other and meet outside, buy that bass, and cut out the middleman.

It’s his bass and your money, but the reason you knew he had a bass to sell was the store. Likewise, it’s “your” listing data, but the reason padmapper is able to hire someone in the third world to read the internet and manually screen scrape the data—or whatever it is they’re going to do—is because craigslist aggregated it for them.

This is very similar to the kerfuffle over allegations that Bing was scraping results from Google rather than organically indexing the web. It’s quite possibly legal, but it’s not particularly admirable.

And I join Phil (disclosure: He once called me a “giant,” so I owe him) in suggesting that arguments that the ends justify the means are suspect.

I’d prefer to see the padmapper folks put up a pirate flag and openly declare war on craigslist. Just come right out and call them the evil empire already.

> the reason padmapper is able to hire someone in the third world to read the internet and manually screen scrape the data—or whatever it is they’re going to do—is because craigslist aggregated it for them.

Here's where the rubber meets the road. As far as I can tell, 3taps is taking data out of the Google Cache and re-aggregating it. While stealing someone's aggregation of data is clearly wrong, re-aggregating data from public sources clearly isn't, both from a legal and moral sense, at least in the US.

The material facts in each ad are public data. The fact that such an ad appeared on Craigslist with such facts is also public data.

> And I join Phil...in suggesting that arguments that the ends justify the means are suspect.

I'm also suspicious of ends justifying the means, but that isn't the whole story here.

> I’d prefer to see the padmapper folks put up a pirate flag and openly declare war on Craigslist. Just come right out and call them the evil empire already.

To me, Craigslist is the one being selfish and shortsighted here. If they back legislation seeking ownership rights over re-aggregation of data, that would be an "evil empire" move. It would be better for everyone if they subvert Padmapper instead of fighting it.

> I’d prefer to see the padmapper folks put up a pirate flag and openly declare war on craigslist.

Perhaps the best moral decision (perhaps), but probably not the best business decision.

Man in the arena, etc.

Yes: the copy of your listing that resides on Craigslist's servers is theirs.

You are free to post the same listing to as many other sites as you want; Craigslist doesn't demand exclusivity. In that regard, your listing remains yours. You have no other claim to the posting on Craiglist itself.

But he said this doesn't involve touching Craiglist's servers at all. I admit I am drawing a blank as to how this would work, but the (seemingly impossible) description he offers seems to get around that objection.

Yes, but this can't reasonably include the material facts concerning the listing: how many bedrooms, the rental amount, the street and cross street. The data I just mentioned and the relationship between them is not something that Craigslist could reasonably own even if they did ask for exclusivity.

It really doesn't matter what you "think", it's their data, you gave it to them, it's on their servers, and it cost them money to host it.

You probably feel like the stuff you upload to Facebook belongs to you too.

I would've totally appreciated and understood a 'bring it on, we'll fight the good fight' style post.

This however comes across as feeling sleazy, shrugging it off and doing it anyhow.

Sorry, this isn't nearly that black and white.

Ignore this guy, you're doing the right thing, and I thank you.

e: To expand, CL obviously picked on you specifically, and it deprives all the apartment hunters of your service when applied to the largest apartment listing available. I could see it if they somehow filled the void with a comparable service, before they turned it off, but by depriving padmapper access, it makes it worse for everyone.

Be careful reading these comments. There's an overwhelming bias on HN in favor of "rules that impede startups are there to be broken", and, of course, 3/4 of the site is going to cheer you on. Remember when Mark Pincus said early stage startups need (as I remember) to lie, cheat, and steal to get off the ground? HN cheered that on too.

Just remember that every nerd that has ever broken any rule --- from seeding a torrent of a movie to scraping a competing site to (yes, go look it up) exploit SQL injection to dump a company's mail spool to the Internet --- has come up with a personally compelling rationale to do it. MAFIAA, "we're helping apartment hunters", "STRATFOR was an evil intelligence company". These are all of a kind.

The fact is, you compete with Craigslist. However much "better" your service is, customers don't agree: they overwhelmingly prefer Craigslist, because that's where the market is: the "killer feature" in your space is the market.

So, to stay viable in the face of competition that has you beaten dead to rights, you've rationalized cheating them.

I don't find what you're doing detestable, or anything like that, but it sure is annoying to watch you pat yourself on the back for it. I'm with Phillip. Have you read a lot of his comments? I have. He's not dumb, nor is he a troll.

Jim Buckmaster calls himself a communist:

  http://www.craigslist.org/about/jim_buckmaster

  Possibly the only CEO ever described as anti-
  establishment, a communist, and a socialistic anarchist, 
  since 2000 Jim has led craigslist to be the most used 
  classifieds in any medium, and one of world's most popular 
  websites, while maintaining its public service mission, 
  non-corporate vibe, and staff of 30-some.

"Non-corporate vibe", like sending out cease and desists. Buckmaster wants to abolish private property. He shouldn't be against the reposting of public property, then.

... and?

You're right, it does come off as arrogant to say that it's that much more efficient, and that that justifies making this decision. I think that the potential for doing good plays into any moral decision, though, so I don't think it's out of place in trying to explain my rationale. My intention definitely isn't to pat myself on the back here.

"It is difficult to get a man to understand something, when his salary depends upon his not understanding it." -- Upton Sinclair (http://en.wikiquote.org/wiki/Upton_Sinclair)

Nah, it's not morally gray. You guys are a search engine. Craigslist may not like what you're doing, but that doesn't make it wrong. Traditional publishers hate Google, despite all the traffic it sends them. So what?

Getting a C+D from Craigslist is a good sign. It means you matter. Hang in there long enough, and Craigslist won't be able to cut you off, because a good chunk of their traffic will come through you.

It's kind of weird to think of Craigslist as stodgy old-media using the courts to defend an out-dated business model because they don't like being disrupted by new technology, but here we are.

Craigslist has every right to do what they want, regardless of whether or not you think its the right or wrong choice. If they choose to spit out C&Ds without self-improvement, then they knowingly run the risk of long-term obsolescence.

Sure. I'm not trying to give advice to Craigslist. It may well be that legal threats are their best strategy. They certainly have the right to sue anyone they want.

I'm just saying that ericd needn't feel guilty for ignoring Craig's wishes. Do you think Craig feels guilty about destroying the newspaper industry?

> Hang in there long enough, and Craigslist won't be able to cut you off, because a good chunk of their traffic will come through you.

If Craiglist was motivated that way, why would they have sent the C+D in the first place? It seems that CL is opposed to anyone violating their TOS, period, regardless of benefit to CL.

Craigslist dominates apartment rental advertising. Landlords post to CL because that's where the renters are looking, and renters look on CL because that's where landlords post apartments. But CL does have competitors. Lots of sites want a piece of that business, they just can't overcome the network effect that protects Craigslist.

Right now, the overwhelming majority of pads on Padmapper are Craigslist posts. But that's just because the overwhelming majority of all advertised apartments are Craigslist posts. If people start using Padmapper in large numbers, it breaks the symmetry of CL's dominance. Landlords are still posting to CL, but renters are finding those posts through Padmapper. It won't be long before landlords realize that those renters will also find posts on Kijiji or Apartments.com or whatever. When that happen's CL is in trouble, because it's revenue comes from landlords in certain cities paying to post.

CL doesn't care about the TOS per se; the TOS are designed to prevent just such a scenario.

That makes great sense. I hadn't seen that angle.

I've used padmapper and its definitely teh number one choice when apartment hunting. Its not even close.

Rant/ Just keep going man. If you stop doing what you do you will lose the millions(?) of people you are helping.

Seriously other apartment sites just don't get it. They SUCK ... suck so bad that I'd rather bang my head on the table than use them.

Without PadMapper, craigslist is worse than the other sites. I don't understand why they just don't buy you outright - thats what should happen if anyone cares about the effing consumer.

/Rant

Please keep going and doing what you are doing.

-Very thankful apartment hunter.

It depends what your intent is. If you're on the consumers' side, people's side, wanting to make their lives easier, better, through efficiency and therefore increased productivity - then being integrated into everything will of course cause this. Best case scenario would be some form of partnership, however Craiglist seems to want to create and control an ecosystem instead of being the management piece, instead of finding a way to survive as a management piece.

I liked that calculation. I was inwardly cheering while reading it. Quantitative ethics FTW.

You had to be suspicious of this calculation, though, right? It certainly aligns nicely with Padmapper's financial interests.

It actually doesn't align, though. I'm not going to go into detail, but my expected finances are probably higher without doing this. That was an argument against doing this.

Should we be suspicious or admiring?

It was the right thing.

Eric, it's not even close to a grey area. Like...not even remotely close to a grey area.

Consider the following:

I leave a bowl of candy on my doorstep for Halloween with a note that says "Please take one piece of candy.".

You, however, have a bunch of friends back home who aren't at my house to get the candy, so you take the entire bowl. You defend this, citing a "grey area" because your friends want the candy, and you're making them happy because you took it from me and gave it to them.

That's what you did.

No grey area. You just wanted the candy.

(Do I support craigslist's decision? No, I think it's stupid, but it's their stupid decision to make.)

Eric, if you are genuinely doing this for the good of the many and not for yourself, then you would use an alternative that is neither "somewhat dickish" (you're admission, not my judgement) nor somewhat cowardly (You are really just hiding behind 3Taps lawyers) if such an alternative were available. Would you not?

I think you have one: http://news.ycombinator.com/item?id=4220106

Hey, I responded there, but I'm pretty confused about what you're proposing. Email me at eric@padmapper.com?

No profit in it. This is one of those "startups".

As somebody currently looking for an apartment, I wholeheartedly agree with his claim.

It's fine if craigslist wants me to use their competing mapping service, I would even pay them a small fee for it, but if they don't have one yet then I just get fucked. Craigslist isn't being "nice" to me, so as long as padmapper isn't doing anything illegal, I really don't care if they're being "nice" to Craigslist.

That's probably a good summary of the way I feel too - like Craigslist defected first in this Prisoner's Dilemma.

By CL's Terms of Use, it's not CL's data. They just post it.

It's clearly a gray area, but let's be clear: IT'S NOT CL'S DATA. Craigslist actually makes it clear that it's the user's data and that when it's posted on CL, it's being posted publicly.

Closer to black than gray.

It's data that a CL user has entrusted to CL. The user could've gone with PadMapper but they didn't. Being posted publicly is irrelevant, since the user agreed to CL's TOS not PadMapper's.

The direction PadMapper should be going in is to forget about the CL data and do things that make it number one over CL. Stealing CL's clients is fair game and in the long run aren't one CL TOS change from being in the wrong again.

Let's be honest here. User's don't post to CL rather than Padmapper because of their TOS.

1. The average user has never read the TOS and doesn't care about it.

2. User's don't post to padmapper because A) they don't know about it and B) posting to craigslist is (until now) a superset of posting to padmapper and craigslist

3. The data Padmapper is taking is your address and posting time. You'd be hard pressed to find more than a handful of people who mind Padmapper reposting such info.

Besides, it's incredibly difficult to steal CL data for apartments. The posters are incredibly technology unsophisticated; I'm amazed how apartment owners can't take a simple 2 minute video walk-through. A lot of places don't even have pictures!

This does not seem like an important distinction. The user implicitly authorized it to be posted on CL by virtue of, well, posting it on CL. They did not authorize any other sites to post it. Or am I missing something?

I don't think PadMapper puts up the entire post written by the user, just fact that the post exists and is in reference to a particular location. Likewise, it's kosher for me to tell Alice that Bob posted a poem on his window so that she can go read it, even though it would not be kosher for me to make her a copy.

If Craigslist is explicitly disclaiming copyright in the data they post, how can they sue for copyright infringement?

The user may be able to. But the users won't do so, at least not in any significant numbers.

> If Craigslist is explicitly disclaiming copyright in the data they post, how can they sue for copyright infringement?

By having the user sign their right to sue over to Craigslist. They're saying "it's your copyright, but you're giving us the enforcement rights to it". It's in their TOS:

> You also expressly grant and assign to CL all rights and causes of action to prohibit and enforce against any unauthorized copying, performance, display, distribution, use or exploitation of, or creation of derivative works from, any content that you post (including but not limited to any unauthorized downloading, extraction, harvesting, collection or aggregation of content that you post).

I'm copying and pasting from a reply I had below, but to respond: "Although I think this is a bad place to stake your claim, the posts themselves are arguably purely public data the second they hit Craigslist. The data conveyed is factual, there is no IP, and it's being disseminated publicly. It's similar to (and yes, there are a lot of ways to poke holes in this analogy, but for the sake of argument...) a town crier shouting an advertisement out in a crowded square and one of the listeners taking that information, traveling to another public square, and repeating it."

> Christ, what an asshole

Downvoted for ad hominem.

One of the wonderful things about HN is that it provides a forum for civil discussion. If that's your style, there are lots of other places on the web that are better venues for you.

BS.

Get over your analysis - this service is farking amazing and Craig is an idiot for trying to thwart it.

Craig has consistently chosen to sit on his hands and do nothing because he is to afraid to mess with the CL money pot.

Things evolve. CL has not evolved hardly at all and after 10 years + it is hurting people for its lack of change.

How exactly is Craigslist "hurting people?"

More that they're not benefitting their users as much as they could otherwise. (That is as good a reason for any for a website to give up.)

I meant: hurting their experience and wasting their time, not harming people.

Eh, works fine for me.

EDIT: Would you say that HN is hurting people through their choice to stick with a poorly-performing language for serving its website?

Outside Developers like/want the CL customer case, their reputation and their page views and see them as a quick, easy path to success. However, you are not entitled to them. Understanding this and moving on are key to doing it yourself and blazing your own path. Attempting to work around access limitations and usage restrictions and then attempting to justify your actions is only going to end badly.

> Outside Developers like/want the CL customer base, their reputation and their page views and see them as a quick, easy path to success. However, you are not entitled to them.

As far as I can see, Padmapper is not using the Craigslist reputation or page views. The data Padmapper is aggregating is treated as public information by Craigslist's own policy. Given these facts, I don't see what he's doing that's wrong, specifically.

It's not public information. The listing data belongs to the original poster who entered into a contractual relationship with CL -- not with PadMapper. If anything the ads are covered by copyright, held by the person who wrote the specific ad. If the service is so great, then start calling the ad posters and offer them the chance to post on PadMapper. Otherwise, it's theft. We can moralize all we want but reproducing someone's ad without their permission is a violation of the poster's rights, even if some of us feel that it's in their best interest. The choice should belong to the person posting the ad.

> It's not public information.

As far as I can tell, it is public information. Craigslist has a claim on the listing and not the information in it.

> The listing data belongs to the original poster who entered into a contractual relationship with CL -- not with PadMapper.

If the data belongs to the original poster and not Craigslist, then the original poster can have a beef with Padmapper, and Craigslist has no claim.

> If anything the ads are covered by copyright, held by the person who wrote the specific ad.

In as far as Padmapper is using the data in the listing, and not the actual listing, I don't see any violation of copyright. Otherwise, one encyclopedia could sue another one for also publishing the same material facts in an article. If your logic held, there could only be one encyclopedia in the world.

> If the service is so great, then start calling the ad posters and offer them the chance to post on PadMapper. Otherwise, it's theft. We can moralize all we want but reproducing someone's ad without their permission is a violation of the poster's rights, even if some of us feel that it's in their best interest. The choice should belong to the person posting the ad.

I don't think you're taking all the implications into account. By this reasoning, Google bots should only put something in the Google Cache if the original poster gives specific permission.

I agree that reproducing someone's ad without their permission is wrong. However, reproducing the material facts of their ad is clearly not immoral or illegal.

Also, if the first person to aggregate public data effectively has a copyright to that data, there would be widespread and harmful implications.

>then the original poster can have a beef with Padmapper, and Craigslist has no claim.

So what you're saying is that, "The original poster can sue PadMapper, but at least CL isn't going to be a plaintiff."

Unfortunately, you don't know what you're talking about:

From the CL TOU: "You also expressly grant and assign to CL all rights and causes of action to prohibit and enforce against any unauthorized copying, performance, display, distribution, use or exploitation of, or creation of derivative works from, any content that you post (including but not limited to any unauthorized downloading, extraction, harvesting, collection or aggregation of content that you post)."

That clause just made CL the agent for the original poster and it expressly grants CL the right to prohibit unauthorized use. RTFM.

This whole situation is excessively asshole-ish. It seems like PadMapper is hell-bent on scraping CL data by any means (or rationalization) necessary. We can debate about CL running their business the way they do, but this argument is starting to look like some pseudo-intellectual freshman philosophy conversation. It's almost like debating Lyndon LaRouche supporters.

What PadMapper really needs to do is consult a lawyer and figure out how to work within the law rather than trying to be cute about skirting the CL TOU. Any investor would run far away from a company that seems so determined to do something without any professional legal advice instead replying upon some sophomoric interpretation of a contract and the law.

This comment: http://news.ycombinator.com/item?id=4160061 outlines a strategy that could work that doesn't involve playing games and skirting Craigslist. It's also a way to actually build a business based on something besides just stealing other people's data. Of course it requires real work and doesn't appeal to someone that thinks code is the answer to every question.

Data posted to craigslist is subject to the CL Terms of Use. If the data comes from CL, it can't be used without CL's permission. There's no gray area.

if data.from_craigslist? cant_use else use

We can moralize and cry all we want, but their Terms of Use are pretty damned clear and the law is pretty damned clear as well.

Returning to the Google example, it's irrelevant, "Google does it" is not a legal defense or even a moral defense, depending on ones opinion of Google..

They're using the Craigslist reputation and page views indirectly. The information is only on Craigslist because people placed it there, and what was their reason for choosing Craigslist in the first place? Reputation and page views.

By that same argument, Google News is using the reputation and page views of The New York Times and The Wall Street Journal. Google as a whole is using the reputation and page view of the entire web.

Christ, what an asshole.

What? Eric is so obviously not an asshole that your comment is just weird.

As someone who knows Eric personally, I can vouch that he's not an asshole. In fact, that is literally how he thinks and those are his real motivations.

He's not the type to come out and call someone else, like Craigslist's CEO, an asshole. In fact, he comes out saying that he admires Craigslist. That's just the way he is.

> step over the moral grey lines of using someone else's data without their consent by claiming my service is better

OK, IANAL, but isn't the content of craiglist's post owned by the author of said post, not the Craiglist itself? (sorry haven't read their TOS/TOC). Just like you own your tweets, not the twitter?

Otherwise, I could keep posting copyrighted material (like books) and then suing Craiglist for copyright infringement (something Vivendi did to Youtube) since CL would claim responsibility/IP rights. I am sure that CL protect itself from acts like that pushing the blame/responsibility at users (I don't think DMCA would work in each and single case). So one could argue that this is each and every CL author right/responsibility to file copyright infringement claim with whomever harvest CL posts, and CL should, for lack of better words, "mind their own business". In other words, they lack copyrights over the content their users are creating. Sure they can remove the post if it violates TOS, but in the best scenario they could email the author "hey we stumbled upon your CL posting on XYZ website, do you want to file a claim?", but really nothing more than that, right?

I know it sound bit weird, but isnt't it what AirBNB and others do: while your local/state law may forbid subletting, AirBNB hands are clean because they are in business of providing a social platform to find and post vacant apartments, not enforce or monitor if their users follow their local renting laws.

Craigslist is being the RIAA/MPAA in this situation. Users (buyers/sellers) want to remix the same content in a different form, and the RIAA/MPAA/Craigslist won't let them. So then it comes down to whether this can be technologically stopped or not.

The thing about morality is that it's much more subjective than law. If someone does something that is legal (or is presumed to be legal), on what authority do you claim their actions to be "morally grey?"

Cartographers, encyclopedicians, and dictionary aggregators have always included some false data as a honeypot: An alleyway might not really exist. An obscure word might have been made up. A famous person might be entirely fictitious. The goal is to catch people copying their books. If you use “original research,” you’d never say that Braythwayt Street runs parallel to Rhodes Avenue in Toronto, you must have copied our map data without permission.

I’m curious as to how padmapper intendeds to solve this problem. If Craigslist seed their listings with something false, that something is entirely craigslist’s copyright. It isn’t a “fact.” They don’t have a right to reproduce it. Lawyers, please chime in.

http://en.wikipedia.org/wiki/Feist_v._Rural

This is (to my knowledge) the most relevant case regarding facts and copyright. Rural made a local phone book, and Feist made an aggregate phone book which wholesale copied Rural's listings (and presumably others).

In this case Rural did seed their phone book with fake listings as you describe. Feist copied the fake listings along with the legitimate listings and still won the case.

https://www.interllective.com/articles/brian/feist-v-rural-w...

Some related legal precedent, discussion, and doctrines. Note in particular the "Sweat of the Brow" doctrine, which holds that rights may be granted "through simple diligence during the creation of a work...Substantial creativity or "originality" is not required." Looks like this doctrine was rejected in Feist v Rural.

I believe padmapper is kind of relying on 3taps to take the hit. 3taps apparently is DMCA compliant.

Long story short I'd expect 3taps to be craigslist legal target.

(though actually I guess 3taps doesn't actually use the data so I'm not sure if what they're doing is technically illegal until padmapper uses it - I'm going to stop armchair lawyering now)

I'm not sure it matters what 3taps says or whether they are DMCA compliant. If I host pirated movies on my server and post in big bold letters, "I promise that you can redistribute them. It's totally legal and I'm responsible for any legal actions.", you're probably still getting sued by the MPAA if you redistribute the movies, and they'll probably win.

I'm not saying Craigslist has any legitimate claim over the data, merely that if they do, accessing it via an indirect means probably doesn't change anything legally.

I think the idea is that Craigslist's claim over the data is categorically different than the MPAA's over movies. The MPAA has a copyright (by proxy) over the movies. Craigslist has a Terms of Service which they assert are legally binding on anyone interacting with their servers, but (I think) they don't claim copyright over the ad.

EDIT: mikeryan, you appear to be right. Upon further reflection, I think the distinction is that PadMapper does not put up the entire post written by the user, just the fact that the post exists and is in reference to a particular location. Likewise, it's kosher for me to tell Alice that Bob posted a poem on his window so that she can go read it, even though it would not be kosher for me to make her a copy.

Their claim:

You automatically grant and assign to CL, and you represent and warrant that you have the right to grant and assign to CL, a perpetual, irrevocable, unlimited, fully paid, fully sub-licensable (through multiple tiers), worldwide license to copy, perform, display, distribute, prepare derivative works from (including, without limitation, incorporating into other works) and otherwise use any content that you post. You also expressly grant and assign to CL all rights and causes of action to prohibit and enforce against any unauthorized copying, performance, display, distribution, use or exploitation of, or creation of derivative works from, any content that you post (including but not limited to any unauthorized downloading, extraction, harvesting, collection or aggregation of content that you post).

There's an interesting subtext here. The key for CL is this line You also expressly grant and assign to CL all rights and causes of action to prohibit and enforce against any unauthorized copying, performance, display, distribution, use or exploitation of, or creation of derivative works from, any content that you post (including but not limited to any unauthorized downloading, extraction, harvesting, collection or aggregation of content that you post)

I believe there was a case with Righthaven[1] where it was found that the copyright owners can assign copyright but can't assign copyright enforcement rights. I wonder if this is the loophole that 3taps is looking to exploit.

[1] https://www.eff.org/press/releases/righthaven-case-ends-vict...

Copyrights can't apply to material facts. And apparently, there is no "database right" in the US either, so they don't have a claim against re-aggregation of their data from public sources.

That's not the goal, no. The point there is that neither PadMapper nor 3taps interact with their servers, so neither has to implicitly agree to their ridiculous TOU in order to summarize the listings. That's the only reason for that.

I don't believe the methodology used to collect the data actually matters does it? The question is fully around the copyright ownership of the content.

I can record TV content from an over the air broadcast and never touch a thing and still not have the right to retransmit the content.

I should have made it clear in the post (I just updated it), but their complaint wasn't a copyright one, but a TOU violation one. PadMapper isn't displaying the listings themselves, it just lists #BR, #BA, Price, and a few other things, and links you to the original to read more.

> I don't believe the methodology used to collect the data actually matters does it?

Actually it does, from both a legal and moral standpoint. It shouldn't be wrong to re-aggregate data from already public sources. That way lies insanity and oppression.

> The question is fully around the copyright ownership of the content.

Actually, it's around ownership of the material facts in the content. Those can't be owned, at least in the US.

You're still violating Craiglist's license and TOU by scraping a copy of the website. Legally, it doesn't matter who hosts the site (or else hosting a site on CloudFront or Amazon would negate copyright protections). The site remains the property of the owner, and use of that site remains under the control of the owner.

It does not matter that you're only using "facts", because those "facts" are data specifically collected by Craiglist pursuant to a license with the original poster. The selection of relevant facts is protected by copyright law in the U.S. As a licensee, Craiglist has the right to enforce this copyright against you if it chooses. Morever, by violating the TOU, you are either directly breaching a contract with Craiglist or committing tortious interference in the contract between CL and the poster.

Either way, you do not have a legal leg to stand on. If Craiglist goes after you, you will be screwed. The first time around, your actions were harmless. But this time around, you are knowingly and deliberately violating Craiglists' rights. That opens the door to punitive damages. Judges and juries do not look kindly upon people who violate others' property rights, especially after being told not to.

Being a "startup" is not an excuse to break the rules or fuck with other people's rights.

"Legally, it doesn't matter who hosts the site"

False. Terms of use and copyright are orthogonal. You can't enforce terms of use on someone who doesn't use your site. Copyright would still be enforceable if Craigslist held the copyright.

"As a licensee, Craiglist has the right to enforce this copyright against you if it chooses."

I know of no legal precedent that backs this up. Craigslist claims this right in its terms of use. I think it's bullshit to scare people away from doing what Padmapper and 3taps are doing.

"Morever, by violating the TOU, you are either directly breaching a contract with Craiglist or committing tortious interference in the contract between CL and the poster."

Neither of those claims are true. The terms of use don't apply to someone who doesn't use the site. Neither Padmapper nor 3taps are causing posters to breach their contracts with Craiglist.

1) Padmapper and 3taps are using Craiglist's site; they are simply used a copy of the site. That difference is not relevant from a legal standpoint given their specific use.

2) It's basic contract and copyright law that a licensee may defend a copyright if necessary to protect their license. See Righthaven, which came down to this very issue: Righthaven was prevented from asserting newspapers' copyrights because it did not have a license to the newspapers' content.

3) You have this backwards. Padmapper and 3taps are (a) the ones breaching the contracts between themselves and Craigslist (see 1) and (b) are interfering in the license agreements between Craiglist and the posters. The posters are not breaching anything.

Don't be an armchair lawyer.

IAAL.

Both 3Taps and Padmapper are potentially liable in this scheme. DMCA compliance does not protect this type of use; the safe harbors are very narrow and specific. It only protects websites (or services) from liability for content posted to the website directly, by users of the website (or service). The DMCA does not protect the website (or service) from content it actively acquires, on its own or on behalf of another party.

3Tap is scraping the Google Cache under some misguided interpretation that the copy of the Craiglist site is not protected by copyright. This is as ridiculous a legal interpretation as Carreon's charges against the Oatmeal for defamation. Google has fair use rights to maintain a copy of Craiglist's site in its cache, but ownership of the content remains with Craiglist. This is no different than if the site were hosted on Akamai or CloudFront servers instead of Craiglist's own servers.

Copyright liability does not stop with the first infringer. Every infringer is potentially liable (which is why the DMCA safe harbors were necessary in the first place).

> IAAL.

Then you should consider the following.

> 3Tap is scraping the Google Cache under some misguided interpretation that the copy of the Craiglist site is not protected by copyright.

Yes, but is 3Taps presenting a copy of the posting or the material facts contained within the posting? The former would be a copyright violation. The latter should not be. I cannot tell without delving into the SDK. I take it you didn't consider this.

I did consider this. Facts are not copyrightable, but unique selections of facts are copyrightable. This is a basic tenet of the copyright decisions regarding databases and phonebooks.

Moreover, even if the copyright does not rise to the level of a copyright violation (if the facts copied are not sufficiently unique -- that would require a look at the CL fields, the 3Tap API, and Padmapper's use), it would still be a TOS violation, which courts have held are enforceable especially against commercial entities.

The US doesn't have database rights, as far as I know. Also, since the Craigslist site isn't visited in the process, there is no TOS violation either.

"ownership of the content remains with Craiglist"

Ownership of the content remains with the person who posted the listing on Craigslist. Craigslist has no recourse via copyright law.

Can you provide a source for this assertion? The terms of service on their website has the standard "you automatically grant us a license so we can distribute it" clause, but then has this additional term directly following:

I don't think such a grant is legally sound, but I'd like to hear evidence to support it. Here's a similar example where the court did not grant the third party standing to sue: https://www.eff.org/press/releases/righthaven-case-ends-vict...

Righthaven, as you pointed out turns on this very issue: Righthaven was not able to enforce someone else's copyright because Righthaven did not have a license to use the copyrighted material. The Righthaven court found that the "license" Righthaven had was merely a "license" to sue for damages, but Righthaven was not actually allowed to use any of the newspapers content.

Craiglist is different. Craiglist has the right to use the copyrighted content of its posters. It is a basic tenant of copyright and contract law that Craiglist therefore has the right to enforce the contract to protect its own license rights.

Honest question: would a client-side implementation be OK? As in, the only person making any requests to the CL servers is the end-user, and then a browser extension/app maps the listings? Is it wrong to automate calls to CL's server from javascript in a local app? If no one else is doing a client-side padmapper, I sort of want to myself.

Along the same lines, how does a company that takes another's data protect its own? If padmapper takes craigslist data under the genuine opinion that it is fully legal for them to do so, can I go ahead and copy all their data? Craigslist couldn't sue me because I am siphoning the data off padmapper's servers, and padmapper couldn't sue me because after all, it is all totally legal and super duper helpful to the end users.

Along the same lines, how does a company that takes another's data protect its own?

If your company has to "protect its data" but in fact, it's really the user's data: this should be a red flag.

Poor google, facebook and craigslist. I wonder if those 3 will ever make it, what with the giant red flag all over their business model.

It's extraordinary, but yes, I mean that.

Google doesn't try to "protect" its data. Facebook and Craigslist do. Yes, there is a red flag on the latter two companies. They have to actively exert themselves to keep their network-effect advantage. Insofar as they do this by generating value for users, they will prosper. When they act against user interests, they act against the preservation of their network advantage. If they go too far or do too little, they will fall.

Google definitely tries to protect their data: if you try to scrape any of their APIs, whether it be for Maps, +, or Search, you will find yourself rapidly hitting quotas. They are very good about denying access to things that don't look like humans, and in cases where the data was extracted using horrendously indirect means (Bing) they resorted to public shaming.

I believe that technique (called 'map bunnies' in the mapmaking domain) has been used to discover copiers, but that subsequent legal/social action against the transgressors is not based on the damages caused by the bunny-copying itself, but the now-detected larger pattern.

IANAL, but I would expect the copier-of-fake-listings to have a number of defenses in such a situation.

It would be hard to prove any real damages from the copying of just those fake listings – any loss of business is due to the copying of the bulk of other real listings. To the extent those other real listings are 'facts', reproducing them may be a protected activity under US legal precedent ('Feist'). The false items don't change the key issue at all. (Statutory damages might still be a problem.)

Further, consider:

• Someone couldn't intentionally hoax a newspaper and then sue the newspaper for having reproduced their creative work of fiction.

• If I tricked you into downloading a file labelled "public-domain video of the 1997 state-of-the-union address", which is actually the 'West Wing' pilot episode, it'd be hard for me to collect damages for copyright violation.

To some extent, passing off material in the form of something else – where that something else would otherwise be legal to disseminate – can easily be considered implicit permission to treat that material as if it were the first thing (true facts, hot news, copyright-unemcumbered information, etc).

Injecting the false data still serves a 'detection' purpose but probably can't serve a 'damages-heightening' purpose. (And, detection isn't at issue in this scenario: PadMapper has announced their intent and technique publicly.)

> Someone couldn't intentionally hoax a newspaper and then sue the newspaper for having reproduced their creative work of fiction.

Your point is cogent, but the example is not. if someone hoazes a newspaper, the are encouraging the newspaper to “copy” them. Craigslist is not hoaxing 3taps or padmapper, just the opposite. In my conjecture, they’re sending C&D letters saying “Do NOT copy our data. Although most of it is not copyrightable, some of it is and cannot be copied from our web site or API. Get your ‘facts' from original sources.”

That is very different from hoaxing or inducing someone to copy and then yelling “gotcha!"

Not a lawyer, but I would be really surprised if that sort of copyright shenanigans held up against any legal challenge. I don't think you can sneak copyrighted stuff into non-copyrightable stuff in order to protect it.

I don't actually think that's relevant.

The point of the "fake streets" on maps is to prove that the rival map maker copied data rather than driving all the roads themselves. Here, there is no doubt that the data in question is originally sourced from Craigslist. There is no need to prove it.

Oh, short story idea there. Someone claims to live in the made up address and generates an identity complete with utility bills... Neal S here we come...

Fred Saberhagen used the concept of a map bunny as a plot device in one of his Berserker short stories.

Can 3taps get me the content of the NY Times website via indirect means, so I can publish it on my website? I did a back of the envelope calculation that a lot of people are losing time by having to go somewhere else to get that content, and it's pissing me off.

This may seem like a good analogy, but it's not. NYTimes owns their data; that's their intellectual property and no one would contest that. The posts on Craigslist are, if anything (see last paragraph), the property of the individual posters, not Craigslist. In fact, Craigslist makes this clear in their Terms of Use and does so largely for legal reasons: they don't want to be liable for potentially illegal posts.

Now, it's definitely a gray area to repost the individuals' content on another site, but what Eric is doing is clearly in line with the intent of the original poster: distribute a listing to get eyeballs. I'm making an assumption here, but I'm pretty sure that he would pull the post if the individual requested as much.

And finally, although I think this is a bad place to stake your claim, the posts themselves are arguably purely public data the second they hit Craigslist. The data conveyed is factual, there is no IP, and it's being disseminated publicly. It's similar to (and yes, there are a lot of ways to poke holes in this analogy, but for the sake of argument...) a town crier shouting an advertisement out in a crowded square and one of the listeners taking that information, traveling to another public square, and repeating it.

It's obviously not OK to post the content of the stories on your site, but you're totally within your fair use rights to write something ABOUT their stories.

> It's obviously not OK to post the content of the stories on your site, but you're totally within your fair use rights to write something ABOUT their stories.

Yes, a better question would be, "Is it okay to index the stories on your site?"

If it's not then Google and all other search engines are wrong.

Only if the NY Times publishes their content in a fashion that makes it hard to read, undiscovered and freakishly difficult to use in any useful manner, and your site makes it readily discoverable and very useful.

As a content creator, I would prefer you bring the issues to my attention, not subvert my protections under copyright law. You don't get to do as you wish with my creations just because you don't like how they're displayed. It doesn't matter whether we're talking about the NYT or some random blog.

Unless Craigslist isn't subject to DMCA they do not own the content. And here is their DMCA page: http://www.craigslist.org/about/infringement.claims

They can, but you didn't say 'fuck' enough.

You're right, I shouldn't curse/write when angry, edited.

I don't understand how CL can even exercise this kind of control. I'm not a lawyer, of course, but it seems reasonable to me that apartment listings would be considered "facts", just like baseball statistics.

So then the CL terms of use are the only obstacle, but can you really be held to those if you didn't read them and weren't somehow forced/encouraged to do so? Perhaps so, but then I would find that a little scary.

Just seems like a bad situation all around, but in my mind Padmapper has the moral (though perhaps not legal, again, not a lawyer) high ground, at least relatively speaking.

Let me try. What you say is correct. The listing is "fact". And CL is not preventing posting this fact to other services. The user can post the same listing to CL, Rent.com and dozen other sites. What they are controlling is access to their servers.

no its very simple. Don't submit your listing to craigslist them. Submit it to a site that is indexable friendly.

I can't help but think you played this poorly.

It sucks CL isn't going to let you get their data, but I really don't think this workaround is going to change anything (although kudos for the neat Google cache hack from 3taps).

I imagine your development efforts would be better spent by leveraging the small but dedicated following you clearly have. There must be ways to take advantage of it without opening yourself up to more litigation.

You may be right, I'm flying by the seat of my pants here (after having talked to some lawyers). My goal isn't to supplant Craigslist, which is what you seem to be suggesting - that's far too big a task to take on, even with my sites' substantial usage, and not really something I'm personally interested in - I think that classifieds are a mostly-solved problem, with the exception of reputation and security. I think CL can get there, they seem to be hiring devs.

Yea it sucks your dealing with CL and not Facebook or you probably would have been acquired.

I say try and move the traffic to pad. It's certainly a big task, but not "far too big". Pick the region where you have the most significant traffic and start there. I haven't looked deeply into your site at all, but I know you have Garry Tans support, ask him what you need to do to get in YC. Get "Craigslist is the Killer" as a headline on TechCrunch. Piggyback the Padlister browser add-on with other common downloads, then push a popup to users posting apartment listings on craigslist saying "Also list on Padmapper?" There are all sorts of things you can try.

There are hundreds of people on HN that wish they had the traffic you have. Use that momentum before it dies off. http://siteanalytics.compete.com/padmapper.com/

I think if you look up "network effect" in the dictionary, it's going to talk about Craigslist. Good luck overcoming that.

A great strategy here would have been to establish a strong relationship with the people at Craigslist from the beginning. No idea if that was tried or not. Seriously, was the C&D letter the first communication exchanged between Padmapper and CL? What a missed opportunity. I wonder if this entire disaster could have been averted by calling the CL guys up early on and taking them out to lunch. Make friends, send them Christmas cards, then when this disagreement comes up you handle it over a round of golf instead of a C&D letter.

Can someone provide more detail on the cache hack? Surely this is going to be an unreliable way to grab the information, considering the speed the cache is updated at?

Also, I'm pretty sure Google's TOS doesn't allow you to do automatic queries and/or scraping.

Yeah, it's impressive that Google hasn't shut down 3taps...I think Google once shut down my IP's access after just 3 automated queries.

I think everyone should just step back and look at the big picture.

This guy is finding any means necessary to add Craiglist data back into his application and Craigslist does not want him to. I think the "gray" area is pretty black and white. Craigslist said no. He's doing it anyway - even if indirectly.

Here's how I see it:

Me: Don't go in my house.

You: But you left the back door unlocked by accident, so I thought I'd just come in and eat your food.

In the real world this would not be acceptable.

Even though we are talking apples, bananas, and oranges, the same ethical perspective should be employed. Craig said no, so we respect his wishes. It's the RIGHT thing to do. If he adds an API someday, then he's sharing the key to his house with us, until then. Doors are locked, too bad, move on.

I think you need to take one more step back. Craigslist doesn't own the listing data on their site. They even say so and for a VERY important reason.

If they own the listing data, they're responsible for it's content. So, in some cases, they'd be responsible for selling sex, drugs, and stolen property. Not to mention responsible for a murder or two.

As soon as they claim they own the user generated content, they'll be opening themselves to thousands of lawsuits for their complicity in thousands of illegal acts.

Reference: http://en.wikipedia.org/wiki/Section_230_of_the_Communicatio...

I'm not talking about who owns information.

My point is that company A is asking company B not to use the content that was submitted directly to company A's site on company B's site.

Company B is finding a round-about way to do it anyway.

In my opinion, this is the wrong thing to do. They should respect the wishes of company A and just not do it because they were told/asked not to.

I believe that just because you "can" doesn't always mean you "should".

So? Microsoft, Autodesk, Dell, EA, etc. would prefer you not be able to sell used copies of software or hardware. As long as it's legal, what's wrong with doing so?

I agree that legal is not equivalent to right, but in this case, I don't see the moral argument against this. Everyone benefits from this; the overall benefit to society is positive.

In my opinion, this is the wrong thing to do. They should respect the wishes of company A and just not do it because they were told/asked not to.

So it's wrong to jailbreak your iPhone because Apple would prefer that you didn't, even though it's explicitly legal?

It it invalid to apply analogies involving physical property to intellectual property. The two are not similar enough for that to be a convincing argument, especially since mere facts aren't protected intellectual property at all.

http://en.wikipedia.org/wiki/Feist_v._Rural

This seems to always be the reply when analogies come up, which is why I said it was apples, bananas, and oranges.

The point being, what's mine is mine, and if I say you can't have it, I want you to respect my wishes. Sure you can probably justify endlessly, or interpret differently, or stretch the truth, or bend the rules, but one business is saying, don't use our data this way, and regardless of this, the other business is finding a way to use the data.

Seems wrong to me.

EDIT:

Because I can't reply deeper to these threads, I have to edit here.

The "mine" above is not intended to be in regards to the information or content on Craigslist. I'm not saying they own it.

I think there is a relationship between the person who posted the information to Craigslist and Craigslist itself. I don't think anyone has a right to become some part of that relationship without both sides agreeing.

Maybe it's because I'm a freelancer. If I have a relationship with a Client to build them something, and someone else decides, of their own accord, that they are now part of that relationship, I have the right to tell them to back off, and I believe they should respect my wishes.

I'm not talking data ownership. I'm not talking about legal matters. I'm saying, human to human, they should just be cool with Craigslist wishes and go another way.

"what's mine is mine"

Craigslist postings aren't Craigslist's in a legally or practically meaningful way. This is the crux of the matter. The owners are the posters, and few if any of them would object to increased visibility.

> The point being, what's mine is mine, and if I say you can't have it, I want you to respect my wishes.

As far as I know, Craigslist doesn't own the facts contained in a listing, nor does it have a claim to the aggregation of such facts, so Craigslist doesn't have a legal or moral claim here.

Yes, many people see it this way, while many see it as black and white in the other direction: CL doesn't own the content on their site (this is a fact) and thus has no say in how others use it (debatable).

Mix black and white together, and you get... gray!

If I throw diamonds from the street onto your front yard, then I just walk away and leave forever, who owns those diamonds? Can your neighbors storm your yard and take them, or are they yours?

Can you now defend your property and keep people away from those diamonds?

If they get them, can you fight to get them back? They were taken from YOUR property after all.

What if I threw them right into your living room?

I'd say a more accurate analogy is like this:

CL: Don't go into my house

You: But you let Mr. Google make an exact copy of everything in your house, how about if I use that?

Mr. Google: Help yourself.

That is exactly the issue in a nutshell, even though the armchair lawyers on HN don't want to believe it.

Craigslist said no. The user that posted the data to Craigslist did not give Padmapper or 3Tap permission to copy their listing. 3Tap and Padmapper did anyway. The Google Cache, for legal purposes is Craigslist's site if it is a cached copy of the Craigslist site. Google has a legitimate fair use argument that the cache is directly related to its primary non-infringing use of search, which is why it gets to cache the site. Google does not get any copyrights to the cached sites; the copyrights remain with the owners.

Thus, the problem remains that 3Tap and Padmapper are violating Craiglist's copyright and its TOS. They could also be violating state and federal laws.

You're incorrect. There are two related issues here: access to Craigslist and potential IP issues. If Craigslist tells you not to access Craigslist, presumably there is some sort of legal remedy if you continue to do so. As a result, Padmapper no longer accesses Craigslist. Neither Padmapper nor 3taps access Craigslist, so this is not an issue. If Google told 3taps to stop accessing their cache in that manner, they'd probably have to stop.

As far as IP goes, Craigslist does not own the copyright to the posts on their site. The posters do. Even if Craigslist wrote all the postings themselves, the factual information (price, number of bedrooms, location) could still be copied because facts are not protected intellectual property.

I don't see how you're any less of an armchair lawyer than we are, and to top it off, you're wrong. (EDIT: Oops, I've come across another one of your comments where you say you're a lawyer.)

> I don't see how you're any less of an armchair lawyer than we are, and to top it off, you're wrong. (EDIT: Oops, I've come across another one of your comments where you say you're a lawyer.)

Considering that, I would expect a more detailed analysis.

I'm not being paid for this analysis. I'm briefly popping in during break periods.

I'm presenting a basic overview of the facts and current state of the law. I limit my comments to what I can type during my breaks. Consequently, my analysis is spread out over multiple comments. If you want a more detailed analysis with every comment, you can pay me or wait for a lawyer with plenty of free time on his hands to summarize things for you.

The founder of 3Taps posted this on Quora last year, in response to a question about disrupting Craigslist:

... the postings in question are public facts about exchanges. Just as any price/supply/demand in a marketplace is open for any and all to notate and republish, so too is the entire set of Craigslist data -- as these offers between seekers and providers are clearly in the public domain. Historically, Craigslist has attempted to block access by others to the comprehensive use of this data. They block many 3rd parties who try to gain access to the data, and sometimes threaten to sue and bankrupt others as if they themselves created the underlying data and hold copyright like property rights over the same.

But public facts are public property. And while some think that predatory Terms of Use demanding that you hand over the Brooklyn Bridge in liquidated damages if you don't comply with some obscure (and potentially constitutionally void) constraint will stand in court -- such absurdities will break if exposed.

A fear, uncertainty,dread approach over access to data breaks down in a world where Google already indexes all of Craigslist data and caches that information all over the internet (for search performance results). If its possible and legal for Google, then why not for any and everyone else to also index and offer access to the same data. In short, Google doesn't get special secondary property rights to privatize public data to the exclusion of anyone else. Equal access to exchange data and search data is a principle in parallel to the notions of net neutrality.

The points above are not a theoretical discourse. Look at 3taps.com/developers to see the execution of this concept. And look at what a 3rd party application (craiggers.com) can do in recreating the whole of Craigslist in a format that gives access to data in a way that is not remotely possible in the legacy Craigslist offering. Craiggers is a perfect example that the function of displaying Craigslist data (rather than gathering it) is a totally distinct (and competitive) marketplace, even if there are still huge network effects in the gathering of Craigslist postings.

Note, Craiggers does NOT disrupt the existing Craigslist revenue model for Craigslist. It simply opens up the field (along with any other developer building on 3taps assisted access to Craigslist data) that wants to build on top of (rather than compete with) the network effects of Craigslist. Think Kayak and Indeed, but now for the whole body of data covered by Craigslist accessible, rather than just a single vertical.

http://www.quora.com/Craigslist/Why-hasnt-another-product-di...

His post is chock full of BS:

> If its possible and legal for Google, then why not for any and everyone else to also index and offer access to the same data.

Because Craigslist isn't suing Google. If they didn't want Google indexing their stuff, Google would comply.

> Google doesn't get special secondary property rights to privatize public data to the exclusion of anyone else.

They can and do. That's why it's beneficial to set your user agent to Google Bot when browsing news sites.

> Equal access to exchange data and search data is a principle in parallel to the notions of net neutrality.

Full of shit. It takes effort to provide the service that Craigslist does. Claiming that you have a right to that data is wrong, and moreover has already been decided by case law (you can find links in other comments on this page).

> And look at what a 3rd party application (craiggers.com) can do in recreating the whole of Craigslist in a format that gives access to data in a way that is not remotely possible in the legacy Craigslist offering.

There is a reason Craigslist doesn't make that data available - it would reduce the utility for sellers of their website, reducing their revenue and potentially drying up their business. Trying to squeeze it out of Google's cache is still copyright infringement.

> Note, Craiggers does NOT disrupt the existing Craigslist revenue model for Craigslist.

Yes, it does. By scraping craigslist in an attempt to undermine their platform you are eliminating their site's relatively utility for buyers, which eliminates the impetus for sellers to list there.

All of your points rely upon the assumption that Craisglist "owns" all of the posts submitted. I'm not saying that's right or wrong, but if that is true then wouldn't that extend to Facebook owning all content submitted to their service, Twitter owning all tweets, Flickr owning all hosted photos, and Stack Overflow owning all submitted answers?

Craigslists owns the unique compilation of their listings. That's what is at stake here.

And there's no copyright infringement as long as your compilation, based on their publicly available data, is also unique, which PadMapper's is. This has lots of precedents dating back to services derived from phonebooks.

I am interested in this. Can you point me to one or more of these precedents? Thanks in advance.

Is it?

In my interpretation it is the content of the listings, not the compilation. He could compile the data from several different sources and present the same result.

What constitutes a "unique compilation" is subject to interpretation in a court of law.

Adding/removing/modifying and changing the arrangement or display of items in a dataset sufficiently constitutes a dataset unique from the one Craigslist offers even if it is largely derivative from the Craigslist dataset.

Craigslist could argue a trespass to chattels tort or file a ToS civil suit, but there isn't much they can do to protect a dataset.

The reason Facebook makes Facebook content largely available only to those who are logged in is to hide behind their ToS and prevent scraping whether centralized or distributed.

As far as I know, in the US they don't.

So what you're suggesting is that if a service put "noindex" in its robots/metatags, they would be somehow be overstepping the bounds of what they can do with their users' content?

Not at all... that's a bit of a strawman since robots.txt/metatags are used by search engines and not the public in general.

Not wanting your data on Google != Denying access to said content

> By scraping craigslist in an attempt to undermine their platform you are eliminating their site's relatively utility for buyers, which eliminates the impetus for sellers to list there.

This is exactly what Craigslist will argue, but it's hard to imagine how that could be true if the eventual downstream destination of the data always sends the user back to Craigslist to finish the process.

I really don't get the whole argument and all this Craigslist bashing. It looks like someone built his house on the grounds of somebody else's ground and now tries to rally up the internet because the owner of the land found out about the illegal house.

Is it not just like asking "why doesn't Google, Ebay etc let me scrape their database"? Just because someone found a nicer way to display, sort, relate the data, does give them neither a legal or nor a moral right to use the data.

The startups who are Craigslist bashing just have a vested interest in getting access to Craigslist content without any ToS. It is more like demanding that a property open up because there is gold on it and they are demanding that it be made available to them because they can serve gold to the market better than the land owner. From what I am reading though, the businesses are offering nothing in the way of compensation and this is more of a property grab.

I'm not disagreeing with you. I'm disagreeing with the foregone conclusion that a more liberal ToS would spell doom for Craigslist. A nicer way to display, sort, relate the data doesn't confer rights, but--as long as the end destination is Craigslist--I don't think providing access is a necessarily a losing proposition for either party.

It doesn't matter. It's a copyright issue. Craigslist has the right to protect their copyright, it can be proven that they have a unique database, and it can also very likely be proven that the scrapers knew they were doing something illegal (e.g. willful infringement) when the scraping took place.

I agree, it doesn't matter, but I think the reasoning is probably wrong.

It would have been better for everyone if Craigslist gave away the data minus whatever is needed to finish the transaction, and include in the TOS that that you must send the user back to Craigslist to actually finish whatever they're trying to do.

It doesn't matter. It's a copyright issue. Craigslist has the right to protect their copyright, it can be proven that they have a unique database, and it can also very likely be proven that the scrapers knew they were doing something illegal (e.g. willful infringement) when the scraping took place.

As far as I know, there is no database copyright in the US.

And you'd be proven wrong by a simple google search on "database copyright"... first hit [1] shows a compilation of laws and court rulings that support copyrighting collection of records... and this stuff has been around for a decade or more.

[1] http://www.bitlaw.com/copyright/database.html

Like Amazon's and Yelp's reviews, Craigslist would probably claim copyright on the posts themselves, not just the aggregation thereof.

> They can and do. That's why it's beneficial to set your user agent to Google Bot when browsing news sites.

Jonathan, you can't be serious about this one. Why on Earth would you go on a public record and suggest someone to illegally impersonalize another company? Don't you know it may be a jailable offense, if Google (or any other company in that matter) decides to go after you??

> It takes effort to provide the service that Craigslist does. Claiming that you have a right to that data is wrong [...]

It is equally wrong to claim you have no right, just because it takes effort to provide Craigslist-level service.

> There is a reason Craigslist doesn't make that data available - it would reduce the utility for sellers of their website, reducing their revenue and potentially drying up their business. Trying to squeeze it out of Google's cache is still copyright infringement.

Uhm, providing one company data in a different template would reduce said company revenue? Excuse me, but under which rock have you been living for the last 20 years? You ever heard of "social sharing"? "like" button? Does "API" ring any bell?? Why do you think most companies provide those?? Most would die for other developers to actually spend their time and effort to build front gates based on their data. I hope you are not working on any startup bro, because you seriously have a shitty point of view!

> Yes, it does. By scraping craigslist in an attempt to undermine their platform [...]

What do you mean by "undermine their platform" ?? Why would you automatically assume that all the OP wants to do is to kill or undermine someones business?? Further, he would have to be retarded to do so! Why would he work on a project that the core is based off of external data and the same time wanted to... kill that data source?? You lack logic here, again.

> By scraping craigslist in an attempt to undermine their platform you are eliminating their site's relatively utility for buyers, which eliminates the impetus for sellers to list there

Yes definitely -- A website where I could see all the pictures of furniture posted on Craiglist on one page which gives me an easy access to scroll down in less than 2 minutes (to actually see what I want to buy), versus spending 45 minutes clicking on each and every post on the original Craigslist website -- yes, definitely because that evil website stealing Craigslist data made me save 40 minutes I will never use Craiglist as either buyer or seller ever again.

What a bullshit!

Holy shit, is this satire? If so, this is a perfect imitation of some of the commenters who go around here pretending that the entire world agrees with their minority viewpoint that neither property nor effort have any established value.

If this isn't satire, well all I can say is i hope you build something awesome one day. I presume your above manifesto can be construed as an invitation to insert myself between you and the userbase you invested so much to accumulate?

If I have a product and a set of users, and you have a way of increasing both my revenue and my users satisfaction by inserting yourself between us, please do. The OP's point is that undermining Craigslist makes no sense for Padmapper, since that's where the data comes from. If people who list apartments get them rented faster because of Padmapper, they like it. If people looking for apartments find them faster because of Padmapper, they like it. If it increases Craigslist's revenue, they like it. If all of these points are positive, it's hard to understand why Craigslist would be against it.

Please recognize the distinction. I'm not saying they're outside their rights to deny this access or that Padmapper is somehow within theirs by violating the ToS, just that if it's a win for everyone, shutting it down doesn't make sense.

Disintermediation -- getting your users used to coming to my site to look at your data is the first step to making you irrelevant and forgettable. PM knows who the renters and the listers are; it's a small step to convince some listers to list with PM first. Maybe PM will agree to repost on CL as well, but as long as listers come to PM first, the relationship between PM and CL is flipped.

Eric might protest that all he wants is to save apartment hunters from minutes of work. The reality is that PM's long-term viability depends on him getting acces to some of that listing fee cash, which almost defintely means reducing CL's revenue. Unless you think he's going to convince agents to pay twice to get in front of an overlapping set of viewers.

It sounds great on paper, but it's not going to stop Craigslist from C&Ding all the developers using the 3Taps API. IANAL and not sure they even could indemnify their customers, but I know I'd want more than a reasoned argument and bold words from my upstream provider if I were to build a business on the 3Taps-provided Craigslist API.

I know the 3Taps people pretty well. They're seasoned entrepreneurs and they definitely know what they're doing with the 3Taps API. One of the founders is a grey haired attorney who's thought this through pretty carefully. They're also prepared to handle any legal issues that arise since they need to be, but also because it would be the best PR ever.

As gray hair as they might be, they come across as rather morally bankrupt and I can't imagine this ending remotely well for anyone in this industry.

Either Google will have to begin applying constraints to cacheing, or the websites will have to apply constraints to Google indexing, or the websites will have to enter into an arms race with the likes of 3taps.

The end result will be a great deal of wasted money, and if 3taps is actually successful, the decimation of the businesses whose data they rely on.

Then what? Is 3taps going to build another Craigslist, for free, and open up their data silo to the internet?

I think this is great. Ownership of user contributed content is a big open question. Now everything is becoming social and crowd-sourced. If this pushes the question to the forefront so an actual judgment is made it will put businesses on some sort of solid ground. Right now a lot of people are building on potential quicksand.

I like flickr's approach. Users own the content and define the copyright. Flickr provides an open API and a TOU which makes it tricky and/or illegal to source the content directly from their servers.

If a billion dollar business relies on people giving them content for free and them keeping the copyright and re-selling the content, then I think maybe they're the morally bankrupt ones.

APIs are everywhere now and API use is becoming quite a big sticking point. I think content API platforms are the future of the Internet and a lot of the big names (Facebook, Yelp, Twitter, Foursquare, etc.) But yes, someone needs to figure out how to monetize being a content platform (in a non-Wikipedia way). Assuming you have the right to take the content, resell it and own the copyright seems like a weak assumption that has only worked so far while people were less savvy about it.

Yes, but are they prepared to handle everyone else's legal issues too? Is doing so in the agreement between them and Padmapper? I'm sure they're great guys, but that doesn't obligate them to protect Padmapper from Craigslist.

Or more likely CL will just tell Google to stop caching their pages.

The argument hinges on Google getting access to Craigslist listings while others are denied. However, that doesn't seem to be true:

http://www.craigslist.org/robots.txt

https://www.google.com/search?q=site%3Aseattle.craigslist.co...

Update: The comment you quoted is a year old and Craigslist only recently blocked Google. http://www.seomoz.org/blog/craigslist-blocks-search-engines Perhaps they decided to block everyone to bolster a legal case against aggregators like Padmapper.

But public facts are public property.

Are they though? In the absence of any terms that specify otherwise, I would expect to retain copyright over anything I post to Craigslist. To play devil's advocate, maybe I wrote a literary masterpiece or uploaded some great photos trying to sell my [random item] - just because I put them on Craigslist doesn't mean I'm releasing them into the public domain. For whatever reason, I may want to retain control over my post, and only give permission to Craigslist to use it.

To take this "public domain" argument to the extreme, does this mean any pictures I put up on Craigslist can be used royalty-free by anyone for any purpose, forever? That's absurd.

Edit: most of the arguments here seem to be around whether Craigslist owns the listings, but I think that's missing the point. The user owns the listing, and none of the aggregators have any way to get the users' consent before reusing their listings. In most cases, yes, of course I'm happy to have my listing show up on aggregators and increase its exposure, but that's not really the point - the point is end users have no control over it.

> does this mean any pictures I put up on Craigslist can be used royalty-free by anyone for any purpose

I think what he's saying is, that the fact that you are publicly selling your picture is in the public domain, a public fact about the economy, and unmonopolizable by any one private subject.

They can be linked to royalty-free forever, because the "publisher" is the server hosting the data.

Even if they did copy and host the image themselves, they would be liable but still be okay because the burden is on the user who owns the data to file a DMCA takedown notice for that content. The worst Craigslist could do here is inform the users (who own the content) that their listing is being shown on Site X and provide a link to the page where the user can file a takedown notice. I don't know about you, but I probably would be happy that the listing is getting more exposure on more sites. I think most users would.

Markets are natural monopolies. Liquidity begets liquidity. The breaking down of these silos is a great thing for price discovery and a better functioning market. The creation of decentralized markets where all listings are available and discoverable is really only a matter of time. When that happens buy and sell offers will be fully free and the idea of a centralized site with free listings like Craigslist will be a quaint notion and we'll all be better off because of it.

Information wants to be free, brother!

It's funny how many people are demanding that Craigslist sell out its user-generated content. I understand why people want this, but isn't CL being incredibly respectful of its users by allowing their data to be used only for the specific purpose for which it was posted and that's it?

I don't think so.

Users are on Craigslist to buy/sell, because it's the biggest place for that. If you post to CL, virtually everyone wants the largest audience possible, so 99.9% of people will be thrilled to have their post copied to other services.

The only annoying thing would be if you take down your post from CL after you've bought/sold your thing, and you continue to get calls/emails from people getting it via copied data. But sites that copy just need to stay up-to-date on whether postings are still up or not...

As someone apartment hunting in NYC I appreciate this. If I wasn't my feelings would be pretty mixed.

Yeah, I went back and forth on it quite a lot. That said, the law protects this sort of thing (facts can't be copyrighted, and that's all PadMapper is showing from CL), and it seems like a pretty large net loss for the world if this isn't continued. But I'm obviously biased.

Facts can't be copyrighted. But how is an advertisement a "fact"? Can I create a website of all the Coca-cola advertisements over the years, and call them facts?

Ads aren't facts. You're still intentionally copying the CL website without the right to do that. Good luck to you.

Edited: Not trying to start a "copyright isn't theft" debate.

No, but if you created a website that showed what prices stores were selling coke for, Coke couldn't do anything. The facts of an ad are not copyrightable.

If you went and did the work to get those prices (called or visited all of the stores in a city, for instance), then you'd be correct. But if you're scraping someone else's website in an automated fashion, and that's provably true, you're entering into the copyrightability of databases.

There's a material difference - collecting data yourself versus copying someone else's collection of data.

As another example, can I go to Amazon.com (without using their API), scrape all the product names and prices off their web site and map the dates and times they change? I would think not. Those too are facts, but you're basically just copying someone else's database.

Can I scrape stock data off of Yahoo finance without having to pay the license fee associated with getting historical stock data?

> As another example, can I go to Amazon.com (without using their API), scrape all the product names and prices off their web site and map the dates and times they change? I would think not. Those too are facts, but you're basically just copying someone else's database.

Sure. There are plenty of services that do this, it's quite handy. Here's the one I use: http://camelcamelcamel.com/

> Can I scrape stock data off of Yahoo finance without having to pay the license fee associated with getting historical stock data?

Sure. They even provide a CSV format to make it easier for you!

http://ichart.finance.yahoo.com/table.csv?s=YHOO&d=0&...

Copying is not stealing! Christ, the music and movie industries have really fucked up the meaning of that word in recent years.

Craigslist's #1 asset is their gigantic amount of classifieds. There's really nothing else going for them.

Any service competing with Craigslist that uses those classifieds might not be stealing them, but its certainly in Craigslist's best interest to keep their most valuable asset safe.

I'm still technically correct, which is the best kind of correct.

A unique database of facts absolutely can be copyrighted. The Billboard charts and SoundScan are a great example, upheld in court. More here: http://www.bitlaw.com/copyright/database.html

The Supreme Court apparently ruled overwise on this case, as discussed above, i.e. here: http://news.ycombinator.com/item?id=4219085

I have a feeling there will be another C+D soon, but since I'm looking for an apartment at this moment, great.

I am annoyed that CL hasn't uttered a single peep about all this. They killed a great app, and didn't even respond to the outcry. A simple 'we don't want to support competitors' would have sufficed, but their silence damns them.

Padmapper already received a C&D... IANAL, but i think a lawsuit may be the next step.

If the circumstances have changed, the appropriate course of action would be a further cease & desist. This would allow Padmapper the opportunity to desist from their activity which is always the right course of action before instituting proceedings.

The C&D mentions not just accessing CL content, but republishing it. That the content was transmitted through third parties is irrelevant for this copyright claim.

It may be irrelevant, however you will get penalised on costs (at least in the UK) if you do not attempt to resolve the dispute prior to issuance of proceedings.

I personally wouldn't want to risk the ire of a judge in issuing proceedings when at least on the face of it, PM complied with at least part of the initial cease & desist sent by CL.

As others have pointed out, it's not Craigslist's content. According to CL's own terms, users own all content posted on the site. So CL has no copyright claim on it.

Can someone explain to me how this works legally? It seems a bit too convenient that you can ignore someone's T&Cs and use an unlicensed third-party service instead.

From what I gather, the Craigslist T&Cs specifically forbid scraping data off their website. They explicitly call out the fact that they do not own the actual data/listings/facts on their website, so those are public domain (or possibly owned by the person who created the listing, who knows). By scraping the data from Google's cache, you're no longer in violation of the T&Cs because you're not scraping from Craigslist's website anymore. You are free to use the data obtained in this method because Craigslist doesn't own the data. (IANAL)

What about breaking Google's TOS by using a bot to scrap data from them?

Do they really have a rule against scraping their scraped data? If so, you'd be in violation of that, but that's your problem with Google now, not Craigslist.

IAAL. And what you just said is not true, at all, in the U.S. It may be true in Somalia or some other country without an established legal system.

Google's cached Craigslist site is, legally, a copy of the Craiglist site, which just happens to be hosted on a different server with permission (under fair use, it is essentially compelled permission).

Craiglist does not own the facts on their website (nobody can own facts), but they do own the data (a filtered or specifically selected set of facts) or a license to use the data. They also have a license to post the listings (which are owned by their users). These licenses do not extend beyond Craiglist.

Craiglist's T&C bars scraping because that is an end-run around both the user's copyright in the listing and Craiglists' licenses in the listings and underlying data.

By scraping from a copy of Craiglist's website, 3Tap and Padmapper are still scraping from Craiglist's website.

1: Stop hatin' on Somalia

2: The US Supreme Court has in fact decided that the 'data' is not not copyrightable in terms of phone book listings. Because they are no longer scraping off Craigslist itself, they are no longer violating terms of service directly. Craigslist doesn't get to control what Google or 3tap allows. See: http://news.ycombinator.com/item?id=4219085

1) Until this year, Somalia did not even have a stable government. According to many reports, it still does not have a stable legal system. It's not hate, it's a simple statement of Somalia's current state.

2) You need to reread your SCOTUS case law. The specific selection, arrangement, and presentation of basic facts is copyrightable. This is why phonebooks and whitepages are copyrightable, even if the information they contain is not.

The key here is the selection of fields--is it sufficiently unique that it would not be the most efficient and obvious way to present such data? With a phone book, you have few fields: name, phone number, and possibly an address. There are only so many ways to select, arrange, and present such information.

With rental postings, there are many more fields. The selection of fields itself can be considered unique, as they tend to very from site to site; address is really the only constant. Consequently, the phone book cases are not strong protections for Padmapper and 3taps. Thus, if 3Taps' api makes available all of Craiglists' fields for copying, that will probably be enough for a violation of Craiglists' license.

I don't think there's a whole lot of caselaw here, so I think it's all a bit of a grey area. One thing I can say from experience is that judges typically frown on "clever hacks" to get around legal obligations. IANAL.

A TOS is only a legal obligation if one enters into said agreement in the first place, which is questionable even when one has signed up for an account. In most cases, it's mere wishful thinking or just extra garbage for lawlyers to bully people with.

The meatspace analogy would be having 100 data technicians transcribe all of the Craigslist ads. You're still stealing the Craigslist content and database, ergo you're still liable for damages.

There is no protection for a database of public facts under US law, contrary to say the UK. Craiglist's data is all public facts.

Posted elsewhere in this thread:

http://www.bitlaw.com/copyright/database.html

http://www.bitlaw.com/source/cases/copyright/feist.html

If you read the second link, you'll see that it is exactly what the OP is doing and it has already been decided in favor of the database owner in case law.

It actually sounds like the opposite was ultimately decided by the supreme court. Two lower courts decided in favor of the DB owner (in this case analogous to Craigslist), but the Supreme Court found that the database did not meet creativity requirements to fall under copyright law (in the example, a company was not allowed to claim copyright over a database of phone listings in alphabetical order).

Applying this example to the Craigslist situation (in a decidedly IANAL fashion), Craigslist would have to be able to argue that the manner in which it "selects", "coordinates" and "arranges" its data is creative and unique enough to be protected by copyright law, and furthermore that PadMapper's use of the data infringed on that specific creative selection, coordination, and arrangement of the data (since the data itself is still not protected by copyright).

For all the people mis-interpreting CL's TOUs: "You automatically grant and assign to CL, and you represent and warrant that you have the right to grant and assign to CL, a perpetual, irrevocable, unlimited, fully paid, fully sub-licensable (through multiple tiers), worldwide license to copy, perform, display, distribute, prepare derivative works from (including, without limitation, incorporating into other works) and otherwise use any content that you post. You also expressly grant and assign to CL all rights and causes of action to prohibit and enforce against any unauthorized copying, performance, display, distribution, use or exploitation of, or creation of derivative works from, any content that you post (including but not limited to any unauthorized downloading, extraction, harvesting, collection or aggregation of content that you post)."

There's obviously a big fat indemnity clause in 3Taps' terms which means you're on the hook for claims made by third parties against 3Taps for loss arising as a result of the usage of the service.

Having said that if CL provides access to 3Taps on terms which allow third parties to take data from 3Taps, then this may be permissible. It of course depends on the terms of 3Taps' access though (I admit to not being completely familiar with the ins and outs of 3Taps' service though).

The problem of course is that CL have already shown a dislike of Padmapper's methods and as a result this workaround is unlikely to meet with CL's approval. Common sense would suffice to tell you that.

Having said that, it's a real shame CL aren't open to discussions over licensing data (maybe they are of course but the price is too high).

Nah, they weren't open to discussing any sort of deal involving the website (they were willing to license data for the mobile app).

Why didn't you go that route? There are so many phones and tablets out there now, you'd have accomplished almost as much public good with far less controversy.

Good question. Basically, it's not really the best way for people to browse this kind of stuff, and there are still lots of people without phones/tablets. You can get a much better picture of things with PM maximized on a 20" screen than you can with a 4" one. The phone is more of a backup for when you're on the road or in a pinch.

The phone is more of a backup for when you're on the road or in a pinch.

My girlfriend is looking for an apartment in Houston, and sometimes browses Craiglist with her iPad.

Yeah, the iPad is a pretty good substitute for those that have one (and I'm happier with the PadMapper iPad app than with the phone apps).

Bold move. Curious if CL's next step will be to add noarchive tags to prevent google cache content from being crawled.

I can't see why they wouldn't do that. It's not like the Google cache offers much utility for Craigslist posts to begin with.

Anyway, it doesn't seem right for Padmapper to be doing this, but for god's sake, could people browsing the primary source for residential ads please have a way of doing that with a map?! That's really all anyone is asking for here. Sheesh.

> I can't see why they wouldn't do that. It's not like the Google cache offers much utility for Craigslist posts to begin with.

Then there's no harm in their putting the tags in.

Uh, okay?

I appreciate that Eric is doing this in the open. The moral argument could go either way, but he is letting everyone know what he is doing and why. He deserves credit for inviting feedback on it. Personally, I hope padmapper does well.

I see some very interesting comments on this thread. I remember not long ago lots of people where upset that people where scraping stackoverflow and getting a higher ranking in google results. This was a threat to S/O's business model model and it made lots of people on this board uneasy.

How is Pad mapper any different? How does the fact that this site has a better UI design somehow give them a pass to use data a company has spent millions of dollars gathering and feels is being used in a way that is a threat to their business model?

Why would you even need to scrape it? It's available as a torrent: http://blog.stackoverflow.com/2009/06/stack-overflow-creativ...

Spent millions gathering? Two basic problems with that mindset. First, spending money to assemble a database of public facts does not magically give you a right to protect your investment in an uncopyrightable asset. Second, I don't know what millions you are talking about since craiglist's only asset is the users, and the users are just a result of tremendous network effects. They had a great idea and got very lucky, but gathering their massive userbase had nothing to do with spending money.

Could someone explain what they disagree with? Do you disagree that databases of public facts are not copyrightable? With the idea that craigslist's investment in their system is irrelevant compared to the value of their userbase? With the idea that craigslist's main success is just being lucky?

The last few places I've acquired we're through padmapper aggregation of CL posts, so when PM got the C&D, I felt compelled to email CraigsList asking them to implement a fee-based API as it would be a win-win-win for everyone involved. Sadly, no reply.

After reading the post and discussion, my conclusion is that Craigslist is making a policy mistake.

They should make the same calculation of public good that ericd made and license 3rd party websites that add value to Craigslist, provided the 3rd parties do not harm the Craigslist infrastructure or business models. Doing otherwise is just holding up progress, and is a losing game in the long run. Facilitating value to users would be a win for them, as it was in the past.

Creating a site where buyers can browse property listings does harm the Craigslist business model. To say otherwise expresses a lack of understanding of online ecosystems.

As far as I can tell, they aren't posting the full listing. They are only aggregating the material facts about the listing. As far as I can tell, short of actually hoofing it to the place, you still have to get on Craigslist to make contact with the landlord.

What Craigslist is afraid of (probably) is Padmapper becoming the first place most people turn to when looking for an apartment. Once that happens, Padmapper can cut Craigslist out of the loop by inviting landlords to make Padmapper listings instead of Craigslist listings. (Note that once Padmapper become to "go to" place for apartment hunters, Padmapper's refusing to continue to host or "index" Craigslist listings helps towards that goal.)

Regardless the morality or legality of Padmapper's latest action, it is pretty clear to those of us who have spent a lot of time observing online ecosystems and software ecosystems that Padmapper's latest action is not in the interests of Craigslist since (even if Craigslist is not out to make money, but rather out to influence the world for the better) the only important source of Craigslist's potential for making money and of Craigslist's influence is Craigslist's being the first place people turn to when apartment hunting and doing other important things.

That is why Craigslist is against Padmapper's latest action even if Padmapper never posts the full Craigslist listing.

And in that, Craigslist is clearly acting in its own interests contrary to the interests of its users.

Contrary to the interests of Padmapper users (some of whom are also Craigslist users.)

The vast majority of Craigslist users don't want to use anything else, nor are aware of anything else.

> that add value to Craigslist

It should be pretty obvious by now that Craig doesn't give a shit about adding value. (I mean this in a good way.) CL has dominated its space for the past 15 years and shows no signs of letting up, so your claim about losing in the long run would seem to be misguided.

> CL has dominated its space for the past 15 years and shows no signs of letting up, so your claim about losing in the long run would seem to be misguided.

Past performance does not guarantee future results.

Although it's tempting to say that CL should make an investment in these 3rd party sites - the most direct solution is for CL to say "We'll license the content for 30% of your equity". Cash element of zero : CL has already invested in making a site that has the raw content that everyone wants.

Eric's confusing method with goal.

Craigslist's past method was to claim TOU violation on the part of PadMapper, denying PM ready use of its content.

Its goal is to maintain market position as the dominant classifieds listing service. TOU is just one way for CL to enforce its market position, and as a commercial entity, it's very likely it will continue to do so.

There's very little reason CL couldn't create an alternate legal claim based on content (the cartographic watermarking example comes to mind) to make multiple infringement claims against PM, which would invoke a whole host of copyright remedies ranging from injunctions and monetary damages to DMCA takedown notices (which could comprise the entire service depending on how construed by PM's hosting provider(s)). There's also the notion of a compilation copyright, in which a collection of CL postings could have standing. While Feist v. Rural Electric holds that a simple compilation of facts doesn't meet the minimal authorship requirements of copyright, a compilation of postings with some level of curation to CL might. I'm not aware of relevant caselaw here.

My own suggestion would be that PM pursue a dual strategy of coming to a business agreement with CL (PM's interface is vastly superior) while curating its own set of rental listings independently (CL is ripe for disruption here).

Disclaimer: I'm a long-time CL user and fan of much of its philosophy. I'm also a recent fan of PM. I hope the two will either work something out cooperatively, or manage to improve apartment and classifieds search by competitive means. I don't particularly care which.

Thank you for doing this. My only problem with it is that you're rewarding craigslist for being obnoxious tools, but ultimately it's better for the users, so that's an acceptable sacrifice.

3taps is pretty awesome. I wish they'd do an AMA or something.

As a satisfied padmapper user, I'm glad to see this, but somewhat troubled at the same time.

The original hubbub about Craigslist pulling padmapper's access, regardless of your stance on Craigslist's actions, was to me a cautionary tale about building apps around external APIs. (See the Twitter drama of the past weeks.)

What happens next if Craigslist does the same thing to 3taps?

So does 3taps gather data with humans or what?

"Any copying, aggregation, display, distribution, performance or derivative use of craigslist or any content posted on craigslist whether done directly or through intermediaries (including but not limited to by means of spiders, robots, crawlers, scrapers, framing, iframes or RSS feeds) is prohibited"

I think this covers Google cache or any other method of gathering data. You're still in violation of their Terms of Use even if you find a clever means of getting around it.

Simply because it exists in the ToS doesn't mean it is legally enforceable. Hopefully the "legally kosher" claim means he had a lawyer look over everything, and conclude that clause in the ToS is for the courts to decide.

That document begins with "By accessing or using craigslist..."

I imagine the argument would be that since you are not accessing or using craigslist directly you are not bound by those terms.

Technically, wouldn't Google Cache be in violation, and therefore 3Taps & PadMapper by extension?

My understanding is that they use the Google cache.

I posted a look at the legal issues on Google+ here: https://plus.google.com/112961607570158342254/posts/HcCuHVG7...

Here's an excerpt from the post: Eric's blog post today says that "using 3taps just makes it so it’s not a TOU issue." The problem is it doesn't. Craigslist's lawyers already thought of this, which is why the TOU also prohibits the "display" and "derivative use" of "any content posted on craigslist," which PadMapper is clearly doing. As Craiglist's lawyer is likely to put it in the next C&D, assent to the TOU is still implied. [...]

If 3taps is in fact scraping Google's cache, they're also violating Google's terms of use, which prohibits any effort "to access [search results] using a method other than the interface and the instructions that we provide." Anyone want to bet on how long it will be before Google blocks them through technical countermeasures, or sends its own C&D? And if 3taps is in the business of the TOU-prohibited "distribution" of Craigslist data, which it appears to be, it should expect to get a C&D from Craigslist too.

"Certainly by the time you read this." Code isn't live yet, at least for me, ericd - is this conversation giving you pause about deploying the new feature, or is it standard ops issues?

Also, this couldn't come at a better time for me. I love padmapper and am moving to the bay area in 3 weeks, I need this, man!

Sorry, was having some sysops issues, thanks for the heads up.

What about a service that lets you post to every barter site at the same time, not just craigslist. Has that been tried? If you could get people to use it (through better UI and more exposure for what they're selling) you could break craigslist's hold on the marketplace.

Padmapper is awesome, way way better than using Craiglist to apartment hunt, even if the data ultimately comes from Craigslist (among others).

Maybe craigslist should buy padmapper instead of getting all lawsuity and take advantage of just how good their interface is for apartment hunting

Not sure if this has been asked, but a philosophical question for Eric: if someone decided to start a padmapper competitor tomorrow, would you be okay with them using your data for free?

I don't see why not. By being a competitor that scrapes PM, they are accepting that there will be a delay in their data versus when it's available on PM. So unless their site was significantly better than PM in some way, in the really competitive apartment hunting markets, they would be strictly inferior to PM.

How do 3taps do it? Some comments say they "use Google cache", which doesn't explain much:

- Google is notorious for fighting bots pretty aggressively, how do 3taps avoid this?

- even if 3taps found a special access to Google cache, it would be trivial for CL to prevent Google or anyone else from caching its listings, thus cutting off the rest of the food chain that feeds on caches; why don't they do it?

Is it possible that 3taps use another method, such as "manually" copy-pasting listings with Mechanical Turk?

Really, really stupid. I mean pretending you want to help people find apartment to justify you're action is ridiculous. You're running a business, be honest about it. Now craigislist do what they want with their data, you are just trying to build a service that "adds value" on top of someone else infrastructure( I mean "add" because I am one of the few who actually like the way cragislist display posts no bloaty js, no ugly pics no maps ... fast and simple).

I find it difficult to believe that you have searched for an apartment in a tough rental market on Craigslist and consider their interface "fast and simple." If you've never had the task of finding an apartment within a mile of a given spot in a big metro area, you're not looking for what PadMapper offers. If you have had that problem, and you've never sworn out loud at Craigslist, you're a saint.

I was looking for an apartment 6 month ago in Reston, VA and I used craigslist for that purpose. I looked for a couple of zipcode (that I heard were good, since I was just moving in the US at that time) and honestly it worked great. Wouldn't it be the same in a big metro area ? just looking for interesting zipcodes ?

You realize in some cities a few blocks makes a GIANT difference in neighborhood, right? Same zip code.

Being able to visually filter those out is a major time saver.

The other issue is where I live, people put all kinds of random keywords in their Craigslist apartment ads to try to get attention. "Just 25 minutes' walk to Foo Square, Bar Square, and Baz Square! Four hours by horse to Lexington, and a short plane ride to London!" If I want to know what's within a 10-minute walk of a given location, I basically have to read every listing with my town's name in it every day and see if they included an address. Or spend 5 seconds glancing at PadMapper.

http://traffic.alexa.com/graph?w=400&h=220&o=f&c...;

Traffic took a great nosedive from the day they announced "bye bye" to CL.

By your own metric, so did traffic for CL, and CL is the 8th most visited site in the US, so their Alexa traffic data is probably more stable than traffic data for PadMapper. http://traffic.alexa.com/graph?w=400&h=220&o=f&c...;

My question is: Why are people using craigslist?

Fortunately for me, I purchased a home before all apartment ads migrated to CL. For the life of me, I don't get why it's so popular.

The site is awful in at least a half dozen different ways.

It's free and easily accessible. Compare to the costs involved in getting an ad listed in a dead tree.

Yes, the site is awful. But it is less awful than the alternatives. Therefore, people use it. Please, on behalf of all apartment hunters everywhere, (but especially in high tech/high rent places like the Bay area or Boston), tell us what the better option is! I say this in all seriousness. The idea that PadMapper was saving vast amount of man hours was not an exaggeration. I wasted many hours clicking on craigslist ads to find my current apartment.

> Why are people using craigslist?

Because everyone else is.

The funny thing is -- PadMapper and 3Taps and the like are actually perpetuating the usefulness of Craigslist:

"everyone scrapes Craigslist, so if you just post it on Craigslist it will be pulled into all those other services automatically"

Ads migrated there because it was free, and a lot more agile than newspaper ads (wait a week for my ad to go live??). They stay because of network effects. And because it's good enough.

I feel like they snagged the community of online classifieds first and have been able to hold on pretty tightly, as people like my dad love that almost nothing has changed.

So, instead of directly scraping CL's data, they are now using an API that scrapes Google's cache of CL's data? That seems kind of sleazy.

Would a client-side padmapper be less morally gray?

Consider, if the situation was reversed, and Craiglist was taking Padmapper's data without permission, would everyone on HN would be supporting Padmapper attempting to block Craiglist from reposting its listings?

In such a scenario, Craiglist reposting the Padmapper listings would be a net gain for user's and posters (even though the posters never gave Craiglist such permission) because the listings would be exposed to an exponentially larger viewership.

Being a startup is not a license (pun intended) to ignore other people's rights. Padmapper is in the wrong here, and if they proceed down this path they could face a ruinous (and fully justified) lawsuit.

>I've always had a lot of respect for what they've done for the world

Did I miss something? What has Craigslist done for the world besides put newspapers out of business?

Using CL I furnished a whole house with super high-quality stuff for less than US$2K. Wouldn't have been realistic pre-CL since traditional classified ads don't have pictures. I also found my house through CL.

Dear PadMapper,

We hate your app and you can't use our API.

Sincerely, Craigslist

+ + +

Dear Cragislist,

Fuck you.

Love, PadMapper

Reply on news.ycombinator.com