Hacker News Reader

Comment by ajju | Hacker News Reader

He crawled a bunch (780 Gigs / ~20 million pages) of 'public' court records that were behind some kind of auth (pay) wall via a free trial and opened them up. Likely as a result of this crawling, the free trial was shut down. The pages he crawled constitute 20% of the total records. The feds then started investigating him.

http://thelede.blogs.nytimes.com/2009/02/13/steal-these-fede...

View on HN

PACER data is now being collected and made available to the public by the RECAP project run by the Center for Information Technology Policy at Princeton.

https://www.recapthelaw.org/how-it-works/

Court records are, in general, public. Not 'public'. Just go to the library. However, PACER costs money and so do the favored online vendors (westlaw and lexis-nexis).

Even if they aren't public. They should be. This stuff can be used as law -- precedents. The only valid objection I see is that there needs to be some process to protect the privacy of the people involved in the case. But that's second-order concern.

They should be public but are behind a wall. Thus, the quotes.

PACER records are public, even that they cost money to access. So are most of the records on Lexis and Westlaw. Westlaw has tried to claim a copyright in certain annotations on the court documents, but they have no copyright in the text of the court records themselves.[1]

[1] Matthew Bender & Co. v. West Publishing Co., 158 F.3d 693 (2d Cir. 1998) - http://bulk.resource.org/courts.gov/c/F3/158/158.F3d.693.97-...

I don't know much about what PACER includes and what exactly is covered by Westlaw and Lexis-Nexis, but Altlaw (http://altlaw.org) is a "free, full-text, searchable database of U.S. case law", and I've heard it being mentioned in the context of Lexis-Nexis. Also has the extra cool aspect of having been done by Stuart Sierra, whom all the Clojure folks around here probably know well.

"The only valid objection I see is that there needs to be some process to protect the privacy of the people involved in the case."

There already is: you can file documents under seal or in redacted form. Most courts require you to do so if you're including information such as social security numbers. See, i.e. http://www.courtinfo.ca.gov/invitationstocomment/documents/s...

Nothing that's available via PACER couldn't be read by walking down to the courthouse and requesting the physical file.

But when lawyers told Mr. Malamud and Mr. Swartz that they appeared to have broken no laws, Mr. Malamud sent Mr. Swartz a message saying, “You should just lay low for a while.”

It's not illegal, so what's going on?

Trying to collect $1.5MM?

Either they think it is illegal or what is more likely is that anytime a sysadmin at a federal facility sees "atypical" behavior (as defined by them) they contact the feds. It is very unlikely that a sysadmin in the federal court system would think about whether the crawling session overloading his server is legal, forget moral and I am sure if he/she complains, the FBI is obligated to investigate.

I am not saying this is what should happen but this is what likely happened.

Well, yeah, and those fees they're missing out on indirectly fund that sysadmin's department.

Interpretation of 'unauthorized access' is murky and there's a lot -- too much -- discretion in how access legitimacy and damages are evaluated. (See also: the submission last week about an author's estimation most people commit 'three felonies a day'.)

In this matter, I'd bet the NYTimes coverage was more valuable in deterring prosecution than any legal counsel or even influential friends -- it meant any trumped-up narrative of nefarious intent would be fighting an uphill battle against the story already started in 'the paper of record'.

Reply on news.ycombinator.com