HTTP cookies, or how not to design protocols | Hacker News Reader

HTTP cookies, or how not to design protocols

lcamtuf.blogspot.com

142 pointstptacek48 comments

That's a nice summary!

I still think allowing cookies to span more than one distinct domain was a mistake. If we had avoided that in the beginning, cookie scope implementations would be dead simple and not much functionality would be lost on the server side. Also, JavaScript cookie manipulation is something we could easily lose for the benefit of every user, web developer and server admin. I postulate there are very few legitimate uses for document.cookie

There may be few legitimate uses for document.cookie, but eliminating it does very little to prevent cross-site scripting.

How about double-posting session ID's from the cookie within AJAX requests to prevent XSRF? I've certainly used document.cookie for that.

The only other scheme I know of to defend against XSRF is to generate secret values for every request, which is not always practical.

I'm not sure I can think of one web app I've tested this year that overloaded the session cookie for an XSRF token; it is straightforward to just generate a random token and embed it in the page, and that also gives you fine-grained control over CSRF protection for forms where you want per-render instead of per-session protection.

You don't need to generate per-request tokens unless you want to; the token is simply yet another thing you stick into the session store and splash out onto the rendered page.

(Also, as my colleague Cory points out: if an attacker can influence the ACTION attribute of the form in any way, you've just coughed up your session token to an attacker in the name of blocking CSRF; is the risk worth the reward?).

I don't think an attacker can influence my ACTION attribute, but just in case, I use only 4 characters from my session token (1MM combinations). I don't think CSRF tokens are easy to brute-force, so that should be long enough.

I don't like keeping per-request state on the server, else I would use random CSRF tokens.

It doesn't need to be per-request; it can just be another 128 bit random number, stored in the session, re-used for every request.

Well, instead of generating two separate tokens, a simpler solution IMHO is overloading the cookie, and it's what I usually see and use.

What are you talking about "if an attacker can influence the ACTION attribute of the form"? If they are running JavaScript in your DOM you are already hosed, they can grab any token you put in your form. There is no sane reason for having ACTION change via request parameters: you would have bigger issues than session stealing. And in any other situation, you should be protected by the browser's cross domain restrictions. I don't understand what you are suggesting.

You can use the HttpOnly flag with a cookie to prevent JavaScript being able to access it. Here's a good summary:

http://www.owasp.org/index.php/HttpOnly

If you are serving a cookie, and JavaScript doesn't need to be able to access it, you should set it to be HttpOnly to try and reduce the effects of an XSS.

It doesn't seem like Michal is sold on HttpOnly and I definitely am not either; most everything you can do with a stolen cookie, you can also do quickly and silently using a captive browser during an XSS attack.

True. Still, given the option of having it turned on or off, I'll turn it on. If only it makes an attack a little more difficult it's worth using.

No cost, potential benefit. I've had to make harder decisions.

The cost is "false sense of security", and believe me, from field work, that is a real cost. A majority of programmer applicants can't solve FizzBuzz, so you're deluded if you think that a good chunk of webdevs don't think "HttpOnly fixes XSS".

The potential benefit is... what, exactly?

It lessens your potential attack surface and comes at no cost. And as security-ignorant devs are concerned (which we all are from time to time) it doesn't magically fix vulnerabilities, but it can alleviate the magnitude of exploits. It's a good idea.

Try to articulate how it lessens the attack surface. It means you can't bank up session tokens; instead, you're only able to do everything that a session token would do for you anyhow. Remember in your explanation that session tokens already expire anyways.

By sending the HttpOnly flag as a server application, I indicate to the browser that it should prevent client-side scripts from obtaining the actual value of the cookie. Hence, it is more difficult for a malicious little piece of smuggled-in XSS code to steal my session id and ship it off to some Nigerian server. Is it circumventable? Yes. Are different browser implementations subverting this scheme? Yes. But is the intention behind it worthwhile? ...I think so.

Of course, that kind of protection is superficial, but please try to remember in your rebuttal there are actual attacks still happening that could have been prevented by setting this simple flag. An insistence on always making secure code that won't have this problem anyway is honorable but not realistic. By the way, it happens on high profile sites, too. It shouldn't, and it makes us web devs uncomfortable even observing it from a distance, but it still happens.

Even disregarding this argument, for me HttpOnly is the way cookies should have been conceived by default. I believe accessing cookie values by JavaScript is pathological. The behavior HttpOnly attempts to trigger on the client seems like good practice to me, and given (again) the point that it comes at no cost to the reasonable developer, turning it on is really a no-brainer. In fact, browsers should turn it on whether the flag is set or not.

It means you can't bank up session tokens; instead, you're only able to do everything that a session token would do for you anyhow.

Oh, you can still use other exploits to bank up tokens, but yes this particular avenue is blocked. You're absolutely right, having the current session to work with is pretty destructive. But I think shipping off session IDs to a hostile server is worse than issuing false requests on behalf of the user in an already open browser window. That's because siphoning session IDs is silent and can lead to problems many days later when you're _not_ online.

Try to articulate [...] Remember in your explanation that session tokens already expire anyways.

Alright, I'll try not to drool and babble more than necessary: for the sake of convenience we are used to very long-lasting session tokens. So when you say they expire, yes, it limits the amount of damage somewhat, but not in any meaningful way considering the typical cookie-based account identifier lasts well beyond a week. Certainly for the purpose of completing my answer I can now say "I remembered" session expiration but it didn't really go anywhere.

Also, JavaScript cookie manipulation is something we could easily lose for the benefit of every user, web developer and server admin

With more and more apps moving to pure JS / CSS/ and HTML front ends the lack of being able to manipulate cookies would kill a good deal of in app storage abilities. Eliminating useful feature to make someone else's job easier is rarely ever a good solution.

In my opinion, session cookies are primarily used as a work around since HTTP Authentication is incredibly ugly. Perhaps if that core problem was addressed, the need for cookies would be dramatically reduced.

I've never understood whats so "ugly" about http authentication. Digest auth can get a little verbose, to be sure, but Basic+SSL is simple, and both are far easier than roll-your-own Cookie auth. In fact, the way most people & frameworks implement cookie auth sessions are no less secure than Basic auth. Haven't the events of the past week demonstrated we should all be using SSL anyways, in which case Basic auth would be fine for everyone.

Basic auth is only simpler than cookie auth if you (a) aren't using a modern web stack (ie, you aren't using ASP.NET, J2EE, Rails, Django, &c) and (b) have only a few users.

If either (a) or (b) do not hold, Basic Auth is at least as hard as cookie auth.

I use HTTP authentication when I want to password protect a bunch of static files, or a third party web app. Using the PAM auth module with Apache is particularly nice. Just drop a few lines in a .htaccess file and it uses your users system credentials, eg /etc/passwd and /etc/shadow.

HTTP Authentication comes in several flavours. I've not done it myself, but I think you can do interesting things to with Kerberos. Good for Intranet systems.

HTTP Authentication does have its places.

In other words, HTTP Authentication is useful in the same sense as FTP is useful: it's outmoded but sometimes convenient.

HTTP Authentication is sessionless. In any scenario where you want to protect something without establishing and maintaining a session, it's useful.

The only large public website that I know of which still uses HTTP Authentication though is http://www.123-reg.co.uk/

What's the browser app scenario in which having a session is a liability, but having a stored HTTP Auth credential isn't?

Actually the elimination of server side session reduces distributed application topology significantly. It is far easier to scale a stateless app than it is a statefull app.

But if your clients include things that aren't browsers, HTTP Auth is the only acceptable choice.

That's not true either; you can just do what big API apps do and generate access tokens. Amazon Web Services don't rely on HTTP-Auth.

Using curl to access AWS is kind of a pain, since you have to hash the body and your keys into a header. For others, you have to set the token in a header, and it (should) change periodically.

Compare to:

    curl --anyauth --user login:password https://example.com/

It's a "1 line of Python" pain, true, but it's also significantly more secure. From a cost/benefit perspective, it's an API, a building block of a software development project, and optimizing it for curl-ability doesn't seem like a major win.

But my point is just: HTTP Auth is not the best-practices answer to non-browser web services auth. Both API-key and signed URLs are both competitive (and probably better) options.

What about for SSL-based APIs? it seems a pretty optimal choice for those.

I don't think it is, for two reasons:

(1) For most large apps, you're not really SSL in the provider's own network, so there's always the cross-app risk if something horrible happens.

(2) It's forcing app-to-app, business-to-business authentication into a username/password mold that doesn't many any real sense; a 128 bit random key (or an SSL client cert) makes more sense anyways.

Against those two problems --- which are marginal, I concede --- HTTP Basic offers... exactly what advantage? I don't see it.

A few problems with basic auth:

    * Browser specific UI, which leads to:

      * No place to put password recovery links

      * No place to put explanatory text

    * No mechanism for logout

If the browsers would allow the login box to be styled via CSS like the rest of the UI, I think you would see more use. We finally went to SAML for our REST service authentication and have been pretty happy with it. THe nice part about SAML is an identity server provides the auth endpoint, and your app proxies the token to the identity server for authentication and authorization, it gets your app or container out of the game of authentication and authorization. Further many app servers can plug into the identity server therefore providing container managed security gates.

HTTP Authentication is dead and is never coming back. Leave aside all protocol design elegance arguments and think about the situation for app developers; form auth is simply better:

* It has a simple reliable log-out button.

* It gives app developers more reliable fine-grained control over the login process because it in no way relies on browser chrome.

* It better supports advanced security and UX idioms, like signup-or-register or SMS-me-a-one-time-login.

* Virtually every web app in the world needs a session-keyed store anyways, and authentication is the easiest of the AAA problems anyhow, so building that one tiny piece into the protocol doesn't solve any problems.

* [ps]

Doubtless there are a myriad of nitpicky arguments about how HTTP Auth can be massaged to mitigate these problems, but who cares? Cookie auth works for everyone. Basic auth manifestly does not. Why would anyone want to expend major effort to take an archaic protocol and make it asymptotically as good as what every web app stack already provides?

[ps]: Cookie auth schemes can also be extended without getting Microsoft, Mozilla, Google, and Apple to agree on anything; this is the [end-to-end argument in systems design] in action.

HTTP Authentication is dead and is never coming back.

What should WebDAV shares use instead? I don't think form auth really works for that.

WebDAV is a good point but may be the exception that proves the rule, since it is itself such a misfeature. Meanwhile, mentally amend my assertion to "HTTP authentication for browser-based web applications is dead and is never coming back".

It's certainly not dead when it comes to RESTful web services APIs. It's often the best choice for authenticating these.

In a browser context, most of the UI criticisms have little to do with the protocol itself and could be easily addressed if browsers would add a little more HTML + javascript API support for doing HTTP auth logins and logouts. Admittedly that looks unlikely to happen in the near future, which is a shame IMO. IIRC they were considering it at one point for HTML5.

I feel like I could win an argument that it's never the best choice for authenticating RESTful services, because it implies that authentication is username/password based, and there are often better idioms for app-to-app / company-to-company authentication than usernames and passwords.

Gotcha.

Is there not any scope for improvement on this front using alternative HTTP auth schemes though? (As far as I'm aware the standard has room for schemes other than Basic and Digest, although I suppose limited to the challenge-response format)

At any rate, it'd be nice if there was an effort to work with the HTTP standard (or if not possible, propose extensions to it) and aim for clean protocol design where at all possible when driving to improve API security.

SSL + HTTP Auth has been pretty good for us, and it's a good fit for the RESTful design goals of the protocol, so I'd be sad to see it thrown out, rather than improved where necessary.

Seriously: the opposite of this sentiment is true. We should be working to hoist stuff out of HTTP. Authentication is a perfect example of something that endpoints should be able to decide without having their options influenced by the protocol.

If there's a single advantage to conveying a username/password in a dedicated HTTP header as opposed to any other way of using HTTP to convey the same information, it hasn't been well articulated to me. Down with HTTP Authentication!

I think you're right. If you look at the standard HTTP header fields, almost all of them demand/specify information about the resource itself, its representation, or the mechanics of the transfer thereof.

The others: Host - necessary for multiplexing over the predominant transport protocol Date - necessary for caching Referer - mostly useless and optional anyway User-Agent - information about the client and essentially useless Server - useless

And then there are the bastard children: Cookie/Set-Cookie - used to turn HTTP into a stateful protocol Authorization/WWW-Authenticate - used to transmit information about the relationship between the client and server.

User-Agent is far from useless. When filling out support forms wouldn't you rather skip all the annoying questions about your OS and browser?

Good point - it's become the de-facto way of communicating client capabilities to the server.

Well, I think you'd want to think carefully before pulling things out of HTTP and forcing HTTP to sit on top of a stateful, session-based protocol in order to achieve them.

The statelessness of HTTP is one of its major strong points, making it easy to load-balance and scale, more easily compatible with a wide range of middleware, etc. The redundancy involved in repeating certain headers is a design trade-off made for a reason.

There's a bunch of trade-offs here in the protocol design though, evidently. Coming at it from a security perspective seemingly leads to a different view of those tradeoffs than (say) Fielding has.

If you have an application with a really pressing need for stateful session-level protocol features though -- perhaps HTTP isn't the best choice anyway?

It's often the best choice for authenticating these

SAML is a better auth mechanism for REST services.

lcamtuf is amazing.

also: Lou Montulli did the blink tag. Hrmph!

He really has gone from genuinely impressive to downright spooky in a pretty short period of time; he might be the smartest person doing browser security today.

That blog is just awesome, for what it's worth. He has yet to turn in a bad post.

I'm actually more enamored of the CAD and fabrication stuff: http://lcamtuf.coredump.cx/guerrilla_cnc1.shtml

Would using javascript's same origin policy have solved 90% of the issues?

On the face of it that seems the case.

Reply on news.ycombinator.com