Nginx JSON hacks | Hacker News Reader

Nginx JSON hacks

gabrielweinberg.com

178 pointsKenan19 comments

I'm not sure I understand how some of these are being used, specifically whether they're being used safely.

API keys usually exist for preventing abuse. By exposing an authenticated API proxy you're allowing anyone to abuse someone else's API using your key, likely leading to them banning you. At a minimum you should implement your own rate limiting.

Likewise, with a JSONP proxy you're allowing other sites to circumvent the browsers' same origin policy to access that API, which could also lead to abuse. At a minimum you should restrict requests to ones with a recognized HTTP Referer header.

Of course with JSONP you also need to trust that the API isn't doing anything malicious, like injecting cookie or other data-stealing JavaScript instead of valid JSON. It would be a good idea to validate that the response is indeed JSON before passing it back to the clients (actually a properly restricted sanitizing JSONP proxy would be a good idea even if the API already provides JSONP)

So use these techniques cautiously...

The article describes how to set up caching in nginx, so not every request to nginx goes to the external service.

The potential for abuse remains.

Then don't use it. As someone who has never had much experience dealing with Nginx I found this article to be very informative. Most people are smart enough not make public their third party apis. Those who aren't will quickly learn of the problems this may have. However, there are some cases where this could be extremely useful, such as an intranet.

I agree the article is very informative and taught me a number of things I didn't know about nginx. Unfortunately letting people learn the hard way tends to not work out well when it comes to security.

Also, your API key may be at risk if you append it to a URL that you proxy.

For instance, we're doing this proxying with Tumblr (sans API key, fortunately) and they occasionally issue a 302 redirect and tack on some parameters. (for caching reasons, I assume) The 'Location' header that is sent back would have your API key in it, so you'd need to detect that and strip it out. Also, if they throw a 500 error they may echo your URL back at you.

I'd probably move this code up into this application layer unless you're very sure about the API you're using.

If you need to proxy through nginx on your own server to pad some external api's json (to enable jsonp), wouldn't you be able to put that server on your own domain? Obviating the need for cross-domain jsonp?

Great article on how to get Nginx to JSONP for you, but I question the idea that setting up a proxy to a paid API with your client key embedded is a good idea... Seems easy to abuse.

I often have to create a service/api to serve (and cache) 3rd party content cause of rate limiting. a few servers much < than 10,000's of clients. This makes a few days of work/testing into a few 10's of minutes.

You can set them to "internal" in the nginx config so they can only be called by other nginx config, not directly by clients. You also need to check the error messages do not leak urls.

Then there is rate limiting etc, depends on the service.

Couldn't the last trick be improved? since you have access to params you could possibly use those as in

    location ^~ /ext_api3/ {
      echo_before_body $arg_callback //+somehow add "(";
      proxy_pass http://api.external.com/;
      echo_after_body ');';
    }

Indeed, the example code isn't actually JSONP, since it hardcodes the callback.

Which I would guess works for his purposes.

JSONP has numerous problems, first it's not JSON -- the object literal is created by executing JS, so you're back to square one with validating your input, the second and more significant problem is that if you ever use JSONP to transmit private data then you've essentially lost the cross origin protection of data present in the browser.

Now all an attacker needs to do is get one of your users with an active session to load a page that does <script src="usersprivatedata.jsonp"></script> and they can gather that data.

I was doing something similar to this article (feeding json directly to JS functions) with PHP. I think this is better for the browser. server side, it's heavier:

    <?php 
    //inject json
    //this puts the json in a div
    //then the js gets the div's innerText and parses it as JSON
    //so there's an object in js (global) with all the JSON data, with jsonInject's key name
    if(isset($jsonInject) && count($jsonInject) > 0){
      foreach($jsonInject as $i=>$json){
        echo "<div style='display:none;' id='jsonInject_{$i}'>{$json}</div>"
        echo "<script>var {$i} = JSON.parse( $('#jsonInject_{$i}')[0].innerText ); </script>";
      }
    }
    ?>

fill the associative $jsonInject array with json strings in your controller. then this goes in your templater. it makes a hidden div, puts the json string in there. then jquery pulls the innerText value, it gets parsed as json and stored in a js var. depending on use, i'll sometimes just pass the results of JSON.parse to an initialize method.

I like the proxy caching. I think apache's mod_proxy and mod_cache could do this but with more config (and resources per request).

the second and more significant problem is that if you ever use JSONP to transmit private data then you've essentially lost the cross origin protection of data present in the browser.

That's assuming the API is using HTTP cookies for authentication, right? If it's not then theres no way for a 3rd party to construct an authenticated request without knowing the credentials.

The real problem here is that a JSONP proxy allows 3rd party sites to access the API from your IP address (and API key if you use the suggestion of an API key proxy). Rate limiting and restricting to specific HTTP Referrers would be prudent.

If anybody is concerned about API rate limiting, here's a nifty solution. http://codetunes.com/2011/07/26/outbound-api-rate-limits-the...

Use node.js instead!

It's true that everything in this article could have been done with node.js, but the nginx configuration is pretty trivial, and nginx is good software. It's stable, it's plenty fast, and you don't have to worry about a new release breaking everything.

(I have nothing against node, mind you. I've used it for something similar to Google Wave, and it was very straightforward and capable.)

Reply on news.ycombinator.com