Hacker News Reader

Comment by jnwatson | Hacker News Reader

Sounds like a lot of work to avoid writing 20 lines of Lua.

I wrote Locomotor[0] to automate the translation of Python code to Lua for Redis. Basically, you add an annotation to a function and the first time the code executes, it's converted to Lua and shipped to the server. You can see an example here[1]. It's far from foolproof and many Python language constructs have not been implemented, but it can handle some relatively complex code.

[0] https://github.com/michaelmior/locomotor

[1] https://github.com/michaelmior/locomotor/blob/master/bench/t...

I'd be curious to see the Lua code that implements this. Anyone care to indulge me?

I'm guessing it would look something like this... I omitted checking if the token is in the 1 minute window. It returns false if the user has reached the rate limit, true if the user has more tokens left. If the user has tokens left, it decrements the token the user has left before returning true.

  local id = "user_1"
  -- Get the number of tokens the user has left
  local tokens = redis.call("HGET", id, "tokens")
  if tokens = 0 then
    -- User has no tokens left
    return false
  else
    -- User has tokens left, decrement token count
    redis.call("HINCRBY", id, "tokens" -1)
    return true
  end

Redis lua scripts block while they are running, so the two redis calls here cannot be interleaved with other reads, as in the token bucket example from the article.

thats really not bad at all.

agreed. I was a bit leery of diving into Lua as I was building http://ratelim.it but it really expands Redis's capabilities dramatically and was easy enough to add.

My apps all write the lua into redis and store the hash when they boot up. Duplicative, but means everybody is on the same page and it's easy to store the lua in the main codebase.

Hey buddy!

Reading all the design and discussion I was. Rey curious how you structured things at a brass tacks storage level.

yoo! https://www.slideshare.net/jdwyah/diy-heroku-using-amazon-ec... does have a bit of a pretty picture, but the basic idea is:

For each rate limit you can choose to be in one of two modes: 1) Redis with a backing store of DynamoDB aka BestEffort since there are failure modes where you could lose an update. In this mode everything expects to happen in Redis, but if we don't find your limit there we check Dynamo. Writes are asynchronously persisted to Dynamo.

2) Token Buckets straight in DynamoDB. This is our Bombproof mode.

(details in https://www.ratelim.it/documentation/safety)

It's worth noting that with either of these you can cache aggressively in the clients whenever the limits have gone over. Both the clients https://github.com/jdwyah/ratelimit-ruby https://github.com/jdwyah/ratelimit-java do that for you.

Reply on news.ycombinator.com