Hacker News Reader

Comment by codefined | Hacker News Reader

You would probably use something like a queue[0] that takes in data from the websocket and dishes it out to lambda functions. You might also use something like Kinesis[1] or other alternatives.

[0] https://aws.amazon.com/sqs/ [1] https://aws.amazon.com/kinesis/

View on HN

Yes of course, or I could send it into Kafka instead (which makes more sense to me). The point is, how would a serverless process looks like which doesn’t have a REST API and does this long term polling of websockets?

Some platforms like Amazon Lambda let you set up functions to consume data from a variety of event sources: https://docs.aws.amazon.com/lambda/latest/dg/invoking-lambda...

How can I consume the Twitter stream API using that?

I don't know this specific API.

Either it works with web-hooks that you could lead to a Lambda via API-Gateway.

Or it needs to pull the data, then you could trigger Lambda via CloudWatch intervals.

Unless the API can stream directly into one of those sources you'd probably need a long-running process, perhaps running on a CaaS like AWS Fargate.

I guess you could argue where to draw the "serverless" line, at functions or containers, but Zeit is calling this container service "serverless" so I think Fargate would fall into the same category. I think it would make sense for Zeit to eventually support long-running containers too (looks like the current max is 30 minutes, I'm not sure how they chose that number)

It depends what you're doing with that stream, most basically you would create a nano/micro EC2 instance that will just trigger Lambda events on every new tweet. Or you could create some more intricate script that does a lot of pre-processing and then stores it in RDS or S3, and with each new update to either of those sources kick off a Lambda.

Reply on news.ycombinator.com