Comment by eduardordm | Hacker News Reader

Comment by eduardordm | Hacker News Reader

Parent Full thread

I built software like this not once, but several times. Server for verifone pos, RPC servers to allow any client to execute natural (it's a programming language) subprograms.

First rule: Do never bother writing your own socket server, there are so many reasons to not build your own server, but just consider that they already exist and are better than what you could build.

That said, it's all about the type of service: For verifone machines server I use a custom protocol which I regret I had to construct every single day. I had to build it because it was a long time ago and connections speeds were too slow for https.

SOAP is a reasonable option if you have text+binary data. Also if clients already use SOAP. I dislike XML deeply, but if you have have complex objects you can combine your web service with XSD schemas and make it great.

Now, the best option: just use dead simple HTTP transporting JSON, it may be REST or not. But anyways, this is the best option you have. It's reliable, tested and fast. I don't know python, but there must be something lighter than django around. Use that with nginx or apache as http server.

Java is pretty cool to build this kind of software. Check out Apache Mina, you can create a rock solid, production ready server in minutes if you decide to roll your own protocol.

Oh, one more thing, if you are doing this to avoid client licensing on databases, you shouldn't. Most of them have specific licenses for connection pooling.

Great, thanks for the advice! HTTP + JSON is exactly what I would love to do. I was just unsure if it was kosher to do (embedding a lightweight web-server and running a daemon on top of it)

This isn't for sharing databases...I just need the info that is in a client's database to perform the required action. Technically, it could be performed all on their end, but I'd prefer this to be a SaaS for a variety of reasons.

This discussion is all about the agent that sits on the clients premises, correct? First define your server side RESTy JSON API. Then write poll based client agent, extract data, transform, PUT to API. Don't bother with embedded http server or rpc etc. Add a --daemonize option (it's _really_ not too complicated) for customers who don't like crontab. And finally customers who don't want to run your agent can write their own against your published API. Maybe they don't want out bound http directly from their db. They could implement different transport and have a shim to your API service.

PS: keep batching in mind when doing API. It'll behoove you and the customer when you get to real RPS.

edit: wait, you want the agent to receive requests for data? From what other clients?

> This discussion is all about the agent that sits on the clients premises, correct?

Correct.

> wait, you want the agent to receive requests for data? From what other clients?

I originally had this in mind, but while working through the problem it is both unnecessarily and a potential friction point (firewall issues with outside clients starting connections, etc). I'll just make the client poll the server at intervals for whatever functions need doing.

> First define your server side RESTy JSON API. Then write poll based client agent, extract data, transform, PUT to API. Don't bother with embedded http server or rpc etc. Add a --daemonize option (it's _really_ not too complicated) for customers who don't like crontab. And finally customers who don't want to run your agent can write their own against your published API. Maybe they don't want out bound http directly from their db. They could implement different transport and have a shim to your API service.

Excellent, this sounds simple and robust. I realized last night that if the client initiates all the connections, all it needs to do is create HTTP connections to URLs. Nothing fancy. Should even be pretty easy to BasicAuth over SSL too, which will be important for security.

I know Python has a few libraries for daemonizing scripts, I'll investigate those after the basic crontab functionality is complete.

Thanks again everyone. Glad to see that I was over-thinking things. Simple is good =)

Reply on news.ycombinator.com