Ask HN: Best practices (and examples) for designing client libraries for APIs? | Hacker News Reader

Ask HN: Best practices (and examples) for designing client libraries for APIs?

97 pointsakudha20 comments

I'm writing a client library for my favorite project management app's API, in PHP. Are there any best practices that I can follow? Can you recommend a good example of such a library (either written by the API provider themselves or by a third party) that I can learn from?

Doesn't need to be in PHP - Java, JS etc works too

View on HN

I usually like to have some Client object that’s responsible for establishing connections and performing unauthenticated interactions with the API. For things that require auth, I usually hang a Login / Authenticate method off of the Client that does whatever auth is necessary (you can even have multiple if there are multiple strategies). Those authentication methods usually return some kind of Session that has whatever token is ultimately produced as state. Authenticated methods hang off the Session object. EDIT: One other nice consequence of this design is that a separate Session makes it possible to have multiple simultaneous authenticated sessions while still maintaining the ability to call unauthenticated methods (e.g. Ping()).

I’ve found this prevents users of the library from having to shuttle around credentials and accidentally calling authenticated endpoints without credentials (since those methods don’t even exist off the Client). Also, if the Client is responsible for managing connections, it can also deal with things like rate limiting and whatnot. Finally, having a separate Session object eases testing, as you don’t have to mock the entirety of the auth flow (think of the complexity of OAuth 2.0) in order to get to a state that you care about. You can simply start with “given an authenticated session...”.

There are some other pieces that can be useful too depending on the abstractions available to you in your language of choice. Sometimes I’ll include a lower-level Request to do basic URL construction given a higher-level map of parameters. Corresponding Response objects can occasionally be useful too in those scenarios to unpack JSON / XML / w/e and present a higher-level construct to your methods.

I agree that a good idea is having a Client object, which main concern is to authenticate to the API and make the first connection. This Client should be in charge of creating instances of each resource available in the API. Don’t try to just offer a replica of the web API: you should figure out which are the most important use cases for the API and then offer them to your end-user. And it’s important to prioritize so you don’t clutter the public interface of the library. If you want to share common features across your resources, you could create a mixin with them.

If you want to check an example, we released just a few days ago a Python-flavored client [1]. The code is readable and still in early stages; hopefully you can borrow a few ideas from there. These guidelines are mainly based on OOP, and Python with its data model (i.e. dunder methods) is flexible enough to offer a great user interface.

[1]: https://github.com/fintoc-com/fintoc-python

Edit: wording

This will only address part of your question, about API design.

I'd highly recommend Josh Bloch's writings. Josh is perhaps best known for the book Effective Java and the Java Collections library. Here is a paper he wrote about good API design (https://dl.acm.org/doi/abs/10.1145/1176617.1176622) and a short interview with him about API design (https://www.infoq.com/articles/API-Design-Joshua-Bloch/).

Brad Myers at CMU also has done research on API usability. See here for more details: http://www.cs.cmu.edu/~NatProg/apiusability.html

Lastly, if you're doing anything remotely related to security, I'd also recommend Matthew Smith's research. He's studied a lot about weaknesses of today's API designs and how they have led to security vulnerabilities. https://ieeexplore.ieee.org/abstract/document/7676144

I've done this a lot, and I'd recommend using HTTPlug here. PHP/composer allows you to require an implementation instead of an actual package, and that lets your users pick the HTTP implementation they'd like to use (Curl/Guzzle/Socket etc).

http://docs.php-http.org/en/latest/httplug/library-developer... is the starting guide and https://packagist.org/providers/psr/http-client-implementati... is all the various providers you will automatically support this way.

There are other pros: you'll be using PSR standards, so it becomes easily extensible. However, there are some limitations, especially if you want to make fairly complex multi-part file uploads (this might have improved since, I'm not sure).

It may not be a pattern that works well in every language, but I have used and quite enjoy this [1] approach from Ruby.

[1]: https://medium.com/rubyinside/building-a-creative-fun-api-cl...

Creating good APIs is about developing a good taste and not surprise the user.

In your case, the first job of designing the API is done for you by the app.

So as a first step, create the foundation layer by mapping the app's API to simple objects that represent the app's API.

Then you can build a layer above to simplify certain operations and build some logic on top.

Think in terms of responsibilities, relationships and who should know about what. That's how you achieve code reuse. When you find yourself reusing your own constructs, it means you've done something right. I find it's often that developers aim for code reusability, but IMHO code reusability is just a natural outcome to good design decisions.

Use your API by writing pseudo code and see if it makes sense.

Keep iterating. Avoid trying to nail it from the first go. IMO, building functionality is more useful than designing a perfect API.

This article could help:

https://www.moesif.com/blog/technical/sdks/Best-Practices-fo...

Personally, for building a client library I'd take a look at OpenAPI Generator (https://github.com/openapitools/openapi-generator) or Swagger Codegen (https://github.com/swagger-api/swagger-codegen) - especially if the API provider has an official OpenAPI/Swagger specification. (If not you can always write a specification yourself). Both tools support PHP among other languages.

I can't recommend OpenAPI/Swagger-based code generation at all based on my experience working with it. I've only used it for Java, Kotlin and TypeScript, but the generated code (usually more then you asked for) only works like 90% of the time and getting it to 100% takes workaround after workaround. A really frustrating experience.

I must say, however, that it may work for simple APIs but even then usually only for generating the model part and not the API part. What kind of worked for me though was generating code for the server in form of a Kotlin interface for a Spring MVC controller. Although, here too, I had to modify the code generator templates to tailor it to my needs.

Regarding API client, when you can successfully generate an API client with code generation, it should be considered a low-level API client upon which one should build a high-level and more user-friendly API client (e.g. object model with actual methods and not only anemic objects).

The clients look terrible and awkward to use thought.

Automated testing is a generically good technique for many things including forcing yourself to have a better API. Code sometimes has to be refactored to make it more testable, and this refactoring invariably implies a better API.

Write automated tests to hit each part of your API and test the functionality of each thing in isolation. If you find the tests frustrating to write, your users will find the API frustrating to use.

I've written more about this here:

https://bad-code-considered-harmful.blogspot.com/2020/05/tes...

By the way, I am not an advocate for TDD. I write my tests at the point when I'm ready to start running them. But I strongly agree with the general claim that TDD makes that says that good unit tests are the closest thing to a panacea you will ever encounter in software development. However, I've also seen projects where the unit tests were just more bloat. Unit tests need to be written to test your API to help you design a better API.

A bunch of good info here already, although it seems some commenters have missed that this is about writing an API client library. Generating from an API spec isn't a bad idea, but can often provide a fairly low level interface that may not always be what's needed.

Here are things I'd look for in a good client library:

- Feels at-home in the language, makes use of language idioms.

- Few dependencies, wide version requirements on language and dependencies, needs to fit into a wide range of use-cases.

- Ability to pass in networking/transport options/session/etc. Needs to work behind proxies, with custom networking requirements, custom headers for proxies, or even allow mocking out HTTP for testing.

- Thorough test suite on range of versions.

- Clear costs of calling functions. Time complexity of functions is unlikely to be the limiting factor, but knowing how many HTTP requests a given function makes could significantly change the usage of the library.

- Authentication in as many ways as possible. Many use-cases have wildly different auth requirements. Google's Python packages are quite good at this.

I’m looking for a resource to gently pass to my client to help coach him up. He wrote all the server APIs, I write one of the mobile apps. His APIs are all massive JSON dumps of columns from the database where most of the fields have no use on mobile. The documentation is an out of date printout of the structures with comments, in Ruby I think.

Worse he doesn’t seem to do any data validation, I recently passed an index instead of an id for one field and ended up with tons of test data that’s broken. Not because of a bug in my code mind you, but because I misunderstood the cryptic comment.

Lastly, he occasionally asks the mobile apps to do processing of the data that’s more easily and safely done on the server. It’s like he misunderstands the role and value of the server in a client server application.

But again he’s my client so I have to be gentle.

Stripe has probably the best and cleanest API design.

And documentation. I would spend some time really understanding their docs and client libraries if you want to see a real world example of the most developer friendly docs available.

https://stripe.com/docs/api

https://stripe.com/docs/api/balance/balance_object?lang=pyth...

Is there any documentation of the client libraries themselves, or just the API?

Yes. It's integrated into the docs. If you look to the right you'll see code samples and you can choose different languages from a drop down.

https://stripe.com/docs/api/balance/balance_retrieve?lang=py...

The client libraries have stand alone documentation as well.

https://github.com/stripe/stripe-python

As a user of your library, don’t make me inherit from your base classes. Allow me to couple loosely to your objects, methods and functions; do not make me write everything centered around your code.

If you are lucky enough that your API vendor provides an OpenAPI (aka Swagger) specification (or possibly API Blueprint or RAML, if nothing else), make sure you build your client to follow the spec. Even if you are possibly aware of some undocumented features, try to avoid relying on them -- they might change or disappear without a notice.

I would also highly recommend to use the spec to dynamically construct your requests. I don't know if PHP has libraries to assist with that, but for example in Python you could use tools like Bravado or Pyotr client.

Honestly, best practices depends on what you are trying to achieve. Why is the API painful to use? Try to address those aspects when creating your client library.

Many API's are easy enough to understand, they don't necessarily get lots of benefit from an official client library. Developers just write the HTTP request code themselves for whichever endpoints they are using. keep it simple if you are first starting.

create a separate file, NameAPI.js, to house your endpoint calling functions. Make it easy for the caller to provide necessary values to those functions. Sometimes API endpoints have a lot of extra optional params/properties. Don't worry about including those in the function interface unless you are actually using them for your use-case--instead choose suitable default params/properties whenever possible.

Then continue to build out your client functions as you need more endpoints or endpoint features.

I would say sometimes its helpful to provide multiple client functions per one endpoint, especially if this endpoint is already loaded with lots of different choices for params/properties. EX: ive seen create user endpoints that require a non-descriptive type property like 1 = basic user 2= admin user etc. well, you could have two client functions createUser vs createAdminUser. In this way, you are taking off some of the burden for the caller to figure out which type number they need to pass, and instead giving them natural language functions which configure and run the HTTP request for them.

A nice to have: make it easy for the caller to pass properties using host language conventions like property name casing. ex: if the host language is in javascript, it may be easier/more consistent for the caller to always provide camelCased params/properties to the function, even if the API endpoint expects snake_case for params/properties (translate the names for your caller).

be clear about how you are handling error conditions in your client functions/package. I think of two basic types of error conditions, network errors vs operational errors. Network errors would be when client has not internet connection or times out for whatever reason. Operational error would be when server DOES send back response but its an error response (like 400's client provided bad values or 500's server is malfunctioning). And will you be throwing errors on network errors AND operational errors? or just throwing on network errors? or not throwing at all (returning some value instead)? basically, how does the caller know when there was an error and what type of error? In Go, i've seen sql clients go through the trouble of returning custom error types for every possible sql error that could happen. That gives the client more potential options to decide how to recover/resume.

If you are first starting, I honestly wouldn't worry about including auth handling within the client. I find it usually confusing more than helpful. But it depends on the protocol and auth method. Perhaps auth IS one of the most painful points of using the API, in that case building some auth management into the client may be helpful. But HTTP + Authorization token is so straightforward and common, developers can manage that auth cred easily in their own way. In that case, just make the token a parameter that must be passed to the client function. That's the simplest way to start.

Next option might be to allow the caller to create an instance of the client for a specific auth cred. Then the caller can use this instance whenever it needs to call client functions given that auth cred, or create multiple instances each with a separate user/service auth cred.

if you are hosting this client as a package for others to use in the community, be sure to stay on top of any API changes which necessitate your client package to be updated. Provide clear documentation about any API changes and how your package has addressed them.

Reply on news.ycombinator.com