We've been fighting this for ages in Webmin/Virtualmin.
We've done the following:
XML-RPC for the core libraries in an almost unwrapped variant (you have to have knowledge of the underlying system to make use of it). Almost no one uses this.
Simple command line utilities that can be called from shell scripts easily, using standard GNU argument conventions ( create-domain.pl --domain virtualmin.com --pass fooby --desc "Cool hosting management software" ). Virtualmin is a system administration tool, so this is our most popular variant...all of our customers know basic shell scripting, so this is a natural fit for them. I suspect we'll eventually make these into commands in the path to make calling them easier (right now they're in our application directory in /usr/libexec or /usr/share).
Remote variant of the above that can be called via simple POST or GET requests: https://domain.tld:10000/virtual-server/remote.cgi?program=l...
This remote one will become RESTful at some point in the near future--I just need to add some rewriting functionality to our webserver to handle mapping /virtual-server/list-domains into the above full path. This one is also very popular, particularly for folks who like to work in PHP. Most of the billing apps that work with Virtualmin use this API.
And, we've just built a Webmin::API Perl module that finds a local Webmin installation and makes the existing API available to Perl programmers.
All but the last one are language agnostic, though not necessarily as tightly coupled as one might like. I've been working on JSON providers for a lot of the menus and such in our product...and the more time I spend with it, the more it seems like an ideal mechanism for building language agnostic tools (every popular language has a JSON parser/deparser and a web client library). Since you can ship out raw data structures in JSON, it's really quite neat. You could build the same data structures in a wrapper for any language pretty trivially. Though the labor involved would be sizable for a large application with a lot of data structures, it's no more intensive than any other method I can think of. I'm working on a blog post about this as we speak, actually.
We've considered using Thrift, as it would also provide a language agnostic data-structure passing system. But I think JSON is easier to deploy, easier for folks to understand and use, and a lot more popular. The only benefit I can think of for using Thrift over JSON would be performance (maybe), so it's no longer on our radar.