The impact on middleware of expanding APIs with Go's interface smuggling | Hacker News Reader

The impact on middleware of expanding APIs with Go's interface smuggling

utcc.utoronto.ca

45 pointszdw17 comments

I had a lot of problems with this in rclone.

Rclone has backends which connect to cloud providers. These all implement a common interface. So far so good.

However some cloud providers can implement optional features, say a server side copy of an object.

Rclone used to use interface upgrades to discover these. Still so far so good.

However rclone has backends which wrap other backends, for example the crypt backend which encrypts file names and data.

This backend has to implement all the optional features - you want to be able to server side copy an encrypted file if possible.

The problem comes when the crypt backend wraps another backend which does not support server side copy.

What rclone used to do here is return a special not implemented error. This works, but isn't ideal because you have to call the method to find out if it is supported and often you'd like to know before that.

I eventually gave up on interface upgrades and resorted to good old function pointers which you can check against nil. When the crypt backend starts up it finds out whether the backend it is wrapping supports server side copies and if not nils the pointer.

A little bit of reflection removed the boiler plate for this and it turned out quite neatly.

If instead there was a way for crypt to remove methods from its method set then that would be a better solution.

The same problem was discovered 15 years ago in Java :)

The first attempt to fix it was to use dynamically generated proxies. The IAdaptable mechanism proposed by Ken Beck is an example of that for Eclipse [1].

This was ultimately fixed by introducing default methods which is as restricted way of implementing Traits [2] in Java 8.

I guess Go will have to follow a similar direction.

[1] https://wiki.eclipse.org/FAQ_How_do_I_use_IAdaptable_and_IAd...

[2] https://en.wikipedia.org/wiki/Trait_(computer_programming)

> The same problem was discovered 15 years ago in Java :)

I swear go is different and doesn't need to learn from mistakes from other languages, because after all, you don't need that with go…

Instead of being arrogant maybe go designers should have listened to the people who told them 10 years ago that these issues were real, and not a product of "bad coding practices". Because this:

> . The Prometheus people try hard to do it anyway, and the result is rather messy and involves a combinatorial explosion of the possible combinations of APIs.

Is madness.

Go has benefitted immensely from not starting out looking like Java from day 1. No one doubts that these edge cases come up from time to time, but how can Go know which features of Java exist because of hard-won experience and which aren’t? Further, even when there is a use case for a given feature, that use case is often marginal and the feature people want would impose on every other use case—in such a case, it’s often better to drop the feature, and Go has profited tremendously in this regard. But yeah, sometimes it misses some features that Java has and it has to add them back in, and the Java people think it’s cause to dunk on Go, but Go is still profiting from all of the misfeatures that are in Java but not in Go. Anyway, people make far too big a deal about things like type checking (if a language is statically typed at all, that’s 95% better than dynamically typed languages and everything else is just bickering about the additional 5%) and other language features and consistently undervalue tooling, static linkage, ecosystem, learning curve, and simplicity.

Go didn’t get everything right, but I’m happy it’s not Java.

> Go didn’t get everything right, but I’m happy it’s not Java.

Go has benefitted immensely from not starting out looking like Java from day 1

But it actually kind of did when it comes to generics.

> Go didn’t get everything right, but I’m happy it’s not Java.

Well a good thing generics are on their way then?

The comparison was between Go and Java circa 2012 when ago was first released. Not between 2012 Go and 1995 Java. Notably, Go has things like closures and implicit interfaces which make it much easier to write more abstract code than pre-generics Java (which depended heavily on downcasting).

I’m not sure if generics will be a net benefit to Go. I often wish I had them, but I rarely need them to make code type safe. Further, I don’t think Go will be able to retain its property that solutions implemented in ago tend to be flat-footed/boring/not-overly-abstract nor the property that different developers tend to approach similar problems in very similar ways (I.e., Go tends to be written in very standard ways). Of course, if you’ve never used Go for a considerable time, you will likely devalue these properties, but they turn out to be really useful, especially in a collaborative setting. Maybe they won’t be lost when generics arrive, or maybe generics will be a net positive. I hope so, but I’m concerned. I think Go is quite a lot nicer to read and write than other languages with generics, and I would be sad if it backslid.

Why stop at Java? Isn't this solved by IUnknown's QueryInterface from COM?

Right... for people who don't know, QueryInterface was the fundamental method client code would call on a COM object to get a particular interface.

The key here, is the object can implement this any way it wants.

A typical "static" object would just return itself as the requested interface for the interfaces it supported. But a proxy object, with a wrapped upstream object, could pass through requests for interfaces it didn't know or need to override to the upstream object.

I don't know Go, but a quick google makes me think type assertion is the mechanism for getting an interface from an object at runtime. So if Go allowed a type to override type assertion, I think that should enable middleware and other proxy objects to behave well.

This mechanism leaped to my mind as well as a pretty common (and simple) design pattern for this particular problem.

AFAIK with package `reflect` it is not possible to attach methods to a dynamically create type. And that seems to be a prerequisite to be able to generate dynamic proxies at runtime.

Java can modify bytecodes since the JVM ClassLoader is a first class construct in Java. Using distinct ClassLoader roots, one can have multiple diverse versions of the same canonical ‘type name’ (e.g. foo.bar.paz.MyList) at the same time in the same running process.

> People can work around this for HTTP middleware because they can make files build only on specific minimum versions of Go. Your package doesn't have this magical power; it's something available only for new APIs in the Go standard library.

It is indeed possible to add build constraints to .go files to have them included for compilation only on specified minimum versions of Go, see [1]

[1] https://golang.org/pkg/go/build/#hdr-Build_Constraints

That's what your quote says. However, what it also says is that functionality is only available for Go versions, and therefore Go standard library versions. It is not functionality available around third-party libraries that are developed independently of the Go release cycle.

Hopefully generics will help with this, if the middleware can deal with T then the calling code can “smuggle” whatever types it wants because it controls T.

This is what happens when you design in a vacuum, ignoring the lessons learned by those who came before you.

I know, every language suffers from this to some degree, but somehow it feels like go gets bitten a lot more than the others (except maybe PHP).

You have to compare everything on balance. By not setting out to reinvent Java or C++ or whatever from data 0, Go doesn’t have a lot of features that chafe developers on a daily basis. But yeah, it means it sometimes misses a feature for some use case. But even then those use cases are often marginal, and omitting the feature is often a better design decision. I’m pretty happy with where Go landed—I’m glad it doesn’t look like C++ or Java or C# even if those languages have some cool features. I think on balance Go is an incredibly productive language that performs well on almost all criteria even if it isn’t top of class in any (except simplicity and perhaps developer velocity).

You already said that.

Reply on news.ycombinator.com