Clojure Protocols can extend final Java classes (like java.lang.String) | Hacker News Reader

Clojure Protocols can extend final Java classes (like java.lang.String)

clojure.org

76 pointsswannodette22 comments

Reminds me of jwz's Java doc on the topic of final variables, and how the access is restricted by compiler convention, not in the JVM:

    System.in, out and err (the stdio streams) are all final
    variables. They didn't used to be, but some clever applet-writer
    realized that you could change them and start intercepting all output
    and do all sorts of nasty stuff. So, the whip-smart folks at Sun went
    and made them final. But hey! Sometimes it's okay to change them! So,
    they also added System.setIn, setOut, and setErr methods to change
    them!

    ``Change a final variable?!'' I hear you cry. Yep. They sneak in
    through native code and change finals now. You might think it'd give
    'em pause to think and realize that other people might also want to
    have public read-only yet privately writable variables, but no.

    Oh, but it gets even better: it turns out they didn't really have to
    sneak in through native code anyway, at least as far as the JVM is
    concerned, since the JVM treats final variables as always writable to
    the class they're defined in! There's no special case for
    constructors: they're just always writable. The javac compiler, on the
    other hand, pretends that they're only assignable once, either in
    static init code for static finals or once per constructor for
    instance variables. It also will optimize access to finals, despite
    the fact that it's actually unsafe to do so.

> some clever applet-writer realized that you could change them and start intercepting all output and do all sorts of nasty stuff.

Can someone explain how this worked? What does he mean by "intercepting all output"?

Replace System.out with an object that responds to all of the same methods, but which does something clever with them. For instance one that adds timestamps on output, logs stuff somewhere, filters, etc.

This is a very powerful trick. However the downside is that it is in the category of things where only one person gets to be clever. Because the second person to replace that object breaks the first. So they decided to discourage that.

As simple as this:

  (defprotocol Foo
    (bar-ify [this]))

  (extend-type java.lang.String
    Foo
    (bar-ify [this] (.concat this "Bar")))

  (bar-ify "Hello world!")

Everything can be redefined interactively at the REPL. Also due to Rich Hickey's mad skills you get pretty much Java method dispatch performance.

EDIT: There is a gross misunderstanding about how this works. The following is invalid and will throw an exception, please take the time to understand this:

  user> (ns baz)
  baz> (bar-ify "foo")

Your changes will not affect use of java.lang.String in other libs.

I'd like to clarify that Protocols just generate a bunch of functions and a Protocol object (and an interface, but that's an implementation/interop detail). The functions are the interesting things in most cases. However, they're also namespaced like everything else. You can do the following though:

  user> (defprotocol Foo (describe [object]))
  user> (extend-type java.lang.String
          Foo 
          (describe [string] "a String"))
  (ns bar)
  bar> (user/describe "String") -> "a String"
  bar> (extend-type java.lang.Object
         user/Foo
         (describe [obj] "an Object"))
  bar> (in-ns 'user)
  user> (describe 3) -> "an Object"
  user> (describe "Hello") -> "a String"

So there is potential for two libraries extending the same things and causing a conflict if the semantics differ, but I'm not too concerned about that. It would be rather rude to extend someone else's protocol to cover someone else's java class without documenting that you do so.

Why's this worth noting? There's no reason you shouldn't be able to extend a type with methods to support a new interface, no matter what that type's author intended. It's not in any sense "dangerous" to existing APIs, as changing final variables could potentially be. It seems like completely natural behavior.

Maybe so, but it's not trivial to do in java. It's noteworthy because it gives clojure a lot more power when dealing with native libraries.

libs you mean. Native libs. :p

iPad sad trombone moment. ;)

It's not in any sense "dangerous" to existing APIs

If your job is to deliver and maintain such an API, then by allowing subclassing you have a larger contract you have to maintain when you want to make changes to the underlying class: rather than just the public interface, you also need to worry about the protected interface. If your API is new, you may want to shepherd your users (by marking a class as final) to favor composition over inheritance to maximize your ability to evolve the implementation in ways that don't break.

Ah, that's true, if extend can see protected members. I didn't realize that was the case.

EDIT: I'm not sure if it is the case or not; I can't find definitive documentation and I don't have a REPL available. Anyone know for sure?

extend does not create any kind of hierarchy, there's no subclassing going on here.

if you try to set a final field you'll get an exception as expected.

protected members when everything is immutable is pointless. Clojure cleanly separates values and identities.

protected members when everything is immutable is pointless.

Except when used to mark the boundary between a stable public API and an internal API that is free to change between releases.

@jjs, The contract is functions not members, and Clojure has a way to declare public and private fns, so what's your argument?

The contract is functions not members, and Clojure has a way to declare public and private fns, so what's your argument?

When I release a new major version of Foo Lib, and your program refers to private members, your program will break when I remove or rename them.

Immutability won't protect you from that.

Except if the public api of foo lib is only functions, which is the case for all clojure libs i've worked with.

EDIT: Ha, i just stumbled on this old yegge article, which is quite appropriate i think. With clojure APIs you think in verbs, so nouns are not that important

http://steve-yegge.blogspot.com/2006/03/execution-in-kingdom...

EDIT2 : perfect quote :

"In the remotest regions, beyond the Functional Kingdoms, lies a fabled realm called Lambda the Ultimate. In this place it is said that there are no nouns at all, only verbs!"

here you go ;)

"In the remotest regions, beyond the Functional Kingdoms, lies a fabled realm called Lambda the Ultimate. In this place it is said that there are no nouns at all, only verbs!"

I already live in that Kingdom, so when I say "members", I mean member variables or member functions.

by allowing subclassing you have a larger contract you have to maintain when you want to make changes to the underlying class: rather than just the public interface, you also need to worry about the protected interface.

In other words, if you allow subclassing, then you also have to worry about the interface that you have explicitly marked visible to subclasses.

Normally a non-problem, by definition.

final is useful when sealing a class that inherits from a parent class with a protected interface, but even if "final" is reduced to an annotation (or even a comment), its presence explicitly limits your end of the API contract.

There is no subclassing going on. Take the time to actually read the documentation provided by the link.

The italicized portion at the top of my comment was a quote from the parent comment.

Does is actually extend the class or simply wrap it with a decorator? I think Groovy takes this approach but I could be wrong.

It does neither. The dispatch logic is on the protocol side, which can be modified. When a class is extended, the protocol function is changed to have an extra instance check. (I don't understand the details in full, though. You'll need to ask Rich Hickey)

AFAIK there is also some kind of caching involved to improve performance.

Reply on news.ycombinator.com