Hacker News Reader

Comment by zaarn | Hacker News Reader

The fun thing about the Go compatibility promise is that it doesn't work as well as the go developers probably imagine; just last week one of my older Go project was discovered to be broken; wouldn't compile anymore. golang.org/x/crypto was responsible, they used some code that wouldn't work in more modern go compilers anymore. Granted, I had not maintained the project in 3 years but I also had everything vendored just in case go decided to break anything upstream, and the broken parts didn't even touch on anything that the compatibility promise excludes.

I think the Go devs should improve in that section a bit.

View on HN

I had the same issue. The issue was a security one https://nvd.nist.gov/vuln/detail/CVE-2019-11840 which required a breaking change to fix so their options where leave everyone vulnerable but could compile or break anyone that uses this code and fix it. I think they made the right choice.

Also golang.org/x/crypto is not a core library so doesn't actually fall under the compatibility guarantee but they seem to try their best not to break it.

Neither of those things applied though; I had x/crypto vendored, so it didn't change it's code either and the CVE is unrelated to the actual code that broke (I used blake2b) and again; it was vendored, so my code wouldn't break but be vulnerable.

How did vendored code that didn't change break your build?

The easiest way to accomplish that is definitely with //go:linkname

You can use a comment of the format '//go:linkname [localname] [method]' to link some function as another one. Notably, this includes unexported private methods in the go runtime and stdlib.

I've seen code in the wild that uses this to grab go's map hash algorithm, get monotonic time, and other things.

The go authors have used this hack themselves in the stdlib often enough because the runtime doesn't expose some knob, and rather than thinking that perhaps other developers may want that knob too, they instead use such hacks, but they also don't support said hacks (because how could you, obviously keeping all private functions stable forever is silly and impossible)

I updated the compiler.

That was golang.org/x/crypto's fault for using internals not covered by the compatibility promise [0], sorry about that.

However, we try to hold x/crypto itself to the same compatibility standard as the standard library, so I think simply updating x/crypto at the same time should have made it a smooth update.

[0]: https://news.ycombinator.com/item?id=23164795

I just compiled a fairly complex (CGo etc...) project a few weeks ago which I hadn't compiled since 2014, and it just worked.

So I guess YMMV...

Of course, not every project will fail, but in my case it failed and I think that is the issue.

Do you remember the specifics? I am not aware of syntax constructs that were removed in Go, I guess ever.

EDIT: I'm sad folks are downvoting for asking a question...

Older versions of the /x/crypto library no longer compile on newer go versions, there wasn't any indication this would happen and I vendored the library to avoid the API breakage in that library. From my standpoint, that completely ruins using that library as after 3 years either I vendored it and it breaks because of something the go compiler does or I didn't vendor it but the API changed. Either way, my program will break eventually and there is no way to avoid it.

Well, that's for the better still.

If you use vulnerable crypto (willing to go with old unpatched vendored code) then you don't really need crypto at all..

@zaarn explained in another comment that the vulnerability did not impact him, he did not exercise that code path.

I understand that, but it's not really relevant.

The code path that failed to build was still related to the included crypto (even if it was due to compiler change).

If that was their only way to fix the crypto issue, then it's totally fine, because that way they fixed it for people who did need and exercize the crypto code path -- and those take priority over people who "bundled some crypto code but didn't exercize that code path and their build still broke".

I exercized a code path that broke, it's not like unrelated code broke, it was code I was actually using in a commit in the upstream library that changed not much for the end user and then was broken by the compiler fairly soon after.

The issue is related to AVX detection in the CPU, which has the reasonable failure path of "don't build it with AVX".

I'm sorry you had this experience. I'm genuinely curious as to what the breakage was so hopefully we can prevent it for some other folks, hence why I asked if you remembered the specifics.

Basically, x/crypto/blake2b received a patch to switch how it detects the presence of AVX and SSE at runtime, the symbols required for that to work with earlier versions where removed. While I understand this is technically covered in the compatibility promise, it's very frustrating when it happens with a library developed by google, especially when, at the time, the general recommendation was to vendor them so as to not suffer from breakage when they update.

I believe this commit has background: https://github.com/golang/crypto/commit/d6449816ce06963d9d13...

To be fair though, that package was depending on a private API to begin with.

Isn't /x/ experimental by definition? They introduced a breaking changed to an experimental library. This does not fall under the scope of the compatibility promise. The compatibility promise works great.

One of those point under "Expectations" can be explain in this page

https://golang.org/doc/go1compat

Nope, none of these points apply to the specific change, atleast as far as I am able to tell. The code that broke did not use unsafe when I last updated it and then it simply didn't work anymore in the current go version.

I agreed with jeffsmith82's comment.

That comment isn't applicable either; I had x/crypto vendored so any fix upstream wouldn't break my app.

I'm afraid blake2 used assembly and runtime.support_avx2, which doesn't seem to be covered by the compatibility promise: assembly is part of the toolchain, and the symbol it used was an internal one.

See:

https://go-review.googlesource.com/c/crypto/+/110355/

https://go-review.googlesource.com/c/crypto/+/110016/

So, my code broke because a library that was provided by the go developers used some unstable feature I had no idea about and had no real way of knowing short of checking a dependency three levels below my project? And to top that, that library was certainly subject to API breakage, the entire reason I vendored it in the first place.

It doesn't seem reasonable to me. When I developed my project, vendoring /x/ projects was recommended exactly due to the breakage, so now it breaks my projects because of recommendations of the golang team.

I believe one of those probably applies.

> Use of package unsafe. Packages that import unsafe may depend on internal properties of the Go implementation. We reserve the right to make changes to the implementation that may break such programs.

I suspect the crypto code you vendored used unsafe somewhere within it (whether to use `//go:linkname` or for other reasons), which would except it from the go1 compatibility promise.

I'm pretty sure x packages are not part of go core.

x packages should still compile on newer go compilers in my opinion. Their compatibility isn't guaranteed if you don't vendor it, which was a recommendation when I started the project and as I pointed out in the comment you're responding to, it was vendored, so it didn't break itself.

Reply on news.ycombinator.com