HNHacker News
TopNewBestAskShowJobs

weberc2

4,767 karma · joined November 25, 2014

submissionscomments
weberc2··on Python at Scale: Strict Modules
I'm not sure (1) exists, probably by definition. And I certainly don't agree that Python makes it easy to "do it correctly". Our Python app has daily 500s due to typing errors. We also suffered for years because we would build magical things that we thought would work in every scenario but ended up being untestable and/or failed to consider numerous edge cases ("what happens if someone inherits from my magical class?") and/or which failed to extend properly ("oops, someone renamed this attribute and now all of our hasattr checks are broken, and the tests didn't catch it because they passed mocks"). Eventually we built a culture that mostly discourages magic/gratuitous dynamism, but it took years and we're still suffering from that legacy code.

These problems simply don't crop up in Go, or at least they're in a different ballpark in terms of frequency and severity. So yeah, Go lacks typesafe generics, but I'll make that tradeoff all day every day in exchange for the maintainability, performance, tooling, distribution, etc improvements that Go offers today. No contest.

weberc2··on Oxford Professor Is Accused of Selling Ancient Texts
Is there anything to substantiate this, or is this a generic "Christian therefore bigot" slur?
weberc2··on Bank of America's CEO says it's saved $2B per year by building its own cloud
No straw men here Greg. Looks like you’ve confused this thread with another.
weberc2··on Apple CEO becomes chairman of China university board
Can you elaborate on the importance and how that factors into Apple’s strategy? Saying that China is important and Tim Cook is Very Smart doesn’t make for a very enlightening comment nor a suitable rebuttal. If you’re onto something, I’d like to know what! :)
weberc2··on Bank of America's CEO says it's saved $2B per year by building its own cloud
No, it wouldn’t make sense for me to make the 95/5 distinction only to prescribe a 100/0 response. My post was a call for sober judgment, which is admittedly not very popular here.
weberc2··on Python at Scale: Strict Modules
> Among those I work with, my opinion seems to be shared.

Yeah, preference distributions are hard to assess. Either of us could be wrong.

> I didn't say otherwise. What I did say is that go adds visual noise that isn't present in python. (and it is noise: the proposal to add try! shows that the error handling style is noisy. It can be basically entirely removed by an automated transformation).

I’m glad we agree that terseness is not readability and visual structure is valuable. How do we meaningfully debate whether some boilerplate is noise or useful visual structure? Why is Python’s implicit propagation of errors elegant and beautiful visual structure while Go’s explicit error handling is ugly noise? Specifically how do we know that you aren’t prejudiced by your disproportionate experience with Python (even assuming my disproportionate experience with Python and preference for Go is an outlier)? What are the criteria?

weberc2··on Bank of America's CEO says it's saved $2B per year by building its own cloud
Why is it better to be locked into your own company’s IT department rather than AWS? I would argue AWS gives a better ROI than 95% of corporate IT departments, and generally less hostile to boot.
weberc2··on Bank of America's CEO says it's saved $2B per year by building its own cloud
> It will continue to rise because it offers flexibility ... The cloud is more expensive for large companies, for most workloads.

The way I read this is that the operating costs are higher in the cloud, but presumably dwarfed by recouped opportunity costs. This makes sense to me, given my limited experience working in large companies where every interaction with IT involved a 2-6 week turnaround time. Need a VM? Fill out a bunch of paperwork and then wait a month. We missed a lot of opportunities that way.

weberc2··on Python at Scale: Strict Modules
> I agree, but if you for instance look at the TypeScript comparison sub-thread, you'll see that all the issues with both the syntax and implementation of the type-system are being aggressively resolved, and likely will all be so by 3.9.

I'm happy to hear that; hopefully the efforts really do address these issues well.

> I find poetry a joy to use. If you want to bypass venvs all together, there's a lot of work to make that a reality, such as https://github.com/David-OConnor/pyflow.

I'll have to check those out, but one inherent problem is that even if these tools really do solve my pain points, adopting them means I'm leaving my org on a relatively small island, isolated from the Python community. If these really are the holy grail, why isn't the broader Python community adopting them? Please don't take this as me looking for something wrong--whatever Python build tool I use, I'll eventually need support and there's a lot to be said for having a thriving community that has almost always run into my exact problem before.

> Python in 3.5 added complete zip app support, which has improved this dramatically from my perspective. Extended by things like shiv https://github.com/linkedin/shiv make it fairly complete.

We're currently using this via pex. It mostly works, but we still run into problems occassionally (system dependencies, for example). Figuring out how to integrate these tools into the broader build process is another problem to solve--we're using `pants` which supports pex out of the box, but we're running into lots of bugs or other problems. I'll keep an eye on shiv.

> This is interesting to me. I prefer async/await in general, because it has become a standard across programming languages and I find it really easy to reason about. I also find channels to be too widely seen as a cure-all, when the only study so far has shown they actually led to an increase bug count. But I don't discount the value of real-parallelism, and am glad to see that Python has been pushing harder on that lately, with things like subshells that allow bypassing the GIL on a single thread.

My biggest issues with async/await are

(1) every package needs an async variant (async boto, async docker, etc etc). We work around this by running them in a thread pool executor, and I think that works, but I don't know if I'm holding the GIL unnecessarily and causing performance issues (fundamentally difficult to diagnose). This is roughly the "what color is my function" problem.

(2) it's really easy to starve the event loop by calling into something that transitively makes a sync call or otherwise just does a lot of CPU-heavy work. We've run into both kinds in production and they've been really hard to troubleshoot (because the requests that time out often aren't the ones that are actually causing the problems).

(3) dynamic typing means it's super easy to forget to await things. Tests should catch this, but we find ourselves writing tests _just_ to catch this (e.g., we now write tests for entrypoints that _just_ `await lib_function(params)`; we would normally not write tests for such simple functions, but now we have to). Static typing is the right way to solve this and mypy does, but mypy has too many other issues (at the moment) for our org.

One substantial criticism of goroutines is that they're less safe than async/await because you need to make sure the code you're running is threadsafe. I appreciate this criticism, but I think it's the right tradeoff for Go's performance aspirations (another great high-performance alternative is Rust's borrow checker, but that's the wrong tradeoff for Go's developer productivity aspirations).

weberc2··on Python at Scale: Strict Modules
> All my projects now use poetry for the full build tooling and I love it. No setup.py needed just include any settings in the standard pyproject.toml file example: https://github.com/timothycrosley/portray/blob/master/pyproj..., which can be generated with poetry's help using poetry init.

We have yet to try poetry in our org. I'm hesitant to stray off the well-trodden path, but it might be worth a shot. Any idea about installing packages with system dependencies? Packages like `pygraphviz` (which depends on the `graphviz` or `graphviz-devel` system library) has always given us a lot of trouble, for example.

> I feel like Python with type hints (for all their current flaws) does give you this exactly.

I've noticed that some editors try to use these hints, but they seem to have a hard time in many cases loading the modules. It's possible that the editor extensions (e.g., VS Code) are just buggy, but it's still a problem. Further, they require that all of your dependencies have annotations or type stubs.

> Personally, I like portray better than anything in the Golang world for this https://timothycrosley.github.io/portray/ I may be biased since I wrote it.

I haven't tried portray.

The killer thing about Go's documentation generation is that it uses type annotations and exposes them in the generated documentation. This is critical because 95% of the reason I'm looking at documentation (especially in Python) is because I need to know the type signatures (and often Python docs omit types, or the types are wrong or vague--e.g., "the type is 'binary'" with no indication if that means a bytestring or a BytesIO or what). This is tablestakes for documentation systems in statically typed languages, but I have yet to find a Python tool that does this well. Further, `godoc.org` also generates links to types including across packages--this is _not_ tablestakes for statically typed languages--so you just have to click the type name and it will take you to the docs for other packages. Further, there is no CI needed to build/publish your documentation; `godoc.org` just needs access to your repo on github or elsewhere (you can run your own godoc.org inside your corporate firewall). Another nice-to-have feature is that documentation is just comments; there's no formal/obscure syntax a la sphinx.

> I write a lot of Python tools so I'm genuinely curious because if there were unfilled needs I would want to address them as one of my 52 projects: https://timothycrosley.com/

Cool. I'll take a look!

weberc2··on Python at Scale: Strict Modules
Like I said, I disagree with this. I suspect this is either because you're very experienced with Python and relatively inexperienced with Go, or perhaps you're simply an outlier. I think if you surveyed developers who are very experienced with Python and have at least a few months of experience with Go, you'll find people say that it's easier to identify issues in Go code--and I think this largely comes down to the role the boilerplate has in visually "framing" or "structuring" (i.e., providing "shape" to) the code.

Have a look at Haskell which goes to great lengths to eliminate boilerplate and I think you'll experience the opposite--Haskell becomes very difficult to read precisely because the code is so dense. Similarly, take the indentation, newlines, etc out of a JavaScript file or JSON blob (minify it, more or less) and see if it's more or less readable as a result. I think you'll find that visual structure is actually important.

weberc2··on Python at Scale: Strict Modules
Yeah, I get it. It’s a little disingenuous of me to say that interface{} qualifies as generics, but I can’t quite put my finger on why it is different than Python. Neither are typesafe (although mypy supports generics, but has many other issues), but in any case typesafe generics would I think improve Go.
weberc2··on Python at Scale: Strict Modules
Go has those features in the reflect package (so as far as I know, Go is just as powerful as Python), but you’re right that they aren’t easy to use. If you do use them, it’s quite clear, and will be addressed in code review so you don’t have nearly as many dynamic typing bugs as Python—it’s not anywhere close.

Very dynamic code shouldn’t be easy; the happy path should encourage clear, simple code. By encouraging people to stay on the happy path, their code is more performant, maintainable, etc and it keeps the average code quality quite high across the ecosystem.

weberc2··on Python at Scale: Strict Modules
These are all fair points. I really enjoy Python, but there are too many things I fight with on a regular basis that simply aren’t issues in Go. It could be so much better if (1) there was a better type system (mypy is unnecessarily shoehorned into the syntax and still very broken—can’t even express recursive types like JSON), (2) a good way to constrain the dynamism so performance could be improved, and (3) a better environment/package management and distribution story (so far pantsbuild.org and PEX files are the best I’ve found). Then there are a long tail of more minor issues, like async/await vs goroutines, real parallelism, etc.
weberc2··on Python at Scale: Strict Modules
Build tooling (“go build” vs setup.py), type checker, text editor support (hovering over a symbol for the type and docstring), documentation generator / godoc.org, dependency management (pip is great but it’s not reproducible; go’s toolchain is only modestly better here IMO), no need for virtualenvs, etc. I’m sure I’m missing several.
weberc2··on Python at Scale: Strict Modules
The opposite is true in my experience. Most of that boilerplate is brackets and indentation, which visually frame the interesting bits, drawing your eye to them. This is, of course, subjective, but I use both regularly and at worst this is not a problem for Go.
weberc2··on Python at Scale: Strict Modules
Pypy is super cool, but it doesn’t solve for maintainability and it only improves performance by one order of magnitude, leaving it 1-2 orders slower than Go. Besides, IMO, goroutines are so much nicer than Python’s async.
weberc2··on Python at Scale: Strict Modules
It _is_ like Python in practice (I use both languages all the time). That’s largely why you see it used in many of the same places as Python. It has dynamic features by way of interface{}, which is every bit as “generic” as what Python has to offer. :) But yes, the error handling is different—values vs exceptions.
weberc2··on Python at Scale: Strict Modules
Yeah, but typically only locally. Like using a for loop instead of a list comprehension, or handling errors. So more keystrokes, but in most cases not more complexity. In some cases (generic programming), Python really is more expressive, but those ~5% of cases aren’t worth the tooling/perf/maintainability tradeoffs most of the time.
weberc2··on Python at Scale: Strict Modules
Sounds like Go. ;) This is a cheeky remark, but I use Python and Go, and Go very much feels like an improved Python in most ways. Especially when it comes to static analysis, build tooling, distribution, performance, etc. In particular, I love that there are no venvs, pipenvs, virtualenvs, pyenvs, wheels, eggs, setuptools, easy_installs, etc.
weberc2··on Working with Errors in Go 1.13
It would be great if Go had sum types for error handling, but errors-as-values is great. And the conventions around error handling are pretty easy to work with—the biggest issue is still the anemic error interface (no standard for stack traces, just a string error message, etc).
weberc2··on Working with Errors in Go 1.13
Pretty sure modules addresses the dependency management concerns, no?
weberc2··on Working with Errors in Go 1.13
I agree that error handling has been one of very few rough points for Go. I think the scorn people pour out onto Go is undeserved. It’s easy to say that they didn’t learn from other languages’ experience, but much of that “failing to learn” is a feature. To its great success, Go failed to learn inheritance, “objects” (as in everything is a fat pointer with data, locks, and vtables), exceptions (as control flow), convoluted build systems, gratuitous design patterns (abstract factory bean), etc. Yeah, Go isn’t perfect, but it’s one of the best languages on the market at the moment.
weberc2··on Working with Errors in Go 1.13
Pretty sure the try proposal was killed. Generics hasn’t seen much headway to my knowledge. Not sure what’s up with it.
weberc2··on Working with Errors in Go 1.13
I’m only familiar with people bashing long Java stack traces because (1) they are often hilariously long, with stateless web applications dumping tenfold the frames of a crashing webbrowser, (2) they contain some number of abstract factory beans (contributing to the length problem) and (3) they show up far more frequently in user-facing applications than just about any other language. Stack traces are great, gratuitous abstraction, buggy code, and exposing stack traces to users are not great.
weberc2··on Bazel 1.0
My 2015 MBP is crawling between Safari, Docker, VS Code, and BitDefender (antivirus). Not sure how bulky the JVM is these days. That said, I wish it supported Python.
weberc2··on Haskell in Production
> There is a whole cottage industry of methods for your discipline of choice: Clean, SOLID, TDD, etc.

I'm not a big OOP fan, but I'm pretty sure Clean and SOLID are principles (or sets of principles), while TDD alone is a practice. I'm also not very familiar with Clean or SOLID, but I'm pretty sure they apply to Haskell as well, and I would expect that Haskell enthusiasts would ordinarily boast that Haskell allows (or encourages) them to write code that is more SOLID with less effort than other languages.

My criticism of Haskell is that there tends to be an obsession with writing the most SOLID/DRY/etc code possible at the expense of productivity. It's a code golf / ego culture.

Yes, this is cultural and not "built into the language" (quotes because I doubt there's a clear distinction between a programming language's culture and the language itself, but that's a debate for another time), but you can't unplug from the culture, because you need the ecosystem to solve any nontrivial problem (so you still have to interface with overly clever code).

Further, even if you could unplug from the culture (perhaps by writing everything in-house under your own improved culture), there are still no 'rails' that encourage naive, flat-footed, dumb code, and that's the kind of code you want 99% of the time. As far as I'm aware, there isn't even a clear standard for writing clear, naive code in Haskell.

weberc2··on Why Isn't Functional Programming the Norm? [video]
Go for it (why would I take offense?). Rust is a fine choice. I'd pick Rust if I ever really needed to eek out every bit of performance out of my system and/or needed safety and was willing to trade quite a lot of developer productivity to get there. Mission critical real-time systems, high-end games, resource-constrained devices, etc are good applications for Rust (and of course there are many others). Rust is a great language and I'd like to use it more often; I just can't justify the productivity hit.

If Rust ever approaches Go's ease of use/learning curve/etc without losing its performance or incurring other costs, I'll happily make the switch for my productivity-sensitive applications as well.

weberc2··on Gitlab: don't discuss politics at work
> On the one had you have permanent negative cultural discrimination that is keeping some people down who don't deserve it, and on the other you have an attempt to counter that with visible positive visible/out-in-the-open affirmative action that is boosting the group that's being discriminated against.

The issue isn't positive vs negative discrimination or else the "fight racism with racism" camp wouldn't cry foul when positive discrimination is applied toward whites or men (in those rare cases when it actually is) nor would there be so much overt negative discrimination toward men and whites (the use of "white men" as a slur, the accusation that men are rapists and need to be trained not to rape, the propensity to blame all problems on whites/men, the existence of work groups and meetups that specifically exclude whites/men, etc). It's pretty clearly that there is some notion that certain genders/races are more deserving of both negative and positive discrimination than others.

> This comment sounds like it doesn't know any history. Affirmative actions have been used globally for a very long time in repose to times & places when discrimination occurs. https://en.wikipedia.org/wiki/Affirmative_action

The confusion is understandable; I should have used a different term. I coined "postive racism" to mean discrimination (specifically negative discrimination, per your definition) toward some group that is perceived to deserve it--those who advocate for the discrimination see it as righteous, hence 'positive'--for the sake of a somewhat familiar, concrete example, Antifa advocates for negative discrimination toward whites, men, etc. I should have used different words since "positive racism" sounds too similar to "positive discrimination" which already has an established meaning (and perhaps "positive racism" also already has an established meaning as well). I also used "discrimination" a lot when I technically meant what you're referring to as "negative discrimination"; in general, my post was ignoring affirmative action or "positive discrimination" because I don't understand it to be even remotely central to the broader national debate on race and social justice.

I would specifically not consider "affirmative action" to be a form of racism, although I think it's ill-considered and probably produces more racial tension, division, and strife than it alleviates. The specific reason I wouldn't consider it a form of racism is that I think racism implies animosity/hatred and not just discrimination. And while there probably are some who support affirmative action to antagonize whites, the overwhelming majority of proponents of AA over time have certainly been genuine in their intentions to reduce suffering.

> Ah, so you do you do that? People have been trying that for a very long time. How well is it working?

It's been working very well. Racism has been declining since the civil rights movement by all indicators. Equality of opportunity is at an all-time high in the history of the US, the West, and the whole world. You promote equality by promoting education, prosperity, and western/judeo-christian values. Chief among those values is individual liberty which naturally gives way to equality, and which is the antithesis to collectivist movements such as white supremacy, fascism/nationalism/national-socialism, communism, and so-called "anti-fascism" and "anti-racism" movements that are popular today.

> When has it worked better than affirmative actions?

I speculate (speculation is all we can do for this question) that it has always worked better (and far better) than affirmative action. As previously mentioned AA has probably not had a net positive impact even if it was well-intentioned.

> And note that proponents of actually fixing the problem would call doing nothing deplorable, because it in effect protects existing known discrimination by refusing to fix it.

Depends on how the problem is defined and one's values (hence the "subjectivity" in my original comment).

If the problem is "too much racism", then most people (including me) who are "proponents of fixing the problem" will naturally disagree with the idea that racism can solve the problem--racism is deplorable.

If the problem is "certain races and sexes deserve to be treated poorer than others" then of course racism will appear to be the solution. It's an inherently racist viewpoint, and people who espouse that viewpoint are (by definition) racists.

weberc2··on Gitlab: don't discuss politics at work
It’s not deplorable to try to end discrimination, it’s deplorable to discriminate on the bases of race and sex, even if you believe that you can fight racism and sexism by way of more racism and sexism. And it’s as ridiculous to think that this approach would reduce the amount of racism/sexism as it is to think that the only other option is to do nothing. The obvious alternative is to discourage racism and promote (real) equality. But even if you do nothing, it’s strictly better than promoting some twisted notion of “positive racism” if only because our society had already been trending away from racism from the start of the civil rights movement right up to the point when “positive racism” became fashionable some 8 years ago or so.
← PreviousPage 13 of 34Next →