Hacker News Reader

Comment by btown | Hacker News Reader

My favorite Python "crime" is that a class that defines __rrshift__, instantiated and used as a right-hand-side, lets you have a pipe operator, regardless of the left-hand-side (as long as it doesn't define __rshift__).

It's reasonably type-safe, and there's no need to "close" your chain - every outputted value as you write the chain can have a primitive type.

    x = Model.objects.all() >> by_pk >> call(dict.values) >> to_list >> get('name') >> _map(str.title) >> log_debug >> to_json

It shines in notebooks and live coding, where you might want to type stream-of-thought in the same order of operations that you want to take place. Need to log where something might be going wrong? Tee it like you're on a command line!

Idiomatic? Absolutely not. Something to push to production? Not unless you like being stabbed with pitchforks. Actually useful for prototyping? 1000%.

View on HN

I spy a functional programmer lurking in this abuse of Python ;)

Looks a lot like function composition with the arguments flipped, which in Haskell is `>>>`. Neat!

But since you’re writing imperative code and binding the result to a variable, you could also compare to `>>=`.

(https://downloads.haskell.org/~ghc/7.6.2/docs/html/libraries...)

Having spent a lot of time lurking on the frustratingly-slow-moving bikeshedding thread for the Javascript pipe operator [0], there's a great irony that a lot of people want a pipe operator because they don't want to deal with function composition in any way, other than just applying a series of operations to their data!

I think there's a big gap pedagogically here. Once a person understands functional programming, these kinds of composition shorthands make for very straightforward and intuitive code.

But, if you're just understanding basic Haskell/Clojure syntax, or stuck in the rabbit hole of "monad is a monoid" style introductions, a beginner could easily start to think: "This functional stuff is really making me need to think in reverse, to need to know my full pipeline before I type a single character, and even perhaps to need to write Lisp-like (g (f x)) style constructs that are quite the opposite of the way my data is flowing."

I'm quite partial to tutorials like Railway Oriented Programming [1] which start from a practical imperative problem, embrace the idea that data and code should feel like they flow in the same direction, and gradually guide the reader to understanding the power of the functional tools they can bring to bear!

If anything, I hope this hack sparks good conversations :)

[0] https://github.com/tc39/proposal-pipeline-operator/issues/91 - 6 years and 793 comments!

[1] https://fsharpforfunandprofit.com/rop/

Witness the chaos you have unloosed upon the world; not even little Lua is safe:

  > =dd-{"Todo","es","bueno"} /call("gsub", "[aeiou]", "i") /call("rep", 2) /call "lower"                  
  {"tiditidi", "isis", "biinibiini"}
  > ps={{x=5, y=7}, {x=5, y=4}, {x=3, y=4}}
  > =dd-ps/get"x"                                       
  {5, 5, 3}
  > =dd-ps/where {y=4}
  {[2] = {y=4, x=5}, [3] = {y=4, x=3}}                                       
  > =dd-ps/group "x"                                         
  {[5] = {{x=5, y=7}, {x=5, y=4}}, [3] = {{x=3, y=4}}}       
  > =dd-ps/group "x"/get(1)                                  
  {[5] = {x=5, y=7}, [3] = {x=3, y=4}}                       
  > =dd-ps/group "x"/map(get"y")                             
  {[5] = {7, 4}, [3] = {4}                                   
  > =dd-ps/group "x"/map(get"y")/map(max)                    
  {[5] = 7, [3] = 4}                                         
  > =dd-ps/group "x"/map(get"y")/map(max)/min                
  4
  > =dd-ps/group "x"/map(get"y")/map(max)/sum                
  11
  > =dd-ps/where(function(p) return 9 < p.x+p.y end)
  {[2] = {y=6, x=8}}                                         
  > function bytes(s) return {s:byte(1,#s)} end
  > =dd-{"hey", "you"}/map(bytes)
  {{104, 101, 121}, {121, 111, 117}}
  > =dd-{"hey", "you"}/map(bytes)/flat/freq
  {[111] = 1, [101] = 1, [104] = 1, [121] = 2, [117] = 1}

I suppose you could use the same trick with __ror__ and | (as long as the left-hand side doesn’t define __or__).

  x = Model.objects.all() | by_pk | call(dict.values) | to_list | get('name') | _map(str.title) | log_debug | to_json

Sadly many things define the __or__ operator, including dicts and sets which are common to find in pipelines. (https://peps.python.org/pep-0584/ was a happy day for everyone but me!)

In practice, rshift gives a lot more flexibility! And you’d rarely chain after a numeric value.

It’s still useful in related situations. The following crime often finds its way into my $PYTHONSTARTUP:

  class more:
      def __ror__(self, other):
          import pydoc
          pydoc.pager(str(other))
  more = more()

and here the low precedence of | is useful.

Or overloading division like https://scapy.net/

Oh god, it's c++ all over again!

Is that supposed to be a bad thing?

IMNSHO: Yes.

It's a sign of the design quality of a programming language when 2 arbitrary features A and B of that language can be combined and the combination will not explode in your face. In python and C++ (and plenty of other languages) you constantly have the risk that 2 features don't combine. Both python and C++ are full of examples where you will learn the hard way: "ah yes, this doesn't work." Or "wow, this is really unexpected".

Well, there is also a question of attitude. Most of the Python programmers don't overload << or >> even though they technically can, while in C++ that's literally the way the standard library does I/O ― and I suspect it leaves an impression on people studying it as one of their first languages that no, it's fine to overload operators however quirkily you want. Overload "custom_string * 1251" to mean "convert string from Windows-1251 to UTF-8"? Sure, why not.

I've seen >> being overloaded in several libraries/frameworks. From the top of my head:

   - Airflow: https://airflow.apache.org/docs/apache-airflow/stable/index.html#dags

   - Diagrams: https://diagrams.mingrammer.com/docs/getting-started/examples

Yes. iostreams overriding << and >> was pretty much universally seen as a bad idea and they eventually abandoned it in C++20/23 with std::format and std::print.

It's usually a good idea for operators to have a specific meaning and not randomly change that meaning depending on the context. If you want to add new operators with new meanings, that's fine. Haskell does that. The downside is people find it really tempting and you end up with a gazillian difficult-to-google custom operators that you have to learn.

Apache beam does it: https://beam.apache.org/documentation/sdks/python-streaming/

I wanted to wash my eyes the first time I saw it.

Wow, this is awesome! Your example is especially fantastic.

Reply on news.ycombinator.com