Universal Jinja: a crazy idea for a Python-ready front end
whatisjasongoldstein.com
whatisjasongoldstein.com
(This is also why "templating languages" in PHP were a special kind of foolish; PHP is a templating language.)
Any PHP project of any reasonable complexity will either use an existing "templating language" (by which you probably mean a framework) or be forced to create some sloppy, ad-hoc version of one, because while yes, PHP is a templating language/framework, it's also a terrible one.
Emitting HTML is no different from any other functional programming task. State in, HTML out. (You had to be a little clever because you had to ob_start/ob_get_buffers/ob_end, but whatever. Still better than the alternatives.)
What hard things, specifically? I've used Twig a lot and never found it to get in the way of anything else I was trying to do.
>You had to be a little clever because you had to ob_start/ob_get_buffers/ob_end, but whatever.
By the time you have some general abstraction to push "state" into and get "HTML" out of, and deal with output buffering, header management, etc, you've started building an ad-hoc framework.
> By the time you have some general abstraction
Sure. It's about a dozen lines long. And the language is the actual language I'm working with, not something that needs its own parser to give me less functionality in the spirit of mollyguarding on behalf of people who make mistakes of classes I categorically refuse to allow in my systems. (Please don't mistake this as me saying "I don't write bugs." I write many, many bugs. But I structure programs such that some bugs are very, very hard not to make obvious and to avoid.)
For similar reasons, this is why almost all of my web stuff these days uses a Node backend and a React/JSX or React Native frontend. Consistency across the whole thing, buttressed by tools that encourage good practice without chopping you off at the knees.
Ok. But to be fair, Twig's parser is extensible, whereas PHP's isn't. And it lets you use array shorthand in PHP versions that don't support it (which is not as much of a problem as it used to be, but to me, more than worth never having to type "Array()" again.)
Have you tried the XHP functionality in Hack? It treats XML as objects and understands it in context (which is something PHP should do natively, but can't.) It seems to solve many of the footgun problems that PHP template languages do, within native code.
The primary restriction is that your output can be modelled as a tree in some way, and what you find is that nearly everything falls into this category. I don't know if anyone's done it yet, but a plain-text renderer is certainly possible, I imagine it would be very similar to a terminal renderer.
JSX transpiles down to a tree of function calls, it's literally nothing more than that, the only time it might be considered XML is in the final rendered content. Syntactically, JSX isn't even valid XML.
It is impressive though how often you have types when I wouldn't expect it.
JSX is a data structure with a syntax similar to XML. The angle brackets are quite similar to brackets around any other data structural literal in any other syntax.
The tag names are references to either in-scope variables/constants (when capitalized), or references to a static set of HTML element names specified by React (when lowercase). This is different from HTML or even XML because they are evaluated at runtime (plain JSX->JS compilation) or validated at compile time (e.g. TypeScript).
The attributes names are keys in the data structure's props, and the values are either static (if specified as string literals) or dynamic (if specified between curly braces). These too can be checked at runtime (with PropTypes) or at compile time (e.g. TypeScript).
By specifying dynamic props (or conditional children), the component will re-render as the values change.
I'm not aware of any other HTML-like or XML-like template language which functions the same way.
Edit: I forgot to mention, dynamic props are also exactly JavaScript/JSX expressions. Basically JSX is a macro that turns the XML-like stuff into React API calls.
The main dlang web framework, vibe.d, uses a 1:1 copy called "diet" templates (You can insert D code to be run on the server, with the arguments given to the HTTP request/response). This works very well IMO, mainly because D is a very good programming language, so you can mix very fast / expressive D code with your regular JS for client side work.
What I like most about it is that i can just use Nim code in my templates. I can't say the same for Django templates or (I guess) Jinja.
One peeve: managing multiple partials for a single page with mixins seems a little more complicated than necessary.
.sep
<div class="sep"></div>
It's another tech that renders just as well on the client as the server.
I'm not sure that's the template language's fault and more than Python is responsible for Python code that goes metaclass happy for no reason.
- the broken mix of pythonisms
- the arbitrary-feeling restrictions
- the oddly missing pieces
https://github.com/pallets/jinja/issues/659
> It was never documented that you can override variables from higher scopes
But you could, and people did... because that's what people do: they try stuff that seems reasonable and if it works they use it. And then someone "fixed" it in a minor release and things started breaking. BTW the error message is
KeyError: 'undefined variable: 0'
I used Jinja2 happily for years on a web app, but that thing pretty much just styled a page for flash to sit in and drew some forms and menus. I use it now via Ansible and have come to despise it through and through.
I'll try to find it again. I obviously ditched the
From what I've seen of other template languages, it resembles Mako the most, though some of the syntax superficially resembles Jinja2.
So if you want to replace a character in a string, that's fine, because `replace` is an instance method:
Changed plus to dot: {{ string.replace('+', '.') }}.
My string is {{ len(string) }} long.
I really started to hate it in Ansible though, where I'm often just trying to get a one-off little operation done that I would rather have obvious right there in the template source rather than having to bounce over to a one-off filter function (or invoke some other outside code), and I can do it in Jinja, but it just looks and feels like hell... which kinda defeats the purpose.
I feel like Jinja was an especially bad fit for Ansible, but as a templating language I feel like it's main strength is it's better than D'jango templates... but that doesn't matter much outside the D'Jango world.
As a logic-ful template language I think it's deficient because it omits the functional parts of Python, which - as discussed above regarding `len()` - are very core to Python, and Jinja adds them back in a different way that requires extra mental overhead to keep track of. I really like the pipes (filters) but it would have been nice to just keep the Python globals as well for consistency.
And if you really want to go logic-less with your templates, I think Mustache(-variants) would be the way to go since they're much more truly "logic-less" and they're supported across a plethora of different runtimes besides Python, giving your much more flexibility. I've never really had the patience / discipline / right use case for logic-less though, so I'm far from an expert in this area.
Jinja is just a really weird mix... it's not logic-less, but it's not Python... it's a funky half-logic mishmash. It obviously makes sense and feels right to the people that make it, and if it makes sense and feels right for you that's awesome, but I hope you can understand how could feel awkward to others.
The brute-forceable admin happened around the time web2py and Django devs started having a few arguments online, like this one [1], which didn't exactly paint web2py in a good light.
[0] https://security-tracker.debian.org/tracker/source-package/w...
[1] https://www.quora.com/Is-web2py-a-good-Python-web-framework
We already have one JS engine in Python (https://github.com/kovidgoyal/dukpy). We even have pluggable renderers (https://pypi.python.org/pypi/PyExecJS). Later we will have webassembly renderers in Python, so you will be able to take frontend frameworks, compile them and execute them in Python.
Finally, we have some fantastic framework that are powerful, yet not that big, such as Vue.js, that already have a pluggable architecture for rendering without a DOM on the server side (https://ssr.vuejs.org/en/basic.html)
The Python community has done much more complicated projects already, and half the work is there, so my guess the reason it hasn't happened yet is we REALLY hate js and most of us just touch it to get the job done and go to the next task.
It's a bit like the async situation. We have the tools, but there is no Django of the asyncio because we love our tools as their are so much. I'm so guilty of this.
The problem is: the rest of the Web is moving ahead, and while it's still relevant to do DRF + Vue.JS right now, one day stuff like meteorjs will be so crazy yet stable it will be hard to compete.
I was hopping webassembly would let us eventually run Python on the client side. But I doubt it will happen. However, I'm thinking more and more than it will allow JS tooling to be plugged in many other languages on the server side.
[0] aka isomorphic, but universal seems to be the new (IMO better, less overloaded) term for that
After learn and forget a few js frameworks, see develop the "compile to javascript" fashion, javascript jumping to the server, the different flavours of MVC, "isomorphic programming", subsets of javascript that compile to whatever and etc..; my gut feeling is that it will never end.
If that is a good or bad thing is left as an exercise for the reader.
In in end, nothing is well integrated.
You would think the whole bloody "one language to rule them all" concept would make the JS stack the most integrated one of all.
Well to my surprise, far from it. It's a mess of hundred of moving unstable badly documented always changing components that your job is to make work together.
There is no django of the JS word. None.
And because of that, most JS projects I worked on were disposable projects, prototype in production.
Now there is no meteor in the Python word neither. But professionally, I need something solid, not an ongoing experiment.
You can make fantastic stuff with JS, but only because the platform is fantastic. The web is genius. The JS ecosystem is terrible.
But wish, I really wish the JS community would pick on the seriousness of other communities and the python one would take more risks and innovations like the JS one.
You can see the project here: https://github.com/Miserlou/django-knockout-modeler
It allows you to take a single Django QuerySet and turn it into a SPA with in-page listing, sorting, filtering with a single Django template tag:
{{ my_query_set|knockout }}
I used in production in a few projects and was very happy. If I were to do this again, I'd re-do it for Vue.js and make the data-syncing bi-directional. But I hope we see more stuff like this.
It's Python to JS in a way that makes sense to me at least. JavaScript and Python have a lot of similarities in the languages imo especially around classes and functions
To run Python in the browser correctly, you need the entire 3Mo Python VM. Transpiling is not enough because Python semantics are complicated as it is very, very rich.
Comparatively, js is incredibly basic.
Write a react clone in Python (complete with "pyx" files), and then use one of the python to js transpilers to compile to js on the frontend. On the backend, you'd use "react" just as another simple template language, whereas on the frontend, you'd also have the reconcilation algorithm.
- operator overriding
- look up interception API
- metaclasses
- embdedded design patterns: iterator, singleton, context manager, etc.
- multiple inheritance
- standardised imports, with hooks
- controlled gc
- dict comprehension
- generator expressions
- type hints
- string formatting language and operations
- much more comfortable lists
- rich built ins (all, any, enumerate, zip...)
- no weird implicit type casting and comparison rules
- stack traces and very explicit error messages
- support of a lot of charsets out of the box
- multiple notations for text and numbers
- types for bytes
- support for imaginary numbers
And so much more.
Because Python is such an easy language to use, it's easy to think it's just a simple scripting language.
It's not.
It is very, very powerful.
But it has such a smooth learning curve and you can be productive in it so quickly a lot of people never need to see it.
You can compare Ruby to Python. Or Go.
But JS ? It's not even on the same planet. The only reason this abomination has this success is because it has an accidental monopoly on the most awesome and popular platform of the human race: the web.
One of the best things about JS is that is has none of these features. Smaller languages are better, and blindly adding features is not the way to improve a language.
Unfortunately, nothing in JS can ever really go away, so it's starting to leave the "small language" sweet spot.
JS is better than Ruby by being smaller and infinitely better specified. Go is a nicer language due to being designed from scratch recently by people with excellent taste.
In your opinion. There are entire platforms, with literally millions of programmers, which say otherwise - like .NET or C++ for example.
My opinion? There's no point in calling a language better without qualifying a context or use-case. There are so many languages, with so many different (takes on) features, that it's frequently impossible to even meaningfully compare them. Much less to say which is better.
In other words, I can agree with your statement, that "smaller languages are better", for example in fitting your tastes or in ease of implementation. I won't agree that smaller languages are "better" universally, however, because it's trivial to show the contrary. It's enough to point to all the "transpilers", template- or macro-systems that are being written all the time, nearly always for small languages. If smaller is universally better, then how come so many people want more features and are willing to invest their time to write a transpiler?
Personally, I like small, elegant languages - but I also like the big, pragmatic languages, the weird and experimental languages, the special-purpose and exotic languages and so on. Each has its uses and may be a good tool in the hands of a professional.
When I say transpiler, I don't mean just mapping one syntax to another. I mean either really compiling to JS, or having a python bytecode interpreter in JS. It is possible to compile very different languages to JS, for example Nim.
I think it should be possible to support 100% of the core language - even metaclasses, why should that feature be more difficult? Of course you would not map Python classes to JS classes! You would map Python classes to something else, like opaque JS objects.
It is not realistic to support 100% of the library, but that is not neccessary - on the frontend, you wouldn't need to use most libraries, and you don't need C FFI. Most of the time, you'll be only pushing JSON or python objects into templates.
So far isomorphic Node + React has been the best (but not perfect) way I've seen to do this. You abstract out the API layer, and now you can define components and their data sources that have the exact same code on the client and server, and it just works. Apollo/GraphQL is probably a giant leap forward in this as well.
Having an option to do hand-glued "isomorphic" data in a few places is probably nicer than Javascript spaghetti on the front end, but it doesn't get you all the way there. Rails has the same problem by the way, the "Rails way" is coffeescript glue to get any kind of modern front end application functionality. Even with React on Rails you have to have to reinvent data fetching, and lose the benefit of a well structured isomorphic codebase.
Until browsers support a different language natively other than Javascript (which will never happen), Node + React is currently the most straightforward path to get there.
I think WebAssembly is promising for this. You could write your app in any language you want, as long as it compiles to WebAssembly.
It works - but there are gotchas:
1. You can't depend on useful pythonisms/javascriptisms such as
{{thing or ""}} vs {{thing || ""}
3. Testing becomes annoying because a template can work in one, and not the other - you have to test both.
4. You don't get the newer html diff style virtual dom rendering which can cause issues with stuff like select boxes. You end up with special case code to handle that, which is more of a hassle.
The conclusions from it all were:
a) You do save time over doing it once in python, once in js
b) You don't save time over doing it all in js - especially if you use a newer functional style js template framework (react/vue/polymer)
c) You can still get the best of both worlds by hooking up your templates in js to be rendered by your backend - eg like https://github.com/reactjs/react-rails does.
In the end for my next project I decided to do an api for the backend in python and the website/frontend in react with SSR, and am really thrilled with the results - very maintainable, testable and simple overall.
The biggest problem is testability. Testing that a template renders correctly means parsing the markup into a DOM and asserting that certain conditions are met. This is exacerbated by the differences between Jinja and Nunjucks, which can be non-trivial and unexpected (e.g., Jinja will render synchronously, while Nunjucks can render asynchronously--weird async behavior may appear). Since your application doesn't have the same entry point to the code (they're in two different languages!), you really need to write two different sets of tests for the same template to get proper coverage. Templates being templates, you also don't get the granular testability of a React component out-of-the-box, which can be pulled out and tested individually.
Templates alone also don't include anything in the way of handling events, component lifecycle, and other things that make your code actually do stuff. You'll need to bind event listeners manually, and those will of course need their own separate tests.
Making Jinja useful also usually involves adding your own plugins and filters. This happens in Python, which means you'll need to write all of those a second time in JavaScript, and test them. Doing this right means comparing the output of the two versions, which isn't necessarily straightforward. Given the complexity of this, I wouldn't bet money that other engineers would go through the hassle of making sure this is done correctly.
I'd encourage the author to write more about the pitfalls of this approach. It's easy to write {{ foo.strip() }} in Jinja and forget that Nunjucks requires {{ foo.trim() }}, or empty arrays being falsey in Jinja and truthy in Nunjucks.
I setup an asset macro so that I could declare static asset dependencies in my templates at the component level and have them injected into the footer of the page automatically.
Adding a few lines of jinja middleware to my test server made it possible to unit test my templates directly. It also made my HTML fixtures much more enjoyable to write, because I had jinja at my disposal.
I'm glad I didn't limit myself to the overlap of the 2 rendering engines. I am now in a place where I can easily migrate to Vue and start killing off my jQuery dependencies.
These techniques are easier with the Pyramid framework.
Server side rendering only makes the first view faster. After that, everything is rendered on the client and faster that way. And if you use caching with progressive web apps or just file caching in general, the first view will be fast after. So you're only optimizing for the first of the first. You can then provide the initial data in the page request so that saves a round trip on the first request while still rendering on the frontend.
I say all this to just point out that there are many ways to speed up your frontend without doing SSR and they are much easier to implement with Python. So I'd rather exhaust all these techniques before implementing SSR. If I get to that point though, I suspect Python might be the slowest part of your stack. And lastly if you are doing SSR for SEO purposes then just use a prebuilt prerender service.
With all that said if there was an easier way to SSR with Python/Django, I'd definitely use it. But Jinja on the frontend doesn't sound like that answer.
The first view is the most important view, and making it faster is often the most beneficial thing you could do for engagement.
Sure if you're using Node.js pull out SSR first since its easier. But for Python right now, it's not the easiest thing.
"Only the Sith deal in absolutes", Obi-Wan Kenobi
ObiWan: deals_in_absolutes ==> Sith
rhapsOdy: mathematician ==> deals_in_absolutes
----------------------------------------------
therefore ...
https://github.com/mozilla/nunjucks/blob/master/CONTRIBUTING...
https://github.com/djc/jasinja
This is in some (conceptual/experiential) ways a precursor to my recent attempt to create a Rusty Jinja-like, which tries hard to take the best ideas from Jinja and mash it up with the best ideas from Rust:
The only thing that doesn't seem clear is why you'd need to combine server- and client-side rendering in a single instance. Jinja is an incredibly useful and well-made tool, and my experience of it with Flask always leaves me with the best impressions of it being exactly what quick webdev should be, but why would you combine it with JS? Surely the use cases are so different that a separate JS library would be just as applicable, and perhaps more fit for purpose.
Making sure the result conforms to the grammar of the output language without any unparser would always involve a parser.
> As soon as I'm looking at more than one programming or markup language in the same file, I'm looking at spaghetti code.
Iain Dooley, December 2011
http://www.workingsoftware.com.au/page/Your_templating_engin...
- just pre-generating everything outside of the template can be very efficients. Especially if you language can't make everything lazy or if you have several representations for the same dataset.
- designers want a bit more freedom that just printing x. Having to go back to the dev team everytime you need a little tweak is terrible
- all templates are not HTML
- it's way easier and faster to prototype
- rendering caching != data caching
- everything is not about display. Linking and injecting resources are a big deal, and putting that outside of the template is a huge pain.
- conditional template inheritance ? includes in loop ?
- stuff like wordpress have entire business based on the fact you can switch templates on the fly without touching the blog code base or without the wp team to know what you are going to need inside the template in advance.
Separation of concerns is a thing that can help designers, no?
I realize you may not be cosigning on everything in the article you're quoting, but this is the author's first suggestion to an alternative to template languages.
I work on an SPA that was built like this. The index.html is over 10k lines long. It contains almost every single piece of the UI. Templating libraries and languages aren't perfect but they offer a better separation of concerns than just "dump it all in one file and write some imperative dom fiddling code to add different states"
I think SPAs are either fundamentally dishonest engineering, in the same way that a microwave wrapped in artificial wood veneer is (and a stainless steel microwave is not) or should result in such a template. If you really think that this is too much, IMO you should not make an SPA in the first place.
Could you rephrase or expound on this metaphor? I have no idea what you mean by "fundamentally dishonest engineering" or what that has to do with SPAs and templates.
So many people jump to heavy JS libs/frameworks to do stuff that can be solved in much more boring/simple ways most of the time. You don't need React/Angular/Vue to submit a form over ajax.
Not all sites need a lot of JS. But your tweeter clone is going to be very boring if you have to hit F5 every 30 seconds and polling doesn't really scale.
My point is that many sites don't really need to be as js heavy as they end up, and people end up solving the wrong problem (how to render all this js server side rather than how to reduce all this js in the first place)
It feels so irrelevant in the world of backend-as-a-service and GraphQL.
...Please forgive my cynicism. I am looking for a reason to like this. Maybe I should just move on.
What got me thinking about these ideas were cases where having a distributed frontend felt overengineered. If you have problems where GraphQL makes sense, this would be a step in the wrong direction.
Would anyone care to comment on the productivity benefits of one or the other?
The main productivity benefit comes from the fact that the query language is a very good fit for the component model. Each component can declare its data dependencies in the form of query fragments, and these get composed into a single query for the entire UI. It means every time you use a component, you can be sure that you'll have the data you need.
I should add that none of this is a unique capability of GraphQL, I know people who've built similar component-data binding with JSON-API, but it's the full package that's compelling. The query language is lovely to work with. You can build GraphQL schemas around the needs of clients, allowing you to hide any weird data modelling or service boundaries you may have in your platform. As an API designer, as well as a front-end developer, I love it.
React does literally none of those things. aiohttp is just a barebones http server with no built-in functionality.
My perceived benefits of Pug/Jade:
* Significantly fewer LOC
* Improved Readability
* A lot of existing tooling thanks to its popularity in the NodeJS community
Looks to be "haml with templating" in a single language. Great idea.