A Case for a New Language
pl-rants.net
pl-rants.net
But most DSL I encountered in the wild were terrible: no community, weak tooling and documentation, plenty of design mistakes and limitations.
And there are so many in the shadows of the corporate caves, lurking in systems created by the cowboy of the moment who though that would be the perfect solution to the problem du jour. It then had to be supported forever, leaving bugs, frustrations, and unsatisfied programmers and users in it's wake.
9 times out of 10, a library would have been a better choice.
But even famous and popular open source DSL can be a mistake. For me the most obvious example is Ansible. Because they needed idempotence, and maybe expected other languages to join in, they created a YAML based DSL.
In the end, you get a terrible coding experience, learning the thing is depressing and can only be leverage for this very tool, plus debugging very complicated deployment files is a huge hassle.
The ironic part ? In the end this DSL just call Python modules. It's just bytecode with extra steps.
They should have skipped the middle man and gone with a well crafted Python API + tooling + doc making hard for you to make something non idempotent.
The type annotations would document the programmer's assumptions and lead to fail-fast behavior. In the presented use case, it would be immediately clear from the run-time type error that the input data isn't in the expected format (as opposed to a typing bug somewhere else in the program).
https://varnish-cache.org/docs/trunk/users-guide/vcl.html
Varnish VCL, accomplishes the job, but far from elegant, of course.
It would be nice, indeed, to be able to start from some template of a rules language.
Where one can customize the 'language syntax' of declaring entities, and what to do with them. And,then, to implement a domain-specific backend (either as code generated, or as interpreted at run-time).
I could not find a framework like that when looking for it, however.
( At the time, I was looking for something allows me to quickly build frontend, and backend runtime in language that can be embedded into java, C++ and javascript).
I started with metalua (this was years ago), and some success but was too difficult at the time to take it further.
Then I tried xText and xtend (so that I could get automatically an IDE.. but again was too difficult).
> > "... The project I am currently working on involves receiving and processing gRPC messages from some A-brand network devices. The messages contain ordered sets of key-value pairs. The meaning and naming conventions for the keys and values varies greatly and is inconsistent across different message types. The problem was how to describe various processing rules for the messages...."
Essentially what I am asking here is: where do we set the line between "scripts" and "configuration files"? Because scripts are simply configuration files with a tree-like structure as they represent programs.
Scripts "configure the execution paths of your program".
A DSL thus is just a configuration file with a set of rules of applying those tree-like structures to each other.
I would say the line is Turing-completeness.
For me, DSL are half way between configuration files and scripts. If they are not Turing complete they are configuration languages. If they are, then they are programming language (with a specific use case).
And I think that it is usually a bad idea to have configuration files in Turing complete languages (with exceptions, it can certainly be done well).
>> I would say the line is Turing-completeness.
>> For me, DSL are half way between configuration files and scripts
_you_ would say.
>> If they are not Turing complete they are configuration languages. If they are, then they are programming language (with a specific use case).
Or, you could say that DSLs are what they are: "Domain Specific Languages". Which means that can either be a set of static definitions a-la configuration file, or on another side they can be scripts.
All of these definitions fit really well the meaning of "Domain Specific Language".
:)
DSLs are nonetheless a "necessary" evil; Most languages are too narrow to be useful for everything, so anywhere you see a boundary in your language -- something that might be done better in another -- your code will be shorter, it will be faster, and you will make fewer bugs.
But what if we can create a language with no boundaries? That language would not need a DSL.
It surprises me that the author used Elixir for implementation but didn't just write a query DSL utilizing the Elixir macro facilities [0]. A macro based DSL likely would be much simpler than writing a new grammar and builds on existing language tooling. Ecto for example does a great job (IMHO) of making a sane DSL for SQL [1]. Given data processing primitives provided by Elixir libraries like Streams and GenStage [2] it seems it'd be pretty straightforward to implement.
Take for example the "segmented query path expression". It is handled fine with a minor modification as an Elixir quoted statement:
`quote do: @query_path//foo/goo[instance='a2']/blah/x[a='S 512']/y`
That yields a well defined AST tree base on the Elixir syntax: ``` {:@, [context: Elixir, import: Kernel], [ {:path, [context: Elixir], [ {:/, [context: Elixir, import: Kernel], [ ... ```
The closures work as well by switching the dollar signs to ampersands: `quote do: select(@path/foo/&x/&goo/z/y/w, { value(&1) < 3 })`
Makes me want to write an Ecto adapter for dealing with Elixir Streams. =)
0: https://hackernoon.com/understanding-elixir-macros-3464e1414... 1: https://github.com/elixir-ecto/ecto 2: https://hexdocs.pm/gen_stage/GenStage.html
Python embedded DSL for image processing / general matrix munging
Define arbitrary execution graphs that compile to GPU or CPU.
Take a SQL query in Python's SQLAlchemy:
s = select([db.func.sum(users.c.id)]).\
.select_from(
Users.outerjoin(Posts, Users.id == Posts.user_id))
where(and_(
users.c.id == a1.c.user_id,
users.c.id == a2.c.user_id,
a1.c.email_address == 'jack@msn.com',
a2.c.email_address == 'jack@yahoo.com'
)).\
group_by(users.c.age)
from account in App.Account,
left_join: t0 in App.Transaction,
on: t0.account_id == account.id
and t0.user_id == ^current_user.id
and t0.deleted == false
and t0.type == "inflow",
where: account.id == ^id and t1.user_id == t0.user_id,
group_by: account.id,
select: {account, {sum(t0.amount), sum(t1.amount)}})
P.S. The examples might not be exactly correct as I cobbled a couple of different ones together.
It's not that easy to define an arbitrary language that allows you to write code that is close to something else completely (like Ecto vs SQL).
Some statically typed languages allow you to build up composable query snippets along the lines of (from a Scala DSL):
val accountQ = for {
(a, t) <- Account leftJoin Transaction (_.id is _.accountId)
if a.id is ... && t.userId ...
groupBy a.id
} yield (a, t.id, t.amount.sum, ...)
val invoiceQ = for {
(a, transId, total) <- accountQ
i <- Invoice join (_.transId is transId)
} yield (a, i, total)
I love query DSLs, the more we move away from string-y SQL the better :)
Right. Maintenance, documentation, training, tooling, packaging, debugging, etc., of a DSL far outstrips its development cost.
When you use regular expressions to solve a problem, you have two problems. And regular expressions have (basically) well understood parsers, logic generators, and so on. When you use a DSL to solve a problem, you have N problems. Maybe that's much, much better than the alternative (query languages are particularly interesting), but people definitely shouldn't stop at "this was easy to make... what's the big deal?"
Itemis, the company behind xtext, is now using MPS.
It supports composable languages and provides IDEs for the language you created with code completion etc.
It is complex, though, so it takes a while to learn.
IIRC I got a grammar working then went off to play with some new shiny thing before trying to get it to generate some code since I didn't really have a use case beyond just playing around though I believe my plan was to get it to spit out AST classes since my ASDL parser/generator is kind of a goofy mix of boost::spirit and moustache.
Also I thought it was going to talk about something like the Blub Paradox: http://wiki.c2.com/?BlubParadox
/foo/*/blah/*/y
For example:
> select(/foo/$x/$goo/z/y/w, { value($1) < 3 })
Could become in JS:
select("/foo/$x/$goo/z/y/w", function(a){ return a<3; })E.g. an SQL engine can potentially use an index to answer x<3. Your library calling your function for every item probably can't do that, or can't guarantee that it can do that.
This makes me really question the author's credibility. Clearly the author is pretty confused about static vs. dynamic type systems, and the value and history of type systems in general.
For example this should be disallowed by the type system.
db_handle h = new DBConnection()
* What if the DNS name doesn't resolve?
* What if the port isn't open?
* What if the password is wrong?
If you're going to assign to the handle you have to prove to the compiler that the constructor will return a valid, working handle.
Result q = h.sendQuery(...)
* What if the connection dropped?
* What if your credentials are no longer valid?
* What if the query is malformed?
x = a / b
Elm damn close to this ideal.
I'm trying to imagine how one could prove to the compiler that an event in the far distant future is valid (without the obvious solution of throwing an exception in the constructor but that is at runtime so...)
Those benefits are orthogonal to dynamic vs. static type checking. Even with a dynamically typed language, you get massive performance, security, and correctness guarantees vs. an untyped language. There's huge merit in having a type system that provides all these benefits but still doesn't guard against runtime errors.
Which brings me to my point: this person is clearly confused about the benefits of type systems, which don't require static typing to add incredible value.
I'm surprised the link doesn't work for you. Any idea why? (To be clear, I'm in no way responsible for the link or its content.)