Data-Driven Development Is a Lie | Hacker News Reader

Data-Driven Development Is a Lie

grishaev.me

95 pointsmgd73 comments

> But they would never give a speech about how they messed up by describing everything with maps.

I think this needs to be emphasized and can easily be transferred to a whole bunch of other topics unrelated to DDD/Clojure.

Nobody ever talks about the negative sides of some new apparently cool thing, because it is assumed that doing so wouldn't attract an audience. While in reality, often cool new thing does only work as advertised in a minority of cases and the negative sides are more important to look at.

We should more often talk about the negative sides of "cool" new approaches.

I had this in a recent python project I joined. Dicts everywhere.

APIs kept breaking, nobody knew what data was passed around, strings instead of ints etc.

Thanks, I hate it.

Dicts are essentially an untyped API, even in a strongly-typed language - at least until you have "dict that contains at least keys X, Y, and Z" as a fundamental type in your type system.

Whether that's a benefit or a liability is in the eye of the beholder...

In a sane statically typed language, using collections without generics is a second degree sin.

I would love to go to a conference where every talk was about how something went horribly wrong. Like two or three days of large scale Daily WTFs. I feel like that would be hugely entertaining and more informative than most regular conferences.

Something like Fuckup Nights (https://en.fuckupnights.com/) but proper conference and with focus on software development, would be really interesting.

Holy shit, FUN looks awesome, and there’s one within driving distance to me coming up! Thanks for sharing that!

EDIT: my comment is wrong because it confuses Data-Driven Design and Data-Oriented Design. Thanks @Narishma for the correction. DOD is pretty great, but I wouldn't push for DDD outside game engines.

---

I see DDD as a counter movement to OOP: expose your fields, don't bake behaviors into your data structures, and accept primitive types instead of instances of the internal class MyData.

To me it doesn't imply "convert your code into data", which seems to be OPs only gripe with it.

What you're talking about is data-oriented design. Data-driven programming is something different.

https://en.wikipedia.org/wiki/Data-oriented_design

https://en.wikipedia.org/wiki/Data-driven_programming

I don't see how this comment section can recover from the confusion.

Most of the comments are either confusing data-driven development (which is possibly bad) for data-oriented design (which is good) and wondering what the author's issue is with the latter; asking or helping to define what data-driven development is; griping that "DDD" is a reserved term; or some combination of those!

Oh no, you're absolutely right. I can't delete my rubbish post, but I've added a disclaimer, thank you very much. Also, it seems that DDD can mean Domain-Driven Design, so that's a confusing namespace to navigate.

https://en.wikipedia.org/wiki/Domain-driven_design

I've done DDD a few times, and every single time was a mistake. The solution fit me like a glove, but nobody else really understood it. Some of my best coworkers could modify it, but they still frequently did things "wrong" and overcomplicated things.

I've basically sworn to never do it again.

I didn't know that I was doing "DDD", though. I was just trying to make it easier to make changes that I knew were common.

So what you've been doing actually? Asking bc DDD has a blurry meaning. The article itself does not expand it well

Like the article, I tried to make it so that someone could edit minimal data to expose fields and validation in a form, instead of having to actually having to code up all the glue code. It involved writing a lot of code that magically determined the right parameters, with the ability to override it if necessary, and too often didn't really have what was necessary to do the job perfectly. But it was good enough and a lot faster than writing the glue code each time.

I swore it off years ago, right before my boss did exactly the same thing for a new system. sigh

Now, Co-pilot can write that glue code in a heartbeat, so it makes even less sense to try to write something like that in the future.

I don't think I could disagree with the article more strongly.

I've written conditional logic trees like this and they're pretty much always better than some hardcoded OOP/imperative code. I think what the author might have missed is that the conditions are labels for some functional result, usually from SQL as the app checks various fields on the user's database tables. So DDD doesn't actually work with data, but filters on data. The labels are to make the intermediate steps readable by humans, like how Prolog rules/predicates work.

In other words, I don't think that the article has said anything, because functional code can always be transformed to look like functions acting on conditions or conditions resulting from functions. At the end of the day, functional code is still a query composed of queries. The important part is to avoid mutable state, which would turn it back into imperative code which is hard to step through and reason about.

I googled the definition of data driven development, and none of the results I looked at explain anything. I don't know if it is because of the concept itself or because internet is becoming more and more shit

In this rare case it's not the internet... there simply is nothing worth coining "Data Driven Design". It's a mantra from within the Clojure community that has given rise to a questionable book named "Data Driven Design".

Agreed. I actually like Clojure and worked on a small project in it several years ago. But, the whole "Data Drive Design" idea literally just boils down to using general-purpose language primitives and data structures instead of defining specific classes/structs/whatever. That's it.

The idea is that if you just use dictionaries for objects instead of traditional, statically typed, classes, you get all of the dictionary APIs for free, so you can easily iterate over the keys, add more keys to the object as you're processing it (as opposed to defining a Foo class and a FooBuilder class, or similar), etc.

The pros and cons are obvious if you've worked in dynamically typed and statically typed languages.

That's not quite what this article is arguing against. This article is more arguing against a functional programming pattern, IMO.

What are the pros? Even in Python, where such an approach is not unheard of, it sounds horrendous. Dataclasses and pydantic became popular for a reason.

I'm not really familiar with data driven development, but in languages with row polymorphism it can be pretty useful to be able to do things like add/remove properties to objects as you go. Personally I wouldn't want to throw out type safety to try to do that in a language without the type system to handle it safely, but I guess I can see people choosing to do that?

A classic rant by Rich Hickey: https://youtu.be/aSEQfqNYNAc?si=XBVI4cAtUlSPSU-v

I say this even though I don’t usually code this way: I think the big benefit for business code is sort of a constrained flexibility.

Objects tend to get really big (the alternative is you have a ton of classes). Think about all the data that makes up one Uber ride. So if I want to do anything with a Ride, I need everything. An even the many classes thing doesn’t really solve that because maybe what I want is cross-cutting. So either way, I just end up having to supply a bunch of stuff I don’t need.

The maps-first approach is more incremental. Every function takes what it needs and gives what it can.

~~search Data Oriented Programming instead.~~

No, that's completely different.

i see. i'm not entirely convinced the author isn't confused between them either...

It seems to mean declarative programming, to the extent possible in a non-declarative programming language.

whatever-driven development is always just about vibes anyway. It shouldn't be taken too literally. The example in this article may have been a good use-case for Clojure's multimethod facilities.

https://clojure.org/reference/multimethods

Let’s not confuse domain-driven design with data driven design.

Most architects would assign the DDD acronym to Domain-Driven Design.

</pedantic-rant>

I align with this. Data driven design is an easier way to say domain driven design. They mean the same thing but because of the former, it’s confusing. DDD ensures you know the domain you’re working with, in language that the business is familiar with. A banking DDD would have Accounts Deposits Withdraws Authorized Users Statements etc.

They absolutely do not mean the same thing.

Data DD

- Account.Insert

- Account.Create

Domain DD

- Account.Open

The former is a data store activity and the latter is a business activity.

Depends on the field, in real-time application and games it’s gonna mostly refer to data driven design more than domain driven design. Honestly haven’t heard domain driven design inside the games field since the early 2010s.

Is software architect a common title in game dev ? It's been a decade since I've looked at the field but it kind of antithetical to the industry as I remember it.

It's been a long time since I've interacted with a software architect. As poor as wikipedia is a source for a layman like myself, I still use it as a first step to explore topics I am not familiar with.

Would software architects be aware of all the various paradigms applicable within a particular language? Much like me trying to pick the right tool to complete a particular project...

https://en.wikipedia.org/wiki/Programming_paradigm

is there a good reason why Domain-Driven Design is not linked in the article above, or just an oversight?

https://en.wikipedia.org/wiki/Domain-driven_design

DDD is not a programming paradigm, it’s a software design approach. You can combine DDD with various programming paradigms such as OOP and FP.

software design approaches are frequently closely tied to programming paradigms, and DDD (while, like Object Oriented Analysis and Design, it can be used with any paradigm) is closely associated with OOP, and at least the early writing on it (if there is any newer that this isn't true of, I haven't seen it) is very tightly coupled with OOP.

That is true to a certain extent, but that doesn’t make it a programming paradigm.

Yes, it was a related observation, not a refutation.

Thanks for the clarification.

So just to spell it out: Domain-driven design (nor Data-driven development per the article title, nor Data-driven design) aren't programming paradigms, they are software design approaches.

Whereas Data-driven programming, as linked from the above wikipedia is infact a paradigm and something else entirely. Alright.

They have 'Data-oriented' listed in the wikipedia paradigm article, but it links to 'Data-oriented design' which is clarified as a software design paradigm, as distinct from a programming paradigm.

I like that the author points out that trying to write a new interpreter is not a good idea. However, to call that DDD, and then to say DDD is a lie is where I would disagree. I guess one could say that it's DDD taken to some extreme, but I would just call it a poor choice as the author points out.

Also I assume that the author is equating DDD with Data-Oriented Programming. Here's a good book on the topic: https://www.manning.com/books/data-oriented-programming?ar=t...

One thing I don't love about the book though is that it tries to demonstrate how to do DOP with Object-Oriented languages, which looks to be difficult to accomplish

You're assuming wrong, they are not talking about data-oriented programming.

DDD usually stands for domain-driven design.[0]

[0]: Or in earlier times for the Data-Display Debugger.

I was reading the title thinking 'What's wrong with domain-driven design?'

Or documentation driven development.

Sure it doesn't stand for a bra size, and you're a bigot for continuing to use the acronym [0]?

[0] https://nitter.net/sarahmei/status/1073234104311734273#m

Wow, that's a thread. I'd like to know who is actually hurt by this. I'm quite liberal, but a lot of the culture policing seems more like a "White Man's Burden" situation rather than a reflection of a legitimate, sizeable complaint from minority groups who stand to be hurt from it. "Latinx" is a prime example, not least of all because the spelling is suited for English (and even then it's awkward) and not Spanish.

I'm confused by this. I've never written Clojure or Lisp. I'm a run-of-the-mill C#, .NET developer. But even I am used to treating LINQ expressions as both code and data. I can take a LINQ expression, modify it if I want, compile it, and execute it against a given runtime context. I would have expected this duality between code and data to be intrinsic to Clojure. Hence my confusion.

C# isn't really my cup of tea, but from what I know about LINQ; I'd say that's code, not data. You couldn't put that expression in a database, read it back and interpret it without some serious hoop jumping.

The lines get blurry in Lisp, since the code is explicitly encoded as a data structure (tree).

LINQ expression trees can be serialized with a custom serializer, but it's a mistake to do so for a different reason: deserializers which materialize code or substitutes for code are major vulnerability vectors. Anyone who has had to upgrade Jackson or remove naive pickling will be familiar.

Arguably, as long as you don’t allow specifying arbitrary types in the serialized format, it should be safe as you can’t otherwise construct gadgets to produce unexpected side effects.

I'll concede your point about saving LINQ in a database.

But you know what I can do, in a day...

Take a rule, written as a function in C#, compile it to an assembly, save it in a database with a name and checksum, and then dynamically load it later when I want to execute it.

ta daaaaaa..... :-)

I take that back, I'd still write a LINQ expression, and compile and save that in the db.

That way I can display the original rule as text to a user and let them modify it.

ta daaaaaa...

A program is written to do something, and you can only think so far about its data without thinking about what you want to do with it.

It's definitely a spectrum though, you may choose to encode more or less of the program logic as data. Take a rule engine/expert system for example, the code is the least interesting part since all of the domain logic is encoded as data.

Formally, the phrase "encode more or less of the program logic as data" seems meaningless, as you cannot implement a Turing machine with static symbols alone. I feel the quoted phrase is intended to capture the idea that a suitable representation of the data can simplify the processing, but that is what is behind the point I am making.

For example, in the case of an expert system, the data provides the specific facts of the domain and the logic is implemented by the program. In choosing to use an existing expert system engine, you have made a decision about how your system will go about achieving your goals, and that places significant constraints on how you should represent your data.

How is using a function not sweeping domain complexity under a rug?

A complaint was that the data-driven development approach created a _poor_ DSL and then a non-DDD solution didn’t use a DSL at all…

What would a solution using a good DSL look like, and how would it differ from the DDD-based approach?

Yeah the Data-Driven part is obvious but the development/design... Approach? Way?

Serendipitous in some sort of a backwards-way that I have personally already gone that way ~a decade ago, with even func/interface-first, and the first example I found is https://github.com/drbig/ricons/blob/master/ricons.go#L57-L5... (it's an example, sorry).

As usual the devil is in the balancing, as in most languages functions won't be data but code. Unless Lisp (...that statement is superficial; homoiconicity is the word).

Usage-context same stuff link-to-code: https://github.com/drbig/ricons/blob/master/symsquare.go#L61... (it's an example too, sorry).

The point is that my opinion is that `declarative` is pretty neat and powerful. Easy? Maybe, but not really.

I am new to Clojure and I am struggling to understand what the point made of the author is.

I understood the Data Orientation of Clojure as building mostly map-shaped, atom-wrapped representation of application state, and then functions and procedures that operate and transact on that state.

The ref to the data being the "backbone" of the application (so data oriented). Is this incorrect?

It seems to me like he's actually criticising the "data programmable engine" (I just finished Joy of Clojure where they describe it) where you feed data to a function to change it's behaviour (as opposed to example passing in a function?)

I am genuinely asking.

My interpretation... the author is arguing against creating a new Domain Specific Language when you could just use the host language.

this is the blogpost version of "could have been an email" :D thanks.

I haven't understood that but then again I don't really understand what the post is about.

I would agree if so with the caveats that it's all fine unless the host language lacks a lot of semantic features that the DSL will add.

Then again, I don't understand the blog post. Is it about programmable programming language?

Seems there are quite a few perspectives around this.

For me, "data-driven" approximately means "Something coaxed into a ~3NF/BCNF schema that the business stakeholders can simultaneously understand".

In my mind, this term doesn't imply any specific language or database technology. What it strongly implies to me is a certain modeling and normalization of the domain such that any reasonable business requirement can be realized within a ~constant cost factor.

A warning worth taking to heart!

Even apart from Clojure and so-called "DDD," there was a kind-of-recent push toward making your application logic as "declarative" as possible. Following that strategy would enable some compelling benefits, mostly related to scaling, but also enabling "self-service" for savvy application users.

The XML comparison is stinging, considering XMLs modern popularity. Is it a case of what's new is old?

You can apply this reasoning to "anything-driven design."

Just stop being dogmatic and choose the right tool for the job.

As a sidenote, I guess spec could be useful for "verifying" the data.

I guess it depends on what you understand by this term. For me defining structA > f1 > f2 > fn > structB is easy to read, explain and modify. Never had issues with it but I did not use a lot of clojure before so maybe I don’t know the exact way.

I guess someone shortened

> "Show me your [code] and conceal your [data structures], and I shall continue to be mystified. Show me your [data structures], and I won't usually need your [code]; it'll be obvious.''

as DDD.

No, the data in DDD is part of the program logic, not data that is being managed by the program, as in your quote.

That is Fred Brooks' quote.

Also my comment was probably incomplete in context... as in how perfectly fine piece of advice could have been abbreviated, misunderstood and became a cult.

All these fads prevent us from producing simple maintainable code.

The author of DDD has openly admitted in a podcast that his book is essentially a scam.

He wanted to write a book about Clojure, but the publisher said that there was too little interest. So he "tricked the readership into reading a book which is about Clojure without saying that it is about Clojure". I'm paraphrasing, but that is the genuine gist of what he said (much to the non-amusement of the interviewer who must have felt deceived as well).

Source: https://changelog.com/podcast/522

I even read this stupid book because there was a hype at work and it is so full of strawman arguments it made me furious.

you prefer statically typed languages?

I prefer books worth reading

Reply on news.ycombinator.com