aard

This is wonderful, thanks! I was unaware of these!

I've been experimenting with making AI generate literate code. The goal is to have the AI produce a bunch of prose alongside the code. A lot of context for the purpose and design of what you are doing usually gets thrown away, but with literate programming you can save it. And this way I can have some hope of understanding the code being created. I've liked it so far.

I am using a literate programming project I built (https://github.com/adam-ard/organic-markdown) and I have an AGENTS.md file that looks like this:

"All the work we do on this project should utilize the organic-markdown literate style that you see explained/demonstrated in the `organic-markdown` project: https://github.com/adam-ard/organic-markdown

Some guidelines for how you should utilize organic-markdown literate programming..."

Then I list a bunch of conventions for making literate code the way I like it.

I've been working on my own version of a literate programming system (https://github.com/adam-ard/organic-markdown). It's kind of a mix of emacs org-mode, jupyter, and Zettelkasten. But, because it's based on standard pandoc-style markdown, you can use it with a much wider range of tools. Any markdown editor will do.

Even though I made it as a toy/proof of concept, it's turned out to be pretty useful for small to medium size projects. As I've used it, I've found all kinds of interesting benefits and helpful usage patterns. I've tried to document some; I hope to do more soon.

--https://rethinkingsoftware.substack.com/p/the-joy-of-literat...

--https://rethinkingsoftware.substack.com/p/organic-markdown-i...

--https://rethinkingsoftware.substack.com/p/dry-on-steroids-wi...

--https://rethinkingsoftware.substack.com/p/literate-testing

--https://www.youtube.com/@adam-ard/videos

The project is at a very early stage, but is finally stable enough that I thought it'd be fun to throw out here and see what people think. It's definitely my own unique spin on literate programming and it's been a lot of fun. See what you think!

Location: Salt Lake City, UT

Remote: Yes, hybrid or onsite also OK

Willing to relocate: Yes

Technologies: C/C++, Python, Embedded Linux, IoT, Docker, Bash, Golang, OpenGL, Qt, CMake, IaC, CI/CD, 3D Graphics/Math, Real-time Systems

Résumé/CV: https://www.linkedin.com/in/adam-ard-03245215/

Email: adam.ard2011@gmail.com

Senior system engineer with 25+ years of experience in robotics, embedded devices, and 3D applications. I've worked on everything from humanoid robots and smart energy systems to AAA console games and graphical simulation tools. I’m passionate about developer experience and recently created Organic Markdown (https://github.com/adam-ard/organic-markdown), a literate programming framework. I write about software design and developer culture at https://rethinkingsoftware.substack.com. Looking for deeply technical roles where software meets hardware and math meets the real world.

I very much support this sentiment! If we want a decentralized internet, we need to stop relying on large companies to manage everything for us. Git was designed to be a p2p system, but we very quickly centralized it with forges like Github. It is very discouraging. Most of the internet is like this now--managed by a handful of very powerful organizations. There is no end to the problems this will cause.

> Sometimes the exciting things are exciting for very valid reasons

Very true! I would also add: Sometimes boring things are boring for reasons that actually make them poor solutions.

For example, some systems require tedious, mind-numbing configuration. This is boring, but also a good reason to NOT use something. If it takes hours and hours of manual tuning, by someone with special training, then the solution is incomplete and at least needs some better defaults and/or documentation. It might be a poor option.

Another example is a system that does not lend itself to automation. Requiring manual interaction is certainly boring, but also does not scale well. It is a valid reason to disqualify a solution.

Boring can often be a smell--an intuition that a solution is not fully solving the problem.

I see your point here, but I want represent a counterpoint to this line of reasoning, from my personal experience. I've been in a lot of situations where someone simply wants their way--in this case they want the organization to choose their personal software preference--and so they call it the "boring" choice.

By calling it boring, they characterize their _preference_ as the majority accepted, mature, obvious decision and anything else is merely software engineers chasing shiny objects. The truth is almost always more nuanced. Both solutions have pros and cons. They trade-off different values that resonate more or less with different people.

So, please be careful with the "it's boring and therefore obviously better" argument. Don't let it be a way to summarily dismiss someone else's preferences in favor of your own--without a deeper discussion of trade-offs. Otherwise it's no better than any other condescending attempt (ex. I'm in charge, so we are doing it this way. No one ever got fired choosing IBM/Microsoft/..) to win an argument without having to present real arguments.

Tim Bryce’s work, such as his “Is PRIDE too rigid?” article, uses analogies with traditional engineering (like jet engine design) to argue that skipping critical design steps can lead to costly rework later, but it stops short of offering rigorous empirical proof or extending the idea to the realm of software.

"Big Design Up Front" in software is the whole reason the Agile movement came about. To put BDUF into Agile is to miss the point completely--they are fundamentally incompatible. Any company that insists on "lots of analysis and design cycles before a single line of code is written" will be left in dust by competitors if all their programmers don't quit first.

There is a lot of confirmation bias in the industry around estimates. If an estimate is ever wrong, it's never taken as evidence that estimation in general is a flawed practice. The failure is always attributed elsewhere, usually to a lack of effort or focus. But if an estimate happens to prove true (even broken clocks are correct twice a day), everyone pats themselves on the back and says, "look how predictable we are. We are so good at estimation."

The autonomy you gain from starting a business is also a big deal. You can work long and hard with low stress when you are the one making decisions. Burnout comes from being powerless and at the mercy of someone else.

Isn't that pretty much textbook Scrum? The language in the Scrum Guide couldn't be much clearer...

"The Product Owner is also accountable for effective Product Backlog management, which includes:

- Developing and explicitly communicating the Product Goal;

- Creating and clearly communicating Product Backlog items;

- Ordering Product Backlog items; and,

- Ensuring that the Product Backlog is transparent, visible and understood. "

And if that wasn't enough...

“…the entire organization must respect their decisions.”

“The Product Owner is one person, not a committee.”

“Those wanting to change the Product Backlog can do so by trying to convince the Product Owner.”

Sounds like "sole authority" to me.

I think we have been stuck with this for so long we have forgotten that any other way is possible. Scrum is just a complicated way to divide labor. No one else at a software company needs a formal process to partition their work. They just sit down, talk and then go do the work. They come up with a light weight way -- a informal process -- to get stuff down. They would never even think to ask themselves, "I wonder what formal process we can follow to assign work?" They just do it.

How about letting engineers do what everyone else at the company does? Get together in teams, divide up the work, and check back later. You are asking for an alternative thing to mandate upon engineers from above -- something formal. I say, just let engineers manage themselves. No mandated process.

The tone of a code review suggestion reflects an underlying assumption about what a code review is (or should be). Often developers disagree on this point. It seems like we fall into one of two camps.

1. Reviewing code is like proofreading a paper. Suggestions are welcome, but there isn't an expectation that all suggestions be followed. The author of the code makes the final decision based on their objectives.

2. A code review is a gate that is closed until all comments have been fully addressed. The author must comply to the liking of all who choose to comment, or they are not authorized to continue.

The tone of review comments will naturally follow from their understanding. Understanding #1 yields comments aimed at persuasion. (Should we, Could we, You might consider) While #2 yields more compulsory language (Change ..., You should). Or passive aggressive language that is meant to be compulsory but softened to sound less aggressive (I would, Could you, shouldn't you).

It is my opinion that understanding #1 is most appropriate. It prevents unnecessary bottlenecks and is more respectful or differing opinions. If developers are peers, than #1 is the understanding the most accurately reflects that.

interesting twitter discussion: https://twitter.com/martinfowler/status/1345375503277023232

I fully agree that the problem is the "JIRA mentality". Specifically, that a developer should be managed with a task list instead of giving them roles/responsibilities in the organization and letting manage the tasks themselves. Sadly, almost all project management tools fall into this trap. A whole different type of tool is necessary to get us out of this rut.

Totally agree. Reminds me of a quote from Peopleware:

"Most managers give themselves excellent grades on knowing when to trust their people and when not to. But in our experience, too many managers err on the side of mistrust. They follow the basic premise that their people may operate completely autonomously, as long as they operate correctly. This amounts to no autonomy at all. The only freedom that has any meaning is the freedom to proceed differently from the way your manager would have proceeded. This is true in a broader sense, too: The right to be right (in your manager’s eyes or in your government’s eyes) is irrelevant; it’s only the right to be wrong that makes you free."

You have a valid point here. Maybe I was too hasty to judge the book by it's summary. From other comments, it sounds like this book takes a pretty reasonable approach.

I am frequently skeptical of efforts to show the "one true way" of programming, because it usually results in some poorly vetted process being forced on my at work. So, I was probably too quick to jump to conclusions.

It actually sounds like this book is good about showing that a lot of our current assumptions about good process are faulty.

The point I was trying to make about the "labour of the cognitariate" was that developer doesn't really produce software, they create the blue-print for the software (source code), and then compilers or interpreters produce the actually software. It may seem like an overly semantic point, but I think it is an important distinction to make. It changes they way you think about software development.

This is good to know. I should jump into the book. I may have gotten the wrong impression from the summary -- the quotes that I shared. I think a scientific effort to look at the many practices that are currently in vogue is very much needed. So many are taken as gospel truths and lorded over people in the name of science-- but they really aren't supported by any rigorous science at all. If this book points that out, than I am all for it.

Especially if the book has more a system thinking approach. Many studies isolate one practice (pair programming, code reviews, etc..) and can show benefits, but they ignore the systems they function in. Apparently opposite approaches, supported by the right personalities and environments can often be equally effective. From your comment, it looks like there may be some analysis like that too.

> "In a competitive market for development work and staff, paying people to learn from mistakes that have already been made by many others is an unaffordable luxury; an engineering approach, derived from evidence, is a lot more cost-effective than craft development."

I disagree with this characterization. Yet again, we developers are being told to just follow the program. That programming is not an artistic or craft endeavor that benefits from experience and intuition. It is like working in a factory where coders just bang out widgets on an assembly line. And the self-appointed thinkers will optimize the process for us.

What is at risk by not allowing developers to "learn from mistakes" is autonomy. Striping developers of their autonomy is the primary cause of poor performance, not an inability to execute so-called "best practices"

Attempts to codify the process of software development always fail, because coding is a design process not a manufacturing one. Developers do their jobs in many different ways, many of which are equally effective. There is more than one way to skin a cat -- especially in creative work.

> "The labour of the cognitariate is the means of production of software systems"

This false assumption is at the base of the problem. The work of the compiler (or scp) are the means of software production. Coding is design. Once the design is complete, the results are compiled and copied to their target environments. In software, production is negligible. Which promotes the misconception that developers are producing software. In actuality they are designing software. The difference may seem subtle, but it is crucial.

One of my favorites:

"If you know what’s in it, you don’t know when it will ship. If you know when it will ship, you don’t know what’s in it."

Thanks for the links to JP Lewis' article. It is excellent material.

I am so glad that Fog Creek is sticking to their guns on developers having private offices. The open office trend that everyone seems to be following now is rather discouraging. As a developer, I find it so hard to concentrate deeply when I need to.

Hence, my suggestion that the product department be used to inform the programmers so they make wise feature decisions. It is not appropriate, however, for product to be in a management position over engineering.

Thanks for pointing out my typo.

Also, how do you account for the many successful software companies that are started by just programmers, with no product direction?