Visual programming with say boxes and lines at its core should not involve an ast.

You need to think outside the box. Your thinking of textual programming as the core and the visual programming as an abstraction above it which is how the industry has been focused since its inception.

https://en.wikipedia.org/wiki/Nassi%E2%80%93Shneiderman_diag...

Some visual programming languages use lines to represent control flow, others use them to represent data flow, and others use a mixture of both. Additionally, different kinds of control and data flow can operate at different frequencies within the same system.

Snap!, which is essentially a visual block based version of Scheme, allows you to pass functions or closures, by wrapping blocks in gray insulating "gaskets" like lambda expressions that delay evaluation, and even supports macros, special forms, continuations, user defined control structures, and threading, just like Scheme.

https://snap.berkeley.edu

Another example is Max/MSP, which primarily uses lines to represent control flow and data flow. However, it also distinguishes between data flow at "simulation tick frequency" and a much higher "signal processing frequency". This means thousands of audio samples can flow along one line at every simulation tick, while only a single piece of data or signal (like a pure data-less control flow "bang") may flow along other lines at a slower simulation tick frequency.

https://en.wikipedia.org/wiki/Max_(software)

In data flow visual programming languages, a data flow box can emit any number of data outputs at once in parallel.

A control flow box typically emits only one control flow output at a time, unless it acts like a "fork" operator. Fork operators, as seen in Petri nets, support concurrent processes by allowing multiple control flow outputs.

https://en.wikipedia.org/wiki/Petri_net

A data flow conditional works like a relay with three inputs (A, B, and Select) and one output. The Select input determines whether A or B is the output.

A control flow conditional, like a traditional flowchart "if", has one or more control flow inputs, a Select data input or embedded expression, and multiple control flow outputs. The Select input or expression chooses which control flow output the "program counter" branches to next.

Pure data flow networks do not have a single explicit "program counter." Instead, they typically evaluate nodes in partial dependency order, which may include loops (introducing a one-cycle feedback delay). Petri nets have multiple concurrent control flow "tokens" that flow between boxes along the lines in parallel.

Another example is Body Electric aka Bounce, which is a data flow visual programming system with relay-like data flow conditionals, but also each box has an implicit "enable" input that you can use to switch it on and off (like a power supply), so when it's turned off, the last calculated outputs are latched and buffered, and can be read by downstream dependencies, but the values are not recalculated during simulation frames when it's not enabled.

https://donhopkins.medium.com/bounce-stuff-8310551a96e3

In Blender geometry nodes, data flows from left to right, while functions can be passed and applied in a way that evaluates right to left against the data flow. That is, functions are passed on the left, but the data is then processed through the function, either once or iteratively.

https://docs.blender.org/manual/en/latest/modeling/geometry_...

New Blender 4.0 Loops!

https://www.youtube.com/watch?v=mr_nQBoJPXw

Functions or operations can be encapsulated within nodes and passed along the data flow. For example, a "Subdivide" node contains a function to subdivide geometry, and this function is applied to the geometry data passed into the node.

Nodes can pass functions as parameters to other nodes. For example, a "Function Input" node can be used to define a custom function that can be passed into another node, such as a "Map Range" node, which applies the function to its input data.

Functions are applied to the data as it flows through the nodes. For example, a "Set Position" node can apply a function that modifies vertex positions based on certain criteria (e.g., noise texture values).

Some nodes, like "Attribute Math" or "Attribute Vector Math," apply mathematical operations to attributes of the geometry, effectively using these operations as functions that transform the data.

Nodes can be configured to apply functions iteratively or conditionally. For example, a "Repeat" node can apply a function multiple times to achieve iterative processing, such as repeated subdivision or transformation.

Conditional nodes, like "Switch" or "Boolean Math," allow functions to be applied based on specific conditions, enabling selective processing of data based on attributes or other criteria.

While data flows left to right, some nodes can evaluate data in a right-to-left manner when applying functions. For instance, a "Function Output" node can send data back up through connected nodes for additional processing before final output.

This evaluation allows for more complex operations where data may need to be processed in multiple stages or cycles.

But this "other" form on first glance can look very different. There are no boxes and lines in C++ or python for example.

The main point of my post is to suggest that there are other equivalent concepts between text and box/line.

The structs/records/tuples approach would work (I think this would correspond with "currying"), but I have this weird idea that certain types of reasoning have a "shape" and that we might be able to more heavily lean on analagous reasoning if we could easily show that shape in correspondence with explanations of our reasoning. I worry that currying everything would force us further away from the "natural" shape (if there even is such a thing).

I've been working on something called plibs (simPlical mad LIBS, since really we're showing a https://en.m.wikipedia.org/wiki/Simplicial_complex here). The functions are represented as tuples of the same arity, and described via sentences with blanks in them (e.g. when you run ____ code in ____ environment you get ____ from stdout). One with three blanks would then appear like a triangle and then I'd fill it in with a gradient indicating which sides are inputs and which are outputs. Functions sharing a mad lib would appear as a regular polygon in the same color (one edge for each blank). Or maybe they're sort of starfish shaped so that each leg can reach out an touch whatever the data representation ends up being.

One could imagine algorithms that generate patchwork quilts or 3D structures (like how protein folding is represented) out of such things. These could be computed only in one direction (from inputs towards outputs) but they could be traversed in any direction if one wanted to explore the relationships.

I'm hoping it will be a nice way of citing your computational sources. You would attach them to any computational result (alongside a scientific paper, perhaps) as a way if saying "here's how I came up with this, rerun it to verify my result". Aside from the computational pathway that yielded the result that's being scrutinized, you could explore adjacent plibs to understand why certain inputs were chosen or to gain other contextual hints.

Now that I'm describing it it sounds a bit like that awful zooming-through-towers-of-data visualization in "Hackers" the movie.

So the "edge" that I was using as a progress bar in my previous description is really a series of paths through a space made of these things. The details of the functions being called (and of the datasets being generated) are attached to the polygonal regions that the paths connect. The paths may merge or branch. It could get unwieldy, so the user would have to be explicit about which plibs to show and which to highlight as a chain of computation.

As for how the represent the intermediate data and their types, I keep changing my mind. Some days I like the stacks of cards thing, some days not. Hopefully something will sick soon, then I'll try to build it.