I build a lot of open source patch editors for older synthesizers. My beef with Sysex is that every manufacturer uses completely different approaches to defining their own proprietary messages with it. For example, nearly every synth in the universe has a sysex message for dumping a patch (a full set of parameter settings) from a synth to another or to a computer; but they define their messages is radically different ways, so I must construct an entirely different set of parsing and emitting tools for every single synthesizer, even within a given manufacturer. It's a nightmare.
So continuing this example, if the MIDI association had gotten together early on and said that MIDI dumps should have a header that looks like THIS and then all the parameters in order, two bytes per parameter with no bit packing whatsoever, no two's complement, and end with a specific checksum, then I'd have written 10x more patch editors so far. I wouldn't have to write custom parsers and dumpers: I'd just provide a list of parameters and their bounds.
Most SYSEX dumps are just dumps of the plain ol' structs that the synth engines are using to drive their output. A lot of synths don't have the processing power to do more than just dump the struct.
So, it wouldn't really make much sense to have them all use the same struct - this can't be enforced too well. Forcing synth mfr's to all use the same struct means that, even if they have their own internal plain-old-structs, they'd need code to dump the SYSEX according to the standard.
I don't think it's processing power: it's stingy RAM utilization. Many bad actors (Kawai, Casio, later Yamaha) did crazy bit-packing of parameters rather than just keep them in a simple array, while the more sane (Oberheim, E-mu, Waldorf, early Yamaha) at least tried to pack in a consistent way. Other bad actors (ahem Korg, as late as 2000) decided to use, shall we say, creative parameter encodings, going even so far as embedding textified versions of parameter numbers into sysex byte streams. And many used all sorts of crazy schemes for banks and patch numbering, most of which are incompatible with one another.
And it's not just encoding: basic synth patch dump features are missing from different models. There are five basic tasks that most synth editors require:
- Change patch - Request a dump from working memory - Dump to working memory - Dump to patch RAM and save - Send a single parameter change (for any parameter)
Manufacturers couldn't even agree to make machines which supported all five of these. Some machines (Yamaha) have no way to write to RAM. Some machines couldn't do single parameter changes. Some machines can't properly change patches in a consistent manner. Some machines have no patch request mechanism. Many machines can't dump to current working memory: only to patch RAM!
The situation is only getting worse. Whereas in the past manufacturers at least attempted a complement of sysex messages, now many manufacturers can't even be bothered to allow access to their machines (Korg, Roland). Others treat their sysex messages as proprietary secrets (Arturia, Digitech, Alesis).
There is only one truly good, shining actor in the open MIDI spec space, and that is Sequential. Which shouldn't be a surprise given who runs it.
This makes no sense. That would also imply a way to discover (and name, and probably provide semantics for) all parameters. That's a huge ask if MIDI (even MIDI 2.0) is the only communication protocol available.
Yes, the first 4 of your list are common. The first one is covered by the core MIDI spec. The 2nd and 3rd have no standard msg, but your complaint seems to be about the contents of the message, which is no business of the requestor. The 4th assumes "patch RAM", which cannot be assumed, as you note, and that seems correct to me.
Why? It's highly standard. About 90% of the synthesizers I've written patch editors for provide exactly this facility. In fact some (PreenFM2, Korg Microsampler, Futursonus Parva) provide only this facility.
> The first one is covered by the core MIDI spec.
Actually it's not. Program Change only works for 128 patches. If a synth has more than 128 (and many do), they must rely optionally on Bank Select, but their definitions of "banks" vary because a bank is not a formally defined concept. Some rationally treat banks as divisions of the patches. Others treat banks as media choices: cards versus RAM versus ROM. Some require that Bank Select be immediately before Program Change with nothing in-between; others do not. Some ignore banks entirely and instead define a "Program Change Table" of 128 slots pointing to arbitrary patches in memory, and then Program Change indicates which patch slot to use.
And there are several major synthesizers (Yamaha TX81Z and DX11 are famous examples) where Program Change is in fact broken and requires unusual workaround hacks. Further, most synths require a program change prior to a patch load: but others (notably the Oberheim Matrix 6 and 1000) require a program change after a patch load. It's a mess.
Back in the 90's, when things like "MIDI Librarians" were common (and widely used), each new device needed to be added to the MIDI Librarian's code to deal with the specifics.
I think you may have misread what I had said. I didn't say that patches had to be the same format or content -- that would be insane.
What I mean is that, if they have to reformat their internal struct to conform to a standard, they don't have the processing power to do this munging. Not that they'd care, as you have noted elsewhere.
This isn't uncommon in serial protocols, but MIDI has been lifted several layers above the UART it was designed for, and one of its strengths for everything but SysEx is that you exactly how big messages are going to be and preallocating space for them is trivial.
That's the reason why you want to break compatibility with MIDI 1 spec? Really? Just like for an HTTP stream, you don't need to know how big the stream is to process it, just when it ends, there is no new problem to solve here.
It's also weird to say SysEx is needed for backwards compat with MIDI 1.0. You can just require a translation layer between MIDI 1.0 sysex and MIDI 2.0 configuration/property exchange for backwards compat. Opaque binary as a part of the protocol does not encourage compatibility among hardware.
Leaded gasoline was successful for decades too, but there comes a time to take things to the next level.
MIDI 1.0 was successful for 30+ years because of its simplicity, so I'm not sure what you are disagreeing with. Building a MIDI payload is extremely simple and serious manufacturers will document their MIDI implementation in public documents.
What part of the MIDI spec did you have hard time with when developing software using the MIDI 1.0 protocol?
Your comment about gasoline has absolutely nothing to do with MIDI.
Allocating memory (by asking the OS) in a realtime context is a no-no. This means guessing the maximum size of a sysex ahead of time or breaking RT programming rules.
The other tricky issue is the way MIDI 1.0 defines the transmission of the LSB and MSB for 14 bit messages. They got it backwards, requiring the receiver to use a timer. Might have been sensible for baked-in-h/w MIDI gear, not so great for general purpose computing.