Synthesis Methods Explained: What Is FM Synthesis? | Hacker News Reader

Synthesis Methods Explained: What Is FM Synthesis?

perfectcircuit.com

The original paper is quite short and readable: https://people.ece.cornell.edu/land/courses/ece4760/Math/GCC...

I love through-zero frequency modulation (TZFM).

If your oscillator can do TZFM you can bring it’s frequency down to zero, where it stops oscillating and then to “negative frequency” where it starts oscillating again, but with a reversed waveform.

This has two main usages:

first you can do normal FM sounds with very deep modulation without getting strange pitch shifts because the frequency is capped before zero.

Second, you can set the oscillator to start at zero and then modulate symmetrically around zero. The sound you get is quite unique — ranging from woody sounds to growling lead sounds. The fundamental pitch is decided by the modulator (as opposed to the carrier in usual FM). Symmetric TZFM takes the modulator signal and integrates it, then adds harmonic content by folding according to the waveshape of the modulator. This works best if the two oscillators are kept in soft or hard sync.

My favourite patch is to use a harmonic oscillator as modulator for a TZFM carrier. Modulating the distribution of sines in the modulator and the frequency of the carries gives such a unique set of timbre controls.

I'm intrigued. Anywhere I can hear some examples of this, or know of any (free) ways I can generate some with software?

Instruo Neoni is one oscillator which can do it. I think one can get on VCV rack (which I think is free).

The Neoni is demonstrated here: https://youtu.be/kkAagua3NN4 (Symmetric TZFM starts around the 57 minute mark).

Here is a track I made using TZFM (I am no muscician, just experimenting): https://hakon.gylterud.net/snd/foot-stomp.flac

The only sound sources are a TZFM oscillator and a kick drum.

I guess it's like time running backwards. Or the speed. FM is basically turning the speed up and down, no reason it can't go backwards too! Edit: or a DJ doing a record scratch

Yes, this is precisely the intuition I have. The oscillator is running through a waveform with a speed corresponding to its frequency. Frequency modulation speeds it up or slows it down. At negative frequency it goes backwards. With symmetric TZFM it spends as much time going backwards as going forwards, and does so in audio rate giving a folding of the modulation signal.

For a couple years I've wanted to write a subtractive synth from scratch in software. But I _cannot_ find any source that explains how to write a low-pass filter, in code, without assuming tons of knowledge about calculus and these odd things like Z-transforms?

Like, I understand FFTs and DCTs. But the explanation for how to construct filters in software is pathetic.

I looked at the code for VCVRack but could not understand it. It's already optimized for SIMD, processing 4 signals at once, which didn't help readability. (I'm sure it's very fast though)

> But the explanation for how to construct filters in software is pathetic.

Pick your favorite FIR filter, set up a ring buffer, input samples go in, at each step you multiply the buffer elements by your filter coefficients, sum, and output the result.

The hard part is calculating the coefficients.

Not really, there's plenty of online tools to do it for you such as http://t-filter.engineerjs.com , and more offline ones: https://docs.scipy.org/doc/scipy/reference/signal.html#filte...

A intuition for creating a lowpass filter ... audio happens in the time domain ... Fourier transform outputs the frequency domain representation of the audio you feed it ... number of audio samples you feed into the FFT call determines the element count of the array returned ... now you have an array of your audio in the frequency domain ... for lowpass crank down to zero array elements above the first quarter of them ... this silences those frequencies ... to render the audio ( generate sound ) feed your edited frequency domain array into an IFFT ( inverse Fourier transform )

This does work, but the sound of it isn’t super pleasing and it limits your filter design options. Checking out time domain IIR filters and methods for generating coefficients gets you flexibility and efficiency with no latency. I posted some links above but the RBJ Cookbook is a good place to start.

https://fiiir.com/ will help you design a filter. Select "Python script" to get some comments on how to apply that filter on your data (look for "convolve").

For more understanding, I do recommend reading up on DSP and Signals concepts, they're pretty critical to know if you're designing filters.

Will Pirkle's books might be close to what you're looking for. He covers Z transforms (which you do ultimately need to understand to code up filters) but also provides implementations you can use as a starting point/to get a better idea of what to do concretely in software.

Was going to recommend them too!

Classic biquad approach with instructions for calculating coefficients of various filter shapes:

https://webaudio.github.io/Audio-EQ-Cookbook/audio-eq-cookbo...

An analog modeling approach, but cross-reference the coefficient calculations!

https://cytomic.com/technical-papers/

https://github.com/ThePJB/okiir/blob/a356b5b09917c0dbd856c9b...

Here's mine lol

Yeah I can't help but think there might be a niche for intermediate level discussion of these topics. I have a burning desire to wrap my head around z transform and other dsp black magic... (Like for example application of Hilbert transform to synthesis, anyone?) A lot of it comes back to understanding the complex plane. Like, I think Z transform relies on the fact that every *e^j2pi wraps around the unit circle again for one sample period.

This was one of the things that was most surprising to me doing software defined radio. It was "how does a finite impulse response filter work?" and "how to I make it work in a specific way?" The interesting question for synths would be how to give it an input that that changes its cut off frequency.

For me, the "aha" moment was when I connected the dots between "averaging" (which is a common way to filter noise out in an embedded computer) and "rolling average" is just summing the previous 'n' samples as sample * 1/n, and a picture of a fir filter that was (C code but it's pretty readable)

   out = 0;
   for (i = 0; i < n; i++) {
     out = out + sample[n-i]/n;
   }
   return(out);

That is a "FIR" filter where the coefficients are all 1/n. Now take that and do the FFT of n samples of n/1 (adding zeros to get the resolution you want). And that is the frequency response of your filter for frequencies between 0 (DC) and sample_rate/2.

For me at least that connected a lot of dots in what I was reading.

You probably want to defer the division until output, and not do the loop each time - instead, just subtract back N samples when adding the current sample.

this depends on how many samples you have and what kind of precision you're working with for the accumulator. as floats get bigger they lose precision.

To be fair, I typically use the MAC in an FPGA to implement this with the coefficient as a fixed point value. As a result the entire step is one cycle, depending on how many MACs are available I will parallelize the algorithim to support all available MAC blocks.

But in C I typically compute the coeficent outside the loop and use it in the multiply sample * (1/coefficient) vs sample/coefficient, even the STM32 microcontrollers have single cycle multiply available.

Z-transform is very similar to Fourier or Laplace transform, but in discrete time. These things are not that hard, if you get FFT already then you can get there if you want.

Check out GNU Radio. In contrary to what the name suggests, it's not just for radio, but can be used for doing complex DSP operations. It includes a nice filter design tool, too.

https://www.gnuradio.org/grcon/grcon18/presentations/The_Bri...

https://w7fu.com/audio-oscillator-project/

The best option to install it on Windows is radioconda: https://github.com/ryanvolz/radioconda

Have a look at the filter code in Csound, for example:

https://github.com/csound/csound/blob/master/Opcodes/butter....

I wrote a subtractive synth in java for my masters project. It was a fascinating domain to learn. For example, how to manage the independent voices, wavetable interpolation, envelopes. I was probably in over my head, and if I recall correctly, my supervisor was warning me about the complexity, since it was a software engineering programme.

The filter was the weirdest, least intuitive part. I ended up porting some open source code to java. This eventually worked, but was essentially impossible to debug without some sort of software oscilloscope.

That was about 18 years ago (yikes), and I still think about DSP from time to time.

The best intuition I have of low pass filters is to imagine it like an averaging, smoothing function which operates on the sample values as a sliding window. But that's not really true - it's not a straight rolling average, instead the sinc function (sin x / x) is used to scale the sample values with respect to time.

The way FIR filters work assumes a finite amount of samples (otherwise it would be technically impossible to know all past and future samples), so instead you pick a time period over which to calculate the output. Since there are always past and future samples included, this leads to a delayed output. Eg for a 100 sample filter window, you'd need to first have 100 samples in order to calculate the first output sample. This is intuitive, since a filter is sort of a smoothing, averaging function.

DSP is mainly mathematics, and so there's not much substitute for understanding someof the theory if you actually want to understand what's going on. It may look intimidating with all the calculus but it's not actually that bad if you just want to understand how the mathematical description of a filter relates to the implementation in terms of the multiplications and additions that are happent.

Ken Shirriff, naturally, has tackled the subject a while back in a multipart analysis of the Yamaha DX7.

https://www.righto.com/search/label/dx7

A couple of years ago, I took a "Music and Computers" course at my undergraduate computer science program. I had known the professor for a while, and he had some incredible projects under his belt. He had built a full fledged Hammond-like MIDI synth using an Arduino Due, all the way from writing the code, and building the stainless steel casing. I took that as an inspiration for my final project, and tried to build a FM synth using ESP32.

I based my project on the Yamaha DX7, which is an amazing feat of musical engineering. I had to write the code, wire together a MIDI Input, and an audio output (using a DAC). It was one of the coolest projects I've ever made, although I did not get too far.

At the core, all you're doing is generating sine waves. Digital Synths in the 80s would not have the computation power to actually compute sine waves, so what you have to do is precompute the values of a sine wave, and store it in memory [0]. Also, you don't get floats, so it's all fixed point arithmetic [1]. FM Synthesis is essentially a carrier wave and a modulator wave [2]. The DX7 goes beyond that with the concept of Operators, which can be mixed and matched in an Algorithm [3].

I managed to make it produce notes, which were in tune. I started making the envelope (attack, decay, sustain and release), but that's where I got stuck and eventually abandoned it.

[0] https://github.com/fercgomes/churrosfm/blob/master/src/Table...

[1] https://github.com/fercgomes/churrosfm/blob/master/src/Oscil...

[2] https://github.com/fercgomes/churrosfm/blob/master/include/A...

[3] https://djjondent.blogspot.com/2019/10/yamaha-dx7-algorithms...

I've written a software emulation of a Yamaha YM2413 (OPLL) for use in a Sega Master System emulator, although that was a couple of years ago. What I remember is a table for the sine wave (1/4 of the full cycle, since the rest can be done by mirroring and inversion), but stored in some logarithmic format so that multiplications can be done through addition, and then a lookup table to convert the output value to an actual integer.

It was cool because there were set values that you could subtract to cut the volume in half, for example. It makes the envelopes into just adding and subtracting values, and getting a cool volume curve out of it.

I remember that I got it reliably producing recognizable output for most register settings, but there are some parameters that I must've gotten wrong, because some settings sound way off from how they do on real hardware.

Site requires a whole bunch of hosts (including Google and Facebook) to be unblocked before any audio plays.

Even worse for me:

Your browser is out of date! Update your browser to view this website correctly. More Information.

and constantly reloads in hope that my browser does update somehow :D

FM is similar to LFO (Low-frequency oscillation) modulation, but at higher frequencies.

With LFO you hear how modulated value changes in time, but with FM, modulation is so fast, that you don't perceive changes as time based changes anymore, but as a different timbre.

FM synth programming can feel like a dark art for starters. It's much less straightforward to dial in a specific sound compared to, say, subtractive synths.

Now imagine a synth that reverse-engineers audio sounds you feed it, adjusting its parameters to match. This would be a game-changer for learning. This recently released FM synth is trying to do just that with AI : https://www.youtube.com/watch?v=3gTYlYG6-R8

That's been around for at least 30 years[1] although there is a strange lack of tools using it, I looked into it a while ago. There's a tool for generating OPN2 patches[2] (sound chip used in Sega Mega Drive) which is what I'm most interested in this technology for but I couldn't get it running right. And a modern method that uses neural networks instead of genetic algorithms but I haven't found any applications of it [3].

[1] https://www.jstor.org/stable/3680541

[2] https://github.com/jpcima/fmprog

[3] http://kth.diva-portal.org/smash/get/diva2:1334639/FULLTEXT0...

p.s. does anyone know why do so many papers on new technologies like this never even provide code samples? what's the point?

Using genetic algorithms or other evolutionary computation methods for this purpose is pretty old. There are two versions. First, there is evolutionary resynthesis, where the system is trying to optimize a synth program to match an existing sound. This can be done automatically: try a bunch of sounds, have some automatic comparator compare them to the target sound, assess them, mix-and-match, try again. Then there is interactive evolution, where the system is trying to come up with sounds interesting to the musician. Here the system proposes and auditions synth patches, and the musician responds with what he thinks are the better ones, and using that information the system mixes and matches and tries again.

Evolutionary resynthesis has been around since at least 1996 (Horner). Aphex Twin was enamored with it for a while. I don't think it's particularly interesting: if you already have a target sound, why are you trying to come up with a program to match it? Why not use the sound itself? But interactive evolution is both useful, more difficult, and interesting, as it makes it possible for the musician to explore the space of synth programs without having to program them (and that's particularly good for FM, since FM is EXTREMELY counterintuitive to program).

I wrote what I think is the best general-purpose interactive evolution style synth programming system right now: Edisyn. It works with a wide range of synthesizers and synthesis types. If you restrict yourself to exactly one synthesizer of your own design, you can get better results still, and there are a number of softsynths out there which do this.

But most interestingly: interactive evolution has a specific inventor: BRIAN ENO! He invented it around 1995 in a letter he wrote to Stewart Brand.

> if you already have a target sound, why are you trying to come up with a program to match it?

Once you have the program matching the sound, you can tweak it and get variations. At least that's what I imagine people would want it for.

>if you already have a target sound, why are you trying to come up with a program to match it

Because then you have it (or something close) in your own synthesizer, as opposed to whatever other synth or natural source you sampled it from, and you can enhance it, adjust it, use filters and others on it, and so on using all its native cabilities.

Sure. But you can manipulate the sample in complex ways with filters, effects, etc. on your sampler or rompler already. I'm not saying there's no reason -- I'm just saying it's a fairly narrow and not hugely compelling one.

>But you can manipulate the sample in complex ways with filters, effects, etc. on your sampler or rompler already.

Much much less.

If you say so. I own an UltraProteus with almost 500 filters and an extensive mod matrix. Pretty sure it can mangle a sample quite a lot.

>If you say so.

Nope, it's objective reality.

>I own an UltraProteus with almost 500 filters and an extensive mod matrix. Pretty sure it can mangle a sample quite a lot.

Which is neither here, nor there. I didn't say you can't change the sample "quite a lot", I said you can change it much less than if it was as a synthesis patch on a synth.

Any fx or filter available to the sample is also available to the synth. The reverse is not true.

As a simple example for an extremely basic thing you can't do: you can't change the sample's inherent attack. The ASDR envelope on the sampler works on top of the raw sample's attack, so you can't, for example, change a slow attack to a fast one.

Fiddling around with FM, I've found it easy to dial in interesting sounds but very difficult to find sounds that are "musical" so it's difficult to actually incorporate them into a piece of music. I guess the wild success of the built-in patches of something like the DX7 proves that this is down to my own lack of skill but FM synthesis seems somewhat chaotic - a small adjustment to one parameter can cause a wild swing in the sound.

Try to use modulation frequencies that are in some sort of integer ratio to the carrier wave; the lower the ratio the less tinny/fizzy the sound will be. Non-integer ratios are enharmonic, good for percussion and SFX but also quickly tailing off into noise.

Underrated comment. This is one of the most inportant parts of FM, imo. Each integer ratio of carrier to modulator has a specific flavor. The "simpler" the ratio the less complex and flavorful the sound.

Therefore, fading from irrational to rational ratios can make a crazy, inharmonic sound "stabilize."

After working for years with FM, i have found a somewhat narrow range of sounds and the layering becomes more inportant than the algorithm. So getting to know of the ratios is important, 1:2, 1:3, 1:4, etc.

As a synth head that teaches synthesizers at university: While I think that is an interesting idea in itself, it is probably not the best idea for learning how FM synthesis works. Learning synths is both about gaining an intution what sounds emerge when you change a set of parameters, as well as having a mental model of the signal flow and how that shapes the sounds.

Both are things that are not helped by having something that creates presets for you — in fact it might even be more productive to look at human made presets if you need guidence.

Also: Synthesizers are also good for creating never before heard, totally out there sounds, and those are hard to reach when you feed it existing stuff. My own personal learning approach always was to use a simple init preset and then create everything manually from there. I called this my "No-presets rule" (meaning I don't use presets that I did not make myswlf for anything, unless ot is a commercial gig). If you then make your 10th preset from scratch you probably understood something already. And then at some point looking at other people's stuff starts to make sense because now you have a set of eyeballs that can actually peeceive the things they did.

Re: your experience teaching synths in university, do you feel there is certain knowledge/concepts that are difficult for those self-teaching sound design?

I do the same "no presets rule", with most being the result of random experimentation and fiddling of parameters. This does a good job at "creating never before heard, totally out there sounds", but I imagine it's trickier the more specific your end goal is.

I think the hardest part is getting the basics straight, and by that I mean things like understanding oscillation produces sound, amplitude influences the spund level while the speed of the oscillation influences the pitch. Understanding that by modulating pitch and amplitude you can already create a ton.

The biggest challenge for students in my experience is grasping all these obscure words and acronyms, with VCOs, LFOs, keytracking, Envelopes, ADSR, Triggers, Gates, CVs, oscillations, overtones, frequency graphs, filter cutoff points, resonance, ...

It is just a lot of concepts at once. Sometimes it can pay of to take a step back and limit one's arsenal and figure out how far you can get only using one or two of those.

Totally. I'm self-taught in this area and I feel like the biggest obstacle for self-study is that the basic concepts are often passed over too quickly (like simply the idea of using phase/the unit circle to represent oscillations, and how all that relates to frequency and period). I recently sat down and tried to make sure I really understood the mathematical modeling part of the whole business and it brings a lot of clarity when tackling more advanced topics. This is kind of true for self-studying any mathematical domain generally. It's easy to pass over the "entry-level" or foundational stuff quickly because on the surface it seems straightforward, but getting those fundamental ideas crystal clear and burned into your brain is absolutely crucial if you're going to comprehend anything that follows.

This is exactly what I think. The basics need to be understood. That doesn't necessarily mean they need to be understood first tho. People that learn synthesizers are first and foremost interested in creating cool sounds, so the best strategy to teach them the (for some: boring) basics, is IMO to repeatedly overwhelm them a little and then step back and explain things throughly.

Another approach, rather than generating randomized offspring, differentiable dsp provides a mechanism to self-correct using a loss function

https://deepai.org/publication/ddx7-differentiable-fm-synthe...

I disagree, given one of my favorite synths FM8, which in particular I've always found easy given its very thoughtful interface mechanisms of an "easy mode" and a "hard mode" both of which have extremely powerful, precise envelopes which can be dialed in really cleanly to visually carve out any kind of sound one is looking for. It's easy to grab a preset, customize to taste, and experiment with it to gain your footing and intuition around how it works.

I also disagree that a synth which "reverse-engineers" the audio sounds you feed it is the right way to go. By doing this rather than learning how to do it from scratch (albeit with training wheels as I've previously described), you rob yourself of the rich experience of learning how to use FM synthesis as applied physical modeling.

Modulate one oscillator with another

The end.

Can vibrato be considered FM synthesis?

Edit: aaah, should have read further into the article

Reply on news.ycombinator.com