Comment by neltnerb | Hacker News Reader

Comment by neltnerb | Hacker News Reader

This reminds me of an undergraduate course on diffusion (atomic scale movement of atoms in a solid).

There are known formal differential equations that let you solve for the exact diffusion profile given geometry, composition, diffusion rate constants, and temperature (basically). So on the homework they asked us to tell them how long it took for the concentration of the dopant to reach 10% at a 10 micron depth.

So of course we all reached for this new math we had learned.

We all got it wrong with the professor commenting that "you should have just used the approximate formula [which was trivial algebra], we only know the diffusion rate to an order of magnitude anyway". This was far more useful as feedback than the 0.2% of our grades we missed credit for.

So I look at this essay and am a bit amused at minimizing calculation error to such an extent in a mathematical model when your resistors have a 5% tolerance and your capacitor is probably at 20% tolerance if you happened to have it in a university EE lab. But I do appreciate the fun of doing it this way too if you're not in a hurry =)

It is crucial for any systems designer to realize that no matter how well you do your math and theory you have to also understand the sensitivity to variability so that you know what math is worth doing and what components we have no choice but to control the tolerances on better.

You can find some fancy software for generating filter networks (similar in concept to this except with more discrete math because component values are discrete...) and ask it to show you sensitivity and you can see exactly how much you'll screw up your perfectly designed 10th order Chebyshev filter if that last resistor is off by 1%...

I had a professor that intentionally baked hard (for the course takers anyway) mathematics into some problems, that were technically solvable during an exam, but that required very good level (they typically included rare/obscure identities). The point was that solving precisely was always a “trap”.

His pet peeve was that engineering wasn’t about finding a precise answer at some point, but about finding good enough answers fast.

I liked that this forced you to stay alert during the problems for simplifications and to understand what you were doing, instead of just droning out method A for solving problem type B.

If the course is not a mathematical course that requires this knowledge then adding obscure identities is focusing on the wrong thing entirely. Some people will simply not see these identities because they might never have used them. Irrelevant for the course but relevant for the grade, that's not a good combo imo.

I read the parent's comment differently from your reading.

I read the parent's comment to say that there was stuff that was barely possible for the best students to solve precisely in the time given, but that there were obviously-acceptable approximations thereof. And that either was a path to full credit.

The professor's goal was to get the students to realize when to use approximations.

It seems like I misread the original comment indeed.

The point was to realize precisely that some problems might have analytical solutions, but they would take too long. If you’re solving problems on pen and paper nowadays, you’re most likely looking for an approximate answer anyway.

Maybe I should add that, as long as the answer was within a specified margin (say 5%), the answer was considered correct.

If you saw that your calculations started to become way too complex, you had already missed something.

Oh interesting application. I have taken a class in microelectronics physics (doping, electron holes, bandgaps, mosfets, bjts) and it was definitely just focusing on simpler approximations and first and second order effects only.

But you are correct in that after my second or third electronics lab I realized that even standard component values vary to much that this approach is a little overkill and non-practical in a lot of cases. However there is a way to bake in discrete component values into the continuous optimization problems by using a continuous/"soft"/differentiable approximation of the min function to make "pockets" of optimal regions close to component values part of a set (for example e series values). I plan to do more writeups exploring this idea as well as looking at more complex applications such as higher order filters as you mentioned (my op-amp filter labs are what motivate me to look into this as I wasted so many hours in open lab trying to get my component values right).

> However there is a way to bake in discrete component values into the continuous optimization problems by using a continuous/"soft"/differentiable approximation of the min function to make "pockets" of optimal regions close to component values part of a set (for example e series values).

Is this the same concept as uncertainty sets in a robust optimization problem?

After some quick searching (https://www.princeton.edu/~aaa/Public/Teaching/ORF523/S16/OR...), I don't think so. It looks like for robust optimization problem you don't know for certain what the objective function is or have uncertain data measurements for the objective function which I don't think is the case here. I could be completely wrong though.

From my understating, if you want to only able to only pick form a discrete set of components for certain variables, you are essentially transforming the problem into a Mixed-Integer Non-Linear Optimization problem (MINLP). I tried to find and easy way to do this with some python optimization libraries but they always needed other packages which were hard to install on Windows. So my solution was to "relax" the discrete constraints by using a continuous approximation of the minimum function when looking at the error from the nearness comment value that is possible. This also lets you assign weighting to have a tradeoff between having realistic comment values vs. having lower error for your main objective.

Yeah, at the least you can just sample from a gaussian distribution around the nominal value; the shape is probably wrong but it's got to be better than assuming no error. Or you could just do error analysis on the formula to find out which terms contribute the most to the final error.

I like that your suggestion is in-line with how we now can use an autoencoder to convert a molecular graph into a continuous latent space which can be used to train a network to predict some property. Now we have a latent space where things that have similar property predictions are near each other in a space we can explore and then decode into new candidate materials/molecules.

Drug discovery, solar panels, I'm looking forward to the field taking off industrially for sure. Applying machine learning and advanced computational techniques to ad-tech is very depressing with problems like these out there.

You are of course right that Monte Carlo analysis is a good way to test your circuit designs' sensitivity to error, but before that, you can just compute the Jacobian of your loss function (or loss vector) with respect to the vector of component values. Then you can take the dot product of that Jacobian with a vector of component value variances to approximate the variance of the loss function; it won't be as accurate as Monte Carlo analysis but it's a lot faster. Maybe that's what you meant by "just do error analysis on the formula"?

Reply on news.ycombinator.com