Pure-Python symbolic regression that rediscovered Kepler's law from 8 data point
github.com
github.com
capacity_SOH ≈ 0.913 − 0.352 · tanh( cycle^((temperature/cycle)^0.485) )
I understand this fits the data, but exponents should be dimensionless, what is temperature/cycle?
In this example temperature would be a magnitude, not a unitful value.
At least in the ISO 8000whatever convention where a value is the product of a unit and a magnitude like most people are use to.
Here is a Terry Tao post with more information[0] on why the convention is there, but as he mentions, in differential geometry and Clifford/Geometric Algebra you do things like add vectors to scalers all the time.
[0] https://terrytao.wordpress.com/2012/12/29/a-mathematical-for...
Physics uses the properties of unitful objects to help avoid errors, it is a convention and not (by itself) a fundamental truth.
If your world is physically realizable one, the benefits make this choice a no brainier, but it is still a convention with some convenient outcomes.
A unitful quantity is actually a tuple, specifically (value = quantity x unit)
In computer science and programming, assuming a priori that individual scalers are only useful in the context of units, or that they will behave in the same way can cause you real issues.
Be careful with conventions and assumptions, use them where they serve you well and avoid them where they block you from finding useful mechanical means to solve problems.
Enforcing dimensional consistency during the search is a known extension (AI Feynman does a version of it) and honestly it's a good roadmap item, a dimensionally sound form would be more trustworthy.
When the temperature difference is small, everithing is linear and you get nice formulas.
When the difference of temperature is big and you have liquids with convection and turbulence, you only get empirical formulas with weird exponents. The correct method would be to give the constants with the correct units, but it's usual to specify unit to measure the data instead, and just enter the numbers in the formula.
After a short search in Google, I got this example https://www.researchgate.net/figure/Equations-used-to-obtain...
And your point about units is the industrial reality: the convention "measure in these units, then use the numbers" is exactly what the tool does internally. A proper dimensional mode stays on my roadmap, but I feel less bad about it now.
What happens if you give the system not only the semi-mayor axis but also the semi-minor axis?
Have you tried with only the 6 planets Kepler know? (I don't expect this to change the result too much.)
Have you tired with noisy data?
to get a new paragraph here.
> b = a·sqrt(1−e²), so corr(a,b) = 1.00000 to 5 decimals
Isn't e different for each planet?
> Mercury is the only planet where they differ by more than 2%
I remember something about Mars been the planet with the most eccentric elipse
> *So fit quality degrades gracefully, symbolic recovery doesn't. Making that part noise-robust is pretty much the open frontier of the whole field, not just of my tool.
Nice. It's a hard problem. Which heuristic are you using to pick the "best" formula?
Isn't e different for each planet?
Yes, each planet got its own e (Mercury 0.206, Venus 0.007, Earth 0.017...). The correlation still comes out at 1.00000 because all eccentricities are small while a spans 0.39 to 30 AU — a ≤2% per-planet wobble is invisible to Pearson across two decades of range. That's exactly what makes it a nasty decoy: almost collinear, but not quite.
Mars been the planet with the most eccentric ellipse
Close — Mercury is actually the most eccentric (0.206), Mars is second (0.093). Funny enough, Mars is the famous one precisely because Kepler derived his laws fighting with Tycho's Mars data: its ellipse was just eccentric enough to kill every circular fit. If Tycho had handed him Venus data instead, we might have waited a while longer.
Which heuristic are you using to pick the "best" formula?
Hold-out validation during evolution, then the old CART-style one-standard-error rule: pick the smallest formula whose validation error is within ~1 SE of the best one. Constants get refit with Levenberg-Marquardt before that comparison, so small forms compete at their best. It also returns the full accuracy-vs-size Pareto front so you can override the choice. And to connect it to your noise question: under noise that 1-SE band is exactly where wrong-but-simpler formulas sneak in and tie the true one — the two problems are really the same problem.
For simple problems as Kepler's law, a quick detour on Desmos will show a perfect fit for power law instantly. In general, there are many important criteria for a better curve fitting (for ex. independent, normal distributed residuals), not just R, so I hope the author has/will incorporate them into the search to create a more robust result.
And agreed on residuals, R2 alone is weak. Hold-out validation is already in there, proper residual diagnostics are a fair ask for the roadmap.
GP_ELITE is a symbolic regression engine in pure Python: given (X, y) data, it searches for a readable mathematical formula linking them, instead of a black-box model.
To show what that means concretely: I gave it nothing but the 8 planets' distance from the Sun and orbital period — 8 data points — and asked for a formula. It returned:
T = a · sqrt(a) (i.e. a^1.5), R² = 1.000000
That's Kepler's Third Law (T² ∝ a³), which took Kepler ~10 years to find in 1618. GP_ELITE found it in ~3 seconds. Reproducible: examples/kepler_demo.py.
v0.2.0 (this week) added the parts that make it reliable: Levenberg-Marquardt constant fitting (constants come back at machine precision — Coulomb's q1·q2/(4πεr²) is recovered exactly), multi-restart with a merged candidate archive, a Pareto front output (the full complexity ↔ accuracy staircase, not just one champion), and a guarded forecasting mode for extrapolating trends beyond your data without the usual GP blow-ups.
Pure Python/NumPy — pip install gp-elite, no compiler, no Julia.
Target Audience
Anyone with small experimental datasets (≤10 variables, 100–5000 points) who wants to understand a relationship, not just predict it: lab engineers, scientists, students. One concrete use case that drove development: battery degradation (SOH) forecasting — the guarded mode gives you an honest bracket of scenarios (a Pareto front from a conservative straight line to richer bounded laws) instead of one overconfident curve. Production-usable for that niche (built-in hold-out validation, regression-tested); not aimed at large-scale ML.
Comparison
vs gplearn (the established pure-Python option): I ran both on the same frozen benchmark — 15 Feynman physics equations, identical data and splits, generous budget for gplearn. Exact symbolic recovery (machine precision): GP_ELITE 10/15 (67%) vs gplearn 6/15 (40%). gplearn recovers the constant-free formulas and stalls as soon as a ½ or a 4π appears (no real constant optimization); LM fitting is what closes that gap. Every number is reproducible: PYTHONHASHSEED=0 python benchmarks/feynman_bench.py 0 15 and benchmarks/duel.py in the repo.
vs PySR / Operon (the state of the art): they are stronger on speed and scale, and I'm not claiming otherwise — but they require a Julia or C++ toolchain. GP_ELITE's whole point is zero barrier: pip install and go.
vs neural nets / gradient boosting: those win on raw accuracy for large data, but give you a black box — GP_ELITE gives you the actual equation.
Honest limits: weak on chaotic targets (tested on Collatz), degrades past ~6 variables with decoy features, and pure Python costs wall-time on big data.
Code (MIT): https://github.com/ariel95500-create/gp-elite
There are infinite number of curves that agree on those 8 points and deviate from Kepler 's law everywhere else. On such 'trajectories' this algorithm would have performed badly.
I do see a great value in conjecturing possible solutions that needs to be verified with domain specific knowledge.
Recall that Ptolemaic epicycles were a great fit, in fact a better fit than Copernicus's heliocentric model. This makes me wary of deep NNs in Physics.
For the epicycles, for me this is exactly the argument for parsimony pressure. Epicycles fit better because you can always add one more circle, same as adding parameters in a NN. They lose on description length, and that is the only defense I have too. And it is not perfect: on noisy data my tool produces its own epicycles, formulas that fit very well and mean nothing. One battery model it gave me predicted the battery heals itself after cycle 264. Great fit, zero physics. So yes, a conjecture machine that a domain expert must verify. No more than that.
There is some misconception in the wild about epicycles models that need not be shared by you specifically. There weren't that many epicycles per orbiting body, but every orbiting body had a few that had to be 'trained' specifically for them.
My fear, and I suspect yours too is that good curve fits done one at a time with mathematical models that are universal approximators (*) rarely, if at all lead to causally explanatory models. In Physics it's the latter that we seek.
(*) Epicycloidal models are a form of Fourier analysis and are a class of universal approximators for periodic trajectories.
And I think we agree on the real problem: a good fit on one problem, made with a universal approximator, is not an explanation. It can be just a compressed description, maybe with a cause behind, maybe not. I cannot fix this, but I added this week a small thing to at least see it: a bootstrap stability check. You resample the data, refit, and look how often the same form comes back. High frequency does not prove anything causal. But low frequency is a good signal that you are only looking at a Fourier-type fit, not a law. The decision stays with the user, the tool only makes it visible.
> That's Kepler's Third Law (T² ∝ a³), which took Kepler ~10 years to find in 1618. GP_ELITE found it in ~3 seconds.
In future post in HN I recommend to avoid that kind of comparisons, because we all know he did that on paper and didn't even have a good set of functions to use a brute force aproach. It feels like unnecessary linkbait and make people write unfriendly comments. Probably it's better:
fake quote> That's Kepler's Third Law (T² ∝ a³), GP_ELITE found it in ~3 seconds.
I couldn't have said it better.