There have been studies where mice were fed a diet of various natural and non-caloric sweeteners. The mice were monitored for blood sugar levels and cholesterol as well as other measures. The mice who ate all three of the most popular artificial sweeteners had worse outcomes on several measures. There is some additional evidence that the effect is largely due to the sweeteners' effect on the composition of gut bacteria. I can't tell you if this holds up (or if it holds for humans) but aspartame does not seem to be harmless in these studies.
Popular coverage: https://www.scientificamerican.com/article/artificial-sweete...
2: https://www.businessinsider.com/artificial-sweeteners-aspart...
Link to one actual study: https://www.mdpi.com/1420-3049/23/2/367