I only focus on index data so it's a bit easier but it's still hard. Lots of scraping, downloading and parsing CSVs, Excel sheets, some puppeteer magic...
Some sources that I use:
- MSCI data (https://www.msci.com/end-of-day-data-search)
- Financial Times website (e.g. https://markets.ft.com/data/indices/tearsheet/summary?s=SP5M...)
- Investing.com (e.g. https://www.investing.com/indices/us-spx-500)
- STOXX
- DAX
- FTSE for FTSE stock indexes (https://research.ftserussell.com/analytics/indexviewer/chart...)
- Yieldbook.com for FTSE bond indxes (https://www.yieldbook.com/m/home/index.shtml) - ICE website for ICE bond indexes
- ...
I'm constantly adding new indexes and sources (see https://backtest.curvo.eu/updates) but it's tough!
Some examples of what Backtest does:
- MSCI World: https://backtest.curvo.eu/portfolio/NoIgsgygwgkgBAdQPYCcA2AT...
- My portfolio: https://backtest.curvo.eu/portfolio/NoIgmg9gTghgdgcgM4AIAK0A...
If the author has any questions, just reach out to me! Happy to share my experiences.