I'm curious about your workflow for reusable scripts with Jenkins. What are the advantages to putting it in Jenkins? How do you set up Jenkins to run these individual scripts?
But this isn’t good, we have Airflow and that’s where these things should be. Their stay inside Jenkins is technical debt, but at least is much better than not having it automated. The advantage is that you made an ad-hoc process less unstructured, you can add post-failure alarms, you don’t have to worry about long running scripts being interrupted by shabby notebooks, you don’t need to spread production credentials to every developer and so on.