The comment under “Future Directions” at the end of that paper would seem to apply to the use of AI in software development, business management, and other areas as well:
“For AI[-assisted mathematics] research, improving research judgement and research-state representation may require training and evaluation on records of mathematics as a process, including failed approaches, strategic decisions, and evolving assessments of evidence. For mathematical practice, while these limitations persist, human expertise is likely to remain most valuable in these global functionalities: deciding when to persist or reframe a direction, and maintaining an accurate representation of accumulated progress.”