I'm active in the security and privacy field, and some conferences have started "artifact reviews" [1][2], where authors can optionally submit the data and code that go along with their work.

It's definitely an improvement, but the devil is in the details. Should these be mandatory? If they're mandatory, then what quality standards should artifacts meet? Who's ensuring those standards are followed? How should proprietary code and sensitive human-subjects research data be handled? There's also the question of what code should be made public: Is it just the analysis that produced the metrics on the paper (e.g., R scripts, Jupyter notebooks, etc.)? Or should it also include the data collection and pre-processing? How about the code that didn't produce immediately publishable results, but could be useful for future work?

[1] https://petsymposium.org/artifacts.php

[2] https://www.usenix.org/conference/usenixsecurity21/call-for-...