but 'building the underlying infrastructure that tons of people use' is not science. in my department we had to fail a phd student because 90% of his work was just implementing bunch of existing methods as a python library. useful, yes; science, no. wasn't his fault, had a shitty supervisor, but making useful tools is not the same as undertaking scientific research.
What could be useful is openness in used tools and software and a way of getting citation counts for software used. It's nothing more than a table. That way the hotness of publication could start to flow for the underlying tools.
no. scientific research is proposing a useful model of an observable phenomenon. this is what you train for during a phd, at least in natural/life sciences: you learn how to test a hypothesis, not an easy skill.
refactoring code or transforming bunch of C++ into a python library is useful, but it's not science.
> What could be useful is openness in used tools and software and a way of getting citation counts for software used. It's nothing more than a table. That way the hotness of publication could start to flow for the underlying tools.
agreed 100%
If you write code that allows science to be done that couldn't be done otherwise then that is science. As a high profile example, a large amount of specialist software was developed for the LHC to allow it to process all the events coming from the detectors.
It sounds like the refactoring here was not really that useful in the first place.
doing a phd -> training to be a scientist.
I worked on software development tools used directly for LHC as part of an internship.
That experience was of zero use when I tried to apply for a PhD later. It did get me several $BIGN internships though.
Make what you want of this story.
For me, our discussion is mainly in where to draw the line around "the process of science". The chair, laptop and coffee machines aren't science. The statistical methods, papers and engineering are. You seem to cut parts of the engineering out, namely the non-novel parts. There's a lot to say for that. But a PhD is proof of apprenticeship as well. I wouldn't grant someone a PhD if all of his work is 'mere retooling'. But in a mainly research papers based PhD-application I wouldn't feel some retooling couldn't be allowed. One could demonstrate scientific craftsmanship in retooling.
if X is e.g. microbiology then it's fair to ask whether (1) some python library proposes something in terms of microbiology, and (2) bunch of biologists should make that decision.
this is why refactoring code is mostly dismissed as 'doing science' by most phd supervisors. sure counts as 'developing skills', which certainly should feature prominently as part of your training, but it cannot be all there is to a project.
However, there's increasingly a role for folks focused more on the scientific computing and methods side. E.g. "how do we constrain X parameters given Y observations" (yes, I just described inverse theory -- that's deliberate). The science isn't solving the problem, it's figuring out what models to use and what the inverted parameters mean. However, solving the problem correctly requires a lot of rather novel work and is very easy to get wrong.
It's similar to many other research staff positions. It's standard to include the person who operated/designed/etc the instrument you're using as an author on papers. Is it that crazy to include the person who developed the numerical methods and implemented the solution as well? For example, I have quite a few friends that stayed on as staff to run the lab or key pieces of equipment. They have tons of "middle author" publications as a result.
However, numerical methods and computing infrastructure and work is much less frequently recognized. This is a step towards changing that.
no-one is proposing that numpy isn't useful or people developing / maintaining tools aren't doing gods work. they have my endless gratitude and try to donate regularly.
however, phd training in my field -- natural/life sciences -- has a specific remit: you learn how to build and test a hypothesis, from start to end. optimising libraries is emphatically not it. as a scientist you should care whether you have a useful model that explains something about the world. this is orthogonal to how neatly you have implemented your linear algebra in python.