If you want to go for a PhD, I suggest reading Sutton's book on RL (at least the important parts for what you want to do), followed by recent RL papers and RL papers in robotics that pick your interest, understand them fully. Aim for journal papers and major conferences only (high impact factor is a first metric that comes to mind).
The main bit is to try to formulate an important research question you're interested in which is not already explored in the vast litterature and plan which experiments you need to carry and what do you need to change if some of them fail.
If you want to engage in a PhD, an important and well studied research question will increase your chances of getting accepted in a PhD program.
The main advantage in doing a PhD in a uni is that you can share and learn knowledge from other students and advisors, as well as have insights in what are ideas worth pursuing and what are small improvement that no one care about.
At uni you usually work on the kind of research meant to improve the knowledge and less on the general engineering problems. Engineering problems are byproduct of the knowledge you are building. A lot of uni work very closely with industries in practical fields like robotics, you'll have plenty of opportunities to apply what you are building directly on engineering problems. But that will not be the main focus.
In my experience it is considered less 'valuable research' to try to apply some SoTA to a specific engineering problem in academy, and usually proposals of this kind are rejected.
Hope this is helpful :)