That's still concerning. If your mental model of a scale is via the notes, the piano has helpful colour coding. Not being able to use that implies that the scales have instead been memorised by how they're played - frets, valves, etc. - that will stop working when a more in-depth application of keys is needed.