F3 is common for "Find Next", F5 for "Refresh". I don't want to use two hands for these.
IntelliJ has defined shortcuts for F10, Shift+F10, Ctrl+F10 and Ctrl+Shift+F10. Adding another modifier to all these isn't great.
You're thinking small here. With QMK keyboard firmware, a single key can send shift-f10 or similar. I have a top layer key that acts as ctrl-super and also as a sticky key (I don't have to hold it) because I configured a bunch of key chords in Sway that start with those keys. Mainly focusing specific programs by appid or class. So, I'll hit that key and then b for browser, v for video (mpv), etc. You can even have keys that directly type out parentheses, no shift needed. The default layout for my keyboard had that for both parens and brackets. I found more useful things to put on those keys, though.
I use a Pinky4 which takes some stuff off the outer edges compared to a 60%, but has more on the inner edges, and so I stuck that key somewhere along the upper right edge of the left half. If I were using qwerty, it'd be near t and g keys.
In addition to being able to run QMK on a keyboard, you could also get a macropad, or just a second keyboard (maybe something small like a planck, could go above your main one), and have all sorts of specific binds like this. I would recommend layers instead for anything not used constantly, but a case can be made for both options. I have mouse keys in a separate layer, letting me hold a modifier with my thumb and then use something similar to an arrow key cluster to move my mouse. I even have all 4 scroll directions mapped, which lets me scroll horizontally, unlike my actual mouse.
Software on the computer really affects use-case I think. In qutebrowser (or firefox + tridactyl) I just hit r to refresh a page. I never miss easy access to f5. I can also copy a page's url with yy instead of doing a dance of ctrl-l and then ctrl-c. I can still hit f5 if I have to. Like if using an unusual program or another computer with my keyboard.
It sounds like due to your software choices, you use the function row a lot more than me. There are some larger split keyboards, but it's worth considering changing some of your behavior/software also.
But whatever, you do you :)
I'm also not sure how to use more layers without dedicating more keys to switch between layers?
For more common layer switches, I extensively use dual-use keys. For example, if I tap "F" then I get the letter "f" but if I hold it, it switches to a vim-like layer. HJKL becomes arrow keys, Y becomes Ctrl+C, etc. The point is that you don't have to dedicate more keys to do layer switching. You can have the same key do different things depending on how it's used.
For me, memorizing has been easy due to incremental change. I didn't come up with my new layout on day one then stick to it forever. I'd find a frequent pain point, tweak the layout, and memorize the tweak. It's easy to remember 2 or 3 key changes at a time.
For shortcuts - I just conciously accepted it takes time to learn. I'd say it was about a month before I stopped reaching for the old shortcuts and started to use the layered shortcuts.
It can also help force you into it by not having the old keys mapped - obviously this comes with judgement on a per key basis. I'm not asking you to unmap "A" - but unmapping/not-mapping the F-keys in their usual place coerces you into using your new mapping and you'll just get the muscle memory eventually.