Yes, they should be discoverable. There are platform conventions for this: on Windows you'll see underlined letters; menus show you the shortcuts. Tab is a pretty much universal "move to next field" shortcut and I don't think that's particularly discoverable if you don't already know it.

There's nothing wrong with a keyboard driven UI having buttons as long as they advertise their key, though. That's one way discoverability is supposed to work. It means you can fall back to the mouse when it makes sense.