How much money is "dip my toes"? If you're talking less than a hundred bucks for some cool toys, you can buy Korg Volca and one of PO-x0 series from Teenage Engineering.
If you're thinking $500+ you can get something like Alesis Micron (designed by Bret Victor, BTW) or Korg Electribe, or Elektron Model:Sample. If you're fine with real-time recording, you can instead get something like Behringer Neutron or Kors Minilogue. You can make entire tracks on those even if all you have is a linear recorder.
If you're in ~$1000+ range you can buy Analog Four or OP-1. The former is a very practical machine, the latter is a marvel of design, very fun and great for experiments.
Or you could go for a workstation. FA-06 has pretty good UX, although it's closer to DAW-on-a-keyboard than other things I mentioned here.
Find a device you like and learn it inside out. It's a very rewarding experience. Also, try to buy a used device. You can save tons of money this way.