As mentioned, Shift-Down will do. (Also, while holding an item, Shift will use if the item is usable, throw otherwise. Usable items can be put down with Shift-Down as well).

These controls are actually explained in the LD levels (door on the right in the very first screen of the game), which you're encouraged to play first!

Still early access, thanks for bearing with us.

Shift+Down is how you pick up items. It's a little finicky though. I think it could stand to be more forgiving on where you're standing to pick them up.