1. Buy a cocktail book. I got one ages ago, so I'm not sure what I'd recommend today.
2. Pick one cocktail that you like and get the ingredients for it. Master it.
3. Slowly branch out to nearby cocktails that only vary by an ingredient or two.
4. Keep exploring that way.
Like other culinary stuff, cocktails tend to be organized into a large web of related drinks. So once you pick a starting point, you can just incrementally explore the graph without having to buy a pile of ingredients.
Eventually you will amass a pile of ingredients. But, fortunately, most last a very long time.
As far as a party goes, it depends a lot on your guests. In particular, their age. Cocktails go through fashion cycles like everything else and younger guests are more dialed in to those so will probably prefer whatever cocktails are big right now. I'm too old to know what that is but I know mules got super popular for a while.
Older guests will generally have a standby or two that they know they like.
If you just want to ensure everyone has a good time, I'd get sufficient beer and wine and then pick a popular cocktail or two. Anyone who doesn't like those particular cocktails can simply default to beer or wine.
If you want to have a "cocktail party" where everyone is encouraged to focus on those, then good coverage would be:
1. Something sweet and fruity for those who don't like a strong liquor taste. Cosmopolitans are good for this. Tropical drinks like mai tais can be good too, though rum might be too strong for some.
2. Something long for people who want a lighter, larger drink. "Long" means mixed with a carbonated beverage. This could be as simple as a Jack and coke or fancier like a Tom Collins. Gin and tonics are great.
3. Something whiskey-based. People who like whiskey really like whiskey. Old fashioneds work well or manhattans if you're a little more adventurous.
Talking to bartenders when you're at a bar is always great way to learn more. Just try not to bug them if they're clearly busy attending to patrons.