A guide to private alpha testing

I'd be curious to learn how others approach private alpha (how long to stay in it, who to include, what to test, how to think about feature additions/pivots, etc...)