You as a "seller" get paid in bitcoin per mhash/s (or whatever the unit is for that algorithm). so if you are hashing "DaggerHashimoto" at 26Mhash/s for 24 hours, you are getting paid x btc per mhash/s
The "buyers" are basically then paying NiceHash to say "I want x ghash/s of this algorithm and I want it going toward this mining pool and I'll pay you X bitcoin for it." So then NiceHash plays the part of the broker and matches that X bitcoin payment up with as many miners as is needed to reach the threshold of ghash/s and takes a little off the top as a fee.
It's not perfect, you may end up taking a sub-optimal algorithm and making less than you could have if you did it perfectly, but it works well enough that I make profit in bitcoin.