Would voxels be easier than vertex-based meshes?

I can imagine you'd have the problem of stray floating voxels then, which isn't as noticeable when it happens with 2D pixels.