I wouldn't wear that weighted vest too much. If you wear it all day, it will have the same adverse effect on your joints as being 20 lbs overweight.
It's fine to use it to add work to body-weight exercises such as pullups, squats, etc. but don't wear it for hours just walking around.