You can get a shot for refusing to work. This can impact good time and will impact your points. Higher points mean you get sent to a higher security (which ironically means less secure/safe for the inmate) facility.