It's not if it's properly structured. That usually means employee having some kind of legal entity (consulting firm) and having proper B2B contract.
If you are treating the contractor as an employee, they are a de facto employee and you should be withholding income tax, paying payroll tax etc..