I highly doubt chatgpt is even remotely near being a lawyer. Lawyering is all about logical argument from a specific context and nitpicky details, which chatgpt is known to be bad at.
Anyways it probably should be illegal if someone is selling "legal services" but not doing it properly for the same reason that if a dr just did what chat-gpt said, that should also be illegal.