I'm worried about how you would verify the legality of the client's business, or the client's identity. I've only talked to this client over the phone. I've googled his name, but found nothing.
His LinkedIn profile also has little info.
It's not like everyone is googlable or should be. I'd say being able to google somebody's name and get tons of results is the exception rather than the rule (and thank $DEITY for that).
Your concern for getting direct access to his website is valid though. If the customer is not using any VCS I'd suggest to him that he starts doing so - there are plenty of options.
If the client can provide you access to a testing/dev environment, where you can safely play with things without breaking production, I'd pick that option as well.
My parents are new to the idea of freelancing, and they're worried that the site might somehow be a disguised site for selling illegal products.
My parents might be a little paranoid.