Roblox CEO: Human reviewers are key to keeping the metaverse safe

On Roblox you can build things either from assets such as meshes, sounds, images, scripts or “models” that can consist of all of those, say a model of a dog that could contain the 3D mesh, but also the sound of it barking and the script to make it move around.

Using such models creators can easily add impressive interactive elements to their games, but it’s hard to know exactly what is in a complex model. Sometimes they contain little trojans that instead of displaying a dog just take over the screen and say “you’ve been hacked” or display endless dialog boxes.

But the more annoying thing they can do is hide inappropriate assets inside. I was team building a level with someone, and they added a model that had an ocean with a battleship on top of it. What they didn’t know, until exploring the included ocean some more, was that under the sea in that model there was also a penis.

This immediately triggered the moderation magic, and they lost their entire account, even though the level hadn’t even been published yet. It was enough to just momentarily have an inappropriate asset in the editor to trigger the ban.