Hacker News Reader

Comment by moonchrome | Hacker News Reader

I wonder how long before games start being developed for server side rendering.

Stuff like MMO for eg. you dont care about latency as much, your payment model already works with it and it solves several major issues :

* no hardware barrier to entry for high end graphics

* no instalation/play anywhere

* optimizations from shared state rendering, potentially advanced rendering techniques (world space lighting) - you can share computation and memory between multiple clients this way

* probably an order of magnitude harder to cheat

View on HN

>I wonder how long before games start being developed for server side rendering.

A while ago I was working on a flappy bird clone (as a test bed for the technology) that kind of did this. The app ran locally on the device, however it created a text-based record of all objects and their movements on the screen, and at the conclusion of each game, had the ability to ask you if you wanted to upload a video of the game you just played to YouTube. If you selected yes, the small text record of the game was uploaded to the server, where it was used to create a video of the game as you played it and upload it.

The idea was that if it were super-easy and used almost no mobile bandwidth to upload video of game sessions where people liked their results enough to share, it would go viral. The technology worked well in tests, but then flappy bird popularity kind of died before I released it. Now I'm working on implementing it in another game.

This is very reminiscent of two things for me:

- the introduction of replays in Halo 3 (which worked the same way - by saving the data of the entire session, one could freely move the camera around the entire map and observe any part of the game at any point in time)

- Super Meat Boy's level-end combined replays (which replayed all of the user's attempts simultaneously, creating a pretty amusing sort of "heatmap" effect)

I think this will eventually become standard for games where replays would be valuable or fun to watch. But I'm not sure about actually rendering live games server-side until we're at a point where input latency is unnoticeable.

Awesome anecdotes. Reminds me of map making and frag videos in Quake, Unreal Tournament, etc. Super Meat Boy's replay feature was a blast through and through. You have good taste!

Server side rendering for live games is a really bad idea. If there's a single hickup with a packet, the whole game will stutter on the client's side and they won't be able to do anything, not move, not look, not pause, whatever.

Server side rendering for replayed games on the other hand is totally fine, not everyone has capture cards. They actually had this service on Halo 3 where you could pay bungie for the number of minutes you wanted footage rendered out at.

nod nod Replays used to be a standard feature of big-budget multiplayer PC video games. I spent many fond hours dissecting exactly how new-to-Descent2 or new-to-Starcraft me got his ass handed to him this time around.

It's a damn shame that more video dev houses don't spend time to create -at a minimum- single-POV client-side replays for their multiplayer games.

Very creative idea. A similar technique is used by a few games for high score validation. The server runs the player inputs through a local simulation of the game to re-calculate the score, thus preventing blatant cheating. TAS is still possible though.

I think TIS-100 and other Zachtronics games do that as well for score validation.

> it created a text-based record of all objects and their movements on the screen,

Also known as what every single ID Engine / GoldSource / Source[1] / Unreal Engine / many more engines do for records and replays, except they do it in a binary format. It's not exactly a novel idea.

https://developer.valvesoftware.com/wiki/DEM_Format

> It's not exactly a novel idea.

True. But:

1) It isn't exactly trivial.

2) Someone who didn't play games in the 1990s, and doesn't like RTS games may very well actually have never played a game with demo recording and playback.

3) Hell, I was unaware that either Source or recent versions of Unreal Engine were capable of demo recording, and I've played the shit out of lots of games that use both engines.

In my case, it was new to me, and I have yet to see this functionality in any mobile game that I have played except for my own.

I do hope you continue to refine your demo playback techniques and bring this functionality to ever more complex games! I also hope that game devs will push hard to take some time to add demo recording and playback to most of the games they build in the future. :D

> I wonder how long before games start being developed for server side rendering.

Haven't you heard about Gaikai (now owned by Sony) that provided cloud-based gaming. https://en.wikipedia.org/wiki/Gaikai

The problem is latency. Now you have input latency from you wireless gamepad to/from your PC/console, the rendering loop CPU/GPU time and then also the latency of your network and the server CPU/GPU time. Good night. If you live in a big hub city near the cloud datacenter you can use it, or if you like slower paced games. But if you like to play Quake (read: very fast paced first person shooter action with 120+fps) forget dreaming about cloud streaming your games now!

Yep, even on my gigabit internet and sitting smack dab in the middle of the USA (well, pretty far South, but centered) latency is still my achilles heel.

I'm pretty sure you would be much better off on either coast. The vast majority of game servers at least tend to be split between LA and the northeast.

Gakai isn't what I'm talking about tho - that's just hosting your gaming PC in the cloud.

Server side rendering would exploit the fact that all of your clients are using the same resources to render the game - so you only need to keep 1 instance of mesh x in memory and you can render it for all clients.

As I said below if you have shared instance worlds like MMO you can do much more sharing, you can have shared effects just like you have a shared simulation for clients - the effects can then be optimized for techniques that might not be viable on consumer hardware - for eg. you might have some sort of world space global illumination technique - like radiance transfer. These techniques aren't usually used because you need a lot of memory + processing power - but you can get server GPUs with >10 GB of memory and more in the future you could have one dedicated GPU that would just compute global illumination and then feed view specific data to each client view rendering thread.

It would completely change the way you render games - right now most effects are done screen space because it's faster (eg. deferred shading) but if you could share world scene state you could get a lot more fancy with world space effects. Also the rendering pipeline would need to be a lot more asynchronous to avoid latency (eg. global illumination could be a separate process that would sync occasionally and not every frame to avoid extra latency in a trade-off for lighting latency).

I think you're overestimating the impact of sharing resources... it wouldn't do much. Lets say you have 10 concurrent users on a machine, that means to keep everyone at 60 fps you need to be rendering at 600+fps. Even with a great graphics card running SLI etc/etc, having ~1.6ms budget you're not going to do much in terms of visual effects.

It's ramping up:

Square Enix's solution https://www.shinra.com/us

Well that's awesome

Darn tootin!

A fun little fact: the name 'Shinra' is a reference to the Shinra Corporation from Final Fantasy VII, one of Square-Enix's most beloved games.

Yes this seems what like the thing I'm talking about - true cloud rendering - not just sticking PC games on to VMs.

Most important: Significantly less cheating! This is huge!

No more client-side hacks (to the .exe) that add draw calls for things like - draw a red rectangle around enemy units, or force draw them blended so that they can be seen, or turn wireframe on, make all significant audio highest volume (so you can hear steps easier), and remove anything else, etc., etc.

It's unlikely for most games, even MMOs. 100ms of input lag (which is a good case) is going to be noticeable to most players. You can always add more bandwidth, but latency is bound by the speed of light.

Economically it'd be a disaster also. Since many games max out resource usage on the machine they're on, you're not going to have a lot of shared hosting. So now you're talking about having an (expensive) machine for each concurrent user.

>You can always add more bandwidth, but latency is bound by the speed of light.

You're always bound by the same latency - your input goes to the server and server returns world state for your PC to render - the only extra latency in this scenario is time required to encode-decode the data which could be offset if the server is faster at rendering than the client PC.

As for economics I already said you can exploit the shared state very much in games if you rework the way rendering works. Right now games focus on camera space rendering because they only care about 1 view output and view space effects are cheapest in this scenario.

If you have shared state rendering suddenly you can process the instance state once per frame for 100s of users just like you simulate the game for 100s of users per instance.

You can have specialized hardware setups (ie. multiple high end GPUs with >10GB of ram) doing dedicated tasks like recomputing lighting/shadows, animation, particle effects, etc. you would need to find a way to make these systems asynchronous to reduce lag so lighting updates might lag a frame or two for eg. behind animation but as usual with rendering there are always clever tricks to fool the eye - and once you have gigabytes of ram at your disposal very different rendering techniques compared to currently used ones become viable for shared state rendering.

Someone already linked a platform that's already doing this - I have no doubt this is the future of VR and gaming at least in some part (maybe you won't be streaming video to the client but some geometric 3D world representation with deltas so that the final rendering can be done client side and you can have low latency rotation for stuff like VR)

> You're always bound by the same latency - your input goes to the server and server returns world state for your PC to render

That's untrue of most games. Most games will accept inputs immediately on the client, and only correct from the server if things get significantly out of sync ("rubber banding"). Dead reckoning is both hard and super important.

> As for economics I already said you can exploit the shared state very much in games if you rework the way rendering works. Right now games focus on camera space rendering because they only care about 1 view output and view space effects are cheapest in this scenario.

The way you say this makes me think you have no idea how rendering works. There is no rendering without a camera. The idea doesn't even make sense.

> recomputing lighting/shadows, animation, particle effects, etc

Particle effects are dependent on graphics card bandwidth and fill rate. No benefit of shared state. Animation is done in vertex shaders. No benefit of shared state. Lighting and shadows are done through GPU buffers. No benefit of shared state.

The only possible system where shared state could (maybe) be useful in the way you describe is some sort of massive ray tracing operation. That wouldn't have anything to do with GPUs, and if you're talking about real time ray tracing the economics of this just got sillier. Now you're basically talking about a super computer cluster.

> I have no doubt this is the future of VR

VR is BY FAR the most sensitive to even minor latency. There's no way you're doing VR over a network.

>That's untrue of most games. Most games will accept inputs immediately on the client, and only correct from the server if things get significantly out of sync ("rubber banding"). Dead reckoning is both hard and super important.

Most games aren't MMOs

>The way you say this makes me think you have no idea how rendering works. There is no rendering without a camera. The idea doesn't even make sense.

What a narrow minded view. There are data structures that can store geometry and lighting information in world space - for example you can have world represented by some sparse voxel data structure and calculate lighting in worlds space - then camera rendering is just raycasting in to the datastructure which is the same for all views. Animation and particles are about updating the world geometry.

>Particle effects are dependent on graphics card bandwidth and fill rate. No benefit of shared state. Animation is done in vertex shaders. No benefit of shared state. Lighting and shadows are done through GPU buffers. No benefit of shared state.

This is because current rendering systems are optimized for client side rendering which is my point. If you discard the notion that the only way to render 3D geometry is using GPU pipeline and triangle rasterization you'll see that there are a lot of possibilities. Unfortunately not a lot of research has been done because rendering 3D polygons fit the constraints we had historically and is really robust, everything is optimized towards it.

>VR is BY FAR the most sensitive to even minor latency. There's no way you're doing VR over a network.

Which is why I said you could stream geometry data updates instead of video - this way your client re-renders to match camera movements but the animation is streamed from the server

> What a narrow minded view. There are data structures that can store geometry and lighting information in world space - for example you can have world represented by some sparse voxel data structure and calculate lighting in worlds space - then camera rendering is just raycasting in to the datastructure which is the same for all views. Animation and particles are about updating the world geometry.

That only works for diffuse lighting. There is more than diffuse lighting. I'm not "narrow minded", I actually work on renderers for a living so I know the actual data structures in use. Things like specular reflections and refractions are entirely dependent on where the viewer is; and calculating lighting information for an entire scene is way less efficient than calculating it for a viewer (see: how deferred renderers work).

> Unfortunately not a lot of research has been done because rendering 3D polygons fit the constraints we had historically and is really robust, everything is optimized towards it.

Huh? There's been decades of research into ray tracing and voxels. Believe me, people have put a lot of thought into how to optimize these things.

>That only works for diffuse lighting.

You can add light ID list to the structure (eg. 1 or 2 ints with byte IDs or w/e) - essentially solving light occlusion problem in world space. You can then do view space specular and you do transparent objects as a separate pass just like you do with deferred.

>Things like specular reflections and refractions are entirely dependent on where the viewer is; and calculating lighting information for an entire scene is way less efficient than calculating it for a viewer (see: how deferred renderers work).

If you're rendering for a single view - my whole point is that if you're rendering for multiple clients then solving lighting world space makes that calculation shared just like gbuffer is an optimization for view space with it's own tradeoffs and workarounds.

>Huh? There's been decades of research into ray tracing and voxels. Believe me, people have put a lot of thought into how to optimize these things.

Compared to triangle rasterization it's nowhere near close - for eg. I've only seen a decent voxel skinning implementation a few years back - realtime is entirely based on it and it's baked in to the hardware pipeline.

It (as with most things) comes down to the implementation. If everything is rendered server-side, and the client is a dumb remote framebuffer, then yes, you'll notice 100ms of input lag. If, on the other hand, environments are composited on the server while the player's avatar and particles are texture-mapped server-side but then motion-tweened and composited into the environment on the client-side (which doesn't require much more work than the dumb remote framebuffer approach), then gameplay feels nearly as snappy as if everything were local.

Already happening.

https://games.amazon.com/games/the-unmaking

"Powered by Amazon’s AppStream -The Unmaking is the first game to unleash the power of the cloud so players can experience thousands of enemies, destructible environments, and an epic, cinematic soundtrack on their Fire tablets."

May or may not be cost effective.

I'm highly skeptical of the viability of your point #3.

http://onlive.com/ tried it

That is different - they weren't using cloud rendering - they were just stuffing games to some sort of VM and streaming the output.

Cloud rendering would be something like you and me play the same game and the server rendering only has 1 instance of every resource used for rendering to reduce memory overhead.

If you have shared instance worlds (eg. MMO) you can then do shared state effects like animation, advanced lighting, etc. and reuse the calculations for each client.

I'm curious how much of a savings you could actually expect to gain from rendering a single scene for multiple cameras at once. My understanding is that, historically, a lot of the work in rendering a scene is camera-dependent, and that a lot of performance optimizations for rendering rely on being able to avoid computing things that aren't visible to the camera. Has that changed significantly over the years, or am I just wrong?

Reply on news.ycombinator.com