A new approach to local multiplayer / splitscreen perspective with raytracing
ph3at.github.io
ph3at.github.io
In what might be a bit of a cursed, uninformed thought - but I'd like to see what happens if each player's perspective could be altered individually. Would it be possible for players to have unique camera perspectives when split, and then reorient to the same perspective when within a given distance to transition to single screen?
From memory (as I have been unable to find a demo video), the LEGO series had some interesting approaches to dealing with this. IIRC LEGO Marvel Super Heroes gave players control of their camera when in the open world, so in Dynamic Splitscreen mode there was a little fade transition when recombining cameras to single screen. Pretty sure there was a little delay too so it wouldn't recombine unnecessarily, and it was typically a more annoying point of the splitscreen as the dividing line would pivot more dramatically - something the raytraced approach would definitely improve!
Imagine a 3rd person action game. A and B split up and have two perspectives. They end up facing in opposite directions, how do you reconcile the camera views?
Maybe you can have some heuristic that only does it if their perspectives are "close enough" but what value does it bring?
For some kind of topdown/isometric type game I could see how that might work.
EDIT: actually read the article after getting curious. All of this is covered, extremely cool article. Don't quite understand ray tracing enough to understanding exactly why it is faster. Is it because rasterisation starts from the camera but ray tracing starts from the light source? so you can amortise more calculations if you start from a light source rather than individual perspectives?
My assumption is that the shared scene and shaders might play a role here, although the direction of the rays will still vary.
The reason for this is that while the primary rays are a coherent bundle, the secondary rays are all over the place. Essentially, as soon as you hit the first object, it bounces in a random direction all over the scene. And for every primary ray, you might end up spawning a dozen secondary rays.
I believe this is also the reason dedicated raytracing hardware is needed for GPUs. While a coherent bundle is fairly trivial to parallelize, incoherent rays are going to absolutely wreck your SIMD performance. You really want dedicated hardware to properly manage this, for example by doing BVH traversal and collecting all secondary rays that hit the same object for a single mostly-coherent render pass.
The problem is Rasterization is a bastard Raytrace. A camera view, represented as a trapezoidal prism, is transformed into a cubic prism and flattened. Every pixel is a parallel "raytrace" along the flattened cube. To get multiple viewports Rasterization creates an entirely new rendering scene, complete with the boilerplate code and API calls, as if it were its own game with its own screen. A four way split was akin to brute-forcing four bastardized raytraces.
Raytracing simplifies this by admitting this is terrible, and thus giving every pixel its own official transform (Raytrace) rather than the ugly pile of hacks above. I think of it as a "Camera Shader", sitting with pixel and vertex shaders as a way to dynamically change a given pixel.
Technically, yes. There's not any major technical hurdle here. You'll probably have to render a bit more than required for each screen, so your shaders don't create seams near merge (because they may behave differently at the edge of whatever they're sampling). Also you'll have to decide what you want to do about on-screen effects (hit notifications etc) on merge. You may get something that looks like seams to our brain's pattern recognition anyways as you get close to merge (a "triangle" looking shape on floor/walls when perspectives only slightly diverge). I suspect the latter is why the game you mentioned had a transition - besides providing a visual to each player that they can now also look at the other half of the screen.
> the LEGO series had some interesting approaches to dealing with this
Indeed, LEGO Indiana Jones (2008) for sure had it on 360, but IIRC LEGO Star Wars I (2005) and II (2006) did not.
Here it is on The Force Awakens https://www.youtube.com/watch?v=T04B2coSN0Y
In some ways, it was awesome, in others, it was terrible!
- When the viewpoints are very close but still split, things from both views are only slightly offset which gave some weird effect like stereoscopic stuff. Visible at 0:07, 0:11, 2:05 in the SW video above.
- FOV is more or less fixed, so on a 16:9 screen with a fixed FOV you get either fixed wide FOV for a vertical-ish split (nice) but also for horizontal-ish (everything is super small), at the cost of a fisheyesque warp; OR you get fixed narrow FOV and you can't see much left-right on vertical-ish split nor up-down on horizontal-ish.
Overall the combination of both made the games extremely headache inducing for me.
> IIRC LEGO Marvel Super Heroes gave players control of their camera when in the open world
Here one can see the open world sections with the POV control, non-dynamic split screen:
https://youtu.be/SPQXmZR7JWo?feature=shared (From about 2:30)
Now get off my lawn.
Was a big fan of the original back in the C64/Amiga days.
It has a pretty awesome soundtrack, imho..
I cannot find anything re this Sphere Spectacle, however.
"In case you’re wondering about the game: what you’re seeing (for the first time) is Sphere Spectacle."
GitHub: https://github.com/W4RH4WK/Raygun
Video: https://www.youtube.com/watch?v=QZSoVidqV2Q
Edit: the sound effects are from an actual glass sphere rolling on and bumping against a metal surface, recorded by a coworker and myself :)
A cool game mechanic would be, when the screen warps the players near eachother if they could exchange items, or attack the other - or kite a monster through the warp at the other player... especially cool if you had to travel far away on the map from the other player to grab the monster to fling at your opponent (without being killed by monster) (or maybe you have to get to a BFG first and frag 'em... through running in opposite directions to your bfg base, ang fighting monsters, and flinging then through the warp until you reach the bfg, which is the only thing which can shoot through the warp...
your side arm is a gun and a tractor whip with a heavy cool down to entangle a monster to throw through the warp, so use it strategically. whomever gets to their bfg firt - now play it on a large square screen, with four players, one per quadrant running to the bfg and flinging monsters through the warps - the center picks a player at random when flung into.
Puzzles that require you to go different directions, to gather components for the puzzle, but you have to first find the warp-thrower to transfer resources, which you have to gather and transfer to your warp partner to solve the puzzle, but you need that BFG to throw resources to the other.
Multiplayer maps where your maps are vastly different, but you have certain "tiles" that are all shared... when two of you run into the same tile - you are warped in multi.... and then you either have to fight or work together.
Some tiles can be "ok we are warped together: what are our goals? "Fight eachother|team against other guy|find a resource| trade| etc"
This mechanic can be so widely used....
I LOVE IT (BTW - I AM one of two of the guys that built the gaming marketing for Intel for proving that a <$1,000 Celeron Machine could satisfactorily play games such as DOOM, Descent, UO, etc.... I know a little bit about gaming. My cohort at intel is an EVP at gaming companies you have all impacted by) (I went tech, networking, medical, etc - he was all in on gaming)
* if you have multiple users on the same physical keyboard, you're very likely to run into the problem where the hardware won't actually register the keypresses simultaneously (some hardware even generates keypress events for keys you didn't actually press!).
* if you plug in multiple keyboards, most input event APIs drop the source index, and hotplug is even less supported. Also this is likely to break single-user-like use cases and sometimes hardware macros.
For example, the average laptop keyboard or membrane keyboard is unlikely to have it - and that's exactly what people will be playing casual splitscreen games on.
That being said, we can assume that people looking for split screen have the equipment to play such games (which I assume it's multiple controllers)
Slightly older (2021) steam post https://store.steampowered.com/news/group/4145017/view/30618... suggests better number depending on game genre. Although considering that whole point of that post is to encourage developers having better controller support, and suspicious category grouping and and qualifiers like "racing and skating games", "many sports and fighting games" and fact that some of those genres are often played with genre specific controllers I would take those number with grain of salt. From the other number of 48m players who have used a controller at least once with ~130m overall active monthly users less than 35% of players with a controller seems likely, considering that not every player from first category is regular active user. And from the same link only 10-15% controller sessions with multiple controllers. So maybe 3-4% with multiple controllers. Although actual number with multiple controllers might be slightly higher compared to session percentage, since many games act buggy with multiple controllers, so it's not uncommon to disconnect all the extra controllers while playing single player games.
From the controller friendly (but also keyboard friendly) PC games I have worked on, 10-30% of players using controller seems reasonable number.
It very much depends on how you define majority. Considering that 4/5 most played games on steam based on current player count are FPS (which on PC are almost exclusively played with mouse and keyboard), not surprising that the "daily gaming session" 10% number is skewed more towards the lower range of estimate. If you exclude FPS numbers are better but even then it's far from majority of PC gamers.
I would exclude rts and fps for the simple fact that those are made with mouse in mind on pc, but fps is challenging because they do work well with controllers on split screen (and you can't play an fps with just keyboard).
I think the good answer is in the stats, 35% of players connected a controller, that's a lot of controllers. There might be more, there might be some with multiple.
But it's a chicken-egg problem. People don't have multiple controllers because they don't make split screen games anymore, most of them require multiple consoles/pcs.
I follow the genre a lot since I want to play with my wife and my friends. It's mostly indie games with the exception of bg3/divinity original sin.
Steam stats covers a lot of pc gamers, so we can and should be relying on that. 130M pc gamers is more than we can ever observe in life, and at least 48M of them have controllers
Steam just proved that at least 48M people have controllers on PC, that is a lot of controllers!
As for non platform users, we literally have no information on them and no way to gather such information, so for the purpose of this discussion, they don't exist. Being an edgecase cuts both ways (yes, steam is that big)
I'm not sure if you read anything between the lines, I remember it was 50% of steam player have a controller, I remember wrong and it's 33%, that's about all I have to say on the stats.
I would not recommend using a keyboard for split screen gameplay, we have a better alternative, especially with the loss of mouse.
From my personal perspective, the keyboard is always a poorly optimized gaming peripherals, it's the mouse that makes a difference. The keyboard is useful when jumping into chat, but that's about it. For a while there were those poor quality gaming boards, made more sense than a keyboard and had an analog stick included, these disappeared from the market sadly, but made more sense
Of course, they did that by dialing down the graphics.
If split screen is not the main focus of the game, the engineering effort to even make these trade-offs would be mostly wasted.
Recent examples are some of the Switch console sellers (Zelda and Pokemon), both of which have framerate issues. Not because the games are that pretty (personal opinion), but because their developers did not prioritize performance.
It's up to you as a player / customer to decide whether you like the trade-off offered.
But seeing their demo of their proposed idea - well, I think I'd much rather just have a hard split between them. The oozy-woozy compression of space just feels a bit...gross, without any particular benefit.
And ultimately, it's still just a split-screen - just one where each of the screens is smaller.
There is also one more option I think I have seen, which is that when the two players are far apart, the camera zooms out. And you can also limit the players in how far apart from each other they can be.
Vaguely related, if you haven't seen it: https://www.youtube.com/watch?v=yqUv2JO2BCs
The cool part was that when the two players were within the range limit, the camera automatically adjusted its pitch to try to keep the players in the same perspective. It was surprisingly smooth.
Playing with my brother, it was pretty fun taking turns
> Rather than having entirely distinct and clearly delineated perspectives for each player, dynamically adjust the camera(s) at a pixel level.
> The most interesting part is what happens in the “no-man’s land” between, where space appears squished as the camera parameters are interpolated.
What does it mean by adjusting camera at pixel level? What is happening in the wrapped area and what does that effect have to do with raytracing? Does this technique bring any performance improvements now that two camera views are being rendered.
Plus I believe there was an (uncommon?) feature where two players could share a single controller to control each character (two shoulder buttons and an analog stick).
(Even with an Action Replay I remember that game being frustrating as hell.)
Not sure if you can actually create a whole game out of that mechanic, but it's a start.
The other jarring thing is that it seemed like a binary on/off thing, whereas it looks like it would be possible to make it a smooth and slow transition depending on the distance between players, make it a bit more surreal.
For the right kind of game I could definitely see this working. In fact you could make a whole game based around this kind of effect, with space being squashed and stretched in different areas etc.
If it also allowed the players some more freedom to move within their slice after the split, and perhaps some level of zoom out before the split, it would end up being much more smooth and fluid.