The Causes Of Lag In PC Gaming And How Your Brain Interprets Lag

By James Stringer

Imagine being in the front row of the mosh pit, ready to see your favorite band. The band comes on stage, you and the rest of the crowd go wild. As the band starts barreling into their first song, you notice something sounds off.

It’s the drummer. He is hitting all the right notes, but his timing seems to be off.

You notice the other band members struggling to keep in sync, they do their best to keep the tempo, but as the band gets more and more out of sync, things really start to fall apart.

This is not what they sounded like on Spotify.

Frame Cadence, Latency and You

Music tempo is described as beats per minute, or BPM for short. To make a 120 BPM beat sound good in our example, you would hit the drum exactly every 0.5 seconds, as this evenly disperses the beats throughout the 60 second period. If you ask your band mates to play along with you, their brains are going to be able to easily identify the simple beat, and it’ll only take them a few seconds to pick up on the tempo and jam along with you.

What about if you didn’t evenly disperse the beat throughout the 60 seconds? Let’s say you hit the drum 120 times very quickly at the beginning, or right at the end? What about if you just drummed at random? One drum here, another 3 there, with no rhyme or rhythm until you got to 120? These could all be measured as “120 beats per minute”, but your band mates would quickly become frustrated and explain that there is no way for them to keep in time with you.

This is exactly like how frame times work in video games, except instead of dealing in minutes, we’re dealing in seconds, and instead of a drum, we have a video game, and instead of band mates, we have you!

It’s All About Feedback Loops

Normally in a video game, you click the mouse, within a millisecond or so, that click is sent to the game engine. You will see the results of that click in the next frame that arrives. Maybe you score a headshot, or maybe you don’t. In any case, in a 60FPS VSync game, the results shouldn’t take any longer than 16.7ms to arrive.

Once your brain gets into the rhythm of understanding that the results will be displayed within 16.7ms, you can start to predict the next appropriate action. For example:

When I click fire with the enemy in my sight, the next frame will show where the bullet hits, because the enemy was moving left, I know the next time I click fire I should move the mouse slightly left if I want a chance to hit them again.

But what if the feedback doesn’t arrive in time?

LAAAAAAAAAAAAAAAAG

You all know the feeling of playing a game on online — you’re either playing on a server that is a little too far away, or your roommate is downloading a movie and you notice your connection becoming unreliable, or even worse, you realize your GPU just can’t handle the new game you installed. You feel disconnected from the game, when you click, the result you wanted doesn’t happen the way you wanted it to, you miss even when you feel like it was a guaranteed hit. You get frustrated.

Let’s take a quick into the two major contributing factors of lag, hardware and network.

Network Performance — A Holy Trinity

BANDWIDTH

The Holy Grail, the reason speedtest.net exists and one of your ISP’s favorite ways to sell you a more expensive plan.

Bandwidth, as you probably know, is the number of megabits per second you can upload and download via the internet. More is always good.

Your bandwidth varies from server to server, location to location. Bandwidth is heavily affected by jitter and packet loss explained below. Contention is also another big factor in limiting bandwidth. Contention means each user is fighting for their share of the server you’re downloading from.

For best results in Parsec, try connecting to your server with the “Bandwidth (Megabits)” set to 5. If things are performing OK, you can try connecting again with 10, then 15.

Packet Loss And Jitter

Ever been on a call in an area with poor coverage, and someone needs to read out an important detail? Perhaps it’s an email address, or a password and one of the characters gets lost in translation, and you need to repeat the whole thing again? That’s packet loss.

Jitter is kind of like packet loss, but instead of a character going totally missing, it just arrives out of order. You might get the first character, then the third, then the forth, and then finally the second. This means that instead of everything being entered as soon as it is received, you need to wait for all of it to arrive and reorder it. Jitter happens a lot on wireless networks, like Wifi and 4G. It can have a real impact on latency and bandwidth, and it’s something we’ve tried to correct for with our proprietary UDP networking protocol.

Parsec is best played when both server and client are connected via network cable to the internet. Wifi and 3G/4G can add significant jitter that causes the game to be unplayable.

Latency

Latency is the time between your click, and the other end receiving that click. On a server 30ms away, it will take your click 15ms to get there, and 15ms to hear back from the server — Latency is usually described as PING. This means when playing on that server, you’re seeing data that’s 30ms old, then up to another 16.7ms for that result to be displayed on screen — meaning you could be waiting up to 46.7ms to find out if you scored a hit or not. It adds up.

Although Parsec does a really great job in high latency environments, nothing beats running Parsec on a low latency connection. If you’re trying to decide which server to try Parsecing to, use this tool to find the datacenter with the lowest PING from you.

Networking Protocol

There are two primary protocols used to transfer data packets across the internet, TCP and UDP.

TCP is the most widely used protocol. Its primary benefit is that for every packet received from the server, the client will acknowledge that it arrived and that the data is not corrupt. This is important where the data being transferred must be without fault, like in banking or online shopping. The only issue with TCP is that its strength in data integrity makes it inherently more latent. Every packet needs to be replied to with “yes I got it, it all looks ok.” That Acknowledgement adds some serious milliseconds, and in gaming, milliseconds count.

UDP prioritizes data getting there as quickly as possible over data integrity. Getting your data to the online game server as quickly as possible means you’re going to be able to get those headshots. UDP is always preferred when the data is live and not adversely effected by faults (like a video game or live video stream).

Parsec built its own protocol on top of UDP for its low latency benefits, your keyboard commands, the video and audio you receive from the server are all transferred securely over UDP. This and the way Parsec captures and displays your game, give Parsec a significant edge over other remote access technologies.

You can read more about why Parsec chose UDP here

Hardware — Maxed Out

Remember the days of scraping by with whatever came with your prebuilt PC that you got from your local electronics chain?

What about the days of looking up tutorials about how to run Skyrim at less than low, then watching a YouTube video about a guy with an SLI setup with more cooling than a cryogenics facility.

One day I’ll be able to afford that — me to myself…one day hasn’t arrived yet.

Graphical lag usually comes from the GPU’s inability to render a frame in less than the refresh rate of the monitor.

On a 60Hz monitor (that’s 60 new images displayed per second), the GPU would need to render a new frame at least every 16.7ms to keep pace. That’s quite a pace to keep consistency wise, especially when there are a ton of players on screen or a few huge explosions.

Often what happens is your frame rate will dip below that when a big explosion happens, you may drop to 50, or even 40FPS. At this moment, it is a lot harder to track your opponent because the time and consistency of the frame rate varies from second to second; this is exacerbated by your monitor’s undying need for a steady frame rate. If your GPU can only manage to render 2/3 of a frame in 16.7ms, it will just send it as is, meaning you only get a portion of the frame and a “Tear” appears. This also happens if your GPU is sending out too many frames per second, the GPU really doesn’t know what to do with all those extra frames other than ship them off and hope they arrive safely.

VSync was introduced to attempt to offer some solution to the GPU not caring about the frequency requirements of the monitor — VSync cleans up visual output by removing ugly tearing and improving frame cadence. With VSync enabled on a 60Hz monitor, the GPU will only send a full frame every 16.7ms. This is all well and good if your game is running at above 60FPS, but if your GPU can’t manage to deliver a frame within 16.7ms, it will be required to wait until next time the monitor comes a-knocking (another 16.7ms).

If you’re prioritizing lag while gaming with Parsec, we recommend disabling the Parsec VSync setting. You can even run at a lower resolution as well as select a lower “Bandwidth (Megabits)” number. If you find that everything is very smooth, experiment with increasing the resolution and bandwidth. In the game settings, you should also opt to disable VSync and “post processing” effects. These settings will all impact image quality, but will improve the responsiveness and fluidness of the game.

What happens when your network and PC hardware are all working well?

Hunting Like Our Ancestors

When the technology gets out of the way, your brain can do what it’s made for. Hunting — a magical mix of reaction, prediction and timing,

Remember those times you played an FPS, and you notice an enemy down range, you track your target, you point your mouse just in front of where you want to hit and you hit fire. BOOM HEADSHOT.

Human race’s long lost past time of hunting to survive gave us the ability to judge accurately the velocity and trajectory of a target, so that we’re able to predict the right direction and speed to fire to connect with our prey. Your primal instinct shows no bounds, and you land top of the leader board. What about when it doesn’t go so well?

Lowly Last

In a simplistic tracking method, you need enough time to be able to discern the speed and direction of the target, what way is it going, and how fast. This becomes much harder the less often you get visual feedback, and even harder when that feedback is not evenly distributed. You could think of this as a deer running behind shrubbery, it may be running at a constant speed, but as the deer runs, trees, vegetation and the landscape obstruct your vision.

Compare this to playing a shooter and tracking your enemy in a low frame per second environment, you take longer to accurately predict the path. It might not be so bad when the target is moving at a constant pace, but as soon as the target starts zigzagging, good luck.

You can see the effect of low frame rates impacting your ability to track the stars in this example. What adds further complication to this is that in a low frame per second environment, the image on the screen is often far behind the target.

Let’s Take An Example Of 30FPS

You can calculate the average frame time by dividing 1000ms by the frame rate = 1000/30 — the average frame time = 33.3ms

As a simplistic equation to demonstrate latency due to low frame rates, the median latency you will be subject to will be 33.3ms/2. Your view is, on average, 16.7ms behind the target.

You can see this displayed very well here — it’s the same test as above, just missing the stars. Notice the UFO’s running at less than 30Hz are seemingly always behind the 60Hz UFO?

One of Parsec’s core strengths is its focus on 60FPS. Unlike other display protocols, Parsec was built by gamers who know that 60FPS is the minimum for a great experience. Parsec uses technology built into your GPU to capture and compress that data into a highly efficient video stream that can even be easily played on a Raspberry Pi

Brain Lag

Like a video camera recording a video, the brain has its own pipeline to turn whatever it sees into something meaningful.

This MIT study was surprised to find that test subjects who were shown an image for 13ms (On a 75Hz monitor) could discern details about the image when questioned by researchers. This surprised the researchers, as the traditional belief was that it takes much longer for a brain to be able to gather enough detail to make some sense of what it’s seeing.

Recognition is one stage of the pipeline, but turning that into a physical reaction is where most latency occurs. For most people in the test, it takes approximately 250ms to traverse the whole pipeline — from seeing an image appear on screen, to clicking your mouse to register that you’ve seen it. You see how you stack up here.

This test is exacerbated by modern image processing techniques — you probably could do much better with an old CRT monitor due to its very low latency characteristics VS LCD panel you’re probably using now. That’s why modern TV’s have a gaming picture mode to reduce image processing.

Parsec helps reduce latency by using an advanced “zero copy” system which takes the frames directly from the GPU as they are being sent to the server display, it then keeps them inside the GPU’s video memory, then uses the GPU’s high performance video encoder to convert them into video. This is done to save microseconds between the data being passed back and forward between the GPU, CPU and system memory.

IRL Brain Prediction Capabilities

Here, catch this!

When you brain sees an object of interest, like a ball flying toward you on a pitch, your brain will do a snap calculation to figure out where the ball may be going, and how fast. The brain also knows that there is a lag between your retina’s and your brain being able to see what’s going on — about 100ms. The magic part is that it calculates the balls current position in real time, 100ms ahead of where it was when the brain saw it.

Read about it here.

Familiarity Reduces Reaction Time

Ever got into a game 6 months or so after everyone else has had it, multiplayer is hard! Because you’re unfamiliar with maps, every corner is a surprise, enemies pop up out of nowhere, and in each one of these skirmishes, the other player manages to come out on top.

If we remember previous in the article, our body’s reaction time is approximately 250ms when dealing with a random event (even when we expect a random event), but once you’re familiar with the map and how other players act inside of it, your brain starts to make more and more predictions to aid your competitive advantage.

If we put a probable value of ~100ms on the time it takes to react to a predicted event, we start to understand how a reduction of computer latency and an increase of frame frequency can aid our performance in a video game. The same applies in the real world. Lebron James has a significantly lower reaction time than you in basketball because he’s seen everything before; whereas, you’re going to feel completely lost on the court. That, and, he’s an incredible athlete.

Take for example, a server with ping time of 30ms, a 16.7ms frame time, and a 100ms reaction time in a map you’re familiar with — you are looking at a minimum delta time between an enemy event and your ability to click the mouse of around 150ms.

150ms is the time it takes Usain Bolt to move just over 1.5 meters (5ft).

If we are reacting to a totally unexpected event, it may be closer to 300ms (30ms + 16ms + 250ms), or 3.1 meters in Usain Bolt’s world.

Parsec captures frames at the same rate they are sent to your server monitor, Parsec does not largely effect fame times from the server as its method is to capture what has already been created. Parsec encodes what would have been sent to the monitor.

Frame times to the client can be impacted by network congestion, packet loss and jitter. This can only be solved by improving the connection quality or reducing the requirements of the connection (eg. dropping the bandwidth from 15Mbps to 5Mbps). That’s why we created our own network protocol to counteract some of the negatives in your network. The video decoder on the client side can also play a role due to its requirement to be able to real time decode the frames — luckily for us, due to the popularity of online video streaming — most recent hardware (since 2011) will be able to easily decode a 1080P60 H264 video.

What Happens If There’s Network Latency, And The Game Lags?

First, you lose. But at a technical level, let’s say that the network latency now jumps to 80ms, and your PC’s graphics are set too high so you’re only getting about 30fps? That’s 33ms between frames. Due to the PC lag and the server latency, you have real trouble being able to predict what’s going on, due to this your reaction time approaches 250ms.

80ms + 33ms + 250ms = 363ms is about 3.8m (12 Ft) in Usain Bolt’s world.

That’s a 3.8M zone in which your enemy could be. Spray and Pray.

Wrapping Up

Delays between you and the video game you’re playing have a real impact on your performance in the game. Although round trip time is an easy vector to measure, it only provides one part of realized latency. Realized Latency is made up of 4 major parts, network latency, hardware latency, display latency, and human reaction latency. Without a user being able to meaningfully improve network latency as it is fixed by location and network type, we can improve realized latency by improving frame rates and frame consistency. Improved frame rates reduce time between feedback. Frame consistency in a familiar environment allows the brain to focus on real time prediction rather than latency heavy reactions.

Parsec’s display protocol was hand crafted to not get in the way of you and your video game sitting on a server 500 miles away. It offers a near local gaming experience, with client hardware and network requirements similar to that of a video streaming app like Netflix or Youtube. Parsec’s technology bridges the gap between cloud gaming the device in front of you.