Frame Rate Test — Parsec Streaming Versus Steam-In-Home-Streaming + VPN For Cloud Gaming

By Benjy Boxer

For many experimental projects in cloud gaming, Steam-in-home-streaming + VPN was the original hack that demonstrated the opportunity and potential for cloud gaming today. Larry Gadea wrote this post that got significant traction on Hackernews and elsewhere and helped create a community of enthusiasts on Reddit for cloud gaming. It was our original inspiration for building Parsec. One of our early goals has been to outperform this hack and to make it substantially easier to play games on any device from anywhere. With Parsec, you can stream your games from a cloud gaming PC, from your personal gaming PC, or connect to a friend for multi-player co-playing over the WAN. As these tests show, Parsec streaming over the internet is much more reliable in hitting 60 FPS versus the Steam-in-home-streaming + VPN set up.

To check out LAN performance, check out this article.

Benchmarking Performance

Our goal with Parsec is to make the lowest latency game streaming technology, prioritizing smooth 60 FPS over everything else. At times, this can result in hiccups in video quality due to internet bandwidth or Wifi, but we want you to be able to play your games anywhere with the highest quality settings possible. If you experience video quality issues, you can also try some of these settings to improve your quality.

We tested Parsec streaming from an AWS GPU server against Steam-in-home-streaming + Hamachi from an AWS GPU server. We were only measuring frame rate performance in this test, we tested this set up under various simulated internet connections to demonstrate the value of our networking protocol.

Why We Tested Against Steam + VPN

The only way to use Steam’s in-home-streaming tool over the WAN is to hack the system to make it think you’re streaming over a local network via a VPN. We will also publish a test of Parsec versus Steam-in-home-streaming in a local area network for benchmarking purposes as well.

THE RESULTS

Under all scenarios tested, Parsec is significantly more likely to maintain 60 FPS versus Steam + VPN set up. These graphs were produced with data from FRAPS. These graphs are used to explain the likelihood of hitting a specific time between frames. A perfect 60 FPS stream will have 100% of its frames land at 16.67 milliseconds. The graphs are density plots produced in R. The area under a density plot is 100%. The concentration of the area of the graph around 16.67 milliseconds explains how likely it is that the streaming solution will result in 60 FPS.

Under Normal Internet Conditions

We simulated poor network performance with Clumsy to produce the next two graphs.

With 3% Packet Loss On The Internet Connection

With 50% Of Packets Out Of Order On The Internet Connection

Postscript — More Details About The Test + Open Sourcing Our Data And Methods

We created this repo to give you insight into the functions and data used to generate the graphs we used in our blog post describing the performance of Parsec versus Steam-in-home-streaming + Hamachi while connecting to a cloud server hosted on AWS. The AWS server was running Windows Server 2016 on a g2.2xlarge.

The g2.2xlarge features (copied directly from AWS):

• High Frequency Intel Xeon E5–2670 (Sandy Bridge) Processors

• High-performance NVIDIA GPUs, each with 1,536 CUDA cores and 4GB of video memory. They use the K520 GRID cards based on the Kepler Architecture. They are old, blame Amazon.

The data was produced using FRAPS connecting to the AWS instance with a Windows 10 client with a GTX 1070 and Intel 6700k. This machine definitely wouldn’t need to use a cloud server to play games ;). The client has a 1Gbps fiber internet connection and is connected via an ethernet cable.

The tests were done playing GRID, Just Cause 3, and Tomb Raider 2013. All games were run in Ultra Low to prevent conflating issues with the GPU and issues with the streaming tools. For GRID, we tested while completing the same track of a race for 1 minute under each scenario. On Tomb Raider 2013, we used the benchmark feature. The AWS K520 card really had a hard time keeping up with Just Cause 3, but the data is included. There are some graphs with the data excluded available in the github repo.