Asynchronous Timewarp on Oculus Rift

The Oculus PC SDK v1.3 implements Asynchronous Timewarp (ATW) on Windows. With the latest drivers and hardware, we reduce judder, deliver consistent low latency, and improve efficiency. All apps benefit from this without having to do anything special.
At GDC 2016 we announced ATW as a new feature on Rift, and we’d like to elaborate on what this means for both users and developers.
Oculus has featured ATW on Gear VR for more than a year and had previously evaluated it for PC. You can read more here.
Since then, we have been working to implement this technology in the Oculus runtime for Rift. ATW has counter intuitive technical requirements and places unique demands on the hardware and software. Oculus developed the specifications and requirements, and drove the development of ATW on PC with Microsoft, NVIDIA and AMD. The results speak for themselves.
Background
Without ATW, VR applications need to be very conservative in how much work they ask of the graphics processor. Complex real-world graphics applications often have varying rendering workload frame-to-frame. Any background CPU or GPU processing can also impact the application’s rendering time of each frame. On a general-purpose multitasking OS like Windows, there is always something in the background that can affect an application’s performance.
Although the term “timewarp” was more recently coined, the basic idea of reprojection existed well before – we shift the rendered image before sending it to the display in order to correct for head motion that occurred after the scene was rendered, reducing the perceived latency. Synchronous timewarp has been in the Oculus SDK since Rift Development Kit 2 (DK2). It helps reduce latency, but the application must still reliably meet frame rate to avoid judder.
Asynchronous Timewarp (ATW)
We can deal with varying workloads and background apps by decoupling timewarp from the rendering loop and running it independently and asynchronously. By relying on CPU and GPU preemption, we should be able to pick up the latest available application frame and timewarp it before each display scanout at 90Hz.

The diagram above illustrates how ATW with GPU preemption can work. In the diagram, the application renders left and right eyes for each frame with the result processed by ATW shortly before it is displayed. In the first frame, rendering completes on time so the behavior is the same as with synchronous timewarp. This should be the typical scenario. In the second frame, application rendering takes longer, causing it to miss the VSync deadline. Without ATW, this would result in judder. However, with ATW, we can pick up the earlier frame and display it again with the headset’s new orientation, eliminating the judder that would occur if it was displayed as-is. The resulting motion appears smooth to the user.
Getting this right took a lot of work. Modern PC GPU hardware and software are optimized for very high throughput, but not for preemption. Windows and several recent GPUs have had basic support for GPU preemption, but when we first tried to use it, it did not work with Microsoft, NVIDIA, and AMD to change OS GPU scheduling, GPU command processor microcode, and GPU kernel driver design to enable ATW. Graphics driver VR extensions in the form of AMD’s Liquid VR and NVIDIA’s VRWorks were developed to support this.
Benefits
The user experiences much smoother virtual reality with ATW. Early measurements of Rift launch titles running without ATW showed apps missing ~5% of their frames. ATW is able to fill in for the majority of these misses, resulting in judder reduction of 20-100x. This functionality comes at no performance cost to the application and requires no code changes. And, this is only the beginning — we are working with partners to improve the effectiveness of ATW going forward.
On Oculus, ATW is always running and it provides insurance against unpredictable application and multitasking operating system behavior. ATW can smooth over jerky rendering glitches like a suspension system in a car can smooth over the bumps. With ATW, we schedule timewarp at a fixed time relative to the frame, so we deliver a fixed, low orientation latency regardless of application performance.
This consistently low orientation latency allows apps to render efficiently by supporting full parallelism between CPU and GPU. Using the PC resources as efficiently as possible, makes it easier for applications to maintain 90fps. Apps that need more time to render will have higher positional latency compared to more efficient programs, but in all cases orientation latency is kept low.
However, ATW is not a silver bullet. Failing to maintain a consistent, full frame rate may produce visible artifacts including noticeable positional judder, particularly in the near field of view. An application that falls below 90fps rendering will get re-warped in time to avoid rotational judder, but while orientation latency is kept low and smooth, animation and player movement may judder in lock-step with missed frames. For these reasons we continue to recommend that developers do not rely on ATW to save them from low frame rate.
The best, most immersive and comfortable experience for users runs consistently at 90fps and developers should target this frame rate aggressively.
Looking forward
ATW will improve as VR drives the PC ecosystem forward. Microsoft Windows 10 build 10586.164 includes features stemming from our collaboration with Microsoft and was designed with virtual reality in mind. While ATW operates well on Windows 7 and 8, you’ll find the most efficient path and best experience on the latest version of Windows 10.
Advances in GPUs will improve on graphics performance and preemption capabilities for ATW. And as hardware evolves, we expect efficiency and latency to continue to improve, providing an even better experience.
For developers, the best news is your apps benefit from ATW without any additional work. The existing techniques to maintain high performance graphics and consistent frame rate still apply, but with ATW, you get consistent low latency, extra CPU and GPU processing headroom, and glitch resistance.
Ultimately, this means better virtual reality for all Oculus users, always. Special thanks to everyone at Microsoft, NVIDIA and AMD for all their support.