Earlier this week, the team behind Ashes of the Singularity released an updated version of its early access game, which updated its features and capabilities. With support for DirectX 11 and DirectX 12, and adding in multiple graphics card support, the game featured a benchmark mode that got quite a lot of attention. We saw stories based on that software posted by Anandtech, Guru3D andExtremeTech, all of which had varying views on the advantages of one GPU or another.
That isn’t the focus of my editorial here today, though.
Shortly after the initial release, a discussion began around results from the Guru3D story that measured frame time consistency and smoothness with FCAT, a capture based testing methodology much like the Frame Rating process we have here at PC Perspective. In that post on ExtremeTech, Joel Hruska claims that the results and conclusion from Guru3D are wrong because the FCAT capture methods make assumptions on the output matching what the user experience feels like. Maybe everyone is wrong?
First a bit of background: I have been working with Oxide and the Ashes of the Singularity benchmark for a couple of weeks, hoping to get a story that I was happy with and felt was complete, before having to head out the door to Barcelona for the Mobile World Congress. That didn’t happen – such is life with an 8-month old. But, in my time with the benchmark, I found a couple of things that were very interesting, even concerning, that I was working through with the developers.
First, the initial implementation of the FCAT overlay, which Oxide should be PRAISED for including since we don’t have and likely won’t have a DX12 universal variant of, was implemented incorrectly, with duplication of color swatches that made the results from capture-based testing inaccurate. I don’t know if Guru3D used that version to do its FCAT testing, but I was able to get some updated EXEs of the game through the developer in order to the overlay working correctly. Once that was corrected, I found yet another problem: an issue of frame presentation order on NVIDIA GPUs that likely has to do with asynchronous shaders. Whether that issue is on the NVIDIA driver side or the game engine side is still being investigated by Oxide, but it’s interesting to note that this problem couldn’t have been found without a proper FCAT implementation.
With all of that under the bridge, I set out to benchmark this latest version of Ashes and DX12 to measure performance across a range of AMD and NVIDIA hardware. The data showed some abnormalities, though. Some results just didn’t make sense in the context of what I was seeing in the game and what the overlay results were indicating. It appeared that Vsync (vertical sync) was working differently than I had seen with any other game on the PC.
TL:DR - ALL games that use DirectX 12 are launched inside a sandbox, the OS handles a lot of the composition, and it's capped at 60Hz with V-Sync enabled regardless of whether you have 144Hz and VRR or not.
Edit: A SHORT HISTORY LESSON
Back in the day when BF4 launched on Xbox One, DICE talked about how they had to design the game around the way the Xbox One did things. They had different resolutions for the game and the UI, which also refreshed at different rates, V-Sync was always on, and the game was "effectively in a sandbox", which we have now come to understand actually means that it's launched in its own virtual instance.
DICE said that some of these features would probably find their way back into DirectX 12. Microsoft at the time also said that the only way they could see Xbox games on the PC was if the environments were the same.
Cue GDC 2014.
This was Microsoft's first open discussion about DirectX 12, and the first time they ever talked about its features. Two lines stand out now that we know how they affected the Ashes benchmarks - "console execution environment" and "conformance and consistency". Microsoft planned all along to make games run this way, despite knowing in advance about technologies like variable refresh rate monitors, super resolution, and VR.
None of those things will be possible in a DX12 game sold inside or outside of the store without some serious workarounds or working with Microsoft to implement them into the OS, and Microsoft doesn't care either way. They have games on their store being sold, they have Hololens ready to serve Windows 10 gamers, so from their perspective, they're all set.
Edited by CataclysmZA - 2/29/16 at 1:28pm