Overclock.net › Forums › Benchmarks › Benchmarking Software and Discussion › [Various] Futuremark's Time Spy DirectX 12 "Benchmark" Compromised. Less Compute/Parallelism than Doom/Aots. Also...
New Posts  All Forums:Forum Nav:

[Various] Futuremark's Time Spy DirectX 12 "Benchmark" Compromised. Less Compute/Parallelism than Doom/Aots. Also... - Page 10

post #91 of 253
Quote:
Originally Posted by iTurn View Post


How isn't news?

Can we not to post anything in this section unless it comes from another website?

Don't worry it's easy. Brb sending an email to wccf
Workstation
(4 items)
 
  
CPUMotherboardGraphicsMonitor
Xeon E5-2690 Supermicro 2011 Nvidia GP100/ Vega FE Dell ultrasharp 4k 
  hide details  
Reply
Workstation
(4 items)
 
  
CPUMotherboardGraphicsMonitor
Xeon E5-2690 Supermicro 2011 Nvidia GP100/ Vega FE Dell ultrasharp 4k 
  hide details  
Reply
post #92 of 253
Quote:
Originally Posted by sugarhell View Post

Don't worry it's easy. Brb sending an email to wccf
You mean WCofTech? Ops, I mean Wccftech rolleyes.gif
post #93 of 253
Quote:
Originally Posted by blue1512 View Post

It's true mate. However it's still an invalid score according to Futuremark, and that's my point here.

I guess my point is why care what Futuremark says when a site like HWBOT that actually gives out prizes doesn't care if tess is modified?

Surely people aren't so naive to buy a video card based off of one benchmark score?
Super P's rig
(20 items)
 
  
CPUMotherboardGraphicsRAM
5960x ASUS X99-A II Asus GTX 1080 Ti Corsair Vengeance DDR4 3000 
Hard DriveHard DriveHard DriveOptical Drive
MyDigitalSSD BPX NVMe Samsung 850 EVO Seagate Momentus XT 500 GB External DVDRW 
CoolingCoolingOSMonitor
EK-XLC Predator 240 Swiftech 240mm Radiator Windows 10 Samsung 40" 4K - UN40KU6290 
KeyboardPowerCaseMouse
G710+ EVGA SuperNOVA 850G2 Fractal Design Define S G700s 
Mouse PadAudioAudioAudio
Vipamz Extended XXXL Asus U7 M-Audio AV40 Sennheiser HD 439 
  hide details  
Reply
Super P's rig
(20 items)
 
  
CPUMotherboardGraphicsRAM
5960x ASUS X99-A II Asus GTX 1080 Ti Corsair Vengeance DDR4 3000 
Hard DriveHard DriveHard DriveOptical Drive
MyDigitalSSD BPX NVMe Samsung 850 EVO Seagate Momentus XT 500 GB External DVDRW 
CoolingCoolingOSMonitor
EK-XLC Predator 240 Swiftech 240mm Radiator Windows 10 Samsung 40" 4K - UN40KU6290 
KeyboardPowerCaseMouse
G710+ EVGA SuperNOVA 850G2 Fractal Design Define S G700s 
Mouse PadAudioAudioAudio
Vipamz Extended XXXL Asus U7 M-Audio AV40 Sennheiser HD 439 
  hide details  
Reply
post #94 of 253
Quote:
Originally Posted by criminal View Post

I guess my point is why care what Futuremark says when a site like HWBOT that actually gives out prizes doesn't care if tess is modified?

Surely people aren't so naive to buy a video card based off of one benchmark score?

You really want an answer to that question? tongue.gif

People think that because nVidia has the performance crown, their 960 is magically faster than a 280X "because it's nVidia". doh.gif Or maybe I'm hanging out with the wrong crowd. redface.gif

But yeah stuff like that just infuriates me.
Edited by magnek - 7/18/16 at 6:11pm
post #95 of 253
Quote:
Originally Posted by rosade View Post

.

If you read DOOM Vulkan patch notes They clearly say Async compute support for Pascal is still on the works and even without that Pascal Cards see 8- 12 % gain already

a) DOOM (Where pascal async is going to be enabled soon)

well you run into benchmarks like these from extremetech and gamegpu that shows regressions on nvidia cards but the usual increase for amd cards.

but then you have others like in the doom vulkan thread that shows an increase. that's been the problem with nvidia, they're not consistent with dx12 / vulkan. some show increases, regressions, or nothing at all. i've even see the occasional post on nvidia's offical forums of users complaining about regression and increases in doom vulkan on their nvidia cards. the only thing so far that has been consistent is time spy which shows no benefit for maxwell and below and a slight increase for pascal.

amd has shown improvements consistently. even in this "nvidia" sided time spy benchmark, if it is truly nvidia sided, still showed a bigger improvement on amd gpu's.
Edited by orlfman - 7/18/16 at 6:14pm
strix
(14 items)
 
  
CPUMotherboardGraphicsRAM
AMD Ryzen R7 1800x Asus X370 Crosshair Vi Hero Sapphire RX 580 Nitro+ LE G.skill Flare X 3200 2x8 16gb 
Hard DriveHard DriveHard DriveCooling
2x Western Digital Black 2TB 2x 512GB Samsung 850 Pro 1x 512GB Samsung 950 Pro Noctua NH-D15S + 2x Corsair ML120's 
OSMonitorKeyboardPower
Windows 10 Education Asus MG279 Ducky One White & Grey - Cherry Reds Evga Supernova G3 750 
CaseMouse
Corsair Air 740 Steelseries Rival 700 
  hide details  
Reply
strix
(14 items)
 
  
CPUMotherboardGraphicsRAM
AMD Ryzen R7 1800x Asus X370 Crosshair Vi Hero Sapphire RX 580 Nitro+ LE G.skill Flare X 3200 2x8 16gb 
Hard DriveHard DriveHard DriveCooling
2x Western Digital Black 2TB 2x 512GB Samsung 850 Pro 1x 512GB Samsung 950 Pro Noctua NH-D15S + 2x Corsair ML120's 
OSMonitorKeyboardPower
Windows 10 Education Asus MG279 Ducky One White & Grey - Cherry Reds Evga Supernova G3 750 
CaseMouse
Corsair Air 740 Steelseries Rival 700 
  hide details  
Reply
post #96 of 253
Quote:
Originally Posted by Bidz View Post

Not really, Pascal and all, it's irrelevant, fact is TimeSpy is not a DX12 benchmark since it's not doing what DX12 is meant to do, it has 1 specific code path and coincidentally it's the code path that's most favorable towards Pascal, but again lets ignore that, since being a single path DX12 benchmark makes it invalid as a DX12 benchmark.

Also no, dynamic loading has nothing to do with async computing, async computing is the capability of performing a task without the need to wait for another task to end (not the exact definition but an accurate enough one), preemption does this by INTERRUPTING another process to perform the new required one, giving a sense of priority. Dynamic load has nothing to do with async computing, it's a separate thing that can benefit async computing but it's not async computing.

Not really , the example I gave was one use case, depending on task priority too it can take over the resources. So a higher priority task can take over more resources on the fly. This with pre-emption makes the NVIDIA implementation in pascal a valid implementation of async compute (In reality pre-emption is not async , it is the value added factor)
Quote:
Originally Posted by orlfman View Post

well you run into benchmarks like these from extremetech and gamegpu that shows regressions on nvidia cards but the usual increase for amd cards.

but then you have others like in the doom vulkan thread that shows an increase. that's been the problem with nvidia, they're not consistent with dx12 / vulkan. some show increases, regressions, or nothing at all. i've even see the occasional post on nvidia's offical forums of users complaining about regression and increases in doom vulkan on their nvidia cards. the only thing so far that has been consistent is time spy which shows no benefit for maxwell and below and a slight increase for pascal.

amd has shown improvements consistently. even in this "nvidia" sided time spy benchmark, if it is truly nvidia sided, still showed a bigger improvement on amd gpu's.

Agreed guess the NVIDIA Vulkan driver is in a state of mess, and I am going to reserve my judgement on that till the official idsoft patch and a better driver.

If you take DOOM out of the equation as I said there is not much fair benchmarks around for DX12

1. RotTR( gameworks, NVIDIA leads, doesn't matter)
2. AotS, Hitman, ( AMD leads, AMD gaming evolved, doesn't matter)
3. Quantum break, total war, absolutely bad ports

So there is no conclusive or neutral stuff around expect time spy and DOOM (if the issues are fixed)
post #97 of 253
Calling timespy neutral is a big strech and Doom is different API. You can discard Ashes if you want. It's pretty neutral, but fringe case and I wouldn't extrapolate from it anyway.

The point is there is no good benchmark. Doom might be for Vulkan after it gets patch for Pascal. And Timespy is useless if you want to use it to compare GCN with anything. It might be good for Kepler and Maxwell comparisons. It's hard to say if it's useful for Pascal. It uses async implementation not used in few games we have and we have yet to see if it will used on other games. So Pascal-Maxwell comparisons might be off too. For overclocking and stability it's not ideal either, at least for GCN.
post #98 of 253
I'm a huge fan of 3dmark benches so this is a bit disappointing to see but is totally understandable from their point of view. If they properly implemented AC it would put Nvidia cards at a big disadvantage and most people are running Nvidia cards, which would be bad for Futuremark's bottom line. You don't really want to release a bench that favors one GPU manufacturer too much over another (especially the one with 80% market share), at least from a business perspective. That said, it really taints the "DX12" focus of the bench because all of those features should be included regardless of which company is better at implementing them. 3dmark11 was a pretty great demo of DX11 at the time iirc and didn't really seem to favor one brand over another (FS was a bit of a step back in that respect)...
post #99 of 253
Quote:
Originally Posted by rosade View Post

.
Pointing out mistakes in the post

1. In the async feature table, comparing GCN to NVIDIA architectures (conveniently excludes pascal)
The Table was done before Pascal was released,
Quote:
2. Pascal supports much more than pre-emption , it actually supports async compute via it's dynamic load balancer
How does this work?
a) There are two Tasks A and B running at the same time.
b) Task A completes before Task B
c)The Dynamic load balancer ensure Task B takes over the resources of Task A
d) This results in usage of "Idle cycles", thus decreasing latency
Exactly in GPUView the AotS test shows how it is doing asynchronous compute executing Graphics and Compute queues at the same time with more compute queues, meanwhile 3dmark time spy shows how whenever a Graphics/3d queues ends there is a space/pause(fences/barriers) and then comes a compute queue


The same way that pre emption works

and using fences cause some time/resources wasted/idling switching contexts
Quote:
  • Minimize the use of barriers and fences
  • We have seen redundant barriers and associated wait for idle operations as a major performance problem for DX11 to DX12 ports
  • The DX11 driver is doing a great job of reducing barriers – now under DX12 you need to do it
  • Any barrier or fence can limit parallelism
  • Don’t insert redundant barriers
  • This limits parallelism
  • This doesn’t allow the driver to pick the worst case of a set of barriers
  • Don’t expect fences to trigger signals/advance at a finer granularity then once per ExecuteCommandLists call.
  • To reiterate: We have seen redundant and/or overly conservative barrier flags and their associated wait for idle operations as a major performance problem for DX11 to DX12 ports.
also
Quote:
  • Submit work in parallel and evenly across several threads/cores to multiple command lists
  • Recording commands is a CPU intensive operation and no driver threads come to the rescue
  • Command lists are not free threaded so parallel work submission means submitting to multiple command lists
  • Be aware of the fact that there is a cost associated with setup and reset of a command list
  • You still need a reasonable number of command lists for efficient parallel work submission
  • Fences force the splitting of command lists for various reasons ( multiple command queues, picking up the results of queries)
  • Try to aim at a reasonable number of command lists in the range of 15-30 or below. Try to bundle those CLs into 5-10 ExecuteCommandLists() calls per frame.
and
Quote:
  • ]Make 3D and compute sections long enough.
  • Switching between compute and 3D queues results in a full flush of all pipelines. The GPU should have spent enough time in one mode to justify the penalty for switching.
  • Beware that there is no active preemption, a long running shader in either engine will stall the transition
.
Quote:
Originally Posted by rosade View Post

.Sound Familiar? Yes that is basic Async compute right there
which this benchmark isnt doing.
Quote:
If you read DOOM Vulkan patch notes They clearly say Async compute support for Pascal is still on the works and even without that Pascal Cards see 8- 12 % gain already
for sure it wont be using asynchronous compute+graphics in parrallel.
Quote:
Hitman, AoTS are AMD Gaming Evolved titles , You can not use them for fair comparison just like RotTR is a gameworks title. And Quantum Break is the worst PC port ever and Total war is a bad title regardless of platform.
why not if they have different path for each GPU vendor? dont they?
Quote:
Which implies there are so far only two neutral DX12/Vulkan benchmarks
a) DOOM (Where pascal async is going to be enabled soon)
b) Time Spy
DOOM is not a bechmark.
Quote:
So I guess there is no conspiracy here , And If people are talking about favoring one architecture , then they should have ignored Hitman and AoTs long ago.
well this happened with AotS
http://www.dsogaming.com/news/oxide-developer-nvidia-was-putting-pressure-on-us-to-disable-certain-settings-in-the-benchmark/

why 3dmark cant be biased if a nuetral dev couldnt use this feature?
Edited by PontiacGTX - 7/19/16 at 5:57pm
  
Reply
  
Reply
post #100 of 253
Quote:
Originally Posted by Bidz View Post

Not really, Pascal and all, it's irrelevant, fact is TimeSpy is not a DX12 benchmark since it's not doing what DX12 is meant to do, it has 1 specific code path and coincidentally it's the code path that's most favorable towards Pascal, but again lets ignore that, since being a single path DX12 benchmark makes it invalid as a DX12 benchmark.

Also no, dynamic loading has nothing to do with async computing, async computing is the capability of performing a task without the need to wait for another task to end (not the exact definition but an accurate enough one), preemption does this by INTERRUPTING another process to perform the new required one, giving a sense of priority. Dynamic load has nothing to do with async computing, it's a separate thing that can benefit async computing but it's not async computing.

The benchmarks don't reflect what you're saying. Pascal gains 5% or so from enabling async whereas AMD cards gain roughly 12%. From the looks of things, some people would only be happy if Nvidia Pascal took hits in performance when async is utilized. The 10-15% gains on the AMD side of things are perfectly consistent with real world performance gains in current DX12 games.
New Posts  All Forums:Forum Nav:
  Return Home
Overclock.net › Forums › Benchmarks › Benchmarking Software and Discussion › [Various] Futuremark's Time Spy DirectX 12 "Benchmark" Compromised. Less Compute/Parallelism than Doom/Aots. Also...