Overclock.net › Forums › Benchmarks › Benchmarking Software and Discussion › [Various] Futuremark's Time Spy DirectX 12 "Benchmark" Compromised. Less Compute/Parallelism than Doom/Aots. Also...
New Posts  All Forums:Forum Nav:

[Various] Futuremark's Time Spy DirectX 12 "Benchmark" Compromised. Less Compute/Parallelism than Doom/Aots. Also... - Page 24

post #231 of 253
Quote:
Originally Posted by Robenger View Post

Why are people pathetic? Please enlighten us.

Let's start with OP, i am not going to go through the whole thread since it mostly dissolves to part of folks pretending it matters, part of folks pretending it's conspiracy and part of folks thinking previous 2 groups are crazy (needless to say, where i land).
Quote:
It seems to be coded to strictly favor Pascal's hack-job async implementation, namely compute preemption, as per nVidia's dx12 "do's"and "dont's".

First and foremost, we obviously get totally understanding and informed opinion on Pascal's async implementation.

Next, it cites rather thought-out list of dos and don'ts, that actually applies in similar form to every concurrent application. Yes, the "GCN optimized" ones too.

Onto the quotes inside, and look, i won't waste much time parsing GPUView output, it's actually getting off-track, but i will question stuff done on the fly
Quote:
Time Spy has a Pre-Emption Packet(black rectangle) in the 3D Queue that shows up every time a compute queue is processed

I have heard confusing stuff about what GPUView actually tracks, so i'll attack on both fronts:

1) Where in D3D12 is it written down how to put pre-emption packet into compute queue, huh?

2) If it's not written down, and as such done by GPU itself on the whim (and is as such tracked by GPUView like that), what is the whole fuss about? It's definitely not Time Spy doing the job, but driver deciding pre-emption would be the approach here. Now, some mention excessive usage of fences and barriers, but well... they are needed at times, to avoid disasters. I'd know, that was part of my diploma (but in application to CPUs).
Quote:
Compute queues as a % of total run time:

Doom: 43.70%
AOTS: 90.45%
Time Spy: 21.38%

Yet, Oxide have claimed AotS spends only about 1/3rd of frametime on compute queue? Where da truth at?
Quote:
I bet they also want 1 engine with 1 path to run on all GPUs to make their Benchmark "valid", to them but it makes it invalid to me since it doesn't use each HW to it's maximum potential, be it NV or AMD or some other GPU.

Well, i forgot that another half of discussions on the matter were arguing precisely that. Well, what if i told you, that tech demo and benchmark are different things?
Quote:
Anyway, I can say that the logical conclusion from this is that Futuremark's benchmark is BOTCHED and biased, not indicative of DX12 capabilities as it should be, but instead restricting them - thus it has arguably no credibility as a BENCHMARK suite.
Quote:
Quotes about pre-emption

Half-correct, but miss the kernel of truth in the other half, could at least waste some time reading 2 paragraphs in Pascal whitepaper, right before pre-emption description, to know what is done, how it's done and why pre-emption improvements matter and where they matter.

Logical conclusion from reading the OP and quotes in it is that OP is BOTCHED and biased, and also lacks understanding of how async compute in Pascal works. Any understanding of it, actually.
post #232 of 253
Quote:
Originally Posted by criminal View Post

I think DX12 will make synthetic benchmarks pointless. Or like someone else said, have "features" test like 3dMark 2000(2001?) that won't run if the hardware doesn't support it.
The old way is for sure a better approach than this. I myself had one of these cards that did not support all tests but yet I did not lose any sleep or rage because they kept the test in for those who actually had a more capable card.
post #233 of 253
Basically any features that AMD shines in are unimportant and discarded in the name of fairness but features that Nvidia shine in like tessellation get touted as crucial to gamer experience and must be jacked up to 11 so AMD users can see for themselves just how crappy their cards really are! thumb.gif
post #234 of 253
The important part to me now is we are supposed to be okay with this since AMD and Nvidia are. Well i got news for AMD in this case. If they are happy with under performing gpu's due to dx12 API mainstream being path then I will gladly not use AMD anymore. I had enough of this with DX11 to last a lifetime. I don't see how one company is okay with under performing hardware due software and the other is happy with limiting a whole brand new API. If i had to guess Nvidia doesn't care about dx12 being limited right now, they love to capitalize on quick replacement intervals.

Right now i am really turned off by both brands if the dx12 future is mainstream like Timespy performance says. I am just super impressed with what Vulkan did with TSSAA and no fps loss. DX12 is looking more like a MS turd left in the sun to bake. I don't care about what's fair in a benchmark at this point, i do care though how bad dx12 is looking so far. I don't know why i am surprised MS is pretty good as sabotage.
post #235 of 253
Quote:
Originally Posted by Majin SSJ Eric View Post

Basically any features that AMD shines in are unimportant and discarded in the name of fairness but features that Nvidia shine in like tessellation get touted as crucial to gamer experience and must be jacked up to 11 so AMD users can see for themselves just how crappy their cards really are! thumb.gif

I guess the way I see it, would it be really be a "benchmark" if they coded a unique path for each gpu? Think about that for a second. How do you compare across AMD/Nvidia gpu's if they each have a unique path that they run? You really couldn't because the settings wouldn't be the same. The best we could hope for would be to compare to the same gpus across different systems. Basically I would only be able to compare my results against a 1070's since I would know I would beat anything 1060 and below and I couldn't really compete with anything 1080 and above. Maxwell or older gpus would be a wash and could only be compared to each other as well. And there would be no reason to compare against AMD cards since they run a different feature set all together.
Super P's rig
(20 items)
 
  
CPUMotherboardGraphicsRAM
5960x ASUS X99-A II Asus GTX 1080 Ti Corsair Vengeance DDR4 3000 
Hard DriveHard DriveHard DriveOptical Drive
MyDigitalSSD BPX NVMe Samsung 850 EVO Seagate Momentus XT 500 GB External DVDRW 
CoolingCoolingOSMonitor
EK-XLC Predator 240 Swiftech 240mm Radiator Windows 10 Samsung 40" 4K - UN40KU6290 
KeyboardPowerCaseMouse
G710+ EVGA SuperNOVA 850G2 Fractal Design Define S G700s 
Mouse PadAudioAudioAudio
Vipamz Extended XXXL Asus U7 M-Audio AV40 Sennheiser HD 439 
  hide details  
Reply
Super P's rig
(20 items)
 
  
CPUMotherboardGraphicsRAM
5960x ASUS X99-A II Asus GTX 1080 Ti Corsair Vengeance DDR4 3000 
Hard DriveHard DriveHard DriveOptical Drive
MyDigitalSSD BPX NVMe Samsung 850 EVO Seagate Momentus XT 500 GB External DVDRW 
CoolingCoolingOSMonitor
EK-XLC Predator 240 Swiftech 240mm Radiator Windows 10 Samsung 40" 4K - UN40KU6290 
KeyboardPowerCaseMouse
G710+ EVGA SuperNOVA 850G2 Fractal Design Define S G700s 
Mouse PadAudioAudioAudio
Vipamz Extended XXXL Asus U7 M-Audio AV40 Sennheiser HD 439 
  hide details  
Reply
post #236 of 253
post #237 of 253
Quote:
Originally Posted by criminal View Post

I guess the way I see it, would it be really be a "benchmark" if they coded a unique path for each gpu? Think about that for a second. How do you compare across AMD/Nvidia gpu's if they each have a unique path that they run? You really couldn't because the settings wouldn't be the same. The best we could hope for would be to compare to the same gpus across different systems. Basically I would only be able to compare my results against a 1070's since I would know I would beat anything 1060 and below and I couldn't really compete with anything 1080 and above. Maxwell or older gpus would be a wash and could only be compared to each other as well. And there would be no reason to compare against AMD cards since they run a different feature set all together.

Different path does not mean different features. The way TimeSpy works right now its like this. Lets make a simple example. AMD cards can handle 40C temps while Nvidia can handle 20C. Because of that TimeSpy to be neutral does not go over 20C. The problem there it can never be neutral.
Ishimura
(21 items)
 
Silent Knight
(13 items)
 
 
CPUMotherboardGraphicsRAM
Intel Core i7 3770K @ 4.6GHz ASRock Z77E-ITX eVGA GTX 1080 Ti Hybrid AVEXIR Blitz 1.1 16GB DDR3-2400MHz CL10 
Hard DriveHard DriveCoolingCooling
SanDisk Ultra II 960GB Toshiba X300 5TB Corsair H100i GTX eVGA Hybrid Water Cooler  
CoolingOSMonitorKeyboard
4x GentleTyphoon AP-15 Windows 10 Pro 64-Bit Philips Brilliance BDM4065UC 4K Razer BlackWidow Chroma  
PowerCaseMouseMouse Pad
eVGA SuperNOVA 750 G3 Define Nano S Logitech G502 Proteus Core PECHAM Gaming Mouse Pad XX-Large 
AudioAudioAudioAudio
Audioengine D1 DAC Mackie CR Series CR3 Audio-Technica ATH-M50 Sennheiser HD 598 
Audio
Sony XB950BT 
CPUMotherboardGraphicsRAM
AMD Phenom II X4 955 @ 4.2GHz ASUS M4A79XTD EVO AMD Radeon HD 7970 3GB @ 1200/1500 2x 4GB G.SKILL Ripjaws X DDR3-1600 
Hard DriveHard DriveHard DriveCooling
OCZ Agility 3 60GB WD Caviar Green 1.5TB 2 x Seagate Barracuda 2TB XSPC Raystorm 
CoolingCoolingOSPower
EK-FC7970 XSPC RS360 Windows 10 Pro 64-Bit Corsair TX750 
Case
NZXT Switch 810  
  hide details  
Reply
Ishimura
(21 items)
 
Silent Knight
(13 items)
 
 
CPUMotherboardGraphicsRAM
Intel Core i7 3770K @ 4.6GHz ASRock Z77E-ITX eVGA GTX 1080 Ti Hybrid AVEXIR Blitz 1.1 16GB DDR3-2400MHz CL10 
Hard DriveHard DriveCoolingCooling
SanDisk Ultra II 960GB Toshiba X300 5TB Corsair H100i GTX eVGA Hybrid Water Cooler  
CoolingOSMonitorKeyboard
4x GentleTyphoon AP-15 Windows 10 Pro 64-Bit Philips Brilliance BDM4065UC 4K Razer BlackWidow Chroma  
PowerCaseMouseMouse Pad
eVGA SuperNOVA 750 G3 Define Nano S Logitech G502 Proteus Core PECHAM Gaming Mouse Pad XX-Large 
AudioAudioAudioAudio
Audioengine D1 DAC Mackie CR Series CR3 Audio-Technica ATH-M50 Sennheiser HD 598 
Audio
Sony XB950BT 
CPUMotherboardGraphicsRAM
AMD Phenom II X4 955 @ 4.2GHz ASUS M4A79XTD EVO AMD Radeon HD 7970 3GB @ 1200/1500 2x 4GB G.SKILL Ripjaws X DDR3-1600 
Hard DriveHard DriveHard DriveCooling
OCZ Agility 3 60GB WD Caviar Green 1.5TB 2 x Seagate Barracuda 2TB XSPC Raystorm 
CoolingCoolingOSPower
EK-FC7970 XSPC RS360 Windows 10 Pro 64-Bit Corsair TX750 
Case
NZXT Switch 810  
  hide details  
Reply
post #238 of 253
Quote:
Originally Posted by ZealotKi11er View Post

Different path does not mean different features. The way TimeSpy works right now its like this. Lets make a simple example. AMD cards can handle 40C temps while Nvidia can handle 20C. Because of that TimeSpy to be neutral does not go over 20C. The problem there it can never be neutral.

It's really exposed itself to be a double edged sword. Damned if they do, damned if they don't.
Big Timmah
(13 items)
 
  
CPUMotherboardGraphicsRAM
Ryzen 5 1600 Asrock x370 Killer SLI/AC Sapphire Radeon Nitro Fury CORSAIR Vengeance LPX 16GB 3200mhz 
Hard DriveCoolingOSMonitor
PNY 480GB SSD PH-TC12DX Black Windows 10 Pro LG 29inch Ultrawide 
KeyboardPowerCaseMouse
Corsair K70 Thermaltake SMART M Series 850W NZXT S340 White Steel ATX Mid Tower Case Wireless Logitech thing 
Mouse Pad
With a supple pad  
  hide details  
Reply
Big Timmah
(13 items)
 
  
CPUMotherboardGraphicsRAM
Ryzen 5 1600 Asrock x370 Killer SLI/AC Sapphire Radeon Nitro Fury CORSAIR Vengeance LPX 16GB 3200mhz 
Hard DriveCoolingOSMonitor
PNY 480GB SSD PH-TC12DX Black Windows 10 Pro LG 29inch Ultrawide 
KeyboardPowerCaseMouse
Corsair K70 Thermaltake SMART M Series 850W NZXT S340 White Steel ATX Mid Tower Case Wireless Logitech thing 
Mouse Pad
With a supple pad  
  hide details  
Reply
post #239 of 253
Quote:
Originally Posted by criminal View Post

I guess the way I see it, would it be really be a "benchmark" if they coded a unique path for each gpu? Think about that for a second. How do you compare across AMD/Nvidia gpu's if they each have a unique path that they run? You really couldn't because the settings wouldn't be the same. The best we could hope for would be to compare to the same gpus across different systems. Basically I would only be able to compare my results against a 1070's since I would know I would beat anything 1060 and below and I couldn't really compete with anything 1080 and above. Maxwell or older gpus would be a wash and could only be compared to each other as well. And there would be no reason to compare against AMD cards since they run a different feature set all together.

It's already not fair. Why do I have to see my hardware not fully used because other hardware can not do what mine can ? I'm pretty sure they can come up with a score system to compensate for the differences. Time Spy is useless. They should continue using dx11 biggrin.gif
Intel Evilnow
(18 items)
 
   
CPUMotherboardGraphicsRAM
i5 2500k 4ghz @ Offset -0.015 Asus P8P67 Evo (bios 3207) Sapphire 280x Tri-x 3GB OC (Stock 1020/1500 Non... G.Skill RipjawsX 2x4gb 1866mhz 9-10-9-28-2n @ 1.5v 
Hard DriveHard DriveHard DriveHard Drive
SHSS37A120G WD5000AAKX-001CA0 WD20EARX WD20EZRZ 
Hard DriveOptical DriveCoolingOS
WD5001AALS-00L3B2 (Now External) ASUS DRW-1814BLT Noctua NH-u12p SE2 Windows 10 Pro 
MonitorKeyboardPowerCase
Asus VH242H Wobbly Stand :) Microsoft Ergo 4000 Enermax Infiniti 650 (28a,28a,30a) Cooler Master haf 912 Advanced 
MouseOther
A4tech x7 F3 Sunbeam RHK-EX-BA Rheobus-Extreme Fan Controlle... 
CPUMotherboardGraphicsRAM
Phenom II x6 1090t BE 3.6/4.0 Turbo@def.volt MSI K9A2 Platinum v1 Sapphire HD6850 1GB 850/1100@def.volt Kingston 2x2gb Hyperx 1066 5-5-5-15 
Hard DriveHard DriveOptical DriveOS
Western Digital WD5001AALS Seagate Barracuda ST3250410AS Asus DRW-1814BLT Windows 7 Ultimate x64 SP1 
MonitorKeyboardPowerCase
Asus VH242H 23.6" Wobbly Stand :D Microsoft Ergo 4000 Enermax Infiniti 650w (28a,28a,30a) Thermaltake Kandalf SuperTower 
Mouse
A4 tech Swop-3 
  hide details  
Reply
Intel Evilnow
(18 items)
 
   
CPUMotherboardGraphicsRAM
i5 2500k 4ghz @ Offset -0.015 Asus P8P67 Evo (bios 3207) Sapphire 280x Tri-x 3GB OC (Stock 1020/1500 Non... G.Skill RipjawsX 2x4gb 1866mhz 9-10-9-28-2n @ 1.5v 
Hard DriveHard DriveHard DriveHard Drive
SHSS37A120G WD5000AAKX-001CA0 WD20EARX WD20EZRZ 
Hard DriveOptical DriveCoolingOS
WD5001AALS-00L3B2 (Now External) ASUS DRW-1814BLT Noctua NH-u12p SE2 Windows 10 Pro 
MonitorKeyboardPowerCase
Asus VH242H Wobbly Stand :) Microsoft Ergo 4000 Enermax Infiniti 650 (28a,28a,30a) Cooler Master haf 912 Advanced 
MouseOther
A4tech x7 F3 Sunbeam RHK-EX-BA Rheobus-Extreme Fan Controlle... 
CPUMotherboardGraphicsRAM
Phenom II x6 1090t BE 3.6/4.0 Turbo@def.volt MSI K9A2 Platinum v1 Sapphire HD6850 1GB 850/1100@def.volt Kingston 2x2gb Hyperx 1066 5-5-5-15 
Hard DriveHard DriveOptical DriveOS
Western Digital WD5001AALS Seagate Barracuda ST3250410AS Asus DRW-1814BLT Windows 7 Ultimate x64 SP1 
MonitorKeyboardPowerCase
Asus VH242H 23.6" Wobbly Stand :D Microsoft Ergo 4000 Enermax Infiniti 650w (28a,28a,30a) Thermaltake Kandalf SuperTower 
Mouse
A4 tech Swop-3 
  hide details  
Reply
post #240 of 253
Quote:
Originally Posted by criminal View Post

I guess the way I see it, would it be really be a "benchmark" if they coded a unique path for each gpu? Think about that for a second. How do you compare across AMD/Nvidia gpu's if they each have a unique path that they run? You really couldn't because the settings wouldn't be the same. The best we could hope for would be to compare to the same gpus across different systems. Basically I would only be able to compare my results against a 1070's since I would know I would beat anything 1060 and below and I couldn't really compete with anything 1080 and above. Maxwell or older gpus would be a wash and could only be compared to each other as well. And there would be no reason to compare against AMD cards since they run a different feature set all together.

There's a very important principle to software development: coverage -- how much of the written code is actually executed. Like-wise, for a benchmark to be useful, it must also exercise hardware feature coverage. Regarding scoring, the overall metric should still be fps.

Take the following 2 implementations of a benchmark:


Top implementation only exercises common features of both hardware. It doesn't have good hardware coverage.
Bottom implementation exercises unique features of both hardware and provides optimal hardware coverage.
Frame-per-frame, both code paths still have to render exactly the same image.

Of course the problem becomes the issue of balance between vendor strengths. How do you prevent one vendor's features from being too heavily weighted in a given benchmark? NVidia's gross over-tessellation in Crysis 2 comes to mind... most likely there needs to be a committee to overlook this balance to establish a baseline. The base criteria can be that the demo achieves 30fps @ 1080p with acceptable artistic results by the committee (and may be by extension the development communities). From there, the hardware is (and drivers are) free to improve upon its specific strengths or embrace those of its competitors. Kronos comes to mind, when considering a committee to determine the baseline balance. Of course, this is all brain-storming... because we need hardware coverage.

It's a good problem to have because it shows that the producers are taking risks to appease us the customers with affordable innovations, rather than expensive refinements. Even JHH said in a presentation to Stanford students that it was NVidia's adaptation to change that allowed them to survive the 90's where everyone was refining fixed-function pipelines. Ever since then to now, change is long over due because we have >$1000 gaming GPUs still refining on aging synchronous computing hardware.
New Posts  All Forums:Forum Nav:
  Return Home
Overclock.net › Forums › Benchmarks › Benchmarking Software and Discussion › [Various] Futuremark's Time Spy DirectX 12 "Benchmark" Compromised. Less Compute/Parallelism than Doom/Aots. Also...