Overclock.net › Forums › Industry News › Software News › [ExtremeTech/PCPer] Futuremark responds to accusations of bias in new DirectX 12 Time Spy test
New Posts  All Forums:Forum Nav:

[ExtremeTech/PCPer] Futuremark responds to accusations of bias in new DirectX 12 Time Spy test

post #1 of 13
Thread Starter 
Quote:
Last week, Futuremark released Time Spy, a new DirectX 12 benchmark that takes full advantage of DX12’s features and capabilities, including asynchronous compute. While the release of a new benchmark is typically of modest interest, there’s been a great deal of confusion, uncertainty, and doubt over Time Spy’s benchmark results and what those results mean. Futuremark has since published an updated and expanded guide to how the benchmark functions and what it’s designed to do

Revisiting asynchronous compute
The debate over whether Time Spy is a valid benchmark, and the questions regarding its implementation of asynchronous compute, speak to a significant amount of confusion in the user community about what asynchronous compute is, how it works, and how it can or should be used in DirectX 12..

Source http://www.extremetech.com/gaming/232122-lies-damned-lies-and-benchmarks-futuremark-responds-to-accusations-of-bias-in-new-directx-12-time-spy-test
Quote:
Update 2 - July 20th @ 6:50pm EDT: We interviewed Jani Joki, Futuremark's Director of Engineering, on our YouTube page. The interview is embed just below this update.

Original post below

The comments of a previous post notified us of an Overclock.net thread, whose author claims that 3DMark's implementation of asynchronous compute is designed to show NVIDIA in the best possible light. At the end of the linked post, they note that asynchronous compute is a general blanket, and that we should better understand what is actually going on.

Source
Edited by PontiacGTX - 7/21/16 at 9:33am
post #2 of 13
First time that joel Hruska releases such a bad article. He even start posting here about the subject .

Time spy looks like it was a dx11 benchmark on development before dx12 and FM wanted to use the hype of dx12 to sell.

I dont think there is a bias on 3dmark that much but their decision to actually follow a way more neutral code path and use the baseline features of dx11/dx12 and just add a bit of async compute only will have a bias towards nvidia gpu.

Even with firestrike on the gpu tests the cpu bottleneck from dx11 was almost irrelevant. That's why they had a combined test.

Now with dx12 they dont have a combined test to actually see dx12 in action.

Also from the comments in this article the HWS units in fiji and polaris can deal with more than just graphics or compute like audio for vr .


Still 3D Mark is a benchmark and all this drama is not necessary. But i would love to see a benchmark that actually push all the new features from vulkan and dx12 and push our hardware to the max.

I think it is an excuse from FM PR that they wanted to have a neutral code path. The nature of dx11 is to have a neutral code and then let the IHVs through the drivers to have specific fixes and optimizations . With dx12 you cant do that actually the developer must do the work of the IHVs now.
Edited by sugarhell - 7/21/16 at 9:46am
Workstation
(4 items)
 
  
CPUMotherboardGraphicsMonitor
Xeon E5-2690 Supermicro 2011 Nvidia GP100/ Vega FE Dell ultrasharp 4k 
  hide details  
Reply
Workstation
(4 items)
 
  
CPUMotherboardGraphicsMonitor
Xeon E5-2690 Supermicro 2011 Nvidia GP100/ Vega FE Dell ultrasharp 4k 
  hide details  
Reply
post #3 of 13
Quote:
Originally Posted by sugarhell View Post

But i would love to see a benchmark that actually push all the new features from vulkan and dx12 and push our hardware to the max.
Me too, but that is not Futuremark/UL's business target.
post #4 of 13
It's been mentioned before but I think it's still a valid point. FM cannot release a new DX12 in which AMD comes out the victor, its simply bad for business.
Big Timmah
(13 items)
 
  
CPUMotherboardGraphicsRAM
Ryzen 5 1600 Asrock x370 Killer SLI/AC Sapphire Radeon Nitro Fury CORSAIR Vengeance LPX 16GB 3200mhz 
Hard DriveCoolingOSMonitor
PNY 480GB SSD PH-TC12DX Black Windows 10 Pro LG 29inch Ultrawide 
KeyboardPowerCaseMouse
Corsair K70 Thermaltake SMART M Series 850W NZXT S340 White Steel ATX Mid Tower Case Wireless Logitech thing 
Mouse Pad
With a supple pad  
  hide details  
Reply
Big Timmah
(13 items)
 
  
CPUMotherboardGraphicsRAM
Ryzen 5 1600 Asrock x370 Killer SLI/AC Sapphire Radeon Nitro Fury CORSAIR Vengeance LPX 16GB 3200mhz 
Hard DriveCoolingOSMonitor
PNY 480GB SSD PH-TC12DX Black Windows 10 Pro LG 29inch Ultrawide 
KeyboardPowerCaseMouse
Corsair K70 Thermaltake SMART M Series 850W NZXT S340 White Steel ATX Mid Tower Case Wireless Logitech thing 
Mouse Pad
With a supple pad  
  hide details  
Reply
post #5 of 13
Who cares? Everyone who says that synthetic benchmarks don't matter should stick to their word and go about their business instead of whipping up drama.
First Build
(10 items)
 
  
CPUMotherboardGraphicsRAM
i3-4370 ASROCK H97M PR04 380x Nitro Corsair 8GB 
Hard DriveHard DriveHard DriveOS
Crucial M500 120GB Seagage 320GB Caviar Blue 1TB Windows 8.1 x86-64 
PowerCase
XFX 550W PS07B 
  hide details  
Reply
First Build
(10 items)
 
  
CPUMotherboardGraphicsRAM
i3-4370 ASROCK H97M PR04 380x Nitro Corsair 8GB 
Hard DriveHard DriveHard DriveOS
Crucial M500 120GB Seagage 320GB Caviar Blue 1TB Windows 8.1 x86-64 
PowerCase
XFX 550W PS07B 
  hide details  
Reply
post #6 of 13
Quote:
Originally Posted by tp4tissue View Post

Dude,, it's just the stupid people who buy Nvidia.. And they can't handle the reality that their GPU is NOT future compatible / properly designed for Async Compute.

Like, that's it, it's a bunch of butthurt Nvidia people who made the wrong decision to buy Nvidia.. biggrin.gif
nvidia IS the future, nvidia MAKES the future wink.gif
The Green Beast
(16 items)
 
  
CPUMotherboardGraphicsRAM
Intel i7 3770K @ 4500 Mhz ASRock Z77 Pro3 Gigabyte GTX 1080 G1 Gaming Crucial Ballistix 2x8GB DDR3-1600 
Hard DriveHard DriveHard DriveCooling
SSD Crucial M550 500GB SSD Samsung 850 Evo 1TB HDD Seagate 7200rpm 3TB Cooler Master Hyper 212 EVO; Xilence X5 
OSMonitorKeyboardPower
Windows 10 Pro 64-bit Acer Predator XB271HU 27" IPS Gsync 1440p 165Hz CM Storm QuickFire XT Cherry Red 800W modular 
CaseMouseMouse PadAudio
Fractal Design Define R4 Black Logitech G900 Chaos Spectrum SteelSeries QcK+ 4mm SK Gaming Realtek On-board 
  hide details  
Reply
The Green Beast
(16 items)
 
  
CPUMotherboardGraphicsRAM
Intel i7 3770K @ 4500 Mhz ASRock Z77 Pro3 Gigabyte GTX 1080 G1 Gaming Crucial Ballistix 2x8GB DDR3-1600 
Hard DriveHard DriveHard DriveCooling
SSD Crucial M550 500GB SSD Samsung 850 Evo 1TB HDD Seagate 7200rpm 3TB Cooler Master Hyper 212 EVO; Xilence X5 
OSMonitorKeyboardPower
Windows 10 Pro 64-bit Acer Predator XB271HU 27" IPS Gsync 1440p 165Hz CM Storm QuickFire XT Cherry Red 800W modular 
CaseMouseMouse PadAudio
Fractal Design Define R4 Black Logitech G900 Chaos Spectrum SteelSeries QcK+ 4mm SK Gaming Realtek On-board 
  hide details  
Reply
post #7 of 13
The Future is DX11 in this case /s
post #8 of 13
Quote:
Originally Posted by Greenland View Post

The Future is DX11 in this case /s

I suggest you read the threads title.
MyCleanPC
(13 items)
 
  
CPUMotherboardGraphicsRAM
i7 4770k ASUS MAXIMUS VI EVGA master blaster Corsair Vengence  
Hard DriveOptical DriveOSMonitor
OCZ SSD raid0 samsung Win 7 Samsung 
PowerCase
Enermax rev 1050 Stacker 832 
  hide details  
Reply
MyCleanPC
(13 items)
 
  
CPUMotherboardGraphicsRAM
i7 4770k ASUS MAXIMUS VI EVGA master blaster Corsair Vengence  
Hard DriveOptical DriveOSMonitor
OCZ SSD raid0 samsung Win 7 Samsung 
PowerCase
Enermax rev 1050 Stacker 832 
  hide details  
Reply
post #9 of 13
Quote:
Originally Posted by DrFPS View Post

Quote:
Originally Posted by Greenland View Post

The Future is DX11 in this case /s

I suggest you read the threads title.

DX 11 sprinkled with politically correct DX 12 bread crumbs then?
 
Metro 2033 review
Metro 2033
CPUMotherboardGraphicsRAM
Core i7-3820 Asus Sabertooth X79 MSI GTX 1060 6 GB Gaming X 16 GB Corsair DDR3 1866 Mhz Dominator 
Hard DriveOptical DriveCoolingOS
Samsung SSD 830 128GB + WD Caviar Black 2TB Sony Optiarc DVD-RW Corsair A70 + Noiseblocker M12-P Windows 7 Home Premium 64-bit 
MonitorKeyboardPowerCase
BenQ RL2455HM Cooler Master Octane Corsair AX750 Professional Modular 80 Plus Gold Cooler Master HAF 912 Plus 
Mouse
Cooler Master Octane 
  hide details  
Reply
 
Metro 2033 review
Metro 2033
CPUMotherboardGraphicsRAM
Core i7-3820 Asus Sabertooth X79 MSI GTX 1060 6 GB Gaming X 16 GB Corsair DDR3 1866 Mhz Dominator 
Hard DriveOptical DriveCoolingOS
Samsung SSD 830 128GB + WD Caviar Black 2TB Sony Optiarc DVD-RW Corsair A70 + Noiseblocker M12-P Windows 7 Home Premium 64-bit 
MonitorKeyboardPowerCase
BenQ RL2455HM Cooler Master Octane Corsair AX750 Professional Modular 80 Plus Gold Cooler Master HAF 912 Plus 
Mouse
Cooler Master Octane 
  hide details  
Reply
post #10 of 13
While Id like to see a benchmark that has the option of a neutral code path or fully optimized option for your choice of vender that's unlikely to happen ever since it would be too much work.
So to me the only real use of synthetic benchmarks is to compare results when your overclocking a card against one of a similar architect to see if your making gains.

The best benchmark will always be look at the game you want to play and buy the card that gives you the best value for your budget for that game.
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Software News
Overclock.net › Forums › Industry News › Software News › [ExtremeTech/PCPer] Futuremark responds to accusations of bias in new DirectX 12 Time Spy test