Overclock.net › Forums › Industry News › Hardware News › [Various] Futuremark Releases 3DMark Time Spy DirectX 12 Benchmark
New Posts  All Forums:Forum Nav:

[Various] Futuremark Releases 3DMark Time Spy DirectX 12 Benchmark - Page 52

post #511 of 772
Quote:
Originally Posted by JackCY View Post

Let's face it we need an OCN benchmark to be happy tongue.gif One that supports all the available features for all the major architectures.

The last thing people want is a benchmark that could be proven to be an unbiased and fair comparison. If we had that we wouldn't be able to argue over the validity of the results.
Computer
(14 items)
 
  
CPUMotherboardGraphicsRAM
i7-4790k Gigabyte Z97X Gaming 7 Nvidia GTX 1080ti FE (EK block) CORSAIR Vengeance 
Hard DriveHard DriveCoolingOS
Samsung 850 Evo Mushkin Reactor Custom Loop Windows 10 
MonitorKeyboardPowerCase
Ben Q BL3201PT Logitech G110 EVGA 1300w Thermatake Core P5 
MouseAudio
LOGITECH G502 PROTEUS CORE Corsair 2100 
  hide details  
Reply
Computer
(14 items)
 
  
CPUMotherboardGraphicsRAM
i7-4790k Gigabyte Z97X Gaming 7 Nvidia GTX 1080ti FE (EK block) CORSAIR Vengeance 
Hard DriveHard DriveCoolingOS
Samsung 850 Evo Mushkin Reactor Custom Loop Windows 10 
MonitorKeyboardPowerCase
Ben Q BL3201PT Logitech G110 EVGA 1300w Thermatake Core P5 
MouseAudio
LOGITECH G502 PROTEUS CORE Corsair 2100 
  hide details  
Reply
post #512 of 772
Quote:
Originally Posted by bigjdubb View Post


The last thing people want is a benchmark that could be proven to be an unbiased and fair comparison. If we had that we wouldn't be able to argue over the validity of the results.

 

:lachen:too true, too true!

     
CPUMotherboardGraphicsRAM
AMD Ryzen 7 1700X @ 4.0 GHz 1.37v o/s  Biostar X370 GT7 Sapphire Radeon RX Vega 64 8GB HBM2 G.SKILL TridentZ RGB Series 16GB (2 x 8GB) @ 32... 
Hard DriveHard DriveHard DriveHard Drive
Biostar M200-240 240GB m.2 SATA SSD (Win) Samsung 850 Pro 256GB SSD (Linux) SanDisk Ultra II 480GB SSD (Games) Seagate 2TB SSHD (Storage) 
Optical DriveCoolingCoolingCooling
LG DVD RW Swiftech H220-X2 Swiftech Komodo RX-ECO Vega Water Block XSPC EX240 Rad 
CoolingOSOSMonitor
Gentle Typhoon 120mm x4 Ubuntu 17.10 w/ custom kernel Windows 10 Pro Pixio PX277 27" 1440p 144Hz DP-Adaptive Sync w... 
KeyboardPowerCaseMouse
JAR-OWL J1 Rainbow LED Mechanical SeaSonic X-Series 1050w Gold+ Corsair 760T Zowie FK1 
Mouse PadAudioAudioAudio
Razer Firefly Plantronics Rig 500e Gaming Headset Marantz Pro MPH-1 Monitoring Headphones S.M.S.L SD-793II Audio DAC + Headphone Amplifier 
Other
Xbox One Controller 
CPUMotherboardGraphicsRAM
Intel Core i7-4790k @ 4.6GHz Asrock Z97 Extreme6 EVGA GTX 1070 SC Corsair Vengeance 16GB (2x8GB) @ 2133 10-10-10-... 
Hard DriveHard DriveOptical DriveCooling
Intel 530 Series 480GB SSD Seagate 2TB SSHD LG DVD RW Cooler Master Hyper 212 Plus 
OSMonitorKeyboardPower
Windows 10 40" Vizio HDTV HP Generic EVGA SuperNOVA NEX 650w 
CaseMouseAudio
Corsair Carbide 300R Logitech Generic HyperX Cloud 
CPUMotherboardGraphicsRAM
AMD FX-6300 @ 4.3GHz 1.35v Biostar TA970 v5.3 MSI HD 6950 2GB G.Skill Ripjaws Z 16GB (4x4GB) 1600 
CoolingCoolingCoolingOS
Cryorig H5 Ultimate Corsair AF120 (White LED) x3 Corsair AF140 (White LED) x2 Windows 10 Pro 
MonitorKeyboardPowerCase
Dell S2440L Microsoft Ergonomic 4000 EVGA B2 850w Corsair 300R 
Mouse
SteelSeries Rival 300 
  hide details  
Reply
     
CPUMotherboardGraphicsRAM
AMD Ryzen 7 1700X @ 4.0 GHz 1.37v o/s  Biostar X370 GT7 Sapphire Radeon RX Vega 64 8GB HBM2 G.SKILL TridentZ RGB Series 16GB (2 x 8GB) @ 32... 
Hard DriveHard DriveHard DriveHard Drive
Biostar M200-240 240GB m.2 SATA SSD (Win) Samsung 850 Pro 256GB SSD (Linux) SanDisk Ultra II 480GB SSD (Games) Seagate 2TB SSHD (Storage) 
Optical DriveCoolingCoolingCooling
LG DVD RW Swiftech H220-X2 Swiftech Komodo RX-ECO Vega Water Block XSPC EX240 Rad 
CoolingOSOSMonitor
Gentle Typhoon 120mm x4 Ubuntu 17.10 w/ custom kernel Windows 10 Pro Pixio PX277 27" 1440p 144Hz DP-Adaptive Sync w... 
KeyboardPowerCaseMouse
JAR-OWL J1 Rainbow LED Mechanical SeaSonic X-Series 1050w Gold+ Corsair 760T Zowie FK1 
Mouse PadAudioAudioAudio
Razer Firefly Plantronics Rig 500e Gaming Headset Marantz Pro MPH-1 Monitoring Headphones S.M.S.L SD-793II Audio DAC + Headphone Amplifier 
Other
Xbox One Controller 
CPUMotherboardGraphicsRAM
Intel Core i7-4790k @ 4.6GHz Asrock Z97 Extreme6 EVGA GTX 1070 SC Corsair Vengeance 16GB (2x8GB) @ 2133 10-10-10-... 
Hard DriveHard DriveOptical DriveCooling
Intel 530 Series 480GB SSD Seagate 2TB SSHD LG DVD RW Cooler Master Hyper 212 Plus 
OSMonitorKeyboardPower
Windows 10 40" Vizio HDTV HP Generic EVGA SuperNOVA NEX 650w 
CaseMouseAudio
Corsair Carbide 300R Logitech Generic HyperX Cloud 
CPUMotherboardGraphicsRAM
AMD FX-6300 @ 4.3GHz 1.35v Biostar TA970 v5.3 MSI HD 6950 2GB G.Skill Ripjaws Z 16GB (4x4GB) 1600 
CoolingCoolingCoolingOS
Cryorig H5 Ultimate Corsair AF120 (White LED) x3 Corsair AF140 (White LED) x2 Windows 10 Pro 
MonitorKeyboardPowerCase
Dell S2440L Microsoft Ergonomic 4000 EVGA B2 850w Corsair 300R 
Mouse
SteelSeries Rival 300 
  hide details  
Reply
post #513 of 772
Quote:
Originally Posted by bigjdubb View Post

The last thing people want is a benchmark that could be proven to be an unbiased and fair comparison. If we had that we wouldn't be able to argue over the validity of the results.

You're not thinking hard/creative enough.

b-b-but this bench was developed on [insert company]'s GPU so how could it not be biased?

benchmark creator is known to have [insert company] bias, so benchmark is invalid

wait for drivers

who cares it's a benchmark not an actual game I mean do you play 3DMark all day?

wait for updated drivers

b-b-but excessive tessellation!

b-b-but excessive async!

i just like Huang's jacket better

we all bleed red
post #514 of 772
Quote:
Originally Posted by dagget3450 View Post

From how i understood it if you can crossfire or sli it then thats all the multigpu action you will get.

Then you clearly dont understand it at all. Please read the link i left. It explains it better and has other sources to read more about EMA options and iGPU working with dGPU.

I realized after the fact you may be talking about Time Spy having a software limitation to do what i ask. In that case thanks for the info. I will try and verify.

thumb.gif
Edited by gapottberg - 7/18/16 at 2:43pm
post #515 of 772
"The last thing people want is a benchmark that could be proven to be an unbiased and fair comparison."

I remember reading about some sociological experiments when I was in college -- and that was long enough ago that I do not pretend to recall all the details with 100% clarity, so consider yourself forewarned. wink.gif

The point of these experiments was to assess how people evaluated competitive situations. They showed football fans identical taped broadcasts of the same football game. Importantly, the people watching the game were fans of one team or another. So let's say this was a Bears vs. Packers game -- they got a group of Green Bay fans and a group of Bears fans together in separate rooms and showed them the same broadcast of the same game.

Here's where it gets interesting. As I recall, both sets of fans reported that the their team had been unfairly penalized. The refs had missed important and obvious calls and the result was a situation that was biased against their own team. Obviously this cannot literally be true -- the game cannot have been unfairly tilted against both teams simultaneously. What the researchers concluded was that in competitive situations our own ability to evaluate objective fairness ceases to exist. The more you care about the outcome (or have predisposed notions about what that outcome should be) the less likely you are to actually evaluate the situation objectively, whether you try to or not.

It's not hard to see this in action. To use a related but distinct example, some people believe that Intel's use of compiler optimizations and rebate practices in the early and mid-2000s constituted an abuse of Intel's market position and monopoly power. Others have argued that this was nothing more than tough competition and that AMD should've sucked it up and created its own compilers and software stack if it wanted to compete effectively. In the former view, AMD was prevented from gaining the market share it should've held by Intel's abuses. In the latter, Intel earned its position at the top of the market by creating a superior product. If it then used its superior position to reinforce and hold on to market dominance, well, that's the "fair" result of investing so much in its own software ecosystem. Where you stand on this question depends on what you think "fair" means in this context.

Part of the reason NV has taken some heat over DX12 is because its DirectX 12 performance is largely flat compared to DX11, whereas AMD's performance often improves significantly. The flip side to this equation is that NV has obviously done a better job of squeezing all of the available performance out of its hardware using the previous-generation API. Is it fair to attack Nvidia for failing to improve DX12 performance if its DX11 performance is already excellent (and remains excellent by comparison?) How do you weight the benefit of improved performance on AMD hardware against the flat-but-still-good performance of NV in DX11 and DX12?

That's the problem with fairness. It's never as objective as we hope -- and when people care strongly about the conclusion their ability to measure whether or not something is fair goes right out the window.
post #516 of 772
The best solution is to create a dynamic scene that use most of the new features of dx12. Then without altering the image quality or the effects of the scene there should be 2 specific hardware paths with the logic to use all the strong points of each architecture. With this way you can compare with the same scene which architecture or gpu is the fastest while providing the same Image quality.

This is the Neutral approach.
Workstation
(4 items)
 
  
CPUMotherboardGraphicsMonitor
Xeon E5-2690 Supermicro 2011 Nvidia GP100/ Vega FE Dell ultrasharp 4k 
  hide details  
Reply
Workstation
(4 items)
 
  
CPUMotherboardGraphicsMonitor
Xeon E5-2690 Supermicro 2011 Nvidia GP100/ Vega FE Dell ultrasharp 4k 
  hide details  
Reply
post #517 of 772
Quote:
Originally Posted by DigiHound View Post


To use a related but distinct example, some people believe that Intel's use of compiler optimizations and rebate practices in the early and mid-2000s constituted an abuse of Intel's market position and monopoly power. Others have argued that this was nothing more than tough competition and that AMD should've sucked it up and created its own compilers and software stack if it wanted to compete effectively. In the former view, AMD was prevented from gaining the market share it should've held by Intel's abuses. In the latter, Intel earned its position at the top of the market by creating a superior product. If it then used its superior position to reinforce and hold on to market dominance, well, that's the "fair" result of investing so much in its own software ecosystem. Where you stand on this question depends on what you think "fair" means in this context.

Pretty sure the courts found this example to be in violation of anti-trust laws and Intel was fined over a billion dollars...
Tsar Bomba
(12 items)
 
Fat Man
(13 items)
 
Little Boy
(29 items)
 
CPUMotherboardRAMHard Drive
Intel Core i7-6700K ASRock Fatal1ty Z170 Gaming-ITX/ac Crucial Ballistix Sport LT 32GB (2 x 16GB) DDR4... Samsung 850 EVO 500GB M.2-2280 SSD 
Hard DriveCoolingCoolingOS
Sandisk Ultra II 960GB 2.5" SSD Zalman CNPS8900 Quiet Noctua NF-F12 PWM Microsoft Windows 10 Pro 
KeyboardPowerCaseMouse
Corsair K95 Corsair SF 600W Silverstone SG05BB-LITE Corsair M65 
CPUMotherboardGraphicsGraphics
AMD A10-7850K GIGABYTE GA-F2A88X-UP4 XFX R9-290A-ENBC Radeon R9 290 XFX R9-290A-ENBC Radeon R9 290 
Hard DriveHard DriveCoolingCooling
Samsung 830 128GB Western Digital Blue 1TB XSPC Raystorm CPU/APU Waterblock - AMD XSPC D5 Photon 170 Reservoir/Pump Combo 
CoolingCoolingCoolingPower
XSPC AX360 Radiator XSPC AX240 Radiator 2x XSPC Razor R9 290X / 290 - Full Cover Water ... EVGA Supernova 1000 P2 
Case
Corsair Carbide Series Air 540 Black  
CPUMotherboardGraphicsGraphics
Intel Core i7-4770K  Asus Maximus VI Extreme Sapphire R9 290 unlocked to 290X Sapphire R9 290 unlocked to 290X 
GraphicsGraphicsRAMHard Drive
Sapphire R9 290 unlocked to 290X Sapphire R9 290 unlocked to 290X G.SKILL Sniper Series 32GB (4 x 8GB) DDR3 2400 2x SAMSUNG 840 EVO 250GB 
Hard DriveOptical DriveCoolingCooling
2x TOSHIBA PH3300U-1I72 3TB Asus BW-12B1ST/BLK/G/AS Blu-Ray/DVD/CD Writer 29x Noctua NF-F12 PWM 2x Swiftech MCP655 + Bitspower Dual D5 Mod Top 
CoolingCoolingCoolingCooling
Phobya Balancer 450 Silver Nickel Reservoir 2x Alphacool NexXxoS XT45 480mm Radiator Alphacool NexXxoS XT45 360mm Radiator Alphacool NexXxoS XT45 240mm Radiator 
CoolingCoolingCoolingCooling
Alphacool NexXxoS ST30 240mm Radiator Alphacool NexXxoS XT45 120mm Radiator 4x Koolance VID-AR290X Radeon VGA Liquid Coolin... EK Supreme LTX Intel CPU Liquid Cooling Block -... 
OSMonitorMonitorMonitor
Microsoft Windows 8.1 Pro Asus VS238H-P 23.0" Asus VS238H-P 23.0" Asus VS238H-P 23.0" 
KeyboardPowerCaseMouse
Corsair Vengeance K95 ENERMAX Maxrevo EMR1500EWT 1500W Corsair Obsidian Series 900D Razer DeathAdder 2013 
Audio
Astro Gaming A50 Circumaural Headset - Black 
  hide details  
Reply
Tsar Bomba
(12 items)
 
Fat Man
(13 items)
 
Little Boy
(29 items)
 
CPUMotherboardRAMHard Drive
Intel Core i7-6700K ASRock Fatal1ty Z170 Gaming-ITX/ac Crucial Ballistix Sport LT 32GB (2 x 16GB) DDR4... Samsung 850 EVO 500GB M.2-2280 SSD 
Hard DriveCoolingCoolingOS
Sandisk Ultra II 960GB 2.5" SSD Zalman CNPS8900 Quiet Noctua NF-F12 PWM Microsoft Windows 10 Pro 
KeyboardPowerCaseMouse
Corsair K95 Corsair SF 600W Silverstone SG05BB-LITE Corsair M65 
CPUMotherboardGraphicsGraphics
AMD A10-7850K GIGABYTE GA-F2A88X-UP4 XFX R9-290A-ENBC Radeon R9 290 XFX R9-290A-ENBC Radeon R9 290 
Hard DriveHard DriveCoolingCooling
Samsung 830 128GB Western Digital Blue 1TB XSPC Raystorm CPU/APU Waterblock - AMD XSPC D5 Photon 170 Reservoir/Pump Combo 
CoolingCoolingCoolingPower
XSPC AX360 Radiator XSPC AX240 Radiator 2x XSPC Razor R9 290X / 290 - Full Cover Water ... EVGA Supernova 1000 P2 
Case
Corsair Carbide Series Air 540 Black  
CPUMotherboardGraphicsGraphics
Intel Core i7-4770K  Asus Maximus VI Extreme Sapphire R9 290 unlocked to 290X Sapphire R9 290 unlocked to 290X 
GraphicsGraphicsRAMHard Drive
Sapphire R9 290 unlocked to 290X Sapphire R9 290 unlocked to 290X G.SKILL Sniper Series 32GB (4 x 8GB) DDR3 2400 2x SAMSUNG 840 EVO 250GB 
Hard DriveOptical DriveCoolingCooling
2x TOSHIBA PH3300U-1I72 3TB Asus BW-12B1ST/BLK/G/AS Blu-Ray/DVD/CD Writer 29x Noctua NF-F12 PWM 2x Swiftech MCP655 + Bitspower Dual D5 Mod Top 
CoolingCoolingCoolingCooling
Phobya Balancer 450 Silver Nickel Reservoir 2x Alphacool NexXxoS XT45 480mm Radiator Alphacool NexXxoS XT45 360mm Radiator Alphacool NexXxoS XT45 240mm Radiator 
CoolingCoolingCoolingCooling
Alphacool NexXxoS ST30 240mm Radiator Alphacool NexXxoS XT45 120mm Radiator 4x Koolance VID-AR290X Radeon VGA Liquid Coolin... EK Supreme LTX Intel CPU Liquid Cooling Block -... 
OSMonitorMonitorMonitor
Microsoft Windows 8.1 Pro Asus VS238H-P 23.0" Asus VS238H-P 23.0" Asus VS238H-P 23.0" 
KeyboardPowerCaseMouse
Corsair Vengeance K95 ENERMAX Maxrevo EMR1500EWT 1500W Corsair Obsidian Series 900D Razer DeathAdder 2013 
Audio
Astro Gaming A50 Circumaural Headset - Black 
  hide details  
Reply
post #518 of 772
Quote:
Pretty sure the courts found this example to be in violation of anti-trust laws and Intel was fined over a billion dollars...

The EU fined Intel roughly $1.4 billion. The US case never went to trial. There's a significant difference between EU and US antitrust law. In the US, you have to prove consumers were harmed by the abuse of monopoly power. In the EU, it's enough to demonstrate that a company abused its market position to harm its competitors, even if consumers themselves were not harmed.

I should've put "In the court of public opinion," to clarify my point, since there were other findings against Intel issued by the Japanese and Korean FTCs, but by bringing up the EU verdict you've actually underscored my point. In the EU, a finding of abuse against other competitors is considered fair grounds to justify a significant penalty. In the US, that's not enough. Again, there are different standards regarding "fairness" in play here.
post #519 of 772
Quote:
Originally Posted by sugarhell View Post

The best solution is to create a dynamic scene that use most of the new features of dx12. Then without altering the image quality or the effects of the scene there should be 2 specific hardware paths with the logic to use all the strong points of each architecture. With this way you can compare with the same scene which architecture or gpu is the fastest while providing the same Image quality.

This is the Neutral approach.

Sounds great. Reality is more complicated

1). Who codes the hardware paths? The benchmark vendor alone, or the benchmark vendor in cooperation with the GPU vendors?

2). What if the benchmark vendor's programmers have more experience in coding for one architecture vs. another? What if one architecture is simply easier to use?

3). Who decides if the benchmark scene has been fairly weighted and allows both architectures to shine via their respective strong points?

4). Who decides if the benchmark has become so optimized and influenced by the GPU vendors that it no longer represents an actual game? Benchmarks are only useful because they tell us something about the performance of shipping titles. If I show benchmark results in 5, 10, or 50 games, it's assumed (by both readers and myself) that these results are representative of other results in other games.

5). Who decides what "an actual game" should look like in the first place?

These aren't small or minor issues. They're the fundamental problems benchmark groups like SPEC or BAPCO grapple with (sometimes more successfully, sometimes less so). I have no doubt that FutureMark could work with AMD and Nvidia to create a benchmark scene that would be hand-coded with separate architectural pathways to take full advantage of every feature and capability of both Pascal and GCN. The problem is, this benchmark would no longer represent real-world scenarios. Instead, it would be a hyper-optimized, best-case scenario for what a developer could create when devoting huge resources to a single test. Even in this case, it would still be just one scene.

To give a really oversimplified example: If Maxwell and Pascal are better at handling small triangles than GCN (and they are), NV will always tend to prioritize small triangles and AMD won't. If GCN is better at async compute, AMD will prefer to use it, while NV won't. If Pascal is better at ambient occlusion, NV will want more of it, AMD will want less.

You can't just say "Code to each architecture's strengths" when the component elements of the scene will inevitably play to those strengths. It's not that simple.
post #520 of 772
Quote:
Originally Posted by DigiHound View Post

Sounds great. Reality is more complicated

1). Who codes the hardware paths? The benchmark vendor alone, or the benchmark vendor in cooperation with the GPU vendors?

2). What if the benchmark vendor's programmers have more experience in coding for one architecture vs. another? What if one architecture is simply easier to use?

3). Who decides if the benchmark scene has been fairly weighted and allows both architectures to shine via their respective strong points?

4). Who decides if the benchmark has become so optimized and influenced by the GPU vendors that it no longer represents an actual game? Benchmarks are only useful because they tell us something about the performance of shipping titles. If I show benchmark results in 5, 10, or 50 games, it's assumed (by both readers and myself) that these results are representative of other results in other games.

5). Who decides what "an actual game" should look like in the first place?

These aren't small or minor issues. They're the fundamental problems benchmark groups like SPEC or BAPCO grapple with (sometimes more successfully, sometimes less so). I have no doubt that FutureMark could work with AMD and Nvidia to create a benchmark scene that would be hand-coded with separate architectural pathways to take full advantage of every feature and capability of both Pascal and GCN. The problem is, this benchmark would no longer represent real-world scenarios. Instead, it would be a hyper-optimized, best-case scenario for what a developer could create when devoting huge resources to a single test. Even in this case, it would still be just one scene.

To give a really oversimplified example: If Maxwell and Pascal are better at handling small triangles than GCN (and they are), NV will always tend to prioritize small triangles and AMD won't. If GCN is better at async compute, AMD will prefer to use it, while NV won't. If Pascal is better at ambient occlusion, NV will want more of it, AMD will want less.

You can't just say "Code to each architecture's strengths" when the component elements of the scene will inevitably play to those strengths. It's not that simple.

It is. And all your logic is wrong because 3dmark was never supposed to represent a real-world scenario.

It's a non interactive dynamic scene with the same animations all the time. The prediction is there because you don't have the interaction from the user. It's a bench and nothing else.

Second all your questions are answered by a simple thing. Games. Games have the same scene with different effects and image quality and now with dx12 and vulkan they have to have different hardware paths for each gpu architecture. If you don't believe me go read all dx12 and vulkan wallpapers. If you don't do that then it's better to stay with dx11.3

To answer your questions :

1) The developers with the help of IHVs

2) Then they shouldnt have a product with the name benchmark. Also that's why they get help from IHVs.

3) It's a company that makes benchmarks. Also they have a level design.concept design, programming design from the start of the development. And believe me when you reach so low level of optimizations and you have your own graphic engine you know all the architectures advantages and disadvantages.

4) 3DMark already does not represent a game. It's a bench and they are useful only for them alone. As i said you cant compare a non interactive with predicted frames scene with a game at all.

5) The concept artists, the main designer, the 3d artists etc etc. If you mean the quality and not the art style then up to the graphic programmers
Workstation
(4 items)
 
  
CPUMotherboardGraphicsMonitor
Xeon E5-2690 Supermicro 2011 Nvidia GP100/ Vega FE Dell ultrasharp 4k 
  hide details  
Reply
Workstation
(4 items)
 
  
CPUMotherboardGraphicsMonitor
Xeon E5-2690 Supermicro 2011 Nvidia GP100/ Vega FE Dell ultrasharp 4k 
  hide details  
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Hardware News
Overclock.net › Forums › Industry News › Hardware News › [Various] Futuremark Releases 3DMark Time Spy DirectX 12 Benchmark