Overclock.net › Forums › Graphics Cards › Graphics Cards - General › How do you like your benchmarks?
New Posts  All Forums:Forum Nav:

How do you like your benchmarks?

post #1 of 6
Thread Starter 
Well some people complain on how a benchmark is set and if it was a fair test so I decided to do a set a of question for education reason:

The GPU benchmarks :
  1. Do you prefer benchmarks with the competitors results ( For example would you like to see a 7970 benchmark test and beside it show the scores of a 680 ) ?
  2. Do you prefer the benchmarks to be show with real time gaming ( the game being played ) benchmark ?
  3. Do you prefer the competitors GPU to be near price point of the current GPU that is being bench marked ?
  4. What would you like to see in the benchmarks ?
  5. Would you like to see overclock benchmarks ?
  6. Do you think its fair if we max overclock the AMD GPU and the MAX overclock the GEFORCE GPU that is available out of the box ( I mean without doing any bios or any opening, just using a program like MSI afterburner ) ?

How can we make a benchmark test more fair and what do you think should be doing benchmarks?
post #2 of 6
Quote:
Do you prefer benchmarks with the competitors results ( For example would you like to see a 7970 benchmark test and beside it show the scores of a 680 ) ?
YES
Quote:
Do you prefer the benchmarks to be show with real time gaming ( the game being played ) benchmark ?
NOT REALLY - there were sometimes when viewing a recorded game play i suspected it was manipulated. i believe you mean having a video recording, no?
Quote:
Do you prefer the competitors GPU to be near price point of the current GPU that is being bench marked ?
YES. and actually despite the age, the previous gens "counterpart" ie 6870/ 7870 or 580/680 if possible - good to see the performance difference. - i know that is done upon first release but to keep monitoring it would show if the increase in performance continues and by how much.
Quote:
What would you like to see in the benchmarks ?
GIRLS!!! and more GIRLS!!!
Quote:
Would you like to see overclock benchmarks ?
YES!
Quote:
Do you think its fair if we max overclock the AMD GPU and the MAX overclock the GEFORCE GPU that is available out of the box ( I mean without doing any bios or any opening, just using a program like MSI afterburner ) ?
YES and NO. it would be good to see an expected overclock performance difference but say the reviewer purposely used a card from the competition was was a fail at overclocking to skew the results; then it would be a bad thing.


ok survey done, do i get a smil3dbd4e4c2e742.gif ?
loon 3.2
(18 items)
 
  
CPUMotherboardGraphicsRAM
i7-3770K Asus P8Z77-V Pro EVGA 980TI SC+ 16Gb PNY ddr3 1866 
Hard DriveHard DriveHard DriveOptical Drive
PNY 1311 240Gb 1 TB Seagate 3 TB WD Blue DVD DVDRW+/- 
CoolingCoolingOSMonitor
EKWB P280 kit EK-VGA supremacy Win X LG 24MC57HQ-P 
KeyboardPowerCaseMouse
Ducky Zero [blues] EVGA SuperNova 750 G2 Stryker M [hammered and drilled] corsair M65 
AudioAudio
SB Recon3D Klipsch ProMedia 2.1  
  hide details  
Reply
loon 3.2
(18 items)
 
  
CPUMotherboardGraphicsRAM
i7-3770K Asus P8Z77-V Pro EVGA 980TI SC+ 16Gb PNY ddr3 1866 
Hard DriveHard DriveHard DriveOptical Drive
PNY 1311 240Gb 1 TB Seagate 3 TB WD Blue DVD DVDRW+/- 
CoolingCoolingOSMonitor
EKWB P280 kit EK-VGA supremacy Win X LG 24MC57HQ-P 
KeyboardPowerCaseMouse
Ducky Zero [blues] EVGA SuperNova 750 G2 Stryker M [hammered and drilled] corsair M65 
AudioAudio
SB Recon3D Klipsch ProMedia 2.1  
  hide details  
Reply
post #3 of 6
Thread Starter 
Quote:
Originally Posted by looniam View Post



ok survey done, do i get a smil3dbd4e4c2e742.gif ?

If you have ID then yes but for now here a rep+
post #4 of 6
Quote:
Do you prefer benchmarks with the competitors results ( For example would you like to see a 7970 benchmark test and beside it show the scores of a 680 ) ?
Most definitely yes.
Quote:
Do you prefer the benchmarks to be show with real time gaming ( the game being played ) benchmark ?
Nope. Rather rendering a set scene - much like the demo benchmarks.
Quote:
Do you prefer the competitors GPU to be near price point of the current GPU that is being bench marked ?
Yes, however refer to question 5.
Quote:
What would you like to see in the benchmarks ?
Split-screen imagery and/or timeline-based logs (GPU clocks, temp, etc) since we know of throttling, etc.
Quote:
Would you like to see overclock benchmarks ?
That would depend. If it comes as an extra, then yes. However if comparing two cards (like you said a 7970 and a 680) it would be nice to have maximum OC of each card results and then easing up the higher-frequency GPU to the top frequency of the other card - some people might enjoy clock for clock comparisons.
Quote:
Do you think its fair if we max overclock the AMD GPU and the MAX overclock the GEFORCE GPU that is available out of the box ( I mean without doing any bios or any opening, just using a program like MSI afterburner ) ?
Yes, already covered in question 5.

Edit: something I consider important regarding GPUs would be driver comparison. If needed, a different table, but I do believe a driver chart under a set of benchmarks per GPU would be awesome. It is, however, a ton of work!
Edited by fragamemnon - 6/6/13 at 4:43am
MEGATRON
(24 items)
 
GIGATRON
(10 items)
 
 
CPUMotherboardGraphicsGraphics
Intel Core i5-3570K @ 4.8/1.32V | Hammer&Vice'd ASUS Sabertooth Z77 PowerColor R9 290 PowerColor R9 290 
RAMHard DriveHard DriveCooling
Samsung MV-3V4G3D/US 4x4GB kit @ 2200MHz 9-10-1... 2x Western Digital Caviar Black 1TB RAID 0 Samsung 840 Evo 250GB 2xAquacomputer Kryographics Hawaii Nickel/Acrylic 
CoolingCoolingCoolingCooling
XSPC Rasa Swiftech MCP-655 Vario Aquacomputer Aqualis 450mm w/ nano & Pump top 2x XSPC RX360; 1x XSPC RS360; 1x Alphacool ST 1... 
CoolingOSMonitorMonitor
EK-Vardar F4-120 8.1 x64; Ubuntu 14.10 Dell U2412M Dell U2412M 
MonitorMonitorKeyboardPower
Dell U2312H BenQ XL2411Z KBC Poker II MX Clears SeaSonic SS-850AM 
CaseMouseMouse PadAudio
Thermaltake Core X9 Mionix AVIOR 7000 Razer Goliathus Aune T1 -> Yamaha A-720 -> Sennheiser HD650 | S... 
CPUCPUMotherboardGraphics
Intel Xeon E5-2650 v4 @ 2.00GHz Intel Xeon E5-2650 v4 @ 2.00GHz SuperMicro X10DAL-i nay 
RAMHard DriveCoolingOS
2x8GB (for now) Kingston DDR4 @ 2133MHz/C16 Samsung 840 120GB EK-Vardar F4 on an Arctic Freezer i11 Ubuntu Server 16.04 
PowerCase
SeaSonic SS850-AM overrated 
  hide details  
Reply
MEGATRON
(24 items)
 
GIGATRON
(10 items)
 
 
CPUMotherboardGraphicsGraphics
Intel Core i5-3570K @ 4.8/1.32V | Hammer&Vice'd ASUS Sabertooth Z77 PowerColor R9 290 PowerColor R9 290 
RAMHard DriveHard DriveCooling
Samsung MV-3V4G3D/US 4x4GB kit @ 2200MHz 9-10-1... 2x Western Digital Caviar Black 1TB RAID 0 Samsung 840 Evo 250GB 2xAquacomputer Kryographics Hawaii Nickel/Acrylic 
CoolingCoolingCoolingCooling
XSPC Rasa Swiftech MCP-655 Vario Aquacomputer Aqualis 450mm w/ nano & Pump top 2x XSPC RX360; 1x XSPC RS360; 1x Alphacool ST 1... 
CoolingOSMonitorMonitor
EK-Vardar F4-120 8.1 x64; Ubuntu 14.10 Dell U2412M Dell U2412M 
MonitorMonitorKeyboardPower
Dell U2312H BenQ XL2411Z KBC Poker II MX Clears SeaSonic SS-850AM 
CaseMouseMouse PadAudio
Thermaltake Core X9 Mionix AVIOR 7000 Razer Goliathus Aune T1 -> Yamaha A-720 -> Sennheiser HD650 | S... 
CPUCPUMotherboardGraphics
Intel Xeon E5-2650 v4 @ 2.00GHz Intel Xeon E5-2650 v4 @ 2.00GHz SuperMicro X10DAL-i nay 
RAMHard DriveCoolingOS
2x8GB (for now) Kingston DDR4 @ 2133MHz/C16 Samsung 840 120GB EK-Vardar F4 on an Arctic Freezer i11 Ubuntu Server 16.04 
PowerCase
SeaSonic SS850-AM overrated 
  hide details  
Reply
post #5 of 6
1. Yes, including cards in a similar price range as well.

2. As representative as realistic scenarios is ideal.

3. Yes. This is an important one. Sometimes two cards are at a similar performance level and therefore get directly compared despite there being a significant price difference.

4. Transparency is very important. There are often benchmarks built into the game or setup by reviewers that give massively different results to real gaming. There are also benchmarks for the same card on the same game with very different performance. The problem here is that the reviewer has often setup their own benchmark and not described their methodology. Just saying the title of the game is not good enough.

There also needs to be a representation of what the quality of the gameplay is like. Min/Max/Avg is not at all sufficient. Minimums can often be just single stutter, maximum is irrelevant and averages are often skewed by long period of high or low framerate. HardOCP's framerate graphs are the best very simple way of showing representative performance. Frametime distribution analysis is harder to understand, especially if you haven't studied statistics at all but give some very important metrics to the quality of performance being produced.

5. Max OC vs max OC as well as max with stock voltage.

6. Ideally doing both is fair. Some people are averse to BIOS flashing for minor gains. OC results don't really have to be that extensive. Just something to give an idea of how It's scaling.
 
Backup / Linux PC
(11 items)
 
HP N54L
(4 items)
 
CPUMotherboardGraphicsGraphics
Bulldozer FX-8120 4.7GHz Asus M5A99X EVO 990X HD7970 3GB 1265/1560MHz HD5770 HTPC 
RAMHard DriveHard DriveHard Drive
Samsung 30nm Green 16GB OCZ Vertex 4 256GB Samsung SPF1 1TB Seagate 3TB 
CoolingCoolingCoolingOS
XSPC RayStorm Aquacomputer AMS 840mm Laing D5 Windows 7 Ultimate 64bit 
MonitorMonitorKeyboardPower
3x Samsung SM2343BW 2048x1152 in eyefinity Samsung 32EH5000 32" 1920x1080 Steelseries 7G Corsair HX750 
CaseMouseAudioAudio
Antec 1200 Logitech G700S Asus Xonar D2X Microlab Solo 7C 2.0 
CPUMotherboardGraphicsRAM
AMD Athlon II X2 260 3.84GHz Gigabyte GA-78LMT-USB3 Nvidia 8800GS 384MB Corsair XMS3 4GB 
Hard DriveOSOSMonitor
Seagate 320GB Windows 8 64bit Kubuntu 12.10 64bit Tulip CRT @ 1920x1200 75Hz 
PowerCaseMouse
Corsair Builder Series 430W Lian-Li PC-A04B Logitech G5 
CPURAMHard DriveOS
AMD Turion II Neo Dual Core 2.2GHz 2GB 250GB Windows 7 Ultimate 64bit 
  hide details  
Reply
 
Backup / Linux PC
(11 items)
 
HP N54L
(4 items)
 
CPUMotherboardGraphicsGraphics
Bulldozer FX-8120 4.7GHz Asus M5A99X EVO 990X HD7970 3GB 1265/1560MHz HD5770 HTPC 
RAMHard DriveHard DriveHard Drive
Samsung 30nm Green 16GB OCZ Vertex 4 256GB Samsung SPF1 1TB Seagate 3TB 
CoolingCoolingCoolingOS
XSPC RayStorm Aquacomputer AMS 840mm Laing D5 Windows 7 Ultimate 64bit 
MonitorMonitorKeyboardPower
3x Samsung SM2343BW 2048x1152 in eyefinity Samsung 32EH5000 32" 1920x1080 Steelseries 7G Corsair HX750 
CaseMouseAudioAudio
Antec 1200 Logitech G700S Asus Xonar D2X Microlab Solo 7C 2.0 
CPUMotherboardGraphicsRAM
AMD Athlon II X2 260 3.84GHz Gigabyte GA-78LMT-USB3 Nvidia 8800GS 384MB Corsair XMS3 4GB 
Hard DriveOSOSMonitor
Seagate 320GB Windows 8 64bit Kubuntu 12.10 64bit Tulip CRT @ 1920x1200 75Hz 
PowerCaseMouse
Corsair Builder Series 430W Lian-Li PC-A04B Logitech G5 
CPURAMHard DriveOS
AMD Turion II Neo Dual Core 2.2GHz 2GB 250GB Windows 7 Ultimate 64bit 
  hide details  
Reply
post #6 of 6
Quote:
Do you prefer benchmarks with the competitors results ( For example would you like to see a 7970 benchmark test and beside it show the scores of a 680 ) ?
Yes, though I'd like to see other cards compared as well. Using the example of the 7970, comparing it to a few cards that are cheaper and some that are more expensive, e.g. 7950 and 7870, then 680, 670, 660 Ti, as well as the 7990, 690 and TITAN.
Quote:
Do you prefer the benchmarks to be show with real time gaming ( the game being played ) benchmark ?
Not sure really. I would probably say no, because it's never going to be exactly the same twice in a row. I'd rather have a fixed scene rendered, like the built-in benchmark option from a lot of games.
Quote:
Do you prefer the competitors GPU to be near price point of the current GPU that is being bench marked ?
Yes, but as above I like to also see cheaper and more expensive cards being compared, that way I get an idea of if it's worth spending the extra money etc.
Quote:
What would you like to see in the benchmarks ?
Not sure, really!
Quote:
Would you like to see overclock benchmarks ?
Yes, BUT not like LinusTechTips where you ONLY get to see overclocked results. I want to see stock results mainly, and then overclocked results afterwards (this seems to be how most review sites do it anyway).
Quote:
Do you think its fair if we max overclock the AMD GPU and the MAX overclock the GEFORCE GPU that is available out of the box ( I mean without doing any bios or any opening, just using a program like MSI afterburner ) ?
I think so. Whilst obviously some cards will overclock better, when reviewing a card you might not know what the average result is, so you'd only have your review sample to go by. By overclocking them as far as they can go, you also take into the account that some GPUs overclock further than others on average, which could be a buying point for a lot of people.
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Graphics Cards - General
Overclock.net › Forums › Graphics Cards › Graphics Cards - General › How do you like your benchmarks?