Overclock.net › Forums › Industry News › Hardware News › [PCper] Frame Rating Dissected: Full Details on Capture-based Graphics Performance Testing
New Posts  All Forums:Forum Nav:

[PCper] Frame Rating Dissected: Full Details on Capture-based Graphics Performance Testing - Page 4

post #31 of 420
Quote:
Originally Posted by SKYMTL View Post

Ultimately, everyone wants to justify their methodology and will say anything to validate it. Add in some technical jargon and any argument sounds like the right one.

I could go on for thousands of words (and I likely will in the coming weeks) as to why NEITHER approach is particularly valid. And at this point, there isn't a "happy medium". Capture cards are certainly not the answer, nor is FRAPS, even though I use it.

The point of these tests is to demonstrate in a visual chart form what people will actually see on their monitors. Do any accomplish this? NO! To be honest, the only way to properly "test" what someone sees is to do a blind testing in front of two systems with a large sample size of physical viewers. That won't happen for every GPU review, that's for sure.

From my understanding, several software solutions are in development but none of them are quite ready. One thing is certain though: this cannot be an exclusive process. Any software MUST be available to the community at large so it can be properly vetted.

Ironically, we have to remember that once AMD gets a handle on the stuttering situation, all of this discussion and development may be for nothing.

There are valid technical reasons as to why FRAPs is not an ideal method, to which both nvidia and AMD agree with. I hope you understand that FRAPs measuring before the rendering pipeline is a huge flaw in testing - Before the GPU driver does anything? How can you even BEGIN to consider such a test to be valid? Testing and benchmarking is the easy part. Most reviewers aren't software developers, however, and they will never understand what goes on behind the scenes at the API level. Again - benchmarking is easy.

Now before you further continue in defense mode, this isn't personal. I like your site very much and I read it quite often. I find your reviews to be excellent and well written. That said, you owe your readers valid testing methodology, to simply brush off these concerns is a major mistake on your part. Again. PRE-RENDER PIPELINE data is a very real concern to which both AMD and nvidia agree.

I do look forward to your followup articles that you mentioned smile.gif Again, nothing personal. I however have to say that the issues with FRAPS are very real and should be taken seriously. Pre-render data is not a valid testing methodology, and it is my belief that you shouldn't be so quick to brush such concerns away.

Perhaps, FCAT will be a better methodology than FRAPs. Although that is developed by nvidia, so i'm not sure how well that will go over with the community. Surely there will be some debates about that.
Edited by xoleras - 3/27/13 at 8:33am
post #32 of 420
I also dont understand how people here play games with no vsync.... Screen tearing alone is a game braking experience...

Oh look im running portal 2 @ 200fps average...
My cards is running so HOT!!!!! i also have it overclock it as well..


If my card can run the game without a single mhz of overclock at the resolution i want to play it... why i should overclock my card? Why i should play a game with no vsync and suffer the horrible screen tearing?

Why i cannot control my cards performance with vsync/overclocks/min fps and avg fps?

Numbers look good on benchmarks, when im playing a game is a different story...

Some people here need alot more to learn..
Edited by zGunBLADEz - 3/27/13 at 8:38am
post #33 of 420
Quote:
Originally Posted by sugarhell View Post

Its like saying dont use nvidia inspector. And amd helps the radeon pro creator.
Quote:
Originally Posted by zGunBLADEz View Post

Because it offers alot more than the traditional ccc?
Its a power tool why we shouldn't be using it?

Its not like is doing damage to my performance..

Like tweaking windows why we should tweak window to our use?


I'm not saying don't use it, nor am I against using it. But aren't you both suggesting use it because AMD hasn't provided proper driver support to begin with?

I appreciate that new games need tweaks to enable SLI/Crossfire if the driver is late but how do you explain Battlefield 3 in the charts I posted previously?
Meh
(11 items)
 
  
CPUMotherboardGraphicsRAM
i7 2600k @4.7GHz Asus P8Z68-V GEN3 GTX Titan 16GB Mushkin Redline Ridgeback 
Hard DriveHard DriveCoolingOS
Crucial M4 128GB  2x 500GB WD Green RAID1 + WD 2TB Green  Noctua Nh-DH 14 Win 8 Pro x64 
MonitorPowerCase
Shimian Achieva 2560x1440p IPS Corsair AX850 Lian-Li PC-A05N 
  hide details  
Reply
Meh
(11 items)
 
  
CPUMotherboardGraphicsRAM
i7 2600k @4.7GHz Asus P8Z68-V GEN3 GTX Titan 16GB Mushkin Redline Ridgeback 
Hard DriveHard DriveCoolingOS
Crucial M4 128GB  2x 500GB WD Green RAID1 + WD 2TB Green  Noctua Nh-DH 14 Win 8 Pro x64 
MonitorPowerCase
Shimian Achieva 2560x1440p IPS Corsair AX850 Lian-Li PC-A05N 
  hide details  
Reply
post #34 of 420
Quote:
Originally Posted by zGunBLADEz View Post

I also dont understand how people here play games with no vsync.... Screen tearing alone is a game braking experience...

After getting the 690 I don't get any screen tearing with vsync off. I couldn't play anything with it off during my 580 SLI days since it would be screen tear galore. Not sure what has changed,
post #35 of 420
Quote:
Originally Posted by SKYMTL View Post

Ultimately, everyone wants to justify their methodology and will say anything to validate it. Add in some technical jargon and any argument sounds like the right one.

I could go on for thousands of words (and I likely will in the coming weeks) as to why NEITHER approach is particularly valid. And at this point, there isn't a "happy medium". Capture cards are certainly not the answer, nor is FRAPS, even though I use it.

The point of these tests is to demonstrate in a visual chart form what people will actually see on their monitors. Do any accomplish this? NO! To be honest, the only way to properly "test" what someone sees is to do a blind testing in front of two systems with a large sample size of physical viewers. That won't happen for every GPU review, that's for sure.

From my understanding, several software solutions are in development but none of them are quite ready. One thing is certain though: this cannot be an exclusive process. Any software MUST be available to the community at large so it can be properly vetted.


Ironically, we have to remember that once AMD gets a handle on the stuttering situation, all of this discussion and development may be for nothing.

I agree that the science behind these methodologies needs to be well understood and agreed upon by the GPU vendors Nvidia and AMD , the platform / OS vendor Microsoft and the people who are going to objectively measure this and publish it to the user community - the tech press.

For the end user all that matters is playability and experience. so subjective opinion is all that matters. if the user feels the Nvidia solution is smoother then its game over. If the user can't tell the difference then not so much a problem.
Fragbox
(14 items)
 
  
CPUMotherboardGraphicsGraphics
Intel Core i5 2400 DH67BL AMD Radeon HD 6900 Series XFX HD 6950 2GB 
RAMRAMHard DriveOptical Drive
Corsair Corsair DDR3 1333 Mhz 2 GB Corsair DDR3 1333 Mhz 2 GB Western Digital Caviar Green SONY DVD-RW AD-7260S 
OSMonitorKeyboardPower
Windows 7 Professional 32 bit BENQ G2420HD Logitech K200 Seasonic VX550 psu 
CaseMouse
ANTEC 200 V2 Logitech mouse 
  hide details  
Reply
Fragbox
(14 items)
 
  
CPUMotherboardGraphicsGraphics
Intel Core i5 2400 DH67BL AMD Radeon HD 6900 Series XFX HD 6950 2GB 
RAMRAMHard DriveOptical Drive
Corsair Corsair DDR3 1333 Mhz 2 GB Corsair DDR3 1333 Mhz 2 GB Western Digital Caviar Green SONY DVD-RW AD-7260S 
OSMonitorKeyboardPower
Windows 7 Professional 32 bit BENQ G2420HD Logitech K200 Seasonic VX550 psu 
CaseMouse
ANTEC 200 V2 Logitech mouse 
  hide details  
Reply
post #36 of 420
Quote:
Originally Posted by Kane2207 View Post


I'm not saying don't use it, nor am I against using it. But aren't you both suggesting use it because AMD hasn't provided proper driver support to begin with?

I appreciate that new games need tweaks to enable SLI/Crossfire if the driver is late but how do you explain Battlefield 3 in the charts I posted previously?

I agree, should be included in CCC
post #37 of 420
Quote:
Originally Posted by xoleras View Post

There are valid technical reasons as to why FRAPs is not an ideal method, to which both nvidia and AMD agree with. I hope you understand that FRAPs measuring before the rendering pipeline is a huge flaw in testing - Before the GPU driver does anything? How can you even BEGIN to consider such a test to be valid? Testing and benchmarking is the easy part. Most reviewers aren't software developers, however, and they will never understand what goes on behind the scenes at the API level. Again - benchmarking is easy.

Now before you further continue in defense mode, this isn't personal. I like your site very much and I read it quite often. I find your reviews to be excellent and well written. That said, you owe your readers valid testing methodology, to simply brush off these concerns is a major mistake on your part. Again. PRE-RENDER PIPELINE data is a very real concern to which both AMD and nvidia agree.

I do look forward to your followup articles that you mentioned smile.gif Again, nothing personal. I however have to say that the issues with FRAPS are very real and should be taken seriously. Pre-render data is not a valid testing methodology, and it is my belief that you shouldn't be so quick to brush such concerns away.

Perhaps, FCAT will be a better methodology than FRAPs. Although that is developed by nvidia, so i'm not sure how well that will go over with the community. Surely there will be some debates about that.

My point isn't to brush off the concerns. Rather, my post was to emphasize one point: a different methodology which isn't FRAPS or FCAT needs to be developed. Like FRAPS, frame capturing gives very little indication of what an end user will actually experience. This goes all the way down to the OS and game coding levels, two points which have been largely overlooked.

We've recently put a substantial amount of money into the development of a viable alternative but it won't be ready for prime time for several months. Until then, I can tell you that frame capturing won't be part of our interim methodology.
Edited by SKYMTL - 3/27/13 at 8:45am
post #38 of 420
Quote:
Originally Posted by Kane2207 View Post


I'm not saying don't use it, nor am I against using it. But aren't you both suggesting use it because AMD hasn't provided proper driver support to begin with?

I appreciate that new games need tweaks to enable SLI/Crossfire if the driver is late but how do you explain Battlefield 3 in the charts I posted previously?

I have my own bf3 fraps frametimes and the trick is vsync or framecap.. It will stop all high frametime and microstutter galore hey im getting frametimes lower than 9ms with no spikes..

But hey i agreed, im also happy that amd is working on this as well.. But lets be honest i never said amd didnt suffer from any of this as well... Im just saying the fixes were out there for those who really care for a long time...

Also, nvidia have their own issues on this as amd...
So saying is all amd is wrong...
The only difference is nvidia have their own vsync thingy already on drivers which amd dont,.
Edited by zGunBLADEz - 3/27/13 at 8:46am
post #39 of 420
Just a side note some of you might or might not like:
Quote:
You may notice that there is a lot of “my” and “our” in this story while also seeing similar results from other websites being released today. While we have done more than a year’s worth of the testing and development on our own tools to help expedite a lot of this time consuming testing, some of the code base and applications were developed with NVIDIA and thus were distributed to other editors recently.

NVIDIA was responsible for developing the color overlay that sits between the game and DirectX (in the same location of the pipeline as FRAPS essentially) as well as the software extractor that reads the captured video file to generate raw information about the lengths of those bars in an XLS file. Obviously, NVIDIA has a lot to gain from this particular testing methodology: its SLI technology looks much better than AMD’s CrossFire when viewed in this light, highlighting the advantages that SLI’s hardware frame metering bring to the table.

Yeah rolleyes.gif

Anyway, this will turn out for the better both on the red and green side, eventually.
Never Finished
(18 items)
 
  
CPUMotherboardGraphicsRAM
Intel i7 6700K ASUS Z170 Pro Gaming Sapphire Tri-X R9 290X 8G Corsair Vengeance LPX DDR4 2666Mhz - (2x8GB) 
Hard DriveCoolingCoolingOS
Samsung 960 EVO M.2 500GB EK Predator 360 EK 250 X3 Reservoir Windows 10 Pro 
MonitorKeyboardPowerCase
Dell U2515H Corsair Raptor K40 Corsair RM650x Fractal Design Define S 
MouseMouse PadAudioAudio
Logitech M500 Gigabyte GP-MP8000 Extreme Accuracy Yamaha HTR-4068 Philips FWB C355 speaker set 
AudioOther
Hyper X Cloud II Samson C01U Pro 
  hide details  
Reply
Never Finished
(18 items)
 
  
CPUMotherboardGraphicsRAM
Intel i7 6700K ASUS Z170 Pro Gaming Sapphire Tri-X R9 290X 8G Corsair Vengeance LPX DDR4 2666Mhz - (2x8GB) 
Hard DriveCoolingCoolingOS
Samsung 960 EVO M.2 500GB EK Predator 360 EK 250 X3 Reservoir Windows 10 Pro 
MonitorKeyboardPowerCase
Dell U2515H Corsair Raptor K40 Corsair RM650x Fractal Design Define S 
MouseMouse PadAudioAudio
Logitech M500 Gigabyte GP-MP8000 Extreme Accuracy Yamaha HTR-4068 Philips FWB C355 speaker set 
AudioOther
Hyper X Cloud II Samson C01U Pro 
  hide details  
Reply
post #40 of 420
Quote:
Originally Posted by Imouto View Post

So they're robbing you your hard earned tearing. Such meanies.

The metric here is the amount of tearing. Think for a second here, what you're promoting here is faster graphics -> more tearing.

I'm with SKYMTL here, you just can't try to measure the gaming experience basing your methodology on how much your screen tears and how delivering the tearing in an even fashion is better than in a messy one. The perceived gaming experience is what matters and this is derailing in the wrong direction.

Also, the guy at PCPer is plain retarded at this point. He promised a V-Sync test and it totally debunked all his theories. In a lame attempt to keep his cool he ran the V-Sync disabled benches at a stupid high rate frame rate and the V-Sync enabled benches at a stupid low frame rate. He's doing things totally backwards.

Yeah just got to the end to the end.............. I typed that when I was still at Sleeping Dogs :/

Seems like if Vsync fixes it for the most part and debunks it comepletely........ then like GunbladeZ says Radeon Pro should pretty much eliminate or alleviate most issues like most CF users have been saying all along....... this bodes well for my 7970 CF biggrin.gif


But to your point about perceiving it vs measuring tears, as a former 5870CF user I had massive issues and the "weaker" 460 SLI was infinitely smoother and vastly better overall experience. I still have those suckers folding on my Aegis tongue.gif while I have long since sold the 5870's. But I know things got a bit better with the 6xxx line and I should get to find out how the 7XXX line does.
Edited by Junkboy - 3/27/13 at 8:54am
Ada
(24 items)
 
Aegis
(9 items)
 
 
CPUMotherboardGraphicsRAM
AMD 1700 Gigabyte AB350 Gaming 3 EVGA GTX 980 ti Classified F4-3600C16D-16GTZ @3200 
Hard DriveHard DriveHard DriveHard Drive
Intel 520 120GB SSD Samsung F2 1TB X2 Samsung F4 2TB Hitachi 1TB X2 
Hard DriveOptical DriveCoolingCooling
Seagate 3 TB Pioneer BDR 2207 Noctua NH D14 Sunbeam Rheobus Extreme 
CoolingOSMonitorMonitor
Enermax 140mm T.B. Vegas Duo x2 Win 10 Pro Extreme Edition Catleap 2B  ASUS VH242H  
KeyboardPowerCaseMouse
Logitech mk260 NZXT Hale90 750 Fractal Design Arc Midi Logitech G9x 
Mouse PadAudioAudioAudio
Steelseries QcK Denon AH D2000 Harman Kardon AVR 2600 Polk RTi8s, Rti4s and CSi3 
CPUMotherboardGraphicsRAM
Intel G645 Asrock z77 Extreme 4 Nvidia Branded GTX 460 SLI 8GB Corsair Vengeance Low profile DDR3 1600 
Hard DriveOSPowerCase
Hitachi 1TB Windows 7 Antec HCG 750 Bitfenix Shinobi Window 
Mouse Pad
Diablo 3 Steel Series.  
  hide details  
Reply
Ada
(24 items)
 
Aegis
(9 items)
 
 
CPUMotherboardGraphicsRAM
AMD 1700 Gigabyte AB350 Gaming 3 EVGA GTX 980 ti Classified F4-3600C16D-16GTZ @3200 
Hard DriveHard DriveHard DriveHard Drive
Intel 520 120GB SSD Samsung F2 1TB X2 Samsung F4 2TB Hitachi 1TB X2 
Hard DriveOptical DriveCoolingCooling
Seagate 3 TB Pioneer BDR 2207 Noctua NH D14 Sunbeam Rheobus Extreme 
CoolingOSMonitorMonitor
Enermax 140mm T.B. Vegas Duo x2 Win 10 Pro Extreme Edition Catleap 2B  ASUS VH242H  
KeyboardPowerCaseMouse
Logitech mk260 NZXT Hale90 750 Fractal Design Arc Midi Logitech G9x 
Mouse PadAudioAudioAudio
Steelseries QcK Denon AH D2000 Harman Kardon AVR 2600 Polk RTi8s, Rti4s and CSi3 
CPUMotherboardGraphicsRAM
Intel G645 Asrock z77 Extreme 4 Nvidia Branded GTX 460 SLI 8GB Corsair Vengeance Low profile DDR3 1600 
Hard DriveOSPowerCase
Hitachi 1TB Windows 7 Antec HCG 750 Bitfenix Shinobi Window 
Mouse Pad
Diablo 3 Steel Series.  
  hide details  
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Hardware News
Overclock.net › Forums › Industry News › Hardware News › [PCper] Frame Rating Dissected: Full Details on Capture-based Graphics Performance Testing