Overclock.net › Forums › Industry News › Hardware News › [PCper] Frame Rating Dissected: Full Details on Capture-based Graphics Performance Testing
New Posts  All Forums:Forum Nav:

[PCper] Frame Rating Dissected: Full Details on Capture-based Graphics Performance Testing - Page 14

post #131 of 420
I don't agree with that statement entirely d3v0.

On one hand the new methodologies some of us are using do display that the visual quality and / or fluidity of scenes can take a tumble when using multi card setups. HOWEVER, regardless of on-display framerate readings, adding a second card will improve perceptible performance in most situations.

What the articles should be saying (and are) is that when using frame monitoring technologies, Crossfire doesn't improve consistent frame delivery. Nor does SLI for that matter. It does improve AVERAGE frame delivery in most cases. With that being said, in the end, this all boils down to end user perception rather that pretty graphs.

What I'm trying to say is that due to the limitations of our eyes (let's call this our "internal refresh rate" for lack of a better term) it is highly unlikely that anyone will perceive latency differences below 30ms. As such, in borderline high / low framerate situations adding a second card WILL make a positive on-screen difference. That is, provided the solution can consistently bring the higher average latencies from a single card setup down to a less perceptible sub-30ms to 35ms mark.

At this point far too many people are jumping to conclusions based upon the information being posted about frame times. Many sites have done a great job of posting all of the relevant information but it seems like few readers are actually "getting it".
Edited by SKYMTL - 3/28/13 at 7:32am
post #132 of 420
Quote:
Originally Posted by d3v0 View Post

People who drank the coolaid believing that adding their second 7970 are funny here; "i certainly noticed I was able to increase details and maintain 30FPS" well yeah, duh. thats what the reviewers say too. The problem is that 30fps readout which you can see on your OSD isn't really 30FPS! Some people are missing the point entirely, simply justifying their purchase. The point of this is objective information....not anecdotal evidence of you reading your FPS readout after installing a second card. Come on. The whole point of the article is to show THAT IS NOT AN EFFECTIVE METHOD OF DETERMINING PERFORMANCE IMPROVEMENT!!

This testing methodology/result flies in the face of what we PC enthusiasts have bowed down to for years, so naturally, its going to take alot of time and adjustment to get used to. We are all intelligent people, and we shouldnt close our minds to new science just because we ...dont like the results.
Yea because you know, because you have tried CFX and seen this for yourself and they are just imagining it rolleyes.gif

Looking at your sig though, it isn't them that drank the "coolaid"..

Like SKY said, there is alot more to this then simple graphs.
Bender
(18 items)
 
  
Reply
Bender
(18 items)
 
  
Reply
post #133 of 420
Quote:
Originally Posted by SKYMTL View Post

I don't agree with that statement entirely d3v0.

On one hand the new methodologies some of us are using do display that the visual quality and / or fluidity of scenes can take a tumble when using multi card setups. HOWEVER, regardless of on-display framerate readings, adding a second card will improve perceptible performance in most situations.

What the articles should be saying (and are) is that when using frame monitoring technologies, Crossfire doesn't improve consistent frame delivery. Nor does SLI for that matter. It does improve AVERAGE frame delivery in most cases. With that being said, in the end, this all boils down to end user perception rather that pretty graphs.

What I'm trying to say is that due to the limitations of our eyes (let's call this our "internal refresh rate" for lack of a better term) it is highly unlikely that anyone will perceive latency differences below 30ms. As such, in borderline high / low framerate situations adding a second card WILL make a positive on-screen difference. That is, provided the solution can consistently bring the higher average latencies from a single card setup down to a less perceptible sub-30ms to 35ms mark.

At this point far too many people are jumping to conclusions based upon the information being posted about frame times. Many sites have done a great job of posting all of the relevant information but it seems like few readers are actually "getting it".

What exactly are people missing then?

You have stated several times in this thread that we are misinterpreting the results and that capture cards are not telling the whole story, but I have not seen you plainly state your reason behind making these remarks. It seems to me that analyzing the frame delivered to the monitor will be nearly 100% correct in quantifying the user experience unless there is some technical issue in capturing the data. Tell me what I am missing here...

Is your point that we will see something different than the capture results when gaming?
Edited by JCPUser - 3/28/13 at 7:59am
post #134 of 420
Quote:
Originally Posted by Mopar63 View Post

Something to consider here, SLI is a MUCH older technology than Crossfire, it has a lot more development behind it.

You see SLI, despite nVidia's attempt to re-write history was actually developed and first used in 1998 by a company known as 3DFX. In fact at the time nVidias found and CEO made many public statements to the effect that multi-card was a gimmick and not the true future of the video card for gaming. (Interesting twist) When nVidia bought out 3DFX they acquired this technology and the fact it was actually pretty well developed then.

Now a lot is being made about how different this SLI is from the one 3DFX had, and I will not argue that but the base principles and technology is what led to the development and this means NVidia did not have to start from scratch but had a solid base of development under them when they reintroduced SLI in 2004.

AMD by contrast did not have this base technology to work from and had to start from scratch, introducing Crossfire near the end of 2005. So in effect AMD is about 7 years in development behind NVidia on this technology.

Now this is not meant as excuse for AMD when it comes to the performance issues we see with this new testing and Crossfire. However it does give a little background. Of note also is the fact of why is anyone surprised? Crossfire testing has consistently been a hit and miss method of GPU speed boosting for a number of years, SLI has had misses as well but has been overall more consistent than Crossfire. These test are really just giving us knowledge we already have.

Actually, there was an ATi card that had two GPUs ages ago: Rage FURY MAXX.
Quote:
Originally Posted by JCPUser View Post

What exactly are people missing then?

You have stated several times in this thread that we are misinterpreting the results and that capture cards are not telling the whole story, but I have not seen you plainly state your reason behind making these remarks. It seems to me that analyzing the frame delivered to the monitor will be nearly 100% correct in quantifying the user experience unless there is some technical issue in capturing the data. Tell me what I am missing here...

Is your point that we will see something different than the capture results when gaming?

A lot of people are saying there's no reason to go CFX at all, when in fact you still do gain performance...Just not in latency.

Think of it like RAM overclocking (Not the actual latency in ns for this, just CAS timings vs Frequency) in which going to CFX is like going from 1333Mhz CAS 9 to 2400Mhz CAS 9, if CFX improved latency then it may be like going from 1333Mhz CAS 9 to 1333Mhz CAS6. (In which average FPS, wherever it's measured, stays the same but latency goes down)

Quite clearly, a balance is needed...Let the latency decrease/increase so it's still below the point where each individual person won't notice it (Which maybe 35ms for me, but 30ms for you to pull numbers out of my rear) while also still increasing FPS. A lot of people seem to assume that saying this is saying that the new method is completely useless...It's not, just like IPC vs Clock Speed or Size vs Speed in storage (ie. SSD vs HDD) you can't just take one and put it on a pedestal, it varies from the application its put into (eg. IPC of an FX-8320/FX-8350 is slightly ahead of an i5 3570k in a small number of apps, but behind in most. Some applications might benefit from having 3/4TB of slower responding space than 256GB/512GB of low latency space, etc) and you need to take both into account at all times.
My system
(23 items)
 
  
CPUMotherboardGraphicsRAM
Intel Core i5 3570k @ 4.5Ghz ASRock Z77 Pro3 Powercolor Radeon HD7950 3GB @ 1150/1350 4x4GB G.Skill Ares 2000Mhz CL9 
Hard DriveHard DriveHard DriveHard Drive
Samsung 840 250GB Western Digital Black 1TB WD1002FAEX Seagate Barracuda 3TB ST3000DM001 Samsung Spinpoint EcoGreen 2TB 
Optical DriveCoolingCoolingCooling
Pioneer DVR-220LBKS Noctua NH-D14 Scythe Gentle Typhoon 1850rpm Corsair AF140 Quiet Edition 
CoolingOSMonitorMonitor
Arcitc Cooling Acclero Twin Turbo II Arch Linux x86-64, amdgpu BenQ G2220HD BenQ G2020HD 
KeyboardPowerCaseMouse
Ducky Shine III Year of the Snake, Cherry Blue Silverstone Strider Plus 600w CoolerMaster CM690 II Black and White SteelSeries Sensei Professional 
Mouse PadAudioOther
Artisan Hien Mid Japan Black Large ASUS Xonar DX NZXT Sentry Mesh 30w Fan Controller 
  hide details  
Reply
My system
(23 items)
 
  
CPUMotherboardGraphicsRAM
Intel Core i5 3570k @ 4.5Ghz ASRock Z77 Pro3 Powercolor Radeon HD7950 3GB @ 1150/1350 4x4GB G.Skill Ares 2000Mhz CL9 
Hard DriveHard DriveHard DriveHard Drive
Samsung 840 250GB Western Digital Black 1TB WD1002FAEX Seagate Barracuda 3TB ST3000DM001 Samsung Spinpoint EcoGreen 2TB 
Optical DriveCoolingCoolingCooling
Pioneer DVR-220LBKS Noctua NH-D14 Scythe Gentle Typhoon 1850rpm Corsair AF140 Quiet Edition 
CoolingOSMonitorMonitor
Arcitc Cooling Acclero Twin Turbo II Arch Linux x86-64, amdgpu BenQ G2220HD BenQ G2020HD 
KeyboardPowerCaseMouse
Ducky Shine III Year of the Snake, Cherry Blue Silverstone Strider Plus 600w CoolerMaster CM690 II Black and White SteelSeries Sensei Professional 
Mouse PadAudioOther
Artisan Hien Mid Japan Black Large ASUS Xonar DX NZXT Sentry Mesh 30w Fan Controller 
  hide details  
Reply
post #135 of 420
I think PCper is a bit harsh saying adding another Radeon adds nothing to the user experience, but the point is that the traditional meter of CF scaling ("90% increase in frame rate, booyah!") is quite ignorant or in some cases self-deceptive.

There is another point worth making: the separation of benchmarking and gaming. Everyone should take note here: choosing a GPU solution for gaming can go wrong if it's based on pure benchmarking preferences. Benchmarking is all about fps numbers and this new method makes it obvious that a good gaming experience is a way more complex objective.
The Air Tunnel
(10 items)
 
  
CPUMotherboardGraphicsRAM
i7-4820K (4500MHz@1.28V) P9X79 Sapphire R9 290 Tri-X New Edition Team Group Vulcan 4x4GB 2133MHz 
Hard DriveCoolingOSPower
2x240GB SSD @RAID 0 Noctua NH-D14 SE2011 Windows 10 Pro Corsair AX750 
CaseAudio
Rosewill Armor EVO hiFace+AudioGD+DT770Pro 
  hide details  
Reply
The Air Tunnel
(10 items)
 
  
CPUMotherboardGraphicsRAM
i7-4820K (4500MHz@1.28V) P9X79 Sapphire R9 290 Tri-X New Edition Team Group Vulcan 4x4GB 2133MHz 
Hard DriveCoolingOSPower
2x240GB SSD @RAID 0 Noctua NH-D14 SE2011 Windows 10 Pro Corsair AX750 
CaseAudio
Rosewill Armor EVO hiFace+AudioGD+DT770Pro 
  hide details  
Reply
post #136 of 420
Quote:
Originally Posted by JCPUser View Post

What exactly are people missing then?

You have stated several times in this thread that we are misinterpreting the results and that capture cards are not telling the whole story, but I have not seen you plainly state your reason behind making these remarks. It seems to me that analyzing the frame delivered to the monitor will be nearly 100% correct in quantifying the user experience unless there is some technical issue in capturing the data. Tell me what I am missing here...

Is your point that we will see something different than the capture results when gaming?

I thought I made it evident but maybe not.

Capture cards, FRAPS and pretty much every other method currently being used do an incomplete job at representing what boils down to an EXPERIENCE. I've asked this many times: how do you benchmark and chart an experience? Simply put, you don't.

There are of course many possibilities which will ultimately help portray an experience. That includes the aforementioned FRAPS, frame capturing and even double-blind testing. My entire point is that frame capturing and / or frame latency testing is just another tool that has to be used alongside the usual frame rate logging. Using only one tool is like trying to build a house with only a hammer; other tools will eventually be needed to get the job done. In this case, the tools need to be used in order to faithfully recreate an experience and results which are valid for end users.

What I am asking for is perspective and I'll give an example which was brought up in another thread.

FRAPS and frame capture cards (see TR's comparison between the two) may show a Crossfire solution spazzing around like crazy with frame latencies all over the place. However, if all of these frame times occur below a certain mark (in this case I believe 35-40ms is imperceptible but for argument's sake, let's say below 30ms) then the actual on-screen effect will be perceived as being completely fluid. This is where perspective comes into the equation since what may LOOK bad isn't necessarily going to be viewed any differently by a gamer, even if latencies hovered around even lower levels.

With this in mind, acting like frame capture is the end-all solution is preposterous in the extreme. As I said before, it is another tool being pushed by NVIDIA which will ultimately help in the effort to demonstrate results. HOWEVER, without that perspective I mentioned, the frame time graphs mean precisely nothing since they'll be taken out of context.
post #137 of 420
Quote:
Originally Posted by specopsFI View Post


There is another point worth making: the separation of benchmarking and gaming. Everyone should take note here: choosing a GPU solution for gaming can go wrong if it's based on pure benchmarking preferences. Benchmarking is all about fps numbers and this new method makes it obvious that a good gaming experience is a way more complex objective.

This.

Part of the reason there is a market for a card like the Titan. Sure, there are faster configurations for the same or less money, but no single or dual card setup offers the experience the Titan does right now.
Super P's rig
(20 items)
 
  
CPUMotherboardGraphicsRAM
5960x ASUS X99-A II Asus GTX 1080 Ti Corsair Vengeance DDR4 3000 
Hard DriveHard DriveHard DriveOptical Drive
MyDigitalSSD BPX NVMe Samsung 850 EVO Seagate Momentus XT 500 GB External DVDRW 
CoolingCoolingOSMonitor
EK-XLC Predator 240 Swiftech 240mm Radiator Windows 10 Samsung 40" 4K - UN40KU6290 
KeyboardPowerCaseMouse
G710+ EVGA SuperNOVA 850G2 Fractal Design Define S G700s 
Mouse PadAudioAudioAudio
Vipamz Extended XXXL Asus U7 M-Audio AV40 Sennheiser HD 439 
  hide details  
Reply
Super P's rig
(20 items)
 
  
CPUMotherboardGraphicsRAM
5960x ASUS X99-A II Asus GTX 1080 Ti Corsair Vengeance DDR4 3000 
Hard DriveHard DriveHard DriveOptical Drive
MyDigitalSSD BPX NVMe Samsung 850 EVO Seagate Momentus XT 500 GB External DVDRW 
CoolingCoolingOSMonitor
EK-XLC Predator 240 Swiftech 240mm Radiator Windows 10 Samsung 40" 4K - UN40KU6290 
KeyboardPowerCaseMouse
G710+ EVGA SuperNOVA 850G2 Fractal Design Define S G700s 
Mouse PadAudioAudioAudio
Vipamz Extended XXXL Asus U7 M-Audio AV40 Sennheiser HD 439 
  hide details  
Reply
post #138 of 420
Quote:
Originally Posted by d3v0 View Post

People who drank the coolaid believing that adding their second 7970 are funny here; "i certainly noticed I was able to increase details and maintain 30FPS" well yeah, duh. thats what the reviewers say too. The problem is that 30fps readout which you can see on your OSD isn't really 30FPS! Some people are missing the point entirely, simply justifying their purchase. The point of this is objective information....not anecdotal evidence of you reading your FPS readout after installing a second card. Come on. The whole point of the article is to show THAT IS NOT AN EFFECTIVE METHOD OF DETERMINING PERFORMANCE IMPROVEMENT!!

This testing methodology/result flies in the face of what we PC enthusiasts have bowed down to for years, so naturally, its going to take alot of time and adjustment to get used to. We are all intelligent people, and we shouldnt close our minds to new science just because we ...dont like the results.

Go activate AA and use a single 7970 on eyefinity or 1440p and up and then use cfx and tell us again that there's no difference on fluidity..

Seriously guys seriously.
LoL
Its a trap!!!!!!!
post #139 of 420
Quote:
Originally Posted by zGunBLADEz View Post

Go activate AA and use a single 7970 on eyefinity or 1440p and up and then use cfx and tell us again that there's no difference on fluidity..

Seriously guys seriously.
LoL
Its a trap!!!!!!!

A second card is rendering the frames, you can run additional detail and higher AA, this much is obvious as AFR shares the work load.

The issue is the current research suggests: whilst you think you're going from 30fps to 60 fps, with all of the little frames less than 21 scan lines high that actually add nothing to the quality of motion or the 'smoothness', the scaling factor becomes more like going from 30fps to 40fps. This is because AMD prioritized latency above all else. If a frame is very slow to render, it could be discarded altogether, hence microstutter.

What will be interesting to see is when AMD fix this by applying some kind of frame metering, will they lose some of the advances in raw FPS seen in the latest drivers? I wouldn't be surprised if the normal metric of absolute FPS drops but you end up with a smoother feeling experience due to better frame pacing.
Meh
(11 items)
 
  
CPUMotherboardGraphicsRAM
i7 2600k @4.7GHz Asus P8Z68-V GEN3 GTX Titan 16GB Mushkin Redline Ridgeback 
Hard DriveHard DriveCoolingOS
Crucial M4 128GB  2x 500GB WD Green RAID1 + WD 2TB Green  Noctua Nh-DH 14 Win 8 Pro x64 
MonitorPowerCase
Shimian Achieva 2560x1440p IPS Corsair AX850 Lian-Li PC-A05N 
  hide details  
Reply
Meh
(11 items)
 
  
CPUMotherboardGraphicsRAM
i7 2600k @4.7GHz Asus P8Z68-V GEN3 GTX Titan 16GB Mushkin Redline Ridgeback 
Hard DriveHard DriveCoolingOS
Crucial M4 128GB  2x 500GB WD Green RAID1 + WD 2TB Green  Noctua Nh-DH 14 Win 8 Pro x64 
MonitorPowerCase
Shimian Achieva 2560x1440p IPS Corsair AX850 Lian-Li PC-A05N 
  hide details  
Reply
post #140 of 420
Quote:
Originally Posted by Kane2207 View Post

A second card is rendering the frames, you can run additional detail and higher AA, this much is obvious as AFR shares the work load.

The issue is the current research suggests: whilst you think you're going from 30fps to 60 fps, with all of the little frames less than 21 scan lines high that actually add nothing to the quality of motion or the 'smoothness', the scaling factor becomes more like going from 30fps to 40fps. This is because AMD prioritized latency above all else. If a frame is very slow to render, it could be discarded altogether, hence microstutter.

What will be interesting to see is when AMD fix this by applying some kind of frame metering, will they lose some of the advances in raw FPS seen in the latest drivers? I wouldn't be surprised if the normal metric of absolute FPS drops but you end up with a smoother feeling experience due to better frame pacing.

I beg to differ to a certain extent. While I agree with the general premise of your post, there are plenty of people here who will say that they can see a difference between 30fps and 60fps, which is a direct result of that "smoothness" factor you mentioned.

The real determining factor of SLI and Crossfire is their ability to enhance fluidity in borderline situations. For example, if a single card runs at 20 fps with minimums of 15, the game is unplayable while adding a second card could boost that to 35 fps with minimums of 30. Many are willing to do that, even if it means interjecting some stuttering. Stuttering I remind you that can be all but eliminated through the use of V-Sync or framerate caps.

Personally, I think making a blanket statement about the validity of Crossfire and / or SLI based purely on frame times is a mistake. It all comes down to what a gamer is willing to sacrifice in order to achieve what they feel is an optimal solution.
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Hardware News
Overclock.net › Forums › Industry News › Hardware News › [PCper] Frame Rating Dissected: Full Details on Capture-based Graphics Performance Testing