Frametime Analysis - .1% lows seem meaningless - Overclock.net - An Overclocking Community

Forum Jump: 

Frametime Analysis - .1% lows seem meaningless

Reply
 
Thread Tools
post #1 of 18 (permalink) Old 11-06-2017, 09:20 AM - Thread Starter
New to Overclock.net
 
cssorkinman's Avatar
 
Join Date: Apr 2009
Posts: 9,454
Rep: 463 (Unique: 263)
I've been seeing a lot of review sites posting this information and on the surface , it appears to be quite useful. However after playing around with frame time analyzers using my own data it became apparent to me that it's not nearly as meaningful as it had appeared.

The case in point




166 fps minimum vs showing a .1% low of 100 because of single frames spread out over the entirety of a sample, ( out of 33,000 in 3 minutes of play) doesn't seem to be very representative of perceptible differences in performance.

Would any of you smart fellows out there have any thoughts on how to make better use of frametime data ? I was wondering how to use those 33000 data points to find any one second of gameplay with the lowest performance?

Any excel or open office wizards out there care to make a tool for doing this?

Perhaps fraps already does this to produce the 166 number? I've always assumed that it simply polls the fps during a given second and reports this instead.

Thanks in advance for any and all replies.
cssorkinman is offline  
Sponsored Links
Advertisement
 
post #2 of 18 (permalink) Old 11-06-2017, 04:46 PM
Iconoclast
 
Blameless's Avatar
 
Join Date: Feb 2008
Posts: 30,086
Rep: 3135 (Unique: 1869)
FRAPS is counting the time of every frame.

166fps minimum is clearly incorrect, and is probably a 1s average. A ~20ms frame is a ~50fps minimum frame rate. The 1% and .01% frame rates are there to cull extreme outliers like that.

I'd argue, that in this case, the 0.1% is far more useful than that misleading average figure, or the single frame out of ~33k frames that falls significantly outside the rest.

...rightful liberty is unobstructed action according to our will within limits drawn around us by the equal rights of others. I do not add 'within the limits of the law,' because law is often but the tyrant's will, and always so when it violates the right of an individual. -- Thomas Jefferson
Blameless is offline  
post #3 of 18 (permalink) Old 11-06-2017, 05:19 PM - Thread Starter
New to Overclock.net
 
cssorkinman's Avatar
 
Join Date: Apr 2009
Posts: 9,454
Rep: 463 (Unique: 263)
Quote:
Originally Posted by Blameless View Post

FRAPS is counting the time of every frame.

166fps minimum is clearly incorrect, and is probably a 1s average. A ~20ms frame is a ~50fps minimum frame rate. The 1% and .01% frame rates are there to cull extreme outliers like that.

I'd argue, that in this case, the 0.1% is far more useful than that misleading average figure, or the single frame out of ~33k frames that falls significantly outside the rest.

The reason I think it isn't a good measure is because you would have to have 100 of those highest frametime images occur in the same one second segement of the bench in order to even come close to the threshold of human perception. If I interpret this correctly , there are only 33 such instances of frames taking that long to render across nearly a 3 minute benchmark.



If I recall the data comes from this section of BF1's single player mode , on a later run.

https://youtu.be/YfhL08s4IMw

FPS is slightly higher whilst not recording and I recall a later run with the cpu cranked up a bit averaging 198 fps with the 200 fps cap.

I appreciate the reply.
cssorkinman is offline  
Sponsored Links
Advertisement
 
post #4 of 18 (permalink) Old 11-06-2017, 05:31 PM
Retired Staff
 
Mr.N00bLaR's Avatar
 
Join Date: Dec 2004
Posts: 6,316
Rep: 262 (Unique: 154)
Quote:
Originally Posted by Blameless View Post

FRAPS is counting the time of every frame.

166fps minimum is clearly incorrect, and is probably a 1s average. A ~20ms frame is a ~50fps minimum frame rate. The 1% and .01% frame rates are there to cull extreme outliers like that.

I'd argue, that in this case, the 0.1% is far more useful than that misleading average figure, or the single frame out of ~33k frames that falls significantly outside the rest.


I think GamersNexus does a good job at explaining it here: https://www.youtube.com/watch?v=uXepIWi4SgM. I also have the g3258 and have seen what he's describing.

All overclock, all the time
Mr.N00bLaR is online now  
post #5 of 18 (permalink) Old 11-06-2017, 05:56 PM
Iconoclast
 
Blameless's Avatar
 
Join Date: Feb 2008
Posts: 30,086
Rep: 3135 (Unique: 1869)
Quote:
Originally Posted by cssorkinman View Post

The reason I think it isn't a good measure is because you would have to have 100 of those highest frametime images occur in the same one second segement of the bench in order to even come close to the threshold of human perception.

100 of those frames would be a full second or more.

Given the frame time distribution of your graph (where every several frames you get a frame that double the frame time) I'd consider those 100fps and 112fps figures way more useful than the 166fps or 193fps figures.

The 0.1% frame rate lets the true outliers be culled and gives you a useful minimum performance figure.
Quote:
Originally Posted by Mr.N00bLaR View Post

I think GamersNexus does a good job at explaining it here: https://www.youtube.com/watch?v=uXepIWi4SgM. I also have the g3258 and have seen what he's describing.

Yes.

In cssorkinman's case, he's not getting any severe dips like that, but if you had a true constant/fixed frame time 193fps output vs. a true 100fps output, the later would be closer to representing the smoothness he'd see than the former.

Now, at 100+fps it's smooth enough regardless and many people may well not be able to tell the difference between 193/166 apparent fps and 112/100 apparent fps, but that doesn't deminish the usefulness of the 0.1% lows as a measurement.

...rightful liberty is unobstructed action according to our will within limits drawn around us by the equal rights of others. I do not add 'within the limits of the law,' because law is often but the tyrant's will, and always so when it violates the right of an individual. -- Thomas Jefferson
Blameless is offline  
post #6 of 18 (permalink) Old 11-06-2017, 05:56 PM - Thread Starter
New to Overclock.net
 
cssorkinman's Avatar
 
Join Date: Apr 2009
Posts: 9,454
Rep: 463 (Unique: 263)
Quote:
Originally Posted by Mr.N00bLaR View Post

Quote:
Originally Posted by Blameless View Post

FRAPS is counting the time of every frame.

166fps minimum is clearly incorrect, and is probably a 1s average. A ~20ms frame is a ~50fps minimum frame rate. The 1% and .01% frame rates are there to cull extreme outliers like that.

I'd argue, that in this case, the 0.1% is far more useful than that misleading average figure, or the single frame out of ~33k frames that falls significantly outside the rest.


I think GamersNexus does a good job at explaining it here: https://www.youtube.com/watch?v=uXepIWi4SgM. I also have the g3258 and have seen what he's describing.

I understand what they are trying to express and I thank you for the link.

I guess it comes down to the way fraps is measuring it - does it keep track of the fps produced for every second and find the lowest and show that value as the min or is it just sampling every so often.

After looking at the min max ave spreadsheet produced it would seem that it is at least averaging across complete seconds ( possibly 10 from the look of it) and considers those .1% lows because it has the average fps being the same as the frame time analyzer.
cssorkinman is offline  
post #7 of 18 (permalink) Old 11-06-2017, 06:00 PM
Iconoclast
 
Blameless's Avatar
 
Join Date: Feb 2008
Posts: 30,086
Rep: 3135 (Unique: 1869)
Quote:
Originally Posted by cssorkinman View Post

I guess it comes down to the way fraps is measuring it - does it keep track of the fps produced for every second and find the lowest and show that value as the min or is it just sampling every so often.

FRAPS times every single frame. It's less than ideal as it doesn't pull info from the very end of the display chain (pure software solutions cannot do this), but should be accurate enough.

ATPipeline_575px.png

Hardware based capture and measurement solutions grab from the very end of the display chain, at the physical display output itself, so can measure what is actually seen.

...rightful liberty is unobstructed action according to our will within limits drawn around us by the equal rights of others. I do not add 'within the limits of the law,' because law is often but the tyrant's will, and always so when it violates the right of an individual. -- Thomas Jefferson
Blameless is offline  
post #8 of 18 (permalink) Old 11-06-2017, 06:08 PM - Thread Starter
New to Overclock.net
 
cssorkinman's Avatar
 
Join Date: Apr 2009
Posts: 9,454
Rep: 463 (Unique: 263)
Quote:
Originally Posted by Blameless View Post

Quote:
Originally Posted by cssorkinman View Post

I guess it comes down to the way fraps is measuring it - does it keep track of the fps produced for every second and find the lowest and show that value as the min or is it just sampling every so often.

FRAPS times every single frame. It's less than ideal as it doesn't pull info from the very end of the display chain (pure software solutions cannot do this), but should be accurate enough.

ATPipeline_575px.png

Hardware based capture and measurement solutions grab from the very end of the display chain, at the physical display output itself, so can measure what is actually seen.


Would that account for frametimes that are lower than the 200 fps limit should allow?
cssorkinman is offline  
post #9 of 18 (permalink) Old 11-06-2017, 06:26 PM
Iconoclast
 
Blameless's Avatar
 
Join Date: Feb 2008
Posts: 30,086
Rep: 3135 (Unique: 1869)
Quote:
Originally Posted by cssorkinman View Post

Would that account for frametimes that are lower than the 200 fps limit should allow?

It could.

...rightful liberty is unobstructed action according to our will within limits drawn around us by the equal rights of others. I do not add 'within the limits of the law,' because law is often but the tyrant's will, and always so when it violates the right of an individual. -- Thomas Jefferson
Blameless is offline  
post #10 of 18 (permalink) Old 11-06-2017, 06:54 PM - Thread Starter
New to Overclock.net
 
cssorkinman's Avatar
 
Join Date: Apr 2009
Posts: 9,454
Rep: 463 (Unique: 263)
Quote:
Originally Posted by Blameless View Post

Quote:
Originally Posted by cssorkinman View Post

Would that account for frametimes that are lower than the 200 fps limit should allow?

It could.

I appreciate your time, thank you. thumb.gif

Now I'm wondering about the cap's effect on frametimes.

I'll keep playing with it
cssorkinman is offline  
Reply

Quick Reply
Message:
Options

Register Now

In order to be able to post messages on the Overclock.net - An Overclocking Community forums, you must first register.
Please enter your desired user name, your email address and other required details in the form below.
User Name:
If you do not want to register, fill this field only and the name will be used as user name for your post.
Password
Please enter a password for your user account. Note that passwords are case-sensitive.
Password:
Confirm Password:
Email Address
Please enter a valid email address for yourself.
Email Address:

Log-in



Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Show Printable Version Show Printable Version
Email this Page Email this Page


Forum Jump: 

Posting Rules  
You may post new threads
You may post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off