Overclock.net banner

[ENGADGET] Samsung caught boosting benchmark performance numbers on Exynos devices (UPDATED)

7K views 100 replies 55 participants last post by  anothergeek 
#1 ·
http://www.engadget.com/2013/07/30/samsung-benchmarks/
Quote:
Our friends at Anandtech started sifting through data for two Exynos octacore variants of the Galaxy S 4, and discovered multiple benchmarks that appear to run those devices at higher CPU and GPU clock speeds than they normally run. Some of the apps included in the report are GLBench 2.5.1, AnTuTu, Quadrant and Linpack. Coincidentally, the site also found code within the GS4 that indicates the existence of a program called "BenchmarkBooster," which is responsible for overclocking the processors when certain apps are running.
Lol that is hilarious.

Update:
Quote:
Update: Samsung has posted a short response on its official Korean-language blog, proclaiming its innocence, saying that the GS4 is designed to run at a higher clock speed on many full-screen apps, and that benchmarking apps simply fall into that category. On the other hand, it hasn't explained the "BenchmarkBooster" code, so we're still waiting on a more detailed explanation from the manufacturer.

2nd update: We now have an English language response from Samsung, which accepts that benchmark apps may run at higher clock speeds than some games, but again denies that this is a deliberate attempt to distort specific scores:

"Under ordinary conditions, the GALAXY S4 has been designed to allow a maximum GPU frequency of 533MHz. However, the maximum GPU frequency is lowered to 480MHz for certain gaming apps that may cause an overload, when they are used for a prolonged period of time in full-screen mode. Meanwhile, a maximum GPU frequency of 533MHz is applicable for running apps that are usually used in full-screen mode, such as the S Browser, Gallery, Camera, Video Player, and certain benchmarking apps, which also demand substantial performance.

The maximum GPU frequencies for the GALAXY S4 have been varied to provide optimal user experience for our customers, and were not intended to improve certain benchmark results. Samsung Electronics remains committed to providing our customers with the best possible user experience."
Moderator Edit: Please use only the original title given in the article. Title has been amended.
 
#4 ·
Quote:
Originally Posted by CynicalUnicorn View Post

Well that's not sketchy at all. Also there's an ad for an S4 at the bottom of the page. Could there be a worse place to put it?
That's not an ad. That's Engadget's "spec box" that pulls the specifications for the device in the article.
 
#6 ·
Quote:
the site also found code within the GS4 that indicates the existence of a program called "BenchmarkBooster," which is responsible for overclocking the processors when certain apps are running
Why are companies this dumb? That's like naming my Prawn folder 'Prawn'.

If I had an app to artificially inflate my scores for benchmark purposes...the last thing I'd call it is 'benchmark booster'.
 
#7 ·
Quote:
Originally Posted by IcedEarth View Post

Why are companies this dumb? That's like naming my Prawn folder 'Prawn'.

If I had an app to artificially inflate my scores for benchmark purposes...the last thing I'd call it is 'benchmark booster'.
Well if you have a folder full of prawn photos then would you not label it prawns? It would make it easier to find.

 
#10 ·
They're not fraudulent at all actually. It's a bit shady, but if those are fraudulent, then all of our CPU benchmarks are fraudulent.

Fraudulent would be like if I ran Heaven 4.0 benchmark with my GTX 670, but photoshopped a higher FPS score onto my results page at the end and used that in place of the real deal.

The fact is, those scores are obtained using the hardware, and oddly enough, that sort of performance is limited to the benchmarks. Nothing a little rooting can't fix.

The scores have not been altered at all.
 
#11 ·
Quote:
Originally Posted by lacrossewacker View Post

They're not fraudulent at all actually. It's a bit shady, but if those are fraudulent, then all of our CPU benchmarks are fraudulent.

Fraudulent would be like if I ran Heaven 4.0 benchmark with my GTX 670, but photoshopped a higher FPS score onto my results page at the end and used that in place of the real deal.

The fact is, those scores are obtained using the hardware, and oddly enough, that sort of performance is limited to the benchmarks. Nothing a little rooting can't fix.

The scores have not been altered at all.
Really depends on how you convey those benchmarks, to be honest. If you're comparing them to other devices at stock and make no note of how your own device isn't, then that's clearly misleading - possibly fraudulent.
 
#13 ·
Quote:
Originally Posted by Miki View Post

Yeah, pretty shady. Doesn't change the fact that Samsung makes good phones, however.

I mean, I've never met one person that said, "Okay, before I buy this, let me benchmark it..." Then based the purchase off of that. In other words, no relevancy whatsoever.
You mean I was a little hasty when I threw my S3 off the wall immediately after reading this?
 
#14 ·
Shady, sure...fraudulent I don't know. Are the programs themselves hard coded or can any program get onto that list and utilize the boost?
 
#15 ·
It is definitely fraudulent. They advertised their phone with specific benchmarks in mind to show off performance - performance that was only obtainable given benchmarking situations and not day to day performance.

It is like AMD giving out benchmarks showing it gets consistent 60 FPS during a game, but upon buying it, you only realize you get 45. Then when you realize you have to overclock it to reach the performance promised, you realize you voided your warranty.
.
Really shady and unethical move by Samsung. Especially considering all the hype about their pr0ce3zz sp33d generated on this forum. Misleading consumers with fake data is a garbage move.
 
#17 ·
Quote:
Originally Posted by j3st3r View Post

It is like AMD giving out benchmarks showing it gets consistent 60 FPS during a game, but upon buying it, you only realize you get 45. Then when you realize you have to overclock it to reach the performance promised, you realize you voided your warranty.
.
Really shady and unethical move by Samsung. Especially considering all the hype about their pr0ce3zz sp33d generated on this forum. Misleading consumers with fake data is a garbage move.
A realization inside a realization? Realiception?
tongue.gif
 
#19 ·
Ok...can now devs of CPU/GPU intensive applications add some code to fool the hardware into thinking it is a benchmark and boost itself already?

Hope they were already doing that, unless all the programmers capable of doing/finding that went to Anandtech
(whoever was left from those able to fool NSA and the Patriot's Act, as all those are proven to be in Apple)

What's next?
x86 CPUs that speedstep to cheat in benchmarks, or GPUs that boost when temps allow and loads require?
Those satan's in samsung might even roll out those little-big core things to cheat even more!
THIS IS OF THE DEVIL!
 
#20 ·
Quote:
Originally Posted by lacrossewacker View Post

They're not fraudulent at all actually. It's a bit shady, but if those are fraudulent, then all of our CPU benchmarks are fraudulent.

Fraudulent would be like if I ran Heaven 4.0 benchmark with my GTX 670, but photoshopped a higher FPS score onto my results page at the end and used that in place of the real deal.

The fact is, those scores are obtained using the hardware, and oddly enough, that sort of performance is limited to the benchmarks. Nothing a little rooting can't fix.

The scores have not been altered at all.
You make a good point.

Frankly, I think this is sweet. I hope to see phones start making this standard, actually. If the company knows the system can be clocked much higher then it should, having it built into the system to overclock itself for gaming, etc. Can't wait for this to be standard.

Software overclocks per application would be insane. You can get amazing battery life app wise. Not like you need a lot of horse power for light operations. And as years go by, the company could just implement a free performance boost with their last update.
 
#21 ·
Quote:
Originally Posted by pcfoo View Post

Ok...can now devs of CPU/GPU intensive applications add some code to fool the hardware into thinking it is a benchmark and boost itself already?

Hope they were already doing that, unless all the programmers capable of doing/finding that went to Anandtech
(whoever was left from those able to fool NSA and the Patriot's Act, as all those are proven to be in Apple)

What's next?
x86 CPUs that speedstep to cheat in benchmarks, or GPUs that boost when temps allow and loads require?
Those satan's in samsung might even roll out those little-big core things to cheat even more!
THIS IS OF THE DEVIL!
Wonder if Apple is doing anything behind the back ground too... Always questionable how other companies get a hold of their GPUs and the performance numbers are radically different, despite a much superior CPU. Let alone the lack of open instruction sets, etc. It's soooo odd!
 
#22 ·
Quote:
Originally Posted by j3st3r View Post

It is definitely fraudulent. They advertised their phone with specific benchmarks in mind to show off performance - performance that was only obtainable given benchmarking situations and not day to day performance.

It is like AMD giving out benchmarks showing it gets consistent 60 FPS during a game, but upon buying it, you only realize you get 45. Then when you realize you have to overclock it to reach the performance promised, you realize you voided your warranty.
.
Really shady and unethical move by Samsung. Especially considering all the hype about their pr0ce3zz sp33d generated on this forum. Misleading consumers with fake data is a garbage move.
I didnt see the initial advertisement you are speaking of but from what you are explaining, it doesnt make sense to me. So Samsung shows off benchmark scores in an advertisement. You buy the phone, run the same benchmark and get the same result, correct? How is that fraudulent?

According to the original article @ anandtech (the source of your source) it states:
Quote:
Running any games, even the most demanding titles, returned a GPU frequency of 480MHz - just like @AndreiF alleged. Samsung never publicly claimed max GPU frequencies for the Exynos 5 Octa (our information came from internal sources), so no harm no foul thus far.
As for their take on the CPU
Quote:
Note that the CPU behavior is different from what we saw on the GPU side however. These CPU frequencies are available for all apps to use, they are simply forced to maximum (and in the case of Snapdragon, all cores are plugged in) in the case of these benchmarks.
It is not mentioned in the engadget article but this is not just for the exynos -
Quote:
Interestingly enough, the same behavior (on the CPU side) can be found on Qualcomm versions of the Galaxy S 4 as well.
Just playing Devil's Advocate here, not really a fanboy of anything. The same thing happened and continues to happen with GPU benchmarks and computer gaming. I guess the uproar is over how it is the first time it was caught in a mobile phone and how blatant it was?
 
#23 ·
Quote:
Originally Posted by Domino View Post

You make a good point.

Frankly, I think this is sweet. I hope to see phones start making this standard, actually. If the company knows the system can be clocked much higher then it should, having it built into the system to overclock itself for gaming, etc. Can't wait for this to be standard.

Software overclocks per application would be insane. You can get amazing battery life app wise. Not like you need a lot of horse power for light operations. And as years go by, the company could just implement a free performance boost with their last update.
No he makes a dumb point actually.

The reality is like this: I am in the market for a 7950. AMD releases benchmarks showcasing their 7950 as having 100 FPS in Skyrim. Upon buying the 7950, I only get 80 FPS. I am like wut why. AMD then says "Oops yeah that is weird but the 7950 we used and labled as a 7950 was actually a 7950 boost edition."

That is not "sweet" and that should not be a "standard" - tricking customers is not "sweet" or "cool". This is a trashy move. But alas, the customer loyalty shows up, even when they are being openly bent over.
 
#25 ·
it isnt being overclocked, it is the reverse, by default the soc is underclocked or prefers the a7 cluster. On specific benchmarks they release the limitations to show best performance possible otherwise it is throttled down probably for power use. I dont know if it is shady, intel had antutu use icc to maximize their scores and they use sdp numbers to lull people into believing the hype, thats shady.
 
This is an older thread, you may not receive a response, and could be reviving an old thread. Please consider creating a new thread.
Top