[G3D] Professional Overclocker Finds A Way To Cheat Benchmarks - Overclock.net - An Overclocking Community

Forum Jump: 

[G3D] Professional Overclocker Finds A Way To Cheat Benchmarks

 
Thread Tools
post #1 of 9 (permalink) Old 01-25-2018, 11:21 AM - Thread Starter
Well, hello there...
 
MacG32's Avatar
 
Join Date: Feb 2010
Location: PT US
Posts: 4,313
Rep: 298 (Unique: 219)
[G3D] Professional Overclocker Finds A Way To Cheat Benchmarks

Quote:
Pro overclocker Ronaldo Buassali, working together with GALAX has published a video in which he shows a way to hide driver settings modifications. This results in false higher end scores. The issue is not isolated to the 3DMark suite.

Source

Primus
(24 items)
pfSense Firewall
(10 items)
CPU
Intel - Core i9-9900K 3.6GHz 8-Core (4.7M-5S GHz Max Boost)
Motherboard
Gigabyte - Z390 Aorus Master LGA1151 Intel ATX
GPU
Gigabyte - GTX 1080 Ti Founders Edition 11GB 352-Bit GDDR5X
RAM
G.Skill - 2x Flare X 32GB (4x8GB) 3200MHz F4-3200C14D-16GFX
Hard Drive
Samsung - 960 Pro 512GB M.2-2280 PCI-E v3.0 x4 NVMe SSD
Hard Drive
Western Digital - Black 6TB 3.5" 7200RPM SATA 6Gb/s
Hard Drive
Samsung - 850 Pro 512GB 2.5" SATA III 3D NAND SSD
Optical Drive
Asus - BW-12B1ST 12X SATA CD/DVD/Blu-ray Writer
Power Supply
EVGA - SuperNOVA P2 650W 80+ Platinum Fully Modular
Cooling
Corsair - Hydro H100i Pro Liquid 240mm All-In-One
Case
Cougar - Panzer Max ATX Full Tower
Operating System
Microsoft - Windows 10 Professional 64-bit
Monitor
Acer - 2x G7 G247HYL 23.8" IPS 4ms 1920 x 1080 60Hz
Keyboard
Cooler Master - MasterKeys Pro S Cherry MX Red w/Ducky Keycaps
Mouse
Logitech - G502 Proteus Spectrum RGB Tunable Gaming
Mousepad
Fellowes - Black w/Microban
Audio
Realtek - ALC1220 - 7.1Ch 32bit/192KHz 120dB SNR HD
Other
Logitech - C920 HD Pro Widescreen 1080p Webcam
Other
Thrustmaster - T.Flight Stick X Joystick
Other
Canon - PIXMA MG3120 All-In-One Copier/Printer/Scanner
Other
Sennheiser - HD 650 Reference Class Headphones
Other
Logitech - Z623 200W 2.1Ch THX-Certified Speakers
Other
Thermal Grizzly - Kryonaut Top Grade Thermal Compound
Other
Shure - MV5 Professional Quality Condenser Microphone
CPU
Intel - Core i7-7700 3.6GHz 4-Core (4.2GHz Max Boost)
Motherboard
ASRock - Z270M-ITX/ac LGA1151 Intel Mini ITX
RAM
Kingston - HyperX Predator 8GB 2400MHz DDR4 CL12
Hard Drive
ADATA - Premier Pro SP600 64GB 2.5" SSD
Power Supply
Silverstone - 300W 80+ Bronze Certified SFX
Cooling
Noctua - NH-L9i 33.8 CFM
Case
Cooler Master - Elite 110 Mini ITX
Operating System
pfSense Latest Version
Monitor
BenQ - GW2270 21.5" 1920x1080 60Hz
Keyboard
Qumax - Scorpius V6 Wired Mini
▲ hide details ▲


MacG32 is online now  
Sponsored Links
Advertisement
 
post #2 of 9 (permalink) Old 01-25-2018, 11:44 AM
Stock *ahem*
 
Quantum Reality's Avatar
 
Join Date: Nov 2008
Posts: 6,380
Rep: 307 (Unique: 237)
I remember a very old version of this issue where if you had PhysX enabled some versions of 3DMark wouldn't take that into account, and your score would look way larger than it should be.
Quantum Reality is offline  
post #4 of 9 (permalink) Old 01-26-2018, 06:32 AM
Tank destroyer and a god
 
Offler's Avatar
 
Join Date: Dec 2012
Posts: 2,225
Rep: 82 (Unique: 64)
Quote: Originally Posted by Kimir View Post
This ain't cheating as HWbot rules allows it. >_>
Thats actually very visible on scores that it was happenning. However it does tamper with benchmark configuration and because of this reason the score is fake.

However its not the only way how to cause benchmark to give better scores, unrelated to the real performance. There is wide range of tricks.

There are methods how to apply configuration set before the test begin, however it might have unwanted effects on the system as a whole.

--- Building in progress * AMD Threadripper 1900x * Asrock X399M Taichi * Radeon VII * Gskill Xflare / Samsung B-die 3200 14-14-14-32 * Samsung 970 PRO 512gb * Fractal Design Node 804 * Seasonic Prime Ultra 850 Titanium *
--- Desktop * AMD Phenom II x6 1090t @ 3,8GHz * ASUS M5A99FX PRO R2.0 * Gigabyte R9 FuryX * A-Data XPG 2.0 / Elpida Hyper MNH 1866 @ 1600 6-6-6-18 * LSI 9211-8i / Raid 0 / 5x Corsair Force 3 GS * Creative Xfi Fatal1ty * Intel I210-T1 * Steelseries 6Gv2 * Logitech MX518 * Samsung U28E590D *
Offler is offline  
post #5 of 9 (permalink) Old 01-26-2018, 07:59 AM
uh oh.
 
Kimir's Avatar
 
Join Date: Jan 2012
Location: France
Posts: 5,049
Rep: 254 (Unique: 159)
Quote: Originally Posted by Offler View Post
Thats actually very visible on scores that it was happenning. However it does tamper with benchmark configuration and because of this reason the score is fake.

However its not the only way how to cause benchmark to give better scores, unrelated to the real performance. There is wide range of tricks.

There are methods how to apply configuration set before the test begin, however it might have unwanted effects on the system as a whole.
Yes it is visible, but that doesn't make it illegal/fake as altering LOD is an allowed optimization as per the rules on HWbot, the platform that run the benchmark competition.

However the fact the sysinfo from Futuremark doesn't detect the tweak because it scan the parameter before the bench run is silly, they could simply add another verification at the end.

You can see on Firestrike ultra top30 that I keep up with, follow the bot rules and LOD is allowed, my own run are with modified LOD (prior to sysinfo change in detection, so it doesn't show up) and latest high score by Gunslinger have the LOD tweak too, he didn't bother to apply those after the detection, because that's stupid to do so when the rules allow it.
The statement that it "results in false higher end scores" is just wrong.
If we are talking about a competition point of view, well everyone in the pro/elite league knows about lod tweak. nothing to see here, moving on.
But if we are talking about the average joe expecting incredible result like Vince when he advertise about the Kinpin card... Well that's a different story.

That video might be new, but that's old news to me. However he (Ronaldo) was the one that reported the bug with platform clock tuning (the RTC bug), making some benchmark illegal to run on win8/10 (see this). So this might make the rules change, or the way the detection of such tweak is done by the benchmark softs.
Kimir is offline  
post #6 of 9 (permalink) Old 01-26-2018, 08:09 AM
Tank destroyer and a god
 
Offler's Avatar
 
Join Date: Dec 2012
Posts: 2,225
Rep: 82 (Unique: 64)
Quote: Originally Posted by Kimir View Post
Yes it is visible, but that doesn't make it illegal/fake as altering LOD is an allowed optimization as per the rules on HWbot, the platform that run the benchmark competition.

However the fact the sysinfo from Futuremark doesn't detect the tweak because it scan the parameter before the bench run is silly, they could simply add another verification at the end.

You can see on Firestrike ultra top30 that I keep up with, follow the bot rules and LOD is allowed, my own run are with modified LOD (prior to sysinfo change in detection, so it doesn't show up) and latest high score by Gunslinger have the LOD tweak too, he didn't bother to apply those after the detection, because that's stupid to do so when the rules allow it.
The statement that it "results in false higher end scores" is just wrong.
If we are talking about a competition point of view, well everyone in the pro/elite league knows about lod tweak. nothing to see here, moving on.
But if we are talking about the average joe expecting incredible result like Vince when he advertise about the Kinpin card... Well that's a different story.

That video might be new, but that's old news to me. However he (Ronaldo) was the one that reported the bug with platform clock tuning (the RTC bug), making some benchmark illegal to run on win8/10 (see this). So this might make the rules change, or the way the detection of such tweak is done by the benchmark softs.
Well i recognize few types of tuning.

1. If you manage to run CS on DX 7.0 to remove all shaders and have upper hand in deathmatch.
If its not normally allowed ingame setting, and you get caught...

2. You disable HPET in Win10 and Bios.
Measuring hit topmost limit (frame limit) and you are so happy for having more FPS. Except without that time benchmarking gets bugged, while you are not getting any real FPS anyway. You are fooling just yourself.

This is exacly the case he mentioned, except because ppl compete in OC scores, its falls to case 1.

3. Overclocking, shorter timing, better drivers
You will get real performance boosts/better efficiency.

Proper benchmarks should test fixed scenarios.

--- Building in progress * AMD Threadripper 1900x * Asrock X399M Taichi * Radeon VII * Gskill Xflare / Samsung B-die 3200 14-14-14-32 * Samsung 970 PRO 512gb * Fractal Design Node 804 * Seasonic Prime Ultra 850 Titanium *
--- Desktop * AMD Phenom II x6 1090t @ 3,8GHz * ASUS M5A99FX PRO R2.0 * Gigabyte R9 FuryX * A-Data XPG 2.0 / Elpida Hyper MNH 1866 @ 1600 6-6-6-18 * LSI 9211-8i / Raid 0 / 5x Corsair Force 3 GS * Creative Xfi Fatal1ty * Intel I210-T1 * Steelseries 6Gv2 * Logitech MX518 * Samsung U28E590D *
Offler is offline  
post #7 of 9 (permalink) Old 01-27-2018, 11:53 AM
OC...D
 
Jpmboy's Avatar
 
Join Date: Apr 2012
Location: Philadelphia
Posts: 25,811
Rep: 1246 (Unique: 553)
Quote: Originally Posted by Offler View Post
Well i recognize few types of tuning.

1. If you manage to run CS on DX 7.0 to remove all shaders and have upper hand in deathmatch.
If its not normally allowed ingame setting, and you get caught...

2. You disable HPET in Win10 and Bios.
Measuring hit topmost limit (frame limit) and you are so happy for having more FPS. Except without that time benchmarking gets bugged, while you are not getting any real FPS anyway. You are fooling just yourself.

This is exacly the case he mentioned, except because ppl compete in OC scores, its falls to case 1.

3. Overclocking, shorter timing, better drivers
You will get real performance boosts/better efficiency.

Proper benchmarks should test fixed scenarios.
and those fixed scenarios are to know how to get the highest score without doing illegal tricks such as RTC hacks or any hack that fools the "timers" involved. LOD and tessellation tweaks in 3D benchmarks using non-driver based tools were introduced mainly because AMD drivers provided these dials in the standard driver-user interface settings, NVidia did not. Tweaks are not tricking the benchmark or code, they are settings provided within the benchmark or code. There is a difference.

ASUS Z370/Z390 Motherboard Support Thread, Coffee Lake OC Guide
Rampage VI Apex / 7980XE / 2 Titan V / 32GB 4000C16/Corsair AX1500i/Intel 900P

For Sale
X299
(25 items)
x99
(18 items)
Z370 Z390
(23 items)
CPU
7980XE
CPU
7740X
CPU
7740X
Motherboard
ASUS Rampage VI Apex
Motherboard
Gigabyte Aorus G9
Motherboard
Gigabyte x299 Gaming
GPU
2x Nvidia Titan V
GPU
GTX 1080
RAM
G. Skill 2x(2x8GB) 3600c15 @4000c16
RAM
Team Group
Hard Drive
INtel Optane 900P
Hard Drive
2x256 Plextor Raid 0
Hard Drive
WD Raptor
Hard Drive
Samsung Sm 961
Power Supply
Corsair AX1500i
Power Supply
EVGA 850 G3
Power Supply
Corsair HX1200i
Cooling
Aquacomputer GiGant 1680 (4x420)
Cooling
Koolance EXC-800
Case
Microcool Banchetto 101 (Alu)
Operating System
Win 10 Pro
Operating System
Win 7 Pro
Monitor
ROG Swift
Keyboard
Ducky
Mouse
Spatha
CPU
6950X
Motherboard
ASUS Rampage V Extreme-10
GPU
Titan Xp
GPU
Titan Xp
RAM
G Skill 8x8GB 3200c14 @3400c13
Hard Drive
Samsung 960 Pro
Optical Drive
ASUS BR/DVD/CD
Power Supply
Corsair AX1500i
Cooling
2x360 Aq Copper Rads
Cooling
Aquaero 6
Cooling
Koolance ERM-3K3U (9-fan mora)
Cooling
Ek Full Cover MB waterblock
Case
CaseLabs Mercury S8
Operating System
Win 10 Pro
Operating System
Win 7 Pro
Monitor
AGON 32 inch
Keyboard
Das Pro
Mouse
Logitech G700
CPU
8700K
CPU
9700K
CPU
8086K
Motherboard
ASUS Maximus X Apex
Motherboard
ASUS Maximus XI Extreme
GPU
GTX 1080
GPU
GTX 1070Ti FTW2
GPU
Titan V
RAM
2x8 GB G.Skill 4400c19
RAM
G.Skill 3600c16 4x6GB
Hard Drive
Intel 750
Optical Drive
PLextor DVD/CD
Power Supply
Corsair AX1500i
Power Supply
Corsair HX 1200i
Power Supply
Seasonic Titanium 850W
Cooling
360Rad + Chiller
Cooling
Koolance ERM-3K3U
Case
Corasir 570
Operating System
Win 10
Operating System
Windows 7 Pro
Monitor
55" 4K
Keyboard
HyperX Alloy FPS Pro
Mouse
SteelSeries Rival
▲ hide details ▲


Jpmboy is online now  
post #8 of 9 (permalink) Old 01-27-2018, 01:45 PM
Daisy Chained Dumb Switch
 
navjack27's Avatar
 
Join Date: Aug 2015
Location: Bath, New York
Posts: 1,227
Rep: 50 (Unique: 39)
Don't even... Disabling hpet... You mean force people to force enable it, causing actual lower performance. I'm sick of that responsiveness breaking, FPS lowering, totally wrong function being banded about as if it should be used by default.

8700k
(10 items)
CPU
Intel Core i7 8700k
Motherboard
Gigabyte Z370 AORUS Gaming 7
GPU
EVGA GeForce GTX 1080 Ti 11GB FTW3 GAMING iCX
RAM
Corsair Vengeance LPX 32GB CMK32GX4M4A2666C15
Hard Drive
SanDisk Ultra II 480GB 2.5"
Hard Drive
PNY CS1311 960GB 2.5"
Power Supply
SeaSonic PRIME Titanium 850W 80+ Titanium Certified Fully-Modular ATX Power Supply
Cooling
Noctua NH-D15
Case
Fractal Design Define S ATX Mid Tower Case
Operating System
Windows 10 Pro
▲ hide details ▲


navjack27 is offline  
post #9 of 9 (permalink) Old 01-27-2018, 04:19 PM
Tank destroyer and a god
 
Offler's Avatar
 
Join Date: Dec 2012
Posts: 2,225
Rep: 82 (Unique: 64)
Quote: Originally Posted by navjack27 View Post
Don't even... Disabling hpet... You mean force people to force enable it, causing actual lower performance. I'm sick of that responsiveness breaking, FPS lowering, totally wrong function being banded about as if it should be used by default.

Well, there are people making Overwatch benchmarks. With HPET... 120-150 fps on lowest settings. Without it, it hits 300 frame limit.

Making a cross reference testing with different FPS measuring method would show, HPET really took away 50% of total PC performance, but nobody ever bothers to be critical on the results? Arent the measurements actually wrong? NAAAAH

--- Building in progress * AMD Threadripper 1900x * Asrock X399M Taichi * Radeon VII * Gskill Xflare / Samsung B-die 3200 14-14-14-32 * Samsung 970 PRO 512gb * Fractal Design Node 804 * Seasonic Prime Ultra 850 Titanium *
--- Desktop * AMD Phenom II x6 1090t @ 3,8GHz * ASUS M5A99FX PRO R2.0 * Gigabyte R9 FuryX * A-Data XPG 2.0 / Elpida Hyper MNH 1866 @ 1600 6-6-6-18 * LSI 9211-8i / Raid 0 / 5x Corsair Force 3 GS * Creative Xfi Fatal1ty * Intel I210-T1 * Steelseries 6Gv2 * Logitech MX518 * Samsung U28E590D *
Offler is offline  
Reply

Quick Reply
Message:
Options

Register Now

In order to be able to post messages on the Overclock.net - An Overclocking Community forums, you must first register.
Please enter your desired user name, your email address and other required details in the form below.
User Name:
If you do not want to register, fill this field only and the name will be used as user name for your post.
Password
Please enter a password for your user account. Note that passwords are case-sensitive.
Password:
Confirm Password:
Email Address
Please enter a valid email address for yourself.
Email Address:

Log-in



Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Show Printable Version Show Printable Version
Email this Page Email this Page


Forum Jump: 

Posting Rules  
You may post new threads
You may post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off