Overclock.net › Forums › Industry News › Software News › [Wired] Google’s Dremel Makes Big Data Look Small
New Posts  All Forums:Forum Nav:

[Wired] Google’s Dremel Makes Big Data Look Small

post #1 of 9
8/16/12 at 9:09am
Thread Starter 
http://www.wired.com/wiredenterprise/2012/08/google-dremel-versus-hadoop/
https://developers.google.com/bigquery/
Quote:
The difference is that Dremel can handle web-sized amounts of data at blazing fast speed. According to Google’s paper, you can run queries on multiple petabytes — millions of gigabytes — in a matter of seconds.
...
“Dremel can execute many queries over such data that would ordinarily require a sequence of MapReduce jobs, but at a fraction of the execution time,” reads Google’s Dremel paper. Hölzle says it can run a query on a petabyte of data in about three seconds.


Woooooooooooooah. This is absolutely amazing..... Google figured out how to query billions of rows in seconds with SQL-like implementation.... in 2006!
Once again...
(13 items)
  		  
CPUMotherboardGraphicsRAM
i7 920 [4.28GHz, HT] Asus P6T + Broadcom NetXtreme II VisionTek HD5850 [900/1200] + Galaxy GT240 2x4GB G.Skill Ripjaw X [1632 MHz] 
Hard DriveOSMonitorKeyboard
Intel X25-M 160GB + 3xRAID0 500GB 7200.12 Window 7 Pro 64 Acer H243H + Samsung 226BW XARMOR-U9BL  
PowerCaseMouseMouse Pad
Antec Truepower New 750W Li Lian PC-V2100 [10x120mm fans] Logitech G9 X-Trac Pro 
View all
  hide details  
Reply
Once again...
(13 items)
  		  
CPUMotherboardGraphicsRAM
i7 920 [4.28GHz, HT] Asus P6T + Broadcom NetXtreme II VisionTek HD5850 [900/1200] + Galaxy GT240 2x4GB G.Skill Ripjaw X [1632 MHz] 
Hard DriveOSMonitorKeyboard
Intel X25-M 160GB + 3xRAID0 500GB 7200.12 Window 7 Pro 64 Acer H243H + Samsung 226BW XARMOR-U9BL  
PowerCaseMouseMouse Pad
Antec Truepower New 750W Li Lian PC-V2100 [10x120mm fans] Logitech G9 X-Trac Pro 
View all
  hide details  
Reply
post #2 of 9
8/16/12 at 9:11am
Thought for a second Google paired with Dremel to come out with a new handheld rotary tool tongue.gif
Skunk_Works
(11 items)
 
To Hot To Handle
(13 items)
  		 
CPUMotherboardGraphicsGraphics
Intel Core i7-2600K Asus P8Z68-V Pro Gen3 NVIDIA GeForce GTX 670 FTW+ NVIDIA GeForce GTX 560Ti 
RAMHard DriveHard DriveOS
Corsair Vengeance Corsair Performance Pro SSD Western Digital Green Windows 7 Pro 
PowerCaseMouse
Corsair AX750 Corsair Obsidean 800D Cyborg R.A.T. 7 
View all
CPUMotherboardGraphicsRAM
Core 2 Duo P8700 Quanta 361B 9600M GT 4GB 
Hard DriveOS
300GB Windows 7 Pro 
View all
  hide details  
Reply
Skunk_Works
(11 items)
 
To Hot To Handle
(13 items)
  		 
CPUMotherboardGraphicsGraphics
Intel Core i7-2600K Asus P8Z68-V Pro Gen3 NVIDIA GeForce GTX 670 FTW+ NVIDIA GeForce GTX 560Ti 
RAMHard DriveHard DriveOS
Corsair Vengeance Corsair Performance Pro SSD Western Digital Green Windows 7 Pro 
PowerCaseMouse
Corsair AX750 Corsair Obsidean 800D Cyborg R.A.T. 7 
View all
CPUMotherboardGraphicsRAM
Core 2 Duo P8700 Quanta 361B 9600M GT 4GB 
Hard DriveOS
300GB Windows 7 Pro 
View all
  hide details  
Reply
post #3 of 9
8/16/12 at 9:12am
  • StormX2
  • ♫♪.ılılıll|̲̅̅●̲̅̅|̲̅̅=̲̅̅|̲̅̅●̲̅̅|llılılı.♫♪
no that is impressive

good find.

Edit, I too was expecting to find a picture of a Dremel with Google on the side ;p
Nameless Traitor
(14 items)
 
Ultimate Rig Contest
(6 items)
 
†he cu|†
(13 items)
 
CPUMotherboardGraphicsRAM
Core I7 920@3.8ghz Asus P6T-Deluxe eVGA GTX 275 Mushkin RedLine: 3x2GB 1600mhz 998805 
Hard DriveCoolingOSMonitor
128GB Crucial M4 SSD + 2 x 250GB + 500GB Extern... Noctua U12P Win7 Ultimate 64-bit 20in Wide LCD 1680x1050 
KeyboardPowerCaseMouse
Ducky OCN 1008 MX Brown MechBoard Corsair CMPSU-750HX HAF 932! Logitech G5 
Mouse Pad
Cool Warhammer Print 
View all
CPUMotherboardGraphicsRAM
i7-3930k Rampage IV Extreme 2011 EVGA Titan Mushking Redline 16GB 4 x 4GB 2133 
Hard DriveCooling
Samsung 840 Pro 512GB Corsair H100i 
View all
CPUMotherboardGraphicsRAM
Opty 165 CCBBE @ 300fsb DFI LanP UT SLI-DR eVGA 8800gts 640mb 2x1gb Corsair XMS TwinX P 
Hard DriveOptical DriveOSMonitor
250 g sataII Max†or NEC DVD+/-DL XP pro 20.1 inch wide 5ms 
KeyboardPowerCaseMouse
Some old Thing OCZ GameXstream 600w Spire Baby! Logitech G5 
Mouse Pad
What? 
View all
  hide details  
Reply
Nameless Traitor
(14 items)
 
Ultimate Rig Contest
(6 items)
 
†he cu|†
(13 items)
 
CPUMotherboardGraphicsRAM
Core I7 920@3.8ghz Asus P6T-Deluxe eVGA GTX 275 Mushkin RedLine: 3x2GB 1600mhz 998805 
Hard DriveCoolingOSMonitor
128GB Crucial M4 SSD + 2 x 250GB + 500GB Extern... Noctua U12P Win7 Ultimate 64-bit 20in Wide LCD 1680x1050 
KeyboardPowerCaseMouse
Ducky OCN 1008 MX Brown MechBoard Corsair CMPSU-750HX HAF 932! Logitech G5 
Mouse Pad
Cool Warhammer Print 
View all
CPUMotherboardGraphicsRAM
i7-3930k Rampage IV Extreme 2011 EVGA Titan Mushking Redline 16GB 4 x 4GB 2133 
Hard DriveCooling
Samsung 840 Pro 512GB Corsair H100i 
View all
CPUMotherboardGraphicsRAM
Opty 165 CCBBE @ 300fsb DFI LanP UT SLI-DR eVGA 8800gts 640mb 2x1gb Corsair XMS TwinX P 
Hard DriveOptical DriveOSMonitor
250 g sataII Max†or NEC DVD+/-DL XP pro 20.1 inch wide 5ms 
KeyboardPowerCaseMouse
Some old Thing OCZ GameXstream 600w Spire Baby! Logitech G5 
Mouse Pad
What? 
View all
  hide details  
Reply
post #4 of 9
8/16/12 at 9:13am
Hahahahaha Facebook Engineering posted the other day "we increased MySQL queries to 2 MILLION a second... from 50k!"

Idiots
Main Rig.
(14 items)
 
Battlefield 2 review.
Battlefield 2 PC Game EA
 
CPUMotherboardGraphicsRAM
Bulldozer FX4100 Gigabyte GA-M68MT-S2 MSI 7770 Kingston Hyper X | 2 x 2 GB 
Hard DriveOptical DriveCoolingOS
WD 2 TB  LiteON DVD/CD R-W Drive AMD Standard cooler Windows 7 Ultimate 
MonitorKeyboardPowerCase
32' Luxor Full HD TV Microsoft comfort curce Antec 450 Watt OcUK Value case 
MouseAudio
Microsoft wireless mouse Logitech speakers & Bass 
View all
  hide details  
Reply
Main Rig.
(14 items)
 
Battlefield 2 review.
Battlefield 2 PC Game EA
 
CPUMotherboardGraphicsRAM
Bulldozer FX4100 Gigabyte GA-M68MT-S2 MSI 7770 Kingston Hyper X | 2 x 2 GB 
Hard DriveOptical DriveCoolingOS
WD 2 TB  LiteON DVD/CD R-W Drive AMD Standard cooler Windows 7 Ultimate 
MonitorKeyboardPowerCase
32' Luxor Full HD TV Microsoft comfort curce Antec 450 Watt OcUK Value case 
MouseAudio
Microsoft wireless mouse Logitech speakers & Bass 
View all
  hide details  
Reply
post #5 of 9
8/16/12 at 9:15am
Thread Starter 
Quote:
Originally Posted by joshd View Post

Hahahahaha Facebook Engineering posted the other day "we increased MySQL queries to 2 MILLION a second... from 50k!"
Idiots
2M a second is still good.

Note that Dremel is OLAP... you can't write or update the data. MySQL is OLTP where data can be changed or inserted.
Once again...
(13 items)
  		  
CPUMotherboardGraphicsRAM
i7 920 [4.28GHz, HT] Asus P6T + Broadcom NetXtreme II VisionTek HD5850 [900/1200] + Galaxy GT240 2x4GB G.Skill Ripjaw X [1632 MHz] 
Hard DriveOSMonitorKeyboard
Intel X25-M 160GB + 3xRAID0 500GB 7200.12 Window 7 Pro 64 Acer H243H + Samsung 226BW XARMOR-U9BL  
PowerCaseMouseMouse Pad
Antec Truepower New 750W Li Lian PC-V2100 [10x120mm fans] Logitech G9 X-Trac Pro 
View all
  hide details  
Reply
Once again...
(13 items)
  		  
CPUMotherboardGraphicsRAM
i7 920 [4.28GHz, HT] Asus P6T + Broadcom NetXtreme II VisionTek HD5850 [900/1200] + Galaxy GT240 2x4GB G.Skill Ripjaw X [1632 MHz] 
Hard DriveOSMonitorKeyboard
Intel X25-M 160GB + 3xRAID0 500GB 7200.12 Window 7 Pro 64 Acer H243H + Samsung 226BW XARMOR-U9BL  
PowerCaseMouseMouse Pad
Antec Truepower New 750W Li Lian PC-V2100 [10x120mm fans] Logitech G9 X-Trac Pro 
View all
  hide details  
Reply
post #6 of 9
8/16/12 at 9:23am
Quote:
Originally Posted by DuckieHo View Post

2M a second is still good.
Note that Dremel is OLAP... you can't write or update the data. MySQL is OLTP where data can be changed or inserted.

Ah.. that makes much more sense thumb.gif
Main Rig.
(14 items)
 
Battlefield 2 review.
Battlefield 2 PC Game EA
 
CPUMotherboardGraphicsRAM
Bulldozer FX4100 Gigabyte GA-M68MT-S2 MSI 7770 Kingston Hyper X | 2 x 2 GB 
Hard DriveOptical DriveCoolingOS
WD 2 TB  LiteON DVD/CD R-W Drive AMD Standard cooler Windows 7 Ultimate 
MonitorKeyboardPowerCase
32' Luxor Full HD TV Microsoft comfort curce Antec 450 Watt OcUK Value case 
MouseAudio
Microsoft wireless mouse Logitech speakers & Bass 
View all
  hide details  
Reply
Main Rig.
(14 items)
 
Battlefield 2 review.
Battlefield 2 PC Game EA
 
CPUMotherboardGraphicsRAM
Bulldozer FX4100 Gigabyte GA-M68MT-S2 MSI 7770 Kingston Hyper X | 2 x 2 GB 
Hard DriveOptical DriveCoolingOS
WD 2 TB  LiteON DVD/CD R-W Drive AMD Standard cooler Windows 7 Ultimate 
MonitorKeyboardPowerCase
32' Luxor Full HD TV Microsoft comfort curce Antec 450 Watt OcUK Value case 
MouseAudio
Microsoft wireless mouse Logitech speakers & Bass 
View all
  hide details  
Reply
post #7 of 9
8/27/12 at 3:24am
Guess it's time to switch from our Hadoop/Hbase lovechild to Dremel!

We use a big data quite heavily at work for market segmentation and targetting via analytics. I can't image what something like Dremel could open up if integrated correctly smile.gif
Gaming PC
(18 items)
  		  
CPUMotherboardGraphicsRAM
Intel Core i7 2600k Asus P8Z68V Pro MSI R7950 Twin Frozr Corsair Vengeance LP 1600mhz 
Hard DriveHard DriveHard DriveOptical Drive
Crucial C300 Samsung Spinpoint F3 Samsung Spinpoint F1 Samsung SH-222AB 
CoolingOSMonitorMonitor
Corsair H100 Windows 7 Professional X64 Dell U2312H Dell U2311H 
KeyboardPowerCaseMouse
Filco Majestouch (Blue) Antec TruePower 650W Corsair Obsidian 800D Logitech G500 
Mouse PadAudio
Steelseries QCK+ Asus Xonar Essence ST 
View all
  hide details  
Reply
Gaming PC
(18 items)
  		  
CPUMotherboardGraphicsRAM
Intel Core i7 2600k Asus P8Z68V Pro MSI R7950 Twin Frozr Corsair Vengeance LP 1600mhz 
Hard DriveHard DriveHard DriveOptical Drive
Crucial C300 Samsung Spinpoint F3 Samsung Spinpoint F1 Samsung SH-222AB 
CoolingOSMonitorMonitor
Corsair H100 Windows 7 Professional X64 Dell U2312H Dell U2311H 
KeyboardPowerCaseMouse
Filco Majestouch (Blue) Antec TruePower 650W Corsair Obsidian 800D Logitech G500 
Mouse PadAudio
Steelseries QCK+ Asus Xonar Essence ST 
View all
  hide details  
Reply
post #8 of 9
8/27/12 at 3:28am
Seems I'm not the only one who thought of a white powertool with coloured letters on it..
My ROG Rig
(17 items)
  		  
CPUMotherboardGraphicsRAM
FX-8120 @4.2ghz  ASUS Crosshair V Formula Sapphire HD7870 XT 2x4gb Red Vengeance 
Hard DriveOptical DriveCoolingOS
Seagate 1tb Lite-On iHas324 NZXT Havik 140 Windows 7 Profession 64-bit 
MonitorKeyboardPowerCase
ASUS MW221 21.5in Monitor 1680x1050 Compaq Corsair AX-850 CoolerMaster HAF X 
MouseMouse PadAudioAudio
Tt eSports Theron Razer Goliathus Control Audioengine A2 Audioengine DS1 
Audio
Xonar Phoebus  
View all
  hide details  
Reply
My ROG Rig
(17 items)
  		  
CPUMotherboardGraphicsRAM
FX-8120 @4.2ghz  ASUS Crosshair V Formula Sapphire HD7870 XT 2x4gb Red Vengeance 
Hard DriveOptical DriveCoolingOS
Seagate 1tb Lite-On iHas324 NZXT Havik 140 Windows 7 Profession 64-bit 
MonitorKeyboardPowerCase
ASUS MW221 21.5in Monitor 1680x1050 Compaq Corsair AX-850 CoolerMaster HAF X 
MouseMouse PadAudioAudio
Tt eSports Theron Razer Goliathus Control Audioengine A2 Audioengine DS1 
Audio
Xonar Phoebus  
View all
  hide details  
Reply
post #9 of 9
8/27/12 at 3:34am
I'm pretty sure Bing has this too...

*pfhahaha* who am I kidding trolololol lachen.gif
My beloved system
(13 items)
  		  
CPUMotherboardGraphicsRAM
Q6600 @ 3.4 (1.4V) GA-P35-DS3 MSI Twin Frozr II/OC GTX560 Ti 2GB OCZ DDR2 800Mhz 
Hard DriveOptical DriveOSMonitor
80GB raptor + 160GB SG + 500GB SG DL Super Writemaster Vista Home 32bit 226BW SyncMaster Samsung 22" 
KeyboardPowerCaseMouse
Logitech 650W Be Quiet! Antec P180 G5 Laser Mouse v2 
View all
  hide details  
Reply
My beloved system
(13 items)
  		  
CPUMotherboardGraphicsRAM
Q6600 @ 3.4 (1.4V) GA-P35-DS3 MSI Twin Frozr II/OC GTX560 Ti 2GB OCZ DDR2 800Mhz 
Hard DriveOptical DriveOSMonitor
80GB raptor + 160GB SG + 500GB SG DL Super Writemaster Vista Home 32bit 226BW SyncMaster Samsung 22" 
KeyboardPowerCaseMouse
Logitech 650W Be Quiet! Antec P180 G5 Laser Mouse v2 
View all
  hide details  
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Software News
Overclock.net › Forums › Industry News › Software News › [Wired] Google’s Dremel Makes Big Data Look Small