Overclock.net › Forums › Industry News › Software News › [Wired] Google’s Dremel Makes Big Data Look Small
New Posts  All Forums:Forum Nav:

[Wired] Google’s Dremel Makes Big Data Look Small

post #1 of 9
Thread Starter 
http://www.wired.com/wiredenterprise/2012/08/google-dremel-versus-hadoop/
https://developers.google.com/bigquery/
Quote:
The difference is that Dremel can handle web-sized amounts of data at blazing fast speed. According to Google’s paper, you can run queries on multiple petabytes — millions of gigabytes — in a matter of seconds.
...
“Dremel can execute many queries over such data that would ordinarily require a sequence of MapReduce jobs, but at a fraction of the execution time,” reads Google’s Dremel paper. Hölzle says it can run a query on a petabyte of data in about three seconds.


Woooooooooooooah. This is absolutely amazing..... Google figured out how to query billions of rows in seconds with SQL-like implementation.... in 2006!
Once again...
(13 items)
 
  
CPUMotherboardGraphicsRAM
i7 920 [4.28GHz, HT] Asus P6T + Broadcom NetXtreme II VisionTek HD5850 [900/1200] + Galaxy GT240 2x4GB G.Skill Ripjaw X [1632 MHz] 
Hard DriveOSMonitorKeyboard
Intel X25-M 160GB + 3xRAID0 500GB 7200.12 Window 7 Pro 64 Acer H243H + Samsung 226BW XARMOR-U9BL  
PowerCaseMouseMouse Pad
Antec Truepower New 750W Li Lian PC-V2100 [10x120mm fans] Logitech G9 X-Trac Pro 
  hide details  
Reply
Once again...
(13 items)
 
  
CPUMotherboardGraphicsRAM
i7 920 [4.28GHz, HT] Asus P6T + Broadcom NetXtreme II VisionTek HD5850 [900/1200] + Galaxy GT240 2x4GB G.Skill Ripjaw X [1632 MHz] 
Hard DriveOSMonitorKeyboard
Intel X25-M 160GB + 3xRAID0 500GB 7200.12 Window 7 Pro 64 Acer H243H + Samsung 226BW XARMOR-U9BL  
PowerCaseMouseMouse Pad
Antec Truepower New 750W Li Lian PC-V2100 [10x120mm fans] Logitech G9 X-Trac Pro 
  hide details  
Reply
post #2 of 9
Thought for a second Google paired with Dremel to come out with a new handheld rotary tool tongue.gif
Skunk_Works
(11 items)
 
unRAID Server
(8 items)
 
To Hot To Handle
(13 items)
 
CPUMotherboardRAMHard Drive
Intel Pentium Processor G3220  Asus H87I-Plus Corsair Vengeance WD Red 3TB 
Hard DriveHard DriveOSCase
WD Red 3TB WD Green 1TB unRAID 5.0 Fractal Design Node 304 
CPUMotherboardGraphicsRAM
Core 2 Duo P8700 Quanta 361B 9600M GT 4GB 
Hard DriveOS
300GB Windows 7 Pro 
  hide details  
Reply
Skunk_Works
(11 items)
 
unRAID Server
(8 items)
 
To Hot To Handle
(13 items)
 
CPUMotherboardRAMHard Drive
Intel Pentium Processor G3220  Asus H87I-Plus Corsair Vengeance WD Red 3TB 
Hard DriveHard DriveOSCase
WD Red 3TB WD Green 1TB unRAID 5.0 Fractal Design Node 304 
CPUMotherboardGraphicsRAM
Core 2 Duo P8700 Quanta 361B 9600M GT 4GB 
Hard DriveOS
300GB Windows 7 Pro 
  hide details  
Reply
post #3 of 9
no that is impressive

good find.

Edit, I too was expecting to find a picture of a Dremel with Google on the side ;p
Nameless Traitor
(14 items)
 
†he cu|†
(13 items)
 
 
CPUMotherboardGraphicsRAM
Core I7 920@3.8ghz Asus P6T-Deluxe GTX 570 1.2gb ;p Mushkin RedLine: 3x2GB 1600mhz 998805 
Hard DriveCoolingOSMonitor
128GB Crucial M4 SSD + 2 x 250GB + 500GB Extern... Noctua U12P Win7 Ultimate 64-bit 20in Wide LCD 1680x1050 
KeyboardPowerCaseMouse
Ducky OCN 1008 MX Brown MechBoard Corsair CMPSU-750HX HAF 932! Logitech G5 
Mouse Pad
Cool Warhammer Print 
CPUMotherboardGraphicsRAM
Opty 165 CCBBE @ 300fsb DFI LanP UT SLI-DR eVGA 8800gts 640mb 2x1gb Corsair XMS TwinX P 
Hard DriveOptical DriveOSMonitor
250 g sataII Max†or NEC DVD+/-DL XP pro 20.1 inch wide 5ms 
KeyboardPowerCaseMouse
Some old Thing OCZ GameXstream 600w Spire Baby! Logitech G5 
Mouse Pad
What? 
  hide details  
Reply
Nameless Traitor
(14 items)
 
†he cu|†
(13 items)
 
 
CPUMotherboardGraphicsRAM
Core I7 920@3.8ghz Asus P6T-Deluxe GTX 570 1.2gb ;p Mushkin RedLine: 3x2GB 1600mhz 998805 
Hard DriveCoolingOSMonitor
128GB Crucial M4 SSD + 2 x 250GB + 500GB Extern... Noctua U12P Win7 Ultimate 64-bit 20in Wide LCD 1680x1050 
KeyboardPowerCaseMouse
Ducky OCN 1008 MX Brown MechBoard Corsair CMPSU-750HX HAF 932! Logitech G5 
Mouse Pad
Cool Warhammer Print 
CPUMotherboardGraphicsRAM
Opty 165 CCBBE @ 300fsb DFI LanP UT SLI-DR eVGA 8800gts 640mb 2x1gb Corsair XMS TwinX P 
Hard DriveOptical DriveOSMonitor
250 g sataII Max†or NEC DVD+/-DL XP pro 20.1 inch wide 5ms 
KeyboardPowerCaseMouse
Some old Thing OCZ GameXstream 600w Spire Baby! Logitech G5 
Mouse Pad
What? 
  hide details  
Reply
post #4 of 9
Hahahahaha Facebook Engineering posted the other day "we increased MySQL queries to 2 MILLION a second... from 50k!"

Idiots
Main Rig.
(14 items)
 
Battlefield 2 review.
Battlefield 2 PC Game EA
 
CPUMotherboardGraphicsRAM
Bulldozer FX4100 Gigabyte GA-M68MT-S2 MSI 7770 Kingston Hyper X | 2 x 2 GB 
Hard DriveOptical DriveCoolingOS
WD 2 TB  LiteON DVD/CD R-W Drive AMD Standard cooler Windows 7 Ultimate 
MonitorKeyboardPowerCase
32' Luxor Full HD TV Microsoft comfort curce Antec 450 Watt OcUK Value case 
MouseAudio
Microsoft wireless mouse Logitech speakers & Bass 
  hide details  
Reply
Main Rig.
(14 items)
 
Battlefield 2 review.
Battlefield 2 PC Game EA
 
CPUMotherboardGraphicsRAM
Bulldozer FX4100 Gigabyte GA-M68MT-S2 MSI 7770 Kingston Hyper X | 2 x 2 GB 
Hard DriveOptical DriveCoolingOS
WD 2 TB  LiteON DVD/CD R-W Drive AMD Standard cooler Windows 7 Ultimate 
MonitorKeyboardPowerCase
32' Luxor Full HD TV Microsoft comfort curce Antec 450 Watt OcUK Value case 
MouseAudio
Microsoft wireless mouse Logitech speakers & Bass 
  hide details  
Reply
post #5 of 9
Thread Starter 
Quote:
Originally Posted by joshd View Post

Hahahahaha Facebook Engineering posted the other day "we increased MySQL queries to 2 MILLION a second... from 50k!"
Idiots
2M a second is still good.

Note that Dremel is OLAP... you can't write or update the data. MySQL is OLTP where data can be changed or inserted.
Once again...
(13 items)
 
  
CPUMotherboardGraphicsRAM
i7 920 [4.28GHz, HT] Asus P6T + Broadcom NetXtreme II VisionTek HD5850 [900/1200] + Galaxy GT240 2x4GB G.Skill Ripjaw X [1632 MHz] 
Hard DriveOSMonitorKeyboard
Intel X25-M 160GB + 3xRAID0 500GB 7200.12 Window 7 Pro 64 Acer H243H + Samsung 226BW XARMOR-U9BL  
PowerCaseMouseMouse Pad
Antec Truepower New 750W Li Lian PC-V2100 [10x120mm fans] Logitech G9 X-Trac Pro 
  hide details  
Reply
Once again...
(13 items)
 
  
CPUMotherboardGraphicsRAM
i7 920 [4.28GHz, HT] Asus P6T + Broadcom NetXtreme II VisionTek HD5850 [900/1200] + Galaxy GT240 2x4GB G.Skill Ripjaw X [1632 MHz] 
Hard DriveOSMonitorKeyboard
Intel X25-M 160GB + 3xRAID0 500GB 7200.12 Window 7 Pro 64 Acer H243H + Samsung 226BW XARMOR-U9BL  
PowerCaseMouseMouse Pad
Antec Truepower New 750W Li Lian PC-V2100 [10x120mm fans] Logitech G9 X-Trac Pro 
  hide details  
Reply
post #6 of 9
Quote:
Originally Posted by DuckieHo View Post

2M a second is still good.
Note that Dremel is OLAP... you can't write or update the data. MySQL is OLTP where data can be changed or inserted.

Ah.. that makes much more sense thumb.gif
Main Rig.
(14 items)
 
Battlefield 2 review.
Battlefield 2 PC Game EA
 
CPUMotherboardGraphicsRAM
Bulldozer FX4100 Gigabyte GA-M68MT-S2 MSI 7770 Kingston Hyper X | 2 x 2 GB 
Hard DriveOptical DriveCoolingOS
WD 2 TB  LiteON DVD/CD R-W Drive AMD Standard cooler Windows 7 Ultimate 
MonitorKeyboardPowerCase
32' Luxor Full HD TV Microsoft comfort curce Antec 450 Watt OcUK Value case 
MouseAudio
Microsoft wireless mouse Logitech speakers & Bass 
  hide details  
Reply
Main Rig.
(14 items)
 
Battlefield 2 review.
Battlefield 2 PC Game EA
 
CPUMotherboardGraphicsRAM
Bulldozer FX4100 Gigabyte GA-M68MT-S2 MSI 7770 Kingston Hyper X | 2 x 2 GB 
Hard DriveOptical DriveCoolingOS
WD 2 TB  LiteON DVD/CD R-W Drive AMD Standard cooler Windows 7 Ultimate 
MonitorKeyboardPowerCase
32' Luxor Full HD TV Microsoft comfort curce Antec 450 Watt OcUK Value case 
MouseAudio
Microsoft wireless mouse Logitech speakers & Bass 
  hide details  
Reply
post #7 of 9
Guess it's time to switch from our Hadoop/Hbase lovechild to Dremel!

We use a big data quite heavily at work for market segmentation and targetting via analytics. I can't image what something like Dremel could open up if integrated correctly smile.gif
post #8 of 9
Seems I'm not the only one who thought of a white powertool with coloured letters on it..
My ROG Rig
(17 items)
 
  
CPUMotherboardGraphicsRAM
FX-8120 @4.2ghz  ASUS Crosshair V Formula Sapphire HD7870 XT 2x4gb Red Vengeance 
Hard DriveOptical DriveCoolingOS
Seagate 1tb Lite-On iHas324 NZXT Havik 140 Windows 7 Profession 64-bit 
MonitorKeyboardPowerCase
ASUS MW221 21.5in Monitor 1680x1050 Compaq Corsair AX-850 CoolerMaster HAF X 
MouseMouse PadAudioAudio
Tt eSports Theron Razer Goliathus Control Audioengine A2 Audioengine DS1 
Audio
Xonar Phoebus  
  hide details  
Reply
My ROG Rig
(17 items)
 
  
CPUMotherboardGraphicsRAM
FX-8120 @4.2ghz  ASUS Crosshair V Formula Sapphire HD7870 XT 2x4gb Red Vengeance 
Hard DriveOptical DriveCoolingOS
Seagate 1tb Lite-On iHas324 NZXT Havik 140 Windows 7 Profession 64-bit 
MonitorKeyboardPowerCase
ASUS MW221 21.5in Monitor 1680x1050 Compaq Corsair AX-850 CoolerMaster HAF X 
MouseMouse PadAudioAudio
Tt eSports Theron Razer Goliathus Control Audioengine A2 Audioengine DS1 
Audio
Xonar Phoebus  
  hide details  
Reply
post #9 of 9
I'm pretty sure Bing has this too...

*pfhahaha* who am I kidding trolololol lachen.gif
My beloved system
(13 items)
 
  
CPUMotherboardGraphicsRAM
Q6600 @ 3.4 (1.4V) GA-P35-DS3 MSI Twin Frozr II/OC GTX560 Ti 2GB OCZ DDR2 800Mhz 
Hard DriveOptical DriveOSMonitor
80GB raptor + 160GB SG + 500GB SG DL Super Writemaster Vista Home 32bit 226BW SyncMaster Samsung 22" 
KeyboardPowerCaseMouse
Logitech 650W Be Quiet! Antec P180 G5 Laser Mouse v2 
  hide details  
Reply
My beloved system
(13 items)
 
  
CPUMotherboardGraphicsRAM
Q6600 @ 3.4 (1.4V) GA-P35-DS3 MSI Twin Frozr II/OC GTX560 Ti 2GB OCZ DDR2 800Mhz 
Hard DriveOptical DriveOSMonitor
80GB raptor + 160GB SG + 500GB SG DL Super Writemaster Vista Home 32bit 226BW SyncMaster Samsung 22" 
KeyboardPowerCaseMouse
Logitech 650W Be Quiet! Antec P180 G5 Laser Mouse v2 
  hide details  
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Software News
Overclock.net › Forums › Industry News › Software News › [Wired] Google’s Dremel Makes Big Data Look Small