Overclock.net › Forums › Software, Programming and Coding › Other Software › Sorting through a Huge CSV list
New Posts  All Forums:Forum Nav:

Sorting through a Huge CSV list

post #1 of 4
Thread Starter 
I have a list of ~22000 lines. I need to check for duplicate items =\\
How can I (in office 2007) sort through this list? They all have names, possible some dupes there but there may also be other labels that are the same in adjacent columns, how can I sort through this mess?
6600K
(12 items)
 
  
CPUMotherboardGraphicsRAM
i5-6600k 4.6GHz Gigabyte z170XP-SLi-CF Sapphire R9-280X Toxic Corsair DDR4 2133MHz @ 3,000MHz 
Hard DriveCoolingOSKeyboard
Intel 535 480GB SSD Noctua NH-D15 Windows 10 Pro 64bit Microsoft Natural 4000 
PowerCaseMouseMouse Pad
Corsair TX950 ThermalTake Armor Logitech G700 Ripper XXL 
  hide details  
Reply
6600K
(12 items)
 
  
CPUMotherboardGraphicsRAM
i5-6600k 4.6GHz Gigabyte z170XP-SLi-CF Sapphire R9-280X Toxic Corsair DDR4 2133MHz @ 3,000MHz 
Hard DriveCoolingOSKeyboard
Intel 535 480GB SSD Noctua NH-D15 Windows 10 Pro 64bit Microsoft Natural 4000 
PowerCaseMouseMouse Pad
Corsair TX950 ThermalTake Armor Logitech G700 Ripper XXL 
  hide details  
Reply
post #2 of 4
do you have any programming skills? it wouldn't be too hard to write a quick program that would traverse through the list and remove dups
Bravo
(13 items)
 
  
CPUMotherboardGraphicsRAM
X4 955 @ 3.6 Asus M479T 4870 1GB 8 GB DDR3 
Hard DriveOSPowerCase
160 GB Win 7 Antec 750w Antec 300 
  hide details  
Reply
Bravo
(13 items)
 
  
CPUMotherboardGraphicsRAM
X4 955 @ 3.6 Asus M479T 4870 1GB 8 GB DDR3 
Hard DriveOSPowerCase
160 GB Win 7 Antec 750w Antec 300 
  hide details  
Reply
post #3 of 4
Use Excel 2007 or Access.

Do you want to remove the duplicates?

If so, use Excel 2007-> Filter -> Advanced Filter -> Unique Records Only.

If you want to use Access, import the data into a table. Then run:
select distinct [key_column] into temptable from [table]

select * from [table] a
left join temptable b
on a.[key_column] = b.[key_column]




If you just want to flag duplicate values, import the file into Excel 2007. Sort the data on the key columns. Then in a free column paste =if(A1=A2,"","Duplicate!") (change A column to whatever column your key is in).
Edited by DuckieHo - 6/25/08 at 7:23am
Once again...
(13 items)
 
  
CPUMotherboardGraphicsRAM
i7 920 [4.28GHz, HT] Asus P6T + Broadcom NetXtreme II VisionTek HD5850 [900/1200] + Galaxy GT240 2x4GB G.Skill Ripjaw X [1632 MHz] 
Hard DriveOSMonitorKeyboard
Intel X25-M 160GB + 3xRAID0 500GB 7200.12 Window 7 Pro 64 Acer H243H + Samsung 226BW XARMOR-U9BL  
PowerCaseMouseMouse Pad
Antec Truepower New 750W Li Lian PC-V2100 [10x120mm fans] Logitech G9 X-Trac Pro 
  hide details  
Reply
Once again...
(13 items)
 
  
CPUMotherboardGraphicsRAM
i7 920 [4.28GHz, HT] Asus P6T + Broadcom NetXtreme II VisionTek HD5850 [900/1200] + Galaxy GT240 2x4GB G.Skill Ripjaw X [1632 MHz] 
Hard DriveOSMonitorKeyboard
Intel X25-M 160GB + 3xRAID0 500GB 7200.12 Window 7 Pro 64 Acer H243H + Samsung 226BW XARMOR-U9BL  
PowerCaseMouseMouse Pad
Antec Truepower New 750W Li Lian PC-V2100 [10x120mm fans] Logitech G9 X-Trac Pro 
  hide details  
Reply
post #4 of 4
Sounds like you're looking through GPS hotspots, good luck with that
My System
(13 items)
 
  
CPUMotherboardGraphicsRAM
QX9650 @ 3.66 EVGA Nvidia 780i 2x EVGA 9600GT 512mb OCZ Reaper 4GB DDR2 1066 
Hard DriveOSPowerCase
2x 500gb Seagate Barracuda 7200.11 Vista Ultimate 64 bit OCZ GameXStream 700W Antec Nine Hundred Black Steel 
  hide details  
Reply
My System
(13 items)
 
  
CPUMotherboardGraphicsRAM
QX9650 @ 3.66 EVGA Nvidia 780i 2x EVGA 9600GT 512mb OCZ Reaper 4GB DDR2 1066 
Hard DriveOSPowerCase
2x 500gb Seagate Barracuda 7200.11 Vista Ultimate 64 bit OCZ GameXStream 700W Antec Nine Hundred Black Steel 
  hide details  
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Other Software
Overclock.net › Forums › Software, Programming and Coding › Other Software › Sorting through a Huge CSV list