Overclock.net › Forums › Software, Programming and Coding › Networking & Security › How to find every URL?
New Posts  All Forums:Forum Nav:

How to find every URL?

post #1 of 3
Thread Starter 
I know there's a program like this, just not sure what it's called.

It basically finds every url and directory within a website.

Example:

I give it "a.com"

and it finds every directory

a.com/a
a.com/b
a.com/c
a.com/d
a.com/e
...
etc

a.com/a/1
a.com/a/2
a.com/a/3
etc

And it finds every single one.

Does anyone know what this kind of program is called, or the name of what I'm thinking of?
    
CPUMotherboardGraphicsRAM
AMD Phenom II X6 1090T Black Edition ASUS M4A89GTD PRO/USB3 XFX HD-577A-ZNFC Radeon HD 5770 G.SKILL 8gb (2x4gb) 1333mhz 
Hard DriveOSMonitorKeyboard
WD Caviar Black WD1002FAEX 1TB 7200 R Windows 7 Home Premium (none yet) Nothing special 
PowerCaseMouse
CORSAIR CMPSU-850TX 850W COOLER MASTER HAF 922 Logitech G700 
  hide details  
Reply
    
CPUMotherboardGraphicsRAM
AMD Phenom II X6 1090T Black Edition ASUS M4A89GTD PRO/USB3 XFX HD-577A-ZNFC Radeon HD 5770 G.SKILL 8gb (2x4gb) 1333mhz 
Hard DriveOSMonitorKeyboard
WD Caviar Black WD1002FAEX 1TB 7200 R Windows 7 Home Premium (none yet) Nothing special 
PowerCaseMouse
CORSAIR CMPSU-850TX 850W COOLER MASTER HAF 922 Logitech G700 
  hide details  
Reply
post #2 of 3
Web Crawler/Spider?
BlueRaven
(13 items)
 
Dead BIOS
(9 items)
 
 
CPUMotherboardGraphicsRAM
Intel Core i7 G74Sx NVIDIA GeForce GTX 560M  Samsung  
RAMRAMHard DriveHard Drive
Samsung  Samsung  Momentus 7200.4 Scorpio Blue 
Mouse
Logitech G700 
  hide details  
Reply
BlueRaven
(13 items)
 
Dead BIOS
(9 items)
 
 
CPUMotherboardGraphicsRAM
Intel Core i7 G74Sx NVIDIA GeForce GTX 560M  Samsung  
RAMRAMHard DriveHard Drive
Samsung  Samsung  Momentus 7200.4 Scorpio Blue 
Mouse
Logitech G700 
  hide details  
Reply
post #3 of 3
You can use HTTrack which downloads the entire site to a local folder and you can break out all the directories. It's not 100% accurate, but it works. Be careful of any copyright violations***
Pwnisher
(15 items)
 
Small Hoss
(14 items)
 
 
CPUMotherboardGraphicsRAM
Intel i7 3770k GIGABYTE GA-Z77X-UP4 TH GTX 460 SE 16GB Samsung MV-3V4G3D/US 
Hard DriveHard DriveOptical DriveCooling
120GB Agility 3 1.5TB Seagate Barracuda LG 14x Blu-ray burner Corsair H100i 
OSMonitorKeyboardPower
Windows 8 Professional Dell Ultrasharp U2410 Logitech Illuminated keyboard Kingwin LZP-750 
CaseMouseMouse Pad
Corsair 650D Logitech G500 RadPadz XT 
CPUMotherboardGraphicsGraphics
Intel Core i3 540 @ 4.2GHz 1.328v Gigabyte GA-P55-UD4P EVGA GTX 460 SE SLI EVGA GTX 460 SE SLI 
RAMHard DriveOSMonitor
G.Skill 8GB 4x 2GB DDR3 OCZ Agility 3 120GB SSD Windows 7 Professional Dell Ultrasharp U2410 sRGB 
KeyboardPowerCaseMouse
Logitech Illuminated Keyboard PC Power & Cooling 750w Antec 1200 Logitech G500 
Mouse PadAudio
RatPadz OnBoard 
  hide details  
Reply
Pwnisher
(15 items)
 
Small Hoss
(14 items)
 
 
CPUMotherboardGraphicsRAM
Intel i7 3770k GIGABYTE GA-Z77X-UP4 TH GTX 460 SE 16GB Samsung MV-3V4G3D/US 
Hard DriveHard DriveOptical DriveCooling
120GB Agility 3 1.5TB Seagate Barracuda LG 14x Blu-ray burner Corsair H100i 
OSMonitorKeyboardPower
Windows 8 Professional Dell Ultrasharp U2410 Logitech Illuminated keyboard Kingwin LZP-750 
CaseMouseMouse Pad
Corsair 650D Logitech G500 RadPadz XT 
CPUMotherboardGraphicsGraphics
Intel Core i3 540 @ 4.2GHz 1.328v Gigabyte GA-P55-UD4P EVGA GTX 460 SE SLI EVGA GTX 460 SE SLI 
RAMHard DriveOSMonitor
G.Skill 8GB 4x 2GB DDR3 OCZ Agility 3 120GB SSD Windows 7 Professional Dell Ultrasharp U2410 sRGB 
KeyboardPowerCaseMouse
Logitech Illuminated Keyboard PC Power & Cooling 750w Antec 1200 Logitech G500 
Mouse PadAudio
RatPadz OnBoard 
  hide details  
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Networking & Security
Overclock.net › Forums › Software, Programming and Coding › Networking & Security › How to find every URL?