Overclock.net › Forums › Software, Programming and Coding › Coding and Programming › Login to OCN with Python
New Posts  All Forums:Forum Nav:

Login to OCN with Python

post #1 of 13
Thread Starter 
Hey fellow programmers. I am trying to write a program in python that monitors my subscription feed and sends me a text when it changes but I need to log on to view it. I have google'd around and every solution I have come across doesn't seem to work with OCN so if anyone could give me any insight or advice, it would be greatly appreciated. Here is what I have so far (still a work in progress):
Code:
#!/usr/bin/env python

import urllib, time, os, smtplib

url = "http://www.overclock.net/users/subscriptions/index/view/activity"
source = urllib.urlopen(url).read()
new_source=source 
counter=0
flag=True

while new_source==source:
    if flag:
        time.sleep(5)
    try:
       new_source = urllib.urlopen(url).read()
    except IOError:
        print "Error: cannot read URL"
    flag=False
    continue 
    counter+=1
  
    flag=True

fromaddr = 'myemail@overclocked.net'
toaddrs = "myphonenumber@messaging.sprintpcs.com"
body = "New Item Available!"

msg = 'From: myemail@overclocked.net\n\n' + body
 
smtp_server = 'email-smtp.us-east-1.amazonaws.com'
smtp_username = '*****'
smtp_password = '*****'
smtp_port = '587'
smtp_do_tls = True

server = smtplib.SMTP(
host = smtp_server,
port = smtp_port,
timeout = 10
)

server.starttls()
server.ehlo()
server.login(smtp_username, smtp_password)
server.sendmail(fromaddr, toaddrs, msg)

The code above works perfectly with pages that you don't need to be logged on to view so I only need help passing login credentials to OCN from python. Here is a bit of code that I found and tried but it didn't work. I don't have much experience at all with urllib and frankly most networking practices within python so I'm stuck on where to go from here.
Code:
#!/usr/bin/env python

import urllib2
import sys
import re
import base64
from urlparse import urlparse

theurl = 'http://www.overclock.net/users/subscriptions/index/view/activity'
# if you want to run this example you'll need to supply
# a protected page with your username and password

username = 'myOCNusername'
password = 'myOCNpassword'

req = urllib2.Request(theurl)
try:
    handle = urllib2.urlopen(req)
except IOError, e:
    # here we *want* to fail
    pass
else:
    # If we don't fail then the page isn't protected
    print "This page isn't protected by authentication."
    sys.exit(1)

if not hasattr(e, 'code') or e.code != 401:
    # we got an error - but not a 401 error
    print "This page isn't protected by authentication."
    print 'But we failed for another reason.'
    sys.exit(1)

authline = e.headers['www-authenticate']
# this gets the www-authenticate line from the headers
# which has the authentication scheme and realm in it


authobj = re.compile(
    r'''(?:\s*www-authenticate\s*:)?\s*(\w*)\s+realm=['"]([^'"]+)['"]''',
    re.IGNORECASE)
# this regular expression is used to extract scheme and realm
matchobj = authobj.match(authline)

if not matchobj:
    # if the authline isn't matched by the regular expression
    # then something is wrong
    print 'The authentication header is badly formed.'
    print authline
    sys.exit(1)

scheme = matchobj.group(1)
realm = matchobj.group(2)
# here we've extracted the scheme
# and the realm from the header
if scheme.lower() != 'basic':
    print 'This example only works with BASIC authentication.'
    sys.exit(1)

base64string = base64.encodestring(
                '%s:%s' % (username, password))[:-1]
authheader =  "Basic %s" % base64string
req.add_header("Authorization", authheader)
try:
    handle = urllib2.urlopen(req)
except IOError, e:
    # here we shouldn't fail if the username/password is right
    print "It looks like the username or password is wrong."
    sys.exit(1)
thepage = handle.read()

Thanks for any and all advice! thumb.gif
Edited by DizZz - 3/31/14 at 5:41pm
Project Arch
(16 items)
 
BigByte
(8 items)
 
Macbook Pro
(8 items)
 
CPUMotherboardGraphicsRAM
Intel i7 5960x @ 4.8ghz Asus X99-E WS 2 x Nvidia Quadro K6000 Corsair Dominator Platinum 64gb 2666 
Hard DriveCoolingOSMonitor
2 x Samsung 850 Pro 512gb {RAID 0} Swiftech H320 w/ Noctua NF-F12 iPPC Arch Linux w/ OpenBox 2 x LG 31MU97 Cinema 4K 
KeyboardPowerCaseMouse
Vortex KBC Poker II PBT EVGA SuperNOVA 1000 G2 Corsair Obsidian 450D Zowie FK1 
Mouse PadAudioAudioOther
Steelseries QCK+ Beyerdynamic DT 770 Pro 80Ω Audioengine D1 Intel X540-T2 NIC 
CPUMotherboardRAMHard Drive
Intel Avoton C2750 Octa-Core ASRock C2750D4I Samsung 64gb ECC 1600mhz 8 x Samsung 850 Pro 1TB 
OSPowerCaseOther
FreeNAS SilverStone ST45SF-G 450w SilverStone DS380 Intel X540-T2 NIC 
CPUGraphicsGraphicsRAM
Intel i7-4960HQ Intel Iris 5200 Pro NVIDIA GT 750M 2gb 16gb (2x8gb) 1600mhz 
Hard DriveMonitorMouseMouse Pad
1tb PCIe SSD 15.4" {2880x1800} IPS SteelSeries Sensei [RAW] SteelSeries QcK Mini 
  hide details  
Reply
Project Arch
(16 items)
 
BigByte
(8 items)
 
Macbook Pro
(8 items)
 
CPUMotherboardGraphicsRAM
Intel i7 5960x @ 4.8ghz Asus X99-E WS 2 x Nvidia Quadro K6000 Corsair Dominator Platinum 64gb 2666 
Hard DriveCoolingOSMonitor
2 x Samsung 850 Pro 512gb {RAID 0} Swiftech H320 w/ Noctua NF-F12 iPPC Arch Linux w/ OpenBox 2 x LG 31MU97 Cinema 4K 
KeyboardPowerCaseMouse
Vortex KBC Poker II PBT EVGA SuperNOVA 1000 G2 Corsair Obsidian 450D Zowie FK1 
Mouse PadAudioAudioOther
Steelseries QCK+ Beyerdynamic DT 770 Pro 80Ω Audioengine D1 Intel X540-T2 NIC 
CPUMotherboardRAMHard Drive
Intel Avoton C2750 Octa-Core ASRock C2750D4I Samsung 64gb ECC 1600mhz 8 x Samsung 850 Pro 1TB 
OSPowerCaseOther
FreeNAS SilverStone ST45SF-G 450w SilverStone DS380 Intel X540-T2 NIC 
CPUGraphicsGraphicsRAM
Intel i7-4960HQ Intel Iris 5200 Pro NVIDIA GT 750M 2gb 16gb (2x8gb) 1600mhz 
Hard DriveMonitorMouseMouse Pad
1tb PCIe SSD 15.4" {2880x1800} IPS SteelSeries Sensei [RAW] SteelSeries QcK Mini 
  hide details  
Reply
post #2 of 13
Did you look through overclock.net page source and page info?
Rampage 3 Formula
(23 items)
 
  
CPUMotherboardGraphicsRAM
Core i7 950 Asus Rampage lll Formula EVGA GTX980Ti ACX2.0 12 GB G-Skill 
Hard DriveHard DriveHard DriveOptical Drive
Intel 320 x 3 RAID 0 Seagate ST31000524NS x 2 Intel 520 Memorex 
CoolingOSMonitorMonitor
Corsair H70 Win7U 64-bit Planar PX2611W Planar PX2611W 
KeyboardPowerCaseMouse
Alps Wave Seasonic Platinum 1000W CM Cosmos S Logitech M510 
AudioAudioAudio
SoundBblaster X-FI Titanium HD Sennheiser PC350 Swan M-200 
  hide details  
Reply
Rampage 3 Formula
(23 items)
 
  
CPUMotherboardGraphicsRAM
Core i7 950 Asus Rampage lll Formula EVGA GTX980Ti ACX2.0 12 GB G-Skill 
Hard DriveHard DriveHard DriveOptical Drive
Intel 320 x 3 RAID 0 Seagate ST31000524NS x 2 Intel 520 Memorex 
CoolingOSMonitorMonitor
Corsair H70 Win7U 64-bit Planar PX2611W Planar PX2611W 
KeyboardPowerCaseMouse
Alps Wave Seasonic Platinum 1000W CM Cosmos S Logitech M510 
AudioAudioAudio
SoundBblaster X-FI Titanium HD Sennheiser PC350 Swan M-200 
  hide details  
Reply
post #3 of 13
Thread Starter 
Quote:
Originally Posted by ar3f View Post

Did you look through overclock.net page source and page info?

Yes I tried but I'm still unsure as to how to pass credentials. This is the important part of the page source but it didn't really help me get any closer to a solution.
Code:
<!-- Search Area -->
        <div id="search-area">
                <div id="login-area">
                                                                <div id="login-box">
                                <form id="loginform" name="loginform" action="/login" method="post">
                                                                                <input type="hidden" name="oldurl" id="oldurl" value="/adminv2/mod_queue/index/">

                                                                                <input type="text" id="username" name="username" placeholder="username or email" value="" />
                                                                                <input type="password" value="" id="password" name="password" placeholder="password" />

                                        <div id="remem-cont">
                                                <label class="check-label" for="rememberme">Remember Me</label>
                                                <input type="checkbox" id="rememberme" name="rememberme" checked="checked">
                                        </div>

                                        <input type="submit" name="Login" value="Login" class="btn"  />                           </form>

                                                                        <div class="forgot-pw"><a id="forgot-pw" href="/users/lost_password">Forgot Password?</a></div>
                                
                        </div>
                                </div>

Project Arch
(16 items)
 
BigByte
(8 items)
 
Macbook Pro
(8 items)
 
CPUMotherboardGraphicsRAM
Intel i7 5960x @ 4.8ghz Asus X99-E WS 2 x Nvidia Quadro K6000 Corsair Dominator Platinum 64gb 2666 
Hard DriveCoolingOSMonitor
2 x Samsung 850 Pro 512gb {RAID 0} Swiftech H320 w/ Noctua NF-F12 iPPC Arch Linux w/ OpenBox 2 x LG 31MU97 Cinema 4K 
KeyboardPowerCaseMouse
Vortex KBC Poker II PBT EVGA SuperNOVA 1000 G2 Corsair Obsidian 450D Zowie FK1 
Mouse PadAudioAudioOther
Steelseries QCK+ Beyerdynamic DT 770 Pro 80Ω Audioengine D1 Intel X540-T2 NIC 
CPUMotherboardRAMHard Drive
Intel Avoton C2750 Octa-Core ASRock C2750D4I Samsung 64gb ECC 1600mhz 8 x Samsung 850 Pro 1TB 
OSPowerCaseOther
FreeNAS SilverStone ST45SF-G 450w SilverStone DS380 Intel X540-T2 NIC 
CPUGraphicsGraphicsRAM
Intel i7-4960HQ Intel Iris 5200 Pro NVIDIA GT 750M 2gb 16gb (2x8gb) 1600mhz 
Hard DriveMonitorMouseMouse Pad
1tb PCIe SSD 15.4" {2880x1800} IPS SteelSeries Sensei [RAW] SteelSeries QcK Mini 
  hide details  
Reply
Project Arch
(16 items)
 
BigByte
(8 items)
 
Macbook Pro
(8 items)
 
CPUMotherboardGraphicsRAM
Intel i7 5960x @ 4.8ghz Asus X99-E WS 2 x Nvidia Quadro K6000 Corsair Dominator Platinum 64gb 2666 
Hard DriveCoolingOSMonitor
2 x Samsung 850 Pro 512gb {RAID 0} Swiftech H320 w/ Noctua NF-F12 iPPC Arch Linux w/ OpenBox 2 x LG 31MU97 Cinema 4K 
KeyboardPowerCaseMouse
Vortex KBC Poker II PBT EVGA SuperNOVA 1000 G2 Corsair Obsidian 450D Zowie FK1 
Mouse PadAudioAudioOther
Steelseries QCK+ Beyerdynamic DT 770 Pro 80Ω Audioengine D1 Intel X540-T2 NIC 
CPUMotherboardRAMHard Drive
Intel Avoton C2750 Octa-Core ASRock C2750D4I Samsung 64gb ECC 1600mhz 8 x Samsung 850 Pro 1TB 
OSPowerCaseOther
FreeNAS SilverStone ST45SF-G 450w SilverStone DS380 Intel X540-T2 NIC 
CPUGraphicsGraphicsRAM
Intel i7-4960HQ Intel Iris 5200 Pro NVIDIA GT 750M 2gb 16gb (2x8gb) 1600mhz 
Hard DriveMonitorMouseMouse Pad
1tb PCIe SSD 15.4" {2880x1800} IPS SteelSeries Sensei [RAW] SteelSeries QcK Mini 
  hide details  
Reply
post #4 of 13
I'm no expert on web-related subjects, but when in doubt, use Wireshark to see exactly what's being passed back and forth. And OCN doesn't use HTTPS for secure login anyway, so you'll be able to see everything in plaintext.
post #5 of 13
Quote:
Originally Posted by The Hundred Gunner View Post

I'm no expert on web-related subjects, but when in doubt, use Wireshark to see exactly what's being passed back and forth. And OCN doesn't use HTTPS for secure login anyway, so you'll be able to see everything in plaintext.

Not specifically aimed at you Gunner, but I've replied to you because I thought you might be interested in the reply smile.gif

Wireshark, while undoubtedly an excellent tool and I understand your logic in suggesting it, I think in this instance it would would be a little overkill for this scenario. However there are similar tools for tracking HTTP/S requests based around setting up a HTTP proxy.

The tool I've used the most is Fiddler, which is free and works exceptionally well. Sadly it's Windows only, but there are cross platforms Java equivalents which also do the job (I've tried WebScarab; which works well enough too).

Honestly though (and this is aimed specifically at the OP), I doubt you'll need a proxy to track HTTP requests as you can tell the from the URL and hyperlink that it's a standard GET request with no query string; so the only way Huddler / OCN could track you would be via IP + user agent (dangerous) or by cookies (which is what every other forum uses). So all you need to do is view the cookies in your Firefox (or whatever browser you use) and paste them verbatim into your Python code (as you probably already know, Firefox allows you to search through cookies site by site within it's standard preference settings window - so you wouldn't even need to install any plugins to check. Chrome also shows cookies in the developer tools).

If that doesn't work, then include a Firefox / Chrome user agent string in your Python script.

The only drawback to this is that you're hard coding your cookies rather than manually logging in. But honest, the time you'd spend writing a log in routine vs how infrequently your session cookies would expire at the server side (and also how easy it is to grab updated cookie strings), it wouldn't be worth your effort in writing extra boilerplate code to log in.

One last thing; if you didn't know this already, OCN has the option to fire you e-mails when subscribed threads have unread content. This might not fit your use case; but it's worth baring in mind if you weren't already aware.
post #6 of 13
Quote:
Originally Posted by Plan9 View Post

Wireshark, while undoubtedly an excellent tool and I understand your logic in suggesting it, I think in this instance it would would be a little overkill for this scenario. However there are similar tools for tracking HTTP/S requests based around setting up a HTTP proxy.

I told you web isn't my forte... I forgot about proxies. There's also OWASP ZAP, but I guess that's a bit more geared toward pentesting compared to WebScarab.
post #7 of 13
Quote:
Originally Posted by The Hundred Gunner View Post

I told you web isn't my forte... I forgot about proxies. There's also OWASP ZAP, but I guess that's a bit more geared toward pentesting compared to WebScarab.

Yeah, ZAP is something different again. FYI, WebScarab is an OWASP project as well. smile.gif
post #8 of 13
Thread Starter 
Awesome thanks for the responses guys! I will and hard code cookies into the program tonight and see if I can get that working.
Project Arch
(16 items)
 
BigByte
(8 items)
 
Macbook Pro
(8 items)
 
CPUMotherboardGraphicsRAM
Intel i7 5960x @ 4.8ghz Asus X99-E WS 2 x Nvidia Quadro K6000 Corsair Dominator Platinum 64gb 2666 
Hard DriveCoolingOSMonitor
2 x Samsung 850 Pro 512gb {RAID 0} Swiftech H320 w/ Noctua NF-F12 iPPC Arch Linux w/ OpenBox 2 x LG 31MU97 Cinema 4K 
KeyboardPowerCaseMouse
Vortex KBC Poker II PBT EVGA SuperNOVA 1000 G2 Corsair Obsidian 450D Zowie FK1 
Mouse PadAudioAudioOther
Steelseries QCK+ Beyerdynamic DT 770 Pro 80Ω Audioengine D1 Intel X540-T2 NIC 
CPUMotherboardRAMHard Drive
Intel Avoton C2750 Octa-Core ASRock C2750D4I Samsung 64gb ECC 1600mhz 8 x Samsung 850 Pro 1TB 
OSPowerCaseOther
FreeNAS SilverStone ST45SF-G 450w SilverStone DS380 Intel X540-T2 NIC 
CPUGraphicsGraphicsRAM
Intel i7-4960HQ Intel Iris 5200 Pro NVIDIA GT 750M 2gb 16gb (2x8gb) 1600mhz 
Hard DriveMonitorMouseMouse Pad
1tb PCIe SSD 15.4" {2880x1800} IPS SteelSeries Sensei [RAW] SteelSeries QcK Mini 
  hide details  
Reply
Project Arch
(16 items)
 
BigByte
(8 items)
 
Macbook Pro
(8 items)
 
CPUMotherboardGraphicsRAM
Intel i7 5960x @ 4.8ghz Asus X99-E WS 2 x Nvidia Quadro K6000 Corsair Dominator Platinum 64gb 2666 
Hard DriveCoolingOSMonitor
2 x Samsung 850 Pro 512gb {RAID 0} Swiftech H320 w/ Noctua NF-F12 iPPC Arch Linux w/ OpenBox 2 x LG 31MU97 Cinema 4K 
KeyboardPowerCaseMouse
Vortex KBC Poker II PBT EVGA SuperNOVA 1000 G2 Corsair Obsidian 450D Zowie FK1 
Mouse PadAudioAudioOther
Steelseries QCK+ Beyerdynamic DT 770 Pro 80Ω Audioengine D1 Intel X540-T2 NIC 
CPUMotherboardRAMHard Drive
Intel Avoton C2750 Octa-Core ASRock C2750D4I Samsung 64gb ECC 1600mhz 8 x Samsung 850 Pro 1TB 
OSPowerCaseOther
FreeNAS SilverStone ST45SF-G 450w SilverStone DS380 Intel X540-T2 NIC 
CPUGraphicsGraphicsRAM
Intel i7-4960HQ Intel Iris 5200 Pro NVIDIA GT 750M 2gb 16gb (2x8gb) 1600mhz 
Hard DriveMonitorMouseMouse Pad
1tb PCIe SSD 15.4" {2880x1800} IPS SteelSeries Sensei [RAW] SteelSeries QcK Mini 
  hide details  
Reply
post #9 of 13
There has to be some code since web browsers (FFox, for ex) can automatically log in to overclock.net.
I'd try to find the procedure in whatever open source for and implement it in Python.

Here's a way from: https://stackoverflow.com/questions/7513569/auto-login-in-python-using-mechanize


from urllib import urlencode

from urllib2 import Request, urlopen

req = Request('www.site.com',urlencode({'user':'userhere', 'password':'passwordhere'}))

open = urlopen(req)

Replace 'user' and 'password' with whatever overclock uses
Edited by ar3f - 4/4/14 at 10:57pm
Rampage 3 Formula
(23 items)
 
  
CPUMotherboardGraphicsRAM
Core i7 950 Asus Rampage lll Formula EVGA GTX980Ti ACX2.0 12 GB G-Skill 
Hard DriveHard DriveHard DriveOptical Drive
Intel 320 x 3 RAID 0 Seagate ST31000524NS x 2 Intel 520 Memorex 
CoolingOSMonitorMonitor
Corsair H70 Win7U 64-bit Planar PX2611W Planar PX2611W 
KeyboardPowerCaseMouse
Alps Wave Seasonic Platinum 1000W CM Cosmos S Logitech M510 
AudioAudioAudio
SoundBblaster X-FI Titanium HD Sennheiser PC350 Swan M-200 
  hide details  
Reply
Rampage 3 Formula
(23 items)
 
  
CPUMotherboardGraphicsRAM
Core i7 950 Asus Rampage lll Formula EVGA GTX980Ti ACX2.0 12 GB G-Skill 
Hard DriveHard DriveHard DriveOptical Drive
Intel 320 x 3 RAID 0 Seagate ST31000524NS x 2 Intel 520 Memorex 
CoolingOSMonitorMonitor
Corsair H70 Win7U 64-bit Planar PX2611W Planar PX2611W 
KeyboardPowerCaseMouse
Alps Wave Seasonic Platinum 1000W CM Cosmos S Logitech M510 
AudioAudioAudio
SoundBblaster X-FI Titanium HD Sennheiser PC350 Swan M-200 
  hide details  
Reply
post #10 of 13
Quote:
Originally Posted by ar3f View Post

There has to be some code since web browsers (FFox, for ex) can automatically log in to overclock.net.
I'd try to find the procedure in whatever open source for and implement it in Python.

Here's a way from: https://stackoverflow.com/questions/7513569/auto-login-in-python-using-mechanize


from urllib import urlencode

from urllib2 import Request, urlopen

req = Request('www.site.com',urlencode({'user':'userhere', 'password':'passwordhere'}))

open = urlopen(req)

Replace 'user' and 'password' with whatever overclock uses

I'd already answered how browsers auto login, via cookies. Browsers wouldn't enter the username and password on each page request.

Your example code would not only fail, but you're also broadcasting confidential login details in clear text, which is grossly insecure.

Edit, in fact even in the link you gave, the top answer stated you need session cookies.
Edited by Plan9 - 4/5/14 at 12:49am
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Coding and Programming
Overclock.net › Forums › Software, Programming and Coding › Coding and Programming › Login to OCN with Python