Help with downloading a file using urllib in python - Page 2 - Overclock.net - An Overclocking Community

Forum Jump: 

Help with downloading a file using urllib in python

Reply
 
Thread Tools
post #11 of 14 (permalink) Old 12-19-2016, 05:39 PM
New to Overclock.net
 
Join Date: Jul 2009
Location: Perth, WA
Posts: 106
Rep: 8 (Unique: 8)
Nice catch biggrin.gif don't know what I'd do without Fiddler! Looks much simpler/cleaner thumb.gif

Big lez is love
cheers.gifAUSSIE OCN CLUBcheers.gif
S0ULphIRE is offline  
Sponsored Links
Advertisement
 
post #12 of 14 (permalink) Old 12-22-2016, 12:31 AM
New to Overclock.net
 
Mrzev's Avatar
 
Join Date: Feb 2008
Location: Texas
Posts: 2,258
Rep: 96 (Unique: 76)
Ohh, i just use the debugging stuff in chrome for that kinda stuff. The Network tab. I have never used or seen fiddler, is that worth checking out?



Mrzev is offline  
post #13 of 14 (permalink) Old 12-24-2016, 09:00 AM - Thread Starter
New to Overclock.net
 
Join Date: Jun 2011
Location: Philly!
Posts: 3,409
Rep: 100 (Unique: 88)
Quote:
Originally Posted by Mrzev View Post

Ohh, i just use the debugging stuff in chrome for that kinda stuff. The Network tab. I have never used or seen fiddler, is that worth checking out?

Yea it's actually pretty good. Setting it up was a little pain though.
theturbofd is offline  
Sponsored Links
Advertisement
 
post #14 of 14 (permalink) Old 12-27-2016, 03:54 PM
New to Overclock.net
 
Mrzev's Avatar
 
Join Date: Feb 2008
Location: Texas
Posts: 2,258
Rep: 96 (Unique: 76)
Well, i tried out the fiddler and it worked out well. Took me a minute to figure out how it worked. Kinda like Wireshark.

But yeah, i actually had to do this today at work. I originally wrote it in python and then i realized i needed to convert it to C# so my asp site can query the info so i can use it like a rest interface.....

Mine was a bit more annoying because it was hitting an ASP site which made things weird. I also spent 2 hours trying to figure out why it wasnt working only to figure out i had txtUsername instead of txtUserName....

My code sucks, but im in a rush and it worked ...


Python Version
Code:
session  = requests.session()
r=session.get(URL,headers=headerVals )
soup=bs(r.content, "html.parser")
dataFields = {'__VIEWSTATE':(soup.find(id="__VIEWSTATE")['value']),
                          '__VIEWSTATEGENERATOR':(soup.find(id="__VIEWSTATEGENERATOR")['value']),
                          '__EVENTVALIDATION':(soup.find(id="__EVENTVALIDATION")['value']),
                          'tzo':'6',
                          'txtUserName':'xxxxxxx',
                          'txtPassword':'xxxxxxx',
                          'Button1':"Login"
                         
                          }
r=session.post(loginURL,data=dataFields,headers=headerVals, cookies=r.cookies.get_dict())
soup=bs(r.content, "html.parser")
print(soup)



This is the C# version
Code:
public ActionResult GetGPS()
        {

            HttpClient client = new HttpClient();
            Uri loginPage = new Uri("http://BLAHBLAH.com/Login.aspx");
            Uri postPage = new Uri("http://BLAHBLAH.com/RANDOMPATH/Login.aspx?ReturnUrl=%2fRANDOMPATH.aspx%2fHistory%3fid%3d10");


            var responseString = client.GetStringAsync(loginPage);

            while (!responseString.IsCompleted)
            {
                Thread.Sleep(1000);
            }

            var values = new Dictionary<string, string>
                {
              {"tzo","6"},
              {"txtUserName","xxxxxx"},
              {"txtPassword","xxxxx"},
              { "Button1","Login"   }
                };

            HtmlDocument document = new HtmlDocument();
            document.LoadHtml(responseString.Result);
            HtmlNodeCollection collection = document.DocumentNode.SelectNodes("//input");
            foreach (HtmlNode link in collection)
            {

                if(link.Id == "__VIEWSTATE")
                {
                    values.Add("__VIEWSTATE", link.Attributes["value"].Value);
                }
                if (link.Id == "__VIEWSTATEGENERATOR")
                {
                    values.Add("__VIEWSTATEGENERATOR", link.Attributes["value"].Value);
                }
                if (link.Id == "__EVENTVALIDATION")
                {
                    values.Add("__EVENTVALIDATION", link.Attributes["value"].Value);
                }
                
            }

            var content = new FormUrlEncodedContent(values);

            var response = client.PostAsync(postPage, content);
            while (!response.IsCompleted)
            {
                Thread.Sleep(1000);
            }


            var responseString2 = response.Result.Content.ReadAsStringAsync();
            while (!responseString2.IsCompleted)
            {
                Thread.Sleep(1000);
            }

            string test = responseString2.Result;
          
            return this.Content(test, "application/json"); ;

        }



Mrzev is offline  
Reply

Quick Reply
Message:
Options

Register Now

In order to be able to post messages on the Overclock.net - An Overclocking Community forums, you must first register.
Please enter your desired user name, your email address and other required details in the form below.
User Name:
If you do not want to register, fill this field only and the name will be used as user name for your post.
Password
Please enter a password for your user account. Note that passwords are case-sensitive.
Password:
Confirm Password:
Email Address
Please enter a valid email address for yourself.
Email Address:

Log-in



Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Show Printable Version Show Printable Version
Email this Page Email this Page


Forum Jump: 

Posting Rules  
You may post new threads
You may post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off