C# HttpwebRequest cookies n' stuff - Overclock.net

kdbolt70 · 06-25-07

Allright, so I'm writing an application to simulate the actions of a user in a web based program we've written. I want to write it for load testing, meaning I'll be hitting the server with thousands of requests at once.

But before I get trigger happy, I need to get my httpwebrequests working properly. The basic idea is I read some XML out of a file that is exactly what would be sent to the server if a user attempted to login though the web. (this was captured through fiddler). I want the server to think I am attempting to log in, with the proper username/password, and send me the proper xml response in return. I had written it properly as described, but I kept getting a "Session Timed Out or Not Available" XML response from the server. So now I'm thinking it needs me to initialize the session by sending a cookie (that is usually recieved initially when loading the login page) with the request to login. My problem is that I am not able to send a blank request(to simulate loading the login page) and apply the cookie recieved in the second web request (to simulate the actual logging in). In fact, I can't seem to get two consecutive requests to run at all. I think if I figure that out I'll be in better shape. I am able to comment out the initial web request/response and just send the login request and it works fine, except for the "Session Timed Out or Not Available". When I try to do the initial web request/response and then do the login, I get an error thrown saying "System.IO.IOException: Unable to write data to the transport connection: An established connection was aborted by the software in your host machine."

Here's code:

Code:

using System;
using System.Net;
using System.IO;
using System.Text;
using System.Web;
using System.Security.Cryptography.X509Certificates;

namespace ping
{
    class SimpleHttpClient
    {
        public static byte[] StrToByteArray(string str)
        {
            System.Text.ASCIIEncoding encoding = new System.Text.ASCIIEncoding();
            return encoding.GetBytes(str);
        }

        [STAThread]
        public void login()
        {
            //string gatewayUrl = "http://p8dev01web:7001/core/pswsapi/pswsapi.asp";
            string inputXmlQueryFilePath = "C:/Documents and Settings/krainbolt/My Documents/Visual Studio 2005/Projects/serverTestApp/serverTestApp/logon.txt";
            StreamWriter output = new StreamWriter(@"C:/Documents and Settings/krainbolt/My Documents/Visual Studio 2005/Projects/serverTestApp/serverTestApp\\Output.txt", false);
            StreamWriter soutput = new StreamWriter(@"C:/Documents and Settings/krainbolt/My Documents/Visual Studio 2005/Projects/serverTestApp/serverTestApp\\startOut.txt", false);
            string cookieHeader = null;


            try
            {
                

                HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://myserver:80/core/");
                request.Method = "Post";
                CookieContainer cookieJar = new CookieContainer();
                request.CookieContainer = cookieJar;
                byte[] requestBytes = StrToByteArray(" ");
                request.ContentLength = requestBytes.Length;
                Stream requestStream = request.GetRequestStream();
                requestStream.Write(requestBytes, 0, requestBytes.Length);
                requestStream.Close();

                HttpWebResponse response = (HttpWebResponse)request.GetResponse();
                Stream responseData = response.GetResponseStream();
                StreamReader reader = new StreamReader(responseData);
                string responseFromServer = reader.ReadToEnd();
                
                cookieHeader = response.Cookies.ToString();
                soutput.Write(cookieHeader);
                soutput.Write(responseFromServer);
                soutput.Write(response.Headers[4]);
                responseData.Close();
                soutput.Close();
                reader.Close();
                response.Close();

                
                
                HttpWebRequest request1 = (HttpWebRequest)WebRequest.Create("http://myserver:80/login.asp");
                // read input XML query from file
                string inputXml;
                using (StreamReader inputQueryReader = new StreamReader(inputXmlQueryFilePath))
                {
                    inputXml = inputQueryReader.ReadToEnd();
                }
             
                byte[] request1Bytes = StrToByteArray(inputXml);
                request1.Method = "Post";
                //request1.CookieContainer.Add(response.Cookies);
                request1.ContentLength = request1Bytes.Length;
                Stream request1Stream = request1.GetRequestStream();
                //output.Write(request1.CookieContainer.ToString());
                request1Stream.Write(request1Bytes, 0, request1Bytes.Length);
                request1Stream.Close();

                HttpWebResponse response1 = (HttpWebResponse)request1.GetResponse();
                Stream response1Data = response1.GetResponseStream();
                StreamReader reader1 = new StreamReader(response1Data);
                string response1FromServer = reader1.ReadToEnd();

                
                output.Write(response1FromServer);
                response1.Close();
                reader1.Close();
                output.Close();

                //
                
            }
            catch (Exception e)
            {
                output.Write("Exception: " + e);
                output.Close();
            }
            
        }

    }
}

I'm thinking there must be a way to keep the connection alive and use the same httpwebrequest object for both calls, rather then use two separate ones. Right now I'm not even trying to deal with the cookies, I just want to get the sequential server requests working. Then I'll deal with either using two objects, or modifying one to send a second request to the server? any suggestions?

kdbolt70 · 06-25-07

heh, probably not to many .NET programmers on here, poo.

kdbolt70 · 06-25-07

Well I've stripped away all the fluff off of my problem, and what I'm really having problems with is how to send a request to the server, get a response, and then send the exact same request (or a different one) using the cookie received by the first response. I have successfully sent 2 sequential requests to the server, but each one has its own cookie. I want the second to send the first's cookie so the server knows it's the same session. Who's got a thought. All the programmers here are old school VB6 and such, not to much knowledge on .NET.

**BFRD** · 06-25-07

I don't think you will be able to reuse the same request. The nature of the request would limit its usability to a single operation.

It looks like a lot of your close methods were out of order. I don't know if that should have a great effect on the outcome, but it is worth a shot. Also any object that implements IDisposable can use the "using" syntax. That will make the structure much easier to read.

Code:

using System;                                                                                                                                                                         
using System.Net;                                                                                                                                                                     
using System.IO;                                                                                                                                                                      
using System.Text;                                                                                                                                                                    
using System.Web;                                                                                                                                                                     
using System.Security.Cryptography.X509Certificates;                                                                                                                                  
 
namespace ping                                                                                                                                                                        
{                                                                                                                                                                                     
  class SimpleHttpClient                                                                                                                                                            
  {                                                                                                                                                                                 
      public static byte[] StrToByteArray(string str)                                                                                                                               
      {                                                                                                                                                                             
          System.Text.ASCIIEncoding encoding = new System.Text.ASCIIEncoding();                                                                                                     
          return encoding.GetBytes(str);                                                                                                                                            
      }                                                                                                                                                                             
 
      [STAThread]                                                                                                                                                                   
      public void login()                                                                                                                                                           
      {                                                                                                                                                                             
          //string gatewayUrl = "http://p8dev01web:7001/core/pswsapi/pswsapi.asp";                                                                                                  
          string inputXmlQueryFilePath = "C:/Documents and Settings/krainbolt/My Documents/Visual Studio 2005/Projects/serverTestApp/serverTestApp/logon.txt";                      
          StreamWriter output = new StreamWriter(@"C:/Documents and Settings/krainbolt/My Documents/Visual Studio 2005/Projects/serverTestApp/serverTestApp\\Output.txt", false);   
          StreamWriter soutput = new StreamWriter(@"C:/Documents and Settings/krainbolt/My Documents/Visual Studio 2005/Projects/serverTestApp/serverTestApp\\startOut.txt", false);
          string cookieHeader = null;                                                                                                                                               
 
 
          try                                                                                                                                                                       
          {                                                                                                                                                                         
              HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://myserver:80/core/");                                                                               
              request.Method = "Post";                                                                                                                                              
              CookieContainer cookieJar = new CookieContainer();                                                                                                                    
              request.CookieContainer = cookieJar;                                                                                                                                  
              byte[] requestBytes = StrToByteArray(" ");                                                                                                                            
              request.ContentLength = requestBytes.Length;                                                                                                                          
 
              using (Stream requestStream = request.GetRequestStream())                                                                                                             
              {                                                                                                                                                                     
                  requestStream.Write(requestBytes, 0, requestBytes.Length);                                                                                                        
              }                                                                                                                                                                     
 
              using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())                                                                                             
              {                                                                                                                                                                     
                  using (Stream responseData = response.GetResponseStream())                                                                                                        
                  {                                                                                                                                                                 
                      using (StreamReader reader = new StreamReader(responseData))                                                                                                  
                      {                                                                                                                                                             
                          string responseFromServer = reader.ReadToEnd();                                                                                                           
 
                          cookieHeader = response.Cookies.ToString();                                                                                                               
                          soutput.Write(cookieHeader);                                                                                                                              
                          soutput.Write(responseFromServer);                                                                                                                        
                          soutput.Write(response.Headers[4]);                                                                                                                       
                          soutput.Close();                                                                                                                                          
                      }                                                                                                                                                             
                  }                                                                                                                                                                 
              }                                                                                                                                                                     
 
 
              HttpWebRequest request1 = (HttpWebRequest)WebRequest.Create("http://myserver:80/login.asp");                                                                          
              // read input XML query from file                                                                                                                                     
              string inputXml;                                                                                                                                                      
              using (StreamReader inputQueryReader = new StreamReader(inputXmlQueryFilePath))                                                                                       
              {                                                                                                                                                                     
                  inputXml = inputQueryReader.ReadToEnd();                                                                                                                          
              }                                                                                                                                                                     
 
              byte[] request1Bytes = StrToByteArray(inputXml);                                                                                                                      
              request1.Method = "Post";                                                                                                                                             
              //request1.CookieContainer.Add(response.Cookies);                                                                                                                     
              request1.ContentLength = request1Bytes.Length;                                                                                                                        
 
              using (Stream request1Stream = request1.GetRequestStream())                                                                                                           
              {                                                                                                                                                                     
                  //output.Write(request1.CookieContainer.ToString());                                                                                                              
                  request1Stream.Write(request1Bytes, 0, request1Bytes.Length);                                                                                                     
              }                                                                                                                                                                     
 
              using (HttpWebResponse response1 = (HttpWebResponse)request1.GetResponse())                                                                                           
              {                                                                                                                                                                     
                  using (Stream response1Data = response1.GetResponseStream())                                                                                                      
                  {                                                                                                                                                                 
                      using (StreamReader reader1 = new StreamReader(response1Data))                                                                                                
                      {                                                                                                                                                             
                          string response1FromServer = reader1.ReadToEnd();                                                                                                         
                          output.Write(response1FromServer);                                                                                                                        
                      }                                                                                                                                                             
                  }                                                                                                                                                                 
              }                                                                                                                                                                     
 
          }                                                                                                                                                                         
          catch (Exception e)                                                                                                                                                       
          {                                                                                                                                                                         
              output.Write("Exception: " + e);                                                                                                                                      
              output.Close();                                                                                                                                                       
          }                                                                                                                                                                         
 
      }                                                                                                                                                                             
 
  }                                                                                                                                                                                 
}

EDIT: I am not sure that declaring the output writers publically like that is the best way to go. However, I can't spend a lot of time looking at it right now. Perhaps a little later I will have some more time.

kdbolt70 · 06-25-07

BFRD, thanks so much for taking a look at it! I was actually able to get the same httpwebrequest to run more than once on the server. And using a cookiecontainer I was able to continue to send the same cookie the second time. Now I'm running into a wall where I'm trying to post to the server, and do it many times in a row, but after the first time I get a "Unable to write data to the transport connection: An established connection was aborted by the software in your host machine." I have no idea if it really is my machine, or it just doesn't like posting more then once. However, when I run it on the loop the entire post call goes out of scope and restarts, so it should have no memory of itself previously logging in, yet it still stops me the second time.

Knitelife · 06-26-07

kdbolt,

Can you post the code you used in your loop. I used to do this type of thing to scrape data from a clients website. I will dig up my old code and see how it compares to what you are doing.

kdbolt70 · 06-26-07

Code:

for (int i = 0; i < x; i++)
                {
                    CookieContainer cookieJar = new CookieContainer();
                    HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://myServer");
                    request.Method = "Get";
                    request.CookieContainer = cookieJar;
                    HttpWebResponse response = (HttpWebResponse)request.GetResponse();
                    Stream responseData = response.GetResponseStream();

                    response.Close();

                    //*---Send Login XML----*

                    request = (HttpWebRequest)WebRequest.Create("http://MyServer ");
                    request.Method = "Post";
                    request.CookieContainer = cookieJar;
                    request.KeepAlive = true;

                    string inputXml;
                    using (StreamReader inputQueryReader = new StreamReader(inputXmlQueryFilePath))
                    {
                        inputXml = inputQueryReader.ReadToEnd();
                    }

                    byte[] requestBytes = StrToByteArray(inputXml);
                    request.ContentLength = requestBytes.Length;
                    Stream requestStream = request.GetRequestStream();
                    requestStream.Write(requestBytes, 0, requestBytes.Length);
                    requestStream.Flush();
                    requestStream.Close();

                    response = (HttpWebResponse)request.GetResponse();
                    responseData = response.GetResponseStream();
                    response.Close();
                    output.Close();
                    //reader.Close();
                    responseData.Close();
                }

It fails on the second execution of

requestStream.Write(requestBytes, 0, requestBytes.Length);

Whats interesting is that it will "Get" just fine, but it will only allow one "Post". When I try the second one it gives me a: "Unable to write data to the transport connection: An established connection was aborted by the software in your host machine."

Knitelife · 06-26-07

Try changing:

Code:

request.KeepAlive = true;

to

Quote:

request.KeepAlive = false;

and see if that makes a difference.

Licht · 06-26-07

Never was much with networking in C#, or any other language. I say stay local! Lol.

I'm gonna stop posting in this thread before i embarrass myself.

		Overclock.net - Overclocking.net > Software, Programming and Coding > Coding and Programming
C# HttpwebRequest cookies n' stuff

06-27-07	#10 (permalink)
kdbolt70 Programmer