Issue with C# & HttpWebRequest/HttpWebResponse & Cookies

Discussion in 'OT Technology' started by Scoob_13, Jun 2, 2003.

  1. Scoob_13

    Scoob_13 Anything is possible, but the odds are astronomica

    Joined:
    Oct 5, 2001
    Messages:
    73,785
    Likes Received:
    38
    Location:
    Fort Worth. Hooray cowgirls.
    :hs:

    First true C# program, and I'm having a huge issue with using HttpWebResponse/Request & Cookies. I'm using a cookie container to store the session cookie, however I cannot for some reason save the cookie to the container & then access the subsequent web pages that I'm attempting to copy so that I can use the data for work related purposes (basically, we need a DB made from the info on this website, and I was tasked with writing a program to save a guy from printing out all 36k records)

    Basically, I'm attempting to get the cookie from the site that webpage is instantiated as, so that the site for webpage2 won't redirect me to the root page, however I can't seem to use the same CookieContainer for both.


    I created a general C# Windows app, made a button called buttonStart on the form, and 2 listBox's (listBox1 & listBox2) for output.

    Code:
    private void buttonStart_Click(object sender, System.EventArgs e)
    		{
    			try
    			{
    				statusBar1.Text = "Beginning Process...";
                    
    				Uri webpage = new Uri("http://www.allregs.com/payoff/index.aspx");
    				Uri webpage2 = new Uri("http://www.allregs.com/payoff/index.aspx?View=Review&SID=10000");
    				Uri webpage3 = new Uri("http://www.yahoo.com");
                    
    				statusBar1.Text = "Requesting " + webpage.ToString() + "...";
    
    				WebRequest newpage = WebRequest.Create(webpage);
    				HttpWebRequest httpnewpage = (HttpWebRequest)newpage;
    				
    				httpnewpage.KeepAlive = true;
    				
    
    				CookieContainer cc = new CookieContainer();
    				httpnewpage.CookieContainer = cc;
    				httpnewpage.CookieContainer.GetCookieHeader(webpage);
    				
    				HttpWebResponse httpresponse = (HttpWebResponse)httpnewpage.GetResponse();
    
    				Stream strm = httpresponse.GetResponseStream();
    				StreamReader sr = new StreamReader(strm);
    				string line;
    				while((line=sr.ReadLine())!= null)
    				{
    					listBox1.Items.Add(line);
    				}
    
    				newpage = WebRequest.Create(webpage2);
    				httpnewpage = (HttpWebRequest)newpage;
    				
    				httpnewpage.CookieContainer = cc;
    				httpnewpage.CookieContainer.Add(webpage,cc.GetCookies(webpage2));
    				//httpnewpage.CookieContainer.GetCookieHeader(webpage2);
    				
    				httpresponse = (HttpWebResponse)httpnewpage.GetResponse();
    
    				strm = httpresponse.GetResponseStream();
    				sr = new StreamReader(strm);
    				
    				while((line=sr.ReadLine())!= null)
    				{
    					listBox2.Items.Add(line);
    				}
    
    
    
    				strm.Close();
    			}
    			catch(Exception ex)
    			{
    				MessageBox.Show("Unable to process request because " + "the following problem occurred:\n" + ex.Message, "Exception Occurred");
    			}
    		}
    
    


    On the plus side, one version of the code that I have is actually turning out to be a damn quick site ripper :o Once I perfect it, I'll post the code so you guys can grab pr0n & whatnot.
     
  2. Scoob_13

    Scoob_13 Anything is possible, but the odds are astronomica

    Joined:
    Oct 5, 2001
    Messages:
    73,785
    Likes Received:
    38
    Location:
    Fort Worth. Hooray cowgirls.
    :hs: No one seems to know how to bypass it. It seems to be an issue with one of the latest updates to IIS 5.0 where it blocks repeated requests from one IP and simply reroutes to the main page.

    Poop.
     

Share This Page