WEB What program do alert sites use?

Discussion in 'OT Technology' started by airbball23, Dec 31, 2007.

  1. airbball23

    airbball23 Rent this Space only $5/mnth

    Joined:
    Jan 13, 2007
    Messages:
    1,489
    Likes Received:
    0
    what program do alert sites like http://wiialerts.com use to tell if a product is in stock or not? is it rss feeds or something?
     
  2. noon

    noon get high and teach me how to listen

    Joined:
    May 4, 2002
    Messages:
    3,384
    Likes Received:
    0
    Location:
    Lawrence, KS
    possibly RSS, but most likely screen scraping
     
  3. 95vr4

    95vr4 OT Supporter

    Joined:
    Oct 6, 2004
    Messages:
    2,513
    Likes Received:
    0
    Location:
    Weddington, NC
    Dont even need to screen scape, I wrote an asp script to monitor one of our competitors prices at work.

    Code:
    url2search="http://www.somewebsite.com/product_info.php?products_id="&prodnumber
    
    url = url2search  
        set xmlhttp = CreateObject("MSXML2.ServerXMLHTTP")  
        on error resume next 
        xmlhttp.open "GET", url, false 
        'objXML.setRequestHeader "User-Agent", ":-0"
        xmlhttp.send ""
    strUPText = UCase(xmlhttp.responseText)
    %>
    That puts the html of the whole page in a string strUPText

    Then I automated it (and crossed my fingers I wouldn't crash one of our servers :x:)

    Code:
    prodnumber=1
    do while prodnumber<500
    ...
    ...
    ...
    prodnumber=prodnumber+1
    loop
    
    Just gotta find unique tags (that are always the same) to search for...a css class or something, for example <h3 class="price">$10</h3>
     
    Last edited: Jan 1, 2008
  4. ez4me2c3d

    ez4me2c3d Cold Member

    Joined:
    Sep 19, 2003
    Messages:
    1,836
    Likes Received:
    0
    Location:
    Minneapolis, MN
    This is how you would get the front page (in HTML) of woot, for a custom woot tracker
    Code:
    #!/usr/bin/env ruby
    require 'net/http'
    
    woot = 'www.woot.com'
    woot_page = '/Default.aspx'
    conn = Net::HTTP::new(woot)
    resp, data = conn.get(woot_page)
    woot_html = data[/<body(.|\s)*<\/body>/]
    The woot_html variable would contain all the html in the body of Default.aspx, then you could parse your way through that variable stripping out the important bits like so (ugly, I know)
    Code:
    woot_item = woot_html[/<h3.+TitleHeader.+h3>/][/>.+</]
    
    woot_price = woot_html[/<span.+PriceSpan.+span>/][/>.+</]
    
    woot_shipping = woot_html[/<span.+shipping.+span>/][/>.+</]
    
    woot_condition = woot_html[/<dd.+ConditionSpan.+dd>/][/>.+</]
    
    woot_image = woot_html[/<img.+SaleImage.+\/>/][/src=".+"/]
    Then you could make it print all that to a page, like so
    Code:
    print "Content-type: text/plain\r\n\r\n"
    
    print "<div id=\"woot\">\n"
    
    print "\t<h3 id=\"woot-item\"><span id=\"woot-title\"" + woot_item + "\/span><\/h3>\n"
    
    print "\t<p id=\"woot-price\"" + woot_price + "\/p>\n"
    
    print "\t<p id=\"woot-shipping\"" + woot_shipping + "\/p>\n"
    
    print "\t<p id=\"woot-condition\"" + woot_condition + "\/p>\n"
    
    print "\t<a href=\"http:\/\/www.woot.com\/\"><img id=\"woot-image\" " + woot_image + " \/><\/a>\n"
    
    print "<\/div>\n"
    Call this page woot.rb and put it on your web server like this:
    http://avholloway.com/cgi-bin/woot.rb

    Then make a another page that will call that woot.rb script on page load, or through AJAX calls like this:
    http://avholloway.com/sandbox/woot/
     
    Last edited: Jan 1, 2008
  5. biawokauns

    biawokauns New Member

    Joined:
    Sep 18, 2001
    Messages:
    19,893
    Likes Received:
    0
    Location:
    Republic of Kalifornia
    good info here
     
  6. 95vr4

    95vr4 OT Supporter

    Joined:
    Oct 6, 2004
    Messages:
    2,513
    Likes Received:
    0
    Location:
    Weddington, NC
    ^^How's that work? I fiddled with .net a little and hated it :). Could't believe what it took just to print a damn "hello world" :eek3:

    This is the way I did it

    Code:
    strStartTag="$"
    strEndTag="</TD>"
    
    intStart = instr(1, strUPText, strStartTag, vbtextcompare)
    if intStart then
        intStart = intStart + len(strStartTag)
        intEnd = InStr(intStart + 1, strUPText, strEndTag, vbtextcompare)
        GetPrice = Mid(strUPText, intStart, intEnd - intStart)
        GetPrice=trim(GetPrice)
    end if
    
     
     
  7. pkr

    pkr Guest

    Is there a way to use digg's api in order to display top news stories like: http://digg.com/add-digg

    the unstyled but without the number of diggs included next to it?
     
  8. 95vr4

    95vr4 OT Supporter

    Joined:
    Oct 6, 2004
    Messages:
    2,513
    Likes Received:
    0
    Location:
    Weddington, NC
    probably not with that javascript api, but it looks like you could use the same type of script me and ez posted to do it. Just grab everything between <h3 id="title0"> and </h3>

    Lmk if you want me to post the whole script so you can fiddle with it
     
  9. JesterFX

    JesterFX New Member

    Joined:
    Oct 10, 2004
    Messages:
    4,557
    Likes Received:
    0
  10. ez4me2c3d

    ez4me2c3d Cold Member

    Joined:
    Sep 19, 2003
    Messages:
    1,836
    Likes Received:
    0
    Location:
    Minneapolis, MN
    It's ugly but I was in a hurry to get it working and not trying to "understand" the RegEx methods in Ruby. I'm new to Ruby and not very good with RegEx, so that is why it's ugly.

    But anyway, the following line works like this

    So the woot_html variable contains all of the HTML between <body> and </body>, and I want to extract the part that contains the woot item heading (this is from today)
    Code:
    ...
    <h3 id="ctl00_ContentPlaceHolder_TitleHeader">Razer Pro|Tone m250 Earphones – 2-Pack</h3>
    ...
    so I use RegEx on the woot_html variable to extract the part I want
    Code:
    woot_item = woot_html[/<h3.+TitleHeader.+h3>/]
    that would give me
    Code:
    woot_item = '<h3 id="ctl00_ContentPlaceHolder_TitleHeader">Razer Pro|Tone m250 Earphones – 2-Pack</h3>'
    But now I want just the innerHTML part right? So I extract that from my new shorter string
    Code:
    woot_item = woot_item[/>.+</]
    Those two steps can be combined into one line
    Code:
    woot_item = woot_html[/<h3.+TitleHeader.+h3>/][/>.+</]
    which gives me
    Code:
    woot_item = '>Razer Pro|Tone m250 Earphones – 2-Pack<'
    What I really should do is match the title in one RegEx command for the extraction, but I don't know how, so I was creative and made a work around for myself
     
  11. 95vr4

    95vr4 OT Supporter

    Joined:
    Oct 6, 2004
    Messages:
    2,513
    Likes Received:
    0
    Location:
    Weddington, NC
    Ohh regular expressions, god I hate reg expressions :hsughr:. I got so frustrated with reg exps, I gave up on this isapi url rewrite plugin I was working on last year.

    Thanks for the expl
     
  12. Ricky

    Ricky █▄ █▄█ █▄ ▀█▄

    Joined:
    Jun 17, 2005
    Messages:
    38,767
    Likes Received:
    6
    :rofl:

    Very smart though. I wonder how much that guy's making
     
  13. 95vr4

    95vr4 OT Supporter

    Joined:
    Oct 6, 2004
    Messages:
    2,513
    Likes Received:
    0
    Location:
    Weddington, NC
    :mamoru: lmao, I see what he did there. I swear if people put half the effort into their own business or something legit as they do scamming these ppc/afilliate marketers, they'd probably do a lot better than nickel and diming amazon/ebay :o
     
  14. ez4me2c3d

    ez4me2c3d Cold Member

    Joined:
    Sep 19, 2003
    Messages:
    1,836
    Likes Received:
    0
    Location:
    Minneapolis, MN
    I don't see what they did there, letmein.
     
  15. Kevin

    Kevin New Member

    Joined:
    Aug 27, 2002
    Messages:
    87,634
    Likes Received:
    0
    Location:
    Michigan
    :hsugh: its simple to print hello world in .net

    .net is worth learnin if you want to program, trust me.
     
  16. 95vr4

    95vr4 OT Supporter

    Joined:
    Oct 6, 2004
    Messages:
    2,513
    Likes Received:
    0
    Location:
    Weddington, NC
    I meant compared to classic it seemed like a pita. I know the basics. I had to make a big data grid for one of our backend admin pages and started with .net so I could use data binding. Ended up using yui-ext library instead though.
     
  17. noon

    noon get high and teach me how to listen

    Joined:
    May 4, 2002
    Messages:
    3,384
    Likes Received:
    0
    Location:
    Lawrence, KS
    dude that IS screen scraping :o
     
  18. CrazyInteg

    CrazyInteg Honda-Acura.net OG

    Joined:
    Dec 30, 2000
    Messages:
    12,205
    Likes Received:
    3
    Location:
    Omaha
    lol (laughing with you, not at you)


    Perl FTW. I wrote a sweet app that adds all my competitors products to a shopping cart until they are sold out. I've only tested it one time to make sure it worked, talked to a lawyer, and decided I shouldn't ever use it.
    Just imagine a 10 minute cron, and a few different ip's.
     
  19. noon

    noon get high and teach me how to listen

    Joined:
    May 4, 2002
    Messages:
    3,384
    Likes Received:
    0
    Location:
    Lawrence, KS
    :rofl: That is hilarious

    Perl is definitely my language of choice for something like this.
     
  20. 95vr4

    95vr4 OT Supporter

    Joined:
    Oct 6, 2004
    Messages:
    2,513
    Likes Received:
    0
    Location:
    Weddington, NC
    Didn't know it was called that, always thought screen scaping was just visual screen scaping (like a captcha bypass). I obviously understand the concept though :o
     
  21. pkr

    pkr Guest

    Thanks! I'll hit you up later about this most likely.
     
  22. int3rnal

    int3rnal Active Member

    Joined:
    Jun 15, 2001
    Messages:
    2,215
    Likes Received:
    0
    Nice, I never knew that technique was called screen scraping. Not trying to hijack, but how hard would it be to perform this:

    http://www.quantico.usmc.mil/weather.asp

    I would like to make something that alerts me, either via email or text message, when the current weather condition changes to blue or red. What would be the best way to go about this?
     
  23. ge0

    ge0 New Member

    Joined:
    Oct 31, 2005
    Messages:
    8,398
    Likes Received:
    0
    Location:
    JERSEY
    I made a PS3 online store checker back in the day

    Code:
    <title>PS3 Online System Scanner</title>
    <head>
    <!--
        The "refresh" meta-tag's content identifies the number of
        seconds to delay before refreshing the current document.
        Notice that the URL tag is not specified, indicating that
        the current document will be refreshed.
    -->
    <meta http-equiv="refresh" content="120">
    </head>
     
    <body>
     
    <?php
     
    $page = implode(file('http://www.circuitcity.com/ssm/PlayStation-3/sem/rpsm/catOid/-16622/N/20012841+20016481+20016622/link/ref/rpem/ccd/categorySpecial.do'));
    preg_match('%SOLD OUT!%', $page, $matches);
    if($matches[0]=="SOLD OUT!")
      {
            if($matches[0] != "SOLD OUT!")
              {
                 
                   
                   $body = "[URL]http://www.circuitcity.com/ssm/PlayStation-3/sem/rpsm/catOid/-16622/N/20012841+20016481+20016622/link/ref/rpem/ccd/categorySpecial.do[/URL]    $body\n\n\n\n\n\n";
                   $sendmail = mail([EMAIL="[email protected]"][email protected][/EMAIL], "PS3 is ONLINE RIGHT NOW!", $body, "From: Contact Form from $name\r\nCc: [EMAIL="[email protected]\r\n"][email protected]\r\n[/EMAIL]");
                   if ($sendmail) {
                                   echo "<br><br><center>PS3!!!!! GET IT!</div>";
                   } else {
                                echo "<br><br><center>There seems to be a problem. Please try again later</center>";
                          }
                    
     
      
              }
             else
              {
                echo("CircuitCity.com is Out of Stock"); ?><br> <?
               } 
       }
    else
       {
                       $body = "[URL]http://www.circuitcity.com/ssm/PlayStation-3/sem/rpsm/catOid/-16622/N/20012841+20016481+20016622/link/ref/rpem/ccd/categorySpecial.do[/URL]    $body\n\n\n\n\n\n";
                   $sendmail = mail([EMAIL="[email protected]"][email protected][/EMAIL], "PS3 is ONLINE RIGHT NOW!", $body, "From: Contact Form from $name\r\nCc: [EMAIL="[email protected]\r\n"][email protected]\r\n[/EMAIL]");
                   if ($sendmail) {
                                   echo "<br><br><center>PS3!!!!! GET IT!</div>";
                   } else {
                                echo "<br><br><center>There seems to be a problem. Please try again later</center>";
                          }
       }
    print_r ($matches);
    ?>
    <p>
    <?php
     
    $page = implode(file('http://www.compusa.com/products/product_info.asp?pfp=SEARCH&Ntt=playstation+3&N=0&Dx=mode+matchall&Nty=1&D=playstation+3&Ntk=All&product_code=343142&Pn=PlayStation_3_Console_60GB_Hard_Drive'));
    preg_match('%SOLD OUT for delivery%', $page, $matches2);
    if($matches2[0]=="SOLD OUT for delivery")
      {
            if($matches2[0] != "SOLD OUT for delivery")
              {
                 
                   
                   $body = "[URL]http://www.compusa.com/products/product_info.asp?pfp=SEARCH&Ntt=playstation+3&N=0&Dx=mode+matchall&Nty=1&D=playstation+3&Ntk=All&product_code=343142&Pn=PlayStation_3_Console_60GB_Hard_Drive[/URL]    $body\n\n\n\n\n\n";
                   $sendmail = mail([EMAIL="[email protected]"][email protected][/EMAIL], "PS3 is ONLINE RIGHT NOW! CIRCUITCITY", $body, "From: Contact Form from $name\r\nCc: [EMAIL="[email protected]\r\n"][email protected]\r\n[/EMAIL]");
                   if ($sendmail) {
                                   echo "<br><br><center>PS3!!!!! GET IT! CIRCUITCITY!!</div>";
                   } else {
                                echo "<br><br><center>There seems to be a problem. Please try again later</center>";
                          }
                    
     
      
              }
             else
              {
                echo("CompUsa.com is Out of Stock"); ?><br> <?
               } 
       }
    else
       {
                       $body = "[URL]http://www.compusa.com/products/product_info.asp?pfp=SEARCH&Ntt=playstation+3&N=0&Dx=mode+matchall&Nty=1&D=playstation+3&Ntk=All&product_code=343142&Pn=PlayStation_3_Console_60GB_Hard_Drive[/URL]    $body\n\n\n\n\n\n";
                   $sendmail = mail([EMAIL="[email protected]"][email protected][/EMAIL], "PS3 is ONLINE RIGHT NOW! CompUSA", $body, "From: Contact Form from $name\r\nCc: [EMAIL="[email protected]\r\n"][email protected]\r\n[/EMAIL]");
                   if ($sendmail) {
                                   echo "<br><br><center>PS3!!!!! GET IT! COMPUSA!!!</div>";
                   } else {
                                echo "<br><br><center>There seems to be a problem. Please try again later</center>";
                          }
       }
    print_r ($matches2);
    ?>
    <p>
    </body>
    
     
  24. Logik

    Logik Livin la vida broka

    Joined:
    Jun 30, 2000
    Messages:
    20,667
    Likes Received:
    1
    Location:
    The Steel City
    simple and works. php ftw
     
  25. Ricky

    Ricky █▄ █▄█ █▄ ▀█▄

    Joined:
    Jun 17, 2005
    Messages:
    38,767
    Likes Received:
    6
    i need to learn some new languages :wtc:
     

Share This Page