WEB PHP - Link Scraping Script v.paypal

Discussion in 'OT Technology' started by maxxpower, Jan 21, 2009.

  1. maxxpower

    maxxpower OG Lauren Crew - Observer OT Supporter

    Joined:
    May 27, 2007
    Messages:
    26,348
    Likes Received:
    0
    I need a script that will scrape my website and return the links on it, specifically from my google adsense boxes on the site.

    The links will need to be returned to a blank page.

    I dont particularly care if its in php or whatever code you wish as long as it will run on a LAMP server.

    Here is an example of one that is scraping my site and returning all the links in an array, but, will not process the links from the adsense boxes.
    http://o2k.org/scrape/scrape3.php

    Shoot me a PM with a price.
     
  2. Pepsi1975

    Pepsi1975 Mod of the Year

    Joined:
    Jan 6, 2005
    Messages:
    47,590
    Likes Received:
    1
    Location:
    Detroit
    ask josh :rofl:
     
  3. Ricky

    Ricky █▄ █▄█ █▄ ▀█▄

    Joined:
    Jun 17, 2005
    Messages:
    38,767
    Likes Received:
    6
    :rofl::o

    I was just messing around trying to learn cURL to do something very similar
     
  4. maxxpower

    maxxpower OG Lauren Crew - Observer OT Supporter

    Joined:
    May 27, 2007
    Messages:
    26,348
    Likes Received:
    0
    :)
    Josh does a ton of work around here for a ton of guys. I know we are all having fun, but he is taking shit too hard.
     
  5. maxxpower

    maxxpower OG Lauren Crew - Observer OT Supporter

    Joined:
    May 27, 2007
    Messages:
    26,348
    Likes Received:
    0
    Do it bro.
    Talk doesnt get me results.
     
  6. maxxpower

    maxxpower OG Lauren Crew - Observer OT Supporter

    Joined:
    May 27, 2007
    Messages:
    26,348
    Likes Received:
    0
    Going to load them into my screensavers so it will automatically allow viewers to see the google ads via the content and link through the screensaver to a relevant ad.

    Example.
    the 135i screensaver loads the google adsense ads for wheels etc. A small banner in the right corner displays the ads but I need a method to grab the link location so it can be passed to the screensaver so when they click on the banner it closes the screensaver and opens the relevant page.

    Way againts googles TOS but i dont give a fuck, this is more about working out the details than the ad service.
     
  7. maxxpower

    maxxpower OG Lauren Crew - Observer OT Supporter

    Joined:
    May 27, 2007
    Messages:
    26,348
    Likes Received:
    0
    Porn referals are easy they always give me a static referral page, but, adsense sucks, it rotates ads/links etc so it has to be more dynamic.
     
  8. maxxpower

    maxxpower OG Lauren Crew - Observer OT Supporter

    Joined:
    May 27, 2007
    Messages:
    26,348
    Likes Received:
    0
    Nope, it uses flash and flash video.
    Can display a webpage but because of different resolutions of monitors, I can get exact locations of corners /gay
     
  9. maxxpower

    maxxpower OG Lauren Crew - Observer OT Supporter

    Joined:
    May 27, 2007
    Messages:
    26,348
    Likes Received:
    0
    WTF i thought we have a dozen "senior php" programmers on this site?
     
  10. maxxpower

    maxxpower OG Lauren Crew - Observer OT Supporter

    Joined:
    May 27, 2007
    Messages:
    26,348
    Likes Received:
    0
    lets try this...



    [​IMG]
     
  11. maxxpower

    maxxpower OG Lauren Crew - Observer OT Supporter

    Joined:
    May 27, 2007
    Messages:
    26,348
    Likes Received:
    0
    fucker will want me to do it in drupal
    :hsugh:
     
  12. maxxpower

    maxxpower OG Lauren Crew - Observer OT Supporter

    Joined:
    May 27, 2007
    Messages:
    26,348
    Likes Received:
    0
    Cant, i tried everything.
    I can "fake" open a browser window and load the ad there, but have issues with scaling.
    Flash doesnt support iframes.
    Flash supports basic html but no way will it load the javascript.
    I had a script that would screen scrape the ad, but that would just display the ad as a jpg imported fine, but no links, so it was no good to me.
    There has to be a way to do this.
     
  13. Pepsi1975

    Pepsi1975 Mod of the Year

    Joined:
    Jan 6, 2005
    Messages:
    47,590
    Likes Received:
    1
    Location:
    Detroit
    remove the sand from you vagina, it was a fucking joke
     
  14. Pepsi1975

    Pepsi1975 Mod of the Year

    Joined:
    Jan 6, 2005
    Messages:
    47,590
    Likes Received:
    1
    Location:
    Detroit
    I love how everyone links it using my photobucket account :rofl:
     
  15. maxxpower

    maxxpower OG Lauren Crew - Observer OT Supporter

    Joined:
    May 27, 2007
    Messages:
    26,348
    Likes Received:
    0
    its the first one that shows up in search :)
     
  16. biawokauns

    biawokauns New Member

    Joined:
    Sep 18, 2001
    Messages:
    19,893
    Likes Received:
    0
    Location:
    Republic of Kalifornia
    you are trying to fuck with data that is outside of your domain. browsers dont like it when you do that (imagine how easy phising would be)

    if it was strictly a iframe, you might have some luck accessing the DOM's node specifically, or even through .contentdocument.
     
  17. Karnejj

    Karnejj “A true conservative is one who can't see any diff OT Supporter

    Joined:
    Jan 9, 2008
    Messages:
    35,585
    Likes Received:
    0
    Location:
    UPGRADED USA Shutdown Today:8 derps
    PM me your site....
     
  18. ge0

    ge0 New Member

    Joined:
    Oct 31, 2005
    Messages:
    8,398
    Likes Received:
    0
    Location:
    JERSEY
    ummmm just do a


    Code:
    $page = implode(file('http://www.yourpage.com/yourpage.php'));
    preg_match('%<a href=\"www.lol.com\">%', $page, $matches);
     
    print_r ($matches);
    
     
    Last edited: Jan 21, 2009
  19. maxxpower

    maxxpower OG Lauren Crew - Observer OT Supporter

    Joined:
    May 27, 2007
    Messages:
    26,348
    Likes Received:
    0
    that wont load the javacript for the ads i dont believe.
     
  20. kingtoad

    kingtoad OT Supporter

    Joined:
    Sep 2, 2003
    Messages:
    55,923
    Likes Received:
    11
    Location:
    Los Angeles
    :rofl:

    scraping :nono:
     
  21. maxxpower

    maxxpower OG Lauren Crew - Observer OT Supporter

    Joined:
    May 27, 2007
    Messages:
    26,348
    Likes Received:
    0
    its for legitimate purposes though
     
  22. Limp_Brisket

    Limp_Brisket New Member

    Joined:
    Jan 2, 2006
    Messages:
    48,422
    Likes Received:
    0
    Location:
    Utah
    lol
     
  23. fl0rine2

    fl0rine2 New Member

    Joined:
    Jan 6, 2005
    Messages:
    1,824
    Likes Received:
    0
    I'm pretty sure the call that the javascript makes takes note of the user's ip visiting your page...you could call all of the information from your ad page via curl, then use some regex to grab all the adsense links into an array...
    but then google would see your server ip each time and would get pissed.

    Not trying to turn your shit around, but have you ever checked out peakclick? http://www.peakclick.com <-- not a referral
    They don't give a shit what you do...simple as hell API to grab the ads and you can display them however you want.

    Some of their keywords pay way more than adsense + you can specify the exact keyword for the ad you want to pull up (ex: only show ads using a high-paying keyword like "casino")
     
  24. maxxpower

    maxxpower OG Lauren Crew - Observer OT Supporter

    Joined:
    May 27, 2007
    Messages:
    26,348
    Likes Received:
    0
    I understand its againts googles TOS, but, I am utilizing the scrape for other ad companies that dont have such a restrictive TOS.
    Yes, I understand that the IP may be recorded for visitors to the site. It is entirely separate from any IP that may click though so I dont think it would be an issue as long as the clicks are unique.
    This has been done already. Google is not interested unless you have existing traffic that warrants such actions. Example is, I did an "i-frame" type implementation in test a few months ago. Where it would open a browser window the size of the google ad over top of my application, the issue is, googles code does not allow linking to a new window. Most "" existing ads when you click on them, it follows that ad on that page.
    I did some prodding and found out that its in googles TOS or some shit that requires this.
    Then I discover that Perez has google ads that click links opening in new windows, preserving his site in the background. When inquiring to google into this new linking, its a "personal" solution worked out between Perez and google.
     
  25. maxxpower

    maxxpower OG Lauren Crew - Observer OT Supporter

    Joined:
    May 27, 2007
    Messages:
    26,348
    Likes Received:
    0
    I think the solution is just to go with a different ad company that has an more open TOS and API.
    I have a couple friends looking at a solution for adsense and if it doesnt work out, f em.
     

Share This Page