WEB wordpress not showing robot meta tags when i select privacy settings...

Discussion in 'OT Technology' started by pharmokan, Aug 26, 2009.

  1. pharmokan

    pharmokan OT Supporter

    Joined:
    Oct 18, 2002
    Messages:
    102,305
    Likes Received:
    495
    Location:
    L.A.
    robots dont show up in head when i tell it in privacy settings "allow bots to index my site to be visible on google :mb:"


    i manually echoed the meta tags in some php file but i forgot where, and they disappeared after i upgraded to 2.8.4

    now why the fuck dont meta tags show up after i set privacy settings to do it
     
  2. brds

    brds OT Supporter

    Joined:
    Jun 26, 2006
    Messages:
    17,652
    Likes Received:
    15
    Location:
    Atlanta
    why are you echoing them? why not hard code in the index's head?
     
  3. fishbulb

    fishbulb Active Member

    Joined:
    Oct 29, 2001
    Messages:
    6,848
    Likes Received:
    4
    Location:
    md
    I thought that setting only controlled what was displayed in robots.txt, not the meta tags themselves?

    you can just manually add them in the theme header file as brds suggested
     
  4. pharmokan

    pharmokan OT Supporter

    Joined:
    Oct 18, 2002
    Messages:
    102,305
    Likes Received:
    495
    Location:
    L.A.
    i manually put it in header.php, thanks for that reminder.

    but thats gay yo

    why cant wordpress put those tags in for you if you select that option in privacy settings

    i dont think that setting controls robots.txt

    whats it gonna put? a single line command:
    User-agent: *

    :weak:
     
  5. fishbulb

    fishbulb Active Member

    Joined:
    Oct 29, 2001
    Messages:
    6,848
    Likes Received:
    4
    Location:
    md
    No, it toggles disallow all. That's how it worked in the past anyways, not sure about now
     
  6. pharmokan

    pharmokan OT Supporter

    Joined:
    Oct 18, 2002
    Messages:
    102,305
    Likes Received:
    495
    Location:
    L.A.
    wordpress does not come with robots.txt you must put it in there yourself
     
  7. Insert Tokens

    Insert Tokens Making Cancer My Bitch OT Supporter

    Joined:
    Jan 12, 2006
    Messages:
    8,330
    Likes Received:
    76
    Location:
    Tasmania
    Wordpress dynamically generates robot.txt response when it gets requested, same way it builds everything else.

    EVERY frequest to wordpress passes through index.php via the main .htaccess and is processed from there. In this case it catches the robots.txt request and serves up whetever you set under the privacy settings.

    Eg from my domain.com/robots.txt even though there's no robots.txt in the root:

     
  8. pharmokan

    pharmokan OT Supporter

    Joined:
    Oct 18, 2002
    Messages:
    102,305
    Likes Received:
    495
    Location:
    L.A.
    how is this possible
     
  9. fishbulb

    fishbulb Active Member

    Joined:
    Oct 29, 2001
    Messages:
    6,848
    Likes Received:
    4
    Location:
    md
    .htaccess as mentioned above
     
  10. Insert Tokens

    Insert Tokens Making Cancer My Bitch OT Supporter

    Joined:
    Jan 12, 2006
    Messages:
    8,330
    Likes Received:
    76
    Location:
    Tasmania
    Look in your htaccess on a wordpress instal..

    Code:
    # BEGIN WordPress
    <IfModule mod_rewrite.c>
    RewriteEngine On
    RewriteBase /
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteRule . /index.php [L]
    </IfModule>
    
    # END WordPress
    
    It diverts EVERYTHING through the index.php.. and then wordpress itself decides how to handle requests. That's how they do friendly URLs, that's how they handle queries, and that's how they handle robots.txt and other miscellaneous stuff.

    If the visitor sends a request for yourdomain.com/robots.txt, it's actually sending a request to index.php asking for robots.txt, which it gets served back dynamically.
     
  11. pharmokan

    pharmokan OT Supporter

    Joined:
    Oct 18, 2002
    Messages:
    102,305
    Likes Received:
    495
    Location:
    L.A.
    so the robots.txt does not need to physically exist in root it gets dynamically generated once someone requests robots.txt you're telling me
     
  12. pharmokan

    pharmokan OT Supporter

    Joined:
    Oct 18, 2002
    Messages:
    102,305
    Likes Received:
    495
    Location:
    L.A.
    thats mad inconvenient for advanced customization
     
  13. pharmokan

    pharmokan OT Supporter

    Joined:
    Oct 18, 2002
    Messages:
    102,305
    Likes Received:
    495
    Location:
    L.A.
    this is getting side tracked a little here though

    i still need to physically insert meta tags for googlebot to actually start indexing

    otherwise if i just relied on privacy settings it would never get indexed because it never adds meta tags to pages telling the bot what the fuck to do
     
  14. Insert Tokens

    Insert Tokens Making Cancer My Bitch OT Supporter

    Joined:
    Jan 12, 2006
    Messages:
    8,330
    Likes Received:
    76
    Location:
    Tasmania
    You can still add other meta-tags via the header if you want, but your original question was re: privacy settings, which is just whether or not Wordpress should use robots.txt to block all bots :)

    What meta tags are you planning on adding that are so important to Google? Google cares more about keywords and content links than meta tags these days.
     
  15. pharmokan

    pharmokan OT Supporter

    Joined:
    Oct 18, 2002
    Messages:
    102,305
    Likes Received:
    495
    Location:
    L.A.
    meta robots index, follow :hsughno:
     
  16. Insert Tokens

    Insert Tokens Making Cancer My Bitch OT Supporter

    Joined:
    Jan 12, 2006
    Messages:
    8,330
    Likes Received:
    76
    Location:
    Tasmania
    Google uses them to look for nofollow, but unless it see's a nofollow it pretty much just follows by default. If you're blocking a page though, absolutely :)

    I'm no SEO expert though, i'm just curious more than anything :)
     
  17. pharmokan

    pharmokan OT Supporter

    Joined:
    Oct 18, 2002
    Messages:
    102,305
    Likes Received:
    495
    Location:
    L.A.
    if that was the case meta robots tags would be deprecated in html5
     
  18. Insert Tokens

    Insert Tokens Making Cancer My Bitch OT Supporter

    Joined:
    Jan 12, 2006
    Messages:
    8,330
    Likes Received:
    76
    Location:
    Tasmania
    Please tell me you're trolling now :hs:
     
  19. pharmokan

    pharmokan OT Supporter

    Joined:
    Oct 18, 2002
    Messages:
    102,305
    Likes Received:
    495
    Location:
    L.A.
    why the fuck would robot meta tags specify "follow, nofollow, index, noindex, all"

    :ugh: obviously they serve as guides for googlebots

    what you're saying they basically look just for the NOFOLLOW other wise they just do a regular follow by default

    follow command seems pretty deprecated to me if what you're saying is true :dunno:
     

Share This Page