how to prevent robots from accessing specific modules

Posted: Tue Dec 07, 2004 8:35 am

Before posting this new topic, I searched all the forums, but no luck.
Please help me on this.
I appreciate your help in advance.

Here is the question.
How can I block robots to access specific nuke modules?

If I want to prevent robots(yahoo bot, googlebot, etc) from accessing chat module, How can I do this?

What I did was like following..
Inside robots.txt, I added the line as..

Disallow: /modules.php?name=4nChat
Disallow: /modules.php?name=Kalender
Disallow: /azkw/modules.php?name=4nChat
Disallow: /azkw/modules.php?name=Kalender

However, the lines have no effect on Bots. They still index above modues..

When I look at indexed pages on google or yahoo, they only shows above modules.

Any help or advice on this will be appreciated greatly.

Thank you all for your hard work!!

Posted: Tue Dec 07, 2004 8:56 am

I am open to any suggetion..
I look at various web sites that talk about robots.txt, googlebot,yahoo bot, googletab, and etc.

Could not find one site on this matter..
"How to prevent robots from accessing specific modules in php-nuke.

Please help me out on this..

By the way,
I am using Google tab to generate static page like URL(searchengine friendly URL)..

I had a little problem with SID at the end of page URL like following.
"hxxp://www.not_real_address.com/postt2004.html&sid=xxxxxxx"
It was easy to fix by chaging a few lines in includes/sessions.php.

If anyone having same problem then look for following code in includes/sessions.php

Code:

$SID = ( $sessionmethod == SESSION_METHOD_GET ) ? 'sid=' . $session_id : '';

to

Code:

if ( $user_id != ANONYMOUS ) {
$SID = ( $sessionmethod == SESSION_METHOD_GET ) ? 'sid=' . $session_id : '';
}

It should be enough to fix the problem since robots are not registered users. So, they don't need to see SID.
Smile

Still looking for the answers for the above posts. Thank you.