phpnuke and search engines

Nuke Soldier Joined: Oct 30, 2004 Posts: 30

hey guys,
I was told that spiders and crawlers can not crawl a phpnuke website. is this true ? if so how do you add your website to a search engine that crawls. i can look in my control panel for my website and i see that spiders and other crawlers have hit the website and use bandwidth each time they hit the website.

i have setup my meta.php and done everything i can to ensure that they are in the search engines. but i notice that the robo.txt file has the line disallow "includes" how does it read the meta.php if its in a folder that is disallowed?

Thanks
Lonk

Site Admin Joined: Jun 18, 2004 Posts: 208

well they can craw but being php its not going to be the best,
i would if i was you install a googletap or if you wait till the weekend i will release updated version.

as googletap makes all your links as html.

and our new googletap also does keyword based urls

like article-this-is-my-topic-1.html

hope this helps

Steve Wink

Nuke Soldier Joined: Oct 30, 2004 Posts: 30

yeah thats great ill wait til the weekend. thanks for the help. can you post a link to it when you get it updated?

Thanks again
lonk

Nuke Soldier Joined: Oct 30, 2004 Posts: 30

hey perfect-games
were you able to update the googletap ?

Nuke Cadet Joined: May 08, 2006 Posts: 1

Heheh, when I saw Perfect games say he was updating google-tap, I jumped.

Hope you can finish it soon, I'd love to give it a try asap.