I am having dire performance problems having rewrite rules in my .htaccess file.
My site get's about 15K-20K hits a day. the load sits between 0.10 and 1.1 fluctuating extensively. Before the .htaccess and rewrites it was sitting pretty much on 0.00 - 0.03 load.
I need to optimise apache more. The first step is to eliminate all the .htaccess files, so apache does not have to do recursive lookups on each request. This alone will drasticly increase performance.
I would love to move them all into my main httpd.conf file.
I can migrate the .htaccess into it no problem.
The site links get rewritten, but they dont work. All pages not found.
I guess I need to yank the header.php condition rules out and chuck it in the httpd.conf also.
Now how, or what's the syntax for that?
Does anyone have any experience in doing this?
Thanks
Imago Captain
Joined: Jan 17, 2003
Posts: 629
Location: Europe
Posted:
Thu Nov 04, 2004 2:45 am
Better play with your robots.txt and allow only several SE to crawl the site. Or sign up for a better machine. My load is steady around 0.5 - with a dozen of Amazon shops rewritten and included in the Nuke Encyclopedia.
Anyway I did some performance tuning and my site performs better now.
Yesterday I received over 140K hits. Page creation times dropped to 4 seconds at times.
I managed to migrate the htaccess file into my main http.conf file, which gave a slight performance increase. I also installed turck-mmcache. This has increased performance quite a bit, most notably the main pages, and some other pages.
Though I have some questions I'm hoping you can answer for me:
1. Does Apache2 deal with rewrites in a better manner?
2. Is it possible to enter a RewriteCondition for Googlebot/MSN/Yahoo etc and have all _other_ ip's/users/agents not go through the rewrite rules, so they can browse the site with the traditional non rewrite way?
Will this break the site in any way? Also if google crawled the links and a user clicks through google to my site will the .html rewrite work for him? and when he browse the site furhter will it all work through .php (non rewriting) ?
Wow thats alot.
Hope you can help me out here.
Thanks
Imago Captain
Joined: Jan 17, 2003
Posts: 629
Location: Europe
Posted:
Sat Nov 13, 2004 4:34 am
Hello,
Don't know about Apache2
Entering any additional rules to .htaccess will probably slow down the performance. You don't need different rules for SE and visitors - rewritten URLs without visitors are useless. For me, the problem is quite the contrary. I have to deny from Google all dynamic links, because, in the last month, Google started to index indiscriminately all links of the type /gates.html?name=Module, etc., down to the reply with quote links on the forum.
That's waste of indexing quota (if they exist at all).
Since our problem is Google, in the robots.txt put the following line
...Entering any additional rules to .htaccess will probably slow down the performance. You don't need different rules for SE and visitors - rewritten URLs without visitors are useless. For me, the problem is quite the contrary. I have to deny from Google all dynamic links, because, in the last month, Google started to index indiscriminately all links of the type /gates.html?name=Module, etc., down to the reply with quote links on the forum...
Beware, this could be a variant of the Santy worm trying to attack your site using Google's search routines.
_________________ "Sic vis pacem para bellum!"
RAF71_Hornet / GibsonXXI
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
