The Ultimate: Google Tap + GZIP'd Nuke Site

Corporal Joined: Nov 30, 2003 Posts: 56

The Ultimate: Google Tap + GZIP'd Nuke Site

And yes, I've managed to do it: GZIP my entire PHP-Nuke site (and I mean virtually every page/block), and I have Google-Tap installed at the same time. What does that mean? That the bots will be crawling all over me - and my bandwidth will barely feel the pressure. GZIP on-the-fly compression on your Nuke site can save an average of 80% or higher bandwidth on HTML only transfers - which is what Google Tap THRIVES on.

Want to learn more? Here's a link that turned me on to compressing my PHP-Nuke site with GZIP compression in the first place:

http://www.karakas-online.de/forum/viewtopic.php?t=130&highlight=accelerate

If you follow the directions in that thread, your HTML transfers will shrink in size by 80% or more. And if you alter more of your "main" PHP files than just index.php (modules.php, admin.php, etc), your entire site will be compressed. Here's a link to test your site's pages when you've got everything installed:

http://www.desilva.biz/gzip-test.php

If you'd like to see my site in action with both Google-Tap and GZIP compression, the address is http://hz.adeptshaven.com. Feel free to copy any url on my site, and paste it into the tester for verification. I am 95% sure when a google-bot (or other bot) hits my site, they are being delivered only HTML content, and that it is compressed down to 1/5th of what it used to be.

Serious hits. Serious ratings. Serious cool. Very Happy

Lieutenant Joined: Jan 23, 2003 Posts: 258 Location: USA

Thanks for the tip! But, where in mainfile.php does...

Code:

ob_start( 'ob_gzhandler' );

...go?

I must be an idiot... Where does the code go in the modules and blocks?

Thanks for any guidance!

Nuke Cops Founder Joined: Nov 14, 2002 Posts: 5939

Very nice! I hope you don't mind if I copy/paste it here too for posterity? If its a problem please let me know and I'll remove it...

Quote:

Want a lightning fast PHP-Nuke? Read on... Wink

Problem: You installed all kinds of modules, blocks and image galleries on your PHP-Nuke. Now, you notice that it has become painfully slow. Crying or Very sad

What can be done to accelarate things a bit?

Idea

Idea: Cacheing! Ask yourself the following questions:

Does your index.php, or the other parts of your PHP-Nuke change more often than, say, once in 15 minutes? Mine does not, if I am not logged in.
Do you really need to see a change the very moment you hit the reload button of your browser? For me, if this change is the number of private messages I got, then I might be interested to see the *real* number. But then again, why not the number as of 5 minutes ago? How important is current content for you? How current is current enough for you? 1 minute, 5 minutes, or even 15 minutes?

If you think a little about it, you will realize that it is not absolutely necessary that your PHP-Nuke does all these conections to the database, querying all that data that most probably did not change the last x minutes, each time you display a page.

So, how about cacheing?

Solution: Enter jpcache: jpcache is a lightweight, full page caching system for PHP, thus reducing server load, as pages are generated less often. It dramatically speeds up serving of your pages, by caching page ouput and returning the chached page instead of compiling the PHP one.

Installation is very simple:

Download it from sourceforge.net.
Extract the files in the includes directory of your PHP-Nuke. A folder "jpcache" will be created containing all the necessary files. Change to that folder.
Read the readme file.
Edit the $includedir variable in the jpcache.php file to reflect the *full* path to the jpcache files:

Code:

// Set the includedir to the jpcache-directory
$includedir = "/usr/local/httpd/htdocs/nuke6/html/includes/jpcache";

Edit the jpcache-config.php file. You basically only need to set the $JPCACHE_TYPE to "file" (remember, we want to avoid database connections Wink

):

Code:

$JPCACHE_TYPE = "file";
//$JPCACHE_TYPE = "mysql";
//$JPCACHE_TYPE = "dbm"; -- Not yet

then coment the next block (you don't need it, since you *did* specified the $JPCACHE_TYPE above Wink

):

Code:

// DOH! Strip out this check for performance if you are sure you did set it.
// if (!isset($JPCACHE_TYPE))
// {
// exit("[jpcache-config.php] No JPCACHE_TYPE has been set!");
// }

Just check the configuration options to see what they do, the defaults are O.K. for me:

Code:

/**
* General configuration options.
*/
$JPCACHE_TIME = 900; // Default number of seconds to cache a page
$JPCACHE_DEBUG = 0; // Turn debugging on/off
$JPCACHE_IGNORE_DOMAIN= 1; // Ignore domain name in request(single site)
$JPCACHE_ON = 1; // Turn caching on/off
$JPCACHE_USE_GZIP = 1; // Whether or not to use GZIP
$JPCACHE_POST = 0; // Should POST's be cached (default OFF)
$JPCACHE_GC = 1; // Probability % of garbage collection
$JPCACHE_GZIP_LEVEL = 9; // GZIPcompressionlevel to use (1=low,9=high)
$JPCACHE_CLEANKEYS = 0; // Set to 1 to avoid hashing storage-key:
// you can easily see cachefile-origin.

Finally, set the directory for the cached files, preferably outside your web server root, say /tmp/jpcache. Create it, set the owner and group equal to the owner and group of your web server and change the permissions to 755 (rwx-r-x-r-x), or even 750 (rwx-r-x----). Then, enter in jpcache-config.php the absolute path to that cache directory:

Code:

/**
* File based caching setting.
*/
$JPCACHE_DIR = "/tmp/jpcache"; // Directory where jpcache must store
// generated files. Please use a dedicated
// directory, and make it writable

That's all! jpcache is now configured correctly. To use it, just open the index.php file of your PHP-Nuke and enter the following lines, after the line "require_once("mainfile.php");" (that's important!):

Code:

require_once("mainfile.php");
if (!isset($user)) {
$cachetimeout=900;
} else {
$cachetimeout=300;
}
require_once "includes/jpcache/jpcache.php";

(I have included the "require_once("mainfile.php");" line in the code above for your orientation).

That's it! You can now tweak the two cacheing intervalls, 900 and 300 seconds, which apply to non-logged-in and logged-in users respectively. Rethink the questions I asked you above. Wink

A cacheing intervall of x seconds means that your server will serve you a cached copy of the page, if the cache copy is not older than x seconds. Since non-logged-in users don't get private messages or similar "events", a longer cacheing intervall is just right for them. Feel free to experiment with these two values, trading actuality of content against serving speed.

Enjoy your new, lightning fast PHP-Nuke!

Nuke Cops Founder Joined: Nov 14, 2002 Posts: 5939

chris wrote:

Oops...I forgot a crucial detail:

You must edit the mainfile.php file too! There, you *must* comment the line

Code:

ob_start("ob_gzhandler");

as follows:

Code:

# ob_start("ob_gzhandler");

Otherwise, you will be compressing twofold, once from PHP-Nuke with the above line and once with jpcache through the configuration line

Code:

$JPCACHE_USE_GZIP = 1; // Whether or not to use GZIP

of jpcache-config.php.

The other way round, turning off gzip in jpcache and leaving the ob_start("ob_gzhandler") line uncommented in PHP-Nuke, works too. It may even be better to choose this way, if you use Apache and mod_gzip (a common configuration).

Nuke Cops Founder Joined: Nov 14, 2002 Posts: 5939

chris wrote:

After reading the above, you may ask yourself if this is all we can do to accelerate PHP-Nuke with cacheing - and the answer is, of course not!

jpcache enables cacheing of the whole page. Remember, we edited index.php to set a different cache expiration time for logged and not logged-in users. The idea behind this was that a user that is not logged in will not have access to most of the dynamic features of PHP-Nuke, thus making cacheing of the whole page for longer intervalls an acceptable compromise between freshness of content and speed of page generation.

But how about our logged-in users? Wouldn't it be nice if we could at least cache some PHP-Nuke blocks we know they don't change that often? This is indeed possible with Cache-Lite of the PEAR library from PHP.

Download Cache-Lite and extract it in the includes folder of your PHP-Nuke installation.

If your PHP does not include the PEAR library, you will need to download download the base PEAR package too. Extract only the PEAR.php file and put it in the same folder as mainfile.php, i.e. your PHP-Nuke base folder.

Cache-Lite needs the PEAR.php file for its error-handling, so you may not notice that you need it until some error occurs. Wink

If your PHP installation includes PEAR, you don't need to bother installing PEAR.php anywhere - it is already there, at the right place. But if your ISP does not include it, the above PEAR.php will do the trick.

That's all! You are now ready to use Cache-Lite. I will talk about this in my next post.

Nuke Cops Founder Joined: Nov 14, 2002 Posts: 5939

chris wrote:

Cache-Lite can easily be used to cache a PHP variable. The easiest application of this is to cache the $content variable of a block that does not change very often - and this is the majority of PHP-Nuke blocks! Idea

To use Cache-Lite in a PHP-Nuke block, open that block in your editor and put the following code

Code:

// include the Cache-Lite package
require_once("includes/Cache_Lite/Lite.php");

// set some variables
$options = array(
"cacheDir" => "/tmp/Cache_Lite/",
"lifeTime" => 900
);

// create a Cache_Lite object
$objCache = new Cache_Lite($options);

// test if there exists a valid cache,
// the ID will be the basename of the block
$fileid = basename($blockfile,".php");
if ($content = $objCache->get($fileid)) {
// add a message indicating this is cached output
$content .= "\n[cached with ID=$fileid]";
}
else
{
// do the block's work here...

after the lines

Code:

if (eregi("block-XXXXXXX.php",$PHP_SELF)) {
Header("Location: index.php");
die();
}

What does the inserted code do?

It includes Cache-Lite. Note that in this example, Cache_Lite is a symbolic link to the actual directory the Lite.php file is in. I created this symbolic link myself, as I did not want to put a version-specific directory name in my code - I would have to change it each time I would upgrade Cache-Lite, this way I will only need to change the symbolic link to show to the right directory.
It first sets the path to the cache directory, in this case /tmp/Cache_Lite/. The directory must exist and must be writable and readable by the web server.
Sets the cache expiration time intervall for the object we are going to cache, in this case to 900 seconds.
Creates a cache object with the above settings.
Uses a computed "file id" as a key to identify the cached object (the cached $content variable). I have decided to use the basename of the PHP-Nuke block , i.e. the filename of the block without the ".php" ending, as the key for the $content variable. This way I don't have to change my code each time I insert the above lines in a new block. The $key will be computed automatically and will be the basename of the block file. Since in PHP-Nuke blocks are guaranteed to have distinct names, this is a good choice.
If a valid cached copy of the object identified by the key exists, it is fetched and fed into the $content variable of the block. I issue an echo to print the cache ID (the key) in this case, just for the start, to give me some idea of how (and if at all) this works. Of course, you can comment the echo later.
In case the cache misses the object (or expiration time is over), you wil have to compute $content from scratch. Put your normal code here.

Of course, we are not done yet - after we compute $content in our block, we will want to cache it for future use. Wink

Thus, after the end of your computations and before you close the block with "?>", put the following to instruct Cache-Lite to cache your $content:

Code:

// ...and save it in the cache for future use
$objCache->save($content, $fileid);
}

This closes the IF-statement we opened with the previous block of lines. It caches the $content variable in our cache object, using as identifier the file ID we derived from the block's filename.

You have to put these two blocks of line in every PHP-Nuke block you plan to cache, but the work is worth it!

For a more in-depth tutorial on Cache-Lite, see Caching With PHP Cache_Lite.

You can use both jpcache and Cache-Lite together in a cache strategy that caches the whole page with jpcache when a user is not logged-in, but caches the least changing blocks (probably most of them) with Cache-Lite when he is. Such a strategy allows enough flexibility and room for experimentation and brings a considerable acceleration to your PHP-Nuke, without, on the other hand, requiring any access to the internals of your PHP installation - something that you will need, if you want to try the ultimate approach to cacheing, which I will present you in a follow-up post.

Stay tuned and happy cacheing! Very Happy

Captain Joined: Oct 20, 2003 Posts: 335

Nuke Cops Founder Joined: Nov 14, 2002 Posts: 5939

chris wrote:

As we know, PHP is an interpreted language. The PHP interpreter compiles your scripts each time you call them. However, the PHP code itself may not change that often in your scripts. Having to compile the code on each call wastes your CPU's time. That's where the idea of cacheing the PHP code (and not only the HTML content) comes in. Idea

The idea of PHP code cacheing lies in the basis of many commercial PHP accelerators, like

Hopefully, there is a free open source alternative: Turck MMCache, a product under the GPL Licence.

Turck MMCache is a free open source PHP accelerator, optimizer, encoder and dynamic content cache for PHP. It increases performance of PHP scripts by caching them in compiled state, so that the overhead of compiling is almost completely eliminated. Also it uses some optimizations to speed up execution of PHP scripts. Turck MMCache typically reduces server load and increases the speed of your PHP code by 1-10 times!

Download the newest version (currently 2.3.23) from the Turck MMCache download page at Sourceforge. There are precompiled binaries for Windows, as well as source packages for Linux and other operating systems.

Before you rush and download it though, let me bring to your attention that the approach taken by a code accelerator like MMCache requires that you install it as a so-called "PHP extension". You will also be configuring options like the amount of shared memory that you want allocated to MMCache, in order to use it for cacheing. These steps are normally beyond the reach of mass hosting accounts on ISPs, so you may be out of luck. However, if you own the web server, or have administration access to it, then MMCache is for you - rush to the download page and get it!

In the following, I will describe the installation process of Turck MMCache for Linux:

Extract the package. It creates an folder with all files. Change to that folder.
Find the place where the phpize program is installed on your system (type which phpize). Mine is on /usr/bin, so what is before the "/bin" is my PHP_PREFIX.
Type the following:

Code:

export PHP_PREFIX="/usr"
$PHP_PREFIX/bin/phpize

You get the output:

Code:

autoheader: `config.h.in' is created
You should update your `aclocal.m4' by running aclocal.

so you run

Code:

aclocal
Now you are ready to create the Makefile with the configure script:

Code:

./configure --enable-mmcache=shared --with-php-config=$PHP_PREFIX/bin/php-config

After that, the Makefile is there and you can do (you must be root for the last step)

Code:

make
make install

This wil finish installation of MMCache on your server. In my next post I will discuss how you use it to squeeze the ultimate performance out of your PHP-Nuke.

Nuke Cops Founder Joined: Nov 14, 2002 Posts: 5939

chris wrote:

After installation, you need to configure MMCache:

Open the php.ini file (/etc/php.ini), find the "extensions" section and put the following there:

Code:

[extension section]
... various other extensions

extension="mmcache.so"
mmcache.shm_size="16"
mmcache.cache_dir="/tmp/mmcache"
mmcache.enable="1"
mmcache.optimizer="1"
mmcache.check_mtime="1"
mmcache.debug="0"
mmcache.filter=""
mmcache.shm_max="0"
mmcache.shm_ttl="0"
mmcache.shm_prune_period="0"
mmcache.shm_only="0"
mmcache.compress="0"
; end of extension section

MMCache uses shared memory for cacheing, as opposed to disk space. This is a reason why it is so fast. The mmcache.shm_size option defines how much shared memory (in MBytes) it should use for cacheing.

Again, you need a cache directory too, which is what mmcache.cache_dir is for. Again, the cache directory should be writable and readable by your web server.

You can read about the other options in the Turck MMCache Homepage. For our purposes, you can leave them at their values as in the example above.

This completes the configuration of MMCache. We stil have to talk about its use - but that's easy, stay tuned! Wink

Nuke Cops Founder Joined: Nov 14, 2002 Posts: 5939

chris wrote:

Question: I am on a virtual server and should not interfere with the other domains hosted there. How do I configure MMCache in this situation?

Answer: Compile MMCache and put it into the extensions directory. Configure php.ini to utilize the extension but set mmcache.optimizer, .compress and .enable all to FALSE.

Then a .htaccess file in your own webspace and use:

Code:

php_value 'mmcache.enable' '1'
php_value 'mmcache.optimizer' '1'
php_value 'mmcache.compress' 1'

This should do the trick.

Nuke Cops Founder Joined: Nov 14, 2002 Posts: 5939

chris wrote:

By following the configuartion steps above, MMCache already works for you: it compiles PHP code and caches it. Whenever a script's code changes, it recompiles it and caches it again. You should already see a noticeable improvement in your response times because of this.

But there is more you can do with MMCache: you can cache a whole page, like with jpcache, or individual parts of them, like with Cache-Lite. Since jpcache uses disk space for cacheing the whole page, you can replace it by MMCache's functionality, which uses shared memory and could thus be up to 5 times faster. Here's how you do it:

Open the index.php file and change the code we put there for jpcache (see a previous post) as follows:

Code:

require_once("mainfile.php");
if (!isset($user)) {
$cachetimeout=900;
mmcache_cache_page($_SERVER['PHP_SELF'].'?GET='.serialize($_GET), $cachetimeout);
} else {
$cachetimeout=-1;
}
# require_once "includes/jpcache/jpcache.php";

As you can see, we commented the jpcache inclusion line and put a call to mmcache_cache_page with the same cache timeout. This will do exactly what jpcache was doing previously - cache the whole page in case the user is not logged-in and leave it uncached in case he is, so that Cache-Lite can cache parts of it. But due to the shared memory implementation, we are going the get an extra speed kick out of it. Smile

mmcache_cache_page is only one of many MMCache functions you can use in your blocks and modules to speed up things. Some other interesting ones are:

mmcache_set_session_handlers(): install the MMCache session handlers. Since PHP 4.2.0 you can install MMCache session handlers in "php.ini" by

Code:

session.save_handler=mmcache
mmcache_cache_output($key, $eval_code, $ttl=0): caches the output of $eval_code in shared memory for $ttl seconds. Output can be removed from cache by calling mmcach_rm() with the same $key. For Example:

Code:

<?php mmcache_cache_output('test', 'echo time(); phpinfo();', 30); ?>
mmcache_cache_result($key, $eval_code, $ttl=0): caches the result of $eval_code in shared memory for $ttl seconds. Result can be removed from cache by calling mmcach_rm() with the same $key. For Example:

Code:

<?php mmcache_cache_output('test', 'time()." Hello";', 30); ?>

There are other uses of MMCache, like its use as an encoder to hide your precious PHP scripts from the spying eyes of competitors - but we are talking open source here, aren't we? Wink

Nuke Cops Founder Joined: Nov 14, 2002 Posts: 5939

chris wrote:

We have seen three approaches to PHP-Nuke acceleration in the previous posts:

jpcache can cache and compress a whole page - on disk
Cache-Lite can cache and compress a whole page or individual PHP-Nuke blocks, modules, variables - on disk (or shared memory too? someone please correct me here!)
MMCache can cache and compress all of the above, plus can do encoding - all in shared memory!

So which tool is the right one for you?

There are a lot of combinations one could try there and without serious testing it is not easy to answer this question definitely. However, from what we have seen already in the previous posts, we can derive some strategies:

If you don't have administration access to your web server, you are left with jpcache and Cache-Lite. Use a combination of them as outlined above, cacheing whole pages with jpcache and individual blocks/modules with Cache-Lite. It's a strategic decision when to use which one - one you will have to make yourself.
If you do have administration access to your web server, MMCache is the way to go. You can use it to cache PHP code and - optionally - a whole page content or just part of it. Whether you should prefer Cache-Lite for the latter, or decide to use MMCache, is something that probably only tests can show.
Ease of deployment can also play a role: with MMCache, you add a line, with Cache-Lite a whole block of lines, to each PHP-Nuke module or block you wish to cache. On the other hand, Cache-Lite may live up to its name and prove itself to be really lightweight in some (or all) situations. Again, you wll have to test.

Let me close by saying that I am interested in any experiences you had in cacheing your PHP-Nuke (err...should we be calling it PHP-Rocket by now? Very Happy

) using the tools I presented. Feel free to post your test or success stories - but also your problems, of course! Smile

Enjoy!

Nuke Cops Founder Joined: Nov 14, 2002 Posts: 5939

chris wrote:

I noticed the following minor annoyance while using Turck MMCache 2.3.22:

At the very top of the page (theme is NukeNews), even before the header, there is a line like

Code:

90032/nuke6/html/index.php?GET=a:0:{}

I have narrowed it down to the call

Code:

mmcache_cache_page($_SERVER["PHP_SELF"]."?GET=".serialize($_GET), $cachetimeout);

that has to be inserted at the start of the index.php file. Currently, I have a slightly different form, but without change in the result:

Code:

require_once("mainfile.php");
if (!isset($user)) {
$cachetimeout=900;
mmcache_cache_page($PHP_SELF."?GET=".serialize($_GET), $cachetimeout);
// mmcache_cache_page($_SERVER["PHP_SELF"]."?GET=".serialize($_GET), $cachetimeout);
} else {
$cachetimeout=-1;
}

Obviously, the output comes directly from mmcache_cahe_page(). But even if I try to catch it in an output buffer and suppress it, like in the following test-cache.php file

Code:

<?php
ob_start();
mmcache_cache_page($_SERVER['PHP_SELF'].'?GET='.serialize($_GET), 30);
$mmcache_output = ob_get_contents();
ob_end_clean();

echo time();
phpinfo();
?>

even then, I get that line at the top of page.

Any hints are welcome.

Captain Joined: Oct 20, 2003 Posts: 335

Very interesting stuff.

ob_start( 'ob_gzhandler' );

This code is already in mainfile.php. It uses the zlib to compress files.
This info is pretty clear- http://www.desilva.biz/php/zlib.html

It mentions GZIP in that thread as an option of the cache program (CACHE your pages, not compress). Better to leave the gzhandler and turn off the gzip in jpcache?

$JPCACHE_USE_GZIP = 1; // Whether or not to use GZIP

Does gzip=zlib? Are they two different animals, or the same?

Captain Joined: Oct 20, 2003 Posts: 335

Hmm. Well, I just did the jpcache part of all this, and I told jpcache NOT to use the gzip cause I left the ob_start( 'ob_gzhandler' ); thingy in there.

I dont notice any difference. Any way to tell if this is doing anything? And, I guess 'of course', the gzip test page (cool idea) doesnt show anything cause I told jpcache not to use gzip.

Maybe its best to leave the gzip in jpcache on? (and comment out the ob_start( 'ob_gzhandler' ); )?

Back to testing....

Someone knowledgeable in this would be a great help.