Clker.com - weblog » server

Caching SQL results with PHP

Mohamed Ibrahim — Mon, 12 May 2008 15:54:36 +0000

I’ve been looking around lately on the best way to cache SQL results in PHP. I found some interesting articles posted in lots of places, but I didn’t find any that exactly matches my needs. The problem I have on hand is basically the same every growing website faces: decreasing mean resource usage per page request.

Now – this is my plan A to keep up with the website’s growth without a lot of hardware upgrades. There is a plan B, but I will keep that to a later post.

Usually, the time consumed by executing the SQL statements is more than 80% of the whole script time. Reducing this time will significantly reduce the resource usage. CPU is not the bottle neck, usually SQL queries result in a significant disk access especially if there’s a lot of inserts/updates/deletes.

The easiest and best way I found is to use PHP’s serialize to store the SQL results. This requires that we fetch all the rows we’re dealing with. So the sequence of operations is to check the cache file, if expired or does not exist then fetch all the rows from the result set, serialize in one variable, write it to the cache file.

require_once "utils.php";
$cache_prefix=‘/tmp/clkercache/’;
function cache_fetch($filename, $expiration=3600)
{
global $cache_prefix;
$filename=$cache_prefix.$filename;
if (!file_exists($filename)) return false;
if (time(0)-filemtime($filename)>$expiration) return false;
$data=unserialize(base64_decode(file_get_contents($filename)));
return $data;
}
function cache_store($filename,$data)
{
global $cache_prefix;
$filename=$cache_prefix.$filename;
@mk_dir($filename);
return file_put_contents($filename,base64_encode(serialize($data)));
}
?>

So the way to use the caching functions would be something similar to this:

if ( !($pictures=cache_fetch("mypictures.cache")) ){
$res=query("select pictureid from pictures order by rating15days desc limit 100");
while ($row=fetch_array($res))
$pictures[]=$row;
cache_store("mypictures.cache",$pictures);
}
?>

So how to do that if you have multiple queries? The way I store my cache files is by giving them suffix or prefix names depending on what is being stores. For example a file that store the pictures that are related to a picture whose id is 1500 being displayed, will be called related-1500.cache. For more complicated cases like searches, I would use the md5sum of the request URI.

The results were good. Using apache bench I tested the homepage. Before any caching it used to support 3 pages per second. After caching the result was 35. I think in my current situation with my current traffic, this is a very good result.

One might think so what does it take to generate pages at a much higher rate – say 300, 400 or even a 1000 or more? Well, that’s my plan B, and I’ll discuss that in more detail in another post. This plan B will generate that much pages per second.

Technorati Tags: optimization, server, sql, php, apache, bench, cache

Running your server is easy, fun but involved

Mohamed Ibrahim — Sun, 02 Mar 2008 01:09:09 +0000

I love using Linux to do my work. My best usage of Linux is my web server, although I recall I read once that Linus never intended for the kernel to be used as a server. He was more focused on using the kernel in desktops. I’ve been running my own web server for almost a year now, which runs two websites mibrahim.net my real estate website, and clker.com a to be online clipart website – we’re halfway there.

The fun part is simply everything just works. You’ll have all the tools you need starting from database engines like postgres, mysql to scripting languages like php, ruby with different types of webservers apache, lighthttpd and others. All the tools you might think of are there and under your own hand. Building your own server is not expensive – around $100 will do it. You don’t need a super quad core machine to produce extremely fast websites, unless you are already getting more than 50 page requests per second and at that point you will need something faster.

The performance bottle neck is never the CPU, it’s the hard drives read or write speeds. You can improve on that using fake RAIDs. Almost all Linux distros offer fake RAIDs and that is the cheapest way to improve the read speed.

Setting up your server is not a hard process. The best distributions that I recommend are Debian and Ubuntu. The reason is the very large library of software that comes with each. I believe that now the full distribution has grown more than 11 CDs. I used to run Debian and switched to Ubuntu a year ago and the reason behind the switch is the faster updates I get from Ubuntu, which enables me to use more recent and updated versions of PHP and the database engines.

The easiest setup is using the Ubuntu server CD, which is not any different from the desktop CD in terms of binaries. The only difference is that it won’t install the X11-server (GUI) and the window managers (gnome or kde) and the install program itself runs over the console and not VGA graphics. I use the server installation, and connect to my server using ssh. I have another old machine that runs Ubutu as well, and is used to run freenx. By that way I keep the server’s memory for the services running, and I can add all the GUI programs I want on this old machine.

Since I greatly benefited from running my own web server, I will share my experiences every now and then when I’ve got time to write.

Technorati Tags: ubuntu, linux, server, apache, database, sql server, database server, web server

Hardware problems – the mystery solved

Mohamed Ibrahim — Tue, 11 Dec 2007 18:34:54 +0000

I previously wrote about hardware problems that were puzzling me, and I finally found the answer. I installed sensors on ubuntu and ran ksensors and just watched what’s going on. The old server that was shutting down suddenly had an opteron dual core processor, which ran at 50 degrees Celsius at no load. I had three websites running there, the most recent addition is clker.com . Due to the large number of people that were trying to access the website, the CPU was loaded and overheated, which inturn lead to it shutting down.

Of course the solution is to put the server in a very cold room, which I don’t have. I resorted to buying a new board and CPU, which does not heat as much as the old ones and transfered the database and pictures to it. Currently the no load temperature is 11 degrees, which I think is great. This new board is a special sale from microcenter, both the board and CPU were for $94 with $15 rebate Sempron 64! I added 1GB of RAM. It really worked much better than my expectations, as I had very very low performance expectations from that CPU

Currently this server is running clker.com only, and will run mibrahim.net today. The CPU load as shown in top does not pass 2% except in spikes. People are still digging as well as search engine robots. So that’s a good value if someone wants to run his/her own starter website.

Technorati Tags: ubuntu, shutting down, temperature, sempron, AMD, performance, linux, web server, server, php, postgresql

Powered by Ubuntu

Mohamed Ibrahim — Wed, 21 Nov 2007 22:49:56 +0000

One of the SVG pictures that had troubles is the powered by Ubuntu logo. In an earlier post I described that I’m doing some SVG cleanup, and still to expect more cleanup in the future. One of the images that I was looking forward to is the powered by Ubuntu logo. So I’m up to some coloring for this night ! I will try to make a unified theme for both the website part, and the wordpress weblog part, and of course will incorporate the powered by Ubuntu logo, since this server is actually powered by Ubuntu.

Technorati Tags: ubuntu, powered, powered by ubuntu, server, webserver, apache, wordpress, theme, css, style