Clker.com – weblog

My server has been suffering from unexplainable apache hangs. Once in a while, apache would stop accepting new connections. It will keep running in memory but all incoming connections will timeout. Since then, my only solution was a bash script that runs every minute and tries to read a text file from the website. If it fails, it would restart apache.

On Fri evening, I replaced APC with Memcache. Although I know that Memcache is little bit slower than APC due to lots of reasons including network overhead, yet it was APC’s turn to get tested in the sequence of tests I’ve been running. It seems that APC was the reason apache was hanging. I know that some big websites like facebook use APC, but maybe they are using a different version. I also know that youtube is using memcahe, so APC was one of my least suspects. The server has been running for the past 48 hours without a crash or a hang, which is more than its average. Usually, it used to hang once every 20 hours or so.

Since I had my own caching functions, that finally called APC, the amount of code changes were very small. Hopefully it will continue running without problems.

Some people think that injuries only happen if you’re doing physically demanding work. However, lots of people who work in the computer / IT field suffer injuries, it’s just that they don’t know it!

Have you ever felt back pain or muscle spasm – may be one that lasted like 6 weeks without being able to turn your head, lift heavy things? Some people think that Carpal Tunnel Syndrome is yet another injury cause by compressing a nerve as a result of using keyboards.

The most common is the back / shoulder / neck pains and it happens frequently to people who spend a lot of time in front of computers & screens. I had a back pain recently, that reduced my activity a lot for the past eight weeks. It was horrible, once I come back from work, I’m almost grounded. That was the fourth time I get this pain, and the second time being so bad to make me go to the doctor & last around 6-8 weeks.

My doctor referred me to a physical therapist, who gave me a good reason for this happening after some tests. As he explained, I suffer from neck muscle atrophy – more on the left side, which is simply weakening in the muscles of the neck due to not using those muscles. As a result, my other muscles had to exert more effort to carry the neck, and the result is pain in the other muscles. I noticed that when I sit and read, I actually lean my head on my hand, so the head is being carried a significant amount of time.

I was advised to do some exercises that should help build the neck muscles and stretch the back muscles that have pain. I also noticed lots of other people doing exercieses while sitting – so I wasn’t alone in this computer related back pain world. Best way to avoid it, stop sitting funny & exercise a couple of time a week & I hope no one else gets it.

Lately, I’ve been having so much unexplainable troubles with apache. After a two or three days of work it suddenly stops accepting connections. A test shows that it is still running with lots of forked children.

After lots of research on the internet I found more than a dozen of possibilities. One case was apache 2.2.9 and php 5.2 combo, with php running as a module. At that point when Apache runs out of MaxClients it stops killing children as they run out of MaxRequestsPerChild. At that point, apache just sits and rejects connections. Issuing a reload did not seem to solve the problem and the only way it would start responding is by restarting the server.

The server was not rebooted approximately for 270 days, which I don’t think should pose any problem. The only problem that I know of is holes in memory pages due to allocation and freeing memory, which may result in kswap running more, however that was not the case as I didn’t see kswap jumping up when I ran top.

Rebooting the server seemed to at least partially resolve this problem. Now when I lower the MaxClients and MaxRequestsPerClient to force apache in this bug, it seems that reloading works. The ulimate test is to leave it running, and see what will happen in a week.

That made me go back to writing my own webserver project. It actually turned to be much simpler that I anticipated, and I think that I will split my spare time part to add some new features in php, and the other to finish this c web server.

I’ve been seriously considering writing my own webserver instead of running apache. Although apache is very good, yet, sometimes I feel things will be much funner if write this website even in C. My reasoning is although PHP is fast and APC provides a further boost, yet at one point I will have to cluster a set of machines and proxy the requests because of various tweeks that need to get done in every piece of software.

So, I started considering seriously how to write a webserver that is optimized for high requests and can make better use of memory, and how to resolve the biggest problem in C, which is crashes due to memory allocation and writing outside arrays without slowing down the C speed even a bit.

I ended up writing a small library does that memory stuff, and a small webserver and I’ve been playing with them for a while and without a surprise, I found that I can serve more than 1000 pages a second very easy without attempting to optimize dealing with strings and just with basic STL string, which in my opinion is very slow.

What I still need to do is to support the CGI stuff, so I can execute the current PHP code and run it beside the C and that will give me more time to port the website pages in phases without stopping the website, or delaying running this server.

If you didn’t know, Flex and actionscript flash imposes a limit on the bitmap size. For some reason, the max width or height cannot exceed 2048 pixels.

When applying a transformation matrix to a bitmap, it seems that Flex generates a temporary bitmap in memory, even for the area that are not going to be displayed. Since there is a limit on the bitmap size, usually it won’t resize or zoom more than a certain amount.

This problem will show if you have a graphical application that views a bitmap, which has a zoom-in feature. The solution that I found to this problem is: after calculating the transformation matrix (scaling from zoom, translation from the scroll bars), calculate the inverse of the matrix. Using your viewport co-ordinates (usually will be 0,0 and width,height) determine the area in the bitmap that will appear if the bitmap was zoomed. Copy that area to a temporary bitmap, Calculate a new transformation matrix composed only of the zoom, and apply it to the temporary image. That’s it!

It was the first time yesterday that I took sometime to understand what is the “average load” that appears with the top shell command. It turns out that it is a very informative number, if used right.

Linux averages the number of processes running, or can run – but may be blocked waiting for I/O. The average is calculated three times: for a minute, 5 minutes and 15 minutes.

In case that no processes were blocked on I/O, this number should always be less than the number of available CPU cores. For insance, if the load average is 2.6 this means that in average 2.6 processes were running or wanted to run. If you have only two cores, this means that there’s in average 0.6 processes that cannot find an available CPU to run, which means that you need to upgrade your machine (or server).

Since this number grows when processes are blocked on I/O, it is nice to know whether there’s any blocked on I/O or not before going ahead and spending the last penny to buy new hardrware. The solution is a tool called atop. If the disks are stressed and cannot keep up with the requests (and consequently processes will be waiting for the disk) you will see the DSK line (or lines if you had more than one) in red.

At that point you should find who is the culprit, and do something to run that process in a better way.

I wrote an blog entry about caching sql with php before, since then I re-wrote the php functions that I’ve been using. The main difference betweek the older functions, and what I will describe here is storing in memory versus the disk.

Read the rest of this entry »

Bug updates stopping the plugin to run on some FreeBSD installtions. Thanks to David Newman with networktest.com for his positive & helpful feedback.

You can download the latest plugin from here.

Just fixed a usability problem. Lots of visitors complained that when they clicked “download” nothing happened. By reviewing the website, I found that I used an up arrow, and a down arrow for voting for the picture with a positve vote, or a negative vote. However, users misunderstood that, and I don’t blame them!

I switched the icons to a green up thumb, and a down red thumb for voting for a picture. Also, added a disk icon, with an arrow pointing down in the area, which has the different download formats.

Hope this helps and reduces confusion.

Last week google released a framework similar to flash, with the only difference of running x86 code. They developed a sandbox to limit the type and paramters of instructions that can be run over the browser. It draws over a framebuffer, which then gets updated on the browser. They demonstrated quake, and opengl applications running inside their native client.

Although they don’t have yet a widget toolkit, yet, after I did some research, I found that it is very easy to write an SDL extension, which means that all the libraries running over SDL can be available including 3D game libraries, animation and many widget libraries.

– Just some thoughts…..