I have a few servers that I look after at work, and having spent too long staring at them day after day, night after night, just to make sure they are working, I decided I needed some better alerting and monitoring.
Being a Ruby fan I looked closely at both 'god' and 'blue-pill', but at the end of the day decided against both. Partly due to concerns raised about memory-usage with a Ruby-based package (although most of these fears seem to have been allayed in recent versions), but also because you really don't want a big footprint on a monitoring product. You should not need to monitor a monitor.
So at the end of a little research I installed 'Monit' on the servers. This has a nice little web interface (secured) that will enable you both to monitor, and restart monitored services. Ideal if you can't SSH onto a server and just need to get things back up and running. The monitoring file syntax is simple and there are examples for the common sorts of services you might want to monitor (MySQL, Postfix, Apache, ...) You can be simply alerted on certain conditions (storage, CPU, #children) or have corrective action taken (restarting a service; stopping it; so on).
I've found at lest one thing to be wary of though. You can monitor a "web process" (e.g. Apache) by checking once per 'monitoring cycle' that your specific port responds to a specific protocol. This can go as far as checking the response from the protocol - e.g. test that page 'home.html' returns some regular expression. Quite nice, but rarely needed - Apache is rock solid in my experience. Simply knowing the process is active is good enough. However, if you *do* choose to run a default 'protocol HTTP' test, then this will 'GET /' from your site. I was looking at my MySQL logs and noticed a connection and a couple of SELECTs every 5 minutes. Baffling, until I noticed they coincided with the Monit probes as seen in the Apache logs. The website in question sits at the base of the domain, and when sending the home page also checks to see if a cookie is present in the database, and if not builds a default response, from default settings held in the database - hence the calls to the database. I'm not happy about polling my database server every 5 minutes (see one of my previous posts!) so I've now reverted to a simple port 80 TCP probe - that just checks the port is open rather than sends in an HTTP request. I feel this is good enough ... as I said, if Apache is there at all, then it's working.
So you might have a large home page with lots of dynamic content. Be aware that that page will be fetched by Monit whenever it checks your web server. Probably not an issue for most, but weigh the cost against the value.
