Alright, so its been a couple of days since i last updated the blog.

This is the story about how i crashed our dedicated server taking down all our client websites along with it.

We run two separate servers, one for development and testing and the other for deployment. The development and testing server isn't stable and has very poor backup options. So we thought we'd create a separate account for testing. So i began fidgeting with the server installing Subversion and Trac. While installing Trac i made an extremely noob mistake, something i should have taken precautions while doing, but didn't. I ran a Perl install command from the root account instead of the client account.

I was already playing with the httpd apache file and so after running the install command when i rebooted, all hell broke loose and apache simply refused to start. This is when i started to panic, if it was the test server the downtime wouldnt have mattered all that much, but considering the fact that it was our deployment server and alot of projects were serving content on the internet was a big problem.

I quickly called up the support of our hosting company and raised a ticket along with it. Thankfully the support team restored the last working copy of the apache httpd file and peace was restored.

Firstly, i cant emphasize enough how important it is if you are running a deployment server that you have good support. We host our server with Liquid Web and kudos to them for the prompt response.

--Eddy