Configuration Management

How best to manage the configuration of the servers

Static Configuration Files These are files such as exim and apache configuration files.
In most cases, these will only be altered when the package is installed.
In most cases, the customised files will not apply to the next version of the package.

Dynamic data files These are password files and network information.
The data is stored in the MySQL database and is managed via the website.
Most of the data will not change very regularly, but when it does, the servers need to be updated soon - otherwise you have to wait 24hrs for a password change.
The files can easily be generated by scripts, but there are lots of scripts on lots of servers each one generating a single configuration file.

Virtualisation

Virtualisation of services would mean that they can be set up once and moved around the servers as hardware is upgraded
Access can be controlled on a per-service level
Does not solve the configuration managment problem
Configurations can be tested on a copy of the service before rolled out for real.

Version Control System

/etc could be put into CVS
A possible structure would have different branches for each server and trunk holds common configuration
Update would be pushed in a similar way to the website ie. on each commit a script is run to push the updates

Advantages

Allows changeing of configuration files in-place with a commit once changes have been made
Easy to roll back to an earlier version or see previous changes on any file

Disadvantages

Its a lot of infrastucture for files that usually don't change after installation and won't be the same when the next version is installed.
People need to remember to do a commit once changes have been made
No per-host permissions
Does not deal with dynamic data files