A Step by Step Guide to Migrating a TeamCity Instance from One Server to Another
If you've been using TeamCity for a number of years, there will come a time where the server it's running on will reach its end of life and your TeamCity instance will need to be moved to a fresh server.
As part of this migration process, you may decide to change OS from Linux to Windows or persistence mechanism from the internal HQL data store, or an Oracle Database to SQL Server or MySQL. Luckily, due to the way that TeamCity has been designed, all of these scenarios are supported.
This step by step guide is going to cover how to migrate to a new Windows Server instance running Microsoft SQL Server.
Step 1: Backup TeamCity Database Data
In order to perform a migration, you first need to understand how TeamCity is architected, especially how TeamCity stores its data in two separate repositories; the Data Directory for server settings, build artefacts, logs & configuration, based on the file system and a second, containing build information, configuration & statistics which is stored in a database (HSQLDB, Microsoft SQL Server, MYSQL, PostgreSQL, Oracle). To perform a migration you need to backup (and restore) both of these data repositories.
There are two main mechanisms for backing up the TeamCity database; the first is using the maintainDB tool – this gives you command line control of the back up process. The tool can be found in <TeamCity Home>\bin directory and exists in the form of a .cmd and a .sh file and is quite simple to use, if you want to back up everything (configuration, database, build logs, personal changes) you would run it using the following switches:
maintainDB backup -C -D -L –P
By default the backup will be created in the <TeamCity Data Directory>\backup directory with the following file name: TeamCity_Backup_<timestamp>.zip, where the timestamp is in the YYYYMMDD_HHMMSS format.
The second, and easier mechanism is to use the TeamCity Admin UI. To access the backup screen you will need navigate to the Administration screen (assuming you are a TC Administrator) and then select the "Backup" option from the secondary navigation. Once you are in the backup screen you can select which items you want to select. I've highlighted the UI which shows you where the backup will end up. Click "Start Backup" to begin:
The screen will update to show you the backup process. Depending on the number of build configurations, amount of historical data etc. – the backup process can take a number of minutes:
Once the backup has completed, TeamCity will display a summary and show the path of the backup which is also a hyperlink that you can click to download the file:
Step 2: Stop TeamCity Services
In order to take a backup of the TeamCity Data Directory, you need exclusive access to the files, so you need to shut down the TeamCity Services.
To stop the TeamCity service, open PowerShell and enter the following command:
Stop-Service TeamCity
Stopping the TeamCity service may take a few minutes.
If you have a Build Agent installed on the same server as the TeamCity Service you will also need to stop this. Use the following command, in a PowerShell console:
Stop-Service TCBuildAgent
Step 3: Backup TeamCity Data Directory
Now that you have exclusive access to the TeamCity Data Directory you can back it up. If you want to find the location of your TeamCity Data Directory, the easiest way is to log into the Administration dashboard and navigate to Server Administration > Global Settings where you will see the following information:
If space is at a premium, there are two directories that can be excluded, highlighted below:
Copy this folder to a temporary directory on the new machine.
Step 4: Install a matching version of TeamCity onto the new server
I've already written A Step by Step Guide to Hosting TeamCity in IIS 7, which contains all the setup instructions.
One additional comment is that the default location for the TeamCity Configuration Directory has changed in TeamCity version 7. In previous versions the default location was %USERPROFILE%\.BuildServer. If you are planning to perform an upgrade post installation, you might want to set this path to be the new default (and much more sensible) location of c:\ProgramData\JetBrains\TeamCity.
Another thing to note is that the ProgramData directory is hidden by default, so make sure you enable showing hidden items in Windows Explorer:
Of course – if this is a production TeamCity environment, I would strongly recommend that you install TeamCity into a non-system disk partition (D: drive for example), to ensure that if your TeamCity artefact repository fills up all available free disk space, it wont take down the server.
Last, yet, very important step: when prompted, do not select to start TeamCity during the install process:
Step 5: Configure Environment Variables for Java
As TeamCity contains it's own, self-contained version of the Java Runtime (yes, TeamCity is a Java application), if you want to run any of the tools from the command line, you'll need to set up Environment Variables that match the install. Open a PowerShell Command prompt and run the following two commands:
[Environment]::SetEnvironmentVariable("JAVA_HOME", "<TEAMCITY_PATH>\jre\bin", "Machine")
[Environment]::SetEnvironmentVariable("JRE_HOME", "<TEAMCITY_PATH>\jre", "Machine")
Step 6: Install SQL Server
I'm not going to cover how to install SQL Server - but there are several other good Step by Step Guides you can use.
Once you have a new SQL Server setup we need to create a new Database called TeamCity. Next, create new user called TeamCity and assign the user the following roles:
- db_datareader
- db_datawriter
- db_owner
Step 7: Install Microsoft SQL Server JDBC Driver 3.0
In order for TeamCity to be able to communicate with Microsoft SQL Server, it requires a JDBC driver. You can download the official Microsoft driver from the Microsoft Download Center.
Unpack the .exe and copy sqljdbc4.jar to <TEAMCITY_DATA_DIRECTORY>\lib\JDBC
Step 8: Create Database Properties file
TeamCity requires a Database.properties file to use during the backup restoration process. Create a new text file, add the following content (inserting your password):
connectionUrl=jdbc:sqlserver://<Server Name>:1433;databaseName=TeamCity
connectionProperties.user=TeamCity
connectionProperties.password=<Password>
and save it to c:\temp\database.properties
Step 9: Restore Database Backup
To restore the database backup we made in step 1, we are going to need to use the maintainDB tool on the new TeamCity environment.
Run the following command, making the changes that represent your environment. N.B. all directory paths must be fully qualified.
C:\TeamCity\bin\maintainDB.cmd restore -F C:\Temp\TeamCityBackup_20130725_092952.zip -A <TEAMCITY_DATA_DIRECTORY> -T C:\Temp\database.properties
If you just want to restore the database (in the example that you want to migrate away from Oracle and to MySQL, you can use the following command line)
C:\TeamCity\bin\maintainDB.cmd restore -F C:\Temp\TeamCityBackup_20130725_092952.zip -T C:\Temp\database.properties -D
If you've done everything correctly, you should see the following output:
One further gotcha; the above screenshots are taken from the TeamCity 7.x version of maintainDB, if you are performing a migration using TeamCity 6.x, maintainDB has a couple of differences.
Firstly, you need to run the tool, from your command prompt rooted in it's working directory, otherwise it has difficulties locating other files it relies on.
Secondly, you have a final manual step to perform – copying across the database.properties file into the config directory:
If you are going to start TeamCity with the database just restored, don't forget to copy this file to
C:\ProgramData\JetBrains\TeamCity\config\database.properties
If you see the following error: "Unexpected exception SQLServerException: SQL error when doing: Connecting to MSSQL":
You may need to enable the TCP/IP protocol for your instance of SQL Server:
Step 10: Merge in Data Folder
Now you're restored the build information, configuration & statistics you now need to restore the server settings, build artefacts, logs & configuration. To make this step as easy as possible, I would recommend using a file / folder comparison tool, such as (the wonderful) Beyond Compare (or something like WinMerge).
You need to perform a diff between your backup of the Data Folder and the newly created one. The main folder that you will need to synchronise is the artefacts directory:
Step 11: Update Build Agent Configuration Settings
Next, you need to update the Build Agents to point to the new TeamCity server. RDP into each Build Agent and open the following file in a text editor:
<AGENT HOME>\buildAgent\conf\buildAgent.properties
Edit the line:
serverUrl=http\://localhost\:81
and change the serverUrl to match your new TeamCity server, remembering to honour the escaping format used.
Next you need to restart the Build Agent for these changes to take effect. Use the following command, in a PowerShell console:
Restart-Service TCBuildAgent
Step 12: Start TeamCity Services
Now you've completed the migration, you need to start the TeamCity services.
Use the following command, in a PowerShell console:
Start-Service TeamCity
Start-Service TCBuildAgent
If you browse to the new TeamCity instance and see an error like:
TeamCity data format version mismatch: configuration version is 454 but database is 418
You need to check that you have copied over the database.properties file!
Step 13: Test
Now the migration is complete, you need to test that everything is working as expected. Commit some changes, run all the build configurations and ensure that the complete as before.
If you see the following symptoms, whereby TeamCity cannot connect to your Git Repositories:
Then you will need to copy the default private key for Git from your original TeamCity Server.
Step 14: Check your TeamCity License
Before you attempt upgrading your TeamCity instance to the latest version, you should check your TeamCity license keys to ensure that they are valid for the latest version:
If not, you are going to need to contact the JetBrains sales team and get a quote for an upgrade, and then go talk to your budget holder to get it approved! Otherwise you will experience the following:
Another thing to note is that email support by JetBrains is only available with a valid License key (i.e. not for the free version).
Step 15: Backup
Now that you've done all the hard work, it would be foolish to attempt a major upgrade without having a backup of your freshly migrated environment. You can just repeat steps 1,2 & 3 from this guide.
Step 16: Upgrade
Now you can perform the upgrade to the latest version of TeamCity.
If you are upgrading to TeamCity 8 – it is worth reviewing the upgrade notes and upgrade steps.
If it is a major version update, you may need to follow the instructions listed in Step 4.
If there is a major data schema change, you will have to enter an administrator token to action the one-way schema upgrade.
Once you've upgraded, repeat the step 13, to ensure everything is working as expected.
Work Smarter, Not Harder.
@HowardvRooijen | @endjin
