Cleaning up TeamCity build configurations
In this blog post I will describe how I went from roughly 150 build configuration, which were copy-pasted from each other, to all of them using a single template and only specifying the differences in each build configuration. TeamCity Kotlin DSL feature was used in order to simplify this normally unbelievably tedious task.
The problem
We run contract tests on TeamCity. With a big number of microservices, we had an even bigger number of contract tests, each with their own build configuration. Unfortunately the build configurations were copy-pasted from each other, with some receiving updates over the time and others not. This issue was ignored so far, because there simply was very little incentive to fix it. However, I needed to pass in a new parameter to ALL build configurations and was faced with the problem of having to do it 150 times. Through the clunky web UI non the less => no way I’m doing this.
The idea
TeamCity has the Versioned Settings feature, allowing you to store all settings as Kotlin code in a git repository. With code I can simply use search and replace queries, meaning it does not matter how many build configurations I have to update, the work stays mostly the same.
This means I need to
- Activate the Versioned Settings feature
- Figure out the template, that fits all build configurations
- Adjust all build configurations to use the template
The implementation
Activate the Versioned Settings feature
Just follow the explanations from the documentation to do this. The current documentation is:
To enable it, go to Project Settings | Versioned Settings | Configuration.
One problem I faced was that when using their Kotlin DSL as the code format, the ids need to be in the portable format, which was not the case. This can be achieved by regenerating all ids. Be mindful of the warning shown before executing this.
Figure out the template
This was simply looking through a few build configurations and seeing which parts were mostly the same. In my case it were the vcs, steps, build features, and agent requirements. There were some differences, which I ascribed to them being outdated.
Adjust all build configurations to use the template
This part was done through a series of search and replace queries. I defined a custom search scope, which only included the build configurations, that should be changed. With this I used the “Replace in Files…” (⌘ + shift + R) with the above scope selected and could just replace it in all places.
Some of my queries were super simple like templates(_Self.buildTypes.DockerSupport)
to templates(_Self.buildTypes.DockerContracttester)
. Others were some cursed regex, that worked most of the time, but required to verify the changes afterwards. Like steps\s*\{\n(.|\n)*(triggers)
to $2
, which removed the steps block. The (triggers)
was needed, because I was not able to capture the closing brace properly, so I captured the next word following the brace and put it back with $2
.
And that’s it. I did this in multiple iterations, starting with a single build configuration and then all the ones from my team and then the ones from other teams. This way I could verify that the changes were correct and that the build configurations still worked, without impacting too many people at once.
Written by Xiang Rong Lin Developer @ Europace