Cleaning up TeamCity build confi­gu­ra­tions

17. April 2023|Tech|
jonathan-borba-h6WeqL81ZEw-unsplash

In this blog post I will describe how I went from roughly 150 build confi­gu­ration, which were copy-pasted from each other, to all of them using a single template and only speci­fying the diffe­rences in each build confi­gu­ration. TeamCity Kotlin DSL feature was used in order to simplify this normally unbelie­vably tedious task.

The problem

We run contract tests on TeamCity. With a big number of micro­ser­vices, we had an even bigger number of contract tests, each with their own build confi­gu­ration. Unfort­u­nately the build confi­gu­ra­tions were copy-pasted from each other, with some receiving updates over the time and others not. This issue was ignored so far, because there simply was very little incentive to fix it. However, I needed to pass in a new parameter to ALL build confi­gu­ra­tions and was faced with the problem of having to do it 150 times. Through the clunky web UI non the less => no way I’m doing this.

The idea

TeamCity has the Versioned Settings feature, allowing you to store all settings as Kotlin code in a git repository. With code I can simply use search and replace queries, meaning it does not matter how many build confi­gu­ra­tions I have to update, the work stays mostly the same.

This means I need to

  1. Activate the Versioned Settings feature
  2. Figure out the template, that fits all build confi­gu­ra­tions
  3. Adjust all build confi­gu­ra­tions to use the template

The imple­men­tation

Activate the Versioned Settings feature

Just follow the explana­tions from the documen­tation to do this. The current documen­tation is:

To enable it, go to Project Settings | Versioned Settings | Confi­gu­ration.

One problem I faced was that when using their Kotlin DSL as the code format, the ids need to be in the portable format, which was not the case. This can be achieved by regene­rating all ids. Be mindful of the warning shown before executing this.

Figure out the template

This was simply looking through a few build confi­gu­ra­tions and seeing which parts were mostly the same. In my case it were the vcs, steps, build features, and agent requi­re­ments. There were some diffe­rences, which I ascribed to them being outdated.

Adjust all build confi­gu­ra­tions to use the template

This part was done through a series of search and replace queries. I defined a custom search scope, which only included the build confi­gu­ra­tions, that should be changed. With this I used the “Replace in Files…” (⌘ + shift + R) with the above scope selected and could just replace it in all places.

Some of my queries were super simple like templates(_Self.buildTypes.DockerSupport) to templates(_Self.buildTypes.DockerContracttester). Others were some cursed regex, that worked most of the time, but required to verify the changes after­wards. Like steps\s*\{\n(.|\n)*(triggers) to $2, which removed the steps block. The (triggers) was needed, because I was not able to capture the closing brace properly, so I captured the next word following the brace and put it back with $2.

And that’s it. I did this in multiple itera­tions, starting with a single build confi­gu­ration and then all the ones from my team and then the ones from other teams. This way I could verify that the changes were correct and that the build confi­gu­ra­tions still worked, without impacting too many people at once.

Written by Xiang Rong Lin Developer @ Europace