Nowadays it is essential to be able to deploy to production in a reasonably short amount of time and in an automated manner. In this post, I want to summarize my experience with creating deployment pipelines and I will try to provide as many details as possible, despite the broad topic. I will focus on two concrete tools I have used in the past years – TeamCity and Octopus Deploy, which have helped me create automated deployment pipelines.

The big picture

We apply trunk-based development, meaning all changes go to one master branch in our version control system (VCS) (we use git on BitBucket). Every developer creates a pull-request (PR) with the desired changes and when approved, they get merged into master. For a PR to be merged, there are two conditions:

1. There should be a successful build in TeamCity on the merge changes before they get merged into master. GitHub, for example, creates a special branch for your PR, which includes the merged changes (if they can be merged).
2. There should not be any incomplete task in the PR. Tasks are created by the reviewers to improve the PR

The master branch is considered stable in terms of deployability, i.e. we can always create a deployable package from master. Such a package is used by Octopus and is deployed automatically on each environment, starting from test and getting promoted to staging and production.

Continuous Integration using TeamCity

The idea of Continuous Integration (CI) is to ensure the proposed changes (by your PR) can be merged into the trunk without negatively affecting the existing code. This is achieved by using rules you and your team has agreed upon. In my team we use C# as a programming language, but these rules are more or less the same in other languages too.

1. The project can be compiled with new changes. If you work with an interpreted language (e.g., PHP), you could perform a syntax check to validate the code.
2. Perform static code analysis to improve your code. This includes style check to enforce common development style in the team. We use FxCop and StyleCop for C#.
3. Run automated tests to ensure the existing code is not broken and new code works as expected. This includes both unit and integration tests (e.g., against a real database). We use NUnit to run automated tests.

TeamCity allows you to create a build configuration where you can define a series of steps to be executed when a build is triggered (after a change in your VCS). We have one build configuration, called Developer Builds, and its steps could look like this:

TeamCity comes with a number of built-in step templates for you, but if you cannot find a suitable one for your needs, TeamCity can be extended with custom ones, called meta-runners. Or you can use the step template for calling a command line utility or PowerShell.

Running the CI pipeline locally

Initially, we started with this approach but encountered some irritations on the way. Often you would want to run the entire pipeline on your computer to see the effect of your changes before committing them to the VCS. You can, of course, push your changes to a custom user branch or fork in your VCS and let the build server run the pipeline for you, but this is slow. Furthermore, it requires that your local development environment is exactly the same as the one set up on the build server. The solution for us was to introduce a script in the project’s repository that runs the whole pipeline. Bash, batch, and PowerShell are only some examples of scripting languages you can use, and TeamCity supports them all by default. We decided to use Cake, which is

[…] a cross-platform build automation system with a C# DSL for tasks such as compiling code, copying files and folders, running unit tests, compressing files and building NuGet packages.

Cake uses a PowerShell bootstrapper, which means it can easily be run both locally and on the TeamCity server. The Cake script we use contains the same activities as mentioned before.

The Cake script downloads the necessary tools, for example, the NUnit runner, to run the tasks inside. The only requirement for the TeamCity server is to have Visual Studio installed. Then our TeamCity CI pipeline changes to this:

We use the same structure for all our projects. This means every project has a build script, which is placed at the same location in the repository and is almost identical to other build scripts in other projects (with few variations like project name). In TeamCity we use templates when creating projects, but every project configuration uses the same single step – calling the build Cake script.

Creating deployable packages

When a PR gets merged into master, it means for us that a deployable package can be created. It does not tell us anything about the correctness of the code, this is yet to be tested when the code gets deployed to a test environment. In TeamCity we have a second build configuration, called Deployment Builds. The sole purpose of this configuration is:

• To create release notes based on all commit messages since the last package
• To create a NuGet package with the assets to be deployed
• To create a release in Octopus and, optionally, to trigger a new deployment to a test environment

Our deployment build configuration in TeamCity looks like this:

There is no built-in step template for creating release notes in TeamCity, but JetBrains has collected a useful meta-runner power pack, where you can find step templates for generating release notes from GitHub, BitBucket, JIRA, etc.

As with the build Cake script, we also have a pack Cake script to create a NuGet package. The pack Cake script is similar to the build one, without the need to run automated tests and perform code analysis. Furthermore, your .NET project needs the OctoPack NuGet package, which creates Octopus-compatible NuGet packages when RunOctoPack property is specified in the MSBuild process.

Octopus Deploy provides meta-runners for TeamCity, so you can do things like creating and deploying releases directly from TeamCity.

Although the deployment build configuration in TeamCity could be triggered when merging to master, we have chosen to do it manually. The reason is we have a limited number of test environments and we want to make sure we don’t perform a deployment while someone else is testing.

Deploying packages with Octopus

Octopus Deploy is a tool that has one main job: it takes a source package (e.g., *.nupkg, *.zip, *.msi) and deploys it to a set of configured environments using a configured pipeline. As one can define a pipeline in TeamCity, so can one in Octopus.

You start by creating your environments and the deployment lifecycle. Usually, you would start by deploying your application to a test environment, then promoting it to a staging environment and at last – to production. Octopus provides a dashboard where you can see which version of your application is deployed on which environment.

The deployment pipeline, as in TeamCity, consists of a series of steps to be executed. Octopus comes with some built-in step templates, but there is also a library with community-written ones. You can write your own too. In the .NET world, one would usually deploy an IIS application or a Windows service, and there are built-in step templates for this. For other languages, e.g., Java, there are similar step templates too.

Configuring package feed

Octopus needs access to a certain package repository (feed). It comes with a built-in feed, where you can push your deployable packages, or you can configure an external feed (e.g., GitHub, Maven, NuGet). We keep our NuGet packages in TeamCity – it exposes a NuGet feed, which Octopus can use to fetch the packages.

When creating a release in Octopus, you give it a version and a link to the desired package from your package feed. You can then deploy this release to your environments according to the project’s lifecycle.

Deployment targets

The machines where your code will be deployed are called deployment targets. In our company, we have predefined (static) virtual machines, but Octopus has integration to Docker and (further) Kubernetes. Each deployment target has a role (a tag) to define what applications it contains (e.g., web, database, service).

Configuration management

One of the fundamental principles of Continuous Delivery (CD) is build once, deploy anywhere. This means you build one version of your application and you deploy the same version to all environments. In practice, however, things like usernames and passwords differ from one environment to another – you have to inject them in the deployment process. Octopus has the notion of variables, which you can restrict to a certain environment. In the deployment process you can then perform variable substitution in your configuration files (e.g., Web.config)

Conclusion

TeamCity and Octopus Deploy are two very similar tools, which you can use to build a CI/CD pipeline for your projects. Both are extensible, flexible and easy to integrate with other popular tools. Both have a community to provide help with potential issues and to vote on new features. Both can be installed on-premise (as we do) or run in the cloud as Software-as-a-Service (SaaS).