Samuel Fisher

← Back

Writing a Terraform Provider in C#

6 min read

Terraform is an infrastructure-as-code tool for provisioning resources. Resources are defined in a language called HCL, and Terraform takes care of creating, updating and deleting these resources that have been defined in code.

It contains a number of built-in resource types for provisioning things such as AWS virtual machines and DNS records, but also allows defining new resource types through custom providers.

Terraform is written in Go, and the official SDK for writing custom providers only supports writing new providers in Go. However, while other languages are not officially supported, it is possible to write custom providers in other languages, including C#. This can be useful when you want to write a provider that makes use of a .NET client library, or if you are more familiar writing C# than Go.

Plugins over RPC

Terraform uses Hashicorp's Go Plugin System over RPC. This is a framework that allows Terraform to call plugins by making gRPC calls to an external process.

It works as follows:

  1. Whenever Terraform needs to call the custom provider, it runs the plugin's binary.
  2. The plugin outputs the gRPC connection details to standard output (stdout).
  3. Terraform connects using these details and calls the gRPC methods it needs.

Since gRPC services can be written in C#, this means that a Terraform provider can be written in C#.

Terraform expects the connection details to be the first thing written to stdout by the plugin process, and they must have the following format:

<plugin protocol>|<app protocol>|<network type>|<address>|<protocol>|<certificate>
  • Plugin protocol: this is the version of the go-plugin protocol that will be used.
  • App protocol: this is the version of the Terraform plugin API that will be used.
  • Network type: setting this to tcp to indicate that Terraform should connect to the plugin over a TCP socket.
  • Address: IP address and port the plugin is listening on.
  • Protocol: we will set this to grpc.
  • Certificate: base64-encoded server certificate that the plugin will provide when Terraform connects to it over TLS. Terraform doesn't validate it other than checking it matches the certificate provided here, so it can be a self-signed certificate.

C# Plugin

This section explains the key points that are required for writing a plugin in C# but does not cover all of the fine details. For a complete example please see the full code on GitHub.

The gRPC part is provided by ASP.NET Core, so we can start by creating a new gRPC project.

dotnet new grpc -o MyTerraformPlugin

This gRPC project needs to implement the Terraform plugin service. The proto file for this can be found here. This can be added to the C# gRPC project by referencing it in the project file:

<ItemGroup>
  <Protobuf Include="Protos\tfplugin5.2.proto" GrpcServices="Server" />
</ItemGroup>

After building the project, a base class for the gRPC service will be generated automatically from this proto file. We can implement this to create a provider.

public class MyTerraformProviderService : Provider.ProviderBase
{
    public override Task<PlanResourceChange.Types.Response> PlanResourceChange(...)
    {
    }

    ...
}

We will need to generate a self-signed certificate to allow Terraform to connect to this service. This can be done using the BouncyCastle library. Generating a certificate is quite verbose, so please see the example on GitHub for details.

Once we have a certificate, we will then tell ASP.NET Core's Kestrel server to use it. This can be done in the CreateHostBuilder method in Program.cs:

public static X509Certificate2 Cert { get; } = ...

...

public static IHostBuilder CreateHostBuilder(string[] args, Action<IServiceCollection, ResourceRegistry> configure) =>
    Host.CreateDefaultBuilder(args)
        .ConfigureWebHostDefaults(webBuilder =>
        {
            webBuilder.ConfigureKestrel(x =>
            {
                // Listen on localhost port 5344
                x.ListenLocalhost(5344, x => x.UseHttps(x =>
                {
                    // Use self-signed certificate generated earlier
                    x.ServerCertificate = Cert;

                    // Don't validate the client certificate
                    x.AllowAnyClientCertificate();
                }));
            });
            ...
        });

Next we need to disable logging to the console, because Terraform reads the connection settings from here and it will fail to run our plugin if there are log messages printed before this. To help with debugging, it is still useful to be able to see log messages though, so logs can be written to a file instead of stdout. Serilog is one such logging library that is able to do this.

Finally we need to add the following code to Startup.cs to write the connection settings to the console. It is important that this isn't written to the console before ASP.NET Core is ready to serve requests, otherwise Terraform will fail to connect. We can make use of the ApplicationStarted lifetime event to wait until the app is ready before writing to the console.

public void Configure(IApplicationBuilder app, IHostApplicationLifetime lifetime, ...)
{
    lifetime.ApplicationStarted.Register(() =>
    {
        logger.LogInformation("Application started.");
        Console.WriteLine($"1|5|tcp|127.0.0.1:5344|grpc|{Convert.ToBase64String(Program.Cert.RawData)}");
    });

    ...

This will always listen on port 5344 which could fail if that port is already in use. An improvement would be to automatically find a free port and listen on that instead.

Use in Terraform

The simplest way to build the provider for usage in Terraform is to use the self-contained single-file publishing mode available with .NET 5.0. This means there is only one file that needs to be copied into the Terraform plugins directory.

To publish the project (assuming it will be run on a 64-bit Linux system):

dotnet publish -r linux-x64 -c release -p:PublishSingleFile=true

Then copy this to the Terraform plugins directory:

# Create plugin directory
mkdir -p ~/.terraform.d/plugins/example.com/example/dotnetsample/1.0.0/linux_amd64/

# Copy binary
cp ./bin/release/net5.0/linux-x64/publish/MyTerraformPlugin ~/.terraform.d/plugins/example.com/example/dotnetsample/1.0.0/linux_amd64/terraform-provider-dotnetsample

To use in a Terraform project, add a reference to the provider in versions.tf:

terraform {
  required_providers {
    dotnetsample = {
      source = "example.com/example/dotnetsample"
      version = "1.0.0"
    }
  }
}

Add to providers.tf:

provider "dotnetsample" {}

You can now create instances of the new resource implemented by the C# plugin. Note that the resource name should be prefixed with the name of the provider in order for Terraform to know which provider is defining that resource type.

Try it out

The source code for a complete example, and a library making it easier to define custom Terraform providers in C# is available on GitHub.

← Back to home