A new client site in Terraform

The result: one client, one entry

You maintain several client sites. Each site is one entry in a variables file. For that entry, Terraform creates a Ping of the home page, a Flow of the checkout funnel up to the payment page, and an availability objective.

With the two clients below, the first terraform apply creates seven resources: three per client, and one shared alerting webhook. One more client is one more entry and three more resources. Nobody clicks in the console.

The full example, with its validations, lives in the provider repository, under examples/agency-client. This page follows its code in a shorter form. The example.com domains are reserved for documentation: replace them with the sites you are allowed to monitor.

• Provider examples on GitHub
• Registry pathlyhq/pathly

webhook_url = "https://hooks.example.com/pathly"

clients = {
  bakery = {
    name         = "Example Bakery"
    home_url     = "https://www.example.com/"
    product_url  = "https://www.example.com/products/sourdough"
    checkout_url = "https://www.example.com/checkout"
    severity     = "critical"
  }

  bookshop = {
    name          = "Example Bookshop"
    home_url      = "https://shop.example.com/"
    product_url   = "https://shop.example.com/books/atlas"
    checkout_url  = "https://shop.example.com/checkout"
    objective_pct = 99.5
  }
}

# PATHLY_API_TOKEN comes from your secret store, never from a file
terraform init
terraform plan
terraform apply

# Plan: 7 to add, 0 to change, 0 to destroy.

The code: a Ping, a Flow and an objective per client

The clients variable is a map of objects. Its key is a short, stable identifier: it becomes part of every resource address and of the client tag. Each resource walks that map with for_each.

The Ping is a pathly_scenario with a url: it checks that the home page answers. The Flow is a pathly_scenario of type browser, with its actions in steps. A real browser opens the product page, adds the product to the cart and opens the payment page. The first step is always a goto.

The Flow stops when the payment page is displayed. No step clicks a pay button, and it only types test values. The selectors and the expected text depend on each site: check them in the browser inspector before you apply.

The availability objective is a pathly_sla_target bound to the Ping. objective_pct sets the target over a rolling window of window_days days. warn_at_budget_ratio warns when 80% of the error budget is spent, while there is still room to act.

The API never returns the steps of a Flow, only their fingerprint (scenario_fingerprint). The code is therefore the only readable copy of the journey: keep it in the repository, reviewed like everything else.

terraform {
  required_providers {
    pathly = {
      source  = "pathlyhq/pathly"
      version = "~> 0.1"
    }
  }
}

provider "pathly" {}

variable "clients" {
  type = map(object({
    name          = string
    home_url      = string
    product_url   = string
    checkout_url  = string
    severity      = optional(string, "major")
    objective_pct = optional(number, 99.9)
  }))
}

resource "pathly_scenario" "home" {
  for_each = var.clients

  name            = "${each.value.name} - home page"
  url             = each.value.home_url
  interval_sec    = 300
  expected_status = 200
  severity        = each.value.severity
  folder          = each.value.name
  tags            = ["terraform", "client:${each.key}"]
}

resource "pathly_scenario" "checkout" {
  for_each = var.clients

  name         = "${each.value.name} - checkout up to payment"
  type         = "browser"
  interval_sec = 3600
  severity     = each.value.severity
  folder       = each.value.name
  tags         = ["terraform", "client:${each.key}", "checkout"]

  steps = [
    {
      op  = "goto"
      url = each.value.product_url
    },
    {
      op       = "click"
      selector = "button[name=add-to-cart]"
    },
    {
      op  = "goto"
      url = each.value.checkout_url
    },
    {
      op   = "assert_text"
      text = "Payment"
    },
  ]
}

resource "pathly_sla_target" "home" {
  for_each = var.clients

  scenario_id          = pathly_scenario.home[each.key].id
  name                 = "${each.value.name} - ${each.value.objective_pct} %"
  objective_pct        = each.value.objective_pct
  window_days          = 30
  exclude_maintenance  = true
  warn_at_budget_ratio = 0.8
}

Alerts: one shared webhook, one tag and one folder per client

A single pathly_webhook receives the failures (run.failed) and the recoveries (run.recovered) of every client. Each delivery is HMAC-signed with a secret returned once only, at creation. Verify that signature on the receiving side, otherwise anyone can forge a fake alert.

A delivery names the scenario by its identifier (monitorId) and by its name (monitorName), not by its tags. To find the client, your receiver compares the identifier with the scenario_ids output, or reads the start of the name: every name begins with the client name.

The client:<key> tag and the folder are there for sorting. The folder keeps the scenarios of one client together in the console. The tag filters the inventory, for example with filter_tag on the pathly_scenarios data source.

Outbound webhooks are not part of every plan. On a plan that does not include them, the API refuses the creation of the webhook.

variable "webhook_url" {
  type = string
}

resource "pathly_webhook" "alerts" {
  url    = var.webhook_url
  events = ["run.failed", "run.recovered"]
}

output "scenario_ids" {
  value = {
    for key, client in var.clients : key => {
      home     = pathly_scenario.home[key].id
      checkout = pathly_scenario.checkout[key].id
    }
  }
}

data "pathly_scenarios" "bakery" {
  filter_tag = "client:bakery"
}

Secrets: key, sensitive variables and state

The API key lives in the PATHLY_API_TOKEN environment variable, supplied by your secret store or by the CI secrets. The provider block stays empty. A key written in a .tf file ends up in the repository, and a key read from a Terraform variable ends up in plaintext in the state.

Give the key the scopes it needs and nothing more: scenarios:write, alerting:write and sla:write to apply. Add maintenance:write only if you manage maintenance windows in the same code. Writing implies reading. For a job that only runs terraform plan, a second key with the :read scopes is enough.

If the funnel of a client requires a sign-in, the test account goes through a variable marked sensitive, kept apart from the clients map: Terraform does not let a sensitive map drive for_each. Create that account for monitoring, with no saved payment method. Never use the account of a real shopper.

The Terraform state holds the webhook secret, because the API only returns it at creation. It also holds the test passwords typed by the Flow. Keep it in a remote backend, encrypted at rest, with restricted access. Do not commit it, and do not publish the secret in a non-sensitive output.

# terraform apply   scenarios:write  alerting:write  sla:write
#                   maintenance:write   (maintenance windows only)
# terraform plan    scenarios:read   alerting:read   sla:read

variable "test_logins" {
  type = map(object({
    username = string
    password = string
  }))
  sensitive = true
  default   = {}
}

  steps = [
    {
      op  = "goto"
      url = "https://shop.example.com/login"
    },
    {
      op       = "fill"
      selector = "input[name=email]"
      value    = var.test_logins[each.key].username
    },
    {
      op       = "fill"
      selector = "input[name=password]"
      value    = var.test_logins[each.key].password
    },
    {
      op       = "click"
      selector = "button[type=submit]"
    },
    # ...goto product page, add to cart, goto checkout, assert_text
  ]

Drift: what exists outside the code

A colleague in a hurry creates a scenario by hand in the console. It escapes the review, and nobody knows it exists. The pathly_scenarios data source reads back the whole inventory of the organisation. The scenarios_outside_terraform output lists what Pathly holds and the code ignores.

An empty list means the code describes everything. A scenario that shows up there is handled in one of two ways: you import it into the code, or you delete it. If other Terraform repositories manage part of the organisation, narrow the inventory with filter_tag or filter_folder.

Schedule a terraform plan every night in CI, with the -detailed-exitcode option. Exit code 0 says nothing moved. Exit code 2 says the plan is not empty: an attribute described in the code changed in the console, or the list of scenarios outside the code changed. The job fails and tells you.

This nightly job only reads: give it a key with the :read scopes. It also needs access to the state backend and the same variables as the job that applies the code.

Two limits to keep in mind. An attribute absent from the code keeps the value set in the console, without showing in the plan. The steps of a Flow cannot be read back through the API. So write in the code every attribute you want to hold.

data "pathly_scenarios" "all" {
  depends_on = [pathly_scenario.home, pathly_scenario.checkout]
}

output "scenarios_outside_terraform" {
  value = [
    for s in data.pathly_scenarios.all.scenarios : {
      id     = s.id
      name   = s.name
      folder = s.folder
    }
    if !contains(
      concat(
        [for home in pathly_scenario.home : home.id],
        [for checkout in pathly_scenario.checkout : checkout.id],
      ),
      s.id,
    )
  ]
}

name: pathly-drift
on:
  schedule:
    - cron: "0 3 * * *"

jobs:
  plan:
    runs-on: ubuntu-latest
    env:
      PATHLY_API_TOKEN: ${{ secrets.PATHLY_API_TOKEN_READ }}
      TF_VAR_webhook_url: ${{ secrets.PATHLY_WEBHOOK_URL }}
      TF_VAR_test_logins: ${{ secrets.PATHLY_TEST_LOGINS }}
    steps:
      - uses: actions/checkout@v4
      - uses: hashicorp/setup-terraform@v3
        with:
          terraform_wrapper: false
      - run: terraform init -input=false
      - run: terraform plan -input=false -lock=false -detailed-exitcode

Add, change, remove or import a client

Add a client: one more entry in the map, a pull request, an apply. The plan shows three resources to create and touches nothing else. That is the check to make in review.

Change a client: edit its entry. A new URL, severity or objective is updated in place. Do not rename the key lightly: it is part of the resource addresses. Terraform would destroy the scenarios and create them again, without their run history. If you have to rename it, declare one moved block per resource.

Remove a client: delete its entry. The plan shows three resources to destroy. Destroyed scenarios take their run history with them: export first what the client has to keep.

Import a site that was created by hand: first add its entry to the map, then import the existing scenario at the address of that entry, with terraform import pathly_scenario.<name> <scenario UUID>. The identifier is a UUID, readable in the scenario URL in the console. Without the import, Terraform would create a second scenario next to the first.

After the import of a Flow, the next apply writes the steps of the code over the recorded ones, since the API does not return them. An availability objective already set in the console for that scenario is adopted by the first apply, without a conflict error.

clients = {
  # ...

  cycles = {
    name         = "Example Cycles"
    home_url     = "https://store.example.org/"
    product_url  = "https://store.example.org/bikes/city"
    checkout_url = "https://store.example.org/checkout"
  }
}

terraform import 'pathly_scenario.home["cycles"]' 00000000-0000-4000-8000-000000000000
terraform import 'pathly_scenario.checkout["cycles"]' 11111111-1111-4111-8111-111111111111
terraform plan

import {
  to = pathly_scenario.home["cycles"]
  id = "00000000-0000-4000-8000-000000000000"
}