New Relic Terraform Provider Setup, Alerts and Dashboards

Set up the New Relic Terraform provider v3, then manage NRQL alert conditions, notification workflows, entity tags, dashboards and workloads as code.

  • Go to the profile of Safoor Safdar
·12 min read

Alert policies and dashboards built by hand in the New Relic UI drift apart between teams and environments. You can’t review a change or rebuild a policy after a bad edit. This guide puts the whole setup in Terraform, using the resources the current provider supports.

Key Takeaways

  • Pin the provider with version = "~> 3.0" and pass account_id, api_key and region through variables or NEW_RELIC_* environment variables.
  • Use newrelic_nrql_alert_condition for alerts. The provider marks newrelic_alert_condition and newrelic_infra_alert_condition as deprecated.
  • Route alerts with a notification destination, a notification channel and a newrelic_workflow. These replace newrelic_alert_channel.
  • Tag entities with newrelic_entity_tags, then group them into a workload with a tag query.

Changes since the 2022 version of this guide

The New Relic Terraform provider is the newrelic/newrelic plugin that lets Terraform create and manage New Relic alerts, notifications, dashboards and tags. The provider is on the 3.x line, with v3.102.0 released in November 2026. Three resources from the original walkthrough are now deprecated:

Old resourceStatusUse instead
newrelic_alert_conditionDeprecatednewrelic_nrql_alert_condition
newrelic_infra_alert_conditionDeprecatednewrelic_nrql_alert_condition
newrelic_alert_channel and newrelic_alert_policy_channelDeprecatednewrelic_notification_destination, newrelic_notification_channel and newrelic_workflow

The provider docs say each deprecated resource will go away in a future major release. The examples below use the replacements.

Before you begin

You need a New Relic account with at least one APM or Browser application reporting data. You also need the Terraform CLI and a New Relic user API key. If Terraform is new to you, read my introduction to IaC and Terraform first.

Bootstrap Terraform with required_providers

Create a project folder and a main.tf file. The required_providers block tells Terraform where to download the provider and which versions to accept.

mkdir newrelic-terraform && cd newrelic-terraform
touch main.tf
terraform {
  required_version = "~> 1.0"

  required_providers {
    newrelic = {
      source  = "newrelic/newrelic"
      version = "~> 3.0"
    }
  }
}

The ~> 3.0 constraint accepts any 3.x release and blocks a jump to 4.0. A new major version can remove the deprecated resources, so you choose when to upgrade. HashiCorp documents the operator in its version constraints guide.

Configure the New Relic provider

The provider needs three settings: account_id, api_key and region. Keep the key out of source control by passing it in as a sensitive variable.

variable "newrelic_account_id" {
  type = number
}

variable "newrelic_api_key" {
  type      = string
  sensitive = true
}

provider "newrelic" {
  account_id = var.newrelic_account_id
  api_key    = var.newrelic_api_key # User API key (NRAK-...)
  region     = "US"                 # US, EU or JP
}
  • account_id is your New Relic account ID.
  • api_key is a user API key. The provider docs show these keys with an NRAK prefix.
  • region accepts US, EU or JP, and defaults to US. Pick EU if you log in at one.eu.newrelic.com.

Set the variables in your shell with export TF_VAR_newrelic_api_key=NRAK-.... The provider also reads its own environment variables if you leave an argument out of the block:

Provider argumentEnvironment variable
account_idNEW_RELIC_ACCOUNT_ID
api_keyNEW_RELIC_API_KEY
regionNEW_RELIC_REGION

Run terraform init to download the provider.

Reference your application with newrelic_entity

The newrelic_entity data source looks up an existing application by name. Later resources use its guid and name.

data "newrelic_entity" "app_apm" {
  name   = "checkout-service" # exact name of your APM application
  domain = "APM"
  type   = "APPLICATION"
}

data "newrelic_entity" "app_browser" {
  name   = "checkout-web" # exact name of your Browser application
  domain = "BROWSER"
  type   = "APPLICATION"
}

Valid domain values are APM, BROWSER, INFRA, MOBILE, SYNTH and EXT. Set ignore_case = true if the capitalization of the name in New Relic doesn’t match yours.

Tag entities with newrelic_entity_tags

newrelic_entity_tags adds key and value tags to any entity by GUID. Tags let you filter in the UI and build workloads from a query. Each tag block takes a key and a list of values.

locals {
  tagged_entities = {
    apm     = data.newrelic_entity.app_apm.guid
    browser = data.newrelic_entity.app_browser.guid
  }
}

resource "newrelic_entity_tags" "app" {
  for_each = local.tagged_entities
  guid     = each.value

  tag {
    key    = "environment"
    values = ["production"]
  }

  tag {
    key    = "team"
    values = ["checkout"]
  }
}

The for_each map applies one tag set to both applications. The entity tags docs warn against using reserved keys that New Relic sets itself, such as accountId. To bring tags you created in the UI under Terraform, import them by GUID with terraform import 'newrelic_entity_tags.app["apm"]' <guid>.

Create a newrelic_alert_policy

An alert policy groups conditions and controls how New Relic rolls incidents into issues. The incident_preference argument sets that rollup.

resource "newrelic_alert_policy" "golden_signals" {
  name                = "Golden signals - checkout"
  incident_preference = "PER_CONDITION_AND_TARGET"
}

PER_CONDITION_AND_TARGET opens one issue per condition and signal, so a CPU spike on two hosts produces two issues. The default is PER_POLICY, which folds all incidents into one issue. Run terraform apply and type yes to create the policy.

Add NRQL alert conditions for the golden signals

Each condition runs a NRQL query on a schedule and compares the result with a threshold. The four conditions below cover traffic, latency, errors and saturation for the checkout application.

# Traffic, fewer than 5 transactions per minute for 5 minutes
resource "newrelic_nrql_alert_condition" "low_throughput" {
  policy_id                    = newrelic_alert_policy.golden_signals.id
  name                         = "Low throughput"
  type                         = "static"
  runbook_url                  = "https://example.com/runbooks/low-throughput"
  violation_time_limit_seconds = 86400
  aggregation_window           = 60
  aggregation_method           = "cadence"
  aggregation_delay            = 120
  fill_option                  = "static"
  fill_value                   = 0

  nrql {
    query = "SELECT count(*) FROM Transaction WHERE appName = '${data.newrelic_entity.app_apm.name}'"
  }

  critical {
    operator              = "below"
    threshold             = 5
    threshold_duration    = 300
    threshold_occurrences = "all"
  }
}

# Latency, 95th percentile response time
resource "newrelic_nrql_alert_condition" "high_latency" {
  policy_id                    = newrelic_alert_policy.golden_signals.id
  name                         = "High p95 response time"
  type                         = "static"
  runbook_url                  = "https://example.com/runbooks/high-latency"
  violation_time_limit_seconds = 86400
  aggregation_window           = 60
  aggregation_method           = "event_flow"
  aggregation_delay            = 120

  nrql {
    query = "SELECT percentile(duration, 95) FROM Transaction WHERE appName = '${data.newrelic_entity.app_apm.name}'"
  }

  critical {
    operator              = "above"
    threshold             = 2
    threshold_duration    = 300
    threshold_occurrences = "all"
  }

  warning {
    operator              = "above"
    threshold             = 1
    threshold_duration    = 300
    threshold_occurrences = "all"
  }
}

# Errors, percentage of failed transactions
resource "newrelic_nrql_alert_condition" "error_percentage" {
  policy_id                    = newrelic_alert_policy.golden_signals.id
  name                         = "High error percentage"
  type                         = "static"
  runbook_url                  = "https://example.com/runbooks/error-rate"
  violation_time_limit_seconds = 86400
  aggregation_window           = 60
  aggregation_method           = "event_flow"
  aggregation_delay            = 120

  nrql {
    query = "SELECT percentage(count(*), WHERE error IS true) FROM Transaction WHERE appName = '${data.newrelic_entity.app_apm.name}'"
  }

  critical {
    operator              = "above"
    threshold             = 5
    threshold_duration    = 300
    threshold_occurrences = "all"
  }

  warning {
    operator              = "above"
    threshold             = 2
    threshold_duration    = 300
    threshold_occurrences = "all"
  }
}

# Saturation, CPU per host, one signal per hostname
resource "newrelic_nrql_alert_condition" "high_cpu" {
  policy_id                    = newrelic_alert_policy.golden_signals.id
  name                         = "High CPU utilization"
  type                         = "static"
  runbook_url                  = "https://example.com/runbooks/high-cpu"
  violation_time_limit_seconds = 86400
  aggregation_window           = 60
  aggregation_method           = "event_flow"
  aggregation_delay            = 120

  nrql {
    query = "SELECT average(cpuPercent) FROM SystemSample WHERE hostname LIKE 'checkout-%' FACET hostname"
  }

  critical {
    operator              = "above"
    threshold             = 90
    threshold_duration    = 300
    threshold_occurrences = "all"
  }
}

A few rules from the newrelic_nrql_alert_condition docs shape these values:

  • threshold_duration must be a multiple of aggregation_window, and between 60 and 86,400 seconds for static conditions.
  • violation_time_limit_seconds force-closes long incidents and accepts 300 to 2,592,000 seconds.
  • The older term, duration and time_function arguments are deprecated. Use critical, warning, threshold_duration and threshold_occurrences.

The throughput condition uses cadence aggregation with a fill value of 0. An app that stops receiving traffic still produces a data point, so the “below 5” threshold can fire. The CPU query uses FACET hostname, and New Relic’s NRQL condition guide treats each facet as a separate signal. Leave SINCE and UNTIL out of alert queries, because the same guide says conditions don’t support them.

Run each query in the New Relic query builder before you pick a threshold. The latency numbers above are placeholders for your own service level.

Route alerts with destinations, channels and workflows

New Relic splits notifications into three resources. A destination holds the address or credentials, a channel holds the message settings, and a workflow decides which issues go to which channel.

Start with an email destination and a channel that points at it:

resource "newrelic_notification_destination" "team_email" {
  name = "Checkout team email"
  type = "EMAIL"

  property {
    key   = "email"
    value = "oncall@example.com,checkout-team@example.com"
  }
}

resource "newrelic_notification_channel" "team_email" {
  name           = "Golden signals email"
  type           = "EMAIL"
  destination_id = newrelic_notification_destination.team_email.id
  product        = "IINT"

  property {
    key   = "subject"
    value = "New Relic alert for checkout"
  }
}

Set product = "IINT" on any channel you attach to a workflow. The workflow docs require it.

Slack needs a different setup. New Relic connects to Slack through an OAuth flow in the browser, so Terraform can’t create a Slack destination. Connect Slack once in the New Relic UI, then look up the destination with a data source:

data "newrelic_notification_destination" "slack" {
  exact_name = "Acme Slack" # name of the Slack destination in New Relic
}

resource "newrelic_notification_channel" "team_slack" {
  name           = "Golden signals Slack"
  type           = "SLACK"
  destination_id = data.newrelic_notification_destination.slack.id
  product        = "IINT"

  property {
    key   = "channelId"
    value = "C0123456789" # Slack channel ID
  }
}

The 2022 version of this post had no way to send warnings and critical alerts to different places. Workflows fix that with an issue filter. New Relic’s workflow docs say to filter on priority HIGH to catch warning events. The pair below sends warnings to Slack and everything else from the policy to email:

resource "newrelic_workflow" "golden_signals_email" {
  name                  = "Golden signals critical to email"
  muting_rules_handling = "DONT_NOTIFY_FULLY_MUTED_ISSUES"

  issues_filter {
    name = "golden-signals-critical"
    type = "FILTER"

    predicate {
      attribute = "labels.policyIds"
      operator  = "EXACTLY_MATCHES"
      values    = [newrelic_alert_policy.golden_signals.id]
    }

    predicate {
      attribute = "priority"
      operator  = "DOES_NOT_EQUAL"
      values    = ["HIGH"]
    }
  }

  destination {
    channel_id = newrelic_notification_channel.team_email.id
  }
}

resource "newrelic_workflow" "golden_signals_slack" {
  name                  = "Golden signals warnings to Slack"
  muting_rules_handling = "DONT_NOTIFY_FULLY_MUTED_ISSUES"

  issues_filter {
    name = "golden-signals-warning"
    type = "FILTER"

    predicate {
      attribute = "labels.policyIds"
      operator  = "EXACTLY_MATCHES"
      values    = [newrelic_alert_policy.golden_signals.id]
    }

    predicate {
      attribute = "priority"
      operator  = "EQUAL"
      values    = ["HIGH"]
    }
  }

  destination {
    channel_id            = newrelic_notification_channel.team_slack.id
    notification_triggers = ["ACTIVATED", "CLOSED"]
  }
}

An issue must match every predicate in the filter. Each channel can belong to one workflow, so give each workflow its own channel.

Build a dashboard with newrelic_one_dashboard

newrelic_one_dashboard defines pages and widgets in HCL. Each widget takes a title, a row and a column, and the docs recommend listing widgets in row and column order to avoid drift.

resource "newrelic_one_dashboard" "website_performance" {
  name        = "Website performance"
  permissions = "public_read_only"

  page {
    name = "Sessions"

    widget_line {
      title  = "Unique user sessions"
      row    = 1
      column = 1
      width  = 8
      height = 3

      nrql_query {
        query = "FROM PageView SELECT uniqueCount(session) WHERE appName = '${data.newrelic_entity.app_browser.name}' TIMESERIES"
      }
    }

    widget_markdown {
      title  = "Dashboard notes"
      row    = 1
      column = 9
      width  = 4
      height = 3

      text = "Owned by the checkout team. Alerts route through the golden signals workflows."
    }
  }
}

resource "newrelic_entity_tags" "website_performance" {
  guid = newrelic_one_dashboard.website_performance.guid

  tag {
    key    = "environment"
    values = ["production"]
  }
}

Widgets use a 12-column grid, so a width of 8 plus a width of 4 fills one row. The dashboard exports a guid, and the second resource tags it like any other entity.

Group tagged entities in a workload

A workload collects related entities into one view with a shared health status. The entity_search_query picks up anything tagged environment = production, including entities you add later.

resource "newrelic_workload" "production" {
  name       = "Checkout production"
  account_id = var.newrelic_account_id

  entity_search_query {
    query = "tags.accountId = '${var.newrelic_account_id}' AND tags.environment = 'production'"
  }

  scope_account_ids = [var.newrelic_account_id]
}

The workload docs note that this resource needs a user API key, which the provider block above already uses.

My approach to monitoring as code

I keep alert and dashboard code in the same repository and pipeline as the infrastructure it watches. A pull request then shows the new service and its alerts together, and a reviewer can catch a missing condition. I worked the same way on platforms where infrastructure and application deployment lived end to end in Terraform. You can read more about my background in cloud reliability.

Before you write any thresholds, decide what users need from the service. My post on starting with SLOs covers that step. The provider documentation documents each resource and argument when you need more than this guide covers.

Frequently asked questions

What is the New Relic Terraform provider?

It’s the newrelic/newrelic plugin for Terraform. It manages New Relic alert policies, NRQL conditions, notification workflows, dashboards, tags and workloads as code. Configure it with account_id, api_key and region.

How do I create a newrelic_alert_policy in Terraform?

Declare a newrelic_alert_policy resource with a name and an optional incident_preference. Attach conditions by passing the policy’s id as policy_id on each newrelic_nrql_alert_condition. Route its alerts with a newrelic_workflow that filters on labels.policyIds.

What does newrelic_entity_tags do?

It adds key and value tags to a New Relic entity, identified by its GUID. You get the GUID from the newrelic_entity data source or from a resource such as a dashboard. Each tag block takes a key and a list of values.

Is newrelic_alert_channel deprecated?

Yes. The provider docs mark newrelic_alert_channel and newrelic_alert_policy_channel as deprecated and due for removal in a future major release. Use newrelic_notification_destination, newrelic_notification_channel and newrelic_workflow instead.