# Terraform Module Design That Scales

> State boundaries, composition, and the CI pipeline that keeps applies off laptops

Canonical: https://everettkildare.com/blog/terraform-module-design-that-scales
Author: Everett Kildare
Published: 2026-06-20
Updated: 2026-11-01

Terraform makes it easy to create infrastructure and surprisingly hard to keep a large codebase maintainable. The problems show up around the eighteen-month mark: modules nobody dares change, state files that take twenty minutes to plan, and a `terraform apply` that everyone runs from their laptop while holding their breath.

These are the structural decisions that prevent that.

## State boundaries decide everything else

Before modules, before naming conventions, decide how state is split. This is the hardest thing to change later.

A single state file for an entire environment is the default and it fails in three ways: plan time grows linearly with resource count, the blast radius of a mistake is the whole environment, and only one person can apply at a time.

Split state along axes that change at different rates:

```text
├── foundation/          # accounts, org policy, DNS zones — changes yearly
│   └── prod/
├── platform/            # VPCs, clusters, shared databases — changes monthly
│   ├── prod/
│   └── staging/
└── services/            # application infrastructure — changes daily
    ├── prod/
    │   ├── api/
    │   └── worker/
    └── staging/
```

The rule: **things that change together live together; things that change at different rates live apart.** A deploy to the API service should not require a plan that evaluates your DNS zones.

Cross-state references go through data sources or a registry, not `terraform_remote_state`:

```hcl
# Avoid — couples the consumer to the producer's internal state layout
data "terraform_remote_state" "platform" {
  backend = "s3"
  config  = { bucket = "tf-state", key = "platform/prod/terraform.tfstate" }
}

# Prefer — a published contract the producer maintains deliberately
data "aws_ssm_parameter" "vpc_id" {
  name = "/platform/prod/vpc/id"
}
```

Remote state reads mean any refactor of the producer's internals can break every consumer. A parameter store entry is an explicit, versionable interface.

## Module design

A good module has a narrow interface and does one thing. The test: can you describe what it does in one sentence without using "and"?

### Structure

```text
modules/postgres-instance/
├── README.md
├── main.tf
├── variables.tf
├── outputs.tf
├── versions.tf
└── examples/
    ├── minimal/
    └── production/
```

`versions.tf` in a reusable module declares minimum versions only. Never an exact pin, and not `~>` either — an upper bound in a shared module makes it uncomposable when two modules disagree. The tight constraint belongs in the root module, as `~>`, with the committed `.terraform.lock.hcl` recording the exact versions chosen:

```hcl
terraform {
  required_version = ">= 1.9"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 5.60"
    }
  }
}
```

### Variables with real validation

Validation catches mistakes at plan time rather than at apply time, twenty minutes in:

```hcl
variable "instance_class" {
  description = "RDS instance class. Must be a memory-optimised class in production."
  type        = string

  validation {
    condition     = can(regex("^db\\.(t4g|m7g|r7g)\\.", var.instance_class))
    error_message = "instance_class must be a Graviton class: db.t4g.*, db.m7g.*, or db.r7g.*."
  }
}

variable "backup_retention_days" {
  description = "Days to retain automated backups."
  type        = number
  default     = 14

  validation {
    condition     = var.backup_retention_days >= 7 && var.backup_retention_days <= 35
    error_message = "backup_retention_days must be between 7 and 35."
  }
}

variable "environment" {
  type = string

  validation {
    condition     = contains(["dev", "staging", "prod"], var.environment)
    error_message = "environment must be one of: dev, staging, prod."
  }
}
```

Use an object type when a variable is really a bundle of related settings — it documents the shape and gives you type checking:

```hcl
variable "maintenance" {
  description = "Maintenance and backup windows, in UTC."

  type = object({
    window        = string
    backup_window = string
    auto_upgrade  = optional(bool, true)
  })

  default = {
    window        = "sun:03:00-sun:04:00"
    backup_window = "01:00-02:00"
  }
}
```

`optional()` with a default is the feature that makes object variables practical — consumers set what they care about.

> [!TIP]
> Every variable gets a `description`. `terraform-docs` generates your README from them, so a described variable documents itself permanently and an undescribed one never gets documented at all.

### Outputs are a public API

Once someone depends on an output, removing it is a breaking change. Export the identifiers consumers genuinely need, not everything you happen to have:

```hcl
output "endpoint" {
  description = "Connection endpoint for the primary instance."
  value       = aws_db_instance.this.address
}

output "security_group_id" {
  description = "Security group attached to the instance; attach client rules here."
  value       = aws_security_group.this.id
}

# Sensitive outputs are marked, never logged.
output "master_password_secret_arn" {
  description = "ARN of the Secrets Manager secret holding the master password."
  value       = aws_secretsmanager_secret.master.arn
  sensitive   = true
}
```

Never output a secret's value. Output the reference to it — state files are plain text, and every consumer's state inherits the secret.

## Composition over configuration

The strongest temptation in Terraform is the mega-module: one module with fifty variables that provisions an entire environment. It is easy to call and impossible to change.

```hcl
# The trap — a module that grows a flag for every requirement
module "environment" {
  source = "./modules/environment"

  enable_bastion            = true
  enable_nat_per_az         = false
  enable_flow_logs          = true
  database_multi_az         = true
  # ... 40 more booleans
}
```

Each flag multiplies the untested combinations. Compose small modules instead:

```hcl
module "network" {
  source = "../../modules/vpc"

  name               = "prod"
  cidr               = "10.40.0.0/16"
  availability_zones = ["eu-west-1a", "eu-west-1b", "eu-west-1c"]
  nat_gateways       = 3
}

module "database" {
  source = "../../modules/postgres-instance"

  identifier         = "prod-primary"
  subnet_ids         = module.network.private_subnet_ids
  vpc_id             = module.network.vpc_id
  instance_class     = "db.r7g.2xlarge"
  environment        = "prod"
}
```

The root module is where environment-specific composition belongs. It should be readable as a description of the environment.

## Handling drift and imports

Real environments accumulate resources created outside Terraform. Since 1.5, `import` blocks make adoption reviewable — the import is planned, not performed imperatively:

```hcl
import {
  to = aws_db_instance.legacy_reporting
  id = "prod-reporting-01"
}

resource "aws_db_instance" "legacy_reporting" {
  # Run `terraform plan -generate-config-out=generated.tf` to scaffold this.
}
```

Detect drift on a schedule rather than discovering it during an incident:

```bash
terraform plan -detailed-exitcode -lock=false
# 0 = no changes, 1 = error, 2 = drift detected
```

Wire exit code 2 into an alert. Drift that nobody notices becomes drift that blocks an urgent change.

## CI/CD

Applies belong in a pipeline, not on laptops. The minimum viable pipeline:

```yaml
name: terraform

on:
  pull_request:
    paths: ['infrastructure/**']

permissions:
  id-token: write      # OIDC — no long-lived cloud credentials
  contents: read
  pull-requests: write

jobs:
  plan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::111122223333:role/terraform-plan
          aws-region: eu-west-1

      - uses: hashicorp/setup-terraform@v3
        with:
          terraform_version: 1.9.8

      - run: terraform init -backend-config=envs/prod.hcl
      - run: terraform fmt -check -recursive
      - run: terraform validate

      - run: tflint --recursive
      - run: trivy config --severity HIGH,CRITICAL .

      - run: terraform plan -out=tfplan -lock-timeout=5m
      - run: terraform show -no-color tfplan > plan.txt

      - uses: actions/github-script@v7
        with:
          script: |
            const plan = require('fs').readFileSync('plan.txt', 'utf8');
            await github.rest.issues.createComment({
              issue_number: context.issue.number,
              owner: context.repo.owner,
              repo: context.repo.repo,
              body: '```\n' + plan.slice(0, 60000) + '\n```'
            });
```

Two details matter. **OIDC federation** removes long-lived cloud credentials from CI entirely. And the plan is **saved and applied as an artefact** — applying a freshly-computed plan after approval means applying something nobody reviewed.

> [!WARNING]
> Plan output can contain sensitive values. Posting it to a pull request on a public repository leaks them. Use `sensitive = true` diligently, and consider posting only the resource change summary rather than the full diff.

## Testing

`terraform test` gives you assertions without deploying, using plan-time evaluation:

```hcl
# tests/defaults.tftest.hcl
variables {
  identifier     = "test-db"
  subnet_ids     = ["subnet-aaa", "subnet-bbb"]
  vpc_id         = "vpc-123"
  environment    = "dev"
  instance_class = "db.t4g.medium"
}

run "encryption_is_enabled_by_default" {
  command = plan

  assert {
    condition     = aws_db_instance.this.storage_encrypted == true
    error_message = "Storage encryption must default to enabled."
  }
}

run "rejects_non_graviton_classes" {
  command = plan

  variables {
    instance_class = "db.m5.large"
  }

  expect_failures = [var.instance_class]
}
```

Test the defaults and the validation rules. Those are where regressions hide — someone changes a default in a refactor and every new database ships unencrypted.

## The short version

- Split state by rate of change, not by resource type.
- Publish cross-stack values through a parameter store, never `terraform_remote_state`.
- Small modules, composed in the root, beat one configurable module.
- Validate variables; the error at plan time is worth ten minutes of writing.
- Never output secret values, only references.
- Apply from CI with OIDC, applying the reviewed plan artefact.
- Alert on drift before it blocks you.

## Sources

- [Terraform 1.5 brings config-driven import and checks](https://www.hashicorp.com/en/blog/terraform-1-5-brings-config-driven-import-and-checks) — `import` blocks arrived in 1.5 as a plannable operation, with `-generate-config-out`.
- [terraform plan command reference](https://developer.hashicorp.com/terraform/cli/commands/plan) — the 0 / 1 / 2 meanings of `-detailed-exitcode`.
- [Tests - Configuration Language](https://developer.hashicorp.com/terraform/language/tests) — `terraform test` (v1.6+), `command = plan`, `assert`, and `expect_failures` on input variables.
- [Type Constraints - Configuration Language](https://developer.hashicorp.com/terraform/language/expressions/type-constraints) — `optional()` object attributes with default values.
- [Version Constraints - Configuration Language](https://developer.hashicorp.com/terraform/language/expressions/version-constraints) — the `~>` operator, and HashiCorp's advice that reusable modules constrain minimum versions only while root modules use `~>`.
- [Dependency Lock File (.terraform.lock.hcl) - Configuration Language](https://developer.hashicorp.com/terraform/language/files/dependency-lock) — the lock file belongs to the configuration as a whole and should be committed to version control.
- [Backup retention period](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_WorkingWithAutomatedBackups.BackupRetention.html) — RDS automated backups can be kept for at most 35 days.
- [Configuring OpenID Connect in Amazon Web Services](https://docs.github.com/en/actions/how-tos/secure-your-work/security-harden-deployments/oidc-in-aws) — OIDC federation from GitHub Actions in place of long-lived cloud credentials.