AWS with Terraform (Day 26)

HCP Terraform Explained: Projects, Workspaces, and a Hands-On Demo

Running Terraform from a laptop works well when you’re learning or managing a small setup. But as soon as you move into team environments or production systems, problems appear quickly—local state files, shared credentials, manual runs, and limited visibility.

On Day 26 of my DevOps journey, I focused on HashiCorp Cloud Platform Terraform (Terraform Cloud) and how it solves these problems by adding centralized state, secure workflows, collaboration, and governance—without changing how Terraform itself works.

This post explains what Terraform Cloud is, how its hierarchy works, and how to use it in real projects.

What Is HashiCorp Cloud Platform Terraform

Terraform Cloud is a managed, web-based service for running Terraform. It acts as a centralized orchestration layer on top of Terraform CLI and code.

Terraform stays Terraform—you still write HCL, run plans, and apply changes. What Terraform Cloud adds is:

  • Centralized state management

  • Secure handling of secrets

  • UI-based run history and audit logs

  • Integration with Git repositories

  • Multiple run workflows

  • A private module registry

Think of Terraform Cloud as Terraform for teams and production.

Why Move Beyond CLI-Only Terraform

Local Terraform works, but it does not scale well. Terraform Cloud addresses the most common pain points.

Centralized State

No more local terraform.tfstate files or shared S3 buckets with custom locking. State is stored, locked, and versioned per workspace automatically.

Secure Secrets

Provider credentials and sensitive values are stored as encrypted workspace environment variables, not in .tfvars or source control.

VCS Integration

Terraform Cloud integrates with GitHub, GitLab, Bitbucket, and Azure DevOps. Commits can automatically trigger plans and applies.

Multiple Workflows

Support for:

  • VCS-driven workflows

  • CLI-driven workflows

  • API-driven workflows

Private Module Registry

Share Terraform modules across the organization without publishing them publicly.

Audit and Visibility

Every plan, apply, diff, and log is visible in the UI—critical for compliance and debugging.

Core Hierarchy: Organization, Project, Workspace

Terraform Cloud uses a simple but powerful hierarchy.

Organization

The top-level boundary. Usually maps to a company or team. All projects, workspaces, and modules live here.

Project

A logical grouping inside an organization. Projects are commonly used to separate:

  • Applications

  • Business units

  • Cloud providers

  • Teams

Example: payments, platform, observability.

Workspace

A workspace represents one Terraform state and one root configuration.

Common patterns:

  • One workspace per environment: dev, test, prod

  • One workspace per application

  • One workspace per Terraform root module

Each workspace contains:

  • Terraform configuration

  • Variables and secrets

  • State file

  • Run history

Terraform Cloud Workflows

Terraform Cloud supports three main workflows. The choice depends on how you want changes to flow into Terraform.

VCS-Driven Workflow

This is the most common and recommended setup.

  • Connect a Git repository

  • Configure the working directory

  • Commits trigger plans automatically

  • Applies can be manual or automatic

Best suited for GitOps-style workflows and production environments.

CLI-Driven Workflow

This workflow lets you keep using your local terminal while Terraform Cloud manages state and runs.

Typical flow:

  1. Authenticate with terraform login

  2. Map your local configuration to a workspace

  3. Run terraform plan and terraform apply locally

  4. Plans and applies run remotely and appear in the UI

This is useful when migrating from local Terraform or when developers prefer CLI-based workflows.

API-Driven Workflow

Runs are triggered programmatically using Terraform Cloud APIs.

Best for:

  • CI/CD pipelines

  • Custom automation platforms

  • Advanced orchestration use cases

Variables, Secrets, and State Management

Terraform Cloud simplifies configuration management.

Terraform Variables

Used for non-sensitive values such as region or instance size. Defined in the workspace UI or via VCS.

Environment Variables

Used for provider credentials such as:

  • AWS_ACCESS_KEY_ID

  • AWS_SECRET_ACCESS_KEY

Mark them as sensitive so values are encrypted and hidden from logs.

Variable Sets

Apply the same variables to multiple workspaces. Ideal for shared credentials or common configuration.

Remote State

Terraform Cloud manages state automatically. When using Terraform Cloud, you do not need an S3 backend.

VCS-Driven Workspace Setup Steps

A typical production setup looks like this:

  1. Create an organization

  2. Create a project

  3. Create a workspace and select VCS-driven workflow

  4. Connect the repository

  5. Set the working directory

  6. Configure auto-apply:

    • Enabled for non-production

    • Disabled for production

  7. Define trigger rules to limit runs to relevant paths

CLI-Driven Workspace Setup

To use Terraform Cloud with the CLI:

  1. Run terraform login and authenticate

  2. Add the cloud configuration block:

    terraform {
  cloud {
    organization = "your-organization"
    workspaces {
      name = "your-workspace-name"
    }
  }
}

  3. Remove any existing backend blocks

  4. Run terraform init

  5. Use terraform plan and terraform apply as usual

Terraform Cloud now manages state and runs remotely.

Common Gotchas and Best Practices

  • Always set provider credentials as workspace environment variables

  • Never mix backend blocks with the cloud block

  • Align Terraform CLI version with the workspace version

  • Avoid auto-apply in production

  • Use variable sets to reduce duplication

  • Review plan output in the UI before applying

Troubleshooting Scenarios

  • No valid credential source found
    Add provider credentials as sensitive environment variables.

  • Conflicting backend and cloud blocks
    Remove the backend block when using Terraform Cloud.

  • Incompatible Terraform version
    Upgrade local Terraform or adjust workspace version settings carefully.

Diagram




Conclusion

Terraform Cloud is not about replacing Terraform—it’s about scaling Terraform safely.

By adding centralized state, secure secrets, auditability, and flexible workflows, Terraform Cloud makes Infrastructure as Code suitable for team collaboration and production governance.

Choosing the right workflow, structuring projects and workspaces cleanly, and enforcing approval gates for production environments makes Terraform both powerful and safe at scale.

Continuing to build real-world DevOps foundations

Here is the session link:



Comments

Post a Comment

Popular posts from this blog

AWS with Terraform (Day 01)

Image
  Terraform Fundamentals – Day 01 (My Learning Notes as a DevOps Engineer) Today I wrapped up Day-01 of my Terraform with AWS series, and even with my DevOps background, this session helped me tighten my fundamentals and rethink how I approach cloud provisioning. Terraform is something I’ve used before, but today was more about understanding why IaC matters at scale and why Terraform continues to dominate in real-world DevOps ecosystems. Understanding Infrastructure as Code (IaC) As a DevOps engineer, IaC is not new to me. But today’s session broke it down in a way that clarified its real value. IaC = Infrastructure represented as code. Instead of building resources manually through AWS/Azure/GCP consoles, we define everything in .tf files. This ensures our setup is predictable, repeatable, and version-controlled. What I liked most is how clearly the contrast was shown: Cloud-Agnostic Tools (Terraform / Pulumi) Work across AWS, Azure, GCP, Kubernetes, GitHub, etc. ...
Read more

AWS with Terraform (Day 27)

Image
Automating AWS Infrastructure Using Terraform and GitHub Actions Infrastructure automation is where DevOps truly becomes real. Writing Terraform code is only half the story—the real value comes when infrastructure changes are version-controlled, reviewed, scanned, approved, and applied automatically . On Day 27 of my DevOps journey, I implemented a production-grade CI/CD pipeline to automate AWS infrastructure using Terraform and GitHub Actions , following best practices used in real-world teams. This blog walks through the architecture, workflow design, safety controls, and lessons learned . Why Automate Terraform with CI/CD? Running Terraform manually from a laptop works for learning, but it quickly breaks down in team and production environments: Local state files are risky and hard to share Credentials scattered across machines No standardized reviews or approvals No security or policy checks before apply No clear audit trail By integrating Terraform with G...
Read more

AWS with Terraform (Day 02)

Image
  AWS with Terraform – Day 02 Deep Dive into Providers, Versioning & the Real “Bridge” Between Code and Cloud Today’s session took me from simply writing Terraform code to actually understanding the engine behind it — the Provider layer . And honestly, once this clicks, Terraform stops feeling like just a tool and starts feeling like a real engineering system. Here’s what I learned and how I’ve started applying it. Providers: The Plugin That Makes Terraform Real The simplest way I now understand it: Terraform writes the story. Providers translate it into cloud actions. Before today, I treated the provider block as just two lines of mandatory code. Now I see it as the translator that turns HCL into actual AWS API calls . Example: I write: resource "aws_s3_bucket" "demo" { bucket = "my-app-demo" } The AWS provider turns that into the exact S3 API request. This layer saves us from manually hitting APIs, building payloads, or juggling ...
Read more