> For the complete documentation index, see [llms.txt](https://docs.boringdata.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.boringdata.io/template-aws-iceberg/project-structure/live.md).

# live/

## Overview

The `live/` directory contains Terragrunt configurations for each environment in this AWS Iceberg template. 

Terragrunt serves as a thin wrapper around Terraform that provides:

* Consistent configuration management across environments
* DRY (Don't Repeat Yourself) infrastructure code
* Dependency handling between modules
* Simplified remote state management

## Folder Structure

The template is built around two primary Terraform modules:

* `base/aws` - Core AWS infrastructure components
* `pipelines/` - Data processing pipeline components

These modules are organized in the `live/` directory with environment-specific configurations:

```
live/
│   ├── base/
│   │   └── aws/
│   │       └── terragrunt.hcl
│   ├── pipelines/
│   │   └── terragrunt.hcl
│   └── root.hcl      # Common configuration shared across all modules
```

Each `terragrunt.hcl` file contains:

* Environment-specific input values
* Terraform provider configuration
* Backend configuration for state management
* Module dependencies

## Deployment Options

There are three ways to deploy this infrastructure:

| Deployment Method           | State Storage    | Recommended Use          |
| --------------------------- | ---------------- | ------------------------ |
| Local with local state      | Local filesystem | Development/testing only |
| Local with remote state     | AWS S3           | Development and staging  |
| GitHub CI with remote state | AWS S3           | Production deployments   |

### 1. Local Deployment with Local State

Best for quick testing and initial development. It is not recommended for shared or production environments.

```bash
export AWS_PROFILE=<your_profile>
export ENVIRONMENT=<environment>
make deploy
```

### 2. Local Deployment with Remote State

Recommended for development work requiring state persistence:

1. First create the S3 remote state bucket by following the instructions in the [production deployment guide](https://github.com/boringdata/boringdata-template-aws-iceberg/blob/main/production-deployment.md)
2. Then run:

```bash
export AWS_PROFILE=<your_profile>
export ENVIRONMENT=<environment>
make deploy
```

### 3. GitHub CI Deployment with Remote State

Recommended for production environments:

1. Configure the remote state following the [production deployment guide](https://github.com/boringdata/boringdata-template-aws-iceberg/blob/main/production-deployment.md)
2. Push your changes to the configured branch
3. GitHub Actions will execute the deployment process automatically

## Deployment Process

The `make deploy` command performs two sequential operations:

1. **Infrastructure Deployment**: Runs `terragrunt run-all apply` to create all infrastructure resources
2. **Container Deployment**: Builds and pushes Docker images for components in `pipelines/ingest` and `pipelines/transform`
3. **Schema Migration**: Runs `make migrate-schemas` to migrate the schemas for the ingestion and transformation layers

### Why a separate step for container deployment?

This approach resolves circular dependencies between infrastructure and container resources:

1. First, Terraform creates the infrastructure (ECR repositories, Lambda functions, etc.)
2. Then, container images are built and pushed to the newly created repositories

This sequence ensures all necessary infrastructure exists before container deployment occurs, resolving the "chicken and egg" problem where:

* ECR repositories must exist before container images can be pushed
* Lambda functions need a reference to container images that don't exist yet

By separating these processes, we ensure proper resource creation while maintaining infrastructure as code principles.

## Resources

* [Terragrunt](https://terragrunt.gruntwork.io/)


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.boringdata.io/template-aws-iceberg/project-structure/live.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.