Boring Data
Template: AWS+Iceberg
Boring DataTemplate: AWS+SnowflakeTemplate: AWS+Iceberg
Template: AWS+Iceberg
Boring DataTemplate: AWS+SnowflakeTemplate: AWS+Iceberg
  • Introduction
    • Overview
    • Key Concepts
    • Get Started
  • Project Structure
    • pipelines/
      • Ingestion: dlt + lambda
      • Transformation: dbt
    • base/aws/
    • live/
  • Guides
    • Add a New Pipeline
    • CI Deployment
  • Help
    • FAQ
Powered by GitBook
On this page
Edit on GitHub
  1. Introduction

Overview

NextKey Concepts

Last updated 14 days ago

Welcome to the AWS + Apache Iceberg Data Stack Template!

This template provides everything you need to build a modern data lakehouse on AWS using Apache Iceberg.

The infrastructure is fully automated with Terraform and includes:

  • An example ingestion pipeline: dlt + lambda

  • A dbt project transformation Iceberg data with Athena

  • AWS Step Functions to orchestrate ingestion + transformation

  • A Github CI workflow

The template is designed to be fully modular and customizable. You can easily swap out any component with your preferred tools:

  • Replace dlt with any other ingestion framework

  • Use an alternative to dbt for transformations

  • Switch to a different orchestration service instead of Step Functions

Next Steps

  1. Key Concepts - Understand the core architecture and components

  2. Get Started - Set up your environment and run your first deployment

Iceberg Stack Overview