<h2>Overview</h2> <p>We are seeking a <strong>Lead Data Engineer</strong> to design and implement the core analytical data platform for a federal permitting modernization initiative.</p> <p>The Lead Data Engineer will establish a modern lakehouse environment using <strong>DuckDB, Apache Iceberg, Amazon S3, and AWS Glue Catalog</strong> and will lead the engineering patterns used to ingest, harmonize, and serve data across multiple source systems. The role will also help implement metadata-driven and agentic approaches to data engineering, including source analysis, mapping, pipeline generation, testing, reconciliation, and quality assurance.</p> <p>This is a hands-on technical leadership role with responsibility for guiding and reviewing the work of other Data Engineers.</p> <p>This position is remote.</p> <p><strong>Duties & Responsibilities</strong></p> <ul> <li>Design and implement the core DuckDB / Apache Iceberg data platform on AWS.</li> <li>Define physical data organization, schema, partitioning, file layout, snapshot, compaction, and optimization strategies for Iceberg tables.</li> <li>Integrate DuckDB with Amazon S3, Apache Iceberg, and AWS Glue Catalog.</li> <li>Establish bronze, silver, and gold data-layer conventions and reusable engineering patterns.</li> <li>Lead and mentor Data Engineers, providing technical direction, code reviews, and engineering guidance.</li> <li>Build and maintain analytical datasets supporting application services, reporting, analytics, and data science.</li> <li>Develop reusable patterns for incremental loads, CDC, schema evolution, late-arriving data, backfills, and historical reconstruction.</li> <li>Work with the Data Architect to implement the common canonical data model and source-to-canonical mappings.</li> <li>Lead migration and harmonization of datasets from multiple source systems.</li> <li>Implement reconciliation processes that demonstrate completeness and accuracy between source and target systems.</li> <li>Establish automated data-quality monitoring, operational observability, and testing.</li> <li>Implement agentic and AI-enabled data engineering patterns supporting activities such as source analysis, mapping, pipeline generation, troubleshooting, reconciliation, and quality assurance.</li> <li>Support integration of structured data with associated document and geospatial datasets.</li> <li>Lead development of certified data products, analytical datasets, and business metrics.</li> <li>Optimize DuckDB queries and Iceberg datasets for performance and efficiency.</li> <li>Develop engineering documentation, operational procedures, and technical handoff materials.</li> </ul> <p><strong>Educational Requirements</strong></p> <ul> <li>Bachelor's degree in Computer Science, Data Engineering, Information Systems, Engineering, or a related field.</li> </ul> <p><strong>Requirements</strong></p> <ul> <li><strong>6+ years</strong> of relevant data engineering or data platform experience, including experience leading or mentoring other engineers.</li> <li>Strong hands-on experience with <strong>DuckDB and Apache Iceberg</strong>.</li> <li>Advanced SQL and strong Python development experience.</li> <li>Experience building cloud-based analytical platforms using object storage and columnar formats such as Parquet.</li> <li>Strong understanding of partitioning, schema evolution, snapshots, compaction, incremental processing, and analytical query optimization.</li> <li>Experience implementing bronze/silver/gold or comparable layered data architectures and transforming heterogeneous enterprise datasets.</li> <li>Experience with AWS data services, particularly Amazon S3 and AWS Glue.</li> <li>Experience designing reusable, metadata-driven ingestion and transformation frameworks.</li> <li>Experience implementing automated data quality, reconciliation, monitoring, and operational controls.</li> <li>Experience with <strong>agentic or AI-enabled data engineering solutions</strong>, including the use of agents or AI-assisted workflows for activities such as data mapping, pipeline generation, validation, testing, or troubleshooting.</li> <li>Hands-on experience with agentic frameworks such as LangChain, LangGraph, or similar frameworks for building tool-using or multi-step AI workflows.</li> <li>Strong troubleshooting, performance tuning, and technical leadership skills.</li> </ul> <p><strong>Bonus Skills</strong></p> <ul> <li>Experience with Kafka or Amazon MSK and CDC architectures.</li> <li>Experience with GeoParquet, PostGIS, or geospatial datasets.</li> <li>Experience with OpenSearch, document stores, or vector search technologies.</li> <li>Experience integrating enterprise SaaS platforms, APIs, databases, and file-based sources.</li> <li>Experience working in the <strong>U.S. Federal Government space</strong>.</li> </ul> <p>Experience with permitting, regulatory, land management, environmental, energy, or natural-resource datasets.</p> <h2>Benefits and Salary</h2> <p>We attract the best people in the business with our competitive benefits package that includes medical, dental and vision coverage, 401k plan with employer contribution, paid holidays, vacation, and tuition reimbursement.</p> <p>If you enjoy being a part of a high performing, p