Senior Databricks Data Engineer

<div class="content-intro"><p><strong>About Us</strong><br>Capital Farm Credit is the largest rural lending cooperative in Texas, serving 192 counties through nearly 70 credit offices. With over $12 billion in assets and more than 600 team members, we provide essential financial services to farmers, ranchers, rural homeowners, and agribusinesses. As part of the nationwide Farm Credit System, we are dedicated to supporting rural communities and agriculture.</p> <p><strong>Why Join Us?</strong><br>We seek motivated individuals who share our core values: commitment, trust, value, and family-like respect. As a customer-owned cooperative, we align employee success with member success, offering competitive pay, growth opportunities, and a supportive environment.</p> <p><strong>Our Benefits:</strong></p> <ul> <li><strong>Incentive Program:</strong> Company-wide, goals-based rewards.</li> <li><strong>Accrued Time Off:</strong> Earn 13 days of annual leave and 15 days of sick leave per year, plus enjoy 10-12 paid holidays annually.</li> <li><strong>Retirement:</strong> 401(k) with up to 9% employer contribution/match.</li> <li><strong>Health Coverage:</strong> Affordable medical, dental, and vision plans.</li> <li><strong>Parental Leave:</strong> 8 weeks of paid parental leave.</li> <li><strong>Life & Disability Insurance:</strong> Employer-paid coverage.</li> <li><strong>Education & Wellness:</strong> Tuition reimbursement and up to $400 for wellness expenses.</li> </ul> <p> </p> <p>At Capital Farm Credit, you’ll find more than a job—you’ll find purpose.</p> <p> </p></div><p><strong><span class="fontstyle2">EDUCATION AND EXPERIENCE</span></strong></p> <ul> <li>Bachelor’s degree in computer science, Information Systems, Engineering, Data Analytics, or a related field, or an equivalent combination of education and experience.</li> <li>Five (5) or more years of hands-on data engineering experience using Azure, Databricks, or comparable cloud data platforms, including two (2) or more years serving as a senior engineer, technical lead, or mentor on data engineering teams or projects.</li> <li>Demonstrated experience in Databricks, Spark, PySpark, Delta Lake, Unity Catalog, SQL, Python, notebooks, jobs, clusters, and Lakehouse architecture is required.</li> <li>Experience with Azure Data Lake Storage (ADLS), Azure SQL, Key Vault, managed identities, ETL/ELT processes, dimensional modeling, and cloud security fundamentals is required.</li> <li>Experience designing data solutions that support analytics, machine learning, artificial intelligence, governed enterprise reporting, and secure data consumption use cases is also required, including knowledge of AI/ML data preparation practices such as feature engineering, model-ready data design, data quality, lineage, and secure access to sensitive data.</li> <li>Databricks Certified Data Engineer Associate certification or an equivalent current Databricks data engineering certification is required;</li> <li>Databricks Certified Data Engineer Professional certification is preferred.</li> </ul> <p> </p> <p><strong><span class="fontstyle0">JOB SUMMARY</span></strong></p> <ul> <li>The Senior Databricks Data Engineer designs, develops, and supports scalable, secure, and governed cloud-based data solutions that enable enterprise analytics, business intelligence, regulatory reporting, artificial intelligence initiatives, and data products. This role serves as a senior technical resource for the enterprise data platform and provides day-to-day technical leadership to a team of data engineers, including work planning and assignment, design and code review, mentoring, and escalation support for complex production issues.</li> <li>In addition to building, maintaining, and optimizing production data pipelines, this position leads the design and development of the organization’s enterprise data platform using Azure and Databricks. The position applies and establishes modern Lakehouse engineering practices and technologies, including Databricks, Apache Spark, PySpark, Delta Lake, Unity Catalog, Azure Data Lake Storage, SQL, and Python, with an emphasis on data governance, performance, automation, security, and production reliability. The Senior Databricks Data Engineer also drives data solution architecture by establishing and applying Lakehouse and Medallion architecture patterns, developing governed and AI-ready data assets, creating reusable engineering frameworks, and supporting secure and consistent data consumption across the organization.</li> </ul> <p><span class="fontstyle2"><span class="fontstyle0"><strong>ESSENTIAL FUNCTIONS</strong></span></span></p> <ul> <li>Provide day-to-day technical leadership and direction to a team of data engineers, including assigning and prioritizing work, estimating effort, tracking delivery, and removing technical obstacles in partnership with the manager.  </li> <li>Mentor and develop engineers through design guidance, pair programming, knowledge sharing, and structured onboarding of new team members, contractors, and partners. </li> <li>Lead technical design sessions and establish engineering standards, patterns, and best practices for ingestion, transformation, governance, deployment, observability, and data consumption.   </li> <li>Own technical quality across the team by co