This role is for one of our clients Pay: 30–100/hour Job Type: Contractor (~15 hrs a week) Location: Remote Schedule: Flexible, you pick the hours and days (including weekends if desired) Required Skills: Python3, Java, Rust, Algorithms, Basics of C++, TypeScript, Bug fixing, Feature implementation, Codebase refactoring, Performance optimization We are seeking an experienced Senior Software Engineer to contribute to the training and evaluation of next-generation artificial intelligence systems. This role is ideal for engineers with strong backend development expertise, excellent problem-solving abilities, and a proven track record of building reliable, scalable, and high-performance software. You will design realistic and challenging coding tasks that assess AI systems' ability to implement features, fix bugs, refactor existing code, and optimize application performance. You will also develop deterministic verification mechanisms that accurately validate solutions, accepting reasonable implementations that meet the requirements while rejecting incorrect or incomplete submissions. The role requires strong software engineering judgment, extensive debugging experience, and a solid understanding of testing methodologies. Previous AI experience is not required; your technical expertise and independent judgment are what matter most. Requirements Key Responsibilities • Coding Task Development: Design challenging, real-world software engineering problems involving feature implementation, bug fixes, code refactoring, and performance optimization. • Reference Solution Development: Create high-quality reference implementations that demonstrate correct, efficient, and maintainable approaches to solving technical problems. • Automated Verification: Build deterministic verifiers, validation scripts, and comprehensive test cases to evaluate solution correctness and ensure reliable task assessment. • Backend Engineering: Apply expertise in backend development, scalable services, APIs, asynchronous processing, and distributed systems to create realistic engineering challenges. • Debugging and Troubleshooting: Analyze complex software defects, investigate system bottlenecks, and identify root causes of unexpected behavior. • Testing Strategy and Quality Assurance: Design effective testing strategies, develop and maintain large test suites, diagnose flaky tests, and improve overall software quality. • Code Review: Evaluate code for correctness, readability, maintainability, performance, security, and adherence to engineering best practices. • Refactoring and Optimization: Develop tasks that assess an AI system's ability to improve legacy code, simplify complex implementations, and optimize resource utilization. • Technical Reasoning Assessment: Evaluate AI-generated reasoning related to architecture, debugging, algorithms, system design, and implementation decisions. • Technical Documentation: Document task requirements, expected behavior, test coverage, solution approaches, and verification criteria clearly and consistently. • Continuous Improvement: Analyze model failures and refine coding tasks and evaluation methods to improve the quality and reliability of AI-generated software. Required Qualifications • Strong professional experience in software engineering, particularly backend development and production-grade systems. • Proficiency in one or more programming languages, including Java, Node.js, Go, Rust, TypeScript, Python, C++, or similar technologies. • Proven experience designing, building, and maintaining scalable services and APIs. • Strong understanding of data structures, algorithms, system design, and fundamental computer science concepts. • Extensive experience with debugging, troubleshooting, performance optimization, and code review. • Practical experience with automated testing, test design, and software quality assurance. • Ability to identify implementation defects, edge cases, inefficient algorithms, and architectural weaknesses. • Strong understanding of code maintainability, refactoring, reliability, and software engineering best practices. • Excellent analytical, problem-solving, and technical communication skills. • Ability to work independently and exercise sound technical judgment when evaluating complex engineering problems. Must-Have Skills • Senior Software Engineering • Backend Development • Scalable Services and API Design • Java, Node.js, Go, Rust, TypeScript, Python, or C++ • Data Structures and Algorithms • Debugging and Root-Cause Analysis • Automated Testing and Test Strategy • Code Review and Refactoring • Performance Optimization • Technical Problem-Solving Good-to-Have Skills • Large-Scale Test Suite Design and Maintenance • Flaky Test Diagnosis and Resolution • Test Automation Frameworks • Continuous Integration and Continuous Delivery (CI/CD) • Distributed Systems and Microservices • System Architecture and Design Patterns • Regression, Integration, and Unit Testing • Deterministic Evaluation and Automated Verification • AI Coding Agent Evaluation • Legacy System Modernization • Production Reliability and Performance Engineering Preferred Qualifications • Demonstrated success delivering scalable, reliable, and high-performance backend applications. • Experience resolving complex software defects and improving system performance. • Proven ability to implement complex features and refactor large codebases. • Strong background in test strategy, automated validation, and improving engineering quality standards. • Experience establishing testing best practices across engineering teams. • Ability to translate complex engineering requirements into clear, reproducible coding tasks and evaluation criteria. Selection Process • Application: Submit your application and complete the required screening questions. • AI Interview: Complete an approximately 30-minute interview focused on your technical expertise and problem-solving approach. • Technical Assessment: Complete a tech