About the Role Lead the development of agent intelligence that helps mechanical engineers complete multi-step workflows in desktop engineering software. You will combine hands-on engineering, technical leadership, and collaboration with user research and domain experts to improve real-world task completion and product value. What You'll Do • Own agent task success metrics, establish baselines, and systematically improve completion rates across engineering workflows. • Design and build reproducible evaluation infrastructure based on validated user stories, including tests for cost efficiency and failure modes. • Set per-task token budgets and track the cost of completed workflows. • Interview engineers with research and domain teammates, document their workflows, and translate validated stories into benchmarks. • Prioritize workflow coverage based on customer value and technical feasibility. • Set the agent architecture for tool calling, state management, error recovery, model routing, and context management. • Lead a small AI engineering team while writing production code, reviewing designs, and unblocking teammates. • Work with product, integrations, and customers during proof-of-concept projects to align agent behavior with real usage. What We're Looking For • At least 7 years of software engineering experience, including 2 or more years building LLM-based agents that take real-world actions. • Strong Python skills and deep experience with LLM application architecture, including model selection, retrieval, context management, orchestration, and tool calling. • Experience building rigorous agent evaluation and benchmarking systems that measure task completion, cost, and failure modes. • Technical leadership experience on a small engineering team, with a track record of reviewing code and continuing to contribute production software. • Experience shipping AI or LLM tools using proprietary engineering data or desktop engineering software. • Familiarity with function calling, tool APIs, tracing or observability, and evaluation tools. • Experience with desktop automation or application APIs, such as COM, and a background in mechanical engineering software, CAD/CAE, PLM, or an adjacent domain. • Understanding of deployment constraints on managed enterprise workstations. Compensation & Benefits Base salary: $160,000 to $250,000 USD annually. Visa sponsorship is not available. Location On-site in San Francisco, California, United States.