Staff Engineer, Test Automation - San Mateo (R5792)

Shield AI is a venture-backed defense-tech company with the mission of protecting service members and civilians with intelligent systems. Its products include Hivemind autonomy software, V-BAT and X-BAT aircraft, and Aechelon simulation and synthetic reality technologies. With offices and facilities across the U.S., Europe, the Middle East, and Asia-Pacific, Shield AI’s technology actively supports operations worldwide. For more information, visit www.shield.ai https://shield.ai/. Follow Shield AI on LinkedIn https://www.linkedin.com/company/shield-ai, X https://x.com/shieldaitech, Instagram https://www.instagram.com/shield_ai, and YouTube https://www.youtube.com/@Shield_AI. JOB DESCRIPTION: Shield AI is a venture-backed defense-tech company with the mission of protecting service members and civilians with intelligent systems. Its products include Hivemind autonomy software, V-BAT and X-BAT aircraft, and Aechelon simulation and synthetic reality technologies. With offices and facilities across the U.S., Europe, the Middle East, and Asia-Pacific, Shield AI’s technology actively supports operations worldwide. For more information, visit www.shield.ai https://shield.ai/. Follow Shield AI on LinkedIn https://www.linkedin.com/company/shield-ai, X https://x.com/shieldaitech, Instagram https://www.instagram.com/shield_ai, and YouTube https://www.youtube.com/@Shield_AI. The Hivemind Software Engineering Integration and Test team is seeking a Staff Automated Test Engineer to provide technical leadership for automated verification across our next-generation autonomy platform. You will define and evolve test architecture, validation infrastructure, MLOps quality systems, and CI/CD pipelines that enable the Hivemind software ecosystem to create, test, and deploy resilient autonomy capabilities for unmanned aircraft and robotic platforms operating in complex, contested, and GPS-denied environments. You will work across production flight code, machine learning models, simulation and synthetic environments, mission planning and orchestration systems, operator-facing Ground Control Station applications, telemetry and data pipelines, cloud-native developer infrastructure, and hardware-in-the-loop systems. In this hands-on Staff role, you will lead complex, cross-functional initiatives, establish automation and verification standards, identify systemic quality risks, and develop scalable test infrastructure. The ideal candidate combines deep Python automation and distributed-systems testing experience with technical leadership and experience validating machine learning systems throughout the data, training, evaluation, deployment, and monitoring lifecycle. What you'll do: - - Own the technical strategy, architecture, and roadmap for automated testing, verification, and MLOps quality across the Hivemind ecosystem. - Design and maintain scalable test frameworks for autonomy software, backend services, APIs, operator-facing applications, and distributed hardware environments. - Lead functional, integration, regression, system, performance, reliability, and end-to-end testing across simulation, edge-compute, software-in-the-loop, and hardware-in-the-loop environments. - Build automated ML validation pipelines covering data quality, training reproducibility, model accuracy, robustness, regression, latency, resource utilization, and system integration. - Establish CI/CD and continuous training workflows that provide versioning and traceability for datasets, models, configurations, evaluation results, and deployment artifacts. - Develop scenario-based validation for autonomy models, including edge cases, degraded sensing or communications, distribution shifts, and representative mission conditions. - Create observability, analytics, and failure-triage capabilities for software behavior, model and data drift, inference health, test results, and production performance. - Build Python automation that improves test execution, parallelization, reporting, environment setup, experiment comparison, and developer productivity. - Create test harnesses, simulators, stubs, mocks, and synthetic data capabilities that improve system testability and coverage. - Collaborate with software, autonomy, machine learning, data, simulation, and systems engineers to define verification strategies and improve designs before implementation. - Develop and govern AI-assisted engineering workflows using coding agents and LLM-based tools for test generation, log analysis, debugging, and failure triage while maintaining security, reproducibility, and traceability. Required qualifications: - - Typically 8+ years of relevant experience in software engineering, test infrastructure, developer tooling, MLOps, systems integration, or systems verification, or an equivalent combination of experience and demonstrated impact. - 5+ years of experience building scalable automation frameworks or developer tooling in Python. - Demonstrated success designing test, CI/CD, or MLOps infrastructure used across multiple engineering teams. - Experience validating machine learning systems across the data, training, evaluation, packaging, deployment, and monitoring lifecycle. - Understanding of ML quality risks such as data leakage, training-serving skew, nondeterminism, distribution shift, drift, model regression, and statistical acceptance criteria. - Experience defining model-performance baselines, automated evaluation suites, release thresholds, and candidate-to-production comparison workflows. - Experience testing GPU-accelerated infrastructure and workloads, including GPU scheduling, allocation, utilization, and resource contention in Kubernetes environments. - Experience with performance benchmarking, profiling, and observability for GPU workloads, including identifying compute, memory, storage, networking, and data-loading bottlenecks. - Experience validating multi-tenant Kubernetes environments, including RBAC, resource quotas, worklo