Systems Test Engineer, End-to-End Validation
Posted
Location
San Francisco
Employment Type
Full time
Location Type
On-site
Department
Software
About the company
Efference provides high-performance perception and compute infrastructure for training and deploying robot models in the cloud. Our vision is to make robotic deployments scalable with shared compute.
We approach the problem by owning the entire path from the image sensor to the datacenter GPU. We shrink the load at the sensor, fix the uplink so robot traffic is fast and reliable, and clock synchronize the system so observations arrive time-aligned, ready for inference.
About the role
You'll validate Efference as a complete system, from image capture through network transport to GPU inference and the return path. You'll build automated tests that expose integration failures, quantify operating limits, and give the team clear evidence for release decisions.
What you’ll do
- Define system acceptance tests with engineering owners, covering observation integrity, synchronization, delivery latency, jitter, throughput, and recovery.
- Build hardware-in-the-loop rigs that combine cameras, client devices, access points, network impairment, and inference workloads in reproducible configurations.
- Instrument timing from exposure to inference response, validating clock alignment and measurement uncertainty before comparing latency across hosts.
- Exercise fleet load, roaming, congestion, packet loss, clock drift, thermal throttling, process crashes, and interrupted updates through stress and endurance tests.
- Correlate device logs, packet traces, and GPU metrics to isolate failures; create minimal reproductions and regression tests with subsystem engineers.
- Maintain a release matrix across hardware, firmware, SDK, and model-serving versions, reporting coverage, results, unresolved defects, and readiness against agreed criteria.
What we are looking for
- Experience validating complex systems that combine hardware, embedded software, networking, or distributed compute.
- Strong Python skills and experience building reliable automated test infrastructure rather than only executing manual test plans.
- Ability to design meaningful load and fault-injection experiments and interpret latency distributions and failure rates.
- Practical debugging across Linux devices and networked services, using logs, traces, packet captures, and lab instruments.
- Careful measurement practice and clear communication of what a test demonstrates, what remains untested, and why a release fails a requirement.
Nice to Have
- Experience with camera timing, PTP, hardware timestamping, network emulation, or GPU inference systems.
- Work on robotics hardware-in-the-loop testing, fleet-scale simulation, or long-duration reliability tests.
- Experience designing CI systems that control physical equipment and preserve complete test artifacts.