Built a minimal template wiring TensorRT-based detection to a JAX NMPC loop under ROS 2 Humble with zero-copy transport; on an Orin NX 16GB it sustains 30 Hz end-to-end (about 28–32 ms vision, about 3 ms iLQR). If you need a starting point for optimizing the full stack and integrating learning-based perception with model-based control, the repo is here (with benchmarks and launch files): https://github.com/jasongray-ai/ros2-perception-nmpc.
On our Orin NX 16GB, I shaved about 3 ms off the “28–32 ms vision” by fusing pre/post steps into a TensorRT plugin and capturing the path in a CUDA Graph. If you’re on ROS 2 Humble zero-copy, keep tensors on-device with loaned messages (Isaac ROS NITROS works) or a hidden host hop sneaks in when you compose nodes. Small caveat: lock clocks (sudo nvpmodel -m 2 && sudo jetson_clocks) or you’ll see 30 Hz jitter once it heats up.
Quick tip that helped me hit 30 Hz consistently on Humble: lock the Orin NX with nvpmodel -m 2 and sudo jetson_clocks — DVFS dips were stretching my “28–32 ms vision” to 40+ ms, . It costs some thermals, but the variance dropped a ton; details here: https://docs.nvidia.com/jetson/archives/r35.3/DeveloperGuide/text/SD/PlatformPowerAndPerformance.html#maximizing-performance. Did you also try donate_argnums in JAX to cut device copies in the iLQR loop?