There is a particular kind of bug report that arrives after a model ships to hardware. It says the detector works in testing and does not work in the field, and it comes with a video where the boxes are late, or jittery, or subtly wrong in a way nobody can describe.
Almost none of these are model problems. They live in the gap between where you trained and where you run, and the gap has a small number of recurring shapes.
1. Training resolution and export resolution disagree
This is the most common one and the easiest to miss, because nothing errors.
You train at one input size. You export to TensorRT at another, maybe because someone picked a round number, maybe because the export script has a different default. The engine builds. Inference runs. Boxes come out. They are just quietly worse, because the network is now seeing objects at a scale distribution it never trained on.
Small objects punish this hardest. If your targets are 80 to 120 pixels in the source frame, a 20 % resolution mismatch can move them below the scale the model learned to detect at. Pin the two numbers together in config and assert on them at export time. It costs one line and removes an entire category of confusion.
2. Preprocessing drifts between the two environments
Your training pipeline resizes, pads and normalises in a particular order, with a particular interpolation, into a particular colour order. Your deployment code was written separately, often in C++, often by someone reading the Python from memory.
Letterbox padding colour, BGR versus RGB, whether normalisation happens before or after the cast to float, whether the resize uses bilinear or nearest: each of these is individually small and they compound. The symptom is a model that is not broken, just consistently a few points worse than the report claims.
The fix is not vigilance. It is a test: run one fixed image through both preprocessing paths and assert the tensors match within tolerance. If they do not, you have found your missing accuracy before it reaches the field.
3. The throughput budget is not where you think
On a workstation the model is the expensive part. On an embedded board it often is not.
Frame capture, colour conversion, the copy from host to device, the copy back, the postprocessing, the NMS, the drawing of overlays: on a constrained target these can add up to more than the forward pass. I have spent more time moving work off the critical path than optimising inference itself. C++ multithreading to overlap capture with inference, CUDA to move colour conversion onto the device, OpenMP for the embarrassingly parallel postprocessing.
Measure the whole pipeline, not the model. If you only profile inference you will optimise the one part that was already fast.
4. FP16 is usually free, until it is not
TensorRT FP16 on Jetson is close to a free speedup for most detection backbones. Most.
The failure mode is narrow but real: layers with a wide dynamic range can saturate, and the effect shows up as a small accuracy drop that is easy to attribute to something else. The discipline is boring and non-negotiable. Run your validation set through the exported engine, not through the PyTorch model, and compare mAP against the training-time number. If the delta is larger than noise, you have a precision problem, and you now know it in the lab rather than in the field.
The underlying point
Every one of these is a mismatch between two environments that were built by different people at different times under different assumptions. None of them is intellectually hard. They persist because the workstation number is reassuring, and nobody wants to be the person who re-measures a result that already looks good.
The habit worth building is simple: the number that counts is the one measured on the target hardware, through the deployment code path, on data the model has not seen. Everything else is an estimate.