Category: Deep Learning

ONNX files in OpenCV

I have been aware of OpenCV’s ‘dnn’ module for some time: Last time we tried to use it in a project was a number of years ago, and it didn’t seem to be ready for what we needed – or perhaps we just misunderstood it and didn’t give it a good enough look.

Aside from that, I’ve been using .ONNX (Open Neural Network eXchange) files for a while now. My standard usage of these is to transport a trained model from PyTorch – for example a ResNet classifier – onto a Jetson Nano, NX or Orin. PyTorch can export as .ONNX, and TensorRT on the Jetson can import them, so it’s been literally an ‘exchange’ file format for me.

However, pulling these two things together, I have recently learned that OpenCV’s ‘dnn’ module can load directly from .ONNX files, specifically including ResNet models such as the ResNet18 classifier I have recently trained for a client.

There are a few ‘tricks’ required to prepare images to be classified, and it took me a fair amount of research (including some trial-and-error, and using ChatGPT – that was a day I can never get back…) but it works now: I can classify images, using a ONNX file, in OpenCV, from either C++ or Python.

This means that models that I originally trained for Jetson hardware can now be used on any platform with OpenCV. I will be testing this on a Raspberry Pi 5 shortly to gauge performance.

Currently, it’s using CPU only – but it does use all CPU cores available – but I believe GPU is also supported given a suitably-compiled OpenCV: I may try that next.

Jetson hardware, and the ‘jetson-inference’ package

I have been involved in several projects very recently (and two ongoing) where we have used NVIDIA ‘Jetson’ hardware (Nano, Xavier / NX, and ConnectTech Rudi NX).  These machines are roughly ‘credit-card sized’ (apart from the Rudi, which has a larger but very ‘rugged’ case) and are ideal for ‘edge’ or embedded systems.

The Jetson hardware is basically a small but powerful GPU, but also including a CPU and small ‘motherboard’ providing the usual USB ports, etc.  They run a modified version of Ubuntu Linux.

In some cases I developed software in-house using OpenCV (C++ and Python).  However, I am also making more and more use of the excellent ‘jetson-inference’ library of deep-learning tools, and have now built up quite a bit of experience in using this library and developing applications and solutions based on it.

In short, it is very good for developing solutions that need:

  • Image classification (e.g. cat vs dog, or labrador vs poodle, or beach vs park)
  • Object detection (i.e. accurate location, and classification of objects – can be trained to recognise new objects, including very small/distant)
  • Pose estimation (e.g. standing, sitting, walking, pointing, waving)

I have now developed a number of solutions that have ‘gone live’ using this hardware and toolkit.  I am also experienced in the ‘back end’ tasks of training new ‘models’ to recognise new, specfic classes of objects, and porting those models to the Jetson hardware.

Please contact me to discuss whether I can help you with your Jetson-based project.  tom [at] alvervalleysoftware.com.