Overview / Usage

The notebook shows how to take an ONNX model, convert/optimize it to OpenVINO IR format, and run it in the OpenVINO runtime. The optimized model is compared against the original ONNX model, for output compatibility and performance evaluation.

Requirements: Python 3.9.x, OpenVINO 2022.2, ONNX Runtime 1.13.1 (on Windows)
NVIDIA GPU (with CUDA 11.6 and cuDNN)

Setup environment following the official installation guide and the steps below to configure a Python Virtual Environment.
For additional details and other examples, please refer to the OpenVINO Notebooks repository. We used the 102-pytorch-onnx-to-openvino notebook as a starting point for this demo.

python -m venv .venv

..venv\scripts\activate
python -m pip install -U pip
pip install wheel
pip install torch==1.12.1+cu116 torchvision==0.13.1+cu116 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu116
pip install openvino-dev[onnx]==2022.2.0
pip install fastseg
pip install ipywidgets
pip install matplotlib

Methodology / Approach

Check slides on https://github.com/deltatrelabs/deltatre-aiday-2022-demo/tree/main/docs

Technologies Used

Intel OpenVino,
ONNX Runtime,
Nebuly,
etc etc

Documents and Presentations

Repository

https://github.com/deltatrelabs/deltatre-aiday-2022-demo

Collaborators