Overview / Usage

Early Access Link: https://mailchi.mp/59618be69e39/infernoengine

Deploying deep learning models can be a tricky affair and one needs to pay attention to not only the deployed code but also to the server hardware and framework optimization. Inferno Engine is a service which will allow anyone to one-click upload and deploy a model, and access it through a secured REST API endpoint.

On-demand inference will allow developers to focus on application architecture rather than on orchestrating model deployment. Deployed models can be accessed anywhere, allowing inference to be performed even on low powered applications like IoT devices and mobile phones.

Currently, this platform supports Keras and Tensorflow, and I wish to add more frameworks eventually. The project is in a closed in-house alpha.

Methodology / Approach

Inferno Engine uses a web based dashboard to help developers upload their models. The models are evaluated and then uploaded to an encrypted storage device. The model is now accessible via the cloud and can be used to perform inference on demand. If the developer wishes to access his/her model again, then he/she creates a POST request which contains authentication parameters and the inputs for the model. The request is authenticated and the model is decrypted. It is then used to perform inference, and the model outputs is sent back as a JSON response.

Technologies Used

Python is used to design the framework. In order to optimize inference on the server CPUs, I use Intel-Tensorflow. Currently, deploying deep learning models made in Keras with a Tensorflow backend is supported.

Other links

• Sign up for Early Access!!
• Starting Guide