Onnx2tensorrt

Self hostable service to convert an ONNX model to TensorRT engine.

Server deployment is mandatory for running this app. The fullstack app does not need to be deployed.

1. Create a Modal account. Low usage should fit within the $30 monthly free tier.

2. Create 2 Cloudflare R2 buckets with access key

3. Create a Modal secret:

ONNX_BUCKET=
TRT_BUCKET=
S3_ENDPOINT=
S3_KEY_ID=
S3_KEY=
REGION_NAME=auto

4. Create .env file in project root for Modal backend deployment:

APP_NAME=
MODAL_WORKSPACE=

5. Install python package and deploy:

poetry install
pnpm run deploy:modal

• Modal - Serverless GPU FAAS platform.
• TensorRT - Build fast ML models for Nvidia GPU