From the course: Rust LLMOps

Unlock this course with a free trial

Join today to access over 25,600 courses taught by industry experts.

Running pretrained PyTorch models in Rust

Running pretrained PyTorch models in Rust - Rust Tutorial

From the course: Rust LLMOps

Start my 1-month free trial Buy for my team

Running pretrained PyTorch models in Rust

“

- [Instructor] Here we have a pre-trained PyTorch model alongside of a Rust Actix microservice. Let's go ahead and dive into the code here. I've got this running inside of GitHub code spaces and I have this rtorchdist directory. So first thing we'll do is CD into here and we'll get this cooking. Now if we look at this directory here, I like to use make files. And you can see here that if I did basically a make dbuild that would build the docker container. If I said make build, this would actually build the release here. And in this case, let's go ahead and say make build. You can see here that it shows me that I can run this particular command. So let's go ahead and do that. Cargo build - j 16. Perfect, we've already built this previously and so we just checked the binary. Now if I want to run this, I can also just type in cargo run and this is a way to actually run the project as well. So the bill just compiles it, the run will run it. And what's nice about this local binary based…

Contents