The deep learning software trilemma - PyTorch Tutorial

And the last bit of just kind of call it the deep learning software ecosystem is similar to the other trilemma for generative modeling is the deep learning software trilemma. I show here the four, I would say, main deep learning libraries. There is JAX, which is a newer library out of Google. There's TensorFlow and Keras, which Keras just had a new version come out, Keras 3.0. And Keras 3.0 is really interesting, because it essentially is a complete rewrite of the library to be backend framework agnostic. Keras used to be really just a more user-friendly layer built on top of TensorFlow. But it has sense in this new rewrite, just like with different PyTorch backends of running PyTorch with the CPU versus the GPU, Keras can actually run using Jax, PyTorch, or TensorFlow kind of seamlessly. So if you're building a new model, or you're doing a new architecture, and you want to leverage the community from all three of these other frameworks, Keras is kind of this unifying abstraction on top of these three. But that being said, I still, at the end of the day, really love PyTorch because it exists kind of in the in-between of all of these aspects. So when you are considering what type of library or framework do I want to do deep learning There's kind of these three aspects to consider of. So, how productionizable is the model or the code? How easy is it to deploy that model to different devices, maybe embedded systems, the web, your server, your data center, what have you? How easy is it to build new models? How developer-friendly is it? And kind of how flexible or cutting-edge is the library? So JAX is really kind of the next-gen library of all libraries listed here. It's incorporated a lot of interesting new research around actually how do we build these performance scalable frameworks. And it's also super flexible. It's much closer to kind of pure NumPy type semantics. It's a little bit lower level than these other four libraries. But that being said, it's a little harder to use. It's squarely in this research point of the trilemma. Keras is interesting because it's the most similar alternative to PyTorch, but it's really developed. If it has one guiding principle to its development, it's make it as easy or user-friendly as possible. That being said, it can be a little bit hard to customize or make things flexible or deploy things, in a sense. And TensorFlow, the dominant library, probably the most commonly used library. But it can be a little bit hard to use, since it's pretty far away from this ergonomic side of things. And PyTorch is this sweet Goldilocks zone of existing in between all of these, in that it's easy enough to develop in. It's developer-friendly enough. But it's also flexible enough. And it has a pretty easy path to deployment. So again, that's why we, and we being me and you in this course and these lessons, have settled on PyTorch, the Hugging Face community. With the libraries, there are certain models that have backends in TensorFlow. They have backends in JAX. But for the most part, the HuggingFace libraries, both diffusers and transformers, are built with PyTorch. And most of the new models are PyTorch first. And then maybe they get ported to TensorFlow or JAX with HuggingFace. But all that being said, all of these libraries are more similar than they're different. I said Keras is maybe too high level and it's not flexible enough. Even though it has a lower level API, it's pretty easy to deploy things with Keras. Take everything with this deep learning software trilemma with a grain of salt. I know if I tweeted this out, There's probably people who would think I'm very wrong or hate on me for putting Keras in this corner that's only ergonomics or putting TensorFlow in the only production corner. This is the same kind of argument of, is Python or R better for data science or statistics argument? They're both good in different ways. And like that, all four of these libraries, they're all good. They all have their niches, but they're all general enough that you can do the same thing in either of these. I just wanted to present this as a way, if you had to force me to categorize all of the deep learning libraries, this is what I would try, this is my attempt at. But with that, we're ready to dive in and get our hands dirty with some PyTorch core code.