Transcript

How do you access generative AI capabilities when you can't reach the Internet? Maybe you're deployed out at the tactical edge without a reliable connection, or maybe you're locked away in a windowless SCIF toiling away on a highly classified air gap network. Been there, done that. But when you get home from work and have your personal computer, you can access and use tools like ChatGPT and Google Bard whenever you want to. The reason you can't use those at work is because those services run in the cloud and have Internet-facing APIs. You might be wondering, can't you just install ChatGPT locally to take it wherever you go? Unfortunately, working with modern LLMs isn't that simple. For one, models like GPT require huge amounts of memory and computational resources. A laptop like this ain't going to handle that. Also, getting those limbs up and running isn't as simple as downloading an app from the App Store. They require additional support infrastructure to host and interface with the models. Hi, I'm Barron Stone, and I'm an AI product manager with Defense Unicorns. We are a modern defense company doing our part to transform how the US defense apparatus buys, builds, delivers, and sustains mission software and AI capabilities to get them into the hands of our war fighters when and where they need them getting AI capabilities. Into the types of systems where the DoD tends to operate is hard. I already mentioned the challenges with disconnected systems, but even if you do have a connection to the Internet, you might not always be allowed to share your sensitive data with a third party service provider. And this is a concern that extends well beyond just the defense sector. It affects other parts of our government and even commercial sectors that deal with sensitive data like finance and healthcare. The team at Defense Unicorns saw this challenge as a technical problem in need of a technical solution. So we built LeapfrogAI. LeapfrogAI is a suite of tools for deploying, operating, and running generative AI capabilities in national security environments which are often resource constrained and egress limited. It gives you the ability to self host LLMs so you can maintain full control over your data to ensure its privacy and security. That data independence makes LeapfrogAI different from many other AI services because there's no need to transfer your data to a third party system. LeapfrogAI provides an API that closely mirrors that of Open AI. That means if you have tools that were built to work with that API, they should function seamlessly using Leapfrog as the back end when fully disconnected from the Internet. Other LeapfrogAI features include a vector database service which enables efficient similarity searches on large scale databases, and generative embeddings, which can be used for semantic similarity clustering and more. Under the hood, LeapfrogAI provides multiple backends to choose from. Including HuggingFace Transformers and C Transformers. Having multiple back end options differentiates leapfrog from many other solutions because that enables it to scale and fit your mission environment. If you have a massive system with tons of RAM and GPUs, you can scale up to use full size models with 10s of billions of parameters. On the other hand, if you're in a resource constrained environment, you can scale down by using C Transformers to run a smaller LLM with four bit quantization. Purely on a CPU, Leapfrog AI currently supports several modalities, with the most common being text-to-text LLMs. You can connect these models with your organization's data to build context aware chat bots to assist with daily tasks. They can also summarize large amounts of information for easy consumption and automate generating documentation to speed up processes. LeapfrogAI also supports Speech to text models which you can use to transcribe audio files into text, and it supports. Text to vector models, which can be used to create embeddings for retrieval augmented generation. That's what LeapfrogAI can do today as of September 2023, but it's still under active development by Defense Unicorns and will continue to add support for more modalities such as text-to-image. LeapfrogAI is available today, and you can download and begin using it for free from our GitHub repository. We also encourage you to engage with our growing community of LeapfrogAI users and collaborators. Which includes organizations from the US Navy, Air Force, and Space Force. Now, I've just said a lot of things about what LeapfrogAI can do, but I also want to acknowledge that LeapfrogAI itself is not a complete end to end solution. It's a foundational platform for hosting AI models. But that platform and the models are just part of the total puzzle. You'll still need to connect it to your mission data and related applications. LeapfrogAI has an open architecture that enables mission specific integration and customization. You can choose to perform that integration yourself, or you can work with our team of AI experts to support your mission.