An open-source alternative to Kubernetes and Slurm for AI container orchestration, supporting NVIDIA, AMD, TPU, and Intel across cloud and on-prem.
Externer Link zu dstack
Munich, DE
dstack hat dies direkt geteilt
dstack 0.20.22 is out 🚀 Tenstorrent Blackhole support has landed. Run dev, training, and inference workloads on Blackhole PCIe cards and systems. Thanks to the Tenstorrent team and Artem Y. for collaborating on this. https://lnkd.in/dYvY-KQh
dstack hat dies direkt geteilt
Training models or serving inference on AMD GPUs? We’ve refreshed the AMD accelerator example in the dstack docs, covering on-prem fleets, cloud GPU provisioning, dev environments, training jobs, and production-grade inference. https://lnkd.in/dn6cR93h
dstack hat dies direkt geteilt
Most GPU workflows come with too much infra overhead. Spin up a VM, install dependencies, copy code over, expose ports, watch logs, and remember to shut everything down after. With JarvisLabs now available as a dstack backend, most of that goes away. You define the machine and the workload in a YAML file. Run dstack apply. That's it. dstack manages the infrastructure lifecycle. JarvisLabs provides the GPUs. We wrote a short tutorial covering the full setup, from connecting your account to running a nanochat training job on H100s Works for training runs, evals, benchmarks, inference services, and GPU dev environments.
dstack 0.20.22 is out 🚀 Tenstorrent Blackhole support has landed. Run dev, training, and inference workloads on Blackhole PCIe cards and systems. Thanks to the Tenstorrent team and Artem Y. for collaborating on this. https://lnkd.in/dYvY-KQh
dstack hat dies direkt geteilt
https://lnkd.in/g8eRFnmv Great job dstack and Matt Bettinger!!!
Training models or serving inference on AMD GPUs? We’ve refreshed the AMD accelerator example in the dstack docs, covering on-prem fleets, cloud GPU provisioning, dev environments, training jobs, and production-grade inference. https://lnkd.in/dn6cR93h
dstack hat dies direkt geteilt
dstack 0.20.21 is out 🚀 ✨ This release adds support for multiple Kubernetes clusters in a single Kubernetes backend. Each kubeconfig context can now be exposed as a separate backend region, making it easier to use dstack across multiple clusters. ✨ We’ve also added a new JarvisLabs.ai backend. You can now provision H100/H200 GPUs on demand via JarvisLabs, with RTX PRO 6000 support to follow. Full release notes: https://lnkd.in/dEuB2TNA
The community asked us for an example of how to use RadixArk Miles with dstack for RL training. Since Miles uses Ray and dstack can run Ray, using Miles with dstack is quite straightforward. Here’s a new example of running Miles on a multi-node cluster provisioned and managed by dstack: https://lnkd.in/d69mkrpZ
dstack hat dies direkt geteilt
JarvisLabs.ai is now a dstack backend. That means: if you've been using dstack to orchestrate runs across GPU clouds, you can now point it at our infrastructure with one YAML block. type: jarvislabs creds: api_key: ... One YAML. One command. GPU running in few seconds. L4, H100, and CPU VMs, more on the way. dstack handles the instance lifecycle. We provide the GPU. If you've been waiting for it, give it a spin.
dstack hat dies direkt geteilt
Deploying production-grade LLM inference on AMD GPU clusters is getting easier. We published a walkthrough on deploying PD-disaggregated inference on multi-node AMD clusters using dstack and SGLang, with LightSeek Foundation's Shepherd Model Gateway (SMG) for routing and Mooncake for KV cache transfer over RoCE. With one dstack service configuration, you can deploy an autoscaling inference endpoint across cloud or on-prem clusters, while keeping the router, prefill, and decode workers orchestrated as part of a single service. Read the post: https://lnkd.in/dqi6CDnc