Q&A: Pulumi’s Joe Duffy on the Renaissance of Infrastructure as Code
Infrastructure as Code (IaC) is hot again. Organizations are realizing that their complex system deployments should be repeatable, in that the blueprints they make for such systems should be expressed as code, so that rolling out, or updating, a system could be automated, saving time and accruing benefits in security, compliance and resilience.
Recently, we caught up with one of the pioneers of this space, Pulumi co-founder and CEO Joe Duffy. We discussed the use of AI, platform engineering, using IaC for running CI/CD systems, and the recent controversy around HashiCorp’s relicensing of the widely-used IaC tool Terraform.
This interview has been edited for clarity and brevity.
How are things with Pulumi?
So things are going really well for us just in terms of market adoption. We’re seeing a lot of businesses want to empower developers more and Pulumi is great for that, because you get to use your favorite language. But also [we see success] with complexity at scale, especially with AI workloads: It’s infinite compute and infinite data. It’s really tough to manage infrastructure at that level of scale. And Pulumi, that’s our forte.
We’ve noticed a surge of interest on Thenewstack.io around Infrastructure-as-Code in the last year, and not just from the recent controversy around HashiCorp and the OpenTofu fork of Terraform…
We hear from our customers that infrastructure-as-code is table stakes all the time now, and that was not the case five years ago. Chef, Puppet, [Red Hat’s] Ansible, SaltStack was the first wave of Infrastructure as Code. And I feel like it’s kind of gone through a renaissance now.
Unlike those IaC tool providers, Pulumi is unique in that the user can define an environment using a regular programming language (such as Python, Java or TypeScript), rather than learning a proprietary language. What was the thinking behind that?
Programming languages were invented to help with complexity in software: to build bigger things out of smaller things to encapsulate complexity, and sharing and reuse common practices.
And I think with the modern world of infrastructure, there’s so many moving pieces. Like if you’re using Kubernetes, or using serverless, you’ve gone from using two virtual machines and a database, which is kind of what you had in Chef. Now if you’re using cloud native stuff, it’s a single environment with probably hundreds of resources you have to manage. And that’s exactly what a programming language is really good at.
I went on the road this year to visit a lot of customers, like British Telecom, Lego, BMW, those sorts of folks. And what struck me is a lot of companies are trying to build the same thing using Infrastructures as Code. They’re trying to build an internal platform for how they deliver cloud applications and infrastructure. Because there’s a lot of people recreating the wheel.
And that’s an area that Pulumi is really focused on: Let’s take those common patterns and make them just work out of the box rather than having to build everything from scratch. But I think our unique approach with engineering and languages really helps with those use cases as well.
It seems like a lot of ask for a system administrator to learn a new programming language just to deploy some resources.
I think the cloud has gone well beyond admin work at this point. The cloud is central to modern software being able to do amazing things. OpenAI would not exist if it weren’t for the fact that they can scale up infinite GPUs in the cloud`.
And I think that’s the shift that we’ve gone through over the last 10 years, from IaC being an administrative task and a cost center, to actually adding value to the business itself. And I think a lot of the folks that Pulumi does well with are people who recognize the cloud as being an accelerant for their business.
How do organizations get started with Pulumi?
Everybody does it differently. A lot of startups and AI companies we work with, they’re like, ‘We know Python, we just need to spin up infrastructure, our developers will do it.’ And then at the Fortune 500 companies, the CIO or CTO has a top-level line item of empowering developers to be self-service. But it has to be done differently. It needs to be done with guardrails for security, compliance and cost. So many of those cases they might prefer a developer portal, where it’s a point-and-click experience versus developers writing infrastructure code. So we see every point along that spectrum.
You recently announced a Backstage plug-in for Pulumi. Backstage is the open source developer portal that many businesses are using to build their internal developer platform. But I never thought of Pulumi as being in the Platform Engineering space. What is going on there?
So for companies that have decided that Backstage will be their internal portal, we want to plug right into that, and allow you to bring the infrastructure patterns and practices into the Backstage context.
We also find that for a lot of businesses Backstage may be overkill. They really just want a developer portal, a service catalog experience. Pulumi Cloud now has that capability. We always want to integrate with what people are already using, rather than trying to boil the ocean. And so we’ve done that.
So Pulumi Cloud is a lightweight alternative to Backstage?
Backstage offers a subset of what our Pulumi Cloud. It also does things like policy-as-code, analytics and insights of your infrastructure usage.
They’re better together. But if you only want the service catalog, and point-to-click provisioning of a cloud application infrastructure, then technically you don’t need Backstage. But they are better together.
I did not know that you had a tool to convert IaC code written in HCL (Hashicorp Configuration Language). Could you tell me about that?
We launched our Migration Hub recently where we will actually convert a customer’s Terraform for them. But we also do have this tool that allows you to do it yourself.
We actually had an incubation project for a while, but we recently we redid the whole thing, rebuilt it from scratch, and made it so it not only converts your HCL code but also adopts infrastructure on the fly. One day, you come into the office, and you got Terraform, and by the end of the day when you leave and it’s all under Pulumi. You’ve never had to rip down any infrastructure. There’s no impact your cloud applications. So we want to make it just super easy.
What we find is a lot of people love the idea of Pulumi, but they wonder how to get started. And so that’s the intent with this, to make it a lot easier.
So you have been seeing users want to migrate from Terraform?
Yes, absolutely. Yeah, quite a bit. In fact, you know, the Terraform relicense significantly increased interest in Pulumi from existing Terraform users. Because a lot of people are nervous about the business source license change.
For those existing Pulumi users, can you tell us what the company is working on now?
There’s a lot of bread and butter, just making sure we’re the best infrastructure as code tool. We’re easy to use, with all the examples and documentation that you need. That’s our core focus.
We’ve started to integrate AI across the board. And that’s, that’s really helping to basically take a fresh approach to some tried-and-true hard problems.
We launched this AI assistant recently, it’s just gotten so much better, even since we launched it. You say, ‘I want this cloud architecture, with a Kubernetes cluster, and NGINX controller and a CloudFlare CDN and A Snowflake database, and it will just like, explain it to you and give you the code. So check that out.
It writes the code. it can explain the code and refactor existing code. That’s actually the hardest part. With Pulumi, you can use a great programming language like Python, but you still have to understand the cloud architecture. And that can be daunting. There’s thousands of services across hundreds of clouds, and this thing understands all of it.
We find today, it’s still [a case of] you draw on the whiteboard that you want a little microservice with a database, and then you go to encode that as infrastructure code, and find you have to understand the entirety of AWS and Azure or Google Cloud. So it’s not as easy as it should be.
But going beyond that, [we want to] help platform teams with all the challenges they see at scale, and that goes beyond just infrastructures code. There’s a lot of other continuous delivery workflows can be made more secure and cost-effective. How do you stay on top of a team of 1,000 people deploying infrastructure every day? That’s really tough.
So CI/CD can be used as a possible use case for Infrastructure as Code. Please address this point.
Most customers we have worked with have figured out continuous integration for applications, maybe using GitHub Actions or GitLab pipelines.
People are coming from a world of thinking that infrastructure delivery and application delivery are totally just disparate things, and they are coming to realize that they need to happen in tandem, in harmony with one another. So that’s where Pulumi can come in. If you use GitHub Actions for application delivery, you can use it for infrastructure as well.
