Cloud Foundry Summit 2015: Diego Update

2. The Go gopher was designed by Renee French

3. DIEGO what’s new with onsi fakhouri

4. last year…

5. last year… story

6. last year…

7. last year… HOPE

8. last year… HOPEA NEW

9. this year….

10. this year…. …it’s complicated

11. this year…. …but there’s a plot twist

12. this year…. …but there’s a plot twist

13. DIEGOstrikes back

14. ? ?

15. ? ?

16. ? a rewrite RUNTIME

17. ? a rewrite RUNTIME DEA HM Warden

18. ? a rewrite RUNTIME (in GO) DEA HM Warden

19. ? DEAGO

20. ? DEAGO

21. ? DIEGO

22. ? DIEGO is a distributed system that orchestrates containerized workloads

27. ? DIEGO is a distributed system that orchestrates containerized workloads Cells

28. ? DIEGO is a distributed system that orchestrates containerized workloads Cells Brain

29. ? DIEGO is a distributed system that orchestrates containerized workloads Cells Brain BBS (currently etcd)

30. ? DIEGO a distributed system that orchestrates containerized workloads Cells Brain BBS (currently etcd)

31. ? DIEGO a distributed system that orchestrates containerized workloads Cells Brain BBS (currently etcd) scheduler

35. ? DIEGO a distributed system that orchestrates containerized workloads Cells Brain BBS (currently etcd) health-monitor

43. ? DIEGO runs a distributed system that orchestrates containerized workloads

44. ? DIEGO runs a distributed system that orchestrates containerized workloads one-off tasks long running processes

45. ? DIEGO runs a distributed system that orchestrates containerized workloads long running processes Task a unit of work runs at most once

46. ? DIEGO runs a distributed system that orchestrates containerized workloads Task LRP a unit of work runs at most once N long-running instances distributed across cells for HA monitored & restarted

47. ? DIEGO runs a distributed system that orchestrates containerized workloads Task LRP generic, platform independent, abstraction

48. ? DIEGO runs a distributed system that orchestrates containerized workloads Task LRP generic, platform independent, abstraction

49. ? DIEGO runs a distributed system that orchestrates containerized workloads Task LRP working today generic, platform independent, abstraction

50. ? DIEGO runs a distributed system that orchestrates containerized workloads Task LRP successful abstraction working today

51. …confusion

52. …confusion = ?

53. …confusion ? ?

54. …confusion ?

55. …confusion

57. ? ?

58. ? isolation ?

59. ? isolation shared resources processA processB processC processD processE processF kernel tenant 1 tenant 2 tenant 3

60. ? isolation shared resources kernel resource isolation namespace isolation processA processB processC processD processE processF tenant 1 tenant 2 tenant 3

61. ? isolation CPU kernel resource isolation namespace isolation processA processB processC processD processE processF tenant 1 tenant 2 tenant 3

62. ? isolation resource isolation namespace isolation CPU processA processB processC processD processE processF tenant 1 tenant 2 tenant 3

63. ? isolation resource isolation namespace isolation processA processB processC processD processE processF tenant 1 tenant 2 tenant 3 CPU

64. ? isolation resource isolation namespace isolation processA processB processC processD processE processF tenant 1 tenant 2 tenant 3 CPU

65. ? isolation resource isolation namespace isolation processA processB processC processD processE processF tenant 1 tenant 2 tenant 3 cgroups CPU

66. ? isolation resource isolation namespace isolation processA processB processC processD processE processF tenant 1 tenant 2 tenant 3 cgroups processD processE processF CPU

67. ? isolation shared resources kernel resource isolation namespace isolation processA processB processC processD processE processF tenant 1 tenant 2 tenant 3

68. ? isolation kernel resource isolation namespace isolation processA processB processC processD processE processF tenant 1 tenant 2 tenant 3 ProcessID

69. ? isolation resource isolation namespace isolation processA processB processC processD processE processF tenant 1 tenant 2 tenant 3 PID 2 3 4 5 6 7

72. ? isolation resource isolation namespace isolation processA processB processC processD processE processF tenant 1 tenant 2 tenant 3 PID 2 3 4 5 6 7 PID namespace

75. ? isolation resource isolation namespace isolation processA processB processC processD processE processF tenant 1 tenant 2 tenant 3 PID shared resources kernel Network Mount User namespaces

76. ? ?

77. ? ? = isolation User Network cgroups PID

78. ? ? ? = isolation PID User Network cgroups

79. ? ? = isolation PID User Network cgroups + contents

80. ? ? = isolation PID User Network cgroups + contents + processes

81. ? ? =

82. ? Tasks LRPs in

83. ? Tasks LRPs in Garden

84. ? Garden allows Diego to programmatically say “make me a container” “put this in it” “then run this” via a platform-agnostic API

85. ? Garden allows Diego’s abstractions to be flexible

86. ? cf push

87. ? cf push haiku here is my source code i do not care how run it on the cloud for me

88. ? app source code Task staging cf push

89. ? cf push compiled asset app + app-specific dependencies assumes a particular execution context cflinuxfs2

90. ? cf push ?

91. ? cf push LRP

92. ? cf push

93. ? cf push cflinuxfs2 preloaded rootfs

94. ? cf push cflinuxfs2 preloaded rootfs download droplet

95. ? cf push cflinuxfs2 preloaded rootfs download droplet start command

96. ? cf push Droplet LRP { memory: 128mb, rootfs: “preloaded:cflinuxfs2”, setup: <download-droplet>, run: {metadata}.start-command }

97. ? cf push Droplet LRP { memory: 128mb, rootfs: “preloaded:cflinuxfs2”, setup: <download-droplet>, run: {metadata}.start-command }

98. ? cf push { memory: 128mb, rootfs: “preloaded:cflinuxfs2”, setup: <download-droplet>, run: {metadata}.start-command } Droplet LRP

99. ? cf push { memory: 128mb, rootfs: “preloaded:cflinuxfs2”, setup: <download-droplet>, run: {metadata}.start-command } Droplet LRP

100. ? cf push

101. ? cf push-docker

103. ? cf push-docker docker image

104. ? cf push-docker docker image docker metadata

105. ? cf push-docker docker image docker metadata docker registry }

106. ? cf push-docker docker image docker metadata docker registry }

107. ? Docker LRP { memory:128mb, rootfs: “docker://docker-image”, run: {docker metadata}.start- command } cf push-docker

111. ? ?

112. ? ? (anything)

113. ? ? (anything)

114. ? ? (anything) (anything)

115. ? ? (anything) (anything)

116. ? ?

117. ? ? appc

119. ? cf push -stack windows

120. ? Garden-Windows resource isolation kernel job object disk quotas namespace isolation user profiles Host Web Core (an isolated IIS instance) Garden-Linux resource isolation cgroups namespace isolation PID Network User Mount

121. ? collaborating with Microsoft Garden-Windows

122. ? Garden-Windows provides a container experience for Windows 2012 that will only get better with Windows 2016 allows us to build a cf push experience

123. ? Garden-Linux Garden-Windows

124. ? Garden-Linux Garden-Windows ? Garden API

125. ? Garden APIGarden-LinuxGarden-Windows

126. ? .net LRP { memory: 128mb, rootfs: “preloaded:windows2012R2”, setup: <download-application> run: {metadata}.start-command }

131. ? 3 different contexts

132. ? 1 cluster

133. ? ?

134. API

135. API

136. architecture code

137. architecture code tests TDD

138. architecture code tests e.g. performance, simulation

139. architecture code tests e.g. performance, simulation

140. architecture code tests

141. architecture code tests (agile architecture)

142. last year…

143. Cells Brain BBS

144. Distributed auction Cells Brain BBS

145. Distributed auction Cells Brain BBS

146. architecture

147. architecture code

148. architecture tests code

149. architecture simulation code

150. architecture 100 cells code simulation

151. architecture 100 cells 200 cells code simulation

152. architecture code++ 100 cells 200 cells simulation

153. architecture code++ 100 cells 200 cells simulation

154. architecture code++ 100 cells 200 cells 400 cells simulation

155. architecture code++++++ 100 cells 200 cells 400 cells simulation

156. architecture code++++++ 100 cells 200 cells 400 cells simulation

157. architecture code++++++ 100 cells 200 cells 400 cells 1000s cells? simulation

160. Cells Brain BBS Distributed auction

161. Cells Brain BBS Centralized HA scheduler

162. API (it’s just simpler this way)

163. API

164. cf push Cloud Controller

165. Cloud Controller DEA cf push stage DEA DEA DEA run

166. Cloud Controller cf push stage run

167. Cloud Controller cf push stage run app-specific

168. Cloud Controller cf push stage run app-specific generic

169. Cloud Controller cf push stage run CC Bridge app-specific generic task LRP

170. Cloud Controller cf push stage run CC Bridge task LRP Cells BrainBBS app-specific generic

173. (DB != API)

174. Cloud Controller cf push stage run CC Bridge task Cells BrainBBS LRP

175. Cloud Controller cf push stage run CC Bridge Cells BrainBBS task LRP

176. Cloud Controller cf push stage run CC Bridge Cells BrainBBS

177. Cloud Controller cf push stage run CC Bridge Cells BrainBBS ReceptorAPI

178. Cloud Controller cf push stage run CC Bridge Cells BrainBBS ReceptorAPI

179. Cloud Controller CC Bridge Cells BrainBBS ReceptorAPI generic consumer

180. Cloud Controller CC Bridge Cells BrainBBS ReceptorAPI generic consumer

181. Cloud Controller CC Bridge Cells BrainBBS ReceptorAPI generic consumer other consumers?

182. ? ?

183. Cells BrainBBS ReceptorAPI Task or LRP

184. Cells BrainBBS ReceptorAPI Task or LRP meh

185. Cells BrainBBS ReceptorAPI Task or LRP gorouter http traffic

186. Cells BrainBBS ReceptorAPI Task or LRP gorouter http traffic loggregator logs

189. vagrant up

190. vagrant up terraform apply

191. vagrant up terraform apply ltc create <app>

192. lattice.cf

193. lattice.cf Local VM

194. lattice.cf Local VM AWS Digital Ocean Google Cloud Platform OpenStack (thanks!)

195. ? ?Why

196. ? CC UAA Diego Loggregator Gorouter Buildpacks Services BOSH

197. ? CC UAA Diego Loggregator Gorouter Buildpacks Services BOSH

198. ? CC UAA Diego Loggregator Gorouter Buildpacks Services BOSH Diego Loggregator Gorouter

199. ? CC UAA Diego Loggregator Gorouter Buildpacks Services BOSH single-tenant

200. ? CC UAA Diego Loggregator Gorouter Buildpacks* Services BOSH docker single-tenant

201. ? CC UAA Diego Loggregator Gorouter Buildpacks* Services BOSH BYOS docker single-tenant

202. ? CC UAA Diego Loggregator Gorouter Buildpacks* Services BOSH no rolling upgrades BYOS docker single-tenant

203. ? cluster root (now go play)

204. ? ?Why

205. ? …is a useful low-barrier solution to real-world problems …makes exploring Diego easy …is a softer onramp to the CF tech stack …allows us to efficiently prototype new ideas for Diego’s future Lattice…

206. ? ?

207. WHEN?

208. “rewrite the DEA” Diego’s scope is much more than

209. WHEN?

210. Diego is running in production on PWS Managing ~5% of the load Running Pivotal’s internal applications

211. WHEN?

212. Diego is in beta while we validate performance at O(~100s) of cells secure Diego’s internal components

213. WHEN?

214. Start using it alongside the DEAs now and give us feedback

215. WHEN?

216. Diego should be out of beta within Q3 (probably)

217. Then what?

218. Placement Constraints top of backlog post-beta

219. cf ssh <app/index> working now, CLI support on the way shell access, port forwarding, scp

220. TCP Routing kicking off with GE Cloud Foundry and IOT protocol support Atul Kshirsagar - Monday 1:30

221. Private Docker Registry in collaboration with SAP Let Diego Manage your Docker Application Georgi Saber - Tuesday 2:10

222. Support for persistence (a long term goal) The Road to Persistence on Cloud Foundry Diego Caleb Miles & Ted Young - Tuesday 11:20

223. Container-Container networking (a long term goal)

224. Condenser lightweight buildpacks for Lattice

225. ? ?

226. github.com/cloudfoundry-incubator/diego lattice.cf xray.cf open-house: monday@1:30

Cloud Foundry Summit 2015: Diego Update

Recommended

More Related Content

What's hot (20)

Similar to Cloud Foundry Summit 2015: Diego Update (20)

More from VMware Tanzu (20)

Recently uploaded (20)

Cloud Foundry Summit 2015: Diego Update