Prying the Cloud Open: Dell Crowbar & OpenStack Rob Hirschfeld

Prying the Cloud Open: Dell Crowbar & OpenStack Rob Hirschfeld (@zehicle), Principal Cloud Architect Joseph B. George (@jbgeorge), Senior Cloud Strategist July 2011, OSCON

Open Cloud Operations Intersection of two major technology waves: Operational Automation (DevOps) Open Source Cloud (OpenStack) Together, a practical approach to cloud scale Manages elastic resources Embraces constant change Productizes best practices Dell Cloud Solutions 2 July 20, 2011

Lessons from Hyper-scale Hyper-scale (1000s of servers) forced customers to change their core assumptions. To operate large systems, customers Embraced scale out design (inconsistency is OK) Eliminated hardware redundancy (too expensive) Shunned manual steps (too slow) Dell Cloud Solutions 3 July 20, 2011

Lessons from Public Clouds Cloud servers forced customers to change their core assumptions. To operate remote virtualized systems, customers Embraced scale out design (to get elastic growth) Lived with weak SLAs (that was all that was offered) Worked around lack of physical access (not available) Dell Cloud Solutions 4 June 12, 2024

Clouds require an Operational Focus Clouds demand significant operational and process controls SW OPS HW Dell Cloud Solutions Operational decisions drive hardware and software decisions We are finding ways to productize operations into best practices 5 July 20, 2011

DevOps Addresses Challenges DevOps is an operational approach that automates system configuration and management. To manage cloud systems, customers Need to manage servers as groups Must respond to rapid infrastructure changes Have repeatable automated deployments Dell Cloud Solutions 6 June 12, 2024

Missing Cloud Infrastructure The cloud market was highly fragmented Amazon dominated but was not public domain VMware was building on their enterprise base Microsoft was taking the PaaS route Many small players but no community projects Service Providers could not agree on APIs Dell Cloud Solutions 7 July 20, 2011

Community Cloud Infrastructure We needed Cloud Infrastructure that Had support from major industry players Was collaboratively developed without a single owner Had an API that was Service Provider license friendly Could be demonstrated to run at scale Was built on open source components Had global reach and support Dell Cloud Solutions 8 July 20, 2011

OpenStack! Dell Cloud Solutions 9 July 20, 2011

Making an OpenStack Cloud ? Operations SW OPS HW ? Expertise Resources ? Dell Cloud Solutions 10 July 20, 2011

Dell OpenStack Cloud Solution Reference Architecture Dell Cloud Solutions SW SW “Crowbar” CloudOps Software OPS OPS HW 11 Services & Consulting July 20, 2011

TECHNICAL CONTENT AHEAD How DevOps is Different Images vs. Layers Dell Cloud Solutions 12 July 20, 2011

Images vs. Layers: Overview Images: Single Unit Integrations Configuration Application Foo Integrations Applications Utilities Operating System Dell Cloud Solutions Layers: Stacked Pieces Application Bar Utilities Operating System 13 July 20, 2011

Images vs. Layers: Lifecycle Images: Replacement Config Config Config I A U O/S I A U O/S I A U O/S Layers: Upgrade I Foo Bar v1 U OS Config I Foo Bar v2 U OS Bar v2 I A U O/S Dell Cloud Solutions 14 July 20, 2011

Images vs. Layers: Details Images Layers Like a Compiled Language Early Binding Tightly Coupled Assembled as a package 1-Step Deployment Big Storage Footprint Cloned from “Golden Image” Single Version Dell Cloud Solutions 15 Like an Interpreted Language Late Binding Loosely Coupled Assembled in parts Multi-Step Deployment Small Footprint (packages) Setup per instance Discrete Versions July 20, 2011

Images vs. Layers: Mo’ Details Images Layers Very Static Operations Manual Setup Low Entry / High Maintenance High Change Operations Automated setup High Entry / Low Maintenance Works well in Enterprise Hardware Sensitive Brittle Typically: Windows Works well in Cloud Hardware Agnostic Elastic Typically: Linux Dell Cloud Solutions 16 July 20, 2011

Images vs. Layers Soup Sandwich Delicious! Fast! Delicious! Fast! Premixed Hard to customize Hard to change Hard to carry Components can be common Every batch is unique Dell Cloud Solutions Flexible Easy to customize Configurable Very portable 17 July 20, 2011

Cloud Deployments are Complex Crowbar has scripts that install the components of OpenStack Nova in a proven pattern that can be customized after the fact. Source: Ken Pepple Confidential 18 06/12/24

CloudOps for OpenStack There are lot of pieces, but they share many elements! I Crowbar Chef U OS I Nova Node U OS Dell Cloud Solutions I U OS I Dash Board U OS I Nova Cntrl U OS I Nova DB U OS I Nova API U OS I Nova Node U OS I Nova Node U OS I Nova Node U OS I Nova Node U OS I Nova Node U OS Nagios 19 July 20, 2011

CloudOps for OpenStack Plus interconnections are highly complex and dynamic I Crowbar Chef U OS I Nova Node U OS Dell Cloud Solutions I U OS I Dash Board U OS I Nova Cntrl U OS I Nova DB U OS I Nova API U OS I Nova Node U OS I Nova Node U OS I Nova Node U OS I Nova Node U OS I Nova Node U OS Nagios 20 July 20, 2011

No soup for you! Confidential 21 06/12/24

WARNING: TECHNICAL CONTENT AHEAD Crowbar Deep Dive Dell Cloud Solutions 22 July 20, 2011

What is Crowbar? Mission: “A Zero Touch Cloud Installer” Servers in boxes to full function cloud in under 2 hours Fast & Flexible Bare metal install including BIOS & RAID config Users can choose how their system is configured (“barclamps”) DevOps Embracing Ongoing Operations Model (DevOps for Clouds) Leverages & Wraps Opscode Chef Open Not specific to OpenStack – Dell using for other Apps Not restricted to Dell hardware Apache 2 licensed Dell Cloud Solutions 23 July 20, 2011

Crowbar Stages of Deployment APIs, User Access, & Ecosystem Partners Cloud Infrastructure & Dell IP Extensions Includes all the components required to implement an entire cloud infrastructure including ecosystems partners Pluggable components deploy cloud infrastructure. Allow for addition of Dell IP and expansion by the community services and customers Core Components & Operating Systems Physical Resources Can integrate with Dell existing products Delivers basic data center services and required cloud infrastructure. Provision bare-metal servers from box to cloud WITHOUT user intervention (other than racking/cabling and some minimal configuration questions) NOT EXCLUSIVE TO DELL HARDWARE * Crowbar has potential to service other programs beyond OpenStack Dell Cloud Solutions 24 July 20, 2011

Crowbar Install State Machine Dell Cloud Solutions 25 July 20, 2011

Modular Design: Barclamps APIs, User Access, & Ecosystem Partners Cloud Infrastructure & Dell IP Extensions Core Components & Operating Systems Physical Resources Nagios Ganglia Dashboard Nova Swift Glance Compellen t EqualLogic Crowbar DNS Logging Deployer NTP Provisioner BIOS Network RAID IPMI * Crowbar has potential to service other programs beyond OpenStack Dell Cloud Solutions 26 July 20, 2011

Live Demo (placeholder) Dell Cloud Solutions 27 July 20, 2011

Live Demo (placeholder) Dell Cloud Solutions 28 July 20, 2011

In Conclusion, Dell Cloud Solutions 29 July 20, 2011

Open Cloud Operations Intersection of two major technology waves: Operational Automation (DevOps) Open Source Cloud (OpenStack) Together, a practical approach to cloud scale Manages elastic resources Embraces constant change Productizes best practices Dell Cloud Solutions 30 July 20, 2011

OpenStack Crowbar Dell Cloud Solutions 31 July 20, 2011

Questions? More info: http://Dell.com/OpenStack - White Papers & More! http://RobHirschfeld.com – Technical & Agile http://JBGeorge.net – Business & Marketing