GRID AND CLOUD COMPUTING Courtesy of Professor Ahmed Ezzat,

GRID AND CLOUD COMPUTING Courtesy of Professor Ahmed Ezzat, Ph.D OpenStack Overview and Introduction http://web.uettaxila.edu.pk/CMS/FALL/2017/teGNCCms/ 1 www.svuca.edu

UNIT 15: OpenStack Overview and Introduction 2

The Story of OpenStack 3

The E-mail That Started It All 4

NASA Nabula Nebula is an open source cloud computing platform that was developed to provide an easily quantifiable and improved alternative to building additional expensive data centers and to provide an easier way for NASA scientists and researchers to share large, complex data sets with external partners and the public. Nebula's high-density architecture allows for a dramatically reduced data center footprint. Each shipping container data center can hold up to 15,000 CPU cores or 15 petabytes (one petabyte equals one million gigabytes), proving 50 percent more energy efficient than traditional data centers. https://www.nasa.gov/open/nebula.htm l 5

The Birth of OpenStack Nebul a 6

COMPANIES 338 15,672 INDIVIDUAL MEMBERS TOTAL DEVELOPERS AVERAGE MONTHLY CONTRIBUTORS 2130 374 COUNTRIES 135 TOP 10 COUNTRIES United States, China, India, Great Britain, France, Russia, Australia, Canada, Japan, Germany CODE CONTRIBUTIONS 115,206 7

What is OpenStack? 8

OpenStack Mission "To produce the ubiquitous Open Source cloud computing platform that will meet the needs of public and private cloud providers regardless of size, by being simple to implement and massively scalable."

OpenStack Founding Principles l l l l l l Apache 2.0 license (OSI), open development process Open design process, 2x year public Design Summits Publicly available open source code repository Open community processes documented and transparent Commitment to drive and adopt open standards Modular design for deployment flexibility via APIs

Gold Members

Community with Broad Commercial Support

OpenStack Release Schedule Essex: April 5, 2012 Diablo: September 22, 2012 Bexar: February 3, 2011 Austin: October 21, 2010 l Nova, Swift l l Initial Release of OpenStack Combining the two projects Nova and Swift l l Cactus: April 15, 2011 Nova, Swift, Glance l Nova, Swift, Glance l Nova, Swift, Glance Nova, Swift, Glance, Horizon, Keystone l OpenStack Compute ready for enterprise private cloud deployments and mid-size service provider deployments Enhanced documentation Easier to install and deploy OpenStack Compute ready for large service provider scale deployments This is the ‘Rackspace-ready’ release; need to communicate Rackspace support and plans for deployment Followed by conference and design summit in Boston in early October

Last Few Releases of OpenStack 16th Release of Openstack, Released on August, 2017 15th Release of Openstack, Released on February, 2017 14th Release of Openstack, Released on October, 2016 13th Release of Openstack, Released on April, 2016 12th Release of Openstack, Released on October, 2015 11th Release of Openstack, Released on April, 2015 10th Release of Openstack, Released on October, 2014 9th Release of Openstack, Released on April, 2014 8th Release of Openstack, Released on October, 2013 7h Release of Openstack, Released on April, 2013 6th Release of Openstack, Released on September, 2012

Why Openstack for Cloud It's Open Source : All of the code for OpenStack is freely available under the Apache 2.0 license. Anyone can run it, build on it, or submit changes back to the project. Who it's for: Enterprises, service providers, government and academic institutions with physical hardware that would like to build a public or private cloud. How it's being used today: Organizations like Cisco WebEx, DreamHost, eBay, The Gap, HP, MercadoLibre, NASA, PayPal, Rackspace and University of Melbourne have deployed OpenStack clouds to achieve control, business agility and cost savings without the licensing fees and terms of proprietary software.

Public Clouds vs. Private Clouds

Private Clouds

Automation and Orchestration of IT Resources 18

19

In a Loosely Coupled Architecture 20

By Leveraging Various Open Source Projects 21

OpenStack Architecture

OpenStack Architecture

Horizon (Dashboard) Horizon is a web-based interface for managing OpenStack services. It provides a graphical user interface for operations such as launching instances, managing networking and setting access controls. Its modular design allows interfacing with other products such as billing, monitoring and additional management tools.

Keystone (Identity) Keystone is the centralized identity service that provides authentication and authorization for other services. Keystone also provides a central catalog of services running in a particular OpenStack cloud. It supports multiple forms of authentication including user name and password credentials, token-based systems, and Amazon Web Services style logins.

Neutron (OpenStack Networking) OpenStack Networking provides connectivity between the interfaces of other OpenStack services, such as Nova. OpenStack Networking is a pluggable architecture, users can create their own networks, control traffic, and connect servers to other networks. A software defined networking service that supports many plugins like Open vSwitch, Cisco UCS/Nexus, QoS etc.

Cinder (Block Storage) Cinder is a service that manages storage volumes for virtual machines. This is persistent block storage for the instances running in Nova. Snapshots can be taken for backing up and data, either for restoring data, or to be used to create new block storage volumes.

Nova (Compute) Compute nodes form the resource core of the OpenStack Compute cloud, providing the processing, memory, network and storage resources to run instances. Nova is a distributed component and interacts with Keystone for authentication, Glance for images and Horizon for web interface. Nova is designed to scale horizontally on standard hardware, downloading images to launch instances as required.

Glance (Image Service) Glance service that acts as a registry for virtual machine images allowing users to copy server images for immediate storage. Images can be used as templates when setting up new servers. Usually the images are stored in the Swift (Object) service.

Swift (Object Storage) Swift service providing object storage which allows users to store and retrieve files. Swift architecture is distributed to allow for horizontal scaling, and to provide redundancy as failure-proofing. Data replication is manage by software, allowing greater scalability and redundancy than dedicated hardware.

Software to provision virtual machines on standard hardware at massive scale OpenStack Compute creating open source software to build public and private clouds Software to reliably store billions of objects distributed across standard hardware OpenStack Object Storage

OpenStack Compute Key Features ReST-based API Asynchronous eventually consistent communication Horizontally and massively scalable Hypervisor agnostic: support for Xen ,XenServer, Hyper-V, KVM, UML and ESX is coming Hardware agnostic: standard hardware, RAID not required

Public Network Server Groups Dual Quad Core RAID 10 Drives 1 GigE Public 1 GigE Private 1 GigE Management Management Example OpenStack Compute Hardware (other models possible) Private Network (intra data center)

User Manager Cloud Controllers: Global state of system, talks to LDAP, OpenStack Object Storage, and compute/storage/network workers through a queue Advanced Technology Attachment over Ethernet / Internet Small Computer Systems Interface ATAoE / iSCSI API: Receives HTTP requests, converts commands to/from API format, and sends requests to cloud controller Host Machines: workers that spawn instances OpenStack Compute Glance: HTTP OpenStack Object Storage for server images

System Components l l l l l l l l l l l l l l API Server: Interface module for command and control requests Designed to be modular to support multiple APIs In current release: OpenStack API, EC2 Compatibility Module Approved blueprint: Open Cloud Computing Interface (OCCI) Message Queue: Broker to handle interactions between services Currently based on RabbitMQ Metadata Storage: ORM Layer using SQLAlchemy for datastore abstraction In current release: MySQL In Diablo: PostgreSQL User Manager: Directory service to store user identities In current release: OpenLDAP, FakeLDAP (with Redis), Database Scheduler: Determines the placement of a new resource requested via the API Modular architecture to allow for optimization Base schedulers included in Bexar: Round-robin, Least busy

System Components (Cont.) l l l l l l l l Compute Worker: Manage compute hosts through commands received on the Message Queue via the API Base features: Run, Terminate, Reboot, Attach/Detach Volume, Get Console Output Network Controller: Manage networking resources on compute hosts through commands received on the Message Queue via the API Support for multiple network models Fixed (Static) IP addresses, VLAN with NAT, DHCP Volume Worker: Interact with iSCSI Targets to manage volumes Base features: Create, Delete, Establish Image Store: Manage and deploy VM images to host machines

New Features in Diablo and Beyond l l l l l l l Quantum: Networking as a Service Developed in the open by Cisco, Nicira, others Burrow: HTTP-based message queue Red Dwarf: Database as a Service Keystone: Integrated, pluggable auth for all OpenStack components Lunr: Volumes as a Service Dashboard: Control nova and other OpenStack components via web

Object Storage Summary 38

Object Storage Key Features Data distributed evenly throughout system REpresentational State Transfer ReSTbased API No central database Scalable to multiple petabytes, billions of objects Hardware agnostic: standard hardware, RAID not required 39

To Load Balancers Example OpenStack Object Storage Hardware Proxies 5 Zones 2 Proxies per 25 Storage Nodes 10 GigE to Proxies 1 GigE to Storage Nodes 24 x 2TB Drives per Storage Node Example Large Scale Deployment -- Many Configs Possible 40

System Components l l l l Proxy Server: Request routing, exposes the public API Replication: Keep the system consistent, handle failures Updaters: Process failed or queued updates Auditors: Verify integrity of objects, containers, and accounts 41

System Components (Cont.) l l l l Account Server: Handles listing of containers, stores as SQLite DB Container Server: Handles listing of objects, stores as SQLite DB Object Server: Blob storage server, metadata kept in xattrs, data in binary format Object location based on hash of name & timestamp 42

Evolution of Object Storage Architecture Version 1: Central DB (Rackspace Cloud Files 2008) Version 2: Fully Distributed (OpenStack Object Storage 2010) 43

Example Small Scale Deployment 44

Learning OpenStack 45

Reading About OpenStack The OpenStack Foundation http://www.openstack.org/ Official OpenStack Documentation http://docs.openstack.org/ The OpenStack Cloud Computing Cookbook (Second Edition) http://www.amazon.com/OpenStack-Cloud-Computing-Cookbook-Jackson/dp/1782167587/ref sr 1 1?s books&ie UTF8&qid 1382033707&sr 11 46

Trying Out OpenStack TryStack (OpenStack Sandbox) http://trystack.org/ OpenStack-based Public Clouds DreamHost http://dreamhost.com/cloud/ HP Public Cloud https://www.hpcloud.com/ Rackspace Public Cloud http://www.rackspace.com/cloud/ 47

Deploying OpenStack OpenStack Distributions Red Hat - http://openstack.redhat.com/ SUSE - https://www.suse.com/products/suse-cloud/ Ubuntu - http://www.ubuntu.com/cloud Packaged Deploys For Different Linux Distros Mirantis - https://fuel.mirantis.com/ Piston Cloud Computing - http://www.pistoncloud.com/openstack-cloud-software/ Rackspace - http://www.rackspace.com/cloud/private/openstack software/ Configuration Management Tools Opscode Chef - https://github.com/opscode/openstack-chef-repo/ Puppet Labs Puppet -http://puppetlabs.com/solutions/cloud-automation/compute/openstack 48

Join the Community Join The OpenStack Community http://www.openstack.org/community/ 49

Assignment #14 What are the names of Service Projects included in latest version of OpenStack (Pike)? 50

Thank You Questions and Comments? http://web.uettaxila.edu.pk/CMS/FALL/2017/teGNCCms/ 51