CLOUD WITH AMAZON

CLOUD WITH AMAZON

2 Amazon Web Services AWS is a collection of remote computing services Elastic Compute Cloud (EC2) provides scalable virtual private servers using Xen. Simple Storage Service (S3) provides Web Service based storage. SimpleDB allows developers to run queries on structured data. It provide "the core functionality of a database." AElastic MapReduce allows developers to easily and cheaply process vast amounts of data. It uses a hosted Hadoop Virtual Private Cloud (VPC) creates a logically isolated set of Amazon EC2 instances which can be connected to an existing network using a VPN And More

3 Amazon’s EC2 Amazon Elastic Compute Cloud (EC2) Web service that provides resizable compute capacity in the cloud An EC2 instance appears physical HW, provides users complete control over nearly entire sw stack, from the kernel upwards Load Variety of operating system Install Custom applications Manage network access permission Run image using as many/few systems as you desire

4 Amazon’s EC2 features Elastic capacity Elastic resource config/reconfig; Elastic num of instances Completely Control Root access/access to console output/data store/ reboot Reliable Multiple locations Elastic IP addresses Secure Firewall config Virtual Private Cloud Performance Auto Scaling Auto local balancing

5 Amazon’s EC2 Instances On-Demand Instances Pay for capacity without long-term commitment Reserved Instances Standard Instances Micro Instances High-Memory Instances High-CPU Instances High-I/O instance High Storage Instances Spot Instances Bit on unused Amazon EC2 capacity, run those instances for as long as their bid exceeds the current Spot Prices

Cloud Computing 6 Amazon’s EC2 Operating STEP 1 Create an Amazon Machine Image (AMI) containing your applications, libraries, data and associated configuration settings. Or use pre-configured, templated images to get up and running immediately. STEP 2 Choose the types of instances and OS, then start, terminate, and monitor as many instances of your AMI as needed, using the web service APIs or the variety of management tools provided. STEP 3 Determine whether you want to run in multiple locations, utilize static IP endpoints, or attach persistent block storage to your instances. NOTE: Pay only for the resources that you actually consume, like instance-hours or data transfer.

7 Amazon’s S3 Amazon Simple Storage Service (S3) Storage for the Internet. Features Unlimited Storage Highly scalable in terms of storage, request rate and concurrent users Reliable Store redundant data in multiple facilities and on multiple devices Secure Flexibility to control who/how/when/where to access the data Performance Choose region to optimize for latency/minimize costs Work with other AWS products EC2/Elastic MR/Amazon Import/Export

8 Example : online photo processing service Photo operation red eye reduction/cropping/customization/re-coloring/teeth whitening, etc Procedure Web server receive request Put request message in the queue Pictures stored in S3 Multiple EC2 instances run photo processing Put back in the queue Return

9 MORE ABOUT AWS

10 Amazon Data solutions RedShift (PostgreSQL) - setup to be able to work with larger data than RDS, optimized over RDS. Relational Database Service(RDS) -- MySQL, Oracl, SQL Server, PostgreSQL DynamoDB – non traditional data base operations (and less scalable "older" Amazon SimpleDB) ElastiCache – cache solution CloudFront – content delivery network Elastic Block Store - backing up instances in an cloud infrastructure as a service for recovery from failure

11 MORE ON USAGE

Self-Scaling Applications End-user requests Load-balancing DNS frontend www www Load monitor www S3 backing store for common data vault To EC2 provisioning system

Self-Scaling Backends Work queue Job launcher To EC2 provisioning system Hadoop master S3 output bucket S3 input bucket (many worker nodes) Data collection processes Front-end nodes

14 CASE STUDIES

15 Case Study 1 Large scale web applications with occasional huge spikes and background processing. Video sharing site Deployment A number of web instances based on demand Table storage for information Many works for processing Blobs storage for large data set

16 Case Study 2 Parallel processing applications Financial modeling at a bank New drug testing simulations in a pharmaceutical company Deployment Web role for access interface Many workers for processing Large data set stored in blobs

17 Case Study 3 Using storage from an on-premises or hosted application Archive old email User log file Deployment Connect on-premises application with Azure

18 Case Study 4 Crawling the web Large web crawl data is stored in S3 Users can submit regular expression to the “search” program – “GTW: grep the web” uses Hadoop to search for data Puts your results in an output bucket and notifies you when it’s ready

20 Case Study 5 - netflix “Netflix runs tens of thousands of AWS EC2 instances, .”

21 Netflix ---develop own management tools “trying to track and manage that number of resources via the AWS Management Console would be unworkable Netflix created its own AWS management tools to manage any of its applications running in AWS. at www.slideshare.net/adrianco/netflix-and-open-source .”

22 Netflix –OSS –management tools Some of it .

23 WAYS TO ACCESS AWS More than the console

24 Accessing AWS AWS Management Console Command-line interface API SDKs (java, python, php, ruby, .net, android, ios,more ) We can use APIs to bridge the gap between hardware and applications. IDE integraton Example Eclipse pluggin develop, debug, integrate, migrate, and deploy Java-based applications that use the AWS resources platform