Python: An Introduction Shubin Liu, Ph.D. Research Computing

Python: An Introduction Shubin Liu, Ph.D. Research Computing Center University of North Carolina at Chapel Hill

Agenda Introduction Running Python Python Programming Data types Control flows Classes, functions, modules Hands-on Exercises The PPT/WORD format of this presentation is available here: http://its2.unc.edu/divisions/rc/training/scientific/ /afs/isis/depts/its/public html/divisions/rc/training/scientific/short courses/ its.unc.edu 2

Course Goals To understand the basic structure and syntax of Python programming language To learn how to run Python scripts on our research computing facility, the Emerald Linux cluster To write your own simple Python scripts. To serve as the starting point for more advanced training on Python coding its.unc.edu 3

What is python? its.unc.edu Object oriented language Interpreted language Supports dynamic data type Independent from platforms Focused on development time Simple and easy grammar High-level internal object data types Automatic memory management It’s free (open source)! 4

Timeline Python born, name picked - Dec 1989 By Guido van Rossum, now at GOOGLE First public release (USENET) - Feb 1991 python.org website - 1996 or 1997 2.0 released - 2000 Python Software Foundation - 2001 2.4 released - 2004 2.5 released – 2006 Current version: 2.6.x its.unc.edu 5

Language properties Everything is an object Modules, classes, functions Exception handling Dynamic typing, polymorphism Static scoping Operator overloading Indentation for block structure its.unc.edu 6

High-level data types Numbers: int, long, float, complex Strings: immutable Lists and dictionaries: containers Other types for e.g. binary data, regular expressions, introspection Extension modules can define new “built-in” data types its.unc.edu 7

Why learn python? Fun-to-use "Scripting language" Object-oriented Highly educational Very easy to learn Powerful, scalable, easy to maintain high productivity Lots of libraries Glue language Interactive front-end for FORTRAN/C/C code its.unc.edu 8

Why learn python? (cont.) Reduce development time Reduce code length Easy to learn and use as developers Easy to understand codes Easy to do team projects Easy to extend to other languages its.unc.edu 9

Where to use python? its.unc.edu System management (i.e., scripting) Graphic User Interface (GUI) Internet programming Database (DB) programming Text data processing Distributed processing Numerical operations Graphics And so on 10

Python vs. Perl Easier to learn important for occasional users More readable code improved code maintenance Fewer “magical” side effects More “safety” guarantees Better Java integration Less Unix bias its.unc.edu 11

Python vs. Java Code 5-10 times more concise Dynamic typing Much quicker development no compilation phase less typing Yes, it runs slower but development is so much faster! Similar (but more so) for C/C Use Python with Java: JPython! its.unc.edu 12

Agenda Introduction Running Python Python Programming Data types Control flows Classes, functions, modules Hands-on Exercises The PPT/WORD format of this presentation is available here: http://its2.unc.edu/divisions/rc/training/scientific/ /afs/isis/depts/its/public html/divisions/rc/training/scientific/short courses/ its.unc.edu 13

Running Python Interactively Start python by typing "python" /afs/isis/pkg/isis/bin/python D (control-D) exits % python D % Comments start with ‘#’ 2 2 #Comment on the same line as text 4 7/3 #Numbers are integers by default 2 x y z 0 #Multiple assigns at once z 0 its.unc.edu 14

Running Python Programs In general % python ./myprogram.py Can also create executable scripts Compose the code in an editor like vi/emacs % vi ./myprogram.py # Python scripts with the suffix .py. Then you can just type the script name to execute % python ./myprogram.py The first line of the program tells the OS how to execute it: #! /afs/isis/pkg/isis/bin/python Make the file executable: % chmod x ./myprogram.py Then you can just type the script name to execute % ./myprogram.py its.unc.edu 15

Running Python Programs Interactively Suppose the file script.py contains the following lines: print 'Hello world' x [0,1,2] Let's run this script in each of the ways described on the last slide: python -i script.py Hello world x [0,1,2] python execfile('script.py') x [0,1,2] its.unc.edu 16

Running Python Programs Interactively Suppose the file script.py contains the following lines: print 'Hello world' x [0,1,2] Let's run this script in each of the ways described on the last slide: python import script # DO NOT add the .py suffix. Script is a module here x Traceback (most recent call last): File " stdin ", line 1, in ? NameError: name 'x' is not defined script.x # to make use of x, we need to let Python know which #module it came from, i.e. give Python its context [0,1,2] its.unc.edu 17

Running Python Programs Interactively # Pretend that script.py contains multiple stored quantities. To promote x(and only x) to the top level context, type the following: python from script import x Hello world x [0,1,2] # To promote all quantities in script.py to the top level context, type from script import * into the interpreter. Of course, if that's what you want, you might as well type python -i script.py into the terminal. from script import * its.unc.edu 18

File naming conventions python files usually end with the suffix .py but executable files usually don’t have the .py extension modules (later) should always have the .py extension its.unc.edu 19

Comments Start with # and go to end of line What about C, C style comments? NOT supported! its.unc.edu 20

Agenda Introduction Running Python Python Programming Data types Control flows Classes, functions, modules Hands-on Exercises The PPT/WORD format of this presentation is available here: http://its2.unc.edu/divisions/rc/training/scientific/ /afs/isis/depts/its/public html/divisions/rc/training/scientific/short courses/ its.unc.edu 21

Python Syntax Much of it is similar to C syntax Exceptions: missing operators: , - no curly brackets,{}, for blocks; uses whitespace different keywords lots of extra features no type declarations! its.unc.edu 22

Simple data types Numbers Integer, floating-point, complex! Strings characters are strings of length 1 Booleans are False or True its.unc.edu 23

Numbers The usual notations and operators 12, 3.14, 0xFF, 0377, (-1 2)*3/4**5, abs(x), 0 x 5 C-style shifting & masking 1 16, x&0xff, x 1, x, x y Integer division truncates :-( 1/2 - 0 # float(1)/2 - 0.5 Long (arbitrary precision), complex 2L**100 - 1267650600228229401496703205376L 1j**2 - (-1 0j) its.unc.edu 24

Strings and formatting i 10 d 3.1415926 s "I am a string!" print "%d\t%f\t%s" % (i, d, s) print “newline\n" print "no newline" its.unc.edu 25

Variables No need to declare Need to assign (initialize) use of uninitialized variable raises exception Not typed if friendly: greeting "hello world" else: greeting 12**2 print greeting Everything is a variable: functions, modules, classes its.unc.edu 26

Reference semantics Assignment manipulates references x y does not make a copy of y x y makes x reference the object y references Very useful; but beware! Example: a [1, 2, 3]; b a a.append(4); print b [1, 2, 3, 4] its.unc.edu 27

Simple data types: operators - * / % (like C) - etc. (no or --) Assignment using but semantics are different! a 1 a "foo" # OK Can also use to concatenate strings its.unc.edu 28

Strings "hello" "world" "helloworld" # concatenation "hello"*3 "hellohellohello" # repetition "hello"[0] "h" # indexing "hello"[-1] "o" # (from end) "hello"[1:4] "ell" # slicing len("hello") 5 # size "hello" "jello" 1 # comparison "e" in "hello" 1 # search New line: "escapes: \n " Line continuation: triple quotes ’’’ Quotes: ‘single quotes’, "raw strings" its.unc.edu 29

Simple Data Types Triple quotes useful for multi-line strings s """ a long . string with "quotes" or anything else""" s ' a long\012string with "quotes" or anything else' len(s) 45 its.unc.edu 30

Methods in string its.unc.edu upper() lower() capitalize() count(s) find(s) rfind(s) strip(), lstrip(), rstrip() replace(a, b) expandtabs() split() join() center(), ljust(), rjust() index(s) 31

Compound Data Type: List List: A container that holds a number of other objects, in a given order Defined in square brackets a [1, 2, 3, 4, 5] print a[1] # number 2 some list [] some list.append("foo") some list.append(12) print len(some list) its.unc.edu # 2 32

List a [99, "bottles of beer", ["on", "the", "wall"]] Flexible arrays, not Lisp-like linked lists Same operators as for strings a b, a*3, a[0], a[-1], a[1:], len(a) Item and slice assignment a[0] 98 a[1:2] ["bottles", "of", "beer"] - [98, "bottles", "of", "beer", ["on", "the", "wall"]] del a[-1] its.unc.edu # - [98, "bottles", "of", "beer"] 33

More list operations a range(5) # [0,1,2,3,4] a.append(5) # [0,1,2,3,4,5] a.pop() # [0,1,2,3,4] 5 a.insert(0, 5.5) a.pop(0) # [5.5,0,1,2,3,4] # [0,1,2,3,4] 5.5 a.reverse() a.sort() its.unc.edu # [4,3,2,1,0] # [0,1,2,3,4] 34

Operations in List its.unc.edu append insert index count sort reverse remove pop extend Indexing e.g., L[i] Slicing e.g., L[1:5] Concatenation e.g., L L Repetition e.g., L * 5 Membership test e.g., ‘a’ in L Length e.g., len(L) 35

Nested List List in a list E.g., s [1,2,3] t [‘begin’, s, ‘end’] t [‘begin’, [1, 2, 3], ‘end’] t[1][1] 2 its.unc.edu 36

Dictionaries Dictionaries: curly brackets What is dictionary? Refer value through key; “associative arrays” Like an array indexed by a string An unordered set of key: value pairs Values of any type; keys of almost any type {"name":"Guido", "age":43, ("hello","world"):1, 42:"yes", "flag": ["red","white","blue"]} d { "foo" : 1, "bar" : 2 } print d["bar"] # 2 some dict {} some dict["foo"] "yow!" print some dict.keys() # ["foo"] its.unc.edu 37

Methods in Dictionary its.unc.edu keys() values() items() has key(key) clear() copy() get(key[,x]) setdefault(key[,x]) update(D) popitem() 38

Dictionary details Keys must be immutable: numbers, strings, tuples of immutables these cannot be changed after creation reason is hashing (fast lookup technique) not lists or other dictionaries these types of objects can be changed "in place" no restrictions on values Keys will be listed in arbitrary order again, because of hashing its.unc.edu 39

Tuples What is a tuple? A tuple is an ordered collection which cannot be modified once it has been created. In other words, it's a special array, a read-only array. How to make a tuple? In round brackets E.g., t () t (1, 2, 3) t (1, ) t 1, a (1, 2, 3, 4, 5) print a[1] # 2 its.unc.edu 40

Operations in Tuple Indexing e.g., T[i] Slicing e.g., T[1:5] Concatenation e.g., T T Repetition e.g., T * 5 Membership test e.g., ‘a’ in T Length e.g., len(T) its.unc.edu 41

List vs. Tuple What are common characteristics? Both store arbitrary data objects Both are of sequence data type What are differences? Tuple doesn’t allow modification Tuple doesn’t have methods Tuple supports format strings Tuple supports variable length parameter in function call. Tuples slightly faster its.unc.edu 42

Data Type Wrap Up Integers: 2323, 3234L Floating Point: 32.3, 3.1E2 Complex: 3 2j, 1j Lists: l [ 1,2,3] Tuples: t (1,2,3) Dictionaries: d {‘hello’ : ‘there’, 2 : 15} its.unc.edu 43

Data Type Wrap Up Lists, Tuples, and Dictionaries can store any type (including other lists, tuples, and dictionaries!) Only lists and dictionaries are mutable All variables are references its.unc.edu 44

Input The raw input(string) method returns a line of user input as a string The parameter is used as a prompt The string can be converted by using the conversion methods int(string), float(string), etc. its.unc.edu 45

File I/O f file("foo", "r") line f.readline() print line, f.close() # Can use sys.stdin as input; # Can use sys.stdout as output. its.unc.edu 46

Files: Input input open(‘data’, ‘r’) Open the file for input S input.read() Read whole file into one String S input.read(N) Reads N bytes (N 1) Returns a list of line strings L input.readlines() its.unc.edu 47

Files: Output output open(‘data’, ‘w’) Open the file for writing output.write(S) Writes the string S to file output.writelines(L) Writes each of the strings in list L to file output.close() Manual close its.unc.edu 48

open() and file() These are identical: f open(filename, "r") f file(filename, "r") The open() version is older The file() version is the recommended way to open a file now uses object constructor syntax (next lecture) its.unc.edu 49

OOP Terminology class -- a template for building objects instance -- an object created from the template (an instance of the class) method -- a function that is part of the object and acts on instances directly constructor -- special "method" that creates new instances its.unc.edu 50

Objects Objects: What is an object? data structure, and functions (methods) that operate on it class thingy: # Definition of the class here, next slide t thingy() t.method() print t.field Built-in data structures (lists, dictionaries) are also objects though internal representation is different its.unc.edu 51

Defining a class class Thingy: """This class stores an arbitrary object.""" def init (self, value): constructor """Initialize a Thingy.""" self.value value def showme(self): method """Print this object to stdout.""" print "value %s" % self.value its.unc.edu 52

Using a class (1) t Thingy(10) # calls init method t.showme() # prints "value 10" t is an instance of class Thingy showme is a method of class Thingy init is the constructor method of class Thingy when a Thingy is created, the init method is called Methods starting and ending with are "special" methods its.unc.edu 53

Using a class (2) print t.value # prints "10" value is a field of class Thingy its.unc.edu t.value 20 # change the field value print t.value # prints "20" 54

"Special" methods All start and end with (two underscores) Most are used to emulate functionality of built-in types in user-defined classes e.g. operator overloading add , sub , mult , . see python docs for more information its.unc.edu 55

Control flow (1) if, if/else, if/elif/else if a 0: print "zero!" elif a 0: print "negative!" else: print "positive!" Notes: blocks delimited by indentation! colon (:) used at end of lines containing control flow its.unc.edu keywords 56

Control flow (3) while loops a 10 while a 0: print a a - 1 its.unc.edu 57

Control flow (4) for loops for a in range(10): print a really a "foreach" loop its.unc.edu 58

Control flow (5) Common for loop idiom: a [3, 1, 4, 1, 5, 9] for i in range(len(a)): print a[i] its.unc.edu 59

Control flow (6) Common while loop idiom: f open(filename, "r") while True: line f.readline() if not line: break # do something with line its.unc.edu 60

Control flow (7): odds & ends continue statement like in C pass keyword: if a 0: pass # do nothing else: # whatever its.unc.edu 61

Defining functions def foo(x): y 10 * x 2 return y All variables are local unless specified as global Arguments passed by value its.unc.edu 62

Executing functions def foo(x): y 10 * x 2 return y print foo(10) its.unc.edu # 102 63

Why use modules? Code reuse Routines can be called multiple times within a program Routines can be used from multiple programs Namespace partitioning Group data together with functions used for that data Implementing shared services or data Can provide global data structure that is accessed by multiple subprograms its.unc.edu 64

Modules Modules are functions and variables defined in separate files Items are imported using from or import from module import function function() import module module.function() Modules are namespaces Can be used to organize variable names, i.e. atom.position atom.position - molecule.position its.unc.edu 65

Modules Access other code by importing modules import math print math.sqrt(2.0) or: from math import sqrt print sqrt(2.0) its.unc.edu 66

Modules or: from math import * print sqrt(2.0) Can import multiple modules on one line: import sys, string, math Only one "from x import y" per line its.unc.edu 67

Example: NumPy Modules http://numpy.scipy.org/ NumPy has many of the features of Matlab, in a free, multiplatform program. It also allows you to do intensive computing operations in a simple way Numeric Module: Array Constructors ones, zeros, identity arrayrange LinearAlgebra Module: Solvers its.unc.edu Singular Value Decomposition Eigenvalue, Eigenvector Inverse Determinant Linear System Solver 68

Arrays and Constructors a ones((3,3),float) print a [[1., 1., 1.], [1., 1., 1.], [1., 1., 1.]] b zeros((3,3),float) b b 2.*identity(3) #" " is overloaded c a b print c [[3., 1., 1.], [1., 3., 1.], [1., 1., 3.]] its.unc.edu 69

Overloaded operators b 2.*ones((2,2),float) #overloaded print b [[2.,2.], [2.,2.]] b b 1 # Addition of a scalar is print b # element-by-element [[3.,3.], [3.,3.]] c 2.*b # Multiplication by a scalar is print c # element-by-element [[6.,6.], [6.,6.]] its.unc.edu 70

Array functions from LinearAlgebra import * a zeros((3,3),float) 2.*identity(3) print inverse(a) [[0.5, 0., 0.], [0., 0.5, 0.], [0., 0., 0.5]] print determinant(inverse(a)) 0.125 print diagonal(a) [0.5,0.5,0.5] print diagonal(a,1) [0.,0.] transpose(a), argsort(), dot() its.unc.edu 71

Eigenvalues from LinearAlgebra import * val eigenvalues(c) val, vec eigenvectors(c) print val [1., 4., 1.] print vec [[0.816, -0.408, -0.408], [0.575, 0.577, 0.577], [-0.324, -0.487, 0.811]] also solve linear equations, singular value decomposition, etc. its.unc.edu 72

Least Squares Fitting Part of Hinsen's Scientific Python module from LeastSquares import * def func(params,x): # y ax 2 bx c return params[0]*x*x params[1]*x params[2] data [] for i in range(10): data.append((i,i*i)) guess (3,2,1) fit params, fit error leastSquaresFit(func,guess,data) print fit params [1.00,0.000,0.00] its.unc.edu 73

FFT from FFT import * data array((1,0,1,0,1,0,1,0)) print fft(data).real [4., 0., 0., 0., 4., 0., 0., 0.]] Also note that the FFTW package ("fastest Fourier transform in the West") has a python wrapper. See notes at the end Python Standard Libraries/Modules: http://docs.python.org/library/ http://its2.unc.edu/dci/dci components/shared apps/pack ages/python packages.html http://pypi.python.org/pypi/ its.unc.edu 74

Command-line arguments import sys print len(sys.argv) # NOT argc # Print all arguments: print sys.argv # Print all arguments but the program # or module name: print sys.argv[1:] its.unc.edu # "array slice" 75

Catching Exceptions #python code a.py x 0 try: print 1/x except ZeroDivisionError, message: print "Can’t divide by zero:" print message python a.py Can't divide by zero: integer division or modulo by zero its.unc.edu 76

Try-Finally: Cleanup f open(file) try: process file(f) finally: f.close() # always executed print "OK" # executed on success only its.unc.edu 77

Raising Exceptions raise IndexError raise IndexError("k out of range") raise IndexError, "k out of range” try: something except: # catch everything print "Oops" raise # reraise its.unc.edu 78

Python: Pros & Cons Pros Free availability (like Perl, Python is open source). Stability (Python is in release 2.6 at this point and, as I noted earlier, is older than Java). Very easy to learn and use Good support for objects, modules, and other reusability mechanisms. Easy integration with and extensibility using C and Java. Cons Smaller pool of Python developers compared to other languages, such as Java Lack of true multiprocessor support Absence of a commercial support point, even for an Open Source project (though this situation is changing) Software performance slow, not suitable for high performance applications its.unc.edu 79

References Python Homepage http://www.python.org Python Tutorial http://docs.python.org/tutorial/ Python Documentation http://www.python.org/doc Python Library References http://docs.python.org/release/2.5.2/lib/lib.html Python Add-on Packages: http://pypi.python.org/pypi its.unc.edu 80

Questions & Comments Please Pleasedirect directcomments/questions comments/questionsabout aboutresearch researchcomputing computingtoto E-mail: E-mail:[email protected] [email protected] Please Pleasedirect directcomments/questions comments/questionspertaining pertainingtotothis thispresentation presentationtoto E-Mail: E-Mail:[email protected] [email protected] The PPT file of this presentation is available here: http://its2.unc.edu/divisions/rc/training/scientific/short courses/Python intro.ppt its.unc.edu

Hands-On Exercises 26 codes at /netscr/training/Python Copy to your own /netscr/ USER Read, understand, and then run them Suggested order: hello, input, print, readwrite number, string test, sort list, dictionary, tuple, function, class loop, fact, calculator, guess, prime number matrix, opt, leastsq The PPT/WORD format of this presentation is available here: http://its2.unc.edu/divisions/rc/training/scientific/ its.unc.edu /afs/isis/depts/its/public html/divisions/rc/training/scientific/short courses/ 82