Basic Computer Coding: Python

Citation preview

BASIC COMPUTER CODING: PYTHON

BIBLIOTEX Digital Library

www.bibliotex.com

BASIC COMPUTER CODING: PYTHON

BIBLIOTEX Digital Library

www.bibliotex.com email: [email protected]

e-book Edition 2019 ISBN: 978-1-98462-684-4 (e-book)

This book contains information obtained from highly regarded resources. Reprinted material sources are indicated. Copyright for individual articles remains with the authors as indicated and published under Creative Commons License. A Wide variety of references are listed. Reasonable efforts have been made to publish reliable data and views articulated in the chapters are those of the individual contributors, and not necessarily those of the editors or publishers. Editors or publishers are not responsible for the accuracy of the information in the published chapters or consequences of their use. The publisher assumes no responsibility for any damage or grievance to the persons or property arising out of the use of any materials, instructions, methods or thoughts in the book. The editors and the publisher have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission has not been obtained. If any copyright holder has not been acknowledged, please write to us so we may rectify.

Notice: Registered trademark of products or corporate names are used only for explanation and identification without intent of infringement.

© 2019 3G E-learning LLC

In Collaboration with 3G E-Learning LLC. Originally Published in printed book format by 3G E-Learning LLC with ISBN 978-1-98462-284-6

EDITORIAL BOARD Aleksandar Mratinković is born on May 5, 1988. in Arandjelovac, Serbia. He has graduated on Economic high school (2007), The College of Tourism in Belgrade (2013), and also has a master degree of Psychology (Faculty of Philosophy, University of Novi Sad). He has been engaged in different fields of psychology (Developmental Psychology, Clinical Psychology, Educational Psychology and Industrial Psychology) and has published several scientific works.

Dan Piestun (PhD) is currently a startup entrepreneur in Israel working on the interface of Agriculture and Biomedical Sciences and was formerly presidentCEO of the National Institute of Agricultural Research (INIA) in Uruguay. Dan is a widely published scientist who has received many honours during his career including being a two-time recipient of the Amit Golda Meir Prize from the Hebrew University of Jerusalem, his areas of expertise includes stem cell molecular biology, plant and animal genetics and bioinformatics. Dan’s passion for applied science and technological solutions did not stop him from pursuing a deep connection to the farmer, his family and nature. Among some of his interest and practices counts enjoying working as a beekeeper and onboard fishing. Felecia Killings is the Founder and CEO of LiyahAmore Publishing, a publishing company committed to providing technical and educational services and products to Christian Authors. She operates as the Senior Editor and Writer, the Senior Writing Coach, the Content Marketing Specialist, Editor-in-Chief to the company’s quarterly magazine, the Executive and Host of an international virtual network, and the Executive Director of the company’s online school for Authors. She is a former high-school English instructor and professional development professor. She possesses a Master of Arts degree in Education and a Bachelor’s degree in English and African American studies. Dr. Sandra El Hajj, Ph.D. in Health Sciences from Nova Southeastern University, Florida, USA is a health professional specialized in Preventive and Global Health. With her 12 years of education obtained from one of the most prominent universities in Beirut, in addition to two leading universities in the State of Florida (USA), Dr. Sandra made sure to incorporate interdisciplinary and multicultural approaches in her work. Her long years of studies helped her create her own miniature world of knowledge linking together the healthcare field with Medical Research, Statistics, Food Technology, Environmental & Occupational Health, Preventive Health and most noteworthy her precious last degree of Global Health. Till today, she is the first and only doctor specialized in Global Health in the Middle East area. Hazem Shawky Fouda has a PhD. in Agriculture Sciences, obtained his PhD. From the Faculty of Agriculture, Alexandria University in 2008, He is working in Cotton Arbitration & Testing General Organization (CATGO).

Fozia Parveen has a Dphil in Sustainable Water Engineering from the University of Oxford. Prior to this she has received MS in Environmental Sciences from National University of Science and Technology (NUST), Islamabad Pakistan and BS in Environmental Sciences from Fatima Jinnah Women University (FJWU), Rawalpindi.

Igor Krunic 2003-2007 in the School of Economics. After graduating in 2007, he went on to study at The College of Tourism, at the University of Belgrade where he got his bachelor degree in 2010. He was active as a third-year student representative in the student parliament.Then he went on the Faculty of science, at the University of Novi Sad where he successfully defended his master’s thesis in 2013. The crown of his study was the work titled Opportunities for development of cultural tourism in Cacak“. Later on, he became part of a multinational company where he got promoted to a deputy director of logistic. Nowadays he is a consultant and writer of academic subjects in the field of tourism. Dr. Jovan Pehcevski obtained his PhD in Computer Science from RMIT University in Melbourne, Australia in 2007. His research interests include big data, business intelligence and predictive analytics, data and information science, information retrieval, XML, web services and service-oriented architectures, and relational and NoSQL database systems. He has published over 30 journal and conference papers and he also serves as a journal and conference reviewer. He is currently working as a Dean and Associate Professor at European University in Skopje, Macedonia.

Dr. Tanjina Nur finished her PhD in Civil and Environmental Engineering in 2014 from University of Technology Sydney (UTS). Now she is working as Post-Doctoral Researcher in the Centre for Technology in Water and Wastewater (CTWW) and published about eight International journal papers with 80 citations. Her research interest is wastewater treatment technology using adsorption process.

Stephen obtained his PhD from the University of North Carolina at Charlotte in 2013 where his graduate research focused on cancer immunology and the tumor microenvironment. He received postdoctoral training in regenerative and translational medicine, specifically gastrointestinal tissue engineering, at the Wake Forest Institute of Regenerative Medicine. Currently, Stephen is an instructor for anatomy and physiology and biology at Forsyth Technical Community College.

Michelle holds a Masters of Business Administration from the University of Phoenix, with a concentration in Human Resources Management. She is a professional author and has had numerous articles published in the Henry County Times and has written and revised several employee handbooks for various YMCA organizations throughout the United States.

Table of Contents Preface

Chapter 1

Introduction to Python

xi

1

Learning Objectives ............................................................................. 1 Introduction.......................................................................................... 2 1.1 Overview of Python ...................................................................... 2 1.1.1 History of Python .................................................................3 1.1.2 Python Features ....................................................................3 1.2 Python Environment Setup .......................................................... 5 1.2.1 Getting Python ......................................................................6 1.2.2 Installing Python ..................................................................6 1.2.3 Setting up PATH ...................................................................7 1.2.4 Python Environment Variables ..........................................8 1.2.5 Running Python....................................................................9 1.3 Basic Syntax of Python................................................................ 11 1.3.1 First Python Program ........................................................11 1.3.2 Python Identifiers ...............................................................13 1.3.3 Reserved Words ..................................................................13 1.3.4 Lines and Indentation ........................................................14 1.3.5 Multi-Line Statements .......................................................16 1.3.6 Quotation in Python ..........................................................16 1.3.7 Comments in Python .........................................................16 1.3.8 Using Blank Lines...............................................................17 1.3.9 Waiting for the User ...........................................................17 1.3.10 Multiple Statements on a Single Line ............................17 1.3.11 Multiple Statement Groups as Suites ............................18 1.3.12 Command Line Arguments ............................................18 1.4 Python Variables .......................................................................... 19 1.4.1 Assigning Values to Variables ..........................................19 1.4.2 Multiple Assignment .........................................................20 1.4.3 Standard Data Types ..........................................................20 1.4.4 Data Type Conversion .......................................................26 1.5 Python Basic Operators .............................................................. 27

1.5.1 Types of Operator ...............................................................28 1.5.2 Python Operators Precedence ..........................................32 Role Model.......................................................................................... 35 Summary............................................................................................. 38 Knowledge Check ............................................................................. 39 Review Questions .............................................................................. 40 References ........................................................................................... 41

Chapter 2

Python Functions, Modules and Packages

43

Learning Objectives ........................................................................... 43 Introduction........................................................................................ 44 2.1 Function in Python ...................................................................... 44 2.1.1 Syntax of Function..............................................................44 2.1.2 Docstring .............................................................................45 2.1.3 The Return Statement ........................................................46 2.1.4 How Function works in Python? .....................................47 2.1.5 Python Function Arguments ............................................47 2.1.6 The Anonymous Functions ...............................................51 2.1.7 The Return Statement ........................................................53 2.2 Python Modules........................................................................... 53 2.2.1 More on Modules ...............................................................55 2.2.2 Standard Modules ..............................................................59 2.3 Python Packages .......................................................................... 62 2.3.1 Importing * From a Package .............................................64 2.3.2 Intra-package References ..................................................66 2.3.3 Packages in Multiple Directories .....................................67 Summary............................................................................................. 68 Knowledge Check ............................................................................. 69 Review Questions .............................................................................. 70 References ........................................................................................... 71

Chapter 3

Dictionaries, Sets, and Files

73

Learning Objectives ........................................................................... 73 Introduction........................................................................................ 74 3.1 Python Dictionaries ..................................................................... 74 3.1.1 Accessing Dictionary Elements ........................................75 3.1.2 Modifying Dictionaries......................................................78

3.1.3 The dict() Constructor........................................................85 3.1.4 Dictionary Methods ...........................................................85 3.1.5 Aliasing and Copying ........................................................86 3.2 Python Sets ................................................................................... 87 3.2.1 Defining a Set ......................................................................88 3.2.2 Set Size and Membership ..................................................92 3.2.3 Methods for Sets .................................................................92 3.2.4 Creating a Set ......................................................................93 3.2.5 Accessing Values in a Set...................................................94 3.2.6 Adding Items to a Set.........................................................94 3.2.7 Removing Item from a Set ................................................95 3.2.8 Union of Sets .......................................................................95 3.2.9 Intersection of Sets .............................................................96 3.2.10 Difference of Sets ..............................................................96 3.2.11 Compare Sets ....................................................................96 3.3 Files ................................................................................................ 97 3.3.1 The open function ..............................................................98 3.3.2 Opening a File that Doesn’t Exist ...................................100 3.3.3 Reading Data from Files ..................................................100 Case Study ........................................................................................ 104 Summary........................................................................................... 108 Knowledge Check ........................................................................... 109 Review Questions ............................................................................ 110 References ......................................................................................... 111

Chapter 4

Exceptions, Unit Testing and Comprehensions

113

Learning Objectives ......................................................................... 113 Introduction...................................................................................... 114 4.1 Exceptions ................................................................................... 114 4.1.1 Handling Exceptions........................................................115 4.1.2 Raising Exceptions ...........................................................120 4.1.3 User-defined Exceptions .................................................120 4.1.4 Defining Clean-up Actions .............................................122 4.1.5 Predefined Clean-up Actions..........................................124 4.2 Unit testing ................................................................................. 125 4.2.1 Basic example ....................................................................127

4.2.2 Command-Line Interface ................................................129 4.2.3 Test Discovery ...................................................................130 4.2.4 Organizing test code ........................................................131 4.2.5 Re-using old test code ......................................................136 4.2.6 Skipping tests and expected failures .............................137 4.3 Comprehensions ........................................................................ 139 4.3.1 List Comprehensions .......................................................139 4.3.2 Dict Comprehensions ......................................................140 4.3.3 Set Comprehensions ........................................................141 4.3.4 Generator Comprehensions ............................................141 Case Study ........................................................................................ 142 Summary .......................................................................................... 146 Knowledge Check ........................................................................... 147 Review Questions ............................................................................ 148 References ......................................................................................... 149

Chapter 5

Object Oriented Programming

151

Learning Objectives ......................................................................... 151 Introduction...................................................................................... 152 5.1 Introduction To Oops In Python ............................................. 153 5.1.1 Classes in Python .............................................................154 5.1.2 Python Objects (Instances) ..............................................154 5.1.3 Instantiating Objects ........................................................157 5.1.4 Instance Methods .............................................................159 5.1.5 Python Object Inheritance ...............................................161 5.2 Methods of OOPS ...................................................................... 168 5.2.1 Inheritance .........................................................................169 5.2.2 Encapsulation....................................................................173 5.2.3 Polymorphism ..................................................................177 5.2.4 Abstraction ........................................................................183 Summary........................................................................................... 194 Knowledge Check ........................................................................... 195 Review Questions ............................................................................ 196 References ......................................................................................... 197

Chapter 6

Python Regular Expression

199

Learning Objectives ......................................................................... 199 Introduction...................................................................................... 200 6.1 Regex Search and Match........................................................... 200 6.1.1 The Match Function .........................................................202 6.1.2 The Search Function.........................................................204 6.1.3 Matching Versus Searching.............................................205 6.1.4 Search and Replace ..........................................................206 6.2 Regular Expression Modifiers: Option Flags......................... 207 6.2.1 Regular Expression Patterns ...........................................208 6.2.2 Regular Expression Examples ........................................211 Case Study ........................................................................................ 217 Summary........................................................................................... 225 Knowledge Check ........................................................................... 226 Review Questions ............................................................................ 227 References ......................................................................................... 228

Chapter 7

Python Multithreading

229

Learning Objectives ......................................................................... 229 Introduction...................................................................................... 230 7.1 Python Threading – Python Multithreading ......................... 231 7.1.1 Getting Started with Python Multithreading ...............232 7.1.2 Python Multithreading Modules for Thread Implementation ...........................................................233 7.1.3 Difference between Multiprocessing and Multithreading ............................................................233 7.2 Functions in Python Multithreading ...................................... 235 7.2.1 Thread-Local Data ............................................................238 7.2.2 Thread Objects ..................................................................239 7.2.3 Lock Objects ......................................................................243 7.2.4 RLock Objects ...................................................................244 7.2.5 Condition Objects .............................................................245 7.2.6 Semaphore Objects ...........................................................248 7.2.7 Event Objects.....................................................................250 7.2.8 Timer Objects ....................................................................251 7.2.9 Barrier Objects...................................................................252 7.2.10 Using locks, Conditions, and Semaphores in the with-statement ..................................................254

Role Model........................................................................................ 256 Summary........................................................................................... 257 Knowledge Check ........................................................................... 258 Review Questions ............................................................................ 259 References ......................................................................................... 260

Index

263

Preface Python is a powerful advanced, object-oriented programming language developed by Guido van Rossum. The simple ease of use syntax makes it the perfect language for beginners trying to learn computer programming. Its high-level built in data structures, combined with dynamic typing and dynamic binding; make it very attractive for Rapid Application Development, as well as for use as a scripting or glue language to connect existing components together. Python’s simple, easy to learn syntax emphasizes readability and therefore reduces the cost of program maintenance. Python supports modules and packages, which encourages program modularity and code reuse. Organization of the Book This book is systematically divided into seven chapters. This is a comprehensive guide on how to get started in Python, why you should learn it and how you can learn it. This hands-on guide takes you through the language a step at a time, beginning with basic programming concepts including with functions, recursion, data structures, and object-oriented design. Chapter 1 presents an introduction to Python. You will learn the Python environment setup, syntax of Python and Python variables. Basic operators of Python are also discussed. Chapter 2 aims to focus on Python functions, modules and packages. In Python, function is a group of related statements that perform a specific task. Functions help break our program into smaller and modular parts. It also describes the python modules and python packages. Chapter 3 begins with python dictionaries. It also explains Python sets and files used in Python. They can be used to read and write text memos, audio clips, Excel documents, saved email messages, and whatever else you happen to have stored on your machine.

Chapter 4 gives an overview of how to use the exceptions? Further, it explains the unit testing used to validate that each unit of the software performs as designed. In last, the chapter focuses on understanding the comprehensions that allow sequences to be built from other sequences. Chapter 5 is aimed to discuss the use of OOPS in python, including various types of methods of OOPS. The programming challenge was seen as how to write the logic, not how to define the data. Object-oriented programming takes the view that what we really care about are the objects we want to manipulate rather than the logic required to manipulate them. Chapter 6 focuses on Python regular expression that helps you match or find other strings or sets of strings, using a specialized syntax held in a pattern. Chapter 7 discusses about Python Multithreading used to implement multithreading in python programs and also used to run multiple threads (tasks, function calls) at the same time. .

CHAPTER 1

INTRODUCTION TO PYTHON “Now, it’s my belief that Python is a lot easier than to teach to students programming and teach them C or C++ or Java at the same time because all the details of the languages are so much harder. Other scripting languages really don’t work very well there either.” ―Guido van Rossum

Learning Objectives After studying this chapter, you will be able to: ■

Overview the Python

■

Learn about Python environment setup

■

Describe the basic syntax of Python

■

Understand Python variables

■

Discuss about basic operators of Python

Basic Computer Coding: Python

INTRODUCTION Python is an interpreted, object-oriented programming language similar to PERL that has gained popularity because of its clear syntax and readability. Python is said to be relatively easy to learn and portable, meaning its statements can be interpreted in a number of operating systems, including UNIX-based systems, Mac OS, MS-DOS, OS/2, and various versions of Microsoft Windows 98. Python was created by Guido van Rossum, a former resident of the Netherlands, whose favorite comedy group at the time was Monty Python’s Flying Circus. The source code is freely available and open for modification and reuse. Python has a significant number of users. A notable feature of Python is its indenting of source statements to make the code easier to read. Python offers dynamic data type, ready-made class, and interfaces to many system calls and libraries. It can be extended, using the C or C++ language. Python can be used as the script in Microsoft’s Active Server Page (ASP) technology. The scoreboard system for the Melbourne (Australia) Cricket Ground is written in Python. Z Object Publishing Environment, a popular Web application server, is also written in the Python language.

1.1 OVERVIEW OF PYTHON Python is a high-level, interpreted, interactive and object-oriented scripting language. Python is designed to be highly readable. It uses English keywords frequently where as other languages use punctuation, and it has fewer syntactical constructions than other languages. ■

Python is Interpreted − Python is processed at runtime by the interpreter. You do not need to compile your program before executing it. This is similar to PERL and PHP.

■

Python is Interactive − You can actually sit at a Python prompt and interact with the interpreter directly to write your programs.

2

Introduction to Python

■

Python is Object-Oriented − Python supports Object-Oriented style or technique of programming that encapsulates code within objects.

■

Python is a Beginner’s Language − Python is a great language for the beginner-level programmers and supports the development of a wide range of applications from simple text processing to WWW browsers to games.

1.1.1 History of Python Python was developed by Guido van Rossum in the late eighties and early nineties at the National Research Institute for Mathematics and Computer Science in the Netherlands. Python is derived from many other languages, including ABC, Modula-3, C, C++, Algol-68, SmallTalk, and Unix shell and other scripting languages. Python is copyrighted. Like Perl, Python source code is now available under the GNU General Public License (GPL). Python is now maintained by a core development team at the institute, although Guido van Rossum still holds a vital role in directing its progress.

1.1.2 Python Features Python’s features include − •

Easy-to-learn − Python has few keywords, simple structure, and a clearly defined syntax. This allows the student to pick up the language quickly.

3

Created by Guido van Rossum and first released in 1991, Python has a design philosophy that emphasizes code readability, notably using significant whitespace. It provides constructs that enable clear programming on both small and large scales. In July 2018, Van Rossum stepped down as the leader in the language community after 30 years.

Basic Computer Coding: Python

■

Easy-to-read − Python code is more clearly defined and visible to the eyes.

■

Easy-to-maintain − Python’s source code is fairly easy-to-maintain.

■

A broad standard library − Python’s bulk of the library is very portable and cross-platform compatible on UNIX, Windows, and Macintosh.

■

Interactive Mode − Python has support for an interactive mode which allows interactive testing and debugging of snippets of code.

■

Portable − Python can run on a wide variety of hardware platforms and has the same interface on all platforms.

■

Extendable − You can add low-level modules to the Python interpreter. These modules enable programmers to add to or customize their tools to be more efficient.

■

Databases − Python provides interfaces to all major commercial databases.

■

GUI Programming − Python supports GUI applications that can be created and ported to many system calls, libraries and windows systems, such as Windows MFC, Macintosh, and the X Window system of Unix.

■

Scalable − Python provides a better structure and support for large programs than shell scripting.

Apart from the above-mentioned features, Python has a big list of good features, few Java is a programming are listed below −

language that produces software for multiple platforms.

■

It supports functional and structured programming methods as well as OOP.

4

Introduction to Python

■

It can be used as a scripting language or can be compiled to byte-code for building large applications.

■

It provides very high-level dynamic data types and supports dynamic type checking.

■

It supports automatic garbage collection.

■

It can be easily integrated with C, C++, COM, ActiveX, CORBA, and Java.

1.2 PYTHON ENVIRONMENT SETUP Python is available on a wide variety of platforms including Linux and Mac OS X. Let’s understand how to set up our Python environment. Open a terminal window and type “python” to find out if it is already installed and which version is installed. ■

Unix (Solaris, Linux, FreeBSD, AIX, HP/UX, SunOS, IRIX, etc.)

■

Win 9x/NT/2000

■

Macintosh (Intel, PPC, 68K)

■

OS/2

■

DOS (multiple versions)

■

PalmOS

■

Nokia mobile phones

■

Windows CE

■

Acorn/RISC OS

■

BeOS

■

Amiga

■

VMS/OpenVMS

■

QNX

■

VxWorks

■

Psion

■

Python has also been ported to the Java and .NET virtual machines

5

Basic Computer Coding: Python

1.2.1 Getting Python The most up-to-date and current source code, binaries, documentation, news, etc., is available on the official website of Python https://www. python.org/ You can download Python documentation from https://www. python.org/doc/. The documentation is available in HTML, PDF, and PostScript formats.

1.2.2 Installing Python Python distribution is available for a wide variety of platforms. You need to download only the binary code applicable for your platform and install Python. If the binary code for your platform is not available, you need a C compiler to compile the source code manually. Compiling the source code offers more flexibility in terms of choice of features that you require in your installation. Here is a quick overview of installing Python on various platforms −

Unix and Linux Installation Here are the simple steps to install Python on Unix/Linux machine. ■

Open a Web browser and go to https://www.python.org/ downloads/.

■

Follow the link to download zipped source code available for Unix/Linux.

■

Download and extract files.

■

Editing the Modules/Setup file if you want to customize some options.

■

run ./configure script

■

make

■

make install

This installs Python at standard location /usr/local/bin and its libraries at /usr/local/lib/pythonXX where XX is the version of Python.

6

Introduction to Python

Windows Installation Here are the steps to install Python on Windows machine. ■

Open a Web browser and go to https://www.python.org/ downloads/.

■

Follow the link for the Windows installer python-XYZ.msi file where XYZ is the version you need to install.

■

To use this installer python-XYZ.msi, the Windows system must support Microsoft Installer 2.0. Save the installer file to your local machine and then run it to find out if your machine supports MSI.

■

Run the downloaded file. This brings up the Python install wizard, which is really easy to use. Just accept the default settings, wait until the install is finished, and you are done.

Macintosh Installation Recent Macs come with Python installed, but it may be several years out of date. See http://www.python.org/download/mac/ for instructions on getting the current version along with extra tools to support development on the Mac. For older Mac OS’s before Mac OS X 10.3 (released in 2003), MacPython is available. Jack Jansen maintains it and you can have full access to the entire documentation at his website − http://www.cwi.nl/~jack/macpython. html. You can find complete installation details for Mac OS installation.

1.2.3 Setting up PATH Programs and other executable files can be in many directories, so operating systems provide a search path that lists the directories that the OS searches for executables. The path is stored in an environment variable, which is a named string maintained by the operating system. This variable contains information available to the command shell and other programs. The path variable is named as PATH in Unix or Path in Windows (Unix is case sensitive; Windows is not).

7

Basic Computer Coding: Python

Unix is a multi-user operating system designed for flexibility and adaptability.

In Mac OS, the installer handles the path details. To invoke the Python interpreter from any particular directory, you must add the Python directory to your path.

Setting path at Unix/Linux To add the Python directory to the path for a particular session in Unix − ■

In the csh shell − type setenv PATH “$PATH:/usr/local/bin/python” and press Enter.

■

In the bash shell (Linux) − type export ATH=”$PATH:/usr/local/bin/python” and press Enter.

■

In the sh or ksh shell − type PATH=”$PATH:/usr/local/bin/python” and press Enter.

■

Note − /usr/local/bin/python is the path of the Python directory

Setting path at Windows Remember C:\Python is the path of the Python directory

To add the Python directory to the path for a particular session in Windows − At the command prompt − type path %path%;C:\ Python and press Enter.

1.2.4 Python Environment Variables Here are important environment variables, which can be recognized by Python −

8

Introduction to Python Sr.No.

Variable & Description

1

PYTHONPATH It has a role similar to PATH. This variable tells the Python interpreter where to locate the module files imported into a program. It should include the Python source library directory and the directories containing Python source code. PYTHONPATH is sometimes preset by the Python installer.

2

PYTHONSTARTUP It contains the path of an initialization file containing Python source code. It is executed every time you start the interpreter. It is named as .pythonrc.py in Unix and it contains commands that load utilities or modify PYTHONPATH.

3

PYTHONCASEOK It is used in Windows to instruct Python to find the first caseinsensitive match in an import statement. Set this variable to any value to activate it.

4

PYTHONHOME It is an alternative module search path. It is usually embedded in the PYTHONSTARTUP or PYTHONPATH directories to make switching module libraries easy.

1.2.5 Running Python There are three different ways to start Python −

Interactive Interpreter You can start Python from Unix, DOS, or any other system that provides you a command-line interpreter or shell window. Enter python the command line. Start coding right away in the interactive interpreter. $python # Unix/Linux or python% # Unix/Linux or C:> python # Windows/DOS Here is the list of all the available command line options − 9

Basic Computer Coding: Python Sr.No.

Option & Description

1

-d It provides debug output.

2

-O It generates optimized bytecode (resulting in .pyo files).

3

-S Do not run import site to look for Python paths on startup.

4

-v verbose output (detailed trace on import statements).

5

-X disable class-based built-in exceptions (just use strings); obsolete starting with version 1.6.

6

-c cmd run Python script sent in as cmd string

7

file run Python script from given file

Script from the Command-line A Python script can be executed at command line by invoking the interpreter on your application, as in the following − $python script.py # Unix/Linux or python% script.py # Unix/Linux or C: >python script.py # Windows/DOS Note − Be sure the file permission mode allows execution.

Integrated Development Environment You can run Python from a Graphical User Interface (GUI) environment as well, if you have a GUI application on your system that supports Python. 10

Introduction to Python

■

Unix − IDLE is the very first Unix IDE for Python.

■

Windows − PythonWin is the first Windows interface for Python and is an IDE with a GUI.

■

Macintosh − The Macintosh version of Python along with the IDLE IDE is available from the main website, downloadable as either MacBinary or BinHex’d files.

If you are not able to set up the environment properly, then you can take help from your system admin. Make sure the Python environment is properly set up and working perfectly fine. Note − All the examples given in subsequent chapters are executed with Python 2.4.3 version available on CentOS flavor of Linux. We already have set up Python Programming environment online, so that you can execute all the available examples online at the same time when you are learning theory. Feel free to modify any example and execute it online.

1.3 BASIC SYNTAX OF PYTHON The Python language has many similarities to Perl, C, and Java. However, there are some definite differences between the languages.

1.3.1 First Python Program Let us execute programs in different modes of programming.

11

Perl is a family of two high-level, generalpurpose, interpreted, dynamic programming languages, Perl 5 and Perl 6.

Basic Computer Coding: Python

Interactive Mode Programming Invoking the interpreter without passing a script file as a parameter brings up the following prompt − $ python Python 2.4.3 (#1, Nov 11 2010, 13:34:43) [GCC 4.1.2 20080704 (Red Hat 4.1.2-48)] on linux2 Type “help”, “copyright”, “credits” or “license” for more information. >>> Type the following text at the Python prompt and press the Enter − >>> print “Hello, Python!” If you are running new version of Python, then you would need to use print statement with parenthesis as in print (“Hello, Python!”);. However in Python version 2.4.3, this produces the following result − Hello, Python!

Script Mode Programming Invoking the interpreter with a script parameter begins execution of the script and continues until the script is finished. When the script is finished, the interpreter is no longer active. Let us write a simple Python program in a script. Python files have extension .py. Type the following source code in a test.py file − print “Hello, Python!” We assume that you have Python interpreter set in PATH variable. Now, try to run this program as follows − $ python test.py This produces the following result − Hello, Python! Let us try another way to execute a Python script. Here is the modified test.py file − #!/usr/bin/python print “Hello, Python!”

12

Introduction to Python

We assume that you have Python interpreter available in /usr/bin directory. Now, try to run this program as follows − $ chmod +x test.py

# This is to make file executable

$./test.py This produces the following result − Hello, Python!

1.3.2 Python Identifiers A Python identifier is a name used to identify a variable, function, class, module or other object. An identifier starts with a letter A to Z or a to z or an underscore (_) followed by zero or more letters, underscores and digits (0 to 9). Python does not allow punctuation characters such as @, $, and % within identifiers. Python is a case sensitive programming language. Thus, Manpower and manpower are two different identifiers in Python. Here are naming conventions for Python identifiers − ■

Class names start with an uppercase letter. All other identifiers start with a lowercase letter.

■

Starting an identifier with a single leading underscore indicates that the identifier is private.

■

Starting an identifier with two leading underscores indicates a strongly private identifier.

■

If the identifier also ends with two trailing underscores, the identifier is a language-defined special name.

1.3.3 Reserved Words The following list shows the Python keywords. These are reserved words and you cannot use them as constant or variable or any other identifier names. All the Python keywords contain lowercase letters only. and

exec

not

assert

finally

or

13

Basic Computer Coding: Python break

for

pass

class

from

print

continue

global

raise

def

if

return

del

import

try

elif

in

while

else

is

with

except

lambda

yield

1.3.4 Lines and Indentation Python provides no braces to indicate blocks of code for class and function definitions or flow control. Blocks of code are denoted by line indentation, which is rigidly enforced. The number of spaces in the indentation is variable, but all statements within the block must be indented the same amount. For example − if True: print “True” else: Remember Do not try to understand the logic at this point of time. Just make sure you understood various blocks even if they are without braces.

print “False” However, the following block generates an error − if True: print “Answer” print “True” else: print “Answer” print “False” Thus, in Python all the continuous lines indented with same number of spaces would form a block. The following example has various statement blocks − 14

Introduction to Python

#!/usr/bin/python import sys try: # open file stream file = open(file_name, “w”) except IOError: print “There was an error writing to”, file_name sys.exit() print “Enter ‘”, file_finish, print “’ When finished” while file_text != file_finish: file_text = raw_input(“Enter text: “) if file_text == file_finish: # close the file file.close break file.write(file_text) file.write(“\n”) file.close() file_name = raw_input(“Enter filename: “) if len(file_name) == 0: print “Next time please enter something” sys.exit() try: file = open(file_name, “r”) except IOError: print “There was an error reading file” sys.exit() file_text = file.read() 15

Basic Computer Coding: Python

file.close() print file_text

1.3.5 Multi-Line Statements Statements in Python typically end with a new line. Python does, however, allow the use of the line continuation character (\) to denote that the line should continue. For example − total = item_one + \ item_two + \ item_three Statements contained within the [], {}, or () brackets do not need to use the line continuation character. For example − days = [‘Monday’, ‘Tuesday’, ‘Wednesday’, ‘Thursday’, ‘Friday’]

1.3.6 Quotation in Python Python accepts single (‘), double (“) and triple (‘’’ or “””) quotes to denote string literals, as long as the same type of quote starts and ends the string. The triple quotes are used to span the string across multiple lines. For example, all the following are legal − word = ‘word’ sentence = “This is a sentence.” paragraph = “””This is a paragraph. It is made up of multiple lines and sentences.”””

1.3.7 Comments in Python A hash sign (#) that is not inside a string literal begins a comment. All characters after the # and up to the end of the physical line are part of the comment and the Python interpreter ignores them. #!/usr/bin/python # First comment print “Hello, Python!” # second comment 16

Introduction to Python

This produces the following result − Hello, Python! You can type a comment on the same line after a statement or expression − name = “Madisetti” # This is again comment You can comment multiple lines as follows − # This is a comment. # This is a comment, too. # This is a comment, too. # I said that already.

1.3.8 Using Blank Lines A line containing only whitespace, possibly with a comment, is known as a blank line and Python totally ignores it. In an interactive interpreter session, you must enter an empty physical line to terminate a multiline statement.

1.3.9 Waiting for the User The following line of the program displays the prompt, the statement saying “Press the enter key to exit”, and waits for the user to take action − #!/usr/bin/python raw_input(“\n\nPress the enter key to exit.”) Here, “\n\n” is used to create two new lines before displaying the actual line. Once the user presses the key, the program ends. This is a nice trick to keep a console window open until the user is done with an application.

1.3.10 Multiple Statements on a Single Line The semicolon ( ; ) allows multiple statements on the single line given that neither statement starts a new code block. Here is a sample snip using the semicolon − 17

Basic Computer Coding: Python

import sys; x = ‘foo’; sys.stdout.write(x + ‘\n’) Code block is a lexical structure of source code which is grouped together.

1.3.11 Multiple Statement Groups as Suites A group of individual statements, which make a single code block are called suites in Python. Compound or complex statements, such as if, while, def, and class require a header line and a suite. Header lines begin the statement (with the keyword) and terminate with a colon ( : ) and are followed by one or more lines which make up the suite. For example − if expression : suite elif expression : suite else : suite

1.3.12 Command Line Arguments Many programs can be run to provide you with some basic information about how they should be run. Python enables you to do this with -h − $ python -h usage: python [option] ... [-c cmd | -m mod | file | -] [arg] ... Options and arguments (and corresponding environment variables): -c cmd : program passed in as string (terminates option list) -d

: debug output from parser (also 18

Introduction to Python

PYTHONDEBUG=x) -E

: ignore environment variables (such as PYTHONPATH)

-h

: print this help message and exit

[ etc. ] You can also program your script in such a way that it should accept various options. Command Line Arguments is an advanced topic and should be studied a bit later once you have gone through rest of the Python concepts.

1.4 PYTHON VARIABLES Variables are nothing but reserved memory locations to store values. This means that when you create a variable you reserve some space in memory. Based on the data type of a variable, the interpreter allocates memory and decides what can be stored in the reserved memory. Therefore, by assigning different data types to variables, you can store integers, decimals or characters in these variables.

1.4.1 Assigning Values to Variables Python variables do not need explicit declaration to reserve memory space. The declaration happens automatically when you assign a value to a variable. The equal sign (=) is used to assign values to variables. The operand to the left of the = operator is the name of the variable and the operand to the right of the = operator is the value stored in the variable. For example − #!/usr/bin/python counter = 100

# An integer assignment

miles

# A floating point

name

= 1000.0 = “John”

# A string

19

Basic Computer Coding: Python

print counter print miles print name Here, 100, 1000.0 and “John” are the values assigned to counter, miles, and name variables, respectively. This produces the following result − 100 1000.0 John

1.4.2 Multiple Assignment Python allows you to assign a single value to several variables simultaneously. For example − a=b=c=1 Here, an integer object is created with the value 1, and all three variables are assigned to the same memory location. You can also assign multiple objects to multiple variables. For example − a,b,c = 1,2,”john” Here, two integer objects with values 1 and 2 are assigned to variables a and b respectively, and one string object with the value “john” is assigned to the variable c.

1.4.3 Standard Data Types The data stored in memory can be of many types. For example, a person’s age is stored as a numeric value and his or her address is stored as alphanumeric characters. Python has various standard data types that are used to define the operations possible on them and the storage method for each of them. Python has five standard data types − ■

Numbers

■

String

■

List 20

Introduction to Python

■

Tuple

■

Dictionary

Python Numbers Number data types store numeric values. Number objects are created when you assign a value to them. For example − var1 = 1 var2 = 10 You can also delete the reference to a number object by using the del statement. The syntax of the del statement is − del var1[,var2[,var3[....,varN]]]] You can delete a single object or multiple objects by using the del statement. For example − del var del var_a, var_b Python supports four different numerical types − ■

int (signed integers)

■

long (long integers, they can also be represented in octal and hexadecimal)

■

float (floating point real values)

■

complex (complex numbers)

Examples Here are some examples of numbers − int

long

float

complex

10

51924361L

0.0

3.14j

100

-0x19323L

15.20

45.j

-786

0122L

-21.9

9.322e-36j

080

0xDEFABCECBDAECBFBAEl

32.3+e18

.876j

-0490

535633629843L

-90.

-.6545+0J

21

Basic Computer Coding: Python

-0x260

-052318172735L

-32.54e100

3e+26J

0x69

-4721885298529L

70.2-E12

4.53e-7j

Python allows you to use a lowercase l with long, but it is recommended that you use only an uppercase L to avoid confusion with the number 1. Python displays long integers with an uppercase L. ■

A complex number consists of an ordered pair of real floating-point numbers denoted by x + yj, where x and y are the real numbers and j is the imaginary unit.

Python Strings Strings in Python are identified as a contiguous set of characters represented in the quotation marks. Python allows for either pairs of single or double quotes. Subsets of strings can be taken using the slice operator ([ ] and [:] ) with indexes starting at 0 in the beginning of the string and working their way from -1 at the end. The plus (+) sign is the string concatenation operator and the asterisk (*) is the repetition operator. For example − #!/usr/bin/python str = ‘Hello World!’ print str

# Prints complete string

print str[0]

# Prints first character of the string

print str[2:5]

# Prints characters starting from 3rd to 5th

print str[2:]

# Prints string starting from 3rd character

print str * 2

# Prints string two times

print str + “TEST” # Prints concatenated string This will produce the following result − Hello World! H llo llo World! 22

Introduction to Python

Hello World!Hello World! Hello World!TEST

Python Lists Lists are the most versatile of Python’s compound data types. A list contains items separated by commas and enclosed within square brackets ([]). To some extent, lists are similar to arrays in C. One difference between them is that all the items belonging to a list can be of different data type. The values stored in a list can be accessed using the slice operator ([ ] and [:]) with indexes starting at 0 in the beginning of the list and working their way to end -1. The plus (+) sign is the list concatenation operator, and the asterisk (*) is the repetition operator. For example − #!/usr/bin/python list = [ ‘abcd’, 786 , 2.23, ‘john’, 70.2 ] tinylist = [123, ‘john’] print list print list[0]

# Prints complete list # Prints first element of the list

print list[1:3] # Prints elements starting from 2nd till 3rd print list[2:] 3rd element

# Prints elements starting from

print tinylist * 2 # Prints list two times print list + tinylist # Prints concatenated lists This produce the following result − [‘abcd’, 786, 2.23, ‘john’, 70.2] abcd 23

Data type is a classification of data which tells the compiler or interpreter how the programmer intends to use the data.

Basic Computer Coding: Python

[786, 2.23] [2.23, ‘john’, 70.2] [123, ‘john’, 123, ‘john’] [‘abcd’, 786, 2.23, ‘john’, 70.2, 123, ‘john’]

Python Tuples A tuple is another sequence data type that is similar to the list. A tuple consists of a number of values separated by commas. Unlike lists, however, tuples are enclosed within parentheses. The main differences between lists and tuples are: Lists are enclosed in brackets ( [ ] ) and their elements and size can be changed, while tuples are enclosed in parentheses ( ( ) ) and cannot be updated. Tuples can be thought of as read-only lists. For example − #!/usr/bin/python tuple = ( ‘abcd’, 786 , 2.23, ‘john’, 70.2 ) tinytuple = (123, ‘john’) print tuple

# Prints complete list

print tuple[0]

# Prints first element of the list

print tuple[1:3]

# Prints elements starting from 2nd till 3rd

print tuple[2:]

# Prints elements starting from 3rd element

print tinytuple * 2

# Prints list two times

print tuple + tinytuple # Prints concatenated lists This produce the following result − (‘abcd’, 786, 2.23, ‘john’, 70.2) abcd (786, 2.23) (2.23, ‘john’, 70.2) (123, ‘john’, 123, ‘john’) (‘abcd’, 786, 2.23, ‘john’, 70.2, 123, ‘john’)

24

Introduction to Python

The following code is invalid with tuple, because we attempted to update a tuple, which is not allowed. Similar case is possible with lists − #!/usr/bin/python tuple = ( ‘abcd’, 786 , 2.23, ‘john’, 70.2 ) list = [ ‘abcd’, 786 , 2.23, ‘john’, 70.2 ] tuple[2] = 1000 list[2] = 1000

# Invalid syntax with tuple # Valid syntax with list

Python Dictionary Python’s dictionaries are kind of hash table type. They work like associative arrays or hashes found in Perl and consist of key-value pairs. A dictionary key can be almost any Python type, but are usually numbers or strings. Values, on the other hand, can be any arbitrary Python object. Dictionaries are enclosed by curly braces ({ }) and values can be assigned and accessed using square braces ([]). For example − #!/usr/bin/python dict = {} dict[‘one’] = “This is one” dict[2]

= “This is two”

tinydict = {‘name’: ‘john’,’code’:6734, ‘dept’: ‘sales’}

print dict[‘one’]

# Prints value for ‘one’ key

print dict[2]

# Prints value for 2 key

print tinydict

# Prints complete dictionary

print tinydict.keys()

# Prints all the keys 25

Basic Computer Coding: Python

print tinydict.values() # Prints all the values This produce the following result − This is one This is two {‘dept’: ‘sales’, ‘code’: 6734, ‘name’: ‘john’} [‘dept’, ‘code’, ‘name’] [‘sales’, 6734, ‘john’] Dictionaries have no concept of order among elements. It is incorrect to say that the elements are “out of order”; they are simply unordered.

1.4.4 Data Type Conversion Sometimes, you may need to perform conversions between the built-in types. To convert between types, you simply use the type name as a function. There are several built-in functions to perform conversion from one data type to another. These functions return a new object representing the converted value. Sr.No.

Function & Description

1

int(x [,base]) Converts x to an integer. base specifies the base if x is a string.

2

long(x [,base] ) Converts x to a long integer. base specifies the base if x is a string.

3

float(x) Converts x to a floating-point number.

4

complex(real [,imag]) Creates a complex number.

5

str(x) Converts object x to a string representation.

6

repr(x) Converts object x to an expression string. 26

Introduction to Python

7

eval(str) Evaluates a string and returns an object.

8

tuple(s) Converts s to a tuple.

9

list(s) Converts s to a list.

10

set(s) Converts s to a set.

11

dict(d) Creates a dictionary. d must be a sequence of (key,value) tuples.

12

frozenset(s) Converts s to a frozen set.

13

chr(x) Converts an integer to a character.

14

unichr(x) Converts an integer to a Unicode character.

15

ord(x) Converts a single character to its integer value.

16

hex(x) Converts an integer to a hexadecimal string.

17

oct(x) Converts an integer to an octal string.

1.5 PYTHON BASIC OPERATORS Operators are the constructs which can manipulate the value of operands. Consider the expression 4 + 5 = 9. Here, 4 and 5 are called operands and + is called operator.

27

Basic Computer Coding: Python

1.5.1 Types of Operator Python language supports the following types of operators. ■

Arithmetic Operators

■

Comparison (Relational) Operators

■

Assignment Operators

■

Logical Operators

■

Bitwise Operators

■

Membership Operators

■

Identity Operators

Python Arithmetic Operators Assume variable a holds 10 and variable b holds 20, then − Operator

Description

Example

+ Addition

Adds values on either side of the operator.

a + b = 30

- Subtraction

Subtracts right hand operand from left hand operand.

a – b = -10

* Multiplication

Multiplies values on either side of the operator

a * b = 200

/ Division

Divides left hand operand by right hand operand

b/a=2

% Modulus

Divides left hand operand by right hand operand and returns remainder

b%a=0

** Exponent

Performs exponential (power) calculation on operators

a**b =10 to the power 20

//

Floor Division - The division of operands 9//2 = 4 and where the result is the quotient in which 9.0//2.0 = 4.0, -11//3 the digits after the decimal point are = -4, -11.0//3 = -4.0 removed. But if one of the operands is negative, the result is floored, i.e., rounded away from zero (towards negative infinity) −

Python Comparison Operators These operators compare the values on either sides of them and decide the relation among them. They are also called Relational operators. 28

Introduction to Python

Assume variable a holds 10 and variable b holds 20, then − Operator Description

Example

==

If the values of two operands are equal, then the condition becomes true.

(a == b) is not true.

!=

If values of two operands are not equal, then condition becomes true.

(a != b) is true.

If values of two operands are not equal, then condition becomes true.

(a b) is true. This is similar to != operator.

>

If the value of left operand is greater than the value of right operand, then condition becomes true.

(a > b) is not true.


=

If the value of left operand is greater than or equal to the value of right operand, then condition becomes true.

(a >= b) is not true.

2 = 15 (means 0000 1111)

Python Logical Operators There are following logical operators supported by Python language. Assume variable a holds 10 and variable b holds 20 then Used to reverse the logical state of its operand. Operator

Description

Example

and Logical AND

If both the operands are true then condition becomes true.

(a and b) is true.

or Logical OR

If any of the two operands are non-zero then condition becomes true.

(a or b) is true.

not Logical NOT

Used to reverse the logical state of its operand.

Not(a and b) is false.

Python Membership Operators Python’s membership operators test for membership in a sequence, such as strings, lists, or tuples. There are two membership operators as explained below − Operator

Description

Example

in

Evaluates to true if it finds a x in y, here in results in a 1 if x variable in the specified sequence is a member of sequence y. and false otherwise.

not in

Evaluates to true if it does not finds a variable in the specified sequence and false otherwise.

x not in y, here not in results in a 1 if x is not a member of sequence y.

Python Identity Operators Identity operators compare the memory locations of two objects. There are two Identity operators explained below − 31

Basic Computer Coding: Python Operator

Description

Example

is

Evaluates to true if the variables on either side of the operator point to the same object and false otherwise.

x is y, here is results in 1 if id(x) equals id(y).

is not

Evaluates to false if the variables on either side of the operator point to the same object and true otherwise.

x is not y, here is not results in 1 if id(x) is not equal to id(y).

1.5.2 Python Operators Precedence The following table lists all operators from highest precedence to lowest. Operator

Description

**

Exponentiation (raise to the power)

~+-

Complement, unary plus and minus (method names for the last two are +@ and -@)

* / % //

Multiply, divide, modulo and floor division

+-

Addition and subtraction

>>

^|

Bitwise exclusive `OR’ and regular `OR’

>=

Comparison operators

== !=

Equality operators

= %= /= //= Assignment operators -= += *= **= is is not

Identity operators

in not in

Membership operators

not or and Logical operators

32

Introduction to Python

Operator precedence affects how an expression is evaluated. x = 7 + 3 * 2; here, x is assigned 13, not 20 because operator * has higher precedence than +, so it first multiplies 3*2 and then adds into 7.

Here, operators with the highest precedence appear at the top of the table, those with the lowest appear at the bottom.

Example #!/usr/bin/python a = 20 b = 10 c = 15 d=5 e=0 e = (a + b) * c / d

#( 30 * 15 ) / 5

print “Value of (a + b) * c / d is “, e e = ((a + b) * c) / d

# (30 * 15 ) / 5

print “Value of ((a + b) * c) / d is “, e e = (a + b) * (c / d);

# (30) * (15/5)

print “Value of (a + b) * (c / d) is “, e e = a + (b * c) / d;

# 20 + (150/5)

print “Value of a + (b * c) / d is “, e When you execute the above program, it produces the following result − 33

Basic Computer Coding: Python

Value of (a + b) * c / d is 90 Value of ((a + b) * c) / d is 90 Value of (a + b) * (c / d) is 90 Value of a + (b * c) / d is 50

34

Introduction to Python

ROLE MODEL Guido van Rossum Guido van Rossum (Dutch: [ˈɣido vɑn ˈrɔsʏm, -səm]; born 31 January 1956) is a Dutch programmer best known as the author of the Python programming language, for which he was the “Benevolent Dictator For Life” (BDFL) until he stepped down from the position in July 2018.

Education and Life Van Rossum was born and raised in the Netherlands, where he received a master’s degree in mathematics and computer science from the University of Amsterdam in 1982. He has a brother, Just van Rossum, who is a type designer and programmer who designed the typeface used in the “Python Powered” logo. Guido lives in Belmont, California, with his wife, Kim Knapp, and their son. According to his home page and Dutch naming conventions, the “van” in his name is capitalized when he is referred to by surname alone, but not when using his first and last name together.

Work While working at the Centrum Wiskunde & Informatica (CWI), Van Rossum wrote and contributed a glob()routine to BSD Unix in 1986 and helped develop the ABC programming language. He once stated, “I try to mention ABC’s influence because I’m indebted to everything I learned during that project and to the people who worked on it.” He also created Grail, an early web browser written in Python, and engaged in discussions about the HTML standard. He has worked for various research institutes, including the Centrum Wiskunde & Informatica (CWI) in the Netherlands, the U.S. National Institute of Standards and Technology (NIST), and the Corporation for National Research Initiatives (CNRI). From 2000 until 2003 he

35

Basic Computer Coding: Python

worked for Zope corporation. In 2003 van Rossum left Zope for Elemental Security. While there he worked on a custom programming language for the organization. From 2005 to December 2012, he worked at Google, where he spent half of his time developing the Python language. In January 2013, he started working for Dropbox.

Python In December 1989, Van Rossum had been looking for a “’hobby’ programming project that would keep [him] occupied during the week around Christmas” as his office was closed when he decided to write an interpreter for a “new scripting language [he] had been thinking about lately: a descendant of ABC that would appeal to Unix/C hackers”. He attributes choosing the name “Python” to “being in a slightly irreverent mood (and [being] a big fan of Monty Python’s Flying Circus)”. He has explained that Python’s predecessor, ABC, was inspired by SETL, noting that ABC co-developer Lambert Meertens had “spent a year with the SETL group at NYU before coming up with the final ABC design”. In July 2018, Van Rossum announced that he would be stepping down from the position of BDFL of the Python programming language.

Computer Programming for Everybody In 1999, Van Rossum submitted a funding proposal to DARPA called “Computer Programming for Everybody,” in which he further defined his goals for Python: ■

An easy and intuitive language just as powerful as major competitors

■

Open source, so anyone can contribute to its development

■

Code that is as understandable as plain English

■

Suitability for everyday tasks, allowing for short development times

Python has grown to become a popular programming language. As of October 2017, it was the second most popular language on GitHub, a social coding website, behind Javascript and ahead of Java. According to a programming language popularity survey 36

Introduction to Python

it is consistently amongst the top 10 most mentioned languages in job postings. Furthermore, Python is consistently in the top 10 most popular languages according to the TIOBE Programming Community Index.

Mondrian At Google, Van Rossum developed Mondrian, a web-based code review system written in Python and used within the company. He named the software after the Dutch painter Piet Mondriaan. He named another related software projectafter Gerrit Rietveld, a Dutch designer.

Dropbox In 2013, Van Rossum started working at the cloud file storage company

Dropbox.

37

Basic Computer Coding: Python

SUMMARY ■

Python is a high-level, interpreted, interactive and objectoriented scripting language. Python is designed to be highly readable. It uses English keywords frequently where as other languages use punctuation, and it has fewer syntactical constructions than other languages.

■

Python distribution is available for a wide variety of platforms. You need to download only the binary code applicable for your platform and install Python.

■

A Python identifier is a name used to identify a variable, function, class, module or other object. An identifier starts with a letter A to Z or a to z or an underscore (_) followed by zero or more letters, underscores and digits (0 to 9).

■

A group of individual statements, which make a single code block are called suites in Python. Compound or complex statements, such as if, while, def, and class require a header line and a suite.

■

Variables are nothing but reserved memory locations to store values. This means that when you create a variable you reserve some space in memory.

■

Python variables do not need explicit declaration to reserve memory space. The declaration happens automatically when you assign a value to a variable. The equal sign (=) is used to assign values to variables.

■

Python’s dictionaries are kind of hash table type. They work like associative arrays or hashes found in Perl and consist of key-value pairs. A dictionary key can be almost any Python type, but are usually numbers or strings. Values, on the other hand, can be any arbitrary Python object.

■

Operators are the constructs which can manipulate the value of operands. Consider the expression 4 + 5 = 9. Here, 4 and 5 are called operands and + is called operator.

38

Introduction to Python

KNOWLEDGE CHECK 1.

2.

3.

4.

Which of the following is correct about Python? a.

Python is a high-level, interpreted, interactive and object-oriented scripting language.

b.

Python is designed to be highly readable.

c.

It uses English keywords frequently where as other languages use punctuation, and it has fewer syntactical constructions than other languages.

d.

All of the above.

Which of the following is correct about Python? a.

It supports functional and structured programming methods as well as OOP.

b.

It can be used as a scripting language or can be compiled to byte-code for building large applications.

c.

It provides very high-level dynamic data types and supports dynamic type checking.

d.

All of the above.

Which of the following environment variable for Python tells the Python interpreter where to locate the module files imported into a program? a.

Pythonpath

b.

Pythonstartup

c.

Pythoncaseok

d.

Pythonhome

Which of the following data types is not supported in python? a.

List

b.

Slice

c.

String

d.

Numbers

39

Basic Computer Coding: Python

5.

Which of the following is correct about tuples in python? a.

A tuple is another sequence data type that is similar to the list.

b.

A tuple consists of a number of values separated by commas.

c.

Unlike lists, however, tuples are enclosed within parentheses.

d.

All of the above.

REVIEW QUESTIONS 1.

What is Python? Name some of the features of python.

2.

What are the purpose of pythonpath, pythonstartup, Pythoncaseok, and pythonhome environment variable?

3.

What are the supported data types in python?

.

What are python’s dictionaries?

5.

What is the difference between tuples and lists in python?

Check Your Result 1. (d)

2. (d)

3. (a)

40

4. (b)

5. (d)

Introduction to Python

REFERENCES 1.

Deily, Ned (28 March 2018). “Python 3.7.0 is now available”. Python Insider. The Python Core Developers. Retrieved 29 March 2018.

2.

Downey, Allen B. (May 2012). Think Python: How to Think Like a Computer Scientist (Version 1.6.6 ed.). ISBN 978-0-521-72596-5.

3.

Guttag, John V. (2016-08-12). Introduction to Computation and Programming Using Python: With Application to Understanding Data. MIT Press. ISBN 978-0-262-52962-4.

4.

Hamilton, Naomi (5 August 2008). “The A-Z of Programming Languages: Python”. Computerworld. Archived from the original on 29 December 2008. Retrieved 31 March 2010.

5.

https://techterms.com/definition/python

6.

https://whatis.techtarget.com/definition/Python

7.

https://www.tutorialspoint.com/python/operators_ precedence_example.htm

8.

h t t p s : / / w w w. t u t o r i a l s p o i n t . c o m / p y t h o n / p y t h o n _ environment.htm

9.

https://www.tutorialspoint.com/python/python_overview. htm

10.

https://www.tutorialspoint.com/python/python_variable_ types.htm

11.

Lutz, Mark (2013). Learning Python (5th ed.). O’Reilly Media. ISBN 978-0-596-15806-4.

12.

Peterson, Benjamin (1 May 2018). “Python 2.7.15 released”. Python Insider. The Python Core Developers. Retrieved 1 May 2018.

13.

Pilgrim, Mark (2004). Dive Into Python. Apress. ISBN 978-1-59059-356-1.

14.

Pilgrim, Mark (2009). Dive Into Python 3. Apress. ISBN 978-1-4302-2415-0. Archived from the original on 201110-17. 41

Basic Computer Coding: Python

15.

Summerfield, Mark (2009). Programming in Python 3 (2nd ed.). Addison-Wesley Professional. ISBN 978-0-32168056-3.

16.

Zadka, Moshe; van Rossum, Guido (11 March 2001). “PEP 238 – Changing the Division Operator”. Python Enhancement Proposals. Python Software Foundation. Retrieved 23 October 2013.

42

CHAPTER 2

PYTHON FUNCTIONS, MODULES AND PACKAGES Everyone knows that any scripting language shootout that doesn’t show Python as the best language is faulty by design. ―Max M

Learning Objectives After studying this chapter, you will be able to: ■

Discuss the function in python

■

Describe the python modules and python packages

Basic Computer Coding: Python

INTRODUCTION A function is a block of organized, reusable code that is used to perform a single, related action. Functions provide better modularity for your application and a high degree of code reusing. A module allows you to logically organize your Python code. Grouping related code into a module makes the code easier to understand and use. A module is a Python object with arbitrarily named attributes that you can bind and reference. A package is a hierarchical file directory structure that defines a single Python application environment that consists of modules and subpackages and sub-subpackages, and so on.

2.1 FUNCTION IN PYTHON In Python, function is a group of related statements that perform a specific task. Functions help break our program into smaller and modular chunks. As our program grows larger and larger, functions make it more organized and manageable. Furthermore, it avoids repetition and makes code reusable.

2.1.1 Syntax of Function def function_name(parameters): “””docstring””” statement(s) Above shown is a function definition which consists of following components. ■

Keyword def marks the start of function header.

■

A function name to uniquely identify it. Function naming follows the same rules of writing identifiers in Python.

■

Parameters (arguments) through which we pass values to a function. They are optional.

■

A colon (:) to mark the end of function header.

44

Python Functions, Modules and Packages

■

Optional documentation string (docstring) to describe what the function does.

■

One or more valid python statements that make up the function body. Statements must have same indentation level (usually 4 spaces).

■

An optional return statement to return A statement is an a value from the function.

instruction that the Python interpreter can execute. We have only seen the assignment statement so far.

Example of a function def greet(name): “””This function greets to the person passed in as parameter””” print(“Hello, “ + name + “. Good morning!”)

How to call a function in python? Once we have defined a function, we can call it from another function, program or even the Python prompt. To call a function we simply type the function name with appropriate parameters. >>> greet(‘Paul’) Hello, Paul. Good morning!

2.1.2 Docstring The first string after the function header is called the docstring and is short for documentation string. It is used to explain in brief, what a function does. Although optional, documentation is a good programming practice. Unless you can remember what you had for dinner last week, always document your code. 45

Basic Computer Coding: Python

In the above example, we have a docstring immediately below the function header. We generally use triple quotes so that docstring can extend up to multiple lines. This string is available to us as __doc__ attribute of the function. For example: Try running the following into the Python shell to see the output. >>> print(greet.__doc__) This function greets to the person passed into the name parameter

2.1.3 The Return Statement The return statement is used to exit a function and go back to the place from where it was called. Syntax of Return return [expression_list] This statement can contain expression which gets evaluated and the value is returned. If there is no expression in the statement or the return statement itself is not present inside a function, then the function will return the None object.

For example: >>> print(greet(“May”)) Hello, May. Good morning! None Here, None is the returned value.

Example of Return def absolute_value(num): 46

Python Functions, Modules and Packages

“””This function returns the absolute value of the entered number””” if num >= 0:

Remember A function that does not have a return statement returns by default something called “None”.

return num else: return -num # Output: 2 print(absolute_value(2)) # Output: 4 print(absolute_value(-4))

2.1.4 How Function works in Python?

2.1.5 Python Function Arguments You can call a function by using the following types of formal arguments − ■

Required arguments

■

Keyword arguments

■

Default arguments

■

Variable-length arguments 47

Google began a project named Unladen Swallow in 2009 with the aim of speeding up the Python interpreter fivefold by using the LLVM, and of improving its multithreading ability to scale to thousands of cores.

Basic Computer Coding: Python

Required Arguments Required arguments are the arguments passed to a function in correct positional order. Here, the number of arguments in the function call should match exactly with the function definition. To call the function printme(), you definitely need to pass one argument, otherwise it gives a syntax error as follows − #!/usr/bin/python # Function definition is here def printme( str ): “This prints a passed string into this function” print str return; # Now you can call printme function printme() When the above code is executed, it produces the following result − Traceback (most recent call last): File “test.py”, line 11, in printme(); TypeError: printme() takes exactly 1 argument (0 given)

Keyword Arguments Keyword arguments are related to the function calls. When you use keyword arguments in a function call, the caller identifies the arguments by the parameter name. This allows you to skip arguments or place them out of order because the Python interpreter is able to use the keywords provided to match the values with parameters. You can also make keyword calls to the printme() function in the following ways − #!/usr/bin/python 48

Python Functions, Modules and Packages

# Function definition is here def printme( str ): “This prints a passed string into this function” print str return; # Now you can call printme function printme( str = “My string”) When the above code is executed, it produces the following result − My string

Parameter are

The following example gives more clear picture. commonly used, and are Note that the order of parameters does not referred to as parameters and arguments—or matter. more formally as a formal parameter and an actual parameter.

#!/usr/bin/python # Function definition is here def printinfo( name, age ): “This prints a passed info into this function” print “Name: “, name print “Age “, age return; # Now you can call printinfo function printinfo( age=50, name=”miki” ) When the above code is executed, it produces the following result − Name: miki Age 50

49

Basic Computer Coding: Python

Default Arguments A default argument is an argument that assumes a default value if a value is not provided in the function call for that argument. The following example gives an idea on default arguments, it prints default age if it is not passed − #!/usr/bin/python # Function definition is here def printinfo( name, age = 35 ): “This prints a passed info into this function” print “Name: “, name print “Age “, age return; # Now you can call printinfo function printinfo( age=50, name=”miki” ) printinfo( name=”miki” ) When the above code is executed, it produces the following result − Name: miki Age 50 Name: miki Age 35

Variable-length Arguments You may need to process a function for more arguments than you specified while defining the function. These arguments are called variable-length arguments and are not named in the function definition, unlike required and default arguments. Syntax for a function with non-keyword variable arguments is this − def functionname([formal_args,] *var_args_tuple ): “function_docstring” function_suite return [expression] 50

Python Functions, Modules and Packages

An asterisk (*) is placed before the variable name that holds the values of all nonkeyword variable arguments. This tuple remains empty if no additional arguments are specified during the function call. Following is a simple example − #!/usr/bin/python # Function definition is here def printinfo( arg1, *vartuple ): “This prints a variable passed arguments” print “Output is: “ print arg1 for var in vartuple: print var return; # Now you can call printinfo function printinfo( 10 ) printinfo( 70, 60, 50 ) When the above code is executed, it produces the following result − Output is: 10 Output is: 70 60 50

2.1.6 The Anonymous Functions These functions are called anonymous because they are not declared in the standard manner by using the def keyword. You can use the lambda keyword to create small anonymous functions. Lambda forms can take any number of arguments but return just one value in the form of an expression. They cannot contain commands or multiple expressions.

51

Basic Computer Coding: Python

An anonymous function cannot be a direct call to print because lambda requires an expression Lambda functions have their own local namespace and cannot access variables other than those in their parameter list and those in the global namespace. Although it appears that lambda’s are a one-line version of a function, they are not equivalent to inline statements in C or C++, whose purpose is by passing function stack allocation during invocation for performance reasons.

Syntax The syntax of lambda functions contains only a single statement, which is as follows − lambda [arg1 [,arg2,.....argn]]:expression Following is the example to show how lambda form of function works − #!/usr/bin/python # Function definition is here sum = lambda arg1, arg2: arg1 + arg2; # Now you can call sum as a function print “Value of total : “, sum( 10, 20 ) print “Value of total : “, sum( 20, 20 ) When the above code is executed, it produces the following result − Value of total : 30 Value of total : 40

52

Python Functions, Modules and Packages

2.1.7 The Return Statement The statement return [expression] exits a function, optionally passing back an expression to the caller. A return statement with no arguments is the same as return None. All the above examples are not returning any value. You can return a value from a function as follows − #!/usr/bin/python # Function definition is here def sum( arg1, arg2 ): # Add both the parameters and return them.” total = arg1 + arg2 print “Inside the function : “, total return total; # Now you can call sum function total = sum( 10, 20 ); print “Outside the function : “, total When the above code is executed, it produces the following result − Inside the function : 30 Outside the function : 30

2.2 PYTHON MODULES Python has a way to put definitions in a file and use them in a script or in an interactive instance of the interpreter. Such a file is called a module; definitions from a module can be imported into other modules or into the main module (the collection of variables that you have access to in a script executed at the top level and in calculator mode). A module is a file containing Python definitions and statements. The file name is the module name with the suffix .py appended. 53

Basic Computer Coding: Python

Within a module, the module’s name (as a string) is available as the value of the global variable __name__. For instance, use your favorite text editor to create a file called fibo. A file name is a name used to uniquely identify py in the current directory with the following a computer file stored in contents: a file system.

# Fibonacci numbers module

def fib(n):

# write Fibonacci series up to n

a, b = 0, 1 while b < n: print b, a, b = b, a+b

def fib2(n): # return Fibonacci series up to n result = [] a, b = 0, 1 while b < n: result.append(b) a, b = b, a+b return result Now enter the Python interpreter and import this module with the following command: >>> import fibo This does not enter the names of the functions defined in fibo directly in the current symbol table; it only enters the module name fibo there. 54

Python Functions, Modules and Packages

Using the module name you can access the functions: >>> fibo.fib(1000) 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 >>> fibo.fib2(100) [1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89] >>> fibo.__name__ ‘fibo’ If you intend to use a function often you can assign it to a local name: >>> fib = fibo.fib >>> fib(500) 1 1 2 3 5 8 13 21 34 55 89 144 233 377

2.2.1 More on Modules A module can contain executable statements as well as function definitions. These statements are intended to initialize the module. They are executed only the first time the module name is encountered in an import statement. (They are also run if the file is executed as a script.) Each module has its own private symbol table, which is used as the global symbol table by all functions defined in the module. Thus, the author of a module can use global variables in the module without worrying about accidental clashes with a user’s global variables. On the other hand, if you know what you are doing you can touch a module’s global variables with the same notation used to refer to its functions, modname.itemname. Modules can import other modules. It is customary but not required to place all import statements at the beginning of a module (or script, for that matter). The imported module names are placed in the importing module’s global symbol table. There is a variant of the import statement that imports names from a module directly into the importing module’s symbol table. For example: 55

Basic Computer Coding: Python

>>> from fibo import fib, fib2 >>> fib(500) 1 1 2 3 5 8 13 21 34 55 89 144 233 377 This does not introduce the module name from which the imports are taken in the local symbol table (so in the example, fibo is not defined). There is even a variant to import all names that a module defines: >>> from fibo import * >>> fib(500) 1 1 2 3 5 8 13 21 34 55 89 144 233 377 This imports all names except those beginning with an underscore (_). Note that in general the practice of importing * from a module or package is frowned upon, since it often causes poorly readable code. However, it is okay to use it to save typing in interactive sessions. If the module name is followed by as, then the name following as is bound directly to the imported module. >>> import fibo as fib >>> fib.fib(500) 0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 This is effectively importing the module in the same way that import fibo will do, with the only difference of it being available as fib. It can also be used when utilising from with similar effects: >>> from fibo import fib as fibonacci >>> fibonacci(500) 0 1 1 2 3 5 8 13 21 34 55 89 144 233 377

Executing Modules as Scripts When you run a Python module with python fibo.py the code in the module will be executed, just as if you imported it, but with the __name__ set to “__main__”. That means that by adding this code at the end of your module: 56

Python Functions, Modules and Packages

if __name__ == “__main__”: import sys fib(int(sys.argv[1])) you can make the file usable as a script as well as an importable module, because the code that parses the command line only runs if the module is executed as the “main” file: $ python fibo.py 50 1 1 2 3 5 8 13 21 34 If the module is imported, the code is not run: >>> import fibo >>> This is often used either to provide a convenient user interface to a module, or for testing purposes (running the module as a script executes a test suite).

The Module Search Path¶ When a module named spam is imported, the interpreter first searches for a built-in module with that name. If not found, it then searches for a file named spam.py in a list of directories given by the variable sys.path. sys.path is initialized from these locations: the directory containing the input script (or the current directory). PYTHONPATH (a list of directory names, with the same syntax as the shell variable PATH). the installation-dependent default. After initialization, Python programs can modify sys.path. The directory containing the script being run is placed at the beginning of the search path, ahead of the standard library path. This means that scripts in that directory will be loaded instead of modules of the same name in the library directory. This is an error unless the replacement is intended. As an important speed-up of the start-up time for short programs that use a lot of standard modules, if a file called spam.pyc exists in the directory where spam.py is found, this is assumed to contain an already-“byte-compiled” version of the module spam. The 57

Basic Computer Coding: Python

modification time of the version of spam.py used to create spam. pyc is recorded in spam.pyc, and the .pyc file is ignored if these don’t match. Normally, you don’t need to do anything to create the spam.pyc file. Whenever spam.py is successfully compiled, an attempt is made to write the compiled version to spam.pyc. It is not an error if this attempt fails; if for any reason the file is not written completely, the resulting spam.pyc file will be recognized as invalid and thus ignored later. The contents of the spam.pyc file are platform independent, so a Python module directory can be shared by machines of different architectures. Some tips for experts: ■

When the Python interpreter is invoked with the -O flag, optimized code is generated and stored in .pyo files. The optimizer currently doesn’t help much; it only removes assert statements. When -O is used, all bytecode is optimized; .pyc files are ignored and .py files are compiled to optimized bytecode.

■

Passing two -O flags to the Python interpreter (-OO) will cause the bytecode compiler to perform optimizations that could in some rare cases result in malfunctioning programs. Currently only __doc__ strings are removed from the bytecode, resulting in more compact .pyo files. Since some programs may rely on having these available, you should only use this option if you know what you’re doing.

■

A program doesn’t run any faster when it is read from a .pyc or .pyo file than when it is read from a .py file; the only thing that’s faster about .pyc or .pyo files is the speed with which they are loaded.

■

When a script is run by giving its name on the command line, the bytecode for the script is never written to a .pyc or .pyo file. Thus, the startup time of a script may be reduced by moving most of its code to a module and having a small bootstrap script that imports that module. It is also possible to name a .pyc or .pyo file directly on the command line.

■

It is possible to have a file called spam.pyc (or spam.pyo when -O is used) without a file spam.py for the same

58

Python Functions, Modules and Packages

module. This can be used to distribute a library of Python code in a form that is moderately hard to reverse engineer. ■

The module compileall can create .pyc files (or .pyo files when -O is used) for all modules in a directory.

2.2.2 Standard Modules¶ Python comes with a library of standard modules, described in a separate document, the Python Library Reference (“Library Reference” hereafter). Some modules are built into the interpreter; these provide access to operations that are not part of the core of the language but are nevertheless built in, either for efficiency or to provide access to operating system primitives such as system calls. The set of such modules is a configuration option which also depends on the underlying platform. For example, the winreg module is only provided on Windows systems. One particular module deserves some attention: sys, which is built into every Python interpreter. The variables sys.ps1 and sys.ps2 define the strings used as primary and secondary prompts: >>> import sys >>> sys.ps1 ‘>>> ‘ >>> sys.ps2 ‘... ‘ >>> sys.ps1 = ‘C> ‘ C> print ‘Yuck!’ Yuck! C> 59

The compileall module finds Python source files and compiles them to the byte-code representation, saving the results in .pyc or .pyo files.

Basic Computer Coding: Python

These two variables are only defined if the interpreter is in interactive mode. The variable sys.path is a list of strings that determines the interpreter’s search path for modules. It is initialized to a default path taken from the environment variable PYTHONPATH, or from a built-in default if PYTHONPATH is not set. You can modify it using standard list operations: >>> import sys >>> sys.path.append(‘/ufs/guido/lib/python’) The dir() Function¶ The built-in function dir() is used to find out which names a module defines. It returns a sorted list of strings: >>> import fibo, sys >>> dir(fibo) [‘__name__’, ‘fib’, ‘fib2’] >>> dir(sys) [‘__displayhook__’, ‘__doc__’, ‘__excepthook__’, ‘__name__’, ‘__ package__’, ‘__stderr__’, ‘__stdin__’, ‘__stdout__’, ‘_clear_type_cache’, ‘_current_frames’, ‘_getframe’, ‘_mercurial’, ‘api_version’, ‘argv’, ‘builtin_module_names’, ‘byteorder’, ‘call_tracing’, ‘callstats’, ‘copyright’, ‘displayhook’, ‘dont_write_bytecode’, ‘exc_clear’, ‘exc_ info’, ‘exc_traceback’, ‘exc_type’, ‘exc_value’, ‘excepthook’, ‘exec_prefix’, ‘executable’, ‘exit’, ‘flags’, ‘float_info’, ‘float_repr_style’, ‘getcheckinterval’, ‘getdefaultencoding’, ‘getdlopenflags’, ‘getfilesystemencoding’, ‘getobjects’, ‘getprofile’, ‘getrecursionlimit’, ‘getrefcount’, ‘getsizeof’, ‘gettotalrefcount’, ‘gettrace’, ‘hexversion’, ‘long_info’, ‘maxint’, ‘maxsize’, ‘maxunicode’, ‘meta_path’, ‘modules’, ‘path’, ‘path_hooks’, ‘path_importer_cache’, ‘platform’, ‘prefix’, ‘ps1’, ‘py3kwarning’, ‘setcheckinterval’, ‘setdlopenflags’, ‘setprofile’, ‘setrecursionlimit’, ‘settrace’, ‘stderr’, ‘stdin’, ‘stdout’, ‘subversion’, 60

Python Functions, Modules and Packages

‘version’, ‘version_info’, ‘warnoptions’] Without arguments, dir() lists the names you have defined currently: >>> a = [1, 2, 3, 4, 5] >>> import fibo >>> fib = fibo.fib >>> dir() [‘__builtins__’, ‘__name__’, ‘__package__’, ‘a’, ‘fib’, ‘fibo’, ‘sys’] Note that it lists all types of names: variables, modules, functions, etc. dir() does not list the names of built-in functions and variables. If you want a list of those, they are defined in the standard module __builtin__: >>> import __builtin__ >>> dir(__builtin__) [‘ArithmeticError’, ‘AssertionError’, ‘AttributeError’, ‘BaseException’, ‘BufferError’, ‘BytesWarning’, ‘DeprecationWarning’, ‘EOFError’, ‘Ellipsis’, ‘EnvironmentError’, ‘Exception’, ‘False’, ‘FloatingPointError’, ‘FutureWarning’, ‘GeneratorExit’, ‘IOError ’, ‘ImportError ’, ‘ImportWarning’, ‘IndentationError’, ‘IndexError’, ‘KeyError’, ‘KeyboardInterrupt’, ‘LookupError ’, ‘NotImplemented’,

‘MemoryError ’,

‘NameError ’,

‘None’,

‘NotImplementedError’, ‘OSError’, ‘OverflowError’, ‘PendingDeprecationWarning’, ‘ReferenceError’, ‘RuntimeError’, ‘RuntimeWarning’, ‘StandardError’, ‘StopIteration’, ‘SyntaxError’, ‘SyntaxWarning’, ‘SystemError’, ‘SystemExit’, ‘TabError’, ‘True’, ‘TypeError’, ‘UnboundLocalError’, ‘UnicodeDecodeError’, ‘UnicodeEncodeError’, ‘UnicodeError’, ‘UnicodeTranslateError’, ‘UnicodeWarning’, ‘UserWarning’, ‘ValueError’, ‘Warning’, 61

Basic Computer Coding: Python

‘ZeroDivisionError’, ‘_’, ‘__debug__’, ‘__doc__’, ‘__import__’, ‘__name__’, ‘__package__’, ‘abs’, ‘all’, ‘any’, ‘apply’, ‘basestring’, ‘bin’, ‘bool’, ‘buffer’, ‘bytearray’, ‘bytes’, ‘callable’, ‘chr’, ‘classmethod’, ‘cmp’, ‘coerce’, ‘compile’, ‘complex’, ‘copyright’, ‘credits’, ‘delattr’, ‘dict’, ‘dir’, ‘divmod’, ‘enumerate’, ‘eval’, ‘execfile’, ‘exit’, ‘file’, ‘filter’, ‘float’, ‘format’, ‘frozenset’, ‘getattr’, ‘globals’, ‘hasattr’, ‘hash’, ‘help’, ‘hex’, ‘id’, ‘input’, ‘int’, ‘intern’, ‘isinstance’, ‘issubclass’, ‘iter’, ‘len’, ‘license’, ‘list’, ‘locals’, ‘long’, ‘map’, ‘max’, ‘memoryview’, ‘min’, ‘next’, ‘object’, ‘oct’, ‘open’, ‘ord’, ‘pow’, ‘print’, ‘property’, ‘quit’, ‘range’, ‘raw_input’, ‘reduce’, ‘reload’, ‘repr’, ‘reversed’, ‘round’, ‘set’, ‘setattr’, ‘slice’, ‘sorted’, ‘staticmethod’, ‘str’, ‘sum’, ‘super’, ‘tuple’, ‘type’, ‘unichr’, ‘unicode’, ‘vars’, ‘xrange’, ‘zip’]

2.3 PYTHON PACKAGES Packages are a way of structuring Python’s module namespace by using “dotted module names”. For example, the module name A.B designates a submodule named B in a package named A. Just like the use of modules saves the authors of different modules from having to worry about each other’s global variable names, the use of dotted module names saves the authors of multi-module packages like NumPy or Pillow from having to worry about each other’s module names. Suppose you want to design a collection of modules (a “package”) for the uniform handling of sound files and sound data. There are many different sound file formats (usually recognized by their extension, for example: .wav, .aiff, .au), so you may need to create and maintain a growing collection of modules for the conversion between the various file formats. There are also many different operations you might want to perform on sound data (such as mixing, adding echo, applying an equalizer function, creating an artificial stereo effect), so in addition you will be writing a never-ending stream of modules to perform these operations. Here’s a possible structure for your package (expressed in terms of a hierarchical filesystem): 62

Python Functions, Modules and Packages

sound/

Top-level package

__init__.py

Initialize the sound package

formats/

Subpackage for file format conversions

__init__.py wavread.py wavwrite.py aiffread.py aiffwrite.py auread.py auwrite.py ... effects/

Subpackage for sound effects

__init__.py echo.py surround.py reverse.py ... filters/

Subpackage for filters __init__.py equalizer.py vocoder.py karaoke.py ...

When importing the package, Python searches through the directories on sys.path looking for the package subdirectory. The __init__.py files are required to make Python treat the directories as containing packages; this is done to prevent directories with a common name, such as string, from unintentionally hiding valid modules that occur later on the module search path. In the simplest case, __init__.py can just be an empty file, but it can also execute initialization code for the package or set the __all__ variable, described later. 63

Basic Computer Coding: Python

Users of the package can import individual modules from the package, for example: import sound.effects.echo This loads the submodule sound.effects.echo. It must be referenced with its full name. sound.effects.echo.echofilter(input, output, delay=0.7, atten=4) An alternative way of importing the submodule is: from sound.effects import echo This also loads the submodule echo, and makes it available without its package prefix, so it can be used as follows: echo.echofilter(input, output, delay=0.7, atten=4) Yet another variation is to import the desired function or variable directly: from sound.effects.echo import echofilter Again, this loads the submodule echo, but this makes its function echofilter() directly available: echofilter(input, output, delay=0.7, atten=4) Note that when using from package import item, the item can be either a submodule (or subpackage) of the package, or some other name defined in the package, like a function, class or variable. The import statement first tests whether the item is defined in the package; if not, it assumes it is a module and attempts to load it. If it fails to find it, an ImportError exception is raised. Contrarily, when using syntax like import item.subitem.subsubitem, each item except for the last must be a package; the last item can be a module or a package but can’t be a class or function or variable defined in the previous item.

2.3.1 Importing * From a Package Now what happens when the user writes from sound.effects import *? Ideally, one would hope that this somehow goes out to the filesystem, finds which submodules are present in the package, and imports them all. This could take a long time and importing

64

Python Functions, Modules and Packages

sub-modules might have unwanted side-effects that should only happen when the sub-module is explicitly imported. The only solution is for the package author to provide an explicit index of the package. The import statement uses the following convention: if a package’s __init__.py code defines a list named __all__, it is taken to be the list of module names that should be imported when from package import * is encountered. It is up to the package author to keep this list up-to-date when a new version of the package is released. Package authors may also decide not to support it, if they don’t see a use for importing * from their package. For example, the file sound/effects/__init__.py could contain the following code: __all__ = [“echo”, “surround”, “reverse”] This would mean that from sound.effects import * would import the three named submodules of the sound package. If __all__ is not defined, the statement from sound.effects import * does not import all submodules from the package sound.effects into the current namespace; it only ensures that the package sound. effects has been imported (possibly running any initialization code in __init__.py) and then imports whatever names are defined in the package. This includes any names defined (and submodules explicitly loaded) by __init__.py. It also includes any submodules of the package that were explicitly loaded by previous import statements. Consider this code: import sound.effects.echo import sound.effects.surround from sound.effects import * In this example, the echo and surround modules are imported in the current namespace because they are defined in the sound.effects package when the from...import statement is executed. (This also works when __all__ is defined.) Although certain modules are designed to export only names that follow certain patterns when you use import *, it is still considered bad practice in production code.

65

Basic Computer Coding: Python

Remember, there is nothing wrong with using from Package import specific_submodule! In fact, this is the recommended notation unless the importing module needs to use submodules with the same name from different packages.

2.3.2 Intra-package References The submodules often need to refer to each other. For example, the surround module might use the echo module. In fact, such references are so common that the import statement first looks in the containing package before looking in the standard module search path. Thus, the surround module can simply use import echo or from echo import echofilter. If the imported module is not found in the current package (the package of which the current module is a submodule), the import statement looks for a top-level module with the given name. When packages are structured into subpackages (as with the sound package in the example), you can use absolute imports to refer to submodules of siblings packages. For example, if the module sound. filters.vocoder needs to use the echo module in the sound.effects package, it can use fromsound.effects import echo. Starting with Python 2.5, in addition to the implicit relative imports described above, you can write explicit relative imports with the from module import nameform of import statement. These explicit relative imports use leading dots to indicate the current and parent packages involved in the relative import. From the surround module for example, you might use: from . import echo from .. import formats from ..filters import equalizer Note that both explicit and implicit relative imports are based on the name of the current module. Since the name of the main module is always “__main__”, modules intended for use as the main module of a Python application should always use absolute imports.

66

Python Functions, Modules and Packages

2.3.3 Packages in Multiple Directories Packages support one more special attribute, __path__. This is initialized to be a list containing the name of the directory holding the package’s __init__.py before the code in that file is executed. This variable can be modified; doing so affects future searches for modules and subpackages contained in the package. While this feature is not often needed, it can be used to extend the set of modules found in a package.

67

Basic Computer Coding: Python

SUMMARY ■

In Python, function is a group of related statements that perform a specific task. Functions help break our program into smaller and modular chunks. As our program grows larger and larger, functions make it more organized and manageable.

■

Required arguments are the arguments passed to a function in correct positional order. Here, the number of arguments in the function call should match exactly with the function definition.

■

A default argument is an argument that assumes a default value if a value is not provided in the function call for that argument.

■

Python has a way to put definitions in a file and use them in a script or in an interactive instance of the interpreter. Such a file is called a module; definitions from a module can be imported into other modules or into the main module (the collection of variables that you have access to in a script executed at the top level and in calculator mode).

■

Packages are a way of structuring Python’s module namespace by using “dotted module names”.

68

Python Functions, Modules and Packages

KNOWLEDGE CHECK 1.

2.

3.

4.

5.

Which of these definitions correctly describes a module? a.

Denoted by triple quotes for providing the specification of certain program elements

b.

Design and implementation of specific functionality to be incorporated into a program

c.

Defines the specification of how it is to be used

d.

Any program that reuses code

Which of the following is the use of function in python? a.

Functions are reusable pieces of programs

b.

Functions don’t provide better modularity for your application

c.

you can’t also create your own functions

d.

All of the mentioned

Which keyword is use for function? a.

Fun

b.

Define

c.

Def

d.

Function

Which of the following is not an advantage of using modules? a.

Provides a means of reuse of program code

b.

Provides a means of dividing up tasks

c.

Provides a means of reducing the size of the program

d.

Provides a means of testing individual parts of the program

Program code making use of a given module is called a ……… of the module. a.

Client

b.

Docstring 69

Basic Computer Coding: Python

c.

Interface

d.

Modularity

REVIEW QUESTIONS 1.

How Function works in Python?

2.

Discuss the python function arguments.

3.

How to create a Python module.

4.

How to create a module that is executable as a standalone script.

5.

What is the difference between a python module and a python package?

Check Your Result 1. (a)

2. (a)

3. (c)

70

4. (c)

5. (a)

Python Functions, Modules and Packages

REFERENCES 1.

http://codefruxtechnology.com/pdf/PythonSyllabus.pdf

2.

https://docs.python.org/3/tutorial/modules.html

3.

https://github.com/PyGithub/PyGithub

4.

https://realpython.com/python-modules-packages/

5.

https://www.codesdope.com/python-boolean/

6.

https://www.geeksforgeeks.org/bool-in-python/

7.

https://www.programiz.com/python-programming/ function-argument

8.

https://www.programiz.com/python-programming/ methods/built-in/bool

9.

https://www.programiz.com/python-programming/ modules

10.

https://www.tutorialspoint.com/python/python_functions. htm

11.

https://www.w3schools.com/python/python_functions.asp

12.

Kuchling, A. M. “Functional Programming HOWTO”. Python v2.7.2 documentation. Python Software Foundation. Retrieved 9 February 2012.

13.

The Python Tutorial. Python Software Foundation. Retrieved 20 February 2012. It is a mixture of the class mechanisms found in C++ and Modula-3

71

72

CHAPTER 3

DICTIONARIES, SETS, AND FILES “Python is an experiment in how much freedom programmers need. Too much freedom and nobody can read another’s code; too little and expressiveness is endangered.” ―Guido van Rossum

Learning Objectives After studying this chapter, you will be able to: ■

Understand python dictionaries

■

Explain python sets

■

Discuss about files

Basic Computer Coding: Python

INTRODUCTION Python dictionaries are something completely different—they are not sequences at all, but are instead known as mappings. Mappings are also collections of other objects, but they store objects by key instead of by relative position. In fact, mappings don’t maintain any reliable left-to-right order; they simply map keys to associated values. Dictionaries, the only mapping type in Python’s core objects set, are also mutable: like lists, they may be changed in place and can grow and shrink on demand. Also like lists, they are a flexible tool for representing collections, but their more mnemonic keys are better suited when a collection’s items are named or labeled—fields of a database record, for example. Sets are constructed from a sequence (or some other iterable object). Since sets cannot have duplicated, there are usually used to build sequence of unique items (e.g., set of identifiers). File objects are Python code’s main interface to external files on your computer. They can be used to read and write text memos, audio clips, Excel documents, saved email messages, and whatever else you happen to have stored on your machine. Files are a core type, but they’re something of an oddball—there is no specific literal syntax for creating them. Rather, to create a file object, you call the built-in open function, passing in an external filename and an optional processing mode as strings.

3.1 PYTHON DICTIONARIES A dictionary is a collection which is unordered, changeable and indexed. In Python dictionaries are written with curly brackets, and they have keys and values.

74

Dictionaries, Sets, and Files

Example: Create and print a dictionary: thisdict =

{

“brand”: “Ford”, “model”: “Mustang”, “year”: 1964 } print(thisdict)

3.1.1 Accessing Dictionary Elements We can call the values of a dictionary by referencing the related keys.

Accessing Data Items with Keys Because dictionaries offer key-value pairs for storing data, they can be important elements in your Python program. If we want to isolate Sammy’s username, we can do so by calling sammy[‘username’]. Let’s print that out: print(sammy[‘username’]) Output 75

Basic Computer Coding: Python

sammy-shark Dictionaries behave like a database in that instead of calling an integer to get a particular index value as you would with a list, you assign a value to a key and can call that key to get its related value. By invoking the key ‘username’ we receive the value of that key, which is ‘sammy-shark’. The remaining values in the sammy dictionary can similarly be called using the same format: sammy[‘followers’] # Returns 987 sammy[‘online’] # Returns True By making use of dictionaries’ key-value pairs, we can reference keys to retrieve values.

Using Methods to Access Elements In addition to using keys to access values, we can also work with some built-in methods: ■

dict.keys() isolates keys

■

dict.values() isolates values

■

dict.items() returns items in a list format of (key, value) tuple pairs

To return the keys, we would use the dict.keys() method. In our example, that would use the variable name and be sammy.keys(). Let’s pass that to a print() method and look at the output: print(sammy.keys()) Output dict_keys([‘followers’, ‘username’, ‘online’]) We receive output that places the keys within an iterable view object of the dict_keys class. The keys are then printed within a list format. This method can be used to query across dictionaries. For example, 76

Dictionaries, Sets, and Files

we can take a look at the common keys shared between two dictionary data structures: sammy = {‘username’: ‘sammy-shark’, ‘online’: Data structure is a data organization, True, ‘followers’: 987} management and

jesse = {‘username’: ‘JOctopus’, ‘online’: False, storage format that enables efficient access ‘points’: 723} for common_key in sammy.keys() & jesse. keys(): print(sammy[common_key], jesse[common_ key]) The dictionary sammy and the dictionary jesse are each a user profile dictionary. Their profiles have different keys, however, because Sammy has a social profile with associated followers, and Jesse has a gaming profile with associated points. The 2 keys they have in common are usernameand online status, which we can find when we run this small program: Output sammy-shark JOctopus True False We could certainly improve on the program to make the output more user-readable, but this illustrates that dict.keys() can be used to check across various dictionaries to see what they share in common or not. This is especially useful for large dictionaries. Similarly, we can use the dict.values() method to query the values in the sammy dictionary, which would be constructed as sammy.values(). Let’s print those out: sammy = {‘username’: ‘sammy-shark’, ‘online’: True, ‘followers’: 987}

77

and modification.

Basic Computer Coding: Python

print(sammy.values()) Output dict_values([True, ‘sammy-shark’, 987]) Both the methods keys() and values() return unsorted lists of the keys and values present in the sammy dictionary with the view objects of dict_keys and dict_values respectively. If we are interested in all of the items in a dictionary, we can access them with the items() method: print(sammy.items()) Output dict_items([(‘online’, True), (‘username’, ‘sammy-shark’), (‘followers’, 987)]) The returned format of this is a list made up of (key, value) tuple pairs with the dict_items view object. We can iterate over the returned list format with a for loop. For example, we can print out each of the keys and values of a given dictionary, and then make it more human-readable by adding a string: for key, value in sammy.items(): print(key, ‘is the key for the value’, value) Output online is the key for the value True followers is the key for the value 987 username is the key for the value sammy-shark The for loop above iterated over the items within the sammy dictionary and printed out the keys and values line by line, with information to make it easier to understand by humans. We can use built-in methods to access items, values, and keys from dictionary data structures.

3.1.2 Modifying Dictionaries Dictionaries are a mutable data structure, so you are able to modify them. In this section, we will go over adding and deleting dictionary elements. 78

Dictionaries, Sets, and Files

Adding and Changing Dictionary Elements Without using a method or function, you can add key-value pairs to dictionaries by using the following syntax: dict[key] = value We’ll look at how this works in practice by adding a key-value pair to a dictionary called usernames: usernames = {‘Sammy’: ‘sammy-shark’, ‘Jamie’: ‘mantisshrimp54’} usernames[‘Drew’] = ‘squidly’ print(usernames) Output {‘Drew’: ‘squidly’, ‘Sammy’: ‘sammy-shark’, ‘Jamie’: ‘mantisshrimp54’} We see now that the dictionary has been updated with the ‘Drew’: ‘squidly’ key-value pair. Because dictionaries may be unordered, this pair may occur anywhere in the dictionary output. If we use the usernames dictionary later in our program file, it will include the additional key-value pair. Additionally, this syntax can be used for modifying the value assigned to a key. In this case, we will reference an existing key and pass a different value to it. Let’s consider a dictionary drew that is one of the users on a given network. We will say that this user got a bump in followers today, so we need to update the integer value passed to the ‘followers’ key. We’ll use the print() function to check that the dictionary was modified. drew = {‘username’: ‘squidly’, ‘online’: True, ‘followers’: 305} 79

Basic Computer Coding: Python

drew[‘followers’] = 342 print(drew) Output {‘username’: ‘squidly’, ‘followers’: 342, ‘online’: True} In the output, we see that the number of followers jumped from the integer value of 305 to 342. We can use this method for adding key-value pairs to dictionaries with user-input. Let’s write a quick program, usernames.py that runs on the command line and allows input from the user to add more names and associated usernames: usernames.py # Define original dictionary usernames = {‘Sammy’: ‘sammy-shark’, ‘Jamie’: ‘mantisshrimp54’} # Set up while loop to iterate while True: # Request user to enter a name print(‘Enter a name:’) # Assign to name variable name = input() # Check whether name is in the dictionary and print feedback if name in usernames: print(usernames[name] + ‘ is the username of ‘ + name) # If the name is not in the dictionary... else: # Provide feedback print(‘I don\’t have ‘ + name + ‘\’s username, what is it?’) 80

Dictionaries, Sets, and Files

# Take in a new username for the associated name username = input() # Assign username value to name key usernames[name] = username # Print feedback that the data was updated print(‘Data updated.’) Let’s run the program on the command line: ■

python usernames.py

When we run the program we’ll get something like the following output: Output Enter a name: Sammy sammy-shark is the username of Sammy Enter a name: Jesse I don’t have Jesse’s username, what is it? JOctopus Data updated. Enter a name: When we are done testing the program, we can press CTRL + C to escape the program. You can set up a trigger to quit the program (such as typing the letter q) with a conditional statement to improve the code. This shows how you can modify dictionaries interactively. With this particular program, as soon as you exit the program with CTRL + 81

Basic Computer Coding: Python

C you’ll lose all your data unless you implement a way to handle reading and writing files. We can also add and modify dictionaries by using the dict.update() method. This varies from the append() method available in lists. In the jesse dictionary below, let’s add the key ‘followers’ and give it an integer value with jesse.update(). Following that, let’s print() the updated dictionary. jesse = {‘username’: ‘JOctopus’, ‘online’: False, ‘points’: 723} jesse.update({‘followers’: 481}) print(jesse) Output {‘followers’: 481, ‘username’: ‘JOctopus’, ‘points’: 723, ‘online’: False} From the output, we can see that we successfully added the ‘followers’: 481 key-value pair to the dictionary jesse. We can also use the dict.update() method to modify an existing key-value pair by replacing a given value for a specific key. Let’s change the online status of Sammy from True to False in the sammy dictionary: sammy = {‘username’: ‘sammy-shark’, ‘online’: True, ‘followers’: 987} sammy.update({‘online’: False}) print(sammy) Output {‘username’: ‘sammy-shark’, ‘followers’: 987, ‘online’: False} The line sammy.update({‘online’: False}) references the existing key ‘online’ and modifies its Boolean value from True to False. When we call to print() the dictionary, we see the update take place in the output. To add items to dictionaries or modify values, we can use wither the dict[key] = value syntax or the method dict.update().

82

Dictionaries, Sets, and Files

Deleting Dictionary Elements Just as you can add key-value pairs and change values within the dictionary data type, you can also delete items within a dictionary.

To remove a key-value pair from a dictionary, we will use the following syntax: del dict[key] Let’s take the jesse dictionary that represents one of the users. We’ll say that Jesse is no longer using the online platform for playing games, so we’ll remove the item associated with the ‘points’ key. Then, we’ll print the dictionary out to confirm that the item was deleted: jesse = {‘username’: ‘JOctopus’, ‘online’: False, ‘points’: 723, ‘followers’: 481} del jesse[‘points’] print(jesse) Output {‘online’: False, ‘followers’: 481}

‘username’:

‘JOctopus’,

The line del jesse[‘points’] removes the keyvalue pair ‘points’: 723 from the jesse dictionary. 83

Syntax is the grammatical structure of sentences. The format in which words and phrases are arranged to create sentences is called syntax.

Basic Computer Coding: Python

If we would like to clear a dictionary of all of its values, we can do so with the dict.clear() method. This will keep a given dictionary in case we need to use it later in the program, but it will no longer contain any items. Let’s remove all the items within the jesse dictionary: jesse = {‘username’: ‘JOctopus’, ‘online’: False, ‘points’: 723, ‘followers’: 481} jesse.clear()

print(jesse) Output {} The output shows that we now have an empty dictionary devoid of key-value pairs. If we no longer need a specific dictionary, we can use del to get rid of it entirely: del jesse print(jesse) When we run a call to print() after deleting the jesse dictionary, we’ll receive the following error: Output ... NameError: name ‘jesse’ is not defined Because dictionaries are mutable data types, they can be added to, modified, and have items removed and cleared.

84

Dictionaries, Sets, and Files The del keyword removes the item with the specified key name: thisdict = { “brand”: “Ford”, “model”: “Mustang”, “year”: 1964 } del thisdict[“model”] print(thisdict)

3.1.3 The dict() Constructor It is also possible to use the dict() constructor to make a dictionary: Example thisdict = dict(brand=”Ford”, model=”Mustang”, year=1964) # note that keywords are not string literals # note the use of equals rather than colon for the assignment print(thisdict)

3.1.4 Dictionary Methods Python has a set of built-in methods that you can use on dictionaries. Method

Description

clear()

Removes all the elements from the dictionary

copy()

Returns a copy of the dictionary

fromkeys()

Returns a dictionary with the specified keys and values

get()

Returns the value of the specified key

items()

Returns a list containing the a tuple for each key value pair

keys()

Returns a list contianing the dictionary’s keys

pop()

Removes the element with the specified key 85

Basic Computer Coding: Python

popitem()

Removes the last inserted key-value pair

setdefault()

Returns the value of the specified key. If the key does not exist: insert the key, with the specified value

update()

Updates the dictionary with the specified key-value pairs

values()

Returns a list of all the values in the dictionary

Remember Although we just used a dictionary to link names to phone numbers in this extended example, we can use a dictionary to link any one type of object to another type of object.

3.1.5 Aliasing and Copying Because dictionaries are mutable, you need to be aware of aliasing. Whenever two variables refer to the same object, changes to one affect the other.

If you want to modify a dictionary and keep a copy of the original, use the copy method. For example, opposites is a dictionary that contains pairs of opposites: >>> opposites = {‘up’: ‘down’, ‘right’: ‘wrong’, ‘true’: ‘false’} >>> an_alias = opposites >>> a_copy = opposites.copy() an_alias and opposites refer to the same object; a_copy refers to a fresh copy of the same dictionary. If we modify alias, opposites is also changed: >>> an_alias[‘right’] = ‘left’ 86

Dictionaries, Sets, and Files

>>> opposites[‘right’] ‘left’ If we modify a_copy, opposites is unchanged: >>> a_copy[‘right’] = ‘privilege’ >>> opposites[‘right’] ‘left’

3.2 PYTHON SETS Sets are a collection of distinct (unique) objects. These are useful to create lists that only hold unique values in the dataset. It is an unordered collection but a mutable one, this is very helpful when going through a huge dataset. x_set = set(‘CAKE&COKE’) y_set = set(‘COOKIE’) print(x_set) {‘A’, ‘&’, ‘O’, ‘E’, ‘C’, ‘K’} print(y_set) # Single unique ‘o’ {‘I’, ‘O’, ‘E’, ‘C’, ‘K’} print(x - y) # All the elements in x_set but not in y_set --------------------------------------------------------------------------NameError

Traceback (most recent call last)

in () ----> 1 print(x - y) # All the elements in x_set but not in y_set

NameError: name ‘x’ is not defined print(x_set|y_set) # Unique elements in x_set or y_set or both {‘C’, ‘&’, ‘E’, ‘A’, ‘O’, ‘K’, ‘I’}

87

Basic Computer Coding: Python

Remember Sets can store anything, not just strings. It’s just easiest to illustrate the set methods using sets of strings.

print(x_set & y_set) # Elements in both x_set and y_set {‘O’, ‘E’, ‘K’, ‘C’}

3.2.1 Defining a Set You can define a set as simple as by naming all of its elements in brackets. The only exception is empty set, which can be created using the function set(). If set(..) has a list, a string or a tuple as a parameter, it will return a set composed of its elements.

Let’s see what all that means, and how you can work with sets in Python. A set can be created in two ways. First, you can define a set with the built-in set() function: x = set() In this case, the argument is an iterable—again, for the moment, think list or tuple—that generates the list of objects to be included in the set. This is analogous to the argument given to the .extend() list method: >>> x = set([‘foo’, ‘bar’, ‘baz’, ‘foo’, ‘qux’]) >>> x {‘qux’, ‘foo’, ‘bar’, ‘baz’} >>> x = set((‘foo’, ‘bar’, ‘baz’, ‘foo’, ‘qux’)) >>> x 88

Dictionaries, Sets, and Files

{‘qux’, ‘foo’, ‘bar’, ‘baz’} Strings are also iterable, so a string can be passed to set() as well. You have already seen that list(s) generates a list of the characters in the string s. Similarly, set(s) generates a set of the characters in s: >>> s = ‘quux’ >>> list(s) [‘q’, ‘u’, ‘u’, ‘x’] >>> set(s) {‘x’, ‘u’, ‘q’} You can see that the resulting sets are unordered: the original order, as specified in the definition, is not necessarily preserved. Additionally, duplicate values are only represented in the set once, as with the string ‘foo’ in the first two examples and the letter ‘u’ in the third. Alternately, a set can be defined with curly braces ({}): x = {, , ..., } When a set is defined this way, each becomes a distinct element of the set, even if it is an iterable. This behavior is similar to that of the .append() list method. Thus, the sets shown above can also be defined like this: >>> x = {‘foo’, ‘bar’, ‘baz’, ‘foo’, ‘qux’} >>> x {‘qux’, ‘foo’, ‘bar’, ‘baz’} >>> x = {‘q’, ‘u’, ‘u’, ‘x’} >>> x 89

String is traditionally a sequence of characters, either as a literal constant or as some kind of variable.

Basic Computer Coding: Python

{‘x’, ‘q’, ‘u’} To recap: ■

The argument to set() is an iterable. It generates a list of elements to be placed into the set.

■

The objects in curly braces are placed into the set intact, even if they are iterable.

Observe the difference between these two set definitions: >>> {‘foo’} {‘foo’} >>> set(‘foo’) {‘o’, ‘f’} A set can be empty. However, recall that Python interprets empty curly braces ({}) as an empty dictionary, so the only way to define an empty set is with the set() function: >>> x = set() >>> type(x)

>>> x set() >>> x = {} >>> type(x)

An empty set is falsy in Boolean context: >>> x = set() >>> bool(x) False >>> x or 1 1 >>> x and 1 90

Dictionaries, Sets, and Files

set() You might think the most intuitive sets would contain similar objects—for example, even numbers or surnames: >>> s1 = {2, 4, 6, 8, 10} >>> s2 = {‘Smith’, ‘McArthur ’, ‘Wilson’, ‘Johansson’} Python does not require this, though. The elements in a set can be objects of different types: >>> x = {42, ‘foo’, 3.14159, None} >>> x {None, ‘foo’, 42, 3.14159} Don’t forget that set elements must be immutable. For example, a tuple may be included in a set: >>> x = {42, ‘foo’, (1, 2, 3), 3.14159} >>> x {42, ‘foo’, 3.14159, (1, 2, 3)} But lists and dictionaries are mutable, so they can’t be set elements: >>> a = [1, 2, 3] >>> {a} Traceback (most recent call last): File “”, line 1, in {a} TypeError: unhashable type: ‘list’ >>> d = {‘a’: 1, ‘b’: 2} >>> {d} Traceback (most recent call last): File “”, line 1, in 91

Earlier versions of Python used a cryptic way to format strings. It is considered deprecated and will eventually disappear from the language.

Basic Computer Coding: Python

{d} TypeError: unhashable type: ‘dict’

3.2.2 Set Size and Membership The len() function returns the number of elements in a set, and the in and not inoperators can be used to test for membership: >>> x = {‘foo’, ‘bar’, ‘baz’} >>> len(x) 3 >>> ‘bar’ in x True >>> ‘qux’ in x False

3.2.3 Methods for Sets add(x) Method Adds the item x to set if it is not already present in the set. people = {“Jay”, “Idrish”, “Archil”} people.add(“Daxit”) -> This will add Daxit in people set. union(s) Method Returns a union of two set.Using the ‘|’ operator between 2 sets is the same as writing set1.union(set2) people = {“Jay”, “Idrish”, “Archil”} vampires = {“Karan”, “Arjun”} population = people.union(vampires) OR population = people|vampires -> Set population set will have components of both people and 92

Dictionaries, Sets, and Files

vampire intersect(s) Method Returns an intersection of two sets.The ‘&’ operator comes can also be used in this case. victims = people.intersection(vampires) -> Set victims will contain the common element of people and vampire difference(s) Method Returns a set containing all the elements of invoking set but not of the second set. We can use ‘-‘ operator here. safe = people.difference(vampires) OR safe = people – vampires -> Set safe will have all the elements that are in people but not vampire clear() Method Empties the whole set. victims.clear() -> Clears victim set However there are two major pitfalls in Python sets: ■

The set doesn’t maintain elements in any particular order.

■

Only instances of immutable types can be added to a Python set.

3.2.4 Creating a Set A set is created by using the set() function or placing all the elements within a pair of curly braces.

93

Basic Computer Coding: Python

When the above code is executed, it produces the following result. Please note how the order of the elements has changed in the result.

3.2.5 Accessing Values in a Set We cannot access individual values in a set. We can only access all the elements together as shown above. But we can also get a list of individual elements by looping through the set.

When the above code is executed, it produces the following result.

3.2.6 Adding Items to a Set We can add elements to a set by using add() method. Again as discussed there is no specific index attached to the newly added element. 94

Dictionaries, Sets, and Files

When the above code is executed, it produces the following result.

3.2.7 Removing Item from a Set We can remove elements from a set by using discard() method. Again as discussed there is no specific index attached to the newly added element.

When the above code is executed, it produces the following result.

3.2.8 Union of Sets The union operation on two sets produces a new set containing all the distinct elements from both the sets. In the below example the element “Wed” is present in both the sets.

When the above code is executed, it produces the following result. Please note the result has only one “wed”.

95

Basic Computer Coding: Python

3.2.9 Intersection of Sets The intersection operation on two sets produces a new set containing only the common elements from both the sets. In the below example the element “Wed” is present in both the sets.

When the above code is executed, it produces the following result. Please note the result has only one “wed”.

3.2.10 Difference of Sets The difference operation on two sets produces a new set containing only the elements from the first set and none from the second set. In the below example the element “Wed” is present in both the sets so it will not be found in the result set.

When the above code is executed, it produces the following result. Please note the result has only one “wed”.

3.2.11 Compare Sets We can check if a given set is a subset or superset of another set. The result is True or False depending on the elements present in the sets. 96

Dictionaries, Sets, and Files

When the above code is executed, it produces the following result. When the above code is executed, it produces the following result.

3.3 FILES Files are traditionally a part of data structures. And although big data is commonplace in the data science industry, a programming language without the capability to store and retrieve previously stored information would hardly be useful. You still have to make use of the all Programming language the data sitting in files across databases and is a formal language, which comprises a set you will learn how to do this. of instructions used to

The syntax to read and write files in Python produce various kinds is similar to other programming languages of output. but a lot easier to handle. Here are some of the basic functions that will help you to work with files using Python: ■

open() to open files in your system, the filename is the name of the file to be opened;

■

read() to read entire files;

■

readline() to read one line at a time;

■

write() to write a string to a file, and return the number of characters written; And 97

Basic Computer Coding: Python

■

close() to close the file.

# File modes (2nd argument): ‘r’(read), ‘w’(write), ‘a’(appending), ‘r+’(both reading and writing) f = open(‘file_name’, ‘w’) # Reads entire file f.read() # Reads one line at a time f.readline() # Writes the string to the file, returning the number of char written f.write(‘Add this line.’) f.close() The second argument in the open() function is the file mode. It allows you to specify whether you want to read (r), write (w), append (a) or both read and write (r+).

3.3.1 The open function The open function takes two arguments. The first is the name of the file, and the second is the mode. Mode ‘w’ means that we are opening the file for writing. Mode ‘r’ means reading, and mode ‘a’ means appending. Let’s begin with an example that shows these three modes in operation:

98

Dictionaries, Sets, and Files

Opening a file creates what we call a file descriptor. In this example, the variable myfile refers to the new descriptor object. Our program calls methods on the descriptor, and this makes changes to the actual file which is located in non-volatile storage. The first line opens the test.txt for writing. If there is no file named test.txt on the disk, it will be created. If there already is one, it will be replaced by the file we are writing and any previous data in it will be lost. To put data in the file we invoke the write method on the file descriptor. We do this three times in the example above, but in bigger programs, the three separate calls to write will usually be replaced by a loop that writes many more lines into the file. The write method returns the number of bytes (characters) written to the file. Closing the file handle tells the system that we are done writing and makes the disk file available for reading by other programs (or by our own program). We finish this example by openning test.txt for reading. We then call the read method, assigning the contents of the file, which is a string, to a variable named contents, and finally print contents to see that it is indeed what we wrote to the file previously. If we want to add to an already existing file, use the append mode.

99

Remember Many common files you may use, such as Word documents or Excel spreadsheets are NOT text documents. They have their own complicated file formats. Text files are ones you can create in simple text editors, by simply typing regular keys without any special features, such as bolding text, different fonts, etc.

Basic Computer Coding: Python

3.3.2 Opening a File that Doesn’t Exist If we try to open a file that doesn’t exist, we get an error:

There is nothing wrong with the syntax of the line that resulted in the error. The error occurred because the file did not exist. Errors like these are called exceptions. Most modern programming languages provide support for dealing with situations like this. The process is called exception handling. In Python, exceptions are handled with the try Error is something ... except statement.

you have done which is considered to be incorrect or wrong, or which should not have been done.

In this example we try to open the data file for reading. If it succeeds, we use the read() method to read the file contents as a string into the variable mydata and close the file. If an IOError exception occurs, we still create mydata as an empty string and continue on with the program.

3.3.3 Reading Data from Files Python file descriptors have three methods for reading in data from a file. We’ve already seen

100

Dictionaries, Sets, and Files

the read() method, which returns the entire contents of the file as a single string. For really big files this may not be what you want. The readline() method returns one line of the file at a time. Each time you call it readline() returns the next line. Calls made to readline() after reaching the end of the file return an empty string (‘’).

This is a handy pattern for our toolbox. In bigger programs, we’d squeeze more extensive logic into the body of the loop at line 8 — for example, if each line of the file contained the name and email address of one of our friends, perhaps we’d split the line into some pieces and call a function to send the friend a party invitation. On line 8 we suppress the newline character that print usually appends to our strings. Why? This is because the string already has its own newline: the readline method in line 3 returns everything up to and including the newline character. This also explains the end-of-file detection logic: when there are no more lines to be read from the file, readline returns an empty string — one that does not even have a newline at the end, hence it’s length is 0.

Turning a File into a List of Lines It is often useful to fetch data from a disk file and turn it into a list of lines. Suppose we have a file containing our friends and their email addresses, one per line in the file. But we’d like the lines sorted into alphabetical order. A good plan is to read everything into a list of lines, then sort the list, and then write the sorted list back to another file:

101

Basic Computer Coding: Python

The readlines method in line 2 reads all the lines and returns a list of the strings. We could have used the template from the previous section to read each line one-at-a-time, and to build up the list ourselves, but it is a lot easier to use the method that the Python implementors gave us!

An Example Many useful line-processing programs will read a text file line-ata-time and do some minor processing as they write the lines to an output file. They might number the lines in the output file, or insert extra blank lines after every 60 lines to make it convenient for printing on sheets of paper, or extract some specific columns only from each line in the source file, or only print lines that contain a specific substring. We call this kind of program a filter. Here is a filter that copies one file to another, omitting any lines that begin with #:

102

Dictionaries, Sets, and Files

The continue statement at line 9 skips over the remaining lines in the current iteration of the loop, but the loop will still iterate. This style looks a bit contrived here, but it is often useful to say “get the lines we’re not concerned with out of the way early, so that we have cleaner more focussed logic in the meaty part of the loop that might be written around line 11.” Thus, if text is the empty string, the loop exits. If the first character of text is a hash mark, the flow of execution goes to the top of the loop, ready to start processing the next line. Only if both conditions fail do we fall through to do the processing at line 11, in this example, writing the line into the new file. Let’s consider one more case: suppose your original file contained empty lines. At line 6 above, would this program find the first empty line in the file, and terminate immediately? No! Recall that readline always includes the newline character in the string it returns. It is only when we try to read beyond the end of the file that we get back the empty string of length 0.

103

Basic Computer Coding: Python

CASE STUDY A Set Partitioning Problem A set partitioning problem determines how the items in one set (S) can be partitioned into smaller subsets. All items in S must be contained in one and only one partition. Related problems are: ■

set packing - all items must be contained in zero or one partitions;

■

set covering - all items must be contained in at least one partition.

In this case study a wedding planner must determine guest seating allocations for a wedding. To model this problem the tables are modelled as the partitions and the guests invited to the wedding are modelled as the elements of S. The wedding planner wishes to maximize the total happiness of all of the tables.

A set partitioning problem may be modelled by explicitly enumerating each possible subset. Though this approach does become intractable for large numbers of items (without using column generation) it does have the advantage that the objective function co-efficients for the partitions can be non-linear expressions (like happiness) and

104

Dictionaries, Sets, and Files

still allow this problem to be solved using Linear Programming. First we use allcombinations() to generate a list of all possible table seatings. #create list of all possible tables possible_tables = [tuple(c) for c in pulp.allcombinations(guests, max_table_size)] Then we create a binary variable that will be 1 if the table will be in the solution, or zero otherwise. #create a binary variable to state that a table setting is used x = pulp.LpVariable.dicts(‘table’, possible_tables, lowBound = 0, upBound = 1, cat = pulp.LpInteger) We create the LpProblem and then make the objective function. Note that happiness function used in this script would be difficult to model in any other way. seating_model = pulp.LpProblem(“Wedding Seating Model”, pulp. LpMinimize) seating_model += sum([happiness(table) * x[table] for table in possible_tables]) We specify the total number of tables allowed in the solution. #specify the maximum number of tables seating_model += sum([x[table] for table in possible_tables]) > >>> 10 * (1/0) Traceback (most recent call last): File “”, line 1, in ZeroDivisionError: division by zero >>> 4 + spam*3 Traceback (most recent call last): File “”, line 1, in NameError: name ‘spam’ is not defined >>> ‘2’ + 2

114

Exceptions, Unit Testing and Comprehensions

Traceback (most recent call last): File “”, line 1, in TypeError: Can’t convert ‘int’ object to str implicitly The last line of the error message indicates what happened. Exceptions come in different types, and the type is printed as part of the message: the types in the example are ZeroDivisionError, NameError and TypeError. The string printed as the exception type is the name of the builtin exception that occurred. This is true for all built-in exceptions, but need not be true for user-defined exceptions (although it is a useful convention). Standard exception names are built-in identifiers (not reserved keywords). The rest of the line provides detail based on the type of exception and what caused it. The preceding part of the error message shows the context where the exception happened, in the form of a stack trace back. In general it contains a stack trace back listing source lines; however, it will not display lines read from standard input.

4.1.1 Handling Exceptions It is possible to write programs that handle selected exceptions. Look at the following example, which asks the user for input until a valid integer has been entered, but allows the user to interrupt the program (using ControlCor whatever the operating system supports); note that a user-generated interruption is signalled by raising the KeyboardInterrupt exception. >>> >>> while True: 115

Stack trace is a report of the active stack frames at a certain point in time during the execution of a program.

Basic Computer Coding: Python

...

try:

...

x = int(input(“Please enter a number: “))

...

break

...

except ValueError:

...

print(“Oops! That was no valid number. Try again...”)

... The try statement works as follows. ■

First, the try clause (the statement(s) between the try and except keywords) is executed.

■

If no exception occurs, the except clause is skipped and execution of the try statement is finished.

■

If an exception occurs during execution of the try clause, the rest of the clause is skipped. Then if its type matches the exception named after the except keyword, the except clause is executed, and then execution continues after the try statement.

■

If an exception occurs which does not match the exception named in the except clause, it is passed on to outer try statements; if no handler is found, it is an unhandled exception and execution stops with a message as shown.

A try statement may have more than one except clause, to specify handlers for different exceptions. At most one handler will be executed. Handlers only handle exceptions that occur in the corresponding try clause, not in other handlers of the same try statement. An except clause may name multiple exceptions as a parenthesized tuple, for example: ... except (RuntimeError, TypeError, NameError): ...

pass

A class in an except clause is compatible with an exception if it is the same class or a base class thereof (but not the other way around — an except clause listing a derived class is not compatible with a base class). For example, the following code will print B, C, D in that order: class B(Exception): pass 116

Exceptions, Unit Testing and Comprehensions

class C(B): pass class D(C): pass for cls in [B, C, D]: try: raise cls() except D: print(“D”) except C: print(“C”) except B: print(“B”) Note that if the except clauses were reversed (with except B first), it would have printed B, B, B — the first matching except clause is triggered. The last except clause may omit the exception name(s), to serve as a wildcard. Use this with extreme caution, since it is easy to mask a real programming error in this way! It can also be used to print an error message and then re-raise the exception (allowing a caller to handle the exception as well): import sys try: f = open(‘myfile.txt’) s = f.readline() i = int(s.strip()) except OSError as err: print(“OS error: {0}”.format(err)) 117

Basic Computer Coding: Python

except ValueError: print(“Could not convert data to an integer.”) except: print(“Unexpected error:”, sys.exc_info()[0]) raise The try … except statement has an optional else clause, which, when present, must follow all except clauses. It is useful for code that must be executed if the try clause does not raise an exception. For example: for arg in sys.argv[1:]: try: f = open(arg, ‘r’) except OSError: print(‘cannot open’, arg) else: print(arg, ‘has’, len(f.readlines()), ‘lines’) f.close() The use of the else clause is better than adding additional code to the try clause because it avoids accidentally catching an exception that wasn’t raised by the code being protected by the try … except statement. When an exception occurs, it may have an associated value, also known as the exception’s argument. The presence and type of the argument depend on the exception type. The except clause may specify a variable after the exception name. The variable is bound to an exception instance with the arguments stored in instance.args. For convenience, the exception instance defines__str__() so the arguments can be printed directly without having to reference .args. One may also instantiate an exception first before raising it and add any attributes to it as desired. >>> >>> try: ...

raise Exception(‘spam’, ‘eggs’) 118

Exceptions, Unit Testing and Comprehensions

... except Exception as inst: ...

print(type(inst))

# the exception instance

...

print(inst.args)

# arguments stored in .args

...

print(inst)

# __str__ allows args to be printed directly, # but may be overridden in exception subclasses

... ...

x, y = inst.args

...

print(‘x =’, x)

...

print(‘y =’, y)

# unpack args

...

(‘spam’, ‘eggs’) (‘spam’, ‘eggs’) x = spam y = eggs If an exception has arguments, they are printed as the last part (‘detail’) of the message for unhandled exceptions. Exception handlers don’t just handle exceptions if they occur immediately in the try clause, but also if they occur inside functions that are called (even indirectly) in the try clause. For example: >>> >>> def this_fails(): ...

x = 1/0

... >>> try: ...

this_fails()

... except ZeroDivisionError as err: ...

print(‘Handling run-time error:’, err)

... Handling run-time error: division by zero

119

Basic Computer Coding: Python

4.1.2 Raising Exceptions The raise statement allows the programmer to force a specified exception to occur. For example: >>> >>> raise NameError(‘HiThere’) Traceback (most recent call last): File “”, line 1, in NameError: HiThere The sole argument to raise indicates the exception to be raised. This must be either an exception instance or an exception class (a class that derives from Exception). If an exception class is passed, it will be implicitly instantiated by calling its constructor with no arguments: raise ValueError # shorthand for ‘raise ValueError()’ If you need to determine whether an exception was raised but don’t intend to handle it, a simpler form of the raise statement allows you to re-raise the exception: >>> >>> try: ...

raise NameError(‘HiThere’)

... except NameError: ...

print(‘An exception flew by!’)

...

raise

... An exception flew by! Traceback (most recent call last): File “”, line 2, in NameError: HiThere

4.1.3 User-defined Exceptions Programs may name their own exceptions by creating a new exception class (see Classes for more about Python classes). Exceptions should 120

Exceptions, Unit Testing and Comprehensions

typically be derived from the Exception class, either directly or indirectly. Exception classes can be defined which do anything any other class can do, but are usually kept simple, often only offering a number of attributes that allow information about the error to be extracted by handlers for the exception. When creating a module that can raise several distinct errors, a common practice is to create a base class for exceptions defined by that module, and subclass that to create specific exception classes for different error conditions: class Error(Exception): “””Base class for exceptions in this module.””” pass class InputError(Error): “””Exception raised for errors in the input. Live Demo #!/usr/bin/python

try: fh = open(“testfile”, “w”) try: fh.write(“This is my test file for exception handling!!”) finally: print “Going to close the file” fh.close() except IOError: print “Error: can\’t find file or read data”

121

Basic Computer Coding: Python

Attributes: expression -- input expression in which the error occurred message -- explanation of the error “”” def __init__(self, expression, message): self.expression = expression self.message = message class TransitionError(Error): “””Raised when an operation attempts a state transition that’s not allowed. Attributes: previous -- state at beginning of transition next -- attempted new state message -- explanation of why the specific transition is not allowed “”” def __init__(self, previous, next, message): self.previous = previous self.next = next self.message = message Most exceptions are defined with names that end in “Error,” similar to the naming of the standard exceptions.

4.1.4 Defining Clean-up Actions The try statement has another optional clause which is intended to define clean-up actions that must be executed under all circumstances. For example: >>> 122

Exceptions, Unit Testing and Comprehensions

>>> try: ...

raise KeyboardInterrupt

... finally: ...

print(‘Goodbye, world!’)

... Goodbye, world! KeyboardInterrupt Traceback (most recent call last): File “”, line 2, in A finally clause is always executed before leaving the try statement, whether an exception has occurred or not. When an exception has occurred in the try clause and has not been handled by an except clause (or it has occurred in an except or else clause), it is re-raised after the finally clause has been executed. The finallyclause is also executed “on the way out” when any other clause of the try statement is left via a break, continue or return statement. A more complicated example: >>> >>> def divide(x, y): ... ... ... ... ... ... ... ...

try: result = x / y except ZeroDivisionError: print(“division by zero!”) else: print(“result is”, result) finally: print(“executing finally clause”)

... >>> divide(2, 1) result is 2.0 123

Return Statement causes execution to leave the current subroutine and resume at the point in the code immediately after where the subroutine was called, known as its return address.

Basic Computer Coding: Python

executing finally clause >>> divide(2, 0) division by zero! executing finally clause >>> divide(“2”, “1”) executing finally clause Traceback (most recent call last): File “”, line 1, in File “”, line 3, in divide TypeError: unsupported operand type(s) for /: ‘str’ and ‘str’

String a long flexible structure made from threads twisted together, which is used to tie, bind, or hang other objects

As you can see, the finally clause is executed in any event. The TypeError raised by dividing two strings is not handled by the except clause and therefore re-raised after the finally clause has been executed. In real world applications, the finally clause is useful for releasing external resources (such as files or network connections), regardless of whether the use of the resource was successful.

4.1.5 Predefined Clean-up Actions Some objects define standard clean-up actions to be undertaken when the object is no longer needed, regardless of whether or not the operation using the object succeeded or failed. Look at the following example, which tries to open a file and print its contents to the screen. for line in open(“myfile.txt”): print(line, end=””) The problem with this code is that it leaves the file open for an indeterminate amount of time after this part of the code has finished executing. This is not an issue in simple scripts, 124

Exceptions, Unit Testing and Comprehensions

but can be a problem for larger applications. The with statement allows objects like files to be used in a way that ensures they are always cleaned up promptly and correctly. with open(“myfile.txt”) as f: for line in f: print(line, end=””) After the statement is executed, the file f is always closed, even if a problem was encountered while processing the lines. Objects which, like files, provide predefined clean-up actions will indicate this in their documentation.

4.2 UNIT TESTING The Python unit testing framework, sometimes referred to as “PyUnit,” is a Python language version of JUnit, by Kent Beck and Erich Gamma. JUnit is, in turn, a Java version of Kent’s Smalltalk testing framework. Each is the de facto standard unit testing framework Test Automation is for its respective language. the use of special unittest supports test automation, sharing of setup and shutdown code for tests, aggregation of tests into collections, and independence of the tests from the reporting framework. The unittest module provides classes that make it easy to support these qualities for a set of tests. To achieve this, unittest supports some important concepts: test fixture A test fixture represents the preparation needed to perform one or more tests, and any associate cleanup actions. This may involve, for example,

125

software (separate from the software being tested) to control the execution of tests and the comparison of actual outcomes with predicted outcomes.

Basic Computer Coding: Python

creating temporary or proxy databases, directories, or starting a server process. test case A test case is the smallest unit of testing. It checks for a specific response to a particular set of inputs. unittest provides a base class, TestCase, which may be used to create new test cases. test suite A test suite is a collection of test cases, test suites, or both. It is used to aggregate tests that should be executed together. test runner A test runner is a component which orchestrates the execution of tests and provides the outcome to the user. The runner may use a graphical interface, a textual interface, or return a special value to indicate the results of executing the tests. The test case and test fixture concepts are supported through the TestCase and FunctionTestCase classes; the former should be used when creating new tests, and the latter can be used when integrating existing test code with a unittest-driven framework. When building test fixtures using TestCase, the setUp() and tearDown() methods can be overridden to provide initialization and cleanup for the fixture. With FunctionTestCase, existing functions can be passed to the constructor for these purposes. When the test is run, the fixture initialization is run first; if it succeeds, the cleanup method is run after the test has been executed, regardless of the outcome of the test. Each instance of the TestCase will only be used to run a single test method, so a new fixture is created for each test. Test suites are implemented by the TestSuite class. This class allows individual tests and test suites to be aggregated; when the suite is executed, all tests added directly to the suite and in “child” test suites are run. A test runner is an object that provides a single method, run(), which accepts a TestCase or TestSuite object as a parameter, and returns a result object. The class TestResult is provided for use as the result object. unittest provides the TextTestRunner as an example test runner which reports test results on the standard error stream by default. Alternate runners can be implemented for other 126

Exceptions, Unit Testing and Comprehensions

environments (such as graphical environments) without any need to derive from a specific class.

4.2.1 Basic example The unittest module provides a rich set of tools for constructing and running tests. This demonstrates that a small subset of the tools suffices to meet the needs of most users. Here is a short script to test three string methods: import unittest class TestStringMethods(unittest.TestCase): def test_upper(self): self.assertEqual(‘foo’.upper(), ‘FOO’) def test_isupper(self): self.assertTrue(‘FOO’.isupper()) self.assertFalse(‘Foo’.isupper()) def test_split(self): s = ‘hello world’ self.assertEqual(s.split(), [‘hello’, ‘world’]) # check that s.split fails when the separator is not a string with self.assertRaises(TypeError): s.split(2) if __name__ == ‘__main__’: unittest.main() A testcase is created by subclassing unittest.TestCase. The three individual tests are defined with methods whose names start with the letters test. This naming convention informs the test runner about which methods represent tests. The crux of each test is a call to assertEqual() to check for an expected result; assertTrue() or assertFalse() to verify a condition; or assertRaises() to verify that a specific exception gets raised. These methods are used instead of the assert statement so the test runner can accumulate all test results and produce a report. 127

Basic Computer Coding: Python

The setUp() and tearDown() methods allow you to define instructions that will be executed before and after each test method. The final block shows a simple way to run the tests. unittest.main() provides a command-line interface to the test script. When run from the command line, the script produces an output that looks like this: ... ---------------------------------------------------------------------Ran 3 tests in 0.000s OK Instead of unittest.main(), there are other ways to run the tests with a finer level of control, less terse output, and no requirement to be run from the command line. For example, the last two lines may be replaced with: suite = unittest.TestLoader(). loadTestsFromTestCase(TestStringMethods) unittest.TextTestRunner(verbosity=2).run(suite) Running the revised script from the interpreter or another script produces the following output: test_isupper (__main__.TestStringMethods) ... ok test_split (__main__.TestStringMethods) ... ok test_upper (__main__.TestStringMethods) ... ok ---------------------------------------------------------------------Ran 3 tests in 0.001s OK The examples show the most commonly used unittest features which are sufficient to meet many everyday testing needs. The remainder of the documentation explores the full feature set from first principles.

128

Exceptions, Unit Testing and Comprehensions

4.2.2 Command-Line Interface The unittest module can be used from the command line to run tests from modules, classes or even individual test methods: python -m unittest test_module1 test_module2 python -m unittest test_module.TestClass python -m unittest test_module.TestClass.test_method You can pass in a list with any combination of module names, and fully qualified class or method names. You can run tests with more detail (higher verbosity) by passing in the -v flag: python -m unittest -v test_module For a list of all the command-line options: python -m unittest -h Changed in version 2.7: In earlier versions it was only possible to run individual test methods and not modules or classes.

Command-line options unittest supports these command-line options: -b, --buffer The standard output and standard error streams are buffered during the test run. Output during a passing test is discarded. Output is echoed normally on test fail or error and is added to the failure messages. -c, --catch Control-C during the test run waits for the current test to end and then reports all the results so far. A second Control-C raises the normal Keyboard Interrupt exception. See Signal Handling for the functions that provide this functionality. -f, --failfast Stop the test run on the first error or failure. New in version 2.7: The command-line options -b, -c and -f were added. 129

Basic Computer Coding: Python

The command line can also be used for test discovery, for running all of the tests in a project or just a subset.

4.2.3 Test Discovery Unittest supports simple test discovery. In order to be compatible with test discovery, all of the test files must be modules or packages importable from the top-level directory of the project (this means that their filenames must be valid identifiers). Test discovery is implemented in TestLoader.discover(), but can also be used from the command line. The basic command-line usage is: cd project_directory python -m unittest discover The discover sub-command has the following options: -v, --verbose Verbose output -s, --start-directory directory Directory to start discovery (. default) -p, --pattern pattern Pattern to match test files (test*.py default) -t, --top-level-directory directory Top level directory of project (defaults to start directory) The -s, -p, and -t options can be passed in as positional arguments in that order. The following two command lines are equivalent: python -m unittest discover -s project_directory -p “*_test.py” python -m unittest discover project_directory “*_test.py” As well as being a path it is possible to pass a package name, for example myproject.subpackage.test, as the start directory. The package name you supply will then be imported and its location on the filesystem will be used as the start directory. Caution: Test discovery loads tests by importing them. Once test discovery has found all the test files from the start directory you specify it turns the paths into package names to import. For example foo/bar/baz.py will be imported as foo.bar.baz. 130

Exceptions, Unit Testing and Comprehensions

If you have a package installed globally and attempt test discovery on a different copy of the package, then the import could happen from the wrong place. If this happens test discovery will warn you and exit. If you supply the start directory as a package name rather than a path to a directory then discover assumes that whichever location it imports from is the location you intended, so you will not get the warning. Test modules and packages can customize test loading and discovery by through the load_tests protocol.

4.2.4 Organizing test code The basic building blocks of unit testing are test cases — single scenarios that must be set up and checked for correctness. In unittest, test cases are represented by instances of unittest’s TestCase class. To make your own test cases you must write subclasses of TestCase, or use FunctionTestCase. An instance of a TestCase-derived class is an object that can completely run a single test method, together with optional set-up and tidy-up code. The testing code of a TestCase instance should be entirely selfcontained, such that it can be run either in isolation or in arbitrary combination with any number of other test cases. The simplest TestCase subclass will simply override the runTest() method in order to perform specific testing code: import unittest class DefaultWidgetSizeTestCase(unittest.TestCase): def runTest(self): widget = Widget(‘The widget’) self.assertEqual(widget.size(), (50, 50), ‘incorrect default size’) Note that in order to test something, we use one of the assert*() methods provided by the TestCase base class. If the test fails, an exception will be raised, and unittest will identify the test case as a failure. Any other exceptions will be treated as errors. This helps you identify where the problem is: failures are caused by incorrect 131

Basic Computer Coding: Python

results - a 5 where you expected a 6. Errors are caused by incorrect code - e.g., a TypeError caused by an incorrect function call. The way to run a test case will be described later. For now, note that to construct an instance of such a test case, we call its constructor without arguments: testCase = DefaultWidgetSizeTestCase()

Subclass “derived class”, heir class, or child class is a modular, derivative class that inherits one or more language entities from one or more other classes (called superclass, base classes, or parent classes).

Now, such test cases can be numerous, and their set-up can be repetitive. In the case, constructing a Widget in each of 100 Widget test case subclasses would mean unsightly duplication. Luckily, we can factor out such set-up code by implementing a method called setUp(), which the testing framework will automatically call for us when we run the test: import unittest class SimpleWidgetTestCase(unittest.TestCase): def setUp(self): self.widget = Widget(‘The widget’) class Default WidgetSizeTestCase(SimpleWidgetTestCase): def runTest(self): self.assertEqual(self.widget.size(), (50,50), ‘incorrect default size’) class Widget Resize Test Case (Simple WidgetTest Case): def runTest(self): self.widget.resize(100,150) self.assertEqual(self.widget.size(), (100,150), 132

Exceptions, Unit Testing and Comprehensions

‘wrong size after resize’) If the setUp() method raises an exception while the test is running, the framework will consider the test to have suffered an error, and the runTest() method will not be executed. Similarly, we can provide a tearDown() method that tidies up after the runTest() method has been run: import unittest class SimpleWidgetTestCase(unittest.TestCase): def setUp(self): self.widget = Widget(‘The widget’) def tearDown(self): self.widget.dispose() self.widget = None If setUp() succeeded, the tearDown() method will be run whether runTest() succeeded or not. Such a working environment for the testing code is called a fixture. Often, many small test cases will use the same fixture. In this case, we would end up subclassing SimpleWidgetTestCase into many small one-method classes such as DefaultWidgetSizeTestCase. This is time-consuming and discouraging, so in the same vein as JUnit, unittest provides a simpler mechanism: import unittest class WidgetTestCase(unittest.TestCase): def setUp(self): self.widget = Widget(‘The widget’) def tearDown(self): self.widget.dispose() self.widget = None def test_default_size(self): self.assertEqual(self.widget.size(), (50,50), ‘incorrect default size’) def test_resize(self): 133

Basic Computer Coding: Python

self.widget.resize(100,150) self.assertEqual(self.widget.size(), (100,150), ‘wrong size after resize’) Here we have not provided a runTest() method, but have instead provided two different test methods. Class instances will now each run one of the test_*() methods, with self.widget created and destroyed separately for each instance. When creating an instance we must specify the test method it is to run. We do this by passing the method name in the constructor: defaultSizeTestCase = WidgetTestCase(‘test_default_size’) resizeTestCase = WidgetTestCase(‘test_resize’) Test case instances are grouped together according to the features they test. unittest provides a mechanism for this: the test suite, represented by unittest’s TestSuite class: widgetTestSuite = unittest.TestSuite() widgetTestSuite.addTest(WidgetTestCase(‘test_default_size’)) widgetTestSuite.addTest(WidgetTestCase(‘test_resize’)) For the ease of running tests, as we will see later, it is a good idea to provide in each test module a callable object that returns a prebuilt test suite: def suite (): suite = unittest.TestSuite() suite.addTest(WidgetTestCase(‘test_default_size’)) suite.addTest(WidgetTestCase(‘test_resize’)) return suite or even: def suite (): tests = [‘test_default_size’, ‘test_resize’] return unittest.TestSuite(map(WidgetTestCase, tests)) Since it is a common pattern to create a TestCase subclass with many similarly named test functions, unittest provides a TestLoader class that can be used to automate the process of creating a test 134

Exceptions, Unit Testing and Comprehensions

suite and populating it with individual tests. For example, suite = u n i t t e s t . Te s t L o a d e r ( ) . loadTestsFromTestCase(WidgetTestCase) will create a test suite that will run WidgetTestCase.test_default_size() and WidgetTestCase.test_resize. TestLoader uses the ‘test’ method name prefix to identify test methods automatically. Often it is desirable to group suites of test cases together, so as to run tests for the whole system at once. This is easy, since TestSuite instances can be added to a TestSuite just as TestCase instances can be added to a TestSuite: suite1 = module1.TheTestSuite() suite2 = module2.TheTestSuite() alltests = unittest.TestSuite([suite1, suite2]) You can place the definitions of test cases and test suites in the same modules as the code they are to test (such as widget.py), but there are several advantages to placing the test code in a separate module, such as test_widget.py: ■

The test module can be run standalone from the command line.

■

The test code can more easily be separated from shipped code.

■

There is less temptation to change test code to fit the code it tests without a good reason.

■

Test code should be modified much less frequently than the code it tests.

■

Tested code can be refactored more easily.

135

That the order in which the various test cases will be run is determined by sorting the test function names with respect to the built-in ordering for strings.

Basic Computer Coding: Python

■

Tests for modules written in C must be in separate modules anyway, so why not be consistent?

■

If the testing strategy changes, there is no need to change the source code.

4.2.5 Re-using old test code Some users will find that they have existing test code that they would like to run from unittest, without converting every old test function to a TestCase subclass. For this reason, unittest provides a FunctionTestCase class. This subclass of TestCase can be used to wrap an existing test function. Set-up and tear-down functions can also be provided. Given the following test function: def testSomething(): something = makeSomething() assert something.name is not None # ... one can create an equivalent test case instance as follows: testcase = unittest.FunctionTestCase(testSomething) If there are additional set-up and tear-down methods that should be called as part of the test case’s operation, they can also be provided like so: testcase = unittest.FunctionTestCase(testSomething, setUp=makeSomethingDB, tearDown=deleteSomethingDB) To make migrating existing test suites easier, unittest supports tests raising AssertionError to indicate test failure. However, it is recommended that you use the explicit TestCase.fail*() and TestCase. assert*() methods instead, as future versions of unittest may treat AssertionError differently. Note: Even though FunctionTestCase can be used to quickly convert an existing test base over to a unittest-based system, this approach is not recommended. Taking the time to set up proper TestCase subclasses will make future test refactorings infinitely easier. 136

Exceptions, Unit Testing and Comprehensions

In some cases, the existing tests may have been written using the doctest module. If so, doctest provides a DocTestSuite class that can automatically build unittest. TestSuite instances from the existing doctest-based tests.

4.2.6 Skipping tests and expected failures Unittest supports skipping individual test methods and even whole classes of tests. In addition, it supports marking a test as an “expected failure,” a test that is broken and will fail, but shouldn’t be counted as a failure on a TestResult. Skipping a test is simply a matter of using the skip() decorator or one of its conditional variants. Basic skipping looks like this: class MyTestCase(unittest.TestCase): @unittest.skip(“demonstrating skipping”) def test_nothing(self): self.fail(“shouldn’t happen”) @unittest.skipIf(mylib.__version__ < (1, 3), “not supported in this library version”) def test_format(self): # Tests that work for only a certain version of the library. pass @unittest.skipUnless(sys.platform.startswith(“win”), “requires Windows”) def test_windows_support(self): # windows specific testing code pass This is the output of running the example in verbose mode: test_format (__main__.MyTestCase) ... skipped ‘not supported in this library version’ test_nothing (__main__.MyTestCase) ... skipped ‘demonstrating skipping’ test_windows_support (__main__.MyTestCase) ... skipped ‘requires 137

Basic Computer Coding: Python

Windows’ --------------------------------------------------------------------Ran 3 tests in 0.005s

The computer algebra system AXIOM (1973) has a similar construct that processes streams, but the first use of the term “comprehension” for such constructs was in Rod Burstall and John Darlington’s description of their functional programming language NPL from 1977.

OK (skipped=3) Classes can be skipped just like methods: @unittest.skip(“showing class skipping”) class MySkippedTestCase(unittest.TestCase): def test_not_run(self): pass TestCase.setUp() can also skip the test. This is useful when a resource that needs to be set up is not available. Expected failures use the expectedFailure() decorator. class ExpectedFailureTestCase(unittest. TestCase): @unittest.expectedFailure def test_fail(self): self.assertEqual(1, 0, “broken”) It’s easy to roll your own skipping decorators by making a decorator that calls skip() on the test when it wants it to be skipped. This decorator skips the test unless the passed object has a certain attribute: def skipUnlessHasattr(obj, attr): if hasattr(obj, attr): return lambda func: func return unittest.skip(“{!r} doesn’t have {!r}”. format(obj, attr)) The following decorators implement test skipping and expected failures: 138

Exceptions, Unit Testing and Comprehensions

unittest.skip(reason) Unconditionally skip the decorated test. reason should describe why the test is being skipped. unittest.skipIf(condition, reason) Skip the decorated test if condition is true. unittest.skipUnless(condition, reason) Skip the decorated test unless condition is true. unittest.expectedFailure() Mark the test as an expected failure. If the test fails when run, the test is not counted as a failure. exception unittest.SkipTest(reason) This exception is raised to skip a test. Usually you can use TestCase.skipTest() or one of the skipping decorators instead of raising this directly. Skipped tests will not have setUp() or tearDown() run around them. Skipped classes will not have setUpClass() or tearDownClass() run.

4.3 COMPREHENSIONS Comprehensions are constructs that allow sequences to be built from other sequences. Types of comprehensions are supported in both Python 2 and Python 3: ■

list comprehensions

■

dictionary comprehensions

■

set comprehensions

■

generator comprehensions

We will discuss them one by one. Once you get the hang of using list comprehensions then you can use any of them easily.

4.3.1 List Comprehensions List comprehensions provide a short and concise way to create lists. It consists of square brackets containing an expression followed by 139

Basic Computer Coding: Python

a for clause, then zero or more for or if clauses. The expressions can be anything, meaning you can put in all kinds of objects in lists. The result would be a new list made after the evaluation of the expression in context of the if and for clauses.

Blueprint variable = [out_exp for out_exp in input_list if out_exp == 2] Here is a short example: multiples = [i for i in range(30) if i % 3 == 0] print(multiples) # Output: [0, 3, 6, 9, 12, 15, 18, 21, 24, 27] This can be really useful to make lists quickly. It is even preferred by some instead of the filter function. List comprehensions really shine when you want to supply a list to a method or function to make a new list by appending to it in each iteration of the for loop. For instance you would usually do something like this: squared = [] for x in range(10): squared.append(x**2) You can simplify it using list comprehensions. For example: squared = [x**2 for x in range(10)]

4.3.2 Dict Comprehensions They are used in a similar way. Here is an example which I found recently: mcase = {‘a’: 10, ‘b’: 34, ‘A’: 7, ‘Z’: 3} mcase_frequency = { k.lower(): mcase.get(k.lower(), 0) + mcase.get(k.upper(), 0) for k in mcase.keys() } 140

Exceptions, Unit Testing and Comprehensions

# mcase_frequency == {‘a’: 17, ‘z’: 3, ‘b’: 34} In the example we are combining the values of keys which are same but in different typecase. I personally do not use dict comprehensions a lot. You can also quickly switch keys and values of a dictionary: {v: k for k, v in some_dict.items()}

4.3.3 Set Comprehensions They are also similar to list comprehensions. The only difference is that they use braces {}. Here is an example: squared = {x**2 for x in [1, 1, 2]} print(squared) # Output: {1, 4}

4.3.4 Generator Comprehensions They are also similar to list comprehensions. The only difference is that they don’t allocate memory for the whole list but generate one item at a time, thus more memory effecient. multiples_gen = (i for i in range(30) if i % 3 == 0) print(multiples_gen) # Output: for x in multiples_gen: print(x) # Outputs numbers

141

Basic Computer Coding: Python

CASE STUDY Limitations of traditional exception Handling Lets look at a typical example of using an http library to access a remote API to fetch some data. We’ll use okhttp but please don’t take this as a hit on the library. It is probably my favorite http library for Java. All Java libraries that throw exceptions will behave in the manner that we will illustrate. A first pass at creating a function that fetches data from a remote API might look like the following: val URL = “http://gd2.mlb.com/components/game/mlb/year_2017/ month_06/day_18/miniscoreboard.json” fun scores(): List? { val httpClient = OkHttpClient() val request = Request.Builder() .url(URL) .build() val response = httpClient.newCall(request).execute() if (response.isSuccessful) { val body = response.body()?.string() val json = ObjectMapper().readValue(body, MinisScoreboardResponse::class.java) return json?.data?.games?.game } else { return null } } That function will happily compile and we might even be bold enough to put it in production (I’ve deployed code to production like that many times). And it might possibly work for days, months or years without an issue. But it might also start failing within hours. We can’t tell, and the code we wrote gives us no indication that something might fail. Furthermore, okhttp’s type signature for 142

Exceptions, Unit Testing and Comprehensions

making an http call does not tell us what exceptions it might throw if something might go wrong. We can draw on past experiences of having written services that interface with third party systems via a network call to know that network issues can occur that will disrupt our services. To be able to resiliently handle these outages we need to handle the possible exceptions that okhttp will throw when something goes wrong. Unfortunately, most of the times we don’t know what those exceptions are until something goes wrong in production or if we read the source code for the third party library we are using. But reading the code for every third party library that our application depends on is not practical. Embarrasingly, one of the things I’ve done when the exceptions that might be thrown are unknown, I use the catch-all java.lang.Exception: val URL = “http://gd2.mlb.com/components/game/mlb/year_2017/ month_06/day_18/miniscoreboard.json” fun scores(): List? { val httpClient = OkHttpClient() try { val request = Request.Builder() .url(URL) .build() val response = httpClient.newCall(request).execute() if (response.isSuccessful) { val body = response.body()?.string() val json = ObjectMapper().readValue(body, MinisScoreboardResponse::class.java) return json?.data?.games?.game } else { return null } } (catch e : Exception) { log.error(“An exception occurred fetching our data from 143

Basic Computer Coding: Python

MLB: “, e) return null } } While that code won’t crash in production, it will fail to alert us of any issues that might be happening that we should be dealing with. It also does not give the parts of our application that are using scores an opportunity to either inform the users in the front end that something went wrong (e.g. MLB is down at the moment; We are experiencing network issues) or try another service that might be a backup for times when the main service is down. We can propagate the exception up the stack and that can give the parts of our business application an opportunity to handle the exceptions however they see fit: val URL = “http://gd2.mlb.com/components/game/mlb/year_2017/ month_06/day_18/miniscoreboard.json” fun scores(): List? { val httpClient = OkHttpClient() try { val request = Request.Builder() .url(URL) .build( val response = httpClient.newCall(request).execute() if (response.isSuccessful) { val body = response.body()?.string() val json = ObjectMapper().readValue(body, MinisScoreboardResponse::class.java) return json?.data?.games?.game } else { return null } } 144

Exceptions, Unit Testing and Comprehensions

(catch e : Exception) { log.error(“An exception occurred fetching our data from MLB: “, e) throw e } }

And we are back to the issues we mentioned earlier: the type signature gives us no indication of what exceptions might be thrown, and every client of scores needs to know and also throw the Exception so that calls from higher up the stack can handle it.

145

Basic Computer Coding: Python

SUMMARY ■

Even if a statement or expression is syntactically correct, it may cause an error when an attempt is made to execute it.

■

The unittest module can be used from the command line to run tests from modules, classes or even individual test methods:

■

The basic building blocks of unit testing are test cases — single scenarios that must be set up and checked for correctness. In unittest, test cases are represented by instances of unittest’s TestCase class.

■

Some users will find that they have existing test code that they would like to run from unittest, without converting every old test function to a TestCase subclass.

■

Unittest supports skipping individual test methods and even whole classes of tests. In addition, it supports marking a test as an “expected failure,” a test that is broken and will fail, but shouldn’t be counted as a failure on a TestResult.

■

List comprehensions provide a short and concise way to create lists. It consists of square brackets containing an expression followed by a for clause, then zero or more for or if clauses.

■

They are also similar to list comprehensions. The only difference is that they use braces {}. Here is an example:

146

Exceptions, Unit Testing and Comprehensions

KNOWLEDGE CHECK 1.

2.

3.

4.

How many except statements can a try-except block have? a.

zero

b.

one

c.

more than one

d.

more than zero

When will the else part of try-except-else be executed? a.

always

b.

when an exception occurs

c.

when no exception occurs

d.

when an exception occurs in to except block

Can one block of except statements handle multiple exception? a.

yes, like except TypeError, SyntaxError [,…].

b.

yes, like except [TypeError, SyntaxError].

c.

no

d.

none of the mentioned

What is the output of the code shown? l=[1,2,3,4,5] [x&1 for x in l]

5.

a.

[1, 1, 1, 1, 1]

b.

[1, 0, 1, 0, 1]

c.

[1, 0, 0, 0, 0]

d.

[0, 1, 0, 1, 0]

What is the output of the code shown below? l1=[1,2,3] l2=[4,5,6] [x*y for x in l1 for y in l2] a.

[4, 8, 12, 5, 10, 15, 6, 12, 18] 147

Basic Computer Coding: Python

b.

[4, 10, 18]

c.

[4, 5, 6, 8, 10, 12, 12, 15, 18]

d.

[18, 12, 6, 15, 10, 5, 12, 8, 4]

REVIEW QUESTIONS 1.

User-Defined Exception in Python.

2.

What are handling exceptions in Python?

3.

What is Unit Testing?

4.

Designing a test case for Python Testing using PyUnit.

5.

The correct expansion of list_1 = [expr(i) for i in list_0 if func(i)]?

Check Your Result 1. (d)

2. (c)

3. (a)

148

4. (b)

5. (c)

Exceptions, Unit Testing and Comprehensions

REFERENCES 1.

C. Szyperski. Component Software: Beyond ObjectOriented Programming. ACM Press and Addison-Wesley, New York, NY, second edition, November 2002.

2.

G. Veccellio and W. M. Thomas. Issues in the assurance of component-based software. In Proc. 2000 International Workshop on Component-Based Software, Carnegie Mellon Software Engineering Institute, 2000.

3.

http://book.pythontips.com/en/latest/exceptions.html

4.

https://python-3-patterns-idioms-test.readthedocs.io/en/ latest/Comprehensions.html

5.

https://www.pythonforbeginners.com/error-handling/ exception-handling-in-python

6.

R. T.Tomita, F. Castor Filho, P.A. de C. Guerra and C. M. F. Rubira. Bellatrix: An environment with architectural support for component-based development (in portuguese). In Proc. IV Brazilian Workshop on Component-Based Development (WDBC’2004), pp. 43-48, João Pessoa, PB, Brazil, Sep. 2004.

7.

Swaroop C.H., A byte of Python, 2005

8.

The Python Language Reference. Python Software Foundation. Retrieved 27 June 2009.

149

CHAPTER 5

OBJECT ORIENTED PROGRAMMING “Object-oriented programming offers a sustainable way to write spaghetti code. It lets you accrete programs as a series of patches.” ―Paul Graham

Learning Objectives After studying this chapter, you will be able to: ■

Explain the introduction to OOPS in python

■

Define the various types of methods of OOPS

Basic Computer Coding: Python

INTRODUCTION Object-oriented programming (OOP) is a programming language model organized around objects rather than “actions” and data rather than logic. Historically, a program has been viewed as a logical procedure that takes input data, processes it, and produces output data. The programming challenge was seen as how to write the logic, not how to define the data. Object-oriented programming takes the view that what we really care about are the objects we want to manipulate rather than the logic required to manipulate them. Examples of objects range from human beings (described by name, address, and so forth) to buildings and floors (whose properties can be described and managed) down to the little widgets on a computer desktop (such as buttons and scroll bars). The first step in OOP is to identify all the objects the programmer wants to manipulate and how they relate to each other, an exercise often known as data modeling. Once an object has been identified, it is generalized as a class of objects (think of Plato’s concept of the “ideal” chair that stands for all chairs) which defines the kind of data it contains and any logic sequences that can manipulate it. Each distinct logic sequence is known as a method. Objects communicate with well-defined interfaces called messages. The concepts and rules used in object-oriented programming provide these important benefits: ■

The concept of a data class makes it possible to define subclasses of data objects that share some or all of the main class characteristics. Called inheritance, this property of OOP forces a more thorough data analysis, reduces development time, and ensures more accurate coding.

■

Since a class defines only the data it needs to be concerned with, when an instance of that class (an object) is run, the code will not be able to accidentally access other program data. This characteristic of data hiding provides greater system security and avoids unintended data corruption.

152

Object Oriented Programming

■

The definition of a class is reuseable not only by the program for which it is initially created but also by other objectoriented programs (and, for this reason, can be more easily distributed for use in networks).

■

The concept of data classes allows a programmer to create any new data type that is not already defined in the language itself.

Simula was the first object-oriented programming language. Java, Python, C++, Visual Basic .NET and Ruby are the most popular OOP languages today. The Java programming language is designed especially for use in distributed applications on corporate networks and the Internet. Ruby is used in many Web applications. Curl, Smalltalk, Delphi and Eiffel are also examples of object-oriented programming languages.

5.1 INTRODUCTION TO OOPS IN PYTHON Object-oriented Programming, or OOP for short, is a programming paradigm which provides a means of structuring programs so that properties and behaviors are bundled into individual objects. For instance, an object could represent a person with a name property, age, address, etc., with behaviors like walking, talking, breathing, and running. Or an email with properties like recipient list, subject, body, etc., and behaviors like adding attachments and sending. Put another way, object-oriented programming is an approach for modeling concrete, real-world things like cars as well as relations between things like companies and employees, students and teachers, etc. OOP models real-world entities as software objects, which have some data associated with them and can perform certain functions. Another common programming paradigm is procedural programming which structures a program like a recipe in that it provides a set of steps, in the form of functions and code blocks, which flow sequentially in order to complete a task. The key takeaway is that objects are at the center of the objectoriented programming paradigm, not only representing the data, as in procedural programming, but in the overall structure of the program as well. 153

Basic Computer Coding: Python

5.1.1 Classes in Python Focusing first on the data, each thing or object is an instance of some class. The primitive data structures available in Python, like numbers, strings, and lists are designed to represent simple things like the cost of something, the name of a poem, and your favorite colors, respectively. What if you wanted to represent something much more complicated? For example, let’s say you wanted to track a number of different animals. If you used a list, the first element could be the animal’s name while the second element could represent its age. How would you know which element is supposed to be which? What if you had 100 different animals? Are you certain each animal has both a name and an age, and so forth? What if you wanted to add other properties to these animals? This lacks organization, and it’s the exact need for classes. Classes are used to create new user-defined data structures that contain arbitrary information about something. In the case of an animal, we could create an Animal() class to track properties about the Animal like the name and age. It’s important to note that a class just provides structure—it’s a blueprint for how something should be defined, but it doesn’t actually provide any real content itself. The Animal() class may specify that the name and age are necessary for defining an animal, but it will not actually state what a specific animal’s name or age is. It may help to think of a class as an idea for how something should be defined.

5.1.2 Python Objects (Instances) While the class is the blueprint, an instance is a copy of the class with actual values, literally an object belonging to a specific class. It’s not an idea anymore; it’s an actual animal, like a dog named Roger who’s eight years old. Put another way, a class is like a form or questionnaire. It defines the needed information. After you fill out the form, your specific copy is an instance of the class; it contains actual information relevant to you. You can fill out multiple copies to create many different instances, but without the form as a guide, you would be lost, not knowing 154

Object Oriented Programming

what information is required. Thus, before you can create individual instances of an object, we must first specify what is needed by defining a class.

How to Define a Class in Python Defining a class is simple in Python: class Dog: pass You start with the class keyword to indicate that you are creating a class, then you add the name of the class (using CamelCase notation, starting with a capital letter.) Also, we used the Python keyword pass here. This is very often used as a place holder where code will eventually go. It allows us to run this code without throwing an error. The above code is correct on Python 3. On Python 2.x (“legacy Python”) you’d use a slightly different class definition: # Python 2.x Class Definition: class Dog(object): pass The (object) part in parentheses specifies the parent class that you are inheriting from. In Python 3 this is no longer necessary because it is the implicit default.

Instance Attributes All classes create objects, and all objects contain characteristics called attributes (referred to as properties in the opening paragraph). Use the __init__() method to initialize (e.g., specify) an object’s initial attributes by giving them their default value (or state). This method must have at least one argument as well as the self variable, which refers to the object itself (e.g., Dog). class Dog: # Initializer / Instance Attributes def __init__(self, name, age):

155

Basic Computer Coding: Python

self.name = name self.age = age In the case of our Dog() class, each dog has a specific name and age, which is obviously important to know for when you start actually creating different dogs. Remember: the class is just for defining the Dog, not actually creating instances of individual dogs with specific names and ages; we’ll get to that shortly. Similarly, the self variable is also an instance of the class. Since instances of a class have varying values we could state Dog.name = name rather than self.name = name. But since not all dogs share the same name, we need to be able to assign different values to different instances. Hence the need for the special self variable, which will help to keep track of individual instances of each class. You will never have to call the __init__() method; it gets called automatically when you create a new ‘Dog’ instance.

Class Attributes While instance attributes are specific to each object, class attributes are the same for all instances—which in this case is all dogs. class Dog: # Class Attribute species = ‘mammal’ # Initializer / Instance Attributes def __init__(self, name, age): self.name = name self.age = age So while each dog has a unique name and age, every dog will be a mammal. Let’s create some dogs…

156

Object Oriented Programming

5.1.3 Instantiating Objects Instantiating is a fancy term for creating a new, unique instance of a class. For example: >>> class Dog: ...

pass

... >>> Dog()

>>> Dog()

>>> a = Dog() >>> b = Dog() >>> a == b False We started by defining a new Dog() class, then created two new dogs, each assigned to different objects. So, to create an instance of a class, you use the class name, followed by parentheses. Then to demonstrate that each instance is actually different, we instantiated two more dogs, assigning each to a variable, then tested if those variables are equal. What do you think the type of a class instance is? >>> class Dog: ...

pass

... >>> a = Dog() >>> type(a)

Let’s look at a slightly more complex example… class Dog: # Class Attribute 157

Basic Computer Coding: Python

species = ‘mammal’ # Initializer / Instance Attributes def __init__(self, name, age): self.name = name self.age = age

# Instantiate the Dog object philo = Dog(“Philo”, 5) mikey = Dog(“Mikey”, 6) # Access the instance attributes print(“{} is {} and {} is {}.”.format( philo.name, philo.age, mikey.name, mikey.age)) # Is Philo a mammal? if philo.species == “mammal”: print(“{0} is a {1}!”.format(philo.name, philo.species)) Save this as dog_class.py, then run the program. You should see: Philo is 5 and Mikey is 6. Philo is a mammal!

What’s Going On? We created a new instance of the Dog() class and assigned it to the variable philo. We then passed it two arguments, “Philo” and 5, which represent that dog’s name and age, respectively. These attributes are passed to the __init__ method, which gets called any time you create a new instance, attaching the name and age to the object. You might be wondering why we didn’t have to pass in the self argument. 158

Object Oriented Programming

This is Python magic; when you create a new instance of the class, Python automatically determines what self is (a Dog in this case) and passes it to the __init__ method.

5.1.4 Instance Methods Instance methods are defined inside a class and are used to get the contents of an instance. They can also be used to perform operations with the attributes of our objects. Like the __init__ method, the first argument is always self: class Dog: # Class Attribute species = ‘mammal’ # Initializer / Instance Attributes def __init__(self, name, age): self.name = name self.age = age # instance method def description(self): return “{} is {} years old”.format(self. name, self.age) # instance method def speak(self, sound): return “{} says {}”.format(self.name, sound) # Instantiate the Dog object mikey = Dog(“Mikey”, 6) 159

Instance is a concrete occurrence of any object, existing usually during the runtime of a computer program. Formally, it is synonymous with “object” as they are each a particular value (realization), and these may be called an instance object; “instance” emphasizes the distinct identity of the object.

Basic Computer Coding: Python

# call our instance methods print(mikey.description()) print(mikey.speak(“Gruff Gruff”)) Save this as dog_instance_methods.py, then run it: Mikey is 6 years old Mikey says Gruff Gruff In the latter method, speak(), we are defining behavior. What other behaviors could you assign to a dog? Look back to the beginning paragraph to see some example behaviors for other objects.

Modifying Attributes You can change the value of attributes based on some behavior: >>> class Email: ...

def __init__(self):

...

self.is_sent = False

...

def send_email(self):

...

self.is_sent = True

... >>> my_email = Email() >>> my_email.is_sent False >>> my_email.send_email() >>> my_email.is_sent True 160

Object Oriented Programming

Here, we added a method to send an email, which updates the is_sent variable to True.

5.1.5 Python Object Inheritance Inheritance is the process by which one class takes on the attributes and methods of another. Newly formed classes are called child classes, and the classes that child classes are derived from are called parent classes. It’s important to note that child classes override or extend the functionality (e.g., attributes and behaviors) of parent classes. In other words, child classes inherit all of the parent’s attributes and behaviors but can also specify different behavior to follow. The most basic type of class is an object, which generally all other classes inherit as their parent. When you define a new class, Python 3 it implicitly uses object as the parent class. So the following two definitions are equivalent: class Dog(object): pass

# In Python 3, this is the same as:

class Dog: pass In Python 2.x there’s a distinction between new-style and old-style classes. I won’t go into detail here, but you’ll generally want to specify object as the parent class to ensure you’re definint a newstyle class if you’re writing Python 2 OOP code.

Dog Park Example Let’s pretend that we’re at a dog park. There are multiple Dog objects engaging in Dog behaviors, each with different attributes. In regular-speak that means some dogs are running, while some 161

Basic Computer Coding: Python

are stretching and some are just watching other dogs. Furthermore, each dog has been named by its owner and, since each dog is living and breathing, each ages. What’s another way to differentiate one dog from another? How about the dog’s breed: >>> class Dog: ... ...

def __init__(self, breed): self.breed = breed

... >>> spencer = Dog(“German Shepard”) >>> spencer.breed ‘German Shepard’ >>> sara = Dog(“Boston Terrier”) >>> sara.breed ‘Boston Terrier’ Each breed of dog has slightly different behaviors. To take these into account, let’s create separate classes for each breed. These are child classes of the parent Dog class.

Extending the Functionality of a Parent Class Create a new file called dog_inheritance.py: # Parent class class Dog:

# Class attribute species = ‘mammal’

162

Object Oriented Programming

# Initializer / Instance attributes def __init__(self, name, age): self.name = name self.age = age

# instance method def description(self): return “{} is {} years old”.format(self.name, self.age)

# instance method def speak(self, sound): return “{} says {}”.format(self.name, sound)

# Child class (inherits from Dog class) class RussellTerrier(Dog): def run(self, speed): return “{} runs {}”.format(self.name, speed)

# Child class (inherits from Dog class) class Bulldog(Dog): def run(self, speed): 163

Basic Computer Coding: Python

return “{} runs {}”.format(self.name, speed)

# Child classes inherit attributes and # behaviors from the parent class jim = Bulldog(“Jim”, 12) print(jim.description())

# Child classes have specific attributes # and behaviors as well print(jim.run(“slowly”)) Read the comments aloud as you work through this program to help you understand what happening, then before you run the program, sees if you can predict the expected output. You should see: Jim is 12 years old Jim runs slowly We haven’t added any special attributes or methods to differentiate a RussellTerrier from a Bulldog, but since they’re now two different classes, we could for instance give them different class attributes defining their respective speeds.

Parent vs. Child Classes The isinstance() function is used to determine if an instance is also an instance of a certain parent class. Save this as dog_isinstance.py: # Parent class class Dog: 164

Object Oriented Programming

# Class attribute species = ‘mammal’ # Initializer / Instance attributes def __init__(self, name, age): self.name = name self.age = age # instance method def description(self): return “{} is {} years old”.format(self.name, self.age) # instance method def speak(self, sound): return “{} says {}”.format(self.name, sound)

# Child class (inherits from Dog() class) class RussellTerrier(Dog): def run(self, speed): return “{} runs {}”.format(self.name, speed)

# Child class (inherits from Dog() class) class Bulldog(Dog): def run(self, speed): return “{} runs {}”.format(self.name, speed)

# Child classes inherit attributes and 165

Basic Computer Coding: Python

# behaviors from the parent class jim = Bulldog(“Jim”, 12) print(jim.description()) # Child classes have specific attributes # and behaviors as well print(jim.run(“slowly”)) # Is jim an instance of Dog()? print(isinstance(jim, Dog)) # Is julie an instance of Dog()? julie = Dog(“Julie”, 100) print(isinstance(julie, Dog)) # Is johnny walker an instance of Bulldog() johnnywalker = RussellTerrier(“Johnny Walker”, 4) print(isinstance(johnnywalker, Bulldog)) # Is julie and instance of jim? print(isinstance(julie, jim)) Output: (‘Jim’, 12) Jim runs slowly True True False Traceback (most recent call last): File “dog_isinstance.py”, line 50, in print(isinstance(julie, jim)) TypeError: isinstance() arg 2 must be a class, type, or tuple of 166

Object Oriented Programming

classes and types Make sense? Both jim and julie are instances of the Dog() class, while johnnywalker is not an instance of the Bulldog() class. Then as a sanity check, we tested if julie is an instance of jim, which is impossible since jim is an instance of a class rather than a class itself—hence the reason for the TypeError.

Overriding the Functionality of a Parent Class Remember that child classes can also override attributes and behaviors from the parent class. For examples: >>> class Dog: ...

species = ‘mammal’

... >>> class SomeBreed(Dog): ...

pass

... >>> class SomeOtherBreed(Dog): ...

species = ‘reptile’

... >>> frank = SomeBreed() >>> frank.species ‘mammal’ >>> beans = SomeOtherBreed() >>> beans.species 167

TypeError is an unintended condition which might manifest in multiple stages of a program’s development. Thus a facility for detection of the error is needed in the type system.

Basic Computer Coding: Python

‘reptile’ The SomeBreed() class inherits the species from the parent class, while the SomeOtherBreed() class overrides the species, setting it to reptile.

5.2 METHODS OF OOPS Python has been an object-oriented language from day one. Because of this, creating and using classes and objects are downright easy. If you don’t have any experience with object-oriented (OO) programming. Methods are functions defined inside the body of a class. They are used to define the behaviors of an object.

Creating Methods in Python class Parrot: # instance attributes def __init__(self, name, age): self.name = name self.age = age # instance method def sing(self, song): return “{} sings {}”.format(self.name, song) def dance(self): return “{} is now dancing”.format(self.name) # instantiate the object blu = Parrot(“Blu”, 10) # call our instance methods print(blu.sing(“’Happy’”)) print(blu.dance()) When we run program, the output will be: 168

Object Oriented Programming

Blu sings ‘Happy’ Blu is now dancing In the above program, we define two methods i.e sing() and dance(). These are called instance method because they are called on an instance object i.e blu.

5.2.1 Inheritance Inheritance is a way of creating new class for using details of existing class without modifying it. The newly formed class is a derived class (or child class). Similarly, the existing class is Base class is the parent class of a derived class. a base class (or parent class). Classes may be used to create other classes. Python Inheritance Syntax A class that is used to create (or derive) another class is called the base class.

class BaseClass: Body of base class class DerivedClass(BaseClass): Body of derived class Derived class inherits features from the base class, adding new features to it. This results into re-usability of code.

Use of Inheritance in Python # parent class class Bird: def __init__(self): print(“Bird is ready”) def whoisThis(self): print(“Bird”) 169

Basic Computer Coding: Python

def swim(self): print(“Swim faster”) # child class class Penguin(Bird): def __init__(self): # call super() function super().__init__() print(“Penguin is ready”) def whoisThis(self): print(“Penguin”) def run(self): print(“Run faster”) peggy = Penguin() peggy.whoisThis() peggy.swim() peggy.run() When we run this program, the output will be: ■

Bird is ready

■

Penguin is ready

■

Penguin

■

Swim faster

■

Run faster

In the above program, we created two classes i.e. Bird (parent class) and Penguin (child class). The child class inherits the functions of parent class. We can see this from swim()method. Again, the child class modified the behavior of parent class. We can see this from 170

Object Oriented Programming

whoisThis() method. Furthermore, we extend the functions of parent class, by creating a new run() method. Additionally, we use super() function before __init__() method. This is because we want to pull the content of __init__() method from the parent class into the child class. To demonstrate the use of inheritance, let us take an example. A polygon is a closed figure with 3 or more sides. Say, we have a class called Polygon defined as follows. class Polygon: def __init__(self, no_of_sides): self.n = no_of_sides self.sides = [0 for i in range(no_of_sides)] def inputSides(self): self.sides = [float(input(“Enter side “+str(i+1)+” : “)) for i in range(self.n)] def dispSides(self): for i in range(self.n): print(“Side”,i+1,”is”,self.sides[i]) This class has data attributes to store the number of sides, n and magnitude of each side as a list, sides. Method inputSides() takes in magnitude of each side and similarly, dispSides() will display these properly. A triangle is a polygon with 3 sides. So, we can created a class called Triangle which inherits from Polygon. This makes all the attributes available in class Polygon readily available in Triangle. We don’t need to define them again (code re-usability). Triangle is defined as follows. class Triangle(Polygon): def __init__(self): Polygon.__init__(self,3)

171

Basic Computer Coding: Python

def findArea(self): a, b, c = self.sides # calculate the semi-perimeter s = (a + b + c) / 2 area = (s*(s-a)*(s-b)*(s-c)) ** 0.5 print(‘The area of the triangle is %0.2f’ %area) However, class Triangle has a new method findArea() to find and print the area of the triangle. Here is a sample run. >>> t = Triangle()

>>> t.inputSides() Enter side 1 : 3 Enter side 2 : 5 Enter side 3 : 4

>>> t.dispSides() Side 1 is 3.0 Side 2 is 5.0 Side 3 is 4.0

>>> t.findArea() The area of the triangle is 6.00 We can see that, even though we did not define methods like inputSides() or dispSides() for class Triangle, we were able to use them. If an attribute is not found in the class, search continues to the base class. This repeats recursively, if the base class is itself derived 172

Object Oriented Programming

from other classes.

5.2.2 Encapsulation Encapsulation is the packing of data and functions operating on that data into a single component and restricting the access to some of the object’s components. Encapsulation means that the internal representation of an object is generally hidden from view outside of the object’s definition. A class is an example of encapsulation as it encapsulates all the data that is member functions, variables etc. Difference between Abstraction and Encapsulation: Abstraction is a mechanism which represent the essential features without including implementation details. Encapsulation: — Information hiding. Abstraction: — Implementation hiding. Python follows the philosophy of we’re all adults here with respect to hiding attributes and methods; i.e. you should trust the other programmers who will use your classes. Use plain attributes whenever possible. You might be tempted to use getter and setter methods instead of attributes, but the only reason to use getters and setters is so you can change the implementation later if you need to. However, Python 2.2 and later allows you to do this with properties:

Protected members Protected member is accessible only from within the class and it’s subclasses. How to accomplish this in Python? The answer is — by convention. By prefixing the name of your member with a single underscore, you’re telling others “don’t touch this, unless you’re a subclass”.

Private members But there is a method in Python to define Private: Add “__” (double underscore) in front of the variable and function name can hide them when accessing them from out of class. 173

Basic Computer Coding: Python

Python doesn’t have real private methods, so one underline in the beginning of a method or attribute means you shouldn’t access this method. But this is just convention. And can still access the variables with single underscore. Also when using double underscore (__).we can still access the private variables An example of accessing private member data.(Using name mangling)

class Person: def __init__(self): self.name = ‘Manjula’ self.__lastname = ‘Dube’ def PrintName(self): return self.name +’ ‘ + self.__lastname

Remember #Outside class The __init__ method is a constructor and runs as soon as an object of a class is instantiated. Its aim is to initialize the object

P = Person() print(P.name) print(P.PrintName()) print(P.__lastname) #AttributeError: ‘Person’ object has no attribute ‘__ lastname’

174

Object Oriented Programming

ccess public variable out of class, succeed Access private variable our of class, fail Access public function but this function access Private variable __B successfully since they are in the same class. An example of accessing private member data. (Using name mangling technique) class SeeMee: def youcanseeme(self): return ‘you can see me’ def __youcannotseeme(self): return ‘you cannot see me’ #Outside class Check = SeeMee() print(Check.youcanseeme()) # you can see me print(Check.__youcannotseeme()) #AttributeError: ‘SeeMee’ object has no attribute ‘__youcannotseeme’ If you need to access the private member function class SeeMee: def youcanseeme(self): return ‘you can see me’ def __youcannotseeme(self): return ‘you cannot see me’ #Outside class Check = SeeMee() print(Check.youcanseeme()) print(Check._SeeMee__youcannotseeme()) 175

Private variables, are variables that are visible only to the class to which they belong.

Basic Computer Coding: Python

#Changing the name causes it to access the function You can still call the method using its mangled name, so this feature doesn’t provide much protection. You should know the following for accessing private members and private functions: ■

When you write to an attribute of an object, that does not exist, the python system will normally not complain, but just create a new attribute.

■

Private attributes are not protected by the Python system. That is by design decision.

■

Private attributes will be masked. The reason is, that there should be no clashes in the inheritance chain. The masking is done by some implicit renaming. Private attributes will have the real name

“___” With that name, it can be accessed from outside. When accessed from inside the class, the name will be automatically changed correctly.

Data Encapsulation in Python Using OOP in Python, we can restrict access to methods and variables. This prevent data from direct modification which is called encapsulation. In Python, we denote private attribute using underscore as prefix i.e single “ _ “ or double “ __“. class Computer:

def __init__(self): self.__maxprice = 900

def sell(self): print(“Selling Price: {}”.format(self.__maxprice))

176

Object Oriented Programming

def setMaxPrice(self, price): self.__maxprice = price

c = Computer() c.sell()

# change the price c.__maxprice = 1000 c.sell()

# using setter function c.setMaxPrice(1000) c.sell() When we run this program, the output will be: Selling Price: 900 Selling Price: 900 Selling Price: 1000 In the above program, we defined a class Computer. We use __init__() method to store the maximum selling price of computer. We tried to modify the price. However, we can’t change it because Python treats the __maxprice as private attributes. To change the value, we used a setter function i.e setMaxPrice() which takes price as parameter.

5.2.3 Polymorphism Polymorphism means that different types respond to the same function. Polymorphism is very useful as it makes programming more intuitive and therefore easier. Polymorphism is a fancy word 177

Basic Computer Coding: Python

that just means the same function is defined on objects of different types. Python provides protocols which is polymorphism under the hood. These implement consistent behavior for built in objects of different type.

Protocols When we introspect an object we have a lot of attributes that take this format: __names__. This section will make many of those clear. Everything is an object and all actions ultimately mean calling functions defined on objects. Protocols are polymorphic functions that are embbedded into python. Most importantly the interpreter is aware of them. Protocols enable: ■

consistency - programmers can rely on intuition

■

special syntax - interpreter translates nice syntax to functions on objects.

■

We will look at two protocols: __contains__ and __iter__

__add__ x + y resolves to x.__add__(y) >>> 1 + 2 3 >>> one = 1 >>> one.__add__(2) 3 >>> ‘1’ + ‘2’ ‘12’ >>> ‘1’.__add__(‘2’) ‘12’ Any object that implements the __add__ function will work with the 178

Object Oriented Programming

+ x syntax.

__contains__ __contains__ is the built in protocol for membership. x in y resolves to y.__contains__(x) When the interpreter encounters ‘b’ in [‘a’, ‘b’] it knows to look for the __contains__ function on the object to the right of in and pass it the object to the left of in as parameter. A list object has that function defined and the interpreter then executes the corresponding code block. All data structures have the concept of membership defined: >>> ‘b’ in [‘a’, ‘b’] True >>> ‘b’ in (‘a’, ‘b’) True >>> ‘b’ in {‘a’: 1, ‘b’: 2} True >>> ‘b’ in {‘a’, ‘b’} True Demonstrating __contains__: >>> [‘a’, ‘b’].__contains__(‘b’) True >>> (‘a’, ‘b’).__contains__(‘b’) 179

Parameter is any characteristic that can help in defining or classifying a particular system. That is, a parameter is an element of a system that is useful, or critical, when identifying the system, or when evaluating its performance, status, condition, etc.

Basic Computer Coding: Python

True >>> {‘a’: 1, ‘b’: 2}.__contains__(‘b’) True >>> {‘a’, ‘b’}.__contains__(‘b’) True Any object that implements the __contains__ function will work with the x in syntax. __iter__ __iter__ is how iteration is implemented in Python. This protocol is a bit more involved than the protocols. Taking this code: >>> number = [1, 2] >>> for i in [1, 2]: ...

print(i)

... 1 2 Roughly here is the sequence of events: * interpreter calls __iter__ on the list object, * an object of type iterator is returned. * interpreter then calls __next__ repeatedly on the iterator * interpreter actions the code in the for loop * interpreter interrupts the loop if a StopIteration Exception occurs. To illustrate: >>> itr_obj = [1, 2].__iter__() >>> type(itr_obj)

>>> itr_obj.__next__() 180

Object Oriented Programming

1 >>> itr_obj.__next__() 2 >>> itr_obj.__next__() Traceback (most recent call last): File “”, line 1, in StopIteration Any object that implements the __iter__ function will work with the for x in : ... syntax.

Exercise Boolean Operators Using introspection functions, which protocol functions do the following syntax resolve to: ■

3>2

■

3>> len(‘hi’) 2 >>> len([1, 2]) 2 181

Basic Computer Coding: Python

Which protocol function is called by the function len on the object it is passed? Polymorphism is an ability (in OOP) to use common interface for multiple form (data types). Suppose, we need to color a shape, there are multiple shape option (rectangle, square, circle). However we could use same method to color any shape. This concept is called Polymorphism.

Using Polymorphism in Python class Parrot: def fly(self): print(“Parrot can fly”) def swim(self): print(“Parrot can’t swim”) class Penguin: def fly(self): print(“Penguin can’t fly”) def swim(self): print(“Penguin can swim”) # common interface def flying_test(bird): bird.fly() #instantiate objects blu = Parrot() peggy = Penguin() 182

Object Oriented Programming

# passing the object flying_test(blu) flying_test(peggy) When we run above program, the output will be: Parrot can fly Penguin can’t fly In the above program, we defined two classes Parrot and Penguin. Each of them have common method fly() method. However, their functions are different. To allow polymorphism, we created common interface i.e flying_test() function that can take any object. Then, we passed the objects blu and peggy in the flying_ test() function, it ran effectively.

5.2.4 Abstraction Data abstraction and encapsulation both are often used as synonyms. Both are nearly synonym because data abstraction is achieved through encapsulation. Abstraction is used to hide internal details and show only functionalities. Abstracting something means to give names to things, so that the name captures the core of what a Modularity is the function or a whole program does. degree to which a As we consider the wide set of things in the world that we would like to represent in our programs, we find that most of them have compound structure. A date has a year, a month, and a day; a geographic position has a latitude and a longitude. To represent positions, we would like our programming language to have the capacity to “glue together” a latitude and longitude to form a pair --- a compound 183

system’s components may be separated and recombined, often with the benefit of flexibility and variety in use.

Basic Computer Coding: Python

data value --- that our programs could manipulate in a way that would be consistent with the fact that we regard a position as a single conceptual unit, which has two parts. The use of compound data also enables us to increase the modularity of the programs. If we can manipulate geographic positions directly as objects in their own right, then we can separate the part of our program that deals with values per se from the details of how those values may be represented. The general technique of isolating the parts of a program that deal with how data are represented from the parts of a program that deal with how those data are manipulated is a powerful design methodology called data abstraction. Data abstraction makes programs much easier to design, maintain, and modify. Data abstraction is similar in character to functional abstraction. When we create a functional abstraction, the details of how a function is implemented can be suppressed, and the particular function itself can be replaced by any other function with the same overall behavior. In other words, we can make an abstraction that separates the way the function is used from the details of how the function is implemented. Analogously, data abstraction is a methodology that enables us to isolate how a compound data object is used from the details of how it is constructed. The basic idea of data abstraction is to structure programs so that they operate on abstract data. That is, our programs should use data in such a way as to make as few assumptions about the data as possible. At the same time, a concrete data representation is defined, independently of the programs that use the data. The interface between these two parts of our system will be a set of functions, called selectors and constructors, that implement the abstract data in terms of the concrete representation. To illustrate this technique, we will consider how to design a set of functions for manipulating rational numbers. As you read the next few sections, keep in mind that most Python code written today uses very high-level abstract data types that are built into the language, like classes, dictionaries, and lists. Since we’re building up an understanding of how these abstractions work, we can’t use them yet ourselves. As a consequence, we will write some code that isn’t Pythonic --- it’s not necessarily the typical way to 184

Object Oriented Programming

implement our ideas in the language. What we write is instructive, however, because it demonstrates how these abstractions can be constructed! Remember that computer science isn’t just about learning to use programming languages, but also learning how they work.

Example: Arithmetic on Rational Numbers Recall that a rational number is a ratio of integers, and rational numbers constitute an important sub-class of real numbers. A rational number like 1/3 or 17/29 is typically written as: / where both the and are placeholders for integer values. Both parts are needed to exactly characterize the value of the rational number. Rational numbers are important in computer science because they, like integers, can be represented exactly. Irrational numbers (like pi or e or sqrt(2)) are instead approximated using a finite binary expansion. Thus, working with rational numbers should, in principle, allow us to avoid approximation errors in our arithmetic. However, as soon as we actually divide the numerator by the denominator, we can be left with a truncated decimal approximation (a float). >>> 1/3 0.3333333333333333 and the problems with this approximation appear when we start to conduct tests: >>> 1/3 == 0.333333333333333300000 # Beware of approximations True How computers approximate real numbers with finite-length decimal expansions is a topic for another class. The important idea here is that by representing rational numbers as ratios of integers, we avoid the approximation problem entirely. Hence, we would like to keep the numerator and denominator separate for the sake of precision, but treat them as a single unit. 185

Basic Computer Coding: Python

We know from using functional abstractions that we can start programming productively before we have an implementation of some parts of our program. Let us begin by assuming that we already have a way of constructing a rational number from a numerator and a denominator. We also assume that, given a rational number, we have a way of extracting (or selecting) its numerator and its denominator. Let us further assume that the constructor and selectors are available as the following three functions: ■

make_rat(n, d) returns the rational number with numerator n and denominator d.

■

numer(x) returns the numerator of the rational number x.

■

denom(x) returns the denominator of the rational number x.

We are using here a powerful strategy of synthesis: wishful thinking. We haven’t yet said how a rational number is represented, or how the functions numer, denom, and make_rat should be implemented. Even so, if we did have these three functions, we could then add, multiply, and test equality of rational numbers by calling them: >>> def add_rat(x, y): nx, dx = numer(x), denom(x) ny, dy = numer(y), denom(y) return make_rat(nx * dy + ny * dx, dx * dy) >>> def mul_rat(x, y): return make_rat(numer(x) * numer(y), denom(x) * denom(y)) >>> def eq_rat(x, y): return numer(x) * denom(y) == numer(y) * denom(x) Now we have the operations on rational numbers defined in terms of the selector functions numer and denom, and the constructor function make_rat, but we haven’t yet defined these functions. What we need is some way to glue together a numerator and a denominator into a unit.

186

Object Oriented Programming

Tuples To enable us to implement the concrete level of our data abstraction, Python provides a compound structure called a tuple, which can be constructed by separating values by commas. Although not strictly required, parentheses almost always surround tuples. >>> (1, 2) (1, 2) The elements of a tuple can be unpacked in two ways. The first way is via our familiar method of multiple assignment. >>> pair = (1, 2) >>> pair (1, 2) >>> x, y = pair >>> x 1 >>> y 2 In fact, multiple assignment has been creating and unpacking tuples all along. A second method for accessing the elements in a tuple is by the indexing operator, written as square brackets. >>> pair[0] 1 >>> pair[1] 2 Tuples in Python (and sequences in most other programming languages) are 0-indexed, meaning that the index 0 picks out the first element, index 1 picks out the second, and so on. One intuition 187

Basic Computer Coding: Python

that underlies this indexing convention is that the index represents how far an element is offset from the beginning of the tuple. The equivalent function for the element selection operator is called getitem, and it also uses 0-indexed positions to select elements from a tuple. >>> from operator import getitem >>> getitem(pair, 0) 1

Operator is a symbol that tells the compiler to perform specific mathematical or logical manipulations.

Tuples are native types, which means that there are built-in Python operators to manipulate them. We’ll return to the full properties of tuples shortly. At present, we are only interested in how tuples can serve as the glue that implements abstract data types. Representing Rational Numbers. Tuples offer a natural way to implement rational numbers as a pair of two integers: a numerator and a denominator. We can implement our constructor and selector functions for rational numbers by manipulating 2-element tuples. >>> def make_rat(n, d): return (n, d) >>> def numer(x): return getitem(x, 0) >>> def denom(x): return getitem(x, 1) A function for printing rational numbers completes our implementation of this abstract data type. 188

Object Oriented Programming

>>> def str_rat(x): “””Return a string ‘n/d’ for numerator n and denominator d.””” return ‘{0}/{1}’.format(numer(x), denom(x)) Together with the arithmetic operations we defined earlier, we can manipulate rational numbers with the functions we have defined. >>> half = make_rat(1, 2) >>> str_rat(half) ‘1/2’ >>> third = make_rat(1, 3) >>> str_rat(mul_rat(half, third)) ‘1/6’ >>> str_rat(add_rat(third, third)) ‘6/9’ As the final example shows, our rational-number implementation does not reduce rational numbers to lowest terms. We can remedy this by changing make_rat. If we have a function for computing the greatest common denominator of two integers, we can use it to reduce the numerator and the denominator to lowest terms before constructing the pair. As with many useful tools, such a function already exists in the Python Library. >>> from fractions import gcd >>> def make_rat(n, d): g = gcd(n, d) return (n//g, d//g) The double slash operator, //, expresses integer division, which rounds down the fractional part of the result of division. Since we know that g divides both n and d evenly, integer division is exact 189

Basic Computer Coding: Python

in this case. Now we have >>> str_rat(add_rat(third, third)) ‘2/3’

Remember The str_rat implementation above uses format strings, which contain placeholders for values. The details of how to use format strings and the format method appear in the formatting strings section of Dive Into Python 3.

as desired. This modification was accomplished by changing the constructor without changing any of the functions that implement the actual arithmetic operations.

Abstraction Barriers Before continuing with more examples of compound data and data abstraction, let us consider some of the issues raised by the rational number example. We defined operations in terms of a constructor make_rat and selectors numer and denom. In general, the underlying idea of data abstraction is to identify for each type of value a basic set of operations in terms of which all manipulations of values of that type will be expressed, and then to use only those operations in manipulating the data. We can envision the structure of the rational number system as a series of layers.

The horizontal lines represent abstraction barriers that isolate different levels of the system. At each level, the barrier separates the functions (above) that use the data abstraction 190

Object Oriented Programming

from the functions (below) that implement the data abstraction. Programs that use rational numbers manipulate them solely in terms of the their arithmetic functions: add_rat, mul_rat, and eq_rat. These, in turn, are implemented solely in terms of the constructor and selectors make_rat, numer, and denom, which themselves are implemented in terms of tuples. The details of how tuples are implemented are irrelevant to the rest of the layers as long as tuples enable the implementation of the selectors and constructor. At each layer, the functions within the box enforce the abstraction boundary because they are the only functions that depend upon both the representation above them (by their use) and the implementation below them (by their definitions). In this way, abstraction barriers are expressed as sets of functions. Abstraction barriers provide many advantages. One advantage is that they makes programs much easier to maintain and to modify. The fewer functions that depend on a particular representation, the fewer changes are required when one wants to change that representation.

The Properties of Data We began the rational-number implementation by implementing arithmetic operations in terms of three unspecified functions: make_rat, numer, and denom. At that point, we could think of the operations as being defined in terms of data objects --- numerators, denominators, and rational numbers --- whose behavior was specified by the latter three functions. But what exactly is meant by data? It is not enough to say “whatever is implemented by the given selectors and constructors.” We need to guarantee that these functions together specify the right behavior. That is, if we construct a rational number x from integers n and d, then it should be the case that numer(x)/denom(x) is equal to n/d. In general, we can think of an abstract data type as defined by some collection of selectors and constructors, together with some behavior conditions. As long as the behavior conditions are met (such as the division property above), these functions constitute a valid representation of the data type. 191

Basic Computer Coding: Python

This point of view can be applied to other data types as well, such as the two-element tuple that we used in order to implement rational numbers. We never actually said much about what a tuple was, only that the language supplied operators to create and manipulate tuples. We can now describe the behavior conditions of two-element tuples, also called pairs, that are relevant to the problem of representing rational numbers. In order to implement rational numbers, we needed a form of glue for two integers, which had the following behavior: If a pair p was constructed from values x and y, then getitem_pair(p, 0) returns x, and getitem_pair(p, 1) returns y.

■

We can implement functions make_pair and getitem_pair that fulfill this description just as well as a tuple. >>> def make_pair(x, y): “””Return a function that behaves like a pair.””” def dispatch(m): if m == 0: return x elif m == 1: return y return dispatch >>> def getitem_pair(p, i): “””Return the element at index i of pair p.””” return p(i) With this implementation, we can create and manipulate pairs. >>> p = make_pair(1, 2) >>> getitem_pair(p, 0) 1 192

Object Oriented Programming

>>> getitem_pair(p, 1) 2 This use of functions corresponds to nothing like our intuitive notion of what data should be. Nevertheless, these functions suffice to represent compound data in our programs. The subtle point to notice is that the value returned by make_pair is a function called dispatch, which takes an argument m and returns either x or y. Then, getitem_pair calls this function to retrieve the appropriate value. The point of exhibiting the functional representation of a pair is not that Python actually works this way (tuples are implemented more directly, for efficiency reasons) but that it could work this way. The functional representation, although obscure, is a perfectly adequate way to represent pairs, since it fulfills the only conditions that pairs need to fulfill. This example also demonstrates that the ability to manipulate functions as values automatically provides us the ability to represent compound data.

193

Terminology invoking “objects” and “oriented” in the modern sense of object-oriented programming made its first appearance at MIT in the late 1950s and early 1960s. In the environment of the artificial intelligence group, as early as 1960, “object” could refer to identified items (LISP atoms) with properties (attributes) Alan Kay was later to cite a detailed understanding of LISP internals as a strong influence on his thinking in 1966.

Basic Computer Coding: Python

SUMMARY ■

Object-oriented programming (OOP) is a programming language model organized around objects rather than “actions” and data rather than logic.

■

Object-oriented Programming, or OOP for short, is a programming paradigm which provides a means of structuring programs so that properties and behaviors are bundled into individual objects.

■

Instance methods are defined inside a class and are used to get the contents of an instance. They can also be used to perform operations with the attributes of our objects.

■

Inheritance is the process by which one class takes on the attributes and methods of another. Newly formed classes are called child classes, and the classes that child classes are derived from are called parent classes.

■

Python has been an object-oriented language from day one. Because of this, creating and using classes and objects are downright easy.

■

Inheritance is a way of creating new class for using details of existing class without modifying it. The newly formed class is a derived class (or child class). Similarly, the existing class is a base class (or parent class).

■

Encapsulation is the packing of data and functions operating on that data into a single component and restricting the access to some of the object’s components. Encapsulation means that the internal representation of an object is generally hidden from view outside of the object’s definition.

■

Polymorphism means that different types respond to the same function. Polymorphism is very useful as it makes programming more intuitive and therefore easier. Polymorphism is a fancy word that just means the same function is defined on objects of different types.

■

Abstraction is used to hide internal details and show only functionalities. Abstracting something means to give names to things, so that the name captures the core of what a function or a whole program does.

194

Object Oriented Programming

KNOWLEDGE CHECK 1.

2.

3.

4.

5.

Which is private member functions access scope? a.

Member functions which can only be used within the class

b.

Member functions which can used outside the class

c.

Member functions which are accessible in derived class

d.

Member functions which can’t be accessed inside the class

Which among the following is true? a.

The private members can’t be accessed by public members of the class

b.

The private members can be accessed by public members of the class

c.

The private members can be accessed only by the private members of the class

d.

The private members can’t be accessed by the protected members of the class

Which member can never be accessed by inherited classes? a.

Private member function

b.

Public member function

c.

Protected member function

d.

All can be accessed

Which syntax among the following shows that a member is private in a class? a.

private: functionName(parameters)

b.

private(functionName(parameters))

c.

private functionName(parameters)

d.

private::functionName(parameters)

How many private member functions are allowed in a class? a.

Only 1

b.

Only 7 195

Basic Computer Coding: Python

c.

Only 255

d.

As many as required

REVIEW QUESTIONS 1.

What are the classes and objects (instances) in python?

2.

How to instantiating the objects? Explain with suitable example.

3.

Discuss on the concept of inheritance.

4.

What the mechanism of encapsulation in OOPS?

Check Your Result 1. (a)

2. (b)

3. (a)

196

4. (c)

5. (d)

Object Oriented Programming

REFERENCES 1.

Beal, V. (2016). What is Polymorphism? Webopedia Definition. [online] Webopedia.com. Available at: http:// www.webopedia.com/TERM/P/polymorphism.html [Accessed 8 May 2016].

2.

http://composingprograms.com/pages/22-data-abstraction. html

3.

http://wla.berkeley.edu/~cs61a/fa11/lectures/functions. html

4.

http://zetcode.com/lang/python/oop/

5.

https://medium.com/@manjuladube/encapsulationabstraction-35999b0a3911

6.

https://medium.com/the-renaissance-developer/python101-object-oriented-programming-part-1-7d5d06833f26

7.

https://pdfs.semanticscholar.org/presentation/6ee1/ add06c8938ed02a1b4a5d01515c234051677.pdf

8.

https://python-intro.readthedocs.io/en/latest/ polymorphism.html

9.

https://pythonprogramminglanguage.com/encapsulation/

10.

h ttp s ://realp y th on .com / p y th on 3 -obje ct-o rie n te d programming/

11.

https://searchmicroservices.techtarget.com/definition/ object-oriented-programming-OOP

12.

https://wizardforcel.gitbooks.io/sicp-in-python/content/9. html

13.

https://www.cs.colorado.edu/~kena/classes/5448/f12/ presentation-materials/li.pdf

14.

https://www.cs.uky.edu/~keen/115/notes/python_classes_ objects.pdf

15.

https://www.johnny-lin.com/pyintro/ed01/free_pdfs/ch07. pdf

16.

https://www.programiz.com/python-programming/ inheritance 197

Basic Computer Coding: Python

17.

https://www.programiz.com/python-programming/objectoriented-programming#methods

18.

Lambert, S. (2012). Quick Tip: The OOP Principle of Encapsulation. [online] Game Development Envato Tuts+. Available at: http://gamedevelopment.tutsplus.com/ tutorials/quick-tip-the-oop-principle-of-encapsulation-gamedev-2187 [Accessed 10 May 2016].

19.

Obbayi, R. (2016). Compare Structured and Object-Oriented Programming: What Are the Real Differences?. [online] Bright Hub. Available at: http://www.brighthub.com/ internet/web-development/articles/82024.aspx [Accessed 6 May 2016].

198

CHAPTER 6

PYTHON REGULAR EXPRESSION “In this beginner-friendly book, called ‘Learn to Program with Minecraft,’ you will learn how to do cool things in Minecraft using the Python programming language. No prior programming experience is needed.” ―Mark Frauenfelder

Learning Objectives After studying this chapter, you will be able to: ■

Understand the regex search and match

■

Learn about regular expression modifiers: option flags

Basic Computer Coding: Python

INTRODUCTION A regular expression is a special sequence of characters that helps you match or find other strings or sets of strings, using a specialized syntax held in a pattern. Regular expressions are widely used in UNIX world. The module re provides full support for Perl-like regular expressions in Python. The re module raises the exception re.error if an error occurs while compiling or using a regular expression. We would cover two important functions, which would be used to handle regular expressions. But a small thing first: There are various characters, which would have special meaning when they are used in regular expression. To avoid any confusion while dealing with regular expressions, we would use Raw Strings as r’expression’.

6.1 REGEX SEARCH AND MATCH Python is a high level open source scripting language. Python’s built-in “re” module provides excellent support for regular expressions, with a modern and complete regex flavor. The only Constant is a significant features missing from Python’s value that cannot regex syntax are atomic grouping, possessive be altered by the quantifiers, and Unicode properties. program during The first thing to do is to import the regexp normal execution, module into your script with import re. i.e., the value is constant. Call re.search(regex, subject) to apply a regex pattern to a subject string. The function returns None if the matching attempt fails, and a Match object otherwise. Since None evaluates to False,

200

Python Regular Expression

you can easily use re.search() in an if statement. The Match object stores details about the part of the string matched by the regular expression pattern. You can set regex matching modes by specifying a special constant as a third parameter to re.search(). re.I or re.IGNORECASE applies the pattern case insensitively. re.S or re.DOTALL makes the dot match newlines. re.Mor re.MULTILINE makes the caret and dollar match after and before line breaks in the subject string. There is no difference between the single-letter and descriptive options, except for the number of characters you have to type in. To specify more than one option, “or” them together with the | operator: re.search(“^a”, “abc”, re.I | re.M). By default, Python’s regex engine only considers the letters A through Z, the digits 0 through 9, and the underscore as “word characters”. Specify the flag re.L or re.LOCALE to make \w match all characters that are considered letters given the current locale settings. Alternatively, you can specify re.U or re.UNICODE to treat all letters from all scripts as word characters. The setting also affects word boundaries. Do not confuse re.search() with re.match(). Both functions do exactly the same, with the important distinction that re.search() will attempt the pattern throughout the string, until it finds a match. re.match() on the other hand, only attempts the pattern at the very start of the string. Basically, re.match(“regex”, subject) is the same as re.search(“\Aregex”, subject). Python 3.4 adds a new re.fullmatch() function. This function only returns a Match object if the regex matches the string entirely. Otherwise 201

Remember re.match() does not require the regex to match the entire string. re.match(“a”, “ab”) will succeed.

Basic Computer Coding: Python

it returns None. re.fullmatch(“regex”, subject) is the same as re.search(“\Aregex\Z”, subject). This is useful for validating user input. If subject is an empty string then fullmatch() evaluates to True for any regex that can find a zero-length match. To get all matches from a string, call re.findall(regex, subject). This will return an array of all non-overlapping regex matches in the string. “Non-overlapping” means that the string is searched through from left to right, and the next match attempt starts beyond the previous match. If the regex contains one or more capturing groups, re.findall() returns an array of tuples, with each tuple containing text matched by all the capturing groups. The overall regex match is not included in the tuple, unless you place the entire regex inside a capturing group. More efficient than re.findall() is re.finditer(regex, subject). It returns an iterator that enables you to loop over the regex matches in the subject string: for m in re.finditer(regex, subject). The for-loop variable m is a Match object with the details of the current match. Unlike re.search() and re.match(), re.findall() and re.finditer() do not support an optional third parameter with regex matching flags. Instead, you can use global mode modifiers at the start of the regex. E.g. “(?i)regex” matches regex case insensitively.

6.1.1 The Match Function This function attempts to match RE pattern to string with optional flags. Here is the syntax for this function − re.match(pattern, string, flags=0) Here is the description of the parameters − Sr.No.

Parameter & Description

1

pattern This is the regular expression to be matched.

202

Python Regular Expression

2

string This is the string, which would be searched to match the pattern at the beginning of string.

3

flags You can specify different flags using bitwise OR (|). These are modifiers, which are listed in the table below.

The re.match function returns a match object on success, None on failure. We use group(num) or groups() function of match object to get matched expression. Sr.No.

Match Object Method & Description

1

group(num=0) This method returns entire match (or specific subgroup num)

2

groups() This method returns all matching subgroups in a tuple (empty if there weren’t any)

#!/usr/bin/python import re

line = “Cats are smarter than dogs”

matchObj = re.match( r’(.*) are (.*?) .*’, line, re.M|re.I)

if matchObj: print “matchObj.group() : “, matchObj.group() print “matchObj.group(1) : “, matchObj.group(1) print “matchObj.group(2) : “, matchObj.group(2) else: print “No match!!”

203

Basic Computer Coding: Python When the above code is executed, it produces following result − matchObj.group() : Cats are smarter than dogs matchObj.group(1) : Cats matchObj.group(2) : smarter

6.1.2 The Search Function This function searches for first occurrence of RE pattern within string with optional flags. Here is the syntax for this function − re.search(pattern, string, flags=0) Here is the description of the parameters − Sr.No.

Parameter & Description

1

pattern This is the regular expression to be matched.

2

string This is the string, which would be searched to match the pattern anywhere in the string.

3

flags You can specify different flags using bitwise OR (|). These are modifiers, which are listed in the table below.

The re.search function returns a match object on success, none on failure. We use group(num) or groups() function of match object to get matched expression. Sr.No.

Match Object Methods & Description

1

group(num=0) This method returns entire match (or specific subgroup num)

204

Python Regular Expression

2

groups() This method returns all matching subgroups in a tuple (empty if there weren’t any)

Example #!/usr/bin/python import re line = “Cats are smarter than dogs”; searchObj = re.search( r’(.*) are (.*?) .*’, line, re.M|re.I) if searchObj: print “searchObj.group() : “, searchObj.group() print “searchObj.group(1) : “, searchObj.group(1) print “searchObj.group(2) : “, searchObj.group(2) else: print “Nothing found!!” When the above code is executed, it produces following result − searchObj.group() : Cats are smarter than dogs searchObj.group(1) : Cats searchObj.group(2) : smarter

6.1.3 Matching Versus Searching Python offers two different primitive operations based on regular expressions: match checks for a match only at the beginning of the string, while search checks for a match anywhere in the string (this is what Perl does by default).

Example #!/usr/bin/python 205

Basic Computer Coding: Python

import re String is traditionally a sequence of characters, either as a literal constant or as some kind of variable.

line = “Cats are smarter than dogs”; matchObj = re.match( r’dogs’, line, re.M|re.I) if matchObj: print “match --> matchObj.group() : “, matchObj.group() else: print “No match!!” searchObj = re.search( r’dogs’, line, re.M|re.I) if searchObj: print “search --> searchObj.group() : “, searchObj.group() else:

Regular expressions originated in 1951, when mathematician Stephen Cole Kleene described regular languages using his mathematical notation called regular sets. These arose in theoretical computer science, in the subfields of automata theory (models of computation) and the description and classification of formal languages.

print “Nothing found!!” When the above code is executed, it produces the following result − No match!! search --> matchObj.group() : dogs

6.1.4 Search and Replace One of the most important re methods that use regular expressions is sub.

Syntax re.sub(pattern, repl, string, max=0) This method replaces all occurrences of the RE pattern in string with repl, substituting all occurrences unless max provided. This method returns modified string. 206

Python Regular Expression

Example #!/usr/bin/python import re phone = “2004-959-559 # This is Phone Number” # Delete Python-style comments num = re.sub(r’#.*$’, “”, phone) print “Phone Num : “, num # Remove anything other than digits num = re.sub(r’\D’, “”, phone) print “Phone Num : “, num When the above code is executed, it produces the following result − Phone Num : 2004-959-559 Phone Num : 2004959559

6.2 REGULAR EXPRESSION MODIFIERS: OPTION FLAGS Regular expression literals may include an optional modifier to control various aspects of matching. The modifiers are specified as an optional flag. You can provide multiple modifiers using exclusive OR (|), as shown previously and may be represented by one of these –

207

Regular expression is a sequence of characters that define a search pattern.

Basic Computer Coding: Python

Sr.No.

Modifier & Description

1

re.I Performs case-insensitive matching.

2

re.L Interprets words according to the current locale. This interpretation affects the alphabetic group (\w and \W), as well as word boundary behavior(\b and \B).

3

re.M Makes $ match the end of a line (not just the end of the string) and makes ^ match the start of any line (not just the start of the string).

4

re.S Makes a period (dot) match any character, including a newline.

5

re.U Interprets letters according to the Unicode character set. This flag affects the behavior of \w, \W, \b, \B.

6

re.X Permits “cuter” regular expression syntax. It ignores whitespace (except inside a set [] or when escaped by a backslash) and treats unescaped # as a comment marker.

6.2.1 Regular Expression Patterns Except for control characters, (+ ? . * ^ $ ( ) [ ] { } | \), all characters match themselves. You can escape a control character by preceding it with a backslash. Following table lists the regular expression syntax that is available in Python − Sr.No.

Pattern & Description

1

^ Matches beginning of line.

2

$ Matches end of line.

208

Python Regular Expression 3

. Matches any single character except newline. Using m option allows it to match newline as well.

4

[...] Matches any single character in brackets.

5

[^...] Matches any single character not in brackets

6

re* Matches 0 or more occurrences of preceding expression.

7

re+ Matches 1 or more occurrence of preceding expression.

8

re? Matches 0 or 1 occurrence of preceding expression.

9

re{ n} Matches exactly n number of occurrences of preceding expression.

10

re{ n,} Matches n or more occurrences of preceding expression.

11

re{ n, m} Matches at least n and at most m occurrences of preceding expression.

12

a| b Matches either a or b.

13

(re) Groups regular expressions and remembers matched text.

14

(?imx) Temporarily toggles on i, m, or x options within a regular expression. If in parentheses, only that area is affected.

15

(?-imx) Temporarily toggles off i, m, or x options within a regular expression. If in parentheses, only that area is affected.

16

(?: re) Groups regular expressions without remembering matched text.

209

Basic Computer Coding: Python 17

(?imx: re) Temporarily toggles on i, m, or x options within parentheses.

18

(?-imx: re) Temporarily toggles off i, m, or x options within parentheses.

19

(?#...) Comment.

20

(?= re) Specifies position using a pattern. Doesn’t have a range.

21

(?! re) Specifies position using pattern negation. Doesn’t have a range.

22

(?> re) Matches independent pattern without backtracking.

23

\w Matches word characters.

24

\W Matches nonword characters.

25

\s Matches whitespace. Equivalent to [\t\n\r\f].

26

\S Matches nonwhitespace.

27

\d Matches digits. Equivalent to [0-9].

28

\D Matches nondigits.

29

\A Matches beginning of string.

30

\Z Matches end of string. If a newline exists, it matches just before newline.

31

\z Matches end of string.

210

Python Regular Expression 32

\G Matches point where last match finished.

33

\b Matches word boundaries when outside brackets. Matches backspace (0x08) when inside brackets.

34

\B Matches nonword boundaries.

35

\n, \t, etc. Matches newlines, carriage returns, tabs, etc.

36

\1...\9 Matches nth grouped subexpression.

37

\10 Matches nth grouped subexpression if it matched already. Otherwise refers to the octal representation of a character code.

6.2.2 Regular Expression Examples Literal characters Sr.No.

Example & Description

1

python Match “python”.

Character classes Sr.No.

Example & Description

1

[Pp]ython Match “Python” or “python”

2

rub[ye] Match “ruby” or “rube”

211

Basic Computer Coding: Python

3

[aeiou] Match any one lowercase vowel

4

[0-9] Match any digit; same as [0123456789]

5

[a-z] Match any lowercase ASCII letter

6

[A-Z] Match any uppercase ASCII letter

7

[a-zA-Z0-9] Match any of the above

8

[^aeiou] Match anything other than a lowercase vowel

9 Digit is a single character in a numeric system. For example, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 are all digits.

[^0-9] Match anything other than a digit

Special Character Classes Sr.No.

Example & Description

1

. Match any character except newline

2

\d Match a digit: [0-9]

3

\D Match a nondigit: [^0-9]

4

\s Match a whitespace character: [ \t\r\ n\f]

5

\S Match nonwhitespace: [^ \t\r\n\f] 212

Python Regular Expression

6

\w Match a single [A-Za-z0-9_]

7

word

character:

\W Match a nonword [^A-Za-z0-9_]

character:

Repetition Cases Sr.No.

Example & Description

1

ruby? Match “rub” or “ruby”: the y is optional

2

ruby* Match “rub” plus 0 or more ys

3

ruby+ Match “rub” plus 1 or more ys

4

\d{3} Match exactly 3 digits

5

\d{3,} Match 3 or more digits

6

\d{3,5} Match 3, 4, or 5 digits

Nongreedy repetition This matches the smallest number of repetitions − Sr.No.

Example & Description

1

Greedy repetition: matches “perl>”

2

Nongreedy: matches “” in “perl>” 213

Basic Computer Coding: Python

Grouping with Parentheses Sr.No.

Example & Description

1

\D\d+ No group: + repeats \d

2

(\D\d)+ Grouped: + repeats \D\d pair

3

([Pp]ython(, )?)+ Match “Python”, “Python, python, python”, etc.

Backreferences This matches a previously matched group again − Sr.No.

Example & Description

1

([Pp])ython&\1ails Match python&pails or Python&Pails

2

([‘”])[^\1]*\1 Single or double-quoted string. \1 matches whatever the 1st group matched. \2 matches whatever the 2nd group matched, etc.

Alternatives Sr.No.

Example & Description

1

python|perl Match “python” or “perl”

2

rub(y|le)) Match “ruby” or “ruble”

3

Python(!+|\?) “Python” followed by one or more ! or one ? 214

Python Regular Expression

Anchors This needs to specify match position. Sr.No.

Example & Description

1

^Python Match “Python” at the start of a string or internal line

2

Python$ Match “Python” at the end of a string or line

3

\APython Match “Python” at the start of a string

4

Python\Z Match “Python” at the end of a string

5

\bPython\b Match “Python” at a word boundary

6

\brub\B \B is nonword boundary: match “rub” in “rube” and “ruby” but not alone

7

Python(?=!) Match “Python”, if followed by an exclamation point.

8

Python(?!!) Match “Python”, if not followed by an exclamation point.

Special Syntax with Parentheses Sr.No.

Example & Description

1

R(?#comment) Matches “R”. All the rest is a comment

2

R(?i)uby Case-insensitive while matching “uby”

215

Basic Computer Coding: Python

3

R(?i:uby) Same as above

4

rub(?:y|le)) Group only without creating \1 backreference

216

Python Regular Expression

CASE STUDY Python Python is an interpreted, dynamically-typed, object-oriented scripting language with a host of built-in data types. It is implemented in C, but in a very object-oriented fashion. The design is a good model for language implementation. The Python interpreter works by loading a source file or reading a line typed at the keyboard, parsing it into an abstract syntax tree, compiling the tree into bytecode, and executing the bytecode. We will concentrate mainly on how the bytecode is executed, such as how inheritance and environments are implemented.

Parsing The parsing process is fairly standard. The basic idea is to first convert the input characters into a more abstract representation like name: x, integer: 7, string: “hello”, less-than-or-equal, etc. The abstracted characters are called tokens. (There are utilities, such as lex, for automatically generating tokenizers, but Python does not use one.) The tokenization process is fully described in the language reference. For example, the statement while(x y) { int x = 2; cout > re.search(‘aaaa’, “alohaaaa”, 0)

b.

>>> re.match(‘aaaa’, “alohaaaa”, 0)

c.

>>> re.match(‘aaa’, “alohaaa”, 0)

d.

>>> re.search(‘aaa’, “alohaaa”, 0)

Which of the following functions clears the regular expression cache? a.

re.sub()

b.

re.pos()

c.

re.purge()

d.

re.subn()

Which of the following functions results in case insensitive matching? a.

re.A

b.

re.U

226

Python Regular Expression

c.

re.I

d.

re.X

REVIEW QUESTIONS 1.

What is the match function?

2.

What is the search function?

3.

Discuss about search and replace.

4.

Discuss about regular expression modifiers: option flags.

5.

Describe the regular expression patterns.

Check Your Result 1. (d)

2. (b)

3. (a)

227

4. (c)

5. (d)

Basic Computer Coding: Python

REFERENCES 1.

A.M. Kuchling (2001-12-21). “PEP 255: Simple Generators”. What’s New in Python 2.2. Python Foundation. Retrieved 2008-09-05.

2.

A.M. Kuchling (2010-07-03). “What’s New in Python 2.7”. Retrieved 2012-10-07.

3.

A.M. Kuchling (December 21, 2001). “PEPs 252 and 253: Type and Class Changes”. What’s New in Python 2.2. Python Foundation. Archived from the original on September 17, 2008. Retrieved September 5, 2008.

4.

Barry Warsaw (2011-11-09). “PEP 404 -- Python 2.8 Unrelease Schedule”. Retrieved 2012-10-07.

5.

Guido van Rossum (January 20, 2009). “The History of Python”. Retrieved March 3, 2018.

6.

Guido van Rossum. “The fate of reduce() in Python 3000”. Artima Developer. Retrieved 2007-03-22.

7.

http://alumni.media.mit.edu/~tpminka/patterns/python/

8.

https://www.regular-expressions.info/python.html

9.

https://www.tutorialspoint.com/python/python_reg_ expressions.htm

10.

Neal Norwitz; Barry Warsaw (2006-06-29). “PEP 361 -Python 2.6 and 3.0 Release Schedule”. Retrieved 201210-07.

11.

Rossum, Guido van van. “Python 3000 FAQ”. artima. com. Retrieved December 27, 2016.

228

CHAPTER 7

PYTHON MULTITHREADING The main languages out of which web applications are built - whether it›s Perl or Python or PHP or any of the other languages - those are all open source languages. So the infrastructure of the web is open source the web as we know it is completely dependent on open source. ―Mitch Kapor

Learning Objectives After studying this chapter, you will be able to: ■

Python threading – Python multithreading

■

Functions in python multithreading

Basic Computer Coding: Python

INTRODUCTION Multithreading is a core concept of software programming that almost all the high-level programming languages support. And Python multithreading is one of the best examples of simplistic implementation of threads. Multithreading (sometimes simply “threading”) is when a program creates multiple threads with execution cycling among them, so one longer-running task does not block all the others. This works well for tasks that can be broken down into smaller subtasks, which can then each be given to a thread to be completed.

In software programming, a thread is the smallest unit of execution with the independent set of instructions. It is a part of the process and operates in the same context sharing program’s runnable resources like memory. A thread has a starting point, an execution sequence, and a result. It has an instruction pointer that holds the current state of the thread and controls what executes next in what order. Similarly, the ability of a process to execute multiple threads parallel is called multithreading. Ideally, multithreading can significantly improve the performance of any program. And Python multithreading mechanism is pretty user-friendly which you can learn quickly.

230

Python Multithreading

7.1 PYTHON THREADING – PYTHON MULTITHREADING Python threading module is used to implement multithreading in python programs. Threading in python is used to run multiple threads (tasks, function calls) at the same time. Note that this does not mean that they are executed on different CPUs. Python threads will not make your program faster if it already uses 100 % CPU time. In that case, you probably want to look into parallel programming. If you are interested in parallel programming with python. Python multiprocessing is one of the similar module that we looked into sometime back.

The threading module builds on the low-level features of thread to make working with threads even easier and more pythonic. Using threads allows a program to run multiple operations concurrently in the same process space. In Computer Science, threads are defined as the smallest unit of work which is scheduled to be done by an Operating System. Some points to consider about Threads are: 231

Python 2.0 was released on 16 October 2000 and had many major new features, including a cycledetecting garbage collector and support for Unicode. With this release, the development process became more transparent and communitybacked.

Basic Computer Coding: Python

■

Threads exists inside a process.

■

Multiple threads can exist in a single process.

■

Threads in same process share the state and memory of the parent process.

This was just a quick overview of Threads in general. This post will mainly focus on the threading module in Python.

7.1.1 Getting Started with Python Multithreading Running several threads is similar to running several different programs concurrently, but with the following benefits − ■

Multiple threads within a process share the same data space with the main thread and can therefore share information or communicate with each other more easily than if they were separate processes.

■

Threads sometimes called light-weight processes and they do not require much memory overhead; they are cheaper than processes.

A thread has a beginning, an execution sequence, and a conclusion. It has an instruction pointer that keeps track of where within its context it is currently running. ■

It can be pre-empted (interrupted)

■

It can temporarily be put on hold (also known as sleeping) while other threads are running - this is called yielding.

Let us start by creating a Python module, named download.py. This file will contain all the functions necessary to fetch the list of images and download them. We will split these functionalities into three separate functions: ■

get_links

■

download_link

■

setup_download_dir

The third function, setup_download_dir, will be used to create a download destination directory if it does not already exist. Imgur’s API requires HTTP (Hypertext Transfer Protocol) requests to bear the Authorization header with the client ID. You can find this client ID from the dashboard of the application that you have 232

Python Multithreading

registered on Imgur, and the response will be JSON encoded. We can use Python’s standard JSON library to decode it. Downloading the image is an even simpler task, as all you have to do is fetch the image by its URL and write it to a file.

7.1.2 Python Multithreading Modules for Thread Implementation Python offers two modules to implement threads in programs. ■

module and

■

module.

For your information, module is deprecated in Python 3 and renamed to module for backward compatibility. But we will explain both methods because many of the users still use legacy Python versions. The key difference between the two modules is that the module implements a thread as a function. On the other hand, the module offers an object-oriented approach to enable thread creation. IronPython, a Python implementation using the .NET framework, does not have a GIL, and neither does Jython, the Java-based implementation. You can find a list of working Python implementations.

7.1.3 Difference between Multiprocessing and Multithreading Multiprocessing and Multithreading both adds performance to the system. Multiprocessing is adding more number of or CPUs/processors 233

HTTP is an application protocol for distributed, collaborative, and hypermedia information systems. HTTP is the foundation of data communication for the World Wide Web.

Basic Computer Coding: Python

to the system which increases the computing speed of the system. Multithreading is allowing a process to create more threads which increase the responsiveness of the system.

Comparison Chart BASIS FOR COMPARISON

MULTIPROCESSING

MULTITHREADING

Basic

Multiprocessing adds CPUs to increase computing power.

Multithreading creates multiple threads of a single process to increase computing power.

Execution

Multiple processes are executed concurrently.

Multiple threads of a single process are executed concurrently.

Creation

Creation of a process is time-consuming and resource intensive.

Creation of a thread is economical in both sense time and resource.

Classification

Multiprocessing can be symmetric or asymmetric.

Multithreading is not classified.

Key Differences Multithreading

between

Multiprocessing

and

■

The key difference between multiprocessing and multithreading is that multiprocessing allows a system to have more than two CPUs added to the system whereas multithreading lets a process generate multiple threads to increase the computing speed of a system.

■

Multiprocessing system executes multiple processes simultaneously whereas, the multithreading system let execute multiple threads of a process simultaneously.

■

Creating a process can consume time and even exhaust the system resources. However creating threads is economical as threads belonging to the same process share the belongings of that process. 234

Python Multithreading

■

Multiprocessing can be classified into symmetric multiprocessing and asymmetric multiprocessing whereas, multithreading is not classified further.

The benefits of multithreading can be gradually increased in multiprocessing environment as multithreading on a multiprocessing system increases parallelism.

7.2 FUNCTIONS IN PYTHON MULTITHREADING The module ‘threading’, for Python, helps us with thread-based parallelism. It constructs higher-level threading interfaces on top of the lower level _thread module. Where _thread is missing, we cannot use threading. For such situations, we have dummy_threading.

We have the following functions in the Python Multithreading module: a. active_count() This returns the number of alive(currently) Thread objects. This is equal to the length the of the list that enumerate() returns. 1. >>> threading.active_count() 235

Remember The fork function creates a copy of the process, all memory pages are copied, open file descriptors are copied etc. All this stuff is intuitive for a UNIX programmer. One important thing that differs the child process from the parent is that the child has only one thread. Cloning the whole process with all threads would be problematic and in most cases not what the programmer wants.

Basic Computer Coding: Python

2 b. current_thread() Based on the caller’s thread of control, this returns the current Thread object. If this thread of control is not through ‘threading’, it returns a dummy thread object with limited functionality. 1. >>> threading.current_thread() 2. 3. c. get_ident() get_ident() returns the current thread’s identifier, which is a nonzero integer. We can use this to index a dictionary of thread-specific data. Apart from that, it has no special meaning. When one thread exits and another creates, Python recycles such an identifier. 1. >>> threading.get_ident() 14352 d. enumerate() This returns a list of all alive(currently) Thread objects. This includes the main thread, daemonic threads, and dummy thread objects created by current_thread(). This obviously does not include terminated threads as well as those that have not begun yet. 1. >>> threading.enumerate() 2. [,

e. main_thread() This method returns the main Thread object. Normally, it is that thread which started the interpreter. 1. >>> threading.main_thread() 2. 236

Python Multithreading

f. settrace(func) settrace() traces a function for all threads we started using ‘threading’. The argument func passes to sys.settrace() for each thread before it calls its run() method. 1. >>> def sayhi(): 2. print(“Hi”) 3. >>> threading.settrace(sayhi) 4. >>> g. setprofile(func) This method sets a profile function for all threads we started from ‘threading’. It passes func to sys.setprofile() for each thread before it calls its run() method. 1. >>> threading.setprofile(sayhi) 2. >>> h. stack_size([size]) stack_size() returns the stack size of a thread when creating new threads. size is the stack size we want to use for subsequently created threads. This must be equal to 0 or a positive integer of value at least 32,768 (32KiB). When not specified, it uses 0. And if it does not support changing thread stack size, it raises a RuntimeError. When we pass an invalid stack size, it raises a ValueError, and does not modify it. The minimum stack size it currently supports to guarantee enough stack space for the interpreter itself is 32KiB. Some platforms may need a minimum stack size of greater than 32KiB. Others may need to allocate in multiples of system memory page size. 1. >>> threading.stack_size() 0 Apart from functions, ‘threading’ also defines a constant. 237

Basic Computer Coding: Python

h. TIMEOUT_MAX This holds the maximum allowed value for this constant, the timeout parameter for blocking functions like Lock.acquire(), Condition. wait(), RLock.acquire(), and others. If we denote a timeout greater than this, it raises an OverflowError. 1. >>> threading.TIMEOUT_MAX 4294967.0 Thread of execution is the smallest sequence of programmed instructions that can be managed independently by a scheduler, which is typically a part of the operating system.

In Java, locks and condition variables are the basic behavior of every object. Whereas in Python, they are individual objects. Here, the class Thread supports some of the functionality of class Thread in Java. However, currently, we have no thread groups, priorities, and we cannot destroy, stop, suspend, resume, or interrupt threads. When we implement the static methods from Java’s Thread, they map to module-level functions. This way, ‘threading’ is much like Java’s threading model in design.

7.2.1 Thread-Local Data That data for which the values are threadspecific, is thread-local. To manage such data, we can create an instance of local/a subclass, and then store attributes on it. 1. >>> mydata=threading.local() 2. >>> mydata.x=7 3. >>> These instance values differ for each thread. We have the following class denoting threadlocal data: class threading.local 238

Python Multithreading

A class that represents thread-local data.

7.2.2 Thread Objects The Thread class that we mentioned earlier in this blog denotes an activity running in a separate thread of control. We can represent this activity either by passing a callable object to the constructor, ot by overriding the method run() in a subclass. You must make sure to not override other methods in a subclass, except for the constructor. In short, only override a class’ __init__() and run() methods. Once the interpreter creates a thread object, we must start its activity by calling its start() method. This will invoke its run() method in a separate thread of control. Once this happens, we consider the thread to be ‘alive’. When run() terminates normally or raising an exception we did not handle, it is no longer alive. To test whether a thread is alive, we may use the method is_alive(). A thread may call another’s join() method. This will block the calling thread until the other terminates. Threads have names, and we can pass these names to the constructor, and even read or modify them. We can flag a thread as a ‘daemon thread’. This means that the whole program exits when only the daemon threads remain. This initial value comes from the creating thread. We can set this flag via the property ‘daemon’, or via the constructor argument ‘daemon’. Daemons abruptly stop at shutdown, and they may not properly release all the resources held. These resources may include open files, database transactions, and others. To stop our threads gracefully, we must make them non-daemonic. It is also preferable to use a suitable signaling process, like an Event. The ‘main thread’ object pertains to the initial thread of control in our program; it is not a daemon. Finally, it is possible that the interpreter creates ‘dummy thread objects’. These are ‘alien threads’ (threads of control started outside ‘threading’, for ex, directly from C code). Such objects have limited functionality, and are always live and daemonic. We cannot join() them. We can also never delete them since it is impossible to detect when they terminate. 239

Basic Computer Coding: Python

This is the class: class threading.Thread(group=None, args=(), kwargs={}, *, daemon=None)

target=None,

name=None,

Take note: ■

Always call the constructor with keyword arguments. It has the following arguments:

■

group must be None. Python reserves this for future extension when we implement a ThreadGroup class.

■

target is a callable object that run() will invoke. The default for this is None, which means it calls nothing.

■

name is the name of the thread. The default for this is “Thread-N”. Here, N is a small decimal number.

■

args is an argument tuple. It helps invoke the target. The default for this is ().

■

Kwargs is a dictionary holding keyword arguments. Even this helps invoke the target. The default for this is {}.

■

daemon decides whether the thread is daemonic. When None, it inherits the daemonic property from the current thread. The default for this is None.

■

Ensure that you invoke the base class constructor(Thread.__ init__()) first if the subclass overrides the constructor.

Thread has the following methods: a. start() This starts thread activity. For a thread object, we can call it maximum once; if we call it again, it raises a RuntimeError. This lets run() for the object invoke in a separate thread of control. 1. >>> threading.Thread.start(threading.current_thread()) 2. Traceback (most recent call last): 3. File “”, line 1, in 4. threading.Thread.start(threading.current_thread()) RuntimeError: threads can only be started once b. run() 240

Python Multithreading

This method explains the thread’s activity. It invokes the callable object we passed to the object’s constructor as the target argument, if it exists. This is with keyword and sequential arguments from kwargs and args. We can override run() in a subclass. c. join(timeout=None) For join() to work, we must wait until the thread terminates. Because when that happens, it blocks the calling thread until the one on which we call join() terminates normally or via an exception we did not handle, or until timeout occurs. When you do provide a timeout (other than None), make sure it is a floating point number. This is so you can pass a timeout in seconds or fractions. So, what is the return value? Well, join() always returns None. Hence, you will need to call is_alive() after calling join() to determine if a timeout happened. If we find out that it is indeed still alive, then we infer that the join() call timed out. However, if timeout is None, or if we did not pass it, this blocks the operation until the thread terminates. We can join() a thread many times. Finally, join() will raise a RuntimeError if we try to join the current thread, because that causes a deadlock. To join() a thread before we start it also causes an error. 1. >>> threading.Thread.join(threading.current_thread()) 2. Traceback (most recent call last): 3. File “”, line 1, in 4. threading.Thread.join(threading.current_thread()) RuntimeError: cannot join current thread d. name This is a string we use for identification; it has no meaning. We can also give the same meaning to multiple threads. The constructor sets the initial name. 1. >>> threading.Thread.name=’First’ 241

Basic Computer Coding: Python

2. 3. >>> e. getName() and setName() These are old getter and setter APIs for name. We use them directly as properties. f. ident If we started the thread, this returns its identifier. Otherwise, it returns None. Note that it is a non-zero integer, like in the get_ ident() function. Python may recycle identifiers when one thread exits and another creates. Such identifiers exist even after a thread exits. g. is_alive() This returns whether the thread is alive. is_ alive() returns true from just before run() starts until just after it terminates. 1. >>> threading.Thread.is_alive(threading. current_thread()) Remember

True h. daemon

Whenever a function wants to modify a variable, it locks that variable. When another function wants to use a variable, it must wait until that variable is unlocked.

daemon is a Boolean value that tells us whether the thread is a daemon. If it is, it returns True. We must set it before we call start(). Otherwise, it raises a RuntimeError. Its initial value comes from the creating thread. The main thread is not a daemon; hence, all threads in the main thread have a default of False for daemon. When only daemon threads remain, the whole program exits. i. isDaemon() and setDaemon()

242

Python Multithreading

These are old getter and setter APIs for daemon. You can use them directly as properties.

7.2.3 Lock Objects A synchronization primitive, a primitive lock does not belong to a certain thread when locked. This is the lowest-level synchronization primitive we currently have in Python, and we implement it using the extension module _thread. Such a lock can be in one of two states: ‘locked’ and ‘unlocked’. When we create a lock, it is in the ‘unlocked’ state. It also has two methods- acquire() and release(). When we want to lock it, acquire() changes its state to ‘locked’, and immediately returns it. If it was ‘locked’ instead, then acquire() blocks until another thread calls release(). This changes the state to ‘unlocked’. Finally, acquire() resets it to ‘locked’, and then returns immediately. If you try to release a lock that is already unlocked, it raises a RuntimeError. These locks also support the CMP(Context Management Protocol). When acquire() blocks more than one thread, only one thread continues when release() resets the state to ‘unlocked’. Which one, you ask? Well, we cannot say. Also, all methods execute atomically. This is the class: class threading.Lock This call implements primitive lock objects. Once a thread acquires a lock, the interpreter blocks further attempts to acquire it. Only after it releases, does any other thread have a chance in acquiring it. Any thread may release a lock. a. acquire(blocking=True, timeout=-1) This method acquires a blocking or non-blocking lock. When blocking=True, it blocks until the lock unlocks. Then, it changes its state to ‘locked’, and returns True. And when it is False, it does not block. A call with blocking=True that blocks, immediately returns False. Otherwise, it sets the lock to ‘locked’ and returns True. timeout is a floating-point argument. When it has a positive value, it 243

Basic Computer Coding: Python

blocks for a maximum of timeout number of seconds; as long as the lock is not acquirable. When it is -1, it denotes an unbounded wait. When blocking is False, we cannot specify timeout. Also, if the lock acquires successfully, it returns True; otherwise, False, like when timeout expires. b. release() This method releases a lock. You can call it from any thread. This means that any thread can release a lock, no matter which thread has acquired it. When ‘locked’, release() resets it to ‘unlocked’, and returns. If other threads wait for it to unlock, only one gets to continue once it unlocks. When we call release() on an ‘unlocked’ lock, it raises a RuntimeError. release() does not return any value.

7.2.4 RLock Objects RLock is very important topic when you learn Python Multithreading. An RLock is a reentrant lock. It is a synchronization primitive that a certain thread can acquire again and again. It does so using concepts like ‘owning thread’ and ‘recursion level’, and locked/ unlocked states. When locked, an RLock belongs to a certain thread; but when unlocked, no thread owns it. Now, how does this work? To lock, a thread calls acquire(). Now that this thread owns the lock, it returns. To unlock it, a thread calls release(). It is also possible to nest acquire()/release() pairs. The outermost release() resets the lock to the ‘unlocked’ state. It also lets another blocked thread to continue. Reentrant locks also support CMP(Context Management Protocol). This is the class: class threading.RLock RLock implements reentrant lock objects. Such a lock only release by the thread holding it. A thread can acquire it again without blocking. However, it must release it once each time it acquires it. It has two methods: a. acquire(blocking=True, timeout=-1) 244

Python Multithreading

acquire() lets us acquire a blocking or non-blocking lock. Without arguments, if the thread already owns the lock, this method ups the recursion level by one, and then returns. If it does not already own it, and another thread owns it, it blocks until the lock ‘unlocks’. And once unlocked, and if it does not belong to any other thread, acquire() declares ownership and sets recursion level to 1, and then returns. If more than one thread waits blocked, at once, only one will get ownership. This method returns no value. Finally, when we set blocking to True, it does the same things we discussed, and then returns True. When blocking is False, however, it doe not block. When a call without arguments blocks, it returns False. Otherwise, it does what it does for a call without arguments, and then returns True. And when we call acquire() with timeout, which is a floatingpoint number, with a positive value, this blocks for a maximum of timeout number of seconds, as long as we cannot acquire the lock. If a thread has acquired it, it returns True; if timeout has elapsed, it returns False. b. release() This method releases a lock and decrements the recursion level. Once the decrement is 0, it resets the lock to the ‘unlocked’ state. This means no thread owns it. If other threads are blocked, only one of them may continue. If the decrement is non-zero, the lock stays in the ‘locked’ state, and belongs to the calling thread. You should only call release() when the calling thread actually owns the lock. If it is already ‘unlocked’, this raises a RuntimeError. release() returns no value. Now lets come to Condition Objects in Python Multithreading.

7.2.5 Condition Objects A condition variable always pertains to a lock, and we can pass it in, or it creates by default. When several such condition variables must share a lock, we can pass it in. But we do not need to exclusively track a lock; it is a part of the condition object. A condition variable follows CMP (Context Management Protocol) in that it uses the 245

Basic Computer Coding: Python

with-statement to acquire the associated lock as long as the enclosed block is alive. acquire() and release() call the lock’s methods. For other methods, we must call them with the associated lock the thread holds. Once wait() releases the lock, it blocks until another thread wakes it up with a call to notify() or notify_all(). After this, wait() acquires the lock again, and then returns. We can also specify a timeout. While notify() awakens one waiting thread, if any, notify_all() awakens all threads waiting for the condition variable. Note that these two methods do not release the lock. So, the threads awakened do not return from wait() immediately. They return only when the calling thread for notify() or notify_all() gives up ownership for the lock. This is the class: class threading.Condition(lock=None) Condition implements condition variable objects. A condition variable lets any number of threads wait until another thread notifies them. If lock is not None, and we do pass it, make sure it is a Lock or RLock object. This should also serve as the underlying lock, otherwise this creates a new RLock object. It has the following methods: a. acquire(*args) This acquires the underlying lock. It calls the corresponding method on it, and returns what that method returns. b. release() This releases the underlying lock. It calls the corresponding method on it, and returns nothing. c. wait(timeout=None) This method waits until a timeout happens or until someone notifies it. If at the time of calling wait(), the calling thread does not own the lock, this raises a RuntimeError. wait() releases the underlying lock, then blocks until a notify()/ notify_all() call for the same condition variable in another thread wakes it up, or until timeout happens. And once this happens, it acquires the lock again, and then returns. 246

Python Multithreading

When we do pass timeout, and that is not None, make sure it is a floating point number denoting a timeout for the operation in seconds or fractions. If the underlying lock is an RLock, its release() method does not release it, because this does not necessarily unlock it if it was acquired multiple times recursively. So, what do we do? We use an internal interface of the RLock class. This unlocks it even when it was recursively acquired many times. Then, we use another internal interface to restore the recursion level when the thread acquires the lock again. wait() returns False if timeout expires. Otherwise, it returns True. d. wait_for(predicate, timeout=None) This method waits until a condition becomes True. The predicate is a callable with a Boolean result. We may provide a timeout to specify a maximum time to wait. wait_for() is a utility method, and it can repeatedly make a call to wait() until the predicate satisfies, or until a timeout happens. It returns the predicate’s last return value, and Recursion occurs when a thing is defined in returns False if the method times out. With this method, the same rules apply as do to wait(). When we call it, the lock must be held, and acquires again on return. This evaluates the predicate with the lock held. e. notify(n=1) notify() wakes up a thread waiting on this condition, if there is any. When we call it, if the calling thread does not own the lock, this raises a RuntimeError. It wakes up a maximum of n threads that wait for the condition variable. If no threads wait, then it is a no-operation(NOP). 247

terms of itself or of its type. Recursion is used in a variety of disciplines ranging from linguistics to logic.

Basic Computer Coding: Python

If at least n threads wait, this implementation will wake exactly n threads up. But we cannot rely on this behavior. An optimized implementation can occasionally wake more than n threads up. f. notify_all() This wakes up all threads that wait on this condition. So, this is like notify(), except that it wakes all waiting threads instead of exactly one. If at the time of calling it, if the calling thread does not own the lock, this raises a RuntimeError.

7.2.6 Semaphore Objects Early Dutch computer scientist Edsger W. Dijkstra invented one of the oldest synchronization primitives. Instead of acquire() and release(), he used P() and V(). What is a semaphore? It is a primitive that lets us manage an internal counter. Each call to acquire() decrements, and each call to release() decrements it. But let us tell you, the counter never goes below zero. When it is 0, acquire() blocks, and waits until a thread makes a call to release(). Semaphores in Python Multithreading support CMP(Context Management Protocol). This is the class we have: class threading.Semaphore(value=1) It implements semaphore objects. A semaphore holds an atomic counter denoting the count of release() calls minus the count of acquire() calls, added to an initial value.acquire() blocks if needed until it can leave the counter non-negative and still return. The default value for the counter is 1. This class implements semaphore objects. A semaphore manages an atomic counter representing the number of release() calls minus the number of acquire() calls, plus an initial value. The acquire() method blocks if necessary until it can return without making the counter negative. If not given, value defaults to 1. value can serve as an initial value for the internal counter. The default for this is 1. If we pass a value less than 0, this raises a ValueError. It has the following methods: 248

Python Multithreading

a. acquire(blocking=True, timeout=None) This acquires a semaphore. When we pass a timeout value other than None, it blocks for a maximum of timeout seconds. If in that interval, acquire() does not complete successfully, it returns False. Otherwise, it returns True. When we call it without arguments, the following cases may be: ■

If, on entry, the internal counter is greater than zero, it decrements it by one, and then returns.

■

If, on entry, the internal counter is zero, it blocks until a call to release() wakes it up. Now that the counter is greater than 0, it decrements it by 1, and then returns True. Each call to release() wakes exactly one thread. We cannot say what order this happens in.

■

When we call it with blocking with a value of False, it does not block. And if a call without arguments blocks, then it returns False. Otherwise, it does the same as when called without arguments, and then returns True.

b. release() This method releases a semaphore, and increments the internal counter by 1. When, on entry, it is 0, and another thread waits for it to grow again, it wakes that thread up. We also have bounded semaphores: class threading.BoundedSemaphore(value=1) This class implements bounded semaphore objects. Such objects ensure that their current values do not exceed their initial values. It this happens, this raises a ValueError. Mostly, semaphores guard resources with limited capacity, for ex., a database server. Where the resource size is fixed, use a bounded semaphore. But if it releases the semaphore way too many times, then you may have a bug in your code. The default for this is 1. Let’s take an example. The main thread initializes the semaphore before spawning any worker threads: 1. >>> maxconnections=5 2. >>> pool_sema=threading.Bounded maxconnections) 249

Semaphore

(value

=

Basic Computer Coding: Python

Now that it is spawned, the worker threads call acquire() and release() when they must connect to the server: 1. >>>with pool_sema: 2. conn=connectdb() 3. try: 4. #use connection 5. finally: 6. conn.close() Using a bounded semaphore lessens the chances of programming errors.

7.2.7 Event Objects An extremely simple tool in Python Multithreading to communicate, it lets one thread play an event, and the other must wait for it. An event object deals an internal flag. The methods set() and clear() allow us to set and reset it to True and False, respectively. Until flag is True, wait() blocks. This is the class: class threading.Event This class implements event objects. An event handles a flag, and we can use the methods set() and clear() to set and reset it to True and False, respectively. Initially, the flag is False. wait() blocks it until it becomes True. It has the following methods: a. is_set() If the internal flag is True, it returns True. b. set() This method sets the internal flag to True, and wakes all threads waiting for it to become True. Once it is True, waiting threads do not block at all. c. clear() 250

Python Multithreading

This resets the internal flag to False. Eventually, waiting threads block until somebody calls set() to set the internal flag to True yet again. d. wait(timeout=None) Until the internal flag is True, this method blocks. On entry, if it is True, it returns immediately. Otherwise, it blocks until another thread makes a call to set() to set the flag to True, or until timeout happens. When timeout exists, and is not Now, make sure it is a floating-point number denoting a timeout for the operation in seconds or fractions. It returns True only if the internal flag is True- either before the call to wait(), or after. This way, wait() always returns True. However, if timeout exists and the operation times out, it returns False.

7.2.8 Timer Objects Timer denotes an action that should run only after a given amount of time; it is a timer in Python Multithreading. This is a subclass of Thread, and we can also use it to learn how to create our own threads. When we call start() on a thread, a timer start with it. We can stop it before it begins, if we call cancel() on it. Before executing, a timer waits for some interval; this may differ from the interval we specify. Take an example: 1. >>> def hello(): 2. print(“Hello”) 3. >>> t=threading.Timer(30.0,hello) 4. >>> t.start() This is the class: class threading.Timer(interval, function, args=None, kwargs=None) It creates a timer that runs a function(with arguments args and kwargs) after intervalseconds. When args is None, it uses an empty list. This is the default. And when kwargs is None, it uses an empty dict. This is a default too. It has one method: 251

Basic Computer Coding: Python

a. cancel() This stops the timer, and then cancels its action. This only works if the timer is waiting. Synchronization refers to one of two distinct but related concepts: synchronization of processes, and synchronization of Data.

Now the last in Python Multithreading is Barrier Objects.

7.2.9 Barrier Objects Barrier is a simple synchronization primitive to a fixed number of threads that must wait for each other. Each thread tries to pass the barrier by making a call to wait; it blocks until all threads have done this. Then, the threads release simultaneously. You can reuse a barrier any number of times for the same number of threads. Let’s take an example. A way to synchronize a client and server thread is: 1. >>> b=threading.Barrier(2,timeout=5) 2. >>> def server(): 3. start_server() b.wait() while True: connection = accept_connection() process_server_connection(connection) 1. >>> def client(): b.wait() while True: connection = make_connection() 252

Python Multithreading

process_client_connection(connection) This is the class: class threading.Barrier(parties, action=None, timeout=None) Barrier creates a barrier object for parties number of threads. When we do pass action, it is a callable that a thread will call when we release it. Finally, timeout is a default value for timeout if we do not specify the same for wait(). It has the following methods: a. wait(timeout=None) wait() passes the barrier. Once all thread parties have called wait(), they all release together. If we do pass a value for timeout, it uses this one, no matter whether we provided a value for the same to the class constructor. It returns any integer value from 0 to parties-1. This is different for each thread. You can use this to choose a thread to do special housekeeping. Take an example: 1. >>> i=barrier.wait() 2. >>> if i==0: 3. #Only one thread must print this 4. print(“passed the barrier”) If we provided an action to the constructor, one thread calls it before releasing. If this raises an error, the barrier sinks into a ‘broken’ state. The same happens if the call times out. Finally, wait() may raise a BrokenBarrierError if the barrier breaks or resets as a thread waits. b. reset() This function resets the barrier to its default, empty state. Any waiting threads receive a BrokenBarrierError. reset() may need external synchronization if other threads with unknown states exist. If it breaks a barrier, just create a new one. c. abort() 253

Basic Computer Coding: Python

abort() puts a barrier into a ‘broken’ state. Consequently, active/future calls to wait() fail with a BrokenBarrierError. To avoid deadlocking an application, we may need to abort it. This is one use-case. Try to create the barrier with a sensible value for timeout so it automatically guards against a thread going haywire. d. parties This returns the number of threads we need to pass the barrier. e. n_waiting This returnd the number of threads that currently wait in the barrier. f. broken This is a Boolean value that is True if the barrier is in a ‘broken’ state. Check out this exception that Barrier may raise: exception threading.BrokenBarrierError This is a subclass of RuntimeError, and it raises if a Barrier object resets or breaks.

7.2.10 Using locks, Conditions, and Semaphores in the with-statement If an object in this module has acquire() and release(), we can use it as a context-manager for a with-statement. When it enters the block, it calls acquire(), and when it exits it, it calls release(). This is the syntax for the same: with some_lock: #do something This is the same as: 1. >>> some_lock.acquire() 2. >>> try: 3. #do something 4. finally: 5. some_lock.release() 254

Python Multithreading

We can currently use Lock, RLock, Condition, Semaphore, and BoundedSemaphore objects as context-managers for with-statements.

255

Basic Computer Coding: Python

ROLE MODEL Edsger Dijkstra: Dutch Computer Scientist Edsger Dijkstra, in full Edsger Wybe Dijkstra, (born May 11, 1930, Rotterdam, Neth.—died Aug. 6, 2002, Nuenen), Dutch computer scientist. He received a Ph.D. from the University of Amsterdam while working at Amsterdam’s Mathematical Center (1952–62). He taught at the Technical University of Eindhoven from 1963 to 1973 and at the University of Texas from 1984. He was widely known for his 1959 solution to the shortest-path problem; his algorithm is still used to determine the fastest way between two points, as in the routing of communication networks and in flight planning. His research on the idea of mutual exclusion in communications led him to suggest in 1968 the concept of computer semaphores, which are used in virtually every modern operating system. A letter he wrote in 1968 was extremely influential in the development of structured programming. He received the Turing Award in 1972.

256

Python Multithreading

SUMMARY ■

Multithreading is a core concept of software programming that almost all the high-level programming languages support. And Python multithreading is one of the best examples of simplistic implementation of threads.

■

In software programming, a thread is the smallest unit of execution with the independent set of instructions. It is a part of the process and operates in the same context sharing program’s runnable resources like memory.

■

A thread has a starting point, an execution sequence, and a result. It has an instruction pointer that holds the current state of the thread and controls what executes next in what order.

■

A thread has a beginning, an execution sequence, and a conclusion. It has an instruction pointer that keeps track of where within its context it is currently running.

■

Threading is one of the most well-known approaches to attaining Python concurrency and parallelism. Threading is a feature usually provided by the operating system. Threads are lighter than processes, and share the same memory space.

■

Multiprocessing and Multithreading both adds performance to the system. Multiprocessing is adding more number of or CPUs/processors to the system which increases the computing speed of the system.

■

In Java, locks and condition variables are the basic behavior of every object. Whereas in Python, they are individual objects. Here, the class Thread supports some of the functionality of class Thread in Java.

257

Basic Computer Coding: Python

KNOWLEDGE CHECK 1.

2.

3.

4.

5.

A thread can be created by using ……………….. class. a.

MultiThread

b.

Thread

c.

Threading

d.

SuperThread

Which Java feature enables to handle multiple tasks simultaneously? a.

Class and Object

b.

Platform Independent

c.

Dynamic Object Initialization

d.

Multi Threading

Which method is used to schedule a thread for execution? a.

start()

b.

init()

c.

run()

d.

resume()

What is called when a function is defined inside a class? a.

Module

b.

Class

c.

Another Function

d.

Method

Which of the following is the use of id() function in python? a.

Id returns the identity of the object

b.

Every object does not have a unique id

c.

All of the mentioned

d.

None of the mentioned

258

Python Multithreading

REVIEW QUESTIONS 1.

What do you understand by multithreading?

2.

How to use the thread module to create threads.

3.

How to use the threading module to create threads.

4.

Differentiate between multithreading.

5.

Explain the functions in python multithreading.

6.

Write the difference between Lock objects and RLock objects.

multiprocessing

Check Your Result 1. (b)

2. (d)

3. (d)

259

4. (d)

5. (a)

and

Basic Computer Coding: Python

REFERENCES 1.

http://www.linuxprogrammingblog.com/threads-andfork-think-twice-before-using-them

2.

h t t p : / / w w w. t u t o r i a l s p o i n t . c o m / p y t h o n / p y t h o n _ multithreading.htm

3.

https://data-flair.training/blogs/python-multithreading/

4.

https://docs.python.org/3/library/threading.html

5.

https://hackernoon.com/synchronization-primitives-inpython-564f89fee732

6.

https://medium.com/@nbosco/multithreading-vsmultiprocessing-in-python-c7dc88b50b5b

7.

https://pymotw.com/2/threading/

8.

https://pythonprogramming.net/threading-tutorialpython/

9.

h t t p s : / / t e c h d i f f e r e n c e s . c o m / d i f f e r e n c e - b e t we e n multiprocessing-and-multithreading.html

10.

https://timber.io/blog/multiprocessing-vs-multithreadingin-python-what-you-need-to-know/

11.

https://www.bogotobogo.com/python/Multithread/ python_multithreading_creating_threads.php

12.

https://www.geeksforgeeks.org/multithreading-pythonset-1/

13.

https://www.geeksforgeeks.org/operating-systemdifference-multitasking-multithreading-multiprocessing/

14.

https://www.journaldev.com/17290/python-threadingmultithreading

15.

https://www.quantstart.com/articles/Parallelising-Pythonwith-Threading-and-Multiprocessing

16.

https://www.remwebdevelopment.com/blog/overview-offorks-threads-and-asynchronous-io-133.html

17.

https://www.shanelynn.ie/using-python-threading-formultiple-results-queue/ 260

Python Multithreading

18.

https://www.techbeamers.com/python-multithreadingconcepts/

19.

https://www.toptal.com/python/beginners-guide-toconcurrency-and-parallelism-in-python

20.

Rauschmayer, Axel. “Chapter 3: The Nature of JavaScript; Influences”. O’Reilly, Speaking JavaScript. Retrieved 15 May 2015.

21.

Smith, Kevin D.; Jewett, Jim J.; Montanaro, Skip; Baxter, Anthony (2 September 2004). “PEP 318 – Decorators for Functions and Methods”. Python Enhancement Proposals. Python Software Foundation. Retrieved 24 February 2012.

22.

http://www.buyya.com/java/Chapter14.pdf

23.

http://softwareramblings.com/2008/06/running-functionsas-threads-in-python.html

24.

Holth, Moore (30 March 2014). “PEP 0441 -- Improving Python ZIP Application Support”. Retrieved 12 November 2015.

25.

Bini, Ola (2007). Practical JRuby on Rails Web 2.0 Projects: bringing Ruby on Rails to the Java platform. Berkeley: APress. p. 3. ISBN 978-1-59059-881-8.

261

INDEX A aliasing 86, 108 Anonymous functions 51 atomic grouping 200, 225 B Backreferences 214 Blueprint 154 C Character classes 211 characters 200, 201, 206, 207, 208, 210, 217, 225, 226 CMP(Context Management Protocol) 243, 244, 248 Computer desktop 152

Computer Science 231 Condition object 245 D data structures 77, 78, 97, 108 data type 2, 19, 23, 24, 26, 40 Default arguments 47 E Encapsulation 173, 176, 194 Execution 230, 232, 238, 257, 258 F File objects 74 Files 74, 97, 100, 108

Basic Computer Coding: Python

G General Public License (GPL) 3 Graphical User Interface (GUI) 10 H HTTP (Hypertext Transfer Protocol) 232 I Inheritance 161, 169, 194 Instance attribute 156, 158, 168 Instance method 159, 194 intersection operation 96 J Java 1, 4, 5, 11, 36 K keys 74, 75, 76, 77, 78, 85, 99, 108, 109, 110 Keyword arguments 47, 48 L Library Reference 59 Literal characters 211 M Mac OS 2, 5, 7, 8 Mappings 74, 108 Match Function 202 Memory space 257 Microsoft Windows 98 2 MS-DOS 2 Multiprocessing 233, 234, 235, 257, 260 mutable data structure 78, 108 N Nongreedy repetition 213

No-operation(NOP) 247 O Object-oriented programming (OOP) 152, 194 Object-Oriented style 3 open function 74, 98 Operating system 256, 257 optional flags 202, 204, 225 P Parallel programming 231 Parent class 155, 161, 164, 166, 167, 168, 169, 170, 171, 194 Parentheses 155, 157, 187 pattern 200, 201, 202, 203, 204, 206, 207, 210, 220, 222, 223, 225 PERL 2 Perl-like regular expressions 200 possessive quantifiers 200, 225 programming languages 97, 100 Programming paradigm 153, 194 Python 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 27, 28, 29, 30, 31, 32, 35, 36, 37, 38, 39, 40, 41, 42, 73, 74, 75, 85, 88, 90, 91, 93, 97, 100, 102, 108, 199, 200, 201, 205, 207, 208, 211, 214, 215, 217, 218, 219, 220, 221, 223, 224, 225, 228 Python code 44, 59 Python dictionaries 74, 108 Python directory 8 Python implementation 233 Python interpreter 45, 47, 48, 54, 58, 59 Python magic 159 Python’s module 62, 68 264

Index

Python’s regex syntax 200, 225 Python threading 231 R Raw Strings 200 Recursion 244, 245, 247 re.error 200 regex 199, 200, 201, 202, 225 Regular expressions 200, 206, 225 re.match() 201, 202 re.search() 201, 202 Return statement 45, 46, 47, 53 re.UNICODE 201

string 78, 85, 88, 89, 97, 98, 99, 100, 101, 103, 108, 110, 200, 201, 202, 203, 204, 205, 206, 208, 210, 214, 215, 217, 221, 225 Synchronization 243, 244, 248, 252, 253, 260 syntax 74, 79, 82, 83, 97, 100 T Thread 233, 235, 236, 238, 239, 240, 241, 242, 251, 257, 258 TypeError 166, 167

S Search Function 204 Self variable 155, 156 Sets 73, 74, 87, 88, 92, 95, 96, 108 Software programming 230, 257

U Unicode properties 200, 225 UNIX 200, 225 UNIX-based systems 2 V Variables 8, 19, 38

265