메인 Hands-On Penetration Testing with Python

Hands-On Penetration Testing with Python

0 / 0
How much do you like this book?
What’s the quality of the file?
Download the book for quality assessment
What’s the quality of the downloaded files?
Enhance your ethical hacking skills to build automated and intelligent systems

About this book

With the current technological and infrastructural shift, penetration testing is no longer a process-oriented activity. Modern-day penetration testing demands lots of automation and innovation; the only language that dominates all its peers is Python. Given the huge number of tools written in Python, and its popularity in the penetration testing space, this language has always been the first choice for penetration testers.

Hands-On Penetration Testing with Python walks you through advanced Python programming constructs. Once you are familiar with the core concepts, you’ll explore the advanced uses of Python in the domain of penetration testing and optimization. You’ll then move on to understanding how Python, data science, and the cybersecurity ecosystem communicate with one another. In the concluding chapters, you’ll study exploit development, reverse engineering, and cybersecurity use cases that can be automated with Python.

By the end of this book, you’ll have acquired adequate skills to leverage Python as a helpful tool to pentest and secure infrastructure, while also creating your own custom exploits.
Packt Publishing
ISBN 13:
PDF, 107.20 MB
다운로드 (pdf, 107.20 MB)

귀하의 관심을 살 수 있습니다 Powered by Rec2Me


주로 사용되는 용어

1 comment
امیر محمد جلیلی
107 MB for a Book? I'm not downloading it, it seems dubious
18 June 2021 (14:05) 

책에 대한 리뷰를 작성하거나 귀하의 독서 경험을 공유할 수 있습니다. 다른 독자들이 귀하가 읽은 책에 대한 의견에 귀를 기울일 것입니다. 개인적으로 책이 마음에 들었거나 그렇지 않았거나 정직하고 상세한 호평은 다른 독자들이 자신에게 적합한 책을 찾는데 도움이 됩니다.
Hands-On Penetration Testing
with Python

Enhance your ethical hacking skills to build automated and
intelligent systems

Furqan Khan


Hands-On Penetration Testing with Python
Copyright © 2019 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form
or by any means, without the prior written permission of the publisher, except in the case of brief quotations
embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented.
However, the information contained in this book is sold without warranty, either express or implied. Neither the
author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to
have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products
mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy
of this information.
Commissioning Editor: Vijin Boricha
Acquisition Editor: Shrilekha Inani
Content Development Editor: Nithin George Varghese
Technical Editor: Mohit Hassija
Copy Editor: Safis Editing
Language Support Editor: Mary McGowan
Project Coordinator: Drashti Panchal
Proofreader: Safis Editing
Indexer: Pratik Shirodkar
Graphics: Tom Scaria
Production Coordinator: Jisha Chirayil
First published: January 2019
Production reference: 1310119
Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
B3 2PB, UK.
ISBN 978-1-78899-082-0


This book is dedicated to my lovely parents!


Mapt is an online digital library that gives you full access to over 5,000 books and videos, as
well as industry leading tools to help you plan your personal development and advance
your career. For more information, please visit our website.

Why subscribe?
Spend less time learnin; g and more time coding with practical eBooks and Videos
from over 4,000 industry professionals
Improve your learning with Skill Plans built especially for you
Get a free eBook or video every month
Mapt is fully searchable
Copy and paste, print, and bookmark content

Did you know that Packt offers eBook versions of every book published, with PDF and
ePub files available? You can upgrade to the eBook version at www.packt.com and as a print
book customer, you are entitled to a discount on the eBook copy. Get in touch with us at
customercare@packtpub.com for more details.
At www.packt.com, you can also read a collection of free technical articles, sign up for a
range of free newsletters, and receive exclusive discounts and offers on Packt books and

About the author

Furqan Khan is a security researcher who loves to innovate in Python, pentesting, ML, AI,
and big data ecosystems.
With a gold medal at both M.Tech and B.Tech, he started off as a research scientist at NITK,
where he developed a web app scanner for the Ministry of IT (India). He then worked as a
security researcher with Paladion Networks and Wipro
Dubai exploring pentesting/exploitation space where he developed tools such as
vulnerability scanner and a threat intelligence platform.
Currently, he is working with Du-Telecom Dubai as a pentesting manager. He has
published and co-authored white papers and journals with Springer and Elsevier, and has
also presented his research and development work at international conferences, including

There is a well-known saying that goes like this: "You are the average of the five
people you spend the most time with." I have the following improvisation to it:
"Allmighty Allah has created you uniquely with the ability to be yourself and yet
acquire wisdom from all the brilliant people around you."
This book would never have been possible without the love and support of my
amazing father, Mr Shakeel Ahmed Khan, my beautiful mother, Mrs Night Khan, my
fantastic brother, Burhan Khan, and my wonderful partner in crime, Zakiya Khan.
I would also like to extend my gratitude to those people who have been the catalyst
for whatever little I have been able to achieve: Mr Walid Kamal, Mrs Santhi Thilagam,
Mr Sayed Wajahat Ali, Mr Gobind Basmall, Mr Siddharth Anbalahan, Mrs Mehmooda
Jan, and Mr Hassan Magray. I am grateful to all of you for believing in me.

About the reviewer
Phil Bramwell acquired the Certified Ethical Hacker and Certified Expert Penetration
Tester certifications at the age of 21. His professional experience includes common criteria
design reviews and testing, network security consulting, penetration testing, and PCI-DSS
compliance auditing for banks, universities, and governments. He later acquired the CISSP
and Metasploit Pro Certified Specialist credentials. Today, he is a cyber security and
cryptocurrency consultant and works in the automotive industry, specializing in malware
detection and analysis.

Packt is searching for authors like you
If you're interested in becoming an author for Packt, please visit authors.packtpub.com
and apply today. We have worked with thousands of developers and tech professionals,
just like you, to help them share their insight with the global tech community. You can
make a general application, apply for a specific hot topic that we are recruiting an author
for, or submit your own idea.

Table of Contents


Chapter 1: Introduction to Python
Technical requirements
Why Python?

About Python – compiled or interpreted
Installing Python
Getting started
Variables and keywords
Variable naming conventions
Python keywords
Python data types
String types
String indexes
String operations through methods and built-in functions
The replace( ) method
Substrings or string slicing
String concatenation and replication
The strip(), lstrip(), and rstrip() methods
The split() method
The find(), index(), upper(), lower(), len(), and count() methods
The in and not in methods
The endswith(), isdigit(), isalpha(), islower(), isupper(), and capitalize() 30
List types
Slicing the lists
Merging and updating lists
Copying lists
Removing elements from lists
Replication with len(), max(), and min()
in and not in
Tuples in Python
Dictionaries in Python
Python operators

Chapter 2: Building Python Scripts
Technical requirements
Conditional statements


Table of Contents

The if condition
The if...else condition
The if...elif condition


The while loop
The for loop

Iteration, iterable, and iterator
A closer look at for loops

Functions and methods in Python
Modules and packages
Generators and comprehensions

Map, Lambda, zip, and filters

Further reading
Chapter 3: Concept Handling
Object-oriented programming in Python
Classes and objects
Class relationships

Access modifiers in Python

Abstract classes

Polymorphism with functions
Polymorphism with classes (abstract classes)

Static, instance, and class methods in Python

Files, directories, and I/O access

File access and manipulation
Renaming and deleting files and accessing directories
Console I/O

Regular expressions in Python
Data manipulation and parsing with XML, JSON, and CSV data
XML data manipulation
JSON data manipulation

Exception handling
Further reading
Chapter 4: Advanced Python Modules
[ ii ]



Table of Contents

Multitasking with threads

Demonic and non-demonic threads
Thread joins and enumeration
Intercommunication between threads
Thread concurrency control

Multitasking with processes

Demonic and non-demonic processes
Process joins, enumeration, and termination
Multiprocess pooling

Socket programming basics
Reverse TCP shells with Python
Further reading
Chapter 5: Vulnerability Scanner Python - Part 1
Introducing Nmap
Building a network scanner with Python


Controlling the Nmap output with the script
Using the Nmap module to conduct Nmap port scanning
Objective and architectural overview
Port scanning
Service scanning
A closer look at the code
Getting started
Executing the code
Database schema for the port scanning portion of the vulnerability scanner
Further reading

Chapter 6: Vulnerability Scanner Python - Part 2
Architectural overview
A closer look at the code




[ iii ]


Table of Contents

Executing the code
Database schema for the service-scanning portion of the vulnerability 206
GUI version of vulnerability scanner

Usage [PTO-GUI]

Scanning modules

Sequential mode
Reconfiguration after discovery is finished
Concurrent mode
Sequential default mode

Pausing and resuming scans

Downloading reports or analyzing when scan would be completed

Further reading
Chapter 7: Machine Learning and Cybersecurity
Machine Learning

Setting up a Machine Learning environment in Kali Linux

Regression-based machine learning models
Simple linear regression

How does the regression model work?

Multiple linear regression

Classification models

Naive Bayes classifier

Summarizing the Naive Bayes classifier
Implementation code

Natural language processing

Using natural language processing with penetration testing
Step 1– tagging the raw data
Step 2–writing the code to train and test our model

Further reading
Chapter 8: Automating Web Application Scanning - Part 1
Automating web application scanning with Burp Suite
Burp automation with Python
SQL injection
Automatic detection of SQL injection with Python

Further reading
Chapter 9: Automated Web Application Scanning - Part 2
[ iv ]



Table of Contents

Stored or Type 1 XSS attacks
Reflected or Type 2 XSS attacks
DOM-based or Type 0 XSS attacks
Automatic detection of XSS with Python


Script in action

Automatically detecting CSRF with Python
Script in action


Automatically detecting clickjacking with Python

SSL stripping (missing HSTS header)

Automatically detecting missing HSTS with Python

Further reading
Chapter 10: Building a Custom Crawler
Setup and installations
Getting started
Crawler code
Urls.py and Views.py code snippet
Code explanation

Driver code – run_crawler.py
Crawler code – crawler.py

Execution of code
Further reading
Chapter 11: Reverse Engineering Linux Applications
Fuzzing Linux applications
Fuzzing in action
Linux and assembly code

Stack buffer overflow in Linux

Exploiting a buffer overflow
Heap buffer overflow in Linux

String format vulnerabilities
Further reading
Chapter 12: Reverse Engineering Windows Applications
Fuzzing Windows applications


Table of Contents

Windows and assembly

Exploiting buffer overflows in Windows
Further reading
Chapter 13: Exploit Development
Scripting exploits over web-based vulnerabilities
Manually executing an LFI exploit
Reverse shell with Netcat
Reverse shell with Python

Exploit development (LFI + RFI)
LFI/RFI exploit code
Executing the LFI exploit
Executing the RFI exploit

Developing a Metasploit module to exploit a network service
Encoding shell codes to avoid detection
Downloading and installing Veil

Further reading
Chapter 14: Cyber Threat Intelligence
Introduction to cyber threat intelligence
Manual threat intelligence
Automated threat intelligence

Cyber threat intelligence platforms
Tools and API

Installing MISP
Threat scoring capability

Threat scoring

Threat scoring weighed file
Threat scoring algorithm
Executing the code

STIX and TAXII and external lookups
External lookups

Further reading
Chapter 15: Other Wonders of Python
Report parsers
Nmap parser

[ vi ]


Table of Contents

Running the code

Nessus parser

Running the code

The need to have custom parsers

Keylogger and exfiltration via sockets
pyxhook – a Linux based Keylogger
pyhook – a Windows-based keylogger

Parsing Twitter tweets
Stealing browser passwords with Python
Python for antivirus-free persistence shells
Further reading
Other Books You May Enjoy



[ vii ]

With so many amazing books out there in the cyber security and Python programming
space, written by brilliant people, what does this book have to offer that's different? It's a
very valid question, so now let's try to answer this.
This book makes a humble attempt to capture the practical and hands-on experience I have
acquired working with Python and the penetration testing space over the past few years. It
is a unique amalgamation of Python, penetration testing/offensive security, defensive
security, and machine learning use cases in the pentesting ecosystem. The book starts off
gently, covering all the key concepts of Python, enabling the reader to acquire a very decent
grasp of Python by the end of the first four chapters, before then clicking into gear and
delving into the hard core automation of penetration testing and cyber security use cases.
Readers will find out how to develop industry standard vulnerability scanners from
scratch, identical to Nessus and Qualys. The book then explores concepts concerning web
application vulnerabilities, their exploitation, and automating web exploitation with
custom tailored exploits. It also affords very deep insights into reverse engineering,
fuzzing, and buffer overflow vulnerabilities in both Windows and Linux environments,
utilizing Python as a centerpiece. There is a section dedicated to custom exploit
development, with a focus on evading anti-virus detection. The book also has a chapter
dedicated to developing a web crawler and its utilization in the cyber security space. The
book also gives decent insights on defensive security concepts, talking about cyber threat
intelligence, and how a custom threat scoring algorithm can be developed. The book
concludes with many other beneficial use cases of Python, such as developing a custom

Who this book is for
If you are a security consultant, developer, or a cyber security enthusiast with little or no
knowledge of Python, and require in-depth insights into how the pentesting ecosystem and
Python combine to create offensive tools, exploits, automate cyber security use cases, and
much more besides, then this book is for you. Hands-On Penetration Testing with Python
guides you through the advanced uses of Python for cyber security and pentesting, helping
you to better understand security loopholes within your infrastructure.


What this book covers
Chapter 1, Introduction to Python, covers the basics of Python, focusing primarily on data

types, variables, expressions, and program structures utilized by Python. The objective is to
familiarize the reader with the basics of the Python programming language with a view to
using and leveraging it in forthcoming chapters.
Chapter 2, Building Python Scripts, covers further concepts of Python that form the basis for

writing a Python script, while also exploring concepts such as functions, modules, loops,
packages, and imports.
Chapter 3, Concept Handling, introduces the reader to other Python-related

concepts, including classes, objects, IO and directory access, regular expressions, exception
handling, and the parsing of CSV, JSON, and XML files.
Chapter 4, Advanced Python Modules, takes the learning process to an advanced level and

explores the power of Python to understand multiprocessing and multithreading concepts,
along with socket programming.
Chapter 5, Vulnerability Scanner Python - Part 1, explores the advanced concepts that are

required in order to make a mini vulnerability scanning engine, which would take the port
scanning results from a custom port scanner build over Nmap and apply various open
source scripts and Metasploit modules, along with Python, Ruby, and NSE scripts. The
results would be aggregated and finally, a report would be drafted for analysts to analyze.
This chapter is very big in terms of complexity and lines of code and is split into two pars.
This part focuses on the automation of port scanning with Python.
Chapter 6, Vulnerability Scanner Python - Part 2, explores the advanced concepts that are

required in order to make a mini vulnerability scanning engine. This chapter is a
continuation of the previous chapter, with the reader learning how to orchestrate a variety
of Kali Linux tools to work together in order to automate the service enumeration phase of
the vulnerability assessment, thereby completing the development of the custom
vulnerability scanner.
Chapter 7, Machine Learning and Cybersecurity, tries to connect the cyber security space with

data science and shed some light as to how we can use machine learning and natural
language processing to automate the manual report analysis phase of penetration testing.
This chapter will also glue all the previous parts together to make a mini penetration testing
toolkit based on the learning we have acquired hitherto.
Chapter 8, Automating Web Application Scanning - Part 1, explains to readers how they can

use Python to automate various web application attack categories, with some of the most
well known being SQL injection, XSS, CSRF, and clickjacking.



Chapter 9, Automated Web Application Scanning - Part 2, is a continuation of the previous

chapter. Here, the reader is going to understand how they can use Python to develop
custom exploits that will exploit the web application and end up giving shell access to a
user using Python.
Chapter 10, Building a Custom Crawler, explains how to build a custom crawler coded in

Python in order to crawl through the application, with and without authentication, while
listing out injection points and web pages of the application under testing. The capabilities
of the crawler can be extended and tailored as per requirements.
Chapter 11, Reverse Engineering Linux Applications and Buffer Overflows , explains how to

perform reverse engineering with Linux applications. The reader is also going to
understand how Python can be used in aiding buffer overflow vulnerabilities in the Linux
environment. The chapter also guides readers on custom exploit development, targeting
buffer overflow vulnerabilities.
Chapter 12, Reverse Engineering Windows Applications, explains how to perform reverse

engineering with Windows applications and how Python can be used in aiding buffer
overflow vulnerabilities in the Windows environment. The chapter also guides readers on
custom exploit development, targeting buffer overflow vulnerabilities.
Chapter 13, Exploit Development, explains how readers can create their own exploits written

in Python, exploits that can be extended as Metasploit modules, and also covers the
encoding of shells to avoid detection.
Chapter 14, Cyber Threat Intelligence, guides readers on how to use Python for cyber threat

intelligence and the gathering of threat feeds, threat scoring, and finally, how to utilize the
information obtained such that the SIEMs, IPS, and IDS systems are powered by the latest
threat information to aid in early detection.
Chapter 15, Other Wonders of Python, covers how to use Python for extracting Google

browser-saved passwords, developing a custom keylogger, parsing Nessus and Nmap
report files, and more.

To get the most out of this book
To get the most out of this book, all that's required is a desire to keep going and understand
every concept in detail before proceeding further.



Download the example code files
You can download the example code files for this book from your account at
www.packt.com. If you purchased this book elsewhere, you can visit
www.packt.com/support and register to have the files emailed directly to you.
You can download the code files by following these steps:
1. in or register at www.packt.com.
the SUPPORT tab.
3. on Code Downloads & Errata.
Enter the
4. name of the book in the Search box and follow the onscreen
Once the file is downloaded, please make sure that you unzip or extract the folder using the
latest version of:
WinRAR/7-Zip for Windows
Zipeg/iZip/UnRarX for Mac
7-Zip/PeaZip for Linux
The code bundle for the book is also hosted on GitHub
at https://github.com/PacktPublishing/Hands-On-Penetration-Testing-withPython. In case there's an update to the code, it will be updated on the existing GitHub
We also have other code bundles from our rich catalog of books and videos available
at https:/ / github. com/ PacktPublishing/ . Check them out!

Download the color images
We also provide a PDF file that has color images of the screenshots/diagrams used in this
book. You can download it here: http:/ / www. packtpub. com/ sites/ default/ files/
downloads/ 9781788990820_ ColorImages. pdf.



Conventions used
There are a number of text conventions used throughout this book.
CodeInText: Indicates code words in text, database table names, folder names, filenames,
file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an
example: "To use the Python Terminal, simply type the python3 command in your
Terminal prompt."

A block of code is set as follows:
if a > b:
print("a is greater")

When we wish to draw your attention to a particular part of a code block, the relevant lines
or items are set in bold:
my_list[0] -> 1
my_List[1] -> "a"
my_list[2] -> [1,2,3]
my_list[2][0] -> 1
my_list[2][2] -> 3
my_list[3] -> {"k1":"v1"}
my_list[3]["k1"] -> "v1"
my_list[3].get("k1") -> "v1

Any command-line input or output is written as follows:
import threading
>>> class a(threading.Thread):
... def __init__(self):
... threading.Thread.__init__(self)
... def run(self):
... print("Thread started")

Bold: Indicates a new term, an important word, or words that you see on screen. For
example, words in menus or dialog boxes appear in the text like this. Here is an example:
"Click on the Start Crawling button."



Warnings or important notes appear like this.

Tips and tricks appear like this.

Get in touch
Feedback from our readers is always welcome.
General feedback: If you have questions about any aspect of this book, mention the book
title in the subject of your message and email us at customercare@packtpub.com.
Errata: Although we have taken every care to ensure the accuracy of our content, mistakes
do happen. If you have found a mistake in this book, we would be grateful if you would
report this to us. Please visit www.packt.com/submit-errata, selecting your book, clicking
on the Errata Submission Form link, and entering the details.
Piracy: If you come across any illegal copies of our works in any form on the internet, we
would be grateful if you would provide us with the location address or website name.
Please contact us at copyright@packt.com with a link to the material.
If you are interested in becoming an author: If there is a topic that you have expertise in,
and you are interested in either writing or contributing to a book, please visit

Please leave a review. Once you have read and used this book, why not leave a review on
the site that you purchased it from? Potential readers can then see and use your unbiased
opinion to make purchase decisions, we at Packt can understand what you think about our
products, and our authors can see your feedback on their book. Thank you!
For more information about Packt, please visit packt.com.



The information within this book is intended to be used only in an ethical manner. Do not
use any information from the book if you do not have written permission from the owner
of the equipment. If you perform illegal actions, you are likely to be arrested and
prosecuted to the full extent of the law. Packt Publishing does not take any responsibility if
you misuse any of the information contained within the book. The information herein must
only be used while testing environments with proper written authorizations from
appropriate persons responsible.


Introduction to Python
This chapter will provide an introduction to Python, focusing primarily on data types,
variables, expressions, and program structures that the Python programming language
follows. The objective of this chapter is to familiarize the reader with the basics of Python so
that they can use it in the upcoming chapters. The chapter will cover the installation of
Python and its dependency manager. We will also start taking a look at scripting in Python.
In this chapter, we will cover the following topics:
An introduction to Python (including its installation and setup)
Basic data types
Sequence data types – lists, dictionaries, tuples
Variables and keywords
Operations and expressions

Technical requirements
Make sure you have the following setup ready before proceeding with this chapter:
A working computer or laptop
An Ubuntu operating system, preferably version 16.04
Python 3.x
A working internet connection

Introduction to Python

Chapter 1

Why Python?
When we think about exploring a new programming language or technology, we often
wonder about the scope of the new technology and how it might benefit us. Let's start this
chapter by thinking about why we might want to use Python and what advantages it might
give us.
To answer this question, we are going to think about current technology trends and not get
into more language-specific features, such as the fact that it is object-oriented, functional,
portable, and interpreted. We have heard these terms before. Let's try to think about why
we might use Python from a strictly industrial standpoint, what the present and future
landscapes of this language might look like, and how the language can serve us. We'll start
by mentioning a few career options that someone involved in computer science might opt
Programmer or software developer
Web developer
Database engineer
Cyber security professional (penetration tester, incident responder, SOC analyst,
malware analyst, security researcher, and so on)
Data scientist
Network engineer
There are many other roles as well, but we'll just focus on the most generic options for the
time being to see how Python fits into them. Let's start off with the role of programmer or
software developer. As of 2018, Python was recorded as the second most popular language
listed in job adverts (https:/ / www. codingdojo. com/ blog/ 7- most- in- demandprogramming- languages- of- 2018/ ). The role of programmer might vary from company to
company, but as a Python programmer, you might be making a software product written in
Python, developing a cyber security tool written in Python (there are tons of these already
in existence that can be found on GitHub and elsewhere in the cyber security community),
prototyping a robot that can mimic humans, engineering a smart home automation product
or utility, and so on. The scope of Python covers every dimension of software development,
from typical software applications to robust hardware products. The reason for this is the
ease of the language to understand, the power of the language in terms of its excellent
library support, which is backed by a huge community, and, of course, the beauty of it
being open source.


Introduction to Python

Chapter 1

Let's move on to the web. In recent years, Python has done remarkably well in terms of its
maturity as a web development language. The most popular full stack web-based
frameworks such as Django, Flask, and CherryPy have made web development with
Python a seamless and clean experience, with lots of learning, customization, and flexibility
on the way. My personal favorite is Django, as it provides a very clean MVC architecture,
where business, logic, and presentation layers are completely isolated, making the
development code much cleaner and easier to manage. With all batteries loaded and
support for ORM and out-the-box support for background task processing with celery,
Django does everything that any other web framework would be capable of doing, while
keeping the native code in Python. Flask and CherryPy are also excellent choices for web
development and come with lots of control over the data flow and customization.
Cyber security is a field that would be incomplete without Python. Every industry within
the cyber security domain is related to Python in one way or another and the majority of
cyber security tools are written in Python. From penetration testing to monitoring security
operations centers, Python is widely used and needed. Python aids penetration testers by
providing them with excellent tools and automation support with which they can write
quick and powerful scripts for a variety of penetration testing activities, from
reconnaissance to exploitation. We will learn about this in great detail throughout the
course of this book.
Machine learning (ML) and artificial intelligence (AI) are buzz words in the tech industry
that we come across frequently nowadays. Python has excellent support for all ML and AI
models. Python, by default in most cases, is the first choice for anyone who wants to learn
ML and AI. The other famous language in this domain is R, but because of Python's
excellent coverage across all the other technology and software development stacks, it is
easier to combine machine learning solutions written in Python with existing or new
products than it is to combine solutions written in R. Python has got amazing machine
learning libraries and APIs such as sciket-learn, NumPy, Pandas, matplotlib, NLTK, and
TensorFlow. Pandas and NumPy have made scientific computations a very easy task,
giving users the flexibility to process huge datasets in memory with an excellent layer of
abstraction, which allows developers and programmers to forget about the background
details and get the job done neatly and efficiently.

[ 10 ]

Introduction to Python

Chapter 1

A few years ago, a typical database engineer would have been expected to know relational
databases such as MySQL, SQL Server, Oracle, PostgreSQL, and so on. Over the past few
years, however, the technology landscape has completely changed. While a typical
database engineer is still supposed to know and be proficient with this database technology
stack, this is no longer enough. With the increasing volume of data, as we enter the era of
big data, traditional databases have to work in conjunction with big data solutions such as
Hadoop or Spark. Having said that, the role of the database engineer has evolved to be one
that includes the skill set of a data analyst. Now, data is not to be fetched and processed
from local database servers—it is to be collected from heterogeneous sources, preprocessed, processed across a distributed cluster or parallel cores, and then stored back
across the distributed cluster of nodes. What we are talking about here is big data analytics
and distributed computing. We mentioned the word Hadoop previously. If you are not
familiar with it, Hadoop is an engine that is capable of processing huge files by spawning
chunks of files across a cluster of computers and then performing an aggregation on the
processed result set, something which is popularly known as a map-reduce operation.
Apache Spark is a new buzzword in the domain of analytics and it claims to be 100 times
faster than the Hadoop ecosystem. Apache Spark has got a Python API for Python
developers called pyspark, using which we can run Apache Spark with native Python
code. It is extremely powerful and having familiarity with Python makes the setup easy
and seamless.
The objective of mentioning the preceding points was to highlight the significance of
Python in the current technological landscape and in the coming future. ML and AI are
likely to be the dominating industries, both of which are primarily powered by Python. For
this reason, there will not be a better time to start reading about and exploring Python and
cyber security with machine learning than now. Let's start our journey into Python by
looking at a few basics.

About Python – compiled or interpreted
Compilers work by converting human-readable code written in high-level programming
languages into machine code, which is then run by the underlying architecture or machine.
If you don't wish to run the code, the compiled version can be saved and executed later on.
It should be noted that the compiler first checks for syntax errors and only creates the
compiled version of the program if none are found. If you have used C, you might have
come across .out files, which are examples of compiled files.

[ 11 ]

Introduction to Python

Chapter 1

In the case of interpreters, however, each line of the program is taken and interpreted from
the source code at runtime and then converted into machine code for execution. Python
falls into the category of interpreted byte code. This means that the Python code is first
translated to an intermediate byte code (a .pyc file). Then, this byte code is interpreted line
by line by the interpreter and executed on the underlying architecture.

Installing Python
Over the course of this book, all of the exercises will be shown on a Linux OS. In my case, I
am using Ubuntu 16.04. You can choose any variant you prefer. We will be using python3
for our exercises, which can be installed as follows:
sudo apt-get install python3
sudo apt-get install python3-pip

The second command installs pip, which is Python's package manager. All open source
Python libraries that do not come as part of the standard installation can be installed with
the help of pip. We will be exploring how to use pip in the upcoming sections.

Getting started
Throughout the course of this book, we will aim to cover advanced and well-known
industry standards in Python, cyber security, penetration testing, and the data science
space. However, as they say, every remarkable journey starts with small steps. Let's go
ahead and start our journey by understanding the basics of Python.

Variables and keywords
Variables, as the name suggests, are placeholders that hold a value. A Python variable is
nothing but a name that can hold a user-defined value during the scope of a Python
program or script. If we compare Python variables to other conventional languages, such as
C, C++, Java, and so on, we will see that they are a little bit different. In the other languages,
we have to associate a data type with the name of the variable. For example, to declare an
integer in C or Java, we have to declare it as int a=2, and the compiler will immediately
reserve two bytes of memory in C and four bytes in Java. It would then name the memory
location as a, which is to be referenced from the program with the value 2 stored in it.
Python, however, is a dynamically typed language, which means that we do not need to
associate a data type with the variable that we will declare or use in our program.

[ 12 ]

Introduction to Python

Chapter 1

A typical Python declaration of an integer might look like a=20. This simply creates a
variable named a and places the value 20 in it. Even if we change the value in the next line
to be a="hello world", it would associate the string hello world with the variable a.
Let's see that in action on the Python Terminal, as follows:

To use the Python Terminal, simply type the python3 command in your Terminal prompt.
Let's think about how this works. Take a look at the following diagram, which compares
statically typed languages with dynamically typed languages:

[ 13 ]

Introduction to Python

Chapter 1

As you can see in the preceding diagrams, in the case of Python, the variable actually holds
a reference to the actual object. Every time the value is changed, a new object is created in
memory and the variable points toward this new object. The previous object is claimed by
the garbage collector.
Having discussed that Python is a dynamically typed language, we must not confuse it
with a weakly typed one. Though Python is dynamically typed, it is also a strongly typed
language, just like Java, C, or C++.
In the following example, we declare a variable, a, of string type and a variable, b, of
integer type:

When we carry out the operation c=a+b, what might happen in a weakly typed language is
that the integer value of b would be typecasted to a string, and the result that was stored in
variable c would have been hello world22. However, because Python is strongly typed,
the function adheres to the type that is associated with the variable. We need to make the
conversion explicitly to carry out any operations of this kind.
Let's take a look at the following example to understand what it means to be a strongly
typed language; we explicitly change the type of variable b and typecast it to a string type
at runtime:

[ 14 ]

Introduction to Python

Chapter 1

Variable naming conventions
Having understood the basics of how variables can be declared and used, let's try to
understand the naming conventions they follow. A variable, also known as an identifier,
can be named by anything that starts with any letter between A-Z, a-z, or an underscore.
This can then be followed by any number of digits or alphanumeric characters.
It must be noted that certain special characters, such as %, @, #, -, and !, are
reserved in Python and can't be used with variables.

Python keywords
Keywords, as the name implies, are certain reserved words that have a predefined meaning
within a particular language implementation. In other languages, we cannot usually name
our variables with the same name as that of the keywords, but Python is a slightly different
case. Although we shouldn't name the variables or identifiers with the same name as those
reserved for keywords, even if we do, the program will not throw any errors and we will
still get an output. Let's try to understand this with the help of a conventional C program
and an equivalent Python script:

It should be noted that this is a simple C program in which we have declared an integer
and used the int identifier to identify it, following which we simply print hello world.

[ 15 ]

Introduction to Python

Chapter 1

When we try to compile the program, however, it throws a compilation error, as shown in
the following screenshot:

Let's try to do the same in a Python shell and see what happens:

It can be seen that the program did not throw any errors when we declared our variable
with the names int and str. Although both int and str are Python keywords, in the
preceding case, we saw that a variable declared with name as int held a string value and a
variable declared with str type held an int value. We also saw how a normal variable, a,
was typecasted from int to string type. From this, it can be established that we can use
reserved words as variables in Python. The downside of this is that if we are to make use of
keywords as variables or identifiers, we are overriding the actual functionality that these
reserved words possess. When we override their actual behavior within the scope of our
program, they will follow the updated or overridden functionality, which is very
dangerous as this would make our code fall out of Python's conventions. This should
always be avoided.

[ 16 ]

Introduction to Python

Chapter 1

Let's extend the preceding example. We know that str() is a built-in Python function, the
purpose of which is to convert a numeric data type into a string type, as we saw for
variable a. Later on, however, we overwrote its functionality and, for the scope of our
program, we assigned it to an integer type. Now, at any point in time during the scope of
this program, if we try to use the str function to convert a numeric type into a string, the
interpreter will throw an error, saying that the int type variables can't be used as methods,
or that they are not callable, as shown in the following screenshot:

The same would hold true for the int method and we would no longer be able to use it to
type cast a string to its equivalent integer.
Now, let's take a look at other types of keywords that are available in Python that we
should try not to use as our variable names. There is a cool way to do this with the Python
code itself, which lets us print the Python keywords in the Terminal window:

[ 17 ]

Introduction to Python

Chapter 1

The import statement is used to import the libraries in Python, just as we use imports for
importing packages in Java. We will get into the details of using imports and loops in
future sections. For now, we will look at what the different Python keywords mean:
false: The Boolean false operator.
none: This is equivalent to Null in other languages.
true: The Boolean true operator.
and: The logical and that can be used with conditions and loops.
as: This is used to assign an alias to a module that we import.
assert: This is used with the objective of debugging code.
break: This exits the loop.
class: This is used to declare a class.
continue: This is the traditional continue statement used with loops that can
be used to continue the execution of a loop.
def: This is used to define a function. Every Python function needs to be
preceded by the def keyword.
del: This is used to delete objects
elif: The conditional else...if statement.
else: The conditional else statement.
except: This is used to catch exceptions.
finally: This is used with exception handling as part of the final block of code
in which we clean our resources.
for: The traditional for loop declaration keyword.
global: This is used to declare and use global variables.
if: The conditional if statement.
import: This is used to import Python libraries, packages, and modules.
in: This is used to search between Python strings, lists, and other objects.
is: This is used to test the identity of an object.
lambda: This is used with Lambda functions.
nonlocal: This is used to declare a variable inside a nested function that is not
local to it.
not: This is a conditional operator.
or: This is another conditional operator.
pass: This is used as a placeholder in Python.
raise: This is used to raise an exception in Python.

[ 18 ]

Introduction to Python

Chapter 1

return: This is used to return from a function.
try: The traditional try keyword that's used with exception handling.
while: This is used with the while loop.
with: This is used with file opening and so on.
yield: This is used with generators.
from: This is used with relative imports.

Throughout this book, we will learn about all the keywords mentioned in this list.

Python data types
Like any other programming language, Python also comes with standard data types. In this
section, we will explore the various powerful data types that Python makes available for us
to use.

Numbers, as the name suggests, covers all the numeric data types, including both integer
and floating data types. Earlier in this chapter, we saw that to use an integer or a float, we
can simply declare the variable and assign an integer or a float value. Now, let's write a
proper Python script and explore how to use numbers. Name the script numbers.py which
is shown as follows:

The preceding screenshot show a simple Python script that adds an integer with a float and
then prints the sum. To run the script, we can type the python3 numbers.py command, as

[ 19 ]

Introduction to Python

Chapter 1

You might have noticed that the command at the beginning of the script says #!
/usr/bin/python. What this line does is make your code executable. After the privileges
of the script have changed and it has been made executable, the command says that if an
attempt is made to execute this script, then we should go ahead and execute it with
python3, which is placed in the /usr/bin/python3 path. This can be seen in the
following example:

If we observe the print command, we can see that the string formatter is %s. To fill it in
with the actual value, the second argument to the print function is passed:

To convert a string into its equivalent integer or float value, we can use the built-in int()
and float() functions.

[ 20 ]

Introduction to Python

Chapter 1

String types
We know that a string is a collection of characters. In Python, string types come under the
sequence category. Strings are really powerful and have many methods that can be used to
perform string manipulation operations. Let's look at the following piece of code, which
introduces us to strings in Python. Strings can be declared within both single and double
quotes in Python:

In the preceding code, we are simply declaring a string called my_str and printing it on the
console window.

String indexes
It must be noted that strings can be accessed as a sequence of characters in Python. Strings
can be thought of as a list of characters. Let's try to print the characters at various indices of
the string, as shown in the following screenshot:

0 gets printed. At index 10, we have an empty space, while at
At index0, the character
index 5, we have the letter m. It should be noted that the sequences are stored in Python
with a starting index 0of
, and the same holds true for the string type.

[ 21 ]

Introduction to Python

Chapter 1

String operations through methods and built-in
In this section, we will look at how to compare two strings, concatenate strings, copy one
string to another, and perform various string manipulation operations with the help of
some methods.

The replace( ) method
The replace method is used to perform string replacement. It returns a new string with
the appropriate replacements. The first argument to the replace method is the string or
character to be replaced within the string, while the second argument is the string or
character with which it is to be replaced:

In the preceding example, we can see that the ! from the original string is replaced by @ and
a new string with the replacement is returned. It should be noted that these changes were
not actually made to the original string, but instead a new string was returned with the
appropriate changes. This can be verified in the following line, where we print the original
string and the old unchanged value, Welcome to python strings !, is printed. The
reason behind this is that strings in Python are immutable, just like they are in Java. This
means that once a string is declared, it can't usually be modified. This isn't always the case,
however. Let's try to change the string and this time try and catch the modifications in the
originally declared string, my_str, as follows:

[ 22 ]

Introduction to Python

Chapter 1

In the preceding code, we were able to modify the original string, as we got the newly
returned string from the replace method in our earlier declared string, my_str. This
might sound contradictory to what we said previously. Let's take a look at how this works
by looking at what happens behind the scenes before and after we call the replace

After replacing the ! with @, this will look as follows:

[ 23 ]

Introduction to Python

Chapter 1

It can be seen in the preceding two illustrations that before the replace method was called,
the my_str string reference pointed toward the actual object that contained an !. Once the
replace() method returned a new string and we updated the existing string variable with
the newly returned object, the older memory object was not overwritten, but instead a new
one was created. The program reference now points toward the newly created object. The
earlier object is in memory and doesn't have any references pointing toward it. This will be
cleaned up by the garbage collector at a later stage.
Another thing we can do is try and change any character in any position of the original
string. We have already seen that the string characters can be accessed by their index, but if
we try to update or change a character at any specific index, an exception will be thrown
and the operation will not be permitted, as shown in the following screenshot:

By default, the replace() method replaces all the occurrences of the replacement string
within the target string. If we only want to replace one or two occurrences of something
within the target string, however, we can pass a third argument to the replace() method
and specify the number of replacement occurrences that we want to have. Let's say we have
the following string:

If we just want the first occurrence of the ! character to be @ and we want the rest to be the
same, this can be achieved as follows:

[ 24 ]

Introduction to Python

Chapter 1

Substrings or string slicing
Obtaining part of the string is a common exercise that we come across frequently in day-today string operations. Languages such as C or Java provide us with dedicated methods
such as substr(st_index,end_index) or subString(st_index,end_index). To
perform the substring operation in Python, there is no dedicated method, but we can
instead use slicing. For example, if we wish to get the first four characters of our original
my_str string, we can achieve this by using operations such as my_str[0:4], as shown in
the following screenshot:

Again, the slice operation returns a new string and the changes are not applied to the
original string. Furthermore, it is worth understanding here that the slicing happens over
n-1 characters, where n is the upper limit, specified as the second parameter, which is four,
in our case. Thus, the actual substring operation will be performed starting 0from index
and ending at index 3, thus returning the string Welc.
Let's take a look at some more examples of slicing:

To get the whole string from index 4, do the following:

To get the string from the start up to index 4, do the following:

To print the whole string with slicing, do the following:

[ 25 ]

Introduction to Python

Chapter 1

To print the characters with a step of 2, do the following:

To print the reverse of the string, do the following:

To print a part of the string in reverse order, to the following:

String concatenation and replication
+ is the concatenation operator that's used in Python to concatenate two strings. As always,
the result of the concatenation is a new string and unless we get the updated string, the
update will not be reflected with the original string object. The + operator is internally
overloaded to perform concatenation of objects when it is used on string types. It is also
used for the addition of two numbers when used on numeric data types, like so:

Interestingly, Python also supports another operator that gets overloaded when used with
string data types. Instead of performing a conventional operation, this operator performs a
variation of the original operation so that the functionality can be replicated across string
data types. Here, we are talking about the multiplication operator, *. This is conventionally
supposed to perform the multiplication of numeric data types, but when it is used on string
data types, it performs a replication operation instead. This is shown in the following code

[ 26 ]

Introduction to Python

Chapter 1

In the preceding case, the multiplication operator actually replicates the Hello world
string stored in the c variable five times, as we specified in the expression. This is a very
handy operation and can be used to generate fuzzing payloads, which we will see in the
later chapters of this book.

The strip(), lstrip(), and rstrip() methods
The strip method is actually used to strip off the white spaces from the input string. By
default, the strip method will strip off the spaces from both the left and right sides of the
string and will return a new string without spaces on both the leading and trailing sides, as
shown in the following screenshot:

However, if we only wish to strip off the left spaces ,we can use the lstrip() method.
Similarly, if we just wish to strip off the right spaces, we can use the rstrip() method.
This is shown as follows:

The split() method
The split method, as the name suggests, is used to split the input string over a particular
delimiter and return a list that contains the words that have been split. We will be looking
at lists in more detail shortly. For now, let's take a look at the following example, where we
have the name, the age, and the salary of an employee in a string separated by commas. If
we wish to obtain this information separately, we can perform a split over ,. The split
function takes the first argument as the delimiter on which the split operation is to be

[ 27 ]

Introduction to Python

Chapter 1

By default, the split operation is performed over a space, that is, if a delimiter is not
specified. This can be seen as follows:

The find(), index(), upper(), lower(), len(), and count()
The find() function is used to search for a character or string within our target string. This
function returns the first index of the string if a match is found. It returns -1 if it does not
find the match:

[ 28 ]

Introduction to Python

Chapter 1

The index() method is identical to the find() method. It returns the first index of the
string if it finds the match and raises an exception if it does not find a match:

The upper() method is used to transform the input string to upper case letters and the
lower() method is used to transform a given string to lowercase letters:

The len() method returns the length of the given string:

The count() method returns the number of occurrences of any character or string that we
wish to count within the target string:

[ 29 ]

Introduction to Python

Chapter 1

The in and not in methods
The in and not in methods are very handy, as they let us perform a quick search on the
sequences. If we wish to check if a certain character or word is present or not present in the
target string, we can use the in and not in methods. These will return True if the word is
present and False otherwise:

The endswith(), isdigit(), isalpha(), islower(), isupper(),
and capitalize() methods
The endswith() method checks whether the given string ends with a specific character or
word that we pass as an argument:

The isdigit() method checks whether the given string is of a digit type or not:

[ 30 ]

Introduction to Python

Chapter 1

The isalpha() method checks whether the given string is of an alphabetic character type
or not:

The islower() method checks whether the string is lowercase, while the isupper()
method checks if the string is uppercase. The capitalize() method puts a given string
into sentence case:

List types
Python does not have array types, but instead offers the list data type. Python lists also fall
under the category of sequences and offer a wide range of functionalities. Coming from a
Java, C, or C++ background, you are likely to find that Python lists are slightly different
from the arrays and list types offered by these languages. In C, C++, or Java, an array is a
collection of elements of similar data types, and this is also the case for Java array lists. This
is different in the case of Python. In Python, a list is a collection of elements that can be of
either homogeneous and heterogeneous data types. This is one of the features that makes
Python lists powerful, robust, and easy to use. We also don't need to specify the size of a
Python list when declaring it. It can grow dynamically to match the number of elements it
contains. Let's see a basic example of using lists:

[ 31 ]

Introduction to Python

Chapter 1

0 and any item can be accessed on the basis of indices, as
Lists in Python start from index
shown in the preceding screenshot. The preceding list is homogeneous, as all the elements
are of string type. We can also have a heterogeneous list, as follows:

For now, we are printing the list elements manually. We can very easily iterate over them
with loops instead, and we will explore that later on. For now, let's try to understand which
operations can be performed on list structures in Python.

Slicing the lists
Slicing is an operation that allows us to extract elements from sequences and lists. We can
slice lists to extract portions that we might be interested in. It must be noted again that the
indexes of slicing are 0-based and that the last index is always considered to be n-1, where
n is the specified last index value. To slice the first five and last five elements from the list,
we can perform the following operation:

Let's see some examples of list slicing and their results:
To get the list from index 4 onwards, do the following:

To get the list elements from the start up to index 4, do the following:

To print the whole list with slicing, do the following:

[ 32 ]

Introduction to Python

Chapter 1

To print the list elements with a step size of 2, do the following:

To print the reverse of the list, do the following:

To print a portion of the list in reverse order, do the following:

Add new elements to list-append(): The append() method is used to add
elements to the list, and the elements to be added are given as an argument to the
append() method. These elements to be added can be of any type. As well as
being a number or a string, the element can be a list in itself:

We can see in the preceding example that we added three elements, 6, 7, and 8, to our
original list using the append() method. Then, we actually added another list containing
three characters that would be stored intact as a list inside the original list. These can be
accessed by specifying the my_list[8] index. In the preceding example, the new list is
added intact to the original list, but is not merged.

[ 33 ]

Introduction to Python

Chapter 1

Merging and updating lists
List merging can be done in two ways in Python. First, we can use the traditional +
operator, which we used previously to concatenate two strings. It does the same when used
on list object types. The other way to achieve this would be by using the extend method,
which takes the new list as an argument to be merged with the existing list. This is shown
in the following example:

To update an element in the list, we can access its index and add the updated value for any
element that we wish to update. For example, if we want to have the string Hello as the 0
element of the list, this can be achieved by assigning
the 0 to the Hello value as

Copying lists
We have seen that Python variables are nothing but references to actual objects. The same
holds true for lists. For this reason, manipulating lists gets a little tricky. By default, if we
copy one list variable to another one by simply using the = operator, it won't actually create
a duplicate or local copy of the list for that variable – instead, it would just create another
reference and point the newly created reference toward the same memory location. Thus,
when we make a change to the copied variable, the same change will be reflected in the
original list. In the following example, we will create new isolated copies, where a change
in the copied variable will not be reflected in the original list:

[ 34 ]

Introduction to Python

Chapter 1

Now, let's look at how can we create a new copy of an existing list so that the changes to the
new one do not cause any changes to the existing one:

Another way to create the isolated copy of the original list is to make use of the copy and
deepcopy functions that are available in Python. A shallow copy constructs a new object
and then inserts references to that object to the objects found in the original list. A deep copy,
on the other hand, constructs a new compound object and then recursively inserts copies of
the objects found in the original list:

Removing elements from lists
We can use the del command to delete either an element from the list or the whole list. The
del command does not return anything. We can also use the pop method to remove
elements from the list. The pop method takes the index of the element that we wish to
remove as an argument:

[ 35 ]

Introduction to Python

Chapter 1

The entire list structure can be deleted as follows:

Replication with len(), max(), and min()
The multiplication operator *, when applied to lists, causes a replication effect of the list
elements. The contents of the list are repeated as many times as indicated by the number
passed to the replication operator:

The len() method gives the length of the Python lists. The max() method returns the
maximum element of the list, while the min() method returns the minimum element of the

[ 36 ]

Introduction to Python

Chapter 1

We can use the max and min methods on the character types as well, but we cannot use
them on a list that has mixed or heterogeneous types. If we do this, we will get an exception
stating that we are trying to compare numbers and characters:

in and not in
The in and not in methods are essential Python operations that can be used against any
sequence type. We saw how these were used previously with strings, where we used them
to search for a string or character within the target string. The in method returns true if
the search is successful and returns false if not. The opposite is the case for the not in
method. The execution is shown as follows:

[ 37 ]

Introduction to Python

Chapter 1

Tuples in Python
A Python tuple is very similar to a Python list. The difference is that it's a read-only
structure, so once it is declared, no modification can be made to the elements of the tuple.
Python tuples can be used as follows:

In the preceding code, we can see that we can access tuples in the same way as we can
access lists, but when we try to change any element of the tuple, it throws us an exception
as a tuple is a read-only structure. If we perform the operations that we performed on lists,
we will see that they work in exactly the same way as tuples:

If a tuple has only one element in it, it has to be declared with a trailing comma. If we do
not add that comma while declaring it, it will be interpreted as a numeric or string data
type, depending on the elements of the tuple. The following example explains this better:

[ 38 ]

Introduction to Python

Chapter 1

A tuple can be converted into a list and can then be operated on as follows:

Dictionaries in Python
Dictionaries are very powerful structures and are widely used in Python. A dictionary is a
key-value pair structure. A dictionary key can be a unique number or string, and the value
can be any Python object. Dictionaries are mutable and can be changed in place. The
following example demonstrates the basics of dictionaries in Python:

A Python dictionary can be declared within curly braces. Each key value pair is separated
by a comma. It should be noted that the keys have to be unique; if we try to repeat the keys,
the old key value pair is overwritten by the new one. From the preceding example, we can
establish that the dictionary keys can be either string or numeric types. Let's try to perform
various operations on dictionaries in Python:
Retrieving the dictionary values with the keys: Dictionary values can be
accessed through the name of the dictionary key. If the name of the key is not
known, we can use loops to iterate through the whole dictionary structure. We
will cover this in the next chapter of this book:

This is one of the many ways to print dictionary values. However, if the key
for which the value we wish to print does not exist in the dictionary, we will
get a key not found exception, as shown in the following screenshot:

[ 39 ]

Introduction to Python

Chapter 1

There is a better way to handle this and avoid these kinds of exceptions. We
can use the get() method provided by the dictionary class. The get()
method takes the key name as the first argument and the default value if the
key is not present as the second argument. Then, instead of throwing an
exception, the default value will be returned if the key is not found. This is
shown in the following screenshot:

In the preceding example, when the k1 key is present in the actual
dictionary, dict1, the value for the k1 key is returned, which is v1. Then, the
k0 key was searched, which was not present originally. In that case, no
exception was raised, but instead the False value was returned, suggesting
that no such key, K0, was actually present. Remember that we can specify
any placeholder as the second argument to the get() method to indicate the
absence of the key we are searching for.
Adding keys and values to the dictionary: Once a dictionary has been declared,
over the course of the code, there could be many occasions in which we want to
modify a dictionary key or add a new dictionary key and value. This can be
achieved as follows. As mentioned earlier, a dictionary value can be any Python
object, so we can have tuples, lists, and dictionary types as values inside a

[ 40 ]

Introduction to Python

Chapter 1

Now, let's add more complex types as values:

These values can be retrieved as normal values by their keys as follows:

Expanding a dictionary with the contents of another dictionary: In the
preceding example,we added a dictionary as a value to an existing dictionary.
We will now see how can we merge two dictionaries into one common or new
dictionary. The update() method can be used to do this:

Keys(): To get all the dictionary keys, we can use the keys() method. This
returns the class instances of the dictionary keys:

[ 41 ]

Introduction to Python

Chapter 1

We can see that the keys method returns an instance of the dict_keys class,
which holds the list of dictionary keys. We can type cast this as a list type as

values(): The values() method returns all the values that are present in the

Items(): This method is actually used to iterate over the dictionary key value
pairs, as it returns a list class instance that contains a list of tuples. Each tuple has
two entries, the first one being the key and the second one being the value:

We can convert the returned class instance into a tuple, list tuple, or list type
as well. The ideal way to do this is to iterate over the items, which we will
see later when we look at loops:

in and not in: The in and not in methods are used to see whether a key is
present in the dictionary or not. By default, the in and not in clauses will
search the dictionary keys, not the values. Take a look at the following example:

[ 42 ]

Introduction to Python

Chapter 1

Order of storing: By default, Python dictionaries are unordered, which means
they are not stored internally in the same order as we define them. The reason for
this is that the dictionaries are stored in dynamic tables called hash tables. As
these tables are dynamic, they can increase and shrink in size. What happens
internally is that a hash value of the key is computed and stored in the table. The
key goes in the first column, while the second column holds the actual value.
Let's take a look at the following example to explain this better:

In the preceding case, we declare a dictionary, a, with the first key as abc
and the second key as abcd. When we print the values, however, we can see
that abcd is stored internally before abc. To explain this, let's assume that
the dynamic table or hash table in which the dictionary is internally stored is
of size 8.
As we mentioned earlier, the keys will be stored as hash values. When we
compute the hash of the abc string and and divide it in a modular fashion by
8, which is the table size, we get the result of 7. If we do the same for abcd,
we get a result of 4. This means that the hash abcd will be stored at index 4,
while the hash abc will be stored at index 7. For this reason, in the listing, we
get abcd listed before abc:

There may be occasions in which two keys arrive at a common value after
the hash(key)%table_size operation, which is called a collision. In this
case, the key to be slotted first is the one that is stored first.
sorted(): If we want our dictionary to be sorted according to the keys, we can
use the built-in sorted method. This can be tweaked to return a list of tuples, with
each tuple having a key at the
and its value at theindex:

[ 43 ]

Introduction to Python

Chapter 1

Removing elements: We can use the conventional del statement to delete any
dictionary item. When we say delete, we mean delete both the key and the value.
Dictionary items work in pairs, so deleting the key would remove the value as
well. Another way to delete an entry is to use the pop() method and pass the
key as an argument. This is shown in the following code snippet:

Python operators
An operator in Python is something that can carry out arithmetic or logical operations on
an expression. The variable on which the operator operates is called the operand. Let's try
to understand the various operators that are available in Python:
Floor Division


a ** b
a // b

[ 44 ]

Introduction to Python

Chapter 1

a = 0 evaluates to a=0
a +=1 evaluates to a = a + 1
a -= 1 evaluates to a = a + 1
a *= 2 evaluates to a = a * 2
a /= 5 evaluates to a = a / 5
a **= 3 evaluates to a = a ** 3
a //= 2 evaluates to a= a // 2 (floor division 2)
a %= 5 evaluates to a= a % 5

Logical operators:
and: True: If both the operands are true, then the condition
becomes true. For example, (a and b) is true.
or: True: If any of the two operands are non-zero, then the
condition becomes true. For example, (a or b) is true.
not: True: This is used to reverse the logical state of its operand.
For example, not (a and b) is false.
Bitwise operators:

Right Shift
Left Shift


a >> b
a << b

In this chapter, we discussed the basics of Python and explored the syntax of the language.
This isn't very different from the languages that you may have studied in the past, such as
C, C ++, or Java. However, it's much easier to use and is really powerful in the cyber
security domain compared to its peers. This chapter formulates the basics of Python and
will help us progress, as some data types such as lists, dictionaries, tuples, and strings are
used heavily throughout the course of this book.
In the next chapter, we will learn about conditions and loops and see how loops can be
used with the data types that we have studied so far.

[ 45 ]

Introduction to Python

Chapter 1

Is Python
1. open source? If so, how is it different from other open source
Who manages
Python and works on further feature enhancements?
Is3.Python faster than Java?
Is Python
object-oriented or functional?
Can I learn
Python quickly if I have little to no experience with any
programming language?
How is6.Python beneficial to me, being a cyber security engineer?
I am a 7.
penetration tester – why do I need to understand AI and machine

[ 46 ]

Building Python Scripts
This chapter will cover the core concepts of all programming languages. This includes
conditional statements, loops, functions, and packages. We will see that these concepts are
pretty much the same in Python as they are in other programming languages, except for
some syntactical differences. But syntax just requires practice; everything else will fall into
line automatically. The topics that we are going to cover in this chapter are as follows:
Conditional statements
Modules and packages
Comprehensions and generators

Technical requirements
Make sure you have the following pre-requisites needed to proceed further:
A working computer or laptop
Ubuntu OS (preferably 16.04)
Python 3.x
A working internet connection

Building Python Scripts

Chapter 2

If you come from a background of a language such as Java, C, or C++, you might be familiar
with the concept of grouping logically connected statements using curly braces. This is not
the case, however, in Python. Instead, the logically connected statements, including classes,
functions, conditional statements, and loops, are grouped using indentation. Indentation
keeps the code clean and easy to read. We shall explore this in more detail in the coming
sections. For now, however, let's say goodbye to braces. I recommend that you use tabs for
indentation, as typing an equal number of spaces in every line becomes very timeconsuming.

Conditional statements
Just like all other languages, to carry out the conditional operations, Python makes use of
conditional statements. The conditional statements supported by Python are as follows:
if condition
if...else condition
else...if conditional ladder, known as elif in Python

Python doesn't support the switch statement.

The if condition
The if condition or the if statement takes a statement and returns either a Boolean True
or a Boolean False value after evaluating the statement. If the condition returns True, the
code proceeding the if statement (equally indented) is executed. If the statement/condition
evaluates to False, then either the else block of code gets executed if there is one, or the
block of code following the if block is executed, so the if block is effectively skipped. Let's
take a look at the if code in action.

[ 48 ]

Building Python Scripts

Chapter 2

From now on, we are going to look at how scripting works. We will either be creating script
files or carrying out exercises. For this reason, go ahead and create a file on gedit or any
editor of your choice and name it if_condition.py. Alternatively, we can type gedit
if_condition.py in the Terminal:

We then type the following code:
if a > b:
print("a is greater")

Now, in order to run this script, we can simply type python3.5 if_condition.py in the

The Python print method by default adds \n to the string to be printed, using which we
can see two outputs in different lines. Note that the syntax of the if statement is as follows:
if <condition> : and then indented code

Whether or not we use brackets with the conditions is up to us. As you can see, the
condition evaluated to True, so the line a is greater was printed. For if conditions in
Python, anything that does not evaluate to 0zero
), False
( , None, or empty would be
treated as True and the code proceeding the if statement will get executed.
Let's see another example of the if condition in conjunction with
the and...or and and...not logical operators.

[ 49 ]

Building Python Scripts

Chapter 2

Let's create another file called if_detailed.py and type the following code:

You might have noticed that at the beginning of the file, we have a statement that reads #!
/usr/bin/python3.5. This means we don't have to type python3.5 every time we
execute the code. It directs the code to use the program placed at /usr/bin/python3.5 to
execute it every time it's executed as an executable. We need to change the permissions of
the file to make it executable. Do this, and then execute the code as follows:

[ 50 ]

Building Python Scripts

Chapter 2

The output produced is self-explanatory. As I mentioned before, anything that doesn't
evaluate to0, False, None, or empty is taken as True and the if block is executed. This
explains why the first three if conditions were evaluated to True and the message was
printed, but the fourth message was not printed. From line 19 onwards, we have made use
of logical operators. In Python, the conjunction operation is carried by the and operator,
which is the same as &&, which we use with C, C++, and Java. For the short circuit Boolean
operator, we have the or keyword in Python, which is the same as || in C, C++, and Java.
Finally, the not keyword provides negation in Python, as ! does in other languages.
It should be noted that in Python, null byte characters are represented by
a reserved keyword, None, which is the same as null in languages such
as Java or C#.

The if...else condition
The if...else condition is pretty much the same as in any other language. If the if
condition evaluates to a True value, the code block indented under if is executed.
Otherwise, the code block indented under the else block is executed:
if a > b:
print("a is Greater")
print("B is either Greater or Equal")

[ 51 ]

Building Python Scripts

Chapter 2

Let's create a file called if_else.py and see how to use it:

Again, the output here is self-explanatory. In this section of code, we explored the usage of
some bitwise operators alongside the if...else code structure. We also used variables,
which are to be printed with the print method. %s is a placeholder, and specifies that the
value of %s should be replaced by a string variable whose value would come immaterially
after the string ends. If we have multiple values to substitute, they can be passed as a tuple
such as %(val1,val2,val3):

[ 52 ]

Building Python Scripts

Chapter 2

The if...elif condition
The if...elif ladder, popularly known as if...else if in other programming languages
such as C, C ++, and Java, has the same function in Python. An if condition let's us specify
a condition alongside the else part of the code. Only if the condition is true is the section
proceeding the conditional statement executed:
if a > b:
print("a is Greater")
elif b > a:
print("B is either Greater or Equal")
print("A and B are equal")

It must be noted that the third else in the preceding code snippet is optional. Even if we
don't specify it, the code works just fine:

[ 53 ]

Building Python Scripts

Chapter 2

Let's create a file named if_el_if.py, and see how this can be used:

Loops are utilities that every programming language has. With the help of loops, we
execute tasks or statements that are repetitive in nature, which, without loops, would take
up lots of lines of code. This, in a way, defeats the purpose of having a programming
language in the first place. If you are familiar with Java, C, or C ++, you might have already
come across while, for, and do...while loops. Python is pretty much the same, except
that it doesn't support do...while loops. Thus, the loops that we are going to study in the
following section in Python are the following:
while loop
for loop

The while loop
Remember that when we discussed lists in the first chapter of the book, we mentioned that
lists can actually hold heterogeneous data types in Python. A list may contain integers,
strings, dictionaries, tuples, or even nested lists. This feature makes lists very powerful and
exceptionally easy and intuitive to use. Let's take a look at the following example:
my_list[0] -> 1
my_List[1] -> "a"
my_list[2] -> [1,2,3]
my_list[2][0] -> 1
my_list[2][2] -> 3
my_list[3] -> {"k1":"v1"}
my_list[3]["k1"] -> "v1"
my_list[3].get("k1") -> "v1

Let's take a closer look at while loops with the help of the following code, which we will
call while_loops.py. We will also see how we can iterate over lists using while loops:

[ 54 ]

Building Python Scripts

Chapter 2

The first portion of the code, lines 2 to 6, depicts a simple usage of the while loop, where
we are printing a statement five times. Note that the condition that is specified for the loop
to be executed can be placed with or without braces, as shown from lines 7 to 10.
In line 12, we declared a list containing numbers, strings, a float, and a nested list as well.
Then, in the final while loop starting at line 14, we iterate over the elements of the list by
setting the loop control variable to be less than the length of the list. Within the loop, we
check for the type of the list variable. The if type (1) returns an integer class, type (a)
returns a string class, and type ([]) returns a list class. When the type is a list, we iterate
over its elements again in a nested while loop, and print each one, as shown from lines 19
to 24:

[ 55 ]

Building Python Scripts

Chapter 2

The for loop
The for loop is very widely used in Python, and it's the default choice whenever we have
to iterate over an inalterable list. Before moving forward with the for loop, let's take a
closer look at what the terms iteration, iterable, and iterator mean in Python.

Iteration, iterable, and iterator
Iteration: An iteration is a process wherein a set of instructions or structures are repeated in
a sequence a specified number of times or until a condition is met. Each time the body of a
loop executes, it's said to complete one iteration.
Iterable: An iterable is an object that has an __iter__ method, which returns an iterator.
An iterator is any object that contains a sequence of elements that can be iterated over, and
then an operation can be performed. Python strings, lists, tuples, dictionaries, and sets are
all iterables as they implement the __iter__ method. Take a look at the following snippet
to see an example:

In the preceding code snippet, we declare a string a and place the value hello value in it.
To see all the built-in methods of any object in Python, we can use the dir(<object>)
method. For strings, this returns us all the operations and methods that can be carried out
on the string type. In the second row, the 5th operation is the iter method we mentioned
previously. It can be seen that iter(a) returns a string iterator:

Likewise, a list object's iter method would return a list iterator, as previously shown.

[ 56 ]

Building Python Scripts

Chapter 2

Iterator: An iterator is an object with a __next__ method. The next method always returns
the next element of the sequence over which the original iter() method is called, starting
from index 0. This is shown in the following code snippet:

As can be seen in the examples for both string and list, the next method over the iterator
will always return us the next element in the sequence or the object that we are iterating
over. It must be noted that the iterator only moves forwards, and if we want iter_a or
list_itr to go back to any element, we must reinitialize the iterator to the original object
or sequence:

A closer look at for loops
The for loops in Python exceed the capabilities of for loops in other programming
languages. When calling upon iterables such as strings, tuples, lists, sets, or dictionaries, the
for loop internally calls the iter to get an iterator. Then, it calls the next method over that
iterator to get the actual element in the iterable. It then calls next repeatedly until a
StopIteration exception is raised, which it would internally handle and get us out of the
loop. The syntax of the for loop is given as follows:
for var in iterable:
statement 1
statement 2
statement n

[ 57 ]

Building Python Scripts

Chapter 2

Let's create a file called for_loops.py, which will explain the basic use of for loops:

In the preceding example, we used the Python range function/method, which helps us
implement the traditional for loop that we learnt in other programming languages such as
C, C ++, or Java. This might look like for i =0 ;i < 5 ;i ++. The range function in
Python takes one mandatory argument and two default ones. The mandatory argument
0, index
specifies the limit of the iteration and, starting from
returns numbers until it
reaches the limit, as seen in lines 3 and 4 of the code. When invoked with two arguments,
the first one serves as the starting point of the range and the last one serves as the end
point, as is depicted in lines 7 and 8 of our code. Finally, when the range function is
invoked with three arguments, the third one serves as the step size, which is equal to
one by default. This is depicted in the following output and lines 12 and 13 of the sample

[ 58 ]

Building Python Scripts

Chapter 2

Let's take a look at another example of a for loop, which we shall use to iterate over all the
iterables that Python defines. This will allow us to explore the real power of for loops. Let's
create a file called for_loops_ad.py:

[ 59 ]

Building Python Scripts

Chapter 2

Earlier, we saw how we can read the values from lists, strings, and tuples. In the preceding
example, we use the for loop to enumerate over strings, lists, and dictionaries. We learned
earlier that the for loop actually invokes the iter method of iterables and then calls the
next method for each iteration. This is shown in the following example:

[ 60 ]

Building Python Scripts

Chapter 2

When we iterate over Python dictionaries using a for loop, by default, the
dictionary keys are returned to us. When we use .items() over
dictionaries, it returns us a tuple each time we iterate over it, with the key
at 0 index of the tuple and the value at the first index.

Functions and methods in Python
Functions and methods are used to design or make a logical unit of code that can be reused
throughout the course of your script or other scripts. Functions actually form the basis of
code reuse and bring modularity to the code structure. They keep the code clean and easier
to modify.
It is advisable to always try to break our logic into small units of code,
each of which is a function. We should try to keep the size of the method
small in terms of the lines of code whenever possible.
The following code represents the basic syntax of defining methods in Python:
def print_message(message):
statement 2

Python methods do not have a return type in their definition as you might have seen in C,
C++, or Java, such as void, in, float, and so on. A Python method may or may not return
a value, but we do not explicitly need to specify that. Methods are very powerful and
flexible in Python.
It should be noted that the default name of every Python script is
main and that this is placed inside a global variable that is accessible
throughout the Python context called __name__. We shall use this in the
coming example.

[ 61 ]

Building Python Scripts

Chapter 2

Let's explore the various ways of invoking methods using our method_basics.py script:

Let's now break this down into smaller pieces and try to understand what has happened:
print_msg1(): This is a basic method that just prints a string on the console. It
is defined at line 2 and invoked at line 19.
print_msg2(): This is a method that takes an argument in the variable message
and then prints that variable value on the screen. Remember that Python
variables do not require a type to be specified, so we can pass any data to the
message variable. This is a Pythonic example of a method that takes a single
argument. Remember that the type of the argument is a Python object and it can
take any value passed to it. The output can be seen in the following screenshot:

[ 62 ]

Building Python Scripts

Chapter 2

print_msg3(): This is a Python method that takes two arguments. It is similar
to the print_msg2() method that we saw previously. The difference is that it
may sometimes return a value. It is also invoked differently. Note that in line 22,
we invoke this method by passing the second parameter as True. This means it
has a return value of True, but we do not invoke it with True as a second
parameter in line 26, so it therefore returns nothing. For this reason, we get None
printed on the screen. In other programming languages, such as C, C++, or Java,
the order of parameters while invoking the method is very important. This is
because the sequence with which we passed the argument should be the same
sequence that is passed to the method. In Python, however, we can invoke the
methods and pass the named parameters during invocation. This means that the
order or sequence doesn't matter, as long as the name matches the name of the
method parameter. This is depicted in line 29, where we are passing a message as
a second parameter, even though it is the first parameter in the method
definition. This works perfectly, as shown in the output.
print_msg4(): This is where we get familiar with Python default parameters
and how they can be used with methods. A default parameter is a variable that is
assigned a default value while a method is declared. If the caller passes on a
value for this parameter or variable, then the default value is overwritten by the
value passed by the caller. If no value is passed for the default parameter during
invocation, then the variable persists the default value to which it was initialized.
The print_msg4() method has got one mandatory argument, m, and two
optional arguments, op1 and op2.

[ 63 ]

Building Python Scripts

Chapter 2

print_msg4('Test Mandatory'): This is invoked at line 31. This indicates
that the Test mandatory string should be passed for the mandatory parameter
and the other two op1 and op2 variables will be initialized to the default values,
as seen in the output.
print_msg4(1,2): This is invoked at line 32. This indicates that an integer
with value=1 should be passed for the mandatory parameter and another
integer with value=2 should be passed for op1. The default value will therefore
be overwritten for op1. op2 will retain the default value, as no value is passed.
print_msg4(2,3,2): This is invoked at line 33. This indicates that an integer
with value=2 should be passed for the mandatory parameter and another
integer with value=3 should be passed for op1 so the default values for op1
and op2 are overwritten.
print_msg4(1,op2='Test'): This is invoked at line 34. The mandatory
parameter receives an integer with value=1. For the second parameter, we are
specifying a named parameter during invocation, so the sequence does not
matter for Test, which will get copied to op2 of the caller.
print_msg4(1,op2=33,op1=44): This is invoked at line 35. The mandatory
parameter receives value=1. For the second parameter, we specify a named
parameter, op2, and for the third parameter, we pass op1. Again, we can see in
the output that the sequence does not matter.
print_msg5(): Usually, in other programming languages, a function or method
can always return one value. If it needs to return multiple values, it must put the
values in an array or another structure and then return them. Python handles this
situation abstractly for us. If you read the code, you might think that the method
is returning multiple values, whereas in reality it's returning a tuple with each
value multiplied by two. This can be validated from the output.

Let's now explore some further methods and ways to pass arguments, using the following
example, methods_adv.py. The following code snippet represents variable-argument type
methods in Python. As can be verified from the output, method_1 takes a normal
sequence of any size as an input, meaning we can pass any number of arguments to the
method. When the method is declared with a parameter preceded by the * sign, all the
passed arguments are translated into a sequence and a tuple object is placed inside args.
On the other hand, when * is used with a parameter while invoking the method, the
parameter type from the sequence is changed and internally each element if sequence is
passed as a single parameter to the caller, as seen in method_1_rev.

[ 64 ]

Building Python Scripts

Chapter 2

Furthermore, when if is used with a parameter in the method declaration, it internally
transforms all the maned parameters into a Python dictionary, with the key as the name
and the value being the same as the value after the = operator. This can be seen
in method_2. Finally, when ** is used with the caller parameter, that parameter
is internally transformed from a Python dictionary to named parameters. This can be
validated with method_2_rev:

[ 65 ]

Building Python Scripts

Chapter 2

Modules and packages
Every Python script is called a module. Python has been designed with reusability and ease
of code in mind. For this reason, every Python file we create becomes a Python module and
is eligible to be invoked or used within any other file or script. You might have learned in
Java how to import classes and reuse them with other classes. The idea is pretty much the
same here, except that we are importing the whole file as a module and we can reuse any
method, class, or variable of the imported file. Let's take a look at an example. We will
create two files, child.py and parent.py, and put the following code in each, as follows:

The first five lines belong to child.py, and the last eight lines belong to
parent.py. We will run the parent, as shown in the output. It should be
noted that the imported file can be given an alias. In our case, we
imported the child and gave it the alias C. Finally, we
called child_method() class of that module from the parent Python
Let's now try to explore Python packages and how they can be used. In Java, a package is
nothing but a folder or directory that collects logically connected class files in Java.
Packages do the same in Python; they collect logically connected Python modules. It is
always recommended to use packages, as this keeps the code clean and makes it reusable
and modular.
As mentioned earlier, a Python package is a normal directory. The only difference is that in
order to make a normal directory behave like a Python package, we must place an empty
__init__.py file inside the directory. This indicates to Python which directories it should
use as packages. Let's go ahead and create a package called shapes. We will place an
empty Python file called __init__.py and another file called area_finder.py inside it:

[ 66 ]

Building Python Scripts

Chapter 2

Let's now put the following code in the area_finder.py file. Let's also create another file
called invoker.py and place it outside the shapes folder that we created. The code of the
invoker is given on the right-hand side of the following image, while the code of
the area_finder is given on the left-hand side:

The preceding code is a straightforward example of how to use packages in Python.We
created a package called shapes and placed a file called area_finder in it, which will
compute the area of a shape. Then, we went ahead and created a file called invoker.py
outside the shapes folder, and imported the area_finder script from the package
in multiple ways (for demonstration purposes). Finally, we used one of the aliases to invoke
the find_area() method.

[ 67 ]

Building Python Scripts

Chapter 2

Generators and comprehensions
A generator is a special kind of iterator in Python. In other words, a Python generator is a
function that returns us a generator iterator by issuing the yield command, which can be
iterated upon. There might be occasions in which we would want a method or function to
return us a series of values, instead of just one. We might, for example, want our method to
partially carry out a task, return the partial results to the caller, and then resume the work
right from the place where it returned the last value. Usually, when a method terminates or
returns a value, its execution begins again from the start. This is what generators try to
address. A generator method returns a value and a control to the caller and then continues
its execution right from where it left off. A generator method is a normal Python method
with a yield statement. The following code snippet, generators.py, explains how
generators can be used:

Note that since genMethod has a yield statement in it, it becomes a generator. Every time
the yield statement is executed, the value of "a" is returned to the caller as well as the
control (remember that generators return series of values). Every time the next() call is
made to the generator method, it resumes its execution from where it left off previously.
We know that every time a yield is executed, the generator method returns a generator
iterator. Thus, as with any iterator, we can use a for loop to iterate over the generator
method. This for loop will continue until it reaches the yield operation in the method. The
same example with a for loop would look as follows:

[ 68 ]

Building Python Scripts

Chapter 2

You might be wondering why we would use generators when the same
result can be achieved with lists. Generators are very memory- and spaceefficient. If a lot of processing is required to generate values, it makes
sense to use generators, because then we only generate values according
to our requirements.
Generator expressions are one-line expressions that can produce generator objects, which
can be iterated over. This means that the same optimization in terms of memory and
processing can be achieved. The following code snippet shows how generator expressions
can be used:

Python comprehensions, commonly known as list comprehensions, are a very powerful
Python utility that comes in handy if we need to perform some manipulation operations
over all or some of the elements of a list. A list comprehension would return a new list with
the modifications applied. Let's say that we have a list of numbers and we want to square
each number in the list.

[ 69 ]

Building Python Scripts

Chapter 2

Let's take a look at two different approaches to this problem:

The code snippet on the left, the more traditional approach, took nine lines. The same code
using comprehensions took us just three lines. A list comprehension is declared within
square brackets and it performs any operation on each element of a list. It then returns a
new list with the modifications. Let's take a look at another example of comprehensions.
This time, we will use an if condition (known as a comprehension filter), and nested loops
with comprehensions. We will name the file list_comp_adv.py and enter the following

The preceding snippet is self-explanatory. It shows us how to use if conditions with
comprehensions (line 4). It also shows us how to use nested loops to add up two lists (line
5). Finally, it shows us how to use dictionaries with comprehensions (line 6).

[ 70 ]

Building Python Scripts

Chapter 2

Map, Lambda, zip, and filters
In this section, we are going to understand some very handy Python functions. These allow
us to carry out quick processing operations on Python iterables such as lists.
Map(): As we saw earlier, list comprehensions come in very handy when we
have to perform an operation on all or some of the elements in a list. The same
can be achieved with the help of the map function. This takes two arguments, the
first being the function that will perform the manipulation on the elements of the
list, and the second being the list itself. The following example, map_usage.py,
demonstrates this:

Lambda(): Lambda functions are small but powerful in-line functions that can be
used for manipulation of data. They can be very useful for small manipulations,
as very little code is required to implement them. Let's look at the same example
again, but this time we will use a Lambda function in place of a normal Python

Zip(): The zip method takes two lists or iterables and aggregates the elements
across multiple iterables. Finally, it returns us a tuple iterator that contains the
aggregation. Let's make use of a simple code, zip_.py, to demonstrate this

[ 71 ]

Building Python Scripts

Chapter 2

Filter() :The filter method is used to filter out the elements of the list that
return true for a particular condition. The filter method takes two arguments,
the first being the method or Lambda function that either returns true or false
for a particular element, and the second being the list or iterable to which that
element belongs. It returns a list that contains the elements for which the
condition evaluated as true. Let's create a file called filter_usage.py, and
add the following content:

In this chapter, we discussed abut conditions, loops, methods, iterators, packages,
generators, and comprehensions. All of these are widely used in Python. The reason why
we covered these topics is because when we get into automating penetration testing and
cyber security test cases later on, we will see these concepts widely used within our code
files. In the next chapter, we will explore the object-oriented nature of Python. We will
explore how to deal with XML, CSV, and JSON data in Python. We will also read about
files, IO, and regular expressions.

Name1.a real-world use case in which generators are used.
Can we 2.
store a function name in a variable and then invoke it via a variable?
Can3.we store a module name in a variable?

Further reading
Generators and Comprehensions: http:/ / jpt- pynotes. readthedocs. io/ en/
latest/ generators- comprehensions. html

Modules: https:/ / docs. python. org/ 3/ tutorial/ modules. html

[ 72 ]

Concept Handling
This chapter will allow us to get familiar with various object-oriented concepts in Python.
We will see that Python can not only be used as a scripting language, but that it also
supports a wide range of object-orien