Profound

Python

Önder Teker

Godoro Publishing

GODORO PUBLISHING

Publisher Certificate No (Turkey): 40946

The Name Of The Book:

Profound Python

Copyright © 2021 Godoro Yayıncılık

The Author Of The Book:

Önder Teker

First Edition, August 2021, Istanbul

ISBN:

978-605-74764-3-2

Profound

Python

Önder Teker

Godoro Yayıncılık

Language & Tools

Python is a language developed to learn and use easily. Its first version is created for development in terminal or command line environments. Afterwards, because of its ease and very rich libraries, it started to be used in many areas. It came out as a simple alternative to some languages such as C/C++, Java and C#. Python began to support complex features over time. However, it keeps its simplicity to a great extent.

Interpretation

There is a main distinction between a development and programming language. Some languages are firstly written in a source language, then the source code is converted to executable machine code. This is called compiling. Then the machine code is executed without source code. Some programming languages execute source code directly. Execution is done by reading source code line by line and running them on the fly. This is called interpretation. Python is an interpreted language.

CPython

A Python language interpreter can be written in any programming language. However, C language is mostly used for this. The language is sometimes called C/C++ and C++ is an advanced or complex version of C. These languages are low-level and run very close to the machine. It works fast but the development process may be very complex and there may be lots of problems if it is not used properly. Even a tiny mistake such as omitting a character may cause the whole machine crash.

An interpreter of Python written in C language is called CPython. There are some other implementations such as Jython written in Java language. CPython is a C application which interprets code written in Python language.

Cython

There is another name, Cython, which can be confused with Cython. Cython is a language very similar to, or almost the same as, Python but compatible with C. It is a language, not a language interpreter. It works compatible with C language. Most Python libraries, especially ones directly accessing to hardware, is written with Cython. The code written in Cython is converted to C, then compiled from C to machine code. So, since it is not interpreted, it can be as fast as C. However, it ensures the code written is compatible with Python language.

C Libraries

Python language is a separate, independent language from C. However, it can use directly or indirectly libraries written in C. Especially libraries that require fast execution and related to hardware are written in C and called from Python. Many C libraries have a Python interface to access them. Some of them use this feature so much that the actual library written in C is used less than the Python interface. The actual library is used only by a small community of developers.

Is C Needed To Know?

Some people have a notion that C or C++ language should be known to use Python properly since Python uses C libraries very often. However, many people learn Python just because they don't want to deal with the complexity of C/C++. Python provides an easy way to use a C library without knowing C. It may be helpful to know how to install or use libraries in C/C++ rather than coding. However this may come into question when advanced topics are needed to learn, not in the phase of learning Python.

Short Syntax

Python language has a short syntax. The section below contains a comparison of some programming languages with Python. For this purpose, a classical application which prints a message such as Hello, World! on the console is written for each language.

C

The simplest application in C language can be written as below:

#include

int main() {

printf(Hello, World!);

  return 0;

}

Java

The same application for Java language is like this:

class HelloWorld {

  public static void main(String[] args) {

    System.out.println(Hello, World!);

  }

}

C#

The application in C# can be written as below:

namespace Hello

{

  class HelloWorld {       

    static void Main(string[] args)

    {

        Console.WriteLine(Hello, World!);

    }

  }

}

Python

The same operation is written in Python as below:

print(Hello, World!)

As it can be easily seen, the Python language is much shorter. The difference is that Python doesn't force a developer to define a class. On the other hand, contrary to functional languages, Python allows developers to define classes or use objects if they want to.

Development Tools

There is much development environment support for the Python language. Some of them are code editors, which contain basic editing operations but don't support language specific tools. These are basically text editors with some extra features such as code colorizing and code completing. On the other hand, there are advanced applications called Integrated Development Environment (IDE). They have features to create applications by using visual tools or clicking some components instead of writing code.

Basic Tools

Some basic tools are explained in this section.

IDLE

grafikler2 Natural development environment for the Python language is called IDLE. It provides basic tools for people to learn the Python language. However, it is simple and contains no advanced features. Advanced applications can be developed but IDLE h no support for them. Developers must code in a simple environment. IDLE is developed by a graphical user interface (GUI) tool named Tkinter, which is a widget library to develop visual applications independent from any operating system.

Text Editors

A Python application can be developed in a text editor such as Notepad++. They recognize the Python language and shows the code in color. Since they provide normal text editing capabilities, many developers use them because they are accustomed to them for some other languages. Especially developers who work in many languages at the same time may prefer code editors.

IPython

There is an environment called IPython which allows development interactively. People can develop Python code and do some mathematical calculations in this tool as if they work in a math application. They allow developers to run a piece of code and see the results immediately. On the other hand, they may be thought of as too low level, too hard to work for people who are used to advanced development environments. However, an environment which contains IPython as a feature may be preferred.

Pip

One of the most used package managers for the Python language is pip. It is used to download and use libraries. The libraries used by a library to be installed are also recursively downloaded. It can be used as a command line or terminal tool. In addition, it is used as a feature by some advanced development environments. Since there are rich libraries about for everything in Python, using a package manager like pip is very important. For many libraries in Python, installation means a simple call to the pip tool.

Advanced Tools

Advanced tools are explained in this section.

Jupyter Notebook

grafikler4 A tool named Jupyter Notebook based on the IPython environment. It provides something called notebook interface, which is writing code as if writing a document. They look like a paper notebook. They are applications between a development environment and word processor. They contain features from both and they provide a simple environment for people who are not developers. They support some other languages such as R or Julia. They are suitable to produce results by trial-and-error techniques. On the other hand, since they allow codes to be saved, they can be used as a development environment.

Spyder

grafikler5 One of the tools to develop Python code is named Spyder. It contains advanced tools especially for data science and machine learning. Even if the development envrionment doesn't have many advanced features, it is used by academicians and data scientist since it have integrated libraries about the topics they are focussed on. A distribution called Anaconda contains Python and R support. Spyder contains libraries such as Numpy, Scipy, Matplotlib and the IPython interactive environment. In addition, lots of open source libraries come together with Spyder and Anaconda.

PyCharm

grafikler6 There is a tool named PyCharm, which is based on IntelliJ used for Java development and developed by a corporation named JetBrains. Android Studio, which is used for Android development is based on IntelliJ. PyCharm provides advanced features developed for Java to Python developers. Especially for developers who develop in Java and Android, it provides a familiar option. It requires JDK (Java Development Kit).

PyDev

grafikler7 Another Python development environment based on Java is named PyDev. It is based on Eclipse. It is suitable for developers of Java Enterprise Edition. It needs a Java environment to be installed.

Python Tools for Visual Studio

An add-on named Python Tools for Visual Studio can be installed on Visual Studio produced by Microsoft. It can be an easy option for C# developers or others using Visual Studio for JavaScript front-end technologies. Visual Studio can be hard to install since it installs lots of tools for .NET platform. It can change the whole Windows operating system. On the other hand, a lighter version of Visual Studio Code can be easier to install.

The Best Environment

New comers to the Python language may prefer to start with IDLE. Many people whose main job is not development prefers Jupyter. For people focussed on data science and machine learning as long as programming or development, Spyder can be recommended.

For people already developing for Java and Android, it may be easier to start with PyCharm. On the other hand, people developing for Java Enterprise with Eclipse may find PyDev easier to adopt. Python Tools for Visual Studio may be a good recommendation for C# or .NET developers. Starting with an environment such as Spyder and then deciding the environment later may be a good solution.

Libraries & Packages

Python language is preferred by some developers because of its ease and power at the same time. However, in some areas such as data science or artificial intelligence, the reason for its wide-spread use is the richness of libraries. This section gives some information about both standard libraries coming with Python and the ones which can be installed on demand.

General Programming

Python language itself has lots of functions and classes. The support for data structures and algorithms to develop complex applications is extremely abundant. Python helps developers perform complex operations in simple ways.

Data Types

Python language supports basic data types such as integer, float, logical / boolean, string. In addition, it has data structures such as arrays and collections. For example; list, set, dictionary / map is supported.

File & Stream

Python contains many functions to do operations in binary or text data. Moreover, it is possible to use files as a data structure. There are functions about the file system, ZIP and archives, files and streams.

Operating System

There are many functions to access operating systems. For example, logging, processes, threads are supported. Python language has advanced support for security and cryptography. Python has very advanced features for operating systems, especially UNIX and Linux. Before Python is used for high-level areas such as data science and artificial intelligence, it was used for operating systems tasks. That's why its support in these areas is advanced. For this reason, other than developers, people working in systems support and data security prefer Python. In addition, it is a language widely used by hackers and crackers.

Network

Among many things with Python, socket programming for TCP/IP can be done. There is extensive support for lots of protocols. POP3, SMTP and IMAP for email; FTP, HTTP and CGI for web are covered. These tools can be useful for retrieving data for data science.

Data Formats

Python language support for data formats such as CSV, XML and JSON. Most of the data used by data science is exported from many applications and imported to the Python language to feed data science. In addition to data formatas, SOAP using XML and REST using JSON is supported.

User Interface

Python language is not rich in visual development. It is used for back-end rather than front-end. On the other hand, in order to develop user interfaces, there are some tools such as Tkinter. In addition, some libraries such Qt, GTK and wxWidgets can be used for GUI. There are support for tools for multimedia such as audio and video.

Numerical & Data Libraries

There are lots of numerical libraries in Python language.

NumPy

There is a library named NumPy in order to do numerical operations. It supports some mathematical operations other than numbers. NumPy contains most functions which can be used by any branch of science. In other words, NumPy is a library with common functions for areas such as math, physics, chemistry, biology, economy, finance. It contains tools to work with multi-dimensional arrays. In other words, it is used for numbers with a large scale. It has hardware related features.

SciPy

SciPy is a library which contains many features which can be used in many scientific fields. Features such as linear algebra, derivation and integration, statistics, optimization are among them. In addition, topics such as interpolation, extrapolation, imaging, signal processing and fast Fourier analysis are supported. SciPy is found over NumPy. So, installing SciPy installs NumPy indirectly.

Pandas

Pandas is a library to make working with data easier. It supports complex data objects. For one-dimensional objects are named as series while two-dimensional ones are called data frames. It has some features to load or save files or remote resources. It helps developers with erroneous and missing values. In Pandas, data is kept in a format ready for processing and visualization.

Requests

One of the libraries used to download data from the Web using HTTP protocol is named Requests. It presentes a better, easier and more advanced version of support which already exists in Python. It contains some tools to work on data. It is suitable for tools such as a web crawler.

Graphics & Visualisation Libraries

In this section, libraries about graphics and visualization is explained.

Matplotlib

Matplolib is used to draw graphics for data. The library contains basic tools for graph creation. Its purpose is to create a solid foundation for drawing operations. The tool contains support for loading and saving images in addition to drawing.

Seaborn

Seaborn is an advanced graphics library for visualization. It contains features for data representation, especially for statistics. This library is created on Matplotlib. However, unlike Matplotlib, drawing of data is performed automatically. Advanced graphics may be produced by just supplying data and titles. This tool produces graphical results which include graphics, diagrams or charts.

Machine Learning Libraries

In this section, libraries about artificial intelligence, especially its subarea machine learning is explained.

Skit-learn

Skit-learn (Science Kit Learning) or SKLearn for short can be used for machine learning. It contains many algorithms about classification, regression or clustering. Some of the algorithms it supports are Artificial Neural Network, K-Nearest Neighbours, Naive Bayesian Classifier, Decision Tree, Random Forest, K-Means Clustering, Genetic Algorithms. This library is based on SciPy and NumPy. So, in order to use it, the libraries mentioned should be used.

TensorFlow

A library named TensorFlow is developed by Google for data flow operations. It supports many features about machine learning, especially artificial neural networks. Other than scientific operations, it has many features for big data and distributed computing.

Package Installer - pip

A tool named pip is a package installer and can be used to install a library or package to a machine. Even if there are some other tools for package management, most of them are used similar to pip.

In order to use pip, it must be installed before as a Python library. Many versions of Python environments install pip together with Python interpreter.

Installation

In order to install pip, a Python script named get-pip.py is downloaded. Any application can be used to download. It can be downloaded by a tool named curl as below:

curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py

The same operations can be performed in a browser by just clicking a link. However, the method explained here must be known because servers in a production environment may not have any visual environment where browsers can run.

After the pip installer is installed, it is executed as a normal Python script as below:

python get-pip.py

This installs the pip tool.

Upgrade

In some cases, last version of pip is required. For example, a library may need a new version to be installed by pip. For this, an upgrade operation is performed. The upgrade is denoted by -U option.

Upgrade can be done in two ways. First, it may be required from pip to upgrade itself. Example:

pip install -U pip

Because the pip can install verything, it can install itself.

An alternative way is requesting an upgrade from the Python interpreter. Example:

python -m pip install -U pip

In the first option, pip reinstalls itself while Python interpreter reinstalls pip in the second option.

Package Installation

In order to install a library or package to Python, the pip tool can be used. For example, a call as below:

pip install mypackage

can install the package given as the last argument. The command will download the files of libraries to the local machine and install it from the local file. In other words, the method above installs a package on-line. When a library is installed, all the libraries it is dependent on are also installed.

Package, Library, Module

Although terms package, library or module have different meanings, it may be used sometimes as synonymous. A module is a Python file used by an application or another module. A package is a module which is installed by a tool. So, the modules developed by a developer and kept in the near directories to the application are not called packages. A package is typically a directory and a module is typically a file. A library may contain more than a few packages and may have more components than a simple package such as native files. A package is used in a Python script by an import statement. A library can contain many packages that should be imported separately. API (Application Programming Interface) is used in some cases for a library but an API is publicly accessible face of a library. In other words, a library may have lots of code which cannot