Intermediate

Understanding the FTP protocol for

transferring files
In this section, you will be introduced to the FTP protocol for transferring files
and the ftplib package for interacting with Python.
The File Transfer Protocol
The File Transfer Protocol (FTP) protocol allows us to make file transfers
through a connection in a network. This is the protocol that we use to connect
remotely to servers and manipulate files. Port 21 is usually used.

The protocol design is defined in such a way that it is not necessary for the
client and server to run on the same platform; any client and any FTP server can
use a different operating system and use the primitives and commands defined in
the protocol to transfer files.

To interact with this protocol, we need two things. The first is a server that is
available for our network—it can be on the same network or maybe on the
internet. The second is a client that can send and receive information from said
server; this client must have the capacity to be able to use the ports specified by
the service and the established authentication.
Introduction to ftplib
Unlike SFTP, FTP uses the plaintext file transfer method. This means any
username or password transferred through the wire can be detected by an
unrelated third party. Even though FTP is a very popular file transfer protocol,
people frequently use this to transfer a file from their PCs to remote servers.

FTPlib is a Python library that will allow us to connection to an FTP server from a
script. To begin, we must have installed Python in our operating system and the
FTPLib package. We can install them on a Linux system in two ways:

pip install ftplib

apt-get install python-ftplib

In Python, ftplib is a built-in module that's used to transfer files to and from the
remote machines. You can create an anonymous FTP client connection with the
FTP() class:

ftp_client = ftplib.FTP(path, username, email)

Then, you can invoke the normal FTP commands, such as the CWD command, to
list the files in a specific directory. To download a binary file, you need to create
a file handler, such as the following:
file_handler = open(DOWNLOAD_FILE_NAME, 'wb')

To retrieve the binary file from the remote host, the syntax shown here can be
used, along with the RETR command:
ftp_client.retrbinary('RETR remote_file_name', file_handler.write)

In the following script, we are trying to connect to the FTP server, ftp.free.fr, to
get get a list of directories with the dir() method, and download a specific file on
that server. To download a file through the ftplib libraries, we will use the
retrbinary method. We need to pass two things to it as an input parameter: the retr
command with the name of the file and a callback function that will be executed
every time a block of data is received. In this case it will write it in a file of the
same name.
You can find the following code in the ftp_download_file.py file:
!/usr/bin/env python3

import ftplib

FTP_SERVER_URL = 'ftp.free.fr'
DOWNLOAD_DIR_PATH = '/mirrors/ftp.kernel.org/linux/kernel/Historic/'
DOWNLOAD_FILE_NAME = 'linux-0.01.tar.gz'

def ftp_file_download(path, username):

# open ftp connection
ftp_client = ftplib.FTP(path, username)
print("Welcome:", ftp_client.getwelcome())
# list the files in the download directory
ftp_client.cwd(DOWNLOAD_DIR_PATH)
print("Current working directory:", ftp_client.pwd())
print("File list at %s:" %path)
files = ftp_client.dir()
print(files)
# download a file
try:
file_handler = open(DOWNLOAD_FILE_NAME, 'wb')
ftp_cmd = 'RETR %s' %DOWNLOAD_FILE_NAME
ftp_client.retrbinary(ftp_cmd,file_handler.write)
file_handler.close()
ftp_client.quit()
except Exception as exception:
print('File could not be downloaded:',exception)

if __name__ == '__main__':
ftp_file_download(path=FTP_SERVER_URL,username='anonymous')

The preceding code illustrates how an anonymous FTP can be downloaded from
ftp.free.fr, which hosts the first Linux kernel version. The FTP() class takes three
arguments, such as the initial filesystem path on the remote server, the username,
and the email address of the ftp user. The FTP.cwd() function is used to change the
directory or folder (change the working directory). In this case, after accessing as
an anonymous user, change the location to the kernel/Historic folder.

For anonymous downloads, no username and password is required. So, the script
can be downloaded from the linux-0.01.tar.gz file, which can be found on the
/mirrors/ftp.kernel.org/linux/kernel/Historic/ path.

In the following screenshot, we can see the execution of the previous script:
Another way to get information about the files and folders in the current location
is to use the retrlines() method, which can indicate the commands to execute.
LIST is a command that's defined by the protocol, as well as others that can also

be applied in this function as RETR, NLST, or MLSD.

For more information on these commands, see RFC 959: http://tools.ietf.org/html/rfc959.html.

The second parameter is the callback function, which is called for each piece of
received data:
def callback(info):
print info
...
ftp.retrlines('LIST', callback)

In this example, instead of using the ntransfercmd() method to apply a RETR

command, we receive data in a byte array. We execute the RETR command to
download the file in binary mode.

You can find the following code in the ftp_download_file_bytes.py file:

#!/usr/bin/env python3

import os, sys

from ftplib import FTP

f = FTP('ftp.free.fr')
f.login()

f.cwd('/mirrors/ftp.kernel.org/linux/kernel/Historic/')
f.voidcmd("TYPE I")

datasock, size = f.ntransfercmd("RETR linux-0.01.tar.gz")

bytes_so_far = 0
fd = open('linux-0.01.tar.gz', 'wb')

while 1:
buf = datasock.recv(2048)
if not buf:
 break
fd.write(buf)
bytes_so_far += len(buf)
print("\rReceived", bytes_so_far, end=' ')
if size:
print("of %d total bytes (%.1f%%)" % (
size, 100 * bytes_so_far / float(size)),end=' ')
else:
print("bytes", end=' ')
sys.stdout.flush()

print()
fd.close()
datasock.close()
f.voidresp()
f.quit()

In this example, we are going to list versions that are available in the Linux
kernel ftp with the dir() method.

You can find the following code in the list_kernel_versions.py file:

#!/usr/bin/env python3

from ftplib import FTP

entries = []
f = FTP('ftp.free.fr')
f.login()
f.cwd('/mirrors/ftp.kernel.org/linux/kernel/')
f.dir(entries.append)
print("%d entries:" % len(entries))
for entry in entries:
print(entry)
f.quit()

In the following screenshot, we can see the execution of the previous script:
Other ftplib functions
These are the main ftplib functions we can use to execute ftp operations:

FTP.getwelcome(): Gets the welcome message

FTP.mkd(route): Creates a directory; it is passed as an input argument to the
route
FTP.rmd(path): Deletes the directory that we pass
FTP.delete(file): Deletes the file that we passed as an input parameter
FTP.pwd(): (Print Working Directory) Returns the current directory where it is
located
FTP.cwd(path): (Change Working Directory) Changes directory
FTP.dir(path): Returns a list of directories
FTP.nlst(path): Returns a list with the file names of the directory
FTP.size(file): Returns the size of the file we passed to it

In this example, we are going to list the versions that are available in the Linux
kernel FTP with the nlst() method.

You can find the following code in the list_kernel_versions_nslt.py file:

!/usr/bin/env python3

from ftplib import FTP

f = FTP('ftp.free.fr')
f.login()
f.cwd('/mirrors/ftp.kernel.org/linux/kernel/')
entries = f.nlst()
entries.sort()
print(len(entries), "entries:")
for entry in entries:
print(entry)
f.quit()
Inspecting FTP packets with
Wireshark
If we capture the FTP session in Wireshark on port 21 of the public network
interface, we can see how the communication happens in plaintext. In the
following example, we can see that after successfully establishing a connection
with a client, the server sends the 230 Welcome to mirror.as35701.net banner message.
Following this, the client will anonymously send a request for login.

In this example, we are using the ftplib module to build a script to determine
whether a server offers anonymous logins.

You can find the following code in the checkFTPanonymousLogin.py file:

import ftplib

def ftpListDirectory(ftp):
try:
dirList = ftp.nlst()
print(dirList)
except:
dirList = []
print('[-] Could not list directory contents.')
print('[-] Skipping To Next Target.')
return
retList = []
for fileName in dirList:
fn = fileName.lower()
if '.php' in fn or '.htm' in fn or '.asp' in fn:
print('[+] Found default page: ' + fileName)
retList.append(fileName)
return retList

def anonymousLogin(hostname):
try:
ftp = ftplib.FTP(hostname)
ftp.login('anonymous', '')
print(ftp.getwelcome())
ftp.set_pasv(1)
print(ftp.dir())
print('\n[*] ' + str(hostname) +' FTP Anonymous Logon Succeeded.')
return ftp
except Exception as e:
print(str(e))
print('\n[-] ' + str(hostname) +' FTP Anonymous Logon Failed.')
return False

host = 'ftp.be.debian.org'
ftp = anonymousLogin(host)
 ftpListDirectory(ftp)

The anonymousLogin() function takes a hostname and returns a Boolean that

describes the availability of anonymous logins. This function tries to create an
FTP connection with anonymous credentials. If successful, it returns
the True value.

In the following screenshot, we can see an example of executing the previous

script over a server that allows anonymous login:

In the following screenshot, we can see packets that are exchanged in the ftp
communication:

In the following screenshot, we can see packets and the request command for
listing files in the ftp server: