Intermediate

Manipulating and retrieving emails

on the server email using IMAP with
imapclient and imaplib
In this section, we will learn about the IMAP protocol and explore the imapclient
and imaplib modules for working with emails with IMAP in Python 3.7.
IMAP protocol
The IMAP protocol does not download messages to your computer—both the
messages and the folders that we have created are kept on the server.

The IMAP protocol is the most advisable when we access our emails from
various devices, or when we are mobile. As a precaution, we must periodically
delete the contents of our account so that it does not exceed the space that's
granted. The drawback of this protocol is that we must always have an internet
connection, even to access and work with old messages.
This protocol has the advantage that, when we connect to read our emails from different
devices, for example, our laptop or smartphone, we know that we can always access all of our
messages, and that the mailbox will be updated. It is also interesting to preserve our privacy
when we read our emails from a public or shared computer, as it does not store information on
the local machine.

For starters, like POP, this protocol is only intended to read emails, not to send
them. The main advantage over this is that you are also prepared to manage
them: being able to organize them in folders or search in the server are inherent
capabilities of the protocol.

Another differential aspect is the architecture that's designed to be accessed from

different computers while keeping copies of our emails synchronized. If, in POP,
we said that the common thing was to erase the messages as we downloaded
them, in IMAP, those messages are kept on the server until we request their
deletion explicitly.

This distributed synchronization is based on the UID that represents a unique

identifier for a given message sequence number, which allows several clients to
access it simultaneously and understand what messages they are manipulating.
To round off this distributed support, clients can access any of the following
connection modes:

Offline mode: It periodically connects to the server to download new

messages and synchronize any changes that may have happened in the
different folders. We have the ability to delete the messages as we
 download them, following a function that's very similar to POP3.
Online mode: It has access to the copy of the server messages exactly
when we need to, synchronizing the changes practically on the fly.
Disconnected mode: Do not confuse this with offline mode. In this case,
the client works with a local copy while they do not have access to the
internet, where they can create/delete/read their emails. The next time you
connect to the internet, these changes will be synchronized with the master
copy of the server.

Since it is based on a model in which messages are normally stored on the server
after being read, IMAP defines an easy way to manage them—with mail trays,
that is, with folders. These follow a tree-like hierarchy, which we are used to in
conventional filesystems. Following the standard we always have, the inbox will
be the main source, but we can create other folders with different attributes. For
example, there are attributes to specify that a folder contains only emails,
(\Noinferiors), or only folders, (\Noselect), but they can also have other attributes
that indicate whether or not new messages exist since the last time we opened it
with (\Marked) and (\Unmarked).

A similar kind of label can have the emails we receive or send. One of the most
used is the one that indicates whether it has been read or not (\Seen), but there are
also others that indicate that the message has been answered (\Answered), that the
message has been highlighted (\Flagged), which is a draft (\ Draft), and so on. All
of this information is saved directly on the server and not on the client as we are
used to, which allows you to perfectly synchronize this metadata between
several clients.

Technically, at a low level, IMAP works very similarly to POP3—a connection

is opened to port 143 of the server, and a conversation begins in ASCII.
Following the custom, Gmail uses another port 993, which is the alternative port
of IMAP if we want the connection to be encrypted under SSL. Once that
connection is created, the client starts sending commands and receiving
responses.

On an IMAP server, email messages are grouped into folders, some of which
will come predefined by an IMAP provider. Once a folder has been selected,
messages can be listed and fetched. Instead of having to download every
message, the client can ask for particular information from a message, such as a
few headers and its message structure, to build a display or summary for the user
to click on, hence pulling message parts and downloading attachments from the
server on demand.
Retrieving emails with imaplib
As we mentioned earlier, accessing emails over the IMAP protocol doesn't
necessarily download them onto the local device.

Python provides a library called imaplib, which can be used for accessing
messages over the IMAP protocol. This library provides the IMAP4() class, which
takes the host and port for implementing this protocol as arguments. The default
port is 143.

The IMAP4_SSL() class has the capacity to connect over an SSL encrypted socket
and provides a secure version of the IMAP4 protocol by using 993 as the default
port.

A typical example of what an IMAP client looks like can be seen here:
mailbox = imaplib.IMAP4_SSL("IMAP_SERVER", "SERVER_PORT")
mailbox.login('username', 'password')
mailbox.select('Inbox')

The previous code will try to initiate an IMAP4 encrypted client session. After
the login() method is successful, you can apply the various methods on the
created object. In the previous code snippet, the select() method has been used.
This will select a user's mailbox. The default mailbox is called inbox.
A full list of methods that are supported by this mailbox object is available on the Python
standard library documentation page, which can be found at https://docs.python.org/3/library/imaplib
.html.

Here, we would like to demonstrate how you can search the mailbox by using
the search() method. It accepts a character set and search criterion parameter. The
character set parameter can be None, where a request for no specific character will
be sent to the server. However, at least one criterion needs to be specified. For
performing an advanced search for sorting the messages, you can use the sort()
method.

We can use a secure IMAP connection for connecting to the server by using the
IMAP4_SSL() class.
If you are using a Gmail account and want to store all of your emails messages
in your Gmail Sent folder, go to the Forwarding and POP/IMAP tab and enable
IMAP.

In the following screenshot, we can see the Gmail configuration for the IMAP
protocol:

You can find the following code in the check_remote_email_imaplib.py file:

#!/usr/bin/env python3

import argparse
import imaplib

def check_email(username,password):
 mailbox = imaplib.IMAP4_SSL('imap.gmail.com', '993')
mailbox.login(username, password)
mailbox.select('Inbox')
type, data = mailbox.search(None, 'ALL')
for num in data[0].split():
type, data = mailbox.fetch(num, '(RFC822)')
print ('Message %s\n%s\n' % (num, data[0][1]))
mailbox.close()
mailbox.logout()

In the previous code block we define check_email() method that establish the
connection with imap gmail server with username and password parameters,
select the inbox for recover messages and search for specific RFC number
protocol inside the mailbox. In the next code block we define our main program
that request information about username and password used for establish the
connection.
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Email Download IMAP')
parser.add_argument('--username', action="store", dest="username")
given_args = parser.parse_args()
username = given_args.username
import getpass
password = getpass.getpass(prompt='Enter your password:')
check_email(username, password)

In this example, an instance of IMPA4_SSL(), that is, the mailbox object, has been
created. Here, we have taken the server address and port as arguments. Upon
successfully logging in with the login() method, you can use the select() method
to choose the mailbox folder that you want to access. In this example, the inbox
folder has been selected. To read the messages, we need to request the data from
the inbox. One way to do that is by using the search() method. Upon successful
reception of some email metadata, we can use the fetch() method to retrieve the
email message envelope part and data. In this example, the RFC 822 type of
standard text message has been sought with the help of the fetch() method.

We can use the Python pretty print or the print module for showing the output on
the screen. Finally, apply the close() and the logout() methods to the mailbox
object.
Retrieving emails with imapclient
IMAPClient is a complete IMAP client library written in Python that uses the
imaplib module from the Python standard library. It provides an API for creating a

connection and reads messages from the inbox folder.

You can install imapclient with the following command:

$ pip install imapclient

The IMAPClient class is the core of the IMAPClient API. You can create a
connection to an IMAP account by instantiating this class and interacting with
the server calling methods on the IMAPClient instance.

The following script shows how to interact with an IMAP server, displaying all
of the messages in the inbox folder and the information related to the message
ID, subject, and date of the message.

You can find the following code in the folder_info_imapclient.py file:

#!/usr/bin/env python3

from imapclient import IMAPClient

import getpass

username = input('Enter your username:')

password = getpass.getpass(prompt='Enter your password:')

server = IMAPClient('imap.gmail.com', ssl=True)

server.login(username, password)
select_info = server.select_folder('INBOX',readonly=True)
for k, v in list(select_info.items()):
print('%s: %r' % (k, v))

server.logout()

In this script, we open an IMAP connection with the IMAPClient and get
information about its capabilities and mailboxes.

You can find the following code in the listing_mailbox_imapclient.py file:

#!/usr/bin/env python3

import sys
from imapclient import IMAPClient
 import getpass

username = input('Enter your username:')

password = getpass.getpass(prompt='Enter your password:')

server = IMAPClient('imap.gmail.com', ssl=True)

try:
server.login('user', 'password')
except server.Error as e:
print('Could not log in:', e)
sys.exit(1)

print('Capabilities:', server.capabilities())
print('Listing mailboxes:')
data = server.list_folders()
for flags, delimiter, folder_name in data:
print(' %-30s%s %s' % (' '.join(str(flags)), delimiter, folder_name))

server.logout()

This could be the output of the previous script, where we can see capabilities and
mailboxes that are available in your Gmail account:
Capabilities: ('UNSELECT', 'IDLE', 'NAMESPACE', 'QUOTA', 'XLIST','AUTH=XOAUTH')
Listing mailboxes:
\Noselect \HasChildren / [Gmail]
\HasChildren \HasNoChildren / [Gmail]/All Mail
\HasNoChildren / [Gmail]/Drafts
\HasChildren \HasNoChildren / [Gmail]/Sent Mail
\HasNoChildren / [Gmail]/Spam
\HasNoChildren / [Gmail]/Starred
\HasChildren \HasNoChildren / [Gmail]/Trash

In this section we have reviewed the imapclient and imaplib modules which provide
the methods can for accessing emails with IMAP protocol.