So how does email actually work?

In this post, I'm going to explore one of the fundamental technologies of the Internet: email. Personally, I've always wondered how email clients work, but I never knew where to start.

There's a lot to unpack:

• How do you manage multiple email accounts?
• What is actually receiving the mail?
• How do all email clients receive the same information?
• How does sending mail work?

In this post, I'll be answering some of those questions. No experience with network programming is necessary for reading this post!

The Basics

I'm going to start from the beginning: you receive an email.¹ The first thing to find out is what received it, and where.

Let's say your email address is hello@icloud.com. The @icloud.com means it's managed by iCloud Mail. iCloud Mail is what's called an email server. It holds all the emails of all @icloud.com email addresses. It's this server that contains the email that was sent to hello@icloud.com (and literally everyone else's)!²

Connecting

So now we know that our email is stored on a remote server that iCloud Mail sets up. Like pretty much everything else, this server conforms to the TCP protocol, so we should be able to connect to it remotely. I'll show how to do so using OpenSSL and the command line.³

$ openssl s_client -connect imap.mail.me.com:993
# A bunch of output telling us that the connection was successful (hopefully)

We're telling OpenSSL that we want to connect to the TCP server located at imap.mail.me.com:993. This is the standard FQDN (fully qualified domain name) that iCloud Mail uses (note that it is not specific to me, in some way).⁴ That is, every email client that wants to get email data from an iCloud account uses this FQDN, which resolves to an IP address via a DNS lookup.⁵ With the IP address, we can connect to the server.

Getting Information

Ok, so now we're connected to the email server. We need to send it messages to request information about our emails. The question becomes: how? They key lies in the name: imap.mail.me.com:993. The magic word is IMAP!

IMAP is a protocol that tells us two things:⁶

1. How to structure the messages to get the information we want
2. What the response will look like

Using our OpenSSL client, we can compose messages by typing in some stuff, and we can press enter to send it.⁷ But you'll likely get a response back telling you that the message you sent was malformed in some way. That's because it didn't conform to the IMAP protocol! Let's look at some messages that do conform to the protocol.

Logging In

In order to access any data, we need to log in to our email account to make sure the email server knows who we are. The IMAP protocol tells us how to compose a message so that the server knows that we want to log in:

C: a1 LOGIN {email} {password}
a1 OK user {email} logged in

Here I'm using C to denote the client message (so if you're trying this at home, omit the C:), and lines without C to denote server output.

If we send that message (substituting in our actual email and password), we'll be logged in and should be able to access our emails by sending more IMAP-compliant messages.

Retrieving Emails

Now we're ready to look at some messages. The first thing we need to do is select a folder. An IMAP folder contains emails. Examples of common folders are:

• INBOX
• Trash
• Spam
• Drafts

This is how you select a folder:

C: g21 SELECT INBOX
* 1 EXISTS
* 0 RECENT
* OK [UNSEEN 1]
* OK [UIDVALIDITY 1451405933]
* OK [UIDNEXT 2272]
* FLAGS (\Answered \Flagged \Draft \Deleted \Seen \Recent $MailFlagBit0 $MailFlagBit1 $MailFlagBit2 $Forwarded Redirected $NotJunk NotJunk)
* OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen \Recent $MailFlagBit0 $MailFlagBit1 $MailFlagBit2 $Forwarded Redirected $NotJunk NotJunk \*)]
g21 OK [READ-WRITE] SELECT completed (took 2 ms)

The main output we're looking for is the first line. It tells us that there's one email in our INBOX folder. Let's take a look at this email:

C: f1 FETCH 1 RFC822.TEXT
* 1 FETCH (RFC822.TEXT {242}
--00000000000055129906262bbfde
Content-Type: text/plain; charset="UTF-8"

Hello, world!

--00000000000055129906262bbfde
Content-Type: text/html; charset="UTF-8"

<div dir="ltr">Hello, world!</div>

--00000000000055129906262bbfde--
)
f1 OK FETCH completed (took 153 ms)

Note that the output comes in two different forms: HTML and plain text.

The FETCH command is very versatile; it's the primary tool email clients use to retrieve the contents of your emails.

Summary

Here are they key takeaways from our exploration of email:

• The domain part of an email designates a remote server to manage that account
• An email client connects to email servers to retrieve information using a protocol
• The most widely used protocol for email servers is IMAP

In future posts, I'll take a dive into sending emails using the SMTP protocol. I hope you enjoyed this post! If you found any problems or have any suggestions, make sure to open an issue on this website's GitHub repository.

References

1. IMAP Specification
2. Wikipedia
3. Apple Support
4. atmail

I recommend reading through some of these sources to get a more thorough understanding of the concepts in this blog post.