What is IMAP?

A brief history of IMAP

IMAP was originally created by Mark Crispin at Stanford in the 1980s. Crispin later moved to the University of Washington and spent 20 years there working on the IMAP specs and reference implementation. The IMAP specification comes in the form of a “Request for Comments,” or RFC, which is basically a memo describing how to implement the protocol that has been adopted as a standard by the Internet Engineering Task Force. RFCs may be revised to make clarifications or changes, and the current RFC that describes IMAP is RFC3501, which was published in 2003. That’s right: the protocol most mail clients use to sync email is over 15 years old!

IMAP was initially designed as a better alternative to an older protocol called POP, with the goal being “to permit manipulation of remote mailboxes as if they were local” — specifically, POP’s mode of operation was to download mail to your local machine and delete it from the server, which made it difficult to manage your mail if you were using multiple computers. This was no problem in the days of timesharing mainframes but became one as personal computers became more prevalent, and you might have a PC at home and work and wanted to browse your email from either place.

The original version of IMAP is lost to history, and IMAP2 (released in 1988) is the first version that made it into the standards. This first version of IMAP only supported “online” operation: it assumed that your mail client would be connected to the server when viewing or modifying messages. You couldn’t use it to sync a copy of your mailbox locally and then update your local copy when you reconnected because messages were only identified by a “sequence” number, and sequence numbers were not persistent across client sessions. One can only imagine that in those days, folks figured that it’d always be the case that whenever you wanted to check your email, you’d be sure to have a reliable network connection. My guess is that it was because computing from mainframes and timesharing systems was essentially logging directly into a server machine, and meant you had an always-on Internet connection at your disposal.

Fast forward a few years to the dial-up era, and it became apparent that hogging your phone line constantly so you could read your email just wasn’t going to cut it. Enter support for “disconnected” operations, which was bolted on to IMAP with the introduction of IMAP4 in 1994. This revision to the protocol added persistent message identifiers (called “UIDs”) and an entire draft spec explaining how you could sync changes made to locally cached data to data stored on a server using IMAP commands. This evolution of the protocol over time is a major reason that implementing IMAP clients is complex in the modern era, as disconnected operation is a required feature for modern apps.

The basic IMAP protocol has remained unchanged since 1996, with all new features since then being implemented in optional extensions to the protocol, some of which have become standards — but adoption of even the most common extensions still varies wildly across email service providers.

When do you need to use IMAP?

Emails serve as a critical link for both personal communication and business operations. Your choice of email protocol can significantly influence how smoothly and effectively these interactions occur. 

IMAP stands out for several reasons, including: 

• Multi-device access: One of the main advantages of IMAP is its ability to synchronize email data across multiple devices seamlessly. Whether you’re an individual checking emails on a smartphone, tablet, and laptop or a business ensuring your workforce can access corporate communication on their office desktops and mobile devices while traveling, IMAP ensures that all messages (read/unread/status) are consistent across all platforms.
• Server-side email management: The centralized, server-based approach of IMAP benefits users who don’t want to download every email to every device they use. It’s especially valuable for businesses requiring a centralized email storage system, allowing for consistent backup, easier administrative controls, and better security measures.
• Real-time synchronization: For businesses, real-time synchronization can mean timely responses to client queries or quick decision-making based on the latest data. For individuals, it ensures they’re always up-to-date, whether switching between devices or checking their emails from different locations.

With an IMAP integration, businesses can offer their users a unified and synchronized email experience, regardless of the device or platform. This integration ensures that users can access the most recent and accurate email data, especially for telehealth, marketplace, recruiting, and other applications that facilitate direct communication, real-time collaboration, and timely decision-making based on up-to-date information.

Integrating IMAP into your application can facilitate features like email search, categorization, and advanced filtering, enhancing an application’s functionality. From a security standpoint, IMAP’s server-side management means that sensitive emails remain on central servers, allowing businesses to implement stringent security protocols and backup strategies. 

Components of IMAP

IMAP has various components that streamline and enhance email management and retrieval capabilities. Let’s delve into the primary and additional components of IMAP. 

Primary components of IMAP 

• Server — The IMAP server stores email messages and allows them to be retrieved by email clients using the IMAP protocol.
• Client — The IMAP client interacts with the server, allowing users to read, delete, or mark their emails as read/unread, reflecting the changes simultaneously across all mail clients.

Additional components of IMAP

• Folders/labels: IMAP allows users to organize their emails into folders (or labels in some email services). This structure is maintained across devices, allowing for better email organization and management.
• Flags/status indicators: These indicate the status of a message, such as whether it’s been read, replied to, flagged for follow-up, etc. They help users and email clients track the state and priority of emails.
• Search capability: IMAP servers can search through emails based on various criteria, allowing clients to fetch specific emails without downloading all messages.
• Authentication: An essential component of IMAP is the authentication mechanism, which ensures only authorized users can access their emails. This often involves username/password combinations but can also include more advanced security methods.
• Encryption/security: Modern IMAP servers often use encryption methods like SSL/TLS to ensure that email data is transferred securely between the client and server.

How does IMAP work?

IMAP operates through a series of commands and responses between the email client and server, ensuring that email data is synchronized and consistent across all devices. 

To better understand how IMAP works, let’s look at connecting to an email account, listing all of the available folders in the mailbox, and downloading a message. Along the way, we’ll identify and explain the relevant parts of the protocol needed to understand what is happening.

You can follow along as we walk through the protocol if you can access a terminal, the openssl command-line utility, and an email account that supports IMAP (including Gmail).

TCP, HTTP & IMAP

Many modern client-server protocols occur between a web browser and a server or an app and an API. These APIs are all implemented on top of a base protocol, HTTP (“hypertext transfer protocol”), which defines the semantics of different types of requests — GET, PUT, POST, etc. HTTP is, in turn, implemented on top of a lower-level protocol called TCP (“transmission control protocol”), which simply guarantees that packets of information are delivered reliably to the destination.

IMAP, like HTTP, is implemented on top of TCP — and it defines the semantics of different types of requests called IMAP commands. IMAP is generally much more concise than HTTP, which was key for IMAP in the early 90s as network connections were very low-bandwidth.

Make IMAP Integration Simple. 

Use the robust Nylas Email API to build engaging email capabilities. 

Build for free

IMAP step-by-step

Let’s get started! Follow along at home in your terminal by typing the commands and subbing in your email account details where relevant. Note that if you’re using Gmail, you may need to disable protection against logins from “less secure apps” for this to work.

The first thing any client needs to do is connect to the remote server on a specific port. These ports can vary because mail servers first accepted plaintext connections and later added support for secure encrypted connections. You can generally find the settings for your provider by checking their help articles. For example, here’s where to find the settings for Gmail. (If you’re curious about the background of why different ports are used, we recommend this fantastic guide from Fastmail.)

Have your settings to make a secure connection? Let’s go!

On the command line, we can connect using the openssl command-line utility. We have to use openssl rather than telnet because otherwise, we could send sensitive data like passwords and private emails in plaintext over the network, leaving them open to network sniffing by an attacker. openssl protects our connection by making a secure connection and verifying the remote server’s certificate before proceeding.

$ openssl s_client -connect imap.gmail.com:993 -crlf [... certificate verification removed for brevity ...]
* OK Gimap ready for requests from 216.38.147.18 h1mb556258870pjq

In the above command, s_client means we’re invoking openssl’s functionality to act as a basic SSL/TLS client, -connect specifies which hostname and port to connect to, and -crlf converts when you press “Enter” in your terminal to the characters expected by the IMAP server to properly end a line. After connecting, openssl checks the SSL/TLS certificate on the server to ensure our connection isn’t being hijacked, and the IMAP server displays a greeting that says it is ready to receive requests from the client.

Commands

IMAP commands generally look like this:

<tag> <command> [<arg1><arg2>…]

We’ll explain these parts in the context of an example command called CAPABILITY. CAPABILITY allows you to ask a server which IMAP extensions it supports:

tag1 CAPABILITY 
* CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 XYZZY SASL-IR AUTH=XOAUTH2 AUTH=PLAIN AUTH=PLAIN-CLIENTTOKEN AUTH=OAUTHBEARER AUTH=XOAUTH 
tag1 OK that's all she wrote! h1mb556258870pjq

Wahoo, we just communicated with an IMAP server! Let’s explain what we sent and what we got back.

A command is similar to a request in HTTP. It’s telling the server to do something or asking it for information. In this case, we ask the server to tell us which capabilities it supports. More on capabilities in just a bit.

The CAPABILITY command has no arguments, but if it did, we’d simply type them in after the command name and before we press enter.

Tagging

Tags (tag1 in this example) are generated by the client, and the server sends the tag back on the final line of the response to a command. Tags can contain any alphanumeric characters and even some symbols and don’t need to contain an ascending integer — though that can be a convenient way to generate unique ones. Server responses that start with * are called untagged responses, meaning they don’t represent the completion of a command requested by the client. In this case, the untagged response is the capability list, and the tagged response is the OK status of the command and a funny message left by the server programmers.

This tagging ability means that it’s possible for a server to handle more than one request at the same time from a client on the same connection and to indicate their completion by sending back the appropriate tag. In practice, many clients don’t support this ability to send concurrent requests and simply block waiting for data to arrive on the open socket after a command is sent.

Capabilities

So what does this list of “capabilities” mean, anyway? A “capability” is a short name for a feature the server supports. Some of these capabilities, like AUTH=PLAIN, are included in the base IMAP4rev1 spec, and others are enabled by extensions. Extensions allow new commands to be added to the IMAP protocol without requiring a new version of the protocol specification. Before logging in, the list of supported capabilities on a server will be different from those after logging in, as there’s little use in listing capabilities that aren’t relevant to the logged-out or logged-in states.

Authentication

Here’s how you authenticate your mail account to an IMAP server:

tag2 LOGIN spang@nylas.com 
tag2 NO [AUTHENTICATIONFAILED] Invalid credentials (Failure)

Server responses

As you can see, server responses don’t always say OK. Here’s a quick guide to the allowed server responses to a command:

OK = success!

NO = failure!

BAD = I didn’t understand what you said!

The server is free to add a bit more information after the response if it so desires, like [AUTHENTICATIONFAILED] Invalid Credentials (Failure).

tag3 LOGIN spang@nylas.com 
* CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 UIDPLUS COMPRESS=DEFLATE ENABLE MOVE CONDSTORE ESEARCH UTF8=ACCEPT LIST-EXTENDED LIST-STATUS LITERAL- SPECIAL-USE APPENDLIMIT=35651584 
tag3 OK spang@nylas.com authenticated (Success)

As you can see, after a successful login, the server sends the client the list of capabilities again, showing what new capabilities have been unlocked due to the login and which old ones have gone away. Some standard extensions in this list are UIDPLUS, MOVE, and CONDSTORE. Some custom extensions supported by Gmail but not elsewhere are ESEARCH and X-GM-EXT-1.

Listing folders

OK, now that we’re logged in, let’s try to do something useful. First, let’s check which folders exist in the mailbox:

tag4 LIST 
tag4 BAD Could not parse command

Whoops! As you can see, if you get the arguments wrong to a command, the server will reject it and say BAD client!

tag5 LIST "" "*" 
* LIST (HasNoChildren) "/" "Call log" 
* LIST (HasNoChildren) "/" "INBOX" 
* LIST (HasChildren Noselect) "/" "[Gmail]" 
* LIST (All HasNoChildren) "/" "[Gmail]/All Mail" 
* LIST (Drafts HasNoChildren) "/" "[Gmail]/Drafts" 
* LIST (HasNoChildren Important) "/" "[Gmail]/Important" 
* LIST (HasNoChildren Sent) "/" "[Gmail]/Sent Mail" 
* LIST (HasNoChildren Junk) "/" "[Gmail]/Spam" 
* LIST (Flagged HasNoChildren) "/" "[Gmail]/Starred" 
* LIST (HasNoChildren Trash) "/" "[Gmail]/Trash" 
* LIST (HasNoChildren) "/" "alabel" 
tag5 OK Success

Folder flags

Here, we can see the folder hierarchy that exists in the mailbox. Folders have special flags indicated in parentheses. Some of these flags are useful for traversing the folder hierarchy, like HasNoChildren and HasChilden, while others are used to denote special characteristics of a given folder, like if it contains drafts (Drafts) or sent messages (Sent). These flags allow clients to tailor their behavior to the mailbox and do things like save sent messages to the right folder no matter which language the user’s mailbox is in.

Sessions

OK, let’s read some emails now. First, because IMAP connections are stateful, we need to open a session on the folder that we care about using the SELECT command:

tag6 SELECT "INBOX" 
* FLAGS (Answered Flagged Draft Deleted Seen $NotPhishing $Phishing) 
* OK [PERMANENTFLAGS (Answered Flagged Draft Deleted Seen $NotPhishing $Phishing *)] Flags permitted. 
* OK [UIDVALIDITY 1] UIDs valid. 
* 55 EXISTS 
* 0 RECENT 
* OK [UIDNEXT 54948] Predicted next UID. 
* OK [HIGHESTMODSEQ 11865059] 
tag6 OK [READ-WRITE] INBOX selected. (Success)

There’s a lot of information the server sends us here. Sessions are a key component of how IMAP works, and a session lasts from when you SELECT a mailbox until you UNSELECT it, SELECT another mailbox, or log out.

When opening a session, this server has sent us, in addition to the OK indicating success:

• a list of active flags in use on messages in the mailbox
• a list of supported flags that could be used on messages in the mailbox
• the UIDVALIDITY – more on this in a bit
• the number of messages in the mailbox
• how many “recent” messages are in the mailbox, which means how many messages are tagged with the Recent flag
• the “predicted next UID”
• a HIGHESTMODSEQ value – more on this in a bit

A few of these items are going to require a bit more explanation. First, UIDs.

UIDs & sequence numbers

A reminder: a long time ago, there was IMAP2, and in IMAP2, messages were identified by sequence numbers. Sequence numbers were literally just that: monotonically increasing numbers, one for each message in a mailbox:

tag7 SEARCH ALL 
* SEARCH 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 
tag7 OK SEARCH completed (Success)

Every time you selected a mailbox, if there were new messages, there would be more sequence numbers. And if you moved messages around and started a new session, the sequence numbers assigned to messages might be different from the ones from the old session.

This very simple scheme worked pretty well if all you did was connect, read your mail online, and disconnect. But it didn’t work if you wanted to connect, download all of your mail locally for offline access, and then reconnect later and have your client sync its local state with the state on the server—because these non-persistent sequence numbers didn’t allow clients to map local mailbox state to remote mailbox state, because messages on each side might have different IDs!

Enter IMAP4 and UIDs. UIDs (unique IDs) are persistent to a message as long as it stays in the same folder and the server’s UIDVALIDITY integer has not increased. So when a client starts a new session on a mailbox, it has to check the UIDVALIDITY and compare it to its local cached value, and if the server’s UIDVALIDITY is higher, it must throw away all cached local UIDs and resync from scratch. Otherwise, it’s free to compare its local cached data to the data on the server and only sync changes.

To retain backwards compatibility, IMAP4 supports both sequence numbers and UIDs, and you can ask for UIDs by prefixing any command that returns message identifiers with UID:

tag8 UID SEARCH ALL 
* SEARCH 53809 54180 54185 54211 54222 54268 54410 54451 54452 54453 54463 54527 54557 54595 54596 54602 54608 54617 54637 54639 54643 54664 54667 54670 54673 54699 54707 54738 54753 54754 54804 54806 54847 54852 54857 54864 54873 54877 54880 54881 54883 54888 54895 54899 54902 54903 54904 54905 54906 54916 54923 54926 54931 54942 54947 
tag8 OK SEARCH completed (Success)

As you can see, unlike sequence numbers, UIDs are not guaranteed to increase in increments of one — gaps develop in the UIDs assigned to a mailbox as messages arrive and are moved around between folders.

Statefulness

While we’re here, one aside about statefulness: Generally, statefulness is not a desirable feature in protocols. Statefulness makes it trickier to use connection pools with IMAP because the SELECT command can be quite slow, and every time a pool consumer checks out a new connection, it needs to ensure that the correct mailbox is selected. Statefulness also means that the application code has to keep track of the state and ensure it’s in sync with the actual state of the connection, or else make another client-server back-and-forth every time the information is needed. Generally, it’s easier to program with stateless protocols than stateful ones.

Fetching message data

OK, let’s finally actually download data from a message!

tag13 UID FETCH 53809 (FLAGS BODY[HEADER.FIELDS (DATE FROM)]) 
* 1 FETCH (UID 53809 FLAGS (Answered Seen) BODY[HEADER.FIELDS (DATE FROM)] {80} From: My Friend <friends4ever@gmail.com> Date: Sun, 18 Feb 2018 10:50:34 -0500 ) 
* 55 EXPUNGE 
* 54 EXISTS 
tag13 OK Success

IMAP provides lots of knobs for allowing clients to request partial data about messages, including the size of a message, flags (like Seen, which means the message has been read, and Answered, which means the message has been replied to), individual headers, and individual MIME parts from within a MIME message, which allows clients to optimize their bandwidth usage by downloading only the data they need. For example, a client can quickly download only basic headers for all the messages in a folder to generate a listing of messages for a user to browse through and then fetch the message bodies on demand.

There are just a couple more concepts we want to cover about IMAP before we wrap up.

Unsolicited responses & IDLE

You may have noticed some strange untagged responses in the previous example:

* 55 EXPUNGE * 54 EXISTS 

This is actually a notification about one message disappearing from the mailbox and another message appearing while we’ve had the mailbox open. It’s valid in IMAP for the server to send these unsolicited messages to a connected client at any time. There’s also a mode you can put an IMAP connection into called IDLE, which is essentially only for handling notifications about changes to a mailbox and is often used by clients to quickly sync new messages coming in.

Disconnected syncing

Disconnected syncing allows email apps to work well in the face of airplane wifi and bad cellular connections. It’s not so much a first-class feature of IMAP as a thing that you can build on top of the protocol with significant effort because IMAP wasn’t originally designed to support it.

The way to do a disconnected sync with IMAP4 in its original form was pretty simplistic: you’d have to take the list of UIDs you have locally for each folder and compare it to the list of UIDs currently in the folder on the server and fetch new messages and delete old ones based on the differences. As mailboxes grew, this became more and more complex for clients to handle, and later extensions like CONDSTORE and QRESYNC improved the flow for disconnected resyncs. But even with these extensions, the process for doing a disconnected resync is complicated to implement—and even today, many IMAP server implementations don’t support the newer extensions, so general-purpose clients must contain conditional logic for different servers. The general reference for implementing disconnected syncing on top of IMAP is RFC4549.

IMAP key features

IMAP boasts various features that enhance its utility in email management across multiple devices.

• Synchronization — IMAP ensures that actions taken on one email client are synchronized with the server, reflecting identical mail status, folders, and organization across all devices.
• Offline accessibility — While IMAP requires an internet connection to receive and send emails, users can view previously downloaded or synchronized messages and compose new ones offline.

While IMAP is great for its synchronization capabilities, it also has inherent technical challenges that developers must navigate. The question of ensuring low latency in operations, particularly when dealing with search functionalities across diverse messages in IMAP, necessitates intricate solutions. Sometimes, this involves implementing additional layers, such as intermediate storage, to address IMAP protocol limitations, ensuring a stable and efficient framework for managing emails.

IMAP advantages and disadvantages

The advantages of IMAP 

• Multi-device accessibility — IMAP’s synchronization ensures seamless email management across multiple devices, reflecting all actions ubiquitously.
• Server-side storage — All emails are stored on the server, preventing data loss even when a device is compromised.
• Efficient bandwidth use — Only headers (not the full message) are downloaded until the user can read a particular email, conserving bandwidth.

The disadvantages of IMAP 

• Server dependency — Dependence on the server for storing emails may be problematic if the server experiences downtime or issues.
• Storage limits — Some email services may impose storage limits on IMAP servers, restricting the amount of mail that can be stored.
• Potential slowness — In some instances, IMAP can be slower than POP since it interacts with the server more frequently to keep mail data synchronized.

IMAP alternatives 

While IMAP offers robust email management, certain scenarios or user preferences might necessitate alternatives.

POP3

Post Office Protocol (POP3) is useful when users prefer to download emails to their local devices. Unlike IMAP, which syncs email status across multiple devices, POP3 simply pulls the email down and doesn’t reflect changes like read or deleted status on other devices.

SMTP

Simple Mail Transfer Protocol (SMTP) is primarily used for sending emails. While it’s not a direct alternative to IMAP in terms of email management, it’s crucial for the actual delivery of emails from one point to another. When setting up an email client, IMAP (or POP3) and SMTP are usually required: IMAP for receiving and managing emails and SMTP for sending them.

Webmail

Webmail allows users to access emails via a web browser, eliminating the need for separate email clients and the complexities of server synchronization.

It’s worth noting that these aren’t necessarily alternatives because one must choose one over the other. In many cases, they’re complementary. For example, someone might use IMAP to manage their email on their phone and computer, SMTP to send emails from those devices, and occasionally use Webmail when they’re on a different device or don’t have access to their primary email client.

IMAP security 

Securing IMAP involves utilizing encrypted connections, secure passwords, and, when possible, two-factor authentication to safeguard email data.

• SSL/TLS encryption: Ensuring the email client’s and server’s connection is encrypted.
• Secure passwords: Implementing robust password policies to enhance account security.

Why use Nylas for IMAP? 

Nylas is not just another email integration tool; it’s a leader with profound expertise in IMAP integrations. Our platform offers developers an easy-to-use path to embed email capabilities into their apps, making the complexities of an IMAP integration simple.

[Nylas] has a unique feature set, and helps us solve problems when it comes to IMAP integrations. [Nylas] is unique! We built our own IMAP gateway initially and it was less overhead using Nylas.

Joy K.

Founder at Joylab

We’ve built a universal email API that tackles the core challenges of IMAP. This ensures faster message searches and seamless synchronization with modern storage systems. The end result? Efficient, user-friendly email solutions.

What sets us apart is our deep understanding of the unique hurdles in IMAP integration. Whether it’s provider-specific quirks or unexpected shifts in IMAP standards, we’ve got it covered. We aim to ensure developers have the resources to craft top-notch, user-focused email platforms.

Moreover, Nylas stands alone in the market with its real-time webhook support for IMAP providers. And, in case of interruptions, Nylas offers a unique feature that allows end-users to replay webhooks. This ensures no important update is missed even if systems face downtimes.