Coutsey to Bob Peers>
The basics.
About IMAP.
IMAP is an email protocol for organizing, storing and retrieving emails on a remote server. It was
developed after POP and is a much more advanced system, one of the main differences being that all the mail is stored on the server so it remains accessible from many
different locations. With POP you have to download the mail to your local computer in order to read it and therefore you cannot synchronize your mail across many
different machines.It may be more complex than POP but there are still only a few core commands we need to know in order to access our mail on an IMAP server.
Before starting it’s important to know a few things:
IMAP command syntax.
Before the actual command is typed into the terminal we need to type a command tag, this could be anything (without spaces) and the server will tag its response
with the tag we give it. This seems to be because IMAP allows multiple connections and so multiple commands, by tagging you know which response refers to which command.
In our case we have only 1 connection and we send single commands so it’s not really relevant, however we need to type something as a tag. I usually just use a period
‘.’ but you could use a number or whatever suits you. To demonstrate the command tag see the two server responses here with the tag (don’t worry about the command itself, it
will be explained soon), in the first one we send ‘. fetch’ and the second one ‘a01a fetch’ getting the same tag back to identify the response:
. fetch 1 fast
* 1 FETCH (FLAGS (Seen hasatt) INTERNALDATE " 1-Feb-2006 08:37:23 -0500" RFC822.SIZE 15013)
. OK Completed (0.000 sec)
ao1a fetch 1 fast
* 1 FETCH (FLAGS (Seen hasatt) INTERNALDATE " 1-Feb-2006 08:37:23 -0500" RFC822.SIZE 15013)
a01a OK Completed (0.000 sec)
Finally, the IMAP commands are not case sensitive, so ‘SELECT inbox’ will work just as well as ‘select INBOX’. For clarity in the code I have typed the commands in
uppercase and the word INBOX in uppercase also.
Mail server address.
The address of your mail server, this will usually be of the form mail.domain.com. You should look at the settings in your email client or documentation about your
email account to get this information.
Security.
In this demonstration we will be sending our account username and password unencrypted over the internet, if this is a major concern to you then you should not follow
this exercise.
Another alternative, if your email provider supports SSL, is to use OpenSSL (which most if not all Linux computers will have), see the ‘Connecting to the host’ section
below for the syntax.
Using telnet.
If you make a mistake in a telnet session you cannot use backspace to delete the entry, you may have to press enter to get an error and then re-type the command or quit and start again.
Connecting to the host.
Insecure login – login using telnet.
By insecure I just mean that your username and password are sent unencrypted over the internet so potentially could be intercepted on the route between your computer and the mail server.
First open up a terminal and type the following, of course replacing mail.myserver.com with the address of your IMAP server, note that the IMAP port used is 143:
telnet mail.myserver.com 143
This should return something like:
telnet mail.myserver.com 143
Trying 66.111.4.160...
Connected to mail.myserver.com (66.111.4.160).
Escape character is '^]'.
* OK IMAP4 ready
Secure login – login using OpenSSL.
To open an SSL session that encrypts all data sent between your computer and the mail server open a teminal and follow these steps, note that we use port 993 here:
openssl s_client -connect mail.myserver.com:993
CONNECTED(00000003)
depth=0 /C=AU/ST=New South Wales/L=Crows Nest/O=Optimal Decisions Group Pty Ltd/CN=mail.messagingengine.com
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 /C=AU/ST=New South Wales/L=Crows Nest/O=Optimal Decisions Group Pty Ltd/CN=mail.messagingengine.com
verify error:num=27:certificate not trusted
verify return:1
depth=0 /C=AU/ST=New South Wales/L=Crows Nest/O=Optimal Decisions Group Pty Ltd/CN=mail.messagingengine.com
verify error:num=21:unable to verify the first certificate
verify return:1
---
Certificate chain
0 s:/C=AU/ST=New South Wales/L=Crows Nest/O=Optimal Decisions Group Pty Ltd/CN=mail.messagingengine .com
i:/C=ZA/ST=Western Cape/L=Cape Town/O=Thawte Consulting cc/OU=Certification Services Division/CN= Thawte Premium Server CA/emailAddress=premium-server@thawte.com
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIDeDCCAuGgAwIBAgIDQBYSMA0GCSqGSIb3DQEBBAUAMIHOMQswCQYDVQQGEwJa
..........................
-----END CERTIFICATE-----
subject=/C=AU/ST=New South Wales/L=Crows Nest/O=Optimal Decisions Group Pty Ltd/CN=mail.messagingeng ine.com
issuer=/C=ZA/ST=Western Cape/L=Cape Town/O=Thawte Consulting cc/OU=Certification Services Division/C N=Thawte Premium Server CA/emailAddress=premium-server@thawte.com
---
No client certificate CA names sent
---
SSL handshake has read 1054 bytes and written 340 bytes
---
New, TLSv1/SSLv3, Cipher is AES256-SHA
Server public key is 1024 bit
SSL-Session:
Protocol : TLSv1
Cipher : AES256-SHA
Session-ID: Session ID
Session-ID-ctx:
Master-Key: Key
Key-Arg : None
Krb5 Principal: None
Start Time: 1140271254
Timeout : 300 (sec)
Verify return code: 21 (unable to verify the first certificate)
---
* OK IMAP4 ready
Once this step is carried out the IMAP commands are identical to those for a normal telnet session.
Logging in.
Next we need to log in using the login command. Type ‘. login’ followed by your username and password separated by spaces.
. login accountname@myserver.com *********
. OK User logged in
LIST command.
To see a list of all the mailboxes on the server we use the list command. The arguments “” “*” simply get all the mailboxes including sub folders.
. list "" "*"
* LIST (HasChildren) "." "INBOX"
* LIST (HasNoChildren) "." "INBOX.Drafts"
* LIST (HasNoChildren) "." "INBOX.Sent Items"
* LIST (HasNoChildren) "." "INBOX.Trash"
* LIST (HasNoChildren) "." "INBOX.test1"
* LIST (HasNoChildren) "." "INBOX.test2"
. OK Completed (0.460 secs 7 calls)
We can see from this output how the mailboxes are arranged like a tree with INBOX being the ‘trunk’. My IMAP provider uses a period (.) as a separator between parent and
child folders so INBOX.Drafts is a child of the INBOX. The HasChildren simply tells us that this folder has sub folders whereas the other folders do not.
The way IMAP works means that all folders are created as subfolders of the INBOX even if your email client is configured not to show it that way.
STATUS command.
This command return some basic information on the folder without selecting the folder, it takes arguments depending on what information you would like returned.
Here are 3 example showing total messages, recent messages and unseen messages.
. status INBOX (messages)
* STATUS INBOX (MESSAGES 2)
. OK Completed
. status INBOX (recent)
* STATUS INBOX (RECENT 0)
. OK Completed
. status INBOX (unseen)
* STATUS INBOX (UNSEEN 0)
. OK Completed
EXAMINE and SELECT commands.
These two commands basically do the same thing, they return information on the folder chosen and then allow us to access the mails stored inside the folder. The
main difference is that EXAMINE returns a read-only reference whereas SELECT is read-write.
. examine INBOX.test2
* FLAGS (Answered Flagged Draft Deleted Seen)
* OK [PERMANENTFLAGS ()]
* 0 EXISTS
* 0 RECENT
* OK [UIDVALIDITY 1138801117]
* OK [UIDNEXT 1]
. OK [READ-ONLY] Completed
. select INBOX.test2
* FLAGS (Answered Flagged Draft Deleted Seen)
* OK [PERMANENTFLAGS (Answered Flagged Draft Deleted Seen *)]
* 0 EXISTS
* 0 RECENT
* OK [UIDVALIDITY 1138801117]
* OK [UIDNEXT 1]
. OK [READ-WRITE] Completed
Note that the only difference in response is the [READ-ONLY] and [READ-WRITE] text. Basically this command just tells us the possible IMAP flags we can set,
EXIST is how many mails are in the folder, RECENT is how many recent mails there are (the SELECT command will remove the RECENT flag since the folder has now been
viewed, this is not the same as the Seen IMAP flag, also note that the EXAMINE command will not reset the RECENT flag).
The RECENT data is what tells an IMAP email client if you have new mails, by clicking on the folder the client sends the SELECT command and the new mail icon
disappears even though the mails are still unread.
CREATE, DELETE and RENAME folders.
It’s very easy to create and delete folders, just make sure you create them as subfolders of the INBOX. For example to create a top level folder called test3 do
do the following.
. create INBOX.test3
. OK Completed
. list "" "*"
* LIST (HasChildren) "." "INBOX"
* LIST (HasNoChildren) "." "INBOX.Drafts"
* LIST (HasNoChildren) "." "INBOX.Sent Items"
* LIST (HasNoChildren) "." "INBOX.Trash"
* LIST (HasNoChildren) "." "INBOX.test1"
* LIST (HasNoChildren) "." "INBOX.test2"
* LIST (HasNoChildren) "." "INBOX.test3" #we created this
. OK Completed (0.420 secs 8 calls)
Conversely we can delete our new folder using the DELETE command. Note that you cannot delete a folder that had subfolders without first deleting the subfolders, also
deleting a folder containing mails will delete all the mails inside so beware!
. delete INBOX.test3
. OK Completed
. list "" "*"
* LIST (HasChildren) "." "INBOX"
* LIST (HasNoChildren) "." "INBOX.Drafts"
* LIST (HasNoChildren) "." "INBOX.Sent Items"
* LIST (HasNoChildren) "." "INBOX.Trash"
* LIST (HasNoChildren) "." "INBOX.test1"
* LIST (HasNoChildren) "." "INBOX.test2"
. OK Completed (0.430 secs 7 calls)
Renaming a folder is just as easy, just type RENAME [current name] [new name]. This will not delete mails as they will just exist in the new folder. Here we rename
folder test1 to linux.
. rename INBOX.test1 INBOX.test3
* OK rename user.accountname.test1 user.accountname.test3
. OK Completed
. list "" "*"
* LIST (HasChildren) "." "INBOX"
* LIST (HasNoChildren) "." "INBOX.Drafts"
* LIST (HasNoChildren) "." "INBOX.Sent Items"
* LIST (HasNoChildren) "." "INBOX.Trash"
* LIST (HasNoChildren) "." "INBOX.linux" #this was test1
* LIST (HasNoChildren) "." "INBOX.test2"
. OK Completed (0.410 secs 7 calls)
FETCH command.
This command is the main command we use to actually access our emails. It has many possible options depending in what you wish to see, message flags, email headers,
text of the body etc. Here we select the INBOX and fetch the emails in a few different ways.
. select INBOX
* FLAGS (Answered Flagged Draft DeleteCLOSE and EXPUNGE commands.d Seen hasatt)
* OK [PERMANENTFLAGS (Answered Flagged Draft Deleted Seen hasatt *)]
* 2 EXISTS
* 0 RECENT
* OK [UIDVALIDITY 1138801043]
* OK [UIDNEXT 3]
. OK [READ-WRITE] Completed
. fetch 1 flags
* 1 FETCH (FLAGS (Seen hasatt))
. OK Completed (0.000 sec)
First we shall fetch the message IMAP flags for all the messages in the folder.
. fetch 1:2 flags
* 1 FETCH (FLAGS (Seen hasatt))
* 2 FETCH (FLAGS (Seen hasatt))
. OK Completed
Note that with all the commands that act upon messages we can select either 1 message by using the message number as in ‘fetch 1 command’ or we can select a range
of messages in the format ‘fetch first:last command’ or all the messages ‘fetch 1:last command’. Also note that we can use ‘*’ to indicate all messages so fetch 1:* will get
all the messages from the first to the last without us knowing how many messages are in the folder.
First we shall fetch using fast, all and full options (these refer to the headers).
. fetch 1 fast
* 1 FETCH (FLAGS (Seen hasatt) INTERNALDATE " 1-Feb-2006 08:37:23 -0500" RFC822.SIZE 15013)
. OK Completed (0.000 sec)
. fetch 1 all
* 1 FETCH (FLAGS (Seen hasatt) INTERNALDATE " 1-Feb-2006 08:37:23 -0500" RFC822.SIZE 15013 ENVELOPE ("Wed, 1 Feb 2006 13:37:19 UT" "IMPORTANT: Click here to begin using your account" (("Email Administrator" NIL "bounce" "myserver.com")) (("Email Administrator" NIL "bounce" "myserver.com")) ((NIL NIL "webmaster" "myserver.com")) (("Joe Bloggs" NIL "accountname" "myserver.com")) NIL NIL NIL "<cmu-lmtpd-28871-1138801043-0@server2.messagingengine.com>"))
. OK Completed (0.000 sec)
. fetch 1 full
* 1 FETCH (FLAGS (Seen hasatt) INTERNALDATE " 1-Feb-2006 08:37:23 -0500" RFC822.SIZE 15013 ENVELOPE ("Wed, 1 Feb 2006 13:37:19 UT" "IMPORTANT: Click here to begin using your account" (("Email Administrator" NIL "bounce" "myserver.com")) (("Email Administrator" NIL "bounce" "myserver.com")) ((NIL NIL "webmaster" "myserver.com")) (("Joe Bloggs" NIL "accountname" "myserver.com")) NIL NIL NIL "<cmu-lmtpd-28871-1138801043-0@server2.messagingengine.com>") BODY ((("TEXT" "PLAIN" NIL NIL NIL "8BIT" 5599 137)("TEXT" "HTML" NIL NIL NIL "8BIT" 7434 141) "ALTERNATIVE")("TEXT" "PLAIN" ("NAME" "This_is_how_attachments_appear.txt") NIL NIL "8BIT" 247 6) "MIXED"))
. OK Completed (0.000 sec)
As you can see this returns differing amounts of data about the IMAP flags, size and ENVELOPE information. It’s maybe more informative to use either ‘fetch message body[header]’
or ‘fetch message rfc822.header’ both of which return the data below.
. fetch 1 rfc822.header
* 1 FETCH (RFC822.HEADER {824}
Return-Path: <nobody@server2.messagingengine.com>
Received: from web2.internal (web2.internal [10.202.2.211])
by server2.messagingengine.com (Cyrus v2.3-alpha) with LMTPA;
Wed, 01 Feb 2006 08:37:23 -0500
X-Sieve: CMU Sieve 2.3
X-Attached: This_is_how_attachments_appear.txt
X-Resolved-to: accountname
X-Mail-from: nobody
Content-Transfer-Encoding: 8bit
Content-Type: multipart/mixed; boundary="_----------=_1138801039165120"
MIME-Version: 1.0
X-Mailer: MIME::Lite 5022 (F2.73; T1.15; A1.64; B3.05; Q3.03)
Date: Wed, 1 Feb 2006 13:37:19 UT
From: "Email Administrator" <bounce@myserver.com>
Reply-To: webmaster@myserver.com
To: "Joe Bloggs" <accountname@myserver.com>
Subject: IMPORTANT: Click here to begin using your account
Message-ID: <cmu-lmtpd-28871-1138801043-0@server2.messagingengine.com>
)
. OK Completed (0.000 sec)
To fetch only some headers we can select the header fields we wish to see.
. fetch 1 (body[header.fields (from to subject date)])
* 1 FETCH (BODY[HEADER.FIELDS (from to subject date)] {195}
Date: Wed, 1 Feb 2006 13:37:19 UT
From: "Email Administrator" <bounce@myserver.com>
To: "Joe Bloggs" <accountname@myserver.co>
Subject: IMPORTANT: Click here to begin using your account
)
. OK Completed (0.000 sec)
To read the body of the email message we can use either ‘fetch message body[text]’ or ‘fetch message rfc822.text’ as shown here.
. fetch 2 rfc822.text
* 2 FETCH (RFC822.TEXT {11658}
This is a multi-part message in MIME format.
--_----------=_1138865560223950
Content-Disposition: inline
Content-Length: 5194
Content-Transfer-Encoding: binary
Content-Type: text/plain
more text here.............
. OK Completed (0.000 sec)
STORE command.
This command allows us to add, remove or replace the IMAP flags on the messages. These are flags that denote a message as replied to, deleted, seen etc. and allow
the message information, as well as the message itself, to be synchronized across different computers. Note that the STORE command causes an automatic FETCH
command of the message flags so we can see the change immediately. There are 3 ways to use STORE:
- STORE message +flags [flag list] – this adds the [flag list] flags to the chosen messages.
- STORE message -flags [flag list] – this removes the [flag list] flags from the chosen messages.
- STORE message flags [flag list] – resets the flags to [flag list] on the chosen messages (the same as removing all flags and then adding [flag list].
The list of flags to add include Answered Flagged Draft Deleted Seen and many more. All the IMAP flags used as part of the standard installation have the
backslash in front of them. However some email clients (Thunderbird is one) also allow you to set labels or mark a message as junk, if you add labels do not use
the backslash. First we shall mark all the messages as deleted.
. store 1:2 flags Deleted
* 1 FETCH (FLAGS (Recent Deleted))
* 2 FETCH (FLAGS (Recent Deleted))
. OK Completed
Next replace the flags with $label1
. store 1:* flags $label1
* FLAGS (Answered Flagged Draft Deleted Seen hasatt Junk label1)
* OK [PERMANENTFLAGS (Answered Flagged Draft Deleted Seen hasatt Junk $label1 *)]
* 1 FETCH (FLAGS ($label1))
* 2 FETCH (FLAGS ($label1))
. OK Completed
Finally we can add the flag NonJunk so that Thunderbird recognises them as not being junk mail.
. store 1:* +flags NonJunk
* FLAGS (Answered Flagged Draft Deleted Seen hasatt NonJunk Junk label1)
* OK [PERMANENTFLAGS (Answered Flagged Draft Deleted Seen hasatt NonJunk Junk $label1 *)]
* 1 FETCH (FLAGS ($label1 NonJunk))
* 2 FETCH (FLAGS ($label1 NonJunk))
. OK Completed
Note that the Deleted flag is used by an IMAP server to mark an email ready for deletion, it is not actually deleted until the server receives either the CLOSE
or EXPUNGE command shown below.
CLOSE and EXPUNGE commands.
Both these commands have the effect of permanently deleting any messages in the current folder marked for deletion with the Deleted flag. EXPUNGE just deletes the
messages but does nothing else (this command is the equivalent of compacting folders in Thunderbird), while CLOSE deletes the messages and deselects the current
folder (you cannot carry out more action on messages until you select a new folder). Assuming the two messages in our INBOX had the Deleted flag set then the output
looks like the following.
. expunge
* 1 EXPUNGE
* 1 EXPUNGE
* 0 EXISTS
* 0 RECENT
. OK Completed
COPY command.
IMAP has no built in move command, when you move a message you actually copy it to another folder and then delete the original. We can easily copy any number of
messages using the COPY message [destination] format. Here I copy both messages from the INBOX (that I already have selected) to INBOX.test2 folder, after that I
select INBOX.test2 to confirm the messages are there. Note that after copying the RECENT flag is reset.
. copy 1:2 inbox.test2
. OK [COPYUID 1138801117 1:2 1:2] Completed
. select inbox.test2
* FLAGS (Answered Flagged Draft Deleted Seen hasatt)
* OK [PERMANENTFLAGS (Answered Flagged Draft Deleted Seen hasatt *)]
* 2 EXISTS
* 2 RECENT
* OK [UIDVALIDITY 1138801117]
* OK [UIDNEXT 3]
. OK [READ-WRITE] Completed
IDLE command.
IDLE allows us to constantly monitor a folder so that we will be instantly be notified if a new message arrives in the current folder. This is of little use
while in a telnet session but I’ll show it here just so you know how it works. First start IDLE on the folder.
. idle
+ idling
The server responds with +idling and will stay this way until either a message is received, we stop idling or carry out another command to break the idle. If non
of these things happen then the connection will eventually time out after a pre-set period depending on your IMAP provider (30 minutes in my case). To stop the IDLE
command use DONE (note this is the only command WITHOUT the preceding command tag).
done
. OK Completed
LSUB, SUBSCRIBE and UNSUBSCRIBE commands.
These are more commands that only really apply to email clients since they involve subscribing to folders, however they are shown here for completeness. First LSUB
works like LIST with the same arguments but returns a list of the currently subscribed folders.
. lsub "" "*"
* LSUB (HasChildren) "." "INBOX"
* LSUB () "." "INBOX.Drafts"
* LSUB () "." "INBOX.Sent Items"
* LSUB () "." "INBOX.Trash"
* LSUB () "." "INBOX.test2"
. OK Completed (0.000 secs 6 calls)
This shows that all folders except INBOX.test3 are currently subscribed, to subscribe a new folder use SUBSCRIBE [foldername].
. subscribe INBOX.test3
. OK Completed
To unsubscribe from this folder use UNSUBSCRIBE [foldername].
. unsubscribe INBOX.test3
. OK Completed
LOGOUT command.
Of course we need to log out of the server, we do this with the LOGOUT command.
. logout
* BYE LOGOUT received
. OK Completed
That’s the main commands covered however there are a few more just 3 of which I’ll mention here as they could be useful.
CAPABILITY, GETQUOTAROOT AND GETACL commands.
These 3 commands return general information on the server environment and your account information. CAPABILITY returns a long list of the mail servers option most of
which are not very exciting, the most important one listed is probably IDLE letting you know that your provider supports the IDLE command. The CHILDREN entry we saw
returned when we did a LIST command (HasChildren or HasNoChildren depending on whether a folder has subfolders).
. capability
* CAPABILITY IMAP4 IMAP4rev1 ACL QUOTA LITERAL+ MAILBOX-REFERRALS NAMESPACE UIDPLUS ID NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND BINARY SORT THREAD=ORDEREDSUBJECT THREAD=REFERENCES ANNOTATEMORE CATENATE IDLE LOGINDISABLED
. OK Completed
GETQUOTAROOT return the amount of space you are using and the amount you have available.
. getquotaroot inbox
* QUOTAROOT inbox user.accountname
* QUOTA user.accountname (STORAGE 31306 2048000)
As you can see I’m using 31Mb but have 2Gb capacity, so plenty to spare! Finally GETACL returns the access control list, basically a list of permission you have on
your mail folders.
. getacl inbox
* ACL inbox accountname lrswipcd admin lrswipcda anyone p
. OK Completed
These letters each refer to a different permission, the letters after the user are that users rights, the full list is explained here:
- l – lookup_flag: mailbox is visible to LIST/LSUB commands
- r – read_flag: SELECT the mailbox, perform CHECK, FETCH, PARTIAL SEARCH, COPY from mailbox
- s – seen_flag: keep seen/unseen information across session
- w – write_flag: STORE flags other than SEEN and DELETED
- i – insert_flag: perform APPEND, COPY into mailbox
- p – post_flag: send mail to submission address for mailbox
- c – create_flag: CREATE new sub-mailboxes in any implementation defined hierarchy
- d – delete_flag: STORE DELETED flag perform EXPUNGE
- a – administer_flag: perform SETACL
