Using imtest and the IMAP protocol

It can be very useful to be able to interact with the IMAP server by manually writing commands, especially when troubleshooting problems. The imtest tool in the Cyrus distribution makes this technique even more useful, as you can authenticate using SASL to servers you might not otherwise be able to access due to security requirements and you can use SSL/TLS encrypted connections. You can also use a different authorization name to your authentication name, allowing you to authenticate using admin's credentials but then essentially "become" a user of your choice for the purposes of the rest of the session.

Much of this page refers to functionality and features that are not part of the IMAP4r1 standard or even specific to Cyrus IMAPd. Some may be useful for working with other IMAP implementations, some will not.

Connecting to the server

The first thing you'll want to do is connect to your IMAP server. I'm going to assume that imtest is on your PATH. If it is not, you can run it as /path/to/cyrus/installation/bin/imtest, or simply add /path/to/cyrus/installation/bin to your PATH. To log in as yourself on the local host, simply run imtest localhost and enter your password when prompted. You should see something like this:

S: * OK bucket.localnet Cyrus IMAP4 v2.1.14 server ready
C: C01 CAPABILITY
S: * CAPABILITY IMAP4 IMAP4rev1 ACL QUOTA LITERAL+ MAILBOX-REFERRALS 
NAMESPACE UIDPLUS ID NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND 
SORT THREAD=ORDEREDSUBJECT THREAD=REFERENCES IDLE X-NETSCAPE
S: C01 OK Completed
Please enter your password:
C: L01 LOGIN craig {9}
S: + go ahead
C: 
S: L01 OK User logged in
Authenticated.
Security strength factor: 0

If so, you're now able to send IMAP commands directly to the server. The one you'll probably need to know first is how to disconnect. Simply type =. logout = (yes, the full stop and space are required). You should see something like this:

. logout
* BYE LOGOUT received
. OK Completed
Connection closed.

You should now find yourself back at the prompt. In general, that's how you can interact with the IMAP server for testing, though normally you'll want to connect a bit differently.

Connection Options and Authentication

Some may not be able to connect using the basic command given above. Perhaps your server requires an encrypted connection, a hashed password, or doesn't listen on the normal port. For that matter, perhaps you're here because you're having trouble getting the server to authenticate users! There are a few command arguments to imtest that you'll need to use. The full list can be found by running imtest -h, and more extensive coverage can be found in the imtest man pages (which you should read).

The most command extra arguments you'll want to use are:

• -p port to connect to an IMAP server listening on a different port (useful when testing settings changes on a production server, as you can have another imapd listen on a different port and use a different config file)
• -s to use IMAP over SSL (imaps, usually tcp/993), which is not to be confused with negotiating SSL/TLS after connecting.
• -u : the authorization name to use. Usually the same as the authentication name, so if you specify one you normally specify both.
• -a : authentication name . See above.
• -w : password to use. Should be the password associated with the authentication username. Warning this may be visible to other users in ps output, etc. You're usually better off entering the password interactively.

Authentication and Authorization

One handy trick you can do is to authenticate as an admin, but provide a different username for authorization. This lets you essentially 'become' that user, and is in concept somewhat similar to root using 'su - username' to "become" a different user. The main advantage is that you can act as any user, without needing to know or change their password. This is especially handy when troubleshooting user-specific problems and when performing batch tasks. You should read the AuthorizationAndAuthentication page for more information on this.

A typical command to authenticate as user 'cyrus' (an admin) but gain the priveleges and essentially the identity of user 'craig' (a normal user) would be:

imtest -s -u craig -a cyrus localhost

Note the use of ssl ( -s ). It is equally valid to use TLS or any other security layer. Note that you must use some sort of security layer to use the PLAIN SASL mechanism, and you must use a SASL mechanism that supports proxy authentication in order to have a different authorization identity from authentication identity.

When you've been authorized as a given user, it's just as if you authenticated as them too. You have all their priveleges (or lack thereof), when you subscribe to a mailbox you subscribe them to that mailbox, and when you flag messages as seen they show up as seen for that user. This is very handy, especially for batch-subscribing users to a new shared mailbox.

IMAP syntax

So, we've done the imtest command, logging in, and authentication to death. You still probably want some information about useful things you can do once connected.

(NOTE: Following needs checking by someone who knows better, I haven't had time to read the entire IMAP4r1 RFC in detail yet) In general, IMAP protocol commands take the form:

tag COMMAND [argument [argument [...]]

where tag is a user-chosen number or string. I usually just use '.' or numbers, counting up from 1, but others use a-z etc. Commands are case-insensitive (at least in Cyrus IMAPd) but the convention is to use UPPERCASE for all commands. Some other things, like mailbox and user names, are case sensitive. For example, these are all OK:

. SUBSCRIBE user.monty
a subscribe user.monty
1 Subscribe user.monty
b33p sUbScRiBe user.monty

... though the latter one might get you laughed to death if anyone caught you using it.

IMAP Commands

Here's a list of common IMAP commands. I've used 'a' to 'z' as the tags, but you can use whatever is appropriate. Non-literal arguments are shown in lower case, so where you see mailbox you would use, for example, user.jcleese or user.brian. Thanks to Ken Murchison for the basis of this summary. Please direct any flames or errors to me (see the bottom of the page). Note: All of the following examples assume that unixhierarchysep is set to no in your imapd.conf. If it is enabled, you need to use / instead of . in all the mailbox paths. A slightly more detailed description of the IMAP commands follows this summary.

• a CAPABILITY: Print some information about what capabilities the mail server has, such as authentication options and ACL support.
• b SELECT mailbox: Opens a mailbox and prints the number of messages flagged as unread.
• c FETCH msgno BODY[HEADER]: Display the headers of message msgno. msgno is a number from 1 to the number of messages in the mailbox as reported by SELECT. Note that the [HEADER] is literal.
• d FETCH msgno BODY[]: Display an entire message.
• e FETCH msgno FLAGS: Display any flags, such as \Seen, \Deleted, and \Flagged that may be set on a message.
• f STORE msgno [-+]FLAGS \FLAGNAME: Sets (with +) or unsets (with -) flag \Flagname on message msgno. A common flag is the \Deleted flag, which marks a message for deletion at the next EXPUNGE, and causes most clients to hide the message or otherwise indicate that it has been deleted.
• g EXPUNGE: Actually delete any messages flagged as \Deleted.
• h CREATE mailbox: Create mailbox
• i DELETE mailbox: Delete . Unlike flagging individual messages as deleted, this is immediate and permanant.
• j RENAME oldmailbox newmailbox: Rename (move) oldmailbox to newmailbox.

There are also some commands which are somewhat or entirely specific to Cyrus IMAPd. While normally run from cyradm, there are times it can be handy to run them via imtest (or telnet). This can be handy if you want to be able to set ACLs on some of your mailboxes without being the admin, for example.

• w RECONSTRUCT mailbox: Attempts to reconstruct a mailbox, much like running reconstruct from the command line. Must be run by an admin(?).
• x SETACL mailbox user acl: Sets acl for user on mailbox.
• y GETACL mailbox: Lists ACLs on =mailbox.
• z DELETEACL mailbox user Removes the ACL for user from mailbox entirely.

One issue that may occasionally confuse is when you issue an apparently valid command, only to have the server respond with BAD Unrecognised command:

SUBSCRIBE user.craig
SUBSCRIBE BAD Unrecognized command

Why? You've missed the command tag, so the server has used UNSUBSCRIBE as the command tag. It doesn't recognise user.craig as a command, so it reports BAD Unrecognised command. This example might clarify things:

x FETCH 1 FLAGS
* 1 FETCH (FLAGS (\Flagged \Seen))
x OK Completed
SUBSCRIBE FETCH 1 FLAGS
* 1 FETCH (FLAGS (\Flagged \Seen))
SUBSCRIBE OK Completed

CAPABILITY

The CAPABILITY command, unsurprisingly, prints a summary of server capabilities. Note the server may choose not to show some capabilities depending on whether or not you have authenticated yet, and what the security level of your connection is. A very common one is that Cyrus IMAPd is usually set not to advertise AUTH=PLAIN on unencrypted connections. This may mean that the capability is not present, or that the server simply chooses not to tell you about it.

Example:

a CAPABILITY
* 43 EXISTS
* 1 RECENT
* CAPABILITY IMAP4 IMAP4rev1 ACL QUOTA LITERAL+ MAILBOX-REFERRALS 
NAMESPACE UIDPLUS ID NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND
SORT THREAD=ORDEREDSUBJECT THREAD=REFERENCES IDLE LOGINDISABLED X-NETSCAPE
a OK Completed

SELECT

The SELECT command opens a mailbox, and prints a summary of its status. You can specify a mailbox relative to your INBOX, or to the global IMAP root. SELECT INBOX and SELECT user.username are equivalent, if username is your authorization name for this connection. Similarly, SELECT INBOX.Sent and SELECT user.username.Sent both select the same mailbox. You can SELECT any mailbox you have permission to see, but mailboxes outside your INBOX herirachy must be specified in names relative to the global IMAP namespace (eg user.someotheruser).

Example:

z SELECT INBOX
* FLAGS (\Answered \Flagged \Draft \Deleted \Seen NonJunk
$Label2 $Label1 $Label3 $Label4 $Label5 Junk $Forwarded $MDNSent)
* OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted
\Seen NonJunk $Label2 $Label1 $Label3 $Label4 $Label5 Junk
$Forwarded $MDNSent \*)]
* 42 EXISTS
* 0 RECENT
* OK [UNSEEN 41]
* OK [UIDVALIDITY 1060587836]
* OK [UIDNEXT 12645]
z OK [READ-WRITE] Completed

FETCH

Retrieves information about a message, or all or part of a message. a FETCH msgno BODY[HEADER] gets just the headers, a FETCH msgno BODY[] gets the entire message. a FETCH msgno FLAGS displays any flags, such as \Seen and \Deleted, that are set on the message. It is also possible to do things like fetch only part of a message - see the IMAP4r1 RFC for details.

Example:

a FETCH 1 FLAGS
* 1 FETCH (FLAGS (\Flagged \Seen))
a OK Completed
b FETCH 1 BODY[HEADER]
* 1 FETCH (BODY[HEADER] {1711}
Return-Path: 
Received: from mail.example.com ([202.0.58.20] helo=linda-1.paradise.net.nz)
        by mail.somewhere.net with esmtp
        for ; Thu, 29 Jan 2004 02:11:28 +0800
Date: Thu, 29 Jan 2004 07:11:17 +1300
From: Remote User 
Subject: Good news
To: Craig Ringer 
Message-id: <003x01x3e5cx$2489x8f0$d000x6x2@EXAMPLE>
MIME-version: 1.0
X-Spam-Flag: false
X-Scanned-By: MIMEDefang
X-Virus-Scanned: ClamAV

)
b OK Completed

STORE

The STORE command can be used to set flags on a message. A common use is to set the \DELETED flag, which marks a message as deleted so that it will be removed from the mailbox when EXPUNGE is next run. Messages flagged as \DELETED are normally hidden by mail clients, or displayed with some indication that they're tagged for deletion (an icon with a red x in it, for example). To set and unset flags with the STORE command, the syntax is: a STORE [+-]FLAGS \FLAGNAME . Use + to set a flag, and - to unset it.

Example:

a FETCH 1 FLAGS
* 1 FETCH (FLAGS (\Flagged \Seen))
a OK Completed
b STORE 1 +FLAGS \DELETED
* 1 FETCH (FLAGS (\Flagged \Deleted \Seen))
b OK Completed
c FETCH 1 FLAGS
* 1 FETCH (FLAGS (\Flagged \Deleted \Seen))
c OK Completed
d STORE 1 -FLAGS \DELETED
* 1 FETCH (FLAGS (\Flagged \Seen))
d OK Completed
e FETCH 1 FLAGS
* 1 FETCH (FLAGS (\Flagged \Seen))
e OK Completed

EXPUNGE

The EXPUNGE command permanantly deletes any messages with the \Deleted flag set from the mailbox. This will change the messsage numbers (but not their UID). It will also print an updated message count.

Example:

. EXPUNGE
* 45 EXISTS
* 1 RECENT
. OK Completed

CREATE and DELETE

Create and delete mailbox with a CREATE mailbox and a DELETE mailbox. mailbox may be specified relative to the INBOX of the logged in user, or relative to the IMAP root.

Example:

a CREATE INBOX.test
a OK Completed
a DELETE INBOX.test
a OK Completed
A CREATE INBOX.test
A OK Completed
a CREATE INBOX.test
a NO Mailbox already exists
a DELETE INBOX.test
a OK Completed

RENAME

Rename, or move, a mailbox. RENAME oldmailbox newmailbox.

Example:

k RENAME INBOX.test INBOX.blah
k OK Completed

RECONSTRUCT

Tells Cyrus IMAPd to attempt to rebuild the indexes of the specified mailbox. This is probably not especially useful, since cyradm and the reconstruct binary are probably much better interfaces. See "man reconstuct" for details.

GETACL, SETACL, DELETEACL

Manipulate ACLs on mailboxes. I won't cover ACL specifications here, as they're documented quite well enough in the docs shipped with Cyrus IMAPd. Again, cyradm is probably a better interface for these commands in normal use.

Example:

2 GETACL INBOX
* ACL INBOX craig lrswipcda
2 OK Completed
3 SETACL INBOX monty lrswipcda
3 OK Completed
4 GETACL INBOX
* ACL INBOX craig lrswipcda monty lrswipcda
4 OK Completed
5 DELETEACL INBOX monty
5 OK Completed
6 GETACL INBOX
* ACL INBOX craig lrswipcda
6 OK Completed

Others

This is not an exhaustive list of IMAP commands; far from it. See the Further Information section at the end of this page.

Telnet / netcat

You can also connect to an IMAP server directly using telnet or netcat - telnet localhost 143 (for example). As far as I know you won't be able to use SSL/TLS or a different authorization name to your authentication name, though. If you connect this way, you'll need to login manually. Warning - your login credentials will not be encrypted, so this is to be avoided in favour of using imtest whenever possible. The command to log in, which you need to send before you can do much on the server, is: . login username password . This will only work if you server permits plaintext logins over insecure links, which many do not.

Further information

It is strongly suggested that you read RFC2060, the IMAP4r1 RFC. The command summaries, in particular, are very, very useful.

References

RFC2060 - IMAP4r1

-- TWikiGuest (Craig Ringer, craig (at) postnewspapers d*t com dot au) - 13 Apr