This documentation is for Dovecot v2.x, see wiki1 for v1.x documentation.
Differences between revisions 5 and 6
Revision 5 as of 2008-04-16 12:50:20
Size: 20029
Comment: Workaround for avelsieve
Revision 6 as of 2008-04-16 20:12:06
Size: 20149
Comment:
Deletions are marked like this. Additions are marked like this.
Line 233: Line 233:
  I experienced exactly the same problem, although in my case it was not TLS-related. The problem is that AvelSieve cannot retrieve the list of stored scripts. The reason is, that dovecot's reply to LISTSCRIPTS is:   I experienced exactly the same problem, although in my case it was not TLS-related. The problem is that AvelSieve cannot retrieve the list of stored scripts. The reason for it is not entirely clear. It looks like dovecot's reply to LISTSCRIPTS is
Line 238: Line 238:
  AvelSieve does not expect the first line and thus fails to get the list of scripts. An easy and dirty way around this is to add the line   although it could also be, that AvelSieve does not fully process previous replies from dovecot. The issue is being investigated.

  In any case, AvelSieve does not
expect the first line and thus fails to get the list of scripts. An easy and dirty way around this is to add the line
Line 242: Line 244:
  to managesieve.lib.php on line 656. The author of AvelSieve has been contacted about this issue.   to managesieve.lib.php on line 656.

Dovecot ManageSieve Server

TableOfContents

Introduction

The [:LDA/Sieve:Sieve plugin] for Dovecot's [:LDA:deliver] LDA expects a user's Sieve script to reside somewhere in the user's directory (~/.dovecot.sieve by default). If the user is to be able to change his sieve script, he needs shell or FTP access to his home directory, which is not always desirable. This is especially applicable to mail servers with virtual users. As a solution, the ManageSieve protocol was proposed to manage sieve scripts on the server without the need for direct file system access by the users. Additionally, the Sieve scripts are compiled before they are installed, making sure that the uploaded script is valid. This prevents a user from inadvertently installing a broken Sieve script. The [http://tools.ietf.org/id/draft-martin-managesieve-08.txt protocol specification] still has a draft status, but it is already supported by quite a few (web)mail clients. Dovecot now supports ManageSieve by means of an external patch and package. Alternatively, an external python-based daemon called [http://woozle.org/~neale/src/pysieved/ pysieved] written by Neale Pickett can be used, but this page only describes the native Dovecot implementation.

Versions and Downloading

The ManageSieve daemon is available for both Dovecot v1.0 and v1.1. There is an important architectural difference between the two implementations however. The v1.0 version is a very large patch that includes another copy of the CMU Sieve library as used by deliver's [:LDA/Sieve:Sieve plugin]. In contrast, the v1.1 version is largely implemented as an external package with a small patch to enable ManageSieve service support in Dovecot itself. The v1.1 implementation no longer includes another copy of the CMU Sieve library: it uses the Sieve plugin package for compilation against the Sieve library. With the advent of Dovecot v2.0, the small patch that is now still necessary for Dovecot v1.1 is likely to disappear completely, as Dovecot will then include support for custom services.

The latest versions of the ManageSieve implementation for Dovecot can be downloaded from the following locations:

The releases are signed with public key 0x3DFBB4F4 which can be found at wwwkeys.pgp.net.

Compiling

v1.0

You first need to apply the downloaded .diff.gz patch to your dovecot-1.0 source tree. This is achieved by executing the following command line inside the source tree ('../patchfile.diff.gz' must be substituted with the location of the patch file you downloaded):

gzip -dc ../pathfile.diff.gz | patch -p1

After applying this patch, the usual ./configure, make, make install sequence is not enough. First the automake/autoconf structure needs to be rebuilt to include the ManageSieve sources in the compilation process. This requires autotools to be installed on your system. When you downloaded using Mercurial, you have a script called autogen.sh in your source tree and you should proceed as specified [:CompilingSource:here]. Otherwise, execute the following command inside the dovecot source tree:

autoreconf -i

Afterwards, you can continue the usual [:CompilingSource:build process].

v1.1

The ManageSieve implementation for v1.1 consists of a patch in .diff.gz format and a separate .tar.gz package. You first need to patch and compile the Dovecot sources. Applying the patch is achieved by executing the following command line inside the dovecot source tree ('../patchfile.diff.gz' must be substituted with the location of the patch file you downloaded):

gzip -dc ../pathfile.diff.gz | patch -p1

Once patched, you can compile Dovecot using the usual [:CompilingSource:build process]. So, unlike the v1.0 implementation, you do not need autotools.

Another prerequisite for compiling the ManageSieve service is a compiled dovecot-sieve-1.1 source tree. Compiling the Sieve plugin is described [:LDA/Sieve:here].

Now that you have both a compiled dovecot and a compiled dovecot-sieve source tree, you can continue building the ManageSieve service. Note that the service will compile against an unpatched dovecot tree, but keep in mind that Dovecot will not know about the existence of ManageSieve without the patch. Unpack the .tar.gz package somewhere and execute the following command sequence inside the unpacked source directory:

./configure --with-dovecot=<dovecot source tree> --with-dovecot-sieve=<dovecot-sieve source tree>
make
sudo make install

The parameters to ./configure represent the following:

  • --with-dovecot=<path>

    • Path to the compiled dovecot-1.1 tree
  • --with-dovecot-sieve=<path>

    • Path to the compiled dovecot-sieve-1.1 tree

Configuring

NOTE: If you have used the sieve plugin before and you have .dovecot.sieve files in user directories, you are advised to make a backup first. Although the managesieve daemon takes care to move these files to the sieve storage before it is substituted with a symbolic link, this is not a very well tested operation, meaning that there is a possibility that existing sieve scripts get lost.

Along with all other binaries that dovecot uses, the managesieve and managesieve-login binaries are installed during make install. The only thing you need to do to activate the ManageSieve support in dovecot is to add managesieve to the protocols= configuration line in your dovecot.conf. The managesieve daemon will listen on port 2000 by default. As the implementation of the managesieve daemon is largely based on the original IMAP implementation, it is very similar in terms of configuration. In addition to most [:MainConfig:mail daemon config settings], the managesieve daemon accepts a few more. The following settings can be configured in the protocol managesieve section:

listen = *:2000
IP or host address where to listen in for connections.
login_executable = /usr/libexec/dovecot/managesieve-login
Login executable location.
mail_executable = /usr/libexec/dovecot/managesieve
managesieve executable location. See mail_executable for IMAP for examples how this could be changed.
managesieve_max_line_length = 65536

Maximum managesieve command line length in bytes. This setting is directly borrowed from IMAP. But, since long command lines are very unlikely with ManageSieve, changing this will not be very useful.

sieve_storage =

This specifies the path to the directory where the uploaded scripts are stored. In terms of '%' variable substitution it is identical to dovecot's mail_location setting used by the mail protocol daemons. Scripts are stored as separate files with extension .sieve, all other files are ignored when scripts are listed by a ManageSieve client. If this setting remains unspecified, the mail_location setting is used as explained below.

sieve = ~/.dovecot.sieve

Specifies the location of the symbolic link pointing to the active script in the sieve storage directory. This must match the [:LDA/Sieve#location:sieve setting used by deliver]. Variable substitution with % is recognized. If a regular file exists at this location, it is moved to the sieve_storage location before the symbolic link is installed. It is renamed to dovecot.orig.sieve and therefore listed as dovecot.orig by a ManageSieve client.

mail_location =
If, for some inobvious reason, the sieve_storage remains unset, the managesieve daemon uses the specification of the mail_location to find out where to store the sieve files. However, this is provided only for backwards compatibility and you should always use the sieve_storage setting in stead.
managesieve_implementation_string = dovecot

To fool ManageSieve clients that are focused on CMU's timesieved you can specify the IMPLEMENTATION capability that the dovecot reports to clients (e.g. 'Cyrus timsieved v2.2.13').

Scripts are stored at the location specified by the sieve_storage setting. The active sieve script is managed as a symbolic link pointing to the active script in the sieve storage direcotory. The location of this symlink can be specified with the sieve setting. Make sure this setting is identical to what [:LDA:deliver] is using for the [:LDA/Sieve:Sieve plugin]. The default location is ~/.dovecot.sieve. Note that if a file starting with '.' is placed inside a Maildir, it will be recognized as a folder, so try to avoid that.

The current version of the managesieve daemon places the script storage directory in the mail folder as specified by the mail_location setting if no sieve_storage is specified. Actually, it is placed in the CONTROL= directory of mail_location if specified, otherwise the sieve directory is placed in the root of the mail location. In a mail or mail control directory, scripts are always stored in a sieve subdirectory. Note that for some mail storage types (e.g. mbox) this script directory is listed as a mail folder, so be sure to put the sieve scripts somewhere else if you can.

A storage location specified by sieve_storage is always generated automatically if it does not exist (as far as the system permits the user to do so; no root privileges are used). This is similar to the behaviour of the mail daemons. Note that when mail_location is used to specify the script storage location, only the sieve subdirectory is generated automatically.

The following provides an example configuration for ManageSieve in dovecot.conf. Only sections relevant to ManageSieve are shown. Refer to dovecot-example.conf in your patched dovecot tree for a full example with comments, but don't forget to add managesieve to the protocols setting if you use it.

# Start imap, pop3 and managesieve services
protocols = imap pop3 managesieve

protocol managesieve {
  # Specify an alternative address:port the daemon must listen on
  # (default: *:2000)
  #listen = localhost:2000

  sieve=~/.dovecot.sieve
  sieve_storage=~/sieve
}

Proxy

Like Dovecot's imapd, the ManageSieve login daemon supports proxying to multiple backend servers. Although the underlying code is copied from the imapd sources for the most part, it has some ManageSieve-specifics that have not seen much testing. The [:PasswordDatabase/ExtraFields/Proxy:proxy configuration wiki page] for POP3 and IMAP should apply to ManageSieve as well.

Troubleshooting

Like Dovecot itself, the ManageSieve service always logs a detailed error message if something goes wrong at the server (refer to [:Logging: Dovecot Logging] for more details): the logs are the first place to look if you suspect something is wrong. To get additional debug messages in your log file, you should set mail_debug=yes in [:MainConfig:dovecot.conf] (inside protocol managesieve {...} if you want to enable this for managesieve only).

If the client commits protocol violations or sends invalid scripts, an error response is provided to the client which is not necessarily logged on the server. A good ManageSieve client presents such error messages to the user.

Manual Login and Script Upload

If you fail to login or upload scripts to the server, it is not necessarily caused by Dovecot or your configuration. It is often best to test your ManageSieve server manually first. This also provides you with the direct error messages from the server without intermission of your client. If you do not use TLS, you can connect using a simple telnet or netcat connection to the configured port. Otherwise you must use a TLS-capable text protocol client like gnutls-cli as described below. Upon connection, the server presents the initial greeting with its capabilities:

"IMPLEMENTATION" "dovecot"
"SASL" "PLAIN"
"SIEVE" "comparator-i;ascii-numeric fileinto reject vacation imapflags notify include envelope body relational regex subaddress copy"
"STARTTLS"
OK "Dovecot ready."

Note that the reported STARTTLS capability means that the server accepts TLS, but, since you are using telnet/netcat, you cannot use this (refer to Manual TLS Login below). The SASL capability lists the available SASL authentication mechanisms. If this list is empty and STARTTLS is available, it probably means that the server forces you to initiate TLS first (as dictated by 'disable_plaintext_auth=yes' in [:MainConfig:dovecot.conf]).

Now you need to log in. Although potentially multiple SASL mechanisms are available, only PLAIN is described here. Authentication is performed using the ManageSieve AUTHENTICATE command. This command typically looks as follows when the PLAIN mechanism is used:

AUTHENTICATE "PLAIN" "<base64-encoded credentials>"

The credentials are the base64-encoded version of the string "\0<username>\0<password" (in which \0 represents the ASCII NUL character). Generating this is cumbersome and a bit daunting for the novice user, so for convenience a simple Perl script is provided to generate the AUTHENTICATE command for you. It is available [attachment:sieve-auth-command.pl here] and used as follows:

sieve-auth-command.pl <username> <password>

The command is written to stdout and you can paste this to your protocol session, e.g.:

AUTHENTICATE "PLAIN" "AHVzZXJuYW1lAHBhc3N3b3Jk"
OK "Logged in."

Now that you are logged in, you can upload a script. This is done using the PUTSCRIPT command. Its first argument is the name for the script and its second argument is a string literal. A string literal starts with a length specification '{<bytes>}' followed by a newline. Thereafter the server expects <bytes> bytes of script data. The following uploads a trivial 6 byte long sieve script that keeps every message (6th byte is the newline character):

PUTSCRIPT "hutsefluts" {6}
keep;
OK "Putscript completed."

Upon successful upload, you should find a file called hutsefluts.sieve in your sieve_storage directory. The script should also be listed by the server as follows when the LISTSCRIPTS command is issued:

LISTSCRIPTS
"hutsefluts"
OK "Listscripts completed."

You can check whether your script is uploaded correctly by downloading it using the GETSCRIPT command. This command accepts the name of the downloaded script as its only parameter:

GETSCRIPT "hutsefluts"
{6}
keep;
OK "Getscript completed."

To let the Sieve plugin use your newly uploaded script, you must activate it using the SETACTIVE command (only one script can be active at any time). The active script is indicated ACTIVE in the LISTSCRIPTS output, e.g.:

SETACTIVE "hutsefluts"
OK "Setactive completed."
LISTSCRIPTS
"hutsefluts" ACTIVE
OK "Listscripts completed.

The symbolic link configured with the sieve setting should now point to the activated script in the sieve_storage directory. If no script is active, this symbolic link is absent.

Manual TLS Login

When TLS needs to be used during manual testing, gnutls-cli provides the means to do so. This command-line utility is part of the GNUTLS distribution and on most systems this should be easy to install. It is used to connect to ManageSieve as follows:

gnutls-cli --starttls -p <port> <host>

This starts the client in plain text mode first. As shown in the previous section, the server presents a greeting with all capabilities of the server. If STARTTLS is listed, you can issue the STARTTLS command as follows:

STARTTLS
OK "Begin TLS negotiation now."

If an OK response is given by the server you can press Ctrl-D to make gnutls-cli start the TLS negotiation. Upon pressing Ctrl-D, gnutls-cli will show information on the negotiated TLS session and finally the first response of the server is shown:

"IMPLEMENTATION" "dovecot"
"SASL" "PLAIN"
"SIEVE" "comparator-i;ascii-numeric fileinto reject vacation imapflags notify include envelope body relational regex subaddress copy"
OK "TLS negotiation successful."

Hereafter, you can continue to authenticate and upload a script as described in the previous section.

Client Problems

If manual efforts to upload a script are successful, but your client still fails, you need to obtain a view on what the client communicates with the server. A common method is to sniff the client protocol session using a tool like ngrep. However, this will not work when TLS is active. If the problem is not specific to TLS, you are advised to temporarily turn off TLS and sniff the plain text protocol. If TLS is part of the issue, you can use Dovecot's [:Debugging/Rawlog: rawlog] facility to see what is going on if the client is logged in. If the authentication is the problem, there is no real nice way to obtain a transcript of the protocol. One way is to run managesieve from inetd, wrapping it into a script that writes the protocol messages somewhere (FIXME: This needs some checking and explanation). Alternatively, if possible, the client can be altered to write its protocol messages somewhere.

Known Issues

  • Although this ManageSieve server should comply with the draft specification of the ManageSieve protocol, quite a few clients don't. This is particularly true for the TLS support. However, now that Cyrus' Timsieved has changed its behaviour towards protocol compliance, all those clients will follow eventually. The following clients are known to have TLS issues:

    Thunderbird Sieve add-on
    (author is working on it)
    KMail + kio_sieve
    Unfortunately, there is no reliable way to provide a workaround for this problem. We will have to wait for the authors of these clients to make the proper adjustments.
  • Other client issues:
    SquirrelMail/AvelSieve
    For some users the Avelsieve client stores scripts but fails to retrieve them later. This problem is somehow hard to reproduce at my end. Someone suggested that it might be TLS-related.

    I experienced exactly the same problem, although in my case it was not TLS-related. The problem is that AvelSieve cannot retrieve the list of stored scripts. The reason for it is not entirely clear. It looks like dovecot's reply to LISTSCRIPTS is "IMPLEMENTATION" "dovecot" [list of scripts]

    although it could also be, that AvelSieve does not fully process previous replies from dovecot. The issue is being investigated.

    In any case, AvelSieve does not expect the first line and thus fails to get the list of scripts. An easy and dirty way around this is to add the line sieve::get_response(); to managesieve.lib.php on line 656.

  • The current implementation of the daemon does not have quota enforcement as recommended in the specification. So keep in mind that malicious users could fill your file system with loads of spurious script files.
  • The ANONYMOUS authentication mechanism is currently not supported and explicitly denied.

Contact Info

None: Pigeonhole/ManageSieve (last edited 2011-01-20 11:04:01 by Stephan Bosch)