This documentation is for Dovecot v2.x, see wiki1 for v1.x documentation.
Differences between revisions 10 and 47 (spanning 37 versions)
Revision 10 as of 2008-04-19 12:11:51
Size: 20428
Comment:
Revision 47 as of 2011-01-20 11:04:01
Size: 1174
Comment: Made page layout and content match Sieve interpreter page.
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
= Dovecot ManageSieve Server = = Pigeonhole ManageSieve Server =
The [[Pigeonhole|Pigeonhole project]] provides [[http://sieve.info|Sieve]] support for Dovecot, which allows users to filter incoming messages by writing scripts specified in the Sieve language (RFC 5228). The Pigeonhole !ManageSieve service is used to manage a user's Sieve script collection. It has the following advantages over doing it directly via filesystem:
Line 3: Line 4:
[[TableOfContents]]  * No need to let users log in via FTP/SFTP/etc, which could be difficult especially with virtual users.
 * !ManageSieve is a [[http://tools.ietf.org/html/rfc5804|standard protocol]], so users can manage their scripts using (hopefully) user-friendly !ManageSieve clients. Many webmails already include a !ManageSieve client.
 * Scripts are compiled before they are installed, which guarantees that the uploaded script is valid. This prevents a user from inadvertently installing a broken Sieve script.
Line 5: Line 8:
== Introduction == == Configuration and Use ==
Line 7: Line 10:
The [:LDA/Sieve:Sieve plugin] for Dovecot's '''''[:LDA:deliver]''''' LDA expects a user's Sieve script to reside somewhere in the user's directory (~/.dovecot.sieve by default). If the user is to be able to change his sieve script, he needs shell or FTP access to his home directory, which is not always desirable. This is especially applicable to mail servers with virtual users. As a solution, the ManageSieve protocol was proposed to manage sieve scripts on the server without the need for direct file system access by the users. Additionally, the Sieve scripts are compiled before they are installed, making sure that the uploaded script is valid. This prevents a user from inadvertently installing a broken Sieve script. The [http://tools.ietf.org/id/draft-martin-managesieve-08.txt protocol specification] still has a draft status, but it is already supported by quite a few (web)mail clients. Dovecot now supports ManageSieve by means of an external patch and package. Alternatively, an external python-based daemon called [http://woozle.org/~neale/src/pysieved/ pysieved] written by Neale Pickett can be used, but this page only describes the native Dovecot implementation.

== Versions and Downloading ==

The ManageSieve daemon is available for both Dovecot v1.0 and v1.1. There is an important architectural difference between the two implementations however. The v1.0 version is a very large patch that includes another copy of the CMU Sieve library as used by deliver's [:LDA/Sieve:Sieve plugin]. In contrast, the v1.1 version is largely implemented as an external package with a small patch to enable ManageSieve service support in Dovecot itself. The v1.1 implementation no longer includes another copy of the CMU Sieve library: it uses the Sieve plugin package for compilation against the Sieve library. With the advent of Dovecot v2.0, the small patch that is now still necessary for Dovecot v1.1 is likely to disappear completely, as Dovecot will then include support for custom services.

The latest versions of the ManageSieve implementation for Dovecot can be downloaded from the following locations:
  * Dovecot v1.0: http://www.rename-it.nl/dovecot/1.0/
   Patch filenames look as follows: '''{{{dovecot-1.0.}}}'''{{{X}}}'''{{{-MANAGESIEVE-v}}}'''{{{<version>}}}'''{{{.diff.gz}}}'''
  * Dovecot v1.1: http://www.rename-it.nl/dovecot/1.1/
   Patch filenames look as follows: '''{{{dovecot-1.1.}}}'''{{{X}}}'''{{{-managesieve-}}}'''{{{<version>}}}'''{{{.diff.gz}}}'''

   Package filenames look as follows: '''{{{dovecot-1.1-managesieve-}}}'''{{{<version>}}}'''{{{.tar.gz}}}'''

The releases are signed with public key 0x3DFBB4F4 which can be found at wwwkeys.pgp.net.

== Compiling ==

=== v1.0 ===

You first need to apply the downloaded {{{.diff.gz}}} patch to your {{{dovecot-1.0}}} source tree. This is achieved by executing the following command line inside the source tree ('{{{../patchfile.diff.gz}}}' must be substituted with the location of the patch file you downloaded):

{{{
gzip -dc ../pathfile.diff.gz | patch -p1
}}}

After applying this patch, the usual {{{./configure}}}, {{{make}}}, {{{make install}}} sequence is not enough. First the {{{automake}}}/{{{autoconf}}} structure needs to be rebuilt to include the ManageSieve sources in the compilation process. This requires {{{autotools}}} to be installed on your system. When you downloaded using Mercurial, you have a script called {{{autogen.sh}}} in your source tree and you should proceed as specified [:CompilingSource:here]. Otherwise, execute the following command inside the dovecot source tree:

{{{
autoreconf -i
}}}

Afterwards, you can continue the usual [:CompilingSource:build process].

=== v1.1 ===

The ManageSieve implementation for v1.1 consists of a patch in {{{.diff.gz}}} format and a separate {{{.tar.gz}}} package. You first need to patch and compile the Dovecot sources. Applying the patch is achieved by executing the following command line inside the dovecot source tree ('{{{../patchfile.diff.gz}}}' must be substituted with the location of the patch file you downloaded):

{{{
gzip -dc ../pathfile.diff.gz | patch -p1
}}}

Once patched, you can compile Dovecot using the usual [:CompilingSource:build process]. So, unlike the v1.0 implementation, you do not need {{{autotools}}}.

Another prerequisite for compiling the ManageSieve service is a compiled dovecot-sieve-1.1 source tree. Compiling the Sieve plugin is described [:LDA/Sieve:here].

Now that you have both a compiled dovecot and a compiled dovecot-sieve source tree, you can continue building the ManageSieve service. Note that the service will compile against an unpatched dovecot tree, but keep in mind that Dovecot will not know about the existence of ManageSieve without the patch. Unpack the {{{.tar.gz}}} package somewhere and execute the following command sequence inside the unpacked source directory:

{{{
./configure --with-dovecot=<dovecot source tree> --with-dovecot-sieve=<dovecot-sieve source tree>
make
sudo make install
}}}

The parameters to {{{./configure}}} represent the following:

 * --with-dovecot=<path>
   Path to the compiled dovecot-1.1 tree


 * --with-dovecot-sieve=<path>
   Path to the compiled dovecot-sieve-1.1 tree

== Configuring ==

'''NOTE''': If you have used the sieve plugin before and you have {{{.dovecot.sieve}}} files in user directories, you are advised to '''make a backup first'''. Although the managesieve daemon takes care to move these files to the sieve storage before it is substituted with a symbolic link, this is not a very well tested operation, meaning that there is a possibility that existing sieve scripts get lost.

Along with all other binaries that dovecot uses, the managesieve and managesieve-login binaries are installed during {{{make install}}}. The only thing you need to do to activate the ManageSieve support in dovecot is to add {{{managesieve}}} to the {{{protocols=}}} configuration line in your dovecot.conf. The managesieve daemon will listen on port 2000 by default. As the implementation of the managesieve daemon is largely based on the original IMAP implementation, it is very similar in terms of configuration. In addition to most [:MainConfig:mail daemon config settings], the managesieve daemon accepts a few more. The following settings can be configured in the {{{protocol managesieve}}} section:

 listen = *:2000:: IP or host address where to listen in for connections.

 login_executable = /usr/libexec/dovecot/managesieve-login:: Login executable location.

 mail_executable = /usr/libexec/dovecot/managesieve:: managesieve executable location. See mail_executable for IMAP for examples how this could be changed.

 managesieve_max_line_length = 65536:: Maximum managesieve command line length in bytes. This setting is directly borrowed from IMAP. But, since long command lines are very unlikely with ManageSieve, changing this will not be very useful.

 sieve_storage = :: This specifies the path to the directory where the uploaded scripts are stored. In terms of '%' variable substitution it is identical to dovecot's {{{mail_location}}} setting used by the mail protocol daemons. Scripts are stored as separate files with extension {{{.sieve}}}, all other files are ignored when scripts are listed by a ManageSieve client. If this setting remains unspecified, the {{{mail_location}}} setting is used as explained below.

 sieve = ~/.dovecot.sieve:: Specifies the location of the symbolic link pointing to the active script in the sieve storage directory. This must match the [:LDA/Sieve#location:sieve setting used by deliver]. Variable substitution with % is recognized. If a regular file exists at this location, it is moved to the {{{sieve_storage}}} location before the symbolic link is installed. It is renamed to {{{dovecot.orig.sieve}}} and therefore listed as {{{dovecot.orig}}} by a ManageSieve client.

 mail_location =:: If, for some inobvious reason, the sieve_storage remains unset, the managesieve daemon uses the specification of the mail_location to find out where to store the sieve files. However, this is provided only for backwards compatibility and you should always use the sieve_storage setting in stead.

 managesieve_implementation_string = dovecot:: To fool ManageSieve clients that are focused on CMU's timesieved you can specify the IMPLEMENTATION capability that the dovecot reports to clients (e.g. '{{{Cyrus timsieved v2.2.13}}}').

Scripts are stored at the location specified by the {{{sieve_storage}}} setting. The active sieve script is managed as a symbolic link pointing to the active script in the sieve storage direcotory. The location of this symlink can be specified with the {{{sieve}}} setting. Make sure this setting is identical to what [:LDA:deliver] is using for the [:LDA/Sieve:Sieve plugin]. The default location is {{{~/.dovecot.sieve}}}. Note that if a file starting with '.' is placed inside a Maildir, it will be recognized as a folder, so try to avoid that.

The current version of the managesieve daemon places the script storage directory in the mail folder as specified by the {{{mail_location}}} setting if no sieve_storage is specified. Actually, it is placed in the {{{CONTROL=}}} directory of {{{mail_location}}} if specified, otherwise the {{{sieve}}} directory is placed in the root of the mail location. In a mail or mail control directory, scripts are always stored in a {{{sieve}}} subdirectory. Note that for some mail storage types (e.g. mbox) this script directory is listed as a mail folder, so be sure to put the sieve scripts somewhere else if you can.

A storage location specified by {{{sieve_storage}}} is always generated automatically if it does not exist (as far as the system permits the user to do so; no root privileges are used). This is similar to the behaviour of the mail daemons. Note that when {{{mail_location}}} is used to specify the script storage location, only the {{{sieve}}} subdirectory is generated automatically.

The following provides an example configuration for ManageSieve in dovecot.conf. Only sections relevant to ManageSieve are shown. Refer to dovecot-example.conf in your patched dovecot tree for a full example with comments, but don't forget to add {{{managesieve}}} to the {{{protocols}}} setting if you use it.

{{{
# Start imap, pop3 and managesieve services
protocols = imap pop3 managesieve

protocol managesieve {
  # Specify an alternative address:port the daemon must listen on
  # (default: *:2000)
  #listen = localhost:2000

  sieve=~/.dovecot.sieve
  sieve_storage=~/sieve
}
}}}

=== Proxy ===

Like Dovecot's imapd, the ManageSieve login daemon supports proxying to multiple backend servers. Although the underlying code is copied from the imapd sources for the most part, it has some ManageSieve-specifics that have not seen much testing. The [:PasswordDatabase/ExtraFields/Proxy:proxy configuration wiki page] for POP3 and IMAP should apply to ManageSieve as well.

== Troubleshooting ==

Like Dovecot itself, '''the ManageSieve service always logs a detailed error message''' if something goes wrong at the server (refer to [:Logging: Dovecot Logging] for more details): the logs are the first place to look if you suspect something is wrong. To get additional debug messages in your log file, you should set {{{mail_debug=yes}}} in [:MainConfig:dovecot.conf] (inside {{{protocol managesieve {...} }}}if you want to enable this for managesieve only).

If the client commits protocol violations or sends invalid scripts, an error response is provided to the client which is not necessarily logged on the server. A good ManageSieve client presents such error messages to the user.

=== Manual Login and Script Upload ===

If you fail to login or upload scripts to the server, it is not necessarily caused by Dovecot or your configuration. It is often best to test your ManageSieve server manually first. This also provides you with the direct error messages from the server without intermission of your client. If you do not use TLS, you can connect using a simple {{{telnet}}} or {{{netcat}}} connection to the configured port. Otherwise you must use a TLS-capable text protocol client like {{{gnutls-cli}}} as described below. Upon connection, the server presents the initial greeting with its capabilities:

{{{
"IMPLEMENTATION" "dovecot"
"SASL" "PLAIN"
"SIEVE" "comparator-i;ascii-numeric fileinto reject vacation imapflags notify include envelope body relational regex subaddress copy"
"STARTTLS"
OK "Dovecot ready."
}}}

Note that the reported {{{STARTTLS}}} capability means that the server accepts TLS, but, since you are using telnet/netcat, you cannot use this (refer to Manual TLS Login below). The {{{SASL}}} capability lists the available SASL authentication mechanisms. If this list is empty and {{{STARTTLS}}} is available, it probably means that the server forces you to initiate TLS first (as dictated by '{{{disable_plaintext_auth=yes}}}' in [:MainConfig:dovecot.conf]).

Now you need to log in. Although potentially multiple SASL mechanisms are available, only {{{PLAIN}}} is described here. Authentication is performed using the ManageSieve {{{AUTHENTICATE}}} command. This command typically looks as follows when the {{{PLAIN}}} mechanism is used:

{{{
AUTHENTICATE "PLAIN" "<base64-encoded credentials>"
}}}

The credentials are the base64-encoded version of the string {{{"\0<username>\0<password"}}} (in which {{{\0}}} represents the ASCII NUL character). Generating this is cumbersome and a bit daunting for the novice user, so for convenience a simple Perl script is provided to generate the {{{AUTHENTICATE}}} command for you. It is available [attachment:sieve-auth-command.pl here] and used as follows:
{{{
sieve-auth-command.pl <username> <password>
}}}

The command is written to stdout and you can paste this to your protocol session, e.g.:
{{{
AUTHENTICATE "PLAIN" "AHVzZXJuYW1lAHBhc3N3b3Jk"
OK "Logged in."
}}}

Now that you are logged in, you can upload a script. This is done using the {{{PUTSCRIPT}}} command. Its first argument is the name for the script and its second argument is a string literal. A string literal starts with a length specification {{{'{<bytes>}'}}} followed by a newline. Thereafter the server expects {{{<bytes>}}} bytes of script data. The following uploads a trivial 6 byte long sieve script that keeps every message (6th byte is the newline character):
{{{
PUTSCRIPT "hutsefluts" {6}
keep;
OK "Putscript completed."
}}}

Upon successful upload, you should find a file called {{{hutsefluts.sieve}}} in your {{{sieve_storage}}} directory. The script should also be listed by the server as follows when the {{{LISTSCRIPTS}}} command is issued:
{{{
LISTSCRIPTS
"hutsefluts"
OK "Listscripts completed."
}}}

You can check whether your script is uploaded correctly by downloading it using the {{{GETSCRIPT}}} command. This command accepts the name of the downloaded script as its only parameter:
{{{
GETSCRIPT "hutsefluts"
{6}
keep;
OK "Getscript completed."
}}}

To let the Sieve plugin use your newly uploaded script, you must activate it using the {{{SETACTIVE}}} command (only one script can be active at any time). The active script is indicated {{{ACTIVE}}} in the {{{LISTSCRIPTS}}} output, e.g.:
{{{
SETACTIVE "hutsefluts"
OK "Setactive completed."
LISTSCRIPTS
"hutsefluts" ACTIVE
OK "Listscripts completed.
}}}

The symbolic link configured with the {{{sieve}}} setting should now point to the activated script in the {{{sieve_storage}}} directory. If no script is active, this symbolic link is absent.

=== Manual TLS Login ===

When TLS needs to be used during manual testing, {{{gnutls-cli}}} provides the means to do so. This command-line utility is part of the GNUTLS distribution and on most systems this should be easy to install. It is used to connect to ManageSieve as follows:

{{{
gnutls-cli --starttls -p <port> <host>
}}}

This starts the client in plain text mode first. As shown in the previous section, the server presents a greeting with all capabilities of the server. If {{{STARTTLS}}} is listed, you can issue the {{{STARTTLS}}} command as follows:
{{{
STARTTLS
OK "Begin TLS negotiation now."
}}}

If an OK response is given by the server you can press {{{Ctrl-D}}} to make {{{gnutls-cli}}} start the TLS negotiation. Upon pressing {{{Ctrl-D}}}, {{{gnutls-cli}}} will show information on the negotiated TLS session and finally the first response of the server is shown:
{{{
"IMPLEMENTATION" "dovecot"
"SASL" "PLAIN"
"SIEVE" "comparator-i;ascii-numeric fileinto reject vacation imapflags notify include envelope body relational regex subaddress copy"
OK "TLS negotiation successful."
}}}

Hereafter, you can continue to authenticate and upload a script as described in the previous section.

=== Client Problems ===

If manual efforts to upload a script are successful, but your client still fails, you need to obtain a view on what the client communicates with the server. A common method is to sniff the client protocol session using a tool like {{{ngrep}}}. However, this will not work when TLS is active. If the problem is not specific to TLS, you are advised to temporarily turn off TLS and sniff the plain text protocol. If TLS is part of the issue, you can use Dovecot's [:Debugging/Rawlog: rawlog] facility to see what is going on if the client is logged in. If the authentication is the problem, there is no real nice way to obtain a transcript of the protocol. One way is to run managesieve from inetd, wrapping it into a script that writes the protocol messages somewhere ('''FIXME''': This needs some checking and explanation). Alternatively, if possible, the client can be altered to write its protocol messages somewhere.

== Known Issues ==

 *Although this ManageSieve server should comply with the draft specification of the ManageSieve protocol, quite a few clients don't. This is particularly true for the TLS support. However, now that Cyrus' Timsieved has changed its behaviour towards protocol compliance, all those clients will follow eventually.
 
 The core of the TLS problem is that a ManageSieve server is required to send an unsolicited CAPABILITY response right after successful TLS negotiation. Older Cyrus servers did not do this and many clients incorporated this protocol error as the standard, meaning that these do not expect the CAPABILITY response and thus fail with subsequent commands.
 
 The following clients are known to have this TLS issue:


  Thunderbird Sieve add-on:: (author is working on it)

  KMail + kio_sieve:: TLS remains broken.

  SquirrelMail/AvelSieve:: For some users the Avelsieve client stores scripts but fails to retrieve them later. This problem is directly caused by AvelSieve's TLS support. A quick way to fix this is not to enable TLS for ManageSieve. AvelSieve stable (v1.0.1) does not have TLS support at all, so you will see this happen only with the development or SVN versions.

  Another issue is that (at least with avelsieve-1.9.7) it is impossible to delete the last rule in a script.

  For avelsieve-1.9.7 you find a patch that fixes these two issues [attachment:avelsieve-1.9.7-dovecot.diff here].

 *Other client issues:
  SmartSieve, WebSieve:: These clients are specifically written for Cyrus timsieved and fail on multiple stages of the protocol when connected to Dovecot ManageSieve.
    ''(Stephan Bosch)'': I intend to look at these in the future, but currently these are very much unavailable for use with Dovecot. But, feel free to fix these yourself :)

 *The current implementation of the daemon does not have quota enforcement as recommended in the specification. So keep in mind that malicious users could fill your file system with loads of spurious script files.
 *The ANONYMOUS authentication mechanism is currently not supported and explicitly denied.

== Contact Info ==

 * Author: Stephan Bosch, stephan@rename-it.nl
 * IRC: Freenode, #dovecot, S[r]us
 * Please use the Dovecot [http://www.dovecot.org/mailinglists.html mailing list] for questions about the ManageSieve service. You don't have to subscribe to it.
 * [[Pigeonhole/Installation|Download and Installation]]
 * [[Pigeonhole/ManageSieve/Configuration|Configuration]]
 * [[Pigeonhole/ManageSieve/Troubleshooting|Troubleshooting]]
 * [[Pigeonhole/ManageSieve/Clients|Client Issues]]

Pigeonhole ManageSieve Server

The Pigeonhole project provides Sieve support for Dovecot, which allows users to filter incoming messages by writing scripts specified in the Sieve language (RFC 5228). The Pigeonhole ManageSieve service is used to manage a user's Sieve script collection. It has the following advantages over doing it directly via filesystem:

  • No need to let users log in via FTP/SFTP/etc, which could be difficult especially with virtual users.
  • ManageSieve is a standard protocol, so users can manage their scripts using (hopefully) user-friendly ManageSieve clients. Many webmails already include a ManageSieve client.

  • Scripts are compiled before they are installed, which guarantees that the uploaded script is valid. This prevents a user from inadvertently installing a broken Sieve script.

Configuration and Use

None: Pigeonhole/ManageSieve (last edited 2019-09-12 08:43:59 by MichaelSlusarz)