Anti-spam software is mostly off-topic for Dovecot, since it is usually done on MTA side. Dovecot does have an way to allow users to train antispam filter by moving mails to spam/ham mailbox, but that's all. Still, this question gets asked once in a while, so here's a page where you're free to give your own opinion about different methods and software. The opinions expressed in this page are not the official opinions of Dovecot authors.
Article on Anti-spam techniques on Wikipedia
SMTP traffic analyzer
Postfix Access Policy Delegation
policyd-weight scores mails based on DNSBLs (RBL/RHSBL), HELO, MAIL FROM and client IP addresses before any queuing is done.
milter-greylist can do greylisting and much more for Sendmail and Postfix. Blacklists, whitelists, and autowhitelists are supported to help reducing the impact of message delays. milter-greylist also features a flexible ACL system that lets you configure complex behaviors : greylist/whitelist/blacklist decision can be made on behalf of sender IP, sender and recipient e-mail addresses, time of day, sender IP presence in a DNSRBL... You can even look up a web service or a LDAP directory to decide what to do, which allows user-modifiable per-recipient configurations. Written in C, no external database required, can sync among multiple MX.
policyd can be integrated with postfix, can do greylisting, auto-whitelisting, HELO checks, RBL checks. Uses a MySQL Database to share its information, so that any of your mail servers know about the greylisting tuples and blacklists/whitelists.
DNS based lookup systems
Wikipedia article on DNSBL
https://blog.returnpath.com/blacklist-basics-the-top-email-blacklists-you-need-to-know-v2/ - A list of some DNSBL providers
Content scanning software
Spam filtering scores can be automatically updated using sa-update
Fast, free and open-source spam filtering system https://www.rspamd.com
Mail/Virus scanner. Can use multiple 'engines' for Virus and Spam scanning, sign and verify DKIM. amavisd-new
DSPAM homepage - Community Driven Antispam Filter (has not been updated since 04/2012)
Cloudmark Authority http://www.cloudmark.com/en/s/products/cloudmark-authority-milter
Cisco Senderbase - http://www.senderbase.org/
Halon Scriptable SMTP - https://halon.io/