MFH: r438222 r438323 r438365 r439618 r439854

This contains updates to both dovecot2 and dovecot2-pigeonhole that
fix bugs and, in dovecot2, a CVE.

Update dovecot to 2.2.29, and bump PORTREVISION for the plugins. Add a
warning to the pkg-message that security.bsd.see_other_uids/gids should
not be enabled if dovecot is storing mail for multiple users concurrently
(PR 218392, submitted by topical).

 * passdb/userdb dict: Don't double-expand %variables in keys. If dict
   was used as the authentication passdb, using specially crafted
   %variables in the username could be used to cause DoS (CVE-2017-2669)
 * When Dovecot encounters an internal error, it logs the real error and
   usually logs another line saying what function failed. Previously the
   second log line's error message was a rather uninformative "Internal
   error occurred. Refer to server log for more information." Now the
   real error message is duplicated in this second log line.
 * lmtp: If a delivery has multiple recipients, run autoexpunging only
   for the last recipient. This avoids a problem where a long
   autoexpunge run causes LMTP client to timeout between the DATA
   replies, resulting in duplicate mail deliveries.
 * config: Don't stop the process due to idling. Otherwise the
   configuration is reloaded when the process restarts.
 * mail_log plugin: Differentiate autoexpunges from regular expunges
 * imapc: Use LOGOUT to cleanly disconnect from server.
 * lib-http: Internal status codes (>9000) are no longer visible in logs
 * director: Log vhost count changes and HOST-UP/DOWN

 + quota: Add plugin { quota_max_mail_size } setting to limit the
   maximum individual mail size that can be saved.
 + imapc: Add imapc_features=delay-login. If set, connecting to the
   remote IMAP server isn't done until it's necessary.
 + imapc: Add imapc_connection_retry_count and
   imapc_connection_retry_interval settings.
 + imap, pop3, indexer-worker: Add (deinit) to process title before
   autoexpunging runs.
 + Added %{encrypt} and %{decrypt} variables
 + imap/pop3 proxy: Log proxy state in errors as human-readable string.
 + imap/pop3-login: All forward_* extra fields returned by passdb are
   sent to the next hop when proxying using ID/XCLIENT commands. On the
   receiving side these fields are imported and sent to auth process
   where they're accessible via %{passdb:forward_*}. This is done only
   if the sending IP address matches login_trusted_networks.
 + imap-login: If imap_id_retain=yes, send the IMAP ID string to
   auth process. %{client_id} expands to it in auth process. The ID
   string is also sent to the next hop when proxying.
 + passdb imap: Use ssl_client_ca_* settings for CA validation.
 - fts-tika: Fixed crash when parsing attachment without
   Content-Disposition header. Broken by 2.2.28. (fixed in FreeBSD ports)
 - trash plugin was broken in 2.2.28 (fixed in FreeBSD ports)
 - auth: When passdb/userdb lookups were done via auth-workers, too much
   data was added to auth cache. This could have resulted in wrong
   replies when using multiple passdbs/userdbs.
 - auth: passdb { skip & mechanisms } were ignored for the first passdb
 - oauth2: Various fixes, including fixes to crashes
 - dsync: Large Sieve scripts (or other large metadata) weren't always
   synced.
 - Index rebuild (e.g. doveadm force-resync) set all mails as \Recent
 - imap-hibernate: %{userdb:*} wasn't expanded in mail_log_prefix
 - doveadm: Exit codes weren't preserved when proxying commands via
   doveadm-server. Almost all errors used exit code 75 (tempfail).
 - ACLs weren't applied to not-yet-existing autocreated mailboxes.
 - Fixed a potential crash when parsing a broken message header.
 - cassandra: Fallback consistency settings weren't working correctly.
 - doveadm director status <user>: "Initial config" was always empty
 - imapc: Various reconnection fixes.

Upgrade mail/dovecot2-pigeonhole to 0.4.18.

Changelog v0.4.18:

+ imapsieve plugin: Implemented the copy_source_after rule action. When
  this is enabled for a mailbox rule, the specified Sieve script is
  executed for the message in the source mailbox during a "COPY" event.
  This happens only after the Sieve script that is executed for the
  corresponding message in the destination mailbox finishes running
  successfully.
+ imapsieve plugin: Added non-standard Sieve environment items for the
  source and destination mailbox.
- multiscript: The execution of the discard script had an implicit
  "keep", rather than an implicit "discard".

Approved by:	adamw (mentor)
Differential Revision:	https://reviews.freebsd.org/D10366

Update to 2.2.29.1.

  - imapc reconnection fix was forgotten from 2.2.29 release, which also
    made "make check" fail in a unit test
  - dict-sql: Merging multiple UPDATEs to a single statement wasn't
    actually working.
  - Fixed building with vpopmail

Upon continuing the deferred implicit keep, the implicit side-effects
(such as imap flags) were not applied.

Obtained from:	https://github.com/dovecot/pigeonhole/commit/3e1a17a286ab0e084577fc267a442cb12aed1cbc
Approved by: adamw (mentor, implicit)

Add an alread-upstreamed patch to fix dovecot-auth wedging with
NTLM authentication.

PR:		218693
Submitted by:	Andriy Syrovenko
Obtained from:	https://github.com/dovecot/core/commit/a319c3201bff1ea7bae3e7ab1fae42e9c4759056

Approved by:	ports-secteam (feld)

11 files changed, 103 insertions, 89 deletions

diff --git a/mail/dovecot2-antispam-plugin/Makefile b/mail/dovecot2-antispam-plugin/Makefile
index 20d9fd0..04a52c9 100644
--- a/mail/dovecot2-antispam-plugin/Makefile
+++ b/mail/dovecot2-antispam-plugin/Makefile
@@ -3,7 +3,7 @@
 
 PORTNAME=	dovecot2-antispam-plugin
 PORTVERSION=	20130429
-PORTREVISION=	25
+PORTREVISION=	26
 CATEGORIES=	mail
 MASTER_SITES=	http://olgeni.olgeni.com/~olgeni/distfiles/ \
 		LOCAL/olgeni
diff --git a/mail/dovecot2-pigeonhole/Makefile b/mail/dovecot2-pigeonhole/Makefile
index 2f80b35..93c27c3 100644
--- a/mail/dovecot2-pigeonhole/Makefile
+++ b/mail/dovecot2-pigeonhole/Makefile
@@ -2,7 +2,7 @@
 # $FreeBSD$
 
 PORTNAME=	dovecot-pigeonhole
-PORTVERSION=	0.4.17
+PORTVERSION=	0.4.18
 PORTREVISION=	1
 CATEGORIES=	mail
 MASTER_SITES=	http://pigeonhole.dovecot.org/releases/${DOVECOTVERSION}/
diff --git a/mail/dovecot2-pigeonhole/distinfo b/mail/dovecot2-pigeonhole/distinfo
index 6042c9b..63bfb81 100644
--- a/mail/dovecot2-pigeonhole/distinfo
+++ b/mail/dovecot2-pigeonhole/distinfo
@@ -1,3 +1,3 @@
-TIMESTAMP = 1488163544
-SHA256 (dovecot-2.2-pigeonhole-0.4.17.tar.gz) = 74d869c7532cbf4fe41e3cc95a1aa6ce32e98f4d423f0d099da1e0fba022dae3
-SIZE (dovecot-2.2-pigeonhole-0.4.17.tar.gz) = 1787177
+TIMESTAMP = 1491958585
+SHA256 (dovecot-2.2-pigeonhole-0.4.18.tar.gz) = dd871bb57fad22795460f613f3c9484a8bf229272ac00956d837a34444f1c3a9
+SIZE (dovecot-2.2-pigeonhole-0.4.18.tar.gz) = 1742357
diff --git a/mail/dovecot2-pigeonhole/files/patch-src_lib-sieve_sieve-result.c b/mail/dovecot2-pigeonhole/files/patch-src_lib-sieve_sieve-result.c
new file mode 100644
index 0000000..e1ebc85
--- /dev/null
+++ b/mail/dovecot2-pigeonhole/files/patch-src_lib-sieve_sieve-result.c
@@ -0,0 +1,42 @@
+From 3e1a17a286ab0e084577fc267a442cb12aed1cbc Mon Sep 17 00:00:00 2001
+From: Stephan Bosch <stephan.bosch@dovecot.fi>
+Date: Fri, 28 Apr 2017 00:02:39 +0200
+Subject: [PATCH] lib-sieve: Fixed bug in handling of deferred implicit keep
+ with implicit side-effects.
+
+Upon continuing the deferred implicit keep, the implicit side-effects (such as imap flags) were not applied.
+---
+ src/lib-sieve/sieve-result.c | 12 +++++++-----
+ 1 file changed, 7 insertions(+), 5 deletions(-)
+
+diff --git a/src/lib-sieve/sieve-result.c b/src/lib-sieve/sieve-result.c
+index 3c896f2..fc75063 100644
+--- src/lib-sieve/sieve-result.c
++++ src/lib-sieve/sieve-result.c
+@@ -969,7 +969,14 @@ static int _sieve_result_implicit_keep
+ 
+ 			rac = rac->next;
+ 		}
++	} else if ( !rollback ) {
++		act_keep.location = kac->action.location;
++		act_keep.mail = kac->action.mail;
++		if ( kac->seffects != NULL )
++			rsef_first = kac->seffects->first_effect;
++	}
+ 
++	if (rsef_first == NULL) {
+ 		/* Apply any implicit side effects if applicable */
+ 		if ( !rollback && hash_table_is_created(result->action_contexts) ) {
+ 			struct sieve_result_action_context *actctx;
+@@ -980,11 +987,6 @@ static int _sieve_result_implicit_keep
+ 			if ( actctx != NULL && actctx->seffects != NULL )
+ 				rsef_first = actctx->seffects->first_effect;
+ 		}
+-	} else if ( !rollback ) {
+-		act_keep.location = kac->action.location;
+-		act_keep.mail = kac->action.mail;
+-		if ( kac->seffects != NULL )
+-			rsef_first = kac->seffects->first_effect;
+ 	}
+ 
+ 	/* Start keep action */
diff --git a/mail/dovecot2/Makefile b/mail/dovecot2/Makefile
index e2a61ae..2d6b168 100644
--- a/mail/dovecot2/Makefile
+++ b/mail/dovecot2/Makefile
@@ -13,10 +13,10 @@
 ######################################################################
 
 PORTNAME=	dovecot
-PORTVERSION=	2.2.28
-PORTREVISION=	2
+PORTVERSION=	2.2.29.1
+PORTREVISION=	1
 CATEGORIES=	mail ipv6
-MASTER_SITES=	https://www.dovecot.org/releases/${PORTVERSION:R}/
+MASTER_SITES=	https://www.dovecot.org/releases/${PORTVERSION:R:R}/
 PKGNAMESUFFIX=	2
 
 MAINTAINER=	adamw@FreeBSD.org
diff --git a/mail/dovecot2/distinfo b/mail/dovecot2/distinfo
index d2fdc3d..372956b 100644
--- a/mail/dovecot2/distinfo
+++ b/mail/dovecot2/distinfo
@@ -1,3 +1,3 @@
-TIMESTAMP = 1487948861
-SHA256 (dovecot-2.2.28.tar.gz) = e0288f59e326ab87cb3881fdabadafe542f4dc7ab9996db13863a439ebbc1f25
-SIZE (dovecot-2.2.28.tar.gz) = 5921992
+TIMESTAMP = 1492013710
+SHA256 (dovecot-2.2.29.1.tar.gz) = ccfa9ffb7eb91e9e87c21c108324b911250c9ffa838bffb64b1caafadcb0f388
+SIZE (dovecot-2.2.29.1.tar.gz) = 5972119
diff --git a/mail/dovecot2/files/patch-fix-ntlm_auth b/mail/dovecot2/files/patch-fix-ntlm_auth
new file mode 100644
index 0000000..28e1040
--- /dev/null
+++ b/mail/dovecot2/files/patch-fix-ntlm_auth
@@ -0,0 +1,36 @@
+From a319c3201bff1ea7bae3e7ab1fae42e9c4759056 Mon Sep 17 00:00:00 2001
+From: Andriy Syrovenko <andriys@gmail.com>
+Date: Mon, 17 Apr 2017 01:14:02 +0300
+Subject: [PATCH] auth: Fixed dovecot/auth hanging when child ntlm_auth crashes
+ while processing an authentication request
+
+---
+ src/auth/mech-winbind.c | 12 +++++++++---
+ 1 file changed, 9 insertions(+), 3 deletions(-)
+
+diff --git a/src/auth/mech-winbind.c b/src/auth/mech-winbind.c
+index 4a65696..c12fb5e 100644
+--- src/auth/mech-winbind.c
++++ src/auth/mech-winbind.c
+@@ -187,12 +187,18 @@ do_auth_continue(struct auth_request *auth_request,
+ 	request->continued = FALSE;
+ 
+ 	while ((answer = i_stream_read_next_line(in_pipe)) == NULL) {
+-		if (in_pipe->stream_errno != 0)
++		if (in_pipe->stream_errno != 0 || in_pipe->eof)
+ 			break;
+ 	}
+ 	if (answer == NULL) {
+-		auth_request_log_error(auth_request, AUTH_SUBSYS_MECH,
+-				       "read(in_pipe) failed: %m");
++		if (in_pipe->stream_errno != 0) {
++			auth_request_log_error(auth_request, AUTH_SUBSYS_MECH,
++					       "read(in_pipe) failed: %m");
++		} else {
++			auth_request_log_error(auth_request, AUTH_SUBSYS_MECH,
++					       "read(in_pipe) failed: "
++					       "unexpected end of file");
++		}
+ 		return HR_RESTART;
+ 	}
+ 
diff --git a/mail/dovecot2/files/patch-src_plugins_fts_fts-parser-tika.c b/mail/dovecot2/files/patch-src_plugins_fts_fts-parser-tika.c
deleted file mode 100644
index 8a7404e..0000000
--- a/mail/dovecot2/files/patch-src_plugins_fts_fts-parser-tika.c
+++ /dev/null
@@ -1,30 +0,0 @@
-From 3751b61dfbc6c141731a740d982fc59918db2482 Mon Sep 17 00:00:00 2001
-From: =?UTF-8?q?Martti=20Rannanj=C3=A4rvi?= <martti.rannanjarvi@dovecot.fi>
-Date: Wed, 1 Mar 2017 12:29:17 +0200
-Subject: [PATCH] fts: Don't add NULL content disposition or type to HTTP
- header
-
----
- src/plugins/fts/fts-parser-tika.c | 9 ++++++---
- 1 file changed, 6 insertions(+), 3 deletions(-)
-
-diff --git a/src/plugins/fts/fts-parser-tika.c b/src/plugins/fts/fts-parser-tika.c
-index 8457f28..385f1d0 100644
---- src/plugins/fts/fts-parser-tika.c
-+++ src/plugins/fts/fts-parser-tika.c
-@@ -158,9 +158,12 @@ fts_parser_tika_try_init(struct mail_user *user, const char *content_type,
- 			fts_tika_parser_response, parser);
- 	http_client_request_set_port(http_req, http_url->port);
- 	http_client_request_set_ssl(http_req, http_url->have_ssl);
--	http_client_request_add_header(http_req, "Content-Type", content_type);
--	http_client_request_add_header(http_req, "Content-Disposition",
--				       content_disposition);
-+	if (content_type != NULL)
-+		http_client_request_add_header(http_req, "Content-Type",
-+					       content_type);
-+	if (content_disposition != NULL)
-+		http_client_request_add_header(http_req, "Content-Disposition",
-+					       content_disposition);
- 	http_client_request_add_header(http_req, "Accept", "text/plain");
- 
- 	parser->http_req = http_req;
diff --git a/mail/dovecot2/files/patch-trash_plugin b/mail/dovecot2/files/patch-trash_plugin
deleted file mode 100644
index 065d3d0..0000000
--- a/mail/dovecot2/files/patch-trash_plugin
+++ /dev/null
@@ -1,48 +0,0 @@
-From 326fb016a23480e4ff8dcc03dc80e76812859bd6 Mon Sep 17 00:00:00 2001
-From: Aki Tuomi <aki.tuomi@dovecot.fi>
-Date: Wed, 1 Mar 2017 09:15:29 +0200
-Subject: [PATCH] trash: Read settings after namespaces are loaded
-
-Fixes Panic: file mail-namespace.c: line 709 (mail_namespace_find): assertion failed: (ns != NULL)
----
- src/plugins/trash/trash-plugin.c | 22 ++++++++++++++++------
- 1 file changed, 16 insertions(+), 6 deletions(-)
-
-diff --git a/src/plugins/trash/trash-plugin.c b/src/plugins/trash/trash-plugin.c
-index 7129d3d..2e12578 100644
---- src/plugins/trash/trash-plugin.c
-+++ src/plugins/trash/trash-plugin.c
-@@ -348,17 +348,27 @@ trash_mail_user_created(struct mail_user *user)
- 	} else {
- 		tuser = p_new(user->pool, struct trash_user, 1);
- 		MODULE_CONTEXT_SET(user, trash_user_module, tuser);
-+	}
-+}
- 
--		if (read_configuration(user, env) == 0) {
--			trash_next_quota_test_alloc =
--				quser->quota->set->test_alloc;
--			quser->quota->set->test_alloc = trash_quota_test_alloc;
--		}
-+static void
-+trash_mail_namespaces_created(struct mail_namespace *namespaces)
-+{
-+	struct mail_user *user = namespaces->user;
-+	struct trash_user *tuser = TRASH_USER_CONTEXT(user);
-+	struct quota_user *quser = QUOTA_USER_CONTEXT(user);
-+	const char *env = mail_user_plugin_getenv(user, "trash");
-+
-+	if (tuser != NULL && read_configuration(user, env) == 0) {
-+		trash_next_quota_test_alloc =
-+			quser->quota->set->test_alloc;
-+		quser->quota->set->test_alloc = trash_quota_test_alloc;
- 	}
- }
- 
- static struct mail_storage_hooks trash_mail_storage_hooks = {
--	.mail_user_created = trash_mail_user_created
-+	.mail_user_created = trash_mail_user_created,
-+	.mail_namespaces_created = trash_mail_namespaces_created,
- };
- 
- void trash_plugin_init(struct module *module)
diff --git a/mail/dovecot2/files/pkg-message.in b/mail/dovecot2/files/pkg-message.in
index ea7220e..1c57ad1 100644
--- a/mail/dovecot2/files/pkg-message.in
+++ b/mail/dovecot2/files/pkg-message.in
@@ -15,6 +15,14 @@
  	dovecot_enable="YES"
 
 
+---------------------------------------------------------------------
+
+To avoid a risk of mailbox corruption, do not enable the
+security.bsd.see_other_uids or .see_other_guids sysctls if Dovecot
+is storing mail for multiple concurrent users (PR 218392).
+
+---------------------------------------------------------------------
+
  If you want to be able to search within attachments using the
  decode2text plugin, you'll need to install textproc/catdoc, and
  one of graphics/xpdf or graphics/poppler-utils.
diff --git a/mail/dovecot2/pkg-plist b/mail/dovecot2/pkg-plist
index 1d146e3..d2d8a41 100644
--- a/mail/dovecot2/pkg-plist
+++ b/mail/dovecot2/pkg-plist
@@ -179,6 +179,7 @@ include/dovecot/hex-dec.h
 include/dovecot/hmac-cram-md5.h
 include/dovecot/hmac.h
 include/dovecot/home-expand.h
+include/dovecot/hook-build.h
 include/dovecot/hostpid.h
 include/dovecot/http-auth.h
 include/dovecot/http-client-private.h
@@ -567,9 +568,12 @@ include/dovecot/userdb-vpopmail.h
 include/dovecot/userdb.h
 include/dovecot/utc-mktime.h
 include/dovecot/utc-offset.h
+include/dovecot/var-expand-private.h
 include/dovecot/var-expand.h
 include/dovecot/wildcard-match.h
 include/dovecot/write-full.h
+lib/dovecot/auth/lib20_auth_var_expand_crypt.a
+lib/dovecot/auth/lib20_auth_var_expand_crypt.so
 lib/dovecot/auth/libauthdb_imap.a
 lib/dovecot/auth/libauthdb_imap.so
 lib/dovecot/doveadm/lib10_doveadm_acl_plugin.a
@@ -627,6 +631,8 @@ lib/dovecot/lib20_quota_clone_plugin.a
 lib/dovecot/lib20_quota_clone_plugin.so
 lib/dovecot/lib20_replication_plugin.a
 lib/dovecot/lib20_replication_plugin.so
+lib/dovecot/lib20_var_expand_crypt.a
+lib/dovecot/lib20_var_expand_crypt.so
 lib/dovecot/lib20_virtual_plugin.a
 lib/dovecot/lib20_virtual_plugin.so
 lib/dovecot/lib20_zlib_plugin.a