Skip to content

nixos/dovecot: 2.3.21.1 -> 2.4.0#388463

Open
Jappie3 wants to merge 7 commits intoNixOS:masterfrom
Jappie3:dovecot
Open

nixos/dovecot: 2.3.21.1 -> 2.4.0#388463
Jappie3 wants to merge 7 commits intoNixOS:masterfrom
Jappie3:dovecot

Conversation

@Jappie3
Copy link
Contributor

@Jappie3 Jappie3 commented Mar 9, 2025

This PR updates the Dovecot version to 2.4. Version 2.3 will be kept around for compatibility, as 2.4 has quite a few breaking changes (https://doc.dovecot.org/2.4.0/installation/upgrade/2.3-to-2.4.html). Updates for all Dovecot plugins present in nixpkgs are also included.

Besides that, the NixOS module for Dovecot is also updated to support the changes to the config format.

Resolves #388262

see #410912 for a followup regarding an RFC42-style settings option

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@Jappie3
Copy link
Contributor Author

Jappie3 commented Mar 9, 2025

just missed @2xsaiko 's comment on the issue:

This would probably a good place to add versioned packages, i.e. dovecot_2_3 and dovecot_2_4, at least for this upgrade. These changes are non-trivial and very likely people will need to change their config for it. Least we could do is not silently break it but have them explicitly change the version.

sounds like a good idea, I'm not an expert at this but I think we can lock the module changes behind system.stateVersion?

@Jappie3
Copy link
Contributor Author

Jappie3 commented Mar 9, 2025

I would also like to explore whether or not we can use settings for all dovecot config instead of some options combined with extraConfig - i.e. NixOS/rfcs#42.

@github-actions github-actions bot added 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. labels Mar 9, 2025
@dblsaiko
Copy link
Contributor

dblsaiko commented Mar 9, 2025

Ah, nice, I just started doing the same thing. Didn't get very far yet though -- just the by-name change below.

sounds like a good idea, I'm not an expert at this but I think we can lock the module changes behind system.stateVersion?

I would just set dovecot to continue being 2.3 for now, or set the module to use 2.3, and have it print a warning when used so people manually update to 2.4. Then in the next release cycle, 2.3 can be removed and hard error.

I'm not sure if this is exactly how this is usually done, but take a look at how it was done for Kanidm for example. #357734

Please also move the package to by-name while you're at it (see dblsaiko@push-mputwrznzqyv).

I would also like to explore whether or not we can use settings for all dovecot config instead of some options combined with extraConfig - i.e. NixOS/rfcs#42.

#286184 is a start for that.

@Jappie3
Copy link
Contributor Author

Jappie3 commented Mar 9, 2025

by-name change... thanks for reminding me!

I would just set dovecot to continue being 2.3 for now, or set the module to use 2.3, and have it print a warning when used so people manually update to 2.4. Then in the next release cycle, 2.3 can be removed and hard error.

we should probably introduce a services.dovecot.package then... set it to 2.3 for 25.05 & change it to throw when 25.11 rolls around nvm that - 2.3 is still supported & I can't immediately find an EOL date... so ig we can just leave it be as an 'older version' for the time being and deprecate it when upstream does

I'm not sure if this is exactly how this is usually done, but take a look at how it was done for Kanidm for example. #357734

ah thanks! The first thing that came to mind for my was the postgres module, but same idea!

https://github.com/NixOS/nixpkgs/blob/20755fa05115c84be00b04690630cb38f0a203ad/nixos/modules/services/databases/postgresql.nix#L529C1-L561C64

#286184 is a start for that.

completely missed this one... I'll do some experimenting to see what's possible while keeping the module backwards-compatible for 2.3!

@dblsaiko
Copy link
Contributor

dblsaiko commented Mar 9, 2025

we should probably introduce a services.dovecot.package then

Right, that would be part of that.

@Jappie3
Copy link
Contributor Author

Jappie3 commented Mar 11, 2025

pulled in your commit & added 2.4 alongside 2.3... not 100% sold on the commit message but good enough for now

both packages build on my machine, haven't checked any functionality (notably the two things mentioned in the original PR description abt the openssl patch and that one hunk)

@ofborg ofborg bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Mar 11, 2025
@dblsaiko
Copy link
Contributor

We'll also have to deal with the plugin packages because they are dovecot version dependent. Here's a proof of concept for pigeonhole: dblsaiko@push-sntmrkppzolr

And a (as of right now very non working) change adding the package option back dblsaiko@push-truxwztylsmm

@Jappie3
Copy link
Contributor Author

Jappie3 commented Mar 12, 2025

uhhh is it preferable to have no broken commits? i.e. stuff that wouldn't eval?
cuz to do so I have to add all the plugin packages together with dovecot_2_4 in a single commit

@ofborg ofborg bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Mar 12, 2025
@Jappie3
Copy link
Contributor Author

Jappie3 commented Mar 12, 2025

fixed both fts-xapian & exporter
flatcurve is apparently included in dovecot now, going by the original commit message in which it was added to nixpkgs (20a503b) and upstream docs (https://doc.dovecot.org/main/core/plugins/fts_flatcurve.html)

dovecot_pigeonhole_2_4 has a build failure i'm too tired to troubleshoot now, dovecot_pigeonhole_2_3 works fine

probably gonna have to rebase all of this later, just wanted to get the code on here in case you or anyone else has a clue wrt that build failure

@Jappie3 Jappie3 force-pushed the dovecot branch 2 times, most recently from f6622ba to 53b2306 Compare March 12, 2025 22:29
@github-actions github-actions bot added 10.rebuild-darwin: 11-100 This PR causes between 11 and 100 packages to rebuild on Darwin. 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. and removed 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. labels Mar 12, 2025
@dblsaiko
Copy link
Contributor

dblsaiko commented Mar 12, 2025

uhhh is it preferable to have no broken commits? i.e. stuff that wouldn't eval? cuz to do so I have to add all the plugin packages together with dovecot_2_4 in a single commit

I'd keep them separate, ordered after the commit for dovecot. (also it would eval, just not build, right?)

dovecot_pigeonhole_2_4 has a build failure i'm too tired to troubleshoot now, dovecot_pigeonhole_2_3 works fine

Huh, does it? Both of them built for me. I'll have to compare your branch to mine.

@Jappie3
Copy link
Contributor Author

Jappie3 commented Mar 13, 2025

I'd keep them separate, ordered after the commit for dovecot. (also it would eval, just not build, right?)

Sounds good, and yeah I think you're right - it would still eval.

Huh, does it? Both of them built for me. I'll have to compare your branch to mine.

Had a typo in all-packages.nix which pointed dovecot_2_4 to 2_3.nix, woops 😅
I'll fix that later today

@Jappie3
Copy link
Contributor Author

Jappie3 commented Mar 13, 2025

dovecot_fts_xapian gave me a build failure on 1.9... upstream released 1.9.1 to fix this, but that tried to install itself to /nix/store for some reason...

the error in question
Making install in src
make[1]: Entering directory '/build/source/src'
make[2]: Entering directory '/build/source/src'
make[2]: Nothing to be done for 'install-exec-am'.
 /nix/store/vz67zaqyfln2jcqwar682kdlrxfx9vq1-coreutils-9.6/bin/mkdir -p '/nix/store/9gp6nrcynrps8bkyp9ql298h560vqcr5-dovecot-2.4.0/lib/dovecot/modules'
 /nix/store/ki4if6b0w5bqv8dc5lrjp8xm7wjy9dlf-bash-5.2p37/bin/bash ../libtool   --mode=install /nix/store/vz67zaqyfln2jcqwar682kdlrxfx9vq1-coreutils-9.6/bin/install -c   lib21_fts_xapian_plugin.la '/nix/st>
libtool: install: /nix/store/vz67zaqyfln2jcqwar682kdlrxfx9vq1-coreutils-9.6/bin/install -c .libs/lib21_fts_xapian_plugin.so /nix/store/9gp6nrcynrps8bkyp9ql298h560vqcr5-dovecot-2.4.0/lib/dovecot/modules/li>
install: cannot create regular file '/nix/store/9gp6nrcynrps8bkyp9ql298h560vqcr5-dovecot-2.4.0/lib/dovecot/modules/lib21_fts_xapian_plugin.so': Permission denied
make[2]: *** [Makefile:520: install-dovecot_moduleLTLIBRARIES] Error 1
make[2]: Leaving directory '/build/source/src'
make[1]: *** [Makefile:755: install-am] Error 2
make[1]: Leaving directory '/build/source/src'
make: *** [Makefile:504: install-recursive] Error 1

I just committed a horrible hack to fix that problem by modifying the Makefile with sed before installing, see 9d4bdfb

The problem is that around line 527 in src/Makefile it tries to do the following:

echo " $(MKDIR_P) '$(DESTDIR)$(dovecot_moduledir)'"; \
$(MKDIR_P) "$(DESTDIR)$(dovecot_moduledir)" || exit 1; \
echo " $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=install $(INSTALL) $(INSTALL_STRIP_FLAG) $$list2 '$(DESTDIR)$(dovecot_moduledir)'"; \
$(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=install $(INSTALL) $(INSTALL_STRIP_FLAG) $$list2 "$(DESTDIR)$(dovecot_moduledir)"; \

with dovecot_moduledir pointing to /nix/store/[...]-dovecot/lib/dovecot/modules/lib21_fts_xapian_plugin.so instead of $out/lib/dovecot/modules (where $out is /nix/store/[...]-dovecot-ftl-xapian-1.9.1).

Following the trail upwards, it's clear that dovecot_moduledir comes from --with-moduledir: reading this back, I'm not sure if dovecot_moduledir comes from moduledir...

src/Makefile.in
417:dovecot_moduledir = @dovecot_moduledir@


$> rg "with-moduledir" configure -A8 -B2
20865:# Check whether --with-moduledir was given.
20866-if test ${with_moduledir+y}
20867-then :
20868-  withval=$with_moduledir; moduledir="$withval"
20869-else case e in #(
20870-  e) moduledir="\$(libdir)/dovecot"
20871-	 ;;
20872-esac
20873-fi


$> rg "with-moduledir" m4/dovecot.m4 -B2 -A6
250-AC_DEFUN([DC_DOVECOT_MODULEDIR],[
251-	AC_ARG_WITH(moduledir,
252:	[  --with-moduledir=DIR    Base directory for dynamically loadable modules],
253-		[moduledir="$withval"],
254-		[moduledir="\$(libdir)/dovecot"]
255-	)
256-	AC_SUBST(moduledir)

which we (nixpkgs) correctly set to $out, not ${dovecot}, using configureFlags:

  configureFlags = [
    "--with-dovecot=${dovecot}/lib/dovecot"
    "--with-moduledir=${placeholder "out"}/lib/dovecot/modules"
  ];

so my guess is that upstream somehow borked this flag and it gets mixed up somewhere with the ${dovecot} path from /nix/store? If that's the case, then we should probably take this upstream to get this resolved.

I want to stress that I am *not* a Make, autotools or m4 expert, this is just my two cents

pinging @Julm and @symphorien as maintainers of this package, maybe you guys have a better understanding of what's going on

@dblsaiko
Copy link
Contributor

Had a typo in all-packages.nix which pointed dovecot_2_4 to 2_3.nix, woops 😅

Oh, oops. I saw that, didn't say anything because I didn't want to be annoying reviewing every commit you make 2 seconds after you push them if there's something wrong with one, then forgot about it when I replied.

(speaking of, please squash that format commit into the parent, I just forgot to format it. my pretty alejandra-style formatting vs evil nixfmt 😔)

@Jappie3
Copy link
Contributor Author

Jappie3 commented Mar 13, 2025

doing a diff on src/Makefile of 1.8.4 and 1.9.1, it's clear that settingsdir was changed to dovecot_moduledir (moduledir with my hacky workaround)... I'll take this upstream as I don't think this is currently implemented as intended

	@$(NORMAL_INSTALL)																	@$(NORMAL_INSTALL)
	@list='$(settings_LTLIBRARIES)'; test -n "$(settingsdir)" || list=; \									     |		@list='$(module_LTLIBRARIES)'; test -n "$(moduledir)" || list=; \
	list2=; for p in $$list; do \																list2=; for p in $$list; do \
	  if test -f $$p; then \																  if test -f $$p; then \
	    list2="$$list2 $$p"; \																    list2="$$list2 $$p"; \
	  else :; fi; \																		  else :; fi; \
	done; \																			done; \
	test -z "$$list2" || { \																test -z "$$list2" || { \
	  echo " $(MKDIR_P) '$(DESTDIR)$(settingsdir)'"; \											     |		  echo " $(MKDIR_P) '$(DESTDIR)$(moduledir)'"; \
	  $(MKDIR_P) "$(DESTDIR)$(settingsdir)" || exit 1; \											     |		  $(MKDIR_P) "$(DESTDIR)$(moduledir)" || exit 1; \
	  echo " $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=install $(INSTALL) $(INSTALL_STRIP_FLAG) $$list2 '$(DESTDIR)$(settingsdir)'";  |		  echo " $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=install $(INSTALL) $(INSTALL_STRIP_FLAG) $$list2 '$(DESTDIR)$(moduledir)'"; \
	  $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=install $(INSTALL) $(INSTALL_STRIP_FLAG) $$list2 "$(DESTDIR)$(settingsdir)"; \	     |		  $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=install $(INSTALL) $(INSTALL_STRIP_FLAG) $$list2 "$(DESTDIR)$(moduledir)"; \
	}																			}

grosjo/fts-xapian#191

@grosjo
Copy link

grosjo commented Mar 15, 2025

Hi @Jappie3 ,
Can you test with latest git ( grosjo/fts-xapian#191 )

Jappie3 added a commit to Jappie3/nixpkgs that referenced this pull request Nov 21, 2025
This patch fixes a permission issue that occurs when saving compiled
sieve scripts sourced from the nix store. Instead of reusing the
read-only permission bits from the nix store, it explicitly uses `0700`
for the directory in which compiled sieve scripts should be saved.

Additional context:
- NixOS#388463 (comment)
- dovecot/pigeonhole#15
Jappie3 added a commit to Jappie3/nixpkgs that referenced this pull request Nov 21, 2025
This patch fixes a permission issue that occurs when saving compiled
sieve scripts sourced from the nix store. Instead of reusing the
read-only permission bits from the nix store, it explicitly uses `0700`
for the directory in which compiled sieve scripts should be saved.

Additional context:
- NixOS#388463 (comment)
- dovecot/pigeonhole#15
@nixpkgs-ci nixpkgs-ci bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Nov 29, 2025
@kfiz
Copy link
Contributor

kfiz commented Dec 5, 2025

@Jappie3 Thanks for this. I'm using this successfully on macOS after fixing a libiconv related issue with the help of @reckenrode by setting NIX_LDFLAGS to "-liconv". Can you rebase to master? Is this ready to merge?

@Jappie3
Copy link
Contributor Author

Jappie3 commented Dec 6, 2025

thanks for testing! You mean NIX_LDFLAGS on the dovecot package I'm assuming?

and no, currently not ready to merge, need to do the above & fix the release notes merge conflict (26.05 instead of 25.11, the latter of which released) (but besides that I think this is ready, been running Dovecot 2.4 for over half a year)

will fix this when I find time, hopefully in one of the next days :)

if you're feeling adventurous, you can also try #410912 :p

@kfiz
Copy link
Contributor

kfiz commented Dec 6, 2025

thanks for testing! You mean NIX_LDFLAGS on the dovecot package I'm assuming?

Yes.

and no, currently not ready to merge, need to do the above & fix the release notes merge conflict (26.05 instead of 25.11, the latter of which released) (but besides that I think this is ready, been running Dovecot 2.4 for over half a year)

will fix this when I find time, hopefully in one of the next days :)

Nice.

if you're feeling adventurous, you can also try #410912 :p

Since I'm using dovecot on macOS, I don't think I qualify as a tester for this. ;)

Jappie3 added a commit to Jappie3/nixpkgs that referenced this pull request Dec 20, 2025
This patch fixes a permission issue that occurs when saving compiled
sieve scripts sourced from the nix store. Instead of reusing the
read-only permission bits from the nix store, it explicitly uses `0700`
for the directory in which compiled sieve scripts should be saved.

Additional context:
- NixOS#388463 (comment)
- dovecot/pigeonhole#15
Jappie3 and others added 6 commits December 20, 2025 15:54
This patch fixes a permission issue that occurs when saving compiled
sieve scripts sourced from the nix store. Instead of reusing the
read-only permission bits from the nix store, it explicitly uses `0700`
for the directory in which compiled sieve scripts should be saved.

Additional context:
- NixOS#388463 (comment)
- dovecot/pigeonhole#15
@nixpkgs-ci nixpkgs-ci bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Dec 20, 2025
kfiz pushed a commit to kfiz/nixpkgs that referenced this pull request Jan 1, 2026
This patch fixes a permission issue that occurs when saving compiled
sieve scripts sourced from the nix store. Instead of reusing the
read-only permission bits from the nix store, it explicitly uses `0700`
for the directory in which compiled sieve scripts should be saved.

Additional context:
- NixOS#388463 (comment)
- dovecot/pigeonhole#15
kfiz pushed a commit to kfiz/nixpkgs that referenced this pull request Jan 4, 2026
This patch fixes a permission issue that occurs when saving compiled
sieve scripts sourced from the nix store. Instead of reusing the
read-only permission bits from the nix store, it explicitly uses `0700`
for the directory in which compiled sieve scripts should be saved.

Additional context:
- NixOS#388463 (comment)
- dovecot/pigeonhole#15
@nixpkgs-ci nixpkgs-ci bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Jan 12, 2026
@kfiz
Copy link
Contributor

kfiz commented Jan 25, 2026

@Jappie3 What do you think? Are we good to go?

@nixpkgs-ci nixpkgs-ci bot added the 8.has: package (update) This PR updates a package to a newer version label Jan 26, 2026
Copy link
Member

@Prince213 Prince213 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please rebase against master.

dovecotConf =
if versionAtLeast cfg.package.version "2.4" then
concatStrings [

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change

warnings = warnAboutExtraConfigCollisions;
warnings =
warnAboutExtraConfigCollisions
++ optional (versionOlder cfg.package.version "2.4") ''
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can add a option to suppress this? Is this necessary in the first place if we already mention 2.4 in release notes?


<!-- To avoid merge conflicts, consider adding your item at an arbitrary place in the list instead. -->

- `services.dovecot` now supports the Dovecot 2.4 configuration format while maintaining compatibility with Dovecot 2.3.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are Nixpkgs users who use Dovecot on darwin, how about we also add an entry to Nixpkgs release notes?

dovecotPkg = cfg.package;
pigeonHolePkg =
if lib.versionAtLeast cfg.package.version "2.4" then
pkgs.dovecot_pigeonhole_2_4
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's not use the explicitly versioned one.

Suggested change
pkgs.dovecot_pigeonhole_2_4
pkgs.dovecot_pigeonhole

mailDriver = mkOption {
type = types.str;
default = "maildir";
example = "maildir";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Default value is actually an example so no need to repeat it.

Suggested change
example = "maildir";

clucene_core_2,
icu75,
libexttextcat,
openldap,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There used to be a withLDAP option, why is it removed?

Comment on lines +159 to +164
"--with-ldap"
"--with-ssl=openssl"
"--with-zlib"
"--with-bzlib"
"--with-lz4"
"--with-ldap"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why two instances of --with-ldap here?

maintainers = with lib.maintainers; [
fpletz
];
teams = [ lib.teams.helsinki-systems ];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This team no longer exists as of #478819.

];
mainProgram = "dovecot";
maintainers = with lib.maintainers; [
fpletz
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding yourself.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this separate package? If not we can put these lines in all-packages.nix, or override dovecot_exporter when needed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For problems with this callPackages pattern, please see previous discussions at #449760 (comment).

@Prince213
Copy link
Member

Working on a "port" at https://github.com/Prince213/nixpkgs/tree/push-ulukpzwkptqu in case rebasing is complicated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

2.status: merge conflict This PR has merge conflicts with the target branch 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: changelog This PR adds or changes release notes 8.has: documentation This PR adds or changes documentation 8.has: module (update) This PR changes an existing module in `nixos/` 8.has: package (update) This PR updates a package to a newer version 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update Request: dovecot 2.3.21.1 → 2.4.0