with local imap: sent mail (Gcc) sometimes not saved

[Christoph Groth]

Hi, I’m using Gnus (from Emacs 24.4.1) with a local dovecot and
mbsync for synchronizing to remote IMAP servers [1]. This works
really rather well. There is, however, one problem: I set the
gcc-self Gnus parameter to save all outgoing mail. Usually this
works flawlessly, but from time to time I cannot find some message
that I know that was sent out successfully (I can see it in my
msmtp log, and I also already got replies to such messages). Do
you have any suggestions how to debug this? I noticed that
dovecot, when used directly through the /usr/lib/dovecot/imap
executable, does not seem to log anything. I tried to add the
command line parameter

-o log_path=/home/myuser/.dovecot.log

but nothing gets logged there.

For now, I've added "2>>$HOME/.dovecot.err" to the end of
nnimap-shell-program. Nothing has appeared there so far, so I'm not too
confident to be able to catch the problem in this way. Also, I'm not
even sure whether it's a dovecot problem at all. It could be also due
to Gnus itself.

It's disquieting to randomly and quietly loose mail. I'd love to fix this.

Christoph


[1] http://roland.entierement.nu/blog/2010/09/08/gnus-dovecot-offlineimap-search-a-howto.html

[Christoph Groth]

Hi,

I've been able to narrow down my problem quite a bit: Messages do
not get stored whenever the subject contains non-ASCII characters,
e.g. the UTF opening quote character (“) or accented characters.

In *Messages* the following line appears:
Couldn't store article in group nnimap+cwg:2014: ((BAD Error in IMAP command APPEND: Invalid arguments.))

The email, however, is sent correctly. It’s only the storing that
fails.

Any ideas on how to debug this further?

Christoph

[Lars Ingebrigtsen]

Post by Christoph Groth
I've been able to narrow down my problem quite a bit: Messages do
not get stored whenever the subject contains non-ASCII characters,
e.g. the UTF opening quote character (“) or accented characters.
Couldn't store article in group nnimap+cwg:2014: ((BAD Error in IMAP
command APPEND: Invalid arguments.))
The email, however, is sent correctly. It’s only the storing that
fails.
Any ideas on how to debug this further?

Try setting `nnimap-record-commands' and look in the *imap log* buffer
after one of these errors. That will tell us what commands nnimap is
sending to the server.

[Christoph Groth]

Thank you, Lars, for your interest. Now that I live in France
this particular issue can be quite annoying...

Post by Lars Ingebrigtsen

Post by Christoph Groth
I've been able to narrow down my problem quite a bit: Messages do
not get stored whenever the subject contains non-ASCII
characters,
e.g. the UTF opening quote character (“) or accented
characters.
Couldn't store article in group nnimap+cwg:2014: ((BAD Error in IMAP
command APPEND: Invalid arguments.))
The email, however, is sent correctly. It’s only the storing that
fails.
Any ideas on how to debug this further?

Try setting `nnimap-record-commands' and look in the *imap log* buffer
after one of these errors. That will tell us what commands
nnimap is
sending to the server.

I tried, this is what gets added to *imap log* when a message
fails to be stored:

22:59:56 [cwg] 597 SELECT "2015"
22:59:57 [cwg] 598 APPEND "2015" {1372}

At the same time the following appears in *Messages*:

Couldn't store article in group nnimap+cwg:2015: ((BAD Error in
IMAP command APPEND: Invalid arguments.))

On the other hand, when a message is stored without problems, the
following
gets added to *imap log*:

22:56:14 [cwg] 496 SELECT "2015"
22:56:14 [cwg] 497 APPEND "2015" {1381}
22:56:14 [cwg] 498 SELECT "2015"
22:56:14 [cwg] 499 UID FETCH 1:* FLAGS
22:56:14 [cwg] 500 SELECT "2015"
22:56:14 [cwg] 501 UID STORE 177 +FLAGS.SILENT (\Seen)

Not sure whether this is of much help. Please let me know if you
have
other ideas.

Another observation:
I noticed the problem only ever with subjects that contain
non-ASCII characters.
It can be reproduced with certain subjects (e.g. "C’est déjà
tard"). On the
other hand, some subjects (e.g. "déjà vu") do not seem to trigger
the error.

Christoph

[Lars Ingebrigtsen]

Post by Christoph Groth
22:59:56 [cwg] 597 SELECT "2015"
22:59:57 [cwg] 598 APPEND "2015" {1372}
Couldn't store article in group nnimap+cwg:2015: ((BAD Error in IMAP
command APPEND: Invalid arguments.))

So it really is reacting to the contents of the message itself (which
follows the APPEND command, but isn't logged here).

Post by Christoph Groth
I noticed the problem only ever with subjects that contain non-ASCII
characters.
It can be reproduced with certain subjects (e.g. "C’est déjà tard").
On the
other hand, some subjects (e.g. "déjà vu") do not seem to trigger the
error.

That's even weirder.

Does this only happen with the Subject header? What if the From header
or the body contains these characters?

[Alan Schmitt]

Post by Lars Ingebrigtsen

Post by Christoph Groth
I noticed the problem only ever with subjects that contain non-ASCII
characters.
It can be reproduced with certain subjects (e.g. "C’est déjà tard").
On the
other hand, some subjects (e.g. "déjà vu") do not seem to trigger the
error.

That's even weirder.
Does this only happen with the Subject header? What if the From header
or the body contains these characters?

The apostrophe ’ may not be latin1, whereas the other characters are.

Alan

[Lars Ingebrigtsen]

The apostrophe ’ may not be latin1, whereas the other characters are.

Aha. So the server is rejecting all non-latin1 characters?

Or perhaps nnimap is encoding stuff in a wonky way. Could you use
wireshark or something to capture what's actually sent?