Performance problem of imap move

[Mickaël Rémond]

Hello,

I am currently trying to trouble shoot performance problem of the nnimap
move methods. It takes more than 30 seconds here to move a mail (I am
using Gmail) on imap on the same server.

imap server is a secondary server.

Here is the result of elp instrument:
gnus-summary-move-article 2 72.684064 36.342032
gnus-activate-group 2 62.887769 31.4438845
gnus-group-get-new-news-this-group 1 62.887473 62.887473
gnus-group-completing-read 2 8.951666 4.475833
gnus-completing-read 2 8.9514789999 4.4757394999
gnus-emacs-completing-read 2 8.951453 4.4757265
gnus-read-move-group-name 2 4.755247 2.3776235
gnus-y-or-n-p 1 4.516911 4.516911
gnus-request-set-mark 2 3.040978 1.520489
gnus-request-move-article 1 1.9702929999 1.9702929999

Any idea what I should do to pinpoint the issue ?

Thanks !

[Mickaël Rémond]

Hello,

Post by Mickaël Rémond
Hello,
I am currently trying to troubleshoot performance problems of the
nnimap move methods. It takes more than 30 seconds here to move a mail
(I am using Gmail) on imap on the same server.

I did more investigations enabling IMAP command logs.
I think the issue is that the last command done during a move is a fetch
of all FLAGS:

474 UID FETCH 1:* FLAGS

I am not sure why this command is performed, but that mailbox has 250k
emails in it, so that may explain why the move is so slow.

Is it risky to remove that FLAGS download ?

[Lars Ingebrigtsen]

Post by Mickaël Rémond
I did more investigations enabling IMAP command logs.
I think the issue is that the last command done during a move is a fetch
474 UID FETCH 1:* FLAGS

Geez. That's the result of a not very thought through calling
sequence. It's calling `gnus-group-get-new-news-this-group' to update
the line of the group we've moved the message to, and that function is
also used for `M-g' to make nnimap completely rescan the group.

There doesn't seem to be any way in nnimap to get the new article count
for a single group without doing a total rescan of the group, and that
is, as you've found out, really slow.

This will have to be refactored a bit, and a new nnimap function will
have to be written, I think...

[Lars Ingebrigtsen]

Post by Lars Ingebrigtsen
This will have to be refactored a bit, and a new nnimap function will
have to be written, I think...

I've now done this, so moving IMAP messages to large groups should be
faster again.

[Tassilo Horn]

Post by Lars Ingebrigtsen

Post by Lars Ingebrigtsen
This will have to be refactored a bit, and a new nnimap function will
have to be written, I think...

I've now done this, so moving IMAP messages to large groups should be
faster again.

I don't know if that's a side-effect of this change but with today's
Gnus version I sometimes get group lines like

38289: nnimap+Server:some.group

where actually there are no unread articles in the group. `M-g' on the
group or another `g' fixes the unread count again.

I also don't have a reproducible recipe but it seems to happen
sporadically with random nnimap groups. Not sure what I did before.
Possibly, it happens when some.group is either my sent-mail or SPAM
group, and previously I've sent a new mail or exited some other group
where I've marked some messages as SPAM which are moved to my SPAM group
as a result...

Bye,
Tassilo

[Steinar Bang]

Post by Tassilo Horn
where actually there are no unread articles in the group. `M-g' on the
group or another `g' fixes the unread count again.

M-g does "UID FETCH 1:* FLAGS", which I guess is what Lars have
removed...? So maybe he removed one too many...?

[Lars Ingebrigtsen]

Post by Tassilo Horn
I don't know if that's a side-effect of this change but with today's
Gnus version I sometimes get group lines like
38289: nnimap+Server:some.group

This is when just `g'-ing in the group buffer? The changes I made
yesterday shouldn't affect that at all.

Post by Tassilo Horn
I also don't have a reproducible recipe but it seems to happen
sporadically with random nnimap groups. Not sure what I did before.
Possibly, it happens when some.group is either my sent-mail or SPAM
group, and previously I've sent a new mail or exited some other group
where I've marked some messages as SPAM which are moved to my SPAM group
as a result...

If you could find a pattern where this happens, that would be nice. The
changes introduced yesterday does change what happens when you copy
articles to an nnimap group, either by Gcc-ing articles to it or
whatnot, so that does sound like a likely scenario...

[Tassilo Horn]

Post by Lars Ingebrigtsen

Post by Tassilo Horn
I don't know if that's a side-effect of this change but with today's
Gnus version I sometimes get group lines like
38289: nnimap+Server:some.group

This is when just `g'-ing in the group buffer? The changes I made
yesterday shouldn't affect that at all.

Post by Tassilo Horn
I also don't have a reproducible recipe but it seems to happen
sporadically with random nnimap groups. Not sure what I did before.
Possibly, it happens when some.group is either my sent-mail or SPAM
group, and previously I've sent a new mail or exited some other group
where I've marked some messages as SPAM which are moved to my SPAM
group as a result...

If you could find a pattern where this happens, that would be nice.

Found one. I'm in nnimap+Server:some.group and write a reply to a mail
in there. My posting styles for that group include (gcc-self . t), so a
copy of the sent reply is added to the group. After exiting some.group,
the unread count is as shown above.

Bye,
Tassilo

[Tassilo Horn]

Post by Tassilo Horn

Post by Lars Ingebrigtsen
If you could find a pattern where this happens, that would be nice.

Found one. I'm in nnimap+Server:some.group and write a reply to a
mail in there. My posting styles for that group include (gcc-self
. t), so a copy of the sent reply is added to the group. After
exiting some.group, the unread count is as shown above.

And in another.group the gcc-self is set to that server's sent-mail
group, and when I write a message in there and exit, the sent-mail group
has the wrong unread count.

So maybe the right thing would be to do the "fast copy/move" for all
messages but the last one where you do the "full ceremony" as before?

Bye,
Tassilo

[Lars Ingebrigtsen]

Post by Tassilo Horn
Found one. I'm in nnimap+Server:some.group and write a reply to a mail
in there. My posting styles for that group include (gcc-self . t), so a
copy of the sent reply is added to the group. After exiting some.group,
the unread count is as shown above.

I've now tried this, and I'm not seeing this problem. But the article
count isn't updating, either, so something isn't working the way it
should.

I Gcc'd to the Test group, and my *imap log* buffer says:

18:26:59 [hermes.netfonds.no] 2741 SELECT "Test".
18:27:00 [hermes.netfonds.no] 2742 APPEND "Test" {422}.
18:27:00 [hermes.netfonds.no] 2743 EXAMINE "Test" (QRESYNC (1286911992 20)).

Which is the right thing, I think. What does yours say?

[Tassilo Horn]

Post by Lars Ingebrigtsen

Post by Tassilo Horn
Found one. I'm in nnimap+Server:some.group and write a reply to a
mail in there. My posting styles for that group include (gcc-self
. t), so a copy of the sent reply is added to the group. After
exiting some.group, the unread count is as shown above.

I've now tried this, and I'm not seeing this problem.

It seems that the problem doesn't always occur but it does at least most
of the time here.

Post by Lars Ingebrigtsen
But the article count isn't updating, either, so something isn't
working the way it should.
18:26:59 [hermes.netfonds.no] 2741 SELECT "Test".
18:27:00 [hermes.netfonds.no] 2742 APPEND "Test" {422}.
18:27:00 [hermes.netfonds.no] 2743 EXAMINE "Test" (QRESYNC (1286911992 20)).
Which is the right thing, I think. What does yours say?

I'm testing now with this very mail which is gcc-ed to my
"nnimap+Fastmail:INBOX.Sent Items" group and to my local
nnml+archive:sent-mails group. I'll report back in a minute.

Bye,
Tassilo

[Tassilo Horn]

Post by Tassilo Horn

Post by Lars Ingebrigtsen
But the article count isn't updating, either, so something isn't
working the way it should.
18:26:59 [hermes.netfonds.no] 2741 SELECT "Test".
18:27:00 [hermes.netfonds.no] 2742 APPEND "Test" {422}.
18:27:00 [hermes.netfonds.no] 2743 EXAMINE "Test" (QRESYNC (1286911992 20)).
Which is the right thing, I think. What does yours say?

I'm testing now with this very mail which is gcc-ed to my
"nnimap+Fastmail:INBOX.Sent Items" group and to my local
nnml+archive:sent-mails group. I'll report back in a minute.

That's what I got:

09:01:32 [mail.messagingengine.com] 1429 SELECT "INBOX.Sent Items"
09:01:32 [mail.messagingengine.com] 1430 APPEND "INBOX.Sent Items" {1953}
09:01:32 [mail.messagingengine.com] 1431 EXAMINE "INBOX.Sent Items" (QRESYNC (1103381936 206930))
09:01:32 [mail.messagingengine.com] 1432 SELECT "INBOX.Sent Items"
09:01:32 [mail.messagingengine.com] 1433 UID STORE 1:2385 +FLAGS.SILENT (\Seen)

And now after exiting the summary, my group line shows

19739: nnimap+Fastmail:INBOX.mailinglists.ding

where 2 or 3 is actually the right unread count. And this time, `g' or
`M-g' don't fix the unread count anymore. I think the only difference
between this time and previous times where `g'/`M-g' fixed the problem
is that this time I exited the group with `q' and still two or three
articles were unread. Before, I always caught up with `c' to exist the
group so the unread cound was zero.

Bye,
Tassilo

[Lars Ingebrigtsen]

Post by Tassilo Horn

Post by Lars Ingebrigtsen
18:26:59 [hermes.netfonds.no] 2741 SELECT "Test".
18:27:00 [hermes.netfonds.no] 2742 APPEND "Test" {422}.
18:27:00 [hermes.netfonds.no] 2743 EXAMINE "Test" (QRESYNC (1286911992 20)).

[...]

Post by Tassilo Horn
09:01:32 [mail.messagingengine.com] 1429 SELECT "INBOX.Sent Items"
09:01:32 [mail.messagingengine.com] 1430 APPEND "INBOX.Sent Items" {1953}
09:01:32 [mail.messagingengine.com] 1431 EXAMINE "INBOX.Sent Items" (QRESYNC (1103381936 206930))

So that's basically the same as what happens in my group...

Post by Tassilo Horn
09:01:32 [mail.messagingengine.com] 1432 SELECT "INBOX.Sent Items"
09:01:32 [mail.messagingengine.com] 1433 UID STORE 1:2385 +FLAGS.SILENT (\Seen)

But that's odd. It's marking articles 1 to 2385 as seen? I would have
expected it to mark at most one article...

Post by Tassilo Horn
And now after exiting the summary, my group line shows
19739: nnimap+Fastmail:INBOX.mailinglists.ding
where 2 or 3 is actually the right unread count. And this time, `g' or
`M-g' don't fix the unread count anymore.

That is even weirder. `M-g' is supposed to do a complete rescan, making
Gnus reflect the state of the messages on the IMAP server totally.

This is very odd.

[Eric Abrahamsen]

Post by Tassilo Horn

Post by Lars Ingebrigtsen

Post by Lars Ingebrigtsen
This will have to be refactored a bit, and a new nnimap function will
have to be written, I think...

I've now done this, so moving IMAP messages to large groups should be
faster again.

I don't know if that's a side-effect of this change but with today's
Gnus version I sometimes get group lines like
38289: nnimap+Server:some.group
where actually there are no unread articles in the group. `M-g' on the
group or another `g' fixes the unread count again.
I also don't have a reproducible recipe but it seems to happen
sporadically with random nnimap groups. Not sure what I did before.
Possibly, it happens when some.group is either my sent-mail or SPAM
group, and previously I've sent a new mail or exited some other group
where I've marked some messages as SPAM which are moved to my SPAM group
as a result...
Bye,
Tassilo

I see something like this (and have for a very long time, this isn't
new) when moving messages between nnimap groups. When I open an imap
group containing new messages, and decide that one of the
(newly-received but now marked "read") messages belongs in a different
group on the same server, I move it to that group, return to the *Group*
buffer, and see that the group I moved the message to now has an
(apparently random) unread count. Hitting "g" or actually entering the
group makes the unread count go away -- the messages aren't actually
marked as unread.

Might be unrelated, though: as I said, it's done this (fairly
reproducibly) for a long time.

[Lars Ingebrigtsen]

Post by Eric Abrahamsen
Might be unrelated, though: as I said, it's done this (fairly
reproducibly) for a long time.

Huh. Hm. Perhaps it depends on what IMAP server is being used? What
server are you seeing this on?

[Eric Abrahamsen]

Post by Lars Ingebrigtsen

Post by Eric Abrahamsen
Might be unrelated, though: as I said, it's done this (fairly
reproducibly) for a long time.

Huh. Hm. Perhaps it depends on what IMAP server is being used? What
server are you seeing this on?

A local dovecot server, synced with gmail via isync.

I've always meant to go and follow the code, but it's never been
annoying enough for me to actually do it. But if you're messing nnimap
at the moment anyway, maybe now's a good time.

[Lars Ingebrigtsen]

Post by Eric Abrahamsen
A local dovecot server, synced with gmail via isync.

Ah. For some reason, it seems that people running local IMAP servers
triggers a lot of bugs in Gnus...

Post by Eric Abrahamsen
I've always meant to go and follow the code, but it's never been
annoying enough for me to actually do it. But if you're messing nnimap
at the moment anyway, maybe now's a good time.

Please do. :-)

[Steinar Bang]

Post by Lars Ingebrigtsen
Ah. For some reason, it seems that people running local IMAP servers
triggers a lot of bugs in Gnus...

Timing-related...?

[Dave Goldberg]

It appears that the change associated with this thread has caused a problem for me at work. After doing a git pull and recompiling (as usual) I noticed some strangeness with some of the groups on our office365 set up. I use a group line format of

%S%p%P%(%G [(%y new,%T !,%I ?)/~%t]%)

A number of groups, though not all, initially showed up with all the article counts showing as a '*'. Running M-g on the group resulted in an error that I regret not recording. I then ran M-c to clear data and M-g again and the article counts showed as ridiculously high and Emacs hung long enough when trying to enter the group that Gnome offered to kill it off for lack of response. I ran git bisect and it determined that this

447c56af7cb2afac14a0f3ed23f85d907c443a5d is the first bad commit
commit 447c56af7cb2afac14a0f3ed23f85d907c443a5d
Author: Lars Ingebrigtsen <***@gnus.org>
Date: Mon Jan 26 13:50:20 2015 +1100

Make moving IMAP articles faster in large groups

* lisp/gnus-group.el (gnus-group-get-new-news-this-group): Explicitly
request rescans when being run interactively.

* lisp/gnus-int.el (gnus-request-group-scan): New backend function.

* lisp/nnimap.el (nnimap-request-scan-group): Implement in on IMAP.

* lisp/nnimap.el (nnimap-request-group): Don't rescan the group here,
because that can be very slow in large groups.

:040000 040000 3164108231527becdb471bfe74ce22bdd3bc5b4f
d424a0d925e35e71fc30863580e2ca949559505e M lisp

While many of the affected groups don't really matter much to me (they mostly represent archives) a couple of them regularly receive messages either as a result of server side splitting or article moves so I'm currently running a detached checkout on the last working commit. I am happy to take some time to try to help debug this but I need some guidance on what to do and what to look for. I've stepped through the lisp debugger in the past but I wouldn't even know where to begin in this case. In particular if I need to look at what's going on IMAP-wise I don't know how to do that.

Thanks,

[Dave Goldberg]

Post by Dave Goldberg
It appears that the change associated with this thread has caused a problem
for me at work. After doing a git pull and recompiling (as usual) I
noticed some strangeness with some of the groups on our office365 set up.
I use a group line format of
%S%p%P%(%G [(%y new,%T !,%I ?)/~%t]%)
A number of groups, though not all, initially showed up with all the article counts showing as a '*'. Running M-g on the group resulted in an error that I regret not recording. I then ran M-c to clear data and M-g again and the article counts showed as ridiculously high and Emacs hung long enough when trying to enter the group that Gnome offered to kill it off for lack of response. I ran git bisect and it determined that this
447c56af7cb2afac14a0f3ed23f85d907c443a5d is the first bad commit
commit 447c56af7cb2afac14a0f3ed23f85d907c443a5d
Date: Mon Jan 26 13:50:20 2015 +1100
Make moving IMAP articles faster in large groups
* lisp/gnus-group.el (gnus-group-get-new-news-this-group): Explicitly
request rescans when being run interactively.
* lisp/gnus-int.el (gnus-request-group-scan): New backend function.
* lisp/nnimap.el (nnimap-request-scan-group): Implement in on IMAP.
* lisp/nnimap.el (nnimap-request-group): Don't rescan the group here,
because that can be very slow in large groups.
:040000 040000 3164108231527becdb471bfe74ce22bdd3bc5b4f
d424a0d925e35e71fc30863580e2ca949559505e M lisp

[...]

The latest checkout seems to have resolved the problem.

Thanks,

[Eric Abrahamsen]

Post by Lars Ingebrigtsen

Post by Eric Abrahamsen
A local dovecot server, synced with gmail via isync.

Ah. For some reason, it seems that people running local IMAP servers
triggers a lot of bugs in Gnus...

Post by Eric Abrahamsen
I've always meant to go and follow the code, but it's never been
annoying enough for me to actually do it. But if you're messing nnimap
at the moment anyway, maybe now's a good time.

Please do. :-)

Doing this bit by bit, first of all here are the imap commands resulting
from moving article 39339 from INBOX to [Gmail].Trash, where it became
article 175. Upon leaving the INBOX group, [Gmail].Trash was marked as
containing 36 unread articles, when in fact it had none -- refreshing
Gnus reset the unread count to the proper value. Stepping through
`gnus-summary-move-article' and `nnimap-request-move-article' didn't
show anything immediately out of the ordinary, but I don't really
understand imap all that well.

In a bit, I'll try it again and step through `nnimap-request-set-mark'.
That's what's being used to update marks on [Gmail].Trash, and seems
like the next most likely place to find bugs. Like I said, though, I
don't know imap all that well, so someone tell me if I should be looking
elsewhere.

10:03:59 [localhost] 128 NOOP
10:03:59 [localhost] 129 NOOP
10:03:59 [localhost] 130 NOOP
10:03:59 [localhost] 131 NOOP
10:03:59 [localhost] 132 NOOP
10:07:19 [localhost] 133 SELECT "INBOX"
10:07:19 [localhost] 134 UID STORE 39339 FLAGS.SILENT (\Seen)
10:07:19 [localhost] 135 UID STORE 39339 -FLAGS.SILENT (\Flagged \Answered gnus-expire gnus-dormant gnus-score gnus-save gnus-download gnus-forward)
10:11:30 [localhost] 136 SELECT "INBOX"
10:11:30 [localhost] 137 UID FETCH 39339 (UID RFC822.SIZE BODYSTRUCTURE BODY.PEEK[HEADER.FIELDS (Subject From Date Message-Id References In-Reply-To Xref To Newsgroups Cc)])
10:12:02 [localhost] 138 UID COPY 39339 "[Gmail].Trash"
10:12:24 [localhost] 139 UID STORE 39339 +FLAGS.SILENT (\Deleted)
10:12:24 [localhost] 140 UID EXPUNGE 39339
10:18:59 [localhost] 141 NOOP
10:18:59 [localhost] 142 NOOP
10:18:59 [localhost] 143 NOOP
10:18:59 [localhost] 144 NOOP
10:18:59 [localhost] 145 NOOP
10:19:18 [localhost] 146 SELECT "[Gmail].Trash"
10:19:18 [localhost] 147 UID STORE 175 +FLAGS.SILENT (\Seen)
10:20:30 [localhost] 148 EXAMINE "[Gmail].Trash" (QRESYNC (1410345743 155))
10:20:43 [localhost] 149 SELECT "INBOX"
10:20:43 [localhost] 150 UID SEARCH SENTBEFORE 21-JAN-2015
10:20:46 [localhost] 151 SELECT "INBOX"
10:20:46 [localhost] 152 UID STORE 39339 +FLAGS.SILENT (\Seen)

[Eric Abrahamsen]

Post by Eric Abrahamsen

Post by Lars Ingebrigtsen

Post by Eric Abrahamsen
A local dovecot server, synced with gmail via isync.

Ah. For some reason, it seems that people running local IMAP servers
triggers a lot of bugs in Gnus...

Post by Eric Abrahamsen
I've always meant to go and follow the code, but it's never been
annoying enough for me to actually do it. But if you're messing nnimap
at the moment anyway, maybe now's a good time.

Please do. :-)

Doing this bit by bit, first of all here are the imap commands resulting
from moving article 39339 from INBOX to [Gmail].Trash, where it became
article 175. Upon leaving the INBOX group, [Gmail].Trash was marked as
containing 36 unread articles, when in fact it had none -- refreshing
Gnus reset the unread count to the proper value. Stepping through
`gnus-summary-move-article' and `nnimap-request-move-article' didn't
show anything immediately out of the ordinary, but I don't really
understand imap all that well.
In a bit, I'll try it again and step through `nnimap-request-set-mark'.
That's what's being used to update marks on [Gmail].Trash, and seems
like the next most likely place to find bugs. Like I said, though, I
don't know imap all that well, so someone tell me if I should be looking
elsewhere.

A little progress: `nnimap-request-set-mark' also looked normal, so I
followed the `gnus-group-get-new-news-this-group' call at the end of
`gnus-summary-move-article' instead.

In the `let' in the middle of that (gnus-group.el:4081 in current Gnus),
we call both `gnus-get-info' and `gnus-active' on the "group" variable.
The return value of `gnus-get-info' indicates a "seen" range of (1 .
177), which is correct. The return value of `gnus-active' is (1 . 211),
which isn't correct, and leads to the bogus unread count.

Off to look at what's happening in `gnus-active'...

E

[Eric Abrahamsen]

Post by Eric Abrahamsen

Post by Eric Abrahamsen

Post by Lars Ingebrigtsen

Post by Eric Abrahamsen
A local dovecot server, synced with gmail via isync.

Ah. For some reason, it seems that people running local IMAP servers
triggers a lot of bugs in Gnus...

Post by Eric Abrahamsen
I've always meant to go and follow the code, but it's never been
annoying enough for me to actually do it. But if you're messing nnimap
at the moment anyway, maybe now's a good time.

Please do. :-)

Doing this bit by bit, first of all here are the imap commands resulting
from moving article 39339 from INBOX to [Gmail].Trash, where it became
article 175. Upon leaving the INBOX group, [Gmail].Trash was marked as
containing 36 unread articles, when in fact it had none -- refreshing
Gnus reset the unread count to the proper value. Stepping through
`gnus-summary-move-article' and `nnimap-request-move-article' didn't
show anything immediately out of the ordinary, but I don't really
understand imap all that well.
In a bit, I'll try it again and step through `nnimap-request-set-mark'.
That's what's being used to update marks on [Gmail].Trash, and seems
like the next most likely place to find bugs. Like I said, though, I
don't know imap all that well, so someone tell me if I should be looking
elsewhere.

A little progress: `nnimap-request-set-mark' also looked normal, so I
followed the `gnus-group-get-new-news-this-group' call at the end of
`gnus-summary-move-article' instead.
In the `let' in the middle of that (gnus-group.el:4081 in current Gnus),
we call both `gnus-get-info' and `gnus-active' on the "group" variable.
The return value of `gnus-get-info' indicates a "seen" range of (1 .
177), which is correct. The return value of `gnus-active' is (1 . 211),
which isn't correct, and leads to the bogus unread count.
Off to look at what's happening in `gnus-active'...

Ugh, it looks at gnus-active-hashtb. I think this is where I pass the
baton to someone else.

[Eric Abrahamsen]

Post by Eric Abrahamsen

Post by Eric Abrahamsen

Post by Eric Abrahamsen

Post by Lars Ingebrigtsen

Post by Eric Abrahamsen
A local dovecot server, synced with gmail via isync.

Ah. For some reason, it seems that people running local IMAP servers
triggers a lot of bugs in Gnus...

Post by Eric Abrahamsen
I've always meant to go and follow the code, but it's never been
annoying enough for me to actually do it. But if you're messing nnimap
at the moment anyway, maybe now's a good time.

Please do. :-)

Doing this bit by bit, first of all here are the imap commands resulting
from moving article 39339 from INBOX to [Gmail].Trash, where it became
article 175. Upon leaving the INBOX group, [Gmail].Trash was marked as
containing 36 unread articles, when in fact it had none -- refreshing
Gnus reset the unread count to the proper value. Stepping through
`gnus-summary-move-article' and `nnimap-request-move-article' didn't
show anything immediately out of the ordinary, but I don't really
understand imap all that well.
In a bit, I'll try it again and step through `nnimap-request-set-mark'.
That's what's being used to update marks on [Gmail].Trash, and seems
like the next most likely place to find bugs. Like I said, though, I
don't know imap all that well, so someone tell me if I should be looking
elsewhere.

A little progress: `nnimap-request-set-mark' also looked normal, so I
followed the `gnus-group-get-new-news-this-group' call at the end of
`gnus-summary-move-article' instead.
In the `let' in the middle of that (gnus-group.el:4081 in current Gnus),
we call both `gnus-get-info' and `gnus-active' on the "group" variable.
The return value of `gnus-get-info' indicates a "seen" range of (1 .
177), which is correct. The return value of `gnus-active' is (1 . 211),
which isn't correct, and leads to the bogus unread count.
Off to look at what's happening in `gnus-active'...

Ugh, it looks at gnus-active-hashtb. I think this is where I pass the
baton to someone else.

Because this is driving me nuts...

I'm able to consistently reproduce the following:

1. (gnus-active "nnimap+PR:[Gmail].Trash") --> (1 . 178) ; correct
2. Move article into Trash
3. (gnus-active "nnimap+PR:[Gmail].Trash") --> (1 . 211) ; wrong
4. Update Gnus, or just the Trash group
5. (gnus-active "nnimap+PR:[Gmail].Trash") --> (1 . 179) ; correct

Guess what's weird! It looks like, in my current session, moving any
article into any group causes `gnus-active' to return (1 . 211) for that
group.

If the group has fewer than that many messages, that manifests as an
immediate unread count, representing the difference.

If the group has more than 211 messages, nothing immediately happens,
but the next time I refresh Gnus the group suddenly has a *whole lot*
more unread messages, and they are actually unread -- I can go into the
group and see them marked as unread.

I just experimented with a different group. Its correct active count was
(1 . 609). I moved a message into it -- didn't see anything funny. Next
time I refreshed Gnus it was marked as containing 399 unread messages,
and `gnus-active' returned the dreaded (1 . 211). When I entered the
group, it asked how many messages I wanted to see, default 403. I hit
return, and the summary buffer contained 399 unread messages. Some of
those messages I had previously ticked, but now they appeared completely
bare. I caught up and exited with "c", and the group went back to
normal. Next time I entered the group, all the old marks (ticks, etc)
were properly restored.

So: gremlins. 211 of them. Next time I restart Gnus I'll try this again,
and see if I get a new magic number. Back to work for now...

E

[Lars Ingebrigtsen]

Post by Eric Abrahamsen
3. (gnus-active "nnimap+PR:[Gmail].Trash") --> (1 . 211) ; wrong

211 is a very suspicious number, because it's the hard-coded return code
on every line. Like here:

(insert (format "211 %d %d %d %S\n"
(- (cdr active) (car active))
(car active)
(cdr active)
group))

Eurhm. Is that statement being interpreted as something else?

[Lars Ingebrigtsen]

Post by Lars Ingebrigtsen
211 is a very suspicious number, because it's the hard-coded return code

There was a missing `erase-buffer' there. Oops. Can you check whether
the fix I just pushed fixes this?

[Eric Abrahamsen]

Post by Lars Ingebrigtsen

Post by Lars Ingebrigtsen
211 is a very suspicious number, because it's the hard-coded return code

There was a missing `erase-buffer' there. Oops. Can you check whether
the fix I just pushed fixes this?

Looking good! Thanks *very* much for getting this. I'll let you know if
anything else goes wonky, but so far it seems okay.

[Tassilo Horn]

Post by Lars Ingebrigtsen

Post by Lars Ingebrigtsen
211 is a very suspicious number, because it's the hard-coded return

There was a missing `erase-buffer' there. Oops. Can you check
whether the fix I just pushed fixes this?

Is it possible that this commit also fixed the "broken unread count
after imap move/copy" issue I've had? At least it didn't happen to me
today.

Bye,
Tassilo

[Lars Ingebrigtsen]

Post by Tassilo Horn

Post by Lars Ingebrigtsen

Post by Lars Ingebrigtsen
211 is a very suspicious number, because it's the hard-coded return

There was a missing `erase-buffer' there. Oops. Can you check
whether the fix I just pushed fixes this?

Is it possible that this commit also fixed the "broken unread count
after imap move/copy" issue I've had? At least it didn't happen to me
today.

Yes, it's quite possible.