gnus uses a cache?

[Sharon Kimble]

I've come to the conclusion that there is some kind of caching going on
with gnus, and that if I close my gnus down when I hit some problems,
and then not open or use it in any form for over 5 hours, the cache
forgets and the problem is solved! The figure of 5 hours is purely
arbitrary, just plucked out of thin air!

Am I right?

Sharon.

[Adam Sjøgren]

Post by Sharon Kimble
Am I right?

It sounds more like you've joined a cargo cult.


Best regards,

Adam

[James Cloos]

SK> I've come to the conclusion that there is some kind of caching going
SK> on with gnus,

There is some caching of articles w/in a given session, but nothing I'm
aware of which would survive a restart.

-JimC

[Dan Christensen]

[Sorry to hijack the thread. Subject adjusted.]

Post by James Cloos
There is some caching of articles w/in a given session, but nothing I'm
aware of which would survive a restart.

Since upgrading Ubuntu and Gnus, I've noticed that the results of
nnmairix searches infiltrate subsequent search results. More precisely,
the second time I do a search, the summary buffer looks correct, but for
some of the articles, when I select them the *Article* buffer shows the
articles from the previous search. The third search may show article
buffers from the first or second search (or both).

My search results are stored in a local imap folder (dovecot), and I
have verified that on disk the correct articles are stored. I have also
verified that if I connect to dovecot via telnet, it displays the
correct article bodies. I am not using the agent (as far as I know).

I have also tried various nnmairix keystrokes to redo searches, etc,
and none of them helped.

Any idea why this is happening? nnmairix used to work beautifully for
me.

Dan

[Dan Christensen]

Post by Dan Christensen
[Sorry to hijack the thread. Subject adjusted.]

Post by James Cloos
There is some caching of articles w/in a given session, but nothing I'm
aware of which would survive a restart.

Since upgrading Ubuntu and Gnus, I've noticed that the results of
nnmairix searches infiltrate subsequent search results. More precisely,
the second time I do a search, the summary buffer looks correct, but for
some of the articles, when I select them the *Article* buffer shows the
articles from the previous search. The third search may show article
buffers from the first or second search (or both).
My search results are stored in a local imap folder (dovecot), and I
have verified that on disk the correct articles are stored. I have also
verified that if I connect to dovecot via telnet, it displays the
correct article bodies. I am not using the agent (as far as I know).
I have also tried various nnmairix keystrokes to redo searches, etc,
and none of them helped.
Any idea why this is happening? nnmairix used to work beautifully for
me.
Dan

I finally figured this out. Setting gnus-keep-backlog to nil solved
the problem. It turns out that by default, gnus caches the most recent
20 articles you have viewed, rather than contacting the server again.
This is true even if you exit and reenter a summary buffer.

Maybe nnmairix should remove articles from this cache when it creates
a search folder? Or bind this variable to nil in nnmairix groups?

Dan

[Eric Abrahamsen]

Post by Dan Christensen

Post by Dan Christensen
[Sorry to hijack the thread. Subject adjusted.]

Post by James Cloos
There is some caching of articles w/in a given session, but nothing I'm
aware of which would survive a restart.

Since upgrading Ubuntu and Gnus, I've noticed that the results of
nnmairix searches infiltrate subsequent search results. More precisely,
the second time I do a search, the summary buffer looks correct, but for
some of the articles, when I select them the *Article* buffer shows the
articles from the previous search. The third search may show article
buffers from the first or second search (or both).
My search results are stored in a local imap folder (dovecot), and I
have verified that on disk the correct articles are stored. I have also
verified that if I connect to dovecot via telnet, it displays the
correct article bodies. I am not using the agent (as far as I know).
I have also tried various nnmairix keystrokes to redo searches, etc,
and none of them helped.
Any idea why this is happening? nnmairix used to work beautifully for
me.
Dan

I finally figured this out. Setting gnus-keep-backlog to nil solved
the problem. It turns out that by default, gnus caches the most recent
20 articles you have viewed, rather than contacting the server again.
This is true even if you exit and reenter a summary buffer.
Maybe nnmairix should remove articles from this cache when it creates
a search folder? Or bind this variable to nil in nnmairix groups?

Hmm, I may have seen this in nnir groups, as well. At least, I've (only
once or twice) had a similar situation: selecting articles in an nnir
search summary buffer, and seeing article bodies from the last search. I
can only assume it's the same problem...

[Eric Abrahamsen]

Post by Eric Abrahamsen

Post by Dan Christensen

Post by Dan Christensen
[Sorry to hijack the thread. Subject adjusted.]

Post by James Cloos
There is some caching of articles w/in a given session, but nothing I'm
aware of which would survive a restart.

Since upgrading Ubuntu and Gnus, I've noticed that the results of
nnmairix searches infiltrate subsequent search results. More precisely,
the second time I do a search, the summary buffer looks correct, but for
some of the articles, when I select them the *Article* buffer shows the
articles from the previous search. The third search may show article
buffers from the first or second search (or both).
My search results are stored in a local imap folder (dovecot), and I
have verified that on disk the correct articles are stored. I have also
verified that if I connect to dovecot via telnet, it displays the
correct article bodies. I am not using the agent (as far as I know).
I have also tried various nnmairix keystrokes to redo searches, etc,
and none of them helped.
Any idea why this is happening? nnmairix used to work beautifully for
me.
Dan

I finally figured this out. Setting gnus-keep-backlog to nil solved
the problem. It turns out that by default, gnus caches the most recent
20 articles you have viewed, rather than contacting the server again.
This is true even if you exit and reenter a summary buffer.
Maybe nnmairix should remove articles from this cache when it creates
a search folder? Or bind this variable to nil in nnmairix groups?

Hmm, I may have seen this in nnir groups, as well. At least, I've (only
once or twice) had a similar situation: selecting articles in an nnir
search summary buffer, and seeing article bodies from the last search. I
can only assume it's the same problem...

Just had it again! See attachment -- the article body shown when
selecting one summary line actually belongs to a different summary line.
This time it seems fairly reproducible (and is messing with
Gnorb-related stuff) so I'll see if I figure it out, starting with
setting gnus-keep-backlog to 0.

[Eric Abrahamsen]

Post by Eric Abrahamsen

Post by Eric Abrahamsen

Post by Dan Christensen

Post by Dan Christensen
[Sorry to hijack the thread. Subject adjusted.]

Post by James Cloos
There is some caching of articles w/in a given session, but nothing I'm
aware of which would survive a restart.

Since upgrading Ubuntu and Gnus, I've noticed that the results of
nnmairix searches infiltrate subsequent search results. More precisely,
the second time I do a search, the summary buffer looks correct, but for
some of the articles, when I select them the *Article* buffer shows the
articles from the previous search. The third search may show article
buffers from the first or second search (or both).
My search results are stored in a local imap folder (dovecot), and I
have verified that on disk the correct articles are stored. I have also
verified that if I connect to dovecot via telnet, it displays the
correct article bodies. I am not using the agent (as far as I know).
I have also tried various nnmairix keystrokes to redo searches, etc,
and none of them helped.
Any idea why this is happening? nnmairix used to work beautifully for
me.
Dan

I finally figured this out. Setting gnus-keep-backlog to nil solved
the problem. It turns out that by default, gnus caches the most recent
20 articles you have viewed, rather than contacting the server again.
This is true even if you exit and reenter a summary buffer.
Maybe nnmairix should remove articles from this cache when it creates
a search folder? Or bind this variable to nil in nnmairix groups?

Hmm, I may have seen this in nnir groups, as well. At least, I've (only
once or twice) had a similar situation: selecting articles in an nnir
search summary buffer, and seeing article bodies from the last search. I
can only assume it's the same problem...

Just had it again! See attachment -- the article body shown when
selecting one summary line actually belongs to a different summary line.
This time it seems fairly reproducible (and is messing with
Gnorb-related stuff) so I'll see if I figure it out, starting with
setting gnus-keep-backlog to 0.

And yes, that worked. My guess is that gnus should just not enter
articles into the backlog if they came from a search-composed group.

Ie, the check at the very top of `gnus-backlog-enter-article' needs to
be expanded. I thought that gnus-ephemeral-group-p might be the right
check, but that doesn't flag nnir/nnmairix groups. I need to run out,
but will look at this more later -- or if anyone knows a nice clean
filter, rather than checking against a bunch of regexps?

Eric

[Alan Schmitt]

Post by Eric Abrahamsen

Post by Eric Abrahamsen

Post by Eric Abrahamsen
Hmm, I may have seen this in nnir groups, as well. At least, I've (only
once or twice) had a similar situation: selecting articles in an nnir
search summary buffer, and seeing article bodies from the last search. I
can only assume it's the same problem...

Just had it again! See attachment -- the article body shown when
selecting one summary line actually belongs to a different summary line.
This time it seems fairly reproducible (and is messing with
Gnorb-related stuff) so I'll see if I figure it out, starting with
setting gnus-keep-backlog to 0.

And yes, that worked. My guess is that gnus should just not enter
articles into the backlog if they came from a search-composed group.

Thank you for that suggestion. This bug has been bugging me for a while
(when I jump to a message from a notmuch search), and I had no idea how
to address it.

Alan

[Eric Abrahamsen]

Post by Dan Christensen

Post by Dan Christensen
[Sorry to hijack the thread. Subject adjusted.]

Post by James Cloos
There is some caching of articles w/in a given session, but nothing I'm
aware of which would survive a restart.

Since upgrading Ubuntu and Gnus, I've noticed that the results of
nnmairix searches infiltrate subsequent search results. More precisely,
the second time I do a search, the summary buffer looks correct, but for
some of the articles, when I select them the *Article* buffer shows the
articles from the previous search. The third search may show article
buffers from the first or second search (or both).
My search results are stored in a local imap folder (dovecot), and I
have verified that on disk the correct articles are stored. I have also
verified that if I connect to dovecot via telnet, it displays the
correct article bodies. I am not using the agent (as far as I know).
I have also tried various nnmairix keystrokes to redo searches, etc,
and none of them helped.
Any idea why this is happening? nnmairix used to work beautifully for
me.
Dan

I finally figured this out. Setting gnus-keep-backlog to nil solved
the problem. It turns out that by default, gnus caches the most recent
20 articles you have viewed, rather than contacting the server again.
This is true even if you exit and reenter a summary buffer.
Maybe nnmairix should remove articles from this cache when it creates
a search folder? Or bind this variable to nil in nnmairix groups?
Dan

Can you or someone else using nnmairix tell me if calling
(gnus-virtual-group-p "nnmairix:your group name") returns t? It's true
for nnvirtual and nnir groups, so that function might make a good guard
inside `gnus-backlog-enter-article'.

Eric

[Dan Christensen]

Post by Eric Abrahamsen

Post by Dan Christensen
I finally figured this out. Setting gnus-keep-backlog to nil solved
the problem. It turns out that by default, gnus caches the most recent
20 articles you have viewed, rather than contacting the server again.
This is true even if you exit and reenter a summary buffer.
Maybe nnmairix should remove articles from this cache when it creates
a search folder? Or bind this variable to nil in nnmairix groups?
Dan

Can you or someone else using nnmairix tell me if calling
(gnus-virtual-group-p "nnmairix:your group name") returns t? It's true
for nnvirtual and nnir groups, so that function might make a good guard
inside `gnus-backlog-enter-article'.
Eric

Unfortunately, that doesn't work:

(gnus-virtual-group-p "nnmairix+mairixserver:nnmairixsearch")
nil

So maybe a combination of checks is needed. Or just check if the
backend is in a banned list which includes nnmairix, nnir, etc.
This is what the registry does in gnus.el:

;; The Gnus registry's ignored groups
(gnus-define-group-parameter
registry-ignore
:type list
:function-document
"Whether this group should be ignored by the registry."
:variable gnus-registry-ignored-groups
:variable-default (mapcar
(lambda (g) (list g t))
'("delayed$" "drafts$" "queue$" "INBOX$"
"^nnmairix:" "^nnir:" "archive"))
...

Or maybe gnus-virtual-group-p should be changed to declare that nnmairix
groups are virtual? This could be accomplished by changing
gnus-valid-select-methods. In gnus.el, it doesn't list nnmairix at all,
but somehow when using nnmairix, an entry for nnmairix gets added to
this list, without the virtual keyword (see below).

I don't know enough about the internals to know the best way to
proceed.

Dan


gnus-valid-select-methods is a variable defined in `gnus.el'.

Value: (("nntp" post address prompt-address physical-address cloud)
("nnspool" post address)
("nnvirtual" post-mail virtual prompt-address)
("nnmbox" mail respool address)
("nnml" post-mail respool address)
("nnmh" mail respool address)
("nndir" post-mail prompt-address physical-address)
("nneething" none address prompt-address physical-address)
("nndoc" none address prompt-address)
("nnbabyl" mail address respool)
("nndraft" post-mail)
("nnfolder" mail respool address)
("nngateway" post-mail address prompt-address physical-address)
("nnweb" none)
("nnrss" none global)
("nnagent" post-mail)
("nnimap" post-mail address prompt-address physical-address respool server-marks cloud)
("nnmaildir" mail respool address server-marks)
("nnnil" none)
("nnmairix" mail address)
("nnir" mail virtual))

[Andreas Schwab]

In gnus.el, it doesn't list nnmairix at all, but somehow when using
nnmairix, an entry for nnmairix gets added to this list, without the
virtual keyword (see below).

diff --git a/lisp/nnmairix.el b/lisp/nnmairix.el
index 0cef699..b2f74e3 100644
--- a/lisp/nnmairix.el
+++ b/lisp/nnmairix.el
@@ -417,7 +417,7 @@ Other back ends might or might not work.")

(nnoo-define-basics nnmairix)

-(gnus-declare-backend "nnmairix" 'mail 'address)
+(gnus-declare-backend "nnmairix" 'mail 'address 'virtual)

(deffoo nnmairix-open-server (server &optional definitions)
;; just set server variables

Andreas.

[Eric Abrahamsen]

Post by Andreas Schwab

In gnus.el, it doesn't list nnmairix at all, but somehow when using
nnmairix, an entry for nnmairix gets added to this list, without the
virtual keyword (see below).

diff --git a/lisp/nnmairix.el b/lisp/nnmairix.el
index 0cef699..b2f74e3 100644
--- a/lisp/nnmairix.el
+++ b/lisp/nnmairix.el
@@ -417,7 +417,7 @@ Other back ends might or might not work.")
(nnoo-define-basics nnmairix)
-(gnus-declare-backend "nnmairix" 'mail 'address)
+(gnus-declare-backend "nnmairix" 'mail 'address 'virtual)
(deffoo nnmairix-open-server (server &optional definitions)
;; just set server variables
Andreas.

After look at this for a while, I do think this is the right solution --
what do you all think? Can someone apply this change?

[Eric Abrahamsen]

Post by Eric Abrahamsen

Post by Andreas Schwab

In gnus.el, it doesn't list nnmairix at all, but somehow when using
nnmairix, an entry for nnmairix gets added to this list, without the
virtual keyword (see below).

diff --git a/lisp/nnmairix.el b/lisp/nnmairix.el
index 0cef699..b2f74e3 100644
--- a/lisp/nnmairix.el
+++ b/lisp/nnmairix.el
@@ -417,7 +417,7 @@ Other back ends might or might not work.")
(nnoo-define-basics nnmairix)
-(gnus-declare-backend "nnmairix" 'mail 'address)
+(gnus-declare-backend "nnmairix" 'mail 'address 'virtual)
(deffoo nnmairix-open-server (server &optional definitions)
;; just set server variables
Andreas.

After look at this for a while, I do think this is the right solution --
what do you all think? Can someone apply this change?

Oh never mind, I guess I didn't look long enough. For any backend
declared as virtual, `gnus-cache-possibly-enter-article' will use
`nnvirtual-find-group-art' to get the real group name.

That means that if a backend is declared 'virtual but isn't actually
nnvirtual, the caching mechanism will use the wrong lookup function.

nnir should use `nnir-article-group'

nnmairix could probably do it with a little massaging of
`nnmairix-determine-original-group-from-path'.

What we'd really want is a properly-abstracted nnoo function that
virtual backends would use to locate the real article.

In the meantime, it would probably be enough to make
`gnus-uncacheable-groups' match nnmairix and nnir groups.

Eric

[Dan Christensen]

Post by Eric Abrahamsen

Post by Eric Abrahamsen

Post by Andreas Schwab

In gnus.el, it doesn't list nnmairix at all, but somehow when using
nnmairix, an entry for nnmairix gets added to this list, without the
virtual keyword (see below).

diff --git a/lisp/nnmairix.el b/lisp/nnmairix.el
index 0cef699..b2f74e3 100644
--- a/lisp/nnmairix.el
+++ b/lisp/nnmairix.el
@@ -417,7 +417,7 @@ Other back ends might or might not work.")
(nnoo-define-basics nnmairix)
-(gnus-declare-backend "nnmairix" 'mail 'address)
+(gnus-declare-backend "nnmairix" 'mail 'address 'virtual)
(deffoo nnmairix-open-server (server &optional definitions)
;; just set server variables
Andreas.

After look at this for a while, I do think this is the right solution --
what do you all think? Can someone apply this change?

Oh never mind, I guess I didn't look long enough. For any backend
declared as virtual, `gnus-cache-possibly-enter-article' will use
`nnvirtual-find-group-art' to get the real group name.
That means that if a backend is declared 'virtual but isn't actually
nnvirtual, the caching mechanism will use the wrong lookup function.

I don't follow you. Isn't the relevant function gnus-backlog-enter-article,
which skips the backlog for groups that are declared virtual?

I haven't actually tested the patch and instead simply disabled the
backlog entirely, but I suspect the patch will work.

Dan

[Eric Abrahamsen]

Post by Dan Christensen

Post by Eric Abrahamsen

Post by Eric Abrahamsen

Post by Andreas Schwab

In gnus.el, it doesn't list nnmairix at all, but somehow when using
nnmairix, an entry for nnmairix gets added to this list, without the
virtual keyword (see below).

diff --git a/lisp/nnmairix.el b/lisp/nnmairix.el
index 0cef699..b2f74e3 100644
--- a/lisp/nnmairix.el
+++ b/lisp/nnmairix.el
@@ -417,7 +417,7 @@ Other back ends might or might not work.")
(nnoo-define-basics nnmairix)
-(gnus-declare-backend "nnmairix" 'mail 'address)
+(gnus-declare-backend "nnmairix" 'mail 'address 'virtual)
(deffoo nnmairix-open-server (server &optional definitions)
;; just set server variables
Andreas.

After look at this for a while, I do think this is the right solution --
what do you all think? Can someone apply this change?

Oh never mind, I guess I didn't look long enough. For any backend
declared as virtual, `gnus-cache-possibly-enter-article' will use
`nnvirtual-find-group-art' to get the real group name.
That means that if a backend is declared 'virtual but isn't actually
nnvirtual, the caching mechanism will use the wrong lookup function.

I don't follow you. Isn't the relevant function gnus-backlog-enter-article,
which skips the backlog for groups that are declared virtual?
I haven't actually tested the patch and instead simply disabled the
backlog entirely, but I suspect the patch will work.

Ha, yes! Between then and now I somehow started looking at the wrong
mechanism altogether. Sorry about that.

I've attached patches for both the backlog edit and the nnmairix edit.

Eric

[Eric Abrahamsen]

Post by Eric Abrahamsen
I've attached patches for both the backlog edit and the nnmairix edit.

...

Post by Eric Abrahamsen
(defun gnus-backlog-enter-article (group number buffer)
(when (and (numberp number)
- (not (string-match "^nnvirtual" group)))
+ (not (gnus-virtual-group-p group)))

I guess gnus-backlog-request-article should have a similar change made?
Dan

Good point! New patches...

[Lars Ingebrigtsen]

Post by Eric Abrahamsen
Good point! New patches...

Thanks; applied.

[Ted Zlatanov]

On Sat, 06 Sep 2014 23:36:24 +0100 Sharon Kimble <***@skimble.plus.com> wrote:

SK> I've come to the conclusion that there is some kind of caching going on
SK> with gnus, and that if I close my gnus down when I hit some problems,
SK> and then not open or use it in any form for over 5 hours, the cache
SK> forgets and the problem is solved! The figure of 5 hours is purely
SK> arbitrary, just plucked out of thin air!

Why don't you describe the problems and how to replicate them? Often
that triggers an "aha" moment.

Ted