check mtime of newsrc.eld before saving it

[Ted Zlatanov]

I sometimes update my newsrc.eld in a directory shared between multiple
machines. I wanted a way to stop Gnus from overwriting the updated
file.

I came up with the following untested patch, which adds a customizable
option to do that check and saves the last modification time. But I
didn't finish testing or commit it because I had two questions:

1) is there a more elegant way to do this with file reverting? It seems
like Emacs can usually tell if the file needs to be reverted.

2) should I make and use a new newsrc pre-save hook?

Let me know what you think...

Thanks
Ted

[Eric Abrahamsen]

Post by Ted Zlatanov
I sometimes update my newsrc.eld in a directory shared between multiple
machines. I wanted a way to stop Gnus from overwriting the updated
file.
I came up with the following untested patch, which adds a customizable
option to do that check and saves the last modification time. But I
1) is there a more elegant way to do this with file reverting? It seems
like Emacs can usually tell if the file needs to be reverted.
2) should I make and use a new newsrc pre-save hook?
Let me know what you think...

I don't know too much about this, but got curious and went looking. So
far as I can tell, Emacs' clever file reversion/overwrite stuff is tied
to files that are visited by buffers (the buffer itself keeps track of
and compares last modification times), and not so much files that are
simply read and written. So I'm guessing your approach is probably the
right thing to do.

You probably knew all that already, but I learned something!

Eric

(elisp) Modification Time

[Ted Zlatanov]

On Tue, 07 Oct 2014 11:49:17 +0800 Eric Abrahamsen <***@ericabrahamsen.net> wrote:

EA> I don't know too much about this, but got curious and went looking. So
EA> far as I can tell, Emacs' clever file reversion/overwrite stuff is tied
EA> to files that are visited by buffers (the buffer itself keeps track of
EA> and compares last modification times), and not so much files that are
EA> simply read and written. So I'm guessing your approach is probably the
EA> right thing to do.

I could have used the buffer-specific stuff but there's actually another
version of the code that saves the newsrc, which uses file renames
instead of overwrites. So it got too complicated and I went back to my
original version. I added a y-or-n prompt and pushed to Git so it's
ready for use; just set `gnus-save-newsrc-file-check-timestamp' and
that's it. If anyone can confirm it works all right for them, I will
add docs.

Ted

[Lars Ingebrigtsen]

Post by Ted Zlatanov
I sometimes update my newsrc.eld in a directory shared between multiple
machines. I wanted a way to stop Gnus from overwriting the updated
file.

[...]

Post by Ted Zlatanov
+(defcustom gnus-save-newsrc-file-check-timestamp nil
+ "Check the modification time of the newsrc.eld file before saving it.

I like it, but I don't think we really need this variable. Instead the
checking should always be on, and

[...]

Post by Ted Zlatanov
+ (error "Couldn't save %s because it was updated" checkfile))))

this should be a `y-or-n-p' instead, I think. That would make Gnus'
behaviour here be pretty consistent with how Emacs otherwise behaves
when writing data to files.

[Ted Zlatanov]

Post by Ted Zlatanov
I sometimes update my newsrc.eld in a directory shared between multiple
machines. I wanted a way to stop Gnus from overwriting the updated
file.
+(defcustom gnus-save-newsrc-file-check-timestamp nil
+ "Check the modification time of the newsrc.eld file before saving it.

LI> I like it, but I don't think we really need this variable. Instead the
LI> checking should always be on

OK, I can make that change. But then I need...

Post by Ted Zlatanov
+ (error "Couldn't save %s because it was updated" checkfile))))

LI> this should be a `y-or-n-p' instead, I think. That would make Gnus'
LI> behaviour here be pretty consistent with how Emacs otherwise behaves
LI> when writing data to files.

...a way to merge the on-disk newsrc.eld with the current one, because
currently aborting is the only way to avoid overwriting either side's
marks.

Something that walks all the groups and merges each entry would be
ideal; if the time of last group exit was recorded somewhere in the
marks, then this would be simply a timestamp compare. Otherwise we may
have to prompt the user for any groups with different marks. Could the
newsrc.eld format be modified to record this?

This is kind of related to the gnus-sync thing.

Ted

[Steinar Bang]

Post by Ted Zlatanov
This is kind of related to the gnus-sync thing.

Yes. Any chance of progress on that...?

[Ted Zlatanov]

Post by Ted Zlatanov
This is kind of related to the gnus-sync thing.

SB> Yes. Any chance of progress on that...?

Yes. I need the IMAP backing in gnus-cloud.el that Lars was working on.

Ted

[Steinar Bang]

Post by Ted Zlatanov

Post by Ted Zlatanov
This is kind of related to the gnus-sync thing.

SB> Yes. Any chance of progress on that...?

Post by Ted Zlatanov
Yes. I need the IMAP backing in gnus-cloud.el that Lars was working on.

Any news about the cloudy stuff?

I upgraded to debian 8.0 "jessie" recently, which bumped couchdb from
1.2.0 to 1.4.0 and now gnus-sync.el doesn't work and I don't have sync
between my gnusen.

I could spend time debugging why gnus-sync.el doesn't work with couchdb
1.4.0, but if gnus-cloud is just around the corner, that would be a
waste of time and effort...? :-)

[Steinar Bang]

Post by Steinar Bang
I upgraded to debian 8.0 "jessie" recently, which bumped couchdb from
1.2.0 to 1.4.0 and now gnus-sync.el doesn't work and I don't have sync
between my gnusen.
I could spend time debugging why gnus-sync.el doesn't work with
couchdb 1.4.0, but if gnus-cloud is just around the corner, that would
be a waste of time and effort...? :-)

I've given up on the couchdb route, all that happens when trying to
initialize couchdb 1.4.0 is that I'm getting strange error messages,
like e.g. a 404 on /sb/_design/lesync (my base is named "sb" and not
"tzz"), and /var/couchdb/couchdb.log fils up with JSON dumps with error
messages I don't understand.

So I'm trying to go the SSH file sync route, and I'm trying use plink as
the tramp method.

I have done this in .emacs:
(when windows-emacs
(require 'tramp)
(setenv "PATH" (concat "C:\\ProgramFiles\\PuTTY;" (getenv "PATH")))
(setq tramp-default-method "plink"))

and I have done this in .gnus.el:
(setq gnus-sync-backend '("/***@sync.mydomain.no:sync/gnus")
gnus-sync-global-vars '(gnus-newsrc-last-checked-date)
gnus-sync-newsrc-groups '("nntp" "nnrss")
gnus-sync-newsrc-offsets '(2 3))

I have done
C-x d /***@sync.mydomain.no:sync RET
and successfully opened the remote directory using plink.

When I do
C-u M-x gnus-sync-save RET
with these settings, nothing happens.

Any idea? Do I have to make the sync file path explicitly use "plink:"?
Or do I need to use an ssh implementation with an executable called
ssh.exe (it doesn't look like there is anything in gnus-sync that cares,
as long as the file can be reached...?)

[Steinar Bang]

Post by Steinar Bang
So I'm trying to go the SSH file sync route, and I'm trying use plink as
the tramp method.
(when windows-emacs
(require 'tramp)
(setenv "PATH" (concat "C:\\ProgramFiles\\PuTTY;" (getenv "PATH")))
(setq tramp-default-method "plink"))
gnus-sync-global-vars '(gnus-newsrc-last-checked-date)
gnus-sync-newsrc-groups '("nntp" "nnrss")
gnus-sync-newsrc-offsets '(2 3))

The problem was that I had made gnus-sync-backend a list containing a
string, instead of just a string. If I did it like this, it worked over
a plink tramp connection:
(setq gnus-sync-backend '("/***@sync.mydomain.no:sync/gnus")
gnus-sync-global-vars '(gnus-newsrc-last-checked-date)
gnus-sync-newsrc-groups '("nntp" "nnrss")
gnus-sync-newsrc-offsets '(2 3))

Very slow compared to the lesync version, though.

[Steinar Bang]

Post by Steinar Bang

Post by Steinar Bang
So I'm trying to go the SSH file sync route, and I'm trying use plink as
the tramp method.
(when windows-emacs
(require 'tramp)
(setenv "PATH" (concat "C:\\ProgramFiles\\PuTTY;" (getenv "PATH")))
(setq tramp-default-method "plink"))
gnus-sync-global-vars '(gnus-newsrc-last-checked-date)
gnus-sync-newsrc-groups '("nntp" "nnrss")
gnus-sync-newsrc-offsets '(2 3))

The problem was that I had made gnus-sync-backend a list containing a
string, instead of just a string. If I did it like this, it worked over

Forgot to correct the lisp code, this is what works:
(setq gnus-sync-backend "/***@sync.mydomain.no:sync/gnus"
gnus-sync-global-vars '(gnus-newsrc-last-checked-date)
gnus-sync-newsrc-groups '("nntp" "nnrss")
gnus-sync-newsrc-offsets '(2 3))

Post by Steinar Bang
Very slow compared to the lesync version, though.

Not really. I just misinterpreted the feedback from the initial save
and thought it was waiting for a response when it was actually finished.

[Sivaram Neelakantan]

[snipped 24 lines]

Post by Steinar Bang

Post by Steinar Bang
Very slow compared to the lesync version, though.

Not really. I just misinterpreted the feedback from the initial save
and thought it was waiting for a response when it was actually finished.

Could you post the full setup and how you got it working, please?

sivaram
--

[Steinar Bang]

Post by Sivaram Neelakantan
Could you post the full setup and how you got it working, please?

First:
- You need a server you can SSH into
- Make sure PuTTY works against this server using public/private key
authentication (I do it without using a passphrase, no idea what
would be needed to make it work with a passphrase)

Setting up PuTTY in this way is left as an exercise for the reader.

However using the value for gnus-sync-backend below, the "Host Name (or
IP address)" field in "Session" should be: ***@sync.mydomain.no

The session name in "Saved Sessions" should be: sync.mydomain.no

The session is then saved.

Secondly the setup og emacs is as follows:
1. Adding plink.exe to emacs's PATH and making plink be the default
remote file method, put this in ~/.emacs:
(require 'tramp)
(setenv "PATH" (concat "C:\\ProgramFiles\\PuTTY;" (getenv "PATH")))
(setq tramp-default-method "plink")
2. Put the following in ~/.gnus.el:
(require 'gnus-sync)
(setq gnus-sync-backend "/***@sync.mydomain.no:sync/gnus"
gnus-sync-global-vars '(gnus-newsrc-last-checked-date)
gnus-sync-newsrc-groups '("nntp" "nnrss")
gnus-sync-newsrc-offsets '(2 3))
3. Make sure "gzip" is in emacs' PATH (I got it from GnuWin32), which I
do the following way:
(setenv "PATH" (concat "c:\\ProgramFiles\\ezwinports\\bin;C:\\ProgramFiles\\GnuWin32\\bin;" (getenv "PATH"))

To initially save all NNTP groups, do: C-u M-x gnus-sync-save RET
Later
- save with: M-x gnus-sync-save RET
- load with: M-x gnus-sync-read RET
- After the gnus-sync-read command has completed, to 'g' in the
*Group* buffer, and the read numbers will be adjusted

[Lars Ingebrigtsen]

Post by Ted Zlatanov
LI> this should be a `y-or-n-p' instead, I think. That would make Gnus'
LI> behaviour here be pretty consistent with how Emacs otherwise behaves
LI> when writing data to files.
...a way to merge the on-disk newsrc.eld with the current one, because
currently aborting is the only way to avoid overwriting either side's
marks.

Well, if you answer "yes" to "file change; write anyway?", Emacs will
overwrite changes made elsewhere to files normally. So this would make
Gnus be pretty consistent with how Emacs works elsewhere, wouldn't it?

[Ted Zlatanov]

On Thu, 05 Feb 2015 14:13:09 +1100 Lars Ingebrigtsen <***@gnus.org> wrote:

LI> Ted Zlatanov <***@lifelogs.com> writes:
LI> this should be a `y-or-n-p' instead, I think. That would make Gnus'
LI> behaviour here be pretty consistent with how Emacs otherwise behaves
LI> when writing data to files.

Post by Ted Zlatanov
...a way to merge the on-disk newsrc.eld with the current one, because
currently aborting is the only way to avoid overwriting either side's
marks.

LI> Well, if you answer "yes" to "file change; write anyway?", Emacs will
LI> overwrite changes made elsewhere to files normally. So this would make
LI> Gnus be pretty consistent with how Emacs works elsewhere, wouldn't it?

Yes, but it's not what users want. For me, at least (I count for two :)

The correct behavior would be to merge the newer newsrc as if it was a
dribble file. I think

Ted

[Lars Ingebrigtsen]

Post by Ted Zlatanov
The correct behavior would be to merge the newer newsrc as if it was a
dribble file. I think

Yes, that would also be a good thing, but probably a separate issue
(that perhaps the cloud stuff may fix)...

[Steinar Bang]

Post by Lars Ingebrigtsen

Post by Ted Zlatanov
The correct behavior would be to merge the newer newsrc as if it was a
dribble file. I think

Yes, that would also be a good thing, but probably a separate issue
(that perhaps the cloud stuff may fix)...

The cloud stuff, we wants it! Yes we doesss!

[Ted Zlatanov]

Post by Ted Zlatanov
+ (error "Couldn't save %s because it was updated" checkfile))))

LI> this should be a `y-or-n-p' instead, I think. That would make Gnus'
LI> behaviour here be pretty consistent with how Emacs otherwise behaves
LI> when writing data to files.

I looked and there is a `y-or-n-p' just above. The newsrc save flow is:

* if the timestamp is OK, proceed
* if the timestamp is newer, `y-or-n-p' to save
* if yes, proceed (overwriting)
* if no, error out

How should it work instead?

Ted

[Lars Ingebrigtsen]

Post by Ted Zlatanov
* if the timestamp is OK, proceed
* if the timestamp is newer, `y-or-n-p' to save
* if yes, proceed (overwriting)
* if no, error out

Hm... I think that's the right work flow, indeed.