discussion and development of piem
 help / color / mirror / Atom feed
* [PATCH 0/3] inject: Add support for skipping messages
@ 2020-08-22 19:01 Kyle Meyer
  2020-08-22 19:01 ` [PATCH 1/3] inject: Abbreviate piem-maildir-directory in message Kyle Meyer
                   ` (2 more replies)
  0 siblings, 3 replies; 4+ messages in thread
From: Kyle Meyer @ 2020-08-22 19:01 UTC (permalink / raw)
  To: piem

When I'm using Gnus to read a thread via nntp and decide I want it in
my regular mail (notmuch), I call piem-inject-thread-into-maildir to
pull the thread into my notmuch database.  If I call it again at a
later time to get new messages, a duplicate message is added for each
message that came in with the previous call.

For notmuch at least, that's not a big deal because those messages are
mostly hidden, but it'd still be nice to avoid.  This series makes
that possible with

    (setq piem-mail-injection-skipif-predicate
          #'piem-notmuch-known-mid-p)

This might also end up being useful outside of avoiding duplicates for
things like this:

    (setq piem-mail-injection-skipif-predicate
          #'<skip message if From: matches X>)

  [1/3] inject: Abbreviate piem-maildir-directory in message
  [2/3] inject: Support skipping messages
  [3/3] notmuch: Extract "known message?" logic to function

 piem-notmuch.el | 32 +++++++++++++----------
 piem.el         | 67 ++++++++++++++++++++++++++++++++++++-------------
 2 files changed, 68 insertions(+), 31 deletions(-)


base-commit: 846c69cae321442bb2f5cefec7000328943c7560
-- 
2.28.0


^ permalink raw reply	[flat|nested] 4+ messages in thread

* [PATCH 1/3] inject: Abbreviate piem-maildir-directory in message
  2020-08-22 19:01 [PATCH 0/3] inject: Add support for skipping messages Kyle Meyer
@ 2020-08-22 19:01 ` Kyle Meyer
  2020-08-22 19:01 ` [PATCH 2/3] inject: Support skipping messages Kyle Meyer
  2020-08-22 19:01 ` [PATCH 3/3] notmuch: Extract "known message?" logic to function Kyle Meyer
  2 siblings, 0 replies; 4+ messages in thread
From: Kyle Meyer @ 2020-08-22 19:01 UTC (permalink / raw)
  To: piem

The message is going to get longer when skip counts are reported, so
try to save some characters.
---
 piem.el | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/piem.el b/piem.el
index 3669cfe..5fc7f04 100644
--- a/piem.el
+++ b/piem.el
@@ -444,7 +444,8 @@ (defun piem--inject-thread-callback (status mid message-only)
               (piem--url-decompress))
             (let ((message-count (piem--write-mbox-to-maildir)))
               (message "%d message(s) for %s moved to %s"
-                       message-count mid piem-maildir-directory))
+                       message-count mid
+                       (abbreviate-file-name piem-maildir-directory)))
             (run-hook-with-args 'piem-after-mail-injection-functions mid)))
       (and (buffer-live-p buffer)
            (kill-buffer buffer)))))
-- 
2.28.0


^ permalink raw reply	[flat|nested] 4+ messages in thread

* [PATCH 2/3] inject: Support skipping messages
  2020-08-22 19:01 [PATCH 0/3] inject: Add support for skipping messages Kyle Meyer
  2020-08-22 19:01 ` [PATCH 1/3] inject: Abbreviate piem-maildir-directory in message Kyle Meyer
@ 2020-08-22 19:01 ` Kyle Meyer
  2020-08-22 19:01 ` [PATCH 3/3] notmuch: Extract "known message?" logic to function Kyle Meyer
  2 siblings, 0 replies; 4+ messages in thread
From: Kyle Meyer @ 2020-08-22 19:01 UTC (permalink / raw)
  To: piem

piem-inject-thread-into-maildir allows pulling a thread from a
public-inbox URL, but repeating the process to get new messages in the
thread leads to duplicates in the local store.  Add a predicate that
the caller can configure to avoid this.

Note that the predicate has the entire message available to make its
decision, but it's called with the message ID for convenience because
that's likely the only information needed to determine if the message
is already in the local store.
---
 piem.el | 63 ++++++++++++++++++++++++++++++++++++++++-----------------
 1 file changed, 45 insertions(+), 18 deletions(-)

diff --git a/piem.el b/piem.el
index 5fc7f04..aa0ec0c 100644
--- a/piem.el
+++ b/piem.el
@@ -145,6 +145,16 @@ (defcustom piem-maildir-directory nil
 If non-nil, this must be an existing Maildir directory."
   :type 'string)
 
+(defcustom piem-mail-injection-skipif-predicate nil
+  "Predicate to decide whether to skip injecting a message.
+
+The function is called with the message ID (no surrounding
+brackets) within a buffer that is narrowed to the message.  The
+function does not need to worry about saving point.  A non-nil
+return value signals that `piem-inject-thread-into-maildir'
+should skip the message."
+  :type 'function)
+
 (defcustom piem-after-mail-injection-functions nil
   "Functions called after writing messages to `piem-maildir-directory'.
 Functions should accept one argument, the message ID given to
@@ -406,7 +416,8 @@ (defun piem-download-and-decompress (url)
     (url-retrieve url #'piem--decompress-callback)))
 
 (defun piem--write-mbox-to-maildir ()
-  (let ((n-messages 0))
+  (let ((n-added 0)
+        (n-skipped 0))
     (while (and (not (eobp))
                 (search-forward "From mboxrd@z" nil t))
       (let* ((beg (line-beginning-position 2))
@@ -417,21 +428,31 @@ (defun piem--write-mbox-to-maildir ()
              (basename (piem-maildir-make-uniq-maildir-id))
              (tmpfile (concat piem-maildir-directory "/tmp/" basename)))
         (goto-char beg)
-        ;; TODO: Consider supporting a caller-specified predicate that
-        ;; could be used, for example, to skip messages already in
-        ;; their local mail.
-        (let ((case-fold-search nil))
-          (while (re-search-forward
-                  (rx line-start ">" (group (zero-or-more ">") "From "))
-                  end t)
-            (replace-match "\\1" t)))
-        (write-region beg end tmpfile nil nil nil 'excl)
-        (piem-maildir-move-tmp-to-new piem-maildir-directory
-                                      basename)
-        (delete-file tmpfile)
-        (cl-incf n-messages)
+        (if (and (functionp piem-mail-injection-skipif-predicate)
+                 (save-excursion
+                   (save-restriction
+                     (narrow-to-region beg end)
+                     (funcall piem-mail-injection-skipif-predicate
+                              (replace-regexp-in-string
+                               "\\`<\\(.*\\)>\\'" "\\1"
+                               (or (save-restriction
+                                     (save-excursion
+                                       (message-narrow-to-head-1)
+                                       (message-fetch-field "message-id" t)))
+                                   (error "Message lacks message-ID")))))))
+            (cl-incf n-skipped)
+          (let ((case-fold-search nil))
+            (while (re-search-forward
+                    (rx line-start ">" (group (zero-or-more ">") "From "))
+                    end t)
+              (replace-match "\\1" t)))
+          (write-region beg end tmpfile nil nil nil 'excl)
+          (piem-maildir-move-tmp-to-new piem-maildir-directory
+                                        basename)
+          (delete-file tmpfile)
+          (cl-incf n-added))
         (goto-char end)))
-    n-messages))
+    (cons n-added n-skipped)))
 
 (defun piem--inject-thread-callback (status mid message-only)
   (let ((buffer (current-buffer)))
@@ -442,9 +463,15 @@ (defun piem--inject-thread-callback (status mid message-only)
             (piem--url-remove-header)
             (unless message-only
               (piem--url-decompress))
-            (let ((message-count (piem--write-mbox-to-maildir)))
-              (message "%d message(s) for %s moved to %s"
-                       message-count mid
+            (pcase-let ((`(,added-count . ,skipped-count)
+                         (piem--write-mbox-to-maildir)))
+              (message "Added %d message%s%s for %s to %s"
+                       added-count
+                       (if (= added-count 1) "" "s")
+                       (if (> skipped-count 0)
+                           (format " (skipping %d)" skipped-count)
+                         "")
+                       mid
                        (abbreviate-file-name piem-maildir-directory)))
             (run-hook-with-args 'piem-after-mail-injection-functions mid)))
       (and (buffer-live-p buffer)
-- 
2.28.0


^ permalink raw reply	[flat|nested] 4+ messages in thread

* [PATCH 3/3] notmuch: Extract "known message?" logic to function
  2020-08-22 19:01 [PATCH 0/3] inject: Add support for skipping messages Kyle Meyer
  2020-08-22 19:01 ` [PATCH 1/3] inject: Abbreviate piem-maildir-directory in message Kyle Meyer
  2020-08-22 19:01 ` [PATCH 2/3] inject: Support skipping messages Kyle Meyer
@ 2020-08-22 19:01 ` Kyle Meyer
  2 siblings, 0 replies; 4+ messages in thread
From: Kyle Meyer @ 2020-08-22 19:01 UTC (permalink / raw)
  To: piem

Notmuch users can set piem-mail-injection-skipif-predicate to the new
function.
---
 piem-notmuch.el | 32 +++++++++++++++++++-------------
 piem.el         |  5 ++++-
 2 files changed, 23 insertions(+), 14 deletions(-)

diff --git a/piem-notmuch.el b/piem-notmuch.el
index 29f4d27..5bf5b09 100644
--- a/piem-notmuch.el
+++ b/piem-notmuch.el
@@ -47,21 +47,27 @@ (defun piem-notmuch-get-mid ()
   "Return the message ID of a `notmuch-show-mode' buffer."
   (notmuch-show-get-message-id 'bare))
 
+(defun piem-notmuch-known-mid-p (mid)
+  "Return non-nil if MID is known to Notmuch.
+The message ID should not include Notmuch's \"id:\" prefix or
+have surrounding brackets."
+  (let ((query (concat "id:" mid)))
+    (equal query
+           (string-trim-right
+            (with-output-to-string
+              (with-current-buffer standard-output
+                (call-process notmuch-command
+                              nil '(t nil) nil
+                              "search" "--output=messages" query)))))))
+
 (defun piem-notmuch-mid-to-thread (mid)
   "Return a function that inserts an mbox for MID's thread."
-  (let ((query (concat "id:" mid)))
-    (when (equal query
-                 (string-trim-right
-                  (with-output-to-string
-                    (with-current-buffer standard-output
-                      (call-process notmuch-command
-                                    nil '(t nil) nil
-                                    "search" "--output=messages" query)))))
-      (lambda ()
-        (call-process notmuch-command
-                      nil '(t nil) nil
-                      "show" "--format=mbox" "--entire-thread=true"
-                      query)))))
+  (when (piem-notmuch-known-mid-p mid)
+    (lambda ()
+      (call-process notmuch-command
+                    nil '(t nil) nil
+                    "show" "--format=mbox" "--entire-thread=true"
+                    (concat "id:" mid)))))
 
 (defun piem-notmuch-am-ready-mbox ()
   "Return a function that inserts an am-ready mbox.
diff --git a/piem.el b/piem.el
index aa0ec0c..5bc86d5 100644
--- a/piem.el
+++ b/piem.el
@@ -152,7 +152,10 @@ (defcustom piem-mail-injection-skipif-predicate nil
 brackets) within a buffer that is narrowed to the message.  The
 function does not need to worry about saving point.  A non-nil
 return value signals that `piem-inject-thread-into-maildir'
-should skip the message."
+should skip the message.
+
+Notmuch users can use `piem-notmuch-known-mid-p' as the predicate
+to skip messages that are already in the Notmuch database."
   :type 'function)
 
 (defcustom piem-after-mail-injection-functions nil
-- 
2.28.0


^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2020-08-22 19:02 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-08-22 19:01 [PATCH 0/3] inject: Add support for skipping messages Kyle Meyer
2020-08-22 19:01 ` [PATCH 1/3] inject: Abbreviate piem-maildir-directory in message Kyle Meyer
2020-08-22 19:01 ` [PATCH 2/3] inject: Support skipping messages Kyle Meyer
2020-08-22 19:01 ` [PATCH 3/3] notmuch: Extract "known message?" logic to function Kyle Meyer

discussion and development of piem

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://inbox.kyleam.com/piem/0 piem/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 piem piem/ https://inbox.kyleam.com/piem \
		piem@inbox.kyleam.com
	public-inbox-index piem

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://news.yhetil.org/yhetil.emacs.piem


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git