discussion and development of piem
 help / color / mirror / code / Atom feed
* [PATCH 0/5] b4: Try to download thread from piem-inboxes URL
@ 2020-08-17  1:33 Kyle Meyer
  2020-08-17  1:33 ` [PATCH 1/5] Use url-http-end-of-headers move to payload Kyle Meyer
                   ` (4 more replies)
  0 siblings, 5 replies; 6+ messages in thread
From: Kyle Meyer @ 2020-08-17  1:33 UTC (permalink / raw)
  To: piem

If a thread can be obtained from the URL returned by piem-inbox-url,
get it from there so that callers don't need to set b4.midmask in
their git configuration.

  [1/5] Use url-http-end-of-headers move to payload
  [2/5] Move "has gunzip?" check to helper
  [3/5] Extract logic from inject-thread-callback
  [4/5] b4: Use a clearer variable name
  [5/5] b4: Try to download thread from piem-inboxes URL

 piem-b4.el | 25 ++++++++++++++++++-------
 piem.el    | 48 +++++++++++++++++++++++++++++++++++++-----------
 2 files changed, 55 insertions(+), 18 deletions(-)


base-commit: ff3b7724a75427c8d73a9b80f9ee5057250479cd
--
2.28.0

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [PATCH 1/5] Use url-http-end-of-headers move to payload
  2020-08-17  1:33 [PATCH 0/5] b4: Try to download thread from piem-inboxes URL Kyle Meyer
@ 2020-08-17  1:33 ` Kyle Meyer
  2020-08-17  1:33 ` [PATCH 2/5] Move "has gunzip?" check to helper Kyle Meyer
                   ` (3 subsequent siblings)
  4 siblings, 0 replies; 6+ messages in thread
From: Kyle Meyer @ 2020-08-17  1:33 UTC (permalink / raw)
  To: piem

---
 piem.el | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/piem.el b/piem.el
index 6a3c2b3..bcbec78 100644
--- a/piem.el
+++ b/piem.el
@@ -33,6 +33,8 @@ (require 'rfc2047)
 (require 'subr-x)
 (require 'url)
 
+(defvar url-http-end-of-headers)
+
 \f
 ;;;; Options
 
@@ -397,7 +399,7 @@ (defun piem--inject-thread-callback (status mid message-only)
         (let ((error-status (plist-get status :error)))
           (if error-status
               (signal (car error-status) (cdr error-status))
-            (search-forward "\n\n")
+            (goto-char (1+ url-http-end-of-headers))
             (delete-region (point-min) (point))
             (unless message-only
               (unless (= 0 (call-process-region nil nil "gunzip" nil t))
-- 
2.28.0


^ permalink raw reply related	[flat|nested] 6+ messages in thread

* [PATCH 2/5] Move "has gunzip?" check to helper
  2020-08-17  1:33 [PATCH 0/5] b4: Try to download thread from piem-inboxes URL Kyle Meyer
  2020-08-17  1:33 ` [PATCH 1/5] Use url-http-end-of-headers move to payload Kyle Meyer
@ 2020-08-17  1:33 ` Kyle Meyer
  2020-08-17  1:33 ` [PATCH 3/5] Extract logic from inject-thread-callback Kyle Meyer
                   ` (2 subsequent siblings)
  4 siblings, 0 replies; 6+ messages in thread
From: Kyle Meyer @ 2020-08-17  1:33 UTC (permalink / raw)
  To: piem

This will be needed in another spot.
---
 piem.el | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/piem.el b/piem.el
index bcbec78..4b35938 100644
--- a/piem.el
+++ b/piem.el
@@ -365,6 +365,13 @@ (defun piem-am-ready-mbox ()
 \f
 ;;;; Maildir injection
 
+(defvar piem--has-gunzip)
+(defun piem-check-gunzip ()
+  "Return non-nil if gunzip is available."
+  (unless (boundp 'piem--has-gunzip)
+    (setq piem--has-gunzip (executable-find "gunzip")))
+  piem--has-gunzip)
+
 (defun piem--write-mbox-to-maildir ()
   (let ((n-messages 0))
     (while (and (not (eobp))
@@ -413,8 +420,6 @@ (defun piem--inject-thread-callback (status mid message-only)
       (and (buffer-live-p buffer)
            (kill-buffer buffer)))))
 
-(defvar piem--has-gunzip)
-
 ;;;###autoload
 (defun piem-inject-thread-into-maildir (mid &optional message-only)
   "Inject thread containing MID into `piem-maildir-directory'.
@@ -428,15 +433,13 @@ (defun piem-inject-thread-into-maildir (mid &optional message-only)
    (list (or (piem-mid)
              (user-error "No message ID found for the current buffer"))
          current-prefix-arg))
-  (unless (or message-only (boundp 'piem--has-gunzip))
-    (setq piem--has-gunzip (executable-find "gunzip")))
   (cond
    ((not piem-maildir-directory)
     (user-error "`piem-maildir-directory' is not configured"))
    ((not (piem-maildir-dir-is-maildir-p piem-maildir-directory))
     (user-error
      "`piem-maildir-directory' does not look like a Maildir directory"))
-   ((not (or message-only piem--has-gunzip))
+   ((not (or message-only (piem-check-gunzip)))
     (user-error "gunzip executable not found")))
   (url-retrieve (concat (or (piem-inbox-url)
                             (user-error
-- 
2.28.0


^ permalink raw reply related	[flat|nested] 6+ messages in thread

* [PATCH 3/5] Extract logic from inject-thread-callback
  2020-08-17  1:33 [PATCH 0/5] b4: Try to download thread from piem-inboxes URL Kyle Meyer
  2020-08-17  1:33 ` [PATCH 1/5] Use url-http-end-of-headers move to payload Kyle Meyer
  2020-08-17  1:33 ` [PATCH 2/5] Move "has gunzip?" check to helper Kyle Meyer
@ 2020-08-17  1:33 ` Kyle Meyer
  2020-08-17  1:33 ` [PATCH 4/5] b4: Use a clearer variable name Kyle Meyer
  2020-08-17  1:33 ` [PATCH 5/5] b4: Try to download thread from piem-inboxes URL Kyle Meyer
  4 siblings, 0 replies; 6+ messages in thread
From: Kyle Meyer @ 2020-08-17  1:33 UTC (permalink / raw)
  To: piem

An upcoming commit will use this same logic in another callback.
---
 piem.el | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/piem.el b/piem.el
index 4b35938..94fa1ad 100644
--- a/piem.el
+++ b/piem.el
@@ -372,6 +372,16 @@ (defun piem-check-gunzip ()
     (setq piem--has-gunzip (executable-find "gunzip")))
   piem--has-gunzip)
 
+(defun piem--url-remove-header ()
+  (goto-char (1+ url-http-end-of-headers))
+  (delete-region (point-min) (point)))
+
+(defun piem--url-decompress ()
+  (unless (= 0 (call-process-region nil nil "gunzip" nil t))
+    (error "Decompressing t.mbox.gz failed"))
+  (delete-region (point) (point-max))
+  (goto-char (point-min)))
+
 (defun piem--write-mbox-to-maildir ()
   (let ((n-messages 0))
     (while (and (not (eobp))
@@ -406,13 +416,9 @@ (defun piem--inject-thread-callback (status mid message-only)
         (let ((error-status (plist-get status :error)))
           (if error-status
               (signal (car error-status) (cdr error-status))
-            (goto-char (1+ url-http-end-of-headers))
-            (delete-region (point-min) (point))
+            (piem--url-remove-header)
             (unless message-only
-              (unless (= 0 (call-process-region nil nil "gunzip" nil t))
-                (error "Decompressing t.mbox.gz failed"))
-              (delete-region (point) (point-max)))
-            (goto-char (point-min))
+              (piem--url-decompress))
             (let ((message-count (piem--write-mbox-to-maildir)))
               (message "%d message(s) for %s moved to %s"
                        message-count mid piem-maildir-directory))
-- 
2.28.0


^ permalink raw reply related	[flat|nested] 6+ messages in thread

* [PATCH 4/5] b4: Use a clearer variable name
  2020-08-17  1:33 [PATCH 0/5] b4: Try to download thread from piem-inboxes URL Kyle Meyer
                   ` (2 preceding siblings ...)
  2020-08-17  1:33 ` [PATCH 3/5] Extract logic from inject-thread-callback Kyle Meyer
@ 2020-08-17  1:33 ` Kyle Meyer
  2020-08-17  1:33 ` [PATCH 5/5] b4: Try to download thread from piem-inboxes URL Kyle Meyer
  4 siblings, 0 replies; 6+ messages in thread
From: Kyle Meyer @ 2020-08-17  1:33 UTC (permalink / raw)
  To: piem

---
 piem-b4.el | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/piem-b4.el b/piem-b4.el
index 3381b01..537abed 100644
--- a/piem-b4.el
+++ b/piem-b4.el
@@ -56,17 +56,17 @@ (defun piem-b4--get-am-files (mid coderepo args)
                   (make-temp-file "piem-b4-" t)))
          (root (concat outdir "m"))
          (mbox-thread (concat root "-piem"))
-         (custom-p nil))
+         (local-mbox-p nil))
     (when-let ((fn (run-hook-with-args-until-success
                     'piem-mid-to-thread-functions mid)))
       (with-temp-file mbox-thread
         (funcall fn)
         (unless (= (point-max) 1)
-          (setq custom-p t))))
+          (setq local-mbox-p t))))
     ;; Move to the coderepo so that we pick up any b4 configuration
     ;; from there.
     (apply #'piem-process-call coderepo piem-b4-b4-executable "am"
-           (and custom-p
+           (and local-mbox-p
                 (concat "--use-local-mbox=" mbox-thread))
            (concat "--outdir=" outdir)
            (concat "--mbox-name=m")
-- 
2.28.0


^ permalink raw reply related	[flat|nested] 6+ messages in thread

* [PATCH 5/5] b4: Try to download thread from piem-inboxes URL
  2020-08-17  1:33 [PATCH 0/5] b4: Try to download thread from piem-inboxes URL Kyle Meyer
                   ` (3 preceding siblings ...)
  2020-08-17  1:33 ` [PATCH 4/5] b4: Use a clearer variable name Kyle Meyer
@ 2020-08-17  1:33 ` Kyle Meyer
  4 siblings, 0 replies; 6+ messages in thread
From: Kyle Meyer @ 2020-08-17  1:33 UTC (permalink / raw)
  To: piem

If a call to piem-b4-am-from-mid fails to generate the thread for a
message ID via piem-mid-to-thread-functions, b4 is called without a
local mbox.  In this case, b4 tries to download the thread from the
URL specified by b4.midmask in the caller's Git configuration.

That works, but it's inconvenient because the user needs to configure
the URL in two places.  If the current buffer is associated with an
inbox in piem-inboxes, try to download the thread from its :url before
falling back to b4's midmask.
---
 piem-b4.el | 19 +++++++++++++++----
 piem.el    | 15 +++++++++++++++
 2 files changed, 30 insertions(+), 4 deletions(-)

diff --git a/piem-b4.el b/piem-b4.el
index 537abed..39d32be 100644
--- a/piem-b4.el
+++ b/piem-b4.el
@@ -47,10 +47,6 @@ (defcustom piem-b4-b4-executable "b4"
 \f
 ;;;; Internals
 
-;; In many cases, we don't really need b4 to download the mbox for us,
-;; as we already have our own mbox to URL mapping.  Perhaps we should
-;; default to using that, but it should still be an option to use b4
-;; so that we honor its customization/URL resolution.
 (defun piem-b4--get-am-files (mid coderepo args)
   (let* ((outdir (file-name-as-directory
                   (make-temp-file "piem-b4-" t)))
@@ -63,6 +59,21 @@ (defun piem-b4--get-am-files (mid coderepo args)
         (funcall fn)
         (unless (= (point-max) 1)
           (setq local-mbox-p t))))
+    ;; `piem-mid-to-thread-functions' didn't generate an mbox.  Next
+    ;; try to download it from a URL at `piem-inboxes'.  Finally, fall
+    ;; back to b4's configuration.
+    (unless local-mbox-p
+      (when-let ((url (piem-inbox-url))
+                 (mid (piem-mid))
+                 (buffer (condition-case nil
+                             (piem-download-and-decompress
+                              (concat url mid "/t.mbox.gz"))
+                           (user-error nil))))
+        (when (buffer-live-p buffer)
+          (with-current-buffer buffer
+            (write-region nil nil mbox-thread))
+          (kill-buffer buffer)
+          (setq local-mbox-p t))))
     ;; Move to the coderepo so that we pick up any b4 configuration
     ;; from there.
     (apply #'piem-process-call coderepo piem-b4-b4-executable "am"
diff --git a/piem.el b/piem.el
index 94fa1ad..0fd1edc 100644
--- a/piem.el
+++ b/piem.el
@@ -382,6 +382,21 @@ (defun piem--url-decompress ()
   (delete-region (point) (point-max))
   (goto-char (point-min)))
 
+(defun piem--decompress-callback (status)
+  (if (plist-get status :error)
+      (kill-buffer (current-buffer))
+    (piem--url-remove-header)
+    (piem--url-decompress)))
+
+(defun piem-download-and-decompress (url)
+  "Retrieve gzipped content at URL and decompress it.
+A buffer with the decompressed content is returned.  A live
+buffer indicates that the request did not result in an error."
+  (unless (piem-check-gunzip)
+    (user-error "gunzip executable not found"))
+  (let ((url-asynchronous nil))
+    (url-retrieve url #'piem--decompress-callback)))
+
 (defun piem--write-mbox-to-maildir ()
   (let ((n-messages 0))
     (while (and (not (eobp))
-- 
2.28.0


^ permalink raw reply related	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2020-08-17  1:33 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-08-17  1:33 [PATCH 0/5] b4: Try to download thread from piem-inboxes URL Kyle Meyer
2020-08-17  1:33 ` [PATCH 1/5] Use url-http-end-of-headers move to payload Kyle Meyer
2020-08-17  1:33 ` [PATCH 2/5] Move "has gunzip?" check to helper Kyle Meyer
2020-08-17  1:33 ` [PATCH 3/5] Extract logic from inject-thread-callback Kyle Meyer
2020-08-17  1:33 ` [PATCH 4/5] b4: Use a clearer variable name Kyle Meyer
2020-08-17  1:33 ` [PATCH 5/5] b4: Try to download thread from piem-inboxes URL Kyle Meyer

Code repositories for project(s) associated with this public inbox

	https://git.kyleam.com/piem/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).