discussion and development of piem
 help / color / mirror / Atom feed
* [PATCH 0/5] b4: Try to download thread from piem-inboxes URL
@ 2020-08-17  1:33 Kyle Meyer
  2020-08-17  1:33 ` [PATCH 1/5] Use url-http-end-of-headers move to payload Kyle Meyer
                   ` (3 more replies)
  0 siblings, 4 replies; 5+ messages in thread
From: Kyle Meyer @ 2020-08-17  1:33 UTC (permalink / raw)
  To: piem

If a thread can be obtained from the URL returned by piem-inbox-url,
get it from there so that callers don't need to set b4.midmask in
their git configuration.

  [1/5] Use url-http-end-of-headers move to payload
  [2/5] Move "has gunzip?" check to helper
  [3/5] Extract logic from inject-thread-callback
  [4/5] b4: Use a clearer variable name
  [5/5] b4: Try to download thread from piem-inboxes URL

 piem-b4.el | 25 ++++++++++++++++++-------
 piem.el    | 48 +++++++++++++++++++++++++++++++++++++-----------
 2 files changed, 55 insertions(+), 18 deletions(-)


base-commit: ff3b7724a75427c8d73a9b80f9ee5057250479cd
--
2.28.0

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH 1/5] Use url-http-end-of-headers move to payload
  2020-08-17  1:33 [PATCH 0/5] b4: Try to download thread from piem-inboxes URL Kyle Meyer
@ 2020-08-17  1:33 ` Kyle Meyer
  2020-08-17  1:33 ` [PATCH 3/5] Extract logic from inject-thread-callback Kyle Meyer
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 5+ messages in thread
From: Kyle Meyer @ 2020-08-17  1:33 UTC (permalink / raw)
  To: piem

---
 piem.el | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/piem.el b/piem.el
index 6a3c2b3..bcbec78 100644
--- a/piem.el
+++ b/piem.el
@@ -33,6 +33,8 @@ (require 'rfc2047)
 (require 'subr-x)
 (require 'url)
 
+(defvar url-http-end-of-headers)
+
 \f
 ;;;; Options
 
@@ -397,7 +399,7 @@ (defun piem--inject-thread-callback (status mid message-only)
         (let ((error-status (plist-get status :error)))
           (if error-status
               (signal (car error-status) (cdr error-status))
-            (search-forward "\n\n")
+            (goto-char (1+ url-http-end-of-headers))
             (delete-region (point-min) (point))
             (unless message-only
               (unless (= 0 (call-process-region nil nil "gunzip" nil t))
-- 
2.28.0


^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH 3/5] Extract logic from inject-thread-callback
  2020-08-17  1:33 [PATCH 0/5] b4: Try to download thread from piem-inboxes URL Kyle Meyer
  2020-08-17  1:33 ` [PATCH 1/5] Use url-http-end-of-headers move to payload Kyle Meyer
@ 2020-08-17  1:33 ` Kyle Meyer
  2020-08-17  1:33 ` [PATCH 4/5] b4: Use a clearer variable name Kyle Meyer
  2020-08-17  1:33 ` [PATCH 5/5] b4: Try to download thread from piem-inboxes URL Kyle Meyer
  3 siblings, 0 replies; 5+ messages in thread
From: Kyle Meyer @ 2020-08-17  1:33 UTC (permalink / raw)
  To: piem

An upcoming commit will use this same logic in another callback.
---
 piem.el | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/piem.el b/piem.el
index 4b35938..94fa1ad 100644
--- a/piem.el
+++ b/piem.el
@@ -372,6 +372,16 @@ (defun piem-check-gunzip ()
     (setq piem--has-gunzip (executable-find "gunzip")))
   piem--has-gunzip)
 
+(defun piem--url-remove-header ()
+  (goto-char (1+ url-http-end-of-headers))
+  (delete-region (point-min) (point)))
+
+(defun piem--url-decompress ()
+  (unless (= 0 (call-process-region nil nil "gunzip" nil t))
+    (error "Decompressing t.mbox.gz failed"))
+  (delete-region (point) (point-max))
+  (goto-char (point-min)))
+
 (defun piem--write-mbox-to-maildir ()
   (let ((n-messages 0))
     (while (and (not (eobp))
@@ -406,13 +416,9 @@ (defun piem--inject-thread-callback (status mid message-only)
         (let ((error-status (plist-get status :error)))
           (if error-status
               (signal (car error-status) (cdr error-status))
-            (goto-char (1+ url-http-end-of-headers))
-            (delete-region (point-min) (point))
+            (piem--url-remove-header)
             (unless message-only
-              (unless (= 0 (call-process-region nil nil "gunzip" nil t))
-                (error "Decompressing t.mbox.gz failed"))
-              (delete-region (point) (point-max)))
-            (goto-char (point-min))
+              (piem--url-decompress))
             (let ((message-count (piem--write-mbox-to-maildir)))
               (message "%d message(s) for %s moved to %s"
                        message-count mid piem-maildir-directory))
-- 
2.28.0


^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH 4/5] b4: Use a clearer variable name
  2020-08-17  1:33 [PATCH 0/5] b4: Try to download thread from piem-inboxes URL Kyle Meyer
  2020-08-17  1:33 ` [PATCH 1/5] Use url-http-end-of-headers move to payload Kyle Meyer
  2020-08-17  1:33 ` [PATCH 3/5] Extract logic from inject-thread-callback Kyle Meyer
@ 2020-08-17  1:33 ` Kyle Meyer
  2020-08-17  1:33 ` [PATCH 5/5] b4: Try to download thread from piem-inboxes URL Kyle Meyer
  3 siblings, 0 replies; 5+ messages in thread
From: Kyle Meyer @ 2020-08-17  1:33 UTC (permalink / raw)
  To: piem

---
 piem-b4.el | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/piem-b4.el b/piem-b4.el
index 3381b01..537abed 100644
--- a/piem-b4.el
+++ b/piem-b4.el
@@ -56,17 +56,17 @@ (defun piem-b4--get-am-files (mid coderepo args)
                   (make-temp-file "piem-b4-" t)))
          (root (concat outdir "m"))
          (mbox-thread (concat root "-piem"))
-         (custom-p nil))
+         (local-mbox-p nil))
     (when-let ((fn (run-hook-with-args-until-success
                     'piem-mid-to-thread-functions mid)))
       (with-temp-file mbox-thread
         (funcall fn)
         (unless (= (point-max) 1)
-          (setq custom-p t))))
+          (setq local-mbox-p t))))
     ;; Move to the coderepo so that we pick up any b4 configuration
     ;; from there.
     (apply #'piem-process-call coderepo piem-b4-b4-executable "am"
-           (and custom-p
+           (and local-mbox-p
                 (concat "--use-local-mbox=" mbox-thread))
            (concat "--outdir=" outdir)
            (concat "--mbox-name=m")
-- 
2.28.0


^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH 5/5] b4: Try to download thread from piem-inboxes URL
  2020-08-17  1:33 [PATCH 0/5] b4: Try to download thread from piem-inboxes URL Kyle Meyer
                   ` (2 preceding siblings ...)
  2020-08-17  1:33 ` [PATCH 4/5] b4: Use a clearer variable name Kyle Meyer
@ 2020-08-17  1:33 ` Kyle Meyer
  3 siblings, 0 replies; 5+ messages in thread
From: Kyle Meyer @ 2020-08-17  1:33 UTC (permalink / raw)
  To: piem

If a call to piem-b4-am-from-mid fails to generate the thread for a
message ID via piem-mid-to-thread-functions, b4 is called without a
local mbox.  In this case, b4 tries to download the thread from the
URL specified by b4.midmask in the caller's Git configuration.

That works, but it's inconvenient because the user needs to configure
the URL in two places.  If the current buffer is associated with an
inbox in piem-inboxes, try to download the thread from its :url before
falling back to b4's midmask.
---
 piem-b4.el | 19 +++++++++++++++----
 piem.el    | 15 +++++++++++++++
 2 files changed, 30 insertions(+), 4 deletions(-)

diff --git a/piem-b4.el b/piem-b4.el
index 537abed..39d32be 100644
--- a/piem-b4.el
+++ b/piem-b4.el
@@ -47,10 +47,6 @@ (defcustom piem-b4-b4-executable "b4"
 \f
 ;;;; Internals
 
-;; In many cases, we don't really need b4 to download the mbox for us,
-;; as we already have our own mbox to URL mapping.  Perhaps we should
-;; default to using that, but it should still be an option to use b4
-;; so that we honor its customization/URL resolution.
 (defun piem-b4--get-am-files (mid coderepo args)
   (let* ((outdir (file-name-as-directory
                   (make-temp-file "piem-b4-" t)))
@@ -63,6 +59,21 @@ (defun piem-b4--get-am-files (mid coderepo args)
         (funcall fn)
         (unless (= (point-max) 1)
           (setq local-mbox-p t))))
+    ;; `piem-mid-to-thread-functions' didn't generate an mbox.  Next
+    ;; try to download it from a URL at `piem-inboxes'.  Finally, fall
+    ;; back to b4's configuration.
+    (unless local-mbox-p
+      (when-let ((url (piem-inbox-url))
+                 (mid (piem-mid))
+                 (buffer (condition-case nil
+                             (piem-download-and-decompress
+                              (concat url mid "/t.mbox.gz"))
+                           (user-error nil))))
+        (when (buffer-live-p buffer)
+          (with-current-buffer buffer
+            (write-region nil nil mbox-thread))
+          (kill-buffer buffer)
+          (setq local-mbox-p t))))
     ;; Move to the coderepo so that we pick up any b4 configuration
     ;; from there.
     (apply #'piem-process-call coderepo piem-b4-b4-executable "am"
diff --git a/piem.el b/piem.el
index 94fa1ad..0fd1edc 100644
--- a/piem.el
+++ b/piem.el
@@ -382,6 +382,21 @@ (defun piem--url-decompress ()
   (delete-region (point) (point-max))
   (goto-char (point-min)))
 
+(defun piem--decompress-callback (status)
+  (if (plist-get status :error)
+      (kill-buffer (current-buffer))
+    (piem--url-remove-header)
+    (piem--url-decompress)))
+
+(defun piem-download-and-decompress (url)
+  "Retrieve gzipped content at URL and decompress it.
+A buffer with the decompressed content is returned.  A live
+buffer indicates that the request did not result in an error."
+  (unless (piem-check-gunzip)
+    (user-error "gunzip executable not found"))
+  (let ((url-asynchronous nil))
+    (url-retrieve url #'piem--decompress-callback)))
+
 (defun piem--write-mbox-to-maildir ()
   (let ((n-messages 0))
     (while (and (not (eobp))
-- 
2.28.0


^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2020-08-17  1:33 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-08-17  1:33 [PATCH 0/5] b4: Try to download thread from piem-inboxes URL Kyle Meyer
2020-08-17  1:33 ` [PATCH 1/5] Use url-http-end-of-headers move to payload Kyle Meyer
2020-08-17  1:33 ` [PATCH 3/5] Extract logic from inject-thread-callback Kyle Meyer
2020-08-17  1:33 ` [PATCH 4/5] b4: Use a clearer variable name Kyle Meyer
2020-08-17  1:33 ` [PATCH 5/5] b4: Try to download thread from piem-inboxes URL Kyle Meyer

discussion and development of piem

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://inbox.kyleam.com/piem/0 piem/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 piem piem/ https://inbox.kyleam.com/piem \
		piem@inbox.kyleam.com
	public-inbox-index piem

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://news.yhetil.org/yhetil.emacs.piem


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git