From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms12 with LMTPS id GP7UFK9rQV/sPQAAsNZ9tg (envelope-from ) for ; Sat, 22 Aug 2020 19:02:07 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id cJ+tEK9rQV+dLQAA1q6Kng (envelope-from ) for ; Sat, 22 Aug 2020 19:02:07 +0000 Received: from pb-smtp21.pobox.com (pb-smtp21.pobox.com [173.228.157.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id E13A39402C2 for ; Sat, 22 Aug 2020 19:02:06 +0000 (UTC) Received: from pb-smtp21.pobox.com (unknown [127.0.0.1]) by pb-smtp21.pobox.com (Postfix) with ESMTP id 92BF6EA00B; Sat, 22 Aug 2020 15:02:05 -0400 (EDT) (envelope-from kyle@kyleam.com) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=pobox.com; h=from:to :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; s=sasl; bh=JtkgYMsI+qXsJqI5GEGm7Mq4w NY=; b=azWog6m9XrGsOgmN6m4Fv13kwqXQ1CTGUyhsKLv6Ip8lDj1jJpJJrR3lG kuowDe/xoKCTfvnW5sTGq3qAKK/lfZtglajgFxURP5kyz7nTsnbZEITU5lgSsUzN E5VHN8fdSxnNX8cDQHXdpKZ4GuYMGrXIMBUVBlB4yIh1Cg2otc= Received: from pb-smtp21.sea.icgroup.com (unknown [127.0.0.1]) by pb-smtp21.pobox.com (Postfix) with ESMTP id 8CA22EA00A; Sat, 22 Aug 2020 15:02:05 -0400 (EDT) (envelope-from kyle@kyleam.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=kyleam.com; h=from:to:subject:date:message-id:in-reply-to:references:mime-version:content-transfer-encoding; s=mesmtp; bh=+RXkYLOaMLS3TE0CqrGu8pfa63Ryo+iNM482SMUPNKo=; b=UvVWZNzd2ZhcxCjNpDj0c0ILdM9gfOJCYiP7H5leaVwoILZJZtUtPfiyf6xaRmYI6kRMbfM79YrqDSR5zW76tgosL1tm0TW66j07oDpFM/7/T4sZ4DmVqAcHEB6rWx2hgoXgo1KIgEalWijNCh4ajoYbRV7tiBcFeYdwQhaxsNQ= Received: from localhost (unknown [45.33.91.115]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by pb-smtp21.pobox.com (Postfix) with ESMTPSA id 03235EA009; Sat, 22 Aug 2020 15:02:03 -0400 (EDT) (envelope-from kyle@kyleam.com) From: Kyle Meyer To: piem@inbox.kyleam.com Subject: [PATCH 2/3] inject: Support skipping messages Date: Sat, 22 Aug 2020 15:01:29 -0400 Message-Id: <20200822190130.20397-3-kyle@kyleam.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200822190130.20397-1-kyle@kyleam.com> References: <20200822190130.20397-1-kyle@kyleam.com> MIME-Version: 1.0 X-Pobox-Relay-ID: F6D904F6-E4A9-11EA-ABD4-843F439F7C89-24757444!pb-smtp21.pobox.com Content-Transfer-Encoding: quoted-printable X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=pobox.com header.s=sasl header.b=azWog6m9; dkim=fail (body hash did not verify) header.d=kyleam.com header.s=mesmtp header.b=UvVWZNzd; dmarc=none; spf=pass (aspmx1.migadu.com: domain of kyle@kyleam.com designates 173.228.157.53 as permitted sender) smtp.mailfrom=kyle@kyleam.com X-Spam-Score: 2.50 X-TUID: gvd1bedYfeE3 piem-inject-thread-into-maildir allows pulling a thread from a public-inbox URL, but repeating the process to get new messages in the thread leads to duplicates in the local store. Add a predicate that the caller can configure to avoid this. Note that the predicate has the entire message available to make its decision, but it's called with the message ID for convenience because that's likely the only information needed to determine if the message is already in the local store. --- piem.el | 63 ++++++++++++++++++++++++++++++++++++++++----------------- 1 file changed, 45 insertions(+), 18 deletions(-) diff --git a/piem.el b/piem.el index 5fc7f04..aa0ec0c 100644 --- a/piem.el +++ b/piem.el @@ -145,6 +145,16 @@ (defcustom piem-maildir-directory nil If non-nil, this must be an existing Maildir directory." :type 'string) =20 +(defcustom piem-mail-injection-skipif-predicate nil + "Predicate to decide whether to skip injecting a message. + +The function is called with the message ID (no surrounding +brackets) within a buffer that is narrowed to the message. The +function does not need to worry about saving point. A non-nil +return value signals that `piem-inject-thread-into-maildir' +should skip the message." + :type 'function) + (defcustom piem-after-mail-injection-functions nil "Functions called after writing messages to `piem-maildir-directory'. Functions should accept one argument, the message ID given to @@ -406,7 +416,8 @@ (defun piem-download-and-decompress (url) (url-retrieve url #'piem--decompress-callback))) =20 (defun piem--write-mbox-to-maildir () - (let ((n-messages 0)) + (let ((n-added 0) + (n-skipped 0)) (while (and (not (eobp)) (search-forward "From mboxrd@z" nil t)) (let* ((beg (line-beginning-position 2)) @@ -417,21 +428,31 @@ (defun piem--write-mbox-to-maildir () (basename (piem-maildir-make-uniq-maildir-id)) (tmpfile (concat piem-maildir-directory "/tmp/" basename))) (goto-char beg) - ;; TODO: Consider supporting a caller-specified predicate that - ;; could be used, for example, to skip messages already in - ;; their local mail. - (let ((case-fold-search nil)) - (while (re-search-forward - (rx line-start ">" (group (zero-or-more ">") "From ")) - end t) - (replace-match "\\1" t))) - (write-region beg end tmpfile nil nil nil 'excl) - (piem-maildir-move-tmp-to-new piem-maildir-directory - basename) - (delete-file tmpfile) - (cl-incf n-messages) + (if (and (functionp piem-mail-injection-skipif-predicate) + (save-excursion + (save-restriction + (narrow-to-region beg end) + (funcall piem-mail-injection-skipif-predicate + (replace-regexp-in-string + "\\`<\\(.*\\)>\\'" "\\1" + (or (save-restriction + (save-excursion + (message-narrow-to-head-1) + (message-fetch-field "message-id"= t))) + (error "Message lacks message-ID"))))= ))) + (cl-incf n-skipped) + (let ((case-fold-search nil)) + (while (re-search-forward + (rx line-start ">" (group (zero-or-more ">") "From "= )) + end t) + (replace-match "\\1" t))) + (write-region beg end tmpfile nil nil nil 'excl) + (piem-maildir-move-tmp-to-new piem-maildir-directory + basename) + (delete-file tmpfile) + (cl-incf n-added)) (goto-char end))) - n-messages)) + (cons n-added n-skipped))) =20 (defun piem--inject-thread-callback (status mid message-only) (let ((buffer (current-buffer))) @@ -442,9 +463,15 @@ (defun piem--inject-thread-callback (status mid mess= age-only) (piem--url-remove-header) (unless message-only (piem--url-decompress)) - (let ((message-count (piem--write-mbox-to-maildir))) - (message "%d message(s) for %s moved to %s" - message-count mid + (pcase-let ((`(,added-count . ,skipped-count) + (piem--write-mbox-to-maildir))) + (message "Added %d message%s%s for %s to %s" + added-count + (if (=3D added-count 1) "" "s") + (if (> skipped-count 0) + (format " (skipping %d)" skipped-count) + "") + mid (abbreviate-file-name piem-maildir-directory))) (run-hook-with-args 'piem-after-mail-injection-functions mid= ))) (and (buffer-live-p buffer) --=20 2.28.0