discussion and development of piem
 help / color / mirror / code / Atom feed
blob e7b4b32e8fe6f0b9eabb318fc1f1089fdd1a5641 19101 bytes (raw)
name: piem.texi 	 # note: path name is non-authoritative(*)

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
 
\input texinfo @c -*-texinfo-*-

@set VERSION 0.2.0 (unreleased)

@setfilename piem.info
@documentencoding UTF-8
@documentlanguage en
@settitle Emacs tools and glue for working with public-inbox archives

@copying
Copyright @copyright{} 2020--2021 all contributors <piem@@inbox.kyleam.com>

Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.3 or
any later version published by the Free Software Foundation; with no
Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.  A
copy of the license is included in the section entitled ``GNU Free
Documentation License''.
@end copying

@dircategory Emacs
@direntry
* piem: (piem).  Emacs tools and glue for working with public-inbox archives
@end direntry

@finalout
@titlepage
@title piem reference manual
@subtitle for version @value{VERSION}
@author Kyle Meyer
@page
@vskip 0pt plus 1filll
@insertcopying
@end titlepage

@contents
@*

@ifnottex
@node Top
@top piem

This manual is for piem version @value{VERSION}.
@end ifnottex

\f
@menu
* Overview::
* Getting started::
* Applying patches::
* Injecting messages into a Maildir directory::
* Contributing::
* Related projects and tools::

Appendices
* GNU Free Documentation License::  The license for this documentation.

Indices
* Key Index::
* Variable Index::
* Lisp Function Index::
* Concept Index::
@end menu

\f
@node Overview
@chapter Overview

piem is a collection of Emacs libraries for working with public-inbox
archives.  As much of the hard work here is already done by other Emacs
libraries---things like mail clients, news readers, Git interfaces, and
even web browsers---piem is mostly about bridging some of these parts
for convenience.

@node public-inbox
@section public-inbox
@cindex public-inbox
@cindex lore

@url{https://public-inbox.org/README.html,public-inbox} is software for
archiving public mailing lists.  Archives can be exposed over HTTP.  As
examples, @url{https://public-inbox.org/meta} serves public-inbox's own
mailing list, and @url{https://lore.kernel.org/lists.html} hosts the
archives of many Linux development mailing lists.

@cindex pull methods
These web archives are good for searching, particularly if you don't
have all of the list's messages on your local machine, or for linking to
a message.  On the other hand, the web interface isn't convenient when
you want to follow new activity on a list.  To do that, you could of
course subscribe to the mailing list, but public-inbox offers a few
``pull methods'' that you can use instead:

@itemize
@item
an atom feed for the list as a whole or for specific searches
@item
read-only NNTP
@item
read-only IMAP (new in upcoming public-inbox v1.6.0)
@end itemize

Finally, archives are exposed as one or more Git repositories,
facilitating replication (see
@url{https://public-inbox.org/reproducibility.html}).  For example, you
can clone the mailing list archives of @samp{git.vger.kernel.org} with

@example
git clone --mirror https://lore.kernel.org/git/0 git/git/0.git
@end example

@noindent
After the initial clone, new messages can be retrieved with
@code{git fetch}.  Unsurprisingly @code{git log} is not a pleasant way
to read a mailing list; instead this method is useful for mirroring the
archive or bulk importing of the messages.  (See
@url{https://public-inbox.org/clients.html} for a list of some tools
designed to work with public-inbox archives.)

\f
@node Getting started
@chapter Getting started
@findex piem-dispatch

@code{piem-dispatch} transient
(see
@ifinfo
@ref{Top,,,transient}
@end ifinfo
@ifnotinfo
@url{https://magit.vc/manual/transient/}
@end ifnotinfo
)
provides an entry point to piem commands.
It's recommended to bind @code{piem-dispatch} to a key.  However, before
most of those commands do anything useful, you need to register inboxes
and activate at least one minor mode.

@node Registering inboxes
@section Registering inboxes
@cindex coderepo
@cindex inbox
@vindex piem-inboxes

A public-inbox archive, referred to as an @dfn{inbox}, is registered by
adding an entry to @code{piem-inboxes}.  Here's an example entry for the
Git project's mailing list:

@lisp
("git"
 :url "https://lore.kernel.org/git/"
 :address "git@@vger.kernel.org"
 :listid "git.vger.kernel.org"
 :coderepo "~/src/git/")
@end lisp

@noindent
The first element is a name for the inbox and will typically match the
name at the end of the @code{:url} value.  Specifying either
@code{:listid} or @code{:address} is important so that a message in a
buffer can be mapped to an inbox in @code{piem-inboxes}.

@code{:coderepo} points to a local Git repository that contains code
related to that archive (in the example above, a local clone of
@url{https://git.kernel.org/pub/scm/git/git.git/}).  This information is
required to apply patches from an archive to a local code repository
(@pxref{Applying patches}).

@node Enabling integration libraries
@section Enabling integration libraries
@findex piem-elfeed-mode
@findex piem-eww-mode
@findex piem-gnus-mode
@findex piem-notmuch-mode

With inboxes defined, the next step is to enable minor modes that teach
particular Emacs modes to link a buffer with a registered inbox.  piem
currently has libraries to support

@itemize
@item EWW
@item Elfeed
@item Gnus
@item Notmuch
@end itemize

For example, if you use notmuch.el to read your mail, you can add
support for applying patches from a Notmuch message buffer by enabling
@code{piem-notmuch-mode} (@pxref{Applying patches}):

@lisp
(piem-notmuch-mode 1)
@end lisp

Help adding support for other modes, especially other mail clients, is
welcome.

\f
@node Applying patches
@chapter Applying patches
@cindex am-ready mbox
@cindex applying patches
@cindex git-am

With @code{piem-inboxes} configured and appropriate integration
libraries enabled, a buffer that can be linked to an inbox can be mapped
to a code repository.  When reading a message in a
@code{notmuch-show-mode} buffer, for example, the list ID can be used to
identify the inbox and thus the associated local code repository.

There are two commands for applying patches:

@table @code

@item piem-am
@findex piem-am
This command tries to extract a patch from the current Notmuch or Gnus
message buffer and can handle an inline patch as well as one or more
patch attachments.

@item piem-b4-am
@findex piem-b4-am
This command relies on the b4 command-line tool to do more sophisticated
processing of the @emph{full thread} (e.g., pulling out the latest
reroll of a series) to generate an mbox that can be fed to @code{git
am}.  It is only compatible with inline patches.

@end table

@node Applying patches contained in a message
@section Applying patches contained in a message

@table @kbd
@findex piem-am
@item M-x piem-am @key{RET} @var{branch} @key{RET} @var{base}
Apply the patch or patches in the current buffer to the associated code
repository.  Before applying, checkout a new branch @var{branch}
starting at @var{base}.
@end table

@findex piem-name-branch-who-what-v
@vindex piem-default-branch-function
You'll be queried for the name of the new branch.  The default name
offered is generated by @code{piem-name-branch-who-what-v}, which uses
the @samp{From:} and @samp{Subject:} headers to construct branch names
like @samp{km/b4-short-subj__v3}.  To use a different function to
generate the completion default, configure
@code{piem-default-branch-function}.

Next you'll be queried for the base to use as the starting point for the
branch.  If the sender specified a base commit for the series, that will
be provided as the default completion candidate.  Entering an empty base
signals to use the current branch of the repository as the base.

@vindex piem-am-create-worktree
@vindex piem-am-read-worktree-function
Rather than applying the patches directly to the associated code
repository, you can create a dedicated worktree by setting
@code{piem-am-create-worktree} to a non-nil value.  Giving a prefix
argument to @code{piem-am} inverts the meaning of
@code{piem-am-create-worktree}; that is, by default a prefix argument is
useful if you generally prefer to work within the configured code
repository but would like to trigger the one-off creation of a worktree
for a particular call.

@cindex magit
@vindex piem-use-magit
When piem loads, it detects whether Magit is loaded and sets
@code{piem-use-magit} accordingly.  If that option is non-nil, piem uses
Magit for some operations, particularly those that are user-facing.
This includes jumping to the Magit status buffer for a code repository
after applying a patch.

@findex piem-am-ready-mbox
Note that the @code{piem-am} command works only for buffers from which
@code{piem-am-ready-mbox} can generate an am-ready mbox, which depends
on the enabled integration libraries.  Currently @code{piem-notmuch} and
@code{piem-gnus} implement the necessary functionality.

@node Using b4 to apply patches
@section Using b4 to apply patches
@cindex b4
@cindex lore

@url{https://git.kernel.org/pub/scm/utils/b4/b4.git,b4} is a
command-line tool for interacting with public-inbox archives.  While
useful for public-inbox archives in general, it is written for Linux
kernel development and focuses on the public-inbox archives hosted at
@url{https://lore.kernel.org}.

It's a fast moving target at the moment, but some of its current
capabilities include

@itemize
@item
downloading the mbox for a thread based on a given message ID
@item
extracting patches from a thread's mbox that can be fed to @code{git am}
@item
submitting and verifying cryptographic attestation for patches
@item
fetching a pull request found in a message ID
@item
generating a thanks email for patches
@end itemize

@noindent
The second item is the focus for piem, though at least some degree of
support for all of the above features will likely be added.
The entry point to applying patches with b4 is the @code{piem-b4-am}
transient.  (See
@ifinfo
@ref{Top,,,transient}
@end ifinfo
@ifnotinfo
@url{https://magit.vc/manual/transient/}
@end ifnotinfo
for more information on using Transient.)

@findex piem-b4-am
@code{piem-b4-am} offers the following actions:

@table @kbd

@item a
@itemx M-x piem-b4-am-from-mid
@findex piem-b4-am-from-mid
@findex piem-mid
@vindex piem-am-create-worktree
@vindex piem-am-read-worktree-function
Generate or download a thread's mbox for the current buffer's message
ID, process it into an am-ready mbox with b4, and then feed it to
@code{git am} called within an associated Git repository.  If a message
ID of the current buffer is not known (i.e. @code{piem-mid} returns
nil), one is read from the caller.  The caller is also queried for the
branch name and base, as described for @code{piem-am} (@pxref{Applying
patches contained in a message}).  And, as with @code{piem-am}, a
worktree can be created by configuring @code{piem-am-create-worktree} to
a non-nil value or by giving a prefix argument.

@findex piem-mid-to-thread-functions
To generate the input thread, first any functions in
@code{piem-mid-to-thread-functions} are tried.  This allows for a thread
to be retrieved from a local store (e.g., the Notmuch database).  If
that fails, the thread is downloaded from the public-inbox URL
associated with the current buffer.  Finally, if an inbox's entry in
@code{piem-inboxes} doesn't specify a URL, @code{b4 am} is called
without a local mbox, letting it download the thread according to its
own configuration.

@item i
@itemx M-x piem-b4-am-ready-from-mid
@findex piem-b4-am-ready-from-mid
Call @code{b4 am} with a given message ID.  This differs from
@code{piem-b4-am-from-mid} in that it is a direct wrapper around a
command-line call to @code{b4 am}.  The caller is always queried for the
message ID, and the final product is an am-ready mbox.  @code{b4} is
responsible for downloading the thread, so the caller must point b4's
configuration option @code{b4.midmask} to the appropriate public-inbox
URL.

@item b
@itemx M-x piem-b4-am-ready-from-mbox
@findex piem-b4-am-ready-from-mbox
Like @code{piem-b4-am-ready-from-mid}, but process a local mbox rather
than identifying the thread based on the specified message ID.

@end table

@node Applying patches without a public-inbox archive
@section Applying patches without a public-inbox archive

Much of the functionality described in the previous sections can work
even if messages aren't available in a public-inbox archive.
@code{piem-am} and @code{piem-b4-am-from-mid} try to generate the
am-ready mbox from a local source (e.g., via Notmuch or Gnus) before
falling back to downloading the thread from a public-inbox archive.

@cindex mailscripts
Also, for those not working with public-inbox archives, it's worth
checking out @url{https://git.spwhitton.name/mailscripts/,mailscripts},
a nice set of Debian-focused tools by Sean Whitton that provides, among
other things, functionality for applying patch series, including
b4-inspired patch extraction.

\f
@node Injecting messages into a Maildir directory
@chapter Injecting messages into a Maildir directory
@cindex Maildir

public-inbox allows you to follow lists through several mechanisms
(@pxref{public-inbox}).  You may prefer different methods for different
projects depending on things like how actively you are following the
development and how high traffic the list is.  For a project you
maintain, perhaps you want to receive every message as regular mail.
For a project you actively follow and occasionally contribute to, you
may prefer to not clutter your local mail store and instead follow via
read-only NNTP or IMAP in Gnus (which may or may not be your MUA).  And
for a project you're new to or are digging into for a particular reason,
HTTP via EWW may be all you need.

@findex piem-inject-thread-into-maildir
@vindex piem-maildir-directory
Depending on your mail setup, a problem with this approach is that it
can be inconvenient to start participating in a thread that you aren't
reading in your regular MUA (e.g., if you use notmuch.el to read your
regular mail but are following a project via NNTP in Gnus).  In this
case, you can use the command @code{piem-inject-thread-into-maildir} to
move the thread's messages into a local Maildir directory
(@code{piem-maildir-directory}).  By default the command downloads the
entire thread for the message ID associated with the current buffer.  A
prefix argument restricts the download to only the message.

@vindex piem-after-mail-injection-functions
After the messages are injected, each function in
@code{piem-after-mail-injection-functions} is called with the message ID
that was used to identify the thread.  This can be used to pop to the
message in your mail client.  For example, Notmuch users may want
something like this:

@lisp
(defun my/notmuch-new-and-show (mid)
  (message "Running notmuch new")
  (call-process notmuch-command nil nil nil "new")
  (notmuch-show (concat "id:"  mid)))

(add-hook 'piem-after-mail-injection-functions
          #'my/notmuch-new-and-show)
@end lisp

@vindex piem-mail-injection-skipif-predicate
@findex piem-notmuch-known-mid-p
To prevent duplicate messages from being written on subsequent calls to
@code{piem-inject-thread-into-maildir}, you can set
@code{piem-mail-injection-skipif-predicate} to a function that returns
non-nil if a message ID is known and should be skipped.  For Notmuch,
@code{piem-notmuch} provides a function that works for this purpose,
@code{piem-notmuch-known-mid-p}:

@lisp
(setq piem-mail-injection-skipif-predicate
      #'piem-notmuch-known-mid-p)
@end lisp

\f
@node Contributing
@chapter Contributing

Patches, bug reports, and other feedback are welcome.  Please send a
plain-text email to @email{piem@@inbox.kyleam.com}.  Messages that
include this address are public and available as public-inbox archives
at @url{https://inbox.kyleam.com/piem}.  Note that this is not a mailing
list.  Updates can be followed through one of public-inbox's pull
methods (@pxref{public-inbox}).  This means it is particularly important
to @emph{not} drop participants when replying.

You can, unsurprisingly, use piem to work on piem by adding an entry
like this to @code{piem-inboxes}.

@lisp
("piem"
 :coderepo "<path/to/local/clone>"
 :address "piem@@inbox.kyleam.com"
 :url "https://inbox.kyleam.com/piem/")
@end lisp

The source repository is available at @url{https://git.kyleam.com/piem}.
Here are some guidelines for sending patches:

@itemize
@item Please send patches inline rather than as attachments.

If you're using @code{git send-email}, you may want to set
@code{sendemail.to} to @code{piem@@inbox.kyleam.com} in your local
repository.

@item Specify the base commit.

This can be done via the @code{--base=} option of @code{git
format-patch} or by configuring @code{format.useAutoBase}.

@item Keep rerolls in the same thread.

In general, prefer to keep iterations of a patch series in the same
thread, labeling rerolls with an appropriate version.

@item Consider adding a range-diff to the cover letter of rerolls.

For a multi-patch series, @code{git range-diff} can often provide a nice
overview of the changes between the previous and last iteration.  Note
that @code{git format-patch} has a @code{--range-diff} convenience
option.

@end itemize

\f
@node Related projects and tools
@chapter Related projects and tools

Here's a (short and incomplete) list of tools that you, as someone that
cared to look at piem's manual, may be interested in---things that
either have some connection to public-inbox or to using an email-based
development workflow in Emacs.  Corrections and additions are welcome.

@itemize

@item
git-email provides an Emacs interface for preparing and sending patches.

@url{https://sr.ht/~yoctocell/git-email/}

@cindex magit
@item
@code{git range-diff} is a built-in Git command for comparing iterations
of a patch series.  If you use Magit, the
@url{https://github.com/magit/magit-tbdiff,magit-tbdiff} extension
provides an interface for @code{git range-diff}.

@url{https://kernel.org/pub/software/scm/git/docs/git-range-diff.html}

@item
grokmirror enables efficient replication of large Git repository
collections.  The HTTP interface for public-inbox exposes a
grokmirror-compatible manifest.js.gz endpoint.

@url{https://git.kernel.org/pub/scm/utils/grokmirror/grokmirror.git}

@cindex mailscripts
@item
mailscripts, mentioned earlier in the manual (@pxref{Applying patches
without a public-inbox archive}), is a ``collection of scripts for
manipulating e-mail on Debian'', including b4-inspired patch extraction
and an Emacs interface for some of the functionality.

@url{https://git.spwhitton.name/mailscripts/}

@end itemize

A list of more public-inbox-related tools is at
@url{https://public-inbox.org/clients.txt}.

\f
@node GNU Free Documentation License
@chapter GNU Free Documentation License
@include fdl-1.3.texi

@node Key Index
@unnumbered Key Index

@printindex ky

@node Variable Index
@unnumbered Variable Index

@printindex vr

@node Lisp Function Index
@unnumbered Function Index

@printindex fn

@node Concept Index
@unnumbered Concept Index

@printindex cp


@bye

debug log:

solving e7b4b32 ...
found e7b4b32 in https://git.kyleam.com/piem/

(*) Git path names are given by the tree(s) the blob belongs to.
    Blobs themselves have no identifier aside from the hash of its contents.^

Code repositories for project(s) associated with this public inbox

	https://git.kyleam.com/piem/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).