From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:203:b4db::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms12 with LMTPS id 2Ku8D/k6zmHVfQAAsNZ9tg (envelope-from ) for ; Thu, 30 Dec 2021 23:04:25 +0000 Received: from out1.migadu.com ([2001:41d0:2:863f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id QODuDvk6zmFAXgAA9RJhRA (envelope-from ) for ; Fri, 31 Dec 2021 00:04:25 +0100 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kyleam.com; s=key1; t=1640905465; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type; bh=JcEBjibh++XbV7ULb1AqaxdJO7eDXS0yrZb45a0sMC8=; b=mFcUvyqOflDRa5SROC7Yy8gSFTmdp/+p1BoxzT9OgErNiqX9LepZBjDSVHeIqYqwrNMvZY JRPZJcTc4yQE+lLafzswgoPInFsCx5pS85i6AgWsfCX0M4qzGPa12PEaduEDuZJs9Tca/K 5ClGCQ8Np73ONwpqPyzoNDLU//zuu1c1x7lMSC8VFo94f+kprpOfVnbRCP4BKstmPgUk8e c3fxAb/UNeRespgU7ij9G511W74+05gLfbWjMCIexRynM0Ej0ZYO3lqlCJSKqtcSeGiiSA 4p3+KhgpYXtD6Kiksu4uPYWqRF/JytHqbOCByWtj0lVj98PKduwE2lJuMsGVBA== From: Kyle Meyer To: meta@public-inbox.org Cc: piem@inbox.kyleam.com Subject: lei q: importing messages when specifying '-f *json*'? Date: Thu, 30 Dec 2021 18:04:23 -0500 Message-ID: <87k0fly9s8.fsf@kyleam.com> MIME-Version: 1.0 Content-Type: text/plain X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: kyleam.com X-TUID: htIhhdruP0V3 The lei-q interface I'm working on for piem [1] consumes JSON output to display search results. From there, an individual message can be shown. If the query is against a remote external, that involves two curl calls to the remote, one for the search and one for the display. I'd like to import hits from the first step into the local store so that I can avoid the second curl call. However, I haven't been able to figure out a way to do this when requesting JSON output. For example [2], say I don't have meta's 20211124154539.350522-1-e@80x24.org locally: $ lei daemon-kill $ export HOME=$(mktemp -d "${TMPDIR:-/tmp}"/pi-testing-XXXXXXX) $ lei q m:20211124154539.350522-1-e@80x24.org # /tmp/pi-testing-3JaSz2K/.config/lei/config created [null] If I search for that message against https://public-inbox.org/meta/ and request JSON output $ lei q -I https://public-inbox.org/meta/ -f ldjson \ m:20211124154539.350522-1-e@80x24.org # /usr/bin/curl -Sf -s -d '' https://public-inbox.org/meta/?x=m&q=m%3A20211124154539.350522-1-e%4080x24.org {"blob":"a8754283bd9e985d6e1156215071be59aa2b5a53",...} then no message ends up in the local store: $ lei q m:20211124154539.350522-1-e@80x24.org [null] In contrast, if I request mboxrd output $ lei q -I https://public-inbox.org/meta/ -f mboxrd \ m:20211124154539.350522-1-e@80x24.org >/dev/null # /usr/bin/curl -Sf -s -d '' https://public-inbox.org/meta/?x=m&q=m%3A20211124154539.350522-1-e%4080x24.org the message is imported to the local store: $ lei q -f ldjson m:20211124154539.350522-1-e@80x24.org {"blob":"a8754283bd9e985d6e1156215071be59aa2b5a53",...} $ git -C $HOME/.local/share/lei/store/local/0.git/ log --oneline ca12a1b (HEAD -> master) [PATCH] eliminate some unused subs I was hoping that --import-remote might do the trick, but that doesn't seem to be the case: $ lei daemon-kill $ export HOME=$(mktemp -d "${TMPDIR:-/tmp}"/pi-testing-XXXXXXX) $ lei q --import-remote -I https://public-inbox.org/meta/ \ -f ldjson m:20211124154539.350522-1-e@80x24.org # /tmp/pi-testing-Ny7KDcB/.config/lei/config created # /usr/bin/curl -Sf -s -d '' https://public-inbox.org/meta/?x=m&q=m%3A20211124154539.350522-1-e%4080x24.org {"blob":"a8754283bd9e985d6e1156215071be59aa2b5a53",...} $ lei q m:20211124154539.350522-1-e@80x24.org [null] Should --import-remote trigger an import in the case above? [1] https://git.kyleam.com/piem/tree/piem-lei.el [2] These are with public-inbox's current master (07cd8973baf).