snakemake mode test failures with emacs 28.2 on Debian

snakemake mode test failures with emacs 28.2 on Debian

[Diane Trout]

Hi,

I noticed there's 3 ert test failures running Emacs 28.2 on Debian
unstable.

My knowledge of elisp is pretty poor so I'm not entirely sure what's
broken, but to start the debugging process here's the errors reported
from running 
M-x ert RET t RET

after having run this command
emacs  -Q -l package --eval "(add-to-list 'package-directory-list
\"/usr/share/emacs/site-lisp/elpa\")" --eval "(add-to-list 'package-
directory-list \"/usr/share/emacs/site-lisp/elpa-src\")" -f package-
initialize -L . -l snakemake-test.el --eval \(ert-run-tests-batch-and-
exit\)

F snakemake-test-check-target
    (ert-test-failed
     ((should
       (snakemake-with-temp-dir
         (snakemake-check-target "aa.out")))
      :form
      (let*
          ((vnew #'...)
           (old
            (symbol-function ...)))
	(unwind-protect
            (progn
              (fset ... vnew)
              (let* ... ...))
          (fset 'message old)))
      :value nil))

F snakemake-test-file-targets-at-point
    (ert-test-failed
     ((should
       (equal
	'("aa.out")
	(snakemake-with-temp-dir
          (with-temp-buffer ... ... ...))))
      :form
      (equal
       ("aa.out")
       nil)
      :value nil :explanation
      (different-types
       ("aa.out")
       nil)))

F snakemake-test-rule-at-point
    (ert-test-failed
     ((should
       (equal
        '("aa")
        (snakemake-with-temp-dir
          (with-temp-buffer ... ... ... ...))))
      :form
      (equal
       ("aa")
       nil)
      :value nil :explanation
      (different-types
       ("aa")
       nil)))

Thanks for any advice you can offer.
Diane

Re: snakemake mode test failures with emacs 28.2 on Debian

[Kyle Meyer]

Diane Trout writes:

> Hi,
>
> I noticed there's 3 ert test failures running Emacs 28.2 on Debian
> unstable.

Thanks for reporting.

With Emacs 28.2 and snakemake 7.7.0 from Guix, the tests pass on my end.
It looks like Debian unstable currently has snakemake 7.12.1-1.  My
guess is that these failures are triggered by a snakemake change.

I'll try to look into that within the next couple of days.

Re: snakemake mode test failures with emacs 28.2 on Debian

[Diane Trout]

On Sat, 2023-01-07 at 19:09 -0500, Kyle Meyer wrote:
> Diane Trout writes:
> 
> > Hi,
> > 
> > I noticed there's 3 ert test failures running Emacs 28.2 on Debian
> > unstable.
> 
> Thanks for reporting.
> 
> With Emacs 28.2 and snakemake 7.7.0 from Guix, the tests pass on my
> end.
> It looks like Debian unstable currently has snakemake 7.12.1-1.  My
> guess is that these failures are triggered by a snakemake change.
> 
> I'll try to look into that within the next couple of days.

I discovered the snakemake ert tests run on my Debian testing system
but fail in the unstable package build environment.

I read through the snakemake.el code and figured out how it was calling
the snakemake executable. I found the problem, and it's not related to
snakemake-mode, it looks like snakemake executable has an
incompatibility with python3.11.

Upstream's solution is here and involves using a more recently updated
depedency.
https://github.com/snakemake/snakemake/issues/1952


$ python3.11 /usr/bin/snakemake --dryrun aa.out
Building DAG of jobs...
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/snakemake/__init__.py", line
730, in snakemake
    success = workflow.execute(
              ^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/snakemake/workflow.py", line
942, in execute
    self.scheduler = JobScheduler(
                     ^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/snakemake/scheduler.py", line
105, in __init__
    from ratelimiter import RateLimiter
  File "/usr/lib/python3/dist-packages/ratelimiter.py", line 36, in
<module>
    class RateLimiter(object):
  File "/usr/lib/python3/dist-packages/ratelimiter.py", line 127, in
RateLimiter
    __aexit__ = asyncio.coroutine(__exit__)
                ^^^^^^^^^^^^^^^^^
AttributeError: module 'asyncio' has no attribute 'coroutine'

$ python3.10 -Wd /usr/bin/snakemake --dryrun aa.out
Building DAG of jobs...
/usr/lib/python3/dist-packages/ratelimiter.py:127: DeprecationWarning:
"@coroutine" decorator is deprecated since Python 3.8, use "async def"
instead
  __aexit__ = asyncio.coroutine(__exit__)
Job stats:
job      count    min threads    max threads
-----  -------  -------------  -------------
aa           1              1              1
total        1              1              1


[Sat Jan  7 19:56:59 2023]
rule aa:
    output: aa.out
    jobid: 0
    reason: Missing output files: aa.out
    resources: tmpdir=/tmp

Job stats:
job      count    min threads    max threads
-----  -------  -------------  -------------
aa           1              1              1
total        1              1              1

Reasons:
    (check individual jobs above for details)
    missing output files:
        aa

This was a dry-run (flag -n). The order of jobs does not reflect the
order of execution.
/usr/lib/python3.10/tempfile.py:999: ResourceWarning: Implicitly
cleaning up <TemporaryDirectory
'/home/diane/.cache/snakemake/snakemake/source-cache/runtime-
cache/tmplqpvu6li'>
  _warnings.warn(warn_message, ResourceWarning)

Re: snakemake mode test failures with emacs 28.2 on Debian

[Kyle Meyer]

Diane Trout writes:

> I discovered the snakemake ert tests run on my Debian testing system
> but fail in the unstable package build environment.
>
> I read through the snakemake.el code and figured out how it was calling
> the snakemake executable. I found the problem, and it's not related to
> snakemake-mode, it looks like snakemake executable has an
> incompatibility with python3.11.

I see.  Thanks for investigating.

> Upstream's solution is here and involves using a more recently updated
> depedency.
> https://github.com/snakemake/snakemake/issues/1952

It looks like the associated fix came in with 50b8f160 (fix: Change
ratelimiter dependency to throttler (#1958), 2022-11-10), which was part
of the v7.18.2 release.  So, the issue should be resolved whenever
unstable gets bumped to the latest Snakemake.