`:setup` form is not called before Eldev tries to use the package from `eldev-use-vc-repository`

[mathrick]

Describe the bug

#91 suggests that :setup can be used to adjust where Eldev looks for the package files when fetching via VC. However, the form is not actually called until after eldev-package-descriptor is called, which makes it impossible to use packages whose header file is not located at the top level of the repository.

Steps to reproduce

1. Add VC repository for magit, and declare it as a dependency

   (eldev-add-extra-dependencies 'prepare 'magit)
   (eldev-use-vc-repository 'magit :github "magit/magit" :commit "v4.0.0" :setup '(setf eldev-project-source-dirs "lisp"))
   

2. Run eldev prepare
3. It fails with No .el files with package headers in ‘/home/mathrick/Dev/magit-p4/.eldev/git/magit/’

Expected behavior

:setup forms should be executed before

Actual behavior

eldev-package-descriptor is called on the fetched source before :setup had a chance to run, which causes it to fail miserably. Backtrace:

Started up on Sun Mar 9 00:15:39 2025
Running on GNU Emacs 27.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.20, cairo version 1.16.0) of 2020-09-19
Emacs source is unknown
Project directory: ‘/home/mathrick/Dev/magit-p4/’
No file ‘/home/mathrick/.config/eldev/config’, not applying user-specific configuration
Loading file ‘Eldev’...
Using package archive ‘gnu’ at ‘https://elpa.gnu.org/packages/’ with priority 300
Using package archive ‘gnu-devel’ at ‘https://elpa.gnu.org/devel/’ with priority 190
Using package archive ‘melpa-stable’ at ‘https://stable.melpa.org/packages/’ with priority 200
Using package archive ‘melpa-unstable’ at ‘https://melpa.org/packages/’ with priority 100
Reading target dependencies from file ‘.eldev/27.1/target-dependencies.build’...
Target dependency information is up-to-date, not saving...
Old Emacs version detected, remapping lisp-data-mode
Loading file ‘Eldev-local’...
Executing command ‘prepare’...
Contents of package archive ‘gnu’ has been fetched already
Contents of package archive ‘melpa-stable’ has been fetched already
Contents of package archive ‘gnu-devel’ has been fetched already
Contents of package archive ‘melpa-unstable’ has been fetched already
Reusing existing Git clone of ‘/home/mathrick/Dev/magit-p4/.eldev/git/magit/’...
Reading internal evaluation cache from file ‘.eldev/27.1/internal-eval.cache’...
Using cached value for form ‘(eldev-project-source-dirs)’ in directory ‘/home/mathrick/Dev/magit-p4/.eldev/git/magit/’: ("/home/mathrick/Dev/magit-p4/.eldev/git/magit/")
Using cached value for form ‘eldev-project-main-file’ in directory ‘/home/mathrick/Dev/magit-p4/.eldev/git/magit/’: nil

Debugger entered--Lisp error: (error "No .el files with package headers in ‘/home/mathrick/Dev/magit-p4/.eldev/git/magit/’")
 signal(error ("No .el files with package headers in ‘/home/mathrick/Dev/magit-p4/.eldev/git/magit/’"))
 eldev-package-descriptor("/home/mathrick/Dev/magit-p4/.eldev/git/magit/")
 eldev--vc-fetch-repository((magit :git "https://github.com/magit/magit.git" :github "magit/magit" :commit "v4.0.0" :setup (setf eldev-project-source-dirs "lisp")) nil)
 eldev--do-install-or-upgrade-dependencies(project nil nil nil t nil nil)
 eldev--install-or-upgrade-dependencies(project nil nil nil t nil nil)
 eldev-load-project-dependencies(nil nil t)
 eldev-prepare()
 apply(eldev-prepare nil)
 eldev--execute-command(("prepare"))
 eldev-cli(("-dt" "--old-deps" "prepare"))
 (kill-emacs (eldev-cli (append (cdr (member "--" command-line-args)) nil)))
 eval((kill-emacs (eldev-cli (append (cdr (member "--" command-line-args)) nil))) t)
 command-line-1(("--execute" "(let ((eldev--emacs-version (format \"%s.%s\" emacs-major-version emacs-minor-version))\n      (eldev--dir           (getenv \"ELDEV_DIR\"))\n      ;; This is intentional.  First, this is in case ELDEV_LOCAL is\n      ;; defined, second, this is just Eldev default for packages.\n      (load-prefer-newer    t))\n  ;; Setting `debug-on-error' would be useful, but it can break many\n  ;; `package-*' functions, since those use `with-demoted-errors' and\n  ;; so `condition-case-unless-debug'.\n  (unless (and (fboundp 'version<=) (version<= \"24.4\" eldev--emacs-version))\n    (error \"Eldev requires Emacs 24.4 or newer\"))\n  (setf package-user-dir\n        (expand-file-name \"bootstrap\"\n                          (expand-file-name eldev--emacs-version\n                                            (if (> (length eldev--dir) 0)\n                                                eldev--dir\n                                              (if (file-directory-p \"~/.eldev\")\n                                                  \"~/.eldev\"\n                                                ;; Duplicating not-yet-available code from `eldev-xdg-cache-home'.\n                                                (expand-file-name \"eldev\"\n                                                                  (let ((eldev--xdg-cache-dir (getenv \"XDG_CACHE_HOME\")))\n                                                                    (if (and eldev--xdg-cache-dir (file-name-absolute-p eldev--xdg-cache-dir))\n                                                                        eldev--xdg-cache-dir\n                                                                      \"~/.cache\")))))))\n        package-directory-list nil\n        package-archives       nil)\n  (require 'package)\n  (package-initialize t)\n  (let ((package-archives '((\"melpa-stable\" . \"http://stable.melpa.org/packages/\")))\n        (archive-name      \"MELPA Stable\")\n        (inhibit-message  t)\n        (eldev-local      (getenv \"ELDEV_LOCAL\"))\n        eldev-pkg\n        requirements)\n    (unless (= (length eldev-local) 0)\n      (if (string-prefix-p \":pa:\" eldev-local)\n          (setf package-archives `((\"bootstrap-pa\" . ,(file-name-as-directory (substring eldev-local (length \":pa:\")))))\n                archive-name     \"a local package archive\")\n        (with-temp-buffer\n          (insert-file-contents (expand-file-name \"eldev.el\" eldev-local))\n          (setf eldev-pkg                    (package-buffer-info)\n                (package-desc-dir eldev-pkg) (expand-file-name eldev-local))\n          ;; Currently Eldev has no external dependencies, but let's be generic.\n          (dolist (requirement (package-desc-reqs eldev-pkg))\n            (unless (package-activate (car requirement))\n              (push requirement requirements))))))\n    (when (if eldev-pkg\n              requirements\n            (not (package-activate 'eldev)))\n      (let ((inhibit-message nil))\n        (message \"Bootstrapping Eldev for Emacs %s from %s...\\n\" eldev--emacs-version archive-name)\n        (when eldev-pkg\n          (message \"Eldev package itself will be used from `%s'\\n\" eldev-local)))\n      ;; See `eldev-retrying-for-robustness'; since Eldev is not bootstrapped yet, we have\n      ;; to inline everything.  No control from command line here.\n      (let* ((all-retry-delays (when (equal (getenv \"CI\") \"true\") '(30 60 120 180 300)))\n             (remaining-delays all-retry-delays))\n        (catch 'obtained-result\n          (while t\n            (condition-case error\n                (throw 'obtained-result (let ((debug-on-error (and debug-on-error (null remaining-delays))))\n                                          ;; See similar workarounds for `package-refresh-contents' in `eldev.el'.\n                                          (let* (failure\n                                                 (failure-catcher (lambda (original archive &rest arguments)\n                                                                    (unless failure\n                                                                      (condition-case-unless-debug error\n                                                                          (apply original archive arguments)\n                                                                        (error (setf failure (cons error (if (consp archive) (car archive) archive)))))))))\n                                            (advice-add 'package--download-one-archive :around failure-catcher)\n                                            (unwind-protect\n                                                (package-refresh-contents)\n                                              (advice-remove 'package--download-one-archive failure-catcher))\n                                            (when failure\n                                              (error \"%s (when updating contents of package archive `%s')\" (error-message-string (car failure)) (cdr failure))))))\n              (error (let ((inhibit-message nil)\n                           (delay           (pop remaining-delays)))\n                       (unless delay\n                         (when all-retry-delays\n                           (message \"Giving up: too many retries already\"))\n                         (signal (car error) (cdr error)))\n                       (message \"%s\" (error-message-string error))\n                       (message \"Assuming this is an intermittent problem, waiting %s before retrying...\\n\"\n                                (if (< delay 60) (format \"%s s\" delay) (format \"%s m\" (/ delay 60))))\n                       (sleep-for delay)\n                       (let ((n (- 5 (length remaining-delays))))\n                         (message \"Retry #%d%s...\" n (if (= n 5) \", the last\" \" of maximum 5\")))))))))\n      (if eldev-pkg\n          (package-download-transaction (package-compute-transaction nil requirements))\n        (package-install 'eldev)))\n    (when eldev-pkg\n      (push `(eldev . (,eldev-pkg)) package-alist)\n      ;; `package--autoloads-file-name' is package-private.\n      (let* ((autoloads-file     (expand-file-name (format \"%s-autoloads\" (package-desc-name eldev-pkg))\n                                                   (package-desc-dir eldev-pkg)))\n             (autoloads-disabler (lambda (do-load file &rest args) (unless (equal file autoloads-file) (apply do-load file args)))))\n        ;; Otherwise old Emacs versions print an ugly error having not found the autoloads file.\n        (advice-add #'load :around autoloads-disabler)\n        (package-activate-1 eldev-pkg)\n        ;; As of commit 1d5b164109b in Emacs repository, `package-activate-1' no longer modifies `load-path',\n        ;; leaving this to the autoloads file.  As we don't have such a file, we have to do that ourselves.\n        (add-to-list 'load-path (package-desc-dir eldev-pkg))\n        (advice-remove #'load autoloads-disabler))))\n  (require 'eldev)\n  (eldev-start-up))" "--execute" "(kill-emacs (eldev-cli (append (cdr (member \"--\" command-line-args)) nil)))" "--" "-dt" "--old-deps" "prepare"))
 command-line()
 normal-top-level()

Environment

• Eldev: 1.11
• Emacs: 27.1
• OS: Ubuntu Linux 20.04

Additional context

Looking at git blame of eldev-vc.el, it doesn't seem that it ever would have worked, starting with the initial implementation in 96d5baa. There are also no tests to verify that it works.

[doublep]

Looking at git blame of eldev-vc.el, it doesn't seem that it ever would have worked, starting with the initial implementation in 96d5baa. There are also no tests to verify that it works.

I now added a test that demonstrates that :setup does make a difference. I guess eldev-project-source-dirs is sort of special in this case, will investigate.

[doublep]

Can you please test if the fix in commit 5706586 solves your problem?

[mathrick]

Thanks for the quick response! It's better, but still fails (magit-p4 is the package I'm using Eldev for):

$ ELDEV_LOCAL=~/Dev/eldev/ eldev prepare
Dependency ‘magit’ is not available
Required by package ‘magit-p4’

And when run with -dt:

$ ELDEV_LOCAL=~/Dev/eldev/ eldev -dt prepare
Started up on Tue Mar 11 23:46:25 2025
Running on GNU Emacs 27.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.20, cairo version 1.16.0) of 2020-09-19
Emacs source is unknown
Project directory: ‘/home/mathrick/Dev/magit-p4/’
No file ‘/home/mathrick/.config/eldev/config’, not applying user-specific configuration
Loading file ‘Eldev’...
Using package archive ‘gnu’ at ‘https://elpa.gnu.org/packages/’ with priority 300
Using package archive ‘gnu-devel’ at ‘https://elpa.gnu.org/devel/’ with priority 190
Using package archive ‘melpa-stable’ at ‘https://stable.melpa.org/packages/’ with priority 200
Using package archive ‘melpa-unstable’ at ‘https://melpa.org/packages/’ with priority 100
Reading target dependencies from file ‘.eldev/27.1/target-dependencies.build’...
Target dependency information is up-to-date, not saving...
Loading file ‘Eldev-local’...
Executing command ‘prepare’...
Contents of package archive ‘gnu’ has been fetched already
Contents of package archive ‘melpa-stable’ has been fetched already
Contents of package archive ‘gnu-devel’ has been fetched already
Contents of package archive ‘melpa-unstable’ has been fetched already
Reusing existing Git clone of ‘/home/mathrick/Dev/magit-p4/.eldev/git/with-editor/’...
Reading internal evaluation cache from file ‘.eldev/27.1/internal-eval.cache’...
Using cached value for form ‘(eldev-project-source-dirs)’ in directory ‘/home/mathrick/Dev/magit-p4/.eldev/git/with-editor/’: ("/home/mathrick/Dev/magit-p4/.eldev/git/with-editor/lisp/")
Using cached value for form ‘eldev-project-main-file’ in directory ‘/home/mathrick/Dev/magit-p4/.eldev/git/with-editor/’: nil
Reusing existing Git clone of ‘/home/mathrick/Dev/magit-p4/.eldev/git/transient/’...
Using cached value for form ‘(eldev-project-source-dirs)’ in directory ‘/home/mathrick/Dev/magit-p4/.eldev/git/transient/’: ("/home/mathrick/Dev/magit-p4/.eldev/git/transient/lisp/")
Using cached value for form ‘eldev-project-main-file’ in directory ‘/home/mathrick/Dev/magit-p4/.eldev/git/transient/’: nil
Reusing existing Git clone of ‘/home/mathrick/Dev/magit-p4/.eldev/git/magit/’...
Using cached value for form ‘(eldev-project-source-dirs)’ in directory ‘/home/mathrick/Dev/magit-p4/.eldev/git/magit/’: ("/home/mathrick/Dev/magit-p4/.eldev/git/magit/lisp/")
Using cached value for form ‘eldev-project-main-file’ in directory ‘/home/mathrick/Dev/magit-p4/.eldev/git/magit/’: nil

Dependency ‘magit’ is not available
Required by package ‘magit-p4’

Finished erroneously on Tue Mar 11 23:46:27 2025

From a quick look at the backtrace, I can see that eldev--plan-install-or-upgrade can't find anything to populate its available list, even though the from-vc value is correctly passed in. I haven't really had the time to look in detail as to why that is.

[doublep]

I was able to locally create a project that uses Magit from VC like this.

File xxx.el:

;;; xxx.el --- Bla bla  -*- lexical-binding: t -*-

;; Version: 1.0
;; Homepage: https://example.com/
;; Package-Requires: ((emacs "24") (magit "0.1"))

;;; Commentary:

(require 'magit)

(provide 'xxx)

;;; xxx.el ends here

File Eldev:

(eldev-use-vc-repository 'magit :git "https://github.com/magit/magit"
                         :setup `(progn (setf eldev-project-source-dirs "lisp"     ; it has source code in a subdirectory
                                              eldev-project-main-file "magit.el")  ; it has several files with package headers, need to set one explicitly
                                        (eldev-use-package-archive 'gnu)
                                        (eldev-use-package-archive 'melpa)))       ; to build a package, Eldev needs to know where to find dependencies

;; Top-level Eldev (for _this_ project) needs to know where to find dependencies of Magit.
;; `eldev-use-vc-repository' only tells it where to find Magit _itself_ (and dependency lookup is not
;; inherited currently).
(eldev-use-package-archive 'gnu)  ; for `compat'
(eldev-use-package-archive 'melpa)

Example run, $ eldev clean all; EMACS=emacs-28 eldev -dt prepare (using an older Emacs just to demonstrate that it also works):

Deleted 1 directory
Started up on Wed Mar 12 22:15:59 2025
Running on GNU Emacs 28.2 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.41, cairo version 1.18.0) of 2025-03-12
Emacs source: rev. 739b5d0e52 of branch ‘HEAD’
Project directory: ‘/home/paul/test/xxx/’
Loading file ‘/home/paul/.eldev/config’...
Loading file ‘Eldev’...
Using package archive ‘gnu’ at ‘https://elpa.gnu.org/packages/’ with priority 300
Using package archive ‘melpa-stable’ at ‘https://stable.melpa.org/packages/’ with priority 200
Using package archive ‘melpa-unstable’ at ‘https://melpa.org/packages/’ with priority 100
No file ‘Eldev-local’, not customizing build
Executing command ‘prepare’...
Cloning Git repository ‘https://github.com/magit/magit’...
Most recent stable release in the repository is tagged ‘v4.3.1’
Initially switching to commit ‘v4.3.1’...
No file ‘.eldev/28.2/internal-eval.cache’, not reading internal evaluation cache
Starting a child Eldev process to evaluate form ‘(eldev-project-source-dirs)’ in directory ‘/home/paul/test/xxx/.eldev/git/magit/’
Evaluated to ‘("/home/paul/test/xxx/.eldev/git/magit/lisp/")’
Starting a child Eldev process to evaluate form ‘eldev-project-main-file’ in directory ‘/home/paul/test/xxx/.eldev/git/magit/’
Evaluated to ‘"magit.el"’
Fetching contents of package archive ‘gnu’...
Using file ‘https://elpa.gnu.org/packages/archive-contents’ from the global cache (at ‘/home/paul/.eldev/global-cache/gnu/archive-contents’)...
Using file ‘https://elpa.gnu.org/packages/archive-contents.sig’ from the global cache (at ‘/home/paul/.eldev/global-cache/gnu/archive-contents.sig’)...
Fetching contents of package archive ‘melpa-stable’...
Using file ‘https://stable.melpa.org/packages/archive-contents’ from the global cache (at ‘/home/paul/.eldev/global-cache/melpa-stable/archive-contents’)...
File ‘https://stable.melpa.org/packages/archive-contents.sig’ is not cached, retrieving...
Not fetching contents of other archive(s) as redundant
[1/7] Installing package ‘seq’ (2.24) from ‘gnu’...
Using file ‘https://elpa.gnu.org/packages/seq-2.24.tar’ from the global cache (at ‘/home/paul/.eldev/global-cache/gnu/seq-2.24.tar’)...
Using file ‘https://elpa.gnu.org/packages/seq-2.24.tar.sig’ from the global cache (at ‘/home/paul/.eldev/global-cache/gnu/seq-2.24.tar.sig’)...
[2/7] Installing package ‘compat’ (30.0.2.0) from ‘gnu’...
Using file ‘https://elpa.gnu.org/packages/compat-30.0.2.0.tar’ from the global cache (at ‘/home/paul/.eldev/global-cache/gnu/compat-30.0.2.0.tar’)...
Using file ‘https://elpa.gnu.org/packages/compat-30.0.2.0.tar.sig’ from the global cache (at ‘/home/paul/.eldev/global-cache/gnu/compat-30.0.2.0.tar.sig’)...
[3/7] Installing package ‘llama’ (0.6.1) from ‘melpa-stable’...
Using file ‘https://stable.melpa.org/packages/llama-0.6.1.tar’ from the global cache (at ‘/home/paul/.eldev/global-cache/melpa-stable/llama-0.6.1.tar’)...
File ‘https://stable.melpa.org/packages/llama-0.6.1.tar.sig’ is not cached, retrieving...
[4/7] Installing package ‘magit-section’ (4.3.1) from ‘melpa-stable’...
Using file ‘https://stable.melpa.org/packages/magit-section-4.3.1.tar’ from the global cache (at ‘/home/paul/.eldev/global-cache/melpa-stable/magit-section-4.3.1.tar’)...
File ‘https://stable.melpa.org/packages/magit-section-4.3.1.tar.sig’ is not cached, retrieving...
[5/7] Installing package ‘transient’ (0.8.5) from ‘gnu’...
Using file ‘https://elpa.gnu.org/packages/transient-0.8.5.tar’ from the global cache (at ‘/home/paul/.eldev/global-cache/gnu/transient-0.8.5.tar’)...
Using file ‘https://elpa.gnu.org/packages/transient-0.8.5.tar.sig’ from the global cache (at ‘/home/paul/.eldev/global-cache/gnu/transient-0.8.5.tar.sig’)...
[6/7] Installing package ‘with-editor’ (3.4.3) from ‘melpa-stable’...
Using file ‘https://stable.melpa.org/packages/with-editor-3.4.3.tar’ from the global cache (at ‘/home/paul/.eldev/global-cache/melpa-stable/with-editor-3.4.3.tar’)...
File ‘https://stable.melpa.org/packages/with-editor-3.4.3.tar.sig’ is not cached, retrieving...
[7/7] Installing package ‘magit’ (4.3.1) from ‘https://github.com/magit/magit’...
Creating a package from ‘https://github.com/magit/magit’
Full command line (in directory ‘/home/paul/test/xxx/.eldev/git/magit/’):
  /home/paul/.local/bin/eldev --setup '(progn (setf eldev-project-source-dirs "lisp" eldev-project-main-file "magit.el") (eldev-use-package-archive '\''gnu) (eldev-use-package-archive '\''melpa))' package --output-dir /tmp/eldev-vc-A6l2Wb --force-version 4.3.1
[1/6] Installing package ‘seq’ (2.24) from ‘gnu’...
[2/6] Installing package ‘compat’ (30.0.2.0) from ‘gnu’...
[3/6] Installing package ‘llama’ (0.6.1) from ‘melpa-stable’...
[4/6] Installing package ‘magit-section’ (4.3.1) from ‘melpa-stable’...
[5/6] Installing package ‘transient’ (0.8.5) from ‘gnu’...
[6/6] Installing package ‘with-editor’ (3.4.3) from ‘melpa-stable’...
Child Eldev process produced no output (other than maybe on stderr)
Finished successfully on Wed Mar 12 22:16:26 2025
Saving internal evaluation cache to file ‘.eldev/28.2/internal-eval.cache’...

The first (1..7) dependency installation is for outer Eldev, i.e. for the example project xxx. The second (1..6) is for inner VC dependency Magit, i.e. it happens when building a package out of it. It's confusing and not exactly simple, but seems to work. Suggestions to make the process more understandable are welcome.

[mathrick]

@doublep: I see. This makes VC repositories of little practical use, IMHO, as it's effectively to impossible to declare all the dependencies. I'm using VC repos in order to test compatibility with the oldest declared versions my package supports, and for it to work, I'd have to declare all those packages as VCs in my Eldev, then transitively in setup forms for each of those dependencies, then deeper still in the nested setup forms for those dependencies, and so on. Unless I'm missing something, that is just unusable.

Would it be possible to redesign them on top of straight.el perhaps? I have most of the code to accomplish that already in my Eldev, and it's relatively straightforward, with the one difficulty being that there's no way to declare which package is to be loaded from the given dir when using eldev-use-local-sources, which is necessary to load git-commit (because it lives in the same directory as magit), hence the need for declare-vc-package.

(defvar straight-base-dir (file-name-as-directory (eldev-cache-dir t)))
;; Ensure a stable location for the lockfile
(defvar straight-profiles `((nil . ,(expand-file-name "package-lockfile.el" eldev-project-dir))))
;; Need straight.el for old versions of dependencies, since `eldev-use-vc-repository' has
;; a showstopper bug: https://github.com/emacs-eldev/eldev/issues/114
(defun bootstrap-straight ()
  (unless (require 'straight nil t)
    (eldev-trace "Bootstrapping straight.el")
    (let ((bootstrap-file
           (expand-file-name "straight/repos/straight.el/bootstrap.el" straight-base-dir))
          (bootstrap-version 7))
      (unless (file-exists-p bootstrap-file)
        (with-current-buffer
            (url-retrieve-synchronously
             "https://raw.githubusercontent.com/radian-software/straight.el/develop/install.el"
             'silent 'inhibit-cookies)
          (goto-char (point-max))
          (eval-print-last-sexp)))
      (load bootstrap-file nil 'nomessage))))

(cl-defun use-straight-repository (recipe &optional (loading-mode 'as-is))
  (straight--with-plist (straight--convert-recipe recipe)
      (package includes)
    (eldev-trace "Installing %s via straight" package)
    (straight-use-package recipe)
    (eldev-use-local-sources (straight--build-dir package) loading-mode)
    (cl-loop for include in (if (listp includes)
                                includes
                              (list includes))
             do (declare-vc-package include (straight--build-dir package) loading-mode))
    (eldev-trace "Installed %s into %s" package (straight--build-dir package))))

(cl-defun declare-vc-package (name dir &optional (loading-mode 'as-is))
  "Declare package NAME as being present in DIR, using LOADING-MODE.

This function is needed when a package lives in the same
directory as another package, such as `git-commit' being included
as a part of `magit', as it won't be detected by
`eldev-package-descriptor'"
  (eldev-trace "Registering included package '%s' at '%s'" name dir)
  (straight-use-package name)
  (let* ((pkg-desc (cl-loop for (file reader . args) in `((,(format "%s.el" name) package-buffer-info)
                                                          (,(format "%s-pkg.el" name) package--read-pkg-desc dir))
                            for path = (expand-file-name file dir)
                            when (file-readable-p path)
                            return (with-temp-buffer
                                     (insert-file-contents path)
                                     (apply reader args))
                            finally (error "No package file for '%s' found in directory '%s'" name dir))))
    (push `(,name ,pkg-desc ,dir ,(expand-file-name dir eldev-project-dir) ,loading-mode)
          eldev--local-sources)))


;; Set up dependencies manually for testing compatibility with old versions of Emacs and libraries
(eldev-defoption use-old-dependencies ()
  :options (--old-deps)
  (eldev-trace "Old dependency mode enabled")
  (bootstrap-straight)
  ;; Need to add git-commit manually for some reason, the built-in recipe doesn't always know about it
  (use-straight-repository '(magit :includes git-commit))
  (use-straight-repository 'magit-section)
  (use-straight-repository 'transient)
  (use-straight-repository 'with-editor)
  (use-straight-repository 'dash))

Note: straight.el doesn't really support :commit, so I'm using an explicit lockfile for my project, which is the official way of pinning a specific version. But it would probably be easy enough either to abuse :branch (not sure whether that'd have any negative consequences), or create a lockfile on the fly if a :commit is given.

[doublep]

as it's effectively to impossible to declare all the dependencies

You don't need to declare the dependencies themselves, only where to look them up, e.g. package archives. It is exactly the same without VC repositories, actually. Your project needs to declare where to find its dependency D (be that another VC repository or MELPA), but it also needs to declare where to search for dependencies of D. In fact, this is also the same for plain Emacs package manager: to install Magit it needs a way to find its dependencies.

I'm using VC repos in order to test compatibility with the oldest declared versions my package supports, and for it to work, I'd have to declare all those packages as VCs in my Eldev, then transitively in setup forms for each of those dependencies, then deeper still in the nested setup forms for those dependencies, and so on. Unless I'm missing something, that is just unusable.

Well, this particular usecase was not foreseen, but see below.

Would it be possible to redesign them on top of straight.el perhaps?

No, sorry. Your usecase is not the only one and for different reasons Eldev needs to have direct control over package installation. I investigated this when adding support for VC repositories in 1.11.

What could be possible is to find out repository URLs automatically, e.g. from MELPA, like straight.el does (I think).

Another possible improvement would be to avoid installing dependencies when simply building a package, as this is apparently not even needed. E.g. to build Magit package you apparently don't need llama and so on, only to install the package. In other words, "inner" Eldev invocation (for Magit) wouldn't need to locate the dependencies, only the "outer" (for your project).

If this is implemented, you'd not need to pass all those eldev-use-... in setup forms at all. The reason this was not important before is that Eldev would either deal with PA-provided dependencies or with local sources that used Eldev themselves, so it was not really important.

I need to investigate this closer, though, then maybe implement this for 1.12.

[mathrick]

You don't need to declare the dependencies themselves, only where to look them up, e.g. package archives. It is exactly the same without VC repositories, actually. Your project needs to declare where to find its dependency D (be that another VC repository or MELPA), but it also needs to declare where to search for dependencies of D. In fact, this is also the same for plain Emacs package manager: to install Magit it needs a way to find its dependencies.

Sure, but you do need to provide some dependencies, and there isn't any way of just using the dependencies that I'm trying to specify with eldev-use-vc-repository in the first place! And if the dependencies that get used are vastly different from the VC deps I'm trying to specify, it will materially affect the outcome. I.e. if I specify one version of transient to use, but another version gets pulled in from PA to build the Magit I also specify an explicit version of, then all kinds of spurious incompatibilities are bound to happen that are purely due to this limitation.

I don't fully understand why Eldev goes through the circuitous route of having to spawn a sub-Emacs to build every package. Does it provide some sort of extra benefits, or is it simply because all of the Eldev functions that deal are currently hardcoded to look for the "current project", and it would be difficult to work around it otherwise? I don't think any of this would be an issue if Eldev just dealt with packages in place, instead of spawning another Emacs.

Another possible improvement would be to avoid installing dependencies when simply building a package, as this is apparently not even needed. E.g. to build Magit package you apparently don't need llama and so on, only to install the package. In other words, "inner" Eldev invocation (for Magit) wouldn't need to locate the dependencies, only the "outer" (for your project).

I suspect this will not be true in the general case, as packages are free to use their deps through eval-when-compile, and many do, which would break this assumption.

[doublep]

I don't fully understand why Eldev goes through the circuitous route of having to spawn a sub-Emacs to build every package.

Basically it tries to emulate how users normally use your package (installed from somewhere). Unless you specify loading modes, but those are only for local sources, not for VC repositories.

Does it provide some sort of extra benefits, or is it simply because all of the Eldev functions that deal are currently hardcoded to look for the "current project", and it would be difficult to work around it otherwise?

It would actually be outright impossible, at least in a robust way. Since Eldev script is a full-fledged program, it can do anything it wants. And it's then impossible to give any sorts of guarantees when using Emacs instances cross-project. The only possibility is to spawn another Emacs if another project needs to be build. (Of course, the "parent" could still spoil it for the "child", but in this case it would have to actively want to do that, not just break the dependency accidentally by changing some Eldev variable that the child didn't expect.)

I suspect this will not be true in the general case, as packages are free to use their deps through eval-when-compile

Building packages doesn't involve compilation: it only happens when packages are installed. Anyway, let me try (when I have time) and see if it really works.