Lengthy detailed GIT and ASDF LISP story


1. INTRO

On the chilling-themed episode today (archive bot failed: archive later), we started to explore a few peoples' git useage and systems.

While a reasonable amount of the episode was me just saying stuff I do-a-lot-every-day or reading others' descriptions, I think there is enough for a modestly interesting how-to-lisp article. Criticism and participation appreciated.

2. GIT

You could not do this, but you also could do this. I noticed that nobody seems very happy with their git flow, up to and including git flow bloggers. Here's what I do.

  1. Make a repository on https://codeberg.org/tfw/ or possibly https://gitlab.com/screwlisp since that's where mdhughes is
  2. =cd ~/gits/= =git clone --bare git@gitlab.com:screwlisp/my-repo-name.git=
  3. =cd ~/working/= =got checkout ~/gits/my-repo-name=
  4. = cd ~/common-lisp/= =ln -s ~/working/my-repo-name=

A less stepful version would be to just clone the repo into =~/common-lisp/= or if specifically only using quicklisp, =~/quicklisp/local-projects/=, but I like to use got's collection of specific, sophisticated behaviours rather than git's do-anything-at-all methodology.

2.1. If a folder like ~/common-lisp/foo/ already exists

  1. =cd ~/common-lisp/foo/=
  2. =git init=
  3. =git remote set-url origin git@codeberg.org:tfw/foo.git=
  4. =git push -f=

Since got and git differ, I won't try and summarise what-you-do-with-them.

Also I'm not addressing special cases.

3. ASDF

Well, let's say you're me, and you decide to remake NUD from the ground up.

This is just what I normally do day-to-day. I guess there's =~/common-lisp/nud/= a git, in whatever your prefered style is.

3.1. nud/nud.asd

Note the implied system definition is for =load-op=, don't worry about it but you can have system operations that do literally anything to a system (the other default conventional one being =test-op=).

(defsystem "nud"
  :depends-on (:nud/user)
  :class :package-inferred-system)

this system-definition-file says that when somebody does something like =(require :nud)= or =(asdf:load-system :nud)=, this means to load whatever =:nud/user= is, by default meaning =~/common-lisp/nud/user.lisp=.

=:class :package-inferred-system= means that every package is one file, and every source file is one package. The first thing in source files is required to be a package definition, in my case =:nud/user= is

(uiop:define-package :nud/user
    (:mix :nud/main :cl))

my convention is that normal systems always have a :my-system/user package that :mixes or :import-froms everything for normal use of that package. This makes it a pretty good choice to load when someone says =(require :nud)=, even if they don't intend to =(in-package :nud/user)= per se. Compare, =mcclim-user=.

3.1.1. Odd(er) things about package-inferred-system

Note that packages are referred to by the path to their source file within the system, not their registered package name. If you want to register a differe package name, you could =(register-system-packages :nud/user '(:nud-user))= in your .asd file. However, you still refer to :nud/user (i.e. its path) in :depends-on, :mix, :use and so forth. However the package name it is Truly Registered Under will be :nud-user rather than the default, :nud/user. Actually, I never do this, since the old-fashioned-style nicknames just take a whole package name as an alias for the package. So I don't, but I could

(uiop:define-package :nud/user
    (:mix :nud/main :cl)
  (:nicknames :nud-user))

And this is similar to having register-system-packages :nud/user to :nud-user, except that I can also refer to the package (truly) using :nud/user, and I would get this when I introspect the package's name I guess.

3.2. Dependencies

As we can see, =:nud/user= mixes =:nud/main=. Mix is like use, except it assumes you always want to shadow prior definitions. So if :nud/main and :cl had an exported name collision, mix shadows :cl's one with :nud/main's one. Otherwise you have to customise :shadowing a lot, potentially.

Since we are =:class :package-inferred-system=, what-to-compile is recursively found from the =:depends-on (:nud/user)= in the system definition asd file.

4. Compiling lisp

The major occupation, I think, is what to do when I modify :nud/main. Let's tell the whole story.

4.1. Start lisp

(slime)

in emacs, I guess. =$ sbcl= in a shell would work too. I really meant the whole story

4.2. Require nud

(require :nud)

4.3. Enter nud/user to use nud

(in-package :nud/user)

This has compiled :nud/main as a dependency of :nud/user. But what if I now modify :nud/main. I often forget to add exports to the package definition.

4.3.1. Change

(uiop:define-package :nud/main
    (:mix :cl)
  (:export #:gird #:move-in #:move-on #:look
           #:} #:}} #:{ #:reg-act #:cursor-set
           #:] #:]] #:[ #:[[
           #:&))

my naming decisions notwithstanding, I have forgotten to export =#:@=, which I added to work like =@= in VI (it executes whatever is in a register, as vi expressions/macros).

But, I forgot to export it.

(uiop:define-package :nud/main
    (:mix :cl)
  (:export #:gird #:move-in #:move-on #:look
           #:} #:}} #:{ #:reg-act #:cursor-set
           #:] #:]] #:[ #:[[
           #:& #:@))

4.3.2. Recompile just :nud/main (and dependencies that changed at all)

(asdf:load-system :nud/main :force t)

the =:force t= is necessary, because asdf's default behaviour is to leave packages that are already loaded alone. Forcing lets it know to overwrite the already-loaded package in places that changed.

Well, now I can use the exported symbol (it's a function)

(nud/main:@ g)

; @ performs an operation on a particularly structured cons tree conventionally named g.

4.3.3. Recompile just :nud/user

But since I only recompiled =:nud/main=, I don't have this newly exported symbol mixed into =:nud/user=, which I would like it to be.

(asdf:load-system :nud/user :force t)

Now, =:nud/user= gets recompiled (now mixing =:nud/main:@=), but =:nud/main= did not need to be recompiled. And =@= is mixed into =:nud/user=.

(@ g)

. In this case I could have just =(asdf:load-system :nud/user :force t)= which would have found and recompiled =:nud/main='s changes, but not doing that was a learning experience.

Note that variables and things unique to the =:nud/user= and =:nud/main= package instances loaded into the repl were preserved, unless they were explicitly overwritten by a source change, so you don't have to interrupt what you were doing at all. I would call this hot-recompiling dependencies.

5. Summary

Other than git, my convention is to always start with an asd like:

5.1. ~/common-lisp/nud/nud.asd

(defsystem "nud"
  :depends-on (:nud/user)
  :class :package-inferred-system)

where the /user package mixes other packages to-be-used, and all source files start with package definitions like

5.2. ~/common-lisp/nud/user.lisp

(uiop:define-package :nud/user
    (:mix :nud/main :cl))

5.3. ~/common-lisp/nud/main.lisp

(uiop:define-package :nud/main
    (:mix :cl)
  (:export #:gird #:move-in #:move-on #:look
           #:} #:}} #:{ #:reg-act #:cursor-set
           #:] #:]] #:[ #:[[
           #:& #:@))

(in-package :nud/main)

6. Other packages referencing single file/packages within this one

Let's say I wanted to drag in :Marklib as currently housed in :arrokoth to choose a specific example with :nud/user.

6.1. ~/common-lisp/nud/user.lisp

(uiop:define-package :nud/user
    (:import-from :arrokoth/marklib)
  (:mix :nud/main :cl))

Er, refering to my specific PR of arrokoth. Note that the package :marklib is a synonym for :arrokoth/marklib, since it has that global nickname. People say don't do this, but I do.

However in asdf's :package-inferred-system, I'm really giving paths to files inside of asdf's central source registry locations, even if I register-system-packages :arrokoth/marklib to truly just be named :marklib. Of course, I could just move marklib to its own system with =~/commmon-lisp/marklib/marklib.asd= and so forth.

</div>

Author: screwlisp

Created: 2025-01-29 Wed 16:26

Validate

Get lispmoo2

Comments

Log in with itch.io to leave a comment.

Episode! Featuring the usual crew in lambdaMOO !