Alternative command runner to make (tasks/just/duty)

[dalito]

Make is problematic for Window users because it is typically not available. It is also quite challenging to write cross-platform shell commands and get escaping right. Make as a full build tool is naturally more complex than required for a simple command runner. Since we only need a command runner here, we may consider some alternatives.

I looked at

• tasks (written in go, cross-platform, yaml-syntax for task-definitions)
• just (written in rust, cross-platform, makefile-like syntax for task-definitions)
• duty (written in Python, cross-platform, python-file syntax for task-definitions, similar to invoke)

For just and tasks binaries are available for all platforms and installation is straight-forward (also on Windows). duty as a Python tool can be installed by pipx. Since linkML-project-cookiecutter users are already instructed to use pipx for cruft the installation of duty as command runner is the simplest one.

The syntax of tasks and just can be learned relatively fast, but I found it clearly more time-consuming than learning to define tasks in Python for duty. In duty I already know how to deal with cross-platform compatibility and have all great Python features like f-strings or path-manipulation available. So my favorite is duty.

I wrote a duties.py file that mostly matches the current makefile. - PR to come...

What do you think in general about this? Does it help? Does it add or reduce complexity?

[dalito]

In the meantime just can be installed via pipx as well. This is a nice surprise since the just devs said "not planned" when I asked for it a while ago. Since just is a much wider used tool, I closed the duty-based PR #121 in favor of a future just-based PR.

[sierra-moxon]

Hi @dalito - what do you think about creating a stand-alone package that takes a makefile as input and spits out a justfile as output?

From today's dev call, we were a bit wary about introducing a new build system at the cookie-cutter level. We understand the need for simplification of the build process for Windows users (and more generally, our cookie-cutter has a lot of functionality that we want to streamline). Perhaps (in the long term) we decrease the need for make or just and move more of this into python with config files?

[dalito]

I think converting from make to just in a reliable way gets too complex quickly. I could not find a good converter to help with creating the justfile from the PR. Also the AI systems were not able to produce good working code based on the makefile as input.

Perhaps (in the long term) we decrease the need for make or just and move more of this into python with config files?

I agree. A simpler config and the core project management commands available in Python (either from linkML or from this cookiecutter repo) would be better than yet another tool.

[dalito]

Some additional notes:

• The main disadvantage of the justfile proposed in Add task definitions for just as alternative command runner #127 is that it still requires access to bash (or sh) which comes with git-for-windows but needs to be put on path (or configured in the justfile). Such small steps are often a hurdle for beginners. Something to note is that just can use other shells / recipes as well, e.g. powershell, but then the recipes would be platform-specific. It can also run Python recipes which could solve the bash-shell-config issue. In Add task definitions for just as alternative command runner #127 I wrote the justfile in a makefile-alike style for easy comparison with the makefile. However, it would also be possible to write a justfile using python recipes (see example) for everything where more logic is involved.

• It does not help that the current makefile expects a non-standard dotenv file (no quotes around strings) but just expects a standard-compliant dotenv file with quotes around strings with spaces. So just and make cannot use exactly the same dotenv config file (until someone fixes encoding/dequoting in the makefile). Should this point get its own issue?

[dalito]

FYI: I have created an experimental new copier-based linkml-project template, which uses just as primary command runner but still includes the Makefile for now. Currently the new template is still compatible with the code here; this means it produces the same project with copier as here with cruft/cookiecutter.

For more see https://github.com/dalito/linkml-project-copier/releases/tag/v0.1.0