diff --git a/.gitea/workflows/build-iso.yaml b/.gitea/workflows/build-iso.yaml index 11e32eb..860c079 100644 --- a/.gitea/workflows/build-iso.yaml +++ b/.gitea/workflows/build-iso.yaml @@ -21,17 +21,10 @@ jobs: run: | pip install PyYAML pykickstart - - name: Generate all recipes + - name: Generate, lint, and validate dishes run: | - cd recipe-generator - python3 generate_recipe.py \ - --manifest recipes_manifest.yaml \ - --output-dir ../recipes/ - - - name: Validate recipes (strict mode) - run: | - cd recipe-generator - python3 generate_recipe.py --validate ../recipes/*.cfg --strict + cd cook + make all build-iso: needs: validate @@ -52,22 +45,27 @@ jobs: run: | mock -r fedora-43-x86_64 --install lorax-lmc-novirt vim-minimal pykickstart livecd-tools + - name: Generate dishes + run: | + cd cook + make all + - name: Copy configuration file to mock run: | - mock -r fedora-43-x86_64 --copyin recipes/live-server_rawhide.cfg /builddir + mock -r fedora-43-x86_64 --copyin dishes/desktop_43_standard_grub_gnome_guest-agents.cfg /builddir - name: Build ISO with livemedia-creator run: | mock -r fedora-43-x86_64 --shell --enable-network --isolation=simple << 'EOF' cd /builddir - livemedia-creator --ks live-server_rawhide.cfg --no-virt --resultdir /var/lmc --project live-server --make-iso --volid live-server --iso-only --iso-name server-43.iso --releasever 43 --macboot + livemedia-creator --ks desktop_43_standard_grub_gnome_guest-agents.cfg --no-virt --resultdir /var/lmc --project phyllomeos --make-iso --volid phyllomeos --iso-only --iso-name desktop-43.iso --releasever 43 --macboot EOF - name: Upload ISO as artifact uses: actions/upload-artifact@v3 with: - name: server-43.iso - path: /var/lmc/server-43.iso + name: desktop-43.iso + path: /var/lmc/desktop-43.iso if-no-files-found: error - name: Cleanup mock environment diff --git a/.gitea/workflows/build-iso_more-recent.yaml b/.gitea/workflows/build-iso_more-recent.yaml index 31eea88..81be60a 100644 --- a/.gitea/workflows/build-iso_more-recent.yaml +++ b/.gitea/workflows/build-iso_more-recent.yaml @@ -7,7 +7,7 @@ on: env: FEDORA_VERSION: 43 - KICKSTART_FILE: live-desktop-hypervisor + KICKSTART_FILE: desktop_43_standard_grub_gnome_guest-agents jobs: checkout: @@ -29,6 +29,11 @@ jobs: run: | mock -r fedora-${{ env.FEDORA_VERSION }}-x86_64 --install lorax-lmc-novirt vim-minimal pykickstart livecd-tools + - name: Generate dishes + run: | + cd cook + make all + - name: Copy configuration file to mock run: | mock -r fedora-${{ env.FEDORA_VERSION }}-x86_64 --copyin dishes/${{ env.KICKSTART_FILE }} /builddir diff --git a/README.md b/README.md index 1d61d03..841905f 100644 --- a/README.md +++ b/README.md @@ -7,188 +7,125 @@ Provided that some dependencies are met (`libvirt` is running on your computer, - Make the script executable: ``` -chmod +x deploy-vm.sh +chmod +x deploy.sh ``` -- Execute it and pick `virtual-desktop-hypervisor` when prompted: +- Execute it and pick a dish, e.g. `desktop_43_standard_grub_gnome_guest-agents`, when prompted: ``` -./deploy-vm.sh +./deploy.sh Executing: ./deploy/core-count.sh System has more than 2 core (nproc --all: 6). [...] -10. virtual-desktop-hypervisor +Available files: +1. desktop_43_standard_grub_gnome [...] -Enter the number of the file you want to select: 10 -You selected: virtual-desktop-hypervisor +8. desktop_43_standard_grub_gnome_guest-agents +[...] +Enter the number of the file you want to select: 8 +You selected: desktop_43_standard_grub_gnome_guest-agents Starting install... Retrieving 'vmlinuz' | 16 MB 00:00:00 Retrieving 'initrd.img' | 161 MB 00:00:05 Allocating 'virtinst-n0km88yy-vmlinuz' | 16 MB 00:00:00 -Transferring 'virtinst-n0km88yy-vmlinuz' | 16 MB 00:00:00 -Allocating 'virtinst-qxr2jxcb-initrd.img' | 161 MB 00:00:00 Transferring 'virtinst-qxr2jxcb-initrd.img' | 161 MB 00:00:00 -Allocating 'virtual-desktop-hypervisor.img' | 10 GB 00:00:00 +Allocating 'desktop_43_standard_grub_gnome_guest-agents.img' | 10 GB 00:00:00 Creating domain... ``` + +`deploy.sh` regenerates the `recipes` and `dishes` (they are build products) before deploying. After a successfull installation, the virtual machine will shutdown and be ready to use when powered on again. ## Repository structure This repository contains such files broken down as: -* `ingredients`๐Ÿฅ‘ ๐Ÿฅฅ ๐Ÿฅญ ๐Ÿฅ ๐Ÿฅฆ ๐Ÿฅฌ ๐Ÿฅ’ ๐Ÿง„: the basic building blocks for assembling Phyllome OS and other derivatives. +* `cook/ingredients`๐Ÿฅ‘ ๐Ÿฅฅ ๐Ÿฅญ ๐Ÿฅ ๐Ÿฅฆ ๐Ÿฅฌ ๐Ÿฅ’ ๐Ÿง„: the basic building blocks for assembling Phyllome OS and other derivatives. -* `recipes`๐Ÿงพ ๐Ÿงฉ: lists of ingredients to compose several editions +* `cook/recipes`๐Ÿงพ ๐Ÿงฉ: lists of ingredients to compose several editions. **Build product**: generated by `make all`. -* `dishes`๐Ÿฅจ ๐Ÿฅ ๐Ÿฅ– ๐Ÿฅง ๐Ÿฅž ๐Ÿฅฏ ๐Ÿง† ๐Ÿง: read-to-consume and standalone kickstart artifacts, which can be used to deploy complete systems +* `cook/dishes`๐Ÿฅจ ๐Ÿฅ ๐Ÿฅ– ๐Ÿฅง ๐Ÿฅž ๐Ÿฅฏ ๐Ÿง† ๐Ÿง: ready-to-consume and standalone kickstart artifacts, which can be used to deploy complete systems. **Build product**: generated by `make all`. -Each ingredient represents a feature or a set of integrated features, such as a specific Desktop Environment or a storage configuration. - - Ingredients prefixed with *live* such as `live-core.cfg` are to be used with live editions only - - *core* ingredients are meant be used in all their respective recipes, *base* ingredients, recommended but optional, and extra provides more stuff (sic) +Each ingredient represents a feature or a set of integrated features, such as a specific Desktop Environment or a storage configuration. ## Development -Using a pull request, you can suggest a modification to an existing ingredient or create a new ingredient from scratch. +Using a pull request, you can suggest a modification to an existing ingredient or create a new ingredient from scratch. The cooking pipeline is fully data-driven: + +* `cook/recipes_manifest.yaml` declares which editions (groups) and which variant matrix to build. +* `cook/recipe_templates.yaml` ("Proteus") wires variant values and flags to ingredient fragments: `base` (always included), `choices` (exactly-one per category), `features` (additive). +* `make all` (in `cook/`) composes recipes, flattens them into dishes with pykickstart in-process, lints the invariants pykickstart can't check, and validates every dish. ### Requirements - `qemu` - `libvirt` - `virt-install` -- `pykickstart` +- `pykickstart` (needed to flatten and validate; `pip install -r cook/requirements.txt`) -### Example 1: add a new package and include it into a recipe +### Example: add a new package and include it into a recipe -- Add [Luanti](https://www.luanti.org/), a free and open-source sandbox video game engine formerly known as Minetest, as a standalone ingredient, using the `echo` command +- Add [Luanti](https://www.luanti.org/), a free and open-source sandbox video game engine formerly known as Minetest, as a standalone ingredient: ``` echo "%packages --exclude-weakdeps # Beginning of the package section. Does not include weak dependencies luanti # Multiplayer infinite-world block sandbox with survival mode -%end # End of the packages section" > ingredients/extra-luanti.cfg +%end # End of the packages section" > cook/ingredients/extra-luanti.ks ``` -Instead of creating a recipe from scratch, let's make a copy of the `virtual-desktop.cfg` recipe, which provide a Desktop environment necessary for *luanti* to function +- Wire it as a new optional feature in `cook/recipe_templates.yaml`: -``` -cp recipes/virtual-desktop.cfg recipes/virtual-desktop-luanti.cfg +```yaml +features: + luanti: extra-luanti.ks ``` -- Add the extra ingredient to the new recipe: - -``` -echo "%include ../ingredients/extra-luanti.cfg # Sandbox video game engine" >> recipes/virtual-desktop-luanti.cfg -``` - -#### Flatten - -- Prepare the dish by following the recipe, a process called 'flattening' +- Regenerate: ``` -ksflatten -c recipes/virtual-desktop-luanti.cfg -o dishes/virtual-desktop-luanti.cfg +cd cook && make all ``` -> If any errors are detected, go back and fix them. +- Every dish that enables the `luanti` feature now contains the new ingredient. To name such dishes distinctly, enable the flag in the manifest, e.g. add `luanti: [true, false]` to a variant list. -It is time to test the new dish! - -#### Kickstart - -- Make the `deploy-vm.sh` script executable +Alternatively, for a bespoke dish not part of the matrix, append the ingredient directly to an existing recipe and re-flatten it: ``` -chmod +x deploy-vm.sh +echo "%include ../ingredients/extra-luanti.ks # Sandbox video game engine" >> cook/recipes/desktop_43_standard_grub_gnome.cfg +cd cook && make generate ``` -- Execute the script +### Example: create a new edition from the existing list of ingredients + +- Print the full ingredient inventory derived from the templates: ``` -./deploy-vm.sh +cd cook && make inventory ``` -- Select the new dish, *virtual-desktop-luanti* +- Define a new edition (a group named e.g. `server`) in `cook/recipes_manifest.yaml` with the desired variants; the cartesian product of its list values generates one dish per combination. + +### Useful targets (in `cook/`) ``` -[...] -Available files: -1. desktop-hypervisor-amdcpu -[...] -14. virtual-desktop-luanti +make all # generate recipes + dishes, lint, and validate (default) +make lint # check the manifest/templates without writing files +make validate # validate existing dishes +make inventory # print the ingredient catalog derived from the templates +make test # run the pytest test suite ``` -- When the installation is done, the machine will shut down - -- Start it again, and ensure that Luanti has correctly been installed - -That's it ! - -### Example 2: Create a new recipe from the existing list of ingredients - -The file `recipes/_list-of-ingredients.cfg` can be copied and edited to create your own remix of Phyllome OS, which itself is a remix of Fedora. - -``` -cp recipes/_list-of-ingredients.cfg recipes/my-new-distro.cfg -``` - -Then edit the said file to include your favorite ingredient - -``` -nano recipes/my-new-distro.cfg -``` - -``` -# __ ____ ____ _____ -# ____ / /_ __ __/ / /___ ____ ___ ___ / __ \/ ___/ -# / __ \/ __ \/ / / / / / __ \/ __ `__ \/ _ \ / / / /\__ \ -# / /_/ / / / / /_/ / / / /_/ / / / / / / __/ / /_/ /___/ / -# / .___/_/ /_/\__, /_/_/\____/_/ /_/ /_/\___/ \____//____/ -# /_/ /____/ - -# The list of ingredients for composing Phyllome OS -# Uncomment lines with "%include" to enable ingredient - -# Installation method -# Exactly one option has to be picked -# %include ../ingredients/core.cfg # Text mode -# %include ../ingredients/live-core.cfg # For live systems only -# Documentation: https://pykickstart.readthedocs.io/en/latest/kickstart-docs.html#graphical-or-text-or-cmdline - -# Storage configuration -# Exactly one option has to be picked -# WARNING !!! Will erase local disks! -# %include ../ingredients/core-storage.cfg # Basic ext4 partition layout for UEFI-based systems -# %include ../ingredients/live-core-storage.cfg # For live systems only -# Documentation: https://pykickstart.readthedocs.io/en/latest/kickstart-docs.html#part-or-partition -[...] -``` - -- Once you are done, you can [flatten](#flatten) the file and [kickstart](#kickstart) it as explained in the previous section. - ## FAQ -If multiple dishes are affected by your ingredient, you can flatten them all +- **I change one ingredient and many dishes are affected** โ€” just run `make all` to regenerate the full matrix; recipes and dishes are derived from the templates, so they cannot drift. -- Navigate to the recipes' directory +- **I want to inspect what a dish contains** โ€” forget `git diff`: generated files are intentionally untracked. Open `cook/recipes/.cfg` for the include list or `cook/dishes/.cfg` for the flattened kickstart. -``` -cd recipes -``` - -- Then use the following - -``` -for filename in *.cfg; do ksflatten -c "$filename" -o "../dishes/$filename"; done -``` - -The following message can safetly be ignored: - -``` -/usr/lib/python3.13/site-packages/pykickstart/commands/partition.py:461: KickstartParseWarning: A partition with the mountpoint / has already been defined. -``` +- **The `%packages` sections are merged and sorted by pykickstart** โ€” that is expected canonical output. ## Acknowledgement