Commit Graph
100 Commits
Author SHA1 Message Date
Lukas Greve 6f5d7478be Fix duplicate storage configuration in recipes
- Move storage options from 'modifiers' to 'required' in recipe_templates.yaml
- Changed required.storage from {'ext4': storage/standard.ks} to {'standard': ..., 'encrypted': ...}
- Removed storage from modifiers section
- This ensures storage: encrypted recipes include only storage/encrypted.ks
- Updated generated recipes and dishes to reflect the fix
2026-03-27 16:06:57 +01:00
Lukas Greve a4bc2967e5 Change: Update recipe generation to use single universal template
- Transition from multi-template (install/live recipe types) to single universal template (proteus)
- Remove recipe_type parameter from RecipeGenerator
- Update manifest format (remove recipe_type field)
- Simplify CLI (remove --type argument, use --output for single generation)
- Filename generation now uses primary modifier as first component
2026-03-27 15:39:30 +01:00
Lukas Greve 3ffb079981 refactor: Move recipe generator to cook/ directory and fix flattening bugs
- Move recipe-generator/ to cook/ for cleaner structure

- Fix ksflatten-relative path conversion to handle all %include paths

- Fix validation exit code to only fail on actual errors

- All recipes now generate and flatten successfully via make all
2026-03-27 14:02:59 +01:00
Lukas Greve 288ed0a249 refactor: Simplify template logic - reduce from 6 to 3 categories, remove variant_type, keep validation, remove tests
Simplified template logic: reduced from 6 categories (required, modifiers, optional, versioned, conditional, flags) to 3 categories (required, modifiers, optional). Removed variant_type and replaced with modifier-based approach (version, bootloader replaced, hardware_support). All validation methods kept for recipe generation and validation. Removed tests directory (tests/test_recipe_generator.py, tests/integration/, tests/container/). Removed non-ISO workflows (validate-recipes, test-generation, container-tests, validate-ingredients). Updated Makefile, requirements.txt, and documentation. All 20 recipes successfully generated, validated, and flattened.
2026-03-27 12:13:51 +01:00
Lukas Greve 789e0e05d4 refactor: Remove deprecated ingredients_dir parameter from RecipeGenerator
Simplify RecipeGenerator.__init__() to accept only templates_file parameter.
The old two-argument pattern RecipeGenerator(ingredients_dir, templates_file)
is no longer supported. All existing code already uses the new single-argument
style.

Changes:
- Remove ingredients_dir_or_templates parameter
- Remove backward compatibility logic
- Remove unused Optional import
- Update DEVELOPMENT.md example

Migration: Change RecipeGenerator(Path('ingredients'), Path('templates.yaml'))
           to RecipeGenerator(Path('templates.yaml'))
2026-03-27 11:07:53 +01:00
Lukas Greve 0a3c2b14d8 docs: Add comprehensive documentation to recipe generator
Add detailed docstrings and inline comments to all recipe generator modules explaining classes, functions, and complex logic in plain English.

Files updated:
- recipe-generator/recipe_generator.py: Module, class, and method docs
- recipe-generator/cli.py: CLI modes, arguments, and workflow docs
- recipe-generator/validators.py: Three-layer validation architecture docs
- recipe-generator/manifest.py: Manifest processing and variant expansion docs
- recipe-generator/generate_recipe.py: Entry point documentation
2026-03-27 11:07:16 +01:00
Lukas Greve 31955a8983 refactor: rename fragments directory to ingredients and update all references
- Directory rename: fragments/ → ingredients/ (54 .ks files)
- Updated all Python scripts:
  * tests/integration/conftest.py - fixture renames
  * tests/integration/test_integration.py - constant and path updates
  * tests/test_recipe_generator.py - updated test assertions
  * recipe-generator/validators.py - updated comments and error messages
  * recipe-generator/recipe_generator.py - updated comment
- Updated all YAML files:
  * recipe-generator/recipe_templates.yaml - 66 path references
  * .gitea/workflows/validate-fragments.yaml → validate-ingredients.yaml
  * .gitea/workflows/test-generation.yaml - path patterns
- Updated scripts:
  * bin/ksflatten-relative - updated path detection
- Updated test file:
  * tests/integration/test_fragments.py → test_ingredients.py
- Updated documentation:
  * DEVELOPMENT.md - simplified references
  * DEVELOPMENT_QUICK.md - updated examples
  * tests/container/README.md - test references
- Regenerated all recipes (16 files) with ingredient paths
- Updated test fixtures (7 files)
- All integration tests pass (5/5)
- All unit tests pass (29/31 - 2 pre-existing failures unrelated)
2026-03-26 20:27:56 +01:00
Lukas Greve 3eb4c75392 refactor: fix linting issues - remove unused kwargs, fix unused variable, and make method public 2026-03-26 19:26:31 +01:00
Lukas Greve 050f41787b Deleted extra dishes 2026-03-26 16:10:16 +01:00
Lukas Greve 05d611276d fix: Add _bare-metal suffix for False boolean modifiers to prevent filename collisions
The manifest uses hyphenated keys (guest-agents, hardware-support) but the filename
generation was looking for underscored keys, causing 16 variants to overwrite 4 files.

Added _get_modifier() helper to normalize both key formats and added _bare-metal suffix
for False values of guest_agents, ensuring unique filenames for all variant combinations.

This restores the intended 24 distinct variants from the manifest and adds support for
all modifiers (bootloader, initial-setup, desktop, security, storage, guest_agents, hardware_support).
2026-03-26 16:09:28 +01:00
Lukas Greve 81c8392874 refactor: Remove filename deduplication to generate all 24 variants
- Removed deduplication logic from cli.py
- All 24 variants now processed (some overwrite with same filename)
- manifests: Added comments explaining cartesian product and filename behavior
- tests: Updated assertions for correct filename generation order
- Added 3 new tests for guest_agents and hardware_support modifiers
2026-03-26 15:53:50 +01:00
Lukas Greve 898c831065 refactor: Modularize recipe generator
- Split generate_recipe.py into cli.py, recipe_generator.py, validators.py, manifest.py
- Removed override logic - modifiers now add fragments rather than replace
- Added filename deduplication to prevent overwriting recipes
- Updated CI workflows and tests to use new module structure
- Made pykickstart a hard dependency
- Removed ingredients/ directory support (only fragments/*.ks now used)
- New layout: 5 modules (~990 lines) vs single 769-line file
2026-03-26 15:31:18 +01:00
Lukas Greve 8f824cb977 Remove deprecated command tracking from generate_recipe
- Remove DEPRECATED_COMMANDS dictionary
- Remove _check_deprecated_commands() method
- Remove deprecated command checks from validation
- Remove test methods for deprecated commands
- Remove DEPRECATED_COMMANDS documentation reference

This shifts responsibility for tracking deprecated kickstart commands away
from the recipe generator, as this is not its primary role.
2026-03-26 14:29:22 +01:00
Lukas Greve d93e84cb09 Fix duplicate storage partitioning when storage=encrypted is defined 2026-03-26 14:13:01 +01:00
Lukas Greve 7f2f150fd2 Automate flattening: convert %ksappend to %include for ksflatten compatibility 2026-03-26 14:04:52 +01:00
Lukas Greve 8c0c3907d4 minor change for where the bootloader is 2026-03-26 11:19:09 +01:00
Lukas Greve 5c9a2535df minor change for where the bootloader is 2026-03-26 11:10:36 +01:00
Lukas Greve f6809e8756 Make initial-setup required in install template, with server as default
- Add initial-setup to required section in install template
- Default: fragments/initial-setup/server/config.ks
- Override: fragments/initial-setup/{desktop|gnome|generic-wayland}/config.ks
- Add initial-setup to override_keys set
2026-03-26 11:09:28 +01:00
Lukas Greve b97b9bb919 Make version required in core, with 43 as default
- Add version to required section in install and live templates
- Set 43 as default version (overridable via modifiers)
- Update CLI default from rawhide to 43
2026-03-26 11:06:29 +01:00
Lukas Greve c2829dc614 Add bootloader modifier to install template
- Add bootloader modifier following security pattern (override, not additive)
- Supports 'grub' and 'systemd-boot' options
- Default uses grub from required, override via modifier

Closes #issue
2026-03-26 10:57:42 +01:00
Lukas Greve 9c905ee3de Move security from modifiers to required in install template
- Security is now enabled by default in all install recipes
- Add override handling to skip required fragment when modifier sets security
- Fix modifier key lookup to handle both hyphen and underscore keys

Closes #issue
2026-03-26 10:49:45 +01:00
Lukas Greve 91cb314467 Reduces the number of generated variants 2026-03-26 10:39:46 +01:00
Lukas Greve c193914435 Fix duplicate recipe filenames by adding hardware-support, guest-agents, and hypervisor-type suffixes
Previously, list modifiers like hardware-support, guest-agents, and hypervisor-type created multiple variants but overwrote the same filenames. Now:

1. Add '_hw' suffix when hardware-support is enabled
2. Add '_ga' suffix when guest-agents is enabled
3. Add CPU type (amdcpu, intelcpu) suffix for hypervisor variants
4. Normalize modifier names from hypervisor-type to hypervisor_type

This ensures all 144 recipes have unique filenames.
2026-03-26 10:34:35 +01:00
Lukas Greve cd203b8799 Refactor security and guest-agents config
- Move guest-agents from hypervisor modifier to independent optional feature
- Merge guest-agents/gui.ks into base.ks
- Remove security from required, add optional security modifier with off option
- Fix hypervisor-type modifier key normalization (underscore→hyphen)
- Add expand_variants() to handle list values in manifest
- Update all tests and expected recipes
2026-03-25 21:18:01 +01:00
Lukas Greve 5b8974a6ed hypervisor: implement base vs desktop variants with GUI packages only for desktop
- Split guest-agents/packages.ks into base.ks (qemu-guest-agent) and gui.ks (spice-vdagent)
- Move hypervisor modifier from variant_type.hypervisor to dedicated modifier section
- Added hypervisor modifier with base (headless) and desktop (GUI) options
- Create hypervisor-desktop variant with desktop: gnome for GUI hypervisors
- Update generate_recipe.py to recognize hypervisor-desktop variant name

This ensures GUI packages (virt-manager, spice-vdagent) are only included
in hypervisors with desktop environments, keeping headless hypervisors minimal.
2026-03-25 20:06:13 +01:00
Lukas Greve 0843127d6a Reorganize fragments: flat structure from fragments/shared/ and fragments/platform/ 2026-03-25 19:40:07 +01:00
Lukas Greve 520657dcee recipe-generator: Update to new template structure with install and live templates
- Replaced individual templates with unified 'install' (desktop, server, hypervisor) and 'live' (live-desktop, live-server) templates

- Added new modifiers: variant_type, hypervisor-type, desktop, initial-setup, virtualization

- Updated generate_recipe.py to handle new modifier structure with additive flags

- Updated recipes_manifest.yaml with new modifier structure

- Added missing fragment files and directories
2026-03-25 19:15:32 +01:00
Lukas Greve 3782777058 refactor: functional separation with deploy/, recipe-generator/, and bin/
- deploy/: deployment bash scripts (4 files)
- recipe-generator/: recipe generation tool (Python + YAML configs)
- bin/: executable wrappers (single generate-recipe entry point)
- Updated deploy.sh, README.md, DEVELOPMENT.md, DEVELOPMENT_QUICK.md
- Updated tests/test_recipe_generator.py and tests/integration/test_integration.py
- Updated CI workflow and Makefile paths
- All 41 tests pass
2026-03-25 09:34:00 +01:00
Lukas Greve 00c4356ff3 Add DEVELOPMENT.md and DEVELOPMENT_QUICK.md
DEVELOPMENT.md comprehensive guide for contributors covering:
- Architecture overview
- Development environment setup
- Fragment development (54 modular .ks files)
- Recipe generation via manifest + templates
- Testing suite (36+ tests)
- CI/CD workflows
- Common workflows + Migration Guide

DEVELOPMENT_QUICK.md quick reference companion
2026-03-24 21:43:01 +01:00
Lukas Greve de961e2dd8 ci: regenerate recipes with %ksappend syntax 2026-03-24 21:29:41 +01:00
Lukas Greve f3031fdf0f fix: update integration tests for new recipe paths 2026-03-24 21:27:47 +01:00
Lukas Greve 5e8afd7d6f feat: Phase 2 (fragment migration), Phase 3 (testing infrastructure), Phase 4 (CI/CD)
Phase 2: Fragment Migration
- Migrated 42 ingredients to 54 fine-grained fragments
- Replaced %include with %ksappend syntax
- Created organized fragment structure (platform/generic-43, platform/generic-rawhide, shared/)
- Updated generator to use fragment paths
- All 16 manifest variants generate successfully

Phase 3: Testing Infrastructure
- Created containerized test runner (tests/container/)
- Added integration test suite (tests/integration/)
- Created golden master fixtures (tests/fixtures/expected_recipes/)
- 41 tests passing (36 unit + 5 integration)
- 54 fragments, 16 recipes validated

Phase 4: CI/CD Integration
- 5 new workflows: validate-recipes, validate-fragments, test-generation, container-tests, build-iso
- Added validation gates before ISO builds
- Works with existing fedora-runner-image
- Local testing support via act_runner
2026-03-24 21:27:29 +01:00
Lukas Greve a0a5de31cc Phase 1: Enhance recipe generator with semantic validation
- Add pykickstart integration for semantic validation
- Support version-specific validation (F42 for Fedora 43, DEVEL for rawhide)
- Track deprecated/removed commands (authconfig, keyboard, langsupport, nfs, parted)
- Add warning for deprecated commands, error for removed commands
- Provide migration suggestions for each deprecated command
- Add --strict flag for CI mode (warnings treated as errors)
- Upgrade validate_recipe_semantic() with enhanced error reporting
- Add _check_deprecated_commands() for command-level validation
- Improve version extraction regex to handle _ and - separators
- Add 5 new test cases for deprecated command detection and version extraction
- Update tests/test_recipe_generator.py with comprehensive validation tests

All 36 tests pass with semantic validation working for generated recipes.
2026-03-24 20:36:53 +01:00
Lukas Greve 1aa264c146 Add recipe generator for automation and DRY recipes
- create generate_recipe.py: CLI tool for generating recipes from templates
- add recipe_templates.yaml: 5 base templates (virtual-desktop, virtual-server, desktop-hypervisor, live-desktop, live-server)
- add recipes_manifest.yaml: 16 recipe variants defined declaratively
- add Makefile: automation targets for generate, validate, test
- add requirements.txt: PyYAML dependency
- add tests/test_recipe_generator.py: 25 pytest tests

This enables:
- Template-based recipe generation with minimal duplication
- One-line variant addition via YAML manifest editing
- Automatic duplicate include prevention
- Ingredient path validation

Generated recipes are committed for reproducibility as per project conventions.
2026-03-24 19:52:34 +01:00
Lukas Greve c1c5fa8d07 Updated dishes, including one with luks encryption 2026-03-24 16:00:32 +01:00
Lukas Greve 404533ddf6 readded pipewire 2026-02-19 19:46:33 +01:00
Lukas Greve 636b8a2f28 updated dishes 2026-02-18 21:34:08 +01:00
Lukas Greve 77787e09b5 switch to just --enable, knowing that --reconfig does not seem to survive a pykickstart flattening 2026-02-18 21:33:56 +01:00
Lukas Greve 88db0b65b1 switch to server-based initial-setup until gnome-initial-setup is fixed 2026-02-18 21:32:53 +01:00
Lukas Greve d4b3f7f236 add gnome-terminal 2026-02-18 21:32:01 +01:00
Lukas Greve a3740d6f1e Improve the script logic
Key improvements made:

Enhanced find_fedora_iso function:

Takes dish_name as parameter to understand which version to select
Parses dish names to identify "rawhide" or numeric versions like "43", "44", "45"
Matches ISO files containing both "Fedora-Everything" and the version information
Falls back gracefully if no exact match is found
Backward compatibility:

Maintained the original find_fedora_iso_old function for fallback behavior
If no matching ISO is found based on dish name, falls back to existing behavior
Smart version matching:

For virtual-desktop_43 → looks for ISO containing "43"
For virtual-desktop_rawhide → looks for ISO containing "Rawhide"
Handles edge cases with proper fallbacks
Improved robustness:

Proper error handling for missing directories
Ensures matched files actually exist and are files (not directories)
Handles multiple version patterns
2026-02-15 19:43:43 +01:00
Lukas Greve 465925d81e new resulting dishes 2026-02-15 19:33:51 +01:00
Lukas Greve 5daa5e91ad new dumb script 2026-02-15 19:33:42 +01:00
Lukas Greve 63e7830c4d rely on dnf groups when possible, to better track upstream. Add new base graphical group.
Add new labwc test ingredient
2026-02-15 19:33:29 +01:00
Lukas Greve 714d27b6dc Merge mandatory and default packages ingredients into core-packages
Rename initial setup desktop to gnome initial setup desktop
2026-02-15 19:31:16 +01:00
Lukas Greve 570d355f1f target Fedora-Everything ISO 2026-02-15 19:28:58 +01:00
Lukas Greve 2e6b0d8498 move to Fedora Everything 2026-02-14 17:46:44 +01:00
Lukas Greve fdae77ef23 just new recipes 2026-02-14 17:46:27 +01:00
Lukas Greve 999ff024d3 add new recipes 2026-02-14 17:46:08 +01:00
Lukas Greve 09dd229908 readd the grub2-mkconfig line until a fix for the desktop is found 2026-02-14 17:44:45 +01:00
Lukas Greve e6ebef1818 fix: it is actually the boot partition that needs to be bigger 2026-02-14 17:44:06 +01:00
Lukas Greve f8ea7db7f0 remove unuseful dishes and update the rest 2026-02-14 17:43:29 +01:00
Lukas Greve 84bcfecd12 fix regression 2026-01-31 15:24:45 +01:00
Lukas Greve c102ab48f2 remove the hypervisor part 2025-12-14 10:09:16 +01:00
Lukas Greve 854ce5d009 new devel recipes 2025-12-14 09:49:37 +01:00
Lukas Greve c39e9b7803 recreate development-ready virtual machine recipes 2025-12-14 09:47:06 +01:00
Lukas Greve a5cbd39b31 default to rawhide instead of fedora 43 2025-12-13 14:40:47 +01:00
Lukas Greve 05cf28c11a Add local ISO detection functionality to deploy-distro.sh 2025-12-11 21:56:56 +01:00
Lukas Greve e018870d0e updated recipes 2025-12-11 20:50:24 +01:00
Lukas Greve 0c76e7626d make it even more insecure, to make it easier to debug 2025-12-11 20:50:13 +01:00
Lukas Greve 28c390f1b4 remove the post section to speed installation process 2025-12-11 20:49:29 +01:00
Lukas Greve ebd35f0b1f virtual dishes are for development-only 2025-12-10 21:31:31 +01:00
Lukas Greve 6832142907 add allow ssh incoming connection 2025-12-10 21:31:01 +01:00
Lukas Greve eae12100a8 correct an awful regression. SELinux is either disabled or in enforcing mode. The enable switch does not work 2025-12-10 21:30:19 +01:00
Lukas Greve bc5f156cd9 All non-virtual dishes / recipes are meant to be shipped with sane security defaults 2025-12-10 21:29:25 +01:00
Lukas Greve 8245bdf3b9 revert back gitignore to include dishes 2025-12-10 21:27:45 +01:00
Lukas Greve cabebbcedf add fast_commit and encrypt filesystem feature for ext4. Encrypt is a prerequisite for fscrypt support. Fast_commit is a faster journaling method. 2025-12-10 17:17:39 +01:00
Lukas Greve b028260090 weak security to enable easier debugging 2025-12-09 22:33:51 +01:00
Lukas Greve 1fbe496b6a the proper syntax for firstboot enabling 2025-12-09 22:33:15 +01:00
Lukas Greve ae4ffdc64d updated gitignore. Let's not track changes to complete kickstart files. It goes in the way 2025-12-09 22:32:40 +01:00
Lukas Greve 35deff93b3 tweak the security settings, to allow root login for debuging purposes (i.e. when initial-setup does not work) 2025-12-09 22:20:47 +01:00
Lukas Greve a186c60e9a add these changes to nano branch 2025-12-09 17:03:18 +01:00
Lukas Greve 9400b239f8 test new ingredient.
Now not working. The idea would be to deploy a system from a squashfs.img image, to avoid having to redownload all packages
2025-11-21 11:04:41 +01:00
Lukas Greve 740557564d test new workflow to release ISO 2025-11-17 16:36:53 +01:00
Lukas Greve f612842e50 explicitly mention fedora 43 2025-11-13 21:28:28 +01:00
Lukas Greve 37c8c925dd updated recipes version 2025-11-13 20:31:05 +01:00
Lukas Greve b721ec7b59 bump to fedora 43 2025-11-13 20:30:11 +01:00
Lukas Greve ab9aa58eb2 fix what looks like an embarassing regression
we need to specify arch and version, or have env  variable somehwhere do it for us. As of now, it seems safer to hardcode it.
2025-11-13 20:29:55 +01:00
Lukas Greve 7fced10c52 switch local to Zurich 2025-11-13 20:05:22 +01:00
Lukas Greve 59c82dd9c3 improve comment 2025-11-13 20:05:07 +01:00
Lukas Greve 461936a841 add some comment 2025-11-13 20:04:49 +01:00
Lukas Greve 7c3571f8e6 update comment to match reality 2025-10-16 12:08:39 +02:00
Lukas Greve 7ffc71043d reverting back to GRUB 2025-10-15 14:47:02 +02:00
Lukas Greve a8dcf4011c moving to no-secure-boot, and more generally a simpler virtual machine definition 2025-10-15 14:46:40 +02:00
Lukas Greve 28b3e927a6 fix comment 2025-10-10 12:58:26 +02:00
Lukas Greve 06caf5b3c7 increase size of efi partition and explicitly list mbr for the location of the bootloader 2025-10-10 12:58:09 +02:00
Lukas Greve 5da0303cf5 add Spice and listen to all interfaces by default 2025-10-10 12:07:52 +02:00
Lukas Greve de48f21335 remove grub conf 2025-10-07 15:35:07 +02:00
Lukas Greve 7ae6714da1 remove grub conf 2025-10-07 15:34:56 +02:00
Lukas Greve 246fba3efa switch to systemd-boot 2025-09-07 20:17:05 +02:00
Lukas Greve 2e56372e6d new systemd-boot ingredient for live systems 2025-09-07 20:15:20 +02:00
Lukas Greve 515950daad move to root directory 2025-09-05 18:04:55 +02:00
Lukas Greve 1fecc5ea1b adjust list of packages
packages related to libvirt are explicitly picked
2025-09-05 18:04:01 +02:00
Lukas Greve 8c2b118c8e add curl 2025-07-13 20:53:28 +02:00
Lukas Greve dd41bcb66b refine the package selection 2025-07-13 20:53:21 +02:00
Lukas Greve e9e70c6ce3 add gitignore file, excluding aider 2025-07-13 20:53:09 +02:00
Lukas Greve 6a043dcd3d move and rename list of ingredients 2025-07-01 21:28:50 +02:00
Lukas Greve cf3ff84a6d new script to install prerequisites on common Linux distribution 2025-07-01 21:28:33 +02:00
Lukas Greve 46bbe1d5e9 add a iommu device in the automated VM deployment and set memballon to disable 2025-06-17 14:51:14 +02:00
Lukas Greve b98f41512d restore variables for setting specific path and networking for session and system URI 2025-06-15 17:14:31 +02:00