Question on organization

[briannielson]

I'm working on migrating our configuration management from puppet over to Cfengine and I would like to do this piecemeal. My first target is our development mysql installation, so I went ahead and got all of my mysql daemon-related code into a single file called mysqld.cf. Inside that file I have a body common control block that defines the bundlesequence and everything is running as expected (boy howdy is it fast!).

Now I want to move onto the next chunk of code (our node server) and I'm at a loss of how to organize it. I wanted to organize things into files grouped by service (e.g. node, mysql, java) to make it more straightforward to use docker containers (e.g. the MySQL container just runs mysqld.cf, the node container just runs node.cf, etc).

We don't use docker containers now, but I want the configuration code for Docker containers to be the same as the configuration code for our bare metal AWS+development instances (or as close to the same as possible...). It will (hopefully) make migration that much smoother in the long run, and means I don't have to rewrite configuration again to get Docker working for local development.

I can invoke Cfengine multiple times from a Makefile for each service to setup on the bare metal instances, and only the relevant service in my Docker containers, but it seems like there should be a way to do this in Cfengine? I played around with a control.cf file that defined the bundlesequence as a variable, then override that variable in a __main__ bundle in each service file, but I couldn't get that working...

Any thoughts? Any resources I should look into?

[atsaloli]

Hi Brian! Welcome! Yes, CFEngine is blazing fast. services/main.cf is the suggested place to put your custom policies That’s in the masterfiles policy framework (MPF) which ships with CFEngine. You don’t HAVE to use it but it provides a lot of nice things. See https://docs.cfengine.com/docs/3.26/reference-masterfiles-policy-framework-services-main.html You can use inputs to pull in multiple files under services, e.g., services/ mysqld.cf Hope this helps! Let us know how it goes. Best, Aleksey

…

On Fri, Aug 29, 2025 at 2:50 PM Brian Bauman ***@***.***> wrote: I'm working on migrating our configuration management from puppet over to Cfengine and I would like to do this piecemeal. My first target is our development mysql installation, so I went ahead and got all of my mysql daemon-related code into a single file called mysqld.cf. Inside that file I have a body common control block that defines the bundlesequence and everything is running as expected (boy howdy is it fast!). Now I want to move onto the next chunk of code (our node server) and I'm at a loss of how to organize it. I wanted to organize things into files grouped by service (e.g. node, mysql, java) to make it more straightforward to use docker containers (e.g. the MySQL container just runs mysqld.cf, the node container just runs node.cf, etc). We don't use docker containers now, but I want the configuration code for Docker containers to be the same as the configuration code for our bare metal AWS+development instances (or as close to the same as possible...). It will (hopefully) make migration that much smoother in the long run, and means I don't have to rewrite configuration again to get Docker working for local development. I can invoke Cfengine multiple times from a Makefile for each service to setup on the bare metal instances, and only the relevant service in my Docker containers, but it seems like there should be a way to do this in Cfengine? I played around with a control.cf file that defined the bundlesequence as a variable, then override that variable in a __main__ bundle in each service file, but I couldn't get that working... Any thoughts? Any resources I should look into? — Reply to this email directly, view it on GitHub <#5872>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAD4GIDXURBFHKKG5QP2S6T3QCVJ5AVCNFSM6AAAAACFFOOAEGVHI2DSMVQWIX3LMV43ERDJONRXK43TNFXW4OZYHAYTMMRWGE> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[nickanderson]

Organizing things is a bit like opinions, everyone has their own, and generally CFEngine doesn't try to make many prescriptions about that.

I would encourage you to checkout build.cfengine.com and consider managing your policy set with the cfbs tooling. Let us know how you progress, reach out for help here, on the mailing list, and or on matrix.

[basvandervlies]

Yes like Nick said build it round `cfbs` then you are the most flexible. You can easily add services, modules and promise types. Maybe this can be inspiration for you. It is all installable via `cfbs`: * https://build.cfengine.com/modules/surf-cfengine-library/ * https://build.cfengine.com/modules/promise-type-docker-compose/ There are many more that are useful and easy to install. Have fun

…

On 29/08/2025 23:43, Nick Anderson wrote: Organizing things is a bit like opinions, everyone has their own, and generally CFEngine doesn't try to make many prescriptions about that. I would encourage you to checkout build.cfengine.com and consider managing your policy set with the cfbs tooling. Let us know how you progress, reach out for help here, on the mailing list, and or on matrix. — Reply to this email directly, view it on GitHub <https://github.com/ cfengine/core#5872#discussioncomment-14259779>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ AADJKB2DSRQD4DWJWZQ7DZ33QDCQDAVCNFSM6AAAAACFFOOAEGVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTIMRVHE3TOOI>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

-- -- Bas van der Vlies | High Performance Computing & Visualization | SURF| Science Park 140 | 1098 XG Amsterdam | T +31 (0) 20 800 1300 | ***@***.*** | www.surf.nl

[briannielson]

Thanks everyone, I wish I could mark more than one comment as an answer.

I think the main lesson I am taking from this is that a sequence of events can be an anti-pattern in the context of a declarative language. While bundlesequence exists, it seems like this is really designed to be used for discrete blocks, e.g. first install mysql, then node, then nginx, etc. The fact I need this is more than likely an indication that my code is written incorrectly!

Organizing things is a bit like opinions, everyone has their own, and generally CFEngine doesn't try to make many prescriptions about that.

To some extent, every language is opinionated. When it takes 100 lines to replace 5 lines of shell scripting, it seems like a good indication that you are going against the grain of what the authors had in mind. That being said:

• Sometimes good enough is as high a bar as is worth achieving, especially when someone else is paying you for the work
• It is hard to know what a good design is when you've only implemented less than five percent of the system

For now, I've created a bundle that runs the other bundles I've made with explicit handles + depends_on, then refactored that into its own bundle with a note to avoid doing the same in the future. It ain't pretty, but it works:

# We shouldn't ever use run_sequence, instead preferring declarative promises!
# However, its sometimes easier to define a sequence of events when
# translating install scripts
bundle agent run_sequence(prefix, sequence_list)
{
  vars:
    "bundle_list" slist => { @(sequence_list) };
    "step_0" string => nth("bundle_list", 0);
    "step_1" string => nth("bundle_list", 1);
...

  classes:
    "has_step_0" expression => not(strcmp("$(step_0)", ""));
    "has_step_1" expression => not(strcmp("$(step_1)", ""));
...

  methods:
    has_step_0::
      "$(prefix)_step_0"
        usebundle => "$(step_0)",
        handle => "$(prefix)_step_0",
        comment => "Sequence step 0: $(step_0)";

    has_step_1::
      "$(prefix)_step_1"
        usebundle => "$(step_1)",
        handle => "$(prefix)_step_1",
        depends_on => { "$(prefix)_step_0" },
        comment => "Sequence step 1: $(step_1)";
...

  reports:
    cfengine_3.verbose_mode::
      "Sequential execution $(prefix) completed";
}

Which gets called as:

bundle agent mysqld_setup
{
  vars:
    "sequence" slist => {
      "mysqld_pre",
      "mysqld_install",
      "mysqld_configure",
      "mysqld_running",
      "mysqld_setup_data"
    };

  methods:
    "MySQL full setup"
      usebundle => run_sequence("MySQL setup", "@(sequence)");

  reports:
    cfengine_3.verbose_mode::
      "MySQL setup sequence completed";
}

We'll see how the code matures after moving more parts of our system off of puppet into cfengine.

Thanks again!

[olehermanse]

@briannielson expanding a bit on what the others recommended, here's a short demo of how I'd set it up in your case;

$ mkdir briannielson-project
$ cd briannielson-project/
$ cfbs init
  ... [ Answer prompts or press enter to use defaults and edit later ]
$ mkdir custom-policy
$ echo "bundle agent entrypoint
> {
>   # Put policy here
> }" > custom-policy/entrypoint.cf
$ cfbs add custom-policy/
Which bundle should be evaluated (added to bundle sequence)?
 1. ./custom-policy/entrypoint.cf:entrypoint (default)
 2. (None)
 [1/2]: 1
Added module: ./custom-policy/
The default commit message is 'Added module './custom-policy/'' - edit it? [yes/y/NO/n] 
Committing using git:

[main 2e7c039] Added module './custom-policy/'
 2 files changed, 16 insertions(+)
 create mode 100644 custom-policy/entrypoint.cf

$ cfbs status
Name:        Example project
Description: Example description
File:        cfbs.json

Modules:
001 masterfiles      @ 3.24.2 / 5c49f1ed1c3a4ecfc46b4b61296ff5dd38991f33 (Downloaded)
002 ./custom-policy/ @ local                                             (Copied)

You project now has 3.24.2 default policy ("masterfiles"), with your custom policy inside ./custom-policy/.
custom-policy and briannielson-project are just example names, of course, you can name things what you like.

In order to get a working, deployable policy set, you need to build it:

$ cfbs build

Modules:
001 masterfiles      @ 5c49f1ed1c3a4ecfc46b4b61296ff5dd38991f33 (Downloaded)
002 ./custom-policy/ @ local                                    (Copied)

Steps:
001 masterfiles      : run 'EXPLICIT_VERSION=3.24.2 EXPLICIT_RELEASE=1 ./prepare.sh -y'
001 masterfiles      : copy './' 'masterfiles/'
002 ./custom-policy/ : directory './' 'masterfiles/services/cfbs/custom-policy/'
002 ./custom-policy/ : policy_files 'services/cfbs/custom-policy/entrypoint.cf'
002 ./custom-policy/ : bundles 'entrypoint'

Generating tarball...

Build complete, ready to deploy 🐿
 -> Directory: out/masterfiles
 -> Tarball:   out/masterfiles.tgz

To install on this machine: sudo cfbs install
To deploy on remote hub(s): cf-remote deploy

You can start editing custom-policy/entrypoint.cf to include or call your bundles. Other policy files can be added to custom-policy/ without doing anything special, just make sure they are committed to git, and add calls to their bundle(s) to entrypoint.cf (and rerun cfbs build after making changes).

The advantages of this approach are many:

• No "mixing" (patching) of default policy with your own, future upgrades will be 1 simple command instead of resolving conflicts.
• As Bas pointed out, it will make it easy to add modules from other people (such as promise types).
• Conversely, you can more easily make reusable modules to share within your organization, or with the rest of the community on build.cfengine.com.
• You can also split up your own policy between different modules / teams / repos, as desired.
• Your git commit history will be well organized, focusing on your policy and edits, not all the changes we make to the default policy.

Let us know if you have any questions!

[nickanderson]

bundle agent mysqld_setup
{
  vars:
    "sequence" slist => {
      "mysqld_pre",
      "mysqld_install",
      "mysqld_configure",
      "mysqld_running",
      "mysqld_setup_data"
    };

  methods:
    "MySQL full setup"
      usebundle => run_sequence("MySQL setup", "@(sequence)");

  reports:
    cfengine_3.verbose_mode::
      "MySQL setup sequence completed";
}

I am not saying you should change anything, I just want to point out some other policy and ideas that you might find useful.

The methodology you use here is part way to a custom "autorun": https://cfengine.com/blog/2022/extending-autorun/ Rather than explicitly enumerating the bundles as you have in sequence you can use bundlesmatching() to find bundles by characteristic of its name or tags that it has. Doing this, you can account for ordering by using sort().