Salt Guide

Salt can be used as a configuration management tool similar to Puppet or Ansible.

This guide describes orcharhino’s Salt support. It describes how to install the Salt plugin on orcharhino, how to integrate it with your Salt master, as well as its usage and supported features.

Note

Salt supports two distinct modes of operation: Clientless (using ssh) or using the Salt minion client software. Currently, the orcharhino Salt plugin supports exclusively the Salt minion approach.

Summary of supported features:

  • Importing Salt states, assigning them to Salt environments, and finally associating those Salt environments with hosts or host groups.

  • Using orcharhino as an external node classifier (ENC) for Salt. (orcharhino parameters are made available as Salt pillars which can then be used as variables in Salt states.)

  • Viewing configuration management reports and facts (aka Salt grains), from within orcharhino.

  • Running an arbitrary Salt function (e.g.: sate.highstate) from the orcharhino interface.

Installation

When installing a new orcharhino it is possible to select Salt as a configuration management system during step 5 of the main orcharhino installation. Choosing this option will install and configure both the Salt plugin and a Salt master on your orcharhino.

To install the Salt Plugin on an existing orcharhino installation, use the following command:

foreman-installer --enable-foreman-plugin-salt --enable-foreman-proxy-plugin-salt

This will install the orcharhino Salt plugin, as well as the smart proxy features for the bundled smart proxy (which is always included in your main orcharhino installation).

If you do not yet have a working Salt master, you will need to install one. The Salt master can either run on your main orcharhino, or on an orcharhino smart proxy with the Salt feature enabled. It is also possible to use an existing Salt master by installing (and configuring) the relevant orcharhino smart proxy features on the existing Salt master host.

For more information on installing a Salt master, please consult the official SaltStack documentation. Alternatively you can simply use the bootstrap instructions found with the official SaltStack package repository.

Note

It is recommended to use the same Salt version on the Salt master as you are using on your Salt minions.

Manual Configuration

The following steps describe how to integrate a Salt master with the orcharhino Salt plugin.

Note

If you select Salt during step 5 of the main orcharhino installation, then these steps are done automatically. The following subsections can be used as a reference for manual installations. This is needed both when adding Salt support to an existing orcharhino installation, as well as when adding an existing Salt master to orcharhino.

Unless otherwise stated, all of the following actions are to be performed on the host that the Salt master is on. This will either be the main orcharhino host, or a host with the relevant orcharhino smart proxy features enabled.

System Configuration

Add the following to the /etc/sudoers:

Cmnd_Alias SALT = /usr/bin/salt, /usr/bin/salt-key
foreman-proxy ALL = NOPASSWD: SALT
Defaults:foreman-proxy !requiretty

Add a user saltuser which will be used for the Salt API:

adduser --no-create-home -s /bin/false -d / saltuser
passwd saltuser
# enter a password for the saltuser twice

Warning

The command adduser saltuser -p <salt_password> won’t work. Using it will prevent you from importing any Salt states later.

Salt Master Configuration

Change the following parts in /etc/salt/master so that Salt will use orcharhino as the external node classifier (ENC):

master_tops:
  ext_nodes: /usr/bin/foreman-node

Enable Salt pillar data for use with orcharhino:

ext_pillar:
  - puppet: /usr/bin/foreman-node

The following construct adds a Salt environment called base which is associated with the /srv/salt directory:

file_roots:
  base:
    - /srv/salt

The autosign mechanism is used to allow Salt minions to connect to the Salt master. For enabling the autosign file which the orcharhino smart proxy will use, change the following line in /etc/salt/master:

autosign_file: /etc/salt/autosign.conf

Add the /etc/salt/autosign.conf file with appropriate ownership and permissions:

touch /etc/salt/autosign.conf
chgrp foreman-proxy /etc/salt/autosign.conf
chmod 660 /etc/salt/autosign.conf

Use the saltuser for the Salt API and specify needed connection settings:

external_auth:
  pam:
    saltuser:
      - '@runner'

rest_cherrypy:
  port: 9191
  host: 0.0.0.0
  ssl_key: /etc/puppetlabs/puppet/ssl/private_keys/orcharhino.<domain>.pem
  ssl_crt: /etc/puppetlabs/puppet/ssl/certs/orcharhino.<domain>.pem

Salt Grains Upload

To enable the upload of Salt grains (analogous to Puppet facts) to orcharhino, you will need to adjust the configuration in /etc/salt/foreman.yaml (on the host that your Salt master is on).

:proto: https
:host: orcharhino.<domain>
:port: 443
:ssl_ca: "/etc/puppetlabs/puppet/ssl/ssl_ca.pem"
:ssl_cert: "/etc/puppetlabs/puppet/ssl/client_cert.pem"
:ssl_key: "/etc/puppetlabs/puppet/ssl/client_key.pem"
:timeout: 10
:salt: /usr/bin/salt
:upload_grains: true

Make sure you adapt this example as appropriate for your installation.

Salt API Configuration

Edit /etc/foreman-proxy/settings.d/salt.yml and configure the API-related settings (make sure to use the correct password for the saltuser):

:use_api: true
:api_auth: pam
:api_url: https://orcharhino.<domain>:9191
:api_username: saltuser
:api_password: <salt_password>

Activation

For the changes to take effect, we need to restart the relevant services. You will need to run the following command on the host that your Salt master is on (either the main orcharhino host, or an orcharhino smart proxy):

systemctl restart salt-master salt-api

You will also need to run the following command on your main orcharhino installation, as well as on any smart proxy based Salt master installation you might have:

foreman-maintain service restart

The first time you install (aka enable) a new smart proxy feature, you will also need to refresh the available smart proxy features from within orcharhino:

In the management UI, go to the smart proxies page (Infrastructure > Smart Proxies) and select the Refresh Action from the drop down menu for the relevant smart proxies.

Minion Setup

Hosts that are to be managed using orcharhino’s Salt features will require the Salt minion client software. The rest of this section provides a best practice recommendation for how to provide such hosts with the needed software, and how to deploy them using orcharhino.

Using orcharhino’s Content Management

Since it is recommended to use the same Salt version on your Salt minions as you are using on your Salt master, we recommend using orcharhino’s content management to provide your hosts with the correct version of this client software.

Note

The following steps are opinionated. They make certain assumptions about your setup and orcharhino usage. Feel free to make adjustments if needed.

The following are step-by-step instructions for how to achieve this:

  1. Create and sync the needed products and repositories.

    1. Create an orcharhino product named SaltStack.

    2. For each supported operating system that you want to install Salt minion client software on, create an orcharhino repository within your new SaltStack product from the previous step.

      Note

      We recommend the following naming schema for these repositories: SaltStack for <os_name> (e.g.: SaltStack for Ubuntu Bionic (18.04)).

      For the Upstream URL for each repository, find the correct SaltStack package repository from the official SaltStack package repository page. Here, correct implies for your desired platform (operating system), as well as in the correct version (the same version used by your Salt master). For example http://repo.saltstack.com/apt/ubuntu/18.04/amd64/2019.2/.

  1. Creating, and publishing the needed content views.

    1. Create an orcharhino content view for each repository from step 1b.

      Note

      We recommend using the same name for the content view, as you did for the relevant repository. Don’t forget to add this repository to the content view.

    2. Publish an initial content view version for each of your content views from the previous step.

  2. Make the new content available, using the relevant composite content views.

    Note

    We recommend creating a single composite content view for each major version of each operating system you are using. That way, all content hosts using a specific major version, of a specific operating system, can all use this content view. Because it is a composite content view, it can be easily extended with additional content, as needed in the future.

    1. Add each of your operating system specific SaltStack content views to the main composite content view for that operating system and version.

    2. Publish a new version of the composite content view from the previous step.

    3. Promote the new content view version from the previous step to your lifecycle environments as appropriate.

    4. If needed, create new activation keys for your composite content view plus lifecycle environment combinations.

Deploying Minion Hosts

Once everything is correctly configured (as laid out in this guide), you can automatically deploy new hosts, that are fully provisioned and configured for Salt usage. All you need to do is to make sure that the Salt Master and Salt Environment fields are set during host creation and that the host will use the content you set up in the previous section.

The preferred way to achieve this is to set up a host group that has these fields set to the desired values, and also uses the correct content view, lifecycle environment, and activation key (as set up in the previous section).

Any new hosts deployed via such a host group, will automatically install and configure the needed Salt minion client software, and will automatically register with the Salt master.

Connection Test

Once you have deployed your first Salt minion, you can verify that the communication between minion and Salt master works using the following command (on your Salt master host):

salt "*" test.ping

Print all grains of all connected Salt minions:

salt "*" grains.items

Usage

As noted above, every orcharhino managed Salt minion host, is always associated with a Salt master, and a Salt environment. The name of the associated Salt environment within orcharhino, must match the actual Salt environment from the file_roots option in the /etc/salt/master file. (This is generally ensured, since the orcharhino environment is chosen from a drop down menu.) Once this association stands (and the Salt minion software is installed), the orcharhino host is also a Salt minion, and by extension ready for Salt usage.

Salt States

A Salt state configures a certain part of a host - e.g. writes the configuration file of a service or installs an application. Salt States can be imported from the Salt master to orcharhino. The Salt master configuration used above uses a Salt environment named base which includes the Salt states stored in /srv/salt/ Navigate to Configure > States > Import in the management UI to import these states. This will display a list of newly found states as follows:

Salt states

Using the Update action on each Salt state, you can change the name of the Salt state and change the Salt environment assignment. To remove a Salt State from orcharhino, use the Delete action. This will only remove the Salt state from orcharhino, not from the disk of the Salt master.

Later on, the states are assigned to a host or to a host group. Salt will apply these states to any hosts they are assigned to whenever state.highstate is run (see Running Salt below).

States assignment to a host / host group

Note

The paths for states and pillars have to be configured in /etc/salt/master. The conventional default is /srv/salt for states and /srv/pillar for pillars.

Salt Keys

The Salt keys interface lists hosts and their Salt keys. You can manually accept, reject, or delete keys from here:

Salt key management

Salt Autosign

The Salt autosign feature can be used to automatically accept signing requests from hosts. This is not normally needed for hosts managed by orcharhino, since such hosts will be supplied with a Salt key during host provisioning. orcharhino is able to manage the Salt autosign entries:

Salt autosign management

Salt Reports

Salt reports are uploaded to orcharhino every 10 minutes via the /etc/cron.d/smart_proxy_salt cron job. This action can also be triggered manually, by running the following command on the host your Salt master is on:

/usr/sbin/upload-salt-reports

Salt reports can be viewed on the configuration management page (navigate to Monitor > Config Management in the management UI). The reports associated with individual hosts, can also be viewed via the relevant host overview page.

ENC Parameters

As mentioned already, orcharhino can be used as an external node classifier (ENC) for Salt. To view what is being made available for Salt usage with some given host, you can press the Salt ENC button on the host overview page of the relevant host. You will see a list of parameters that are made available for Salt usage as pillar data.

You can also check, what parameters are truly available on the Salt side, by running the following command on your Salt master:

salt '*' pillar.items

If you feel some parameter is missing, you may need to refresh the pillar data first:

salt '*' saltutil.refresh_pillar

Running Salt

The Salt plugin, leverages orcharhino’s remote execution features, to allow users to run arbitrary Salt functions.

Most commonly, users will simply want to run salt.highstate on one or more minions. This will apply all relevant Salt states. It can be done via the host overview page of individual minions, by selecting Run Salt from the Schedule Remote Job drop down menu.

Note

Older versions of the Salt plugin provided a Run Salt button that did not use orcharhino’s remote execution features. It is possible to fall back to this old behaviour by changing the salt_hide_run_salt_button setting in the Salt tab of the settings page.

Triggering a remote execution job will take you to a corresponding job overview page:

Run Salt using Schedule Remote Job

If you select a host from the list in the screenshot above, you can see the output of the remote job in real time:

state.highstate output

Remote jobs are recorded on the jobs page, and can be viewed there later.

Running the state.highstate function will also generate a Salt report as detailed above (remember that reports are only uploaded to orcharhino every 10 minutes).

As mentioned already you can use orcharhino’s remote execution features to run arbitrary Salt functions. This includes an optional test mode, that will tell you what would happen without actually changing anything. To use either feature, perform the following steps:

  1. Navigate to Hosts > All Hosts > Your Host > Schedule Remote Job.

  2. For the Job category field, select Salt from the drop down menu.

  3. For the Job template field, select Salt Run function from the drop down menu.

  4. Fill in the function you want to trigger (e.g.: pillar.items or test.ping).

  5. (optional) Click “Display advanced fields” and enable “test-run”.

  6. Run the job by clicking the Submit button.

You can also schedule remote jobs, or let them run recurrently. For more information regarding orcharhino’s remote execution plugin, see the remote execution guide.

Alternatively, you can also achieve recurrent actions using the native Salt way. For example, you can schedule hourly state.highstate runs on individual minions by adding the following to /etc/salt/minion:

schedule:
  highstate:
    function: state.highstate
    minutes: 60

For more information see the Job Management section from the official Salt Stack documentation.

Final Example

This example uses a Salt state to manage the /etc/motd file on one or more Salt minions. In doing so we demonstrate the use of orcharhino as an external node classifier (ENC), as well as the use of Salt grains (which are analogous to Puppet facts).

First we will create an orcharhino global parameter named vendor_name with the string “orcharhino” as its value.

Next, we will add a new Salt state named motd to our Salt master. To do so, we will create the /srv/salt/motd/ directory, and add two files to it.

The /srv/salt/motd/init.sls file, a Salt state file, will contain the following:

/etc/motd:
  file.managed:
    - user: root
    - group: root
    - mode: 0644
    - source: salt://motd/motd.template
    - template: jinja

The /srv/salt/motd/motd.template file, a template referenced from our Salt state file, will contain the following:

Welcome to {{ grains['fqdn'] }} Powered by {{ salt['pillar.get']('vendor_name') }}

As you can see, we are attempting to access the fqdn Salt grain from within this template. We are also attempting to retrieve the vendor_name parameter from the Salt pillar.

Having created the new motd Salt state, we need to import it into orcharhino (see Salt States above).

Next, verify that Salt has been given access to the vendor_name parameter, by running either of the following commands on your Salt master:

salt '*' pillar.items | grep -A 1 vendor_name
salt '*' pillar.get vendor_name

If the output does not include the value of the vendor_name parameter (“orcharhino”), we may need to refresh the pillar data first (see also ENC Parameters above):

salt '*' saltutil.refresh_pillar

You can now add the motd Salt state to your minion host, or a relevant host group, and apply it by running state.highstate (see Running Salt above).

Finally, verify the contents of /etc/motd on any target minions, and our proof of concept example is complete!

Other Resources