Securing orcharhino

orcharhino uses SELinux and firewalld.

Configuring the Firewall on orcharhino

This is not an exhaustive introduction or documentation of firewalld. Please refer to the official firewalld documentation or the firewalld documentation of your operating system vendor before making any changes to your firewall configuration.

orcharhino uses firewalld. It is enabled by default on every Kickstart installation and available on Alma Linux 8, Oracle Linux 8, Red Hat Enterprise Linux 8, and Rocky Linux 8. The firewall daemon can be enabled by running systemctl enable firewalld and started/stopped running systemctl {start|stop} firewalld. Running systemctl status firewalld shows whether the firewall service is currently running; firewall-cmd --help shows the inbuilt documentation. firewalld relies on zones and services:

  • Zones define the level of trust of a connection. One interface, source, or connection belongs to exactly one zone; one zone can be used by many interfaces, sources, or connections. By default, zone example files are located at /usr/lib/firewalld/zones/ and describe an individual zone. firewall-cmd --list-all-zones lists all available zones on your machine. For more information about zones in firewalld, see the official documentation or the firewalld documentation of your operating system vendor.

  • Services bundle and allow traffic regarding certain ports and protocols. By default, SSH, DNS, HTTP, and HTTPS services are added to the public zone. An orcharhino tailor-made service is defined at /usr/lib/firewalld/services/orcharhino.xml and specifies several ports and protocols, for example port 80 with tcp protocol for the orcharhino web server.

    Do not overwrite /usr/lib/firewalld/services/orcharhino.xml as this file is being managed by orcharhino itself and will be overwritten with every upgrade.

Enabling the firewalld Service

You can enable the orcharhino service in firewalld on your orcharhino Server manually:

$ firewall-cmd --permanent --zone=internal --add-service=orcharhino

The internal firewall zone is created by the orcharhino installer. Run firewall-cmd --get-active-zones to see all zones that are currently active. Use --permanent to ensure your change is persistent after rebooting.

Adding a Custom Service

firewall-cmd --zone=internal --list-all lists all services of the internal zone. firewall-cmd --permanent --zone=work --add-service=orcharhino allows you to enable the orcharhino service for the work zone permanently, that is persistent after rebooting. You may specify a network interface with the --add-interface=My_Interface option, for example --add-interface=eth0. Ensure to reload the firewalld configuration afterwards: firewall-cmd --complete-reload. This can be reverted by running firewall-cmd --permanent --zone=work --remove-service=orcharhino. You can also add or disable arbitrary services like HTTP. For more information about services in firewalld, see the official documentation or the firewalld documentation of your operating system vendor.

Opening a Custom Port

You can open specific ports on your orcharhino Server using the following command: firewall-cmd --permanent --remove-port=5000/tcp, for example for Katello for Docker registry. Run firewall-cmd --complete-reload to reload the firewalld configuration. Revert removing a port by using the --add-port=PORT/PROTOCOL option.

Altering the Firewall Configuration

You can change the firewall configuration by adding a custom service. Write a custom service and save it in the /etc/firewalld/services/ directory. Run firewall-cmd --permanent --zone=internal --new-service-from-file=my_service.xml to add your service to the internal zone and reload the firewalld configuration: firewall-cmd --complete-reload.

Ensure to not remove the SSH service unless you have other means of accessing your orcharhino.

Restricting User Access

You can restrict user access to orcharhino using roles and filters.

You can also restrict access using a location and organization context. This limits the user’s access to orcharhino’s resources, for example hosts and job templates.

Dedicated Remote Execution User

If you want to run job templates against your orcharhino Server or orcharhino Proxies, then orcharhino’s remote execution features require SSH access. To make setting up the required SSH access easy, ATIX provides the orcharhino-rex-user RPM package.

Installing the orcharhino-rex-user RPM package allows any sufficiently privileged orcharhino management UI user to run arbitrary commands against the orcharhino server as root. By installing this package, you are opting in to the security implications.

Mitigation strategies include:

  • Restricting job template access for any regular orcharhino users.

  • Adopting more restrictive sudo rules for the or-rex user in /etc/sudoers.d/or_rex_user. Note that future orcharhino updates may revert your local changes.

  • Uninstalling the orcharhino-rex-user package after use. This will completely revert the changes introduced by installing the package:

    $ dnf remove orcharhino-rex-user

Install the package to set up the or-rex linux system user, along with sudo rules, and the required authorized_keys file:

$ dnf install orcharhino-rex-user

You will also need to set a host parameter for orcharhino’s remote execution features to use the new user:

  1. Navigate to All Hosts > Hosts.

  2. Select the host you have installed the user on, for example orcharhino.example.com or orcharhino-proxy.network2.example.com.

  3. Click the Edit button.

  4. On the Parameters tab, add the host parameter remote_execution_ssh_user of type string and with value or-rex.

Your sufficiently privileged orcharhino users can now run job templates against the relevant target host.

Running Remote Execution Jobs on orcharhino Server

When you execute a remote job targeting your orcharhino Server, ensure your orcharhino Server is in a subnet with the correct smart proxy set for remote execution. This prevents remote execution jobs that you run on your orcharhino Server itself to try and use every other orcharhino Proxy first. It saves a lot of time, especially in scenarios with multiple orcharhino Proxies, as you no longer have to wait until the remote job has ran into the SSH timeout for each orcharhino Proxy.

Procedure

  1. Navigate to Hosts > All Hosts and select your orcharhino Server.

  2. On the Interfaces tab, click Edit.

  3. Select a subnet in the IPv4 Subnet drop down menu.

  4. Navigate to Infrastructure > Subnets and select the subnet of your orcharhino Server.

  5. On the Remote Execution tab, select your orcharhino Server.

  6. Amend the public SSH key of the foreman-proxy user of the orcharhino Server to the list of authorized keys of the root user of the orcharhino Server:

    $ cat /var/lib/foreman-proxy/ssh/id_rsa_foreman_proxy.pub >> /root/.ssh/authorized_keys

    This allows for the execution of arbitrary commands as root user on your orcharhino Server for all orcharhino users that can run job templates.

  7. Ensure to remove the public SSH key from the /root/.ssh/authorized_keys file after running the job template.

    $ sed -i /foreman-proxy@/d /root/.ssh/authorized_keys

The following procedure allows you to run remote execution on hosts without a subnet configuration.

Procedure

  1. Navigate to Administer > Settings.

  2. On the RemoteExecution tab, ensure Enable Global Proxy and Fallback to Any Proxy are set to their default values.

    If you require a fallback to any orcharhino Proxy, remote execution jobs try every smart proxy with remote execution enabled. For example, if you have ten orcharhino Proxies, remote execution tries to connect with all ten orcharhino Proxies and only after all ten orcharhino Proxies returned whether they can connect to the target host, it selects an orcharhino Proxy that works and starts the job. This implies that the remote execution job has to wait for each unsuccessful connection to run into a timeout.

Restricting Access to Job Templates

Job templates can be used to execute arbitrary commands on any system that accepts the SSH key used by orcharhino’s remote execution features. By default, this will include all managed systems as many orcharhino features depend on it. In addition, ATIX provides job templates intended to target the orcharhino host itself to automate orcharhino administration.

As a result, it is highly recommend to limit job template access for standard users. You can disable job template access completely by ensuring your standard users do not have access to either the Remote Execution Manager or the Remote Execution User roles. Since users will often need access to at least some job templates, you can also use a combination of the Remote Execution User role along with filters to limit users to specific templates and/or specific hosts, allowing for almost arbitrarily fine grained permissions management.

The example below uses a filter to exclude users from using the Run Command - SSH Default and Run Command - Ansible Default templates. These templates are particularly versatile, since even users that may not edit any job templates (Remote Execution Manager role) can use them to run arbitrary commands with job parameters.

Procedure

  1. Navigate to Administer > Roles and select Clone from the drop down menu of the Remote Execution User role. Select a new name of the role and save it to orcharhino.

  2. Edit this role and adjust its filters on the Filters tab:

    • Edit the Job invocation resource:

      Set the permission from unlimited to host.name != orcharhino.example.com and host.name != orcharhino-proxy.network2.example.com. This will limit job invocations to not use the orcharhino or orcharhino proxy as target host.

    • Edit the Template invocation resource:

      Set the permission from unlimited to job_category != Commands and job_category != "Ansible Commands".

    • Ensure your limited user has access to your cloned Remote Execution User role only.

Using a Custom User for Remote Execution

You can use a custom user for remote execution using SSH or Ansible. By default, orcharhino uses the root user to connect to managed hosts.

Procedure

  1. Navigate to Configure > Host Groups and select a host group.

    Alternatively, navigate to Configure > Global Parameters and click Create Parameter to enable a custom user for all newly provisioned hosts.

  2. On the Parameters tab, click Add Parameter.

  3. In the Name field, enter remote_execution_create_user.

  4. From the Type drop down menu, select boolean.

  5. In the Value field, enter true.

  6. Add a second parameter called remote_execution_ssh_user of type string and with value My_Custom_SSH_User_Name.

  7. Optional: Add a third parameter called remote_execution_ssh_password of type string and with value My_Custom_SSH_Password.

    Only set a password for your custom REX user if you plan to login yourself. orcharhino uses key-based authentication for SSH.

  8. Click Submit to add the parameter.

Your custom user can now use sudo to run commands as root user without password query.

If you use bootstrap.py to register hosts to orcharhino, ensure the user specified using --rex-user already exists on your host. The bootstrap.py script does not create this user. For more information, run ./bootstrap.py --help.

Changing the default SSH Key for Remote Execution

By default, each Smart Proxy, that is, orcharhino Server and orcharhino Proxies, creates its own RSA SSH key pair. You can find the SSH key on your orcharhino Server or orcharhino Proxy under /var/lib/foreman-proxy/ssh/ that gets deployed to newly provisioned hosts by default according to your provisioning templates. You can change the default SSH key, for example if you do not want to use RSA SSH keys.

Procedure on orcharhino Server

  1. Copy the SSH key pair to your orcharhino Server:

    $ scp My_SSH_Key root@orcharhino.example.com:/var/lib/foreman-proxy/ssh/
$ scp My_SSH_Key.pub root@orcharhino.example.com:/var/lib/foreman-proxy/ssh/

  2. Set the path to your SSH private key:

    $ sed -i "s/^  ssh_identity_file: .*$/  ssh_identity_file: My_SSH_Key/g" /etc/foreman-installer/scenarios.d/katello-answers.yaml

  3. Optional: Verify orcharhino Server uses your custom SSH key:

    $ cat /etc/foreman-installer/scenarios.d/katello-answers.yaml | grep "ssh_identity_file"

  4. Run orcharhino-installer:

    $ orcharhino-installer
Procedure on orcharhino Proxies

  1. Copy the SSH key pair to your orcharhino Proxy:

    $ scp My_SSH_Key root@orcharhino-proxy.network2.example.com:/var/lib/foreman-proxy/ssh/
$ scp My_SSH_Key.pub root@orcharhino-proxy.network2.example.com:/var/lib/foreman-proxy/ssh/

  2. Set the path to your SSH private key:

    $ sed -i "s/^  ssh_identity_file: .*$/  ssh_identity_file: My_SSH_Key/g" /etc/foreman-installer/scenarios.d/foreman-proxy-content-answers.yaml

  3. Optional: Verify orcharhino Proxy uses your custom SSH key:

    $ cat /etc/foreman-installer/scenarios.d/foreman-proxy-content-answers.yaml | grep "ssh_identity_file"

  4. Run orcharhino-installer:

    $ orcharhino-installer

  5. Navigate to Infrastructure > Smart Proxies.

  6. In the Actions column, select your orcharhino Proxy and click Refresh.