Removing orphaned content

If you synchronize content to orcharhino Server or orcharhino Proxy Servers and no longer wish to distribute it to hosts, you can get rid of content on orcharhino to free up disk space.

Orphaned content: Orphaned content is content that is not referenced in any published content view version and older in minutes than the Orphaned Content Protection Time setting. By default, orcharhino removes orphaned content once a week as defined in /etc/cron.d/katello.

Configuring custom cron jobs on orcharhino

You can set up a cron job on your orcharhino Server to remove orphaned content regularly.

Procedure

On your orcharhino Server, duplicate the existing cron job:

# cat /etc/cron.d/katello | sed "/alternate.content/d" > /etc/cron.d/custom-orphan-cleanup

Adjust the interval:
```
# vim /etc/cron.d/custom-orphan-cleanup
```
For more information, see crontab.guru or man cron.

Recovering from a full disk

The following procedure describes how to resolve the situation when a logical volume (LV) with the Pulp database on it has no free space.

Procedure

Let running Pulp tasks finish but do not trigger any new ones as they can fail due to the full disk.

Ensure that the LV with the /var/lib/pulp directory on it has sufficient free space. Here are some ways to achieve that:

Remove orphaned content:
```
# foreman-rake katello:delete_orphaned_content RAILS_ENV=production
```
This is run weekly so it will not free much space.
Change the download policy from Immediate to On Demand for as many repositories as possible and remove already downloaded packages.

Grow the file system on the LV with the /var/lib/pulp directory on it.

If you use an untypical file system (other than for example ext3, ext4, or xfs), you might need to unmount the file system so that it is not in use. In that case, complete the following steps:

Stop orcharhino services:
```
# orcharhino-maintain service stop
```
Grow the file system on the LV.
Start orcharhino services:
```
# orcharhino-maintain service start
```

If some Pulp tasks failed due to the full disk, run them again.

Reclaiming PostgreSQL space

The PostgreSQL database can use a large amount of disk space especially in heavily loaded deployments. Use this procedure to reclaim some of this disk space on orcharhino.

Procedure

Stop all services, except for the postgresql service:

# orcharhino-maintain service stop --exclude postgresql

Switch to the postgres user and reclaim space on the database:
```
# su - postgres -c 'vacuumdb --full --all'
```
Start the other services when the vacuum completes:
```
# orcharhino-maintain service start
```

Content settings

Setting Default Value Description

Setting	Default Value	Description
Default HTTP Proxy		Default HTTP Proxy for syncing content.
CDN SSL version		SSL version used to communicate with the CDN.
Default synced OS provisioning template	Kickstart default	Default provisioning template for operating systems created from synced content.
Default synced OS finish template	Kickstart default finish	Default finish template for new operating systems created from synced content.
Default synced OS user-data	Kickstart default user data	Default user data for new operating systems created from synced content.
Default synced OS PXELinux template	Kickstart default PXELinux	Default PXELinux template for new operating systems created from synced content.
Default synced OS PXEGrub template	Kickstart default PXEGrub	Default PXEGrub template for new operating systems created from synced content.
Default synced OS PXEGrub2 template	Kickstart default PXEGrub2	Default PXEGrub2 template for new operating systems created from synced content.
Default synced OS iPXE template	Kickstart default iPXE	Default iPXE template for new operating systems created from synced content.
Default synced OS partition table	Kickstart default	Default partitioning table for new operating systems created from synced content.
Default synced OS kexec template	Discovery ATIX AG kexec	Default kexec template for new operating systems created from synced content.
Default synced OS Atomic template	Atomic Kickstart default	Default provisioning template for new atomic operating systems created from synced content.
Manifest refresh timeout	1200	Timeout when refreshing a manifest (in seconds).
Subscription connection enabled	Yes	Can communicate with the ATIX AG Portal for subscriptions.
Installable errata from Content View	No	Calculate errata host status based only on errata in a host’s content view and lifecycle environment.
Restrict Composite Content View promotion	No	If this is enabled, a composite content view cannot be published or promoted, unless the content view versions that it includes exist in the target environment.
Check services before actions	Yes	Check the status of backend services such as pulp and candlepin before performing actions?
Batch size to sync repositories in	100	How many repositories should be synced concurrently on a orcharhino Proxy. A smaller number may lead to longer sync times. A larger number will increase dynflow load.
Sync orcharhino Proxies after Content View promotion	Yes	Whether or not to auto sync orcharhino Proxies after a content view promotion.
Default Custom Repository download policy	`immediate`	Default download policy for custom repositories. Either `immediate` or `on_demand`.
Default ATIX AG Repository download policy	`on_demand`	Default download policy for enabled ATIX AG repositories. Either `immediate` or `on_demand`.
Default orcharhino Proxy download policy	`on_demand`	Default download policy for orcharhino Proxy syncs. Either `inherit`, `immediate`, or `on_demand`.
Pulp export destination filepath	`/var/lib/pulp/katello-export`	On-disk location for exported repositories.
Pulp 3 export destination filepath	`/var/lib/pulp/exports`	On-disk location for Pulp 3 exported repositories.
Pulp client key	`/etc/pki/katello/private/pulp-client.key`	Path for SSL key used for Pulp server authentication.
Pulp client cert	`/etc/pki/katello/certs/pulp-client.crt`	Path for SSL certificate used for Pulp server authentication.
Sync Connection Timeout	300	Total timeout in seconds for connections when syncing.
Delete Host upon unregister	No	When unregistering a host using subscription-manager, also delete the host record. Managed resources linked to the host such as virtual machines and DNS records might also be deleted.
Subscription manager name registration fact		When registering a host using subscription-manager, force use the specified fact for the host name (in the form of `fact.fact`).
Subscription manager name registration fact strict matching	No	If this is enabled, and `register_hostname_fact` is set and provided, registration looks for a new host by name only using that fact, and skips all hostname matching.
Default Location subscribed hosts	Default Location	Default location where new subscribed hosts are stored after registration.
Expire soon days	120	The number of days remaining in a subscription before you are reminded about renewing it.
Content View Dependency Solving Default	No	The default dependency solving value for new content views.
Host Duplicate DMI UUIDs	[]	If hosts fail to register because of duplicate Desktop Management Interface (DMI) UUIDs, add their comma-separated values here. Subsequent registrations generate a unique DMI UUID for the affected hosts.
Host Profile Assume	Yes	Enable new host registrations to assume registered profiles with matching hostname if the registering DMI UUID is not used by another host.
Host Profile Can Change In Build	No	Enable host registrations to bypass Host Profile Assume if the host is in build mode.
Host Can Re-Register Only In Build	No	Enable hosts to re-register only when they are in build mode.
Host Tasks Workers Pool Size	5	Number of workers in the pool to handle the execution of host-related tasks. When set to 0, the default queue is used. Restart of the dynflowd/foreman-tasks service is required.
Applicability Batch Size	50	Number of host applicability calculations to process per task.
Autosearch	Yes	For pages that support it, automatically perform the search while typing in search input.
Autosearch delay	500	If Autosearch is enabled, delay in milliseconds before executing searches while typing.
Pulp bulk load size	2000	The number of items fetched from a single paged Pulp API call.
Upload profiles without Dynflow	Yes	Enable Katello to update host installed packages, enabled repositories, and module inventory directly instead of wrapped in Dynflow tasks (try turning off if Puma processes are using too much memory).
Orphaned Content Protection Time	1440	Time in minutes to consider orphan content as orphaned.
Prefer registered through proxy for remote execution	No	Prefer using a proxy to which a host is registered when using remote execution.
Allow deleting repositories in published content views	Yes	Enable removal of repositories that the user has previously published in one or more content view versions.

Default HTTP Proxy

Default HTTP Proxy for syncing content.

CDN SSL version

SSL version used to communicate with the CDN.

Default synced OS provisioning template

Kickstart default

Default provisioning template for operating systems created from synced content.

Default synced OS finish template

Kickstart default finish

Default finish template for new operating systems created from synced content.

Default synced OS user-data

Kickstart default user data

Default user data for new operating systems created from synced content.

Default synced OS PXELinux template

Kickstart default PXELinux

Default PXELinux template for new operating systems created from synced content.

Default synced OS PXEGrub template

Kickstart default PXEGrub

Default PXEGrub template for new operating systems created from synced content.

Default synced OS PXEGrub2 template

Kickstart default PXEGrub2

Default PXEGrub2 template for new operating systems created from synced content.

Default synced OS iPXE template

Kickstart default iPXE

Default iPXE template for new operating systems created from synced content.

Default synced OS partition table

Kickstart default

Default partitioning table for new operating systems created from synced content.

Default synced OS kexec template

Discovery ATIX AG kexec

Default kexec template for new operating systems created from synced content.

Default synced OS Atomic template

Atomic Kickstart default

Default provisioning template for new atomic operating systems created from synced content.

Manifest refresh timeout

1200

Timeout when refreshing a manifest (in seconds).

Subscription connection enabled

Yes

Can communicate with the ATIX AG Portal for subscriptions.

Installable errata from Content View

Calculate errata host status based only on errata in a host’s content view and lifecycle environment.

Restrict Composite Content View promotion

If this is enabled, a composite content view cannot be published or promoted, unless the content view versions that it includes exist in the target environment.

Check services before actions

Yes

Check the status of backend services such as pulp and candlepin before performing actions?

Batch size to sync repositories in

100

How many repositories should be synced concurrently on a orcharhino Proxy. A smaller number may lead to longer sync times. A larger number will increase dynflow load.

Sync orcharhino Proxies after Content View promotion

Yes

Whether or not to auto sync orcharhino Proxies after a content view promotion.

Default Custom Repository download policy

immediate

Default download policy for custom repositories. Either immediate or on_demand.

Default ATIX AG Repository download policy

on_demand

Default download policy for enabled ATIX AG repositories. Either immediate or on_demand.

Default orcharhino Proxy download policy

on_demand

Default download policy for orcharhino Proxy syncs. Either inherit, immediate, or on_demand.

Pulp export destination filepath

/var/lib/pulp/katello-export

On-disk location for exported repositories.

Pulp 3 export destination filepath

/var/lib/pulp/exports

On-disk location for Pulp 3 exported repositories.

Pulp client key

/etc/pki/katello/private/pulp-client.key

Path for SSL key used for Pulp server authentication.

Pulp client cert

/etc/pki/katello/certs/pulp-client.crt

Path for SSL certificate used for Pulp server authentication.

Sync Connection Timeout

300

Total timeout in seconds for connections when syncing.

Delete Host upon unregister

When unregistering a host using subscription-manager, also delete the host record. Managed resources linked to the host such as virtual machines and DNS records might also be deleted.

Subscription manager name registration fact

When registering a host using subscription-manager, force use the specified fact for the host name (in the form of fact.fact).

Subscription manager name registration fact strict matching

If this is enabled, and register_hostname_fact is set and provided, registration looks for a new host by name only using that fact, and skips all hostname matching.

Default Location subscribed hosts

Default Location

Default location where new subscribed hosts are stored after registration.

Expire soon days

120

The number of days remaining in a subscription before you are reminded about renewing it.

Content View Dependency Solving Default

The default dependency solving value for new content views.

Host Duplicate DMI UUIDs

[]

If hosts fail to register because of duplicate Desktop Management Interface (DMI) UUIDs, add their comma-separated values here. Subsequent registrations generate a unique DMI UUID for the affected hosts.

Host Profile Assume

Yes

Enable new host registrations to assume registered profiles with matching hostname if the registering DMI UUID is not used by another host.

Host Profile Can Change In Build

Enable host registrations to bypass Host Profile Assume if the host is in build mode.

Host Can Re-Register Only In Build

Enable hosts to re-register only when they are in build mode.

Host Tasks Workers Pool Size

Number of workers in the pool to handle the execution of host-related tasks. When set to 0, the default queue is used. Restart of the dynflowd/foreman-tasks service is required.

Applicability Batch Size

Number of host applicability calculations to process per task.

Autosearch

Yes

For pages that support it, automatically perform the search while typing in search input.

Autosearch delay

500

If Autosearch is enabled, delay in milliseconds before executing searches while typing.

Pulp bulk load size

2000

The number of items fetched from a single paged Pulp API call.

Upload profiles without Dynflow

Yes

Enable Katello to update host installed packages, enabled repositories, and module inventory directly instead of wrapped in Dynflow tasks (try turning off if Puma processes are using too much memory).

Orphaned Content Protection Time

1440

Time in minutes to consider orphan content as orphaned.

Prefer registered through proxy for remote execution

Prefer using a proxy to which a host is registered when using remote execution.

Allow deleting repositories in published content views

Yes

Enable removal of repositories that the user has previously published in one or more content view versions.

PostgreSQL tuning

PostgreSQL is the primary SQL based database that is used by orcharhino for the storage of persistent context across a wide variety of tasks that orcharhino does. The database sees an extensive usage is usually working on to provide the orcharhino with the data which it needs for its smooth functioning. This makes PostgreSQL a heavily used process which if tuned can have a number of benefits on the overall operational response of orcharhino.

The PostgreSQL authors recommend disabling Transparent Hugepage on servers running PostgreSQL. For more information, see Disable Transparent Hugepage.

You can apply a set of tunings to PostgreSQL to improve its response times, which will modify the postgresql.conf file.

Procedure

Append /etc/foreman-installer/custom-hiera.yaml to tune PostgreSQL:
```
postgresql::server::config_entries:
  max_connections: 1000
  shared_buffers: 2GB
  work_mem: 8MB
  autovacuum_vacuum_cost_limit: 2000
```
You can use this to effectively tune down your orcharhino instance irrespective of a tuning profile.
Apply your changes to orcharhino Server. For more information, see Applying configurations.

In the above tuning configuration, there are a certain set of keys which we have altered:

max_connections: The key defines the maximum number of connections that can be accepted by the PostgreSQL processes that are running.
shared_buffers: The shared buffers define the memory used by all the active connections inside PostgreSQL to store the data for the different database operations. An optimal value for this will vary between 2 GiB to a maximum of 25% of your total system memory depending upon the frequency of the operations being conducted on orcharhino.
work_mem: The work_mem is the memory that is allocated on per process basis for PostgreSQL and is used to store the intermediate results of the operations that are being performed by the process. Setting this value to 8 MB should be more than enough for most of the intensive operations on orcharhino.
autovacuum_vacuum_cost_limit: The key defines the cost limit value for the vacuuming operation inside the autovacuum process to clean up the dead tuples inside the database relations. The cost limit defines the number of tuples that can be processed in a single run by the process. ATIX AG recommends setting the value to 2000 as it is for the medium, large, extra-large, and extra-extra-large profiles, based on the general load that orcharhino pushes on the PostgreSQL server process.

The text and illustrations on this page are licensed by ATIX AG under a Creative Commons Attribution Share Alike 4.0 International ("CC BY-SA 4.0") license. This page also contains text from the official Foreman documentation which uses the same license ("CC BY-SA 4.0").