Pacemaker feature set

From ClusterLabs

Besides a release version, Pacemaker has a separate "CRM feature set". This version number applies to the communication between cluster nodes, and is used to avoid problems in mixed-version clusters.

Most of the time, CRM feature sets are for Pacemaker's internal use. However, they can also be used by resource agents and other external code to detect support for various features. The table below lists those features.

Pacemaker CRM Feature Sets
CRM feature set Available in Pacemaker release Features
3.10.2 unreleased crm_resource --validate and --force-check arguments accept an optional argument for OCF check level
3.10.1 unreleased pacemakerd accepts --output-as (text, none, or xml) and --output-to options, mainly for use with the --features argument
3.10.0 2.1.0 Pacemaker supports OCF resource agents' reload-agent action
3.9.0 2.1.0 "Promoted" and "Unpromoted" are accepted as role names; crm_attribute accepts the --promotion option; crm_resource accepts the --promoted option; promotable clone notify actions will have notify_promoted_resource, notify_unpromoted_resource, notify_promoted_uname, and notify_unpromoted_uname environment variables; crm_mon XML output for bans includes a "promoted-only" attribute
3.8.0 2.1.0 crm_simulate accepts --show-attrs and --show-failcounts options
3.7.4 2.1.0 crm_simulate accepts --output-as (text, none, or xml) and --output-to options, and its output is formatted more similarly to crm_mon's
3.7.3 2.1.0 CIB_file="-" for standard input is supported by tools that support CIB_file
3.7.2 2.1.0 crm_verify accepts --output-as (text, none, or xml) and --output-to options; crmadmin drops -E/--election and -K/--kill options
3.7.1 2.1.0 support for node type argument to crmadmin --nodes
3.7.0 2.1.0 support for "critical" resource meta-attribute and "influence" colocation option
3.6.4 2.1.0 crm_resource offers --digests option
3.6.3 2.1.0 crm_resource accepts --output-as (text or xml) and --output-to
3.6.2 2.1.0 crmadmin accepts --output-as (text or xml) and --output-to
3.6.1 2.0.5 cibsecret returns standard Pacemaker exit status codes
3.6.0 2.0.5
3.5.0 2.0.5 "integer" supported as value for "type" in rules, and "number" compares as double-precision floating-point value
3.4.1 2.0.5 crmadmin supports -P/--pacemakerd and -i/--ipc-name options
3.4.0 2.0.5 op_expression and rsc_expression supported in rules; "demote" supported as value for "no-quorum-policy" and "on-fail" options; priority-fencing-delay cluster option supported; stonith_admin supports --delay option; crm_mon supports --node and --resource options; crm_rule --check can now check "date_spec" entries that contain "years" but not "moon"
3.3.0 2.0.4 fence-reaction, shutdown-lock, and shutdown-lock-limit cluster options supported; crm_mon supports --include and --exclude options
3.2.0 2.0.3 new tool crm_rule; crm_mon and stonith_admin support --output-as and --output-to options
3.1.0 2.0.0

(Note: in some cases, the specified features were available in the prior Pacemaker release, but the listed feature set version is the first that includes them.)

How CRM feature sets are numbered

The major version increases when nodes with different versions would not work together. Rolling upgrades would not be possible in this case.

The minor version increases when mixed-version clusters are allowed only during rolling upgrades. If one of the older nodes drops out of the cluster for any reason, it will not be able to rejoin until it is upgraded. In this case, you should run a mixed-version cluster only during a small upgrade window.

The minor-minor version number is ignored by Pacemaker. Clusters with different versions should work without problems. This number allows resource agents and other external code to detect cluster support for various features.

Pacemaker ensures that the longest-running node is the cluster’s DC. This ensures new features are not enabled until all nodes are upgraded to support them.