Pacemaker 2.1 Changes

Pacemaker 2.1 is in development with an expected release in mid-2021. Item marked "proposed" here may or may not end up in the final release.

Pacemaker 2.1 will be a minor Pacemaker release that preserves backward compatibility for both rolling upgrades and the C API, which means there should be no major concerns for individuals or distributions who upgrade. The minor version bump is intended to draw attention to larger changes than usual, including changes that may affect any custom scripts users have written.

As with Pacemaker 2.0, Corosync 2 or later is supported as the underlying cluster layer, and rolling upgrades from Pacemaker 1.1.11 or later are possible as long as the cluster layer stays at the same Corosync major version (2 or 3).

Since backward compatibility will be maintained, there will be no need to backport future changes to the 2.0 series, and there will be no more 2.0 series releases.

Build process changes

  • Raise minimum dependency versions required to build Pacemaker to these versions (all released 6 or more years ago and available in recent and even many not-so-recent versions of all major Linux distributions):
    • Python 3.2 or later (support for Python 2.7 has been removed)
    • glib 2.32.0 or later
    • libqb 0.17.0 or later
    • GnuTLS 2.12.0 or later (to enable Pacemaker Remote support)
    • rpm 4.11.0 or later (if building RPMs)
    • C library must provide setenv() and unsetenv()
  • configure script:
    • Change the default for --enable-legacy-links from yes to no (which means that symbolic links to the pacemaker 1.1 daemon names, such as crmd instead of pacemaker-controld, will no longer be installed by default). However, a link to the old pacemaker_remoted name will now always be created, regardless of the value of this option, to ensure that container images with Pacemaker 2 can be used in bundles on cluster nodes running Pacemaker 1.
    • The configure script already supports a --with-runstatedir option that is ignored if the installed autoconf version supports the --runstatedir option. Until recently, only Debian patched its autoconf package to support --runstatedir, but autoconf 2.70 was released in December 2020 with support, so as time goes on, more builds will need to use the standard --runstatedir instead of --with-runstatedir.
    • Remove support for the deprecated --pkgname and --pkg-name options, which have no effect
    • Remove support for the --with-brand option, which was only needed for Publican (see "Documentation changes" below)
    • Remove support for the --enable-ansi option, which hasn't worked in a very long time
    • Remove support for the --enable-no-stack option, which builds only the scheduler
  • Move version.m4 to the m4 subdirectory, and move some build side effects to the libltdl subdirectory. This is uninteresting unless you update an existing git checkout and want to rebuild, in which case you first need to run: rm -rf aclocal.m4 autom4te.cache config.guess config.log config.status config.sub configure depcomp install-sh libltdl libtool ltmain.sh missing py-compile test-driver; ./autogen.sh
  • Remove support for the deprecated and undocumented SUSE_ACL_COMPAT compile-time option. (Official SUSE builds have never used this option.)
  • Remove the clustermon.sh script, which was the pre-1.1.15 way of doing alerts.
  • Remove the nonfunctional ocf:pacemaker:pingd resource agent (the separate and widely used ocf:pacemaker:ping agent will remain).
  • Remove the cibmon utility, which was a no-longer-useful holdover from the heartbeat days typically installed in /usr/libexec/pacemaker.
  • Make "subtree" the default for RPMDEST in GNUmakefile, which means that the "make rpm" target will put all artifacts in a dedicated "rpm/" subdirectory rather than a mix of the top-level directory and system defaults. You can run "make RPMDEST=toplevel rpm" to get the previous behavior, though that will be considered deprecated and eventually removed.
  • (proposed) The pacemaker-cts package (containing the Cluster Test Suite) will likely be split into two packages, pacemaker-cts-exerciser for what's needed on the CTS exerciser host, and pacemaker-cts-node for what's needed on the cluster nodes, so the dependencies can be better separated. The pacemaker-cts package will remain as a dummy package depending on the other two, for backward compatibility with current workflows.

Logging changes

  • If Pacemaker is built with libqb 2.0 or later, the detail log will use millisecond-resolution timestamps. (If for some reason this causes a problem for some users, we could potentially make this behavior optional.)

Cluster Information Base (CIB) changes

  • (proposed) The concurrent-fencing cluster option might change its default from false to true, and the option might be deprecated (and removed in a future release, acting as always true).
  • (proposed) The undocumented can_fail operation meta-attribute (which serves no useful purpose) will be officially deprecated (and removed in a future version).
  • (proposed) Pacemaker will stop adding the last-run attribute to <lrm_rsc_op> entries in the <status> section. (This attribute hasn't been added to recurring operations since version 1.1.5, and is always identical to last-rc-change for non-recurring operations.) As a result, last-run will no longer appear in crm_mon XML output either.
  • (proposed) The obscure and useless remove-after-stop cluster option will be officially deprecated, with a warning logged if it is changed from its default. It will be removed in a future version.

Tool changes

All

  • More commands support the --output-as and --output-to options, including XML output. These include crmadmin and crm_resource (in addition to crm_mon and stonith_admin, which already did). The intent is that the XML output will remain stable across releases as much as possible, for reliable parsing by scripts, while the text output may change from release to release for human-friendliness.
  • Some tools currently have (or had) undocumented support for long option abbreviation; for example, if the long option is "--upgrade", then any shorter abbreviation ("--upgr", "--up", etc.) would also work as long as it is unique for the tool. This support has been removed from many tools and is now considered deprecated for remaining tools, for which it will be dropped in future releases.
  • Most short options that take arguments currently make the space after the short option optional, for example "-X saved.xml" or "-Xsaved.xml". Specifying a short option without a space is now considered deprecated; a future release will print a warning when this is used, and an even later release will drop support.

crm_mon

  • (proposed) The --simple-status/-s option (generating nagios plugin-style output) will likely be deprecated. Nagios/icinga/etc. users are recommended to use a monitoring community-supplied plug-in instead.
  • (proposed) The deprecated --as-html, --as-xml, --web-cgi, and --disable-ncurses options might print a deprecation warning when used (and showing the currently supported equivalents). crm_mon.rng, the XML schema corresponding to --as-xml, would also be deprecated.

crm_resource

  • The -a/--constraints and -A/--stack options (to display location and colocation constraints related to a given resource) now have considerably different (and hopefully more readable) text output.
  • (proposed) The crm_resource --xml-file option might be deprecated (setting CIB_file in the environment is the preferred means of accomplishing the same thing)

Other

  • (proposed) OCF resource agents will be displayed as ocf:PROVIDER:AGENT rather than ocf::PROVIDER:AGENT in crm_mon and crm_simulate output and pacemaker-schedulerd logs.
  • (proposed) crm_simulate will no longer automatically redirect stderr to stdout when --verbose is given
  • (proposed) The nonfunctional crm_report -s option (which exists only for compatibility with the ancient hb_report tool) and its -x option (which turns on line-by-line debugging) will likely be deprecated, and removed in a future release.
  • (proposed) crmadmin, when given the -q/--quiet option, currently prints its usual output to stdout and a briefer version to stderr. For consistency with other tools, crmadmin --quiet will now print only the brief version, to stdout.
  • (proposed) The use of certain undocumented and deprecated command-line options, such as cibadmin --host, may now print a deprecation warning (with the currently preferred alternative when possible).
  • The output of the pacemakerd --features command better reflects the build options currently available. (Feature listings are not guaranteed to stay the same from release to release, and are intended only for troubleshooting.)

Documentation changes

The source for the Pacemaker "books" (Clusters from Scratch, Pacemaker Explained, etc.) has been switched from using asciidoc source and Publican to using reStructuredText source and Sphinx. This is mainly because Publican is a dead project and distributions are dropping it, but it will also give us a cleaner, more modern look.

The --with-brand configure option has been removed. The books will now be built based on whether the sphinx-build and inkscape commands are available at build time.

The --with doc RPM build option remains, to decide whether to enable python3-sphinx and inkscape as build requirements for the pacemaker-doc subpackage (effectively controlling whether the books are built or not). The previous build requirement for publican has been removed. (The build requirement of asciidoc or asciidoctor remains, for generating the deprecated text file documentation, but that may change.)

The directory structure for the generated books has changed:

Generated content 2.0 location 2.1 location
Books built locally (in source directory) doc/TITLE/publish/FORMAT/en-US/ doc/sphinx/TITLE/_build/FORMAT/
Installed books (in package documentation directory a.k.a. docdir) TITLE/FORMAT/en-US/ TITLE/FORMAT/
Uploaded books (in RSYNC_DEST directory) pacemaker/doc/en-US/Pacemaker/2.0/FORMAT/TITLE/ pacemaker/doc/2.1/TITLE/FORMAT

New role terminology

(proposed)

With Pacemaker 2.0.0, we renamed "master/slave resources" to "promotable clone resources". However, the role names themselves were left as "Master" and "Slave" due to the difficulty of changing all the code in time. With Pacemaker 2.1.0, we hopefully will be able to change these to "promoted" and "unpromoted".

For the most part, we can remain backward compatible:

  • The configuration will accept either the old names or new names
  • Resource agents that support the notify action will be provided notification environment variables with both old and new names (for example, OCF_RESKEY_CRM_meta_notify_promoted_resource and OCF_RESKEY_CRM_meta_notify_master_resource -- though the exact new names have not yet been decided)

There will be some places where only the new names will be provided:

  • Status output (including crm_mon XML output)
  • Log messages
  • Documentation

The C API will continue to use the old terminology in some places, since we are not breaking API backward compatibility with this release. Those instances will likely be deprecated with 2.1.0 and changed with the next compatibility break.

New Features

Noncritical resources

Colocation constraints may take a new "influence" option that determines whether the dependent resource influences the location of the main resource, if the main resource is already active. The default of true preserves the previous behavior. Setting it to false has the effect of making the dependent resource stop if it reaches its migration-threshold in failures, rather than cause the main resource to move to another node.

Resources may take a new "critical" meta-attribute that serves as a default for all colocation constraints involving the resource as the dependent, as well as all groups involving the resource.

As an example, a large, important database that takes a long time to start and stop may have a less important, occasionally used reporting tool colocated or grouped with it. In this situation, it might be preferable to stop the reporting tool if it fails too many times, rather than cause a database outage to move both resources. In this case, either the colocation constraint could be marked with influence=false, or the reporting tool resource could be give the meta-attribute critical=false, to achieve the desired effect.

Development changes

If you make changes to the Pacemaker source code, or build from the master branch, this could affect you.

  • (proposed) The cts source subdirectory might be renamed "tests", since it now contains more than just the CTS code.
  • (proposed, but unlikely) We might rename the "master" branch of the git repository to something like "main" or "next". Anyone who has an existing fork and/or checkout of the repository would have a dangling reference to the old name. If you have a fork, you can do this from your checkout of your fork:
git branch -m master main
git push -u origin main
# Now go to the github page for your fork, select Settings then Branches,
# and change the default branch to main. Then continue below ...
git push origin --delete master
git remote set-head origin -a

If you have more than one checkout of your fork, repeat the first and last commands from the above in each one. If you only have checkouts (not a fork) of the upstream repository, you can update each one like:

git fetch --all
git remote set-head origin -a
git branch --set-upstream-to origin/main
git branch -m master main