Advanced Node Failure Detection

The Advanced node failure detection function can reduce the number of failure scenarios that result in cluster partitions.

A Hardware Management Console (HMC) can be used with advanced node failure detection to prevent cluster partitions when a cluster node has actually failed.

Before you begin

Consult the requirements and restrictions before setting up advanced node failure detection in a cluster:

Using HMC with a Representational state transfer (REST) server requires a HMC minimum version of V8R8.5.0 to implement and configure advanced node failure detection. See Planning advanced node failure detection for hardware and software requirements.
The Add cluster monitor (ADDCLUMON) command must be used with the representational state transfer (REST) server. The PowerHA® graphical interface does not support REST servers.
Check the QSSLPCL system value. Verify that it is set correctly for the release currently running.

NOTE: An incorrect value in QSSLPCL may result in a CPFBBCB diagnostic message with reason code 4.

To allow a an HMC using REST server to notify IBM i cluster nodes of sudden partition changes or system failures, communication between the HMC and the cluster nodes must be enabled. A digital certificate from the HMC is required and a secure certificate keystore and access to the certificate, if necessary, must be created. This certificate from the HMC is copied and installed on every node in the cluster that requires monitoring.

The setup instructions include steps for creating a *SYSTEM certificate keystore. This keystore may have already been created. If so, the password is required. Ask your IBM® i administrator for the keystore and access information.

About this task

These steps guide you through obtaining the digital certificate of your HMC, storing it and referencing it to allow advanced node failure detection for the cluster node.

IMPORTANT: This guide describes steps making use of features of both HMC and of the Digital Certificate Manager. Changes to either of these products may cause portions of this guide to become invalid. If you suspect such changes are preventing you from following the steps outlined in this guide successfully, contact your technical support provider.

Procedure

Create a *SYSTEM certificate store to hold the digital certificates

To create the *SYSTEM certificate store, use the following steps:

Creating the *SYSTEM certificate store

Procedure

In a web browser, enter http://mysystem:2001/dcm, where mysystem is the host name or IP address of the system. This opens IBM Digital Certificate Manager for i.
Log in with an IBM i profile with sufficient authority.
Click on Create Certificate Store on the left-hand navigation menu
On the right-hand side of the page select *SYSTEM.

Note: If the *SYSTEM option is not available in the list, it indicates that there is a *SYSTEM store already created on this system, and these steps have already been performed.

Create a password for the *SYSTEM store and click Create.

Note: The password is case-sensitive. It is recommended not to use special characters. This password is not attached to a user profile and it will not lock you out of the system after too many attempts.

Result

The *SYSTEM certificate store is created on the node.

Open the IBM Navigator for i and click Internet Configurations.
On the Internet Configurations page, click Digital Certificate Manager. You need to enter your user profile and password.
In the Digital Certificate Manager page, click Create New Certificate Store.
In the page that appears, you should have an option for *SYSTEM. Make sure that the button is selected and click Continue. If the *SYSTEM option is not there, you already have a *SYSTEM store created. Skip forward to: Select the *SYSTEM certificate store below.
Select No - Do not create a certificate in the certificate store.
Create a password for the *SYSTEM store and click Continue. The password is case-sensitive. It is recommended not to use special characters. This password is not attached to a user profile and it will not lock you out of the system after too many retries. You have successfully created the *SYSTEM store.

Determine the type of Certificate Used by the HMC

Importing Certificates into the System Certificate Store

Follow the instructions depending on the type of certificate used by the HMC.

Importing Self-Signed Certificates into the System Certificate Store

Extract the self-signed certificates to the IBM i

Begin by extracting the digital certificates for the HMC and copying them to the IBM® i system in the cluster node with these steps:

Sign on your IBM i system and open the command line display.
In the command line display, enter CALL QP2TERM to enter the PASE shell environment.
Retrieve the digital certificates from the HMC with this command:
```
openssl s_client -showcerts -connect HMC_name:443 2>/dev/null | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' | awk '/-BEGIN CERTIFICATE-/{a++}{print > "HMC_name"a".pem"}'
```
Replace HMC_name with the name of your system's HMC. This copies the certificates into files named HMC_name1.pem … HMC_nameN.pem, where N is the number of certificates copied from your system's HMC.
Press F3 to exit the QP2TERM environment.
Run the following command for each of certificate file to convert the CCSID to 819 (ASCII)
```
CHGATR OBJ('HMC_nameX.pem') ATR(*CCSID) VALUE(819).
```

Select the *SYSTEM certificate store in Digital Certificate Manager

Open the IBM Navigator for i and click Internet Configurations.
On the Internet Configurations page, click Digital Certificate Manager. You need to enter your user profile and password.
Click Select a Certificate Store and select the *SYSTEM option, click continue.
Sign in with the password for the certificate store and click Continue, then Manage Certificates.

Import the HMC certificates into the *SYSTEM certificate store.

Select Import certificate and click Continue. If your HMC has only one certificate, perform these steps for that certificate. If your HMC has multiple certificates, perform these steps for each certificate except the first certificate (HMC_name1.pem), starting with the last certificate and moving backwards through the list of certificates. For example, if there are three certificates: HMC_name1.pem, HMC_name2.pem, and HMC_name3.pem, perform these steps for HMC_name3.pem first, then for HMC_name2.pem.
Select Certificate Authority (CA) and click Continue.
Enter the path name of the certificate you want to import. For example, the path and file name may be /HMC_name1.pem. Click Continue.

Importing a Trusted Certificate into the System Certificate Store

To import a trusted certificate, follow the instructions in the IBM Documentation for Digital Certificate Manager.

Results

The selected security certificate is imported into the certificate store.

What to do next

After importing the certificates, sign on to your IBM i and use the command line to run the Add cluster monitor (ADDCLUMON) command to run the cluster configuration steps. For additional information about ADDCLUMON, see the Add Cluster Monitor (ADDCLUMON) command in the Knowledge Center.