Advanced Automatic Node Failure Detection (7.5)

A Hardware Management Console (HMC) can be used with advanced node failure detection to prevent cluster partitions when a cluster node has actually failed. The Advanced node failure detection function can reduce the number of failure scenarios that result in cluster partitions.

Before you begin

Consult the requirements and restrictions before setting up advanced node failure detection in a cluster:

Using HMC with a Representational state transfer (REST) server requires a HMC minimum version of V8R8.5.0 to implement and configure advanced node failure detection.
Check the QSSLPCL system value. Verify that it is set correctly for the release currently running.

NOTE: An incorrect value in QSSLPCL may result in a CPFBBCB diagnostic message with reason code 4.

The setup instructions include steps for creating a *SYSTEM certificate keystore. This keystore may have already been created. If so, the password is required. Ask your IBM i administrator for the keystore and access information.

Procedure

These steps guide you through obtaining the digital certificate of your HMC, storing it and referencing it to allow advanced node failure detection for the cluster node.

Create a *SYSTEM certificate store to hold the digital certificates

To create the *SYSTEM certificate store, use the following steps:

Procedure

In a web browser, enter http://mysystem:2001/dcm, where mysystem is the host name or IP address of the system. This opens IBM Digital Certificate Manager for i.
Log in with an IBM i profile with sufficient authority.
Click on Create Certificate Store on the left-hand navigation menu
On the right-hand side of the page select *SYSTEM.

Note: If the *SYSTEM option is not available in the list, it indicates that there is a *SYSTEM store already created on this system, and these steps have already been performed.

Create a password for the *SYSTEM store and click Create.

Note: The password is case-sensitive. It is recommended not to use special characters. This password is not attached to a user profile and it will not lock you out of the system after too many attempts.

Result

The *SYSTEM certificate store is created on the node.

Determine the type of Certificate Used by the HMC

Importing Certificates into the System Certificate Store

Follow the instructions depending on the type of certificate used by the HMC.

Extract the self-signed certificates to the IBM i

Begin by extracting the digital certificates for the HMC and copying them to the IBM® i system in the cluster node with these steps:

Sign on your IBM i system and open the command line display.
In the command line display, enter CALL QP2TERM to enter the PASE shell environment.
Retrieve the digital certificates from the HMC with this command:
openssl s_client -showcerts -connect HMC_name:443 2>/dev/null | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' | awk '/-BEGIN CERTIFICATE-/{a++}{print > "HMC_name"a".pem"}'
Replace HMC_name with the name of your system's HMC. This copies the certificates into files named HMC_name1.pem … HMC_nameN.pem, where N is the number of certificates copied from your system's HMC.
Press F3 to exit the QP2TERM environment.
Run the following command for each of certificate file to convert the CCSID to 819 (ASCII)
CHGATR OBJ('HMC_nameX.pem') ATR(*CCSID) VALUE(819).

Select the *SYSTEM certificate store in Digital Certificate Manager

Open the IBM Navigator for i and click Internet Configurations.
On the Internet Configurations page, click Digital Certificate Manager. You need to enter your user profile and password.
Click Select a Certificate Store and select the *SYSTEM option, click continue.
Sign in with the password for the certificate store and click Continue, then Manage Certificates.

Import the HMC certificates into the *SYSTEM certificate store.

Select Import certificate and click Continue. If your HMC has only one certificate, perform these steps for that certificate. If your HMC has multiple certificates, perform these steps for each certificate except the first certificate (HMC_name1.pem), starting with the last certificate and moving backwards through the list of certificates. For example, if there are three certificates: HMC_name1.pem, HMC_name2.pem, and HMC_name3.pem, perform these steps for HMC_name3.pem first, then for HMC_name2.pem.
Select Certificate Authority (CA) and click Continue.
Enter the path name of the certificate you want to import. For example, the path and file name may be /HMC_name1.pem. Click Continue.

Importing a Trusted Certificate into the System Certificate Store

To import a trusted certificate, follow the instructions in the IBM Documentation for Digital Certificate Manager.

Adding a Cluster Monitor to Monitor for Node Failures

The Add Cluster Monitor (ADDCLUMON) command is used to register an HMC with a particular cluster node. The following example adds a cluster monitor:

 ADDCLUMON CLUSTER(MYCLU) 
           NODE(PROD) 
           TYPE(*RESTSVR)
           RESTSVR('hmc1.example.com' 'myhmcuser' 'myhmcpassword')

This command specifies to add a cluster monitor to the node named PROD in cluster MYCLU. This monitor will run on cluster node PROD and connect to the hmc hmc1.example.com with the credentials specified.

If desired, repeat this process for additional HMCs in the environment.

Results

Advanced node failure detection with cluster monitors is configured. This can be verified using one of the following:

The Work with Cluster (WRKCLU) menu, followed by option 6 to work with cluster nodes, followed by option 6 on a cluster node to see the monitors that cluster node has.
The Display Cluster Information (DSPCLUINF) command. For example DSPCLUINF and pressing enter until the cluster monitor list is displayed.