Page Comparison

...

Overview:
This enhancement only applies to the Cray CLE 6.0 systems.and the instructions below only apply to installing PBS Professional on the Cray CLE 6.0 system.

Behavior change:
During install, check to see if the system is a CLE 6.0, and if it is, then pbs_postinstall will not be called.  The externally visible behavior change on Cray (CLE 6.0 and newer) will be that some PBS directories and files will not be created nor configured (e.g. PBS_HOME and /etc/pbs.conf). 

Installation instructions for PBS Professional on CLE 6.0:
These instructions rely on Cray's Image Management and Provisioning System (IMPS) features.  These instructions are based on the information found in the section that covers "Install Third-Party Software with a Custom Image" of the Cray document S-2559 (https://pubs.cray.com/content/S-2559/CLE%206.0.UP04/xctm-series-software-installation-and-configuration-guide-cle-60up04-s-2559-rev-b/install-third-party-software-with-a-custom-image-recipe there is also a PDF available).  Other Cray documents that may be of use are listed below as references.  Please be aware that newer versions of the documents may exist.

Instruction to Install PBS on CLE 6.0 with a custom image:

1. Follow Cray's instructions for creating a repository (Refer to the section on "Install Third-Party Software with a Custom Image" in the Software Installation and Configuration Guide in Cray document S-2559)
2. Follow Cray's instructions for creating a package collection in Cray document S-2559
   1. Use the pbspro_server-* RPM.
3. Follow Cray's instructions (in Cray document S-2559) for creating a new recipe and updating it with the PBS RPM.  
   Cray requires that each recipe be based on the type of node where the image will run.  For example, a service node would be contain the service image (e.g. service_cle_6.0up01_sles_12_x86-64_ari) as a base sub-recipe.  Therefore, if PBS server/scheduler/comm will be run on the sdb and the PBS MoM on the login nodes, at least 2 different recipes will be needed.  One recipe for the server/sched/comm and one recipe for the MoMs.
   1. In addition to adding the PBS RPM, add the other RPMs that PBS requires:
      
      1. In order to satisfy /bin/sh, also add sle-server_12sp212sp3_x86-64
         smw# recipe update -r sle-server_12sp3_x86-64 <recipe name>
      2. As Cray's instructions state, the base and base_updates repository pairs should be added.  This means sle-server_12sp3_x86-64_updates also needs to be added.
         smw# recipe update -r sle-server_12sp3_x86-64_updates <recipe name>
   2. Follow Cray's instructions (in Cray document S-2559) to validate the image recipe
   3. repeat as needed for the different types of nodes.
4. Follow Cray's instructions (in Cray document S-2559) to build and push/package the image.
5. Create any desired configuration sets.  Please refer to Cray’s documentation on configuration sets and the configurator for more information.
   The Cray ansible plays that would potentially need editing are "Persistent Dirs" for PBS_HOME (/var/spool/pbs) and "simple sync" for /etc/pbs.conf.
   WARNING: Note that unlike in prior Cray CLE versions (CLE 5.2 and earlier), with CLE 6.0 /var/spool is not persistent by default. So therefore, when you reboot a node, any configuration changes you may have made will be gone unless you use Cray’s cfgset command to make particular directories (for example /var/spool/pbs, /etc/pbs.conf, et cetera) persistent.   
6. Follow Cray's instructions (in Cray document S-2559) to assign the new boot image to the nodes that need the image that contains PBS Professional.  This would also include the nodes that don't run a PBS Professional daemon, but do need the client commands.  Include the configuration set (from step 5) that contains the information that you want to be persistent.
   smw# cnode update -i  /var/opt/cray/imps/boot_images/<image name>.cpio -c <configuration set name> <cname-of-node-to-update or group-to-update>
7. Follow Cray's instructions to reboot the system (in Cray document S-2559)
8. Log on to the nodes that are hosting PBS daemonsFor initial install only, remove $PBS_HOME (as specified in /etc/pbs.conf)
   For example: # rm -rf /var/spool/pbs
   
   1. Put the Cray compute nodes in batch mode (e.g. xtprocadmin -k m batch)
   2. Start PBS on that node:
      For example, on the server node:
      # PBS_DATA_SERVICE_USER=<non-root user for the database> /etc/init.d/pbs start
9. At this point PBS will be up and running, connected to ALPS and ready to be configured (for example, add moms to the server, set licenses, etc.)

...