Forum discussion (EDD review).

Overview:

In order to support Cray features like requesting particular segments, or some number of segments, PBS created one vnode for each NUMA node.
The customer has requested the ability to change the way PBS reports the Cray nodes, so that they can have fewer vnodes to deal with and not
encounter some of the differences that multiple vnodes per host brings.
PBS shall provide the option to allow a vnode to be created per compute node on a Cray.
PBS shall continue to support the backwards compatible behavior of creating one vnode per NUMA node.

Interface: alps_create_vnode_per_numa

• Visibility: Public
• Change Control: Experimental
• Details:
  • New interface: new mom_priv/config variable.
  • alps_create_vnode_per_numa is a Boolean. Setting it to "TRUE" will cause PBS to create one vnode per Cray numa node. This is the 
    default behavior when alps_create_vnode_per_numa is not set in mom_priv/config. Setting it to "FALSE" will cause PBS to create
    one vnode per Cray compute node.
  • Note: when alps_create_vnode_per_numa is set to "FALSE", it will not be possible for PBS to schedule based on segments.

Administrator's instructions.

• When changing from using one type of vnode create versus another, you must first:
  
  • Quiesce the system (ensure there are no running jobs).
  • Remove all existing vnodes of the vntype cray_compute. The easiest way to do this is to delete all vnodes using 
    • qmgr -c "delete node @default"
  • Set the mom_priv/config variable to desired setting in each MoM.
  • HUP or restart the MoM.
  • Add the MoMs back using qmgr: qmgr -c "create node <mom name>"

Definition of technical terms, spelling out acronyms and abbreviations.