...
Extend PBS to allow users to submit jobs with a node-filter (node_filter) expression.
- Details:
- Example: qsub -lselect=3:ncpus=2:mem=18gb -lnode_filter=“ncpus<=8 and model==Skylake or ncpus>=8 and model==Haswell” job.scr
- A user can specify a node filter with each of their jobs and this filter will help scheduler to filter out nodes that this job is allowed to run on.
- There is a new built-in resource “node_filter”. This resource is of type string. Users/operator/manager has privileges to read/write this resource. It is not a host level resource.
- node_filter is an expression created by using resources_available on the nodes.
- node_filter can be a collection of multiple filters separated by 'or' operator. The scheduler will cull the nodes by applying filters and run the job as soon as it finds the node solution.
- If there are many filters separated by 'or' operator, the scheduler will start applying filter. After applying each filter scheduler will try to find a node solution for the job, if it is able to find one then it will use it otherwise it will discard the result of the filter and applies the next filter on all the nodes again. The scheduler may choose to apply filters in any order.
- Users can specify a node filter with node resources using operators like "==,<, >, <=, >=, !=, and, or" Example: qsub -lselect=3:ncpus=2:mem=18gb -lnode_filter=“ncpus<=8 and model==Skylake or ncpus>=8 and model==Haswell” job.scr
- While applying the filter, if scheduler encounters a resource in the filter that is not present in the 'resources' line of sched config then the scheduler will ignore that resource while filtering the nodes.
- Caveats:
If node_filter is used in conjunction with node_group_key, node_filter will take precedence over node_group_key set on server or queue.- node_filter and -lplace=group cannot be used together in the same job. Such a job submission will fail with error "node filter cannot be used with placement grouping"With node_filter in effect, placement grouping can be achieved by users when they specify desired filters. Like place=group=color can be changed to -lnode_filter="color=Green or color=Red". Since node_filter is more flexible and users can specify multiple expressions in it, -lplace=group is marked as obsolete and it is recommended to use node_filter instead.