...
PBS in its current form releases all the consumable resources requested by the job when it is suspended. In most cases suspended job But, in reality when system is out of swap space, a suspended job's process holds on to the memory it would have consumed and just releases ncpus (because kernel stops the process), in some cases admin might have configured an alternate suspend signal which would make the job release a few resources (like licenses) upon suspension. Therefore, it would be better if PBS has a way for admins to specify what all resources can be released from a job upon suspension.
Interface 1: New scheduler server attribute to specify which resources can be released.
- Visibility: Public
- Change Control: Stable
- Details:
- A new scheduler server attribute “res_released_on_susp” can be used to specify a list of comma separated list of resource names that can be release when a job is suspended.
- This scheduler server attribute is of type “array_string” and can only be set by a manager.
- If a manager is trying to set the attribute with a resource which is non existent then following error will be thrown on the console by qmgr command -
# qmgr -c "s s sched res_released_on_susp = ‘ncpus, abcd'"
...
qmgr: Error (15035) returned from server
- By default, this attribute is unset and If unset, after suspending the job PBS will release all the consumable resources requested by the job.If
- By default this attribute is unset PBS will get back to default behavior of releasing all the consumable resources upon job suspension.
- PBS manager can also add/remove resources to/from "res_released_on_susp" attribute by using "+="/"-=" operators.
- The resources specified in this new scheduler server attribute will be released (provided job has requested for them) every time a job is suspended (by preemption or qsig).
...
•This job attribute is populated only if “res_released_on_susp” scheduler server attribute is set and has a list of legitimate resources to be released.
•This attribute is set by scheduler server whenever it tries to preempt preempts a job using suspension. Scheduler will populate this job attribute by sending a ModifyJob batch request to server.
...
Interface 3: New Job attribute “resource_released_list”
...
•This attribute is of type “resource” “resource_list” and can only be read by user/operator/manager. This attribute is internally set by server when a job is suspended.
...
•This job attribute is populated only if “res_released_on_susp” scheduler server attribute is set and has a list of legitimate resources to be released.
...
- Visibility: Public
- Change Control: Stable Experimental
- Details:
- If server is unable to populate “resources_released” job attribute while suspending a job then it will log following log message at LOG_INFO log level.
“Unable to create resource released list”
Interface 5: New
...
- Visibility: Public
- Change Control: Stable
- Details:
- If scheduler is unable to populate “resources_released” job attribute while suspending a job then it will log following log message at LOG_INFO log level.
...
Interface 6: New error message while deleting a custom resource
- Visibility: Public
- Change Control: Stable
- Details:
- If an admin tries to delete a custom resource that is part of the res_released_on_susp scheduler server attribute then qmgr command will fail with “resource busy” error code.
...