Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Ticket

Jira Legacy
serverJIRA (pbspro.atlassian.net)
serverId32008a99-7831-3ff8-9638-3db0cd01164d
keyPP-228
was addressed by removing the code to restart the Python interpreter. This change introduced a significant memory leak when queuejob, modifyjob, and movejob hooks are present. This code will be restored in such a way that the administrator has more control and visibility over Python restarts. The change control is being defined as experimental because the long term plan is to address the memory leak in the pbs Python extension itself. Once completed, all of these interfaces become unnecessary.

Interfaces:

  • Interface 1: PBSpython_PYTHONrestart_RESTARTmax_MAX_HOOKShooks
    • Visibility: Public
    • Change Control: Experimental
    • Synopsis: Add configuration variable PBS_PYTHON_RESTART_MAX_HOOKSserver attribute python_restart_max_hooks
    • Detail: In the past there were macros defined to control how often the Python interpreter was restarted in the server. Allow the administrator to override the hook count limit. The attribute may be set by a manager through qmgr. Default value is 100 when the server attribute is unset, as it was previously. This variable may be set in PBS_CONF_FILE or in the environment prior to starting the server.
  • Interface 2: PBSpython_PYTHONrestart_RESTARTmax_MAX_OBJECTSobjects
    • Visibility: Public
    • Change Control: Experimental
    • Synopsis: Add configuration variable PBS_PYTHON_RESTART_MAX_OBJECTSserver attribute python_restart_max_objects
    • Detail: In the past there were macros defined to control how often the Python interpreter was restarted in the server. Allow the administrator to override the object count limit. This attribute may be set by a manager through qmgr. Default value is 1000 when the server attribute is unset, as it was previously. This variable may be set in PBS_CONF_FILE or in the environment prior to starting the server.
  • Interface 3: PBSpython_PYTHONrestart_RESTARTmin_MIN_INTERVALinterval
    • Visibility: Public
    • Change Control: Experimental
    • Synopsis: Add configuration variable PBS_PYTHON_RESTART_MIN_INTERVALserver attribute python_restart_min_interval
    • Detail: This is a new configuration variable server attribute that controls the minimum interval between Python restarts, expressed in seconds (pbs.duration). Default behavior is to avoid restarting the server more than once every 30 seconds when this attribute is unset. Administrators may increase this value at the cost of additional server memory usage. This variable attribute may be set in PBS_CONF_FILE or in the environment prior to starting the serverby a manager through qmgr.
  • Interface 4: Log message
    • Visibility: Public
    • Change Control: Experimental
    • Synopsis: PBSpython_PYTHONrestart_RESTART_MAX_HOOKS: max_hooks is now 100
    • Detail: A DEBUG3 message is being added to log the value of PBSpython_PYTHONrestart_RESTART_MAX_HOOKS the first time that _pbs_python_event_set() is calledmax_hooks when it changes. The message will be printed to the server log.
  • Interface 5: Log message
    • Visibility: Public
    • Change Control: Experimental
    • Synopsis: PBSpython_PYTHONrestart_RESTART_MAX_OBJECTS: max_objects is now 1000
    • Detail: A DEBUG3 message is being added to log the value of PBSpython_PYTHONrestart_RESTART_MAX_OBJECTS the first time that _pbs_python_event_set() is calledmax_objects when it changes. The message will be printed to the server log.
  • Interface 6: Log message
    • Visibility: Public
    • Change Control: Experimental
    • Synopsis: PBSpython_PYTHONrestart_RESTART_MIN_INTERVAL: min_interval is now 30
    • Detail: A DEBUG3 message is being added to log the value of PBSpython_PYTHONrestart_RESTART_MAX_HOOKS the first time that _pbs_python_event_set() is calledmin_interval when it changes. The message will be printed to the server log.
  • Interface 7: Log message
    • Visibility: Public
    • Change Control: Experimental
    • Synopsis: Restarting Python interpreter to reduce mem usage
    • Detail: A DEBUG2 message is being restored to indicate the Python interpreter is being restarted. The message will be printed to the server log. This message existed prior to this design document, but was never properly documented.
  • Interface 8: Log message
    • Visibility: Public
    • Change Control: Experimental
    • Synopsis: Current memory usage: VmSize=388200kB, VmRSS=98824kB
    • Detail: A DEBUG2 message is being added to log the contents of /proc/self/statm prior to server restart. The message will be printed to the server log. For systems not supporting /proc/self/statm, the string "unknown" will be printed in place of the memory information.

...