View Source

Interface 1: As an admin or user, I would like qstat to be able to output full job attribute information in json format so that I can more easily write scripts consuming this information.
- Visibility: Public
- Change Control: Stable
- Details: qstat -f will use -F json option to print the output in JSON format(http://www.json.org/).
- Similar to pbsnodes -av -F json, timestamp, pbs_version and pbs_server will be used as the prolog of the output.
- > Timestamp - system time in seconds since the Epoch. Server name - default server name.
  > The server name is based on the values configured in pbs.conf. This will follow the order of precedence when specifying server name in pbs.conf
- Along with full job attributes this new interface is applicable to full server attributes (qstat -Bf -F json) and full queue attributes (qstat -Qf -F json)
- - ```
  Usage:
```
```
  qstat -f -F json
```
```
  qstat -f -F json <job_id>
```
- Example:
  {
  "timestamp":1479277336,
  "pbs_version":"14.1",
  "pbs_server":"vbox",
  "Jobs":{
  "1.vbox":{
  "Job_Name":"STDIN",
  "Job_Owner":"root@vbox",
  "job_state":"Q",
  "queue":"workq",
  "server":"vbox",
  "Checkpoint":"u",
  "ctime":"Fri Nov 11 17:57:05 2016",
  "Error_Path":"vbox:/home/nithinj/Tasks/overlay/STDIN.e1",
  "Hold_Types":"n",
  "Join_Path":"n",
  "Keep_Files":"n",
  "Mail_Points":"a",
  "mtime":"Fri Nov 11 17:57:05 2016",
  "Output_Path":"vbox:/home/nithinj/Tasks/overlay/STDIN.o1",
  "Priority":0,
  "qtime":"Fri Nov 11 17:57:05 2016",
  "Rerunable":"True"
     "Resource_List":{
                                 "ncpus":1,
         "nodect":1,
       "place":"pack",
         "select":"1:ncpus=1"
      },
                           "schedselect":"1:ncpus=1",
  "substate":10,
  "Variable_List":{
  "PBS_O_HOME":"/root",
  "PBS_O_LANG":"en_US.utf8",
  "PBS_O_LOGNAME":"nithinj",
  "PBS_O_PATH":"/root/perl5/bin:/usr/lib64/qt- 3.3/bin:/home/nithinj/perl5/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:/usr/pbs/bin:/home/nithinj/.local/bin:/home/nithinj/bin:/opt/pbs/default/bin:/usr/local/git/bin:/opt/pbs/default/bin:/opt/pbs/default/sbin:/usr/local/git/bin",
  "PBS_O_MAIL":"/var/spool/mail/nithinj",
  "PBS_O_SHELL":"/bin/bash",
  "PBS_O_WORKDIR":"/home/nithinj/Tasks/overlay",
  "PBS_O_SYSTEM":"Linux",
  "PBS_O_QUEUE":"workq",
  "PBS_O_HOST" :"vbox"
  },
  "euser":"root",
  "egroup":"root",
  "queue_rank":1,
  "queue_type":"E",
  "comment":"Not Running: Not enough free nodes available",
  "etime":"Fri Nov 11 17:57:05 2016",
  "Submit_arguments":"-- /bin/sleep 1000",
  "executable":"<jsdl-hpcpa:Executable>/bin/sleep</jsdl-hpcpa:Executable>",
  "argument_list":"<jsdl-hpcpa:Argument>1000</jsdl-hpcpa:Argument>",
  "project":"_pbs_project_default"
  }
  }
  }

Special case (when user variables contains special characters): Json output with special characters

- Interface 2: As an admin or user, I would like qstat to be able to output full job attribute information in delimiter separated value format so that I can more easily write scripts consuming this information.
  - Visibility: Public
  - Change Control: Stable
  - Details: qstat -f will use -F dsv option to print the output in delimiter separated value format.
  - Default delimiter will be a pipe.("|")
  - The user can provide custom delimiter using -D option. (character or string)
  - Backslash ("\") is used as the escape character on Unix/Linux if delimiter itself is appearing in the value.
  - Caret ("^") is used as the escape character on Windows.
  - newline is used as the separator between multiple jobs. Using the newline as the delimiter will lead to undefined behavior.
  - Since feed characters could break the dsv format, they are converted into the following if appears in the value.
  - Along with full job attributes this new interface is applicable to full server attributes (qstat -Bf -F dsv) and full queue attributes (qstat -Qf -F dsv)
    newline → '\'+'n' and formfeed → '\'+'f' (plus indicated that it is a concatenation)
  - ```
  Usage:
```
```
  qstat -f -F dsv
```
```
  qstat -f -F dsv <job_id>
```
```
  qstat -f -F dsv -D,     (where "," is the delimiter)
```
- Example:
  qstat -f -Fdsv
  Job Id: 1.vbox|Job_Name = STDIN|Job_Owner = root@vbox|job_state = Q|queue = workq|server = vbox|Checkpoint = u|ctime = Fri Nov 11 17:57:05 2016|Error_Path = vbox:/home/nithinj/Tasks/overlay/STDIN.e1|Hold_Types = n|Join_Path = n|Keep_Files = n Mail_Points = a|mtime = Fri Nov 11 17:57:05 2016|Output_Path = vbox:/home/nithinj/Tasks/overlay/STDIN.o1|Priority = 0|qtime = Fri Nov 11 17:57:05 2016|Rerunable = True|Resource_List.ncpus = 1|Resource_List.nodect = 1|Resource_List.place = pack|Resource_List.select = 1:ncpus=1|schedselect = 1:ncpus=1|substate = 10|Variable_List = PBS_O_HOME=/root,PBS_O_LANG=en_US.utf8, PBS_O_LOGNAME=nithinj, PBS_O_PATH=/root/perl5/bin:/usr/lib64/qt-3.3/bin:/home/nithinj/perl5/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:/usr/pbs/bin:/home/nithinj/.local/bin:/home/nithinj/bin:/opt/pbs/default/bin:/usr/local/git/bin:/opt/pbs/default/bin:/opt/pbs/default/sbin:/usr/local/git/bin,PBS_O_MAIL=/var/spool/mail/nithinj,PBS_O_SHELL=/bin/bash, PBS_O_WORKDIR=/home/nithinj/Tasks/overlay,PBS_O_SYSTEM=Linux, PBS_O_QUEUE=workq,PBS_O_HOST=vbox|euser = root|egroup = root|queue_rank = 1|queue_type = E|comment = Not Running: Not enough free nodes available|etime = Fri Nov 11 17:57:05 2016|Submit_arguments = -- /bin/sleep 1000|executable = <jsdl-hpcpa:Executable>/bin/sleep</jsdl-hpcpa:Executable>|argument_list = <jsdl-hpcpa:Argument>1000</jsdl-hpcpa:Argument>|project = _pbs_project_default
  Job Id: 2.vbox|Job_Name = STDIN|Job_Owner = root@vbox|job_state = Q|queue = workq|server = vbox|Checkpoint = u|ctime = Fri Nov 11 17:57:05 2016|Error_Path = vbox:/home/nithinj/Tasks/overlay/STDIN.e2|Hold_Types = n|Join_Path = n|Keep_Files = n Mail_Points = a|mtime = Fri Nov 11 17:57:05 2016|Output_Path = vbox:/home/nithinj/Tasks/overlay/STDIN.o2|Priority = 0|qtime = Fri Nov 11 17:57:05 2016|Rerunable = True|Resource_List.ncpus = 1|Resource_List.nodect = 1|Resource_List.place = pack|Resource_List.select = 1:ncpus=1|schedselect = 1:ncpus=1|substate = 10|Variable_List = PBS_O_HOME=/root,PBS_O_LANG=en_US.utf8, PBS_O_LOGNAME=nithinj, PBS_O_PATH=/root/perl5/bin:/usr/lib64/qt-3.3/bin:/home/nithinj/perl5/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:/usr/pbs/bin:/home/nithinj/.local/bin:/home/nithinj/bin:/opt/pbs/default/bin:/usr/local/git/bin:/opt/pbs/default/bin:/opt/pbs/default/sbin:/usr/local/git/bin,PBS_O_MAIL=/var/spool/mail/nithinj,PBS_O_SHELL=/bin/bash, PBS_O_WORKDIR=/home/nithinj/Tasks/overlay,PBS_O_SYSTEM=Linux, PBS_O_QUEUE=workq,PBS_O_HOST=vbox|euser = root|egroup = root|queue_rank = 1|queue_type = E|comment = Not Running: Not enough free nodes available|etime = Fri Nov 11 17:57:05 2016|Submit_arguments = -- /bin/sleep 1000|executable = <jsdl-hpcpa:Executable>/bin/sleep</jsdl-hpcpa:Executable>|argument_list = <jsdl-hpcpa:Argument>1000</jsdl-hpcpa:Argument>|project = _pbs_project_default

- Interface 3: New argument valid in conjunction with qstat -f

- Visibility: Public
- Change Control: Stable
- Details: User/Admin is able to request a the output of qstat to return the full job output with each attribute on its own line (i.e. not splitting up the value of an attribute across multiple lines if it is greater than 78 chars) This is done by using "-w" in conjunction with "-f"
  - ```
  Usage:
```
```
  qstat -fw
```
```
  qstat -fw <job_id>
```
- Using -w with -f is now accepted. All other usages of -w shall behave as previously defined for qstat
- Since feed characters could break the one line output format, they are converted into the following.
  newline → '\'+'n' and formfeed → '\'+'f' (plus indicated that it is a concatenation)
- Along with full job attributes this new interface is applicable to full server attributes (qstat -Bf -w) and full queue attributes (qstat -Qf -w)
- Example output:
- [centos@pbs-server ~]$ /tmp/qstat -fw 88
  Job Id: 88.pbs-server
      Job_Name = swarm-worker
      Job_Owner = centos@pbs-server
      resources_used.cpupercent = 0
      resources_used.cput = 00:00:00
      resources_used.mem = 367016kb
      resources_used.ncpus = 1
      resources_used.swarm_id = pbs-server
      resources_used.swarm_node_status = {Resources: {Used: {cpu_time: 622, memory: 366908k}, Total: {ncpu: 2, memory: 256mb}}, Containers: {Paused: 0, Running: 20, Total: 22, Stopped: 2}}
      resources_used.vmem = 4316kb
      resources_used.walltime = 23:53:03
      job_state = R
      queue = workq
      server = pbs-server
      Checkpoint = u
      ctime = Mon Oct 17 21:07:50 2016
      Error_Path = pbs-server:/home/centos/swarm-worker.e88
      exec_host = pbs-node01/0
      exec_vnode = (pbs-node01:ncpus=1)
      Hold_Types = n
      Join_Path = n
      Keep_Files = n
      Mail_Points = a
      mtime = Tue Oct 18 21:00:51 2016
      Output_Path = pbs-server:/home/centos/swarm-worker.o88
      Priority = 0
      qtime = Mon Oct 17 21:07:50 2016
      Rerunable = True
      Resource_List.ncpus = 1
      Resource_List.nodect = 1
      Resource_List.place = excl
      Resource_List.select = 1:ncpus=1
      stime = Mon Oct 17 21:07:50 2016
      session_id = 3690
      jobdir = /home/centos
      substate = 42
      Variable_List = PBS_O_SYSTEM=Linux,PBS_O_SHELL=/bin/bash,PBS_O_HOME=/home/centos,PBS_O_HOST=pbs-server,swarm_id=pbs-server,PBS_O_LOGNAME=centos,PBS_O_WORKDIR=/home/centos,PBS_O_LANG=en_US.UTF-8,PBS_O_PATH=/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/pbs/bin:/opt/pbs/bin:/opt/pbs/sbin:/opt/pbs/unsupported:/home/centos/.local/bin:/home/centos/bin,PBS_O_QUEUE=workq,PBS_O_MAIL=/var/spool/mail/centos
      comment = Job run at Mon Oct 17 at 21:07 on (pbs-node01:ncpus=1)
      etime = Mon Oct 17 21:07:50 2016
      run_count = 1
      Submit_arguments = -N swarm-worker -lplace=excl -lselect=1:ncpus=1 -v swarm_id=pbs-server -- /bin/sleep 660000
      executable = <jsdl-hpcpa:Executable>/bin/sleep</jsdl-hpcpa:Executable>
      argument_list = <jsdl-hpcpa:Argument>660000</jsdl-hpcpa:Argument>
      project = _pbs_project_default

NOTE:

All the three new interfaces will be supported only with full set of attributes (" -f " option). This would include every vaild combination of "-f" (Ex: "qstat -Qf" or "qstat -Bf")
Using it with other optargs will throw the already existing "conflicting options" error and prints the usage on the screen.

Follow the community discussion for json and dsv options

Community discussion for qstat -fw option

Interface 1: As an admin or user, I would like qstat to be able to output full job attribute information in json format so that I can more easily write scripts consuming this information.

Interface 2: As an admin or user, I would like qstat to be able to output full job attribute information in delimiter separated value format so that I can more easily write scripts consuming this information.

Interface 3: New argument valid in conjunction with qstat -f