2017-05-11 10:23:33,450 INFO input command: pbs_benchpress -f pbs_resc_used_single_node.py -o /mnt/hgfs/altair/PP-745_unfixed.txt 2017-05-11 10:23:33,450 INFO param: None 2017-05-11 10:23:33,450 INFO ptl version: 1.0.0 2017-05-11 10:23:33,451 INFO platform: Linux centosvm 3.10.0-514.10.2.el7.x86_64 #1 SMP Fri Mar 3 00:04:05 UTC 2017 x86_64 x86_64 2017-05-11 10:23:33,451 INFO python version: 2.7.5 2017-05-11 10:23:33,451 INFO user: dilipk 2017-05-11 10:23:33,451 INFO -------------------------------------------------------------------------------- 2017-05-11 10:23:33,451 INFO Cleaning up temporary files 2017-05-11 10:23:33,462 INFO ====================================================================== 2017-05-11 10:23:33,463 INFO suite name: TestPbsPP745 2017-05-11 10:23:33,463 INFO ====================================================================== 2017-05-11 10:23:33,463 INFO ================================ 2017-05-11 10:23:33,463 INFO Entered TestPbsPP745 setUpClass 2017-05-11 10:23:33,463 INFO ================================ 2017-05-11 10:23:33,463 INFOCLI2 centosvm: id pbsuser 2017-05-11 10:23:33,468 INFOCLI2 centosvm: id pbsuser1 2017-05-11 10:23:33,472 INFOCLI2 centosvm: id pbsuser2 2017-05-11 10:23:33,476 INFOCLI2 centosvm: id pbsuser3 2017-05-11 10:23:33,489 INFO server centosvm: server operating mode set to cli 2017-05-11 10:23:33,489 INFOCLI centosvm: /opt/pbs/bin/qstat -Bf centosvm 2017-05-11 10:23:33,502 INFO server centosvm: version 14.0.1 2017-05-11 10:23:33,502 INFO expect action: created new action kicksched 2017-05-11 10:23:33,503 INFO expect action: added action kicksched to server centosvm 2017-05-11 10:23:33,529 INFOCLI2 centosvm: sudo -H /opt/pbs/sbin/pbsfs 2017-05-11 10:23:33,548 INFOCLI2 centosvm: sudo -H /usr/bin/cat /var/spool/pbs/sched_priv/resource_group 2017-05-11 10:23:33,559 INFOCLI centosvm: /opt/pbs/bin/qmgr -c list sched 2017-05-11 10:23:33,573 INFOCLI2 centosvm: sudo -H /usr/bin/cat /var/spool/pbs/sched_priv/holidays 2017-05-11 10:23:33,594 INFO server centosvm: server operating mode set to cli 2017-05-11 10:23:33,595 INFOCLI centosvm: /opt/pbs/bin/qstat -Bf centosvm 2017-05-11 10:23:33,607 INFO server centosvm: version 14.0.1 2017-05-11 10:23:33,612 INFO ================================= 2017-05-11 10:23:33,613 INFO Completed TestPbsPP745 setUpClass 2017-05-11 10:23:33,613 INFO ================================= 2017-05-11 10:23:33,613 INFO test name: test_epilogue_single_node (tests.functional.pbs_resc_used_single_node.TestPbsPP745)... 2017-05-11 10:23:33,613 INFO test start time: Thu May 11 10:23:33 2017 2017-05-11 10:23:33,613 INFO test docstring: Test accumulatinon of resources of a single node job from an exechost_epilogue hook. The string values specified are used by mom as is; that is, they're not converted into JSON format. 2017-05-11 10:23:33,613 INFO =========================== 2017-05-11 10:23:33,613 INFO Entered TestPbsPP745 setUp 2017-05-11 10:23:33,613 INFO =========================== 2017-05-11 10:23:33,614 INFOCLI centosvm: /opt/pbs/bin/qstat -Bf centosvm 2017-05-11 10:23:33,628 INFO status on centosvm: server 2017-05-11 10:23:33,628 INFOCLI centosvm: /opt/pbs/bin/qstat -Bf centosvm 2017-05-11 10:23:33,641 INFO manager on centosvm: set server {'managers': (3, 'dilipk@*')} 2017-05-11 10:23:33,641 INFOCLI centosvm: sudo -H /opt/pbs/bin/qmgr -c set server managers-=dilipk@* 2017-05-11 10:23:33,667 INFO manager on centosvm: set server {'managers': (2, 'dilipk@*')} 2017-05-11 10:23:33,667 INFOCLI centosvm: sudo -H /opt/pbs/bin/qmgr -c set server managers+=dilipk@* 2017-05-11 10:23:33,694 INFO server centosvm: reverting configuration to defaults 2017-05-11 10:23:33,694 INFOCLI centosvm: /opt/pbs/bin/qstat -Bf centosvm 2017-05-11 10:23:33,706 INFO select on centosvm: __ALL__ 2017-05-11 10:23:33,706 INFOCLI centosvm: /opt/pbs/bin/qselect 2017-05-11 10:23:33,717 INFOCLI centosvm: /opt/pbs/bin/qstat -f @centosvm 2017-05-11 10:23:33,731 INFO expect on server centosvm: job_state set 0 job ... OK 2017-05-11 10:23:33,731 INFOCLI centosvm: /opt/pbs/bin/pbs_rstat -f 2017-05-11 10:23:33,742 INFO manager on centosvm: unset server ['pbs_license_max', 'pbs_license_min'] 2017-05-11 10:23:33,743 INFOCLI centosvm: /opt/pbs/bin/qmgr -c unset server pbs_license_max,pbs_license_min 2017-05-11 10:23:33,758 INFOCLI centosvm: sudo -H /opt/pbs/bin/qmgr -c list hook 2017-05-11 10:23:33,779 INFOCLI centosvm: /opt/pbs/bin/qstat -Qf @centosvm 2017-05-11 10:23:33,792 INFO manager on centosvm: delete queue workq 2017-05-11 10:23:33,792 INFOCLI centosvm: /opt/pbs/bin/qmgr -c delete queue workq 2017-05-11 10:23:33,810 INFOCLI centosvm: /opt/pbs/bin/qstat -Qf workq@centosvm 2017-05-11 10:23:33,821 INFO expect on server centosvm: unset queue workq ... OK 2017-05-11 10:23:33,821 INFO manager on centosvm: create queue workq {'started': 'True', 'queue_type': 'Execution', 'enabled': 'True'} 2017-05-11 10:23:33,822 INFOCLI centosvm: /opt/pbs/bin/qmgr -c create queue workq started=True,queue_type=Execution,enabled=True 2017-05-11 10:23:33,840 INFOCLI centosvm: /opt/pbs/bin/qstat -Qf workq@centosvm 2017-05-11 10:23:33,850 INFO expect on server centosvm: started set True || queue_type set Execution || enabled set True queue workq ... OK 2017-05-11 10:23:33,850 INFO manager on centosvm: set server {'pbs_license_linger_time': '3600', 'default_queue': 'workq', 'flatuid': 'True'} 2017-05-11 10:23:33,851 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server pbs_license_linger_time=3600,default_queue=workq,flatuid=True 2017-05-11 10:23:33,870 INFO status on centosvm: resource 2017-05-11 10:23:33,871 INFO manager on centosvm: list resource 2017-05-11 10:23:33,871 INFOCLI centosvm: /opt/pbs/bin/qmgr -c list resource 2017-05-11 10:23:33,882 INFOCLI status on centosvm: server license_count 2017-05-11 10:23:33,882 INFOCLI centosvm: /opt/pbs/bin/qstat -Bf centosvm 2017-05-11 10:23:33,894 INFO server: centosvm licensed 2017-05-11 10:23:33,921 INFOCLI2 centosvm: sudo -H /usr/bin/cat /var/spool/pbs/server_priv/comm.lock 2017-05-11 10:23:33,953 INFOCLI2 centosvm: sudo -H /usr/bin/cat /var/spool/pbs/sched_priv/sched.lock 2017-05-11 10:23:33,965 INFO scheduler centosvm: reverting configuration to defaults 2017-05-11 10:23:33,965 INFO manager on centosvm: list sched 2017-05-11 10:23:33,966 INFOCLI centosvm: /opt/pbs/bin/qmgr -c list sched 2017-05-11 10:23:33,977 INFO manager on centosvm: unset sched ['sched_cycle_length'] 2017-05-11 10:23:33,977 INFOCLI centosvm: /opt/pbs/bin/qmgr -c unset sched sched_cycle_length 2017-05-11 10:23:33,993 INFOCLI2 centosvm: sudo -H /usr/bin/cat /var/spool/pbs/sched_priv/dedicated_time 2017-05-11 10:23:34,008 INFOCLI2 centosvm: which cp 2017-05-11 10:23:34,011 INFOCLI2 centosvm: sudo -H /usr/bin/cp -p /tmp/PtlPbs6_SQY4 /var/spool/pbs/sched_priv/dedicated_time 2017-05-11 10:23:34,022 INFOCLI2 centosvm: which chmod 2017-05-11 10:23:34,030 INFOCLI2 centosvm: sudo -H /usr/bin/chmod 0644 /var/spool/pbs/sched_priv/dedicated_time 2017-05-11 10:23:34,042 INFOCLI2 centosvm: which chown 2017-05-11 10:23:34,045 INFOCLI2 centosvm: sudo -H /usr/bin/chown root /var/spool/pbs/sched_priv/dedicated_time 2017-05-11 10:23:34,056 INFOCLI2 centosvm: which chgrp 2017-05-11 10:23:34,059 INFOCLI2 centosvm: sudo -H /usr/bin/chgrp root /var/spool/pbs/sched_priv/dedicated_time 2017-05-11 10:23:34,071 INFOCLI2 centosvm: cmp /opt/pbs/etc/pbs_resource_group /var/spool/pbs/sched_priv/resource_group 2017-05-11 10:23:34,075 ERROR err: ['cmp: /var/spool/pbs/sched_priv/resource_group: Permission denied'] 2017-05-11 10:23:34,075 INFOCLI2 centosvm: sudo -H /usr/bin/cp -p /opt/pbs/etc/pbs_resource_group /var/spool/pbs/sched_priv/resource_group 2017-05-11 10:23:34,087 INFOCLI2 centosvm: sudo -H /usr/bin/chmod 0644 /var/spool/pbs/sched_priv/resource_group 2017-05-11 10:23:34,100 INFO scheduler centosvm: reverting holidays file to default 2017-05-11 10:23:34,100 INFOCLI2 centosvm: cmp /opt/pbs/etc/pbs_holidays /var/spool/pbs/sched_priv/holidays 2017-05-11 10:23:34,103 ERROR err: ['cmp: /var/spool/pbs/sched_priv/holidays: Permission denied'] 2017-05-11 10:23:34,103 INFOCLI2 centosvm: sudo -H /usr/bin/cp -p /opt/pbs/etc/pbs_holidays /var/spool/pbs/sched_priv/holidays 2017-05-11 10:23:34,118 INFOCLI2 centosvm: sudo -H /usr/bin/chmod 0644 /var/spool/pbs/sched_priv/holidays 2017-05-11 10:23:34,129 INFO scheduler centosvm: Parsing holidays file 2017-05-11 10:23:34,130 INFOCLI2 centosvm: sudo -H /usr/bin/cat /var/spool/pbs/sched_priv/holidays 2017-05-11 10:23:34,140 INFOCLI2 centosvm: cmp /opt/pbs/etc/pbs_sched_config /var/spool/pbs/sched_priv/sched_config 2017-05-11 10:23:34,143 ERROR err: ['cmp: /var/spool/pbs/sched_priv/sched_config: Permission denied'] 2017-05-11 10:23:34,144 INFOCLI2 centosvm: sudo -H /usr/bin/cp -p /opt/pbs/etc/pbs_sched_config /var/spool/pbs/sched_priv/sched_config 2017-05-11 10:23:34,156 INFOCLI2 centosvm: sudo -H /usr/bin/chmod 0644 /var/spool/pbs/sched_priv/sched_config 2017-05-11 10:23:34,167 INFO scheduler centosvm: sent signal -HUP 2017-05-11 10:23:34,167 INFOCLI2 centosvm: sudo -H kill -HUP 58281 2017-05-11 10:23:34,180 INFOCLI2 centosvm: which rm 2017-05-11 10:23:34,183 INFOCLI2 centosvm: sudo -H /usr/bin/rm /var/spool/pbs/sched_priv/usage 2017-05-11 10:23:34,197 ERROR err: ['/usr/bin/rm: cannot remove \xe2\x80\x98/var/spool/pbs/sched_priv/usage\xe2\x80\x99: No such file or directory'] 2017-05-11 10:23:34,224 INFOCLI2 centosvm: sudo -H /usr/bin/cat /var/spool/pbs/sched_priv/sched.lock 2017-05-11 10:23:34,235 INFOCLI2 centosvm: sudo -H cmp /opt/pbs/sbin/pbs_mom /opt/pbs/sbin/pbs_mom.cpuset 2017-05-11 10:23:34,262 INFOCLI2 centosvm: sudo -H /usr/bin/cat /var/spool/pbs/mom_priv/mom.lock 2017-05-11 10:23:34,276 INFOCLI2 centosvm: sudo -H /opt/pbs/sbin/pbs_mom --version 2017-05-11 10:23:34,290 INFO mom centosvm: reverting configuration to defaults 2017-05-11 10:23:34,290 INFOCLI2 centosvm: sudo -H /usr/bin/rm -f /var/spool/pbs/mom_priv/epilogue 2017-05-11 10:23:34,305 INFOCLI2 centosvm: sudo -H /usr/bin/rm -f /var/spool/pbs/mom_priv/prologue 2017-05-11 10:23:34,317 INFOCLI2 centosvm: sudo -H ls /var/spool/pbs/mom_priv/config.d 2017-05-11 10:23:34,331 INFOCLI2 centosvm: sudo -H /usr/bin/cp -p /tmp/PtlPbssTx2oy /var/spool/pbs/mom_priv/config 2017-05-11 10:23:34,344 INFOCLI2 centosvm: sudo -H /usr/bin/chmod 0644 /var/spool/pbs/mom_priv/config 2017-05-11 10:23:34,355 INFOCLI2 centosvm: sudo -H /usr/bin/chown root /var/spool/pbs/mom_priv/config 2017-05-11 10:23:34,368 INFOCLI2 centosvm: sudo -H /usr/bin/chgrp root /var/spool/pbs/mom_priv/config 2017-05-11 10:23:34,380 INFO mom centosvm: sent signal -HUP 2017-05-11 10:23:34,381 INFOCLI2 centosvm: sudo -H kill -HUP 58278 2017-05-11 10:23:34,405 INFOCLI2 centosvm: sudo -H /usr/bin/cat /var/spool/pbs/mom_priv/mom.lock 2017-05-11 10:23:34,418 INFOCLI centosvm: /opt/pbs/bin/pbsnodes -s centosvm -v -a 2017-05-11 10:23:34,434 INFO server centosvm: server operating mode set to cli 2017-05-11 10:23:34,434 INFOCLI centosvm: /opt/pbs/bin/qstat -Bf centosvm 2017-05-11 10:23:34,446 INFO server centosvm: version 14.0.1 2017-05-11 10:23:34,446 INFO status on centosvm: node centosvm 2017-05-11 10:23:34,447 INFOCLI centosvm: /opt/pbs/bin/pbsnodes -s centosvm -v centosvm 2017-05-11 10:23:34,459 INFOCLI centosvm: /opt/pbs/bin/pbsnodes -s centosvm -v centosvm 2017-05-11 10:23:34,473 INFO expect on server centosvm: state = free node centosvm ... OK 2017-05-11 10:23:34,474 INFO ============================ 2017-05-11 10:23:34,474 INFO Completed TestPbsPP745 setUp 2017-05-11 10:23:34,474 INFO ============================ 2017-05-11 10:23:34,474 INFO manager on centosvm as root: delete resource foo_str 2017-05-11 10:23:34,474 INFOCLI centosvm: sudo -H -u root /opt/pbs/bin/qmgr -c delete resource foo_str 2017-05-11 10:23:34,497 ERROR err: ['qmgr obj=foo_str svr=default: Unknown resource', 'qmgr: Error (15035) returned from server'] 2017-05-11 10:23:34,497 INFO manager on centosvm as root: delete resource foo_f 2017-05-11 10:23:34,497 INFOCLI centosvm: sudo -H -u root /opt/pbs/bin/qmgr -c delete resource foo_f 2017-05-11 10:23:34,518 ERROR err: ['qmgr obj=foo_f svr=default: Unknown resource', 'qmgr: Error (15035) returned from server'] 2017-05-11 10:23:34,518 INFO manager on centosvm as root: delete resource foo_i 2017-05-11 10:23:34,518 INFOCLI centosvm: sudo -H -u root /opt/pbs/bin/qmgr -c delete resource foo_i 2017-05-11 10:23:34,543 ERROR err: ['qmgr obj=foo_i svr=default: Unknown resource', 'qmgr: Error (15035) returned from server'] 2017-05-11 10:23:34,543 INFO manager on centosvm as root: delete resource foo_str2 2017-05-11 10:23:34,543 INFOCLI centosvm: sudo -H -u root /opt/pbs/bin/qmgr -c delete resource foo_str2 2017-05-11 10:23:34,564 ERROR err: ['qmgr obj=foo_str2 svr=default: Unknown resource', 'qmgr: Error (15035) returned from server'] 2017-05-11 10:23:34,565 INFO manager on centosvm as root: delete resource foo_str3 2017-05-11 10:23:34,565 INFOCLI centosvm: sudo -H -u root /opt/pbs/bin/qmgr -c delete resource foo_str3 2017-05-11 10:23:34,587 ERROR err: ['qmgr obj=foo_str3 svr=default: Unknown resource', 'qmgr: Error (15035) returned from server'] 2017-05-11 10:23:34,587 INFOCLI centosvm: /opt/pbs/bin/qstat -Bf centosvm 2017-05-11 10:23:34,601 INFO server centosvm: stopping Server on host centosvm 2017-05-11 10:23:34,601 INFOCLI2 centosvm: sudo -H /usr/bin/cat /var/spool/pbs/server_priv/server.lock 2017-05-11 10:23:34,612 INFOCLI2 centosvm: sudo -H kill -TERM 58707 2017-05-11 10:23:34,623 INFOCLI2 centosvm: sudo -H /usr/bin/cat /var/spool/pbs/server_priv/server.lock 2017-05-11 10:23:39,901 INFO service: starting pbs_server 2017-05-11 10:23:39,902 INFOCLI centosvm: sudo -H /opt/pbs/sbin/pbs_server 2017-05-11 10:23:46,498 INFOCLI2 centosvm: sudo -H /usr/bin/cat /var/spool/pbs/server_priv/server.lock 2017-05-11 10:23:46,576 INFOCLI centosvm: /opt/pbs/bin/qstat -Bf centosvm 2017-05-11 10:23:46,599 INFO mom centosvm: stopping MoM on host centosvm 2017-05-11 10:23:46,599 INFOCLI2 centosvm: sudo -H kill -TERM 58278 2017-05-11 10:23:46,619 INFOCLI2 centosvm: sudo -H /usr/bin/cat /var/spool/pbs/mom_priv/mom.lock 2017-05-11 10:23:46,679 INFO service: starting pbs_mom -d /var/spool/pbs 2017-05-11 10:23:46,680 INFOCLI centosvm: sudo -H /opt/pbs/sbin/pbs_mom -d /var/spool/pbs 2017-05-11 10:23:46,798 INFOCLI2 centosvm: sudo -H /usr/bin/cat /var/spool/pbs/mom_priv/mom.lock 2017-05-11 10:23:46,827 INFOCLI2 centosvm: sudo -H /usr/bin/rm -rf /var/spool/pbs/mom_priv/config.d 2017-05-11 10:23:46,840 INFO manager on centosvm: delete node 2017-05-11 10:23:46,840 INFOCLI centosvm: /opt/pbs/bin/qmgr -c delete node @default 2017-05-11 10:23:46,855 INFO manager on centosvm: create node centosvm 2017-05-11 10:23:46,855 INFOCLI centosvm: /opt/pbs/bin/qmgr -c create node centosvm 2017-05-11 10:23:46,873 INFOCLI centosvm: /opt/pbs/bin/pbsnodes -s centosvm -v centosvm 2017-05-11 10:23:46,887 INFO expect on server centosvm: state = free node centosvm got: state = state-unknown,down 2017-05-11 10:23:47,391 INFOCLI centosvm: /opt/pbs/bin/pbsnodes -s centosvm -v centosvm 2017-05-11 10:23:47,431 INFO expect on server centosvm: state = free node centosvm attempt: 2 got: state = state-unknown,down 2017-05-11 10:23:47,934 INFOCLI centosvm: /opt/pbs/bin/pbsnodes -s centosvm -v centosvm 2017-05-11 10:23:47,976 INFO expect on server centosvm: state = free node centosvm attempt: 3 got: state = state-unknown,down 2017-05-11 10:23:48,478 INFOCLI centosvm: /opt/pbs/bin/pbsnodes -s centosvm -v centosvm 2017-05-11 10:23:48,541 INFO expect on server centosvm: state = free node centosvm attempt: 4 ... OK 2017-05-11 10:23:48,542 INFO manager on centosvm: set server {'job_history_enable': 'True'} 2017-05-11 10:23:48,543 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server job_history_enable=True 2017-05-11 10:23:48,571 INFO manager on centosvm as root: create resource foo_str2 {'flag': 'h', 'type': 'string'} 2017-05-11 10:23:48,572 INFOCLI centosvm: sudo -H -u root /opt/pbs/bin/qmgr -c create resource foo_str2 flag=h,type=string 2017-05-11 10:23:48,621 INFO manager on centosvm as root: create resource foo_str3 {'flag': 'h', 'type': 'string'} 2017-05-11 10:23:48,621 INFOCLI centosvm: sudo -H -u root /opt/pbs/bin/qmgr -c create resource foo_str3 flag=h,type=string 2017-05-11 10:23:48,655 INFO manager on centosvm as root: create resource stra {'flag': 'h', 'type': 'string_array'} 2017-05-11 10:23:48,655 INFOCLI centosvm: sudo -H -u root /opt/pbs/bin/qmgr -c create resource stra flag=h,type=string_array 2017-05-11 10:23:48,687 INFO status on centosvm: hook 2017-05-11 10:23:48,687 INFO manager on centosvm: list hook 2017-05-11 10:23:48,688 INFOCLI centosvm: sudo -H /opt/pbs/bin/qmgr -c list hook 2017-05-11 10:23:48,707 INFO status on centosvm: hook 2017-05-11 10:23:48,708 INFO manager on centosvm: list hook 2017-05-11 10:23:48,708 INFOCLI centosvm: sudo -H /opt/pbs/bin/qmgr -c list hook 2017-05-11 10:23:48,736 INFO manager on centosvm: create hook start 2017-05-11 10:23:48,736 INFOCLI centosvm: sudo -H /opt/pbs/bin/qmgr -c create hook start 2017-05-11 10:23:48,757 INFO manager on centosvm: set hook start {'enabled': 'True', 'event': 'exechost_startup'} 2017-05-11 10:23:48,757 INFOCLI centosvm: sudo -H /opt/pbs/bin/qmgr -c set hook start enabled=True,event=exechost_startup 2017-05-11 10:23:48,785 INFOCLI centosvm: sudo -H /opt/pbs/bin/qmgr -c list hook start 2017-05-11 10:23:48,806 INFO expect on server centosvm: enabled set True || event set exechost_startup hook start ... OK 2017-05-11 10:23:48,806 INFO manager on centosvm: import hook start {'content-type': 'application/x-python', 'input-file': '/tmp/PtlPbsPIjZKR', 'content-encoding': 'default'} 2017-05-11 10:23:48,807 INFOCLI centosvm: sudo -H /opt/pbs/bin/qmgr -c import hook start application/x-python default /tmp/PtlPbsPIjZKR 2017-05-11 10:23:48,832 INFO server centosvm: imported hook body --- import pbs e=pbs.event() localnode=pbs.get_local_nodename() e.vnode_list[localnode].resources_available['foo_i'] = 7 e.vnode_list[localnode].resources_available['foo_f'] = 5.0 e.vnode_list[localnode].resources_available['foo_str'] = "seventyseven" e.vnode_list[localnode].resources_available['foo_str2'] = "seven" --- 2017-05-11 10:23:48,833 INFO mom centosvm: sent signal -HUP 2017-05-11 10:23:48,833 INFOCLI2 centosvm: sudo -H kill -HUP 60071 2017-05-11 10:23:48,846 INFO mom centosvm log match: searching for "resourcedef;copy hook-related file"... OK 2017-05-11 10:23:48,847 INFO test_epilogue_single_node 2017-05-11 10:23:48,847 INFO status on centosvm: hook 2017-05-11 10:23:48,847 INFO manager on centosvm: list hook 2017-05-11 10:23:48,847 INFOCLI centosvm: sudo -H /opt/pbs/bin/qmgr -c list hook 2017-05-11 10:23:48,870 INFO status on centosvm: hook 2017-05-11 10:23:48,871 INFO manager on centosvm: list hook 2017-05-11 10:23:48,871 INFOCLI centosvm: sudo -H /opt/pbs/bin/qmgr -c list hook 2017-05-11 10:23:48,891 INFO manager on centosvm: create hook epi 2017-05-11 10:23:48,892 INFOCLI centosvm: sudo -H /opt/pbs/bin/qmgr -c create hook epi 2017-05-11 10:23:48,916 INFO manager on centosvm: set hook epi {'enabled': 'True', 'event': 'execjob_epilogue'} 2017-05-11 10:23:48,916 INFOCLI centosvm: sudo -H /opt/pbs/bin/qmgr -c set hook epi enabled=True,event=execjob_epilogue 2017-05-11 10:23:48,947 INFOCLI centosvm: sudo -H /opt/pbs/bin/qmgr -c list hook epi 2017-05-11 10:23:48,968 INFO expect on server centosvm: enabled set True || event set execjob_epilogue hook epi ... OK 2017-05-11 10:23:48,969 INFO manager on centosvm: import hook epi {'content-type': 'application/x-python', 'input-file': '/tmp/PtlPbscWJTKa', 'content-encoding': 'default'} 2017-05-11 10:23:48,969 INFOCLI centosvm: sudo -H /opt/pbs/bin/qmgr -c import hook epi application/x-python default /tmp/PtlPbscWJTKa 2017-05-11 10:23:48,995 INFO server centosvm: imported hook body --- import pbs e=pbs.event() pbs.logmsg(pbs.LOG_DEBUG, "executed epilogue hook") e.job.resources_used["vmem"] = pbs.size("9gb") e.job.resources_used["foo_i"] = 9 e.job.resources_used["foo_f"] = 0.09 e.job.resources_used["foo_str"] = '{"seven":7}' e.job.resources_used["foo_str3"] = """{"a":6,"b":"some value #$%^&*@","c":54.4,"d":"32.5gb"}""" e.job.resources_used["foo_str2"] = "seven" e.job.resources_used["cput"] = 10 e.job.resources_used["stra"] = '"glad,elated","happy"' --- 2017-05-11 10:23:48,996 INFO job: executable set to /bin/sleep with arguments: 100 2017-05-11 10:23:48,996 INFO job: executable set to /bin/sleep with arguments: 10 2017-05-11 10:23:48,996 INFOCLI centosvm: sudo -H -u pbsuser /opt/pbs/bin/qsub -l place=scatter -l walltime=10 -l select=1:ncpus=1 -- /bin/sleep 10 2017-05-11 10:23:49,025 INFO submit to centosvm as pbsuser: job 0.centosvm OrderedDict([('Resource_List.place', 'scatter'), ('Resource_List.walltime', 10), ('Resource_List.select', '1:ncpus=1')]) 2017-05-11 10:23:49,025 INFO server centosvm: expect offset set to 10 2017-05-11 10:23:59,026 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:23:59,070 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm 2017-05-11 10:23:59,573 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:23:59,620 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:23:59,647 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 2 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:00,149 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:00,196 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:00,222 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 3 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:00,724 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:00,772 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:00,795 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 4 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:01,298 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:01,347 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:01,386 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 5 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:01,888 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:01,936 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:01,968 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 6 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:02,470 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:02,520 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:02,548 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 7 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:03,051 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:03,100 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:03,137 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 8 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:03,640 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:03,687 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:03,725 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 9 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:04,227 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:04,275 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:04,308 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 10 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:04,809 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:04,860 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:04,888 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 11 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:05,391 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:05,439 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:05,475 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 12 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:05,978 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:06,026 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:06,054 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 13 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:06,558 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:06,605 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:06,643 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 14 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:07,147 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:07,195 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:07,232 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 15 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:07,734 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:07,781 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:07,815 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 16 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:08,319 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:08,367 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:08,397 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 17 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:08,901 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:08,948 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:08,977 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 18 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:09,480 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:09,529 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:09,566 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 19 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:10,068 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:10,116 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:10,144 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 20 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:10,645 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:10,694 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:10,736 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 21 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:11,238 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:11,285 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:11,321 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 22 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:11,822 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:11,868 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:11,900 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 23 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:12,404 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:12,489 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:12,529 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 24 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:13,032 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:13,084 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:13,121 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 25 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:13,626 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:13,673 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:13,712 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 26 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:14,215 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:14,262 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:14,288 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 27 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:14,790 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:14,838 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:14,863 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 28 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:15,366 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:15,414 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:15,445 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 29 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:15,948 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:15,995 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:16,033 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 30 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:16,534 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:16,581 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:16,613 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 31 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:17,116 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:17,162 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:17,193 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 32 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:17,696 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:17,745 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:17,785 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 33 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:18,287 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:18,333 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:18,366 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 34 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:18,868 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:18,931 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:18,957 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 35 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:19,460 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:19,510 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:19,532 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 36 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:20,034 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:20,083 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:20,109 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 37 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:20,612 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:20,656 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:20,682 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 38 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:21,185 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:21,234 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:21,263 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 39 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:21,766 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:21,806 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:21,828 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 40 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:22,331 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:22,381 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:22,410 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 41 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:22,914 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:22,962 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:22,999 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 42 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:23,503 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:23,552 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:23,581 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 43 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:24,082 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:24,130 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:24,156 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 44 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:24,658 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:24,684 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:24,705 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 45 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:25,207 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:25,253 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:25,282 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 46 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:25,783 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:25,833 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:25,870 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 47 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:26,373 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:26,420 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:26,463 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 48 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:26,967 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:27,020 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:27,059 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 49 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:27,562 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:27,611 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:27,647 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 50 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:28,149 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:28,198 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:28,236 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 51 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:28,738 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:28,786 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:28,824 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 52 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:29,327 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:29,375 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:29,416 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 53 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:29,917 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:29,960 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:29,988 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 54 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:30,491 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:30,540 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:30,577 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 55 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:31,081 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:31,126 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:31,151 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 56 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:31,653 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:31,703 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:31,740 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 57 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:32,243 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:32,291 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:32,330 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 58 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:32,833 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:32,881 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:32,922 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 59 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:33,427 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:33,476 INFOCLI centosvm: /opt/pbs/bin/qstat -x -f 0.centosvm 2017-05-11 10:24:33,504 INFO expect on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 60 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:34,005 INFOCLI centosvm: /opt/pbs/bin/qmgr -c set server scheduling=True 2017-05-11 10:24:34,056 INFO ERROR 2017-05-11 10:24:34,056 INFO Set {'PBS_SERVER': 'centosvm'} in /etc/pbs.conf 2017-05-11 10:24:34,062 INFOCLI2 centosvm: sudo -H /usr/bin/chown root /tmp/PtlPbsaGNuuf 2017-05-11 10:24:34,081 INFOCLI2 centosvm: sudo -H /usr/bin/chgrp root /tmp/PtlPbsaGNuuf 2017-05-11 10:24:34,111 INFOCLI2 centosvm: sudo -H /usr/bin/rm /tmp/PtlPbsaGNuuf 2017-05-11 10:24:34,124 INFO ============================== 2017-05-11 10:24:34,124 INFO Entered TestPbsPP745 tearDown 2017-05-11 10:24:34,125 INFO ============================== 2017-05-11 10:24:34,125 INFO =============================== 2017-05-11 10:24:34,125 INFO Completed TestPbsPP745 tearDown 2017-05-11 10:24:34,125 INFO =============================== 2017-05-11 10:24:34,125 INFO manager on centosvm as root: delete resource foo_str 2017-05-11 10:24:34,125 INFOCLI centosvm: sudo -H -u root /opt/pbs/bin/qmgr -c delete resource foo_str 2017-05-11 10:24:34,170 INFO manager on centosvm as root: delete resource foo_f 2017-05-11 10:24:34,170 INFOCLI centosvm: sudo -H -u root /opt/pbs/bin/qmgr -c delete resource foo_f 2017-05-11 10:24:34,213 INFO manager on centosvm as root: delete resource foo_i 2017-05-11 10:24:34,213 INFOCLI centosvm: sudo -H -u root /opt/pbs/bin/qmgr -c delete resource foo_i 2017-05-11 10:24:34,252 INFO manager on centosvm as root: delete resource foo_str2 2017-05-11 10:24:34,252 INFOCLI centosvm: sudo -H -u root /opt/pbs/bin/qmgr -c delete resource foo_str2 2017-05-11 10:24:34,291 INFO manager on centosvm as root: delete resource foo_str3 2017-05-11 10:24:34,292 INFOCLI centosvm: sudo -H -u root /opt/pbs/bin/qmgr -c delete resource foo_str3 2017-05-11 10:24:34,330 INFOCLI centosvm: /opt/pbs/bin/qstat -Bf centosvm 2017-05-11 10:24:34,345 INFO server centosvm: stopping Server on host centosvm 2017-05-11 10:24:34,345 INFOCLI2 centosvm: sudo -H kill -TERM 60057 2017-05-11 10:24:34,356 INFOCLI2 centosvm: sudo -H /usr/bin/cat /var/spool/pbs/server_priv/server.lock 2017-05-11 10:24:39,573 INFO service: starting pbs_server 2017-05-11 10:24:39,574 INFOCLI centosvm: sudo -H /opt/pbs/sbin/pbs_server 2017-05-11 10:24:45,124 INFOCLI2 centosvm: sudo -H /usr/bin/cat /var/spool/pbs/server_priv/server.lock 2017-05-11 10:24:45,187 INFOCLI centosvm: /opt/pbs/bin/qstat -Bf centosvm 2017-05-11 10:24:45,202 INFO mom centosvm: stopping MoM on host centosvm 2017-05-11 10:24:45,202 INFOCLI2 centosvm: sudo -H kill -TERM 60071 2017-05-11 10:24:45,214 INFOCLI2 centosvm: sudo -H /usr/bin/cat /var/spool/pbs/mom_priv/mom.lock 2017-05-11 10:24:45,240 INFO service: starting pbs_mom -d /var/spool/pbs 2017-05-11 10:24:45,240 INFOCLI centosvm: sudo -H /opt/pbs/sbin/pbs_mom -d /var/spool/pbs 2017-05-11 10:24:45,306 INFOCLI2 centosvm: sudo -H /usr/bin/cat /var/spool/pbs/mom_priv/mom.lock 2017-05-11 10:24:45,354 INFO 2017-05-11 10:24:45,354 INFO ====================================================================== 2017-05-11 10:24:45,354 INFO ERROR: test_epilogue_single_node (tests.functional.pbs_resc_used_single_node.TestPbsPP745) 2017-05-11 10:24:45,355 INFO ___m_oo_m___ 2017-05-11 10:24:45,355 INFO Traceback (most recent call last): File "/home/dilipk/workspace/pbspro/test/tests/functional/pbs_resc_used_single_node.py", line 149, in test_epilogue_single_node extend='x', offset=10, id=jid) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8418, in expect extend, level=level, msg=" ".join(msg)) File "/usr/lib/python2.7/site-packages/PbsTestLab-1.0.0-py2.7.egg/ptl/lib/pbs_testlib.py", line 8227, in expect raise PtlExpectError(rc=1, rv=False, msg=_msg) PtlExpectError: rc=1, rv=False, msg=expected on server centosvm: resources_used.stra = "glad,elated","happy" || resources_used.cput = 00:00:10 || resources_used.foo_i = 9 || resources_used.foo_str2 = seven || resources_used.ncpus = 1 || job_state = F || resources_used.foo_f = 0.09 || resources_used.foo_str = '{"seven": 7}' || resources_used.vmem = 9gb job 0.centosvm attempt: 60 got: resources_used.foo_str = {"seven":7} 2017-05-11 10:24:45,360 INFO ================================================================================ error: test_epilogue_single_node (tests.functional.pbs_resc_used_single_node.TestPbsPP745) Test cases with failures: TestPbsPP745.test_epilogue_single_node Test suites with failures: TestPbsPP745 run: 1, succeeded: 0, failed: 0, errors: 1, skipped: 0, timedout: 0 Tests run in 0:01:11.891498 2017-05-11 10:24:45,360 INFO Cleaning up temporary files