pbs_benchpress does not throw error when value of --cumulative-tc-failure-threshold is greater than the value of --tc-failure-threshold

Description

pbs_benchpress does not throw error when value of --cumulative-tc-failure-threshold is greater than the value of --tc-failure-threshold

For example:

Test suites with inserted failures for testing:
FailSmokeResvTest_ext, (2 failures, 2 pass)
FailSmokeTest, (3 failures, 2 pass)
OneFailSmokeTest (1 failures)

Both the commands below exit without any error output, but with exit status 1:

saritah@x19-64-sles11-altix:~/ptlpost/tests> /opt/pbs/python/bin/python /tmp/skh/OPTL/bin/pbs_benchpress -t FailSmokeTest,OneFailSmokeTest,FailSmokeResvTest_ext --cumulative-tc-failure-threshold=2 --max-postdata-threshold=1 --tc-failure-threshold=3 --post-analysis-data /home/saritah/out14 -o /home/saritah/31jul_out14
saritah@x19-64-sles11-altix:~/ptlpost/tests> echo $?
1
saritah@x19-64-sles11-altix:~/ptlpost/tests> /opt/pbs/python/bin/python /tmp/skh/OPTL/bin/pbs_benchpress -t FailSmokeTest,OneFailSmokeTest,FailSmokeResvTest_ext --cumulative-tc-failure-threshold=5 --post-analysis-data /home/saritah/out16 -o /home/saritah/31jul_out16
saritah@x19-64-sles11-altix:~/ptlpost/tests> echo $?
1
saritah@x19-64-sles11-altix:~/ptlpost/tests>

In both the cases the value of --cumulative-tc-failure-threshold is greater than the specified or default value of --tc-failure-threshold value.

This is not in accordance with the expected behavior mentioned in PP-302: Implement save of PBS data for post-run analysis (https://pbspro.atlassian.net/wiki/display/PD/PP-302%3A+Implement+save+of+PBS+data+for+post-run+analysis):

<count> should be greater or equal to 'tc-failure-threshold', if <count> is less than 'tc-failure-threshold' then PTL will bail out with error message "Value for cumulative-tc-failure-threshould should be greater or equal to 'tc-failure-threshold'". <count>==0 will disable this threshold.

Acceptance Criteria

None

Status

Assignee

Unassigned

Reporter

sarita kh

Severity

2-Medium

OS

None

Start Date

None

Pull Request URL

None

Story Points

1

Components

Priority

High
Configure