Test Framework should allow testers to run a test for multiple times
Follow the PBS Pro Design Document Guidelines.
Description:
New pbs_benchpress options
--repeat_tests=<integer>" for how many times test repeat
--repeat-tests-delay=<integer> delay between two repetition
PR Link: https://github.com/openpbs/openpbs/pull/1908
Purpose:
Test repetition is used to find race condition, reproduce bug and run stress test.
Details:
Each repetition includes all submitted tests. next repetition starts after finishing of all tests of first repetition.
- before starting each repetition a message is printed "tests are running (repetition number) time"
If submits three tests then in first repetition all three tests run and after finishing all three tests, next repetition starts and again all three tests run.
- Tester can use delay between two repetition
- default delay is 0 second
At the end we get message " All tests are repeated (repetion count) times"
- we get summary of all repetition at the end.
Example:
I/P
$ pbs_benchpress -t testsuite_1.t1,testsuite_2.t2,testsuite_3.t3 --repeat_tests=3 --repeat-test --repeat-tests-delay=2
O/P
===================
Test are running (1) time
===================
t1:setup start
run
teardown
t2:setup start
run
teardown
t3:setup start
run
teardown
===================
Test are running (2) time
===================
t1:setup start
run
teardown
t2:setup start
run
teardown
t3:setup start
run
teardown
t3:setup start
run
teardown
===================
Test are running (3) time
===================
t1:setup start
run
teardown
t2:setup start
run
teardown
t3:setup start
run
teardown
=============================================
All tests are repeated 3 times
=============================================
run: 9, succeeded: 9, failed: 0, errors: 0, skipped: 0, timedout: 0
Project Documentation Main Page