This page appears in the PTL Developer's Guide.
Excerpt |
---|
You use pbs_benchpress to run or list test suites and test cases. How to run pbs_benchpressGo to pbspro/test/tests in your cloned GitHub directory structure. Syntax: pbs_benchpress <option(s) PrivilegeBy default, pbs_benchpress automatically makes its current user a PBS Manager, via "qmgr -c 's s managers += < current user>". Options to pbs_benchpress--cumulative-tc-failure-threshold=<count> Cumulative test case failure threshold. Defaults to 100. Set count to 0 to disable this threshold. If not zero, must be greater than or equal to 'tc-failure-threshold' --db-access=<path to DB parameter keyword file> Path to a file that defines DB parameters (PostreSQL only); can be absolute or relative. See https://www.postgresql.org/docs/current/static/libpq-connect.html#LIBPQ-PARAMKEYWORDS. You probably need to specify the following parameters: dbname: database name user: user name used to authenticate password: password used to authenticate host: database host address port: connection port number --db-name=<database name> Database where test information and results metrics are stored for later processing. Defaults to ptl_test_results.db. --db-type=<database type> Can be one of "html", "file", "sqlite", "pgsql". Defaults to "html". --eval-tags='<Python expression>' Selects only tests where Python expression evaluates to True. Can be applied multiple times. Example: 'smoke and (not regression)' - Selects all tests which have "smoke" tag and don't have "regression" tag 'priority>4' - Selects all tests which have "priority" tag and where the value of priority is greater than 4 --exclude=<tests to exclude> Comma-separated string of tests to exclude -f <test files to run> Comma-separated list of test file names -F Sets logging format to include timestamp and level --follow-child If set, walks the test hierarchy and runs each test. Starts from current directory. -g <path to test suite file> Absolute path to file containing comma-separated list of test suites Used for very long list of tests --genhtml-bin=<path to genhtml binary> Defaults to "genhtml": PTL assumes genhtml is in PATH -h Displays usage information -i Prints test name and docstring to stdout, then exits. -l <log level> Specifies log level to use. Log levels are documented in PTL. -L Prints the following to stdout, then exits: test suite, file, module, suite docstring, tests, and number of test suites and test cases in current directory and all its subdirectories. --lcov-baseurl=<url> Base URL for lcov HTML report. Uses url as base URL. --lcov-bin=<path to lcov binary> Defaults to "lcov": PTL assumes lcov is in PATH --lcov-data=<path to directory> Path to directory containing .gcno files; can be absolute or relative --lcov-nosrc If set, do not include PBS source in coverage analysis. By default, PBS source is included in coverage analysis --lcov-out=<path to output directory> Output path for HTML report. Default path is TMPDIR/pbscov-YYYYMMDD_HHmmSS --list-tags Recursively list all tags in tests in current directory and its subdirectories. --log-conf=<path to logging config file> Absolute path to logging configuration file. No default. File format specified in https://docs.python.org/2/library/logging.config.html#configuration-file-format. --max-postdata-threshold=<count> Max number of diagnostic tool runs per testsuite. Diagnostic tool is run on failure. Defaults to 10 sets of output from diagnostic tool. Set count to 0 to disable this threshold. --max-pyver=<version> Maximum Python version --min-pyver=<version> Minimum Python version -o <logfile> Name of log file. Logging output is always printed stdout; this option specifies that it should be printed to specified log file as well. -p <test parameter> Comma-separated list of <key>=<value> pairs. Note that the comma cannot be used in value. Pre-defined pairs are in 'Custom parameters:' in http://pbspro.org/ptldocs/ptl.html#ptl.utils.pbs_testsuite.PBSTestSuite but you can add your own along with those. --param-file=<parameter file> If set, gets test parameters from file Overrides -p --post-analysis-data=<post-analysis data directory> Path to directory where diagnostic tool output is placed. --stop-on-failure If set, stop when any test fails -t <test suites> Comma-separated list of test suites to run --tags=<list of tags> Selects only tests that have tag tag. Can be applied multiple times. Format: [!]<tag>[,<tag>] Examples: smoke - Selects all tests which have "smoke" tag !smoke - Selects all tests which don't have "smoke" tag smoke,regression - Selects all tests which have both "smoke" and "regression" tag --tags-info Lists all selected test suites When used with --verbose, lists all test cases Must be used with --tags or --eval-tags You can use -t or --exclude to limit selection --tc-failure-threshold=<count> Test case failure threshold per testsuite Defaults to 10 To disable, set count to 0 --timeout=<seconds> Duration after which no test suites are run --verbose Shows verbose output Must be used with -i, -L or --tags-info --version Prints version number to stdout and exits |