doc/testing.md
author ihse
Fri, 09 Mar 2018 19:12:07 +0100
changeset 49200 a19a6228cdb2
parent 48058 c5eefa465e37
child 49293 9e1c1889136e
permissions -rw-r--r--
8199405: Bump lowest supported gcc to 4.8 Reviewed-by: erikj
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
44511
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
     1
% Testing OpenJDK
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
     2
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
     3
## Using the run-test framework
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
     4
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
     5
This new way of running tests is developer-centric. It assumes that you have
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
     6
built a jdk locally and want to test it. Running common test targets is simple,
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
     7
and more complex ad-hoc combination of tests is possible. The user interface is
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
     8
forgiving, and clearly report errors it cannot resolve.
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
     9
45266
29347d0c7122 8180651: Make target to run tests on exploded image
erikj
parents: 44983
diff changeset
    10
The main target "run-test" uses the jdk-image as the tested product. There is
29347d0c7122 8180651: Make target to run tests on exploded image
erikj
parents: 44983
diff changeset
    11
also an alternate target "exploded-run-test" that uses the exploded image
29347d0c7122 8180651: Make target to run tests on exploded image
erikj
parents: 44983
diff changeset
    12
instead. Not all tests will run successfully on the exploded image, but using
29347d0c7122 8180651: Make target to run tests on exploded image
erikj
parents: 44983
diff changeset
    13
this target can greatly improve rebuild times for certain workflows.
29347d0c7122 8180651: Make target to run tests on exploded image
erikj
parents: 44983
diff changeset
    14
44511
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    15
Some example command-lines:
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    16
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    17
    $ make run-test-tier1
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    18
    $ make run-test-jdk_lang JTREG="JOBS=8"
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    19
    $ make run-test TEST=jdk_lang
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    20
    $ make run-test-only TEST="gtest:LogTagSet gtest:LogTagSetDescriptions" GTEST="REPEAT=-1"
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    21
    $ make run-test TEST="hotspot/test:hotspot_gc" JTREG="JOBS=1;TIMEOUT=8;VM_OTIONS=-XshowSettings -Xlog:gc+ref=debug"
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    22
    $ make run-test TEST="jtreg:hotspot/test:hotspot_gc hotspot/test/native_sanity/JniVersion.java"
45266
29347d0c7122 8180651: Make target to run tests on exploded image
erikj
parents: 44983
diff changeset
    23
    $ make exploded-run-test TEST=hotspot_tier1
44511
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    24
45618
74f490ba783a 8179537: Update testing.md for more clarity regarding JTReg configuration
ihse
parents: 44983
diff changeset
    25
### Configuration
74f490ba783a 8179537: Update testing.md for more clarity regarding JTReg configuration
ihse
parents: 44983
diff changeset
    26
74f490ba783a 8179537: Update testing.md for more clarity regarding JTReg configuration
ihse
parents: 44983
diff changeset
    27
To be able to run JTReg tests, `configure` needs to know where to find the
74f490ba783a 8179537: Update testing.md for more clarity regarding JTReg configuration
ihse
parents: 44983
diff changeset
    28
JTReg test framework. If it is not picked up automatically by configure, use
74f490ba783a 8179537: Update testing.md for more clarity regarding JTReg configuration
ihse
parents: 44983
diff changeset
    29
the `--with-jtreg=<path to jtreg home>` option to point to the JTReg framework.
74f490ba783a 8179537: Update testing.md for more clarity regarding JTReg configuration
ihse
parents: 44983
diff changeset
    30
Note that this option should point to the JTReg home, i.e. the top directory,
74f490ba783a 8179537: Update testing.md for more clarity regarding JTReg configuration
ihse
parents: 44983
diff changeset
    31
containing `lib/jtreg.jar` etc. (An alternative is to set the `JT_HOME`
74f490ba783a 8179537: Update testing.md for more clarity regarding JTReg configuration
ihse
parents: 44983
diff changeset
    32
environment variable to point to the JTReg home before running `configure`.)
74f490ba783a 8179537: Update testing.md for more clarity regarding JTReg configuration
ihse
parents: 44983
diff changeset
    33
44511
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    34
## Test selection
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    35
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    36
All functionality is available using the run-test make target. In this use
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    37
case, the test or tests to be executed is controlled using the `TEST` variable.
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    38
To speed up subsequent test runs with no source code changes, run-test-only can
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    39
be used instead, which do not depend on the source and test image build.
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    40
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    41
For some common top-level tests, direct make targets have been generated. This
45618
74f490ba783a 8179537: Update testing.md for more clarity regarding JTReg configuration
ihse
parents: 44983
diff changeset
    42
includes all JTReg test groups, the hotspot gtest, and custom tests (if
44511
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    43
present). This means that `make run-test-tier1` is equivalent to `make run-test
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    44
TEST="tier1"`, but the latter is more tab-completion friendly. For more complex
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    45
test runs, the `run-test TEST="x"` solution needs to be used.
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    46
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    47
The test specifications given in `TEST` is parsed into fully qualified test
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    48
descriptors, which clearly and unambigously show which tests will be run. As an
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    49
example, `:tier1` will expand to `jtreg:jdk/test:tier1
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    50
jtreg:langtools/test:tier1 jtreg:nashorn/test:tier1 jtreg:jaxp/test:tier1`. You
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    51
can always submit a list of fully qualified test descriptors in the `TEST`
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    52
variable if you want to shortcut the parser.
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    53
45618
74f490ba783a 8179537: Update testing.md for more clarity regarding JTReg configuration
ihse
parents: 44983
diff changeset
    54
### JTReg
44511
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    55
45618
74f490ba783a 8179537: Update testing.md for more clarity regarding JTReg configuration
ihse
parents: 44983
diff changeset
    56
JTReg test groups can be specified either without a test root, e.g. `:tier1`
44511
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    57
(or `tier1`, the initial colon is optional), or with, e.g.
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    58
`hotspot/test:tier1`, `jdk/test:jdk_util`.
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    59
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    60
When specified without a test root, all matching groups from all tests roots
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    61
will be added. Otherwise, only the group from the specified test root will be
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    62
added.
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    63
45618
74f490ba783a 8179537: Update testing.md for more clarity regarding JTReg configuration
ihse
parents: 44983
diff changeset
    64
Individual JTReg tests or directories containing JTReg tests can also be
44511
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    65
specified, like `hotspot/test/native_sanity/JniVersion.java` or
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    66
`hotspot/test/native_sanity`. You can also specify an absolute path, to point
45618
74f490ba783a 8179537: Update testing.md for more clarity regarding JTReg configuration
ihse
parents: 44983
diff changeset
    67
to a JTReg test outside the source tree.
44511
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    68
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    69
As long as the test groups or test paths can be uniquely resolved, you do not
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    70
need to enter the `jtreg:` prefix. If this is not possible, or if you want to
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    71
use a fully qualified test descriptor, add `jtreg:`, e.g.
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    72
`jtreg:hotspot/test/native_sanity`.
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    73
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    74
### Gtest
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    75
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    76
Since the Hotspot Gtest suite is so quick, the default is to run all tests.
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    77
This is specified by just `gtest`, or as a fully qualified test descriptor
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    78
`gtest:all`.
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    79
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    80
If you want, you can single out an individual test or a group of tests, for
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    81
instance `gtest:LogDecorations` or `gtest:LogDecorations.level_test_vm`. This
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    82
can be particularly useful if you want to run a shaky test repeatedly.
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    83
48058
c5eefa465e37 8192995: run-test gtest should use all jvm variants, not just "server"
ihse
parents: 47216
diff changeset
    84
For Gtest, there is a separate test suite for each JVM variant. The JVM variant
c5eefa465e37 8192995: run-test gtest should use all jvm variants, not just "server"
ihse
parents: 47216
diff changeset
    85
is defined by adding `/<variant>` to the test descriptor, e.g.
c5eefa465e37 8192995: run-test gtest should use all jvm variants, not just "server"
ihse
parents: 47216
diff changeset
    86
`gtest:Log/client`. If you specify no variant, gtest will run once for each JVM
c5eefa465e37 8192995: run-test gtest should use all jvm variants, not just "server"
ihse
parents: 47216
diff changeset
    87
variant present (e.g. server, client). So if you only have the server JVM
c5eefa465e37 8192995: run-test gtest should use all jvm variants, not just "server"
ihse
parents: 47216
diff changeset
    88
present, then `gtest:all` will be equivalent to `gtest:all/server`.
c5eefa465e37 8192995: run-test gtest should use all jvm variants, not just "server"
ihse
parents: 47216
diff changeset
    89
44511
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    90
## Test results and summary
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    91
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    92
At the end of the test run, a summary of all tests run will be presented. This
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    93
will have a consistent look, regardless of what test suites were used. This is
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    94
a sample summary:
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    95
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    96
    ==============================
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    97
    Test summary
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    98
    ==============================
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
    99
       TEST                                          TOTAL  PASS  FAIL ERROR
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   100
    >> jtreg:jdk/test:tier1                           1867  1865     2     0 <<
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   101
       jtreg:langtools/test:tier1                     4711  4711     0     0
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   102
       jtreg:nashorn/test:tier1                        133   133     0     0
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   103
    ==============================
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   104
    TEST FAILURE
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   105
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   106
Tests where the number of TOTAL tests does not equal the number of PASSed tests
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   107
will be considered a test failure. These are marked with the `>> ... <<` marker
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   108
for easy identification.
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   109
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   110
The classification of non-passed tests differs a bit between test suites. In
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   111
the summary, ERROR is used as a catch-all for tests that neither passed nor are
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   112
classified as failed by the framework. This might indicate test framework
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   113
error, timeout or other problems.
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   114
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   115
In case of test failures, `make run-test` will exit with a non-zero exit value.
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   116
45618
74f490ba783a 8179537: Update testing.md for more clarity regarding JTReg configuration
ihse
parents: 44983
diff changeset
   117
All tests have their result stored in `build/$BUILD/test-results/$TEST_ID`,
44511
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   118
where TEST_ID is a path-safe conversion from the fully qualified test
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   119
descriptor, e.g. for `jtreg:jdk/test:tier1` the TEST_ID is
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   120
`jtreg_jdk_test_tier1`. This path is also printed in the log at the end of the
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   121
test run.
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   122
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   123
Additional work data is stored in `build/$BUILD/test-support/$TEST_ID`. For
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   124
some frameworks, this directory might contain information that is useful in
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   125
determining the cause of a failed test.
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   126
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   127
## Test suite control
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   128
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   129
It is possible to control various aspects of the test suites using make control
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   130
variables.
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   131
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   132
These variables use a keyword=value approach to allow multiple values to be
45618
74f490ba783a 8179537: Update testing.md for more clarity regarding JTReg configuration
ihse
parents: 44983
diff changeset
   133
set. So, for instance, `JTREG="JOBS=1;TIMEOUT=8"` will set the JTReg
44511
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   134
concurrency level to 1 and the timeout factor to 8. This is equivalent to
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   135
setting `JTREG_JOBS=1 JTREG_TIMEOUT=8`, but using the keyword format means that
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   136
the `JTREG` variable is parsed and verified for correctness, so
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   137
`JTREG="TMIEOUT=8"` would give an error, while `JTREG_TMIEOUT=8` would just
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   138
pass unnoticed.
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   139
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   140
To separate multiple keyword=value pairs, use `;` (semicolon). Since the shell
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   141
normally eats `;`, the recommended usage is to write the assignment inside
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   142
qoutes, e.g. `JTREG="...;..."`. This will also make sure spaces are preserved,
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   143
as in `JTREG="VM_OTIONS=-XshowSettings -Xlog:gc+ref=debug"`.
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   144
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   145
(Other ways are possible, e.g. using backslash: `JTREG=JOBS=1\;TIMEOUT=8`.
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   146
Also, as a special technique, the string `%20` will be replaced with space for
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   147
certain options, e.g. `JTREG=VM_OTIONS=-XshowSettings%20-Xlog:gc+ref=debug`.
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   148
This can be useful if you have layers of scripts and have trouble getting
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   149
proper quoting of command line arguments through.)
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   150
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   151
As far as possible, the names of the keywords have been standardized between
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   152
test suites.
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   153
45618
74f490ba783a 8179537: Update testing.md for more clarity regarding JTReg configuration
ihse
parents: 44983
diff changeset
   154
### JTReg keywords
44511
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   155
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   156
#### JOBS
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   157
The test concurrency (`-concurrency`).
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   158
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   159
Defaults to TEST_JOBS (if set by `--with-test-jobs=`), otherwise it defaults to
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   160
JOBS, except for Hotspot, where the default is *number of CPU cores/2*, but
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   161
never more than 12.
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   162
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   163
#### TIMEOUT
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   164
The timeout factor (`-timeoutFactor`).
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   165
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   166
Defaults to 4.
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   167
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   168
#### TEST_MODE
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   169
The test mode (`-agentvm`, `-samevm` or `-othervm`).
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   170
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   171
Defaults to `-agentvm`.
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   172
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   173
#### ASSERT
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   174
Enable asserts (`-ea -esa`, or none).
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   175
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   176
Set to `true` or `false`. If true, adds `-ea -esa`. Defaults to true, except
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   177
for hotspot.
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   178
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   179
#### VERBOSE
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   180
The verbosity level (`-verbose`).
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   181
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   182
Defaults to `fail,error,summary`.
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   183
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   184
#### RETAIN
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   185
What test data to retain (`-retain`).
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   186
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   187
Defaults to `fail,error`.
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   188
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   189
#### MAX_MEM
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   190
Limit memory consumption (`-Xmx` and `-vmoption:-Xmx`, or none).
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   191
45618
74f490ba783a 8179537: Update testing.md for more clarity regarding JTReg configuration
ihse
parents: 44983
diff changeset
   192
Limit memory consumption for JTReg test framework and VM under test. Set to 0
44511
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   193
to disable the limits.
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   194
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   195
Defaults to 512m, except for hotspot, where it defaults to 0 (no limit).
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   196
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   197
#### OPTIONS
45618
74f490ba783a 8179537: Update testing.md for more clarity regarding JTReg configuration
ihse
parents: 44983
diff changeset
   198
Additional options to the JTReg test framework.
44511
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   199
45618
74f490ba783a 8179537: Update testing.md for more clarity regarding JTReg configuration
ihse
parents: 44983
diff changeset
   200
Use `JTREG="OPTIONS=--help all"` to see all available JTReg options.
44511
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   201
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   202
#### JAVA_OPTIONS
45618
74f490ba783a 8179537: Update testing.md for more clarity regarding JTReg configuration
ihse
parents: 44983
diff changeset
   203
Additional Java options to JTReg (`-javaoption`).
44511
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   204
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   205
#### VM_OPTIONS
45618
74f490ba783a 8179537: Update testing.md for more clarity regarding JTReg configuration
ihse
parents: 44983
diff changeset
   206
Additional VM options to JTReg (`-vmoption`).
44511
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   207
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   208
### Gtest keywords
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   209
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   210
#### REPEAT
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   211
The number of times to repeat the tests (`--gtest_repeat`).
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   212
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   213
Default is 1. Set to -1 to repeat indefinitely. This can be especially useful
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   214
combined with `OPTIONS=--gtest_break_on_failure` to reproduce an intermittent
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   215
problem.
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   216
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   217
#### OPTIONS
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   218
Additional options to the Gtest test framework.
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   219
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   220
Use `GTEST="OPTIONS=--help"` to see all available Gtest options.
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   221
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   222
---
44983
4e216c0475ee 8179453: Add a proper SetupProcessMarkdown
ihse
parents: 44511
diff changeset
   223
# Override some definitions in the global css file that are not optimal for
4e216c0475ee 8179453: Add a proper SetupProcessMarkdown
ihse
parents: 44511
diff changeset
   224
# this document.
44511
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   225
header-includes:
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   226
 - '<style type="text/css">pre, code, tt { color: #1d6ae5; }</style>'
f3c27401a860 8177955: Add testing documentation
ihse
parents:
diff changeset
   227
---