18 $ make run-test-jdk_lang JTREG="JOBS=8" |
18 $ make run-test-jdk_lang JTREG="JOBS=8" |
19 $ make run-test TEST=jdk_lang |
19 $ make run-test TEST=jdk_lang |
20 $ make run-test-only TEST="gtest:LogTagSet gtest:LogTagSetDescriptions" GTEST="REPEAT=-1" |
20 $ make run-test-only TEST="gtest:LogTagSet gtest:LogTagSetDescriptions" GTEST="REPEAT=-1" |
21 $ make run-test TEST="hotspot:hotspot_gc" JTREG="JOBS=1;TIMEOUT=8;VM_OPTIONS=-XshowSettings -Xlog:gc+ref=debug" |
21 $ make run-test TEST="hotspot:hotspot_gc" JTREG="JOBS=1;TIMEOUT=8;VM_OPTIONS=-XshowSettings -Xlog:gc+ref=debug" |
22 $ make run-test TEST="jtreg:test/hotspot:hotspot_gc test/hotspot/jtreg/native_sanity/JniVersion.java" |
22 $ make run-test TEST="jtreg:test/hotspot:hotspot_gc test/hotspot/jtreg/native_sanity/JniVersion.java" |
|
23 $ make run-test TEST="micro:java.lang.reflect" MICRO="FORK=1;WARMUP_ITER=2" |
23 $ make exploded-run-test TEST=tier2 |
24 $ make exploded-run-test TEST=tier2 |
24 |
25 |
25 ### Configuration |
26 ### Configuration |
26 |
27 |
27 To be able to run JTReg tests, `configure` needs to know where to find the |
28 To be able to run JTReg tests, `configure` needs to know where to find the |
28 JTReg test framework. If it is not picked up automatically by configure, use |
29 JTReg test framework. If it is not picked up automatically by configure, use |
29 the `--with-jtreg=<path to jtreg home>` option to point to the JTReg framework. |
30 the `--with-jtreg=<path to jtreg home>` option to point to the JTReg framework. |
30 Note that this option should point to the JTReg home, i.e. the top directory, |
31 Note that this option should point to the JTReg home, i.e. the top directory, |
31 containing `lib/jtreg.jar` etc. (An alternative is to set the `JT_HOME` |
32 containing `lib/jtreg.jar` etc. (An alternative is to set the `JT_HOME` |
32 environment variable to point to the JTReg home before running `configure`.) |
33 environment variable to point to the JTReg home before running `configure`.) |
|
34 |
|
35 To be able to run microbenchmarks, `configure` needs to know where to find |
|
36 the JMH dependency. Use `--with-jmh=<path to JMH jars>` to point to a directory |
|
37 containing the core JMH and transitive dependencies. The recommended dependencies |
|
38 can be retrieved by running `sh make/devkit/createJMHBundle.sh`, after which |
|
39 `--with-jmh=build/jmh/jars` should work. |
33 |
40 |
34 ## Test selection |
41 ## Test selection |
35 |
42 |
36 All functionality is available using the run-test make target. In this use |
43 All functionality is available using the run-test make target. In this use |
37 case, the test or tests to be executed is controlled using the `TEST` variable. |
44 case, the test or tests to be executed is controlled using the `TEST` variable. |
95 For Gtest, there is a separate test suite for each JVM variant. The JVM variant |
102 For Gtest, there is a separate test suite for each JVM variant. The JVM variant |
96 is defined by adding `/<variant>` to the test descriptor, e.g. |
103 is defined by adding `/<variant>` to the test descriptor, e.g. |
97 `gtest:Log/client`. If you specify no variant, gtest will run once for each JVM |
104 `gtest:Log/client`. If you specify no variant, gtest will run once for each JVM |
98 variant present (e.g. server, client). So if you only have the server JVM |
105 variant present (e.g. server, client). So if you only have the server JVM |
99 present, then `gtest:all` will be equivalent to `gtest:all/server`. |
106 present, then `gtest:all` will be equivalent to `gtest:all/server`. |
|
107 |
|
108 ### Microbenchmarks |
|
109 |
|
110 Which microbenchmarks to run is selected using a regular expression |
|
111 following the `micro:` test descriptor, e.g., `micro:java.lang.reflect`. This |
|
112 delegates the test selection to JMH, meaning package name, class name and even |
|
113 benchmark method names can be used to select tests. |
|
114 |
|
115 Using special characters like `|` in the regular expression is possible, but |
|
116 needs to be escaped multiple times: `micro:ArrayCopy\\\\\|reflect`. |
100 |
117 |
101 ## Test results and summary |
118 ## Test results and summary |
102 |
119 |
103 At the end of the test run, a summary of all tests run will be presented. This |
120 At the end of the test run, a summary of all tests run will be presented. This |
104 will have a consistent look, regardless of what test suites were used. This is |
121 will have a consistent look, regardless of what test suites were used. This is |
228 #### OPTIONS |
245 #### OPTIONS |
229 Additional options to the Gtest test framework. |
246 Additional options to the Gtest test framework. |
230 |
247 |
231 Use `GTEST="OPTIONS=--help"` to see all available Gtest options. |
248 Use `GTEST="OPTIONS=--help"` to see all available Gtest options. |
232 |
249 |
|
250 ### Microbenchmark keywords |
|
251 |
|
252 #### FORK |
|
253 Override the number of benchmark forks to spawn. Same as specifying `-f <num>`. |
|
254 |
|
255 #### ITER |
|
256 Number of measurement iterations per fork. Same as specifying `-i <num>`. |
|
257 |
|
258 #### TIME |
|
259 Amount of time to spend in each measurement iteration, in seconds. Same as |
|
260 specifying `-r <num>` |
|
261 |
|
262 #### WARMUP_ITER |
|
263 Number of warmup iterations to run before the measurement phase in each fork. |
|
264 Same as specifying `-wi <num>`. |
|
265 |
|
266 #### WARMUP_TIME |
|
267 Amount of time to spend in each warmup iteration. Same as specifying `-w <num>`. |
|
268 |
|
269 #### RESULTS_FORMAT |
|
270 Specify to have the test run save a log of the values. Accepts the same values |
|
271 as `-rff`, i.e., `text`, `csv`, `scsv`, `json`, or `latex`. |
|
272 |
|
273 #### VM_OPTIONS |
|
274 Additional VM arguments to provide to forked off VMs. Same as `-jvmArgs <args>` |
|
275 |
|
276 #### OPTIONS |
|
277 Additional arguments to send to JMH. |
|
278 |
233 --- |
279 --- |
234 # Override some definitions in the global css file that are not optimal for |
280 # Override some definitions in the global css file that are not optimal for |
235 # this document. |
281 # this document. |
236 header-includes: |
282 header-includes: |
237 - '<style type="text/css">pre, code, tt { color: #1d6ae5; }</style>' |
283 - '<style type="text/css">pre, code, tt { color: #1d6ae5; }</style>' |