38 containing `lib/jtreg.jar` etc. (An alternative is to set the `JT_HOME` |
38 containing `lib/jtreg.jar` etc. (An alternative is to set the `JT_HOME` |
39 environment variable to point to the JTReg home before running `configure`.) |
39 environment variable to point to the JTReg home before running `configure`.) |
40 |
40 |
41 To be able to run microbenchmarks, `configure` needs to know where to find |
41 To be able to run microbenchmarks, `configure` needs to know where to find |
42 the JMH dependency. Use `--with-jmh=<path to JMH jars>` to point to a directory |
42 the JMH dependency. Use `--with-jmh=<path to JMH jars>` to point to a directory |
43 containing the core JMH and transitive dependencies. The recommended dependencies |
43 containing the core JMH and transitive dependencies. The recommended dependencies |
44 can be retrieved by running `sh make/devkit/createJMHBundle.sh`, after which |
44 can be retrieved by running `sh make/devkit/createJMHBundle.sh`, after which |
45 `--with-jmh=build/jmh/jars` should work. |
45 `--with-jmh=build/jmh/jars` should work. |
46 |
46 |
47 ## Test selection |
47 ## Test selection |
48 |
48 |
49 All functionality is available using the `test` make target. In this use case, |
49 All functionality is available using the `test` make target. In this use case, |
200 proper quoting of command line arguments through.) |
200 proper quoting of command line arguments through.) |
201 |
201 |
202 As far as possible, the names of the keywords have been standardized between |
202 As far as possible, the names of the keywords have been standardized between |
203 test suites. |
203 test suites. |
204 |
204 |
|
205 ### General keywords (TEST_OPTS) |
|
206 |
|
207 Some keywords are valid across different test suites. If you want to run |
|
208 tests from multiple test suites, or just don't want to care which test suite specific |
|
209 control variable to use, then you can use the general TEST_OPTS control variable. |
|
210 |
|
211 There are also some keywords that applies globally to the test runner system, |
|
212 not to any specific test suites. These are also available as TEST_OPTS keywords. |
|
213 |
|
214 #### JOBS |
|
215 |
|
216 Currently only applies to JTReg. |
|
217 |
|
218 #### TIMEOUT_FACTOR |
|
219 |
|
220 Currently only applies to JTReg. |
|
221 |
|
222 #### VM_OPTIONS |
|
223 |
|
224 Applies to JTReg, GTest and Micro. |
|
225 |
|
226 #### JAVA_OPTIONS |
|
227 |
|
228 Applies to JTReg, GTest and Micro. |
|
229 |
|
230 #### AOT_MODULES |
|
231 |
|
232 Applies to JTReg and GTest. |
|
233 |
|
234 #### JCOV |
|
235 |
|
236 This keywords applies globally to the test runner system. If set to `true`, it |
|
237 enables JCov coverage reporting for all tests run. To be useful, the JDK under |
|
238 test must be run with a JDK built with JCov instrumentation (`configure |
|
239 --with-jcov=<path to directory containing lib/jcov.jar>`, `make jcov-image`). |
|
240 |
|
241 The simplest way to run tests with JCov coverage report is to use the special |
|
242 target `jcov-test` instead of `test`, e.g. `make jcov-test TEST=jdk_lang`. This |
|
243 will make sure the JCov image is built, and that JCov reporting is enabled. |
|
244 |
|
245 The JCov report is stored in `build/$BUILD/test-results/jcov-output`. |
|
246 |
|
247 Please note that running with JCov reporting can be very memory intensive. |
|
248 |
205 ### JTReg keywords |
249 ### JTReg keywords |
206 |
250 |
207 #### JOBS |
251 #### JOBS |
208 The test concurrency (`-concurrency`). |
252 The test concurrency (`-concurrency`). |
209 |
253 |
210 Defaults to TEST_JOBS (if set by `--with-test-jobs=`), otherwise it defaults to |
254 Defaults to TEST_JOBS (if set by `--with-test-jobs=`), otherwise it defaults to |
211 JOBS, except for Hotspot, where the default is *number of CPU cores/2*, but |
255 JOBS, except for Hotspot, where the default is *number of CPU cores/2*, but |
212 never more than 12. |
256 never more than 12. |
213 |
257 |
214 #### TIMEOUT |
258 #### TIMEOUT_FACTOR |
215 The timeout factor (`-timeoutFactor`). |
259 The timeout factor (`-timeoutFactor`). |
216 |
260 |
217 Defaults to 4. |
261 Defaults to 4. |
218 |
262 |
219 #### TEST_MODE |
263 #### TEST_MODE |
243 Limit memory consumption for JTReg test framework and VM under test. Set to 0 |
287 Limit memory consumption for JTReg test framework and VM under test. Set to 0 |
244 to disable the limits. |
288 to disable the limits. |
245 |
289 |
246 Defaults to 512m, except for hotspot, where it defaults to 0 (no limit). |
290 Defaults to 512m, except for hotspot, where it defaults to 0 (no limit). |
247 |
291 |
|
292 #### KEYWORDS |
|
293 |
|
294 JTReg kewords sent to JTReg using `-k`. Please be careful in making sure that |
|
295 spaces and special characters (like `!`) are properly quoted. To avoid some |
|
296 issues, the special value `%20` can be used instead of space. |
|
297 |
|
298 #### EXTRA_PROBLEM_LISTS |
|
299 |
|
300 Use additional problem lists file or files, in addition to the default |
|
301 ProblemList.txt located at the JTReg test roots. |
|
302 |
|
303 If multiple file names are specified, they should be separated by space (or, to |
|
304 help avoid quoting issues, the special value `%20`). |
|
305 |
|
306 The file names should be either absolute, or relative to the JTReg test root of |
|
307 the tests to be run. |
|
308 |
|
309 |
248 #### OPTIONS |
310 #### OPTIONS |
249 Additional options to the JTReg test framework. |
311 Additional options to the JTReg test framework. |
250 |
312 |
251 Use `JTREG="OPTIONS=--help all"` to see all available JTReg options. |
313 Use `JTREG="OPTIONS=--help all"` to see all available JTReg options. |
252 |
314 |
253 #### JAVA_OPTIONS |
315 #### JAVA_OPTIONS |
254 Additional Java options to JTReg (`-javaoption`). |
316 Additional Java options to JTReg (`-javaoption`). |
255 |
317 |
256 #### VM_OPTIONS |
318 #### VM_OPTIONS |
257 Additional VM options to JTReg (`-vmoption`). |
319 Additional VM options to JTReg (`-vmoption`). |
|
320 |
|
321 #### AOT_MODULES |
|
322 |
|
323 Generate AOT modules before testing for the specified module, or set of |
|
324 modules. If multiple modules are specified, they should be separated by space |
|
325 (or, to help avoid quoting issues, the special value `%20`). |
258 |
326 |
259 ### Gtest keywords |
327 ### Gtest keywords |
260 |
328 |
261 #### REPEAT |
329 #### REPEAT |
262 The number of times to repeat the tests (`--gtest_repeat`). |
330 The number of times to repeat the tests (`--gtest_repeat`). |
267 |
335 |
268 #### OPTIONS |
336 #### OPTIONS |
269 Additional options to the Gtest test framework. |
337 Additional options to the Gtest test framework. |
270 |
338 |
271 Use `GTEST="OPTIONS=--help"` to see all available Gtest options. |
339 Use `GTEST="OPTIONS=--help"` to see all available Gtest options. |
|
340 |
|
341 #### AOT_MODULES |
|
342 |
|
343 Generate AOT modules before testing for the specified module, or set of |
|
344 modules. If multiple modules are specified, they should be separated by space |
|
345 (or, to help avoid quoting issues, the special value `%20`). |
272 |
346 |
273 ### Microbenchmark keywords |
347 ### Microbenchmark keywords |
274 |
348 |
275 #### FORK |
349 #### FORK |
276 Override the number of benchmark forks to spawn. Same as specifying `-f <num>`. |
350 Override the number of benchmark forks to spawn. Same as specifying `-f <num>`. |