doc/testing.md
branchdatagramsocketimpl-branch
changeset 58678 9cf78a70fa4f
parent 54570 93b702d2a0cb
child 58679 9c3209ff7550
--- a/doc/testing.md	Thu Oct 17 20:27:44 2019 +0100
+++ b/doc/testing.md	Thu Oct 17 20:53:35 2019 +0100
@@ -23,7 +23,7 @@
     $ make test-jdk_lang JTREG="JOBS=8"
     $ make test TEST=jdk_lang
     $ make test-only TEST="gtest:LogTagSet gtest:LogTagSetDescriptions" GTEST="REPEAT=-1"
-    $ make test TEST="hotspot:hotspot_gc" JTREG="JOBS=1;TIMEOUT=8;VM_OPTIONS=-XshowSettings -Xlog:gc+ref=debug"
+    $ make test TEST="hotspot:hotspot_gc" JTREG="JOBS=1;TIMEOUT_FACTOR=8;VM_OPTIONS=-XshowSettings -Xlog:gc+ref=debug"
     $ make test TEST="jtreg:test/hotspot:hotspot_gc test/hotspot/jtreg/native_sanity/JniVersion.java"
     $ make test TEST="micro:java.lang.reflect" MICRO="FORK=1;WARMUP_ITER=2"
     $ make exploded-test TEST=tier2
@@ -180,11 +180,11 @@
 variables.
 
 These variables use a keyword=value approach to allow multiple values to be
-set. So, for instance, `JTREG="JOBS=1;TIMEOUT=8"` will set the JTReg
+set. So, for instance, `JTREG="JOBS=1;TIMEOUT_FACTOR=8"` will set the JTReg
 concurrency level to 1 and the timeout factor to 8. This is equivalent to
-setting `JTREG_JOBS=1 JTREG_TIMEOUT=8`, but using the keyword format means that
+setting `JTREG_JOBS=1 JTREG_TIMEOUT_FACTOR=8`, but using the keyword format means that
 the `JTREG` variable is parsed and verified for correctness, so
-`JTREG="TMIEOUT=8"` would give an error, while `JTREG_TMIEOUT=8` would just
+`JTREG="TMIEOUT_FACTOR=8"` would give an error, while `JTREG_TMIEOUT_FACTOR=8` would just
 pass unnoticed.
 
 To separate multiple keyword=value pairs, use `;` (semicolon). Since the shell
@@ -192,7 +192,7 @@
 qoutes, e.g. `JTREG="...;..."`. This will also make sure spaces are preserved,
 as in `JTREG="VM_OPTIONS=-XshowSettings -Xlog:gc+ref=debug"`.
 
-(Other ways are possible, e.g. using backslash: `JTREG=JOBS=1\;TIMEOUT=8`.
+(Other ways are possible, e.g. using backslash: `JTREG=JOBS=1\;TIMEOUT_FACTOR=8`.
 Also, as a special technique, the string `%20` will be replaced with space for
 certain options, e.g. `JTREG=VM_OPTIONS=-XshowSettings%20-Xlog:gc+ref=debug`.
 This can be useful if you have layers of scripts and have trouble getting
@@ -306,6 +306,14 @@
 The file names should be either absolute, or relative to the JTReg test root of
 the tests to be run.
 
+#### RUN_PROBLEM_LISTS
+
+Use the problem lists to select tests instead of excluding them.
+
+Set to `true` or `false`.
+If `true`, JTReg will use `-match:` option, otherwise `-exclude:` will be used.
+Default is `false`.
+
 
 #### OPTIONS
 Additional options to the JTReg test framework.
@@ -399,6 +407,20 @@
     $ export LANG="en_US" && make test TEST=...
     $ make test JTREG="VM_OPTIONS=-Duser.language=en -Duser.country=US" TEST=...
 
+### PKCS11 Tests
+
+It is highly recommended to use the latest NSS version when running PKCS11 tests.
+Improper NSS version may lead to unexpected failures which are hard to diagnose.
+For example, sun/security/pkcs11/Secmod/AddTrustedCert.java may fail on Ubuntu
+18.04 with the default NSS version in the system.
+To run these tests correctly, the system property `test.nss.lib.paths` is required
+on Ubuntu 18.04 to specify the alternative NSS lib directories.
+For example:
+
+    $ make test TEST="jtreg:sun/security/pkcs11/Secmod/AddTrustedCert.java" JTREG="JAVA_OPTIONS=-Dtest.nss.lib.paths=/path/to/your/latest/NSS-libs"
+
+For more notes about the PKCS11 tests, please refer to test/jdk/sun/security/pkcs11/README.
+
 ---
 # Override some definitions in the global css file that are not optimal for
 # this document.