Merge
authorduke
Wed, 05 Jul 2017 23:12:15 +0200
changeset 44587 d88553af25b0
parent 44586 fe7e6fd5ccf7 (current diff)
parent 44583 0ebe9abfb687 (diff)
child 44603 29d8dc0132ab
Merge
--- a/.hgtags-top-repo	Thu Apr 13 20:35:45 2017 +0000
+++ b/.hgtags-top-repo	Wed Jul 05 23:12:15 2017 +0200
@@ -408,3 +408,4 @@
 21b063d75b3edbffb9bebc8872d990920c4ae1e5 jdk-9+162
 c38c6b270ccc8e2b86d1631bcf42248241b54d2c jdk-9+163
 7810f75d016a52e32295c4233009de5ca90e31af jdk-9+164
+aff4f339acd40942d3dab499846b52acd87b3af1 jdk-9+165
--- a/common/conf/jib-profiles.js	Thu Apr 13 20:35:45 2017 +0000
+++ b/common/conf/jib-profiles.js	Wed Jul 05 23:12:15 2017 +0200
@@ -882,7 +882,7 @@
         jtreg: {
             server: "javare",
             revision: "4.2",
-            build_number: "b05",
+            build_number: "b07",
             checksum_file: "MD5_VALUES",
             file: "jtreg_bin-4.2.zip",
             environment_name: "JT_HOME",
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/common/doc/testing.html	Wed Jul 05 23:12:15 2017 +0200
@@ -0,0 +1,112 @@
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml">
+<head>
+  <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+  <meta http-equiv="Content-Style-Type" content="text/css" />
+  <meta name="generator" content="pandoc" />
+  <title>Testing OpenJDK</title>
+  <style type="text/css">code{white-space: pre;}</style>
+  <link rel="stylesheet" href="http://openjdk.java.net/page.css" type="text/css" />
+  <style type="text/css">pre, code, tt { color: #1d6ae5; }</style>
+  <style type="text/css">pre { font-size: 10pt; }</style>
+</head>
+<body>
+<div id="header">
+<h1 class="title">Testing OpenJDK</h1>
+</div>
+<div id="TOC">
+<ul>
+<li><a href="#using-the-run-test-framework">Using the run-test framework</a></li>
+<li><a href="#test-selection">Test selection</a><ul>
+<li><a href="#jtreg">JTreg</a></li>
+<li><a href="#gtest">Gtest</a></li>
+</ul></li>
+<li><a href="#test-results-and-summary">Test results and summary</a></li>
+<li><a href="#test-suite-control">Test suite control</a><ul>
+<li><a href="#jtreg-keywords">JTreg keywords</a></li>
+<li><a href="#gtest-keywords">Gtest keywords</a></li>
+</ul></li>
+</ul>
+</div>
+<h2 id="using-the-run-test-framework">Using the run-test framework</h2>
+<p>This new way of running tests is developer-centric. It assumes that you have built a jdk locally and want to test it. Running common test targets is simple, and more complex ad-hoc combination of tests is possible. The user interface is forgiving, and clearly report errors it cannot resolve.</p>
+<p>Some example command-lines:</p>
+<pre><code>$ make run-test-tier1
+$ make run-test-jdk_lang JTREG=&quot;JOBS=8&quot;
+$ make run-test TEST=jdk_lang
+$ make run-test-only TEST=&quot;gtest:LogTagSet gtest:LogTagSetDescriptions&quot; GTEST=&quot;REPEAT=-1&quot;
+$ make run-test TEST=&quot;hotspot/test:hotspot_gc&quot; JTREG=&quot;JOBS=1;TIMEOUT=8;VM_OTIONS=-XshowSettings -Xlog:gc+ref=debug&quot;
+$ make run-test TEST=&quot;jtreg:hotspot/test:hotspot_gc hotspot/test/native_sanity/JniVersion.java&quot;</code></pre>
+<h2 id="test-selection">Test selection</h2>
+<p>All functionality is available using the run-test make target. In this use case, the test or tests to be executed is controlled using the <code>TEST</code> variable. To speed up subsequent test runs with no source code changes, run-test-only can be used instead, which do not depend on the source and test image build.</p>
+<p>For some common top-level tests, direct make targets have been generated. This includes all JTreg test groups, the hotspot gtest, and custom tests (if present). This means that <code>make run-test-tier1</code> is equivalent to <code>make run-test TEST=&quot;tier1&quot;</code>, but the latter is more tab-completion friendly. For more complex test runs, the <code>run-test TEST=&quot;x&quot;</code> solution needs to be used.</p>
+<p>The test specifications given in <code>TEST</code> is parsed into fully qualified test descriptors, which clearly and unambigously show which tests will be run. As an example, <code>:tier1</code> will expand to <code>jtreg:jdk/test:tier1 jtreg:langtools/test:tier1 jtreg:nashorn/test:tier1 jtreg:jaxp/test:tier1</code>. You can always submit a list of fully qualified test descriptors in the <code>TEST</code> variable if you want to shortcut the parser.</p>
+<h3 id="jtreg">JTreg</h3>
+<p>JTreg test groups can be specified either without a test root, e.g. <code>:tier1</code> (or <code>tier1</code>, the initial colon is optional), or with, e.g. <code>hotspot/test:tier1</code>, <code>jdk/test:jdk_util</code>.</p>
+<p>When specified without a test root, all matching groups from all tests roots will be added. Otherwise, only the group from the specified test root will be added.</p>
+<p>Individual JTreg tests or directories containing JTreg tests can also be specified, like <code>hotspot/test/native_sanity/JniVersion.java</code> or <code>hotspot/test/native_sanity</code>. You can also specify an absolute path, to point to a JTreg test outside the source tree.</p>
+<p>As long as the test groups or test paths can be uniquely resolved, you do not need to enter the <code>jtreg:</code> prefix. If this is not possible, or if you want to use a fully qualified test descriptor, add <code>jtreg:</code>, e.g. <code>jtreg:hotspot/test/native_sanity</code>.</p>
+<h3 id="gtest">Gtest</h3>
+<p>Since the Hotspot Gtest suite is so quick, the default is to run all tests. This is specified by just <code>gtest</code>, or as a fully qualified test descriptor <code>gtest:all</code>.</p>
+<p>If you want, you can single out an individual test or a group of tests, for instance <code>gtest:LogDecorations</code> or <code>gtest:LogDecorations.level_test_vm</code>. This can be particularly useful if you want to run a shaky test repeatedly.</p>
+<h2 id="test-results-and-summary">Test results and summary</h2>
+<p>At the end of the test run, a summary of all tests run will be presented. This will have a consistent look, regardless of what test suites were used. This is a sample summary:</p>
+<pre><code>==============================
+Test summary
+==============================
+   TEST                                          TOTAL  PASS  FAIL ERROR
+&gt;&gt; jtreg:jdk/test:tier1                           1867  1865     2     0 &lt;&lt;
+   jtreg:langtools/test:tier1                     4711  4711     0     0
+   jtreg:nashorn/test:tier1                        133   133     0     0
+==============================
+TEST FAILURE</code></pre>
+<p>Tests where the number of TOTAL tests does not equal the number of PASSed tests will be considered a test failure. These are marked with the <code>&gt;&gt; ... &lt;&lt;</code> marker for easy identification.</p>
+<p>The classification of non-passed tests differs a bit between test suites. In the summary, ERROR is used as a catch-all for tests that neither passed nor are classified as failed by the framework. This might indicate test framework error, timeout or other problems.</p>
+<p>In case of test failures, <code>make run-test</code> will exit with a non-zero exit value.</p>
+<p>All tests have their result stored in <code>build/$BUILD/test-result/$TEST_ID</code>, where TEST_ID is a path-safe conversion from the fully qualified test descriptor, e.g. for <code>jtreg:jdk/test:tier1</code> the TEST_ID is <code>jtreg_jdk_test_tier1</code>. This path is also printed in the log at the end of the test run.</p>
+<p>Additional work data is stored in <code>build/$BUILD/test-support/$TEST_ID</code>. For some frameworks, this directory might contain information that is useful in determining the cause of a failed test.</p>
+<h2 id="test-suite-control">Test suite control</h2>
+<p>It is possible to control various aspects of the test suites using make control variables.</p>
+<p>These variables use a keyword=value approach to allow multiple values to be set. So, for instance, <code>JTREG=&quot;JOBS=1;TIMEOUT=8&quot;</code> will set the JTreg concurrency level to 1 and the timeout factor to 8. This is equivalent to setting <code>JTREG_JOBS=1 JTREG_TIMEOUT=8</code>, but using the keyword format means that the <code>JTREG</code> variable is parsed and verified for correctness, so <code>JTREG=&quot;TMIEOUT=8&quot;</code> would give an error, while <code>JTREG_TMIEOUT=8</code> would just pass unnoticed.</p>
+<p>To separate multiple keyword=value pairs, use <code>;</code> (semicolon). Since the shell normally eats <code>;</code>, the recommended usage is to write the assignment inside qoutes, e.g. <code>JTREG=&quot;...;...&quot;</code>. This will also make sure spaces are preserved, as in <code>JTREG=&quot;VM_OTIONS=-XshowSettings -Xlog:gc+ref=debug&quot;</code>.</p>
+<p>(Other ways are possible, e.g. using backslash: <code>JTREG=JOBS=1\;TIMEOUT=8</code>. Also, as a special technique, the string <code>%20</code> will be replaced with space for certain options, e.g. <code>JTREG=VM_OTIONS=-XshowSettings%20-Xlog:gc+ref=debug</code>. This can be useful if you have layers of scripts and have trouble getting proper quoting of command line arguments through.)</p>
+<p>As far as possible, the names of the keywords have been standardized between test suites.</p>
+<h3 id="jtreg-keywords">JTreg keywords</h3>
+<h4 id="jobs">JOBS</h4>
+<p>The test concurrency (<code>-concurrency</code>).</p>
+<p>Defaults to TEST_JOBS (if set by <code>--with-test-jobs=</code>), otherwise it defaults to JOBS, except for Hotspot, where the default is <em>number of CPU cores/2</em>, but never more than 12.</p>
+<h4 id="timeout">TIMEOUT</h4>
+<p>The timeout factor (<code>-timeoutFactor</code>).</p>
+<p>Defaults to 4.</p>
+<h4 id="test_mode">TEST_MODE</h4>
+<p>The test mode (<code>-agentvm</code>, <code>-samevm</code> or <code>-othervm</code>).</p>
+<p>Defaults to <code>-agentvm</code>.</p>
+<h4 id="assert">ASSERT</h4>
+<p>Enable asserts (<code>-ea -esa</code>, or none).</p>
+<p>Set to <code>true</code> or <code>false</code>. If true, adds <code>-ea -esa</code>. Defaults to true, except for hotspot.</p>
+<h4 id="verbose">VERBOSE</h4>
+<p>The verbosity level (<code>-verbose</code>).</p>
+<p>Defaults to <code>fail,error,summary</code>.</p>
+<h4 id="retain">RETAIN</h4>
+<p>What test data to retain (<code>-retain</code>).</p>
+<p>Defaults to <code>fail,error</code>.</p>
+<h4 id="max_mem">MAX_MEM</h4>
+<p>Limit memory consumption (<code>-Xmx</code> and <code>-vmoption:-Xmx</code>, or none).</p>
+<p>Limit memory consumption for JTreg test framework and VM under test. Set to 0 to disable the limits.</p>
+<p>Defaults to 512m, except for hotspot, where it defaults to 0 (no limit).</p>
+<h4 id="options">OPTIONS</h4>
+<p>Additional options to the JTreg test framework.</p>
+<p>Use <code>JTREG=&quot;OPTIONS=--help all&quot;</code> to see all available JTreg options.</p>
+<h4 id="java_options">JAVA_OPTIONS</h4>
+<p>Additional Java options to JTreg (<code>-javaoption</code>).</p>
+<h4 id="vm_options">VM_OPTIONS</h4>
+<p>Additional VM options to JTreg (<code>-vmoption</code>).</p>
+<h3 id="gtest-keywords">Gtest keywords</h3>
+<h4 id="repeat">REPEAT</h4>
+<p>The number of times to repeat the tests (<code>--gtest_repeat</code>).</p>
+<p>Default is 1. Set to -1 to repeat indefinitely. This can be especially useful combined with <code>OPTIONS=--gtest_break_on_failure</code> to reproduce an intermittent problem.</p>
+<h4 id="options-1">OPTIONS</h4>
+<p>Additional options to the Gtest test framework.</p>
+<p>Use <code>GTEST=&quot;OPTIONS=--help&quot;</code> to see all available Gtest options.</p>
+</body>
+</html>
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/common/doc/testing.md	Wed Jul 05 23:12:15 2017 +0200
@@ -0,0 +1,207 @@
+% Testing OpenJDK
+
+## Using the run-test framework
+
+This new way of running tests is developer-centric. It assumes that you have
+built a jdk locally and want to test it. Running common test targets is simple,
+and more complex ad-hoc combination of tests is possible. The user interface is
+forgiving, and clearly report errors it cannot resolve.
+
+Some example command-lines:
+
+    $ make run-test-tier1
+    $ make run-test-jdk_lang JTREG="JOBS=8"
+    $ make run-test TEST=jdk_lang
+    $ make run-test-only TEST="gtest:LogTagSet gtest:LogTagSetDescriptions" GTEST="REPEAT=-1"
+    $ make run-test TEST="hotspot/test:hotspot_gc" JTREG="JOBS=1;TIMEOUT=8;VM_OTIONS=-XshowSettings -Xlog:gc+ref=debug"
+    $ make run-test TEST="jtreg:hotspot/test:hotspot_gc hotspot/test/native_sanity/JniVersion.java"
+
+## Test selection
+
+All functionality is available using the run-test make target. In this use
+case, the test or tests to be executed is controlled using the `TEST` variable.
+To speed up subsequent test runs with no source code changes, run-test-only can
+be used instead, which do not depend on the source and test image build.
+
+For some common top-level tests, direct make targets have been generated. This
+includes all JTreg test groups, the hotspot gtest, and custom tests (if
+present). This means that `make run-test-tier1` is equivalent to `make run-test
+TEST="tier1"`, but the latter is more tab-completion friendly. For more complex
+test runs, the `run-test TEST="x"` solution needs to be used.
+
+The test specifications given in `TEST` is parsed into fully qualified test
+descriptors, which clearly and unambigously show which tests will be run. As an
+example, `:tier1` will expand to `jtreg:jdk/test:tier1
+jtreg:langtools/test:tier1 jtreg:nashorn/test:tier1 jtreg:jaxp/test:tier1`. You
+can always submit a list of fully qualified test descriptors in the `TEST`
+variable if you want to shortcut the parser.
+
+### JTreg
+
+JTreg test groups can be specified either without a test root, e.g. `:tier1`
+(or `tier1`, the initial colon is optional), or with, e.g.
+`hotspot/test:tier1`, `jdk/test:jdk_util`.
+
+When specified without a test root, all matching groups from all tests roots
+will be added. Otherwise, only the group from the specified test root will be
+added.
+
+Individual JTreg tests or directories containing JTreg tests can also be
+specified, like `hotspot/test/native_sanity/JniVersion.java` or
+`hotspot/test/native_sanity`. You can also specify an absolute path, to point
+to a JTreg test outside the source tree.
+
+As long as the test groups or test paths can be uniquely resolved, you do not
+need to enter the `jtreg:` prefix. If this is not possible, or if you want to
+use a fully qualified test descriptor, add `jtreg:`, e.g.
+`jtreg:hotspot/test/native_sanity`.
+
+### Gtest
+
+Since the Hotspot Gtest suite is so quick, the default is to run all tests.
+This is specified by just `gtest`, or as a fully qualified test descriptor
+`gtest:all`.
+
+If you want, you can single out an individual test or a group of tests, for
+instance `gtest:LogDecorations` or `gtest:LogDecorations.level_test_vm`. This
+can be particularly useful if you want to run a shaky test repeatedly.
+
+## Test results and summary
+
+At the end of the test run, a summary of all tests run will be presented. This
+will have a consistent look, regardless of what test suites were used. This is
+a sample summary:
+
+    ==============================
+    Test summary
+    ==============================
+       TEST                                          TOTAL  PASS  FAIL ERROR
+    >> jtreg:jdk/test:tier1                           1867  1865     2     0 <<
+       jtreg:langtools/test:tier1                     4711  4711     0     0
+       jtreg:nashorn/test:tier1                        133   133     0     0
+    ==============================
+    TEST FAILURE
+
+Tests where the number of TOTAL tests does not equal the number of PASSed tests
+will be considered a test failure. These are marked with the `>> ... <<` marker
+for easy identification.
+
+The classification of non-passed tests differs a bit between test suites. In
+the summary, ERROR is used as a catch-all for tests that neither passed nor are
+classified as failed by the framework. This might indicate test framework
+error, timeout or other problems.
+
+In case of test failures, `make run-test` will exit with a non-zero exit value.
+
+All tests have their result stored in `build/$BUILD/test-result/$TEST_ID`,
+where TEST_ID is a path-safe conversion from the fully qualified test
+descriptor, e.g. for `jtreg:jdk/test:tier1` the TEST_ID is
+`jtreg_jdk_test_tier1`. This path is also printed in the log at the end of the
+test run.
+
+Additional work data is stored in `build/$BUILD/test-support/$TEST_ID`. For
+some frameworks, this directory might contain information that is useful in
+determining the cause of a failed test.
+
+## Test suite control
+
+It is possible to control various aspects of the test suites using make control
+variables.
+
+These variables use a keyword=value approach to allow multiple values to be
+set. So, for instance, `JTREG="JOBS=1;TIMEOUT=8"` will set the JTreg
+concurrency level to 1 and the timeout factor to 8. This is equivalent to
+setting `JTREG_JOBS=1 JTREG_TIMEOUT=8`, but using the keyword format means that
+the `JTREG` variable is parsed and verified for correctness, so
+`JTREG="TMIEOUT=8"` would give an error, while `JTREG_TMIEOUT=8` would just
+pass unnoticed.
+
+To separate multiple keyword=value pairs, use `;` (semicolon). Since the shell
+normally eats `;`, the recommended usage is to write the assignment inside
+qoutes, e.g. `JTREG="...;..."`. This will also make sure spaces are preserved,
+as in `JTREG="VM_OTIONS=-XshowSettings -Xlog:gc+ref=debug"`.
+
+(Other ways are possible, e.g. using backslash: `JTREG=JOBS=1\;TIMEOUT=8`.
+Also, as a special technique, the string `%20` will be replaced with space for
+certain options, e.g. `JTREG=VM_OTIONS=-XshowSettings%20-Xlog:gc+ref=debug`.
+This can be useful if you have layers of scripts and have trouble getting
+proper quoting of command line arguments through.)
+
+As far as possible, the names of the keywords have been standardized between
+test suites.
+
+### JTreg keywords
+
+#### JOBS
+The test concurrency (`-concurrency`).
+
+Defaults to TEST_JOBS (if set by `--with-test-jobs=`), otherwise it defaults to
+JOBS, except for Hotspot, where the default is *number of CPU cores/2*, but
+never more than 12.
+
+#### TIMEOUT
+The timeout factor (`-timeoutFactor`).
+
+Defaults to 4.
+
+#### TEST_MODE
+The test mode (`-agentvm`, `-samevm` or `-othervm`).
+
+Defaults to `-agentvm`.
+
+#### ASSERT
+Enable asserts (`-ea -esa`, or none).
+
+Set to `true` or `false`. If true, adds `-ea -esa`. Defaults to true, except
+for hotspot.
+
+#### VERBOSE
+The verbosity level (`-verbose`).
+
+Defaults to `fail,error,summary`.
+
+#### RETAIN
+What test data to retain (`-retain`).
+
+Defaults to `fail,error`.
+
+#### MAX_MEM
+Limit memory consumption (`-Xmx` and `-vmoption:-Xmx`, or none).
+
+Limit memory consumption for JTreg test framework and VM under test. Set to 0
+to disable the limits.
+
+Defaults to 512m, except for hotspot, where it defaults to 0 (no limit).
+
+#### OPTIONS
+Additional options to the JTreg test framework.
+
+Use `JTREG="OPTIONS=--help all"` to see all available JTreg options.
+
+#### JAVA_OPTIONS
+Additional Java options to JTreg (`-javaoption`).
+
+#### VM_OPTIONS
+Additional VM options to JTreg (`-vmoption`).
+
+### Gtest keywords
+
+#### REPEAT
+The number of times to repeat the tests (`--gtest_repeat`).
+
+Default is 1. Set to -1 to repeat indefinitely. This can be especially useful
+combined with `OPTIONS=--gtest_break_on_failure` to reproduce an intermittent
+problem.
+
+#### OPTIONS
+Additional options to the Gtest test framework.
+
+Use `GTEST="OPTIONS=--help"` to see all available Gtest options.
+
+---
+# Override some definitions in http://openjdk.java.net/page.css that are
+# unsuitable for this document.
+header-includes:
+ - '<style type="text/css">pre, code, tt { color: #1d6ae5; }</style>'
+ - '<style type="text/css">pre { font-size: 10pt; }</style>'
+---
--- a/make/RunTests.gmk	Thu Apr 13 20:35:45 2017 +0000
+++ b/make/RunTests.gmk	Wed Jul 05 23:12:15 2017 +0200
@@ -160,6 +160,7 @@
 ifeq ($(TEST), )
   $(info No test selection given in TEST!)
   $(info Please use e.g. 'run-test TEST=tier1' or 'run-test-tier1')
+  $(info See common/doc/testing.[md|html] for help)
   $(error Cannot continue)
 endif
 
@@ -182,6 +183,7 @@
 
 ifneq ($(UNKNOWN_TEST), )
   $(info Unknown test selection: '$(UNKNOWN_TEST)')
+  $(info See common/doc/testing.[md|html] for help)
   $(error Cannot continue)
 endif
 
--- a/make/UpdateBuildDocs.gmk	Thu Apr 13 20:35:45 2017 +0000
+++ b/make/UpdateBuildDocs.gmk	Wed Jul 05 23:12:15 2017 +0200
@@ -47,6 +47,7 @@
 # Remaining parameters are named arguments. These include:
 #   SOURCE_FILE  The markdown source file
 #   TARGET_DIR   The directory where to store the generated html file
+#   OPTIONS      Additional options to pandoc
 #
 SetupMarkdownToHtml = $(NamedParamsMacroTemplate)
 define SetupMarkdownToHtmlBody
@@ -65,12 +66,13 @@
 	$$(call LogInfo, Converting $$(notdir $1) to html)
 	$$(call MakeDir, $$($1_TARGET_DIR) $$(MAKESUPPORT_OUTPUTDIR)/markdown)
 	$$(call ExecuteWithLog, $$(MAKESUPPORT_OUTPUTDIR)/markdown/$1, \
-	    $$(PANDOC) -f markdown -t html --standalone '$$<' -o '$$@')
-	TOO_LONG_LINES=`$$(GREP) -E -e '^.{80}.+$$$$' $$<` ; \
-	if [ "x$$TOO_LONG_LINES" != x ]; then \
+	    $$(PANDOC) $$($1_OPTIONS) -f markdown -t html --standalone \
+	    --css 'http://openjdk.java.net/page.css' '$$<' -o '$$@')
+	TOO_LONG_LINES=`$$(GREP) -E -e '^.{80}.+$$$$' $$<` || true ; \
+	if [ "x$$$$TOO_LONG_LINES" != x ]; then \
 	  $$(ECHO) "Warning: Unsuitable markdown in $$<:" ; \
 	  $$(ECHO) "The following lines are longer than 80 characters:" ; \
-	  $$(GREP) -E -e '^.{80}.+$$$$' $$< ; \
+	  $$(GREP) -E -n -e '^.{80}.+$$$$' $$< || true ; \
 	fi
 
   $1 := $$($1_OUTPUT_FILE)
@@ -80,12 +82,17 @@
 
 ################################################################################
 
-BUILD_DOCS_DIR := $(TOPDIR)/common/doc
-BUILD_DOCS_MD_FILE := building.md
+DOCS_DIR := $(TOPDIR)/common/doc
 
 $(eval $(call SetupMarkdownToHtml, building, \
-  SOURCE_FILE := $(BUILD_DOCS_DIR)/$(BUILD_DOCS_MD_FILE), \
-  TARGET_DIR := $(BUILD_DOCS_DIR), \
+  SOURCE_FILE := $(DOCS_DIR)/building.md, \
+  TARGET_DIR := $(DOCS_DIR), \
+))
+
+$(eval $(call SetupMarkdownToHtml, testing, \
+  SOURCE_FILE := $(DOCS_DIR)/testing.md, \
+  TARGET_DIR := $(DOCS_DIR), \
+  OPTIONS := --toc, \
 ))
 
 ################################################################################
--- a/test/lib/jdk/test/lib/JDKToolLauncher.java	Thu Apr 13 20:35:45 2017 +0000
+++ b/test/lib/jdk/test/lib/JDKToolLauncher.java	Wed Jul 05 23:12:15 2017 +0200
@@ -24,9 +24,7 @@
 package jdk.test.lib;
 
 import java.util.ArrayList;
-import java.util.Arrays;
 import java.util.List;
-import jdk.test.lib.process.ProcessTools;
 
 /**
  * A utility for constructing command lines for starting JDK tool processes.
@@ -59,7 +57,6 @@
         } else {
             executable = JDKToolFinder.getTestJDKTool(tool);
         }
-        vmArgs.addAll(Arrays.asList(ProcessTools.getPlatformSpecificVMArgs()));
     }
 
     /**
--- a/test/lib/jdk/test/lib/cli/CommandLineOptionTest.java	Thu Apr 13 20:35:45 2017 +0000
+++ b/test/lib/jdk/test/lib/cli/CommandLineOptionTest.java	Wed Jul 05 23:12:15 2017 +0200
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2014, 2016, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2014, 2017, Oracle and/or its affiliates. All rights reserved.
  * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
  *
  * This code is free software; you can redistribute it and/or modify it
@@ -28,6 +28,7 @@
 import java.util.Collections;
 import java.util.function.BooleanSupplier;
 
+import jdk.test.lib.management.InputArguments;
 import jdk.test.lib.process.ExitCode;
 import jdk.test.lib.process.ProcessTools;
 import jdk.test.lib.process.OutputAnalyzer;
@@ -105,7 +106,7 @@
                     throws Throwable {
         List<String> finalOptions = new ArrayList<>();
         if (addTestVMOptions) {
-            Collections.addAll(finalOptions, ProcessTools.getVmInputArgs());
+            Collections.addAll(finalOptions, InputArguments.getVmInputArgs());
             Collections.addAll(finalOptions, Utils.getTestJavaOpts());
         }
         Collections.addAll(finalOptions, options);
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/test/lib/jdk/test/lib/management/InputArguments.java	Wed Jul 05 23:12:15 2017 +0200
@@ -0,0 +1,41 @@
+/*
+ * Copyright (c) 2017, Oracle and/or its affiliates. All rights reserved.
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This code is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 only, as
+ * published by the Free Software Foundation.
+ *
+ * This code is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+ * version 2 for more details (a copy is included in the LICENSE file that
+ * accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License version
+ * 2 along with this work; if not, write to the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
+ *
+ * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
+ * or visit www.oracle.com if you need additional information or have any
+ * questions.
+ */
+
+package jdk.test.lib.management;
+
+import java.lang.management.ManagementFactory;
+import java.lang.management.RuntimeMXBean;
+import java.util.List;
+
+public class InputArguments {
+    /**
+     * Gets the array of strings containing input arguments passed to the VM
+     *
+     * @return arguments
+     */
+    public static String[] getVmInputArgs() {
+        RuntimeMXBean runtime = ManagementFactory.getRuntimeMXBean();
+        List<String> args = runtime.getInputArguments();
+        return args.toArray(new String[args.size()]);
+    }
+}
--- a/test/lib/jdk/test/lib/process/ProcessTools.java	Thu Apr 13 20:35:45 2017 +0000
+++ b/test/lib/jdk/test/lib/process/ProcessTools.java	Wed Jul 05 23:12:15 2017 +0200
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2013, 2016, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2013, 2017, Oracle and/or its affiliates. All rights reserved.
  * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
  *
  * This code is free software; you can redistribute it and/or modify it
@@ -28,13 +28,10 @@
 import java.io.InputStream;
 import java.io.OutputStream;
 import java.io.PrintStream;
-import java.lang.management.ManagementFactory;
-import java.lang.management.RuntimeMXBean;
 import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.Collections;
 import java.util.concurrent.CountDownLatch;
-import java.util.List;
 import java.util.Map;
 import java.util.concurrent.ExecutionException;
 import java.util.concurrent.Future;
@@ -45,7 +42,6 @@
 import java.util.stream.Collectors;
 
 import jdk.test.lib.JDKToolFinder;
-import jdk.test.lib.Platform;
 import jdk.test.lib.Utils;
 
 public final class ProcessTools {
@@ -66,23 +62,23 @@
     }
 
     /**
-    * Pumps stdout and stderr from running the process into a String.
-    *
-    * @param processHandler ProcessHandler to run.
-    * @return Output from process.
-    * @throws IOException If an I/O error occurs.
-    */
+     * Pumps stdout and stderr from running the process into a String.
+     *
+     * @param processHandler ProcessHandler to run.
+     * @return Output from process.
+     * @throws IOException If an I/O error occurs.
+     */
     public static OutputBuffer getOutput(ProcessBuilder processBuilder) throws IOException {
         return getOutput(processBuilder.start());
     }
 
     /**
-    * Pumps stdout and stderr the running process into a String.
-    *
-    * @param process Process to pump.
-    * @return Output from process.
-    * @throws IOException If an I/O error occurs.
-    */
+     * Pumps stdout and stderr the running process into a String.
+     *
+     * @param process Process to pump.
+     * @return Output from process.
+     * @throws IOException If an I/O error occurs.
+     */
     public static OutputBuffer getOutput(Process process) throws IOException {
         ByteArrayOutputStream stderrBuffer = new ByteArrayOutputStream();
         ByteArrayOutputStream stdoutBuffer = new ByteArrayOutputStream();
@@ -304,38 +300,14 @@
     public static long getProcessId() throws Exception {
         return ProcessHandle.current().getPid();
     }
-    /**
-     * Gets the array of strings containing input arguments passed to the VM
-     *
-     * @return arguments
-     */
-    public static String[] getVmInputArgs() {
-        RuntimeMXBean runtime = ManagementFactory.getRuntimeMXBean();
-        List<String> args = runtime.getInputArguments();
-        return args.toArray(new String[args.size()]);
-    }
 
-    /**
-     * Get platform specific VM arguments (e.g. -d64 on 64bit Solaris)
-     *
-     * @return String[] with platform specific arguments, empty if there are
-     *         none
-     */
-    public static String[] getPlatformSpecificVMArgs() {
-
-    if (Platform.is64bit() && Platform.isSolaris()) {
-            return new String[] { "-d64" };
-        }
-
-        return new String[] {};
-    }
 
 
     /**
      * Create ProcessBuilder using the java launcher from the jdk to be tested and
      * with any platform specific arguments prepended
      */
-    public static ProcessBuilder createJavaProcessBuilder(String... command) throws Exception {
+    public static ProcessBuilder createJavaProcessBuilder(String... command) {
         return createJavaProcessBuilder(false, command);
     }
 
@@ -348,12 +320,11 @@
      * @param command Arguments to pass to the java command.
      * @return The ProcessBuilder instance representing the java command.
      */
-    public static ProcessBuilder createJavaProcessBuilder(boolean addTestVmAndJavaOptions, String... command) throws Exception {
+    public static ProcessBuilder createJavaProcessBuilder(boolean addTestVmAndJavaOptions, String... command) {
         String javapath = JDKToolFinder.getJDKTool("java");
 
         ArrayList<String> args = new ArrayList<>();
         args.add(javapath);
-        Collections.addAll(args, getPlatformSpecificVMArgs());
 
         args.add("-cp");
         args.add(System.getProperty("java.class.path"));
@@ -386,26 +357,6 @@
 
     /**
      * Executes a test jvm process, waits for it to finish and returns the process output.
-     * The default jvm options from the test's run command, jtreg, test.vm.opts and test.java.opts, are added.
-     * The java from the test.jdk is used to execute the command.
-     *
-     * The command line will be like:
-     * {test.jdk}/bin/java {test.fromRun.opts} {test.vm.opts} {test.java.opts} cmds
-     *
-     * @param cmds User specifed arguments.
-     * @return The output from the process.
-     */
-    public static OutputAnalyzer executeTestJvmAllArgs(String... cmds) throws Throwable {
-        List<String> argsList = new ArrayList<>();
-        String[] testArgs = getVmInputArgs();
-        Collections.addAll(argsList, testArgs);
-        Collections.addAll(argsList, Utils.addTestJavaOpts(cmds));
-        ProcessBuilder pb = createJavaProcessBuilder(argsList.toArray(new String[argsList.size()]));
-        return executeProcess(pb);
-    }
-
-    /**
-     * Executes a test jvm process, waits for it to finish and returns the process output.
      * The default jvm options from jtreg, test.vm.opts and test.java.opts, are added.
      * The java from the test.jdk is used to execute the command.
      *