# HG changeset patch # User jjg # Date 1507680172 25200 # Node ID d18df41954ba72e430f4b40c994404aa6f4cd916 # Parent 8cb132b3a01619a3e0e3f0a29ba79db6b14bfcbd 8187521: In some corner cases the javadoc tool can reuse id attribute Reviewed-by: bpatel, ksrini diff -r 8cb132b3a016 -r d18df41954ba src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/AbstractExecutableMemberWriter.java --- a/src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/AbstractExecutableMemberWriter.java Tue Oct 10 09:55:14 2017 -0700 +++ b/src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/AbstractExecutableMemberWriter.java Tue Oct 10 17:02:52 2017 -0700 @@ -313,7 +313,8 @@ * @return the 1.4.x style anchor for the executable element. */ protected String getErasureAnchor(ExecutableElement executableElement) { - final StringBuilder buf = new StringBuilder(name(executableElement) + "("); + final StringBuilder buf = new StringBuilder(writer.anchorName(executableElement)); + buf.append("("); List parameters = executableElement.getParameters(); boolean foundTypeVariable = false; for (int i = 0; i < parameters.size(); i++) { diff -r 8cb132b3a016 -r d18df41954ba src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/HtmlDocletWriter.java --- a/src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/HtmlDocletWriter.java Tue Oct 10 09:55:14 2017 -0700 +++ b/src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/HtmlDocletWriter.java Tue Oct 10 17:02:52 2017 -0700 @@ -33,6 +33,7 @@ import javax.lang.model.element.AnnotationMirror; import javax.lang.model.element.AnnotationValue; import javax.lang.model.element.Element; +import javax.lang.model.element.ElementKind; import javax.lang.model.element.ExecutableElement; import javax.lang.model.element.ModuleElement; import javax.lang.model.element.Name; @@ -74,6 +75,7 @@ import jdk.javadoc.internal.doclets.formats.html.markup.HtmlStyle; import jdk.javadoc.internal.doclets.formats.html.markup.HtmlTag; import jdk.javadoc.internal.doclets.formats.html.markup.HtmlTree; +import jdk.javadoc.internal.doclets.formats.html.markup.HtmlVersion; import jdk.javadoc.internal.doclets.formats.html.markup.RawHtml; import jdk.javadoc.internal.doclets.formats.html.markup.StringContent; import jdk.javadoc.internal.doclets.toolkit.AnnotationTypeWriter; @@ -1468,20 +1470,18 @@ if (isProperty) { return executableElement.getSimpleName().toString(); } - String signature = utils.signature(executableElement); - StringBuilder signatureParsed = new StringBuilder(); - int counter = 0; - for (int i = 0; i < signature.length(); i++) { - char c = signature.charAt(i); - if (c == '<') { - counter++; - } else if (c == '>') { - counter--; - } else if (counter == 0) { - signatureParsed.append(c); - } + String member = anchorName(executableElement); + String erasedSignature = utils.makeSignature(executableElement, true, true); + return member + erasedSignature; + } + + public String anchorName(Element member) { + if (member.getKind() == ElementKind.CONSTRUCTOR + && configuration.isOutputHtml5()) { + return ""; + } else { + return utils.getSimpleName(member); } - return utils.getSimpleName(executableElement) + signatureParsed.toString(); } public Content seeTagToContent(Element element, DocTree see) { diff -r 8cb132b3a016 -r d18df41954ba src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/markup/HtmlDocWriter.java --- a/src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/markup/HtmlDocWriter.java Tue Oct 10 09:55:14 2017 -0700 +++ b/src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/markup/HtmlDocWriter.java Tue Oct 10 17:02:52 2017 -0700 @@ -59,7 +59,8 @@ public static final String CONTENT_TYPE = "text/html"; - DocPath pathToRoot; + private final HtmlConfiguration configuration; + private final DocPath pathToRoot; /** * Constructor. Initializes the destination file name through the super @@ -68,8 +69,9 @@ * @param configuration the configuration for this doclet * @param filename String file name. */ - public HtmlDocWriter(BaseConfiguration configuration, DocPath filename) { + public HtmlDocWriter(HtmlConfiguration configuration, DocPath filename) { super(configuration, filename); + this.configuration = configuration; this.pathToRoot = filename.parent().invert(); Messages messages = configuration.getMessages(); messages.notice("doclet.Generating_0", @@ -80,7 +82,9 @@ * Accessor for configuration. * @return the configuration for this doclet */ - public abstract BaseConfiguration configuration(); + public BaseConfiguration configuration() { + return configuration; + } public Content getHyperLink(DocPath link, String label) { return getHyperLink(link, new StringContent(label), false, "", "", ""); @@ -166,8 +170,6 @@ * @return a valid HTML name string. */ public String getName(String name) { - StringBuilder sb = new StringBuilder(); - char ch; /* The HTML 4 spec at http://www.w3.org/TR/html4/types.html#h-6.2 mentions * that the name/id should begin with a letter followed by other valid characters. * The HTML 5 spec (draft) is more permissive on names/ids where the only restriction @@ -178,8 +180,14 @@ * substitute it accordingly, "_" and "$" can appear at the beginning of a member name. * The method substitutes "$" with "Z:Z:D" and will prefix "_" with "Z:Z". */ + + if (configuration.isOutputHtml5()) { + return name.replaceAll(" +", ""); + } + + StringBuilder sb = new StringBuilder(); for (int i = 0; i < name.length(); i++) { - ch = name.charAt(i); + char ch = name.charAt(i); switch (ch) { case '(': case ')': diff -r 8cb132b3a016 -r d18df41954ba src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/markup/HtmlTree.java --- a/src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/markup/HtmlTree.java Tue Oct 10 09:55:14 2017 -0700 +++ b/src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/markup/HtmlTree.java Tue Oct 10 17:02:52 2017 -0700 @@ -181,36 +181,63 @@ return s; } - /** - * A set of ASCII URI characters to be left unencoded. + /* + * The sets of ASCII URI characters to be left unencoded. + * See "Uniform Resource Identifier (URI): Generic Syntax" + * IETF RFC 3986. https://tools.ietf.org/html/rfc3986 */ - public static final BitSet NONENCODING_CHARS = new BitSet(256); + public static final BitSet MAIN_CHARS; + public static final BitSet QUERY_FRAGMENT_CHARS; static { - // alphabetic characters - for (int i = 'a'; i <= 'z'; i++) { - NONENCODING_CHARS.set(i); - } - for (int i = 'A'; i <= 'Z'; i++) { - NONENCODING_CHARS.set(i); + BitSet alphaDigit = bitSet(bitSet('A', 'Z'), bitSet('a', 'z'), bitSet('0', '9')); + BitSet unreserved = bitSet(alphaDigit, bitSet("-._~")); + BitSet genDelims = bitSet(":/?#[]@"); + BitSet subDelims = bitSet("!$&'()*+,;="); + MAIN_CHARS = bitSet(unreserved, genDelims, subDelims); + BitSet pchar = bitSet(unreserved, subDelims, bitSet(":@")); + QUERY_FRAGMENT_CHARS = bitSet(pchar, bitSet("/?")); + } + + private static BitSet bitSet(String s) { + BitSet result = new BitSet(); + for (int i = 0; i < s.length(); i++) { + result.set(s.charAt(i)); } - // numeric characters - for (int i = '0'; i <= '9'; i++) { - NONENCODING_CHARS.set(i); - } - // Reserved characters as per RFC 3986. These are set of delimiting characters. - String noEnc = ":/?#[]@!$&'()*+,;="; - // Unreserved characters as per RFC 3986 which should not be percent encoded. - noEnc += "-._~"; - for (int i = 0; i < noEnc.length(); i++) { - NONENCODING_CHARS.set(noEnc.charAt(i)); - } + return result; + } + + private static BitSet bitSet(char from, char to) { + BitSet result = new BitSet(); + result.set(from, to + 1); + return result; } + private static BitSet bitSet(BitSet... sets) { + BitSet result = new BitSet(); + for (BitSet set : sets) { + result.or(set); + } + return result; + } + + /** + * Apply percent-encoding to a URL. + * This is similar to {@link java.net.URLEncoder} but + * is less aggressive about encoding some characters, + * like '(', ')', ',' which are used in the anchor + * names for Java methods in HTML5 mode. + */ private static String encodeURL(String url) { + BitSet nonEncodingChars = MAIN_CHARS; StringBuilder sb = new StringBuilder(); for (byte c : url.getBytes(Charset.forName("UTF-8"))) { - if (NONENCODING_CHARS.get(c & 0xFF)) { + if (c == '?' || c == '#') { + sb.append((char) c); + // switch to the more restrictive set inside + // the query and/or fragment + nonEncodingChars = QUERY_FRAGMENT_CHARS; + } else if (nonEncodingChars.get(c & 0xFF)) { sb.append((char) c); } else { sb.append(String.format("%%%02X", c & 0xFF)); diff -r 8cb132b3a016 -r d18df41954ba test/langtools/jdk/javadoc/doclet/lib/JavadocTester.java --- a/test/langtools/jdk/javadoc/doclet/lib/JavadocTester.java Tue Oct 10 09:55:14 2017 -0700 +++ b/test/langtools/jdk/javadoc/doclet/lib/JavadocTester.java Tue Oct 10 17:02:52 2017 -0700 @@ -37,6 +37,8 @@ import java.lang.ref.SoftReference; import java.lang.reflect.InvocationTargetException; import java.lang.reflect.Method; +import java.nio.charset.Charset; +import java.nio.charset.UnsupportedCharsetException; import java.nio.file.Files; import java.util.Arrays; import java.util.ArrayList; @@ -46,6 +48,7 @@ import java.util.HashMap; import java.util.List; import java.util.Map; +import java.util.Objects; import java.util.function.Function; @@ -150,6 +153,9 @@ /** The output directory used in the most recent call of javadoc. */ protected File outputDir; + /** The output charset used in the most recent call of javadoc. */ + protected Charset charset = Charset.defaultCharset(); + /** The exit code of the most recent call of javadoc. */ private int exitCode; @@ -158,6 +164,8 @@ /** A cache of file content, to avoid reading files unnecessarily. */ private final Map> fileContentCache = new HashMap<>(); + /** The charset used for files in the fileContentCache. */ + private Charset fileContentCacheCharset = null; /** Stream used for logging messages. */ protected final PrintStream out = System.out; @@ -293,13 +301,46 @@ out.println("Running javadoc (run " + javadocRunNum + ")..."); } + outputDir = new File("."); + String charsetArg = null; + String docencodingArg = null; + String encodingArg = null; for (int i = 0; i < args.length - 2; i++) { - if (args[i].equals("-d")) { - outputDir = new File(args[++i]); - break; + switch (args[i]) { + case "-d": + outputDir = new File(args[++i]); + break; + case "-charset": + charsetArg = args[++i]; + break; + case "-docencoding": + docencodingArg = args[++i]; + break; + case "-encoding": + encodingArg = args[++i]; + break; } } + + // The following replicates HtmlConfiguration.finishOptionSettings0 + // and sets up the charset used to read files. + String cs; + if (docencodingArg == null) { + if (charsetArg == null) { + cs = (encodingArg == null) ? "UTF-8" : encodingArg; + } else { + cs = charsetArg; + } + } else { + cs = docencodingArg; + } + try { + charset = Charset.forName(cs); + } catch (UnsupportedCharsetException e) { + charset = Charset.defaultCharset(); + } + out.println("args: " + Arrays.toString(args)); // log.setOutDir(outputDir); @@ -637,6 +678,10 @@ * @return the file in string format */ private String readFile(File baseDir, String fileName) throws Error { + if (!Objects.equals(fileContentCacheCharset, charset)) { + fileContentCache.clear(); + fileContentCacheCharset = charset; + } try { File file = new File(baseDir, fileName); SoftReference ref = fileContentCache.get(file); @@ -644,7 +689,8 @@ if (content != null) return content; - content = new String(Files.readAllBytes(file.toPath())); + // charset defaults to a value inferred from latest javadoc run + content = new String(Files.readAllBytes(file.toPath()), charset); fileContentCache.put(file, new SoftReference<>(content)); return content; } catch (FileNotFoundException e) { diff -r 8cb132b3a016 -r d18df41954ba test/langtools/jdk/javadoc/doclet/testAnchorNames/TestAnchorNames.java --- a/test/langtools/jdk/javadoc/doclet/testAnchorNames/TestAnchorNames.java Tue Oct 10 09:55:14 2017 -0700 +++ b/test/langtools/jdk/javadoc/doclet/testAnchorNames/TestAnchorNames.java Tue Oct 10 17:02:52 2017 -0700 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2013, 2016, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2013, 2017, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -23,29 +23,37 @@ /* * @test - * @bug 8025633 8025524 8081854 + * @bug 8025633 8025524 8081854 8187521 * @summary Test for valid name attribute in HTML anchors. * @author Bhavesh Patel - * @library ../lib + * @library /tools/lib ../lib * @modules jdk.javadoc/jdk.javadoc.internal.tool - * @build JavadocTester + * @build toolbox.ToolBox JavadocTester * @run main TestAnchorNames */ +import java.io.IOException; +import java.nio.charset.Charset; +import java.nio.file.Path; +import java.nio.file.Paths; + +import toolbox.*; + public class TestAnchorNames extends JavadocTester { - private static final String[] ARGS = new String[] { - - }; + public final ToolBox tb; + public static void main(String... args) throws Exception { + TestAnchorNames tester = new TestAnchorNames(); + tester.runTests(m -> new Object[] { Paths.get(m.getName()) }); + } - public static void main(String[] args) throws Exception { - TestAnchorNames tester = new TestAnchorNames(); - tester.runTests(); + public TestAnchorNames() { + tb = new ToolBox(); } @Test - void test() { - javadoc("-d", "out", + void testHtml4(Path ignore) { + javadoc("-d", "out-html4", "-sourcepath", testSrc, "-source", "8", //so that '_' can be used as an identifier "-use", @@ -153,11 +161,169 @@ "_"); // The marker name conversion should only affect HTML anchors. It should not - // affect the lables. + // affect the labels. checkOutput("pkg1/RegClass.html", false, " Z:Z_", " Z:Z:Dfield", " Z:Z_field_In_Class", " S_:D:D:D:D:DINT"); } + + @Test + void testHtml5(Path ignore) { + javadoc("-d", "out-html5", + "-sourcepath", testSrc, + "-source", "8", //so that '_' can be used as an identifier + "-use", + "-html5", + "pkg1"); + checkExit(Exit.OK); + + // Test some section markers and links to these markers + checkOutput("pkg1/RegClass.html", true, + "", + "", + "", + "", + "", + "", + "", + "", + "", + ""); + + // Test some members and link to these members + checkOutput("pkg1/RegClass.html", true, + //The marker for this appears in the serialized-form.html which we will + //test below + ""); + + // Test some fields + checkOutput("pkg1/RegClass.html", true, + "", + "", + "", + "", + "", + "", + "", + "", + "", + "", + "", + "", + "", + ""); + + checkOutput("pkg1/DeprMemClass.html", true, + "", + ""); + + // Test constructor + checkOutput("pkg1/RegClass.html", true, + "", + ""); + + // Test some methods + checkOutput("pkg1/RegClass.html", true, + "", + "", + "", + "", + "", + "", + "", + "", + "", + "", + "", + "", + "", + ""); + + checkOutput("pkg1/DeprMemClass.html", true, + "", + ""); + + // Test enum + checkOutput("pkg1/RegClass.Te$t_Enum.html", true, + "", + ""); + + // Test nested class + checkOutput("pkg1/RegClass._NestedClas$.html", true, + "", + ""); + + // Test class use page + checkOutput("pkg1/class-use/DeprMemClass.html", true, + ""); + + // Test deprecated list page + checkOutput("deprecated-list.html", true, + "", + ""); + + // Test constant values page + checkOutput("constant-values.html", true, + ""); + + // Test serialized form page + checkOutput("serialized-form.html", true, + //This is the marker for the link that appears in the pkg1.RegClass.html page + ""); + + // Test member name index page + checkOutput("index-all.html", true, + "", + "$", + "_"); + } + + /** + * The following test is somewhat simplistic, but it is useful + * in conjunction with the W3C Validation Service at https://validator.w3.org/nu/#file + * @param base A working directory for this method, in which some UTF-8 source files + * will be generated + * @throws IOException if there is a problem generating the source files + */ + @Test + void testNonAscii(Path base) throws IOException { + Path src = base.resolve("src"); + tb.writeJavaFiles(src, + "package p; public class Def {\n" + + " public int \u00e0\u00e9;\n" // a`e' + + " public void \u00c0\u00c9() { }\n" // A`E' + + " public int \u03b1\u03b2\u03b3;\n" // alpha beta gamma + + " public void \u0391\u0392\u0393() { }\n" // ALPHA BETA GAMMA + + "}", + "package p; \n" + + "/**\n" + + " * {@link Def#\u00e0\u00e9 àé}
\n" + + " * {@link Def#\u00c0\u00c9() ÀÉ}
\n" + + " * {@link Def#\u03b1\u03b2\u03b3 αβγ}
\n" + + " * {@link Def#\u0391\u0392\u0393() ΑΒΓ}
\n" + + " */\n" + + "public class Ref { }"); + + javadoc("-d", "out-nonAscii", + "-sourcepath", src.toString(), + "-html5", + "-encoding", "utf-8", + "p"); + checkExit(Exit.OK); + + checkOutput("p/Def.html", true, + "
", + "", + "", + ""); + + checkOutput("p/Ref.html", true, + "àé", + "ÀÉ", + "αβγ", + "ΑΒΓ"); + + } } diff -r 8cb132b3a016 -r d18df41954ba test/langtools/jdk/javadoc/doclet/testDocEncoding/TestDocEncoding.java --- a/test/langtools/jdk/javadoc/doclet/testDocEncoding/TestDocEncoding.java Tue Oct 10 09:55:14 2017 -0700 +++ b/test/langtools/jdk/javadoc/doclet/testDocEncoding/TestDocEncoding.java Tue Oct 10 17:02:52 2017 -0700 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2002, 2016, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2002, 2017, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -37,6 +37,8 @@ * @run main TestDocEncoding */ +import java.nio.charset.Charset; + public class TestDocEncoding extends JavadocTester { public static void main(String... args) throws Exception { @@ -53,6 +55,13 @@ "pkg"); checkExit(Exit.OK); + checkOutput("stylesheet.css", true, + "body {\n" + + " background-color:#ffffff;"); + + // reset the charset, for a negative test, that the -docencoding + // was effective and that the output is not in UTF-8. + charset = Charset.forName("UTF-8"); checkOutput("stylesheet.css", false, "body {\n" + " background-color:#ffffff;"); diff -r 8cb132b3a016 -r d18df41954ba test/langtools/jdk/javadoc/doclet/testSearch/TestSearch.java --- a/test/langtools/jdk/javadoc/doclet/testSearch/TestSearch.java Tue Oct 10 09:55:14 2017 -0700 +++ b/test/langtools/jdk/javadoc/doclet/testSearch/TestSearch.java Tue Oct 10 17:02:52 2017 -0700 @@ -23,7 +23,8 @@ /* * @test - * @bug 8141492 8071982 8141636 8147890 8166175 8168965 8176794 8175218 8147881 8181622 8182263 8074407 + * @bug 8141492 8071982 8141636 8147890 8166175 8168965 8176794 8175218 8147881 + * 8181622 8182263 8074407 8187521 * @summary Test the search feature of javadoc. * @author bpatel * @library ../lib @@ -64,7 +65,7 @@ checkExit(Exit.OK); checkInvalidUsageIndexTag(); checkSearchOutput(true); - checkSingleIndex(true); + checkSingleIndex(true, false); checkSingleIndexSearchTagDuplication(); checkJqueryAndImageFiles(true); checkSearchJS(); @@ -86,7 +87,7 @@ checkExit(Exit.ERROR); checkDocLintErrors(); checkSearchOutput(true); - checkSingleIndex(true); + checkSingleIndex(true, false); checkSingleIndexSearchTagDuplication(); checkJqueryAndImageFiles(true); checkSearchJS(); @@ -128,7 +129,7 @@ "-use", "pkg", "pkg1", "pkg2", "pkg3"); checkExit(Exit.OK); checkSearchOutput(true); - checkSingleIndex(true); + checkSingleIndex(true, true); checkSingleIndexSearchTagDuplication(); checkJqueryAndImageFiles(true); checkSearchJS(); @@ -280,7 +281,9 @@ "
"); } - void checkSingleIndex(boolean expectedOutput) { + void checkSingleIndex(boolean expectedOutput, boolean html5) { + String html_span_see_span = html5 ? "html%3Cspan%3Esee%3C/span%3E" : "html-span-see-/span-"; + // Test for search tags markup in index file. checkOutput("index-all.html", expectedOutput, "
" @@ -313,7 +316,7 @@ + "#nested%7B@indexnested_tag_test%7D\">nested {@index nested_tag_test} - " + "Search tag in pkg.AnotherClass.ModalExclusionType.NO_EXCLUDE
", "
html <span> see </span> - Search " + + "#" + html_span_see_span + "\">html <span> see </span> - Search " + "tag in pkg.AnotherClass.ModalExclusionType.APPLICATION_EXCLUDE
", "
quoted" + " - Search tag in pkg.AnotherClass.CONSTANT1
",