hotspot/agent/doc/index.html
author minqi
Mon, 12 Nov 2012 14:03:53 -0800
changeset 14477 95e66ea71f71
parent 1 489c9b5090e2
permissions -rw-r--r--
6830717: replay of compilations would help with debugging Summary: When java process crashed in compiler thread, repeat the compilation process will help finding root cause. This is done with using SA dump application class data and replay data from core dump, then use debug version of jvm to recompile the problematic java method. Reviewed-by: kvn, twisti, sspitsyn Contributed-by: yumin.qi@oracle.com
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
1
489c9b5090e2 Initial load
duke
parents:
diff changeset
     1
<html>
489c9b5090e2 Initial load
duke
parents:
diff changeset
     2
<head>
489c9b5090e2 Initial load
duke
parents:
diff changeset
     3
<title>
489c9b5090e2 Initial load
duke
parents:
diff changeset
     4
Using HotSpot Serviceability Agent (SA)
489c9b5090e2 Initial load
duke
parents:
diff changeset
     5
</title>
489c9b5090e2 Initial load
duke
parents:
diff changeset
     6
</head>
489c9b5090e2 Initial load
duke
parents:
diff changeset
     7
<body>
489c9b5090e2 Initial load
duke
parents:
diff changeset
     8
<h1>Using HotSpot Serviceability Agent (SA)</h1>
489c9b5090e2 Initial load
duke
parents:
diff changeset
     9
489c9b5090e2 Initial load
duke
parents:
diff changeset
    10
<h3>HSDB GUI</h3>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    11
<p>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    12
The top-level GUI program using the HotSpot Serviceability Agent APIs is
489c9b5090e2 Initial load
duke
parents:
diff changeset
    13
called <b>HSDB</b>, the "HotSpot Debugger". To run it, type "hsdbproc.sh" 
489c9b5090e2 Initial load
duke
parents:
diff changeset
    14
or "hsdbwindbg.bat" or 64-bit variants (on Unix, Windows platforms 
489c9b5090e2 Initial load
duke
parents:
diff changeset
    15
respectively). More info. on HSDB GUI are in <a href="hsdb.html">hsdb.html</a>.
489c9b5090e2 Initial load
duke
parents:
diff changeset
    16
</p>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    17
489c9b5090e2 Initial load
duke
parents:
diff changeset
    18
<h3>SA Modes</h3>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    19
<p>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    20
There are three modes for the SA debugger: 
489c9b5090e2 Initial load
duke
parents:
diff changeset
    21
<ul>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    22
<li>attaching to a local process,
489c9b5090e2 Initial load
duke
parents:
diff changeset
    23
<li>opening a core file, and 
489c9b5090e2 Initial load
duke
parents:
diff changeset
    24
<li>attaching to a remote "debug server". 
489c9b5090e2 Initial load
duke
parents:
diff changeset
    25
</ul>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    26
<p>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    27
The remote case requires two programs to be running on the remote machine:
489c9b5090e2 Initial load
duke
parents:
diff changeset
    28
the rmiregistry (see the script "start-rmiregistry.sh" in this directory;
489c9b5090e2 Initial load
duke
parents:
diff changeset
    29
run this in the background) and the debug server (see the script
489c9b5090e2 Initial load
duke
parents:
diff changeset
    30
"start-debug-server-proc.sh"), in that order. start-rmiregistry.sh takes no
489c9b5090e2 Initial load
duke
parents:
diff changeset
    31
arguments; start-debug-server-proc.sh (or -windbg.bat) takes as argument 
489c9b5090e2 Initial load
duke
parents:
diff changeset
    32
the process ID or the executable and core file names to allow remote debugging
489c9b5090e2 Initial load
duke
parents:
diff changeset
    33
of. 
489c9b5090e2 Initial load
duke
parents:
diff changeset
    34
</p>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    35
489c9b5090e2 Initial load
duke
parents:
diff changeset
    36
<h3>Command line HSDB</h3>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    37
<p>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    38
There are also command line HSDB variants ("clhsdbproc.sh" or "clhsdbwindbg.bat"
489c9b5090e2 Initial load
duke
parents:
diff changeset
    39
or 64-bit variants). There is also a JavaScript based command line interface
489c9b5090e2 Initial load
duke
parents:
diff changeset
    40
called "jsdbproc.sh" [or "jsdbwindbg.bat" or 64-bit variants]. More details on 
489c9b5090e2 Initial load
duke
parents:
diff changeset
    41
command line interfaces can be found in 
489c9b5090e2 Initial load
duke
parents:
diff changeset
    42
<ul>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    43
<li><a href="clhsdb.html">clhsdb.html</a>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    44
<li><a href="jsdb.html">jsdb.html</a>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    45
</ul> 
489c9b5090e2 Initial load
duke
parents:
diff changeset
    46
</p>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    47
489c9b5090e2 Initial load
duke
parents:
diff changeset
    48
<h3>Other command line tools</h3>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    49
<p>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    50
The following table lists all SA command line tools. &lt;xxx&gt;windbg.bat 
489c9b5090e2 Initial load
duke
parents:
diff changeset
    51
files are for Windows. .sh files are for Solaris. &lt;xxx&gt;64.sh are for 
489c9b5090e2 Initial load
duke
parents:
diff changeset
    52
64 bit debugees.
489c9b5090e2 Initial load
duke
parents:
diff changeset
    53
</p>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    54
489c9b5090e2 Initial load
duke
parents:
diff changeset
    55
<table border="1">
489c9b5090e2 Initial load
duke
parents:
diff changeset
    56
<tr>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    57
<th>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    58
Tool
489c9b5090e2 Initial load
duke
parents:
diff changeset
    59
</th>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    60
<th>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    61
Description
489c9b5090e2 Initial load
duke
parents:
diff changeset
    62
</th>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    63
</tr>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    64
489c9b5090e2 Initial load
duke
parents:
diff changeset
    65
<tr>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    66
<td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    67
dumpflagsproc.sh,
489c9b5090e2 Initial load
duke
parents:
diff changeset
    68
dumpflagsproc64.sh,
489c9b5090e2 Initial load
duke
parents:
diff changeset
    69
dumpflagswindbg.bat
489c9b5090e2 Initial load
duke
parents:
diff changeset
    70
dumpflagswindbg64.bat
489c9b5090e2 Initial load
duke
parents:
diff changeset
    71
</td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    72
<td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    73
dumps name and value of all -XX JVM command line arguments passed
489c9b5090e2 Initial load
duke
parents:
diff changeset
    74
to debuggee. 
489c9b5090e2 Initial load
duke
parents:
diff changeset
    75
</td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    76
</tr>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    77
489c9b5090e2 Initial load
duke
parents:
diff changeset
    78
<tr>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    79
<td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    80
<a name="dumpsysprops"></a>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    81
dumpsyspropsproc.sh,
489c9b5090e2 Initial load
duke
parents:
diff changeset
    82
dumpsyspropsproc64.sh,
489c9b5090e2 Initial load
duke
parents:
diff changeset
    83
dumpsyspropswindbg.bat
489c9b5090e2 Initial load
duke
parents:
diff changeset
    84
dumpsyspropswindbg64.bat
489c9b5090e2 Initial load
duke
parents:
diff changeset
    85
</td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    86
<td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    87
This prints name and value of Java level System properties.
489c9b5090e2 Initial load
duke
parents:
diff changeset
    88
</td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    89
</tr>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    90
489c9b5090e2 Initial load
duke
parents:
diff changeset
    91
<tr>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    92
<td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    93
<a name="heapdump"></a>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    94
heapdumpproc.sh,
489c9b5090e2 Initial load
duke
parents:
diff changeset
    95
heapdumpproc64.sh,
489c9b5090e2 Initial load
duke
parents:
diff changeset
    96
heapdumpwindbg.bat
489c9b5090e2 Initial load
duke
parents:
diff changeset
    97
heapdumpwindbg64.bat
489c9b5090e2 Initial load
duke
parents:
diff changeset
    98
</td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
    99
<td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   100
Dumps heap in a file in hprof binary format.
489c9b5090e2 Initial load
duke
parents:
diff changeset
   101
</td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   102
</tr>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   103
489c9b5090e2 Initial load
duke
parents:
diff changeset
   104
<tr>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   105
<td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   106
<a name="heapsum"></a>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   107
heapsumproc.sh,
489c9b5090e2 Initial load
duke
parents:
diff changeset
   108
heapsumproc64.sh,
489c9b5090e2 Initial load
duke
parents:
diff changeset
   109
heapsumwindbg.bat
489c9b5090e2 Initial load
duke
parents:
diff changeset
   110
heapsumwindbg64.bat
489c9b5090e2 Initial load
duke
parents:
diff changeset
   111
</td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   112
<td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   113
Prints summary information on Java heap.
489c9b5090e2 Initial load
duke
parents:
diff changeset
   114
</td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   115
</tr>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   116
489c9b5090e2 Initial load
duke
parents:
diff changeset
   117
489c9b5090e2 Initial load
duke
parents:
diff changeset
   118
<tr>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   119
<td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   120
jcoreproc.sh,
489c9b5090e2 Initial load
duke
parents:
diff changeset
   121
jcoreproc64.sh,
489c9b5090e2 Initial load
duke
parents:
diff changeset
   122
jcorewindbg.bat
489c9b5090e2 Initial load
duke
parents:
diff changeset
   123
jcorewindbg64.bat
489c9b5090e2 Initial load
duke
parents:
diff changeset
   124
</td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   125
<td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   126
This can retrieve .class files from the debuggee.
489c9b5090e2 Initial load
duke
parents:
diff changeset
   127
set the environment variable <b>JCORE_PACKAGES</b> to comman separated list of
489c9b5090e2 Initial load
duke
parents:
diff changeset
   128
packages whose classes have to be retrieved from the core file.
489c9b5090e2 Initial load
duke
parents:
diff changeset
   129
</td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   130
</tr>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   131
489c9b5090e2 Initial load
duke
parents:
diff changeset
   132
<tr>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   133
<tr>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   134
<td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   135
jstackproc.sh,
489c9b5090e2 Initial load
duke
parents:
diff changeset
   136
jstackproc64.sh,
489c9b5090e2 Initial load
duke
parents:
diff changeset
   137
jstackwindbg.bat
489c9b5090e2 Initial load
duke
parents:
diff changeset
   138
jstackwindbg64.bat
489c9b5090e2 Initial load
duke
parents:
diff changeset
   139
</td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   140
<td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   141
used to get java stack trace for all java threads.
489c9b5090e2 Initial load
duke
parents:
diff changeset
   142
</td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   143
</tr>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   144
489c9b5090e2 Initial load
duke
parents:
diff changeset
   145
<tr>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   146
<td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   147
jhistoproc.sh,
489c9b5090e2 Initial load
duke
parents:
diff changeset
   148
jhistoproc64.sh,
489c9b5090e2 Initial load
duke
parents:
diff changeset
   149
jhistowindbg.bat
489c9b5090e2 Initial load
duke
parents:
diff changeset
   150
jhistowindbg64.bat
489c9b5090e2 Initial load
duke
parents:
diff changeset
   151
</td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   152
<td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   153
used to get object histogram of java heap.
489c9b5090e2 Initial load
duke
parents:
diff changeset
   154
</td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   155
</tr>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   156
489c9b5090e2 Initial load
duke
parents:
diff changeset
   157
<tr>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   158
<td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   159
permstatproc.sh,
489c9b5090e2 Initial load
duke
parents:
diff changeset
   160
permstatproc64.sh,
489c9b5090e2 Initial load
duke
parents:
diff changeset
   161
permstatwindbg.bat
489c9b5090e2 Initial load
duke
parents:
diff changeset
   162
permstatwindbg64.bat
489c9b5090e2 Initial load
duke
parents:
diff changeset
   163
</td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   164
<td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   165
To gather statistics on perm. generation.
489c9b5090e2 Initial load
duke
parents:
diff changeset
   166
</td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   167
</tr>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   168
489c9b5090e2 Initial load
duke
parents:
diff changeset
   169
<a name="mixed_pstack"></a>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   170
<tr>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   171
<tr>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   172
<td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   173
pstackproc.sh,
489c9b5090e2 Initial load
duke
parents:
diff changeset
   174
pstackproc64.sh,
489c9b5090e2 Initial load
duke
parents:
diff changeset
   175
pstackwindbg.bat
489c9b5090e2 Initial load
duke
parents:
diff changeset
   176
pstackwindbg64.bat
489c9b5090e2 Initial load
duke
parents:
diff changeset
   177
</td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   178
<td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   179
This is cross platform mixed mode pstack utility. This works on any (non-java as well) process, core dump. For java process and core dumps, this prints both java and C/C++ frames. 
489c9b5090e2 Initial load
duke
parents:
diff changeset
   180
</td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   181
</tr>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   182
489c9b5090e2 Initial load
duke
parents:
diff changeset
   183
<tr>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   184
<td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   185
pmapproc.sh,
489c9b5090e2 Initial load
duke
parents:
diff changeset
   186
pmapproc64.sh,
489c9b5090e2 Initial load
duke
parents:
diff changeset
   187
pmapwindbg.bat
489c9b5090e2 Initial load
duke
parents:
diff changeset
   188
pmapwindbg64.bat
489c9b5090e2 Initial load
duke
parents:
diff changeset
   189
</td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   190
<td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   191
This is cross platform Solaris pmap-like utility. 
489c9b5090e2 Initial load
duke
parents:
diff changeset
   192
</td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   193
</tr>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   194
489c9b5090e2 Initial load
duke
parents:
diff changeset
   195
<tr>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   196
<td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   197
soqlproc.sh,
489c9b5090e2 Initial load
duke
parents:
diff changeset
   198
soqlproc64.sh,
489c9b5090e2 Initial load
duke
parents:
diff changeset
   199
soqlwindbg.bat
489c9b5090e2 Initial load
duke
parents:
diff changeset
   200
soqlwindbg64.bat
489c9b5090e2 Initial load
duke
parents:
diff changeset
   201
</td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   202
<td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   203
This is command line SOQL - Simple Object Query Language tool.
489c9b5090e2 Initial load
duke
parents:
diff changeset
   204
SOQL is SQL-like query language to query Java heap.
489c9b5090e2 Initial load
duke
parents:
diff changeset
   205
</td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   206
<tr>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   207
<td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   208
start-debug-server-proc.sh,
489c9b5090e2 Initial load
duke
parents:
diff changeset
   209
start-debug-server-proc64.sh,
489c9b5090e2 Initial load
duke
parents:
diff changeset
   210
start-debug-server-windbg.bat,
489c9b5090e2 Initial load
duke
parents:
diff changeset
   211
start-debug-server-windbg64.bat,
489c9b5090e2 Initial load
duke
parents:
diff changeset
   212
start-rmiregistry.bat,
489c9b5090e2 Initial load
duke
parents:
diff changeset
   213
start-rmiregistry64.bat,
489c9b5090e2 Initial load
duke
parents:
diff changeset
   214
start-rmiregistry.sh
489c9b5090e2 Initial load
duke
parents:
diff changeset
   215
start-rmiregistry64.sh
489c9b5090e2 Initial load
duke
parents:
diff changeset
   216
</td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   217
<td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   218
These scripts are used to run SA remotely. 
489c9b5090e2 Initial load
duke
parents:
diff changeset
   219
</td>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   220
</tr>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   221
</table>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   222
14477
95e66ea71f71 6830717: replay of compilations would help with debugging
minqi
parents: 1
diff changeset
   223
<h3>C2 Compilation Replay</h3>
95e66ea71f71 6830717: replay of compilations would help with debugging
minqi
parents: 1
diff changeset
   224
<p>
95e66ea71f71 6830717: replay of compilations would help with debugging
minqi
parents: 1
diff changeset
   225
When a java process crashes in compiled method, usually a core file is saved.
95e66ea71f71 6830717: replay of compilations would help with debugging
minqi
parents: 1
diff changeset
   226
The C2 replay function can reproduce the compiling process in the core.
95e66ea71f71 6830717: replay of compilations would help with debugging
minqi
parents: 1
diff changeset
   227
<a href="c2replay.html">c2replay.html</a>
95e66ea71f71 6830717: replay of compilations would help with debugging
minqi
parents: 1
diff changeset
   228
1
489c9b5090e2 Initial load
duke
parents:
diff changeset
   229
<h3>Debugging transported core dumps</h3>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   230
<p>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   231
When a core dump is moved from the machine where it was produced to a
489c9b5090e2 Initial load
duke
parents:
diff changeset
   232
difference machine, it may not always be possible for SA to debug the same.
489c9b5090e2 Initial load
duke
parents:
diff changeset
   233
More info. on debugging on transported core dumps is in
489c9b5090e2 Initial load
duke
parents:
diff changeset
   234
<a href="transported_core.html">transported_core.html</a>.
489c9b5090e2 Initial load
duke
parents:
diff changeset
   235
</p>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   236
489c9b5090e2 Initial load
duke
parents:
diff changeset
   237
<h3>SA Bugs</h3>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   238
<p>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   239
Not all of the possible states of target VMs have been tested (or
489c9b5090e2 Initial load
duke
parents:
diff changeset
   240
supportable) with SA. For example, the SA will probably not work at all 
489c9b5090e2 Initial load
duke
parents:
diff changeset
   241
if it freezes the target VM during certain phases of GC. When filing bugs
489c9b5090e2 Initial load
duke
parents:
diff changeset
   242
a pointer to a core file (see gcore(1)) which the SA can not handle well 
489c9b5090e2 Initial load
duke
parents:
diff changeset
   243
is best.
489c9b5090e2 Initial load
duke
parents:
diff changeset
   244
</p>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   245
489c9b5090e2 Initial load
duke
parents:
diff changeset
   246
</body>
489c9b5090e2 Initial load
duke
parents:
diff changeset
   247
</html>