|
1
|
1 |
<HTML>
|
|
|
2 |
|
|
|
3 |
|
|
|
4 |
<HEAD>
|
|
|
5 |
<TITLE>
|
|
|
6 |
Using The HotSpot Serviceability Agent
|
|
|
7 |
</TITLE>
|
|
|
8 |
</HEAD>
|
|
|
9 |
|
|
|
10 |
<BODY TEXT="#000000" BGCOLOR="#FFFFFF">
|
|
|
11 |
<H1>
|
|
|
12 |
Using The HotSpot Serviceability Agent
|
|
|
13 |
</H1>
|
|
|
14 |
|
|
|
15 |
<P>
|
|
|
16 |
<H2>
|
|
|
17 |
Contents
|
|
|
18 |
</H2>
|
|
|
19 |
</P>
|
|
|
20 |
|
|
|
21 |
<UL>
|
|
|
22 |
<LI> <A HREF = "#INTRODUCTION">Introduction</A>
|
|
|
23 |
<LI> <A HREF = "#SOURCES">Organization of the sources</A>
|
|
|
24 |
<LI> <A HREF = "#RUNNING">Running HSDB</A>
|
|
|
25 |
<LI> <A HREF = "#NOTES">Notes</A>
|
|
|
26 |
</UL>
|
|
|
27 |
|
|
|
28 |
<H2>
|
|
|
29 |
<A NAME="INTRODUCTION">
|
|
|
30 |
Introduction
|
|
|
31 |
</A>
|
|
|
32 |
</H2>
|
|
|
33 |
|
|
|
34 |
<P>
|
|
|
35 |
The HotSpot Serviceability Agent (SA) is a set of Java APIs which
|
|
|
36 |
mirror the internal APIs of the HotSpot VM and which can be used to
|
|
|
37 |
examine the state of a HotSpot VM.
|
|
|
38 |
</P>
|
|
|
39 |
|
|
|
40 |
<P>
|
|
|
41 |
The system understands the layout of certain VM data structures and is
|
|
|
42 |
able to traverse these structures in an examination-only fashion; that
|
|
|
43 |
is, it does not rely on being able to run code in the target VM. For
|
|
|
44 |
this reason it transparently works with either a running VM or a core
|
|
|
45 |
file.
|
|
|
46 |
</P>
|
|
|
47 |
|
|
|
48 |
<P>
|
|
|
49 |
The system can reconstruct information about Java frames on the stack
|
|
|
50 |
and objects in the heap. Many of the important data structures in the
|
|
|
51 |
VM like the CodeCache, Universe, StubQueue, Frames, and VFrames have
|
|
|
52 |
been exposed and have relatively complete (or at least useful)
|
|
|
53 |
implementations.
|
|
|
54 |
</P>
|
|
|
55 |
|
|
|
56 |
<P>
|
|
|
57 |
A small graphical debugger called HSDB (the "HotSpot Debugger") has
|
|
|
58 |
been written using these APIs. It provides stack memory dumps
|
|
|
59 |
annotated with method invocations, compiled-code inlining (if
|
|
|
60 |
present), interpreted vs. compiled code, interpreter codelets (if
|
|
|
61 |
interpreted), and live oops from oop-map information. It also provides
|
|
|
62 |
a tree-based oop inspector. More information will be added as
|
|
|
63 |
necessary; please <A HREF = "mailto:kenneth.russell@eng">send
|
|
|
64 |
email</A> with suggestions on what would be useful.
|
|
|
65 |
</P>
|
|
|
66 |
|
|
|
67 |
<P>
|
|
|
68 |
The SA currently only works on Solaris. It uses dbx to connect to the
|
|
|
69 |
remote process or core file and communicates with a small piece of
|
|
|
70 |
code (an "import module") loaded into the debugger.
|
|
|
71 |
</P>
|
|
|
72 |
|
|
|
73 |
<H2>
|
|
|
74 |
<A NAME="SOURCES">
|
|
|
75 |
Organization of the sources
|
|
|
76 |
</A>
|
|
|
77 |
</H2>
|
|
|
78 |
|
|
|
79 |
<P>
|
|
|
80 |
The Java-side source code, which is the bulk of the SA, is in
|
|
|
81 |
src/share/vm/agent. The organization of the sun.jvm.hotspot package
|
|
|
82 |
hierarchy mirrors the organization in the VM. This should allow
|
|
|
83 |
engineers familiar with the HotSpot sources to quickly understand how
|
|
|
84 |
the SA works and to make modifications if necessary. To build these
|
|
|
85 |
sources, cd to src/share/vm/agent and type "make".
|
|
|
86 |
</P>
|
|
|
87 |
|
|
|
88 |
<P>
|
|
|
89 |
|
|
|
90 |
The SA on Solaris works by communicating with a small piece of code
|
|
|
91 |
(an "import module") loaded into dbx. The source code for this import
|
|
|
92 |
module is in src/os/solaris/agent. To build this library, cd to
|
|
|
93 |
src/os/solaris/agent and type "make 32bit" or "make 64bit". The
|
|
|
94 |
distinction is necessary because the SPARC version of dbx ships with
|
|
|
95 |
two versions of its executable, and depending on which architecture
|
|
|
96 |
(v8 or v9) the debugger is running on selects the appropriate
|
|
|
97 |
executable. The SA tries the v8 version first, but if you are running
|
|
|
98 |
on a v9 machine you must provide both versions to the SA.
|
|
|
99 |
</P>
|
|
|
100 |
|
|
|
101 |
<P>
|
|
|
102 |
|
|
|
103 |
The system is currently hardwired to look on jano for its dbx
|
|
|
104 |
executable and import module. The relevant directory structure looks
|
|
|
105 |
like this:
|
|
|
106 |
|
|
|
107 |
<UL>
|
|
|
108 |
<LI> .../hotspot/sa/
|
|
|
109 |
<UL>
|
|
|
110 |
<LI> solaris/
|
|
|
111 |
<UL>
|
|
|
112 |
<LI> sparc/
|
|
|
113 |
<UL>
|
|
|
114 |
<LI> bin/
|
|
|
115 |
<UL>
|
|
|
116 |
<LI> dbx: symlink to (v8) dbx 7.0 executable
|
|
|
117 |
</UL>
|
|
|
118 |
</UL>
|
|
|
119 |
<UL>
|
|
|
120 |
<LI> lib/
|
|
|
121 |
<UL>
|
|
|
122 |
<LI> libsvc_agent_dbx.so: 32-bit version of import module
|
|
|
123 |
</UL>
|
|
|
124 |
</UL>
|
|
|
125 |
<LI> sparcv9/
|
|
|
126 |
<UL>
|
|
|
127 |
<LI> lib/
|
|
|
128 |
<UL>
|
|
|
129 |
<LI> libsvc_agent_dbx.so: 32-bit version of import module
|
|
|
130 |
</UL>
|
|
|
131 |
</UL>
|
|
|
132 |
</UL>
|
|
|
133 |
</UL>
|
|
|
134 |
</UL>
|
|
|
135 |
</P>
|
|
|
136 |
|
|
|
137 |
<P>
|
|
|
138 |
The code which builds up path names to these executables is contained
|
|
|
139 |
in sun.jvm.hotspot.HotSpotAgent.java. There are hardcoded paths in
|
|
|
140 |
this file to jano, but the rest of the system is isolated from this.
|
|
|
141 |
</P>
|
|
|
142 |
|
|
|
143 |
<P>
|
|
|
144 |
(It would be nice to have a panel in the debugger allowing
|
|
|
145 |
configuration of all of the known operating systems' options; right
|
|
|
146 |
now Solaris is the only supported OS, making that easier.)
|
|
|
147 |
</P>
|
|
|
148 |
|
|
|
149 |
<H2>
|
|
|
150 |
<A NAME="RUNNING">
|
|
|
151 |
Running HSDB
|
|
|
152 |
</A>
|
|
|
153 |
</H2>
|
|
|
154 |
|
|
|
155 |
<P>
|
|
|
156 |
An installation of HSDB has been placed on jano. To access it, add the
|
|
|
157 |
following directory to your PATH:
|
|
|
158 |
</P>
|
|
|
159 |
|
|
|
160 |
<P>
|
|
|
161 |
<PRE>
|
|
|
162 |
/net/jano/export/disk05/hotspot/sa/bin/common
|
|
|
163 |
</PRE>
|
|
|
164 |
</P>
|
|
|
165 |
|
|
|
166 |
<P>
|
|
|
167 |
To start the debugger, type "hsdb".
|
|
|
168 |
</P>
|
|
|
169 |
|
|
|
170 |
<P>
|
|
|
171 |
Alternatively, you can start a local build of the debugger by building
|
|
|
172 |
the sources in src/share/vm/agent, cd'ing to that directory, and
|
|
|
173 |
typing "java sun.jvm.hotspot.HSDB".
|
|
|
174 |
</P>
|
|
|
175 |
|
|
|
176 |
<P>
|
|
|
177 |
There are three modes for the debugger: attaching to a local process,
|
|
|
178 |
opening a core file, and attaching to a remote "debug server". The
|
|
|
179 |
remote case requires two programs to be running on the remote machine:
|
|
|
180 |
the rmiregistry (see the script "start-rmiregistry" in this directory;
|
|
|
181 |
run this in the background) and the debug server (see the script
|
|
|
182 |
"start-debug-server"), in that order. start-rmiregistry takes no
|
|
|
183 |
arguments; start-debug-server takes as argument the process ID or the
|
|
|
184 |
executable and core file names to allow remote debugging of. Make sure
|
|
|
185 |
you do NOT have a CLASSPATH environment variable set when you run
|
|
|
186 |
these scripts. (The classes put into the rmiregistry are in sun.*, and
|
|
|
187 |
there are permissions problems if they aren't placed on the boot
|
|
|
188 |
classpath.)
|
|
|
189 |
</P>
|
|
|
190 |
|
|
|
191 |
<P>
|
|
|
192 |
NOTE that the SA currently only works against VMs on Solaris/SPARC.
|
|
|
193 |
Remote debugging of Solaris/SPARC VMs on arbitrary platforms is
|
|
|
194 |
possible using the debug server; select "Connect to debug server..."
|
|
|
195 |
in HSDB.
|
|
|
196 |
</P>
|
|
|
197 |
|
|
|
198 |
<P>
|
|
|
199 |
Once the debugger has been launched, the threads list is displayed.
|
|
|
200 |
The current set of functionality allows:
|
|
|
201 |
</P>
|
|
|
202 |
|
|
|
203 |
<UL>
|
|
|
204 |
<LI> Browsing of the annotated stack memory ("Stack Memory" button). It
|
|
|
205 |
is currently annotated with the following information:
|
|
|
206 |
<UL>
|
|
|
207 |
<LI> Method names of the Java frames and their extents (supporting
|
|
|
208 |
inlined compiled methods)
|
|
|
209 |
<LI> Locations and types of oops, found using the oop map information
|
|
|
210 |
from compiled methods (interpreter oop maps coming soon)
|
|
|
211 |
<LI> If a Java frame was interrupted by a signal (e.g., because of a
|
|
|
212 |
crash), annotates the frame with the signal name and number
|
|
|
213 |
<LI> Interpreter codelet descriptions for interpreted frames
|
|
|
214 |
</UL>
|
|
|
215 |
<LI> Finding which thread or threads caused a crash (currently
|
|
|
216 |
identified by the presence of a signal handler frame)
|
|
|
217 |
<LI> Browsing of oops using the Oop Inspector.
|
|
|
218 |
<LI> Browsing of the java.lang.Thread object's oop.
|
|
|
219 |
<LI> Object histogram and inspection of objects therein.
|
|
|
220 |
</UL>
|
|
|
221 |
</P>
|
|
|
222 |
|
|
|
223 |
<P>
|
|
|
224 |
More functionality is planned. Please <A HREF =
|
|
|
225 |
"mailto:kenneth.russell@eng">send email</A> with suggestions on what
|
|
|
226 |
would be useful, with any questions or comments, or if the debugger
|
|
|
227 |
crashes.
|
|
|
228 |
</P>
|
|
|
229 |
|
|
|
230 |
<H2>
|
|
|
231 |
<A NAME="NOTES">
|
|
|
232 |
Notes
|
|
|
233 |
</A>
|
|
|
234 |
</H2>
|
|
|
235 |
|
|
|
236 |
<P>
|
|
|
237 |
HSDB does not suspend the system at a safepoint, but at an arbitrary
|
|
|
238 |
point. This means that many of the invariants in the VM's code are not
|
|
|
239 |
followed.
|
|
|
240 |
</P>
|
|
|
241 |
|
|
|
242 |
<P>
|
|
|
243 |
As it happens, most of the places where the code ported over from the
|
|
|
244 |
VM has failed involve the topmost frame on the stack. Some
|
|
|
245 |
modifications have been made to allow the system to recognize
|
|
|
246 |
problematic situations.
|
|
|
247 |
</P>
|
|
|
248 |
|
|
|
249 |
<P>
|
|
|
250 |
Certainly, not all of the failure modes of the debugger have been
|
|
|
251 |
found. Please <A HREF = "mailto:kenneth.russell@eng">send email</A> if
|
|
|
252 |
HSDB throws an exception. The best debugging aid in these situations
|
|
|
253 |
is a core file since it is a static view of the VM to which we can
|
|
|
254 |
then adapt the debugger code, as opposed to having to try to suspend
|
|
|
255 |
the VM over and over to reproduce the failure. gcore (1) is a useful
|
|
|
256 |
tool. (NOTE: do not try gcore with any application using the DGA X
|
|
|
257 |
server extension (example: Java2Demo); the kernel will panic. See bug
|
|
|
258 |
4343237.)
|
|
|
259 |
</P>
|
|
|
260 |
|
|
|
261 |
</BODY>
|
|
|
262 |
</HTML>
|