1
+ − 1 <HTML>
+ − 2
+ − 3
+ − 4 <HEAD>
+ − 5 <TITLE>
+ − 6 Using The HotSpot Serviceability Agent
+ − 7 </TITLE>
+ − 8 </HEAD>
+ − 9
+ − 10 <BODY TEXT="#000000" BGCOLOR="#FFFFFF">
+ − 11 <H1>
+ − 12 Using The HotSpot Serviceability Agent
+ − 13 </H1>
+ − 14
+ − 15 <P>
+ − 16 <H2>
+ − 17 Contents
+ − 18 </H2>
+ − 19 </P>
+ − 20
+ − 21 <UL>
+ − 22 <LI> <A HREF = "#INTRODUCTION">Introduction</A>
+ − 23 <LI> <A HREF = "#SOURCES">Organization of the sources</A>
+ − 24 <LI> <A HREF = "#RUNNING">Running HSDB</A>
+ − 25 <LI> <A HREF = "#NOTES">Notes</A>
+ − 26 </UL>
+ − 27
+ − 28 <H2>
+ − 29 <A NAME="INTRODUCTION">
+ − 30 Introduction
+ − 31 </A>
+ − 32 </H2>
+ − 33
+ − 34 <P>
+ − 35 The HotSpot Serviceability Agent (SA) is a set of Java APIs which
+ − 36 mirror the internal APIs of the HotSpot VM and which can be used to
+ − 37 examine the state of a HotSpot VM.
+ − 38 </P>
+ − 39
+ − 40 <P>
+ − 41 The system understands the layout of certain VM data structures and is
+ − 42 able to traverse these structures in an examination-only fashion; that
+ − 43 is, it does not rely on being able to run code in the target VM. For
+ − 44 this reason it transparently works with either a running VM or a core
+ − 45 file.
+ − 46 </P>
+ − 47
+ − 48 <P>
+ − 49 The system can reconstruct information about Java frames on the stack
+ − 50 and objects in the heap. Many of the important data structures in the
+ − 51 VM like the CodeCache, Universe, StubQueue, Frames, and VFrames have
+ − 52 been exposed and have relatively complete (or at least useful)
+ − 53 implementations.
+ − 54 </P>
+ − 55
+ − 56 <P>
+ − 57 A small graphical debugger called HSDB (the "HotSpot Debugger") has
+ − 58 been written using these APIs. It provides stack memory dumps
+ − 59 annotated with method invocations, compiled-code inlining (if
+ − 60 present), interpreted vs. compiled code, interpreter codelets (if
+ − 61 interpreted), and live oops from oop-map information. It also provides
+ − 62 a tree-based oop inspector. More information will be added as
+ − 63 necessary; please <A HREF = "mailto:kenneth.russell@eng">send
+ − 64 email</A> with suggestions on what would be useful.
+ − 65 </P>
+ − 66
+ − 67 <P>
+ − 68 The SA currently only works on Solaris. It uses dbx to connect to the
+ − 69 remote process or core file and communicates with a small piece of
+ − 70 code (an "import module") loaded into the debugger.
+ − 71 </P>
+ − 72
+ − 73 <H2>
+ − 74 <A NAME="SOURCES">
+ − 75 Organization of the sources
+ − 76 </A>
+ − 77 </H2>
+ − 78
+ − 79 <P>
+ − 80 The Java-side source code, which is the bulk of the SA, is in
+ − 81 src/share/vm/agent. The organization of the sun.jvm.hotspot package
+ − 82 hierarchy mirrors the organization in the VM. This should allow
+ − 83 engineers familiar with the HotSpot sources to quickly understand how
+ − 84 the SA works and to make modifications if necessary. To build these
+ − 85 sources, cd to src/share/vm/agent and type "make".
+ − 86 </P>
+ − 87
+ − 88 <P>
+ − 89
+ − 90 The SA on Solaris works by communicating with a small piece of code
+ − 91 (an "import module") loaded into dbx. The source code for this import
+ − 92 module is in src/os/solaris/agent. To build this library, cd to
+ − 93 src/os/solaris/agent and type "make 32bit" or "make 64bit". The
+ − 94 distinction is necessary because the SPARC version of dbx ships with
+ − 95 two versions of its executable, and depending on which architecture
+ − 96 (v8 or v9) the debugger is running on selects the appropriate
+ − 97 executable. The SA tries the v8 version first, but if you are running
+ − 98 on a v9 machine you must provide both versions to the SA.
+ − 99 </P>
+ − 100
+ − 101 <P>
+ − 102
+ − 103 The system is currently hardwired to look on jano for its dbx
+ − 104 executable and import module. The relevant directory structure looks
+ − 105 like this:
+ − 106
+ − 107 <UL>
+ − 108 <LI> .../hotspot/sa/
+ − 109 <UL>
+ − 110 <LI> solaris/
+ − 111 <UL>
+ − 112 <LI> sparc/
+ − 113 <UL>
+ − 114 <LI> bin/
+ − 115 <UL>
+ − 116 <LI> dbx: symlink to (v8) dbx 7.0 executable
+ − 117 </UL>
+ − 118 </UL>
+ − 119 <UL>
+ − 120 <LI> lib/
+ − 121 <UL>
+ − 122 <LI> libsvc_agent_dbx.so: 32-bit version of import module
+ − 123 </UL>
+ − 124 </UL>
+ − 125 <LI> sparcv9/
+ − 126 <UL>
+ − 127 <LI> lib/
+ − 128 <UL>
+ − 129 <LI> libsvc_agent_dbx.so: 32-bit version of import module
+ − 130 </UL>
+ − 131 </UL>
+ − 132 </UL>
+ − 133 </UL>
+ − 134 </UL>
+ − 135 </P>
+ − 136
+ − 137 <P>
+ − 138 The code which builds up path names to these executables is contained
+ − 139 in sun.jvm.hotspot.HotSpotAgent.java. There are hardcoded paths in
+ − 140 this file to jano, but the rest of the system is isolated from this.
+ − 141 </P>
+ − 142
+ − 143 <P>
+ − 144 (It would be nice to have a panel in the debugger allowing
+ − 145 configuration of all of the known operating systems' options; right
+ − 146 now Solaris is the only supported OS, making that easier.)
+ − 147 </P>
+ − 148
+ − 149 <H2>
+ − 150 <A NAME="RUNNING">
+ − 151 Running HSDB
+ − 152 </A>
+ − 153 </H2>
+ − 154
+ − 155 <P>
+ − 156 An installation of HSDB has been placed on jano. To access it, add the
+ − 157 following directory to your PATH:
+ − 158 </P>
+ − 159
+ − 160 <P>
+ − 161 <PRE>
+ − 162 /net/jano/export/disk05/hotspot/sa/bin/common
+ − 163 </PRE>
+ − 164 </P>
+ − 165
+ − 166 <P>
+ − 167 To start the debugger, type "hsdb".
+ − 168 </P>
+ − 169
+ − 170 <P>
+ − 171 Alternatively, you can start a local build of the debugger by building
+ − 172 the sources in src/share/vm/agent, cd'ing to that directory, and
+ − 173 typing "java sun.jvm.hotspot.HSDB".
+ − 174 </P>
+ − 175
+ − 176 <P>
+ − 177 There are three modes for the debugger: attaching to a local process,
+ − 178 opening a core file, and attaching to a remote "debug server". The
+ − 179 remote case requires two programs to be running on the remote machine:
+ − 180 the rmiregistry (see the script "start-rmiregistry" in this directory;
+ − 181 run this in the background) and the debug server (see the script
+ − 182 "start-debug-server"), in that order. start-rmiregistry takes no
+ − 183 arguments; start-debug-server takes as argument the process ID or the
+ − 184 executable and core file names to allow remote debugging of. Make sure
+ − 185 you do NOT have a CLASSPATH environment variable set when you run
+ − 186 these scripts. (The classes put into the rmiregistry are in sun.*, and
+ − 187 there are permissions problems if they aren't placed on the boot
+ − 188 classpath.)
+ − 189 </P>
+ − 190
+ − 191 <P>
+ − 192 NOTE that the SA currently only works against VMs on Solaris/SPARC.
+ − 193 Remote debugging of Solaris/SPARC VMs on arbitrary platforms is
+ − 194 possible using the debug server; select "Connect to debug server..."
+ − 195 in HSDB.
+ − 196 </P>
+ − 197
+ − 198 <P>
+ − 199 Once the debugger has been launched, the threads list is displayed.
+ − 200 The current set of functionality allows:
+ − 201 </P>
+ − 202
+ − 203 <UL>
+ − 204 <LI> Browsing of the annotated stack memory ("Stack Memory" button). It
+ − 205 is currently annotated with the following information:
+ − 206 <UL>
+ − 207 <LI> Method names of the Java frames and their extents (supporting
+ − 208 inlined compiled methods)
+ − 209 <LI> Locations and types of oops, found using the oop map information
+ − 210 from compiled methods (interpreter oop maps coming soon)
+ − 211 <LI> If a Java frame was interrupted by a signal (e.g., because of a
+ − 212 crash), annotates the frame with the signal name and number
+ − 213 <LI> Interpreter codelet descriptions for interpreted frames
+ − 214 </UL>
+ − 215 <LI> Finding which thread or threads caused a crash (currently
+ − 216 identified by the presence of a signal handler frame)
+ − 217 <LI> Browsing of oops using the Oop Inspector.
+ − 218 <LI> Browsing of the java.lang.Thread object's oop.
+ − 219 <LI> Object histogram and inspection of objects therein.
+ − 220 </UL>
+ − 221 </P>
+ − 222
+ − 223 <P>
+ − 224 More functionality is planned. Please <A HREF =
+ − 225 "mailto:kenneth.russell@eng">send email</A> with suggestions on what
+ − 226 would be useful, with any questions or comments, or if the debugger
+ − 227 crashes.
+ − 228 </P>
+ − 229
+ − 230 <H2>
+ − 231 <A NAME="NOTES">
+ − 232 Notes
+ − 233 </A>
+ − 234 </H2>
+ − 235
+ − 236 <P>
+ − 237 HSDB does not suspend the system at a safepoint, but at an arbitrary
+ − 238 point. This means that many of the invariants in the VM's code are not
+ − 239 followed.
+ − 240 </P>
+ − 241
+ − 242 <P>
+ − 243 As it happens, most of the places where the code ported over from the
+ − 244 VM has failed involve the topmost frame on the stack. Some
+ − 245 modifications have been made to allow the system to recognize
+ − 246 problematic situations.
+ − 247 </P>
+ − 248
+ − 249 <P>
+ − 250 Certainly, not all of the failure modes of the debugger have been
+ − 251 found. Please <A HREF = "mailto:kenneth.russell@eng">send email</A> if
+ − 252 HSDB throws an exception. The best debugging aid in these situations
+ − 253 is a core file since it is a static view of the VM to which we can
+ − 254 then adapt the debugger code, as opposed to having to try to suspend
+ − 255 the VM over and over to reproduce the failure. gcore (1) is a useful
+ − 256 tool. (NOTE: do not try gcore with any application using the DGA X
+ − 257 server extension (example: Java2Demo); the kernel will panic. See bug
+ − 258 4343237.)
+ − 259 </P>
+ − 260
+ − 261 </BODY>
+ − 262 </HTML>