Static Analysis for Rooting and Heap Write Hazards¶
Treeherder can run two static analysis builds: the full browser (linux64-haz), just the JS shell (linux64-shell-haz). They show up on treeherder as H
and SM(H)
.
Diagnosing a hazard failure¶
The first step is to look at what sort of hazard is being reported. There are two types that cause the job to fail: stack rooting hazards for garbage collection, and heap write thread safety hazards for stylo.
The summary output will include either the string <N> rooting hazards detected
or <N> heap write hazards detected out of <M> allowed
. See the appropriate section below for each.
Diagnosing a rooting hazards failure¶
Click on the H
build link, select the âArtifactsâ pane on the bottom left, and download the public/build/hazards.txt.gz
file.
Example snippet:
Function 'jsopcode.cpp:uint8 DecompileExpressionFromStack(JSContext*, int32, int32, class JS::Handle<JS::Value>, int8**)' has unrooted 'ed' of type 'ExpressionDecompiler' live across GC call 'uint8 ExpressionDecompiler::decompilePC(uint8*)' at js/src/jsopcode.cpp:1866
js/src/jsopcode.cpp:1866: Assume(74,75, !__temp_23*, true)
js/src/jsopcode.cpp:1867: Assign(75,76, return := 0)
js/src/jsopcode.cpp:1867: Call(76,77, ed.~ExpressionDecompiler())
GC Function: uint8 ExpressionDecompiler::decompilePC(uint8*)
JSString* js::ValueToSource(JSContext*, class JS::Handle<JS::Value>)
uint8 js::Invoke(JSContext*, JS::Value*, JS::Value*, uint32, JS::Value*, class JS::MutableHandle<JS::Value>)
uint8 js::Invoke(JSContext*, JS::CallArgs, uint32)
JSScript* JSFunction::getOrCreateScript(JSContext*)
uint8 JSFunction::createScriptForLazilyInterpretedFunction(JSContext*, class JS::Handle<JSFunction*>)
uint8 JSRuntime::cloneSelfHostedFunctionScript(JSContext*, class JS::Handle<js::PropertyName*>, class JS::Handle<JSFunction*>)
JSScript* js::CloneScript(JSContext*, class JS::Handle<JSObject*>, class JS::Handle<JSFunction*>, const class JS::Handle<JSScript*>, uint32)
JSObject* js::CloneStaticBlockObject(JSContext*, class JS::Handle<JSObject*>, class JS::Handle<js::StaticBlockObject*>)
js::StaticBlockObject* js::StaticBlockObject::create(js::ExclusiveContext*)
js::Shape* js::EmptyShape::getInitialShape(js::ExclusiveContext*, js::Class*, js::TaggedProto, JSObject*, JSObject*, uint32, uint32)
js::Shape* js::EmptyShape::getInitialShape(js::ExclusiveContext*, js::Class*, js::TaggedProto, JSObject*, JSObject*, uint64, uint32)
js::UnownedBaseShape* js::BaseShape::getUnowned(js::ExclusiveContext*, js::StackBaseShape*)
js::BaseShape* js_NewGCBaseShape(js::ThreadSafeContext*) [with js::AllowGC allowGC = (js::AllowGC)1u]
js::BaseShape* js::gc::NewGCThing(js::ThreadSafeContext*, uint32, uint64, uint32) [with T = js::BaseShape; js::AllowGC allowGC = (js::AllowGC)1u; size_t = long unsigned int]
void js::gc::RunDebugGC(JSContext*)
void js::MinorGC(JSRuntime*, uint32)
GC
This means that a rooting hazard was discovered at js/src/jsopcode.cpp
line 1866, in the function DecompileExpressionFromStack
(it is prefixed with the filename because itâs a static function.) The problem is that there is an unrooted variable ed
that holds an ExpressionDecompiler
live across a call to decompilePC
. âLiveâ means that the variable is used after the call to decompilePC
returns. decompilePC
may trigger a GC according to the static call stack given starting from the line beginning with âGC Function:
â.
The hazard itself has some barely comprehensible Assume(...)
and Call(...)
gibberish that describes the exact data flow path of the variable into the function call. That stuff is rarely useful â usually, youâll only need to look at it if itâs complaining about a temporary and you want to know where the temporary came from. The type ExpressionDecompiler
is believed to hold pointers to GC-controlled objects of some sort. The analysis currently does not describe the exact field it is worried about.
To unpack this a little, the analysis is saying the following can happen:
ExpressionDecompiler
contains some pointer to a GC thing. For example, it might have a fieldobj
of typeJSObject*
.DecompileExpressionFromStack
is called.A pointer is stored in that field of the
ed
variable.decompilePC
is invoked, which callsValueToSource
, which callsInvoke
, which eventually callsjs::MinorGC
During the resulting garbage collection, the object pointed to by
ed.obj
is moved to a different location. All pointers stored in the JS heap are updated automatically, as are all rooted pointers.ed.obj
is not, because the GC doesnât know about it.After
decompilePC
returns, something accessesed.obj
. This is now a stale pointer, and may refer to just about anything â the wrong object, an invalid object, or whatever. As TeX would say, badness 10000.
Diagnosing a heap write hazard failure¶
For the thread unsafe heap write analysis, a hazard means that some Gecko_* function calls, directly or indirectly, code that writes to something on the heap, or calls an unknown function that might write to something on the heap. The analysis requires quite a few annotations to describe things that are actually safe. This section will be expanded as we gain more experience with the analysis, but here are some common issues:
Adding a new Gecko_* function: often, you will need to annotate any outparams or owned (thread-local) parameters in the
treatAsSafeArgument
function injs/src/devtools/rootAnalysis/analyzeHeapWrites.js
.Calling some libc function: if you add a call to some random libc function (eg
sin()
orfloor()
orceil()
, though the latter two are already annotated), the analysis will report an âExternal Functionâ. Add it tocheckExternalFunction
, assuming it doesnât have the possibility of writing to shared heap memory.If you call some non-returning (crashing) function that the analysis doesnât know about, youâll need to add it to
ignoreContents
.
On the other hand, you might have a real thread safety issue on your hands. Shared caches are common problems. Fix it.
Analysis implementation¶
These builds do the following:
set up a build environment and run the analysis within it, then upload the resulting files
compile an optimized JS shell to later run the analysis
compile the browser with gcc, using a slightly modified version of the sixgill (http://svn.sixgill.org) gcc plugin
produce a set of
.xdb
files describing everything encountered during the compilationanalyze the
.xdb
files with scripts injs/src/devtools/rootAnalysis
Running the analysis¶
Pushing to try¶
The easiest way to run an analysis is to push to try with mach try fuzzy -q "'haz"
(or, if the hazards of interest are contained entirely within js/src
, use mach try fuzzy -q "'shell-haz"
for a much faster result). The expected turnaround time for linux64-haz is just under 1.5 hours (~20 minutes for hazard-linux64-shell-haz
).
The output will be uploaded and an output file hazards.txt.xz
will be placed into the âArtifactsâ info pane on treeherder.
Running locally¶
The rooting hazard analysis may be run using mach.
So you broke the analysis by adding a hazard. Now what?¶
Backout, fix the hazard, or (final resort) update the expected number of hazards in js/src/devtools/rootAnalysis/expect.browser.json
(but donât do that).
The most common way to fix a hazard is to change the variable to be a Rooted
type, as described in RootingAPI.h
For more complicated cases, ask on the Matrix channel (see spidermonkey.dev for contact info). If you donât get a response, ping sfink or jonco for rooting hazards, bholley or sfink for heap write hazards.