Linkers & Bootstrap

Running the lab module with -logLevel DEBUG shows the order in which 2.13.1 invokes the linkers for one permutation. The extra Precache manifest line is the custom linker of Custom linkers:

The first line is a pre-linker, the second is the primary linker, and the rest are post-linkers. The same six lines are repeated for every permutation and once more for the final link after all permutations are compiled. The Developer Guide says post-linkers run "in reverse lexical order of their name". The 2.13.1 source and this log say otherwise: StandardLinkerContext collects the post-linkers in the order in which their add-linker elements were processed and then reverses that list ("Get all the post-linkers IN REVERSE ORDER"), and the Javadoc of Linker says the exact order is determined by the order of the add-linker tags. A linker added last (here precacheManifest, added by the module that inherits it) therefore runs first among the post-linkers. That matters when one post-linker reads the output of another: write the dependency into the order of your inherits and add-linker lines, and look at the DEBUG log to confirm it.

Source: GWT 2.13.1 source — StandardLinkerContext.java, Javadoc — Linker

Compiled with the default settings, the Bookshelf (two locales and two user agents, so six permutations, and one split point for the admin area) writes this module directory:

The mapping file is the key to the sequence. It lists, for each permutation file, the property values that select it, and the same table is compiled into the nocache.js as calls such as k([Zb,Ub],$b) in the obfuscated script.

Source: GWT — FAQ Debugging and Compiling: The .nocache.js File

The sibling page on the compiler shows the permutation matrix and how -style changes it. This page starts after that: it is about what the browser does with the files.

The selection script calls window.__gwtStatsEvent(event) at every step, if the page defines that function before the script runs. An event has moduleName, sessionId, subSystem, evtGroup, millis and type; __gwtStatsSessionId identifies the page load. This is the supported hook for startup timing, and it is how the lab recorded the real sequence of the Bookshelf:

Source: GWT — Lightweight Metrics

The events that arrived, in order, for the default (obfuscated) Bookshelf build:

moduleStartup:moduleRequested is the moment installScript starts the download and moduleEvalStart/moduleEvalEnd surround the evaluation of the compiled code in the iframe. A build that installs the script with a src attribute (see the next section) adds one more event, moduleStartup:scriptTagAdded, when the script element has been added to the iframe. The numbers you would report in a startup dashboard are the differences between these events; Compile reports and optimization is the place for the code-size side of the same story.

A host page is any HTML that has a script element pointing at the nocache.js. The selection script waits for document.body before it creates the iframe, so the tag may sit in the head. Three details of the template are worth knowing:

Here is the DOM of the Bookshelf (the direct_install build) after the module started, from a headless Chrome dump (trimmed), showing the extra elements the bootstrap and the module added:

Source: GWT 2.13.1 source — installLocationIframe.js

The link element is the stylesheet from <stylesheet src> of the Clean theme, added by step 6; the style element is the injected CSS of the application’s CssResource. Both matter for the Content Security Policy section.

Do the error hooks work? The lab removed the permutation files of the badge module and loaded a host page that registers gwt:onLoadErrorFn:

The result depends on the install path:

The cause is in installScriptEarlyDownload.js of 2.13.1: the onerror handler of the download script builds its message with a variable code that exists only in another function, so the handler throws instead of calling __errFn. The installScriptDirect.js variant defines the handler inside the function that has code, and it works. Until that is fixed, do not rely on gwt:onLoadErrorFn to report a failed download in a default build; watch for a missing moduleStartup:moduleEvalEnd event in __gwtStatsEvent (or a timeout) instead. The cause was found in the source and confirmed by the run on 2.13.1; it was not checked against a newer release.

Source: GWT 2.13.1 source — installScriptEarlyDownload.js, GWT 2.13.1 source — installScriptDirect.js

The default linker also emits the hook that makes Super Dev Mode work with the same host page. When the page protocol is http: or file: and sessionStorage has an entry named __gwtDevModeHook:<module>, the script validates the stored URL against a whitelist and loads it instead of the compiled code. The default whitelist, from CrossSiteIframeLinker, is http://(localhost|127\.0\.0\.1)(:\d+)?/.*; anything else is ignored and logged as Ignoring non-whitelisted Dev Mode URL: .... Three configuration properties control the hook: devModeRedirectEnabled (default true), devModeUrlWhitelistRegexp and the protocol rule, which a custom linker can change by overriding getJsDevModeRedirectHookPermitted. The Javadoc of the protocol hook recommends refusing the redirect on https and on host names that end users visit; see Development mode and debugging for the Super Dev Mode workflow and its security notes. The query parameter ?gwt.codesvr= belongs to the classic Dev Mode and makes the script load <module>.devmode.js instead of a permutation; the classic plugin-based mode is legacy.

Source: GWT 2.13.1 source — CrossSiteIframeLinker.java, GWT 2.13.1 source — DevModeRedirectHook.js

xsiframe (class CrossSiteIframeLinker, description "Cross-Site-Iframe") "uses an iframe to hold the code and a script tag to download the code". Because the code is fetched with a script element and never with XMLHttpRequest, the module directory can be on another origin (see Cross-site loading), and because the compiled code lives in its own iframe the module’s globals do not collide with the page’s or with another module’s, and code-split fragments can be installed into the same frame later.

How the code reaches the iframe is a choice of the property installCode, whose default is "is the output obfuscated":

The first characters of a Bookshelf-sized permutation file show the difference:

The comment in installScriptEarlyDownload.js describes the mechanism: the download starts immediately, and the contents are installed into the iframe once the document body is ready. The consequence that matters today is the second half of the table: inline script text is not allowed by a strict Content Security Policy. That is why direct_install exists.

Two further properties of xsiframe are worth knowing. First, it does not support <script src="..."/> elements in a *.gwt.xml file, because it has no place to put them in the host page: the compile fails unless you set xsiframe.failIfScriptTag to FALSE (see Pitfalls). Second, it uses the ScriptTagLoadingStrategy for code splitting, because CrossSiteIframeLinker.gwt.xml switches the AsyncFragmentLoader.LoadingStrategy with <when-linker-added name="xsiframe"/>; the default strategy of every other linker, XhrLoadingStrategy, downloads fragments with XMLHttpRequest. The <when-linker-added> condition was added in 2.9.0, according to the release notes.

Source: GWT 2.13.1 source — CrossSiteIframeLinker.java, GWT 2.13.1 source — CrossSiteIframeLinker.gwt.xml, GWT 2.13.1 source — AsyncFragmentLoader.gwt.xml

direct_install (class DirectInstallLinker) "adds a script tag directly to the iframe rather than downloading the JavaScript code as a list of strings"; its Javadoc says it is "exactly like CrossSiteIframeLinker except that it defaults to `installCode=false`". You turn it on with one line in the module:

Because the primary linker is replaced, the other linkers (the policy manifest, symbol maps and so on) still run. The 2.5.1 release notes introduced the linker with the remark that applications built with it "should work in a page where inline scripts are forbidden", which was written for Chrome extensions and is exactly the Content Security Policy case. The lab confirms it for 2.13.1, with one exception (code splitting) that is the subject of Content Security Policy.

The price of the src install is visible in the cross-site section: the script element carries crossOrigin='' when the page has a host name, and so a module directory on another origin needs an Access-Control-Allow-Origin header.

Source: GWT 2.13.1 source — DirectInstallLinker.java, GWT — Linkers: Available linkers

sso (class SingleScriptLinker, description "Single Script") writes one JavaScript file that contains both the bootstrap and the compiled code: no iframe and no separate permutation file. That makes it the right choice for exactly one situation: a module that has one permutation and no split points, for example a small widget script that you want to ship as a single file to other sites. Both conditions are checked at link time, and the lab hit both errors:

Both compiles exit with status 1. Fixing the first error means collapsing every property that has more than one value, in the module XML:

The result is one file, badge.nocache.js, of 12,259 bytes (the whole program is in it), and nothing else in the module directory. Fixing user.agent to one value is a trade: the code is compiled for that one value, so test the result in every browser you support. Use sso for a module whose audience you control; for an application with locales, use xsiframe or direct_install.

Source: GWT 2.13.1 source — SingleScriptLinker.java

std (IFrameLinker, "IFrame") and xs (XSLinker, "Cross-Site") are what the book’s loader chapter describes. The release notes of 2.7.0 (RC1) deprecated both: "IFrameLinker and XSLinker are deprecated because they don’t work in Super Dev Mode". Both still exist in 2.13.1, and both print a deprecation warning, three times for one compile:

The warning is controlled by the configuration property iframe.linker.suppressDeprecationWarning (default false), but suppressing it is not a fix. What they write is also different from the default:

Neither linker works with a strict Content Security Policy (see Content Security Policy), and std has an extra problem that the lab reproduced: the std bootstrap looks the iframe up with getElementById(<module name>), so a host page that has an element with the module name as its id fails before the application starts:

The same host page, with the same <div id="badge">, starts normally under xsiframe and xs, which only create the frame and never look it up. The book’s gwt.js loader, module.nocache.html, .cache.html files and nocache-xs.js belong to this family or to an even older one; none of them is written by a 2.13.1 build with the default linker (see Where older material is outdated).

Source: GWT 2.13.1 source — IFrameLinker.java, GWT 2.13.1 source — XSLinker.java, GWT — Release notes

The Bookshelf wants an offline shell, and a service worker needs the list of files to precache. The module directory contains hashed permutation files, so the list changes with every build and must be generated. This linker is a post-linker that waits for the final link, collects every public file, and writes <module>.precache.json:

The linker class is not client code. It must not be inside a <source path> of any module (the compiler would try to translate it), it needs gwt-dev on its compile classpath, and it must be on the classpath of the compiler when the module is built. In a Maven project the simplest layout is a separate *-linker module (or a library jar) that the gwt-app module depends on; the lab compiled the class with javac --release 17 against gwt-dev and put the output directory on the compiler classpath. The registration lives in a small module that applications inherit:

The application module then only needs <inherits name="com.example.bookshelf.linkers.PrecacheManifest"/>. Compiled with the Badge module, the module directory gained badge.precache.json:

Source: GWT — Linkers: Custom linkers, Javadoc — ArtifactSet

What the run shows, and what a real implementation must handle:

Besides the primary linkers, GWT ships a few more that applications switch on by inheriting a module. The Developer Guide separates them into "internal" linkers that applications do not name, and the pre-compression linker that they do:

The precompress linker was run on the lab module. The module directory afterwards:

A web server that supports precompressed files (Server integration and deployment has the caching and compression rules) can serve the .gz files with Content-Encoding: gzip and skip compressing at request time.

Source: GWT 2.13.1 source — Precompress.gwt.xml, GWT — Linkers: Internal linkers

CrossSiteIframeLinker reads configuration properties that a module can set, which is lighter than a subclass. Each one was checked in the 2.13.1 source:

A subclass can override more: getJsInstallLocation (where the iframe goes), getJsModuleFunctionErrorCatch (wrap the bootstrap in a try/catch and run your code on failure), shouldUseSelfForWindowAndDocument (the Javadoc says it is for worker threads), and shouldIncludeBootstrapInPrimaryFragment.

Source: GWT 2.13.1 source — CrossSiteIframeLinker.java, Javadoc — CrossSiteIframeLinker

A module with GWT.runAsync split points has a third script source, and it is an inline one for every xsiframe-family build, including direct_install. A split fragment is wrapped by the linker as $wnd.<module>.runAsyncCallbackN("...the code as a string...") (CrossSiteIframeLinker.wrapDeferredFragment), and __installRunAsyncCode in runAsync.js installs the string as the text of a script element in the iframe. The lab used a module with one split point, SplitEntryPoint:

Source: MDN — Content Security Policy

So for an application that uses code splitting (the Bookshelf’s admin area does), script-src 'self' alone is not a complete policy even with direct_install: use a nonce with strict-dynamic, or accept unsafe-inline for scripts. Application code that injects scripts has the same constraint. ScriptInjector.fromString is inline text too, and the SafeHtml page measured that it copies the nonce of the first script element that has one; Code splitting covers the fragments themselves.

The measurements give a policy the Bookshelf can use with its split admin area, its XSRF-protected RPC and the default linker, where the server writes the nonce into the host page as in the dynamic host page example:

Run on the real Bookshelf module (the XSRF call failed because no server was running), the direct_install build put its shell into the page under script-src 'self' and under the nonce policy, and the obfuscated default build did so under the nonce policy and not under script-src 'self'. The admin area’s fragment was not opened in that run; the split-point result above is from the single-split-point lab module.

The sibling GWT-specific CSP facts, already measured on the SafeHtml page, are not repeated here: the gwt.cspCompatModeEnabled binding property (default false) that makes widgets avoid javascript: targets, StyleInjector having no nonce support, and the 2.12.0 release note "Fix CSP issues in linkers and dev mode".

Source: MDN — Content Security Policy, W3C — Content Security Policy Level 3, GWT 2.13.1 source — runAsync.js