Web browser

The architecture of the web browser in the LUWRAIN platform is designed to transform standard HTML/CSS web pages into an accessible, linear, and spatially aware format suitable for blind users. It relies on a hybrid approach: it uses the JavaFX WebEngine for fetching, parsing, and rendering the DOM, while utilizing JavaScript injections to extract geometric bounding boxes and monitor dynamic changes.

Here is a detailed breakdown of the core architectural components, class roles, and data flows.

Core Architectural Concepts

The primary goal of this architecture is to convert a hierarchical DOM tree into a list of spatially calculated text blocks. This is achieved through three main phases:

1. DOM Traversal and Block Collection: Walking through the HTML nodes and grouping text into logical blocks based on HTML tags.
2. Geometry Extraction: Using JavaScript to query the actual screen coordinates (bounding rectangles) of each DOM node.
3. Text Formatting and Spatial Alignment: Calculating text layout based on available width and adjusting the vertical positioning of blocks to prevent overlaps.

Block Collection Pipeline

The process of parsing the DOM is handled by the BlocksCollector and its concrete implementation WebKitBlocksCollector. This is an abstract generic class that provides a stack-based algorithm for traversing a tree structure. It separates nodes into markup nodes (which create structural boundaries) and text nodes (which contain actual content).

public void process(N node)
{
    try {
        if (isTextNode(node))
	{
            if (!blocksStack.isEmpty())
                addTextToBlock(node, blocksStack.getLast());
            return;
        }
        final var children = getChildNodes(node);
        if (isMarkupNode(node))
	{
            markupStack.addLast(node);
            for(final var c: children) process(c);
            markupStack.pollLast();
            return;
        }
        blocksStack.addLast(createBlock(node));
        for(final var c: children) process(c);
        final B block = blocksStack.pollLast();
        if (saveBlock(block)) this.blocks.add(block);
    } catch(Throwable e) {
        log(e.getMessage());
    }
}

The WebKitBlocksCollector class extends BlocksCollector specifically for the JavaFX WebEngine. It interacts with the W3C DOM API (e.g., HTMLDocument, Node). It defines the rules for which HTML elements are considered markup (e.g., HTMLAnchorElementImpl, HTMLBRElementImpl) and which are ignored (e.g., HTMLScriptElementImpl).

Geometry Extraction via JavaScript Injection

Since JavaFX does not provide a direct API to get the rendered screen coordinates of DOM nodes easily, LUWRAIN injects a custom JavaScript file (injection.js) into the page. The injected script recursively walks the DOM and uses getBoundingClientRect() to extract the exact visual position of elements. It also calculates a hash of the content to detect dynamic changes.

this.scanDOM = function () {
    var nodeList = this.nodewalk(document, 0);
    var result = [];
    for (var i = 0; i < nodeList.length; i++) {
        var nodeData = {
            node: nodeList[i],
            rect: (nodeList[i].getBoundingClientRect ? 
                   nodeList[i].getBoundingClientRect() : null),
            hash: this.getNodeHash(nodeList[i]),
            text: this.getNodeContent(nodeList[i])
        };
        result.push(nodeData);
    };
    return result;
};

On the Java side, WebKitGeom executes this script via engine.executeScript(injection). It parses the returned JSObject into a map of Node objects to GeomEntry objects. A GeomEntry is a simple data container holding the x, y, width, and height of a node.

Text Layout and Line Building

Once blocks are collected and their geometries are known, the text must be formatted to fit requested app layout.

The WebKitBlockBase class represents a collected block of text. It contains a list of Run objects (continuous text strings). The method buildLines() calculates how to split these runs into a Fragment and group them into a Line based on the available width (right - left). It intelligently looks for spaces to perform word wrapping via the findNextBreak() method.

int findNextBreak(int[] breaks, int continueFrom, int wholeLen, int availableSpace) {
    int left;
    for(left = 0; left + 1 < breaks.length && breaks[left + 1] <= continueFrom; left++);
    int right = breaks.length - 1;
    while(right > left && breaks[right] - continueFrom > availableSpace)
        right--;
    return right > left ? breaks[right] : -1;
}

Spatial Geometry Calculation

Because web layouts can be complex (floating elements, absolute positioning), blocks might overlap visually. BlockGeom builds a two-dimensional dependency matrix to prevent this in the linearized accessible view. It checks if blocks intersect horizontally and adjusts their top coordinates to ensure they flow sequentially without overlapping.

public void process()
{
    Arrays.sort(blocks);
    buildMatrix();
    for(int i = 0; i < count; i++)
        for(int j = i + 1; j < count; j++) {
            final Block b1 = blocks[i], b2 = blocks[j];
            final int space = matrix[i][j];
            if (space < 0) continue;
            b2.top = Math.max(b2.top, b1.top + b1.height + space - 1);
        }
}

Orchestration and Mutation Observation

The WebKitBlocks class is the main orchestrator class. It initializes the WebKitBlocksCollector, triggers the scaling of blocks, commands the building of lines, and applies the BlockGeom processing.

Furthermore, it handles dynamic web pages (like SPAs). When the JavaFX Worker.State reaches SUCCEEDED, it binds the Java instance of WebKitBlocks to the JavaScript window object. This allows the JavaScript polling mechanism (defined in injection.js via setTimeout) to notify the Java backend when the DOM changes (e.g., when a node's hash or bounding rect changes), setting the needsToBeUpdated flag.