Forking Chrome to show HTML into SVG

November eleventh 2022

I have been engaged on a program referred to as html2svg, it converts net pages to SVG. It is based mostly on a fork of Chromium to assist fashionable net requirements. This publish explains most patches.

Take an image

Chromium is constructed on high of Blink: an HTML engine forked from WebKit, and Skia: a 2D engine additionally utilized in Firefox and Android.

Blink consumes the HTML enter, and Skia produces the graphical output. The Chromium compositor (cc) is in between, however we’ll ignore it for now.

With a view to assist a number of platforms and targets, Skia is constructed to render right into a back-end: it could possibly be its GPU renderer referred to as Ganesh, its software program rasterizer, or perhaps a PDF file. That is how Chromium can work with or with out a GPU and export net pages to PDF information with excessive constancy.

Skia additionally has an experimental SVG back-end, and that is what we’ll use to construct html2svg!

To get began, we’ll must discover a strategy to render the web page into an SVG canvas and expose it underneath a JS API.

The Chromium docs explains how one can export an .skp file utilizing the --enable-gpu-benchmarking flag.

An .skp file is a binary illustration of an SkPicture, a C++ class containing Skia drawing directions that may be replayed into any canvas by way of its playback() methodology.

Wanting on the code we are able to discover that it makes use of cc::Layer::GetPicture() to get an SkPicture:

// Recursively serializes the layer tree.
// Every layer within the tree is serialized right into a separate skp file
// within the given listing.
void Serialize(const cc::Layer* root_layer) {
    for (auto* layer : *root_layer->layer_tree_host()) {
        sk_sp<const SkPicture> image = layer->GetPicture();
        if (!image)
            proceed;

Declare a operate

We’ll add a brand new international JS API into Chromium to get began, let’s name it getPageContentsAsSVG().

The GPU extension we’re utilizing to generate .skp information registers itself in content material::RenderFrameImpl::DidClearWindowObject() which is sensible: it has entry to the rendering knowledge, and it is referred to as proper after window, the worldwide object, is created. Including the next on the finish of this methodology is sufficient to get our international operate registered.

// Get entry to the JS VM for this course of (every tab is a course of)
v8::Isolate* isolate = blink::MainThreadIsolate();
// Automated v8::Native destruction
v8::HandleScope handle_scope(isolate);
// Get the JS context for the present tab
v8::Native<v8::Context> context = GetWebFrame()->MainWorldScriptContext();
// Automated context entry/exit
v8::Context::Scope context_scope(context);
// Get the worldwide object (window)
v8::Native<v8::Object> international = context->World();

// Create a brand new JS operate binding
v8::Native<v8::FunctionTemplate> fn = v8::FunctionTemplate::New(
    isolate,
    [](const v8::FunctionCallbackInfo<v8::Worth>& args) {
        v8::Isolate* isolate = blink::MainThreadIsolate();

        args.GetReturnValue().Set(
            v8::String::NewFromUtf8(isolate, "think about that is svg").ToLocalChecked()
        );
    }
);

// Register the operate as getPageContentsAsSVG()
international->Set(
    context,
    v8::String::NewFromUtf8(isolate, "getPageContentsAsSVG").ToLocalChecked(),
    fn->GetFunction(context).ToLocalChecked()
).Examine();

Let’s run Chromium, open the debugger and check out.. and it really works! Now we have to do some precise work contained in the operate.

Render to SVG

// Get entry to the principle JS VM for this course of (every tab is a course of)
v8::Isolate* isolate = blink::MainThreadIsolate();
// Automated v8::Native destruction
v8::HandleScope handle_scope(isolate);
// Get the WebLocalFrame for the present v8 Context
auto* body = WebLocalFrame::FrameForCurrentContext();
// Get entry to the foundation rendering layer
auto* root = body->LocalRoot()->FrameWidget()->LayerTreeHost()->root_layer();

// Go over every sub-layer
for (auto* layer : *root->layer_tree_host()) {
    // Get vectorial knowledge for this layer
    auto image = layer->GetPicture();

    // Skip if we get there is no such thing as a knowledge
    if (!image) {
        proceed;
    }

    // Create a reminiscence stream to avoid wasting the SVG content material
    SkDynamicMemoryWStream stream;
    // Create an SVG canvas with the size of the layer
    auto canvas = SkSVGCanvas::Make(image->cullRect(), &stream);

    // Draw the layer knowledge into the SVG canvas
    canvas->drawPicture(image.get());

    // Allocate a buffer to carry the SVG knowledge
    auto dimension = stream.bytesWritten();
    auto* bytes = new char[size];

    // Copy from the stream to the buffer
    stream.copyTo(static_cast<void *>(bytes));

    // Return the information to the JS world
    args.GetReturnValue().Set(
        // Copy the UTF-8 buffer into an UTF-16 JS string
        v8::String::NewFromUtf8(isolate, bytes, v8::NewStringType::kNormal, dimension).ToLocalChecked()
    );

    // Launch the allotted knowledge
    delete[] bytes;

    // Do not course of another layers
    break;
}

This would possibly not work as a result of blink::WebFrameWidget::LayerTreeHost() is non-public. Let’s make content material::RenderFrameImpl a good friend class:

  // GPU benchmarking extension wants entry to the LayerTreeHost
  good friend class GpuBenchmarkingContext;

+ // Enable RenderFrameImpl to entry the LayerTreeHost for html2svg
+ good friend class content material::RenderFrameImpl;

Linking error now, we have to bundle SkSVGCanvas:

- # Take away unused util sources.
- sources -= [ "//third_party/skia/src/utils/SkParsePath.cpp" ]
+ # Add SVG dependencies for html2svg
+ deps += [ "//third_party/expat" ]
+ sources += [
+     "//third_party/skia/src/xml/SkDOM.cpp",
+     "//third_party/skia/src/svg/SkSVGCanvas.cpp",
+     "//third_party/skia/src/svg/SkSVGDevice.cpp",
+     "//third_party/skia/src/xml/SkXMLParser.cpp",
+     "//third_party/skia/src/xml/SkXMLWriter.cpp",
+ ]

getPageContentsAsSVG() does output one thing that resembles SVG, however there may be an error opening it: XML closing tags are lacking.

SkSVGCanvas closes tags when its destructor is named:

SkSVGDevice::~SkSVGDevice() {
    // Pop order is necessary.
    whereas (!fClipStack.empty()) {
        fClipStack.pop_back();
    }
}

Let’s wrap it in a scope:

// Create a reminiscence stream to avoid wasting the SVG content material
SkDynamicMemoryWStream stream;

{
    // Create an SVG canvas with the size of the layer
    auto canvas = SkSVGCanvas::Make(image->cullRect(), &stream);

    // Draw the layer knowledge into the SVG canvas
    canvas->drawPicture(image.get());
}

// Allocate a buffer to carry the SVG knowledge
auto dimension = stream.bytesWritten();
auto* bytes = new char[size];

Higher, however the textual content is rendered with a serif font and has bizarre kerning.

The serif font is brought on by lacking fonts, we are able to repair that by including a fallback to the font-family attribute of <textual content> components:

- if (!familyName.isEmpty()) {
-     this->addAttribute("font-family", familyName);
- }
+ familyName.appendf(
+     (familyName.isEmpty() ? "%s" : ", %s"),
+     "-apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Helvetica, Arial, sans-serif, 'Apple Colour Emoji', 'Segoe UI Emoji', 'Segoe UI Image'"
+ );
+
+ this->addAttribute("font-family", familyName);

The bizarre kerning seems as a result of the place of every character is ready, we are able to workaround this by solely setting the place of the primary character:

- fPosXStr.appendf("%.8g, ", place.fX);
- fPosYStr.appendf("%.8g, ", place.fY);
+ if (fPosXStr.isEmpty()) {
+     fPosXStr.appendf("%.8g", place.fX);
+ }
+
+ if (fPosYStr.isEmpty()) {
+     fPosYStr.appendf("%.8g", place.fY);
+ }

Little higher! Listed below are some outcomes:

Repair the T in Twitter

So the T letter is lacking from twitter.com, simply the T letter. Hmm okay bizarre, this is how Blink and Skia handles font knowledge:

1. Blink hundreds fonts from the system or remotely and maps them into the SkTypeface class
2. SkTypeface makes use of a font back-end internally: CoreText for macOS, DirectWrite for Home windows, and FreeType for Linux.
3. The font back-end parses uncooked font information and exports an array of supported UTF-32 code factors and their vectorial representations, the place within the array is named the glyph ID.
4. Blink passes the textual content to HarfBuzz which returns a set of glyph IDs and their relative positions, they’re handed to Skia which picks the vectorial knowledge and renders it.

Step 4 is required to assist ligatures and a few languages. Arabic characters for instance, must be rendered in a different way based mostly on their place in a phrase. This is the reason HarfBuzz is between Blink and Skia: it shapes a string of unicode characters right into a set of glyph IDs and their positions.

Take my title written in latin and arabic script, add an area between letters and the characters are rendered in a different way in arabic. The unicode characters don’t change, however their graphical illustration do.

A few of this logic is applied in HarfBuzz, and a few is applied in font information by way of tables:

• CMAP: the character map desk, map glyph ID and its platform encoding ID
• GSUB: the glyph substitution desk, map a number of glyph IDs to at least one yet another glyph IDs, that is the place ligatures are declared

A font implementing a ligature for fi may have a GSUB entry to exchange the glyphs for f and i with a particular glyph for fi, and this particular glyph will very probably map to a particular character within the CMAP desk.

And that is the issue, getGlyphToUnicodeMap() simply goes over the CMAP desk, the T from Twitter may be very probably applied as a substitution on the GSUB desk, which maps to a particular character, which will not map to a legitimate Unicode codepoint.

Our back-end is SVG, it’s imagined to deal with textual content shaping already, so we have to bypass HarfBuzz. Textual content shaping is dealt with by blink::Font::DrawText(), which ship knowledge to HarfBuzz after which calls blink::Font::DrawBlobs() with the glyph knowledge. We’ll use SkTextBlob::MakeFromString() to construct a textual content blob of nominal glyphs from a string. These will map 1:1 to a unicode codepoint, permitting the SVG viewer to deal with textual content shaping. This is what the patch seems to be like:

-  CachingWordShaper word_shaper(*this);
-  ShapeResultBuffer buffer;
-  word_shaper.FillResultBuffer(run_info, &buffer);
-  ShapeResultBloberizer::FillGlyphs bloberizer(
-      GetFontDescription(), device_scale_factor > 1.0f, run_info, buffer,
-      draw_type == Font::DrawType::kGlyphsOnly
-          ? ShapeResultBloberizer::Kind::kNormal
-          : ShapeResultBloberizer::Kind::kEmitText);
-  DrawBlobs(canvas, flags, bloberizer.Blobs(), level, node_id);
+  // Bypass HarfBuzz textual content shaping for html2svg
+  auto blob = SkTextBlob::MakeFromString(
+    StringView(run_info.run.ToStringView(), run_info.from, run_info.to - run_info.from).
+      ToString().
+      Utf8().
+      c_str(),
+    PrimaryFont()->
+      PlatformData().
+      CreateSkFont(false, &font_description_)
+  );
+
+  if (node_id != cc::kInvalidNodeId) {
+    canvas->drawTextBlob(blob, level.x(), level.y(), node_id, flags);
+  } else {
+    canvas->drawTextBlob(blob, level.x(), level.y(), flags);
+  }

Floor compositing

>
 
 
 
 
 

textual content = predominant.onCreateDevice()
textual content.drawText("chocolatine", 0, 0)
gradient = predominant.onCreateDevice()
gradient.drawRect(0, 0, 500, 150)
textual content.drawDevice(gradient, SkBlendMode::DstIn)
predominant.drawDevice(textual content, SkBlendMode::Over)

Testing mui.com I seen some textual content components with a gradient impact didn’t render. It is because SkSVGDevice doesn’t implement floor compositing: drawing right into a floor, after which rendering this floor utilizing a Porter-Duff compositing operation.

We have to implement SkBaseDevice::onCreateDevice() and SkBaseDevice::drawDevice() to assist this. On the GPU renderer it creates a texture, on the CPU renderer it allocates a buffer, and on SVG we’ll use <g>.

I will not go into an excessive amount of particulars on the implementation because it most likely deserves its personal publish, however mainly we use <g> components to create textures, <use> to show them, and <feComposite> to mix them.

Earlier than	After

Render the entire web page

Solely the primary ~6,000 pixels are rendered. We have to get the compositor to attract the entire web page, and blink::WebLocalFrame::capturePaintPreview() does precisely that! It appears to have been applied for the phising classifier, there was a Chromium weblog publish about it. Mixed with cc::PaintRecorder we are able to get it to render into our canvas.

cc::PaintRecorder recorder;
auto rect = SkRect::MakeWH(width, top);

body->CapturePaintPreview(
    gfx::Rect(0, 0, width, top),
    recorder.beginRecording(rect),
    false,
    false
);

auto canvas = SkSVGCanvas::Make(rect, &stream);

recorder.finishRecordingAsPicture()->Playback(canvas.get());

We now have one other downside, the macOS scrollbar additionally will get rendered into the SVG! Suprisingly, it is totally vectorized. We are able to work round this by including some CSS within the Electron code:

const fashion = doc.createElement('fashion')

fashion.innerHTML = `
    physique::-webkit-scrollbar,
    physique::-webkit-scrollbar-track,
    physique::-webkit-scrollbar-thumb {
        show: none;
    }
`

doc.head.appendChild(fashion)

Assist shadows

One factor lacking is shadows. Skia would not explicitly assist drawing them, however it gives the 2 predominant components: gaussian blur and clipping.

We have to add some code to deal with the maskFilter property of SkPaint, it accommodates the SkBlurMaskFilter used for bluring:

+    if (const SkMaskFilter* mf = paint.getMaskFilter()) {
+        SkMaskFilterBase::BlurRec maskBlur;
+
+        if (as_MFB(mf)->asABlur(&maskBlur) && maskBlur.fStyle == kNormal_SkBlurStyle) {
+            SkString maskfilterID = fResourceBucket->addColorFilter();
+
+            AutoElement filterElement("filter", fWriter);
+
+            filterElement.addAttribute("id", maskfilterID);
+
+            AutoElement floodElement("feGaussianBlur", fWriter);
+
+            floodElement.addAttribute("stdDeviation", maskBlur.fSigma);
+
+            assets.fMaskFilter.printf("url(#%s)", maskfilterID.c_str());

The clipping code needed to be refactored as a result of shadows prolong outdoors of ordinary clipping bounds, however we’ll skip it as it’s concerned.

Vectorize <canvas>

What if we might additionally vectorize 2D <canvas> components managed by JavaScript? Seems, Chromium has this functionality built-in for printing:

// For 2D Canvas, there are two methods of render Canvas for printing:
// show record or picture snapshot. Show record permits higher PDF printing
// and we favor this methodology.
// Listed below are the necessities for show record for use:
//    1. We will need to have had a full repaint of the Canvas after beforeprint
//       occasion has been fired. In any other case, we do not have a PaintRecord.
//    2. CSS property 'image-rendering' should not be 'pixelated'.

// show record rendering: we replay the final full PaintRecord, if Canvas
// has been redraw since beforeprint occurred.
if (IsPrinting() && IsRenderingContext2D() && canvas2d_bridge_) {

All we’d like is to make IsPrinting() return true:

  bool HTMLCanvasElement::IsPrinting() const {
-   return GetDocument().BeforePrintingOrPrinting();
+   return true;
  }

And there you go, SVG pacman based mostly on the MDN <canvas> demo!

Last ideas

That is all for now, html2svg is dwell on GitHub, test it out!