HarfBuzz Study: Subsetter

Behdad Esfahbod & Garret Rieger
9 April 2025

Introduction

Requirements

Other subsetters

Prior papers and presentations

Design

Read/write instead of decompile/modify/compile

Input specification

Subset Planning and Execution

Glyph closure

Variation space pruning, a.k.a. the instancer

Serialization

Repacking

Implementation

hb_subset_input_t

hb_subset_plan_t

Preprocessing

Table sanitization caching

cmap caching

CFF and CFF2 caching

Parsed charstrings

Compacted charstrings

seac presence

Benchmarks

Conclusions

Introduction

In this paper we present an overview of the harfbuzz-subset font subsetting library. Font subsetting is the act of creating a new font from an existing font, with functionality that is the same as the original font, but only for a reduced space of the supported space of the original font. This can be, most commonly, to reduce the set of Unicode codepoints (eg. languages) supported by the font. Or the supported OpenType features. Or a variable-font’s design-space. Or to remove hinting information. Etc.

The aim of this writeup is to better inform the klippa Rust-based font subsetting implementation, especially in the preprocessing stage of the HarfBuzz subsetter.

Requirements

There are four primary use-cases for a font subsetting library:

  1. File size reduction for web-font serving when the full range of the font’s functionality is not requested, for example, used extensively by Google Fonts

  2. File size reduction for operating system fonts and application size reduction, when the full range of the font’s functionality is not requested, for example, used extensively by Google Android and Flutter.

  3. Printing to PDF, for example in Skia, used by Chrome & Android.

  4. Limiting a font’s capabilities by type foundries, to sell cheaper versions of the font with limited functionality.

The four use-cases come with different requirements. All of which are supported by the HarfBuzz subsetter. Namely:

  1. Web-font serving requires support for shrinking input Unicode supported codepoints, OpenType features needed, as well as variation design space shrinkage. It also needs to be ideally extremely low-latency and high throughput, since in many scenarios it is done on-the-fly.

  2. The requirements are like web-font serving, but these are done offline ahead-of-time, so the latency & throughput are not as big of a concern.

  3. Printing requires specifying the desired glyph set to retain, instead of the Unicode set. It also requires fetching the mapping from the original font’s glyph indices to the subset font’s glyph indices for the retained glyphs.

  4. Limiting a font requires being able to subtract features from the subset input specification, instead of a purely additive model.

Other subsetters

Prior papers and presentations

Design

Read/write instead of decompile/modify/compile

HarfBuzz subsetter was roughly modeled after the FontTools subsetter, in that it performs the following operations in order:

In the FontTools subsetter, the font is first loaded into memory into Python objects, the subset operation applied to these Python objects, and then stored as binary via compilation.

Since HarfBuzz’s font table “parsing” is zero-copy, ie. objects are just read off the font-file mapped memory region directly as needed, instead of loading / parsing them into memory in custom data-structures, it made sense to extend that to the subsetter as well. So, the HarfBuzz subsetter simply loops over the input font, trims it to specification, and serializes the output font.

This sped up development significantly, because HarfBuzz as a shaper already had all the code to read font tables. So we just needed to implement a serialize method for each object to write it out, and a subset method to filter the input and redirect it to the output. This, obviously, saved time and memory on the decompile side of things, being zero-copy.

Input specification

A subset operation is primarily described by listing the various parts / aspects of the font that should be retained. These are broken down into three main types:

  1. Sets of integers:

    1. Glyph indices,

    2. Unicode codepoints,

    3. Name IDs,

    4. Table tags to drop,

    5. Layout feature and script tags.

  2. A bitmap of option flags that modify the subsetter behaviour. For example you can configure hints to be retained or dropped. Of particular importance are (see the linked option list for a complete listing):

    1. Retain glyph ids: the generated subsetted will not change any glyph indices used in the input font. It will just place empty glyphs for the glyphs not retained by the subset operation.

    2. Drop hints: drops all data from the font related to hinting.

  3. A variable-font’s design-space can be reduced by giving a list of ranges to retain, and optionally new default values, for each font design axis.

Combinations of the input sets are supported, for example you can give both a list of glyphs and Unicode codepoints that should be retained and the produced subset will be a union of the two.

Subset Planning and Execution

The subsetting operation is broken up into two main phases: subset planning and execution. In the public API these two steps can be run separately if desired. Some users of the subsetting library do not need the final subset bytes and instead just need the information (e.g. glyph closure) generated during the planning phase.

Subset planning takes the subset input specification and then determines what specific parts of the input font will need to be retained in the generated subset in order to retain functional equivalence. It does things like:

The subset plan is used to generate information needed to perform the subset that spans multiple font tables. Where information is only needed within a single table that will typically be calculated during the subset execution phase.

The subset execution phase takes the input font and subset plan and generates the final subset font bytes working table by table. Subsetting execution per table aims to be O(N) where N is the number of glyphs in the output subset. To achieve this we aim, where possible, to avoid parsing/loading parts of input font tables that will not be retained in the output subset.

Glyph closure

Given the input font and the subset specification we need to determine the complete set of glyphs which may be reachable by any content which is a subset of the subset specification. For the generated subset to maintain functional equivalence it must retain all of these glyphs. Generating this set of glyphs is called glyph closure.

At high level the closure process works by iteratively checking the current set of reachable glyphs against all of the various rules throughout the font that may substitute in glyphs (eg. GSUB table) and determining if additional glyphs are reachable. This repeats until the set of reachable glyphs becomes stable.

Variation space pruning, a.k.a. the instancer

In addition to removing unused data from the input font, the harfbuzz subsetter also supports modifying the variation design space supported by the font. This operation is called (rather clumsily) instancing. The input can specify a new design space (one range per axis) and the font’s variable data will be pruned to only those ranges. Instancing operations can be grouped into five levels of increasing complexity:

Harfbuzz currently supports all levels. Instancing occurs during the subset execution phase alongside the non-instancing subsetting operations.

Serialization

The final operation in subset execution is to serialize the actual bytes. To facilitate this we use a common serialization helper, hb_serialize_context_t, to construct the tables. There’s two main types of serialization:

  1. Simple records: these tables are mostly just an array of records. Serialization is just a matter of appending the retained records together.

  2. Object graphs: these tables contain graphs of objects with offsets forming the edges. The serializer maintains a stack of objects in order to produce a topological ordering and automatically deduplicates and shares objects when possible. A more detailed overview of how it works can be found here.

Repacking

A common pattern in the encoding of font tables with complex data structures is to have fixed size offsets (16, 24, or 32bit) from objects to other objects. As a result, in many cases it is possible to end up needing offsets that overflow the available fixed offset sizes (common in practice with the 16-bit offsets). Repacking is the process of re-arranging and modifying the serialization to eliminate offset overflows. For a more detailed look at the problem see:

In HarfBuzz, when serializing the output tables, we default to using a simple and fast topological sort that does not attempt to prevent overflows. If after serialization is finished there was one or more overflowing offsets, then the repacker is invoked to fix them. Since the vast majority of subsetting operations will not result in overflows, this makes the typical case fast and ensures we only pay for offset overflow resolution when it is actually needed.

The repacker operates on a generic object graph and as such is not specific to one table (or to fonts at all, for that matter). It can resolve overflows in any of the table structures that utilize offsets. It does however utilize some table specific techniques for the GSUB and GPOS tables (namely table splitting and extension-lookup promotion).

Since the HarfBuzz repacker is a sophisticated piece of engineering that was undesirable to reimplement in FontTools, the FontTools subsetter can optionally call into HarfBuzz just for offset overflow resolution using the HB repacker.

Implementation

hb_subset_input_t

The core of subset input is a list of integer sets specifying the various parts / aspects of the input font which should be retained. These are stored in the set_t structure which is indexed by the hb_subset_sets_t enum. This allows new sets to be added to the public API by only adding a new value to the enum.

The other main configuration mechanism on hb_subset_input_t is the option flags which is a flag bitmap using the hb_subset_flags_t enum. Again, this allows us to introduce new flags to the public API by only adding a new enum value.

Lastly, the hb_subset_input_t stores a list of variable-font design-space ranges that inform the instancing portion of the subset operation.

hb_subset_plan_t

The subset plan is a pretty large structure that stores a wide variety of information used to drive the subset execution. A mostly complete list of the information stored in the structure can be seen in hb-subset-plan-member-list.hh.

In general where there is information needed during subsetting that is used by more than one font table, we compute it during planning and store it in the subset plan.

Preprocessing

Subsetting large fonts (for example, NotoSansCJK-VF.otf.ttc [~32MB] / NotoSansCJK-VF.ttf.ttc [~38MB]) is an expensive procedure. A lot of this time is spent in loading the cmap table of these fonts into an hb_set_t and hb_map_t.

If a client (like the Google Fonts server for example) can hold onto an hb_face_t and subset it multiple times to different specifications (hb_subset_input_t’s), then it is in theory possible to throw more memory at the problem and cache some data to the hb_face_t to speed up subsetting. That is exactly what the hb_subset_preprocess_face() API does. It returns a new hb_face_t that is functionally equivalent to the input hb_face_t, but provides much faster hb_subset() function calls.

In the rest of this section we enumerate what is being cached and why. The cached data is specified in hb-subset-accelerator.hh. The way the accelerator-building is done is to perform a special subset operation with the keep-everything flag on, which instructs the subsetter to keep all data, while also turning on a flag to build the accelerator as it goes about the subsetting. The resulting subset face is the output of hb_subset_preprocess_face() and has the accelerator attached to it.

Table sanitization caching

The zero-copy table loading model in HarfBuzz involves an initial round of sanitization to make sure the table offsets do not point out of range. This sanitization mechanism is fairly fast, but still, it is work that need not be done again and again. So, the subset accelerator has a hb_map_t cache of hb_tag_t table tags to sanitized hb_blob_t table data. The cache is guarded by a mutex and lazily populated for each table upon first request.

cmap caching

One of the goals of the subsetter is that the runtime should ideally be bounded by the size of the output file, not input. This, however, is not possible because the cmap table almost always needs to be fully loaded before any subsetting can be done. As such, caching the cmap table provides one of the biggest speed-ups, especially for simple, non-variable, TrueType-flavored fonts. This cache applies to all supported font formats since they all map Unicode codepoints to glyphs using the cmap table.

List of cached items:

These mostly speed-up the two hotspots _populate_gids_to_retain and _populate_unicodes_to_retain.

CFF and CFF2 caching

CFF and CFF2 subset accelerators are heavily used for subsetting and are expensive to construct (require parsing various parts of the CFF/CFF2 table) so we store these to avoid needing to reconstruct them.

A subset accelerator for CFF(2) subclasses from the common CFF(2) accelerator, which has parsed encoding, charset, nameIndex, topDictIndex, subroutines & charstring indices etc.

The subset accelerator adds to the common accelerator new members that are useful to speed up subsetting. That is stored in the cff_subset_accelerator_t structure, and includes: parsed and compacted charstrings and subroutines. Let me expand on those two terms:

Parsed charstrings

CFF subroutines and glyph data are encoded in the CharString format, which encodes the computer program including operands and operators of a stack machine. The operands (numbers) are encoded first, using a variable-width encoding, and they are pushed onto the runtime stack by the interpreter, followed by an operator (one or two bytes) which consumes operands from the stack. The common CFF accelerator does not parse the charstrings whatsoever. It executes them as needed for shaping or glyph extents extraction or some subsetting operations and whatnot.

When subsetting we need to parse the charstrings into a vector of operations, with links back to their encoded strings. This parsed list is needed for operations like dropping hinting, subsetting without desubroutinizing, and other operations.

While normal subsetting only parses the used glyph charstrings and subroutines, by pre-parsing all of them and storing them in the accelerator, we can speed up subsetting considerably.

Compacted charstrings

Since each operation (operator/operand) in a charstring is encoded with just one or two bytes typically, this parsed vector has significant memory overhead since it will be at least 16 bytes on a 64-bit environment because of the pointer back to the bytes and the bytes length plus misc bits and alignment. See parsed_cs_op_t.

This can blow up the memory required for the accelerator significantly, for large CJK fonts. Like, >300MB of memory. We found a great optimization opportunity, which is to compact the parsed charstrings in the following way.

Since the need for the parsed charstrings is to handle operations like subroutine closure and removing hinting data, we only care about those operators/operands that contribute to those (subroutine calls, hinting). The bulk of the charstring is actual drawing commands, and we simply want to copy them out verbatim (except for when instancing CFF2 fonts), so we do not actually need a separate item in the parsed vector for each operator/operand that is not relevant to subroutine calls or hinting. As such, we can merge neighboring items to significantly reduce the number of items in the parsed vector, and then shrink the vector allocation as well. This not only reduces the memory usage significantly, it has a meaningful speed-up as well since we have to deal with a smaller workload. This is done in parsed_cs_str_t::compact().

seac presence

The seac operator is one of those niche and quasi-deprecated operators that can appear at the very end of the charstring, and if present, requires extra processing. Since the operator appears at the end of the charstring, even in cases where the charstring need not be parsed, we had to do a whole pass over it just to see if it ends with a seac. Since seac is rare in modern fonts, we simply cache whether the font uses any seac at all. This saves a pass over the charstrings in certain cases.

Benchmarks

A performance comparison of subsetting with and without preprocessing can be found here. The results show that preprocessing results in substantial speedups over a wide variety of font technologies and subsetting operations. Notably we see significant speedups (90%+ reductions in times) for large CJK fonts being subsetted to small sets of codepoints. This speedup is primarily due to eliminating full parsing of the cmap table.

Conclusions

In this document we gave an overview of the HarfBuzz subsetter, its use-cases, high-level design, and performance optimizations that make it the industry-standard high-performance subsetter that it is. There is a Google-led effort to port the Open Source font production & consumption stacks (written in Python and C/C++ respectively) to the safe Rust programming language, and we hope this document can speed up the development of the HarfBuzz subsetter successor in Rust, namely Klippa.