Skip to content

GitLab

Last edited by Vincent Wei
Page history
This is an old version of this page. You can view the most recent version or browse the history.

Showing Text in Complex or Mixed Scripts

How to lay out, shape, and render text in complex or mixed scripts.

Table of Contents

Overview

As described in Using Enhanced Font Interfaces, in order to support complex or mixed scripts, we tuned and enhanced MiniGUI's font interfaces in version 4.0.0.

But this is only a small part of the whole story. To show a text paragraph in complex/mixed scripts, we must implement the complete APIs which conforming to Unicode standards and specifications, including the Unicode Bidirectional Algorithm (UAX#9), Unicode Line Breaking Algorithm (UAX#14), Unicode Normalization Forms (UAX#15), Unicode Script Property (UAX#24), Unicode Text Segmentation (UAX#29), Unicode Vertical Text Layout (UAX#50), and so on.

On the basis of full support for Unicode, we provides APIs for laying out, shaping, and rendering the text in complex or mixed scripts. This document describes how to use these APIs.

Before we continue, let's clarify a few terms and concepts first.

A script means a set of letters of one human language, for example, Latin, Han, Arabic, Hangul, and so on.

The writing system means the specific writing convention of a language/script. For example, we write letters in Latin from left to right horizontally then from top to bottom vertically, while we write traditional Chinese words from top to bottom vertically then from right to left horizontally.

The common languages, such as English and modern Chinese, are standard scripts. In these scripts, there is a one-to-one relationship between an encoded character (e.g. 0x4E2D in UTF-8) and the glyph (中) that represents it. And we write the characters in standard scripts always from left to right horizontally then from top to bottom vertically.

A glyph is a set of data which represents a specific character in a visual or printable form. In computer, a glyph may be a bitmap or a vector path data.

Generally, one font contains a lot of glyphs in bitmap or vector path data for the characters in a specific language or script or a few similar languages or scripts. For example, today, most fonts for East Asia markets contain almost all glyphs for Chinese (both traditional and simplified), Japanese, and Korea characters.

Systems and applications that handle the standard scripts do not need to make a distinction between character processing and glyph processing. In working with text in a standard script, it is most often convenient to think only in terms of character processing, or simply text processing: that is, the sequential rendering of glyphs representing character codes as input in logical order.

The term complex script refers to any writing system that requires some degree of character reordering and/or glyph processing to display, print or edit. In other words, scripts for which Unicode logical order and nominal glyph rendering of codepoints do not result in acceptable text. Such scripts, examples of which are Arabic and the numerous Indic scripts descended from the Brahmi writing system, are generally identifiable by their morphographic characteristics: the changing of the shape or position of glyphs as determined by their relationship to each other. It should be noted that such processing is not optional, but is essential to correctly rendering text in these scripts. Additional glyph processing to render appropriately sophisticated typography may be desirable beyond the minimum required to make the text readable.

The situation becomes more complicated when we process a text in mixed scripts. That is, the text contains characters from different scripts, for example, Latin, Arabic, and Chinese.

General Process

Text in Standard Scripts

If you can determine the text to process is in standard scripts, you can still to use MiniGUI GDI functions (TextOut, DrawText and so on) to show the text. For more information, please refer to:

http://www.minigui.com/doc-api-ref-minigui-sa-4.0.0/html/group__text__output__fns.html

Text in Complex Scripts

Before dealing with the bidirectional algorithm and the glyph shaping, we first need to divide the text into paragraphs and get the relevant properties of the characters, such as whether there is a breaking opportunity before or after each character.

Moreover, all of the above processing needs to be done under the Unicode character set.

Therefore, you call GetUCharsUntilParagraphBoundary function to convert a multi-byte string to a Unicode string under the specified white space rule. For example, you can convert a general C string in UTF-8 or GB18030 charset to a Uchar32 string by calling this function.

    LOGFONT* lf;
    int left_len_text;

    char* text = "中华人民共和国万岁！\n中国共产党万岁！";

    if (!(lf = CreateLogFontForMChar2UChar("utf-8"))) {
        _ERR_PRINTF("%s: failed to create logfont for utf-8 charset\n",
                __FUNCTION__);
        exit(1);
    }

    left_len_text = strlen(text);
    while (left_len_text > 0) {
        Uchar32* ucs = NULL;
        int consumed, n;

        consumed = GetUCharsUntilParagraphBoundary(lf, text, left_len_text,
                WSR_NOWRAP, &ucs, &n);
        if (consumed > 0) {

            // got a paragraph and go on.
            ...

            // when we are done.
            free (ucs);
            ucs = NULL;
        }

        text += consumed;
        left_len_text -= consumed;
    }

    DestroyLogFont(lf);

In the above code, we call CreateLogFontForMChar2UChar function to create a dummy logfont object only for converting C-string in UTF-8 to Uchar32 string. The logfont object created by CreateLogFontForMChar2UChar will expense a minimal memory. But note that if you use the logfont object returned by this function to show a text, you will see nothing.

Obviously, the original text has two paragraphs. So we use a while loop to deal with a paragraph in one loop.

Next, you should call UStrGetBreaks to get the breaking opportunities of each characters in the paragraph. This information are very important to the layout algorithm.

If the text is in a standard script, like Latin or Chinese, you can call GetGlyphsExtentFromUChars function to lay out the paragraph to a line. This function returns a glyph string which can fit in a line with the specified maximal extent and rendering flags. After this, you call DrawGlyphStringEx function to draw the glyph string to the specific position of a DC.

If there are characters left to lay out, you can call this function again, until there is no any character left.

For a complete sample, please refer to:

https://github.com/VincentWei/mg-tests/blob/master/4.0/drawglyphstringex.c

If the text is in complex and/or mixed scripts, like Arabic, Thai, and Indic, you should create a TEXTRUNS object first by calling CreateTextRuns function, then initialize the shaping engine for laying out the text.

For the usage sample of TEXTRUNS object, please refer to:

https://github.com/VincentWei/mg-tests/blob/master/4.0/createtextruns.c

MiniGUI provides two types of shaping engine. One is the basic shaping engine. The corresponding function is InitBasicShapingEngine. The other is called complex shaping engine, which is based on HarfBuzz. The corresponding function is InitComplexShapingEngine. The latter one can give you a better shaping result.

For the sample of the basic shaping engine, please refer to:

https://github.com/VincentWei/mg-tests/blob/master/4.0/basicshapingengine.c

Note that, to use the complex shaping engine, you need to compile MiniGUI with the option --enable-complexscripts, and HarfBuzz should be installed in your system. Also note that, you should use TrueType fonts with the complex shaping engine.

For the sample of the complex shaping engine, please refer to:

https://github.com/VincentWei/mg-tests/blob/master/4.0/complexshapingengine.c

After this, you should call CreateLayout to create a LAYOUT object for laying out the text, then call LayoutNextLine to lay out the lines one by one.

You can render the laid out lines by calling DrawLayoutLine function.

Finally, you call DestroyLayout and DestroyTextRuns to destroy the layout object and text runs object.

Before rendering the glyphs laid out, you can also call GetLayoutLineRect to get the line rectangle, or call CalcLayoutBoundingRect to get the bounding rectangle of one paragraph.

For the sample for layout object, please refer to:

https://github.com/VincentWei/mg-tests/blob/master/4.0/createlayout.c

For the detailed API description related to complex/mixed scripts, please refer to:

http://www.minigui.com/doc-api-ref-minigui-sa-4.0.0/html/group__complex__scripts.html

Internals

These new APIs provide a very flexible implementation for your apps to process the complex scripts. The implementation is derived from LGPL'd Pango, but we optimize and simplify the original implementation in the following respects:

  • We split the layout process into two stages. We get the text runs (Pango items) in the first stage, and the text runs will keep as constants for subsequent different layouts. In the second stage, we create a layout object for a set of specific layout parameters, and generates the lines one by one for the caller. This is useful for an app like browser, it can reuse the text runs if the output width or height changed, and it is no need to re-generate the text runs because of the size change of the output rectangle.

  • We use MiniGUI's fontname for the font attributes of text, and leave the font selection and the glyph generating to MiniGUI's logfont module. In this way, we simplify the layout process greatly.

  • We always use Uchar32 string for the whole layout process. So the code and the structures are clearer than original implementation.

  • We provide two shaping engines for rendering the text. One is a basic shaping engine and other is the complex shaping engine based on HarfBuzz. The former can be used for some simple applications.

Restrictions