Skip to content

GitLab

Last edited by Vincent Wei
Page history

Supporting and Using Extra Input Messages

How to support modern input devices and use the extra input messages in your MiniGUI apps.

Table of Contents

Overview

In MiniGUI 4.0.0, we introduce the extra input messages to support modern input devices including multiple touch panel, gesture, joystick, tablet tool, table pad, and even switch.

The extra input messages have the prefix MSG_EXIN_. If a MiniGUI app want to handle these extra input events such as gestures, you need to handle the MSG_EXIN_XXX messages in your window procedure.

Currently, there are two built-in IAL engines which can generates the extra input messages:

  • The IAL engine of libinput to support all modern input devices on a Linux box. This engine runs on libinput introduced by Free Desktop project.

  • The enhanced IAL engine of random to generate extra input messages automatically for testing.

You can also write your own IAL engines to generate the extra messages. Please see the implementation of libinput and random engines for the details.

This document describes how to handle the extra input messages and how to use libinput and random engines, or how to writing your own IAL engine to support a modern input devices, for example, a multi-touch panel.

Using the Extra Input Messages

In addition to the standard keyboard and mouse messages, MiniGUI generates extra input messages for input events from other input devices, including multi-touch panel, tablet pad, joystick, and so on. We call these messages as 'extra input messages'.

The extra input messages can be classified into the following types:

  • Axis messages: the messages generated by a pointer axis like mouse wheel.
    • MSG_EXIN_AXIS
  • Button messages: the messages generated by a button on joystick. The buttons other than left, right, and middle buttons on a mouse will be treated as generic buttons.
    • MSG_EXIN_BUTTONDOWN
    • MSG_EXIN_BUTTONUP
  • Multi-touch messages: the messages generated by a multi-touch panel.
    • MSG_EXIN_TOUCH_DOWN
    • MSG_EXIN_TOUCH_UP
    • MSG_EXIN_TOUCH_MOTION
    • MSG_EXIN_TOUCH_CANCEL
    • MSG_EXIN_TOUCH_FRAME
  • Gesture messages: the gesture messages.
    • MSG_EXIN_GESTURE_SWIPE_BEGIN
    • MSG_EXIN_GESTURE_SWIPE_UPDATE
    • MSG_EXIN_GESTURE_SWIPE_END
    • MSG_EXIN_GESTURE_PINCH_BEGIN
    • MSG_EXIN_GESTURE_PINCH_UPDATE
    • MSG_EXIN_GESTURE_PINCH_END
  • Tablet tool messages: the messages generated by a tablet tool.
    • MSG_EXIN_TABLET_TOOL_AXIS
    • MSG_EXIN_TABLET_TOOL_PROXIMITY
    • MSG_EXIN_TABLET_TOOL_TIP
    • MSG_EXIN_TABLET_TOOL_BUTTON
    • MSG_EXIN_END_CHANGES
  • Tablet pad messages: the messages generated by a tablet pad.
    • MSG_EXIN_TABLET_PAD_BUTTON
    • MSG_EXIN_TABLET_PAD_RING
    • MSG_EXIN_TABLET_PAD_STRIP
    • MSG_EXIN_END_CHANGES
  • Switch messages: the messages generated by a switch.
    • MSG_EXIN_SWITCH_TOGGLE
  • User-defined messages: the messages generated by a user-defined device.
    • MSG_EXIN_USER_BEGIN
    • MSG_EXIN_USER_UPDATE
    • MSG_EXIN_USER_END

All extra input messages will be sent to the current active main window. In your window procedure, you can handle the messages to reflect user's input, for example, zooming in or out a picture.

For the complete description of the messages, please refer to:

http://www.minigui.com/doc-api-ref-minigui-sa-4.0.0/html/group__extra__input__msgs.html

For the sample to handle the extra input messages, please refer to:

https://github.com/VincentWei/mg-tests/tree/master/extra-input

Compile-time Configuration

There are two configure options related to the libinput and random IAL engines :

  • --enable-libinputial or --disable--libinputial to enable or disable the libinput IAL engine; enabled by default. Note that the libinput IAL engine is only available on Linux, and you need to install the libinput 1.10.0 or later first.
  • --enable-randomial or --disable--randomial to enable or disable the random IAL engine; disabled by default. Note that random engine does not attached to any real input devices, it generates the extra input events randomly.

Run-time Configuration

For libinput engine, we introduce a new section in MiniGUI runtime configuration:

[libinput]
seat=seat0

The key libinput.seat specifies the seat identifier, the default is seat0.

For random engine, we introduce a new section in MiniGUI runtime configuration:

[random]
logfile=events.out
eventtypes=mouse-keyboard-button-gesture-stouch
minkeycode=1
maxkeycode=128
minbtncode=0x100
maxbtncode=0x1ff

The MiniGUI runtime configuration key random.logfile specifies the log file which will store the input events generated by this engine. If MiniGUI failed to open the log file, the log feature will be disabled.

The MiniGUI runtime configuration key random.eventtypes specifies the input event types which will be generated by this IAL engine, in the following pattern:

<event-type>[-<event-type>]*

The <event-type> can be one of the following values:

  • mouse: mouse.
  • keyboard: keyboard.
  • button: buttons.
  • single_touch: touch pen or single touch panel.
  • multi_touch: multiple touch panel.
  • gesture: gesture.
  • tablet_tool: tablet tool.
  • tablet_pad: tablet pad.
  • switch: switch.

The MiniGUI ETC key random.minkeycode specifies the minimal key code which can be generated by the engine if keyboard is included.

The MiniGUI ETC key random.maxkeycode specifies the maximal key code which can be generated by the engine if keyboard is included.

The MiniGUI ETC key random.minbtncode specifies the minimal button code which can be generated by the engine if button is included.

The MiniGUI ETC key random.maxbtncode specifies the maximal key code which can be generated by the engine if button is included.

For invalid random.eventtyps, the engine uses mouse as default.

For invalid random.minkeycode, and/or random.maxkeycode key values, the engine uses SCANCODE_ESCAPE, and SCANCODE_MICMUTE respectively.

For invalid random.minbtncode, and/or random.maxbtncode key values, use 0x100 (BTN_MISC defined by Linux kernel), and 0x2ff (KEY_MAX defined by Linux kernel) respectively.

This engine maintains a state machine for each input event type, and generates a reasonable event sequence for each type. If and only if an event sequence finished or cancelled, the engine switch to another event type randomly.

Note that currently, the following event types (in random engine) are not implemented:

  • multi_touch
  • tablet_tool
  • tablet_pad
  • switch

Writing an IAL Engine to Support Extra Input Messages

On Linux, it is better to use libinput to support all input devices including multi-touch panel and table tool. However, for an embedded or IoT device, it may be too heavy to use libinput or there is no way to run libinput (e.g., when you run MiniGUI on a real-time operating system).

For this situation, you need to implement your own IAL engine to support the extra input messages. The steps are as follow:

  1. Use --enable-customial option to configure MiniGUI to include custom IAL engine.
  2. Implement the following external stubs outside MiniGUI to initialize and terminate your custom IAL engine:
  3. Change MiniGUI run-time configuration to specify the key system.ial_engine to be custom.
BOOL InitCustomInput (INPUT* input, const char* mdev, const char* mtype);
void TermCustomInput (void);

In InitCustomInput function, you should fill the operations (the callbacks) of input argument, the pointer to a INPUT structure:

typedef struct tagINPUT {
    char*   id;

    // Initialization and termination
    BOOL (*init_input) (struct tagINPUT *input, const char* mdev, const char* mtype);
    void (*term_input) (void);

    // Mouse operations
    int  (*update_mouse) (void);
    void (*get_mouse_xy) (int* x, int* y);
    void (*set_mouse_xy) (int x, int y);
    int  (*get_mouse_button) (void);
    void (*set_mouse_range) (int minx, int miny, int maxx, int maxy);
    void (*suspend_mouse) (void);
    int (*resume_mouse) (void);

    // Keyboard operations
    int  (*update_keyboard) (void);
    const char* (*get_keyboard_state) (void);
    void (*suspend_keyboard) (void);
    int (*resume_keyboard) (void);
    void (*set_leds) (unsigned int leds);

    // Event loop
    int (*wait_event) (int which, int maxfd, fd_set *in, fd_set *out,
            fd_set *except, struct timeval *timeout);

    // New wait event method for getting extra input events; since 4.0.0
    int (*wait_event_ex) (int maxfd, fd_set *in, fd_set *out,
            fd_set *except, struct timeval *timeout, EXTRA_INPUT_EVENT* extra);
} INPUT;

The key to support extra input messages is the new operation: wait_event_ex. If you implement wait_event_ex, you can set wait_event to be NULL.

In the implementation of wait_event_ex, when there is an extra input message, you should:

  1. Fill the fields of extra structure, e.g.:
    extra->event = IAL_EVENT_AXIS;
    extra->wparam = MAKELONG(AXIS_SCROLL_VERTICAL, AXIS_SOURCE_WHEEL);
    extra->wparam = MAKELONG(AXIS_SCROLL_HORIZONTAL, AXIS_SOURCE_WHEEL);
    extra->lparam = MAKELONG(mouse_event->sv, mouse_event->dsv);
  1. Make sure the return value of this operation has the bit of IAL_EVENT_EXTRA set:
    retval |= IAL_EVENT_EXTRA;

For more information, please refer to the libinput and/or random IAL engines:

https://github.com/VincentWei/minigui/blob/master/src/ial/linux-libinput.c https://github.com/VincentWei/minigui/blob/master/src/ial/random.c