Thursday 18 August 2016

Project conclusion

This summer I have been developing a new keyboard system for Mixxx as part of this year's Google summer of Code program. My initial project was to make a keyboard mapping GUI, but the backend turned out to be more complex than expected. To be more specific, it was the internationalization that stretched the original plan as described in the project proposal. However, although I did not make the GUI, the keyboard does work. And all its mappings work on all supported keyboard layouts because of a new key-sequence translation system I made.

The code I've written during this GSoC can be found here:
PULL REQUEST

Keyboard Controller

The base of this new keyboard system is the KeyboardController, a new Mixxx controller type whose life cycle is regulated in the ControllerManager, just like every other controller. Its task is to communicate with the KeyboardEventFilter, which is installed to various widgets as an event filter listening for key events as well for keyboard layout change- and focus events. It is updated by KeyboardController about current mappings whenever the mappings change in KeyboardController's current KeyboardControllerPreset.

When the user presses a key sequence known to KeyboardEventFilter, it passes the ConfigKey bound to the pressed key sequence to KeyboardController, which then does or does not set the Mixxx control, depending on whether the controller is enabled or not. The user can enable / disable it both via controller preferences and via main menu bar's Enable Keyboard Shortcuts checkbox.

KeyboardShortcuts
In Mixxx, we distinct ordinary keyboard shortcuts from shortcuts that are bound directly to QActions with QAction::setShortcut(). These type of shortcuts belong to the [KeyboardShortcuts] group and are processed neither by KeyboardController nor by KeboardEventFilter, but by the QActions themselves. Also, they are not translated by our own translation system but by QTranslator. This translation is possible because these mappings are Qt Standard Shortcuts. These [KeyboardShortcuts] shortcuts are managed by the KeyboardShortcutsUpdater, which is sent a signal when new mappings are loaded. 

Tooltips
Keyboard shortcut info in tooltips is updated by the TooltipShortcutsUpdater, which is sent a signal whenever new mappings are loaded.

Keyboard presets and key translations

Keyboard mappings are stored to disk as *.kbd.xml files and can hold mappings for multiple keyboard layouts. However, it usually just contains info for one layout. That's because Mixxx is now able to translate key sequence from one keyboard layout to the other, runtime. Therefore, it's now possible to start Mixxx with layout en_US, switch to ru_RU and then to el_GR and the keyboard and its mappings will keep working on each switch. Naturally, tooltips are also updated.

The translation system uses layout translation tables, which are generated by a tool I made that reads the current keyboard layout map from X and generated both layouts.cpp and layouts.h. The tool uses the X11 API and therefore it does only work on Linux.

Migration

The kbdcfg_to_kbdxml tool can be given multiple legacy config files (one per layout) and outputs one single *.kbd.xml file. The tool is also given the layout translation tables file, so that it can explicitly add mappings for specific layouts or leave them out, depending on whether the specific mapping is translatable or not. This tool is used to create Keyboard-Default.kbd.xml, the default keyboard preset.

Tuesday 12 July 2016

Keyboard Layouts

An important step in the process of making a successful keyboard controller is making sure it will work out of the box for most of the users, regardless of where they come from and which keyboard layout is used. The current approach in Mixxx is pretty straight forward. We have different mappings for different keyboard mappings; 12 *.kbd.cfg files, to be specific. Mixxx chooses between one of these based on the current locale or defaults to the American English keyboard layout.

One preset file, for all layouts

Having multiple files is ok for the current implementation, but it doesn't entirely fit in the controller preset architecture. It should, because KeyboardController is a controller, and it doesn't fit the controller preset architecture, because all those 12 files are the same preset, in essence. They all represent the Mixxx default mapping. Therefore, it would be weird to get, for example, the Greek preset listed as an option, being an American. And what would the naming be like? Default Mapping_en_US, Default Mapping_ru_RU, Default Mapping_el_GR? It would be a bit messy, especially considering that users will also be making custom presets. However, if we just ship one keyboard preset: "Default Mapping" it would be much cleaner. This file would just hold mapping information for one keyboard layout. If we had some kind of layouts lookup table, Mixxx could then translate the shortcuts to the current locale, if found in the layouts lookup table. This could be done when loading the preset.

Layouts XML file

In order for Mixxx to translate keyboard shortcuts from one keyboard layout to another, it needs to know about both layouts. It needs to have some kind of unique key ID number per key, which characters will be bound to, for different layouts. For example, we could give the key that is right next to the 'Tab' key, ID: 17. This key will hold the character 'Q' for American English keyboard layouts (QWERTY), and  'A' for French keyboard layouts (AZERTY). The layouts XML file holds this information.

Let's do a breakdown on the following XML snippet:

<KeyboardLayoutsResourceFile>

  <layouts>
    <lang>en_US</lang>
    <lang>fr_FR</lang>
    <lang>ru_RU</lang>
  </layouts>

  <key key_id="1">
    <char dead_key="0" lang="en_US" modifier="NONE">`</char>
    <char dead_key="0" lang="en_US" modifier="SHIFT">~</char>
    <char dead_key="0" lang="fr_FR" modifier="NONE">²</char>
    <char dead_key="0" lang="fr_FR" modifier="SHIFT" />
    <char dead_key="0" lang="ru_RU" modifier="NONE">ё</char>
    <char dead_key="0" lang="ru_RU" modifier="SHIFT">Ё</char>
  </key>

  <key key_id="2">
    <char dead_key="0" lang="en_US" modifier="NONE">1</char>
    <char dead_key="0" lang="en_US" modifier="SHIFT">!</char>
    <char dead_key="0" lang="fr_FR" modifier="NONE">&amp;</char>
    <char dead_key="0" lang="fr_FR" modifier="SHIFT">1</char>
    <char dead_key="0" lang="ru_RU" modifier="NONE">1</char>
    <char dead_key="0" lang="ru_RU" modifier="SHIFT">!</char>
  </key>

  <key key_id="3">
    <char dead_key="0" lang="en_US" modifier="NONE">2</char>
    <char dead_key="0" lang="en_US" modifier="SHIFT">@</char>
    <char dead_key="0" lang="fr_FR" modifier="NONE">é</char>
    <char dead_key="0" lang="fr_FR" modifier="SHIFT">2</char>
    <char dead_key="0" lang="ru_RU" modifier="NONE">2</char>
    <char dead_key="0" lang="ru_RU" modifier="SHIFT">&quot;</char>
  </key>

  <!-- ... Etc ... -->

  <key key_id="51">
    <char dead_key="0" lang="en_US" modifier="NONE">n</char>
    <char dead_key="0" lang="en_US" modifier="SHIFT">N</char>
    <char dead_key="0" lang="fr_FR" modifier="NONE">n</char>
    <char dead_key="0" lang="fr_FR" modifier="SHIFT">N</char>
    <char dead_key="0" lang="ru_RU" modifier="NONE">т</char>
    <char dead_key="0" lang="ru_RU" modifier="SHIFT">Т</char>
  </key>

  <key key_id="52">
    <char dead_key="0" lang="en_US" modifier="NONE">m</char>
    <char dead_key="0" lang="en_US" modifier="SHIFT">M</char>
    <char dead_key="0" lang="fr_FR" modifier="NONE">,</char>
    <char dead_key="0" lang="fr_FR" modifier="SHIFT">?</char>
    <char dead_key="0" lang="ru_RU" modifier="NONE">ь</char>
    <char dead_key="0" lang="ru_RU" modifier="SHIFT">Ь</char>
  </key>

  <key key_id="53">
    <char dead_key="0" lang="en_US" modifier="NONE">,</char>
    <char dead_key="0" lang="en_US" modifier="SHIFT">&lt;</char>
    <char dead_key="0" lang="fr_FR" modifier="NONE">;</char>
    <char dead_key="0" lang="fr_FR" modifier="SHIFT">.</char>
    <char dead_key="0" lang="ru_RU" modifier="NONE">б</char>
    <char dead_key="0" lang="ru_RU" modifier="SHIFT">Б</char>
  </key>
</KeyboardLayoutsResourceFile>

KeyboardLayoutsResourceFile
The root element, which is parent node to all other elements.

Layouts
A little manifest node, telling which keyboard layouts this layouts XML file holds information of.

Lang
Parented to layouts, the content of this element is a lowercase ISO 631 language code and an uppercase ISO 3166 country code, separated by an underscore. For example, en_US, for American English, or en_GB, for Brittish, es_ES, for Spanish, or el_GR, for Greek. This follows the QLocale name naming format.

Key
Element representing one physical keyboard key. Each key element is identified by a unique number, based on the key's scancode. This key ID is stored in the key's key_ID attribute.

Char
Tells about which character is bound to the key element, which it is parented to. It specifies the keyboard layout in the lang attribute, which should be the same as one of the keyboard layouts specified in the layouts manifest element.

When targeting the key with ID 2, on an American English layout, the key sequence that will be read by Qt, is just 1. But when targeting the same key, with the shift modifier, the key sequence is Shift+!, not Shift+1. That's why we also need to know which character is bound to the key ID with the shift modifier. The modifier is specified in the modifier attribute.

On some keyboard layouts, some keys that don't do anything and are stored in the OS, until another key is pressed: dead keys. Sometimes a key is not a dead key with modifiers, but it is without, or vice-versa. This is why we need to store whether this char is a dead key or not, in the dead_key attribute. It is a dead key when it is set to 1, and it is not when not set to 1 (everything other than 1 is considered as a not-dead key.

Finally, some keys don't exist on some keyboard layouts. Take the key that is just above the tab key, with ID 1. On French keyboard layouts, this key, with the shift modifier, doesn't exist. In this case, the char element is self-closing.

Key IDs


Layouts editor

It can be a pain to have to create this layout resource files by hand. Fortunately, the layouts editor does the tedious work for you. It is a little tool that makes it easy setting up new layouts and adding them to a layouts resource file.

Here is a little demo, when opening layouts.xml, containing all keyboard layouts supported by Mixxx, as of now.


Some keys are yellow, those are dead keys. Keys can be marked as dead keys by right clicking a key and clicking on Dead key. Keys can be setup by clicking on a key and typing the key that corresponds to the selected key on the GUI. When a key is typed, the focus moves automatically to the following key, so that you can just begin at key 1, and start pressing all keys from left to right till you arrive at key 55.

When pressing shift, the keyboard reveals which characters are bound to the keys with the shift modifier. To setup keys with the shift modifier, do the exact same thing, holding shift.

If a key doesn't exist on your keyboard layout, just let the key open. You can reset a key by pressing Del on your keyboard or by right clicking a key and selecting Reset.

Keyboard presets, new format

Now that we can potentially translate shortcuts from one keyboard layout to another, there is no need anymore to store information for all keyboard layouts. In the new format, each key sequence is given a keyboard layout which it belongs to and a key ID. Then, when a user with keyboard layout ru_RU goes and loads a preset, Mixxx will first look for a key sequence for ru_RU. If it can't find it, it will translate the key sequence from whatever language is present, to Russian. Why add the possibility of overloading key sequences for different layouts? We need to; sometimes a key sequence can not be translated because the key to translate is a dead key in the targeted layout.

Let's breakdown the following XML snippet:

<MixxxKeyboardPreset mixxxVersion="2.0.1+" schemaVersion="1">

  <info>
    <name>Example</name>
    <author>Jordi</author>
    <description>Example preset based on legacy mapping files en_US and fr_FR</description>
  </info>

  <controller>
    <group name="[Microphone]">
      <control action="talkover">
        <keyseq key_id="1" lang="en_US">`</keyseq>
        <keyseq key_id="45" lang="fr_FR">&lt;</keyseq>
      </control>
    </group>

    <group name="[Master]">
      <control action="crossfader_down">
        <keyseq key_id="35" lang="en_US">g</keyseq>
      </control>

      <control action="crossfader_up_small">
        <keyseq key_id="36" lang="en_US">Shift+h</keyseq>
      </control>
    </group>

    <group name="[Channel1]">
      <control action="back">
        <keyseq key_id="31" lang="en_US">a</keyseq>
      </control>

      <control action="reverse">
        <keyseq key_id="31" lang="en_US">Shift+a</keyseq>
      </control>

      <control action="fwd">
        <keyseq key_id="32" lang="en_US">s</keyseq>
      </control>

      <control action="beatsync">
        <keyseq key_id="2" lang="en_US">1</keyseq>
      </control>
    </group>

    <group name="[AutoDJ]">
      <control action="shuffle_playlist">
        <keyseq key_id="universal_key" lang="en_US">Shift+F9</keyseq>
      </control>
      
      <control action="enabled">
        <keyseq key_id="universal_key" lang="en_US">Shift+F12</keyseq>
      </control>
    </group>

    <group name="[KeyboardShortcuts]">
      <control action="FileMenu_LoadDeck1">
        <keyseq>Ctrl+o</keyseq>
      </control>
      
      <control action="FileMenu_LoadDeck2">
        <keyseq>Ctrl+Shift+O</keyseq>
      </control>
    </group>
  </controller>
</MixxxKeyboardPreset>

MixxxKeyboardPreset
The root element, which is parent node to all other elements. This is not unique to keyboard controller presets. Each controller preset XML has this root element.

Info
Contains information that will be visible in preferences -> controllers, when opening a preset. This is also, not unique to keyboard controller presets. Each controller preset should include an info tag, otherwise, it won't load.

Controller
The actual mapping information will be stored inside of this controller element.

Keyseq
Contains a key sequence. The lang attribute stores the keyboard layout this key sequence is valid for. Mixxx will use both this and the key ID stored in the key_id attribute if it needs to translate this key sequence to another layout. Note that some key_id's value is universal_key, which tells Mixxx that it doesn't need to worry about translating the key sequence. Namely, these key sequences are the same for all keyboard layouts. This is true for keys as Space, Enter, Backspace, or the arrow keys.

Control
This element is a parent to keyseq elements. It tells Mixxx which action to execute when the key sequence is pressed (or a translated one). Mixxx puts together a ConfigKey where the group is the group name, and the item is the action attribute. The group name is retrieved from the name attribute of the group element to which this element is parented to.

Keyboard shortcuts
Key sequences of controls belonging to the [KeyboardShortcuts] group are translated in Mixxx using Qt::tr(). Therefore, these keyseq elements won't include nor a key_id attribute, nor a lang attribute.


Migration from *.cfg to *.xml

Converting the 12 *.kbd.cfg files located in mixxx/res/keyboard by hand is really tedious and would take me ages. That's why I made a little conversion tool that translates one or more legacy files to one XML file. This utility is also given a layouts resource file (made with the layouts editor). This way it can take into account which <keyseq> elements to add and which ones to leave out.

Here is a little demo that shows how to add legacy files, choose a layouts resource file, set a preset name and save the preset.



Tuesday 14 June 2016

Keyboard Controller working!

Last week I have been working on making the keyboard controller functional, and I have made some good progress. Although the controls are still set in the keyboard event filter (which will be moved to the keyboard controller very soon), the mapping info is not owned by the keyboard event filter anymore. It's now owned by the keyboard controller preset, which stores the key combinations with the actions that it should trigger. As soon as a new preset is loaded, the keyboard event filter is sent a signal and gets updated about the new preset.

Keyboard preset format

The presets the keyboard controller uses are instances of KeyboardControllerPreset, which rely on *.kbd.xml files. These XML files start of with an info block.
<?xml version="1.0" encoding="utf-8"?>

<MixxxKeyboardPreset schemaVersion="1" mixxxVersion="2.0.0+">
  <info>
    <name>Keyboard Default</name>
    <author>Mixxx</author>
    <description>Default keyboard mapping.</description>
    <wiki>mixxx.org/.../appendix.html#appendix-keyboard</wiki>
  </info>
This <infoblock is required, otherwise it won't even get enumerated and thus will not be listed as a legitimate preset. The block's format is basically the same as for each preset of each other controller type. Though some presets also include <forumsand <devices>, I believe that that information doesn't apply for keyboard presets. However, if I'm proven wrong and we do need those, it can be added really easily.

The data where it all goes about is in the <controllerblock, which contains <groupnodes, listing <controlnodes holding information about which key sequence triggers what action.
<controller>
    <group name="[AutoDJ]">
      <control action="shuffle_playlist" keyseq="Shift+F9" />
      <control action="skip_next" keyseq="Shift+F10" />
      <control action="fade_now" keyseq="Shift+F11" />
      <control action="enabled" keyseq="Shift+F12" />
    </group>

    <group name="[Master]">
      <control action="crossfader_up" keyseq="h" />
      <control action="crossfader_down" keyseq="g" />
      <control action="crossfader_up_small" keyseq="Shift+h" />
      <control action="crossfader_down_small" keyseq="Shift+g" />
    </group>

    <group name="[Microphone]">
      <control action="talkover" keyseq="`" />
    </group>

    <group name="[PreviewDeck1]">
      <control action="LoadSelectedAndPlay" keyseq="Alt+Return" />
      <control action="start_stop" keyseq="Return" />
      <control action="back" keyseq="Alt+Left" />
      <control action="fwd" keyseq="" />
    </group>

    <group name="[Channel1]">
      <control action="play" keyseq="d" />
      <control action="cue_set" keyseq="Shift+d" />
      <control action="cue_default" keyseq="f" />
      <control action="cue_gotoandstop" keyseq="Shift+f" />
    </group>

    <!-- etc... -->

Keyboard preset file handler

The XML file is firstly enumerated by PresetInfoEnumerator, which was added the new preset extension. Then, when the user navigates to the keyboard controller in controller preferences and then selects a preset, the XML document is parsed by KeyboardPresetFileHandler. Each action is read and is stored in a multihash, from which the key is the key sequence (ConfigValueKbd), and the values are the bound actions (ConfigKey). The keyboard preset file handler is not only responsible for loading,  but also for saving presets, where it does basically the same task, but then in reverse.

Tuesday 31 May 2016

Keyboard Controller

Last week I started making the new controller type; KeyboardController. It is already recognized by Mixxx as a legit Controller and listed under Controllers in the Mixxx preferences window. It is not functional yet, though.

KeyboardController

This class subclasses Controller, which it implements all the pure virtual methods from. It's mostly boilerplate code yet. Just enough to make it compile.
Using as EventFilter?
I also started migrating all the methods and fields from KeyboardEventFilter to KeyboardController, with the idea that we could use it as event filter on all the widgets that have currently the KeyboardEventFilter installed on. It turns out though that we couldn't use the KeyboardController as event filter, because it lives in another thread. We could make it thread-save, but that could make Mixxx seem unresponsive.

The new approach is to let the KeyboardEventFilter live, but let it emit signals of what key sequence was pressed instead of setting controls directly. That will be done by the KeyboardController.

KeyboardEnumerator

Every controller type needs a enumerator. This ControllerEnumerator handles discovery and enumeration of controllers. For instance, the MidiController lists all connected MIDI devices, and the HidController does that with HID devices. Since we only support one keyboard, the KeyboardEnumerator returns a list with just one KeyboardController. If we are ever going to support multiple keyboards, this list could be larger.

KeyboardControllerPreset

The KeyboardControllerPreset subclasses ControllerPreset and implements all its pure virtual methods. Just like KeyboardController, this class consists entirely of boilerplate code. It was needed to be able to implement KeyboardController::visit(KeyboardControllerPreset *) : void, which I BTW also implemented in HidController MidiController and BulkController.

Monday 23 May 2016

Let the coding begin!

Today is the official coding start date for the GSoC. According to the planning I made in the proposal, I will spend the first two weeks making the API for the keyboard controller. As with many other software projects though, the unexpected is the rule rather than the exception. The unforseen, which in this case is supporting the possibility to script the keyboard, is in my opinion too important to postpone. As a consequence to this, the planned two weeks will be stretched a little bit. It would be great to have the keyboard completely migrated to the controller variant before the midterm evaluation, and after the midterm focus on the GUI for the newly created controller. Time will tell if this is realistic or not.

But for this week, I will be making the base for the new controller. That is, making a unfunctional controller that Mixxx can detect.

Wednesday 11 May 2016

Keymapping GUI for Mixxx

This year I can proudly say that I have been given a chance to contribute to Mixxx, the most advanced free DJ software available. In the first instance I will be developing from May 23th till August 29th, as part of the GSoC 2016 program. Although the main work will be done this summer, I do not rule out further contributions to Mixxx after the GSoC.

Project

The goal of this project is to make it as easy as possible for the users of Mixxx to customize the keyboard mapping to their taste, without having to modify a *.cfg file and having to restart Mixxx, which is currently a real pain. In order to achieve that, the idea is to make a GUI for the keyboard mapping. There are now two main controller types: MIDI and HID. This project is about developing the third one: Keyboard.

PROJECT PROPOSAL


I will be posting further details this weekend!