Page History

NxWM::CNxConsole Keyboard Input

I recently decided that I wanted to connect a USB keyboard and use that to drive NxWM::CNxConsole windows. I realized that I had completely forgotten how that work and had to reverse engineer my own code to figure how to do this.

Since I went through that effort to analyze the keyboard path, I decided that I should publish this as a Wiki page. If for no other reason so that I won't have to do this analysis again in the future.

...

Players

Let's look at the major players in the keyboard data transfer. This is much more complex than you might initially think:

Special Drivers

• NxConsole Device. The NX console input comes through a special device driver that is registered at /dev/nxcon0 as part of early initialization.

Kernel Threads

• NX Server Thread. The NX Server is the graphics server command. It receives messages from various sources, performs graphics actions, and forwards graphic event messages to the correct window. Most of the time, the NX Server Thread was waiting on a message queue to receive the next graphics event.

• NxConsole Threads. Each NxConsole has a thread that was started when each NxWM::CNxConsole instance was created by NxWM. Each NxWM::CNxConsole thread opens the NxConsole driver at /dev/nxcon0 and redirects stdin, stdout, and stderr to/from that special device. Normally, the NxWM::CNxConsole thread is stopped, just waiting on read for keyboard input to complete.

Application Threads

• NxWidgets Window Event Handler Thread. CNxServer::listener() is an application thread started by NxWidgets each time a new window is opened. It receives window messages from the NX server and dispatches the messages accordingly.

...

Now here is the sequence of events to get keyboard input from the stdin device to the correct NxConsole.

1. Application Space / NxWidgets Window Event Handler Thread

Let's start with the initial state of the NX Server Thread. Initially, it will just want for messages from the NX Server.

...

• nuttx/libnx/nxwm/nx_eventhandler. nx_eventhandler() waits to receive a message from the NX server. Each window has its own message queue; each window instance has its own nx_eventhandler() waiting for messages.

2. Application Space / Keyboard Listener Thread

Here are the immediate events that happen when the keyboard data is entered. The Keyboard Listener Thread wakes up and forwards the Keyboard data to the the NX Server. Only the NX Server knows which window should get the keyboard input.

...

• nuttx\libnx\nxmu\nx_kbdin.c. This library function just hides the NX server messaging implementation. It sends the NX_SVRMSG_KBDIN to the NX server thread.

3. Kernel Space / NX Server

The NX Server wakes up, receives the keyboard message, and forwards it to the appropriate window.

...

• nuttx/graphics/nxconsole/nxmu_kbdin.c. nxmu_kbdin() simply sends the NX_CLIMSG_KBDIN to the appropriate windows client via the client message queue associated with the window.

4. Application Space / NxWidgets Window Event Handler Thread

The Windows client wakes up when the keyboard message is received. It forwards the keyboard data to "/dev/nxcon0/" so that is can be available to the NxConsole window.

...

NOTE: Here is a violation of the Application and Kernel Space boundaries. nxcon_kbdin.c built into Kernel Space but it is called from Application Space. The solution is to (1) move nxcon_kbdin() to libnx/ and (2) it should then communicate with the /dev/nxcon9 driver via ioctl calls. This will become a problem some day.

5. Kernel Space / NxConsole Thread

Finally,

• nuttx/graphics/nxconsole/nxcon_kbdin.c. The receipt and enqueuing of keyboard data by nxcon_kbdin() wakes up any threads waiting in the nxcon_read() method. This is how the NxConsole gets its keyboard input.

Mouse Input

Almost everything said here applies to mouse/touchscreen input as well. If we were to replace the names keyboard to mouse, kbdin to mousein, etc. you have a pretty good description of how mouse/touchscreen input works.

...