Skip to main content

Hand gestures, driving everything with the camera

Mnemosyne OS can follow your hand through the webcam. You move the board, open an app or close a window without touching the mouse. Everything happens on your machine: the camera image never leaves Mnemosyne OS.

Turning the camera on​

Open Settings › Gestures and allow the camera. The page offers a tutorial: it shows each gesture on a small practice board, step by step. Gestures do not touch your real board while the tutorial is open.

The basic gestures​

  • Pinch thumb and index, then move: the board moves.
  • Pinch at the top-left corner of a window: only that window moves.
  • Pinch with both hands, then spread or bring them together: the board zooms.
  • Open hand, fingers together, then spread them as wide as you can: the window in front goes full screen. The same gesture puts it back.
  • Closed fist, held a moment: the window in front closes.
  • Index and middle finger up, move up or down: the content scrolls.
  • Three fingers up, held: the app pad opens.

A cheat-sheet in a corner of the screen lists these gestures while the camera runs.

Gestures inside an app​

Some apps take the gestures their own way when they are in full screen. The Neural Map in 3D turns when you pinch and move, and comes closer when you move your hand toward the camera. An app from the Hub can do the same: the cheat-sheet then shows its gestures first, under its name.

An app can also have its own actions, such as an exploded view. You choose the pose of each action in the My gestures tab, under « Your apps' gestures ». Show the pose three times, then validate. An app never sees the camera or your hand: it only receives the gesture to apply.

Teaching your own poses​

The My gestures tab lists every gesture, then offers « Teach your own pose ». Poses Mnemosyne OS already uses are refused: pinch, two fingers, three fingers, open hand and fist.

Resetting everything​

At the bottom of Settings › Gestures, « Reset everything » turns the camera off and erases your settings, your calibration and your taught poses. Press twice to confirm. The hand-tracking model ships inside the app: there is nothing to download.