Skip to main content

Voice commands to review

The orb sometimes misses a command. It then treats the sentence as a question. This window lists those sentences so you can label them. A small local model, d1-omni-600M from Liquid AI, rereads what you said to the orb. It lists each sentence that looks like a command. You answer once per sentence: command or not.

The review runs on Windows x64 only for now.

Opening it​

In the Voice page of Settings, the Voice command review section has a button, Open the commands to review. The window can also be added to the menu bar from Add a widget, where its icon is labelled Crosscheck labels. See Voice settings.

Turning the review on​

The window and the settings section share the same controls:

  • A download button fetches the model. Nothing downloads before you click.
  • Review on and Review off switch the review.
  • Remove the model deletes it.

With the review on, a sentence is listed when the model reads a command with at least 90 % confidence.

Labelling sentences​

Sentences to review lists each sentence with three facts: the transcript, what the orb did with it, and what the model read with its confidence. Each sentence offers:

  • It is a command, then a Command type;
  • Not a command;
  • Correct the text, to fix a misheard sentence;
  • Never share, to keep the sentence off any upload.

Filters sort the list: To review, Commands, Not commands, Shared and All.

Statistics​

The Statistics block counts the sentences analysed, listed, reviewed and shared. It also shows how many of the reviewed sentences were real commands, and how often the model picked the right command type.

Contributing to the dataset​

Contribute to the dataset sends the sentences you tick to the public dataset Mnemosyne-OS/orb-intents on Hugging Face. The flow has three steps:

  1. Tick Select for sharing on each sentence. A display name is optional.
  2. Press the button that reviews the file. The window shows the exact file before anything leaves.
  3. Press Send. The app opens a pull request on the dataset, with the Hugging Face token of your account.

A sentence marked Never share never goes into the file. Without a valid token, the window asks you to add one in the Models page of Settings.

The labels are stored on this machine, in ~/.mnemosyne/orb-notebook.json.

From afar​

Zoom the canvas out and the window follows the orb live. The last sentence is shown large: what you said, what the orb did, what the model read, and whether the sentence joined the list. The sentences before it sit underneath, one line each.