The orb, your OS by voice

The orb is the assistant you talk to. You speak, the orb answers aloud, and short commands act on the app before any model is asked. Since 1.9.0 the orb also drives your apps. Since 1.9.1 it drives the views inside an app.

The assistant carries the name you chose at onboarding. You can change it in Settings › Profile. This page calls it "the assistant".

The voices themselves, the engines and cloning are on the voice page. Every switch of Settings › Voice is on the Settings › Voice page.

Call the orb

1. Press the ∞ button

   The ∞ button sits at the start of the dock. Its tooltip reads Call the orb. The orb comes onto the board, above your windows. Press it again to send the orb away.

2. Accept the message once

   The first time, a dialog titled Before you talk out loud explains that answers can contain mistakes, and that what you say goes to the model chosen in Settings. Tick I have read this text and I accept it., then I accept. The date is recorded. You can read the text again in Settings › Voice › Orb, card The orb's message.

3. Speak

   The orb listens, transcribes, then answers aloud and in writing. Say "goodbye" or press Esc to end.

Other doors to the orb:

• The command palette has Call the orb and Voice assistant. The second one opens the full-screen assistant. Either entry can get a key in Settings › Shortcuts.
• The orb studio (Settings › Voice › Orb, card Appearance) has a group Pin the orb. Its switch Above every app keeps a small orb outside Mnemosyne OS, in front of the other windows. Click it to talk and drag it by its ring.
• The wake word opens the orb when you say the assistant's name.

What you can say

Short commands run in the app itself, with no model call. A command has at most 7 words. Politeness such as "can you" or "please" does not count. A longer sentence is conversation and goes to the model.

You say	What happens	Languages
"open my agenda", "show notes", "launch Atlas"	The window or the app opens.	All 7
"close notes", or "close" alone	That window closes, or the one you used last.	All 7
"full screen", "normal size"	The named window, or the last one, is maximized or restored.	All 7
"goodbye", "bye", "stop"	The orb ends the conversation.	All 7
"add a task buy bread"	The task is written to your To-do.	FR, EN, ES
"new chat", "grid", "governance"	A new chat, the windows in a grid, the Governance window.	FR, EN, ES
"arm Sentinel", "disarm Sentinel"	Sentinel arms or disarms.	FR, EN, ES
"what's the weather in Boston"	A weather card, from MET Norway.	FR, EN, ES
"what am I looking at", "read my screen"	The screen dialog opens with your question.	FR, EN, ES

The seven languages are English, French, Spanish, German, Portuguese, Russian and Chinese. The first time the orb opens in German, Portuguese, Russian or Chinese, a dialog says The orb is learning your language and offers a bug report.

A command said first, followed by a question, does both: "open my agenda and tell me what's on Friday" opens the Agenda, then the model answers.

Your own phrases and routines

Settings › Commands lets you teach a phrase. The phrase then runs an action of your choice, such as launching a program. The Routines tab chains steps under one phrase: go to a desktop, open windows, launch a program, play a playlist. A routine that launches a program asks you on screen, once per program.

Orders for your local agent

A sentence that starts with the assistant's name is an order, for example "Mnemosyne, summarize this folder". The app hands it to your local agent in MnemoHermes and shows a card for your approval. The orb says I'm waiting for your go: the card is on the board. While the task runs, "stop" or "cancel" stops it.

The orb drives your apps

A sentence that asks an app for more than opening goes to the app itself. These English sentences are in the app's own tests:

• "please add bread to my shopping list", in the To-do;
• "fly to Tokyo" and "switch to thermal vision", in God's Eye View, the views of an app (1.9.1);
• "enable the satellites", in the same app.

The tests hold many more sentences in French: adding and moving an appointment, searching notes, turning a 3D model in Atlas. The words that spot a gesture are mostly French, so other languages reach this route less often.

The app makes one call to a fast model with the actions that app declares. Then the steps run through the same door your agents use. A gesture that worked once is replayed later without a model call.

• A bare "open Atlas" stays instant and free. Only the extra words reach the model.
• An action that spends credits is never run from the orb. A misheard sentence must never buy a picture.
• When a required detail is missing, the orb asks one question and waits for your answer.
• When no app can do it, the sentence goes back to the chat.
• When the orb is attached to an app, it drives that app only. A line says so, and its × frees the orb.

The To-do, the Agenda and Notes answer aloud. The orb says, for example, Ticked: followed by the task, or Your week: followed by the appointments. When an app cannot do what you asked, the orb says That app can't do this.

The wake word

Beta

The card carries a beta badge in the app.

Say the assistant's name and the orb opens. The whole sentence, name included, becomes the orb's first turn: "Mnemosyne, open my agenda" works in one breath.

1. Choose a local transcription engine

   The wake word confirms the name with the orb's own transcription. It needs a local engine, picked in Settings › Voice › Listening, card Transcription. With cloud transcription the card says Cloud transcription is active: the wake word needs a local engine.

2. Install the engine

   In the Wake word card, press Install. The download is about 165 MB, once.

3. Turn it on

   Switch on Listen for followed by the assistant's name. A light shows in the title bar while it listens.

4. Spell the name if needed

   When the engine does not know the name, the card says so. Write how the name sounds with common words in the spelling field below the switch. The app already knows one spelling: "Mnemosyne" is heard as "némo zine".

How it listens:

• A voice detector runs all the time. Only the start of a spoken segment is checked for the name. Then the orb's transcription confirms the name came first.
• The name must open the sentence. "I saw Mnemosyne yesterday" does not wake it. A short word may come before it: "hey", "ok", "okay", "hé", "oye", "dis", "alors".
• The sound stays in memory on this computer and is never written to disk. Nothing is sent before the name is confirmed.
• A false wake opens nothing. The card counts the day: wakes, confirmed, false.
• Listening pauses while the computer is locked, and while another feature holds the microphone.

The engine pins one small French speech model (Vosk). Names are matched with the words that model knows, which is why the spelling field exists.

Voice first

When you talk to the orb, the voice gets the graphics card first. Under the orb's status a line reads Images and indexing are paused during the conversation. They resume a minute after it ends.

What waits while the voice works:

• the image search of Theia, and its indexing pass;
• background embeddings;
• the night's dreams;
• the DocWatch queue.

Transcription, chat dictation and read-aloud all count as voice. Everything resumes 60 seconds after the last voice activity. If you stopped Theia yourself, it stays stopped for 30 minutes, even after the voice ends. The wake word engine runs on its own and does not hold the card.

Voice command review (in test)

In test

Released in 2.2.0 with a beta badge. Windows x64 only.

Sometimes the orb takes a command for a question. A local decision model, d1-omni-600M from Liquid AI, rereads those sentences after the turn. It lists the ones that look like a command. It never acts and never speaks.

1. Download the model

   Settings › Voice › Orb, card Voice command review, press Download. The model weighs about 407 MB, plus a small server. It runs on the processor, at low priority.

2. Turn it on

   Switch it On. A sentence is listed when the model reads a command with at least 90 % confidence.

3. Review the sentences

   Open the commands to review opens the window Voice commands to review. Each sentence waits for one answer: It is a command or Not a command. You can also Correct the text.

The window also keeps statistics and a live view of the last sentence. Under Contribute to the dataset you can send selected sentences to the dataset Mnemosyne-OS/orb-intents on Hugging Face. The app opens a pull request with your own Hugging Face token, after you review the file. Nothing is shared unless you select it and press Send. Never share keeps a sentence out for good. While the dataset is not open, the window says The public dataset is not open yet. The sentences stay selected.

What the orb reads

Settings › Voice › Orb, card What the orb reads, sets two things:

• The manual: Every answer, Only when asked or Never.
• Vaults: with no vault ticked, the orb searches every vault that answers. Tick some to limit it.

See also

• Settings › Commands: taught phrases and routines.
• Voice commands to review: the review window, control by control.
• The voice: engines, cloning, voice mode.
• Morning briefing and evening journal: the orb reads the briefing aloud.
• Sentinel: armed and disarmed by voice.