Skip to main content

Settings → NVIDIA

One switch, for one thing: whether the local GGUF models chosen on the AI & Models page run on your NVIDIA card. The rail entry is NVIDIA; the page is titled NVIDIA Settings, with the green NVIDIA eye, and its subtitle reads GPU acceleration for local AI models.

The switch saves as soon as you flip it. There is no Save button.

Detection​

Once the configuration has loaded, a banner says what the app found:

  • ✓ NVIDIA GPU detected: followed by the model and its VRAM in MB, on a green tint;
  • ⓘ No NVIDIA GPU detected. CUDA needs a GeForce or RTX card — leave this off to keep the automatic Vulkan/CPU backend.

Detection is best-effort (it matches the names the system reports) and it does not gate the switch: you can flip it either way.

Enable CUDA acceleration​

Enable CUDA acceleration: Prefer your NVIDIA GPU for local models — falls back to CPU automatically if CUDA is unavailable.

What it writes is worth knowing. On asks for an automatic backend: CUDA is tried first, then Vulkan or the CPU. Off pins the CPU. The switch never asks for CUDA alone, so turning it on cannot leave you with a model that refuses to load. Anything that is not an explicit CPU choice reads as on, so on a fresh install the switch is on.

If the setting cannot be written, the switch reverts and the page prints Could not save the setting: with the reason.

Only where the CUDA backend exists

The packaged installer ships the CPU and Vulkan backends for local models; the CUDA binaries were left out to keep the download small. On such a build the switch changes nothing visible: the automatic backend falls back on its own. It helps where the CUDA backend is present.

Why enable it​

The page's own four reasons:

  • Much faster local inference — CUDA can generate answers several times quicker than on CPU.
  • Offloads the heavy math to the GPU, so your CPU stays free for the rest of the system.
  • Shorter, more efficient bursts of computation instead of pinning every CPU core for minutes.
  • Everything stays 100% local — bigger models become usable without any cloud API.

Restart​

After any change: ⚠ Restart the app to apply the new GPU backend. with an amber Restart now button (Restarting…). The new backend is only used after a restart.

The other GPU switches​

This page concerns the local language models only. Three other pages have their own, separate, GPU choice:

  • the Images page switches the image embedder to CUDA (Switch to GPU);
  • the OCR page has a GPU engine, MinerU, behind the Engramm license;
  • the Appearance page picks the graphics processor used for rendering (the Neural Map, for instance), which is unrelated to inference.

The System diagnostics widget shows what the machine reports.