Skip to content
← Back to writing

April 5, 2026

OmniVoice just said hello from my Intel Arc

I ported OmniVoice, a k2-fsa speech model, to run on an Intel Arc GPU. Six files changed, and an honest account of what I did and did not touch.

It's late and OmniVoice just said hello from my Intel Arc.

What upstream looked like when I started: the device picker in OmniVoice v0.1.2 knows CUDA, MPS, and CPU — an Intel Arc falls through to CPU. That's the gap the port fills.

I have been collecting Intel Arc cards over the last few months. An A310, an A380, two Arc Pro B50s, and a B70 waiting on a pickup shelf at Micro Center. That's a lot of VRAM across two architectures, and until tonight almost none of it was doing anything useful for AI.

The reason is the tooling. Every interesting AI project on GitHub says the same thing in its install instructions: requires CUDA, NVIDIA GPU, CUDA 12.x. Search for "how to run X on Intel Arc" and you get three forum threads, two of them ending in someone giving up, and a 2024 blog post that predates Battlemage. Intel has the hardware. They ship a PyTorch XPU backend and OpenVINO. What they don't have is a community porting the good stuff over.

So tonight I sat down and did one small piece of that work myself.

Picking a target

I was reading new releases and hit OmniVoice. It comes out of the k2-fsa project (the Next-gen Kaldi people), not Xiaomi, and I want to be accurate about that since I credited it wrong the first time I talked about it. Zero-shot text to speech, a long list of languages, voice cloning from a short reference clip, voice design from attributes like gender and accent. Apache-2.0, recent, and the code reads cleanly.

It's a good weekend target for one reason: it's small. 45 Python files, about half a megabyte of source. No C++ extensions, no custom CUDA kernels. Just a transformers-based model with PyTorch glue. If I couldn't get this one onto the B50, I probably couldn't get anything onto it.

What actually needed to change

I scanned the codebase for every CUDA assumption and the list was short. Three CLI entry points (infer.py, infer_batch.py, demo.py) each carried their own copy of a get_best_device() helper that knew about CUDA, MPS, and CPU but not XPU. The model's ASR loader picked fp16 only when the device string started with cuda. The batch worker hardcoded cuda:{id} as its per-rank device string.

That was it. No deeper surgery.

The first move was to stop having three copies of the same detector. I put one get_best_device() in omnivoice/utils/common.py with XPU in the priority chain, plus a count-returning variant for the batch worker, and had the three CLI files import from it:

# omnivoice/utils/common.py  (added)
def get_best_device():
    """Auto-detect the best available device: CUDA > XPU > MPS > CPU."""
    if torch.cuda.is_available():
        return "cuda"
    if hasattr(torch, "xpu") and torch.xpu.is_available():
        return "xpu"
    if torch.backends.mps.is_available():
        return "mps"
    return "cpu"


def get_best_device_with_count():
    """Auto-detect best device and return (device_type, device_count)."""
    if torch.cuda.is_available():
        return "cuda", torch.cuda.device_count()
    if hasattr(torch, "xpu") and torch.xpu.is_available():
        return "xpu", torch.xpu.device_count()
    if torch.backends.mps.is_available():
        return "mps", 1
    return "cpu", 1

The hasattr(torch, "xpu") guard matters: a CPU-only torch build doesn't have the xpu namespace at all, so calling torch.xpu.is_available() blind would throw on a machine that has no XPU support compiled in.

Each entry point then lost its private copy. Here is infer.py, which is representative of all three:

# BEFORE: omnivoice/cli/infer.py
from omnivoice.utils.common import str2bool


def get_best_device():
    """Auto-detect the best available device: CUDA > MPS > CPU."""
    if torch.cuda.is_available():
        return "cuda"
    if torch.backends.mps.is_available():
        return "mps"
    return "cpu"
# AFTER
from omnivoice.utils.common import get_best_device, str2bool

The ASR dtype check was a one-line widen. fp16 should be on for XPU the same as CUDA:

# omnivoice/models/omnivoice.py
asr_dtype = (
    torch.float16 if str(self.device).startswith(("cuda", "xpu")) else torch.float32
)

The batch worker had two CUDA assumptions. It called its old two-value get_best_device(), and it built the worker device string as f"cuda:{device_id}". Both got generalized so the device type flows through instead of being assumed:

# omnivoice/cli/infer_batch.py
-    worker_device = f"cuda:{device_id}"
+    worker_device = f"{device_type}:{device_id}"
-    device_type, num_devices = get_best_device()
+    device_type, num_devices = get_best_device_with_count()

The whole change is six files, 57 insertions and 32 deletions. Most of the net add is documentation, and I committed it as 8f802e9 on the intel-xpu branch.

A note on what I deliberately did not touch, because it surprised me how little was needed. I left pyproject.toml alone. Upstream pins the CUDA wheel index there, and you do override it at install time, but you do it on the command line, not by editing the project file, so the fork stays clean for everyone who is still on NVIDIA. I left the flex_attention import alone too. It's only used on the packed-sequence training path, inference never reaches it, and Flash Attention 2 being CUDA-only is handled by transformers itself: when the flash kernel is missing it falls back to SDPA automatically. That fallback is the reason this port works without rewriting a single line of attention code.

Installing and verifying

The install is the part that isn't in the source diff. You get a torch build with XPU support from Intel's wheel index, then install OmniVoice on top:

pip install torch torchaudio --index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/us/
python -c "import torch; print(torch.xpu.is_available(), torch.xpu.device_count())"

On the B50 that printed True 1 and the device name came back as Intel(R) Arc(TM) Pro B50 Graphics. The framework layer was alive. I documented the driver install, the wheel index, and this verification step as a collapsible "Intel Arc GPU (XPU)" section in the README so the next person doesn't have to reverse-engineer it.

The moment of truth

I wrote a small test script: load the model, detect the device, generate one sentence with voice design, write a WAV. First run threw a validation error because I guessed a voice attribute wrong (it wants moderate pitch, not medium pitch), so I fixed that, ran it again, and got audio.

I clicked the WAV. It played: a male American voice saying hello back from Intel's XPU backend, and I rewound it and played it again.

One honest caveat about provenance. The device-detection diff above is in the committed code at SHA 8f802e9, so you can check it. The test script itself, and the timing numbers I quote below, are from my own local runs and never got committed to the repo, so treat the RTF figures as my reported results rather than something you can reproduce from the branch.

What the numbers mean

RTF is real-time factor: generation time divided by audio duration. Under 1.0 is faster than real time. The published NVIDIA reference for OmniVoice is well under that. My warm run on the B50, on Windows, with no torch.compile, no quantization, no tuning of any kind, came in slower than real time.

That's not a number for a marketing page. Tonight was only about proving an Intel Arc GPU can run OmniVoice at all, on the native PyTorch XPU backend, with no CPU fallback and no emulation. That's done. A slower-than-real-time first pass on an entry-level workstation card running Windows with zero optimization is a starting line, not a verdict.

Linux is generally faster than Windows for Intel XPU. The bigger Battlemage parts move data faster than the B50. Quantization cuts memory pressure. torch.compile with the XPU backend is landing in newer PyTorch. Every one of those is on the table for the next pass.

Why this is worth doing

Intel just launched the Arc Pro B70 at $949 with 32 GB of VRAM, against an NVIDIA workstation part in the same class that costs roughly twice as much for less memory. The hardware story is compelling. The software story is almost nonexistent. Search for any new AI project plus "Intel Arc" and you find nothing. That gap is where I want to spend some time.

I'm not a kernel engineer and I'm not going to hand-write flash attention for XPU. What I am good at is the unglamorous part: reading a codebase, finding where it assumes CUDA, and dropping in a fallback that works on other backends. That's what tonight was: a few hours, six files, a working port.

The next target is probably a video generation model, then something else that catches my eye. The plan is simple: open a PR to upstream k2-fsa/OmniVoice so the fork doesn't have to exist long term, run the multilingual test before I call the port complete, and test on the B70 when I pick it up. The diff is small and additive, so the PR shouldn't be controversial.

Tonight the first one works. The WAV file says hello back. Tomorrow I start on the next.


Fork: github.com/smashingtags/OmniVoice/tree/intel-xpu
Upstream: github.com/k2-fsa/OmniVoice
Commit: 8f802e9 (intel-xpu)
Hardware: Intel Arc Pro B50 (16 GB, Battlemage) on Windows 11