Local Voice to Text
Local voice to text is a very appealing idea. Especially when it runs offline.
Nerd-Dictation
Nerd Dictation is an offline Speech to Text utility for desktop Linux.
It uses a Python script to activate Voice to text translation supported by VOSP API static language (neural) model that does the voice recognition.
sudo apt install python3-pip python3-xlib xdotool libasound2-dev` # dependencies suggested by Gemini
sudo apt install python3.12-venv # required to create a virtual envmt
python3 -m venv virtualenvs/vosk # created a vosk virtualenv as global libraries are disallowed
source virtualenvs/vosk/bin/activate # activated the venv
pip3 install vosk # Nerd-dictation installation instruction 1
cd devenv # local preference -- all my installations in one place
git clone https://github.com/ideasman42/nerd-dictation.git
cd nerd-dictation/
wget https://alphacephei.com/kaldi/models/vosk-model-small-en-us-0.15.zip
unzip vosk-model-small-en-us-0.15.zip
mv vosk-model-small-en-us-0.15 model
Apparently there were other dependencies too:
FileNotFoundError: [Errno 2] No such file or directory: 'parec'
- parec - PulseAudio Recorder, PipeWire command-line utilities for audio capture
sudo apt install pulseaudio-utils
which parec # verification check
# /usr/bin/parec
source virtualenvs/vosk/bin/activate
echo $XDG_SESSION_TYPE
echo $XDG_CURRENT_DESKTOP
Getting nerd-dictation to run on Wayland GNOME desktop proved intractable.
It didn’t generate any output:
./nerd-dictation begin --vosk-model-dir=./model --sample-rate=48000
X11 vs Wayland
Windowing systems (display protocols) manage how graphical applications appear on a computer screen.
X11 is older (from 1984). Wayland is a modern protocol that combines the display server and window manager into a secure compositor.
Window placement is handled entirely within the display server itself (X11) or the compositor (Wayland). Wayland is more secure in that it effectively sandboxes applications. X11 had a global access that allowed one application to intercept keystrokes and window positioning of another.
X11 uses xdotools for handling window/keystroke communication. While Wayland uses ydotools.
xdotool interacts directly with the X11 server protocol.
To bypass Wayland’s display constraints (global interaction is blocked),
ydotool bypasses the display server entirely by talking directly to the Linux kernel’s user input framework (/dev/uinput) to simulate a physical keyboard.
While ydotool works at the kernel level (/dev/uinput), its background daemon (ydotoold) usually runs as root. To use it without typing sudo every time you speak a word into nerd-dictation, you have to grant your user account access to the ydotool control socket or the input group.
This opens up a security loophole wrt the user account being hacked.
There are middle ground alternatives such as like dotool or wtype, but not without their pitfalls.
Why the fuss about xdotool
By default, nerd-dictation tries to inject text using xdotool, but this doesn’t work on Wayland due to security restrictions, so I need ydotool for nerd-dictation to simulate keystrokes on a GNOME Wayland desktop.
So I tried…
sudo apt update && sudo apt install ydotool -y
# ydotool simulates a hardware keyboard, it requires access to the system's user input driver
# Add your user account to the input group
sudo usermod -aG input $USER
# Apply a system rule so your user can write to the virtual device without root permission:
echo 'KERNEL=="uinput", GROUP="input", MODE="0660", OPTIONS+="static_node=uinput"' | sudo tee /etc/udev/rules.d/80-uinput.rules
# Load the virtual input kernel module
sudo modprobe uinput
# log out from GNOME or restart computer to apply group permission above
# launch ydotoold background daemon
ydotoold &
# explicitly flag ydotool as your input simulation driver
./nerd-dictation begin --vosk-model-dir=./model --simulate-input-tool=YDOTOOL
I logged out as instructed but it hung and I cold-booted leading to a corrupted data partition.
Had to install repair tools:
sudo apt update && sudo apt install e2fsprogs ntfs-3g exfatprogs hfsprogs
to enable Repair Partition in Gnome Disks.
Repair seemed to have fixed it, but I lost my appetite for unstable OS and security loopholes.
So I rolled back…
sudo killall ydotoold
sudo rm /etc/udev/rules.d/80-uinput.rules
sudo gpasswd -d $USER input
#Removing my user from group input
sudo apt purge ydotool -y && sudo apt autoremove -y
sudo udevadm control --reload-rules && sudo udevadm trigger
Got rid of nerd-dictation too:
rm -rf ~/devenv/nerd-dictation
rm -rf ~/.config/nerd-dictation # the model files are unpacked here
rm -rf ~/virtualenvs/vosk
# i skipped this last one
sudo apt purge xdotool pulseaudio-utils python3.12-venv libasound2-dev -y
Vosk API standalone
Decided to at least get voice recognition working somehow. But was tripped by driver level misconfig in the end…
python3 -m venv virtualenvs/vosk
source virtualenvs/vosk/bin/activate
pip3 install vosk
cd devenv
git clone https://github.com/alphacep/vosk-api
# --- microphone test dependencies (sounddevice, PortAudio)
pip3 install sounddevice
sudo apt-get install portaudio19-dev # installs: libjack-jackd2-dev libpkgconf3 libportaudio2 libportaudiocpp0 pkg-config pkgconf pkgconf-bin portaudio19-
python3 ./test_simple.py test.wav # downloads vosk-model-small-en-us-0.15.zip (39 MB) and unzips it to: ~/.cache/vosk/
The audio capture on laptop mic still failed.
Debugging
Check capture quality by recording a 10 second wav from the command line:
arecord -d 10 -f cd -t wav sample.wav
Turned out the culprit was some form of driver/config mismatch which again proved intractable.
Laptop mic recording was terribly scratchy.
Airbass Boult headphone over bluetooth – almost as bad.
Headphone plugged in over TRRS jack (audio port) -0 no audio capture except a hiss.
Check sound card levels
alsamixer
Card and Chip: PipeWire – modern audio management that mimics older sound systems as a virtual layer.
Changing hardware gains directly inside alsamixer doesn’t fix the sound issues.
Trying with pavucontrol
sudo apt update && sudo apt install pavucontrol
pavucontrol
Tried to hardcode ALSA kernel module to bypass auto-detection mistakes and manually treat TRRS combo port as a distinct hardware headset mic.
# 1. in system's ALSA base configuration file
sudo nano /etc/modprobe.d/alsa-base.conf
# 2. added this as the last line
options snd-hda-intel model=headset-mic
Finally decided it was not worth the wrangling.