FluidAudio Plugin for NativePHP Mobile#

On-device speech recognition via FluidAudio (Apple Core ML).

iOS 18+ only — the Android bridge functions answer UNSUPPORTED_PLATFORM.

Two modes:

Batch file ASR — initializeAsr() downloads/loads the model pack, then transcribeFile($path) transcribes an audio file.
Mic streaming — startStreaming() loads a Parakeet EOU model and streams partial transcripts from the microphone until stopStreaming().

All calls are non-blocking: they return {accepted, requestId} immediately and deliver results through events correlated by requestId.

Installation#

Copied!

composer require vikas5914/fluidaudio
php artisan native:plugin:register vikas5914/fluidaudio

composer require vikas5914/fluidaudio
php artisan native:plugin:register vikas5914/fluidaudio

Usage (PHP)#

Copied!

use Vikas5914\Fluidaudio\Facades\FluidAudio;
use Vikas5914\Fluidaudio\Models\AsrModel;
use Vikas5914\Fluidaudio\Models\StreamingModel;
 
// Batch ASR: load models, then transcribe a file.
$init = FluidAudio::initializeAsr(['version' => AsrModel::MultilingualV3->value]);
// ... wait for ModelLoadProgress(status: 'ready', progress: 100) ...
$result = FluidAudio::transcribeFile('/path/to/audio.wav');
// ... wait for TranscriptionComplete / TranscriptionError ...
 
// Streaming: start the mic, stop to get the final transcript.
$stream = FluidAudio::startStreaming(['variant' => StreamingModel::Eou160ms->value]);
// ... StreamingStarted / StreamingUpdate(isFinal: false) ... ...
$stop = FluidAudio::stopStreaming();
// ... StreamingUpdate(isFinal: true) carries the final text ...
 
// Housekeeping
$available = FluidAudio::isAsrAvailable(); // bool
$info = FluidAudio::getSystemInfo();       // null off-device
$progress = FluidAudio::modelLoadProgress(); // last known download state, null off-device
FluidAudio::cancel($requestId);            // cancel one in-flight request
FluidAudio::cancel();                      // cancel everything (also stops streaming)
FluidAudio::cleanup();                     // release models and stop audio

use Vikas5914\Fluidaudio\Facades\FluidAudio;
use Vikas5914\Fluidaudio\Models\AsrModel;
use Vikas5914\Fluidaudio\Models\StreamingModel;

// Batch ASR: load models, then transcribe a file.
$init = FluidAudio::initializeAsr(['version' => AsrModel::MultilingualV3->value]);
// ... wait for ModelLoadProgress(status: 'ready', progress: 100) ...
$result = FluidAudio::transcribeFile('/path/to/audio.wav');
// ... wait for TranscriptionComplete / TranscriptionError ...

// Streaming: start the mic, stop to get the final transcript.
$stream = FluidAudio::startStreaming(['variant' => StreamingModel::Eou160ms->value]);
// ... StreamingStarted / StreamingUpdate(isFinal: false) ... ...
$stop = FluidAudio::stopStreaming();
// ... StreamingUpdate(isFinal: true) carries the final text ...

// Housekeeping
$available = FluidAudio::isAsrAvailable(); // bool
$info = FluidAudio::getSystemInfo();       // null off-device
$progress = FluidAudio::modelLoadProgress(); // last known download state, null off-device
FluidAudio::cancel($requestId);            // cancel one in-flight request
FluidAudio::cancel();                      // cancel everything (also stops streaming)
FluidAudio::cleanup();                     // release models and stop audio

initializeAsr() defaults to AsrModel::MultilingualV3 (v3); the other option is AsrModel::EnglishV2 (v2). startStreaming() defaults to StreamingModel::Eou160ms; Eou320ms and Eou1280ms trade latency for accuracy. Invalid values throw InvalidArgumentException, as does an empty path passed to transcribeFile().

Off-device (web/preview, no nativephp_call), the *Accepted methods return a fabricated {accepted: true, requestId} envelope; getSystemInfo() and cleanup() return null, isAsrAvailable() returns false.

Listening for Events#

Copied!

use Native\Mobile\Attributes\OnNative;
use Vikas5914\Fluidaudio\Events\ModelLoadProgress;
use Vikas5914\Fluidaudio\Events\StreamingStarted;
use Vikas5914\Fluidaudio\Events\StreamingUpdate;
use Vikas5914\Fluidaudio\Events\TranscriptionComplete;
use Vikas5914\Fluidaudio\Events\TranscriptionError;
 
class Transcriber extends Component
{
    #[OnNative(ModelLoadProgress::class)]
    public function onProgress(string $status, int $progress, ?string $requestId = null, string $type = 'asr') {}
 
    #[OnNative(TranscriptionComplete::class)]
    public function onComplete(string $text, ?string $requestId = null) {}
 
    #[OnNative(StreamingStarted::class)]
    public function onStarted(?string $requestId = null) {}
 
    #[OnNative(StreamingUpdate::class)]
    public function onUpdate(string $text, string $volatile, string $confirmed, bool $isFinal, ?string $requestId = null) {}
 
    #[OnNative(TranscriptionError::class)]
    public function onError(string $message, ?string $requestId = null, ?string $code = null) {}
}

use Native\Mobile\Attributes\OnNative;
use Vikas5914\Fluidaudio\Events\ModelLoadProgress;
use Vikas5914\Fluidaudio\Events\StreamingStarted;
use Vikas5914\Fluidaudio\Events\StreamingUpdate;
use Vikas5914\Fluidaudio\Events\TranscriptionComplete;
use Vikas5914\Fluidaudio\Events\TranscriptionError;

class Transcriber extends Component
{
    #[OnNative(ModelLoadProgress::class)]
    public function onProgress(string $status, int $progress, ?string $requestId = null, string $type = 'asr') {}

    #[OnNative(TranscriptionComplete::class)]
    public function onComplete(string $text, ?string $requestId = null) {}

    #[OnNative(StreamingStarted::class)]
    public function onStarted(?string $requestId = null) {}

    #[OnNative(StreamingUpdate::class)]
    public function onUpdate(string $text, string $volatile, string $confirmed, bool $isFinal, ?string $requestId = null) {}

    #[OnNative(TranscriptionError::class)]
    public function onError(string $message, ?string $requestId = null, ?string $code = null) {}
}

ModelLoadProgress carries status (downloading / compiling / ready), progress (0–100) and type (asr / streaming). StreamingUpdate carries partial text with confirmed empty until the final update (isFinal: true).

Choosing and Downloading Models#

The plugin ships two separate model packs. List them with models() and let the user pick one of each:

Copied!

$catalog = FluidAudio::models();
// ['asr' => [['id' => 'v2', 'label' => 'English v2'], ['id' => 'v3', ...]],
//  'streaming' => [['id' => 'parakeet-eou-160ms', 'label' => '160 ms', 'latencyMs' => 160], ...]]

$catalog = FluidAudio::models();
// ['asr' => [['id' => 'v2', 'label' => 'English v2'], ['id' => 'v3', ...]],
//  'streaming' => [['id' => 'parakeet-eou-160ms', 'label' => '160 ms', 'latencyMs' => 160], ...]]

Batch pack (asr): v2 (English) or v3 (multilingual). Used for file transcripts. Download it with FluidAudio::initializeAsr(['version' => 'v3']).
Live pack (streaming): Parakeet EOU at 160/320/1280 ms latency. Used for mic streaming. Download it up front with FluidAudio::downloadStreamingModel(['variant' => 'parakeet-eou-160ms']), or skip this — the first startStreaming() downloads it by itself.

Both downloads report through ModelLoadProgress (type is asr or streaming) and stay cached on-device, so each pack downloads once.

Model Download Progress#

Two ways to read progress:

Live events — listen for ModelLoadProgress as above. Best for progress bars while a download runs.
Snapshot function — FluidAudio::modelLoadProgress() returns the last known state without waiting for an event. Best on mount(), e.g. when the user navigates back mid-download:

Copied!

$progress = FluidAudio::modelLoadProgress();
// ['asr' => ['status' => 'downloading', 'progress' => 42, 'requestId' => '...'],
//  'streaming' => ['status' => 'idle', 'progress' => 0]]

$progress = FluidAudio::modelLoadProgress();
// ['asr' => ['status' => 'downloading', 'progress' => 42, 'requestId' => '...'],
//  'streaming' => ['status' => 'idle', 'progress' => 0]]

Each entry carries status (idle / downloading / compiling / ready), progress (0–100) and the requestId it belongs to (absent when idle). Returns null off-device.

Usage (JavaScript)#

For Inertia + Vue/React apps, import the client from resources/js/index.js (published to npm as @vikas5914/fluidaudio):

Copied!

import { fluidAudio } from '@vikas5914/fluidaudio';
 
// Batch ASR
await fluidAudio.initializeAsr({ version: 'v3' });
await fluidAudio.transcribeFile('/path/to/audio.wav');
 
// Streaming
await fluidAudio.downloadStreamingModel({ variant: 'parakeet-eou-160ms' });
await fluidAudio.startStreaming({ variant: 'parakeet-eou-160ms' });
await fluidAudio.stopStreaming();
 
// Housekeeping
await fluidAudio.isAsrAvailable();
await fluidAudio.getSystemInfo();
await fluidAudio.cancel(requestId);
await fluidAudio.cleanup();

import { fluidAudio } from '@vikas5914/fluidaudio';

// Batch ASR
await fluidAudio.initializeAsr({ version: 'v3' });
await fluidAudio.transcribeFile('/path/to/audio.wav');

// Streaming
await fluidAudio.downloadStreamingModel({ variant: 'parakeet-eou-160ms' });
await fluidAudio.startStreaming({ variant: 'parakeet-eou-160ms' });
await fluidAudio.stopStreaming();

// Housekeeping
await fluidAudio.isAsrAvailable();
await fluidAudio.getSystemInfo();
await fluidAudio.cancel(requestId);
await fluidAudio.cleanup();

Every function mirrors a FluidAudio.* bridge call and resolves with the {accepted, requestId} envelope (or the sync payload for reads). Results arrive via native events correlated by requestId.

Permissions#

iOS declares NSMicrophoneUsageDescription and the audio background mode in the manifest. No Android permissions are needed: the Android bridge is an explicit stub until a future release adds on-device support.

Environment Variables#

None. The plugin needs no API keys or secrets.

Support#

Report issues at github.com/Vikas5914/fluidaudio/issues or mail [email protected].

License#

MIT