live-audio¶
第9章 · 多模态与实时交互 · 配套项目
chapter9/live-audio
项目说明¶
Live Voice Chat Demo¶
A real-time voice chat demo featuring speech-to-text, AI conversation, and text-to-speech capabilities. The application supports multiple AI service providers and provides a seamless conversational experience with minimal latency.
This is the companion code for 实验 9-1「构建传统语音 Agent」 in 《深入理解 AI Agent》第 9 章. It implements the cascaded voice pipeline (VAD → ASR → LLM → TTS) discussed there: the frontend captures the microphone and streams audio over a WebSocket; the backend runs Silero VAD to detect end-of-speech (~500 ms of silence), then routes the utterance through pluggable ASR, LLM, and TTS providers and streams synthesized audio back for playback.
Features¶
- 🎤 Real-time voice input with Voice Activity Detection (VAD)
- 🤖 AI-powered conversations with multiple provider support
- 🔊 Text-to-speech synthesis
- ⚡ Low-latency audio streaming
- 📊 Real-time latency monitoring and logging
- 🎯 WebSocket-based communication
- 🔧 Flexible provider selection for ASR, LLM, and TTS services
Supported AI Providers¶
ASR (Automatic Speech Recognition)¶
- OpenAI Whisper: High accuracy, excellent language support
- SenseVoice (via Siliconflow): Low latency, cost-effective, auto language detection
LLM (Large Language Model)¶
- OpenAI GPT-4o: Excellent reasoning, balanced performance
- OpenRouter GPT-4o: No geographic restrictions, unified interface
- OpenRouter Gemini: Fast response, optimized for real-time chat
- ARK Doubao: Low latency in China, optimized for Chinese language
TTS (Text-to-Speech)¶
- CosyVoice2 (via Siliconflow): Natural voice synthesis, multiple system voices
Architecture Overview¶
The system consists of a frontend-backend architecture with real-time audio processing and pluggable provider architecture:
Frontend (Next.js)¶
- Audio Capture: Uses Web Audio API to capture microphone input
- Audio Processing: Client-side audio processing and streaming to backend
- WebSocket Communication: Sends audio stream to backend and receives responses
- Audio Playback: Plays back TTS audio responses from the backend
Backend (Node.js)¶
- WebSocket Server: Handles real-time audio streaming and client connections
- Voice Activity Detection: Server-side Silero VAD processing to detect speech boundaries with high accuracy
- Multi-Provider Support: Flexible ASR, LLM, and TTS provider integration
- Provider Factories: Dynamic provider creation and switching capabilities
Data Flow¶
User Speech → WebSocket → Backend VAD → Multi-Provider STT → Multi-Provider LLM → TTS → Audio Response
Ports¶
| Component | Port | Notes |
|---|---|---|
| Backend (WebSocket server) | 8848 | Set by LISTEN_PORT in backend/config.js. The frontend connects to ws://localhost:8848. |
| Frontend (Next.js dev server) | 3000 | Open http://localhost:3000 in the browser. |
The frontend learns the backend port from the WEBSOCKET_PORT environment variable (see frontend/.env.example). It must match the backend's LISTEN_PORT.
Prerequisites¶
- Node.js (v16 or higher)
- npm or yarn
- FFmpeg - Required for audio processing and format conversion
- Google Chrome (recommended) - Best performance and compatibility for real-time audio
- Not recommended: Safari, Edge, or other browsers due to WebAudio API limitations
- API keys from the supported providers (see Configuration section)
Installing FFmpeg¶
macOS (using Homebrew)¶
Ubuntu/Debian¶
Windows¶
- Download from https://ffmpeg.org/download.html
- Or use Chocolatey:
choco install ffmpeg - Make sure
ffmpegis in your PATH
Project Structure¶
/backend
- server.js: Main WebSocket server with provider integration
- config.js: Multi-provider configuration settings
- utils/
- providers/
- asrProviders.js: ASR provider implementations (OpenAI, Siliconflow)
- llmProviders.js: LLM provider implementations (OpenAI, OpenRouter, ARK)
- vad.js: Voice Activity Detection implementation
- speechToText.js: Provider-aware STT service
- textProcessor.js: Text preprocessing utilities
- tests/
- provider-tests.js: Comprehensive provider testing
- run-tests.js: Test runner with environment validation
- utils/providers/: Provider configuration (ASR / LLM / TTS)
- package.json: Backend dependencies and scripts
/frontend
- pages/: Next.js pages
- index.tsx: Main application interface
- components/: Reusable UI components
- public/: Static assets
- audioWorklet.js: Audio processing and VAD implementation
- next.config.js: Next.js configuration
- tailwind.config.js: Tailwind CSS settings
- package.json: Frontend dependencies and scripts
Installation¶
- Clone the repository
- Install backend dependencies:
- Install frontend dependencies:
- Download the Silero VAD model (already included in this repo at
backend/models/silero_vad.onnx; only needed if missing): - Configure the frontend's WebSocket port (defaults to 8848 if omitted):
After installing, verify your environment (Node version, FFmpeg, VAD model, provider keys) without needing a microphone or browser:
This prints which prerequisites are satisfied and which selected providers have their API keys set. It exits non-zero only if a hard prerequisite (Node < 16, missing FFmpeg, or missing VAD model) is absent.
Configuration¶
Provider-Based Configuration¶
The system now supports multiple AI service providers for maximum flexibility. You can mix and match different providers for ASR, LLM, and TTS services.
1. Environment Variables Setup¶
Set up your API keys as environment variables:
# Required for OpenAI services
export OPENAI_API_KEY="your-openai-api-key"
# Required for OpenRouter services
export OPENROUTER_API_KEY="your-openrouter-api-key"
# Required for ARK (Doubao) services
export ARK_API_KEY="your-ark-api-key"
# Required for Siliconflow services (ASR and TTS)
export SILICONFLOW_API_KEY="your-siliconflow-api-key"
# For future use
export ANTHROPIC_API_KEY="your-anthropic-api-key"
2. Provider Selection¶
-
This repo already ships a ready-to-edit
backend/config.js. If it is missing (e.g. a fresh checkout that ignores it), copy the example first: -
Edit
backend/config.jsto select your preferred providers:const config = { // Provider Selection - Choose your preferred providers ASR_PROVIDER: 'siliconflow', // 'openai' (whisper-1) or 'siliconflow' (SenseVoice) LLM_PROVIDER: 'openrouter', // 'openrouter' (gpt-5.6-luna, default), 'openai', 'openrouter-gemini', 'ark' TTS_PROVIDER: 'siliconflow', // 'siliconflow' (CosyVoice2) // API Keys (loaded from environment variables) OPENAI_API_KEY: process.env.OPENAI_API_KEY, OPENROUTER_API_KEY: process.env.OPENROUTER_API_KEY, ARK_API_KEY: process.env.ARK_API_KEY, SILICONFLOW_API_KEY: process.env.SILICONFLOW_API_KEY, // ... other configuration options };
3. Recommended Provider Combinations¶
Default / Recommended (works anywhere with an OpenRouter key)¶
ASR_PROVIDER: 'siliconflow', // SenseVoice
LLM_PROVIDER: 'openrouter', // openai/gpt-5.6-luna via OpenRouter (avoids gpt-5.6* org verification)
TTS_PROVIDER: 'siliconflow', // CosyVoice2
For Real-time Performance (Low Latency in China)¶
ASR_PROVIDER: 'siliconflow', // SenseVoice
LLM_PROVIDER: 'ark', // Doubao (fast in China); or 'openrouter' for gpt-5.6-luna
TTS_PROVIDER: 'siliconflow', // CosyVoice2
For Best Accuracy¶
ASR_PROVIDER: 'openai', // Accurate Whisper
LLM_PROVIDER: 'openrouter', // openai/gpt-5.6-luna via OpenRouter
TTS_PROVIDER: 'siliconflow' // CosyVoice2
4. API Key Requirements¶
You only need the API keys for the providers you plan to use:
| Provider | ASR | LLM | TTS | Required API Key |
|---|---|---|---|---|
| OpenAI | ✅ Whisper | ✅ gpt-5.6-luna | ❌ | OPENAI_API_KEY |
| OpenRouter | ❌ | ✅ gpt-5.6-luna, Gemini | ❌ | OPENROUTER_API_KEY |
| ARK (Doubao) | ❌ | ✅ Doubao | ❌ | ARK_API_KEY |
| Siliconflow | ✅ SenseVoice | ❌ | ✅ CosyVoice2 | SILICONFLOW_API_KEY |
5. Configuration Validation¶
The system includes comprehensive validation and testing tools:
# Test all configured providers
npm run test:providers
# Run the full test suite with environment validation
node run-tests.js
Legacy Configuration Support¶
The system maintains backward compatibility with the previous hardcoded configuration format, but using the new provider selection is strongly recommended for better flexibility.
Usage¶
-
Set up your API keys (see Configuration section)
-
Configure your preferred providers in
backend/config.js -
(Optional) Verify your setup:
cd backend && npm run check -
Start the backend server (WebSocket server on port 8848):
You should seeServer is running on 0.0.0.0:8848. -
Start the frontend development server (on port 3000):
-
Open http://localhost:3000 in your browser (Chrome recommended)
-
Click "Start Recording" and grant microphone permission to begin a conversation
Expected behavior: after you finish speaking, the backend detects ~500 ms of silence (VAD), transcribes your speech (ASR), streams an LLM reply, and synthesizes it back as audio (TTS) that plays automatically. The on-screen log panel shows per-stage latency (WebSocket RTT, transcription, LLM, TTS). If you start speaking again while the assistant is talking, playback is interrupted.
Testing¶
Provider Testing¶
Test individual providers and all combinations:
cd backend
# Test all providers with your API keys
node run-tests.js
# Test specific providers only
npm run test:providers
# Install test dependencies if needed
npm install
The test suite will automatically skip providers for which you don't have API keys configured.
Test Coverage¶
- ✅ ASR provider functionality (OpenAI Whisper, SenseVoice)
- ✅ LLM provider functionality (OpenAI, OpenRouter GPT-4o, OpenRouter Gemini, ARK Doubao)
- ✅ TTS provider functionality (CosyVoice2 via Siliconflow)
- ✅ All provider combinations (8 ASR+LLM combinations)
- ✅ Dynamic provider switching
- ✅ Error handling and fallback mechanisms
Troubleshooting¶
Common Issues¶
- Missing API Keys: Ensure required environment variables are set
- FFmpeg Not Found: Ensure FFmpeg is installed and available in your system PATH
- Test with:
ffmpeg -version - If not found, refer to the FFmpeg installation instructions above
- Network Issues: Check connectivity to API endpoints
- Rate Limiting: Consider switching providers or implementing retry logic
- Geographic Restrictions: Use OpenRouter for global access
- ONNX Runtime Issues: The backend uses ONNX Runtime for voice activity detection
- Usually resolved by the
onnxruntime-nodepackage automatically - On some systems, you may need additional system libraries
Performance Optimization¶
- Low Latency: Use Siliconflow ASR + OpenRouter Gemini
- High Accuracy: Use OpenAI ASR + OpenAI LLM
- China Deployment: Use Siliconflow ASR + ARK LLM
For provider configuration, see backend/config.js.example and the provider implementations under backend/utils/providers/.
License¶
MIT
源代码¶
backend/package-lock.json¶
{
"name": "livechat-backend",
"version": "1.0.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "livechat-backend",
"version": "1.0.0",
"license": "ISC",
"dependencies": {
"axios": "^1.7.5",
"emoji-regex": "^10.4.0",
"express": "^4.21.1",
"fluent-ffmpeg": "^2.1.3",
"form-data": "^4.0.0",
"franc": "^6.2.0",
"node-wav": "^0.0.2",
"onnxruntime-node": "^1.22.0-rev",
"wav": "^1.0.2",
"ws": "^8.18.0"
},
"devDependencies": {
"mocha": "^10.2.0"
}
},
"node_modules/accepts": {
"version": "1.3.8",
"resolved": "https://registry.npmjs.org/accepts/-/accepts-1.3.8.tgz",
"integrity": "sha512-PYAthTa2m2VKxuvSD3DPC/Gy+U+sOA1LAuT8mkmRuvw+NACSaeXEQ+NHcVF7rONl6qcaxV3Uuemwawk+7+SJLw==",
"license": "MIT",
"dependencies": {
"mime-types": "~2.1.34",
"negotiator": "0.6.3"
},
"engines": {
"node": ">= 0.6"
}
},
"node_modules/adm-zip": {
"version": "0.5.16",
"resolved": "https://registry.npmjs.org/adm-zip/-/adm-zip-0.5.16.tgz",
"integrity": "sha512-TGw5yVi4saajsSEgz25grObGHEUaDrniwvA2qwSC060KfqGPdglhvPMA2lPIoxs3PQIItj2iag35fONcQqgUaQ==",
"license": "MIT",
"engines": {
"node": ">=12.0"
}
},
"node_modules/ansi-colors": {
"version": "4.1.3",
"resolved": "https://registry.npmjs.org/ansi-colors/-/ansi-colors-4.1.3.tgz",
"integrity": "sha512-/6w/C21Pm1A7aZitlI5Ni/2J6FFQN8i1Cvz3kHABAAbw93v/NlvKdVOqz7CCWz/3iv/JplRSEEZ83XION15ovw==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=6"
}
},
"node_modules/ansi-regex": {
"version": "5.0.1",
"resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-5.0.1.tgz",
"integrity": "sha512-quJQXlTSUGL2LH9SUXo8VwsY4soanhgo6LNSm84E1LBcE8s3O0wpdiRzyR9z/ZZJMlMWv37qOOb9pdJlMUEKFQ==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=8"
}
},
"node_modules/ansi-styles": {
"version": "4.3.0",
"resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-4.3.0.tgz",
"integrity": "sha512-zbB9rCJAT1rbjiVDb2hqKFHNYLxgtk8NURxZ3IZwD3F6NtxbXZQCnnSi1Lkx+IDohdPlFp222wVALIheZJQSEg==",
"dev": true,
"license": "MIT",
"dependencies": {
"color-convert": "^2.0.1"
},
"engines": {
"node": ">=8"
},
"funding": {
"url": "https://github.com/chalk/ansi-styles?sponsor=1"
}
},
"node_modules/anymatch": {
"version": "3.1.3",
"resolved": "https://registry.npmjs.org/anymatch/-/anymatch-3.1.3.tgz",
"integrity": "sha512-KMReFUr0B4t+D+OBkjR3KYqvocp2XaSzO55UcB6mgQMd3KbcE+mWTyvVV7D/zsdEbNnV6acZUutkiHQXvTr1Rw==",
"dev": true,
"license": "ISC",
"dependencies": {
"normalize-path": "^3.0.0",
"picomatch": "^2.0.4"
},
"engines": {
"node": ">= 8"
}
},
"node_modules/argparse": {
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/argparse/-/argparse-2.0.1.tgz",
"integrity": "sha512-8+9WqebbFzpX9OR+Wa6O29asIogeRMzcGtAINdpMHHyAg10f05aSFVBbcEqGf/PXw1EjAZ+q2/bEBg3DvurK3Q==",
"dev": true,
"license": "Python-2.0"
},
"node_modules/array-flatten": {
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/array-flatten/-/array-flatten-1.1.1.tgz",
"integrity": "sha512-PCVAQswWemu6UdxsDFFX/+gVeYqKAod3D3UVm91jHwynguOwAvYPhx8nNlM++NqRcK6CxxpUafjmhIdKiHibqg==",
"license": "MIT"
},
"node_modules/async": {
"version": "0.2.10",
"resolved": "https://registry.npmjs.org/async/-/async-0.2.10.tgz",
"integrity": "sha512-eAkdoKxU6/LkKDBzLpT+t6Ff5EtfSF4wx1WfJiPEEV7WNLnDaRXk0oVysiEPm262roaachGexwUv94WhSgN5TQ=="
},
"node_modules/asynckit": {
"version": "0.4.0",
"resolved": "https://registry.npmjs.org/asynckit/-/asynckit-0.4.0.tgz",
"integrity": "sha512-Oei9OH4tRh0YqU3GxhX79dM/mwVgvbZJaSNaRk+bshkj0S5cfHcgYakreBjrHwatXKbz+IoIdYLxrKim2MjW0Q==",
"license": "MIT"
},
"node_modules/axios": {
"version": "1.7.9",
"resolved": "https://registry.npmjs.org/axios/-/axios-1.7.9.tgz",
"integrity": "sha512-LhLcE7Hbiryz8oMDdDptSrWowmB4Bl6RCt6sIJKpRB4XtVf0iEgewX3au/pJqm+Py1kCASkb/FFKjxQaLtxJvw==",
"license": "MIT",
"dependencies": {
"follow-redirects": "^1.15.6",
"form-data": "^4.0.0",
"proxy-from-env": "^1.1.0"
}
},
"node_modules/balanced-match": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-1.0.2.tgz",
"integrity": "sha512-3oSeUO0TMV67hN1AmbXsK4yaqU7tjiHlbxRDZOpH0KW9+CeX4bRAaX0Anxt0tx2MrpRpWwQaPwIlISEJhYU5Pw==",
"dev": true,
"license": "MIT"
},
"node_modules/binary-extensions": {
"version": "2.3.0",
"resolved": "https://registry.npmjs.org/binary-extensions/-/binary-extensions-2.3.0.tgz",
"integrity": "sha512-Ceh+7ox5qe7LJuLHoY0feh3pHuUDHAcRUeyL2VYghZwfpkNIy/+8Ocg0a3UuSoYzavmylwuLWQOf3hl0jjMMIw==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=8"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/body-parser": {
"version": "1.20.3",
"resolved": "https://registry.npmjs.org/body-parser/-/body-parser-1.20.3.tgz",
"integrity": "sha512-7rAxByjUMqQ3/bHJy7D6OGXvx/MMc4IqBn/X0fcM1QUcAItpZrBEYhWGem+tzXH90c+G01ypMcYJBO9Y30203g==",
"license": "MIT",
"dependencies": {
"bytes": "3.1.2",
"content-type": "~1.0.5",
"debug": "2.6.9",
"depd": "2.0.0",
"destroy": "1.2.0",
"http-errors": "2.0.0",
"iconv-lite": "0.4.24",
"on-finished": "2.4.1",
"qs": "6.13.0",
"raw-body": "2.5.2",
"type-is": "~1.6.18",
"unpipe": "1.0.0"
},
"engines": {
"node": ">= 0.8",
"npm": "1.2.8000 || >= 1.4.16"
}
},
"node_modules/boolean": {
"version": "3.2.0",
"resolved": "https://registry.npmjs.org/boolean/-/boolean-3.2.0.tgz",
"integrity": "sha512-d0II/GO9uf9lfUHH2BQsjxzRJZBdsjgsBiW4BvhWk/3qoKwQFjIDVN19PfX8F2D/r9PCMTtLWjYVCFrpeYUzsw==",
"deprecated": "Package no longer supported. Contact Support at https://www.npmjs.com/support for more info.",
"license": "MIT"
},
"node_modules/brace-expansion": {
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.0.2.tgz",
"integrity": "sha512-Jt0vHyM+jmUBqojB7E1NIYadt0vI0Qxjxd2TErW94wDz+E2LAm5vKMXXwg6ZZBTHPuUlDgQHKXvjGBdfcF1ZDQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"balanced-match": "^1.0.0"
}
},
"node_modules/braces": {
"version": "3.0.3",
"resolved": "https://registry.npmjs.org/braces/-/braces-3.0.3.tgz",
"integrity": "sha512-yQbXgO/OSZVD2IsiLlro+7Hf6Q18EJrKSEsdoMzKePKXct3gvD8oLcOQdIzGupr5Fj+EDe8gO/lxc1BzfMpxvA==",
"dev": true,
"license": "MIT",
"dependencies": {
"fill-range": "^7.1.1"
},
"engines": {
"node": ">=8"
}
},
"node_modules/browser-stdout": {
"version": "1.3.1",
"resolved": "https://registry.npmjs.org/browser-stdout/-/browser-stdout-1.3.1.tgz",
"integrity": "sha512-qhAVI1+Av2X7qelOfAIYwXONood6XlZE/fXaBSmW/T5SzLAmCgzi+eiWE7fUvbHaeNBQH13UftjpXxsfLkMpgw==",
"dev": true,
"license": "ISC"
},
"node_modules/buffer-alloc": {
"version": "1.2.0",
"resolved": "https://registry.npmjs.org/buffer-alloc/-/buffer-alloc-1.2.0.tgz",
"integrity": "sha512-CFsHQgjtW1UChdXgbyJGtnm+O/uLQeZdtbDo8mfUgYXCHSM1wgrVxXm6bSyrUuErEb+4sYVGCzASBRot7zyrow==",
"license": "MIT",
"dependencies": {
"buffer-alloc-unsafe": "^1.1.0",
"buffer-fill": "^1.0.0"
}
},
"node_modules/buffer-alloc-unsafe": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/buffer-alloc-unsafe/-/buffer-alloc-unsafe-1.1.0.tgz",
"integrity": "sha512-TEM2iMIEQdJ2yjPJoSIsldnleVaAk1oW3DBVUykyOLsEsFmEc9kn+SFFPz+gl54KQNxlDnAwCXosOS9Okx2xAg==",
"license": "MIT"
},
"node_modules/buffer-fill": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/buffer-fill/-/buffer-fill-1.0.0.tgz",
"integrity": "sha512-T7zexNBwiiaCOGDg9xNX9PBmjrubblRkENuptryuI64URkXDFum9il/JGL8Lm8wYfAXpredVXXZz7eMHilimiQ==",
"license": "MIT"
},
"node_modules/buffer-from": {
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/buffer-from/-/buffer-from-1.1.2.tgz",
"integrity": "sha512-E+XQCRwSbaaiChtv6k6Dwgc+bx+Bs6vuKJHHl5kox/BaKbhiXzqQOwK4cO22yElGp2OCmjwVhT3HmxgyPGnJfQ==",
"license": "MIT"
},
"node_modules/bytes": {
"version": "3.1.2",
"resolved": "https://registry.npmjs.org/bytes/-/bytes-3.1.2.tgz",
"integrity": "sha512-/Nf7TyzTx6S3yRJObOAV7956r8cr2+Oj8AC5dt8wSP3BQAoeX58NoHyCU8P8zGkNXStjTSi6fzO6F0pBdcYbEg==",
"license": "MIT",
"engines": {
"node": ">= 0.8"
}
},
"node_modules/call-bind-apply-helpers": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/call-bind-apply-helpers/-/call-bind-apply-helpers-1.0.1.tgz",
"integrity": "sha512-BhYE+WDaywFg2TBWYNXAE+8B1ATnThNBqXHP5nQu0jWJdVvY2hvkpyB3qOmtmDePiS5/BDQ8wASEWGMWRG148g==",
"license": "MIT",
"dependencies": {
"es-errors": "^1.3.0",
"function-bind": "^1.1.2"
},
"engines": {
"node": ">= 0.4"
}
},
"node_modules/call-bound": {
"version": "1.0.3",
"resolved": "https://registry.npmjs.org/call-bound/-/call-bound-1.0.3.tgz",
"integrity": "sha512-YTd+6wGlNlPxSuri7Y6X8tY2dmm12UMH66RpKMhiX6rsk5wXXnYgbUcOt8kiS31/AjfoTOvCsE+w8nZQLQnzHA==",
"license": "MIT",
"dependencies": {
"call-bind-apply-helpers": "^1.0.1",
"get-intrinsic": "^1.2.6"
},
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/camelcase": {
"version": "6.3.0",
"resolved": "https://registry.npmjs.org/camelcase/-/camelcase-6.3.0.tgz",
"integrity": "sha512-Gmy6FhYlCY7uOElZUSbxo2UCDH8owEk996gkbrpsgGtrJLM3J7jGxl9Ic7Qwwj4ivOE5AWZWRMecDdF7hqGjFA==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=10"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/chalk": {
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/chalk/-/chalk-4.1.2.tgz",
"integrity": "sha512-oKnbhFyRIXpUuez8iBMmyEa4nbj4IOQyuhc/wy9kY7/WVPcwIO9VA668Pu8RkO7+0G76SLROeyw9CpQ061i4mA==",
"dev": true,
"license": "MIT",
"dependencies": {
"ansi-styles": "^4.1.0",
"supports-color": "^7.1.0"
},
"engines": {
"node": ">=10"
},
"funding": {
"url": "https://github.com/chalk/chalk?sponsor=1"
}
},
"node_modules/chalk/node_modules/supports-color": {
"version": "7.2.0",
"resolved": "https://registry.npmjs.org/supports-color/-/supports-color-7.2.0.tgz",
"integrity": "sha512-qpCAvRl9stuOHveKsn7HncJRvv501qIacKzQlO/+Lwxc9+0q2wLyv4Dfvt80/DPn2pqOBsJdDiogXGR9+OvwRw==",
"dev": true,
"license": "MIT",
"dependencies": {
"has-flag": "^4.0.0"
},
"engines": {
"node": ">=8"
}
},
"node_modules/chokidar": {
"version": "3.6.0",
"resolved": "https://registry.npmjs.org/chokidar/-/chokidar-3.6.0.tgz",
"integrity": "sha512-7VT13fmjotKpGipCW9JEQAusEPE+Ei8nl6/g4FBAmIm0GOOLMua9NDDo/DWp0ZAxCr3cPq5ZpBqmPAQgDda2Pw==",
"dev": true,
"license": "MIT",
"dependencies": {
"anymatch": "~3.1.2",
"braces": "~3.0.2",
"glob-parent": "~5.1.2",
"is-binary-path": "~2.1.0",
"is-glob": "~4.0.1",
"normalize-path": "~3.0.0",
"readdirp": "~3.6.0"
},
"engines": {
"node": ">= 8.10.0"
},
"funding": {
"url": "https://paulmillr.com/funding/"
},
"optionalDependencies": {
"fsevents": "~2.3.2"
}
},
"node_modules/cliui": {
"version": "7.0.4",
"resolved": "https://registry.npmjs.org/cliui/-/cliui-7.0.4.tgz",
"integrity": "sha512-OcRE68cOsVMXp1Yvonl/fzkQOyjLSu/8bhPDfQt0e0/Eb283TKP20Fs2MqoPsr9SwA595rRCA+QMzYc9nBP+JQ==",
"dev": true,
"license": "ISC",
"dependencies": {
"string-width": "^4.2.0",
"strip-ansi": "^6.0.0",
"wrap-ansi": "^7.0.0"
}
},
"node_modules/collapse-white-space": {
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/collapse-white-space/-/collapse-white-space-2.1.0.tgz",
"integrity": "sha512-loKTxY1zCOuG4j9f6EPnuyyYkf58RnhhWTvRoZEokgB+WbdXehfjFviyOVYkqzEWz1Q5kRiZdBYS5SwxbQYwzw==",
"license": "MIT",
"funding": {
"type": "github",
"url": "https://github.com/sponsors/wooorm"
}
},
"node_modules/color-convert": {
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/color-convert/-/color-convert-2.0.1.tgz",
"integrity": "sha512-RRECPsj7iu/xb5oKYcsFHSppFNnsj/52OVTRKb4zP5onXwVF3zVmmToNcOfGC+CRDpfK/U584fMg38ZHCaElKQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"color-name": "~1.1.4"
},
"engines": {
"node": ">=7.0.0"
}
},
"node_modules/color-name": {
"version": "1.1.4",
"resolved": "https://registry.npmjs.org/color-name/-/color-name-1.1.4.tgz",
"integrity": "sha512-dOy+3AuW3a2wNbZHIuMZpTcgjGuLU/uBL/ubcZF9OXbDo8ff4O8yVp5Bf0efS8uEoYo5q4Fx7dY9OgQGXgAsQA==",
"dev": true,
"license": "MIT"
},
"node_modules/combined-stream": {
"version": "1.0.8",
"resolved": "https://registry.npmjs.org/combined-stream/-/combined-stream-1.0.8.tgz",
"integrity": "sha512-FQN4MRfuJeHf7cBbBMJFXhKSDq+2kAArBlmRBvcvFE5BB1HZKXtSFASDhdlz9zOYwxh8lDdnvmMOe/+5cdoEdg==",
"license": "MIT",
"dependencies": {
"delayed-stream": "~1.0.0"
},
"engines": {
"node": ">= 0.8"
}
},
"node_modules/content-disposition": {
"version": "0.5.4",
"resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-0.5.4.tgz",
"integrity": "sha512-FveZTNuGw04cxlAiWbzi6zTAL/lhehaWbTtgluJh4/E95DqMwTmha3KZN1aAWA8cFIhHzMZUvLevkw5Rqk+tSQ==",
"license": "MIT",
"dependencies": {
"safe-buffer": "5.2.1"
},
"engines": {
"node": ">= 0.6"
}
},
"node_modules/content-type": {
"version": "1.0.5",
"resolved": "https://registry.npmjs.org/content-type/-/content-type-1.0.5.tgz",
"integrity": "sha512-nTjqfcBFEipKdXCv4YDQWCfmcLZKm81ldF0pAopTvyrFGVbcR6P/VAAd5G7N+0tTr8QqiU0tFadD6FK4NtJwOA==",
"license": "MIT",
"engines": {
"node": ">= 0.6"
}
},
"node_modules/cookie": {
"version": "0.7.1",
"resolved": "https://registry.npmjs.org/cookie/-/cookie-0.7.1.tgz",
"integrity": "sha512-6DnInpx7SJ2AK3+CTUE/ZM0vWTUboZCegxhC2xiIydHR9jNuTAASBrfEpHhiGOZw/nX51bHt6YQl8jsGo4y/0w==",
"license": "MIT",
"engines": {
"node": ">= 0.6"
}
},
"node_modules/cookie-signature": {
"version": "1.0.6",
"resolved": "https://registry.npmjs.org/cookie-signature/-/cookie-signature-1.0.6.tgz",
"integrity": "sha512-QADzlaHc8icV8I7vbaJXJwod9HWYp8uCqf1xa4OfNu1T7JVxQIrUgOWtHdNDtPiywmFbiS12VjotIXLrKM3orQ==",
"license": "MIT"
},
"node_modules/core-util-is": {
"version": "1.0.3",
"resolved": "https://registry.npmjs.org/core-util-is/-/core-util-is-1.0.3.tgz",
"integrity": "sha512-ZQBvi1DcpJ4GDqanjucZ2Hj3wEO5pZDS89BWbkcrvdxksJorwUDDZamX9ldFkp9aw2lmBDLgkObEA4DWNJ9FYQ==",
"license": "MIT"
},
"node_modules/debug": {
"version": "2.6.9",
"resolved": "https://registry.npmjs.org/debug/-/debug-2.6.9.tgz",
"integrity": "sha512-bC7ElrdJaJnPbAP+1EotYvqZsb3ecl5wi6Bfi6BJTUcNowp6cvspg0jXznRTKDjm/E7AdgFBVeAPVMNcKGsHMA==",
"license": "MIT",
"dependencies": {
"ms": "2.0.0"
}
},
"node_modules/decamelize": {
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/decamelize/-/decamelize-4.0.0.tgz",
"integrity": "sha512-9iE1PgSik9HeIIw2JO94IidnE3eBoQrFJ3w7sFuzSX4DpmZ3v5sZpUiV5Swcf6mQEF+Y0ru8Neo+p+nyh2J+hQ==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=10"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/define-data-property": {
"version": "1.1.4",
"resolved": "https://registry.npmjs.org/define-data-property/-/define-data-property-1.1.4.tgz",
"integrity": "sha512-rBMvIzlpA8v6E+SJZoo++HAYqsLrkg7MSfIinMPFhmkorw7X+dOXVJQs+QT69zGkzMyfDnIMN2Wid1+NbL3T+A==",
"license": "MIT",
"dependencies": {
"es-define-property": "^1.0.0",
"es-errors": "^1.3.0",
"gopd": "^1.0.1"
},
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/define-properties": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/define-properties/-/define-properties-1.2.1.tgz",
"integrity": "sha512-8QmQKqEASLd5nx0U1B1okLElbUuuttJ/AnYmRXbbbGDWh6uS208EjD4Xqq/I9wK7u0v6O08XhTWnt5XtEbR6Dg==",
"license": "MIT",
"dependencies": {
"define-data-property": "^1.0.1",
"has-property-descriptors": "^1.0.0",
"object-keys": "^1.1.1"
},
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/delayed-stream": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/delayed-stream/-/delayed-stream-1.0.0.tgz",
"integrity": "sha512-ZySD7Nf91aLB0RxL4KGrKHBXl7Eds1DAmEdcoVawXnLD7SDhpNgtuII2aAkg7a7QS41jxPSZ17p4VdGnMHk3MQ==",
"license": "MIT",
"engines": {
"node": ">=0.4.0"
}
},
"node_modules/depd": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/depd/-/depd-2.0.0.tgz",
"integrity": "sha512-g7nH6P6dyDioJogAAGprGpCtVImJhpPk/roCzdb3fIh61/s/nPsfR6onyMwkCAR/OlC3yBC0lESvUoQEAssIrw==",
"license": "MIT",
"engines": {
"node": ">= 0.8"
}
},
"node_modules/destroy": {
"version": "1.2.0",
"resolved": "https://registry.npmjs.org/destroy/-/destroy-1.2.0.tgz",
"integrity": "sha512-2sJGJTaXIIaR1w4iJSNoN0hnMY7Gpc/n8D4qSCJw8QqFWXf7cuAgnEHxBpweaVcPevC2l3KpjYCx3NypQQgaJg==",
"license": "MIT",
"engines": {
"node": ">= 0.8",
"npm": "1.2.8000 || >= 1.4.16"
}
},
"node_modules/detect-node": {
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/detect-node/-/detect-node-2.1.0.tgz",
"integrity": "sha512-T0NIuQpnTvFDATNuHN5roPwSBG83rFsuO+MXXH9/3N1eFbn4wcPjttvjMLEPWJ0RGUYgQE7cGgS3tNxbqCGM7g==",
"license": "MIT"
},
"node_modules/diff": {
"version": "5.2.0",
"resolved": "https://registry.npmjs.org/diff/-/diff-5.2.0.tgz",
"integrity": "sha512-uIFDxqpRZGZ6ThOk84hEfqWoHx2devRFvpTZcTHur85vImfaxUbTW9Ryh4CpCuDnToOP1CEtXKIgytHBPVff5A==",
"dev": true,
"license": "BSD-3-Clause",
"engines": {
"node": ">=0.3.1"
}
},
"node_modules/dunder-proto": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/dunder-proto/-/dunder-proto-1.0.1.tgz",
"integrity": "sha512-KIN/nDJBQRcXw0MLVhZE9iQHmG68qAVIBg9CqmUYjmQIhgij9U5MFvrqkUL5FbtyyzZuOeOt0zdeRe4UY7ct+A==",
"license": "MIT",
"dependencies": {
"call-bind-apply-helpers": "^1.0.1",
"es-errors": "^1.3.0",
"gopd": "^1.2.0"
},
"engines": {
"node": ">= 0.4"
}
},
"node_modules/ee-first": {
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/ee-first/-/ee-first-1.1.1.tgz",
"integrity": "sha512-WMwm9LhRUo+WUaRN+vRuETqG89IgZphVSNkdFgeb6sS/E4OrDIN7t48CAewSHXc6C8lefD8KKfr5vY61brQlow==",
"license": "MIT"
},
"node_modules/emoji-regex": {
"version": "10.4.0",
"resolved": "https://registry.npmjs.org/emoji-regex/-/emoji-regex-10.4.0.tgz",
"integrity": "sha512-EC+0oUMY1Rqm4O6LLrgjtYDvcVYTy7chDnM4Q7030tP4Kwj3u/pR6gP9ygnp2CJMK5Gq+9Q2oqmrFJAz01DXjw==",
"license": "MIT"
},
"node_modules/encodeurl": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/encodeurl/-/encodeurl-2.0.0.tgz",
"integrity": "sha512-Q0n9HRi4m6JuGIV1eFlmvJB7ZEVxu93IrMyiMsGC0lrMJMWzRgx6WGquyfQgZVb31vhGgXnfmPNNXmxnOkRBrg==",
"license": "MIT",
"engines": {
"node": ">= 0.8"
}
},
"node_modules/es-define-property": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/es-define-property/-/es-define-property-1.0.1.tgz",
"integrity": "sha512-e3nRfgfUZ4rNGL232gUgX06QNyyez04KdjFrF+LTRoOXmrOgFKDg4BCdsjW8EnT69eqdYGmRpJwiPVYNrCaW3g==",
"license": "MIT",
"engines": {
"node": ">= 0.4"
}
},
"node_modules/es-errors": {
"version": "1.3.0",
"resolved": "https://registry.npmjs.org/es-errors/-/es-errors-1.3.0.tgz",
"integrity": "sha512-Zf5H2Kxt2xjTvbJvP2ZWLEICxA6j+hAmMzIlypy4xcBg1vKVnx89Wy0GbS+kf5cwCVFFzdCFh2XSCFNULS6csw==",
"license": "MIT",
"engines": {
"node": ">= 0.4"
}
},
"node_modules/es-object-atoms": {
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/es-object-atoms/-/es-object-atoms-1.1.1.tgz",
"integrity": "sha512-FGgH2h8zKNim9ljj7dankFPcICIK9Cp5bm+c2gQSYePhpaG5+esrLODihIorn+Pe6FGJzWhXQotPv73jTaldXA==",
"license": "MIT",
"dependencies": {
"es-errors": "^1.3.0"
},
"engines": {
"node": ">= 0.4"
}
},
"node_modules/es6-error": {
"version": "4.1.1",
"resolved": "https://registry.npmjs.org/es6-error/-/es6-error-4.1.1.tgz",
"integrity": "sha512-Um/+FxMr9CISWh0bi5Zv0iOD+4cFh5qLeks1qhAopKVAJw3drgKbKySikp7wGhDL0HPeaja0P5ULZrxLkniUVg==",
"license": "MIT"
},
"node_modules/escalade": {
"version": "3.2.0",
"resolved": "https://registry.npmjs.org/escalade/-/escalade-3.2.0.tgz",
"integrity": "sha512-WUj2qlxaQtO4g6Pq5c29GTcWGDyd8itL8zTlipgECz3JesAiiOKotd8JU6otB3PACgG6xkJUyVhboMS+bje/jA==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=6"
}
},
"node_modules/escape-html": {
"version": "1.0.3",
"resolved": "https://registry.npmjs.org/escape-html/-/escape-html-1.0.3.tgz",
"integrity": "sha512-NiSupZ4OeuGwr68lGIeym/ksIZMJodUGOSCZ/FSnTxcrekbvqrgdUxlJOMpijaKZVjAJrWrGs/6Jy8OMuyj9ow==",
"license": "MIT"
},
"node_modules/escape-string-regexp": {
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/escape-string-regexp/-/escape-string-regexp-4.0.0.tgz",
"integrity": "sha512-TtpcNJ3XAzx3Gq8sWRzJaVajRs0uVxA2YAkdb1jm2YkPz4G6egUFAyA3n5vtEIZefPk5Wa4UXbKuS5fKkJWdgA==",
"license": "MIT",
"engines": {
"node": ">=10"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/etag": {
"version": "1.8.1",
"resolved": "https://registry.npmjs.org/etag/-/etag-1.8.1.tgz",
"integrity": "sha512-aIL5Fx7mawVa300al2BnEE4iNvo1qETxLrPI/o05L7z6go7fCw1J6EQmbK4FmJ2AS7kgVF/KEZWufBfdClMcPg==",
"license": "MIT",
"engines": {
"node": ">= 0.6"
}
},
"node_modules/express": {
"version": "4.21.2",
"resolved": "https://registry.npmjs.org/express/-/express-4.21.2.tgz",
"integrity": "sha512-28HqgMZAmih1Czt9ny7qr6ek2qddF4FclbMzwhCREB6OFfH+rXAnuNCwo1/wFvrtbgsQDb4kSbX9de9lFbrXnA==",
"license": "MIT",
"dependencies": {
"accepts": "~1.3.8",
"array-flatten": "1.1.1",
"body-parser": "1.20.3",
"content-disposition": "0.5.4",
"content-type": "~1.0.4",
"cookie": "0.7.1",
"cookie-signature": "1.0.6",
"debug": "2.6.9",
"depd": "2.0.0",
"encodeurl": "~2.0.0",
"escape-html": "~1.0.3",
"etag": "~1.8.1",
"finalhandler": "1.3.1",
"fresh": "0.5.2",
"http-errors": "2.0.0",
"merge-descriptors": "1.0.3",
"methods": "~1.1.2",
"on-finished": "2.4.1",
"parseurl": "~1.3.3",
"path-to-regexp": "0.1.12",
"proxy-addr": "~2.0.7",
"qs": "6.13.0",
"range-parser": "~1.2.1",
"safe-buffer": "5.2.1",
"send": "0.19.0",
"serve-static": "1.16.2",
"setprototypeof": "1.2.0",
"statuses": "2.0.1",
"type-is": "~1.6.18",
"utils-merge": "1.0.1",
"vary": "~1.1.2"
},
"engines": {
"node": ">= 0.10.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/express"
}
},
"node_modules/fill-range": {
"version": "7.1.1",
"resolved": "https://registry.npmjs.org/fill-range/-/fill-range-7.1.1.tgz",
"integrity": "sha512-YsGpe3WHLK8ZYi4tWDg2Jy3ebRz2rXowDxnld4bkQB00cc/1Zw9AWnC0i9ztDJitivtQvaI9KaLyKrc+hBW0yg==",
"dev": true,
"license": "MIT",
"dependencies": {
"to-regex-range": "^5.0.1"
},
"engines": {
"node": ">=8"
}
},
"node_modules/finalhandler": {
"version": "1.3.1",
"resolved": "https://registry.npmjs.org/finalhandler/-/finalhandler-1.3.1.tgz",
"integrity": "sha512-6BN9trH7bp3qvnrRyzsBz+g3lZxTNZTbVO2EV1CS0WIcDbawYVdYvGflME/9QP0h0pYlCDBCTjYa9nZzMDpyxQ==",
"license": "MIT",
"dependencies": {
"debug": "2.6.9",
"encodeurl": "~2.0.0",
"escape-html": "~1.0.3",
"on-finished": "2.4.1",
"parseurl": "~1.3.3",
"statuses": "2.0.1",
"unpipe": "~1.0.0"
},
"engines": {
"node": ">= 0.8"
}
},
"node_modules/find-up": {
"version": "5.0.0",
"resolved": "https://registry.npmjs.org/find-up/-/find-up-5.0.0.tgz",
"integrity": "sha512-78/PXT1wlLLDgTzDs7sjq9hzz0vXD+zn+7wypEe4fXQxCmdmqfGsEPQxmiCSQI3ajFV91bVSsvNtrJRiW6nGng==",
"dev": true,
"license": "MIT",
"dependencies": {
"locate-path": "^6.0.0",
"path-exists": "^4.0.0"
},
"engines": {
"node": ">=10"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/flat": {
"version": "5.0.2",
"resolved": "https://registry.npmjs.org/flat/-/flat-5.0.2.tgz",
"integrity": "sha512-b6suED+5/3rTpUBdG1gupIl8MPFCAMA0QXwmljLhvCUKcUvdE4gWky9zpuGCcXHOsz4J9wPGNWq6OKpmIzz3hQ==",
"dev": true,
"license": "BSD-3-Clause",
"bin": {
"flat": "cli.js"
}
},
"node_modules/fluent-ffmpeg": {
"version": "2.1.3",
"resolved": "https://registry.npmjs.org/fluent-ffmpeg/-/fluent-ffmpeg-2.1.3.tgz",
"integrity": "sha512-Be3narBNt2s6bsaqP6Jzq91heDgOEaDCJAXcE3qcma/EJBSy5FB4cvO31XBInuAuKBx8Kptf8dkhjK0IOru39Q==",
"license": "MIT",
"dependencies": {
"async": "^0.2.9",
"which": "^1.1.1"
},
"engines": {
"node": ">=18"
}
},
"node_modules/follow-redirects": {
"version": "1.15.9",
"resolved": "https://registry.npmjs.org/follow-redirects/-/follow-redirects-1.15.9.tgz",
"integrity": "sha512-gew4GsXizNgdoRyqmyfMHyAmXsZDk6mHkSxZFCzW9gwlbtOW44CDtYavM+y+72qD/Vq2l550kMF52DT8fOLJqQ==",
"funding": [
{
"type": "individual",
"url": "https://github.com/sponsors/RubenVerborgh"
}
],
"license": "MIT",
"engines": {
"node": ">=4.0"
},
"peerDependenciesMeta": {
"debug": {
"optional": true
}
}
},
"node_modules/form-data": {
"version": "4.0.1",
"resolved": "https://registry.npmjs.org/form-data/-/form-data-4.0.1.tgz",
"integrity": "sha512-tzN8e4TX8+kkxGPK8D5u0FNmjPUjw3lwC9lSLxxoB/+GtsJG91CO8bSWy73APlgAZzZbXEYZJuxjkHH2w+Ezhw==",
"license": "MIT",
"dependencies": {
"asynckit": "^0.4.0",
"combined-stream": "^1.0.8",
"mime-types": "^2.1.12"
},
"engines": {
"node": ">= 6"
}
},
"node_modules/forwarded": {
"version": "0.2.0",
"resolved": "https://registry.npmjs.org/forwarded/-/forwarded-0.2.0.tgz",
"integrity": "sha512-buRG0fpBtRHSTCOASe6hD258tEubFoRLb4ZNA6NxMVHNw2gOcwHo9wyablzMzOA5z9xA9L1KNjk/Nt6MT9aYow==",
"license": "MIT",
"engines": {
"node": ">= 0.6"
}
},
"node_modules/franc": {
"version": "6.2.0",
"resolved": "https://registry.npmjs.org/franc/-/franc-6.2.0.tgz",
"integrity": "sha512-rcAewP7PSHvjq7Kgd7dhj82zE071kX5B4W1M4ewYMf/P+i6YsDQmj62Xz3VQm9zyUzUXwhIde/wHLGCMrM+yGg==",
"license": "MIT",
"dependencies": {
"trigram-utils": "^2.0.0"
},
"funding": {
"type": "github",
"url": "https://github.com/sponsors/wooorm"
}
},
"node_modules/fresh": {
"version": "0.5.2",
"resolved": "https://registry.npmjs.org/fresh/-/fresh-0.5.2.tgz",
"integrity": "sha512-zJ2mQYM18rEFOudeV4GShTGIQ7RbzA7ozbU9I/XBpm7kqgMywgmylMwXHxZJmkVoYkna9d2pVXVXPdYTP9ej8Q==",
"license": "MIT",
"engines": {
"node": ">= 0.6"
}
},
"node_modules/fs.realpath": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/fs.realpath/-/fs.realpath-1.0.0.tgz",
"integrity": "sha512-OO0pH2lK6a0hZnAdau5ItzHPI6pUlvI7jMVnxUQRtw4owF2wk8lOSabtGDCTP4Ggrg2MbGnWO9X8K1t4+fGMDw==",
"dev": true,
"license": "ISC"
},
"node_modules/fsevents": {
"version": "2.3.3",
"resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz",
"integrity": "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==",
"dev": true,
"hasInstallScript": true,
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": "^8.16.0 || ^10.6.0 || >=11.0.0"
}
},
"node_modules/function-bind": {
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/function-bind/-/function-bind-1.1.2.tgz",
"integrity": "sha512-7XHNxH7qX9xG5mIwxkhumTox/MIRNcOgDrxWsMt2pAr23WHp6MrRlN7FBSFpCpr+oVO0F744iUgR82nJMfG2SA==",
"license": "MIT",
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/get-caller-file": {
"version": "2.0.5",
"resolved": "https://registry.npmjs.org/get-caller-file/-/get-caller-file-2.0.5.tgz",
"integrity": "sha512-DyFP3BM/3YHTQOCUL/w0OZHR0lpKeGrxotcHWcqNEdnltqFwXVfhEBQ94eIo34AfQpo0rGki4cyIiftY06h2Fg==",
"dev": true,
"license": "ISC",
"engines": {
"node": "6.* || 8.* || >= 10.*"
}
},
"node_modules/get-intrinsic": {
"version": "1.2.7",
"resolved": "https://registry.npmjs.org/get-intrinsic/-/get-intrinsic-1.2.7.tgz",
"integrity": "sha512-VW6Pxhsrk0KAOqs3WEd0klDiF/+V7gQOpAvY1jVU/LHmaD/kQO4523aiJuikX/QAKYiW6x8Jh+RJej1almdtCA==",
"license": "MIT",
"dependencies": {
"call-bind-apply-helpers": "^1.0.1",
"es-define-property": "^1.0.1",
"es-errors": "^1.3.0",
"es-object-atoms": "^1.0.0",
"function-bind": "^1.1.2",
"get-proto": "^1.0.0",
"gopd": "^1.2.0",
"has-symbols": "^1.1.0",
"hasown": "^2.0.2",
"math-intrinsics": "^1.1.0"
},
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/get-proto": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/get-proto/-/get-proto-1.0.1.tgz",
"integrity": "sha512-sTSfBjoXBp89JvIKIefqw7U2CCebsc74kiY6awiGogKtoSGbgjYE/G/+l9sF3MWFPNc9IcoOC4ODfKHfxFmp0g==",
"license": "MIT",
"dependencies": {
"dunder-proto": "^1.0.1",
"es-object-atoms": "^1.0.0"
},
"engines": {
"node": ">= 0.4"
}
},
"node_modules/glob": {
"version": "8.1.0",
"resolved": "https://registry.npmjs.org/glob/-/glob-8.1.0.tgz",
"integrity": "sha512-r8hpEjiQEYlF2QU0df3dS+nxxSIreXQS1qRhMJM0Q5NDdR386C7jb7Hwwod8Fgiuex+k0GFjgft18yvxm5XoCQ==",
"deprecated": "Glob versions prior to v9 are no longer supported",
"dev": true,
"license": "ISC",
"dependencies": {
"fs.realpath": "^1.0.0",
"inflight": "^1.0.4",
"inherits": "2",
"minimatch": "^5.0.1",
"once": "^1.3.0"
},
"engines": {
"node": ">=12"
},
"funding": {
"url": "https://github.com/sponsors/isaacs"
}
},
"node_modules/glob-parent": {
"version": "5.1.2",
"resolved": "https://registry.npmjs.org/glob-parent/-/glob-parent-5.1.2.tgz",
"integrity": "sha512-AOIgSQCepiJYwP3ARnGx+5VnTu2HBYdzbGP45eLw1vr3zB3vZLeyed1sC9hnbcOc9/SrMyM5RPQrkGz4aS9Zow==",
"dev": true,
"license": "ISC",
"dependencies": {
"is-glob": "^4.0.1"
},
"engines": {
"node": ">= 6"
}
},
"node_modules/global-agent": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/global-agent/-/global-agent-3.0.0.tgz",
"integrity": "sha512-PT6XReJ+D07JvGoxQMkT6qji/jVNfX/h364XHZOWeRzy64sSFr+xJ5OX7LI3b4MPQzdL4H8Y8M0xzPpsVMwA8Q==",
"license": "BSD-3-Clause",
"dependencies": {
"boolean": "^3.0.1",
"es6-error": "^4.1.1",
"matcher": "^3.0.0",
"roarr": "^2.15.3",
"semver": "^7.3.2",
"serialize-error": "^7.0.1"
},
"engines": {
"node": ">=10.0"
}
},
"node_modules/globalthis": {
"version": "1.0.4",
"resolved": "https://registry.npmjs.org/globalthis/-/globalthis-1.0.4.tgz",
"integrity": "sha512-DpLKbNU4WylpxJykQujfCcwYWiV/Jhm50Goo0wrVILAv5jOr9d+H+UR3PhSCD2rCCEIg0uc+G+muBTwD54JhDQ==",
"license": "MIT",
"dependencies": {
"define-properties": "^1.2.1",
"gopd": "^1.0.1"
},
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/gopd": {
"version": "1.2.0",
"resolved": "https://registry.npmjs.org/gopd/-/gopd-1.2.0.tgz",
"integrity": "sha512-ZUKRh6/kUFoAiTAtTYPZJ3hw9wNxx+BIBOijnlG9PnrJsCcSjs1wyyD6vJpaYtgnzDrKYRSqf3OO6Rfa93xsRg==",
"license": "MIT",
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/has-flag": {
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/has-flag/-/has-flag-4.0.0.tgz",
"integrity": "sha512-EykJT/Q1KjTWctppgIAgfSO0tKVuZUjhgMr17kqTumMl6Afv3EISleU7qZUzoXDFTAHTDC4NOoG/ZxU3EvlMPQ==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=8"
}
},
"node_modules/has-property-descriptors": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/has-property-descriptors/-/has-property-descriptors-1.0.2.tgz",
"integrity": "sha512-55JNKuIW+vq4Ke1BjOTjM2YctQIvCT7GFzHwmfZPGo5wnrgkid0YQtnAleFSqumZm4az3n2BS+erby5ipJdgrg==",
"license": "MIT",
"dependencies": {
"es-define-property": "^1.0.0"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/has-symbols": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/has-symbols/-/has-symbols-1.1.0.tgz",
"integrity": "sha512-1cDNdwJ2Jaohmb3sg4OmKaMBwuC48sYni5HUw2DvsC8LjGTLK9h+eb1X6RyuOHe4hT0ULCW68iomhjUoKUqlPQ==",
"license": "MIT",
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/hasown": {
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/hasown/-/hasown-2.0.2.tgz",
"integrity": "sha512-0hJU9SCPvmMzIBdZFqNPXWa6dqh7WdH0cII9y+CyS8rG3nL48Bclra9HmKhVVUHyPWNH5Y7xDwAB7bfgSjkUMQ==",
"license": "MIT",
"dependencies": {
"function-bind": "^1.1.2"
},
"engines": {
"node": ">= 0.4"
}
},
"node_modules/he": {
"version": "1.2.0",
"resolved": "https://registry.npmjs.org/he/-/he-1.2.0.tgz",
"integrity": "sha512-F/1DnUGPopORZi0ni+CvrCgHQ5FyEAHRLSApuYWMmrbSwoN2Mn/7k+Gl38gJnR7yyDZk6WLXwiGod1JOWNDKGw==",
"dev": true,
"license": "MIT",
"bin": {
"he": "bin/he"
}
},
"node_modules/http-errors": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/http-errors/-/http-errors-2.0.0.tgz",
"integrity": "sha512-FtwrG/euBzaEjYeRqOgly7G0qviiXoJWnvEH2Z1plBdXgbyjv34pHTSb9zoeHMyDy33+DWy5Wt9Wo+TURtOYSQ==",
"license": "MIT",
"dependencies": {
"depd": "2.0.0",
"inherits": "2.0.4",
"setprototypeof": "1.2.0",
"statuses": "2.0.1",
"toidentifier": "1.0.1"
},
"engines": {
"node": ">= 0.8"
}
},
"node_modules/iconv-lite": {
"version": "0.4.24",
"resolved": "https://registry.npmjs.org/iconv-lite/-/iconv-lite-0.4.24.tgz",
"integrity": "sha512-v3MXnZAcvnywkTUEZomIActle7RXXeedOR31wwl7VlyoXO4Qi9arvSenNQWne1TcRwhCL1HwLI21bEqdpj8/rA==",
"license": "MIT",
"dependencies": {
"safer-buffer": ">= 2.1.2 < 3"
},
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/inflight": {
"version": "1.0.6",
"resolved": "https://registry.npmjs.org/inflight/-/inflight-1.0.6.tgz",
"integrity": "sha512-k92I/b08q4wvFscXCLvqfsHCrjrF7yiXsQuIVvVE7N82W3+aqpzuUdBbfhWcy/FZR3/4IgflMgKLOsvPDrGCJA==",
"deprecated": "This module is not supported, and leaks memory. Do not use it. Check out lru-cache if you want a good and tested way to coalesce async requests by a key value, which is much more comprehensive and powerful.",
"dev": true,
"license": "ISC",
"dependencies": {
"once": "^1.3.0",
"wrappy": "1"
}
},
"node_modules/inherits": {
"version": "2.0.4",
"resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.4.tgz",
"integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==",
"license": "ISC"
},
"node_modules/ipaddr.js": {
"version": "1.9.1",
"resolved": "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-1.9.1.tgz",
"integrity": "sha512-0KI/607xoxSToH7GjN1FfSbLoU0+btTicjsQSWQlh/hZykN8KpmMf7uYwPW3R+akZ6R/w18ZlXSHBYXiYUPO3g==",
"license": "MIT",
"engines": {
"node": ">= 0.10"
}
},
"node_modules/is-binary-path": {
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/is-binary-path/-/is-binary-path-2.1.0.tgz",
"integrity": "sha512-ZMERYes6pDydyuGidse7OsHxtbI7WVeUEozgR/g7rd0xUimYNlvZRE/K2MgZTjWy725IfelLeVcEM97mmtRGXw==",
"dev": true,
"license": "MIT",
"dependencies": {
"binary-extensions": "^2.0.0"
},
"engines": {
"node": ">=8"
}
},
"node_modules/is-extglob": {
"version": "2.1.1",
"resolved": "https://registry.npmjs.org/is-extglob/-/is-extglob-2.1.1.tgz",
"integrity": "sha512-SbKbANkN603Vi4jEZv49LeVJMn4yGwsbzZworEoyEiutsN3nJYdbO36zfhGJ6QEDpOZIFkDtnq5JRxmvl3jsoQ==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/is-fullwidth-code-point": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/is-fullwidth-code-point/-/is-fullwidth-code-point-3.0.0.tgz",
"integrity": "sha512-zymm5+u+sCsSWyD9qNaejV3DFvhCKclKdizYaJUuHA83RLjb7nSuGnddCHGv0hk+KY7BMAlsWeK4Ueg6EV6XQg==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=8"
}
},
"node_modules/is-glob": {
"version": "4.0.3",
"resolved": "https://registry.npmjs.org/is-glob/-/is-glob-4.0.3.tgz",
"integrity": "sha512-xelSayHH36ZgE7ZWhli7pW34hNbNl8Ojv5KVmkJD4hBdD3th8Tfk9vYasLM+mXWOZhFkgZfxhLSnrwRr4elSSg==",
"dev": true,
"license": "MIT",
"dependencies": {
"is-extglob": "^2.1.1"
},
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/is-number": {
"version": "7.0.0",
"resolved": "https://registry.npmjs.org/is-number/-/is-number-7.0.0.tgz",
"integrity": "sha512-41Cifkg6e8TylSpdtTpeLVMqvSBEVzTttHvERD741+pnZ8ANv0004MRL43QKPDlK9cGvNp6NZWZUBlbGXYxxng==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=0.12.0"
}
},
"node_modules/is-plain-obj": {
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/is-plain-obj/-/is-plain-obj-2.1.0.tgz",
"integrity": "sha512-YWnfyRwxL/+SsrWYfOpUtz5b3YD+nyfkHvjbcanzk8zgyO4ASD67uVMRt8k5bM4lLMDnXfriRhOpemw+NfT1eA==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=8"
}
},
"node_modules/is-unicode-supported": {
"version": "0.1.0",
"resolved": "https://registry.npmjs.org/is-unicode-supported/-/is-unicode-supported-0.1.0.tgz",
"integrity": "sha512-knxG2q4UC3u8stRGyAVJCOdxFmv5DZiRcdlIaAQXAbSfJya+OhopNotLQrstBhququ4ZpuKbDc/8S6mgXgPFPw==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=10"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/isarray": {
"version": "0.0.1",
"resolved": "https://registry.npmjs.org/isarray/-/isarray-0.0.1.tgz",
"integrity": "sha512-D2S+3GLxWH+uhrNEcoh/fnmYeP8E8/zHl644d/jdA0g2uyXvy3sb0qxotE+ne0LtccHknQzWwZEzhak7oJ0COQ==",
"license": "MIT"
},
"node_modules/isexe": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/isexe/-/isexe-2.0.0.tgz",
"integrity": "sha512-RHxMLp9lnKHGHRng9QFhRCMbYAcVpn69smSGcq3f36xjgVVWThj4qqLbTLlq7Ssj8B+fIQ1EuCEGI2lKsyQeIw==",
"license": "ISC"
},
"node_modules/js-yaml": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.1.0.tgz",
"integrity": "sha512-wpxZs9NoxZaJESJGIZTyDEaYpl0FKSA+FB9aJiyemKhMwkxQg63h4T1KJgUGHpTqPDNRcmmYLugrRjJlBtWvRA==",
"dev": true,
"license": "MIT",
"dependencies": {
"argparse": "^2.0.1"
},
"bin": {
"js-yaml": "bin/js-yaml.js"
}
},
"node_modules/json-stringify-safe": {
"version": "5.0.1",
"resolved": "https://registry.npmjs.org/json-stringify-safe/-/json-stringify-safe-5.0.1.tgz",
"integrity": "sha512-ZClg6AaYvamvYEE82d3Iyd3vSSIjQ+odgjaTzRuO3s7toCdFKczob2i0zCh7JE8kWn17yvAWhUVxvqGwUalsRA==",
"license": "ISC"
},
"node_modules/locate-path": {
"version": "6.0.0",
"resolved": "https://registry.npmjs.org/locate-path/-/locate-path-6.0.0.tgz",
"integrity": "sha512-iPZK6eYjbxRu3uB4/WZ3EsEIMJFMqAoopl3R+zuq0UjcAm/MO6KCweDgPfP3elTztoKP3KtnVHxTn2NHBSDVUw==",
"dev": true,
"license": "MIT",
"dependencies": {
"p-locate": "^5.0.0"
},
"engines": {
"node": ">=10"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/log-symbols": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/log-symbols/-/log-symbols-4.1.0.tgz",
"integrity": "sha512-8XPvpAA8uyhfteu8pIvQxpJZ7SYYdpUivZpGy6sFsBuKRY/7rQGavedeB8aK+Zkyq6upMFVL/9AW6vOYzfRyLg==",
"dev": true,
"license": "MIT",
"dependencies": {
"chalk": "^4.1.0",
"is-unicode-supported": "^0.1.0"
},
"engines": {
"node": ">=10"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/matcher": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/matcher/-/matcher-3.0.0.tgz",
"integrity": "sha512-OkeDaAZ/bQCxeFAozM55PKcKU0yJMPGifLwV4Qgjitu+5MoAfSQN4lsLJeXZ1b8w0x+/Emda6MZgXS1jvsapng==",
"license": "MIT",
"dependencies": {
"escape-string-regexp": "^4.0.0"
},
"engines": {
"node": ">=10"
}
},
"node_modules/math-intrinsics": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/math-intrinsics/-/math-intrinsics-1.1.0.tgz",
"integrity": "sha512-/IXtbwEk5HTPyEwyKX6hGkYXxM9nbj64B+ilVJnC/R6B0pH5G4V3b0pVbL7DBj4tkhBAppbQUlf6F6Xl9LHu1g==",
"license": "MIT",
"engines": {
"node": ">= 0.4"
}
},
"node_modules/media-typer": {
"version": "0.3.0",
"resolved": "https://registry.npmjs.org/media-typer/-/media-typer-0.3.0.tgz",
"integrity": "sha512-dq+qelQ9akHpcOl/gUVRTxVIOkAJ1wR3QAvb4RsVjS8oVoFjDGTc679wJYmUmknUF5HwMLOgb5O+a3KxfWapPQ==",
"license": "MIT",
"engines": {
"node": ">= 0.6"
}
},
"node_modules/merge-descriptors": {
"version": "1.0.3",
"resolved": "https://registry.npmjs.org/merge-descriptors/-/merge-descriptors-1.0.3.tgz",
"integrity": "sha512-gaNvAS7TZ897/rVaZ0nMtAyxNyi/pdbjbAwUpFQpN70GqnVfOiXpeUUMKRBmzXaSQ8DdTX4/0ms62r2K+hE6mQ==",
"license": "MIT",
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/methods": {
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/methods/-/methods-1.1.2.tgz",
"integrity": "sha512-iclAHeNqNm68zFtnZ0e+1L2yUIdvzNoauKU4WBA3VvH/vPFieF7qfRlwUZU+DA9P9bPXIS90ulxoUoCH23sV2w==",
"license": "MIT",
"engines": {
"node": ">= 0.6"
}
},
"node_modules/mime": {
"version": "1.6.0",
"resolved": "https://registry.npmjs.org/mime/-/mime-1.6.0.tgz",
"integrity": "sha512-x0Vn8spI+wuJ1O6S7gnbaQg8Pxh4NNHb7KSINmEWKiPE4RKOplvijn+NkmYmmRgP68mc70j2EbeTFRsrswaQeg==",
"license": "MIT",
"bin": {
"mime": "cli.js"
},
"engines": {
"node": ">=4"
}
},
"node_modules/mime-db": {
"version": "1.52.0",
"resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.52.0.tgz",
"integrity": "sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg==",
"license": "MIT",
"engines": {
"node": ">= 0.6"
}
},
"node_modules/mime-types": {
"version": "2.1.35",
"resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.35.tgz",
"integrity": "sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw==",
"license": "MIT",
"dependencies": {
"mime-db": "1.52.0"
},
"engines": {
"node": ">= 0.6"
}
},
"node_modules/minimatch": {
"version": "5.1.6",
"resolved": "https://registry.npmjs.org/minimatch/-/minimatch-5.1.6.tgz",
"integrity": "sha512-lKwV/1brpG6mBUFHtb7NUmtABCb2WZZmm2wNiOA5hAb8VdCS4B3dtMWyvcoViccwAW/COERjXLt0zP1zXUN26g==",
"dev": true,
"license": "ISC",
"dependencies": {
"brace-expansion": "^2.0.1"
},
"engines": {
"node": ">=10"
}
},
"node_modules/mocha": {
"version": "10.8.2",
"resolved": "https://registry.npmjs.org/mocha/-/mocha-10.8.2.tgz",
"integrity": "sha512-VZlYo/WE8t1tstuRmqgeyBgCbJc/lEdopaa+axcKzTBJ+UIdlAB9XnmvTCAH4pwR4ElNInaedhEBmZD8iCSVEg==",
"dev": true,
"license": "MIT",
"dependencies": {
"ansi-colors": "^4.1.3",
"browser-stdout": "^1.3.1",
"chokidar": "^3.5.3",
"debug": "^4.3.5",
"diff": "^5.2.0",
"escape-string-regexp": "^4.0.0",
"find-up": "^5.0.0",
"glob": "^8.1.0",
"he": "^1.2.0",
"js-yaml": "^4.1.0",
"log-symbols": "^4.1.0",
"minimatch": "^5.1.6",
"ms": "^2.1.3",
"serialize-javascript": "^6.0.2",
"strip-json-comments": "^3.1.1",
"supports-color": "^8.1.1",
"workerpool": "^6.5.1",
"yargs": "^16.2.0",
"yargs-parser": "^20.2.9",
"yargs-unparser": "^2.0.0"
},
"bin": {
"_mocha": "bin/_mocha",
"mocha": "bin/mocha.js"
},
"engines": {
"node": ">= 14.0.0"
}
},
"node_modules/mocha/node_modules/debug": {
"version": "4.4.1",
"resolved": "https://registry.npmjs.org/debug/-/debug-4.4.1.tgz",
"integrity": "sha512-KcKCqiftBJcZr++7ykoDIEwSa3XWowTfNPo92BYxjXiyYEVrUQh2aLyhxBCwww+heortUFxEJYcRzosstTEBYQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"ms": "^2.1.3"
},
"engines": {
"node": ">=6.0"
},
"peerDependenciesMeta": {
"supports-color": {
"optional": true
}
}
},
"node_modules/mocha/node_modules/ms": {
"version": "2.1.3",
"resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz",
"integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",
"dev": true,
"license": "MIT"
},
"node_modules/ms": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/ms/-/ms-2.0.0.tgz",
"integrity": "sha512-Tpp60P6IUJDTuOq/5Z8cdskzJujfwqfOTkrwIwj7IRISpnkJnT6SyJ4PCPnGMoFjC9ddhal5KVIYtAt97ix05A==",
"license": "MIT"
},
"node_modules/n-gram": {
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/n-gram/-/n-gram-2.0.2.tgz",
"integrity": "sha512-S24aGsn+HLBxUGVAUFOwGpKs7LBcG4RudKU//eWzt/mQ97/NMKQxDWHyHx63UNWk/OOdihgmzoETn1tf5nQDzQ==",
"license": "MIT",
"funding": {
"type": "github",
"url": "https://github.com/sponsors/wooorm"
}
},
"node_modules/negotiator": {
"version": "0.6.3",
"resolved": "https://registry.npmjs.org/negotiator/-/negotiator-0.6.3.tgz",
"integrity": "sha512-+EUsqGPLsM+j/zdChZjsnX51g4XrHFOIXwfnCVPGlQk/k5giakcKsuxCObBRu6DSm9opw/O6slWbJdghQM4bBg==",
"license": "MIT",
"engines": {
"node": ">= 0.6"
}
},
"node_modules/node-wav": {
"version": "0.0.2",
"resolved": "https://registry.npmjs.org/node-wav/-/node-wav-0.0.2.tgz",
"integrity": "sha512-M6Rm/bbG6De/gKGxOpeOobx/dnGuP0dz40adqx38boqHhlWssBJZgLCPBNtb9NkrmnKYiV04xELq+R6PFOnoLA==",
"license": "MIT",
"engines": {
"node": ">=4.4.0"
}
},
"node_modules/normalize-path": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/normalize-path/-/normalize-path-3.0.0.tgz",
"integrity": "sha512-6eZs5Ls3WtCisHWp9S2GUy8dqkpGi4BVSz3GaqiE6ezub0512ESztXUwUB6C6IKbQkY2Pnb/mD4WYojCRwcwLA==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/object-inspect": {
"version": "1.13.4",
"resolved": "https://registry.npmjs.org/object-inspect/-/object-inspect-1.13.4.tgz",
"integrity": "sha512-W67iLl4J2EXEGTbfeHCffrjDfitvLANg0UlX3wFUUSTx92KXRFegMHUVgSqE+wvhAbi4WqjGg9czysTV2Epbew==",
"license": "MIT",
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/object-keys": {
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/object-keys/-/object-keys-1.1.1.tgz",
"integrity": "sha512-NuAESUOUMrlIXOfHKzD6bpPu3tYt3xvjNdRIQ+FeT0lNb4K8WR70CaDxhuNguS2XG+GjkyMwOzsN5ZktImfhLA==",
"license": "MIT",
"engines": {
"node": ">= 0.4"
}
},
"node_modules/on-finished": {
"version": "2.4.1",
"resolved": "https://registry.npmjs.org/on-finished/-/on-finished-2.4.1.tgz",
"integrity": "sha512-oVlzkg3ENAhCk2zdv7IJwd/QUD4z2RxRwpkcGY8psCVcCYZNq4wYnVWALHM+brtuJjePWiYF/ClmuDr8Ch5+kg==",
"license": "MIT",
"dependencies": {
"ee-first": "1.1.1"
},
"engines": {
"node": ">= 0.8"
}
},
"node_modules/once": {
"version": "1.4.0",
"resolved": "https://registry.npmjs.org/once/-/once-1.4.0.tgz",
"integrity": "sha512-lNaJgI+2Q5URQBkccEKHTQOPaXdUxnZZElQTZY0MFUAuaEqe1E+Nyvgdz/aIyNi6Z9MzO5dv1H8n58/GELp3+w==",
"dev": true,
"license": "ISC",
"dependencies": {
"wrappy": "1"
}
},
"node_modules/onnxruntime-common": {
"version": "1.22.0",
"resolved": "https://registry.npmjs.org/onnxruntime-common/-/onnxruntime-common-1.22.0.tgz",
"integrity": "sha512-vcuaNWgtF2dGQu/EP5P8UI5rEPEYqXG2sPPe5j9lg2TY/biJF8eWklTMwlDO08iuXq48xJo0awqIpK5mPG+IxA==",
"license": "MIT"
},
"node_modules/onnxruntime-node": {
"version": "1.22.0-rev",
"resolved": "https://registry.npmjs.org/onnxruntime-node/-/onnxruntime-node-1.22.0-rev.tgz",
"integrity": "sha512-9vh50/mnwauFUex0NYyyLf9pmRp8q6DVMG8K+xtoXv68SSB9bESa1bEbWLqfUncgB3XucQaOV+wfMPcqANMYhQ==",
"hasInstallScript": true,
"license": "MIT",
"os": [
"win32",
"darwin",
"linux"
],
"dependencies": {
"adm-zip": "^0.5.16",
"global-agent": "^3.0.0",
"onnxruntime-common": "1.22.0"
}
},
"node_modules/p-limit": {
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/p-limit/-/p-limit-3.1.0.tgz",
"integrity": "sha512-TYOanM3wGwNGsZN2cVTYPArw454xnXj5qmWF1bEoAc4+cU/ol7GVh7odevjp1FNHduHc3KZMcFduxU5Xc6uJRQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"yocto-queue": "^0.1.0"
},
"engines": {
"node": ">=10"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/p-locate": {
"version": "5.0.0",
"resolved": "https://registry.npmjs.org/p-locate/-/p-locate-5.0.0.tgz",
"integrity": "sha512-LaNjtRWUBY++zB5nE/NwcaoMylSPk+S+ZHNB1TzdbMJMny6dynpAGt7X/tl/QYq3TIeE6nxHppbo2LGymrG5Pw==",
"dev": true,
"license": "MIT",
"dependencies": {
"p-limit": "^3.0.2"
},
"engines": {
"node": ">=10"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/parseurl": {
"version": "1.3.3",
"resolved": "https://registry.npmjs.org/parseurl/-/parseurl-1.3.3.tgz",
"integrity": "sha512-CiyeOxFT/JZyN5m0z9PfXw4SCBJ6Sygz1Dpl0wqjlhDEGGBP1GnsUVEL0p63hoG1fcj3fHynXi9NYO4nWOL+qQ==",
"license": "MIT",
"engines": {
"node": ">= 0.8"
}
},
"node_modules/path-exists": {
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/path-exists/-/path-exists-4.0.0.tgz",
"integrity": "sha512-ak9Qy5Q7jYb2Wwcey5Fpvg2KoAc/ZIhLSLOSBmRmygPsGwkVVt0fZa0qrtMz+m6tJTAHfZQ8FnmB4MG4LWy7/w==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=8"
}
},
"node_modules/path-to-regexp": {
"version": "0.1.12",
"resolved": "https://registry.npmjs.org/path-to-regexp/-/path-to-regexp-0.1.12.tgz",
"integrity": "sha512-RA1GjUVMnvYFxuqovrEqZoxxW5NUZqbwKtYz/Tt7nXerk0LbLblQmrsgdeOxV5SFHf0UDggjS/bSeOZwt1pmEQ==",
"license": "MIT"
},
"node_modules/picomatch": {
"version": "2.3.1",
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-2.3.1.tgz",
"integrity": "sha512-JU3teHTNjmE2VCGFzuY8EXzCDVwEqB2a8fsIvwaStHhAWJEeVd1o1QD80CU6+ZdEXXSLbSsuLwJjkCBWqRQUVA==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=8.6"
},
"funding": {
"url": "https://github.com/sponsors/jonschlinkert"
}
},
"node_modules/proxy-addr": {
"version": "2.0.7",
"resolved": "https://registry.npmjs.org/proxy-addr/-/proxy-addr-2.0.7.tgz",
"integrity": "sha512-llQsMLSUDUPT44jdrU/O37qlnifitDP+ZwrmmZcoSKyLKvtZxpyV0n2/bD/N4tBAAZ/gJEdZU7KMraoK1+XYAg==",
"license": "MIT",
"dependencies": {
"forwarded": "0.2.0",
"ipaddr.js": "1.9.1"
},
"engines": {
"node": ">= 0.10"
}
},
"node_modules/proxy-from-env": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/proxy-from-env/-/proxy-from-env-1.1.0.tgz",
"integrity": "sha512-D+zkORCbA9f1tdWRK0RaCR3GPv50cMxcrz4X8k5LTSUD1Dkw47mKJEZQNunItRTkWwgtaUSo1RVFRIG9ZXiFYg==",
"license": "MIT"
},
"node_modules/qs": {
"version": "6.13.0",
"resolved": "https://registry.npmjs.org/qs/-/qs-6.13.0.tgz",
"integrity": "sha512-+38qI9SOr8tfZ4QmJNplMUxqjbe7LKvvZgWdExBOmd+egZTtjLB67Gu0HRX3u/XOq7UU2Nx6nsjvS16Z9uwfpg==",
"license": "BSD-3-Clause",
"dependencies": {
"side-channel": "^1.0.6"
},
"engines": {
"node": ">=0.6"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/randombytes": {
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/randombytes/-/randombytes-2.1.0.tgz",
"integrity": "sha512-vYl3iOX+4CKUWuxGi9Ukhie6fsqXqS9FE2Zaic4tNFD2N2QQaXOMFbuKK4QmDHC0JO6B1Zp41J0LpT0oR68amQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"safe-buffer": "^5.1.0"
}
},
"node_modules/range-parser": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/range-parser/-/range-parser-1.2.1.tgz",
"integrity": "sha512-Hrgsx+orqoygnmhFbKaHE6c296J+HTAQXoxEF6gNupROmmGJRoyzfG3ccAveqCBrwr/2yxQ5BVd/GTl5agOwSg==",
"license": "MIT",
"engines": {
"node": ">= 0.6"
}
},
"node_modules/raw-body": {
"version": "2.5.2",
"resolved": "https://registry.npmjs.org/raw-body/-/raw-body-2.5.2.tgz",
"integrity": "sha512-8zGqypfENjCIqGhgXToC8aB2r7YrBX+AQAfIPs/Mlk+BtPTztOvTS01NRW/3Eh60J+a48lt8qsCzirQ6loCVfA==",
"license": "MIT",
"dependencies": {
"bytes": "3.1.2",
"http-errors": "2.0.0",
"iconv-lite": "0.4.24",
"unpipe": "1.0.0"
},
"engines": {
"node": ">= 0.8"
}
},
"node_modules/readable-stream": {
"version": "1.1.14",
"resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-1.1.14.tgz",
"integrity": "sha512-+MeVjFf4L44XUkhM1eYbD8fyEsxcV81pqMSR5gblfcLCHfZvbrqy4/qYHE+/R5HoBUT11WV5O08Cr1n3YXkWVQ==",
"license": "MIT",
"dependencies": {
"core-util-is": "~1.0.0",
"inherits": "~2.0.1",
"isarray": "0.0.1",
"string_decoder": "~0.10.x"
}
},
"node_modules/readdirp": {
"version": "3.6.0",
"resolved": "https://registry.npmjs.org/readdirp/-/readdirp-3.6.0.tgz",
"integrity": "sha512-hOS089on8RduqdbhvQ5Z37A0ESjsqz6qnRcffsMU3495FuTdqSm+7bhJ29JvIOsBDEEnan5DPu9t3To9VRlMzA==",
"dev": true,
"license": "MIT",
"dependencies": {
"picomatch": "^2.2.1"
},
"engines": {
"node": ">=8.10.0"
}
},
"node_modules/require-directory": {
"version": "2.1.1",
"resolved": "https://registry.npmjs.org/require-directory/-/require-directory-2.1.1.tgz",
"integrity": "sha512-fGxEI7+wsG9xrvdjsrlmL22OMTTiHRwAMroiEeMgq8gzoLC/PQr7RsRDSTLUg/bZAZtF+TVIkHc6/4RIKrui+Q==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/roarr": {
"version": "2.15.4",
"resolved": "https://registry.npmjs.org/roarr/-/roarr-2.15.4.tgz",
"integrity": "sha512-CHhPh+UNHD2GTXNYhPWLnU8ONHdI+5DI+4EYIAOaiD63rHeYlZvyh8P+in5999TTSFgUYuKUAjzRI4mdh/p+2A==",
"license": "BSD-3-Clause",
"dependencies": {
"boolean": "^3.0.1",
"detect-node": "^2.0.4",
"globalthis": "^1.0.1",
"json-stringify-safe": "^5.0.1",
"semver-compare": "^1.0.0",
"sprintf-js": "^1.1.2"
},
"engines": {
"node": ">=8.0"
}
},
"node_modules/safe-buffer": {
"version": "5.2.1",
"resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.2.1.tgz",
"integrity": "sha512-rp3So07KcdmmKbGvgaNxQSJr7bGVSVk5S9Eq1F+ppbRo70+YeaDxkw5Dd8NPN+GD6bjnYm2VuPuCXmpuYvmCXQ==",
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/feross"
},
{
"type": "patreon",
"url": "https://www.patreon.com/feross"
},
{
"type": "consulting",
"url": "https://feross.org/support"
}
],
"license": "MIT"
},
"node_modules/safer-buffer": {
"version": "2.1.2",
"resolved": "https://registry.npmjs.org/safer-buffer/-/safer-buffer-2.1.2.tgz",
"integrity": "sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg==",
"license": "MIT"
},
"node_modules/semver": {
"version": "7.7.2",
"resolved": "https://registry.npmjs.org/semver/-/semver-7.7.2.tgz",
"integrity": "sha512-RF0Fw+rO5AMf9MAyaRXI4AV0Ulj5lMHqVxxdSgiVbixSCXoEmmX/jk0CuJw4+3SqroYO9VoUh+HcuJivvtJemA==",
"license": "ISC",
"bin": {
"semver": "bin/semver.js"
},
"engines": {
"node": ">=10"
}
},
"node_modules/semver-compare": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/semver-compare/-/semver-compare-1.0.0.tgz",
"integrity": "sha512-YM3/ITh2MJ5MtzaM429anh+x2jiLVjqILF4m4oyQB18W7Ggea7BfqdH/wGMK7dDiMghv/6WG7znWMwUDzJiXow==",
"license": "MIT"
},
"node_modules/send": {
"version": "0.19.0",
"resolved": "https://registry.npmjs.org/send/-/send-0.19.0.tgz",
"integrity": "sha512-dW41u5VfLXu8SJh5bwRmyYUbAoSB3c9uQh6L8h/KtsFREPWpbX1lrljJo186Jc4nmci/sGUZ9a0a0J2zgfq2hw==",
"license": "MIT",
"dependencies": {
"debug": "2.6.9",
"depd": "2.0.0",
"destroy": "1.2.0",
"encodeurl": "~1.0.2",
"escape-html": "~1.0.3",
"etag": "~1.8.1",
"fresh": "0.5.2",
"http-errors": "2.0.0",
"mime": "1.6.0",
"ms": "2.1.3",
"on-finished": "2.4.1",
"range-parser": "~1.2.1",
"statuses": "2.0.1"
},
"engines": {
"node": ">= 0.8.0"
}
},
"node_modules/send/node_modules/encodeurl": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/encodeurl/-/encodeurl-1.0.2.tgz",
"integrity": "sha512-TPJXq8JqFaVYm2CWmPvnP2Iyo4ZSM7/QKcSmuMLDObfpH5fi7RUGmd/rTDf+rut/saiDiQEeVTNgAmJEdAOx0w==",
"license": "MIT",
"engines": {
"node": ">= 0.8"
}
},
"node_modules/send/node_modules/ms": {
"version": "2.1.3",
"resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz",
"integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",
"license": "MIT"
},
"node_modules/serialize-error": {
"version": "7.0.1",
"resolved": "https://registry.npmjs.org/serialize-error/-/serialize-error-7.0.1.tgz",
"integrity": "sha512-8I8TjW5KMOKsZQTvoxjuSIa7foAwPWGOts+6o7sgjz41/qMD9VQHEDxi6PBvK2l0MXUmqZyNpUK+T2tQaaElvw==",
"license": "MIT",
"dependencies": {
"type-fest": "^0.13.1"
},
"engines": {
"node": ">=10"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/serialize-javascript": {
"version": "6.0.2",
"resolved": "https://registry.npmjs.org/serialize-javascript/-/serialize-javascript-6.0.2.tgz",
"integrity": "sha512-Saa1xPByTTq2gdeFZYLLo+RFE35NHZkAbqZeWNd3BpzppeVisAqpDjcp8dyf6uIvEqJRd46jemmyA4iFIeVk8g==",
"dev": true,
"license": "BSD-3-Clause",
"dependencies": {
"randombytes": "^2.1.0"
}
},
"node_modules/serve-static": {
"version": "1.16.2",
"resolved": "https://registry.npmjs.org/serve-static/-/serve-static-1.16.2.tgz",
"integrity": "sha512-VqpjJZKadQB/PEbEwvFdO43Ax5dFBZ2UECszz8bQ7pi7wt//PWe1P6MN7eCnjsatYtBT6EuiClbjSWP2WrIoTw==",
"license": "MIT",
"dependencies": {
"encodeurl": "~2.0.0",
"escape-html": "~1.0.3",
"parseurl": "~1.3.3",
"send": "0.19.0"
},
"engines": {
"node": ">= 0.8.0"
}
},
"node_modules/setprototypeof": {
"version": "1.2.0",
"resolved": "https://registry.npmjs.org/setprototypeof/-/setprototypeof-1.2.0.tgz",
"integrity": "sha512-E5LDX7Wrp85Kil5bhZv46j8jOeboKq5JMmYM3gVGdGH8xFpPWXUMsNrlODCrkoxMEeNi/XZIwuRvY4XNwYMJpw==",
"license": "ISC"
},
"node_modules/side-channel": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/side-channel/-/side-channel-1.1.0.tgz",
"integrity": "sha512-ZX99e6tRweoUXqR+VBrslhda51Nh5MTQwou5tnUDgbtyM0dBgmhEDtWGP/xbKn6hqfPRHujUNwz5fy/wbbhnpw==",
"license": "MIT",
"dependencies": {
"es-errors": "^1.3.0",
"object-inspect": "^1.13.3",
"side-channel-list": "^1.0.0",
"side-channel-map": "^1.0.1",
"side-channel-weakmap": "^1.0.2"
},
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/side-channel-list": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/side-channel-list/-/side-channel-list-1.0.0.tgz",
"integrity": "sha512-FCLHtRD/gnpCiCHEiJLOwdmFP+wzCmDEkc9y7NsYxeF4u7Btsn1ZuwgwJGxImImHicJArLP4R0yX4c2KCrMrTA==",
"license": "MIT",
"dependencies": {
"es-errors": "^1.3.0",
"object-inspect": "^1.13.3"
},
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/side-channel-map": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/side-channel-map/-/side-channel-map-1.0.1.tgz",
"integrity": "sha512-VCjCNfgMsby3tTdo02nbjtM/ewra6jPHmpThenkTYh8pG9ucZ/1P8So4u4FGBek/BjpOVsDCMoLA/iuBKIFXRA==",
"license": "MIT",
"dependencies": {
"call-bound": "^1.0.2",
"es-errors": "^1.3.0",
"get-intrinsic": "^1.2.5",
"object-inspect": "^1.13.3"
},
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/side-channel-weakmap": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/side-channel-weakmap/-/side-channel-weakmap-1.0.2.tgz",
"integrity": "sha512-WPS/HvHQTYnHisLo9McqBHOJk2FkHO/tlpvldyrnem4aeQp4hai3gythswg6p01oSoTl58rcpiFAjF2br2Ak2A==",
"license": "MIT",
"dependencies": {
"call-bound": "^1.0.2",
"es-errors": "^1.3.0",
"get-intrinsic": "^1.2.5",
"object-inspect": "^1.13.3",
"side-channel-map": "^1.0.1"
},
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/sprintf-js": {
"version": "1.1.3",
"resolved": "https://registry.npmjs.org/sprintf-js/-/sprintf-js-1.1.3.tgz",
"integrity": "sha512-Oo+0REFV59/rz3gfJNKQiBlwfHaSESl1pcGyABQsnnIfWOFt6JNj5gCog2U6MLZ//IGYD+nA8nI+mTShREReaA==",
"license": "BSD-3-Clause"
},
"node_modules/statuses": {
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/statuses/-/statuses-2.0.1.tgz",
"integrity": "sha512-RwNA9Z/7PrK06rYLIzFMlaF+l73iwpzsqRIFgbMLbTcLD6cOao82TaWefPXQvB2fOC4AjuYSEndS7N/mTCbkdQ==",
"license": "MIT",
"engines": {
"node": ">= 0.8"
}
},
"node_modules/stream-parser": {
"version": "0.3.1",
"resolved": "https://registry.npmjs.org/stream-parser/-/stream-parser-0.3.1.tgz",
"integrity": "sha512-bJ/HgKq41nlKvlhccD5kaCr/P+Hu0wPNKPJOH7en+YrJu/9EgqUF+88w5Jb6KNcjOFMhfX4B2asfeAtIGuHObQ==",
"license": "MIT",
"dependencies": {
"debug": "2"
}
},
"node_modules/string_decoder": {
"version": "0.10.31",
"resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-0.10.31.tgz",
"integrity": "sha512-ev2QzSzWPYmy9GuqfIVildA4OdcGLeFZQrq5ys6RtiuF+RQQiZWr8TZNyAcuVXyQRYfEO+MsoB/1BuQVhOJuoQ==",
"license": "MIT"
},
"node_modules/string-width": {
"version": "4.2.3",
"resolved": "https://registry.npmjs.org/string-width/-/string-width-4.2.3.tgz",
"integrity": "sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==",
"dev": true,
"license": "MIT",
"dependencies": {
"emoji-regex": "^8.0.0",
"is-fullwidth-code-point": "^3.0.0",
"strip-ansi": "^6.0.1"
},
"engines": {
"node": ">=8"
}
},
"node_modules/string-width/node_modules/emoji-regex": {
"version": "8.0.0",
"resolved": "https://registry.npmjs.org/emoji-regex/-/emoji-regex-8.0.0.tgz",
"integrity": "sha512-MSjYzcWNOA0ewAHpz0MxpYFvwg6yjy1NG3xteoqz644VCo/RPgnr1/GGt+ic3iJTzQ8Eu3TdM14SawnVUmGE6A==",
"dev": true,
"license": "MIT"
},
"node_modules/strip-ansi": {
"version": "6.0.1",
"resolved": "https://registry.npmjs.org/strip-ansi/-/strip-ansi-6.0.1.tgz",
"integrity": "sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A==",
"dev": true,
"license": "MIT",
"dependencies": {
"ansi-regex": "^5.0.1"
},
"engines": {
"node": ">=8"
}
},
"node_modules/strip-json-comments": {
"version": "3.1.1",
"resolved": "https://registry.npmjs.org/strip-json-comments/-/strip-json-comments-3.1.1.tgz",
"integrity": "sha512-6fPc+R4ihwqP6N/aIv2f1gMH8lOVtWQHoqC4yK6oSDVVocumAsfCqjkXnqiYMhmMwS/mEHLp7Vehlt3ql6lEig==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=8"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/supports-color": {
"version": "8.1.1",
"resolved": "https://registry.npmjs.org/supports-color/-/supports-color-8.1.1.tgz",
"integrity": "sha512-MpUEN2OodtUzxvKQl72cUF7RQ5EiHsGvSsVG0ia9c5RbWGL2CI4C7EpPS8UTBIplnlzZiNuV56w+FuNxy3ty2Q==",
"dev": true,
"license": "MIT",
"dependencies": {
"has-flag": "^4.0.0"
},
"engines": {
"node": ">=10"
},
"funding": {
"url": "https://github.com/chalk/supports-color?sponsor=1"
}
},
"node_modules/to-regex-range": {
"version": "5.0.1",
"resolved": "https://registry.npmjs.org/to-regex-range/-/to-regex-range-5.0.1.tgz",
"integrity": "sha512-65P7iz6X5yEr1cwcgvQxbbIw7Uk3gOy5dIdtZ4rDveLqhrdJP+Li/Hx6tyK0NEb+2GCyneCMJiGqrADCSNk8sQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"is-number": "^7.0.0"
},
"engines": {
"node": ">=8.0"
}
},
"node_modules/toidentifier": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/toidentifier/-/toidentifier-1.0.1.tgz",
"integrity": "sha512-o5sSPKEkg/DIQNmH43V0/uerLrpzVedkUh8tGNvaeXpfpuwjKenlSox/2O/BTlZUtEe+JG7s5YhEz608PlAHRA==",
"license": "MIT",
"engines": {
"node": ">=0.6"
}
},
"node_modules/trigram-utils": {
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/trigram-utils/-/trigram-utils-2.0.1.tgz",
"integrity": "sha512-nfWIXHEaB+HdyslAfMxSqWKDdmqY9I32jS7GnqpdWQnLH89r6A5sdk3fDVYqGAZ0CrT8ovAFSAo6HRiWcWNIGQ==",
"license": "MIT",
"dependencies": {
"collapse-white-space": "^2.0.0",
"n-gram": "^2.0.0"
},
"funding": {
"type": "github",
"url": "https://github.com/sponsors/wooorm"
}
},
"node_modules/type-fest": {
"version": "0.13.1",
"resolved": "https://registry.npmjs.org/type-fest/-/type-fest-0.13.1.tgz",
"integrity": "sha512-34R7HTnG0XIJcBSn5XhDd7nNFPRcXYRZrBB2O2jdKqYODldSzBAqzsWoZYYvduky73toYS/ESqxPvkDf/F0XMg==",
"license": "(MIT OR CC0-1.0)",
"engines": {
"node": ">=10"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/type-is": {
"version": "1.6.18",
"resolved": "https://registry.npmjs.org/type-is/-/type-is-1.6.18.tgz",
"integrity": "sha512-TkRKr9sUTxEH8MdfuCSP7VizJyzRNMjj2J2do2Jr3Kym598JVdEksuzPQCnlFPW4ky9Q+iA+ma9BGm06XQBy8g==",
"license": "MIT",
"dependencies": {
"media-typer": "0.3.0",
"mime-types": "~2.1.24"
},
"engines": {
"node": ">= 0.6"
}
},
"node_modules/unpipe": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/unpipe/-/unpipe-1.0.0.tgz",
"integrity": "sha512-pjy2bYhSsufwWlKwPc+l3cN7+wuJlK6uz0YdJEOlQDbl6jo/YlPi4mb8agUkVC8BF7V8NuzeyPNqRksA3hztKQ==",
"license": "MIT",
"engines": {
"node": ">= 0.8"
}
},
"node_modules/utils-merge": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/utils-merge/-/utils-merge-1.0.1.tgz",
"integrity": "sha512-pMZTvIkT1d+TFGvDOqodOclx0QWkkgi6Tdoa8gC8ffGAAqz9pzPTZWAybbsHHoED/ztMtkv/VoYTYyShUn81hA==",
"license": "MIT",
"engines": {
"node": ">= 0.4.0"
}
},
"node_modules/vary": {
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/vary/-/vary-1.1.2.tgz",
"integrity": "sha512-BNGbWLfd0eUPabhkXUVm0j8uuvREyTh5ovRa/dyow/BqAbZJyC+5fU+IzQOzmAKzYqYRAISoRhdQr3eIZ/PXqg==",
"license": "MIT",
"engines": {
"node": ">= 0.8"
}
},
"node_modules/wav": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/wav/-/wav-1.0.2.tgz",
"integrity": "sha512-viHtz3cDd/Tcr/HbNqzQCofKdF6kWUymH9LGDdskfWFoIy/HJ+RTihgjEcHfnsy1PO4e9B+y4HwgTwMrByquhg==",
"license": "MIT",
"dependencies": {
"buffer-alloc": "^1.1.0",
"buffer-from": "^1.0.0",
"debug": "^2.2.0",
"readable-stream": "^1.1.14",
"stream-parser": "^0.3.1"
}
},
"node_modules/which": {
"version": "1.3.1",
"resolved": "https://registry.npmjs.org/which/-/which-1.3.1.tgz",
"integrity": "sha512-HxJdYWq1MTIQbJ3nw0cqssHoTNU267KlrDuGZ1WYlxDStUtKUhOaJmh112/TZmHxxUfuJqPXSOm7tDyas0OSIQ==",
"license": "ISC",
"dependencies": {
"isexe": "^2.0.0"
},
"bin": {
"which": "bin/which"
}
},
"node_modules/workerpool": {
"version": "6.5.1",
"resolved": "https://registry.npmjs.org/workerpool/-/workerpool-6.5.1.tgz",
"integrity": "sha512-Fs4dNYcsdpYSAfVxhnl1L5zTksjvOJxtC5hzMNl+1t9B8hTJTdKDyZ5ju7ztgPy+ft9tBFXoOlDNiOT9WUXZlA==",
"dev": true,
"license": "Apache-2.0"
},
"node_modules/wrap-ansi": {
"version": "7.0.0",
"resolved": "https://registry.npmjs.org/wrap-ansi/-/wrap-ansi-7.0.0.tgz",
"integrity": "sha512-YVGIj2kamLSTxw6NsZjoBxfSwsn0ycdesmc4p+Q21c5zPuZ1pl+NfxVdxPtdHvmNVOQ6XSYG4AUtyt/Fi7D16Q==",
"dev": true,
"license": "MIT",
"dependencies": {
"ansi-styles": "^4.0.0",
"string-width": "^4.1.0",
"strip-ansi": "^6.0.0"
},
"engines": {
"node": ">=10"
},
"funding": {
"url": "https://github.com/chalk/wrap-ansi?sponsor=1"
}
},
"node_modules/wrappy": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz",
"integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==",
"dev": true,
"license": "ISC"
},
"node_modules/ws": {
"version": "8.18.0",
"resolved": "https://registry.npmjs.org/ws/-/ws-8.18.0.tgz",
"integrity": "sha512-8VbfWfHLbbwu3+N6OKsOMpBdT4kXPDDB9cJk2bJ6mh9ucxdlnNvH1e+roYkKmN9Nxw2yjz7VzeO9oOz2zJ04Pw==",
"license": "MIT",
"engines": {
"node": ">=10.0.0"
},
"peerDependencies": {
"bufferutil": "^4.0.1",
"utf-8-validate": ">=5.0.2"
},
"peerDependenciesMeta": {
"bufferutil": {
"optional": true
},
"utf-8-validate": {
"optional": true
}
}
},
"node_modules/y18n": {
"version": "5.0.8",
"resolved": "https://registry.npmjs.org/y18n/-/y18n-5.0.8.tgz",
"integrity": "sha512-0pfFzegeDWJHJIAmTLRP2DwHjdF5s7jo9tuztdQxAhINCdvS+3nGINqPd00AphqJR/0LhANUS6/+7SCb98YOfA==",
"dev": true,
"license": "ISC",
"engines": {
"node": ">=10"
}
},
"node_modules/yargs": {
"version": "16.2.0",
"resolved": "https://registry.npmjs.org/yargs/-/yargs-16.2.0.tgz",
"integrity": "sha512-D1mvvtDG0L5ft/jGWkLpG1+m0eQxOfaBvTNELraWj22wSVUMWxZUvYgJYcKh6jGGIkJFhH4IZPQhR4TKpc8mBw==",
"dev": true,
"license": "MIT",
"dependencies": {
"cliui": "^7.0.2",
"escalade": "^3.1.1",
"get-caller-file": "^2.0.5",
"require-directory": "^2.1.1",
"string-width": "^4.2.0",
"y18n": "^5.0.5",
"yargs-parser": "^20.2.2"
},
"engines": {
"node": ">=10"
}
},
"node_modules/yargs-parser": {
"version": "20.2.9",
"resolved": "https://registry.npmjs.org/yargs-parser/-/yargs-parser-20.2.9.tgz",
"integrity": "sha512-y11nGElTIV+CT3Zv9t7VKl+Q3hTQoT9a1Qzezhhl6Rp21gJ/IVTW7Z3y9EWXhuUBC2Shnf+DX0antecpAwSP8w==",
"dev": true,
"license": "ISC",
"engines": {
"node": ">=10"
}
},
"node_modules/yargs-unparser": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/yargs-unparser/-/yargs-unparser-2.0.0.tgz",
"integrity": "sha512-7pRTIA9Qc1caZ0bZ6RYRGbHJthJWuakf+WmHK0rVeLkNrrGhfoabBNdue6kdINI6r4if7ocq9aD/n7xwKOdzOA==",
"dev": true,
"license": "MIT",
"dependencies": {
"camelcase": "^6.0.0",
"decamelize": "^4.0.0",
"flat": "^5.0.2",
"is-plain-obj": "^2.1.0"
},
"engines": {
"node": ">=10"
}
},
"node_modules/yocto-queue": {
"version": "0.1.0",
"resolved": "https://registry.npmjs.org/yocto-queue/-/yocto-queue-0.1.0.tgz",
"integrity": "sha512-rVksvsnNCdJ/ohGc6xgPwyN8eheCxsiLM8mxuE/t/mOVqJewPuO1miLpTHQiRgTKCLexL4MeAFVagts7HmNZ2Q==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=10"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
}
}
}
backend/package.json¶
{
"name": "livechat-backend",
"version": "1.0.0",
"main": "server.js",
"scripts": {
"test": "mocha tests/*.js --timeout 120000",
"test:providers": "mocha tests/provider-tests.js --timeout 120000",
"test:asr": "mocha tests/test-asr-providers.js --timeout 120000",
"test:llm": "mocha tests/test-llm-providers.js --timeout 120000",
"test:tts": "mocha tests/test-tts-providers.js --timeout 120000",
"check": "node check-setup.js",
"start": "node server.js"
},
"keywords": [],
"author": "",
"license": "ISC",
"description": "",
"dependencies": {
"axios": "^1.7.5",
"emoji-regex": "^10.4.0",
"express": "^4.21.1",
"fluent-ffmpeg": "^2.1.3",
"form-data": "^4.0.0",
"franc": "^6.2.0",
"node-wav": "^0.0.2",
"onnxruntime-node": "^1.22.0-rev",
"wav": "^1.0.2",
"ws": "^8.18.0"
},
"devDependencies": {
"mocha": "^10.2.0"
}
}
frontend/package.json¶
{
"name": "livechat-frontend",
"version": "0.1.0",
"private": true,
"scripts": {
"dev": "next dev",
"build": "cross-env NODE_ENV=production next build",
"start": "cross-env NODE_ENV=production next start",
"lint": "next lint"
},
"dependencies": {
"@radix-ui/react-slot": "^1.0.2",
"@types/node": "18.15.11",
"@types/react": "18.0.33",
"@types/react-dom": "18.0.11",
"class-variance-authority": "^0.6.0",
"clsx": "^1.2.1",
"lucide-react": "^0.130.1",
"next": "13.3.0",
"react": "18.2.0",
"react-dom": "18.2.0",
"react-markdown": "^9.0.1",
"react-syntax-highlighter": "^15.6.1",
"remark-gfm": "^4.0.0",
"tailwind-merge": "^1.10.0",
"tailwindcss-animate": "^1.0.5",
"typescript": "5.0.4",
"web-audio-resampler": "^1.0.1"
},
"devDependencies": {
"@tailwindcss/typography": "^0.5.15",
"autoprefixer": "^10.4.14",
"cross-env": "^7.0.3",
"eslint": "9.30.1",
"eslint-config-next": "15.3.5",
"postcss": "^8.4.21",
"tailwindcss": "^3.3.1"
}
}
frontend/tsconfig.json¶
{
"compilerOptions": {
"baseUrl": ".",
"paths": {
"@/*": ["./*"]
},
"lib": [
"dom",
"dom.iterable",
"esnext"
],
"allowJs": true,
"skipLibCheck": true,
"strict": false,
"forceConsistentCasingInFileNames": true,
"noEmit": true,
"incremental": true,
"esModuleInterop": true,
"module": "esnext",
"moduleResolution": "node",
"resolveJsonModule": true,
"isolatedModules": true,
"jsx": "preserve"
},
"include": [
"next-env.d.ts",
"**/*.ts",
"**/*.tsx"
],
"exclude": [
"node_modules"
]
}
backend/check-setup.js¶
#!/usr/bin/env node
/**
* check-setup.js — Headless environment verifier for the Live Voice Chat demo (实验 9-1).
*
* Runs WITHOUT any API keys, microphone, or browser. It only checks that the
* local prerequisites for the cascaded VAD -> ASR -> LLM -> TTS pipeline are in
* place, and reports which providers you have credentials for, so a reader can
* confirm their setup before opening the browser UI.
*
* Usage:
* node check-setup.js
*
* Exit code 0 means the backend can start; non-zero means a hard prerequisite
* (Node version, VAD model, or a loadable config) is missing.
*/
const { execFileSync } = require('child_process');
const fs = require('fs');
const path = require('path');
let hardFailures = 0;
let warnings = 0;
const ok = (msg) => console.log(` ✅ ${msg}`);
const warn = (msg) => { console.log(` ⚠️ ${msg}`); warnings++; };
const fail = (msg) => { console.log(` ❌ ${msg}`); hardFailures++; };
console.log('🔍 Live Voice Chat — setup check (实验 9-1)');
console.log('='.repeat(60));
// 1. Node version (README requires v16+)
console.log('\nNode.js runtime:');
const major = parseInt(process.versions.node.split('.')[0], 10);
if (major >= 16) {
ok(`Node ${process.version} (>= v16 required)`);
} else {
fail(`Node ${process.version} is too old; v16 or higher is required`);
}
// 2. FFmpeg (used by server.js to convert incoming audio for ASR)
console.log('\nFFmpeg (audio format conversion):');
try {
const out = execFileSync('ffmpeg', ['-version'], { encoding: 'utf8' });
ok(`ffmpeg found: ${out.split('\n')[0]}`);
} catch (e) {
fail('ffmpeg not found on PATH. Install it (e.g. `brew install ffmpeg`) — see README.');
}
// 3. Silero VAD model file
console.log('\nSilero VAD model:');
const modelPath = path.join(__dirname, 'models', 'silero_vad.onnx');
if (fs.existsSync(modelPath)) {
const kb = (fs.statSync(modelPath).size / 1024).toFixed(0);
ok(`silero_vad.onnx present (${kb} KB)`);
} else {
fail(`Missing ${modelPath}. Download it — see README "Download the Silero VAD model".`);
}
// 4. Config loads, and report selected providers + key availability
console.log('\nConfiguration & providers:');
let config;
try {
config = require('./config');
ok('config.js loaded');
} catch (e) {
fail(`config.js failed to load: ${e.message}`);
}
if (config) {
// Map each selected provider to the env var its credentials come from.
const keyFor = {
asr: (config.ASR_PROVIDERS[config.ASR_PROVIDER] || {}).apiKey,
llm: (config.LLM_PROVIDERS[config.LLM_PROVIDER] || {}).apiKey,
tts: (config.TTS_PROVIDERS[config.TTS_PROVIDER] || {}).apiKey,
};
const stageLine = (stage, provider, envVar) => {
if (!provider || !envVar) {
fail(`${stage.toUpperCase()}: provider "${provider}" is not defined in config.js`);
return;
}
const val = process.env[envVar];
const placeholder = !val || /your-.*-api-key-here/.test(val);
if (placeholder) {
warn(`${stage.toUpperCase()}: provider "${provider}" selected, but ${envVar} is not set`);
} else {
ok(`${stage.toUpperCase()}: provider "${provider}" ready (${envVar} set)`);
}
};
stageLine('asr', config.ASR_PROVIDER, keyFor.asr);
stageLine('llm', config.LLM_PROVIDER, keyFor.llm);
stageLine('tts', config.TTS_PROVIDER, keyFor.tts);
console.log('\nServer will listen on:');
ok(`ws://${config.LISTEN_HOST}:${config.LISTEN_PORT} (WebSocket) — frontend must use WEBSOCKET_PORT=${config.LISTEN_PORT}`);
}
// Summary
console.log('\n' + '='.repeat(60));
if (hardFailures === 0 && warnings === 0) {
console.log('✅ Setup looks good. Start the backend with: npm start');
} else if (hardFailures === 0) {
console.log(`⚠️ Prerequisites OK, but ${warnings} provider key(s) missing.`);
console.log(' The backend will start; set the missing API key(s) before recording.');
} else {
console.log(`❌ ${hardFailures} hard prerequisite(s) missing — fix these before running the backend.`);
}
process.exit(hardFailures === 0 ? 0 : 1);
backend/config.js¶
const config = {
// API Keys
OPENAI_API_KEY: process.env.OPENAI_API_KEY || 'your-openai-api-key-here',
OPENROUTER_API_KEY: process.env.OPENROUTER_API_KEY || 'your-openrouter-api-key-here',
ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY || 'your-anthropic-api-key-here',
ARK_API_KEY: process.env.ARK_API_KEY || 'your-ark-api-key-here',
SILICONFLOW_API_KEY: process.env.SILICONFLOW_API_KEY || 'your-siliconflow-api-key-here',
// Provider Selection
ASR_PROVIDER: 'siliconflow', // 'openai' (whisper-1) or 'siliconflow' (SenseVoice)
LLM_PROVIDER: 'openrouter', // 'openrouter' (gpt-5.6-luna, default), 'openai', 'openrouter-gemini', 'ark'
TTS_PROVIDER: 'siliconflow', // 'siliconflow' (CosyVoice2, keep current)
// ASR Configuration
ASR_PROVIDERS: {
openai: {
apiUrl: 'https://api.openai.com/v1/audio/transcriptions',
model: 'whisper-1',
apiKey: 'OPENAI_API_KEY'
},
siliconflow: {
apiUrl: 'https://api.siliconflow.cn/v1/audio/transcriptions',
model: 'FunAudioLLM/SenseVoiceSmall',
apiKey: 'SILICONFLOW_API_KEY'
}
},
// LLM Configuration
LLM_PROVIDERS: {
// OpenRouter with a current cheap flagship chat model (default / recommended:
// gpt-5.6* on OpenAI direct needs org verification, OpenRouter avoids that step).
openrouter: {
apiUrl: 'https://openrouter.ai/api/v1/chat/completions',
model: 'openai/gpt-5.6-luna',
apiKey: 'OPENROUTER_API_KEY'
},
openai: {
apiUrl: 'https://api.openai.com/v1/chat/completions',
model: 'gpt-5.6-luna',
apiKey: 'OPENAI_API_KEY'
},
'openrouter-gpt': {
apiUrl: 'https://openrouter.ai/api/v1/chat/completions',
model: 'openai/gpt-5.6-luna',
apiKey: 'OPENROUTER_API_KEY'
},
'openrouter-gemini': {
apiUrl: 'https://openrouter.ai/api/v1/chat/completions',
model: 'google/gemini-3.5-flash',
apiKey: 'OPENROUTER_API_KEY'
},
ark: {
apiUrl: 'https://ark.cn-beijing.volces.com/api/v3/chat/completions',
model: 'doubao-seed-1-6-flash-250615',
apiKey: 'ARK_API_KEY'
}
},
// TTS Configuration (keep current)
TTS_PROVIDERS: {
siliconflow: {
apiUrl: 'https://api.siliconflow.cn/v1/audio/speech',
model: 'FunAudioLLM/CosyVoice2-0.5B',
voice: 'FunAudioLLM/CosyVoice2-0.5B:diana',
apiKey: 'SILICONFLOW_API_KEY'
}
},
// Legacy support (will be deprecated)
LLM_MODEL: 'gpt-5.6-luna',
LLM_API_URL: 'https://api.openai.com/v1/chat/completions',
STT_API_URL: 'https://api.openai.com/v1/audio/transcriptions',
STT_MODEL: 'whisper-1',
TTS_API_URL: 'https://api.siliconflow.cn/v1/audio/speech',
// Common Configuration
VISION_MAX_TOKENS: 4096,
// Silero VAD Configuration
VAD_THRESHOLD: 0.5, // Speech probability threshold for Silero VAD (0.0 to 1.0)
VAD_FRAME_LENGTH: 512, // Frame length for VAD analysis (samples)
VAD_MIN_SPEECH_DURATION: 250, // Minimum speech duration in ms
VAD_MAX_SILENCE_DURATION: 500, // Maximum silence duration before ending speech in ms
AUDIO_SAMPLE_RATE: 16000, // Sample rate for audio processing (required for Silero VAD)
AUDIO_CHUNK_SIZE: 4096, // Audio chunk size for processing
// Server Configuration
LISTEN_PORT: 8848,
LISTEN_HOST: '0.0.0.0',
SYSTEM_PROMPT: 'You are a helpful AI assistant.',
CANCEL_PLAYBACK_TIME_THRESHOLD: 3000,
};
module.exports = config;
backend/run-tests.js¶
#!/usr/bin/env node
/**
* Test runner for provider tests
* This script sets up the environment and runs comprehensive tests for all provider combinations
*
* Usage:
* node run-tests.js
*
* Environment variables required:
* - OPENROUTER_API_KEY: OpenRouter API key
* - ANTHROPIC_API_KEY: Anthropic API key
* - ARK_API_KEY: ARK (Doubao) API key
* - SILICONFLOW_API_KEY: Siliconflow API key
* - OPENAI_API_KEY: OpenAI API key (optional if using others)
*/
const { spawn } = require('child_process');
const path = require('path');
console.log('🧪 Starting Provider Tests for Live Audio Backend');
console.log('=' .repeat(60));
// Check environment variables
const requiredEnvVars = {
'OPENROUTER_API_KEY': 'OpenRouter API key',
'ANTHROPIC_API_KEY': 'Anthropic API key',
'ARK_API_KEY': 'ARK (Doubao) API key',
'SILICONFLOW_API_KEY': 'Siliconflow API key'
};
const missingKeys = [];
const availableKeys = [];
Object.entries(requiredEnvVars).forEach(([key, description]) => {
if (process.env[key]) {
availableKeys.push(`✅ ${description}`);
} else {
missingKeys.push(`❌ ${description} (${key})`);
}
});
console.log('\n🔑 API Key Status:');
availableKeys.forEach(key => console.log(` ${key}`));
missingKeys.forEach(key => console.log(` ${key}`));
if (process.env.OPENAI_API_KEY) {
console.log(` ✅ OpenAI API key (optional)`);
}
console.log('\n📋 Test Plan:');
console.log(' 1. ASR Provider Tests (OpenAI Whisper, SenseVoice)');
console.log(' 2. LLM Provider Tests (OpenAI, OpenRouter GPT-4o, OpenRouter Gemini, ARK Doubao)');
console.log(' 3. Integration Tests (All ASR+LLM combinations)');
console.log(' 4. Provider Switching Tests');
console.log('\n🚀 Running Tests...\n');
// Run the tests
const testProcess = spawn('npm', ['run', 'test:providers'], {
stdio: 'inherit',
cwd: __dirname,
env: process.env
});
testProcess.on('close', (code) => {
console.log('\n' + '='.repeat(60));
if (code === 0) {
console.log('✅ All tests completed successfully!');
console.log('\n📊 Test Summary:');
console.log(' - Provider creation and configuration ✓');
console.log(' - ASR transcription functionality ✓');
console.log(' - LLM chat completion functionality ✓');
console.log(' - Provider integration ✓');
console.log(' - Dynamic provider switching ✓');
} else {
console.log(`❌ Tests failed with exit code ${code}`);
console.log('\n🔧 Troubleshooting:');
console.log(' 1. Ensure all required API keys are set as environment variables');
console.log(' 2. Check network connectivity to API endpoints');
console.log(' 3. Verify API key permissions and quotas');
console.log(' 4. Check the test output above for specific error details');
}
process.exit(code);
});
testProcess.on('error', (error) => {
console.error('❌ Failed to start test process:', error);
process.exit(1);
});
backend/server.js¶
const express = require('express');
const http = require('http');
const WebSocket = require('ws');
const axios = require('axios');
const config = require('./config');
const { preprocessSentence } = require('./utils/textProcessor');
const VoiceActivityDetector = require('./utils/vad');
const SpeechToTextService = require('./utils/speechToText');
const { LLMProviderFactory } = require('./utils/providers/llmProviders');
const fs = require('fs');
const path = require('path');
const ffmpeg = require('fluent-ffmpeg');
// Import franc dynamically at the top level
let franc;
(async () => {
const francModule = await import('franc');
franc = francModule.franc;
})();
const app = express();
const server = http.createServer(app);
const wss = new WebSocket.Server({ server });
// Create a connection handler class to manage state for each connection
class ConnectionHandler {
constructor(ws) {
this.ws = ws;
this.messageHistory = [];
this.currentLLMRequest = null;
this.latestImage = null;
this.currentTTSRequest = null;
this.lastProcessedTranscript = null;
this.playbackStartTime = null;
this.lastUserMessageId = null;
this.currentMessageId = null; // Track current message ID
this.llmOrTTSIsWorking = false;
this.ttsQueue = []; // Queue for pending TTS requests
this.isTTSProcessing = false; // Flag to track if TTS is currently processing
this.lastAudioEndTime = null; // Track when the last audio chunk will finish playing
this.audioTotalDuration = 0; // Track total duration of queued audio
// Add properties for audio recording
this.audioChunks = [];
this.isRecording = false;
this.recordingStartTime = null;
this.recordingPath = path.join(__dirname, 'recordings');
// Create recordings directory if it doesn't exist
if (!fs.existsSync(this.recordingPath)) {
fs.mkdirSync(this.recordingPath, { recursive: true });
}
this.audioFormat = null; // Add this property
this.expectedPlaybackEndTime = null; // Add this property
this.lastSyncedHistoryLength = 0; // Track how much of history has been synced
this.lastSyncedHistory = []; // Keep track of last synced state
// Initialize VAD and STT services
this.vad = new VoiceActivityDetector();
this.sttService = new SpeechToTextService();
// Initialize LLM provider
this.initializeLLMProvider();
// VAD processing state
this.isProcessingSTT = false;
this.pendingAudioBuffer = Buffer.alloc(0);
this.setupWebSocketHandlers();
// Start periodic cleanup of temp files
this.cleanupInterval = setInterval(() => {
this.sttService.cleanupTempFiles();
}, 5 * 60 * 1000); // Every 5 minutes
}
/**
* Initialize LLM provider based on configuration
*/
initializeLLMProvider() {
try {
const providerName = config.LLM_PROVIDER || 'openai';
this.llmProvider = LLMProviderFactory.createProvider(providerName, config, config);
console.log(`LLM Provider initialized: ${providerName}`);
} catch (error) {
console.error('Failed to initialize LLM provider:', error);
throw error;
}
}
/**
* Switch LLM provider dynamically
* @param {string} providerName - Provider name to switch to
*/
switchLLMProvider(providerName) {
try {
this.llmProvider = LLMProviderFactory.createProvider(providerName, config, config);
console.log(`LLM Provider switched to: ${providerName}`);
} catch (error) {
console.error('Failed to switch LLM provider:', error);
throw error;
}
}
setupWebSocketHandlers() {
this.ws.on('message', this.handleMessage.bind(this));
this.ws.on('close', this.handleClose.bind(this));
this.ws.on('error', this.handleError.bind(this));
}
async handleMessage(message) {
try {
// Try to parse as JSON first
const jsonMessage = JSON.parse(message);
// Handle ping message by sending back pong with same timestamp
if (jsonMessage.type === 'ping') {
this.ws.send(JSON.stringify({
type: 'pong',
timestamp: jsonMessage.timestamp
}));
return;
}
// Handle image message
if (jsonMessage.type === 'image') {
this.latestImage = jsonMessage.data;
console.log('Received new image from camera');
return;
}
// Handle other ping message by sending back pong with same timestamp
if (jsonMessage.type === 'ping') {
this.ws.send(JSON.stringify({
type: 'pong',
timestamp: jsonMessage.timestamp
}));
return;
}
console.log('Received JSON message:', jsonMessage);
} catch (e) {
// If not JSON, treat as binary data
if (Buffer.isBuffer(message)) {
// Start recording if not already started
if (!this.isRecording) {
this.startRecording();
}
// Store the audio chunk
this.audioChunks.push(message);
// Process audio through VAD
this.processAudioWithVAD(message);
} else {
console.error('Received unexpected data type:', typeof message);
}
}
}
handleClose() {
console.log('Client disconnected');
this.cleanup();
}
handleError(error) {
console.error('WebSocket error:', error);
this.cleanup();
}
cleanup() {
if (this.currentLLMRequest) {
this.currentLLMRequest.cancel();
}
if (this.currentTTSRequest) {
this.currentTTSRequest.cancel();
}
this.ttsQueue = [];
this.lastAudioEndTime = null;
this.audioTotalDuration = 0;
// Clean up VAD state (now async)
if (this.vad) {
this.vad.cleanup().catch(error => {
console.error('Error cleaning up VAD:', error);
});
}
// Clear cleanup interval
if (this.cleanupInterval) {
clearInterval(this.cleanupInterval);
}
// Save any remaining recording
if (this.isRecording) {
this.isRecording = false;
this.saveRecording();
this.audioChunks = [];
}
// Clear pending audio buffer
this.pendingAudioBuffer = Buffer.alloc(0);
}
logEvent(type, details = {}) {
const timestamp = new Date().toISOString();
console.log(`[${timestamp}] Event: ${type}`, {
...details,
connectionId: this.ws._socket.remoteAddress
});
}
// Add helper method to generate message IDs
generateMessageId() {
return Math.random().toString(36).substring(2, 15);
}
/**
* Process audio chunk through VAD
* @param {Buffer} audioChunk - Raw audio data
*/
async processAudioWithVAD(audioChunk) {
try {
// Process audio through VAD (now async)
const vadResults = await this.vad.processAudioChunk(audioChunk);
for (const result of vadResults) {
if (result.type === 'speech_end') {
// Speech segment ended, process with STT
await this.processSpeechSegment(result.audioData, result.duration);
}
}
} catch (error) {
console.error('Error processing audio with VAD:', error);
}
}
/**
* Process speech segment with STT
* @param {Buffer} audioData - Speech audio data
* @param {number} duration - Duration of speech in ms
*/
async processSpeechSegment(audioData, duration) {
if (this.isProcessingSTT) {
console.log('STT already processing, skipping this segment');
return;
}
try {
this.isProcessingSTT = true;
// Check if audio has sufficient content
if (!this.sttService.hasSufficientAudio(audioData)) {
console.log('Insufficient audio content for STT');
return;
}
console.log(`Processing speech segment: ${duration}ms, ${audioData.length} bytes`);
// Send processing start notification
this.ws.send(JSON.stringify({
type: 'stt_start',
duration: duration,
timestamp: Date.now()
}));
// Generate message ID for this speech segment
this.currentMessageId = this.generateMessageId();
// Transcribe audio
const result = await this.sttService.transcribeAudio(audioData);
if (result.success && result.text.trim()) {
const transcript = result.text.trim();
// Send transcript result
this.ws.send(JSON.stringify({
type: 'transcript',
text: transcript,
isFinal: true,
messageId: this.currentMessageId,
language: result.language,
duration: result.duration,
confidence: result.confidence
}));
this.logEvent('transcript', {
text: transcript,
isFinal: true,
messageId: this.currentMessageId,
language: result.language,
sttDuration: result.duration
});
// Only process if this transcript is different from the last one we processed
if (transcript !== this.lastProcessedTranscript) {
this.lastProcessedTranscript = transcript;
// Update message history
const lastMessage = this.messageHistory[this.messageHistory.length - 1];
const isLastMessageUser = lastMessage &&
(lastMessage.role === 'user' || lastMessage.role === 'transcript');
if (isLastMessageUser) {
// Replace the last user message
this.messageHistory = [
...this.messageHistory.slice(0, -1),
{ role: 'user', content: transcript, messageId: this.currentMessageId }
];
} else {
// Append new message
this.messageHistory.push({
role: 'user',
content: transcript,
messageId: this.currentMessageId
});
}
this.syncChatHistory();
this.generateAIResponse();
}
} else if (!result.success) {
console.error('STT failed:', result.error);
this.ws.send(JSON.stringify({
type: 'stt_error',
error: result.error,
timestamp: Date.now()
}));
}
} catch (error) {
console.error('Error processing speech segment:', error);
this.ws.send(JSON.stringify({
type: 'stt_error',
error: error.message,
timestamp: Date.now()
}));
} finally {
this.isProcessingSTT = false;
}
}
async generateAIResponse() {
this.llmOrTTSIsWorking = true;
try {
this.currentLLMRequest = axios.CancelToken.source();
this.ws.send(JSON.stringify({ type: 'llm_start' }));
this.logEvent('llm_start');
let hasReceivedFirstToken = false;
let hasReceivedFirstSentence = false;
let currentSentence = '';
let isFirstSentence = true;
// Keep only the last 20 messages for context
const recentHistory = this.messageHistory.slice(-20);
let messages = [
{ role: 'system', content: config.SYSTEM_PROMPT },
...recentHistory.map(({ role, content }) => ({ role, content }))
];
// If we have a latest image, include it in the messages
if (this.latestImage) {
// Find the last user message
const lastUserMessageIndex = messages.findIndex(msg => msg.role === 'user');
if (lastUserMessageIndex !== -1) {
// Add image to the user's message
messages[lastUserMessageIndex] = {
role: 'user',
content: [
{
type: 'image_url',
image_url: {
url: this.latestImage,
detail: 'low'
}
},
{
type: 'text',
text: messages[lastUserMessageIndex].content
}
]
};
}
// Clear the image after using it
this.latestImage = null;
}
// Add initial empty assistant message
this.messageHistory.push({
role: 'assistant',
content: '',
messageId: this.currentMessageId
});
this.syncChatHistory();
let accumulatedContent = ''; // Track all content received so far
// Use LLM provider for chat completion
const providerResult = await this.llmProvider.createChatCompletion(messages, {
max_tokens: config.VISION_MAX_TOKENS,
cancelToken: this.currentLLMRequest.token
});
if (!providerResult.success) {
throw new Error(providerResult.error);
}
const response = providerResult.response;
let buffer = ''; // Add this buffer to store incomplete lines
response.data.on('data', async chunk => {
try {
buffer += chunk.toString();
const lines = buffer.split('\n');
// Keep the last line if it's incomplete
buffer = lines.pop() || '';
for (const line of lines) {
const trimmedLine = line.trim();
if (!trimmedLine || trimmedLine === '[DONE]') continue;
if (!trimmedLine.startsWith('data: ')) continue;
try {
const jsonData = JSON.parse(trimmedLine.replace('data: ', ''));
const content = jsonData.choices[0]?.delta?.content || '';
if (!content) continue;
if (!hasReceivedFirstToken) {
hasReceivedFirstToken = true;
this.ws.send(JSON.stringify({
type: 'llm_first_token',
messageId: this.currentMessageId
}));
}
currentSentence += content;
accumulatedContent += content;
if (this.isCompleteSentence(currentSentence, isFirstSentence)) {
// Update message history with accumulated content after each complete sentence
const lastMessage = this.messageHistory[this.messageHistory.length - 1];
lastMessage.content = accumulatedContent;
this.syncChatHistory();
const completedSentence = currentSentence;
currentSentence = '';
isFirstSentence = false;
this.ws.send(JSON.stringify({
type: 'llm_sentence',
text: completedSentence,
messageId: this.currentMessageId
}));
if (!hasReceivedFirstSentence) {
hasReceivedFirstSentence = true;
this.ws.send(JSON.stringify({
type: 'llm_first_sentence',
messageId: this.currentMessageId
}));
}
await this.synthesizeAndStreamAudio(completedSentence);
}
} catch (parseError) {
console.error('Error parsing JSON data:', parseError);
continue;
}
}
} catch (error) {
console.error('Error processing LLM stream:', error);
}
});
// Handle any remaining data in the buffer when the stream ends
response.data.on('end', async () => {
try {
if (buffer) {
const trimmedLine = buffer.trim();
if (trimmedLine && trimmedLine !== '[DONE]' && trimmedLine.startsWith('data: ')) {
const jsonData = JSON.parse(trimmedLine.replace('data: ', ''));
const content = jsonData.choices[0]?.delta?.content || '';
if (content) {
currentSentence += content;
accumulatedContent += content;
}
}
}
// Handle any remaining content
if (currentSentence.trim()) {
await this.synthesizeAndStreamAudio(currentSentence);
accumulatedContent += currentSentence;
// Update message history with final content
const lastMessage = this.messageHistory[this.messageHistory.length - 1];
lastMessage.content = accumulatedContent;
this.syncChatHistory();
}
this.ws.send(JSON.stringify({
type: 'ai_response_complete',
messageId: this.currentMessageId
}));
} catch (error) {
console.error('Error processing final LLM data:', error);
}
});
} catch (error) {
this.llmOrTTSIsWorking = false;
if (!axios.isCancel(error)) {
console.error('Error generating AI response:', error);
this.ws.send(JSON.stringify({ type: 'error', message: 'Error generating AI response' }));
this.logEvent('error', { message: 'Error generating AI response', error: error.toString() });
}
}
}
isCompleteSentence(sentence, isFirstSentence) {
const trimmedSentence = sentence.trim();
// Check if we're inside a markdown code block
if (trimmedSentence.includes('```')) {
const count = (trimmedSentence.match(/```/g) || []).length;
// If the count is odd, we're still inside a code block
if (count % 2 !== 0) {
return false;
}
}
// Check if the sentence is a function call
if (trimmedSentence.includes('<function>')) {
return trimmedSentence.includes('</function>');
}
// Check for newline
if (sentence.endsWith('\n')) {
return true;
}
// Check for period, but not if it's a numbered list item
if (trimmedSentence.endsWith('.')) {
if (/[0-9]+\.$/.test(trimmedSentence)) {
return false;
}
return true;
}
// Check for question mark or exclamation mark
if (trimmedSentence.endsWith('?') || trimmedSentence.endsWith('!')) {
return true;
}
// Check for Chinese punctuation
if (trimmedSentence.endsWith('。') || trimmedSentence.endsWith('？') || trimmedSentence.endsWith('！')) {
return true;
}
// Check for semicolons (both English and Chinese)
if (trimmedSentence.endsWith(';') || trimmedSentence.endsWith('；')) {
return true;
}
// Check if sentence ends with an emoji
if (trimmedSentence.length > 0) {
const lastChar = trimmedSentence.slice(-2); // Take last 2 chars for emoji
const emojiRegex = /[\u{1F300}-\u{1F9FF}]|[\u{2600}-\u{26FF}]/u;
if (emojiRegex.test(lastChar)) {
return true;
}
}
// First sentence should be as short as possible
if (isFirstSentence) {
if (trimmedSentence.endsWith(',') || trimmedSentence.endsWith('，')) {
return true;
}
}
return false;
}
async synthesizeAndStreamAudio(text) {
try {
// Skip empty or whitespace-only strings
if (!text || !text.trim()) {
console.log('Skipping TTS for empty string');
return;
}
// Add text to the queue
this.ttsQueue.push(text);
// Try to process the queue
await this.processTTSQueue();
} catch (error) {
if (!axios.isCancel(error)) {
console.error('Error in TTS:', error);
this.logEvent('error', { message: 'TTS error', error: error.toString() });
}
this.isTTSProcessing = false;
}
}
// Helper method to calculate audio duration from WAV buffer
calculateAudioDuration(audioBuffer) {
if (audioBuffer.length < 44) return 0;
const sampleRate = audioBuffer.readUInt32LE(24);
const numChannels = audioBuffer.readUInt16LE(22);
const bitsPerSample = audioBuffer.readUInt16LE(34);
const dataSize = audioBuffer.length - 44; // Subtract WAV header size
// Calculate duration in milliseconds
const duration = Math.floor(
(dataSize * 8 * 1000) / (sampleRate * numChannels * bitsPerSample)
);
return duration;
}
// Add method to start a new recording
startRecording() {
this.audioChunks = [];
this.isRecording = true;
this.recordingStartTime = new Date();
}
// Add method to save the recording
saveRecording() {
if (this.audioChunks.length === 0) return;
const timestamp = this.recordingStartTime.toISOString().replace(/[:.]/g, '-');
const fileName = `recording_${timestamp}.wav`;
const filePath = path.join(this.recordingPath, fileName);
// Create WAV header
const dataSize = this.audioChunks.reduce((acc, chunk) => acc + chunk.length, 0);
const header = Buffer.alloc(44);
// RIFF chunk descriptor
header.write('RIFF', 0);
header.writeUInt32LE(36 + dataSize, 4);
header.write('WAVE', 8);
// fmt sub-chunk
header.write('fmt ', 12);
header.writeUInt32LE(16, 16); // Subchunk1Size
header.writeUInt16LE(1, 20); // AudioFormat (PCM)
header.writeUInt16LE(1, 22); // NumChannels (Mono)
header.writeUInt32LE(16000, 24); // SampleRate
header.writeUInt32LE(32000, 28); // ByteRate (SampleRate * NumChannels * BitsPerSample/8)
header.writeUInt16LE(2, 32); // BlockAlign (NumChannels * BitsPerSample/8)
header.writeUInt16LE(16, 34); // BitsPerSample
// data sub-chunk
header.write('data', 36);
header.writeUInt32LE(dataSize, 40);
// Write header and audio data to file
const writeStream = fs.createWriteStream(filePath);
writeStream.write(header);
for (const chunk of this.audioChunks) {
writeStream.write(chunk);
}
writeStream.end();
this.logEvent('recording_saved', { filePath, size: dataSize + 44 });
}
async detectLanguage(text) {
try {
if (!franc) {
const francModule = await import('franc');
franc = francModule.franc;
}
const detectedLang = franc(text, { minLength: 1 });
const langMap = {
'cmn': 'zh',
'eng': 'en',
'jpn': 'jp',
'zho': 'zh',
'chi': 'zh',
'und': 'en'
};
return langMap[detectedLang] || 'en';
} catch (error) {
console.error('Language detection error:', error);
return 'en';
}
}
// Add this method to process TTS queue
async processTTSQueue() {
if (this.isTTSProcessing || this.ttsQueue.length === 0) return;
this.isTTSProcessing = true;
const now = Date.now();
// Calculate remaining audio duration
const remainingDuration = this.lastAudioEndTime
? Math.max(0, this.lastAudioEndTime - now)
: 0;
// Only process next TTS if remaining audio is less than 5 seconds
if (remainingDuration <= 5000) {
const text = this.ttsQueue[0];
const ttsStartTime = Date.now();
try {
this.ws.send(JSON.stringify({ type: 'tts_start' }));
this.logEvent('tts_start');
const detectedLang = await this.detectLanguage(text);
this.logEvent('language_detection', { text, detectedLang });
const processedText = preprocessSentence(text, detectedLang);
// Skip TTS if processed text is empty
if (!processedText.trim()) {
this.ttsQueue.shift(); // Remove empty text from queue
this.isTTSProcessing = false;
// Process next item in queue if any
if (this.ttsQueue.length > 0) {
setTimeout(() => this.processTTSQueue(), 100);
}
return;
}
this.currentTTSRequest = axios.CancelToken.source();
const response = await axios({
method: 'post',
url: config.TTS_API_URL,
data: {
"model": (config.TTS_PROVIDERS?.[config.TTS_PROVIDER]?.model) || "FunAudioLLM/CosyVoice2-0.5B",
"input": text,
"voice": (config.TTS_PROVIDERS?.[config.TTS_PROVIDER]?.voice) || "FunAudioLLM/CosyVoice2-0.5B:diana",
"response_format": "mp3",
"sample_rate": 32000,
"stream": true,
"speed": 1,
"gain": 0
},
headers: {
"Authorization": `Bearer ${config.SILICONFLOW_API_KEY}`,
"Content-Type": "application/json"
},
responseType: 'stream',
cancelToken: this.currentTTSRequest.token,
});
let headerBuffer = Buffer.alloc(0);
let headerSent = false;
let totalDataSize = 0;
response.data.on('data', async chunk => {
if (!headerSent) {
headerBuffer = Buffer.concat([headerBuffer, chunk]);
if (headerBuffer.length >= 44) {
const sampleRate = headerBuffer.readUInt32LE(24);
const numChannels = headerBuffer.readUInt16LE(22);
const bitsPerSample = headerBuffer.readUInt16LE(34);
this.audioFormat = { sampleRate, numChannels, bitsPerSample };
const needsResampling = sampleRate !== 16000 || numChannels !== 1;
if (needsResampling) {
this.audioBuffer = Buffer.from(headerBuffer);
headerBuffer = Buffer.alloc(0);
headerSent = true;
return;
}
const bytesPerSample = bitsPerSample / 8;
const samplesPerSecond = sampleRate * numChannels;
const bytesPerSecond = samplesPerSecond * bytesPerSample;
const chunkSize = Math.floor(bytesPerSecond * 0.05); // 50ms worth of audio data
this.ws.send(JSON.stringify({
type: 'audio_start',
format: {
sampleRate,
numChannels,
bitsPerSample
}
}));
// Set playback start time immediately
if (!this.playbackStartTime) {
this.playbackStartTime = Date.now();
}
// Start sending audio data immediately (skip the header and data chunk header)
if (headerBuffer.length > 44) {
let remainingData = headerBuffer.slice(44);
// Skip data chunk header (8 bytes) if present
if (remainingData.length >= 8 &&
remainingData.slice(0, 4).toString() === 'data') {
remainingData = remainingData.slice(8);
}
while (remainingData.length >= chunkSize) {
const chunk = remainingData.slice(0, chunkSize);
this.ws.send(chunk);
totalDataSize += chunk.length;
remainingData = remainingData.slice(chunkSize);
}
headerBuffer = remainingData.length > 0 ? remainingData : Buffer.alloc(0);
}
headerSent = true;
}
} else {
if (this.audioFormat.sampleRate !== 16000 || this.audioFormat.numChannels !== 1) {
this.audioBuffer = Buffer.concat([this.audioBuffer, chunk]);
} else {
const bytesPerSample = this.audioFormat.bitsPerSample / 8;
const samplesPerSecond = this.audioFormat.sampleRate * this.audioFormat.numChannels;
const bytesPerSecond = samplesPerSecond * bytesPerSample;
const chunkSize = Math.floor(bytesPerSecond * 0.05);
let audioData = Buffer.concat([headerBuffer, chunk]);
headerBuffer = Buffer.alloc(0);
while (audioData.length >= chunkSize) {
const chunkToSend = audioData.slice(0, chunkSize);
this.ws.send(chunkToSend);
totalDataSize += chunkSize;
audioData = audioData.slice(chunkSize);
}
if (audioData.length > 0) {
headerBuffer = audioData;
}
}
}
});
response.data.on('end', async () => {
try {
if (this.audioFormat.sampleRate !== 16000 || this.audioFormat.numChannels !== 1) {
// Create temporary files for resampling
const inputPath = path.join(__dirname, `temp_input_${Date.now()}.wav`);
const outputPath = path.join(__dirname, `temp_output_${Date.now()}.wav`);
// Write the input audio buffer to a temporary file
fs.writeFileSync(inputPath, this.audioBuffer);
// Create a promise to handle the ffmpeg conversion
await new Promise((resolve, reject) => {
ffmpeg(inputPath)
.toFormat('wav')
.outputOptions([
'-acodec pcm_s16le', // Set codec to 16-bit PCM
'-ar 16000', // Set sample rate to 16kHz
'-ac 1' // Set to mono channel
])
.on('error', (err) => {
console.error('FFmpeg error:', err);
reject(err);
})
.on('end', () => resolve())
.save(outputPath);
});
// Read the resampled audio
const resampledBuffer = fs.readFileSync(outputPath);
// Clean up temporary files
fs.unlinkSync(inputPath);
fs.unlinkSync(outputPath);
// Send audio start with resampled format
this.ws.send(JSON.stringify({
type: 'audio_start',
format: {
sampleRate: 16000,
numChannels: 1,
bitsPerSample: 16
}
}));
// Set playback start time
if (!this.playbackStartTime) {
this.playbackStartTime = Date.now();
}
// Find the data chunk in the resampled buffer
let dataStart = 44; // Start looking after standard WAV header
while (dataStart < resampledBuffer.length - 8) {
if (resampledBuffer.readUInt32LE(dataStart) === 0x61746164) { // "data" chunk ID
dataStart += 8; // Skip "data" ID (4 bytes) and chunk size (4 bytes)
break;
}
dataStart++;
}
// Calculate chunk size for 50ms of audio at 16kHz mono
const chunkSize = Math.floor(16000 * 2 * 0.05); // 16kHz * 16-bit * 50ms
// Send audio data in chunks, starting after the data header
let currentPos = dataStart;
while (currentPos < resampledBuffer.length) {
const end = Math.min(currentPos + chunkSize, resampledBuffer.length);
const audioChunk = resampledBuffer.slice(currentPos, end);
this.ws.send(audioChunk);
totalDataSize += audioChunk.length;
currentPos += chunkSize;
}
} else {
// For non-resampled audio, process the remaining audio buffer
const audioData = this.audioBuffer;
if (audioData) {
// Calculate chunk size for 50ms of audio
const chunkSize = Math.floor(16000 * 2 * 0.05);
// Send audio data in chunks
let currentPos = dataStart;
while (currentPos < audioData.length) {
const end = Math.min(currentPos + chunkSize, audioData.length);
const audioChunk = audioData.slice(currentPos, end);
this.ws.send(audioChunk);
totalDataSize += audioChunk.length;
currentPos += chunkSize;
}
}
}
const synthesisTime = Date.now() - ttsStartTime;
this.ws.send(JSON.stringify({
type: 'tts_complete',
synthesisTime
}));
this.ws.send(JSON.stringify({ type: 'audio_end' }));
this.logEvent('tts_complete', { synthesisTime });
// Calculate audio duration and update expected playback end time
const audioDuration = Math.floor((totalDataSize * 8 * 1000) / (16000 * 1 * 16));
this.expectedPlaybackEndTime = (this.expectedPlaybackEndTime || Date.now()) + audioDuration;
this.lastAudioEndTime = Date.now() + audioDuration;
// Remove the processed text from queue
this.ttsQueue.shift();
this.isTTSProcessing = false;
// Process next item in queue if any
if (this.ttsQueue.length > 0) {
setTimeout(() => this.processTTSQueue(), 100);
}
} catch (error) {
console.error('Error processing audio:', error);
this.logEvent('error', { message: 'Audio processing error', error: error.toString() });
// Clean up state
this.ttsQueue.shift();
this.isTTSProcessing = false;
}
});
response.data.on('error', (error) => {
console.error('Error in TTS stream:', error);
this.logEvent('error', { message: 'TTS stream error', error: error.toString() });
this.ttsQueue.shift(); // Remove failed text from queue
this.isTTSProcessing = false;
});
} catch (error) {
if (!axios.isCancel(error)) {
console.error('Error in TTS:', error);
this.logEvent('error', { message: 'TTS error', error: error.toString() });
}
this.ttsQueue.shift(); // Remove failed text from queue
this.isTTSProcessing = false;
}
} else {
this.isTTSProcessing = false;
// Try again later if there's still too much audio in the queue
setTimeout(() => this.processTTSQueue(), 100);
}
}
// Update the syncChatHistory method to implement delta sync
syncChatHistory() {
const currentHistory = this.messageHistory;
const lastHistory = this.lastSyncedHistory;
// Find the index where histories start to differ
let diffStartIndex = 0;
while (diffStartIndex < lastHistory.length &&
diffStartIndex < currentHistory.length &&
JSON.stringify(lastHistory[diffStartIndex]) === JSON.stringify(currentHistory[diffStartIndex])) {
diffStartIndex++;
}
// Get the new or modified messages
const updatedMessages = currentHistory.slice(diffStartIndex);
// Send delta update
this.ws.send(JSON.stringify({
type: 'chat_history_delta',
startIndex: diffStartIndex,
messages: updatedMessages
}));
// Update last synced state
this.lastSyncedHistory = JSON.parse(JSON.stringify(currentHistory));
}
}
// Handle new WebSocket connections
wss.on('connection', (ws) => {
console.log('Client connected');
new ConnectionHandler(ws);
});
server.listen(config.LISTEN_PORT, config.LISTEN_HOST, () => {
console.log(`Server is running on ${config.LISTEN_HOST}:${config.LISTEN_PORT}`);
});
backend/test_llm.js¶
const axios = require('axios');
const config = require('./config');
async function testLLMEndpoint() {
console.log('Testing LLM endpoint...');
console.log(`URL: ${config.LLM_API_URL}`);
console.log(`Model: ${config.LLM_MODEL}`);
const testPrompt = "Tell me a short joke.";
try {
const response = await axios.post(
config.LLM_API_URL,
{
model: config.LLM_MODEL,
messages: [
{ role: "user", content: testPrompt }
],
stream: true,
max_tokens: 100
},
{
headers: {
'Authorization': `Bearer ${config.OPENAI_API_KEY}`,
'Content-Type': 'application/json',
},
responseType: 'stream'
}
);
console.log('Connection established successfully');
let fullResponse = '';
response.data.on('data', chunk => {
const lines = chunk.toString().split('\n');
for (const line of lines) {
if (line.trim() === '') continue;
if (line.trim() === 'data: [DONE]') continue;
if (!line.startsWith('data: ')) continue;
try {
const jsonData = JSON.parse(line.replace('data: ', ''));
const content = jsonData.choices[0]?.delta?.content || '';
fullResponse += content;
process.stdout.write(content); // Stream the response
} catch (e) {
console.error('Error parsing chunk:', e);
console.error('Raw chunk:', line);
}
}
});
response.data.on('end', () => {
console.log('\n\nFull response received:', fullResponse);
console.log('\nTest completed successfully');
});
response.data.on('error', (error) => {
console.error('Stream error:', error);
});
} catch (error) {
console.error('\nError testing LLM endpoint:', error.message);
if (error.response) {
console.error('Response status:', error.response.status);
console.error('Response headers:', JSON.stringify(error.response.headers, null, 2));
}
// Log safe error properties
const safeError = {
message: error.message,
name: error.name,
stack: error.stack,
code: error.code,
status: error.status
};
console.error('\nError details:', JSON.stringify(safeError, null, 2));
}
}
// Run the test
testLLMEndpoint();
backend/tests/provider-tests.js¶
const assert = require('assert');
const fs = require('fs');
const path = require('path');
// Import provider factories
const { ASRProviderFactory } = require('../utils/providers/asrProviders');
const { LLMProviderFactory } = require('../utils/providers/llmProviders');
// Import services
const SpeechToTextService = require('../utils/speechToText');
// Mock configuration for testing
const testConfig = {
// API Keys from environment variables
OPENAI_API_KEY: process.env.OPENAI_API_KEY,
OPENROUTER_API_KEY: process.env.OPENROUTER_API_KEY,
ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY,
ARK_API_KEY: process.env.ARK_API_KEY,
SILICONFLOW_API_KEY: process.env.SILICONFLOW_API_KEY,
// Provider configurations
ASR_PROVIDERS: {
openai: {
apiUrl: 'https://api.openai.com/v1/audio/transcriptions',
model: 'whisper-1',
apiKey: 'OPENAI_API_KEY'
},
siliconflow: {
apiUrl: 'https://api.siliconflow.cn/v1/audio/transcriptions',
model: 'FunAudioLLM/SenseVoiceSmall',
apiKey: 'SILICONFLOW_API_KEY'
}
},
LLM_PROVIDERS: {
openai: {
apiUrl: 'https://api.openai.com/v1/chat/completions',
model: 'gpt-5.6-luna',
apiKey: 'OPENAI_API_KEY'
},
'openrouter-gpt': {
apiUrl: 'https://openrouter.ai/api/v1/chat/completions',
model: 'openai/gpt-5.6-luna',
apiKey: 'OPENROUTER_API_KEY'
},
'openrouter-gemini': {
apiUrl: 'https://openrouter.ai/api/v1/chat/completions',
model: 'google/gemini-3.5-flash',
apiKey: 'OPENROUTER_API_KEY'
},
ark: {
apiUrl: 'https://ark.cn-beijing.volces.com/api/v3/chat/completions',
model: 'doubao-seed-1-6-flash-250615',
apiKey: 'ARK_API_KEY'
}
},
// Audio configuration
AUDIO_SAMPLE_RATE: 16000,
VISION_MAX_TOKENS: 4096
};
/**
* Create a sample audio buffer for testing
* This creates a simple sine wave audio buffer
*/
function createTestAudioBuffer(durationSeconds = 2, sampleRate = 16000) {
const numSamples = durationSeconds * sampleRate;
const buffer = Buffer.alloc(numSamples * 2); // 16-bit = 2 bytes per sample
// Generate a simple sine wave at 440Hz
const frequency = 440;
for (let i = 0; i < numSamples; i++) {
const sample = Math.sin(2 * Math.PI * frequency * i / sampleRate) * 0.5;
const intSample = Math.round(sample * 32767);
buffer.writeInt16LE(intSample, i * 2);
}
return buffer;
}
/**
* Test suite for ASR providers
*/
describe('ASR Provider Tests', function() {
this.timeout(60000); // 60 second timeout for API calls
const asrProviders = ['openai', 'siliconflow'];
asrProviders.forEach(providerName => {
describe(`${providerName} ASR Provider`, function() {
let provider;
before(function() {
// Skip test if API key is not available
const apiKeyName = testConfig.ASR_PROVIDERS[providerName].apiKey;
if (!testConfig[apiKeyName]) {
this.skip();
}
try {
provider = ASRProviderFactory.createProvider(providerName, testConfig, testConfig);
} catch (error) {
console.error(`Failed to create ${providerName} provider:`, error);
this.skip();
}
});
it('should be created successfully', function() {
assert(provider, 'Provider should be created');
assert(provider.config, 'Provider should have config');
assert(provider.apiKey, 'Provider should have API key');
});
it('should transcribe test audio', async function() {
const audioBuffer = createTestAudioBuffer(2); // 2 seconds of audio
const tempDir = path.join(__dirname, '../temp');
// Ensure temp directory exists
if (!fs.existsSync(tempDir)) {
fs.mkdirSync(tempDir, { recursive: true });
}
const result = await provider.transcribe(audioBuffer, tempDir);
assert(result, 'Should return a result');
assert(typeof result.success === 'boolean', 'Should have success field');
assert(typeof result.text === 'string', 'Should have text field');
assert(result.provider === providerName, 'Should have correct provider name');
console.log(`${providerName} ASR Result:`, {
success: result.success,
text: result.text.substring(0, 100) + (result.text.length > 100 ? '...' : ''),
language: result.language,
provider: result.provider
});
});
});
});
describe('SpeechToTextService Integration', function() {
asrProviders.forEach(providerName => {
it(`should work with ${providerName} provider`, async function() {
// Skip test if API key is not available
const apiKeyName = testConfig.ASR_PROVIDERS[providerName].apiKey;
if (!testConfig[apiKeyName]) {
this.skip();
}
// Override config for this test
const originalConfig = require('../config');
Object.assign(originalConfig, testConfig);
originalConfig.ASR_PROVIDER = providerName;
const sttService = new SpeechToTextService();
const audioBuffer = createTestAudioBuffer(2);
const result = await sttService.transcribeAudio(audioBuffer);
assert(result, 'Should return a result');
assert(typeof result.success === 'boolean', 'Should have success field');
assert(typeof result.text === 'string', 'Should have text field');
console.log(`STT Service with ${providerName}:`, {
success: result.success,
text: result.text.substring(0, 100) + (result.text.length > 100 ? '...' : ''),
provider: result.provider
});
});
});
});
});
/**
* Test suite for LLM providers
*/
describe('LLM Provider Tests', function() {
this.timeout(60000); // 60 second timeout for API calls
const llmProviders = ['openai', 'openrouter-gpt', 'openrouter-gemini', 'ark'];
const testMessages = [
{ role: 'system', content: 'You are a helpful AI assistant.' },
{ role: 'user', content: 'Hello! Please respond with a short greeting.' }
];
llmProviders.forEach(providerName => {
describe(`${providerName} LLM Provider`, function() {
let provider;
before(function() {
// Skip test if API key is not available
const apiKeyName = testConfig.LLM_PROVIDERS[providerName].apiKey;
if (!testConfig[apiKeyName]) {
this.skip();
}
try {
provider = LLMProviderFactory.createProvider(providerName, testConfig, testConfig);
} catch (error) {
console.error(`Failed to create ${providerName} provider:`, error);
this.skip();
}
});
it('should be created successfully', function() {
assert(provider, 'Provider should be created');
assert(provider.config, 'Provider should have config');
assert(provider.apiKey, 'Provider should have API key');
});
it('should generate chat completion', async function() {
const result = await provider.createChatCompletion(testMessages, {
max_tokens: 100
});
assert(result, 'Should return a result');
assert(typeof result.success === 'boolean', 'Should have success field');
assert(result.provider === providerName.split('-')[0], 'Should have correct provider name');
if (result.success) {
assert(result.response, 'Should have response object');
console.log(`${providerName} LLM Result: Success`);
} else {
console.log(`${providerName} LLM Result:`, result.error);
}
});
it('should stream chat completion', async function() {
const result = await provider.createChatCompletion(testMessages, {
max_tokens: 50,
stream: true
});
assert(result, 'Should return a result');
if (result.success) {
assert(result.response, 'Should have response object');
assert(result.response.data, 'Should have data stream');
// Test streaming by collecting some data
let receivedData = false;
const timeout = setTimeout(() => {
if (!receivedData) {
console.log(`${providerName}: No data received within timeout`);
}
}, 10000);
result.response.data.on('data', (chunk) => {
receivedData = true;
clearTimeout(timeout);
console.log(`${providerName} LLM Streaming: Received data chunk`);
});
result.response.data.on('error', (error) => {
clearTimeout(timeout);
console.error(`${providerName} LLM Streaming Error:`, error.message);
});
} else {
console.log(`${providerName} LLM Streaming Result:`, result.error);
}
});
});
});
});
/**
* Integration tests for all provider combinations
*/
describe('Provider Integration Tests', function() {
this.timeout(120000); // 2 minute timeout for integration tests
const testCombinations = [
{ asr: 'openai', llm: 'openai', description: 'OpenAI ASR + OpenAI LLM' },
{ asr: 'openai', llm: 'openrouter-gpt', description: 'OpenAI ASR + OpenRouter GPT' },
{ asr: 'openai', llm: 'openrouter-gemini', description: 'OpenAI ASR + OpenRouter Gemini' },
{ asr: 'openai', llm: 'ark', description: 'OpenAI ASR + ARK Doubao' },
{ asr: 'siliconflow', llm: 'openai', description: 'SenseVoice ASR + OpenAI LLM' },
{ asr: 'siliconflow', llm: 'openrouter-gpt', description: 'SenseVoice ASR + OpenRouter GPT' },
{ asr: 'siliconflow', llm: 'openrouter-gemini', description: 'SenseVoice ASR + OpenRouter Gemini' },
{ asr: 'siliconflow', llm: 'ark', description: 'SenseVoice ASR + ARK Doubao' }
];
testCombinations.forEach(combination => {
it(`should work with ${combination.description}`, async function() {
// Check if API keys are available
const asrApiKey = testConfig.ASR_PROVIDERS[combination.asr].apiKey;
const llmApiKey = testConfig.LLM_PROVIDERS[combination.llm].apiKey;
if (!testConfig[asrApiKey] || !testConfig[llmApiKey]) {
this.skip();
}
try {
// Create providers
const asrProvider = ASRProviderFactory.createProvider(combination.asr, testConfig, testConfig);
const llmProvider = LLMProviderFactory.createProvider(combination.llm, testConfig, testConfig);
// Test ASR
const audioBuffer = createTestAudioBuffer(2);
const tempDir = path.join(__dirname, '../temp');
if (!fs.existsSync(tempDir)) {
fs.mkdirSync(tempDir, { recursive: true });
}
const asrResult = await asrProvider.transcribe(audioBuffer, tempDir);
assert(asrResult.success !== undefined, 'ASR should return result');
// Test LLM with a simple message
const messages = [
{ role: 'system', content: 'You are a helpful assistant.' },
{ role: 'user', content: 'Say hello in one word.' }
];
const llmResult = await llmProvider.createChatCompletion(messages, {
max_tokens: 10
});
assert(llmResult.success !== undefined, 'LLM should return result');
console.log(`Integration Test - ${combination.description}:`, {
asr: { success: asrResult.success, provider: asrResult.provider },
llm: { success: llmResult.success, provider: llmResult.provider }
});
} catch (error) {
console.error(`Integration test failed for ${combination.description}:`, error.message);
throw error;
}
});
});
});
/**
* Provider switching tests
*/
describe('Provider Switching Tests', function() {
it('should switch ASR providers dynamically', function() {
const originalConfig = require('../config');
Object.assign(originalConfig, testConfig);
const sttService = new SpeechToTextService();
const originalProvider = sttService.getProviderInfo();
// Try switching to different provider
const availableProviders = ['openai', 'siliconflow'];
const newProvider = availableProviders.find(p => p !== originalProvider.provider);
if (newProvider && testConfig[testConfig.ASR_PROVIDERS[newProvider].apiKey]) {
sttService.switchProvider(newProvider);
const newProviderInfo = sttService.getProviderInfo();
assert(newProviderInfo.provider !== originalProvider.provider, 'Provider should change');
console.log('ASR Provider switched from', originalProvider.provider, 'to', newProviderInfo.provider);
} else {
console.log('Skipping ASR provider switching test - insufficient API keys');
}
});
});
// Export test configuration for use in other test files
module.exports = {
testConfig,
createTestAudioBuffer
};
backend/tests/test-asr-providers.js¶
const assert = require('assert');
const fs = require('fs');
const path = require('path');
// Import ASR provider factory
const { ASRProviderFactory } = require('../utils/providers/asrProviders');
// Test configuration
const testConfig = {
OPENAI_API_KEY: process.env.OPENAI_API_KEY,
SILICONFLOW_API_KEY: process.env.SILICONFLOW_API_KEY,
ASR_PROVIDERS: {
openai: {
apiUrl: 'https://api.openai.com/v1/audio/transcriptions',
model: 'whisper-1',
apiKey: 'OPENAI_API_KEY'
},
siliconflow: {
apiUrl: 'https://api.siliconflow.cn/v1/audio/transcriptions',
model: 'FunAudioLLM/SenseVoiceSmall',
apiKey: 'SILICONFLOW_API_KEY'
}
},
AUDIO_SAMPLE_RATE: 16000
};
/**
* Create a test audio buffer with spoken content
* This creates a simple sine wave that simulates audio content
*/
function createTestAudioBuffer(durationSeconds = 3, sampleRate = 16000) {
const numSamples = durationSeconds * sampleRate;
const buffer = Buffer.alloc(numSamples * 2); // 16-bit = 2 bytes per sample
// Generate a more complex wave pattern to simulate speech
for (let i = 0; i < numSamples; i++) {
// Mix multiple frequencies to simulate speech-like content
const t = i / sampleRate;
const sample =
0.3 * Math.sin(2 * Math.PI * 300 * t) + // Base frequency
0.2 * Math.sin(2 * Math.PI * 600 * t) + // Harmonic
0.1 * Math.sin(2 * Math.PI * 1200 * t) + // Higher harmonic
0.05 * (Math.random() - 0.5); // Noise for realism
const intSample = Math.round(sample * 16000); // Scale to 16-bit range
buffer.writeInt16LE(intSample, i * 2);
}
return buffer;
}
/**
* ASR Provider Individual Tests
*/
describe('ASR Providers - Individual Testing', function() {
this.timeout(120000); // 2 minute timeout for API calls
const tempDir = path.join(__dirname, '../temp');
before(function() {
// Ensure temp directory exists
if (!fs.existsSync(tempDir)) {
fs.mkdirSync(tempDir, { recursive: true });
}
});
describe('OpenAI Whisper ASR Provider', function() {
let provider;
before(function() {
if (!testConfig.OPENAI_API_KEY) {
console.log('⚠️ Skipping OpenAI ASR tests - OPENAI_API_KEY not found');
this.skip();
}
try {
provider = ASRProviderFactory.createProvider('openai', testConfig, testConfig);
console.log('✅ OpenAI ASR Provider created successfully');
} catch (error) {
console.error('❌ Failed to create OpenAI ASR provider:', error);
this.skip();
}
});
it('should initialize with correct configuration', function() {
assert(provider, 'Provider should be created');
assert.strictEqual(provider.config.model, 'whisper-1', 'Should use whisper-1 model');
assert.strictEqual(provider.config.apiUrl, 'https://api.openai.com/v1/audio/transcriptions', 'Should use correct API URL');
assert(provider.apiKey, 'Should have API key');
console.log('📋 OpenAI ASR Config:', {
model: provider.config.model,
apiUrl: provider.config.apiUrl
});
});
it('should transcribe test audio successfully', async function() {
const audioBuffer = createTestAudioBuffer(3); // 3 seconds of test audio
console.log('🎵 Testing OpenAI ASR with', audioBuffer.length, 'bytes of audio data');
const startTime = Date.now();
const result = await provider.transcribe(audioBuffer, tempDir);
const duration = Date.now() - startTime;
console.log('📊 OpenAI ASR Results:', {
success: result.success,
transcriptionTime: `${duration}ms`,
textLength: result.text ? result.text.length : 0,
language: result.language,
confidence: result.confidence,
provider: result.provider
});
assert(result, 'Should return a result');
assert(typeof result.success === 'boolean', 'Should have success field');
assert(typeof result.text === 'string', 'Should have text field');
assert.strictEqual(result.provider, 'openai', 'Should identify as openai provider');
if (result.success) {
console.log('✅ OpenAI ASR transcription successful');
if (result.text.length > 0) {
console.log('📝 Transcribed text preview:', result.text.substring(0, 100) + (result.text.length > 100 ? '...' : ''));
}
} else {
console.log('❌ OpenAI ASR transcription failed:', result.error);
}
});
it('should handle empty audio gracefully', async function() {
const emptyBuffer = Buffer.alloc(0);
const result = await provider.transcribe(emptyBuffer, tempDir);
console.log('🔍 OpenAI ASR Empty Audio Test:', {
success: result.success,
error: result.error,
provider: result.provider
});
assert(result, 'Should return a result');
assert(typeof result.success === 'boolean', 'Should have success field');
// Empty audio should typically fail or return empty text
if (!result.success) {
assert(result.error, 'Should have error message for empty audio');
}
});
});
describe('SenseVoice (Siliconflow) ASR Provider', function() {
let provider;
before(function() {
if (!testConfig.SILICONFLOW_API_KEY) {
console.log('⚠️ Skipping Siliconflow ASR tests - SILICONFLOW_API_KEY not found');
this.skip();
}
try {
provider = ASRProviderFactory.createProvider('siliconflow', testConfig, testConfig);
console.log('✅ Siliconflow ASR Provider created successfully');
} catch (error) {
console.error('❌ Failed to create Siliconflow ASR provider:', error);
this.skip();
}
});
it('should initialize with correct configuration', function() {
assert(provider, 'Provider should be created');
assert.strictEqual(provider.config.model, 'FunAudioLLM/SenseVoiceSmall', 'Should use SenseVoiceSmall model');
assert.strictEqual(provider.config.apiUrl, 'https://api.siliconflow.cn/v1/audio/transcriptions', 'Should use correct API URL');
assert(provider.apiKey, 'Should have API key');
console.log('📋 Siliconflow ASR Config:', {
model: provider.config.model,
apiUrl: provider.config.apiUrl
});
});
it('should transcribe test audio successfully', async function() {
const audioBuffer = createTestAudioBuffer(3); // 3 seconds of test audio
console.log('🎵 Testing Siliconflow ASR with', audioBuffer.length, 'bytes of audio data');
const startTime = Date.now();
const result = await provider.transcribe(audioBuffer, tempDir);
const duration = Date.now() - startTime;
console.log('📊 Siliconflow ASR Results:', {
success: result.success,
transcriptionTime: `${duration}ms`,
textLength: result.text ? result.text.length : 0,
language: result.language,
confidence: result.confidence,
provider: result.provider
});
assert(result, 'Should return a result');
assert(typeof result.success === 'boolean', 'Should have success field');
assert(typeof result.text === 'string', 'Should have text field');
assert.strictEqual(result.provider, 'siliconflow', 'Should identify as siliconflow provider');
if (result.success) {
console.log('✅ Siliconflow ASR transcription successful');
if (result.text.length > 0) {
console.log('📝 Transcribed text preview:', result.text.substring(0, 100) + (result.text.length > 100 ? '...' : ''));
}
} else {
console.log('❌ Siliconflow ASR transcription failed:', result.error);
}
});
it('should handle different audio lengths', async function() {
const shortAudio = createTestAudioBuffer(1); // 1 second
const longAudio = createTestAudioBuffer(5); // 5 seconds
console.log('🎵 Testing Siliconflow ASR with different audio lengths');
const shortResult = await provider.transcribe(shortAudio, tempDir);
const longResult = await provider.transcribe(longAudio, tempDir);
console.log('📊 Audio Length Tests:', {
short: { success: shortResult.success, textLength: shortResult.text.length },
long: { success: longResult.success, textLength: longResult.text.length }
});
assert(shortResult, 'Should handle short audio');
assert(longResult, 'Should handle long audio');
assert(typeof shortResult.success === 'boolean', 'Short audio should have success field');
assert(typeof longResult.success === 'boolean', 'Long audio should have success field');
});
});
describe('ASR Provider Comparison', function() {
it('should compare provider performance', async function() {
const availableProviders = [];
if (testConfig.OPENAI_API_KEY) {
availableProviders.push('openai');
}
if (testConfig.SILICONFLOW_API_KEY) {
availableProviders.push('siliconflow');
}
if (availableProviders.length < 2) {
console.log('⚠️ Skipping provider comparison - need at least 2 providers');
this.skip();
}
const audioBuffer = createTestAudioBuffer(3);
const results = {};
console.log('🏁 Comparing ASR provider performance...');
for (const providerName of availableProviders) {
const provider = ASRProviderFactory.createProvider(providerName, testConfig, testConfig);
const startTime = Date.now();
const result = await provider.transcribe(audioBuffer, tempDir);
const duration = Date.now() - startTime;
results[providerName] = {
success: result.success,
duration: duration,
textLength: result.text ? result.text.length : 0,
error: result.error
};
}
console.log('📊 ASR Provider Performance Comparison:');
Object.entries(results).forEach(([provider, result]) => {
console.log(` ${provider}:`, {
success: result.success ? '✅' : '❌',
time: `${result.duration}ms`,
textLength: result.textLength,
error: result.error || 'none'
});
});
// Find fastest successful provider
const successfulProviders = Object.entries(results).filter(([_, result]) => result.success);
if (successfulProviders.length > 0) {
const fastest = successfulProviders.reduce((prev, curr) =>
prev[1].duration < curr[1].duration ? prev : curr
);
console.log(`🏆 Fastest provider: ${fastest[0]} (${fastest[1].duration}ms)`);
}
});
});
after(function() {
// Cleanup temp files
try {
const files = fs.readdirSync(tempDir);
files.forEach(file => {
if (file.startsWith('audio_')) {
fs.unlinkSync(path.join(tempDir, file));
}
});
console.log('🧹 Cleaned up temporary audio files');
} catch (error) {
console.warn('⚠️ Failed to cleanup temp files:', error.message);
}
});
});
module.exports = {
testConfig,
createTestAudioBuffer
};
backend/tests/test-llm-providers.js¶
const assert = require('assert');
// Import LLM provider factory
const { LLMProviderFactory } = require('../utils/providers/llmProviders');
// Test configuration
const testConfig = {
OPENAI_API_KEY: process.env.OPENAI_API_KEY,
OPENROUTER_API_KEY: process.env.OPENROUTER_API_KEY,
ARK_API_KEY: process.env.ARK_API_KEY,
SILICONFLOW_API_KEY: process.env.SILICONFLOW_API_KEY,
LLM_PROVIDERS: {
openai: {
apiUrl: 'https://api.openai.com/v1/chat/completions',
model: 'gpt-5.6-luna',
apiKey: 'OPENAI_API_KEY'
},
'openrouter-gpt': {
apiUrl: 'https://openrouter.ai/api/v1/chat/completions',
model: 'openai/gpt-5.6-luna',
apiKey: 'OPENROUTER_API_KEY'
},
'openrouter-gemini': {
apiUrl: 'https://openrouter.ai/api/v1/chat/completions',
model: 'google/gemini-3.5-flash',
apiKey: 'OPENROUTER_API_KEY'
},
ark: {
apiUrl: 'https://ark.cn-beijing.volces.com/api/v3/chat/completions',
model: 'doubao-seed-1-6-flash-250615',
apiKey: 'ARK_API_KEY'
}
},
VISION_MAX_TOKENS: 4096
};
// Test messages for different scenarios
const testMessages = {
simple: [
{ role: 'system', content: 'You are a helpful AI assistant.' },
{ role: 'user', content: 'Hello! Please respond with exactly the word "SUCCESS" to confirm you are working.' }
],
conversation: [
{ role: 'system', content: 'You are a conversational AI assistant.' },
{ role: 'user', content: 'What is the capital of France?' },
{ role: 'assistant', content: 'The capital of France is Paris.' },
{ role: 'user', content: 'What is its population?' }
],
creative: [
{ role: 'system', content: 'You are a creative writing assistant.' },
{ role: 'user', content: 'Write a very short story about a robot learning to paint. Keep it under 50 words.' }
]
};
/**
* Collect streaming response data
*/
async function collectStreamingResponse(response, timeout = 30000) {
return new Promise((resolve, reject) => {
let buffer = '';
let accumulatedContent = '';
let firstTokenTime = null;
let tokenCount = 0;
const timeoutHandle = setTimeout(() => {
reject(new Error('Streaming response timeout'));
}, timeout);
response.data.on('data', (chunk) => {
try {
if (!firstTokenTime) {
firstTokenTime = Date.now();
}
buffer += chunk.toString();
const lines = buffer.split('\n');
// Keep the last line if it's incomplete
buffer = lines.pop() || '';
for (const line of lines) {
const trimmedLine = line.trim();
if (!trimmedLine || trimmedLine === '[DONE]') continue;
if (!trimmedLine.startsWith('data: ')) continue;
try {
const jsonData = JSON.parse(trimmedLine.replace('data: ', ''));
const content = jsonData.choices[0]?.delta?.content || '';
if (content) {
accumulatedContent += content;
tokenCount++;
}
} catch (parseError) {
// Skip malformed JSON
continue;
}
}
} catch (error) {
reject(error);
}
});
response.data.on('end', () => {
clearTimeout(timeoutHandle);
resolve({
content: accumulatedContent,
tokenCount: tokenCount,
firstTokenTime: firstTokenTime,
success: true
});
});
response.data.on('error', (error) => {
clearTimeout(timeoutHandle);
reject(error);
});
});
}
/**
* LLM Provider Individual Tests
*/
describe('LLM Providers - Individual Testing', function() {
this.timeout(120000); // 2 minute timeout for API calls
describe('OpenAI GPT-4o LLM Provider', function() {
let provider;
before(function() {
if (!testConfig.OPENAI_API_KEY) {
console.log('⚠️ Skipping OpenAI LLM tests - OPENAI_API_KEY not found');
this.skip();
}
try {
provider = LLMProviderFactory.createProvider('openai', testConfig, testConfig);
console.log('✅ OpenAI LLM Provider created successfully');
} catch (error) {
console.error('❌ Failed to create OpenAI LLM provider:', error);
this.skip();
}
});
it('should initialize with correct configuration', function() {
assert(provider, 'Provider should be created');
assert.strictEqual(provider.config.model, 'gpt-5.6-luna', 'Should use gpt-5.6-luna model');
assert.strictEqual(provider.config.apiUrl, 'https://api.openai.com/v1/chat/completions', 'Should use correct API URL');
assert(provider.apiKey, 'Should have API key');
console.log('📋 OpenAI LLM Config:', {
model: provider.config.model,
apiUrl: provider.config.apiUrl
});
});
it('should generate simple chat completion', async function() {
console.log('🤖 Testing OpenAI LLM with simple message');
const startTime = Date.now();
const result = await provider.createChatCompletion(testMessages.simple, {
max_tokens: 100
});
const responseTime = Date.now() - startTime;
console.log('📊 OpenAI LLM Simple Test:', {
success: result.success,
responseTime: `${responseTime}ms`,
provider: result.provider
});
assert(result, 'Should return a result');
assert(typeof result.success === 'boolean', 'Should have success field');
assert.strictEqual(result.provider, 'openai', 'Should identify as openai provider');
if (result.success) {
assert(result.response, 'Should have response object');
console.log('✅ OpenAI LLM simple completion successful');
} else {
console.log('❌ OpenAI LLM simple completion failed:', result.error);
}
});
it('should handle streaming chat completion', async function() {
console.log('🌊 Testing OpenAI LLM streaming');
const startTime = Date.now();
const result = await provider.createChatCompletion(testMessages.simple, {
max_tokens: 50,
stream: true
});
if (!result.success) {
console.log('❌ OpenAI LLM streaming setup failed:', result.error);
assert(false, 'Streaming setup should succeed');
return;
}
try {
const streamResult = await collectStreamingResponse(result.response);
const totalTime = Date.now() - startTime;
const timeToFirstToken = streamResult.firstTokenTime ? streamResult.firstTokenTime - startTime : 0;
console.log('📊 OpenAI LLM Streaming Results:', {
success: streamResult.success,
totalTime: `${totalTime}ms`,
timeToFirstToken: `${timeToFirstToken}ms`,
tokenCount: streamResult.tokenCount,
contentLength: streamResult.content.length
});
assert(streamResult.success, 'Streaming should be successful');
assert(streamResult.content.length > 0, 'Should receive content');
assert(streamResult.tokenCount > 0, 'Should receive tokens');
if (streamResult.content.length > 0) {
console.log('📝 Generated content preview:', streamResult.content.substring(0, 100) + (streamResult.content.length > 100 ? '...' : ''));
}
console.log('✅ OpenAI LLM streaming successful');
} catch (streamError) {
console.error('❌ OpenAI LLM streaming failed:', streamError.message);
assert(false, 'Streaming should not fail: ' + streamError.message);
}
});
it('should handle conversation context', async function() {
console.log('💬 Testing OpenAI LLM with conversation context');
const result = await provider.createChatCompletion(testMessages.conversation, {
max_tokens: 100
});
console.log('📊 OpenAI LLM Conversation Test:', {
success: result.success,
provider: result.provider
});
assert(result, 'Should return a result');
assert(typeof result.success === 'boolean', 'Should have success field');
if (result.success) {
console.log('✅ OpenAI LLM conversation handling successful');
} else {
console.log('❌ OpenAI LLM conversation handling failed:', result.error);
}
});
});
describe('OpenRouter GPT LLM Provider', function() {
let provider;
before(function() {
if (!testConfig.OPENROUTER_API_KEY) {
console.log('⚠️ Skipping OpenRouter GPT tests - OPENROUTER_API_KEY not found');
this.skip();
}
try {
provider = LLMProviderFactory.createProvider('openrouter-gpt', testConfig, testConfig);
console.log('✅ OpenRouter GPT Provider created successfully');
} catch (error) {
console.error('❌ Failed to create OpenRouter GPT provider:', error);
this.skip();
}
});
it('should initialize with correct configuration', function() {
assert(provider, 'Provider should be created');
assert.strictEqual(provider.config.model, 'openai/gpt-5.6-luna', 'Should use openai/gpt-5.6-luna model');
assert.strictEqual(provider.config.apiUrl, 'https://openrouter.ai/api/v1/chat/completions', 'Should use OpenRouter API URL');
assert(provider.apiKey, 'Should have API key');
console.log('📋 OpenRouter GPT Config:', {
model: provider.config.model,
apiUrl: provider.config.apiUrl
});
});
it('should generate chat completion via OpenRouter', async function() {
console.log('🤖 Testing OpenRouter GPT');
const startTime = Date.now();
const result = await provider.createChatCompletion(testMessages.simple, {
max_tokens: 100
});
const responseTime = Date.now() - startTime;
console.log('📊 OpenRouter GPT Test:', {
success: result.success,
responseTime: `${responseTime}ms`,
provider: result.provider
});
assert(result, 'Should return a result');
assert(typeof result.success === 'boolean', 'Should have success field');
assert.strictEqual(result.provider, 'openrouter', 'Should identify as openrouter provider');
if (result.success) {
console.log('✅ OpenRouter GPT completion successful');
} else {
console.log('❌ OpenRouter GPT completion failed:', result.error);
}
});
it('should handle streaming via OpenRouter', async function() {
console.log('🌊 Testing OpenRouter GPT streaming');
const result = await provider.createChatCompletion(testMessages.simple, {
max_tokens: 50,
stream: true
});
if (!result.success) {
console.log('❌ OpenRouter GPT streaming setup failed:', result.error);
return; // Don't fail the test, just log
}
try {
const streamResult = await collectStreamingResponse(result.response);
console.log('📊 OpenRouter GPT Streaming:', {
success: streamResult.success,
tokenCount: streamResult.tokenCount,
contentLength: streamResult.content.length
});
if (streamResult.success) {
console.log('✅ OpenRouter GPT streaming successful');
}
} catch (streamError) {
console.log('❌ OpenRouter GPT streaming error:', streamError.message);
}
});
});
describe('OpenRouter Gemini LLM Provider', function() {
let provider;
before(function() {
if (!testConfig.OPENROUTER_API_KEY) {
console.log('⚠️ Skipping OpenRouter Gemini tests - OPENROUTER_API_KEY not found');
this.skip();
}
try {
provider = LLMProviderFactory.createProvider('openrouter-gemini', testConfig, testConfig);
console.log('✅ OpenRouter Gemini Provider created successfully');
} catch (error) {
console.error('❌ Failed to create OpenRouter Gemini provider:', error);
this.skip();
}
});
it('should initialize with correct configuration', function() {
assert(provider, 'Provider should be created');
assert.strictEqual(provider.config.model, 'google/gemini-3.5-flash', 'Should use gemini-3.5-flash model');
assert.strictEqual(provider.config.apiUrl, 'https://openrouter.ai/api/v1/chat/completions', 'Should use OpenRouter API URL');
assert(provider.apiKey, 'Should have API key');
console.log('📋 OpenRouter Gemini Config:', {
model: provider.config.model,
apiUrl: provider.config.apiUrl
});
});
it('should generate chat completion with Gemini', async function() {
console.log('🤖 Testing OpenRouter Gemini');
const startTime = Date.now();
const result = await provider.createChatCompletion(testMessages.simple, {
max_tokens: 100
});
const responseTime = Date.now() - startTime;
console.log('📊 OpenRouter Gemini Test:', {
success: result.success,
responseTime: `${responseTime}ms`,
provider: result.provider
});
assert(result, 'Should return a result');
assert(typeof result.success === 'boolean', 'Should have success field');
assert.strictEqual(result.provider, 'openrouter', 'Should identify as openrouter provider');
if (result.success) {
console.log('✅ OpenRouter Gemini completion successful');
} else {
console.log('❌ OpenRouter Gemini completion failed:', result.error);
}
});
it('should handle creative tasks with Gemini', async function() {
console.log('🎨 Testing OpenRouter Gemini creative task');
const result = await provider.createChatCompletion(testMessages.creative, {
max_tokens: 150
});
console.log('📊 OpenRouter Gemini Creative Test:', {
success: result.success,
provider: result.provider
});
if (result.success) {
console.log('✅ OpenRouter Gemini creative task successful');
} else {
console.log('❌ OpenRouter Gemini creative task failed:', result.error);
}
});
});
describe('ARK Doubao LLM Provider', function() {
let provider;
before(function() {
if (!testConfig.ARK_API_KEY) {
console.log('⚠️ Skipping ARK Doubao tests - ARK_API_KEY not found');
this.skip();
}
try {
provider = LLMProviderFactory.createProvider('ark', testConfig, testConfig);
console.log('✅ ARK Doubao Provider created successfully');
} catch (error) {
console.error('❌ Failed to create ARK Doubao provider:', error);
this.skip();
}
});
it('should initialize with correct configuration', function() {
assert(provider, 'Provider should be created');
assert.strictEqual(provider.config.model, 'doubao-seed-1-6-flash-250615', 'Should use doubao model');
assert.strictEqual(provider.config.apiUrl, 'https://ark.cn-beijing.volces.com/api/v3/chat/completions', 'Should use ARK API URL');
assert(provider.apiKey, 'Should have API key');
console.log('📋 ARK Doubao Config:', {
model: provider.config.model,
apiUrl: provider.config.apiUrl
});
});
it('should generate chat completion with Doubao', async function() {
console.log('🤖 Testing ARK Doubao');
const startTime = Date.now();
const result = await provider.createChatCompletion(testMessages.simple, {
max_tokens: 100
});
const responseTime = Date.now() - startTime;
console.log('📊 ARK Doubao Test:', {
success: result.success,
responseTime: `${responseTime}ms`,
provider: result.provider
});
assert(result, 'Should return a result');
assert(typeof result.success === 'boolean', 'Should have success field');
assert.strictEqual(result.provider, 'ark', 'Should identify as ark provider');
if (result.success) {
console.log('✅ ARK Doubao completion successful');
} else {
console.log('❌ ARK Doubao completion failed:', result.error);
}
});
it('should handle Chinese language tasks', async function() {
console.log('🇨🇳 Testing ARK Doubao with Chinese');
const chineseMessages = [
{ role: 'system', content: '你是一个有用的AI助手。' },
{ role: 'user', content: '请用一句话介绍北京。' }
];
const result = await provider.createChatCompletion(chineseMessages, {
max_tokens: 100
});
console.log('📊 ARK Doubao Chinese Test:', {
success: result.success,
provider: result.provider
});
if (result.success) {
console.log('✅ ARK Doubao Chinese handling successful');
} else {
console.log('❌ ARK Doubao Chinese handling failed:', result.error);
}
});
});
describe('LLM Provider Performance Comparison', function() {
it('should compare provider response times', async function() {
const availableProviders = [];
if (testConfig.OPENAI_API_KEY) availableProviders.push('openai');
if (testConfig.OPENROUTER_API_KEY) {
availableProviders.push('openrouter-gpt', 'openrouter-gemini');
}
if (testConfig.ARK_API_KEY) availableProviders.push('ark');
if (availableProviders.length < 2) {
console.log('⚠️ Skipping provider comparison - need at least 2 providers');
this.skip();
}
const results = {};
console.log('🏁 Comparing LLM provider performance...');
for (const providerName of availableProviders) {
try {
const provider = LLMProviderFactory.createProvider(providerName, testConfig, testConfig);
const startTime = Date.now();
const result = await provider.createChatCompletion(testMessages.simple, {
max_tokens: 50
});
const duration = Date.now() - startTime;
results[providerName] = {
success: result.success,
duration: duration,
error: result.error
};
} catch (error) {
results[providerName] = {
success: false,
duration: 0,
error: error.message
};
}
}
console.log('📊 LLM Provider Performance Comparison:');
Object.entries(results).forEach(([provider, result]) => {
console.log(` ${provider}:`, {
success: result.success ? '✅' : '❌',
time: `${result.duration}ms`,
error: result.error || 'none'
});
});
// Find fastest successful provider
const successfulProviders = Object.entries(results).filter(([_, result]) => result.success);
if (successfulProviders.length > 0) {
const fastest = successfulProviders.reduce((prev, curr) =>
prev[1].duration < curr[1].duration ? prev : curr
);
console.log(`🏆 Fastest LLM provider: ${fastest[0]} (${fastest[1].duration}ms)`);
}
// At least one provider should work
assert(successfulProviders.length > 0, 'At least one LLM provider should be successful');
});
});
});
module.exports = {
testConfig,
testMessages,
collectStreamingResponse
};
backend/tests/test-tts-providers.js¶
const assert = require('assert');
const axios = require('axios');
const fs = require('fs');
const path = require('path');
// Test configuration
const testConfig = {
SILICONFLOW_API_KEY: process.env.SILICONFLOW_API_KEY,
TTS_PROVIDERS: {
siliconflow: {
apiUrl: 'https://api.siliconflow.cn/v1/audio/speech',
model: 'FunAudioLLM/CosyVoice2-0.5B',
voice: 'FunAudioLLM/CosyVoice2-0.5B:diana',
apiKey: 'SILICONFLOW_API_KEY'
}
}
};
// Test texts for different scenarios
const testTexts = {
simple: 'Hello, this is a simple text-to-speech test.',
multilingual: 'Hello world. 你好世界. こんにちは世界.',
punctuation: 'Testing punctuation: question? exclamation! comma, period.',
numbers: 'The year is 2025, and the time is 12:34 PM.',
long: 'This is a longer text to test the text-to-speech synthesis capability. It contains multiple sentences and should demonstrate the natural flow of speech generation. The quality and naturalness of the audio output will be evaluated.'
};
/**
* TTS Provider for CosyVoice2 via Siliconflow
*/
class SiliconflowTTSProvider {
constructor(config, apiKey) {
this.config = config;
this.apiKey = apiKey;
}
async synthesize(text, options = {}) {
try {
const response = await axios({
method: 'post',
url: this.config.apiUrl,
data: {
model: this.config.model,
input: text,
voice: options.voice || this.config.voice,
response_format: options.format || 'mp3',
sample_rate: options.sampleRate || 32000,
stream: options.stream || false,
speed: options.speed || 1.0,
gain: options.gain || 0
},
headers: {
'Authorization': `Bearer ${this.apiKey}`,
'Content-Type': 'application/json'
},
responseType: 'arraybuffer',
timeout: 60000 // 60 second timeout
});
const result = {
success: true,
audioData: Buffer.from(response.data),
format: options.format || 'mp3',
sampleRate: options.sampleRate || 32000,
provider: 'siliconflow',
timestamp: Date.now()
};
return result;
} catch (error) {
console.error('Siliconflow TTS error:', error.response?.data || error.message);
return {
success: false,
audioData: null,
error: error.response?.data?.error?.message || error.message,
provider: 'siliconflow',
timestamp: Date.now()
};
}
}
}
/**
* TTS Provider Individual Tests
*/
describe('TTS Providers - Individual Testing', function() {
this.timeout(120000); // 2 minute timeout for API calls
const tempDir = path.join(__dirname, '../temp');
before(function() {
// Ensure temp directory exists
if (!fs.existsSync(tempDir)) {
fs.mkdirSync(tempDir, { recursive: true });
}
});
describe('CosyVoice2 (Siliconflow) TTS Provider', function() {
let provider;
before(function() {
if (!testConfig.SILICONFLOW_API_KEY) {
console.log('⚠️ Skipping Siliconflow TTS tests - SILICONFLOW_API_KEY not found');
this.skip();
}
try {
const providerConfig = testConfig.TTS_PROVIDERS.siliconflow;
provider = new SiliconflowTTSProvider(providerConfig, testConfig.SILICONFLOW_API_KEY);
console.log('✅ Siliconflow TTS Provider created successfully');
} catch (error) {
console.error('❌ Failed to create Siliconflow TTS provider:', error);
this.skip();
}
});
it('should initialize with correct configuration', function() {
assert(provider, 'Provider should be created');
assert.strictEqual(provider.config.model, 'FunAudioLLM/CosyVoice2-0.5B', 'Should use CosyVoice2 model');
assert.strictEqual(provider.config.voice, 'FunAudioLLM/CosyVoice2-0.5B:diana', 'Should use diana voice');
assert.strictEqual(provider.config.apiUrl, 'https://api.siliconflow.cn/v1/audio/speech', 'Should use correct API URL');
assert(provider.apiKey, 'Should have API key');
console.log('📋 Siliconflow TTS Config:', {
model: provider.config.model,
voice: provider.config.voice,
apiUrl: provider.config.apiUrl
});
});
it('should synthesize simple text to speech', async function() {
console.log('🎵 Testing Siliconflow TTS with simple text');
const startTime = Date.now();
const result = await provider.synthesize(testTexts.simple);
const synthesisTime = Date.now() - startTime;
console.log('📊 Siliconflow TTS Simple Test:', {
success: result.success,
synthesisTime: `${synthesisTime}ms`,
audioSize: result.audioData ? `${result.audioData.length} bytes` : 'none',
format: result.format,
sampleRate: result.sampleRate,
provider: result.provider
});
assert(result, 'Should return a result');
assert(typeof result.success === 'boolean', 'Should have success field');
assert.strictEqual(result.provider, 'siliconflow', 'Should identify as siliconflow provider');
if (result.success) {
assert(result.audioData, 'Should have audio data');
assert(result.audioData.length > 0, 'Audio data should not be empty');
assert.strictEqual(result.format, 'mp3', 'Should return MP3 format');
// Save test audio file
const testAudioPath = path.join(tempDir, `tts_simple_${Date.now()}.mp3`);
fs.writeFileSync(testAudioPath, result.audioData);
console.log('💾 Saved test audio to:', testAudioPath);
console.log('✅ Siliconflow TTS simple synthesis successful');
} else {
console.log('❌ Siliconflow TTS simple synthesis failed:', result.error);
}
});
it('should handle multilingual text', async function() {
console.log('🌍 Testing Siliconflow TTS with multilingual text');
const result = await provider.synthesize(testTexts.multilingual);
console.log('📊 Siliconflow TTS Multilingual Test:', {
success: result.success,
audioSize: result.audioData ? `${result.audioData.length} bytes` : 'none',
provider: result.provider
});
assert(result, 'Should return a result');
assert(typeof result.success === 'boolean', 'Should have success field');
if (result.success) {
assert(result.audioData, 'Should have audio data for multilingual text');
assert(result.audioData.length > 0, 'Multilingual audio data should not be empty');
// Save multilingual test audio
const testAudioPath = path.join(tempDir, `tts_multilingual_${Date.now()}.mp3`);
fs.writeFileSync(testAudioPath, result.audioData);
console.log('💾 Saved multilingual audio to:', testAudioPath);
console.log('✅ Siliconflow TTS multilingual synthesis successful');
} else {
console.log('❌ Siliconflow TTS multilingual synthesis failed:', result.error);
}
});
it('should handle different audio formats and settings', async function() {
console.log('⚙️ Testing Siliconflow TTS with different settings');
const settings = [
{ format: 'mp3', sampleRate: 24000, speed: 1.0 },
{ format: 'mp3', sampleRate: 32000, speed: 1.2 },
{ format: 'mp3', sampleRate: 16000, speed: 0.8 }
];
for (const setting of settings) {
const result = await provider.synthesize(testTexts.simple, setting);
console.log(`📊 TTS Settings Test (${setting.format}, ${setting.sampleRate}Hz, ${setting.speed}x):`, {
success: result.success,
audioSize: result.audioData ? `${result.audioData.length} bytes` : 'none',
error: result.error || 'none'
});
assert(result, 'Should return a result');
assert(typeof result.success === 'boolean', 'Should have success field');
if (result.success) {
assert(result.audioData, 'Should have audio data');
assert(result.audioData.length > 0, 'Audio data should not be empty');
// Save test audio with settings info
const filename = `tts_settings_${setting.sampleRate}hz_${setting.speed}x_${Date.now()}.mp3`;
const testAudioPath = path.join(tempDir, filename);
fs.writeFileSync(testAudioPath, result.audioData);
console.log(`💾 Saved settings test audio to: ${testAudioPath}`);
}
}
console.log('✅ Siliconflow TTS settings variation tests completed');
});
it('should handle punctuation and numbers correctly', async function() {
console.log('🔢 Testing Siliconflow TTS with punctuation and numbers');
const punctuationResult = await provider.synthesize(testTexts.punctuation);
const numbersResult = await provider.synthesize(testTexts.numbers);
console.log('📊 Siliconflow TTS Punctuation Test:', {
success: punctuationResult.success,
audioSize: punctuationResult.audioData ? `${punctuationResult.audioData.length} bytes` : 'none'
});
console.log('📊 Siliconflow TTS Numbers Test:', {
success: numbersResult.success,
audioSize: numbersResult.audioData ? `${numbersResult.audioData.length} bytes` : 'none'
});
assert(punctuationResult, 'Should handle punctuation');
assert(numbersResult, 'Should handle numbers');
if (punctuationResult.success && numbersResult.success) {
console.log('✅ Siliconflow TTS punctuation and numbers handling successful');
}
});
it('should handle longer text synthesis', async function() {
console.log('📝 Testing Siliconflow TTS with longer text');
const startTime = Date.now();
const result = await provider.synthesize(testTexts.long);
const synthesisTime = Date.now() - startTime;
console.log('📊 Siliconflow TTS Long Text Test:', {
success: result.success,
synthesisTime: `${synthesisTime}ms`,
audioSize: result.audioData ? `${result.audioData.length} bytes` : 'none',
textLength: testTexts.long.length
});
assert(result, 'Should return a result');
assert(typeof result.success === 'boolean', 'Should have success field');
if (result.success) {
assert(result.audioData, 'Should have audio data for long text');
assert(result.audioData.length > 0, 'Long text audio data should not be empty');
// Long text should produce more audio data
const expectedMinSize = 50000; // Roughly 50KB minimum for longer text
if (result.audioData.length > expectedMinSize) {
console.log('✅ Long text produced appropriate amount of audio data');
}
// Save long text audio
const testAudioPath = path.join(tempDir, `tts_long_${Date.now()}.mp3`);
fs.writeFileSync(testAudioPath, result.audioData);
console.log('💾 Saved long text audio to:', testAudioPath);
console.log('✅ Siliconflow TTS long text synthesis successful');
} else {
console.log('❌ Siliconflow TTS long text synthesis failed:', result.error);
}
});
it('should handle empty or invalid text gracefully', async function() {
console.log('🔍 Testing Siliconflow TTS with edge cases');
const emptyResult = await provider.synthesize('');
const spaceResult = await provider.synthesize(' ');
const specialResult = await provider.synthesize('!@#$%^&*()');
console.log('📊 Siliconflow TTS Edge Cases:', {
empty: { success: emptyResult.success, error: emptyResult.error || 'none' },
spaces: { success: spaceResult.success, error: spaceResult.error || 'none' },
special: { success: specialResult.success, error: specialResult.error || 'none' }
});
// These should either work or fail gracefully
assert(emptyResult, 'Should handle empty string');
assert(spaceResult, 'Should handle whitespace');
assert(specialResult, 'Should handle special characters');
console.log('✅ Siliconflow TTS edge cases handled');
});
it('should measure synthesis performance metrics', async function() {
console.log('🏁 Testing Siliconflow TTS performance metrics');
const testText = testTexts.simple;
const iterations = 3;
const results = [];
for (let i = 0; i < iterations; i++) {
const startTime = Date.now();
const result = await provider.synthesize(testText);
const duration = Date.now() - startTime;
if (result.success) {
results.push({
duration: duration,
audioSize: result.audioData.length,
success: true
});
} else {
results.push({
duration: duration,
audioSize: 0,
success: false,
error: result.error
});
}
}
const successfulResults = results.filter(r => r.success);
if (successfulResults.length > 0) {
const avgDuration = successfulResults.reduce((sum, r) => sum + r.duration, 0) / successfulResults.length;
const avgAudioSize = successfulResults.reduce((sum, r) => sum + r.audioSize, 0) / successfulResults.length;
const minDuration = Math.min(...successfulResults.map(r => r.duration));
const maxDuration = Math.max(...successfulResults.map(r => r.duration));
console.log('📊 Siliconflow TTS Performance Metrics:', {
successRate: `${successfulResults.length}/${iterations}`,
avgSynthesisTime: `${Math.round(avgDuration)}ms`,
minSynthesisTime: `${minDuration}ms`,
maxSynthesisTime: `${maxDuration}ms`,
avgAudioSize: `${Math.round(avgAudioSize)} bytes`,
textLength: testText.length
});
// Performance expectations
assert(avgDuration < 10000, 'Average synthesis time should be under 10 seconds');
assert(avgAudioSize > 1000, 'Should produce reasonable amount of audio data');
console.log('✅ Siliconflow TTS performance metrics acceptable');
} else {
console.log('❌ No successful TTS synthesis results for performance testing');
}
assert(successfulResults.length > 0, 'At least one synthesis attempt should succeed');
});
});
describe('TTS Provider Integration', function() {
it('should integrate with live audio system', function() {
if (!testConfig.SILICONFLOW_API_KEY) {
console.log('⚠️ Skipping TTS integration test - SILICONFLOW_API_KEY not found');
this.skip();
}
// Simulate the TTS configuration that would be used in the live system
const liveConfig = {
TTS_API_URL: 'https://api.siliconflow.cn/v1/audio/speech',
TTS_PROVIDERS: testConfig.TTS_PROVIDERS,
TTS_PROVIDER: 'siliconflow',
SILICONFLOW_API_KEY: testConfig.SILICONFLOW_API_KEY
};
assert(liveConfig.TTS_API_URL, 'Should have TTS API URL');
assert(liveConfig.TTS_PROVIDERS.siliconflow, 'Should have Siliconflow TTS provider config');
assert(liveConfig.SILICONFLOW_API_KEY, 'Should have Siliconflow API key');
console.log('📋 Live System TTS Integration Config:', {
provider: liveConfig.TTS_PROVIDER,
model: liveConfig.TTS_PROVIDERS.siliconflow.model,
voice: liveConfig.TTS_PROVIDERS.siliconflow.voice,
hasApiKey: !!liveConfig.SILICONFLOW_API_KEY
});
console.log('✅ TTS provider integration configuration valid');
});
});
after(function() {
// Cleanup temp audio files
try {
const files = fs.readdirSync(tempDir);
let cleanedCount = 0;
files.forEach(file => {
if (file.startsWith('tts_') && file.endsWith('.mp3')) {
fs.unlinkSync(path.join(tempDir, file));
cleanedCount++;
}
});
console.log(`🧹 Cleaned up ${cleanedCount} temporary TTS audio files`);
} catch (error) {
console.warn('⚠️ Failed to cleanup temp TTS files:', error.message);
}
});
});
module.exports = {
testConfig,
testTexts,
SiliconflowTTSProvider
};
backend/utils/providers/asrProviders.js¶
const axios = require('axios');
const FormData = require('form-data');
const fs = require('fs');
const path = require('path');
/**
* Base ASR Provider class
*/
class BaseASRProvider {
constructor(config, apiKey) {
this.config = config;
this.apiKey = apiKey;
}
/**
* Convert raw audio buffer to WAV format
* @param {Buffer} audioBuffer - Raw PCM audio data
* @param {Object} options - Audio format options
* @returns {Buffer} WAV formatted audio data
*/
createWavBuffer(audioBuffer, options = {}) {
const sampleRate = options.sampleRate || 16000;
const channels = options.channels || 1;
const bitsPerSample = options.bitsPerSample || 16;
const byteRate = sampleRate * channels * bitsPerSample / 8;
const blockAlign = channels * bitsPerSample / 8;
const dataSize = audioBuffer.length;
const fileSize = 36 + dataSize;
const header = Buffer.alloc(44);
// RIFF header
header.write('RIFF', 0);
header.writeUInt32LE(fileSize, 4);
header.write('WAVE', 8);
// fmt chunk
header.write('fmt ', 12);
header.writeUInt32LE(16, 16); // PCM format chunk size
header.writeUInt16LE(1, 20); // PCM format
header.writeUInt16LE(channels, 22);
header.writeUInt32LE(sampleRate, 24);
header.writeUInt32LE(byteRate, 28);
header.writeUInt16LE(blockAlign, 32);
header.writeUInt16LE(bitsPerSample, 34);
// data chunk
header.write('data', 36);
header.writeUInt32LE(dataSize, 40);
return Buffer.concat([header, audioBuffer]);
}
async transcribe(audioBuffer, options = {}) {
throw new Error('transcribe method must be implemented by subclass');
}
}
/**
* OpenAI Whisper ASR Provider
*/
class OpenAIASRProvider extends BaseASRProvider {
async transcribe(audioBuffer, tempDir, options = {}) {
try {
// Create WAV buffer
const wavBuffer = this.createWavBuffer(audioBuffer, {
sampleRate: 16000,
channels: 1,
bitsPerSample: 16
});
// Create temporary file
const tempFileName = `audio_${Date.now()}_${Math.random().toString(36).substring(2)}.wav`;
const tempFilePath = path.join(tempDir, tempFileName);
// Write audio to temporary file
fs.writeFileSync(tempFilePath, wavBuffer);
try {
// Create form data
const formData = new FormData();
formData.append('file', fs.createReadStream(tempFilePath));
formData.append('model', this.config.model);
formData.append('response_format', 'json');
if (options.language) {
formData.append('language', options.language);
}
if (options.prompt) {
formData.append('prompt', options.prompt);
}
// Make API request
const response = await axios({
method: 'post',
url: this.config.apiUrl,
data: formData,
headers: {
'Authorization': `Bearer ${this.apiKey}`,
...formData.getHeaders()
},
timeout: 30000 // 30 second timeout
});
const result = {
success: true,
text: response.data.text || '',
language: response.data.language || 'unknown',
duration: response.data.duration || 0,
confidence: response.data.confidence || 1.0,
timestamp: Date.now(),
provider: 'openai'
};
console.log('OpenAI ASR Result:', {
text: result.text,
language: result.language,
duration: result.duration
});
return result;
} finally {
// Clean up temporary file
try {
fs.unlinkSync(tempFilePath);
} catch (cleanupError) {
console.warn('Failed to cleanup temp file:', cleanupError.message);
}
}
} catch (error) {
console.error('OpenAI ASR error:', error.response?.data || error.message);
return {
success: false,
text: '',
error: error.response?.data?.error?.message || error.message,
timestamp: Date.now(),
provider: 'openai'
};
}
}
}
/**
* SenseVoice via Siliconflow ASR Provider
*/
class SiliconflowASRProvider extends BaseASRProvider {
async transcribe(audioBuffer, tempDir, options = {}) {
try {
// Create WAV buffer
const wavBuffer = this.createWavBuffer(audioBuffer, {
sampleRate: 16000,
channels: 1,
bitsPerSample: 16
});
// Create temporary file
const tempFileName = `audio_${Date.now()}_${Math.random().toString(36).substring(2)}.wav`;
const tempFilePath = path.join(tempDir, tempFileName);
// Write audio to temporary file
fs.writeFileSync(tempFilePath, wavBuffer);
try {
// Create form data
const formData = new FormData();
formData.append('file', fs.createReadStream(tempFilePath));
formData.append('model', this.config.model);
formData.append('response_format', 'json');
// SenseVoice specific parameters
if (options.language) {
formData.append('language', options.language);
} else {
// SenseVoice supports auto language detection
formData.append('language', 'auto');
}
if (options.prompt) {
formData.append('prompt', options.prompt);
}
// Make API request
const response = await axios({
method: 'post',
url: this.config.apiUrl,
data: formData,
headers: {
'Authorization': `Bearer ${this.apiKey}`,
...formData.getHeaders()
},
timeout: 30000 // 30 second timeout
});
const result = {
success: true,
text: response.data.text || '',
language: response.data.language || 'unknown',
duration: response.data.duration || 0,
confidence: response.data.confidence || 1.0,
timestamp: Date.now(),
provider: 'siliconflow'
};
console.log('SenseVoice ASR Result:', {
text: result.text,
language: result.language,
duration: result.duration
});
return result;
} finally {
// Clean up temporary file
try {
fs.unlinkSync(tempFilePath);
} catch (cleanupError) {
console.warn('Failed to cleanup temp file:', cleanupError.message);
}
}
} catch (error) {
console.error('SenseVoice ASR error:', error.response?.data || error.message);
return {
success: false,
text: '',
error: error.response?.data?.error?.message || error.message,
timestamp: Date.now(),
provider: 'siliconflow'
};
}
}
}
/**
* ASR Provider Factory
*/
class ASRProviderFactory {
static createProvider(providerName, config, globalConfig) {
const providerConfig = config.ASR_PROVIDERS[providerName];
if (!providerConfig) {
throw new Error(`ASR provider ${providerName} not found in configuration`);
}
// Get API key from global config
const apiKey = globalConfig[providerConfig.apiKey];
if (!apiKey) {
throw new Error(`API key ${providerConfig.apiKey} not found in configuration`);
}
switch (providerName) {
case 'openai':
return new OpenAIASRProvider(providerConfig, apiKey);
case 'siliconflow':
return new SiliconflowASRProvider(providerConfig, apiKey);
default:
throw new Error(`Unsupported ASR provider: ${providerName}`);
}
}
}
module.exports = {
BaseASRProvider,
OpenAIASRProvider,
SiliconflowASRProvider,
ASRProviderFactory
};
backend/utils/providers/llmProviders.js¶
const axios = require('axios');
/**
* Base LLM Provider class
*/
class BaseLLMProvider {
constructor(config, apiKey) {
this.config = config;
this.apiKey = apiKey;
}
async createChatCompletion(messages, options = {}) {
throw new Error('createChatCompletion method must be implemented by subclass');
}
/**
* Prepare headers for API request
* @returns {Object} Headers object
*/
getHeaders() {
return {
'Authorization': `Bearer ${this.apiKey}`,
'Content-Type': 'application/json'
};
}
/**
* Prepare request payload
* @param {Array} messages - Chat messages
* @param {Object} options - Additional options
* @returns {Object} Request payload
*/
getRequestPayload(messages, options = {}) {
return {
model: this.config.model,
messages: messages,
stream: true,
max_tokens: options.max_tokens || 4096,
...options
};
}
}
/**
* OpenAI LLM Provider
*/
class OpenAILLMProvider extends BaseLLMProvider {
async createChatCompletion(messages, options = {}) {
try {
const payload = this.getRequestPayload(messages, options);
const headers = this.getHeaders();
console.log('OpenAI LLM Request:', {
model: payload.model,
messagesCount: messages.length,
stream: payload.stream
});
const response = await axios.post(
this.config.apiUrl,
payload,
{
headers: headers,
cancelToken: options.cancelToken,
responseType: 'stream'
}
);
return {
success: true,
response: response,
provider: 'openai'
};
} catch (error) {
console.error('OpenAI LLM error:', error.response?.data || error.message);
return {
success: false,
error: error.response?.data?.error?.message || error.message,
provider: 'openai'
};
}
}
}
/**
* OpenRouter LLM Provider (supports both GPT-4o and Gemini)
*/
class OpenRouterLLMProvider extends BaseLLMProvider {
getHeaders() {
return {
'Authorization': `Bearer ${this.apiKey}`,
'Content-Type': 'application/json',
'HTTP-Referer': 'https://live-audio-chat.local',
'X-Title': 'Live Audio Chat'
};
}
async createChatCompletion(messages, options = {}) {
try {
const payload = this.getRequestPayload(messages, options);
const headers = this.getHeaders();
console.log('OpenRouter LLM Request:', {
model: payload.model,
messagesCount: messages.length,
stream: payload.stream
});
const response = await axios.post(
this.config.apiUrl,
payload,
{
headers: headers,
cancelToken: options.cancelToken,
responseType: 'stream'
}
);
return {
success: true,
response: response,
provider: 'openrouter'
};
} catch (error) {
console.error('OpenRouter LLM error:', error.response?.data || error.message);
return {
success: false,
error: error.response?.data?.error?.message || error.message,
provider: 'openrouter'
};
}
}
}
/**
* ARK (Doubao) LLM Provider
*/
class ARKLLMProvider extends BaseLLMProvider {
getHeaders() {
return {
'Authorization': `Bearer ${this.apiKey}`,
'Content-Type': 'application/json'
};
}
getRequestPayload(messages, options = {}) {
// ARK API format is similar to OpenAI but may have slight differences
return {
model: this.config.model,
messages: messages,
stream: true,
max_tokens: options.max_tokens || 4096,
temperature: options.temperature || 0.7,
...options
};
}
async createChatCompletion(messages, options = {}) {
try {
const payload = this.getRequestPayload(messages, options);
const headers = this.getHeaders();
console.log('ARK (Doubao) LLM Request:', {
model: payload.model,
messagesCount: messages.length,
stream: payload.stream
});
const response = await axios.post(
this.config.apiUrl,
payload,
{
headers: headers,
cancelToken: options.cancelToken,
responseType: 'stream'
}
);
return {
success: true,
response: response,
provider: 'ark'
};
} catch (error) {
console.error('ARK (Doubao) LLM error:', error.response?.data || error.message);
return {
success: false,
error: error.response?.data?.error?.message || error.message,
provider: 'ark'
};
}
}
}
/**
* LLM Provider Factory
*/
class LLMProviderFactory {
static createProvider(providerName, config, globalConfig) {
const providerConfig = config.LLM_PROVIDERS[providerName];
if (!providerConfig) {
throw new Error(`LLM provider ${providerName} not found in configuration`);
}
// Get API key from global config
const apiKey = globalConfig[providerConfig.apiKey];
if (!apiKey) {
throw new Error(`API key ${providerConfig.apiKey} not found in configuration`);
}
switch (providerName) {
case 'openai':
return new OpenAILLMProvider(providerConfig, apiKey);
case 'openrouter':
case 'openrouter-gpt':
case 'openrouter-gemini':
return new OpenRouterLLMProvider(providerConfig, apiKey);
case 'ark':
return new ARKLLMProvider(providerConfig, apiKey);
default:
throw new Error(`Unsupported LLM provider: ${providerName}`);
}
}
}
module.exports = {
BaseLLMProvider,
OpenAILLMProvider,
OpenRouterLLMProvider,
ARKLLMProvider,
LLMProviderFactory
};
backend/utils/speechToText.js¶
const fs = require('fs');
const path = require('path');
const config = require('../config');
const { ASRProviderFactory } = require('./providers/asrProviders');
class SpeechToTextService {
constructor() {
this.tempDir = path.join(__dirname, '../temp');
this.ensureTempDirectory();
// Initialize ASR provider based on configuration
this.initializeProvider();
}
/**
* Initialize ASR provider based on configuration
*/
initializeProvider() {
try {
const providerName = config.ASR_PROVIDER || 'openai';
this.asrProvider = ASRProviderFactory.createProvider(providerName, config, config);
console.log(`ASR Provider initialized: ${providerName}`);
} catch (error) {
console.error('Failed to initialize ASR provider:', error);
throw error;
}
}
/**
* Switch ASR provider dynamically
* @param {string} providerName - Provider name to switch to
*/
switchProvider(providerName) {
try {
this.asrProvider = ASRProviderFactory.createProvider(providerName, config, config);
console.log(`ASR Provider switched to: ${providerName}`);
} catch (error) {
console.error('Failed to switch ASR provider:', error);
throw error;
}
}
/**
* Ensure temp directory exists
*/
ensureTempDirectory() {
if (!fs.existsSync(this.tempDir)) {
fs.mkdirSync(this.tempDir, { recursive: true });
}
}
/**
* Transcribe audio using the configured ASR provider
* @param {Buffer} audioBuffer - Raw audio data
* @param {Object} options - Transcription options
* @returns {Promise<Object>} Transcription result
*/
async transcribeAudio(audioBuffer, options = {}) {
if (!this.asrProvider) {
throw new Error('ASR provider not initialized');
}
try {
const result = await this.asrProvider.transcribe(audioBuffer, this.tempDir, options);
return result;
} catch (error) {
console.error('Transcription error:', error);
return {
success: false,
text: '',
error: error.message,
timestamp: Date.now(),
provider: this.asrProvider.config ? 'unknown' : 'uninitialized'
};
}
}
/**
* Clean up old temporary files
*/
cleanupTempFiles() {
try {
const files = fs.readdirSync(this.tempDir);
const now = Date.now();
const maxAge = 10 * 60 * 1000; // 10 minutes
files.forEach(file => {
const filePath = path.join(this.tempDir, file);
const stats = fs.statSync(filePath);
if (now - stats.mtime.getTime() > maxAge) {
try {
fs.unlinkSync(filePath);
console.log('Cleaned up old temp file:', file);
} catch (error) {
console.warn('Failed to cleanup old temp file:', file, error.message);
}
}
});
} catch (error) {
console.warn('Failed to cleanup temp directory:', error.message);
}
}
/**
* Check if audio buffer has sufficient content for transcription
* @param {Buffer} audioBuffer - Audio buffer to check
* @returns {boolean} True if buffer has sufficient content
*/
hasSufficientAudio(audioBuffer) {
if (!audioBuffer || audioBuffer.length === 0) {
return false;
}
// Check minimum duration (at least 0.1 seconds of audio)
const minSamples = config.AUDIO_SAMPLE_RATE * 0.1; // 0.1 seconds
const minBytes = minSamples * 2; // 16-bit = 2 bytes per sample
if (audioBuffer.length < minBytes) {
return false;
}
// Since we're using Silero VAD for speech detection, we trust its decision
// and only check for minimum duration. The energy check is redundant and
// can reject valid speech that Silero VAD correctly identified.
return true;
}
/**
* Get current provider information
* @returns {Object} Current provider information
*/
getProviderInfo() {
if (!this.asrProvider) {
return { provider: 'none', status: 'not initialized' };
}
return {
provider: this.asrProvider.constructor.name,
model: this.asrProvider.config.model,
apiUrl: this.asrProvider.config.apiUrl,
status: 'ready'
};
}
}
module.exports = SpeechToTextService;
backend/utils/textProcessor.js¶
const emojiRegex = require('emoji-regex');
// Remove emoji from the sentence
function removeEmoji(sentence) {
const regex = emojiRegex();
return sentence.replace(regex, ' ').trim();
}
// Convert markdown to plain text
function markdownToText(markdown) {
let text = markdown;
// Remove links, keeping only the link text
text = text.replace(/\[([^\]]+)\]\([^\)]+\)/g, '$1');
// Remove headers
text = text.replace(/^#+\s*/gm, '');
// Remove bold and italic markers
text = text.replace(/\*\*/g, '').replace(/\*/g, '')
.replace(/__/g, '').replace(/_/g, '');
// Remove blockquotes
text = text.replace(/^>\s*/gm, '');
// Remove horizontal rules
text = text.replace(/[-*_]{3,}/g, '');
// Remove list markers
text = text.replace(/^[-*+]\s*/gm, '');
// Remove code block markers
text = text.replace(/```/g, '');
return text.trim();
}
// Convert numbers to words
function numberToWords(num) {
const ones = ['', 'one', 'two', 'three', 'four', 'five', 'six', 'seven', 'eight', 'nine'];
const tens = ['', '', 'twenty', 'thirty', 'forty', 'fifty', 'sixty', 'seventy', 'eighty', 'ninety'];
const teens = ['ten', 'eleven', 'twelve', 'thirteen', 'fourteen', 'fifteen', 'sixteen', 'seventeen', 'eighteen', 'nineteen'];
const scales = ['', 'thousand', 'million', 'billion'];
if (num === 0) return 'zero';
function convertGroup(n) {
let result = '';
if (n >= 100) {
result += ones[Math.floor(n / 100)] + ' hundred ';
n %= 100;
}
if (n >= 20) {
result += tens[Math.floor(n / 10)] + ' ';
n %= 10;
if (n > 0) {
result += ones[n] + ' ';
}
} else if (n >= 10) {
result += teens[n - 10] + ' ';
} else if (n > 0) {
result += ones[n] + ' ';
}
return result;
}
let result = '';
let groupIndex = 0;
while (num > 0) {
const group = num % 1000;
if (group !== 0) {
result = convertGroup(group) + scales[groupIndex] + ' ' + result;
}
num = Math.floor(num / 1000);
groupIndex++;
}
return result.trim();
}
// Pronounce special characters
function pronounceSpecialCharacters(text, isCodeBlock = false) {
const specialCharMap = {
'@': 'at',
'#': 'hash',
'$': 'dollar',
'%': 'percent',
'^': 'caret',
'&': 'ampersand',
'*': 'asterisk',
'_': 'underscore',
'=': 'equals',
'+': 'plus',
'[': 'left square bracket',
']': 'right square bracket',
'{': 'left curly brace',
'}': 'right curly brace',
'|': 'vertical bar',
'\\': 'backslash',
'<': 'less than',
'>': 'greater than',
'/': 'slash',
'`': 'backtick',
'~': 'tilde',
};
const punctuationMap = {
'!': 'exclamation',
'.': 'dot',
',': 'comma',
'?': 'question mark',
';': 'semicolon',
':': 'colon',
'"': 'double quote',
"'": 'single quote',
'-': 'minus',
'(': 'left parenthesis',
')': 'right parenthesis',
};
let processedText = text;
// Replace special characters
Object.entries(specialCharMap).forEach(([char, pronunciation]) => {
processedText = processedText.replace(new RegExp(char.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'), 'g'), ` ${pronunciation} `);
});
// Replace punctuation if in code block
if (isCodeBlock) {
Object.entries(punctuationMap).forEach(([char, pronunciation]) => {
processedText = processedText.replace(new RegExp(char.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'), 'g'), ` ${pronunciation} `);
});
}
return processedText;
}
// Pronounce numbers in text
function pronounceNumbers(text, language) {
if (language.startsWith('zh')) return text;
return text.replace(/(\d+\.?\d*)/g, match => {
const num = parseFloat(match);
if (isNaN(num)) return match;
if (match.includes('.')) {
const [integer, decimal] = match.split('.');
return `${numberToWords(parseInt(integer))} point ${decimal.split('').map(d => numberToWords(parseInt(d))).join(' ')}`;
}
return numberToWords(parseInt(match));
});
}
// Remove emotional indicators
function removeEmotions(text) {
return text.replace(/\*[a-zA-Z0-9 -]*\*/g, '').trim();
}
// Process code blocks
function pronounceCodeBlock(text) {
return text.replace(/`([^`\n]+)`|```(?:[\s\S]*?)```/g, (match) => {
const content = match.startsWith('```')
? match.slice(3, -3)
: match.slice(1, -1);
return pronounceSpecialCharacters(content, true);
});
}
// Main preprocessing function
function preprocessSentence(sentence, language = 'en') {
let processed = sentence;
processed = pronounceCodeBlock(processed);
processed = markdownToText(processed);
processed = pronounceNumbers(processed, language);
processed = removeEmotions(processed);
processed = pronounceSpecialCharacters(processed);
processed = removeEmoji(processed);
return processed.trim();
}
module.exports = {
preprocessSentence
};
backend/utils/vad.js¶
const config = require('../config');
const ort = require('onnxruntime-node');
const path = require('path');
const fs = require('fs');
class VoiceActivityDetector {
constructor(options = {}) {
this.threshold = options.threshold || config.VAD_THRESHOLD || 0.5;
this.frameLength = options.frameLength || config.VAD_FRAME_LENGTH || 512;
this.minSpeechDuration = options.minSpeechDuration || config.VAD_MIN_SPEECH_DURATION || 250;
this.maxSilenceDuration = options.maxSilenceDuration || config.VAD_MAX_SILENCE_DURATION || 500;
this.sampleRate = options.sampleRate || config.AUDIO_SAMPLE_RATE || 16000;
// Silero VAD specific parameters
this.sileroFrameLength = 512; // Fixed frame length for Silero VAD
this.sileroSampleRate = 16000; // Fixed sample rate for Silero VAD
// State variables
this.isSpeaking = false;
this.speechStartTime = null;
this.lastSpeechTime = null;
this.audioBuffer = Buffer.alloc(0);
this.speechBuffer = Buffer.alloc(0);
// ONNX Runtime session
this.session = null;
this.isInitialized = false;
this.initializationPromise = null;
// Silero VAD state
this.state = null;
this.sr = null;
// Initialize the Silero VAD
this.initializationPromise = this.initializeSileroVAD();
}
/**
* Initialize the Silero VAD ONNX model
*/
async initializeSileroVAD() {
try {
console.log('Initializing Silero VAD...');
// Load the ONNX model
const modelPath = path.join(__dirname, '../models/silero_vad.onnx');
if (!fs.existsSync(modelPath)) {
throw new Error(`Silero VAD model not found at ${modelPath}`);
}
// Create ONNX Runtime session
this.session = await ort.InferenceSession.create(modelPath, {
executionProviders: ['cpu'],
graphOptimizationLevel: 'all',
enableMemPattern: true
});
// Initialize state tensors for Silero VAD
this.resetSileroState();
this.isInitialized = true;
console.log('Silero VAD initialized successfully');
} catch (error) {
console.error('Failed to initialize Silero VAD:', error);
throw error;
}
}
/**
* Reset Silero VAD state tensors
*/
resetSileroState() {
// Initialize state tensor for Silero VAD
// The state tensor combines h and c LSTM hidden states
this.state = new ort.Tensor('float32', new Float32Array(2 * 1 * 128).fill(0.0), [2, 1, 128]);
this.sr = new ort.Tensor('int64', new BigInt64Array([BigInt(this.sileroSampleRate)]), [1]);
}
/**
* Convert Buffer to Float32Array for Silero VAD
* @param {Buffer} buffer - PCM 16-bit audio buffer
* @returns {Float32Array} Normalized audio samples
*/
bufferToFloat32Array(buffer) {
const samples = new Float32Array(buffer.length / 2);
for (let i = 0; i < samples.length; i++) {
// Convert 16-bit PCM to normalized float (-1 to 1)
const sample = buffer.readInt16LE(i * 2);
samples[i] = sample / 32768.0;
}
return samples;
}
/**
* Run Silero VAD inference on audio chunk
* @param {Float32Array} audioSamples - Normalized audio samples
* @returns {Promise<number>} Speech probability (0-1)
*/
async runSileroVAD(audioSamples) {
if (!this.session || !this.isInitialized) {
throw new Error('Silero VAD not initialized');
}
try {
// Create input tensor
const inputTensor = new ort.Tensor('float32', audioSamples, [1, audioSamples.length]);
// Run inference
const feeds = {
input: inputTensor,
state: this.state,
sr: this.sr
};
const results = await this.session.run(feeds);
// Update state tensor for next inference
this.state = results.stateN;
// Get speech probability
const speechProb = results.output.data[0];
return speechProb;
} catch (error) {
console.error('Error running Silero VAD inference:', error);
return 0.0; // Return silence probability on error
}
}
/**
* Process audio chunk and detect voice activity using Silero VAD
* @param {Buffer} audioChunk - Raw audio data
* @returns {Promise<Array>} VAD result with detection status and audio data
*/
async processAudioChunk(audioChunk) {
// Wait for initialization if not complete
if (!this.isInitialized) {
await this.initializationPromise;
}
const currentTime = Date.now();
const results = [];
// Add to buffer
this.audioBuffer = Buffer.concat([this.audioBuffer, audioChunk]);
// Process audio in chunks suitable for Silero VAD (512 samples)
const chunkSize = this.sileroFrameLength * 2; // 16-bit = 2 bytes per sample
while (this.audioBuffer.length >= chunkSize) {
const chunk = this.audioBuffer.slice(0, chunkSize);
this.audioBuffer = this.audioBuffer.slice(chunkSize);
try {
// Convert to Float32Array for Silero VAD
const audioSamples = this.bufferToFloat32Array(chunk);
// Run Silero VAD inference
const speechProb = await this.runSileroVAD(audioSamples);
// Check if speech is detected
const isVoiceActive = speechProb > this.threshold;
if (isVoiceActive) {
if (!this.isSpeaking) {
// Speech started
this.isSpeaking = true;
this.speechStartTime = currentTime;
this.speechBuffer = Buffer.alloc(0);
console.log('Silero VAD: Speech started (prob:', speechProb.toFixed(3), ')');
}
this.lastSpeechTime = currentTime;
this.speechBuffer = Buffer.concat([this.speechBuffer, chunk]);
} else if (this.isSpeaking) {
// Check if silence duration exceeds threshold
const silenceDuration = currentTime - this.lastSpeechTime;
if (silenceDuration > this.maxSilenceDuration) {
// Speech ended
const speechDuration = currentTime - this.speechStartTime;
if (speechDuration >= this.minSpeechDuration) {
console.log('Silero VAD: Speech ended', {
duration: speechDuration,
bufferSize: this.speechBuffer.length
});
results.push({
type: 'speech_end',
audioData: this.speechBuffer,
duration: speechDuration,
timestamp: currentTime
});
}
this.isSpeaking = false;
this.speechStartTime = null;
this.lastSpeechTime = null;
this.speechBuffer = Buffer.alloc(0);
} else {
// Still in speech, add chunk to buffer
this.speechBuffer = Buffer.concat([this.speechBuffer, chunk]);
}
}
} catch (error) {
console.error('Error processing audio chunk with Silero VAD:', error);
// Continue processing other chunks
}
}
return results;
}
/**
* Force end current speech session
* @returns {Promise<Object|null>} Final speech data if available
*/
async forceEndSpeech() {
if (this.isSpeaking && this.speechBuffer.length > 0) {
const currentTime = Date.now();
const speechDuration = currentTime - this.speechStartTime;
if (speechDuration >= this.minSpeechDuration) {
const result = {
type: 'speech_end',
audioData: this.speechBuffer,
duration: speechDuration,
timestamp: currentTime
};
this.isSpeaking = false;
this.speechStartTime = null;
this.lastSpeechTime = null;
this.speechBuffer = Buffer.alloc(0);
return result;
}
}
this.reset();
return null;
}
/**
* Reset VAD state
*/
reset() {
this.isSpeaking = false;
this.speechStartTime = null;
this.lastSpeechTime = null;
this.audioBuffer = Buffer.alloc(0);
this.speechBuffer = Buffer.alloc(0);
// Reset Silero VAD state
if (this.isInitialized) {
this.resetSileroState();
}
}
/**
* Get current VAD state
* @returns {Object} Current state information
*/
getState() {
return {
isSpeaking: this.isSpeaking,
speechDuration: this.speechStartTime ? Date.now() - this.speechStartTime : 0,
bufferSize: this.speechBuffer.length,
threshold: this.threshold,
isInitialized: this.isInitialized
};
}
/**
* Cleanup resources
*/
async cleanup() {
if (this.session) {
try {
await this.session.release();
} catch (error) {
console.error('Error releasing ONNX session:', error);
}
this.session = null;
}
this.state = null;
this.sr = null;
this.isInitialized = false;
this.reset();
}
}
module.exports = VoiceActivityDetector;
frontend/next.config.js¶
const path = require('path');
/** @type {import('next').NextConfig} */
const nextConfig = {
reactStrictMode: true,
output: 'export',
env: {
WEBSOCKET_PORT: process.env.WEBSOCKET_PORT || '8848',
IS_PRODUCTION: process.env.NODE_ENV === 'production',
},
// Since we're using static export, we need to disable image optimization
images: {
unoptimized: true,
},
webpack: (config) => {
config.resolve.alias = {
...config.resolve.alias,
'@': path.resolve(__dirname),
};
return config;
}
}
module.exports = nextConfig
frontend/postcss.config.js¶
frontend/public/audioWorklet.js¶
class AudioProcessor extends AudioWorkletProcessor {
constructor() {
super();
this.inputBuffer = [];
// Remove VAD parameters - now using server-side Silero VAD only
// The client-side VAD was primarily for debugging and is no longer needed
}
process(inputs, outputs, parameters) {
const input = inputs[0];
if (!input || !input[0]) return true;
// Convert to mono
const monoInput = new Float32Array(input[0].length);
for (let i = 0; i < input[0].length; i++) {
let sum = 0;
for (let channel = 0; channel < input.length; channel++) {
sum += input[channel][i];
}
monoInput[i] = sum / input.length;
}
// Process in chunks - removed VAD processing
const CHUNK_SIZE = 1024;
this.inputBuffer.push(...monoInput);
while (this.inputBuffer.length >= CHUNK_SIZE) {
const chunk = this.inputBuffer.slice(0, CHUNK_SIZE);
this.inputBuffer = this.inputBuffer.slice(CHUNK_SIZE);
// Convert to 16-bit PCM
const pcmData = new Int16Array(chunk.length);
for (let i = 0; i < chunk.length; i++) {
const s = Math.max(-1, Math.min(1, chunk[i]));
pcmData[i] = s < 0 ? s * 0x8000 : s * 0x7FFF;
}
// Send the data - server-side Silero VAD will handle speech detection
this.port.postMessage(pcmData.buffer, [pcmData.buffer]);
}
return true;
}
}
class EchoProcessor extends AudioWorkletProcessor {
constructor() {
super();
this.audioBuffer = [];
this.playbackPosition = 0;
this.isPlaying = false;
this.sampleRate = 16000;
this.isMuted = false;
this.outputBufferSize = 2048;
this.outputBuffer = new Float32Array(this.outputBufferSize);
this.outputBufferPosition = 0;
this.hasNotifiedQueueEmpty = false; // Track if we've sent the queue empty notification
this.port.onmessage = (event) => {
if (event.data instanceof Float32Array) {
if (!this.isMuted) {
const audioData = event.data;
const newBuffer = new Float32Array(audioData.length);
newBuffer.set(audioData);
if (!this.isPlaying) {
this.audioBuffer = Array.from(newBuffer);
this.playbackPosition = 0;
this.outputBufferPosition = 0;
this.hasNotifiedQueueEmpty = false; // Reset notification flag when starting new playback
} else {
this.audioBuffer.push(...Array.from(newBuffer));
}
this.isPlaying = true;
}
} else if (event.data.type === 'clear') {
// Clear the buffer and stop playback
this.audioBuffer = [];
this.playbackPosition = 0;
this.outputBufferPosition = 0;
this.isPlaying = false;
this.isMuted = true;
this.hasNotifiedQueueEmpty = false;
// Notify that the queue is empty after clearing
this.port.postMessage({ type: 'queue_empty' });
} else if (event.data.type === 'unmute') {
this.isMuted = false;
this.hasNotifiedQueueEmpty = false;
} else if (event.data.type === 'mute') {
this.isMuted = true;
this.audioBuffer = [];
this.playbackPosition = 0;
this.isPlaying = false;
this.hasNotifiedQueueEmpty = false;
// Notify that the queue is empty after muting
this.port.postMessage({ type: 'queue_empty' });
}
};
}
process(inputs, outputs, parameters) {
const output = outputs[0];
// If muted or not playing, output silence
if (this.isMuted || !this.isPlaying || this.audioBuffer.length === 0) {
for (let channel = 0; channel < output.length; channel++) {
output[channel].fill(0);
}
// Send queue_empty notification if we haven't already
if (this.isPlaying && !this.hasNotifiedQueueEmpty) {
this.port.postMessage({ type: 'queue_empty' });
this.hasNotifiedQueueEmpty = true;
this.isPlaying = false;
}
return true;
}
const outputChannel = output[0];
const bufferSize = outputChannel.length;
if (this.isPlaying && this.audioBuffer.length > 0) {
// Fill the output buffer
for (let i = 0; i < bufferSize; i++) {
if (this.playbackPosition < this.audioBuffer.length) {
const sample = this.audioBuffer[this.playbackPosition];
for (let channel = 0; channel < output.length; channel++) {
output[channel][i] = sample;
}
this.playbackPosition++;
} else {
// End of buffer reached
for (let channel = 0; channel < output.length; channel++) {
output[channel][i] = 0;
}
// If we've played everything, reset and notify
if (this.playbackPosition >= this.audioBuffer.length && !this.hasNotifiedQueueEmpty) {
this.isPlaying = false;
this.playbackPosition = 0;
this.audioBuffer = [];
this.port.postMessage({ type: 'queue_empty' });
this.hasNotifiedQueueEmpty = true;
}
}
}
} else {
// Output silence if we're not playing
for (let channel = 0; channel < output.length; channel++) {
output[channel].fill(0);
}
}
return true;
}
}
registerProcessor('audio-processor', AudioProcessor);
registerProcessor('echo-processor', EchoProcessor);
frontend/tailwind.config.js¶
/** @type {import('tailwindcss').Config} */
module.exports = {
darkMode: ["class"],
content: [
'./pages/**/*.{js,ts,jsx,tsx,mdx}',
'./components/**/*.{js,ts,jsx,tsx,mdx}',
'./app/**/*.{js,ts,jsx,tsx,mdx}',
'./src/**/*.{js,ts,jsx,tsx,mdx}',
],
theme: {
container: {
center: true,
padding: "2rem",
screens: {
"2xl": "1400px",
},
},
extend: {
colors: {
border: "hsl(var(--border))",
background: "hsl(var(--background))",
foreground: "hsl(var(--foreground))",
},
keyframes: {
"accordion-down": {
from: { height: 0 },
to: { height: "var(--radix-accordion-content-height)" },
},
"accordion-up": {
from: { height: "var(--radix-accordion-content-height)" },
to: { height: 0 },
},
"fade-in": {
'0%': { opacity: 0 },
'100%': { opacity: 1 },
},
"slide-in": {
'0%': { transform: 'translateY(5px)', opacity: 0 },
'100%': { transform: 'translateY(0)', opacity: 1 },
}
},
animation: {
"accordion-down": "accordion-down 0.2s ease-out",
"accordion-up": "accordion-up 0.2s ease-out",
"fade-in": "fade-in 0.3s ease-out",
"slide-in": "slide-in 0.3s ease-out",
},
typography: (theme) => ({
DEFAULT: {
css: {
'--tw-prose-body': theme('colors.gray.900'),
'--tw-prose-headings': theme('colors.gray.900'),
'--tw-prose-links': theme('colors.blue.600'),
'--tw-prose-code': theme('colors.gray.900'),
maxWidth: 'none',
color: 'var(--tw-prose-body)',
fontSize: '1rem',
lineHeight: '1.75',
p: {
marginTop: '1em',
marginBottom: '1em',
fontSize: '1rem',
'&:first-child': {
marginTop: 0,
},
'&:last-child': {
marginBottom: 0,
},
},
'ul, ol': {
paddingLeft: '1.5em',
marginTop: '0.5em',
marginBottom: '0.5em',
},
li: {
marginTop: '0.25em',
marginBottom: '0.25em',
fontSize: '1rem',
lineHeight: '1.5',
p: {
marginTop: '0.375em',
marginBottom: '0.375em',
},
},
'h1, h2, h3, h4': {
color: 'var(--tw-prose-headings)',
marginTop: '1.5em',
marginBottom: '0.5em',
fontSize: '1.25rem',
fontWeight: '600',
lineHeight: '1.3',
'&:first-child': {
marginTop: 0,
},
},
pre: {
margin: '0.5em 0',
padding: '0.5em',
backgroundColor: 'transparent',
borderRadius: '0.375rem',
fontSize: '0.875rem',
lineHeight: '1.5',
overflowX: 'auto',
},
code: {
color: 'var(--tw-prose-code)',
backgroundColor: theme('colors.gray.100'),
padding: '0.2em 0.4em',
borderRadius: '0.25rem',
fontSize: '0.875rem',
fontWeight: '400',
},
'pre code': {
backgroundColor: 'transparent',
padding: 0,
fontSize: '0.875rem',
color: 'inherit',
fontWeight: '400',
},
blockquote: {
borderLeftWidth: '4px',
borderLeftColor: theme('colors.gray.200'),
paddingLeft: '1em',
fontStyle: 'italic',
marginTop: '1em',
marginBottom: '1em',
fontSize: '1rem',
},
hr: {
marginTop: '2em',
marginBottom: '2em',
},
a: {
color: 'var(--tw-prose-links)',
textDecoration: 'underline',
'&:hover': {
color: theme('colors.blue.700'),
},
},
table: {
width: '100%',
marginTop: '1em',
marginBottom: '1em',
borderCollapse: 'collapse',
fontSize: '0.875rem',
lineHeight: '1.5',
},
'th, td': {
padding: '0.5em',
borderWidth: '1px',
borderColor: theme('colors.gray.200'),
},
},
},
}),
},
},
plugins: [
require('@tailwindcss/typography'),
require("tailwindcss-animate")
],
}
frontend/lib/utils.ts¶
import { type ClassValue, clsx } from "clsx"
import { twMerge } from "tailwind-merge"
export function cn(...inputs: ClassValue[]) {
return twMerge(clsx(inputs))
}