Ascend/SmartMeeting
4
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial SmartMeeting codebase

Browse Source
This commit is contained in:
Defeng 2026-07-28 14:20:12 +08:00
commit ce1b054746
370 changed files with 139907 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

35
.gitignore vendored Normal file
View File

@ -0,0 +1,35 @@
# Local configuration and secrets
.env
.env.*
!.env.example
config.env
# Python caches and virtual environments
__pycache__/
*.py[cod]
.pytest_cache/
.mypy_cache/
.venv/
venv/
# Logs, databases, and runtime data
*.log
*.db
deploy/data/
logs/
uploads/
# Local recordings and generated media
*.wav
*.mp3
*.flac
# Build and editor artifacts
node_modules/
dist/
build/
*.egg-info/
.idea/
.vscode/
.DS_Store
*** End Patch

207
README.md Normal file
View File

@ -0,0 +1,207 @@
# SmartMeeting 智能会议
这是一个基于 Docker Compose 的智能会议部署项目，包含 Speakr 前后端、Paraformer ASR、声纹向量提取服务，以及实时 WebSocket 转写桥接服务。
## 目录结构
```text
/storage01/home/xll/202607/voice
├── docker-compose.yml # 统一启动文件
├── config.env # 运行环境变量，主要改这里
├── asr/main.py # ASR 适配服务代码
├── speakr/ # Speakr 前后端代码
├── voice-dected/ # 声纹向量提取服务代码
├── fastapi_wss/ # 实时 WebSocket 桥接服务代码
└── deploy/data/ # 运行数据，已被 git 忽略
```
## 启动和停止
启动全部服务：
```bash
cd /storage01/home/xll/202607/voice
docker compose up -d
```
查看状态和日志：
```bash
docker compose ps
docker compose logs -f
```
停止服务：
```bash
docker compose down
```
## 服务和端口
| 服务 | 容器名 | 端口 | 作用 |
| --- | --- | --- | --- |
| Speakr | `voice-speakr` | `8899` | 页面和后端 |
| ASR | `asr-test` | `59805` | Paraformer HTTP / WebSocket 识别 |
| Python 实时 ASR | `funasr-online` | `10096` | FunASR Python WebSocket，保留用于回退 |
| 官方 Runtime 实时 ASR | `funasr-runtime-2pass` | `10097` | FunASR 官方 C++/ONNX 2pass WebSocket |
| 声纹提取 | `voice-dected` | `18000` | `/extract_embedding` 声纹向量服务 |
| 实时桥接 | `fastapi-wss` | `10095` | 页面实时 ASR WebSocket 桥接 |
| Redis | `voice-redis` | `16380` | 实时桥接的缓存/会话存储 |
页面访问地址：
```text
http://192.168.0.46:8899/tool/speakr/
```
浏览器麦克风录音需要安全上下文。局域网 IP 的 HTTP 页面可能无法调用麦克风。最快的本机访问方式是开 SSH 隧道：
```bash
ssh -L 8899:127.0.0.1:8899 xll@192.168.0.46
```
然后浏览器打开：
```text
http://localhost:8899/tool/speakr/
```
## 配置文件
主要修改这个文件：
```text
/storage01/home/xll/202607/voice/config.env
```
改服务地址、模型地址、LLM 配置、声纹匹配阈值时，优先改 `config.env`。修改后执行：
```bash
cd /storage01/home/xll/202607/voice
docker compose up -d
```
重要变量：
```env
ASR_BASE_URL=http://asr-test:59805
FUNASR_WEBSOCKET_IP=ws://fastapi-wss:10095/websocket_offline
TARGET_WS_URL=ws://funasr-runtime-2pass:10095
# 如需回退到 Python 版实时 ASR，可改为：
# TARGET_WS_URL=ws://funasr-online:10096
VOICE_DETECTED_URL=http://voice-dected:8000
SPEAKER_DET_URL=http://voice-dected:8000/extract_embedding
REDIS_URL=redis://redis:6379/0
TEXT_MODEL_BASE_URL=http://192.168.0.46:59800/v1
TEXT_MODEL_NAME=46-qwen3.5-35B
TEXT_MODEL_API_KEY=none
VOICEPRINT_STORE=/data/asr_voiceprints.json
VOICEPRINT_MATCH_THRESHOLD=0.45
REALTIME_VOICEPRINT_ENABLED=true
REALTIME_VOICEPRINT_WINDOW_SECONDS=2.0
```
说明：
- Compose 内部服务互相访问时，用 `asr-test`、`voice-dected`、`redis` 这种服务名。
- 宿主机或外部机器访问时，用 `192.168.0.46:59805`、`192.168.0.46:18000`、`192.168.0.46:10095` 这种宿主机端口。
- `FUNASR_WEBSOCKET_IP` 是 Speakr 后端实时录音代理要连接的 WebSocket 地址。
- `TARGET_WS_URL` 是 `fastapi-wss` 再往后连接的 ASR WebSocket 地址。
- `VOICEPRINT_MATCH_THRESHOLD` 是声纹匹配阈值。值越低越容易匹配，值越高越不容易误匹配。
- `REALTIME_VOICEPRINT_WINDOW_SECONDS` 是实时声纹匹配窗口，当前为 2 秒；识别抖动时可调到 3-5 秒。
## 运行数据
运行数据放在：
```text
/storage01/home/xll/202607/voice/deploy/data/
```
常见子目录：
```text
deploy/data/speakr/uploads/ # 上传和录音文件
deploy/data/speakr/instance/ # Speakr SQLite 数据库
deploy/data/asr/ # ASR 声纹库
deploy/data/redis/ # Redis 持久化数据
```
这些运行数据已被 `.gitignore` 忽略，不会提交到仓库。
## 声纹流程
Speakr 声纹管理页面会调用 ASR 适配层的兼容接口：
```text
POST /voice_insert
GET /get_voice_name
DELETE /delete_voice
```
ASR 适配层会把声纹向量保存到：
```text
deploy/data/asr/asr_voiceprints.json
```
转录时，`/asr` 会先把上传音频转成 `16k 单声道 wav`，再调用 `voice-dected` 提取声纹向量，并和已录入声纹做相似度匹配。如果分数超过 `VOICEPRINT_MATCH_THRESHOLD`，返回匹配到的说话人名称。
目前这个逻辑是“整段音频匹配一个最像的人”。如果一段录音里多人轮流说话，还需要进一步接入分段说话人识别。
## 常用检查命令
检查 ASR：
```bash
curl http://127.0.0.1:59805/health
```
检查声纹列表：
```bash
curl http://127.0.0.1:59805/get_voice_name
```
检查 Speakr 配置：
```bash
curl http://127.0.0.1:8899/tool/speakr/api/config
```
检查实时 WebSocket 链路：
```bash
docker exec -it fastapi-wss python - <<'PY'
import asyncio, json, websockets
async def main():
async with websockets.connect("ws://127.0.0.1:10095") as ws:
await ws.send(json.dumps({"mode": "2pass", "is_speaking": True}))
await ws.send(json.dumps({"type": "stop", "is_speaking": False}))
print(await ws.recv())
asyncio.run(main())
PY
```
## Git 操作
仓库地址：
```text
https://gitea.zkzdht.com/Ascend/SmartMeeting.git
```
常用命令：
```bash
cd /storage01/home/xll/202607/voice
git status
git add .
git commit -m "更新说明"
git push
```

1002
asr/main.py Normal file
View File

File diff suppressed because it is too large Load Diff

668
asr/main.py.bak Normal file
View File

@ -0,0 +1,668 @@
"""
FunASR Paraformer 语音识别 FastAPI 服务
ASR 模型: speech_seaco_paraformer_large_asr_nat-zh-cn-16k-common-vocab8404-pytorch
VAD 模型: speech_fsmn_vad_zh-cn-16k-common-pytorch
PUNC 模型: punc_ct-transformer_zh-cn-common-vocab272727-pytorch
端口: 59805
"""
import os
import re
import json
import asyncio
import tempfile
import logging
import subprocess
import wave
from typing import Optional
import requests
import uvicorn
from fastapi import FastAPI, UploadFile, File, Form, HTTPException, Request, WebSocket, WebSocketDisconnect
from fastapi.responses import JSONResponse, PlainTextResponse
from funasr import AutoModel
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)
_CN_NUM = {
"零": 0,
"一": 1,
"幺": 1,
"二": 2,
"两": 2,
"三": 3,
"四": 4,
"五": 5,
"六": 6,
"七": 7,
"八": 8,
"九": 9,
}
_CN_UNIT = {
"十": 10,
"百": 100,
"千": 1000,
"万": 10000,
"亿": 100000000,
}
_CN_PATTERN = re.compile(r"[零一幺二两三四五六七八九十百千万亿]+")
def _cn2num(s: str) -> int:
total, section, num = 0, 0, 0
for ch in s:
if ch in _CN_NUM:
num = _CN_NUM[ch]
elif ch in _CN_UNIT:
unit = _CN_UNIT[ch]
if unit >= 10000:
section = (section + (num or 1)) * unit
total += section
section = 0
else:
section += (num or 1) * unit
num = 0
return total + section + num
def cn_to_arabic(text: str) -> str:
return _CN_PATTERN.sub(lambda m: str(_cn2num(m.group())), text)
def read_txt_file(file_path: str) -> str:
try:
with open(file_path, "r", encoding="utf-8") as file:
return file.read()
except FileNotFoundError:
logger.warning(f"找不到热词文件: {file_path}")
return ""
except Exception as e:
logger.warning(f"读取热词文件失败: {e}")
return ""
app = FastAPI(
title="Paraformer ASR API",
description="基于 FunASR Paraformer 的中文语音识别服务",
version="1.0.0",
)
model = None
VOICE_DETECTED_URL = os.environ.get("VOICE_DETECTED_URL", "http://127.0.0.1:18000")
VOICEPRINT_STORE = os.environ.get("VOICEPRINT_STORE", "/tmp/asr_voiceprints.json")
VOICEPRINT_MATCH_THRESHOLD = float(os.environ.get("VOICEPRINT_MATCH_THRESHOLD", "0.45"))
REALTIME_PARTIAL_INTERVAL_SECONDS = float(os.environ.get("REALTIME_PARTIAL_INTERVAL_SECONDS", "2.0"))
REALTIME_PARTIAL_MIN_SECONDS = float(os.environ.get("REALTIME_PARTIAL_MIN_SECONDS", "1.5"))
def _voiceprint_key(database_name: str, collection_name: str) -> str:
return f"{database_name or 'NB'}::{collection_name or 'voice'}"
def _load_voiceprints() -> dict:
if not os.path.exists(VOICEPRINT_STORE):
return {}
try:
with open(VOICEPRINT_STORE, "r", encoding="utf-8") as f:
data = json.load(f)
return data if isinstance(data, dict) else {}
except Exception as e:
logger.warning(f"读取声纹库失败，将使用空库: {e}")
return {}
def _save_voiceprints(data: dict):
os.makedirs(os.path.dirname(VOICEPRINT_STORE) or ".", exist_ok=True)
tmp_path = f"{VOICEPRINT_STORE}.tmp"
with open(tmp_path, "w", encoding="utf-8") as f:
json.dump(data, f, ensure_ascii=False)
os.replace(tmp_path, VOICEPRINT_STORE)
def _extract_embedding(audio_path: str, filename: str) -> list:
with open(audio_path, "rb") as f:
response = requests.post(
f"{VOICE_DETECTED_URL.rstrip('/')}/extract_embedding",
files={"wav_file": (filename or "voice.wav", f, "audio/wav")},
timeout=120,
)
response.raise_for_status()
payload = response.json()
embedding = payload.get("embedding")
if not isinstance(embedding, list):
raise ValueError("voice-dected 返回中没有有效 embedding")
return embedding
def _convert_to_embedding_wav(audio_path: str) -> str:
output = tempfile.NamedTemporaryFile(delete=False, suffix=".wav")
output_path = output.name
output.close()
try:
subprocess.run(
[
"ffmpeg",
"-y",
"-i",
audio_path,
"-acodec",
"pcm_s16le",
"-ar",
"16000",
"-ac",
"1",
output_path,
],
check=True,
capture_output=True,
text=True,
timeout=60,
)
return output_path
except Exception:
if os.path.exists(output_path):
os.unlink(output_path)
raise
def _cosine_similarity(a: list, b: list) -> float:
if not a or not b or len(a) != len(b):
return -1.0
dot = sum(float(x) * float(y) for x, y in zip(a, b))
norm_a = sum(float(x) * float(x) for x in a) ** 0.5
norm_b = sum(float(y) * float(y) for y in b) ** 0.5
if norm_a == 0 or norm_b == 0:
return -1.0
return dot / (norm_a * norm_b)
def _match_voiceprint(audio_path: str, filename: str, database_name: str = "NB", collection_name: str = "voice"):
store = _load_voiceprints()
records = store.get(_voiceprint_key(database_name, collection_name), {})
if not records:
return None, None
embedding_path = _convert_to_embedding_wav(audio_path)
try:
embedding = _extract_embedding(embedding_path, "match.wav")
finally:
if os.path.exists(embedding_path):
os.unlink(embedding_path)
best_name = None
best_score = -1.0
for name, record in records.items():
score = _cosine_similarity(embedding, record.get("embedding", []))
if score > best_score:
best_name = name
best_score = score
logger.info(f"声纹匹配结果: name={best_name}, score={best_score:.4f}, threshold={VOICEPRINT_MATCH_THRESHOLD}")
if best_name and best_score >= VOICEPRINT_MATCH_THRESHOLD:
return best_name, best_score
return None, best_score
def load_model():
global model
if model is None:
logger.info("正在加载 Paraformer 模型...")
model = AutoModel(
model="/app/models/seacomodel",
vad_model="/app/models/vadmodel",
punc_model="/app/models/ctpuncmodel",
)
logger.info("模型加载完成")
return model
@app.on_event("startup")
async def startup_event():
load_model()
@app.get("/")
async def root():
return {
"status": "ok",
"message": "Paraformer ASR 服务运行中",
}
@app.get("/health")
async def health():
return {"status": "healthy"}
def _write_pcm_wav(audio_bytes: bytes) -> str:
with tempfile.NamedTemporaryFile(delete=False, suffix=".wav") as tmp:
tmp_path = tmp.name
with wave.open(tmp_path, "wb") as wav_file:
wav_file.setnchannels(1)
wav_file.setsampwidth(2)
wav_file.setframerate(16000)
wav_file.writeframes(audio_bytes)
return tmp_path
def _text_delta(previous: str, current: str) -> str:
previous = previous or ""
current = current or ""
if not current or current == previous:
return ""
if current.startswith(previous):
return current[len(previous):]
max_overlap = min(len(previous), len(current), 80)
for size in range(max_overlap, 0, -1):
if previous[-size:] == current[:size]:
return current[size:]
return current
async def run_asr_websocket(websocket: WebSocket):
await websocket.accept(subprotocol="binary")
audio_chunks = []
partial_chunks = []
last_partial_at = 0.0
try:
while True:
message = await websocket.receive()
if "bytes" in message and message["bytes"]:
audio_chunks.append(message["bytes"])
partial_chunks.append(message["bytes"])
audio_seconds = sum(len(chunk) for chunk in audio_chunks) / 32000.0
partial_seconds = sum(len(chunk) for chunk in partial_chunks) / 32000.0
now = asyncio.get_running_loop().time()
if (
partial_seconds >= REALTIME_PARTIAL_MIN_SECONDS
and now - last_partial_at >= REALTIME_PARTIAL_INTERVAL_SECONDS
):
last_partial_at = now
partial_audio = b"".join(partial_chunks)
partial_chunks = []
partial_path = None
try:
partial_path = _write_pcm_wav(partial_audio)
result = await run_asr(partial_path, language="zh", response_format="json")
partial_text = result.get("text", "") if isinstance(result, dict) else ""
if partial_text:
await websocket.send_json({
"mode": "2pass-online",
"text": partial_text,
"timestamp": str(int(audio_seconds * 1000)),
"is_final": False,
})
except Exception as e:
logger.warning(f"实时部分识别失败: {e}")
finally:
if partial_path and os.path.exists(partial_path):
os.unlink(partial_path)
continue
if "text" in message and message["text"]:
try:
payload = json.loads(message["text"])
except Exception:
payload = {}
if payload.get("type") == "stop" or payload.get("is_speaking") is False:
break
continue
if message.get("type") == "websocket.disconnect":
break
except WebSocketDisconnect:
pass
text = ""
tmp_path = None
try:
if audio_chunks:
tmp_path = _write_pcm_wav(b"".join(audio_chunks))
result = await run_asr(tmp_path, language="zh", response_format="json")
if isinstance(result, dict):
text = result.get("text", "")
try:
await websocket.send_json({
"mode": "2pass-offline",
"text": text,
"is_final": True,
})
except Exception as e:
logger.warning(f"Skip final transcript because websocket is closed: {e}")
finally:
if tmp_path and os.path.exists(tmp_path):
os.unlink(tmp_path)
try:
await websocket.close()
except Exception:
pass
@app.websocket("/ws/asr")
async def websocket_asr(websocket: WebSocket):
await run_asr_websocket(websocket)
@app.websocket("/")
async def websocket_asr_root(websocket: WebSocket):
await run_asr_websocket(websocket)
def make_speakr_response(text: str, language: str = "zh", speaker: str = "SPEAKER_00"):
text = (text or "").strip()
return {
"text": text,
"language": language,
"segments": [
{
"start": None,
"end": None,
"text": text,
"speaker": speaker or "SPEAKER_00",
}
] if text else [],
}
async def run_asr(
audio_path: str,
language: str = "zh",
response_format: str = "json",
):
try:
hotword = read_txt_file("/app/hotword.txt")
result = model.generate(input=audio_path, hotword=hotword)
if result and len(result) > 0:
text = result[0].get("text", "")
else:
text = ""
text = cn_to_arabic(text)
logger.info(f"识别结果: {text}")
if response_format == "text":
return PlainTextResponse(content=text)
return {
"text": text,
"language": language,
}
except Exception as e:
logger.error(f"识别出错: {e}")
raise HTTPException(status_code=500, detail=f"识别失败: {str(e)}")
@app.post("/api/v1/audio/transcriptions")
async def transcribe(
request: Request,
file: Optional[UploadFile] = File(None),
language: Optional[str] = Form("zh"),
response_format: Optional[str] = Form("json"),
):
"""
语音转文字接口，兼容两种方式：
1. multipart/form-data 上传文件：
- file: 音频文件
- language: 默认 zh
- response_format: json/text
2. application/json 传服务器本地文件路径：
{
"file_path": "/app/audio/test.wav",
"language": "zh",
"response_format": "json"
}
"""
tmp_path = None
should_delete = False
try:
content_type = request.headers.get("content-type", "").lower()
if "application/json" in content_type:
body = await request.json()
file_path = body.get("file_path")
language = body.get("language", language)
response_format = body.get("response_format", response_format)
if not file_path:
raise HTTPException(status_code=400, detail="缺少 file_path")
if not os.path.exists(file_path):
raise HTTPException(status_code=400, detail=f"文件不存在: {file_path}")
if not os.path.isfile(file_path):
raise HTTPException(status_code=400, detail=f"不是有效文件: {file_path}")
audio_path = file_path
logger.info(f"处理服务器本地文件: {audio_path}")
else:
if file is None:
raise HTTPException(status_code=400, detail="缺少上传文件 file")
suffix = os.path.splitext(file.filename or "")[1] or ".wav"
with tempfile.NamedTemporaryFile(delete=False, suffix=suffix) as tmp:
content = await file.read()
if not content:
raise HTTPException(status_code=400, detail="上传文件为空")
tmp.write(content)
tmp_path = tmp.name
should_delete = True
audio_path = tmp_path
logger.info(
f"处理上传文件: {file.filename}, 大小: {len(content)} bytes, 临时路径: {audio_path}"
)
return await run_asr(
audio_path=audio_path,
language=language or "zh",
response_format=response_format or "json",
)
finally:
if should_delete and tmp_path and os.path.exists(tmp_path):
os.unlink(tmp_path)
@app.post("/v1/audio/transcriptions")
async def transcribe_v1(
request: Request,
file: Optional[UploadFile] = File(None),
language: Optional[str] = Form("zh"),
response_format: Optional[str] = Form("json"),
):
return await transcribe(
request=request,
file=file,
language=language,
response_format=response_format,
)
@app.post("/asr")
async def transcribe_speakr_compatible(
audio_file: UploadFile = File(...),
language: Optional[str] = Form("zh"),
database_name: Optional[str] = Form("NB"),
collection_name: Optional[str] = Form("voice"),
):
"""
Speakr 兼容接口：
- 上传字段：audio_file
- 返回字段：segments，其中每个 segment 含 text/speaker/start/end
"""
tmp_path = None
try:
suffix = os.path.splitext(audio_file.filename or "")[1] or ".wav"
with tempfile.NamedTemporaryFile(delete=False, suffix=suffix) as tmp:
content = await audio_file.read()
if not content:
raise HTTPException(status_code=400, detail="上传文件为空")
tmp.write(content)
tmp_path = tmp.name
logger.info(
f"处理 Speakr /asr 上传文件: {audio_file.filename}, 大小: {len(content)} bytes, 临时路径: {tmp_path}"
)
result = await run_asr(
audio_path=tmp_path,
language=language or "zh",
response_format="json",
)
speaker_name = "SPEAKER_00"
try:
matched_name, score = _match_voiceprint(
tmp_path,
audio_file.filename or "audio.wav",
database_name or "NB",
collection_name or "voice",
)
if matched_name:
speaker_name = matched_name
except Exception as e:
logger.warning(f"声纹匹配失败，使用默认说话人: {e}")
return make_speakr_response(
text=result.get("text", "") if isinstance(result, dict) else "",
language=result.get("language", language or "zh") if isinstance(result, dict) else language or "zh",
speaker=speaker_name,
)
finally:
if tmp_path and os.path.exists(tmp_path):
os.unlink(tmp_path)
@app.get("/get_voice_name")
async def get_voice_name(
database_name: str = "NB",
collection_name: str = "voice",
):
store = _load_voiceprints()
records = store.get(_voiceprint_key(database_name, collection_name), {})
return {
"code": 0,
"success": True,
"message": "ok",
"data": [
{"name": name, "voice_name": name}
for name in sorted(records.keys())
],
}
@app.post("/voice_insert")
async def voice_insert(
voice_name: str = Form(...),
database_name: str = Form("NB"),
collection_name: str = Form("voice"),
voice_file: UploadFile = File(...),
):
tmp_path = None
try:
suffix = os.path.splitext(voice_file.filename or "")[1] or ".wav"
with tempfile.NamedTemporaryFile(delete=False, suffix=suffix) as tmp:
content = await voice_file.read()
if not content:
raise HTTPException(status_code=400, detail="上传声纹文件为空")
tmp.write(content)
tmp_path = tmp.name
embedding = _extract_embedding(tmp_path, voice_file.filename or f"{voice_name}.wav")
store = _load_voiceprints()
key = _voiceprint_key(database_name, collection_name)
records = store.setdefault(key, {})
records[voice_name] = {
"name": voice_name,
"database_name": database_name,
"collection_name": collection_name,
"embedding": embedding,
}
_save_voiceprints(store)
return {
"code": 0,
"success": True,
"message": "voice inserted",
"data": {"name": voice_name, "embedding_size": len(embedding)},
}
except requests.RequestException as e:
logger.error(f"调用 voice-dected 失败: {e}")
raise HTTPException(status_code=502, detail=f"voice-dected 调用失败: {e}")
except Exception as e:
logger.error(f"写入声纹失败: {e}")
raise HTTPException(status_code=500, detail=f"写入声纹失败: {e}")
finally:
if tmp_path and os.path.exists(tmp_path):
os.unlink(tmp_path)
@app.delete("/delete_voice")
async def delete_voice(
name: str,
database_name: str = "NB",
collection_name: str = "voice",
):
store = _load_voiceprints()
key = _voiceprint_key(database_name, collection_name)
records = store.setdefault(key, {})
existed = name in records
records.pop(name, None)
_save_voiceprints(store)
return {
"code": 0,
"success": True,
"message": "voice deleted" if existed else "voice not found",
"data": {"name": name, "deleted": existed},
}
if __name__ == "__main__":
uvicorn.run(
"main:app",
host="0.0.0.0",
port=59805,
reload=False,
workers=1,
log_level="info",
)

0
comn Normal file
View File

41
deploy/docker-compose.yml Normal file
View File

@ -0,0 +1,41 @@
services:
speakr:
build:
context: ..
dockerfile: deploy/speakr.runtime.Dockerfile
image: voice-speakr:runtime
container_name: voice-speakr
restart: unless-stopped
working_dir: /app
environment:
PYTHONPATH: /app
SQLALCHEMY_DATABASE_URI: sqlite:////data/instance/transcriptions.db
UPLOAD_FOLDER: /data/uploads
USE_ASR_ENDPOINT: "true"
ASR_BASE_URL: http://192.168.0.46:59805
TEXT_MODEL_BASE_URL: http://192.168.0.46:59800/v1
TEXT_MODEL_NAME: 46-qwen3.5-35B
TEXT_MODEL_API_KEY: none
volumes:
- ../speakr:/app
- ./data/speakr/uploads:/data/uploads
- ./data/speakr/instance:/data/instance
ports:
- "8899:8899"
depends_on:
- voice-dected
voice-dected:
build:
context: ..
dockerfile: deploy/voice-dected.runtime.Dockerfile
image: voice-dected:runtime
container_name: voice-dected
restart: unless-stopped
working_dir: /app
environment:
PYTHONPATH: /app
volumes:
- ../voice-dected/src:/app
ports:
- "18000:8000"

29
deploy/speakr.runtime.Dockerfile Normal file
View File

@ -0,0 +1,29 @@
FROM python:3.11-slim
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \
PIP_NO_CACHE_DIR=1 \
PYTHONPATH=/app \
FLASK_APP=src/app.py \
SQLALCHEMY_DATABASE_URI=sqlite:////data/instance/transcriptions.db \
UPLOAD_FOLDER=/data/uploads
WORKDIR /app
RUN apt-get update && apt-get install -y --no-install-recommends \
ffmpeg \
libgomp1 \
curl \
&& rm -rf /var/lib/apt/lists/*
COPY speakr/requirements.txt /tmp/requirements.txt
RUN pip install --upgrade pip setuptools wheel \
&& pip install --index-url https://download.pytorch.org/whl/cpu torch \
&& pip install -r /tmp/requirements.txt \
&& rm -rf /root/.cache/pip
RUN mkdir -p /data/uploads /data/instance
EXPOSE 8899
CMD ["sh", "-c", "mkdir -p /data/uploads /data/instance && python -c \"from src.app import app; from src.clients import db; app.app_context().push(); db.create_all()\" && exec gunicorn --workers ${GUNICORN_WORKERS:-3} --threads ${GUNICORN_THREADS:-8} --bind 0.0.0.0:8899 --timeout ${GUNICORN_TIMEOUT:-600} src.app:app"]

7
deploy/start.sh Normal file
View File

@ -0,0 +1,7 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")"
mkdir -p data/speakr/uploads data/speakr/instance
docker compose up -d --build
docker compose ps

27
deploy/voice-dected.runtime.Dockerfile Normal file
View File

@ -0,0 +1,27 @@
FROM python:3.8-slim
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \
PIP_NO_CACHE_DIR=1 \
PYTHONPATH=/app
WORKDIR /app
RUN apt-get update && apt-get install -y --no-install-recommends \
libsndfile1 \
sox \
libsox-dev \
curl \
&& rm -rf /var/lib/apt/lists/*
RUN pip install --upgrade pip setuptools wheel \
&& pip install --index-url https://download.pytorch.org/whl/cpu torch torchaudio
COPY voice-dected/requirements.txt /tmp/requirements.txt
RUN grep -v -E '^[[:space:]]*(torch|torchaudio)[[:space:]]*$' /tmp/requirements.txt > /tmp/requirements-no-torch.txt \
&& pip install -i https://pypi.tuna.tsinghua.edu.cn/simple -r /tmp/requirements-no-torch.txt \
&& rm -rf /root/.cache/pip
EXPOSE 8000
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]

226
docker-compose.yml Normal file
View File

@ -0,0 +1,226 @@
services:
speakr:
image: voice-speakr:runtime
container_name: voice-speakr
restart: unless-stopped
working_dir: /app
env_file:
- ./config.env
environment:
PYTHONPATH: /app
SQLALCHEMY_DATABASE_URI: sqlite:////data/instance/transcriptions.db
UPLOAD_FOLDER: /data/uploads
USE_ASR_ENDPOINT: "true"
ASR_BASE_URL: http://asr-test:59805
volumes:
- ./speakr:/app
- ./deploy/data/speakr/uploads:/data/uploads
- ./deploy/data/speakr/instance:/data/instance
ports:
- "8899:8899"
depends_on:
- asr-test
- voice-dected
command:
- sh
- -c
- >
mkdir -p /data/uploads /data/instance &&
python -c "from src.app import app; from src.clients import db; app.app_context().push(); db.create_all()" &&
exec gunicorn --workers $${GUNICORN_WORKERS:-3} --threads $${GUNICORN_THREADS:-8}
--bind 0.0.0.0:8899 --timeout $${GUNICORN_TIMEOUT:-600} src.app:app
voice-dected:
image: voice-dected:runtime
container_name: voice-dected
restart: unless-stopped
working_dir: /app
env_file:
- ./config.env
environment:
PYTHONPATH: /app
volumes:
- ./voice-dected/src:/app
ports:
- "18000:8000"
command:
- uvicorn
- app:app
- --host
- 0.0.0.0
- --port
- "8000"
asr-test:
image: paraformer-asr:latest
container_name: asr-test
restart: unless-stopped
runtime: nvidia
gpus:
- driver: nvidia
device_ids:
- "7"
capabilities:
- gpu
working_dir: /app
env_file:
- ./config.env
environment:
DEVICE: cuda
CUDA_VISIBLE_DEVICES: "0"
NVIDIA_VISIBLE_DEVICES: "7"
VOICE_DETECTED_URL: http://voice-dected:8000
VOICEPRINT_STORE: /data/asr_voiceprints.json
volumes:
- ./asr/main.py:/app/main.py:ro
- /storage01/home/xll/audio:/app/audio:ro
- /storage01/home/xll/weixiu-hj/seacomodel:/app/models/seacomodel:ro
- /storage01/home/xll/weixiu-hj/vadmodel:/app/models/vadmodel:ro
- /storage01/home/xll/weixiu-hj/ctpuncmodel:/app/models/ctpuncmodel:ro
- /storage01/home/xll/weixiu-hj/speech_seaco_paraformer_large_asr_nat-zh-cn-16k-common-vocab8404-pytorch/example/hotword.txt:/app/hotword.txt:ro
- ./deploy/data/asr:/data
ports:
- "59805:59805"
depends_on:
- voice-dected
- fastapi-wss
command:
- uvicorn
- main:app
- --host
- 0.0.0.0
- --port
- "59805"
- --workers
- "1"
funasr-online:
image: paraformer-asr:latest
container_name: funasr-online
restart: unless-stopped
runtime: nvidia
gpus:
- driver: nvidia
device_ids:
- "7"
capabilities:
- gpu
working_dir: /app/funasr_runtime
env_file:
- ./config.env
environment:
DEVICE: cuda
CUDA_VISIBLE_DEVICES: "0"
NVIDIA_VISIBLE_DEVICES: "7"
MODELSCOPE_CACHE: /models
TORCH_HOME: /models/torch
volumes:
- ./funasr_runtime:/app/funasr_runtime:ro
- ./deploy/data/funasr-models:/models
ports:
- "10096:10096"
command:
- python
- funasr_wss_server.py
- --host
- 0.0.0.0
- --port
- "10096"
- --ngpu
- "1"
- --device
- cuda
- --ncpu
- "4"
- --asr_model_online
- iic/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-online
- --vad_model
- iic/speech_fsmn_vad_zh-cn-16k-common-pytorch
- --punc_model
- iic/punc_ct-transformer_zh-cn-common-vad_realtime-vocab272727
- --worker_threads
- "4"
- --concurrent_asr_online
- "1"
- --concurrent_asr_offline
- "1"
- --concurrent_vad
- "1"
- --concurrent_punc
- "1"
- --certfile
- ""
- --keyfile
- ""
funasr-runtime-2pass:
image: registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-online-cpu-0.1.13
container_name: funasr-runtime-2pass
restart: unless-stopped
working_dir: /workspace/FunASR/runtime
volumes:
- ./deploy/data/funasr-runtime-models:/workspace/models
ports:
- "10097:10095"
command:
- bash
- -lc
- >
touch /workspace/models/hotwords.txt &&
exec /workspace/FunASR/runtime/websocket/build/bin/funasr-wss-server-2pass
--download-model-dir /workspace/models
--vad-dir damo/speech_fsmn_vad_zh-cn-16k-common-onnx
--model-dir damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-onnx
--online-model-dir damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-online-onnx
--punc-dir damo/punc_ct-transformer_zh-cn-common-vad_realtime-vocab272727-onnx
--lm-dir damo/speech_ngram_lm_zh-cn-ai-wesp-fst
--itn-dir thuduj12/fst_itn_zh
--decoder-thread-num 4
--model-thread-num 1
--io-thread-num 1
--port 10095
--hotword /workspace/models/hotwords.txt
--certfile ""
--keyfile ""
fastapi-wss:
image: fastapi-wss:runtime
container_name: fastapi-wss
restart: unless-stopped
working_dir: /app/src
env_file:
- ./config.env
environment:
TARGET_WS_URL: ws://funasr-runtime-2pass:10095
REDIS_URL: redis://redis:6379/0
VOICEPRINT_STORE: /data/asr/asr_voiceprints.json
SPEAKER_DET_URL: http://voice-dected:8000/extract_embedding
REALTIME_VOICEPRINT_ENABLED: "true"
REALTIME_VOICEPRINT_WINDOW_SECONDS: "2.0"
volumes:
- ./fastapi_wss:/app
- ./deploy/data/asr:/data/asr:ro
ports:
- "10095:10095"
depends_on:
- funasr-runtime-2pass
- voice-dected
- redis
command:
- python
- a2a_wss.py
redis:
image: redis:7-alpine
container_name: voice-redis
restart: unless-stopped
volumes:
- ./deploy/data/redis:/data
ports:
- "16380:6379"
command:
- redis-server
- --appendonly
- "yes"

9
fastapi_wss/.claude/settings.local.json Normal file
View File

@ -0,0 +1,9 @@
{
"permissions": {
"allow": [
"Bash(nc:*)",
"Bash(curl:*)",
"Bash(timeout 5 curl -v -H \"Connection: Upgrade\" -H \"Upgrade: websocket\" -H \"Sec-WebSocket-Version: 13\" -H \"Sec-WebSocket-Key: test\" http://172.18.127.124:10096/)"
]
}
}

5
fastapi_wss/.env.example Normal file
View File

@ -0,0 +1,5 @@
TARGET_WS_URL= ws://10.100.3.22:10095
REDIS_URL=redis://localhost:6379/0
LLM_API_KEY=http://192.168.0.46:59800/v1
LLM_BASE_URL=http://192.168.0.46:59800/v1
LLM_MODEL=

29
fastapi_wss/Dockerfile Normal file
View File

@ -0,0 +1,29 @@
# 使用轻量级的 Python 3.10 镜像
FROM python:3.10-slim
# 设置工作目录
WORKDIR /app
# 设置环境变量
# 1. 确保 Python 输出直接打印到终端，不进行缓冲（方便查看日志）
# 2. 避免生成 pyc 文件
ENV PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1
# 复制依赖文件并安装
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt && \
apt-get update && \
apt-get install -y vim && \
rm -rf /var/lib/apt/lists/*
# 复制当前目录下的所有代码到容器中
COPY . .
WORKDIR /app/src
# 暴露端口 (对应代码中的 LOCAL_PORT)
EXPOSE 10095
# 启动 a2a_wss 服务
CMD ["python", "a2a_wss.py"]

33
fastapi_wss/requirements.txt Normal file
View File

@ -0,0 +1,33 @@
# WebSocket 智能语义分析服务 - 依赖包列表
#
# 安装方法：
# pip install -r requirements.txt
#
# Python 版本要求：3.8+
# WebSocket 通信库
websockets>=11.0
# 异步 HTTP 客户端（可选，用于扩展功能）
aiohttp>=3.8.0
# Redis 异步客户端
redis>=5.0.0
# Redis C 扩展（性能优化）
hiredis
# OpenAI API 客户端（用于调用大语言模型）
openai>=1.0.0
# 数值计算库（用于未来的特征处理）
numpy
python-dotenv
# 注意：以下为 Python 标准库，无需安装：
# - asyncio (异步编程)
# - json (JSON 处理)
# - os (操作系统接口)
# - re (正则表达式)
# - logging (日志记录)
# - typing (类型提示)
# - dataclasses (数据类)

425
fastapi_wss/src/README.md Normal file
View File

@ -0,0 +1,425 @@
# WebSocket 智能语义分析服务 - 长期记忆增强版
## 项目概述
这是一个基于 WebSocket 和 Agent-to-Agent (A2A) 架构的智能语音识别服务，具备说话人识别和长期记忆功能。系统能够在同一场会议中，通过学习每个说话人的发言特征，持续提高说话人识别的准确率。
## 核心特性
### 1. 实时语音识别处理
- 接收前端音频流，转发到 ASR 引擎
- 支持双通道模式（2pass）和 PCM 音频格式
- 实时推送识别结果到前端
### 2. 多 Agent 协作架构
- **AuditorAgent**: 判断说话人是否完成发言
- **ProfilerAgent**: 识别说话人身份（支持长期记忆）
### 3. 长期记忆功能 ⭐
- 为每个说话人建立特征档案
- 持续学习说话人的发言风格和用词习惯
- 利用会议历史上下文提高识别准确率
### 4. 完善的日志系统 📝
- 自动创建日志目录，分级记录运行信息
- 控制台输出：INFO 级别（运行状态）
- 完整日志文件：DEBUG 级别（详细调试信息）
- 错误日志文件：ERROR 级别（错误和异常）
- 所有日志包含时间戳和上下文信息
---
## 日志系统说明
### 日志文件位置
程序运行时会在项目根目录下创建 `logs` 文件夹，包含以下日志文件：
```
logs/
├── a2a_wss.log # 完整日志（DEBUG 级别）
└── a2a_wss_error.log # 错误日志（ERROR 级别）
```
### 日志级别说明
| 级别 | 说明 | 使用场景 | 输出位置 |
|------|------|----------|----------|
| **DEBUG** | 调试信息 | 详细的程序执行流程、变量值等 | 仅日志文件 |
| **INFO** | 一般信息 | 关键业务流程、状态变更 | 控制台 + 日志文件 |
| **ERROR** | 错误信息 | 异常、错误堆栈 | 控制台 + 日志文件 + 错误日志 |
### 日志内容示例
```
2025-01-07 14:30:15 - root - INFO - 启动 WebSocket 智能语义分析服务: ws://0.0.0.0:10095
2025-01-07 14:30:15 - root - INFO - Agent 模型: Qwen3-32B
2025-01-07 14:30:15 - root - INFO - ASR 服务地址: ws://localhost:59805/ws/asr
2025-01-07 14:30:15 - root - INFO - Redis 地址: redis://localhost:6379/0
2025-01-07 14:30:20 - root - INFO - 客户端已连接
2025-01-07 14:30:20 - root - INFO - 已连接到 ASR 服务: ws://localhost:59805/ws/asr
2025-01-07 14:30:22 - root - DEBUG - 收到 ASR 识别结果: 各位好，今天我们召开月度工作总结会议...
2025-01-07 14:30:23 - root - INFO - [Auditor] 发言完成检测通过 - 会话ID: room_101
2025-01-07 14:30:23 - root - DEBUG - 获取说话人特征库 - 会话ID: room_101, 说话人数量: 0
2025-01-07 14:30:24 - root - INFO - [Profiler] 识别说话人: 主持人 - 会话ID: room_101
```
### 日志配置
日志系统在代码中自动配置（无需手动配置），包含：
1. **日志格式**：`时间 - 记录器名称 - 级别 - 消息`
2. **编码**：UTF-8（支持中文）
3. **自动创建目录**：首次运行时自动创建 `logs` 文件夹
4. **错误追踪**：ERROR 级别日志包含完整的异常堆栈信息
### 日志查看建议
- **实时监控**：`tail -f logs/a2a_wss.log`
- **查看错误**：`cat logs/a2a_wss_error.log`
- **搜索关键词**：`grep "识别说话人" logs/a2a_wss.log`
- **统计分析**：`grep "ERROR" logs/a2a_wss_error.log | wc -l`
---
## 长期记忆功能详解
### 功能说明
长期记忆功能使得系统能够在同一场会议中，记住每个说话人的发言特征，并利用这些历史信息来辅助后续的说话人识别。
### 工作流程
```
┌─────────────┐
│ 开始会议 │
└──────┬──────┘
│
▼
┌─────────────────────┐
│ 接收语音片段 │
└──────┬──────────────┘
│
▼
┌─────────────────────┐
│ 累积到缓冲区 │
└──────┬──────────────┘
│
▼
┌─────────────────────┐
│ Auditor: 判断发言 │
│ 是否完成 │
└──────┬──────────────┘
│
▼ (发言完成)
┌──────────────────────────────┐
│ 获取长期记忆 │
│ (所有说话人的历史特征) │
└──────┬───────────────────────┘
│
▼
┌──────────────────────────────┐
│ Profiler: 识别说话人 │
│ - 分析会议历史记录 │
│ - 对比说话人特征库 │
│ - 判断对话流程和上下文 │
└──────┬───────────────────────┘
│
▼
┌──────────────────────────────┐
│ 更新长期记忆 │
│ (保存当前发言到特征库) │
└──────┬───────────────────────┘
│
▼
┌─────────────────────┐
│ 推送结果到前端 │
└─────────────────────┘
```
### 核心改进
#### 1. Redis 存储扩展
**新增方法**：
- `save_speaker_profile(sid, speaker, text, features)`: 保存说话人特征到长期记忆
- 保留每个说话人最近 10 条发言记录
- 存储发言文本片段（前 200 字符）
- 记录时间戳和可选的特征向量
- `get_speaker_profiles(sid)`: 获取所有说话人的历史特征
- 返回字典格式：`{说话人: [历史记录列表]}`
- 用于 Agent 分析时参考
#### 2. ProfilerAgent 增强
**改进的识别逻辑**：
- **历史记录分析**：查看最近 15 条对话，理解会议上下文
- **对话流程分析**：判断当前发言是对之前谁的话题的回应
- **说话风格匹配**：对比每个角色的用词习惯、句式结构、语气
- **角色职责判断**：根据发言内容是否符合角色职责范围
- **话题连贯性**：当前发言与该角色历史话题的连贯性
**提示词优化**：
```python
# 增强版系统提示词包含4个分析维度：
1. 对话流程分析
2. 说话风格匹配
3. 角色职责判断
4. 话题连贯性
```
#### 3. A2AOrchestrator 集成
在 `process_asr_fragment()` 方法中集成长期记忆：
```python
# 1. 获取长期记忆（说话人特征库）
speaker_profiles = await self.redis.get_speaker_profiles(session_id)
# 2. 使用长期记忆识别说话人
speaker = await self.profiler.identify(buf, history, speaker_profiles)
# 3. 保存到长期记忆（持续学习）
await self.redis.save_speaker_profile(session_id, speaker, buf)
```
---
## 配置说明
### 环境变量
| 变量名 | 说明 | 默认值 |
|--------|------|--------|
| `TARGET_WS_URL` | ASR 服务的 WebSocket 地址 | `ws://localhost:59805/ws/asr` |
| `REDIS_URL` | Redis 连接地址 | `redis://localhost:6379/0` |
| `LLM_API_KEY` | 大语言模型 API Key | `sk-xxxx` |
| `LLM_BASE_URL` | 大语言模型服务地址 | `http://172.18.127.124:9997/v1` |
| `LLM_MODEL` | 使用的模型名称 | `Qwen3-32B` |
### 候选说话人列表
在代码中配置（`CANDIDATE_SPEAKERS`）：
```python
CANDIDATE_SPEAKERS = [
"局长",
"主持人",
"商务专员",
"市场专员",
"控制要素席",
"空中侦察席"
]
```
---
## Redis 数据结构
### 键命名规范
```
meeting:buffer:{session_id} # 会话文本缓冲
meeting:history:{session_id} # 会话历史记录
meeting:speaker_profile:{session_id}:{speaker} # 说话人特征库
```
### 数据类型
- **缓冲区**: String (累积的未完成发言)
- **历史记录**: List (最多 20 条)
- **说话人特征**: List (每个说话人最多 10 条)
---
## 使用示例
### 安装依赖
首先确保已安装 Python 3.8+，然后安装所需依赖：
```bash
# 在项目根目录下执行
pip install -r requirements.txt
```
### 启动服务
```bash
# 在 src 目录下执行
python a2a_wss.py
```
服务将监听 `ws://0.0.0.0:10095`
### 连接和消息格式
**前端发送配置**：
```json
{
"mode": "2pass",
"chunk_size": [10, 10, 10],
"wav_format": "pcm"
}
```
**前端发送音频**: 二进制 PCM 数据
**后端返回结果**：
```json
{
"mode": "offline-speaker",
"type": "asr_with_speaker",
"segments": [{
"id": 0,
"speaker": "局长",
"embedding": null,
"seek": 0,
"full_text": "今天我们讨论一下市场推广方案"
}],
"speaker": "局长"
}
```
---
## 长期记忆的工作示例
### 会议初期（无记忆）
```
历史记录: [暂无]
说话人特征库: [暂无]
当前发言: "各位好，今天我们召开月度工作总结会议"
识别结果: 主持人 (基于内容特征)
```
### 会议中期（有部分记忆）
```
历史记录:
[1] 主持人: 各位好，今天我们召开月度工作总结会议
[2] 局长: 本月工作总体进展顺利
[3] 商务专员: 我们完成了三个重要合同的签署
[4] 市场专员: 市场推广活动覆盖了五个新城市
说话人特征库:
- 【主持人】(发言1次): 各位好，今天我们召开月度工作总结会议
- 【局长】(发言1次): 本月工作总体进展顺利
- 【商务专员】(发言1次): 我们完成了三个重要合同的签署
- 【市场专员】(发言1次): 市场推广活动覆盖了五个新城市
当前发言: "关于下个月的计划，我们需要重点关注客户反馈"
识别结果: 市场专员
分析:
- 延续了市场专员之前的话题（市场推广）
- 符合市场专员的职责范围（客户反馈）
- 用词风格与历史记录匹配
```
### 会议后期（丰富记忆）
```
历史记录: [15条完整对话记录]
说话人特征库:
- 【局长】(发言4次): 本月工作总体进展顺利; 很好，各部门配合默契;
这次会议很有成效; 最后强调一下安全问题
- 【主持人】(发言3次): 各位好，今天我们召开月度工作总结会议;
下面请商务专员汇报; 时间关系，今天的会议就到这里
- 【商务专员】(发言5次): 我们完成了三个重要合同的签署;
这三个合同总金额达500万; 客户对我们的方案很满意;
下季度重点跟进大客户; 需要技术部门的支持
...
当前发言: "技术部门会全力支持商务工作，确保项目交付质量"
识别结果: (未出现在候选列表中，或需要进一步判断)
分析:
- 提到了"技术部门"，可能是技术相关角色
- 对之前商务专员的话题做出回应
- 风式偏向技术人员的特点（强调"质量"、"交付"）
```
---
## 性能优化
### 1. 数据限制
- 历史记录：最多保留 20 条
- 每个说话人特征：最多保留 10 条
- LLM 上下文：使用最近 15 条历史记录
### 2. 异步处理
- Agent 处理使用后台任务，不阻塞主循环
- 连接关闭时等待任务完成（最多 5 秒）
### 3. Redis 连接池
- 使用连接池提高并发性能
- 自动编码/解码 JSON 数据
---
## 技术栈
- **Python 3.8+**
- **websockets**: WebSocket 通信
- **redis**: 会话状态和长期记忆存储
- **openai**: 大语言模型调用
- **logging**: 日志记录和系统监控
- **asyncio**: 异步编程
---
## 代码结构
```
a2a_wss.py
├── 基础配置 (1-48行)
├── RedisManager 类 (52-199行)
│ ├── 会话缓冲管理
│ ├── 历史记录管理
│ └── 说话人特征库管理 ⭐新增
├── Agent 基类 (154-208行)
├── AuditorAgent 类 (211-235行)
├── ProfilerAgent 类 (238-369行) ⭐增强
│ ├── 长期记忆提示词
│ ├── 会议历史分析
│ └── 说话人特征匹配
├── A2AOrchestrator 类 (372-343行) ⭐集成
│ └── process_asr_fragment() 方法
└── WebSocket 处理 (352-450行)
```
---
## 注意事项
1. **Redis 依赖**: 确保 Redis 服务正常运行
2. **LLM 可用性**: 确保大语言模型服务可访问
3. **内存管理**: 长时间运行的会议会累积数据，建议定期清理
4. **识别准确率**: 长期记忆随时间累积，会议初期识别准确率可能较低
---
## 未来改进方向
1. **跨会议记忆**: 支持多场会议之间的说话人特征迁移
2. **特征工程**: 提取更多说话人特征（语速、停顿模式等）
3. **置信度评分**: 为每次识别添加置信度分数
4. **自适应学习**: 根据识别准确率动态调整特征权重
5. **多模态融合**: 结合声学特征提高识别准确率
---
## 作者和许可
本项目为智能语音识别系统的核心组件，采用 Agent-to-Agent 编排模式。
最后更新: 2025-01-07

769
fastapi_wss/src/a2a_wss copy.py Normal file
View File

@ -0,0 +1,769 @@
"""
WebSocket 智能语义分析服务
功能说明：
1. 作为 WebSocket 代理，接收前端音频数据并转发到 ASR 引擎
2. 实时处理 ASR 识别结果，应用多 Agent 协作（A2A）模式
3. Agent 功能：
- AuditorAgent: 判断说话人是否完成发言
- ProfilerAgent: 识别当前说话人的身份
4. 使用 Redis 存储会话缓冲和历史记录
5. 将处理后的带说话人身份的结果推送回前端
架构模式：Agent-to-Agent (A2A) 编排
"""
import asyncio
import json
import os
import websockets
import re
import logging
from pathlib import Path
from typing import List, Optional, Dict, Set
from dataclasses import dataclass
from dotenv import load_dotenv
try:
import redis.asyncio as redis
except ModuleNotFoundError:
redis = None
try:
from openai import AsyncOpenAI
except ModuleNotFoundError:
AsyncOpenAI = None
# ========================
# 1. 日志配置
# ========================
ENV_PATH = Path(__file__).resolve().parent.parent / ".env"
load_dotenv(ENV_PATH)
def get_env(name: str, default: str) -> str:
return os.getenv(name, default).strip()
def get_env_bool(name: str, default: bool = False) -> bool:
value = os.getenv(name)
if value is None:
return default
return value.strip().lower() in {"1", "true", "yes", "on"}
def setup_logging():
"""配置日志系统"""
# 创建日志目录
log_dir = "logs"
os.makedirs(log_dir, exist_ok=True)
# 配置日志格式
log_format = "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
date_format = "%Y-%m-%d %H:%M:%S"
# 配置根日志记录器
logger = logging.getLogger()
logger.setLevel(logging.INFO)
# 清除现有处理器
logger.handlers.clear()
# 控制台处理器
console_handler = logging.StreamHandler()
console_handler.setLevel(logging.INFO)
console_formatter = logging.Formatter(log_format, datefmt=date_format)
console_handler.setFormatter(console_formatter)
logger.addHandler(console_handler)
# 文件处理器（所有日志）
file_handler = logging.FileHandler(
f"{log_dir}/a2a_wss.log",
encoding="utf-8"
)
file_handler.setLevel(logging.DEBUG)
file_formatter = logging.Formatter(log_format, datefmt=date_format)
file_handler.setFormatter(file_formatter)
logger.addHandler(file_handler)
# 错误日志文件处理器
error_handler = logging.FileHandler(
f"{log_dir}/a2a_wss_error.log",
encoding="utf-8"
)
error_handler.setLevel(logging.ERROR)
error_formatter = logging.Formatter(log_format, datefmt=date_format)
error_handler.setFormatter(error_formatter)
logger.addHandler(error_handler)
return logger
# 初始化日志系统
logger = setup_logging()
# ========================
# 2. 基础配置
# ========================
# 目标 ASR 服务的 WebSocket 地址（负责语音识别）
# 注意：移除末尾的斜杠，避免路径匹配问题
TARGET_WS_URL = get_env("TARGET_WS_URL", "ws://localhost:59805/ws/asr")
# 本地服务监听地址和端口（接收前端连接）
LOCAL_HOST = get_env("LOCAL_HOST", "0.0.0.0")
LOCAL_PORT = int(get_env("LOCAL_PORT", "10095"))
# Redis 连接配置（用于存储会话状态和历史记录）
REDIS_URL = get_env("REDIS_URL", "redis://localhost:6379/0")
# LLM（大语言模型）配置（用于 Agent 推理）
LLM_API_KEY = get_env("LLM_API_KEY", "none")
LLM_BASE_URL = get_env("LLM_BASE_URL", "http://192.168.0.46:59800/v1")
LLM_MODEL = get_env("LLM_MODEL", "46-qwen3.5-35B")
USE_LLM_ROLE_IDENTIFICATION = get_env_bool("USE_LLM_ROLE_IDENTIFICATION", False)
# 候选说话人列表（用于身份识别 Agent）
CANDIDATE_SPEAKERS = ["局长", "主持人", "商务专员", "市场专员", "控制要素席", "空中侦察席"]
# ========================
# 3. Redis 管理器
# ========================
class RedisManager:
"""
Redis 会话管理器
职责：
- 管理与 Redis 的连接池
- 存储每个会话的文本缓冲（累积不完整的识别片段）
- 存储历史对话记录（用于上下文理解）
"""
def __init__(self, url):
"""
初始化 Redis 连接池
Args:
url: Redis 连接 URL
"""
if redis is None:
raise RuntimeError("redis package is not installed")
# 使用连接池提高并发性能，decode_responses=True 自动将字节转为字符串
self.pool = redis.ConnectionPool.from_url(url, decode_responses=True)
# 键前缀，用于命名隔离，避免与其他业务冲突
self.prefix = "meeting:"
async def _get_conn(self):
"""从连接池获取一个 Redis 连接"""
return redis.Redis(connection_pool=self.pool)
async def get_buffer(self, sid):
"""
获取指定会话的文本缓冲
Args:
sid: 会话 ID（Session ID）
Returns:
该会话当前累积的文本片段
"""
r = await self._get_conn()
val = await r.get(f"{self.prefix}buffer:{sid}") or ""
return val
async def append_buffer(self, sid, text):
"""
向会话缓冲追加新的文本片段
Args:
sid: 会话 ID
text: 新识别的文本片段
"""
r = await self._get_conn()
# 追加文本并添加空格分隔
await r.append(f"{self.prefix}buffer:{sid}", text + " ")
async def clear_buffer(self, sid):
"""
清空指定会话的缓冲区
用途：当判断发言完成后，清空缓冲准备下一次发言
Args:
sid: 会话 ID
"""
r = await self._get_conn()
await r.delete(f"{self.prefix}buffer:{sid}")
async def push_history(self, sid, role, content):
"""
将完成的对话推入历史记录
Args:
sid: 会话 ID
role: 说话人角色
content: 对话内容
"""
r = await self._get_conn()
key = f"{self.prefix}history:{sid}"
# 将对话记录推入列表右端
await r.rpush(key, json.dumps({"role": role, "content": content}))
# 只保留最近 20 条记录，避免历史过长影响性能
await r.ltrim(key, -20, -1)
async def get_history(self, sid):
"""
获取会话的历史对话记录
Args:
sid: 会话 ID
Returns:
历史对话列表，每项包含 role 和 content
"""
r = await self._get_conn()
items = await r.lrange(f"{self.prefix}history:{sid}", 0, -1)
return [json.loads(i) for i in items]
async def save_speaker_profile(self, sid, speaker, text, features=None):
"""
保存说话人的特征到长期记忆
Args:
sid: 会话 ID
speaker: 说话人名称
text: 发言文本
features: 可选的特征字典（如关键词、语速等）
"""
r = await self._get_conn()
key = f"{self.prefix}speaker_profile:{sid}:{speaker}"
# 构造特征记录
profile_entry = {
"text": text[:200], # 只保留前200个字符作为示例
"timestamp": asyncio.get_event_loop().time(),
"features": features or {}
}
# 将记录推入该说话人的历史列表
await r.rpush(key, json.dumps(profile_entry))
# 只保留最近10条该说话人的记录
await r.ltrim(key, -10, -1)
async def get_speaker_profiles(self, sid):
"""
获取所有说话人的历史特征记录
Args:
sid: 会话 ID
Returns:
字典，key为说话人名称，value为该说话人的历史记录列表
"""
r = await self._get_conn()
# 扫描所有说话人的profile键
pattern = f"{self.prefix}speaker_profile:{sid}:*"
profiles = {}
async for key in r.scan_iter(match=pattern):
# 从键名中提取说话人名称
speaker = key.decode() if isinstance(key, bytes) else key
speaker = speaker.split(":")[-1]
# 获取该说话人的所有历史记录
items = await r.lrange(key, 0, -1)
profiles[speaker] = [json.loads(i) for i in items]
return profiles
async def close(self):
"""关闭连接池，释放资源（修复 DeprecationWarning）"""
await self.pool.disconnect()
# ========================
# 3. Agent 实现
# ========================
class BaseAgent:
"""
Agent 基类
提供所有 Agent 的通用能力：
- 调用 LLM 进行推理
- 提取 LLM 返回的 XML 标签结果
"""
def __init__(self, client: AsyncOpenAI):
"""
初始化 Agent
Args:
client: OpenAI 异步客户端
"""
self.client = client
async def _call_llm(self, system_prompt: str, user_content: str) -> str:
"""
调用大语言模型进行推理
Args:
system_prompt: 系统提示词（定义 Agent 的角色和行为）
user_content: 用户输入内容
Returns:
LLM 返回的文本结果，失败时返回空字符串
"""
try:
response = await self.client.chat.completions.create(
model=LLM_MODEL,
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_content}
],
temperature=0.1, # 低温度使输出更确定、一致
)
return response.choices[0].message.content
except Exception as e:
logger.error(f"LLM调用失败: {e}", exc_info=True)
return ""
def _extract_result(self, text: str) -> str:
"""
从 LLM 返回结果中提取 <result> 标签内容
Args:
text: LLM 返回的完整文本
Returns:
提取的结果，如果没有标签则返回原文
"""
match = re.search(r'<result>(.*?)</result>', text, re.DOTALL | re.IGNORECASE)
return match.group(1).strip() if match else text.strip()
class AuditorAgent(BaseAgent):
"""
审计 Agent：判断当前发言是否完成
功能：
- 分析累积的文本缓冲
- 判断说话人是否已经完成一次完整的发言
- 判断依据：语义完整性、停顿暗示等
"""
# 系统提示词：定义 Agent 的角色和输出格式
SYSTEM_PROMPT = "你是一个对话状态监测专家。请判断发言是否结束，放入 <result>true/false</result> 中。"
async def check(self, text: str) -> bool:
"""
判断给定文本是否代表一次完整的发言
Args:
text: 当前累积的文本片段
Returns:
True 表示发言完成，False 表示发言未完成（继续累积）
"""
res = await self._call_llm(self.SYSTEM_PROMPT, f"内容：{text}")
return "true" in self._extract_result(res).lower()
class ProfilerAgent(BaseAgent):
"""
分析 Agent：识别说话人身份
功能：
- 基于文本内容和说话风格识别说话人
- 利用历史对话上下文提高识别准确率
- 利用长期记忆（说话人特征库）增强识别
- 从预定义的候选说话人列表中选择
"""
# 系统提示词：定义识别任务和候选说话人列表
SYSTEM_PROMPT = f"你是一个会议身份识别专家。请从 {CANDIDATE_SPEAKERS} 中选择发言人，放入 <result>名称</result> 中。"
# 增强版提示词：结合长期记忆
SYSTEM_PROMPT_WITH_MEMORY = f"""你是一个会议身份识别专家。请根据以下信息识别发言人：
候选说话人列表：{CANDIDATE_SPEAKERS}
=== 会议历史记录（按时间顺序）===
{{history}}
=== 说话人特征库（长期记忆）===
{{speaker_profiles}}
=== 当前待识别的发言 ===
{{current_text}}
=== 识别指南 ===
请仔细分析并综合判断：
1. **对话流程分析**：
- 查看历史记录中的对话顺序和上下文
- 判断当前发言是对之前谁的话题的回应、延续或补充
- 注意对话的逻辑关系和互动模式
2. **说话风格匹配**：
- 每个角色的用词习惯、句式结构、语气的特点
- 专有名词、技术术语的使用习惯
- 提问方式、陈述方式、指令方式的差异
3. **角色职责判断**：
- 根据各角色的职责范围判断发言内容的合理性
- 例如：局长通常做总结和决策，主持人负责控场，专员负责具体业务汇报
4. **话题连贯性**：
- 当前发言与该角色之前发言的话题是否连贯
- 是否体现了该角色应有的专业领域关注点
将识别结果放入 <result>说话人名称</result> 中。"""
async def identify(self, text: str, history: List[Dict], speaker_profiles: Dict = None) -> str:
"""
识别当前文本的说话人身份
Args:
text: 当前发言文本
history: 历史对话记录（用于上下文理解）
speaker_profiles: 说话人特征库（长期记忆），key为说话人名称，value为历史记录列表
Returns:
识别出的说话人名称
"""
# 如果有长期记忆，使用增强版提示词
if speaker_profiles:
# 使用更多历史记录（最多15条），保持完整的会议上下文
h_str = self._format_history(history[-15:])
# 构建说话人特征库摘要
profiles_summary = self._format_profiles(speaker_profiles)
prompt = self.SYSTEM_PROMPT_WITH_MEMORY.format(
history=h_str or "暂无历史记录",
speaker_profiles=profiles_summary,
current_text=text
)
res = await self._call_llm(prompt, "")
return self._extract_result(res)
else:
# 原有的简单识别逻辑
h_str = "\n".join([f"{h['role']}: {h['content']}" for h in history[-5:]])
res = await self._call_llm(self.SYSTEM_PROMPT, f"历史：{h_str}\n当前：{text}")
return self._extract_result(res)
def _format_history(self, history: List[Dict]) -> str:
"""
格式化历史对话记录，添加序号和时间顺序
Args:
history: 历史对话列表
Returns:
格式化后的对话文本
"""
if not history:
return "暂无历史记录"
lines = []
for idx, h in enumerate(history, 1):
content_preview = h['content'][:150] # 只显示前150字符
if len(h['content']) > 150:
content_preview += "..."
lines.append(f"[{idx}] {h['role']}: {content_preview}")
return "\n".join(lines)
def _format_profiles(self, speaker_profiles: Dict) -> str:
"""
格式化说话人特征库为可读文本
Args:
speaker_profiles: 说话人特征字典
Returns:
格式化后的文本摘要
"""
if not speaker_profiles:
return "暂无说话人特征记录"
summary_lines = []
for speaker, profiles in speaker_profiles.items():
# 提取该说话人最近5条发言的文本片段
recent_texts = [p['text'] for p in profiles[-5:]]
texts_str = "; ".join(recent_texts)
# 统计该说话人的发言次数
speaking_count = len(profiles)
summary_lines.append(f"- 【{speaker}】(发言{speaking_count}次): {texts_str}")
return "\n".join(summary_lines)
# ========================
# 4. A2A 编排器
# ========================
class A2AOrchestrator:
"""
Agent-to-Agent 编排器
职责：
- 协调多个 Agent 的协作流程
- 处理 ASR 识别结果，依次调用 Auditor 和 Profiler
- 管理会话状态（缓冲和历史记录）
- 将最终结果推送给前端
"""
def __init__(self, client: AsyncOpenAI, redis_mgr: RedisManager):
"""
初始化编排器
Args:
client: OpenAI 异步客户端
redis_mgr: Redis 管理器实例
"""
self.redis = redis_mgr
self.auditor = AuditorAgent(client) # 审计 Agent
self.profiler = ProfilerAgent(client) # 分析 Agent
async def process_asr_fragment(self, session_id: str, fragment: str, websocket):
"""
处理 ASR 识别的文本片段（核心流程）
流程：
1. 将新片段追加到缓冲区
2. 让 Auditor 判断发言是否完成
3. 如果完成，让 Profiler 使用长期记忆识别说话人
4. 将识别结果保存到长期记忆
5. 将结果存入历史，清空缓冲
6. 推送结果到前端
Args:
session_id: 会话 ID
fragment: ASR 识别的新文本片段
websocket: 前端 WebSocket 连接（用于推送结果）
"""
# 1. 追加新片段到缓冲区
await self.redis.append_buffer(session_id, fragment)
buf = await self.redis.get_buffer(session_id)
# 2. 检查发言是否完成
if await self.auditor.check(buf):
logger.info(f"[Auditor] 发言完成检测通过 - 会话ID: {session_id}")
# 3. 获取长期记忆（说话人特征库）
speaker_profiles = await self.redis.get_speaker_profiles(session_id)
logger.debug(f"获取说话人特征库 - 会话ID: {session_id}, 说话人数量: {len(speaker_profiles)}")
# 4. 识别说话人身份（使用长期记忆）
history = await self.redis.get_history(session_id)
speaker = await self.profiler.identify(buf, history, speaker_profiles)
logger.info(f"[Profiler] 识别说话人: {speaker} - 会话ID: {session_id}")
# 5. 保存到长期记忆（更新说话人特征库）
await self.redis.save_speaker_profile(session_id, speaker, buf)
# 6. 保存到历史记录并清空缓冲
await self.redis.push_history(session_id, speaker, buf)
await self.redis.clear_buffer(session_id)
# 7. 构造结果消息
segments = [{
"id": 0,
"speaker": speaker,
"embedding": None, # 占位（预留字段）
"seek": 0,
"full_text": buf.strip()
}]
msg = {
"mode": "offline-speaker",
"type": "asr_with_speaker",
"segments": segments,
"speaker": speaker
}
# 推送结果到前端（失败时静默处理）
try:
await websocket.send(json.dumps(msg, ensure_ascii=False))
except:
pass
def build_acoustic_speaker_message(data: Dict) -> Optional[Dict]:
text = (data.get("text") or "").strip()
speaker = (data.get("spk_name") or "").strip()
if not text or not speaker or speaker.lower() == "unknown":
return None
segment = {
"id": 0,
"speaker": speaker,
"embedding": None,
"seek": 0,
"full_text": text,
"start": None,
"end": None,
}
msg = {
"mode": "offline-speaker",
"type": "asr_with_speaker",
"source": "acoustic",
"speaker": speaker,
"spk_score": data.get("spk_score"),
"text": text,
"wav_name": data.get("wav_name"),
"is_final": data.get("is_final", True),
"segments": [segment],
}
if "timestamp" in data:
msg["timestamp"] = data["timestamp"]
if "sentence_info" in data:
msg["sentence_info"] = data["sentence_info"]
return msg
# ========================
# 5. WebSocket 处理与任务追踪
# ========================
async def forward_audio(websocket):
"""
WebSocket 连接处理函数（核心入口）
职责：
1. 接受前端 WebSocket 连接
2. 建立到 ASR 服务的下游连接
3. 双向转发消息（前端 -> ASR，ASR -> 前端）
4. 拦截 ASR 结果并触 Agent 处理
5. 确保所有后台任务完成后才退出（防止数据丢失）
Args:
websocket: 前端 WebSocket 连接对象
"""
logger.info("客户端已连接")
client = None
redis_mgr = None
orchestrator = None
use_llm_role_identification = USE_LLM_ROLE_IDENTIFICATION
if use_llm_role_identification:
if AsyncOpenAI is None:
logger.error("openai package is not installed; LLM role identification disabled")
use_llm_role_identification = False
elif redis is None:
logger.error("redis package is not installed; LLM role identification disabled")
use_llm_role_identification = False
else:
# 初始化 LLM 客户端、Redis 管理器和编排器
client = AsyncOpenAI(api_key=LLM_API_KEY, base_url=LLM_BASE_URL)
redis_mgr = RedisManager(REDIS_URL)
orchestrator = A2AOrchestrator(client, redis_mgr)
# 用于追踪后台任务，防止连接关闭时任务被强制销毁
background_tasks: Set[asyncio.Task] = set()
try:
# 连接到目标 ASR 服务
# 添加连接参数以提高鲁棒性
async with websockets.connect(
TARGET_WS_URL,
close_timeout=10, # 关闭超时
ping_timeout=20, # ping 超时
max_queue=1024, # 消息队列大小
subprotocols=["binary"],
) as target_ws:
logger.info(f"已连接到 ASR 服务: {TARGET_WS_URL}")
# 发送 ASR 配置（双通道模式，PCM 音频格式）
config = {"mode": "2pass", "chunk_size": [10, 10, 10], "wav_format": "pcm"}
await target_ws.send(json.dumps(config))
logger.debug(f"发送 ASR 配置: {config}")
async def frontend_to_target():
"""
上行转发：前端音频数据 -> ASR 服务
"""
async for message in websocket:
await target_ws.send(message)
async def target_to_frontend():
"""
下行转发：ASR 识别结果 -> 前端
同时拦截结果触发 Agent 处理
"""
async for message in target_ws:
# 1. 透传 ASR 原始结果到前端（字幕显示）
await websocket.send(message)
# 2. 如果是文本消息，尝试触发 Agent 处理
if isinstance(message, str):
try:
data = json.loads(message)
# 只处理最终识别结果（2pass-offline 模式）
if data.get("mode") == "2pass-offline":
text = data.get("text", "").strip()
speaker_msg = build_acoustic_speaker_message(data)
if speaker_msg is not None:
await websocket.send(json.dumps(speaker_msg, ensure_ascii=False))
if text and use_llm_role_identification and orchestrator is not None:
logger.debug(f"收到 ASR 识别结果: {text[:50]}...")
# 创建后台任务处理 Agent 逻辑（不阻塞主循环）
t = asyncio.create_task(
orchestrator.process_asr_fragment("room_101", text, websocket)
)
background_tasks.add(t)
# 任务完成后自动从追踪集合中移除
t.add_done_callback(background_tasks.discard)
except Exception as e:
logger.error(f"处理 ASR 结果时出错: {e}", exc_info=True)
# 并发运行两个方向的数据流
await asyncio.gather(frontend_to_target(), target_to_frontend())
except websockets.exceptions.InvalidMessage as e:
logger.error(f"WebSocket 握手失败 - ASR 服务可能未正常运行或 URL 配置错误: {e}")
logger.error(f"请检查: 1) ASR 服务是否在 {TARGET_WS_URL} 正常运行")
logger.error(f" 2) ASR 服务的 URL 路径是否正确")
logger.error(f" 3) ASR 服务是否需要特定的认证或配置")
except ConnectionRefusedError:
logger.error(f"连接被拒绝 - 无法连接到 ASR 服务 {TARGET_WS_URL}")
logger.error(f"请检查 ASR 服务是否已启动")
except Exception as e:
logger.error(f"连接错误: {type(e).__name__}: {e}", exc_info=True)
finally:
# --- 优雅退出处理 ---
if background_tasks:
logger.info(f"等待 {len(background_tasks)} 个后台 Agent 任务完成...")
# 等待所有剩余任务跑完，最多等 5 秒（防止无限等待）
await asyncio.wait(background_tasks, timeout=5)
logger.debug("所有后台任务已完成或超时")
# 释放资源
if client is not None:
await client.close()
if redis_mgr is not None:
await redis_mgr.close()
logger.info("客户端已断开连接")
async def main():
"""
主函数：启动 WebSocket 服务器
"""
logger.info(f"启动 WebSocket 智能语义分析服务: ws://{LOCAL_HOST}:{LOCAL_PORT}")
logger.info(f"Agent 模型: {LLM_MODEL}")
logger.info(f"ASR 服务地址: {TARGET_WS_URL}")
logger.info(f"Redis 地址: {REDIS_URL}")
# 启动服务器并持续运行（asyncio.Future() 永不完成）
async with websockets.serve(forward_audio, LOCAL_HOST, LOCAL_PORT):
await asyncio.Future()
if __name__ == "__main__":
try:
asyncio.run(main())
except KeyboardInterrupt:
pass

769
fastapi_wss/src/a2a_wss copy.py.bak-202607150815 Normal file
View File

@ -0,0 +1,769 @@
"""
WebSocket 智能语义分析服务
功能说明：
1. 作为 WebSocket 代理，接收前端音频数据并转发到 ASR 引擎
2. 实时处理 ASR 识别结果，应用多 Agent 协作（A2A）模式
3. Agent 功能：
- AuditorAgent: 判断说话人是否完成发言
- ProfilerAgent: 识别当前说话人的身份
4. 使用 Redis 存储会话缓冲和历史记录
5. 将处理后的带说话人身份的结果推送回前端
架构模式：Agent-to-Agent (A2A) 编排
"""
import asyncio
import json
import os
import websockets
import re
import logging
from pathlib import Path
from typing import List, Optional, Dict, Set
from dataclasses import dataclass
from dotenv import load_dotenv
try:
import redis.asyncio as redis
except ModuleNotFoundError:
redis = None
try:
from openai import AsyncOpenAI
except ModuleNotFoundError:
AsyncOpenAI = None
# ========================
# 1. 日志配置
# ========================
ENV_PATH = Path(__file__).resolve().parent.parent / ".env"
load_dotenv(ENV_PATH)
def get_env(name: str, default: str) -> str:
return os.getenv(name, default).strip()
def get_env_bool(name: str, default: bool = False) -> bool:
value = os.getenv(name)
if value is None:
return default
return value.strip().lower() in {"1", "true", "yes", "on"}
def setup_logging():
"""配置日志系统"""
# 创建日志目录
log_dir = "logs"
os.makedirs(log_dir, exist_ok=True)
# 配置日志格式
log_format = "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
date_format = "%Y-%m-%d %H:%M:%S"
# 配置根日志记录器
logger = logging.getLogger()
logger.setLevel(logging.INFO)
# 清除现有处理器
logger.handlers.clear()
# 控制台处理器
console_handler = logging.StreamHandler()
console_handler.setLevel(logging.INFO)
console_formatter = logging.Formatter(log_format, datefmt=date_format)
console_handler.setFormatter(console_formatter)
logger.addHandler(console_handler)
# 文件处理器（所有日志）
file_handler = logging.FileHandler(
f"{log_dir}/a2a_wss.log",
encoding="utf-8"
)
file_handler.setLevel(logging.DEBUG)
file_formatter = logging.Formatter(log_format, datefmt=date_format)
file_handler.setFormatter(file_formatter)
logger.addHandler(file_handler)
# 错误日志文件处理器
error_handler = logging.FileHandler(
f"{log_dir}/a2a_wss_error.log",
encoding="utf-8"
)
error_handler.setLevel(logging.ERROR)
error_formatter = logging.Formatter(log_format, datefmt=date_format)
error_handler.setFormatter(error_formatter)
logger.addHandler(error_handler)
return logger
# 初始化日志系统
logger = setup_logging()
# ========================
# 2. 基础配置
# ========================
# 目标 ASR 服务的 WebSocket 地址（负责语音识别）
# 注意：移除末尾的斜杠，避免路径匹配问题
TARGET_WS_URL = get_env("TARGET_WS_URL", "ws://localhost:10096")
# 本地服务监听地址和端口（接收前端连接）
LOCAL_HOST = get_env("LOCAL_HOST", "0.0.0.0")
LOCAL_PORT = int(get_env("LOCAL_PORT", "10095"))
# Redis 连接配置（用于存储会话状态和历史记录）
REDIS_URL = get_env("REDIS_URL", "redis://localhost:6379/0")
# LLM（大语言模型）配置（用于 Agent 推理）
LLM_API_KEY = get_env("LLM_API_KEY", "none")
LLM_BASE_URL = get_env("LLM_BASE_URL", "http://192.168.0.46:59800/v1")
LLM_MODEL = get_env("LLM_MODEL", "46-qwen3.5-35B")
USE_LLM_ROLE_IDENTIFICATION = get_env_bool("USE_LLM_ROLE_IDENTIFICATION", False)
# 候选说话人列表（用于身份识别 Agent）
CANDIDATE_SPEAKERS = ["局长", "主持人", "商务专员", "市场专员", "控制要素席", "空中侦察席"]
# ========================
# 3. Redis 管理器
# ========================
class RedisManager:
"""
Redis 会话管理器
职责：
- 管理与 Redis 的连接池
- 存储每个会话的文本缓冲（累积不完整的识别片段）
- 存储历史对话记录（用于上下文理解）
"""
def __init__(self, url):
"""
初始化 Redis 连接池
Args:
url: Redis 连接 URL
"""
if redis is None:
raise RuntimeError("redis package is not installed")
# 使用连接池提高并发性能，decode_responses=True 自动将字节转为字符串
self.pool = redis.ConnectionPool.from_url(url, decode_responses=True)
# 键前缀，用于命名隔离，避免与其他业务冲突
self.prefix = "meeting:"
async def _get_conn(self):
"""从连接池获取一个 Redis 连接"""
return redis.Redis(connection_pool=self.pool)
async def get_buffer(self, sid):
"""
获取指定会话的文本缓冲
Args:
sid: 会话 ID（Session ID）
Returns:
该会话当前累积的文本片段
"""
r = await self._get_conn()
val = await r.get(f"{self.prefix}buffer:{sid}") or ""
return val
async def append_buffer(self, sid, text):
"""
向会话缓冲追加新的文本片段
Args:
sid: 会话 ID
text: 新识别的文本片段
"""
r = await self._get_conn()
# 追加文本并添加空格分隔
await r.append(f"{self.prefix}buffer:{sid}", text + " ")
async def clear_buffer(self, sid):
"""
清空指定会话的缓冲区
用途：当判断发言完成后，清空缓冲准备下一次发言
Args:
sid: 会话 ID
"""
r = await self._get_conn()
await r.delete(f"{self.prefix}buffer:{sid}")
async def push_history(self, sid, role, content):
"""
将完成的对话推入历史记录
Args:
sid: 会话 ID
role: 说话人角色
content: 对话内容
"""
r = await self._get_conn()
key = f"{self.prefix}history:{sid}"
# 将对话记录推入列表右端
await r.rpush(key, json.dumps({"role": role, "content": content}))
# 只保留最近 20 条记录，避免历史过长影响性能
await r.ltrim(key, -20, -1)
async def get_history(self, sid):
"""
获取会话的历史对话记录
Args:
sid: 会话 ID
Returns:
历史对话列表，每项包含 role 和 content
"""
r = await self._get_conn()
items = await r.lrange(f"{self.prefix}history:{sid}", 0, -1)
return [json.loads(i) for i in items]
async def save_speaker_profile(self, sid, speaker, text, features=None):
"""
保存说话人的特征到长期记忆
Args:
sid: 会话 ID
speaker: 说话人名称
text: 发言文本
features: 可选的特征字典（如关键词、语速等）
"""
r = await self._get_conn()
key = f"{self.prefix}speaker_profile:{sid}:{speaker}"
# 构造特征记录
profile_entry = {
"text": text[:200], # 只保留前200个字符作为示例
"timestamp": asyncio.get_event_loop().time(),
"features": features or {}
}
# 将记录推入该说话人的历史列表
await r.rpush(key, json.dumps(profile_entry))
# 只保留最近10条该说话人的记录
await r.ltrim(key, -10, -1)
async def get_speaker_profiles(self, sid):
"""
获取所有说话人的历史特征记录
Args:
sid: 会话 ID
Returns:
字典，key为说话人名称，value为该说话人的历史记录列表
"""
r = await self._get_conn()
# 扫描所有说话人的profile键
pattern = f"{self.prefix}speaker_profile:{sid}:*"
profiles = {}
async for key in r.scan_iter(match=pattern):
# 从键名中提取说话人名称
speaker = key.decode() if isinstance(key, bytes) else key
speaker = speaker.split(":")[-1]
# 获取该说话人的所有历史记录
items = await r.lrange(key, 0, -1)
profiles[speaker] = [json.loads(i) for i in items]
return profiles
async def close(self):
"""关闭连接池，释放资源（修复 DeprecationWarning）"""
await self.pool.disconnect()
# ========================
# 3. Agent 实现
# ========================
class BaseAgent:
"""
Agent 基类
提供所有 Agent 的通用能力：
- 调用 LLM 进行推理
- 提取 LLM 返回的 XML 标签结果
"""
def __init__(self, client: AsyncOpenAI):
"""
初始化 Agent
Args:
client: OpenAI 异步客户端
"""
self.client = client
async def _call_llm(self, system_prompt: str, user_content: str) -> str:
"""
调用大语言模型进行推理
Args:
system_prompt: 系统提示词（定义 Agent 的角色和行为）
user_content: 用户输入内容
Returns:
LLM 返回的文本结果，失败时返回空字符串
"""
try:
response = await self.client.chat.completions.create(
model=LLM_MODEL,
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_content}
],
temperature=0.1, # 低温度使输出更确定、一致
)
return response.choices[0].message.content
except Exception as e:
logger.error(f"LLM调用失败: {e}", exc_info=True)
return ""
def _extract_result(self, text: str) -> str:
"""
从 LLM 返回结果中提取 <result> 标签内容
Args:
text: LLM 返回的完整文本
Returns:
提取的结果，如果没有标签则返回原文
"""
match = re.search(r'<result>(.*?)</result>', text, re.DOTALL | re.IGNORECASE)
return match.group(1).strip() if match else text.strip()
class AuditorAgent(BaseAgent):
"""
审计 Agent：判断当前发言是否完成
功能：
- 分析累积的文本缓冲
- 判断说话人是否已经完成一次完整的发言
- 判断依据：语义完整性、停顿暗示等
"""
# 系统提示词：定义 Agent 的角色和输出格式
SYSTEM_PROMPT = "你是一个对话状态监测专家。请判断发言是否结束，放入 <result>true/false</result> 中。"
async def check(self, text: str) -> bool:
"""
判断给定文本是否代表一次完整的发言
Args:
text: 当前累积的文本片段
Returns:
True 表示发言完成，False 表示发言未完成（继续累积）
"""
res = await self._call_llm(self.SYSTEM_PROMPT, f"内容：{text}")
return "true" in self._extract_result(res).lower()
class ProfilerAgent(BaseAgent):
"""
分析 Agent：识别说话人身份
功能：
- 基于文本内容和说话风格识别说话人
- 利用历史对话上下文提高识别准确率
- 利用长期记忆（说话人特征库）增强识别
- 从预定义的候选说话人列表中选择
"""
# 系统提示词：定义识别任务和候选说话人列表
SYSTEM_PROMPT = f"你是一个会议身份识别专家。请从 {CANDIDATE_SPEAKERS} 中选择发言人，放入 <result>名称</result> 中。"
# 增强版提示词：结合长期记忆
SYSTEM_PROMPT_WITH_MEMORY = f"""你是一个会议身份识别专家。请根据以下信息识别发言人：
候选说话人列表：{CANDIDATE_SPEAKERS}
=== 会议历史记录（按时间顺序）===
{{history}}
=== 说话人特征库（长期记忆）===
{{speaker_profiles}}
=== 当前待识别的发言 ===
{{current_text}}
=== 识别指南 ===
请仔细分析并综合判断：
1. **对话流程分析**：
- 查看历史记录中的对话顺序和上下文
- 判断当前发言是对之前谁的话题的回应、延续或补充
- 注意对话的逻辑关系和互动模式
2. **说话风格匹配**：
- 每个角色的用词习惯、句式结构、语气的特点
- 专有名词、技术术语的使用习惯
- 提问方式、陈述方式、指令方式的差异
3. **角色职责判断**：
- 根据各角色的职责范围判断发言内容的合理性
- 例如：局长通常做总结和决策，主持人负责控场，专员负责具体业务汇报
4. **话题连贯性**：
- 当前发言与该角色之前发言的话题是否连贯
- 是否体现了该角色应有的专业领域关注点
将识别结果放入 <result>说话人名称</result> 中。"""
async def identify(self, text: str, history: List[Dict], speaker_profiles: Dict = None) -> str:
"""
识别当前文本的说话人身份
Args:
text: 当前发言文本
history: 历史对话记录（用于上下文理解）
speaker_profiles: 说话人特征库（长期记忆），key为说话人名称，value为历史记录列表
Returns:
识别出的说话人名称
"""
# 如果有长期记忆，使用增强版提示词
if speaker_profiles:
# 使用更多历史记录（最多15条），保持完整的会议上下文
h_str = self._format_history(history[-15:])
# 构建说话人特征库摘要
profiles_summary = self._format_profiles(speaker_profiles)
prompt = self.SYSTEM_PROMPT_WITH_MEMORY.format(
history=h_str or "暂无历史记录",
speaker_profiles=profiles_summary,
current_text=text
)
res = await self._call_llm(prompt, "")
return self._extract_result(res)
else:
# 原有的简单识别逻辑
h_str = "\n".join([f"{h['role']}: {h['content']}" for h in history[-5:]])
res = await self._call_llm(self.SYSTEM_PROMPT, f"历史：{h_str}\n当前：{text}")
return self._extract_result(res)
def _format_history(self, history: List[Dict]) -> str:
"""
格式化历史对话记录，添加序号和时间顺序
Args:
history: 历史对话列表
Returns:
格式化后的对话文本
"""
if not history:
return "暂无历史记录"
lines = []
for idx, h in enumerate(history, 1):
content_preview = h['content'][:150] # 只显示前150字符
if len(h['content']) > 150:
content_preview += "..."
lines.append(f"[{idx}] {h['role']}: {content_preview}")
return "\n".join(lines)
def _format_profiles(self, speaker_profiles: Dict) -> str:
"""
格式化说话人特征库为可读文本
Args:
speaker_profiles: 说话人特征字典
Returns:
格式化后的文本摘要
"""
if not speaker_profiles:
return "暂无说话人特征记录"
summary_lines = []
for speaker, profiles in speaker_profiles.items():
# 提取该说话人最近5条发言的文本片段
recent_texts = [p['text'] for p in profiles[-5:]]
texts_str = "; ".join(recent_texts)
# 统计该说话人的发言次数
speaking_count = len(profiles)
summary_lines.append(f"- 【{speaker}】(发言{speaking_count}次): {texts_str}")
return "\n".join(summary_lines)
# ========================
# 4. A2A 编排器
# ========================
class A2AOrchestrator:
"""
Agent-to-Agent 编排器
职责：
- 协调多个 Agent 的协作流程
- 处理 ASR 识别结果，依次调用 Auditor 和 Profiler
- 管理会话状态（缓冲和历史记录）
- 将最终结果推送给前端
"""
def __init__(self, client: AsyncOpenAI, redis_mgr: RedisManager):
"""
初始化编排器
Args:
client: OpenAI 异步客户端
redis_mgr: Redis 管理器实例
"""
self.redis = redis_mgr
self.auditor = AuditorAgent(client) # 审计 Agent
self.profiler = ProfilerAgent(client) # 分析 Agent
async def process_asr_fragment(self, session_id: str, fragment: str, websocket):
"""
处理 ASR 识别的文本片段（核心流程）
流程：
1. 将新片段追加到缓冲区
2. 让 Auditor 判断发言是否完成
3. 如果完成，让 Profiler 使用长期记忆识别说话人
4. 将识别结果保存到长期记忆
5. 将结果存入历史，清空缓冲
6. 推送结果到前端
Args:
session_id: 会话 ID
fragment: ASR 识别的新文本片段
websocket: 前端 WebSocket 连接（用于推送结果）
"""
# 1. 追加新片段到缓冲区
await self.redis.append_buffer(session_id, fragment)
buf = await self.redis.get_buffer(session_id)
# 2. 检查发言是否完成
if await self.auditor.check(buf):
logger.info(f"[Auditor] 发言完成检测通过 - 会话ID: {session_id}")
# 3. 获取长期记忆（说话人特征库）
speaker_profiles = await self.redis.get_speaker_profiles(session_id)
logger.debug(f"获取说话人特征库 - 会话ID: {session_id}, 说话人数量: {len(speaker_profiles)}")
# 4. 识别说话人身份（使用长期记忆）
history = await self.redis.get_history(session_id)
speaker = await self.profiler.identify(buf, history, speaker_profiles)
logger.info(f"[Profiler] 识别说话人: {speaker} - 会话ID: {session_id}")
# 5. 保存到长期记忆（更新说话人特征库）
await self.redis.save_speaker_profile(session_id, speaker, buf)
# 6. 保存到历史记录并清空缓冲
await self.redis.push_history(session_id, speaker, buf)
await self.redis.clear_buffer(session_id)
# 7. 构造结果消息
segments = [{
"id": 0,
"speaker": speaker,
"embedding": None, # 占位（预留字段）
"seek": 0,
"full_text": buf.strip()
}]
msg = {
"mode": "offline-speaker",
"type": "asr_with_speaker",
"segments": segments,
"speaker": speaker
}
# 推送结果到前端（失败时静默处理）
try:
await websocket.send(json.dumps(msg, ensure_ascii=False))
except:
pass
def build_acoustic_speaker_message(data: Dict) -> Optional[Dict]:
text = (data.get("text") or "").strip()
speaker = (data.get("spk_name") or "").strip()
if not text or not speaker or speaker.lower() == "unknown":
return None
segment = {
"id": 0,
"speaker": speaker,
"embedding": None,
"seek": 0,
"full_text": text,
"start": None,
"end": None,
}
msg = {
"mode": "offline-speaker",
"type": "asr_with_speaker",
"source": "acoustic",
"speaker": speaker,
"spk_score": data.get("spk_score"),
"text": text,
"wav_name": data.get("wav_name"),
"is_final": data.get("is_final", True),
"segments": [segment],
}
if "timestamp" in data:
msg["timestamp"] = data["timestamp"]
if "sentence_info" in data:
msg["sentence_info"] = data["sentence_info"]
return msg
# ========================
# 5. WebSocket 处理与任务追踪
# ========================
async def forward_audio(websocket):
"""
WebSocket 连接处理函数（核心入口）
职责：
1. 接受前端 WebSocket 连接
2. 建立到 ASR 服务的下游连接
3. 双向转发消息（前端 -> ASR，ASR -> 前端）
4. 拦截 ASR 结果并触 Agent 处理
5. 确保所有后台任务完成后才退出（防止数据丢失）
Args:
websocket: 前端 WebSocket 连接对象
"""
logger.info("客户端已连接")
client = None
redis_mgr = None
orchestrator = None
use_llm_role_identification = USE_LLM_ROLE_IDENTIFICATION
if use_llm_role_identification:
if AsyncOpenAI is None:
logger.error("openai package is not installed; LLM role identification disabled")
use_llm_role_identification = False
elif redis is None:
logger.error("redis package is not installed; LLM role identification disabled")
use_llm_role_identification = False
else:
# 初始化 LLM 客户端、Redis 管理器和编排器
client = AsyncOpenAI(api_key=LLM_API_KEY, base_url=LLM_BASE_URL)
redis_mgr = RedisManager(REDIS_URL)
orchestrator = A2AOrchestrator(client, redis_mgr)
# 用于追踪后台任务，防止连接关闭时任务被强制销毁
background_tasks: Set[asyncio.Task] = set()
try:
# 连接到目标 ASR 服务
# 添加连接参数以提高鲁棒性
async with websockets.connect(
TARGET_WS_URL,
close_timeout=10, # 关闭超时
ping_timeout=20, # ping 超时
max_queue=1024, # 消息队列大小
subprotocols=["binary"],
) as target_ws:
logger.info(f"已连接到 ASR 服务: {TARGET_WS_URL}")
# 发送 ASR 配置（双通道模式，PCM 音频格式）
config = {"mode": "2pass", "chunk_size": [10, 10, 10], "wav_format": "pcm"}
await target_ws.send(json.dumps(config))
logger.debug(f"发送 ASR 配置: {config}")
async def frontend_to_target():
"""
上行转发：前端音频数据 -> ASR 服务
"""
async for message in websocket:
await target_ws.send(message)
async def target_to_frontend():
"""
下行转发：ASR 识别结果 -> 前端
同时拦截结果触发 Agent 处理
"""
async for message in target_ws:
# 1. 透传 ASR 原始结果到前端（字幕显示）
await websocket.send(message)
# 2. 如果是文本消息，尝试触发 Agent 处理
if isinstance(message, str):
try:
data = json.loads(message)
# 只处理最终识别结果（2pass-offline 模式）
if data.get("mode") == "2pass-offline":
text = data.get("text", "").strip()
speaker_msg = build_acoustic_speaker_message(data)
if speaker_msg is not None:
await websocket.send(json.dumps(speaker_msg, ensure_ascii=False))
if text and use_llm_role_identification and orchestrator is not None:
logger.debug(f"收到 ASR 识别结果: {text[:50]}...")
# 创建后台任务处理 Agent 逻辑（不阻塞主循环）
t = asyncio.create_task(
orchestrator.process_asr_fragment("room_101", text, websocket)
)
background_tasks.add(t)
# 任务完成后自动从追踪集合中移除
t.add_done_callback(background_tasks.discard)
except Exception as e:
logger.error(f"处理 ASR 结果时出错: {e}", exc_info=True)
# 并发运行两个方向的数据流
await asyncio.gather(frontend_to_target(), target_to_frontend())
except websockets.exceptions.InvalidMessage as e:
logger.error(f"WebSocket 握手失败 - ASR 服务可能未正常运行或 URL 配置错误: {e}")
logger.error(f"请检查: 1) ASR 服务是否在 {TARGET_WS_URL} 正常运行")
logger.error(f" 2) ASR 服务的 URL 路径是否正确")
logger.error(f" 3) ASR 服务是否需要特定的认证或配置")
except ConnectionRefusedError:
logger.error(f"连接被拒绝 - 无法连接到 ASR 服务 {TARGET_WS_URL}")
logger.error(f"请检查 ASR 服务是否已启动")
except Exception as e:
logger.error(f"连接错误: {type(e).__name__}: {e}", exc_info=True)
finally:
# --- 优雅退出处理 ---
if background_tasks:
logger.info(f"等待 {len(background_tasks)} 个后台 Agent 任务完成...")
# 等待所有剩余任务跑完，最多等 5 秒（防止无限等待）
await asyncio.wait(background_tasks, timeout=5)
logger.debug("所有后台任务已完成或超时")
# 释放资源
if client is not None:
await client.close()
if redis_mgr is not None:
await redis_mgr.close()
logger.info("客户端已断开连接")
async def main():
"""
主函数：启动 WebSocket 服务器
"""
logger.info(f"启动 WebSocket 智能语义分析服务: ws://{LOCAL_HOST}:{LOCAL_PORT}")
logger.info(f"Agent 模型: {LLM_MODEL}")
logger.info(f"ASR 服务地址: {TARGET_WS_URL}")
logger.info(f"Redis 地址: {REDIS_URL}")
# 启动服务器并持续运行（asyncio.Future() 永不完成）
async with websockets.serve(forward_audio, LOCAL_HOST, LOCAL_PORT):
await asyncio.Future()
if __name__ == "__main__":
try:
asyncio.run(main())
except KeyboardInterrupt:
pass

1257
fastapi_wss/src/a2a_wss.py Normal file
View File

File diff suppressed because it is too large Load Diff

958
fastapi_wss/src/a2a_wss.py.bak-202607150815 Normal file
View File

@ -0,0 +1,958 @@
"""
WebSocket 智能语义分析服务
功能说明：
1. 作为 WebSocket 代理，接收前端音频数据并转发到 ASR 引擎
2. 实时处理 ASR 识别结果，应用多 Agent 协作（A2A）模式
3. Agent 功能：
- AuditorAgent: 判断说话人是否完成发言
- ProfilerAgent: 识别当前说话人的身份
4. 使用 Redis 存储会话缓冲和历史记录
5. 将处理后的带说话人身份的结果推送回前端
架构模式：Agent-to-Agent (A2A) 编排
"""
import asyncio
import json
import os
import websockets
import re
import logging
from pathlib import Path
from typing import List, Optional, Dict, Set
from dataclasses import dataclass
from dotenv import load_dotenv
try:
import redis.asyncio as redis
except ModuleNotFoundError:
redis = None
try:
from openai import AsyncOpenAI
except ModuleNotFoundError:
AsyncOpenAI = None
# ========================
# 1. 日志配置
# ========================
ENV_PATH = Path(__file__).resolve().parent.parent / ".env"
load_dotenv(ENV_PATH)
def get_env(name: str, default: str) -> str:
return os.getenv(name, default).strip()
def get_env_bool(name: str, default: bool = False) -> bool:
value = os.getenv(name)
if value is None:
return default
return value.strip().lower() in {"1", "true", "yes", "on"}
def get_env_list(name: str, default: str) -> List[str]:
raw = get_env(name, default)
return [item.strip() for item in raw.split(",") if item.strip()]
def setup_logging():
"""配置日志系统"""
# 创建日志目录
log_dir = "logs"
os.makedirs(log_dir, exist_ok=True)
# 配置日志格式
log_format = "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
date_format = "%Y-%m-%d %H:%M:%S"
# 配置根日志记录器
logger = logging.getLogger()
logger.setLevel(logging.INFO)
# 清除现有处理器
logger.handlers.clear()
# 控制台处理器
console_handler = logging.StreamHandler()
console_handler.setLevel(logging.INFO)
console_formatter = logging.Formatter(log_format, datefmt=date_format)
console_handler.setFormatter(console_formatter)
logger.addHandler(console_handler)
# 文件处理器（所有日志）
file_handler = logging.FileHandler(
f"{log_dir}/a2a_wss.log",
encoding="utf-8"
)
file_handler.setLevel(logging.DEBUG)
file_formatter = logging.Formatter(log_format, datefmt=date_format)
file_handler.setFormatter(file_formatter)
logger.addHandler(file_handler)
# 错误日志文件处理器
error_handler = logging.FileHandler(
f"{log_dir}/a2a_wss_error.log",
encoding="utf-8"
)
error_handler.setLevel(logging.ERROR)
error_formatter = logging.Formatter(log_format, datefmt=date_format)
error_handler.setFormatter(error_formatter)
logger.addHandler(error_handler)
return logger
# 初始化日志系统
logger = setup_logging()
# ========================
# 2. 基础配置
# ========================
# 目标 ASR 服务的 WebSocket 地址（负责语音识别）
# 注意：移除末尾的斜杠，避免路径匹配问题
TARGET_WS_URL = get_env("TARGET_WS_URL", "ws://localhost:10096")
# 本地服务监听地址和端口（接收前端连接）
LOCAL_HOST = get_env("LOCAL_HOST", "0.0.0.0")
LOCAL_PORT = int(get_env("LOCAL_PORT", "10095"))
# Redis 连接配置（用于存储会话状态和历史记录）
REDIS_URL = get_env("REDIS_URL", "redis://localhost:6379/0")
# LLM（大语言模型）配置（用于 Agent 推理）
LLM_API_KEY = get_env("LLM_API_KEY", "none")
LLM_BASE_URL = get_env("LLM_BASE_URL", "http://192.168.0.46:59800/v1")
LLM_MODEL = get_env("LLM_MODEL", "46-qwen3.5-35B")
USE_LLM_ROLE_IDENTIFICATION = get_env_bool("USE_LLM_ROLE_IDENTIFICATION", False)
LLM_ROLE_CANDIDATES = get_env_list(
"LLM_ROLE_CANDIDATES",
"主持人,正方辩手,反方辩手,计时员,评委,嘉宾,未知",
)
LLM_ROLE_MIN_TEXT_LEN = int(get_env("LLM_ROLE_MIN_TEXT_LEN", "8"))
LLM_ROLE_CACHE_CONFIDENCE = float(get_env("LLM_ROLE_CACHE_CONFIDENCE", "0.82"))
LLM_ROLE_HISTORY_MAX_CHARS = int(get_env("LLM_ROLE_HISTORY_MAX_CHARS", "1200"))
# 候选说话人列表（用于身份识别 Agent）
CANDIDATE_SPEAKERS = ["局长", "主持人", "商务专员", "市场专员", "控制要素席", "空中侦察席"]
# ========================
# 3. Redis 管理器
# ========================
class RedisManager:
"""
Redis 会话管理器
职责：
- 管理与 Redis 的连接池
- 存储每个会话的文本缓冲（累积不完整的识别片段）
- 存储历史对话记录（用于上下文理解）
"""
def __init__(self, url):
"""
初始化 Redis 连接池
Args:
url: Redis 连接 URL
"""
if redis is None:
raise RuntimeError("redis package is not installed")
# 使用连接池提高并发性能，decode_responses=True 自动将字节转为字符串
self.pool = redis.ConnectionPool.from_url(url, decode_responses=True)
# 键前缀，用于命名隔离，避免与其他业务冲突
self.prefix = "meeting:"
async def _get_conn(self):
"""从连接池获取一个 Redis 连接"""
return redis.Redis(connection_pool=self.pool)
async def get_buffer(self, sid):
"""
获取指定会话的文本缓冲
Args:
sid: 会话 ID（Session ID）
Returns:
该会话当前累积的文本片段
"""
r = await self._get_conn()
val = await r.get(f"{self.prefix}buffer:{sid}") or ""
return val
async def append_buffer(self, sid, text):
"""
向会话缓冲追加新的文本片段
Args:
sid: 会话 ID
text: 新识别的文本片段
"""
r = await self._get_conn()
# 追加文本并添加空格分隔
await r.append(f"{self.prefix}buffer:{sid}", text + " ")
async def clear_buffer(self, sid):
"""
清空指定会话的缓冲区
用途：当判断发言完成后，清空缓冲准备下一次发言
Args:
sid: 会话 ID
"""
r = await self._get_conn()
await r.delete(f"{self.prefix}buffer:{sid}")
async def push_history(self, sid, role, content):
"""
将完成的对话推入历史记录
Args:
sid: 会话 ID
role: 说话人角色
content: 对话内容
"""
r = await self._get_conn()
key = f"{self.prefix}history:{sid}"
# 将对话记录推入列表右端
await r.rpush(key, json.dumps({"role": role, "content": content}))
# 只保留最近 20 条记录，避免历史过长影响性能
await r.ltrim(key, -20, -1)
async def get_history(self, sid):
"""
获取会话的历史对话记录
Args:
sid: 会话 ID
Returns:
历史对话列表，每项包含 role 和 content
"""
r = await self._get_conn()
items = await r.lrange(f"{self.prefix}history:{sid}", 0, -1)
return [json.loads(i) for i in items]
async def save_speaker_profile(self, sid, speaker, text, features=None):
"""
保存说话人的特征到长期记忆
Args:
sid: 会话 ID
speaker: 说话人名称
text: 发言文本
features: 可选的特征字典（如关键词、语速等）
"""
r = await self._get_conn()
key = f"{self.prefix}speaker_profile:{sid}:{speaker}"
# 构造特征记录
profile_entry = {
"text": text[:200], # 只保留前200个字符作为示例
"timestamp": asyncio.get_event_loop().time(),
"features": features or {}
}
# 将记录推入该说话人的历史列表
await r.rpush(key, json.dumps(profile_entry))
# 只保留最近10条该说话人的记录
await r.ltrim(key, -10, -1)
async def get_speaker_profiles(self, sid):
"""
获取所有说话人的历史特征记录
Args:
sid: 会话 ID
Returns:
字典，key为说话人名称，value为该说话人的历史记录列表
"""
r = await self._get_conn()
# 扫描所有说话人的profile键
pattern = f"{self.prefix}speaker_profile:{sid}:*"
profiles = {}
async for key in r.scan_iter(match=pattern):
# 从键名中提取说话人名称
speaker = key.decode() if isinstance(key, bytes) else key
speaker = speaker.split(":")[-1]
# 获取该说话人的所有历史记录
items = await r.lrange(key, 0, -1)
profiles[speaker] = [json.loads(i) for i in items]
return profiles
async def close(self):
"""关闭连接池，释放资源（修复 DeprecationWarning）"""
await self.pool.disconnect()
# ========================
# 3. Agent 实现
# ========================
class BaseAgent:
"""
Agent 基类
提供所有 Agent 的通用能力：
- 调用 LLM 进行推理
- 提取 LLM 返回的 XML 标签结果
"""
def __init__(self, client: AsyncOpenAI):
"""
初始化 Agent
Args:
client: OpenAI 异步客户端
"""
self.client = client
async def _call_llm(self, system_prompt: str, user_content: str) -> str:
"""
调用大语言模型进行推理
Args:
system_prompt: 系统提示词（定义 Agent 的角色和行为）
user_content: 用户输入内容
Returns:
LLM 返回的文本结果，失败时返回空字符串
"""
try:
response = await self.client.chat.completions.create(
model=LLM_MODEL,
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_content}
],
temperature=0.1, # 低温度使输出更确定、一致
)
return response.choices[0].message.content
except Exception as e:
logger.error(f"LLM调用失败: {e}", exc_info=True)
return ""
def _extract_result(self, text: str) -> str:
"""
从 LLM 返回结果中提取 <result> 标签内容
Args:
text: LLM 返回的完整文本
Returns:
提取的结果，如果没有标签则返回原文
"""
match = re.search(r'<result>(.*?)</result>', text, re.DOTALL | re.IGNORECASE)
return match.group(1).strip() if match else text.strip()
class AuditorAgent(BaseAgent):
"""
审计 Agent：判断当前发言是否完成
功能：
- 分析累积的文本缓冲
- 判断说话人是否已经完成一次完整的发言
- 判断依据：语义完整性、停顿暗示等
"""
# 系统提示词：定义 Agent 的角色和输出格式
SYSTEM_PROMPT = "你是一个对话状态监测专家。请判断发言是否结束，放入 <result>true/false</result> 中。"
async def check(self, text: str) -> bool:
"""
判断给定文本是否代表一次完整的发言
Args:
text: 当前累积的文本片段
Returns:
True 表示发言完成，False 表示发言未完成（继续累积）
"""
res = await self._call_llm(self.SYSTEM_PROMPT, f"内容：{text}")
return "true" in self._extract_result(res).lower()
class ProfilerAgent(BaseAgent):
"""
分析 Agent：识别说话人身份
功能：
- 基于文本内容和说话风格识别说话人
- 利用历史对话上下文提高识别准确率
- 利用长期记忆（说话人特征库）增强识别
- 从预定义的候选说话人列表中选择
"""
# 系统提示词：定义识别任务和候选说话人列表
SYSTEM_PROMPT = f"你是一个会议身份识别专家。请从 {CANDIDATE_SPEAKERS} 中选择发言人，放入 <result>名称</result> 中。"
# 增强版提示词：结合长期记忆
SYSTEM_PROMPT_WITH_MEMORY = f"""你是一个会议身份识别专家。请根据以下信息识别发言人：
候选说话人列表：{CANDIDATE_SPEAKERS}
=== 会议历史记录（按时间顺序）===
{{history}}
=== 说话人特征库（长期记忆）===
{{speaker_profiles}}
=== 当前待识别的发言 ===
{{current_text}}
=== 识别指南 ===
请仔细分析并综合判断：
1. **对话流程分析**：
- 查看历史记录中的对话顺序和上下文
- 判断当前发言是对之前谁的话题的回应、延续或补充
- 注意对话的逻辑关系和互动模式
2. **说话风格匹配**：
- 每个角色的用词习惯、句式结构、语气的特点
- 专有名词、技术术语的使用习惯
- 提问方式、陈述方式、指令方式的差异
3. **角色职责判断**：
- 根据各角色的职责范围判断发言内容的合理性
- 例如：局长通常做总结和决策，主持人负责控场，专员负责具体业务汇报
4. **话题连贯性**：
- 当前发言与该角色之前发言的话题是否连贯
- 是否体现了该角色应有的专业领域关注点
将识别结果放入 <result>说话人名称</result> 中。"""
async def identify(self, text: str, history: List[Dict], speaker_profiles: Dict = None) -> str:
"""
识别当前文本的说话人身份
Args:
text: 当前发言文本
history: 历史对话记录（用于上下文理解）
speaker_profiles: 说话人特征库（长期记忆），key为说话人名称，value为历史记录列表
Returns:
识别出的说话人名称
"""
# 如果有长期记忆，使用增强版提示词
if speaker_profiles:
# 使用更多历史记录（最多15条），保持完整的会议上下文
h_str = self._format_history(history[-15:])
# 构建说话人特征库摘要
profiles_summary = self._format_profiles(speaker_profiles)
prompt = self.SYSTEM_PROMPT_WITH_MEMORY.format(
history=h_str or "暂无历史记录",
speaker_profiles=profiles_summary,
current_text=text
)
res = await self._call_llm(prompt, "")
return self._extract_result(res)
else:
# 原有的简单识别逻辑
h_str = "\n".join([f"{h['role']}: {h['content']}" for h in history[-5:]])
res = await self._call_llm(self.SYSTEM_PROMPT, f"历史：{h_str}\n当前：{text}")
return self._extract_result(res)
def _format_history(self, history: List[Dict]) -> str:
"""
格式化历史对话记录，添加序号和时间顺序
Args:
history: 历史对话列表
Returns:
格式化后的对话文本
"""
if not history:
return "暂无历史记录"
lines = []
for idx, h in enumerate(history, 1):
content_preview = h['content'][:150] # 只显示前150字符
if len(h['content']) > 150:
content_preview += "..."
lines.append(f"[{idx}] {h['role']}: {content_preview}")
return "\n".join(lines)
def _format_profiles(self, speaker_profiles: Dict) -> str:
"""
格式化说话人特征库为可读文本
Args:
speaker_profiles: 说话人特征字典
Returns:
格式化后的文本摘要
"""
if not speaker_profiles:
return "暂无说话人特征记录"
summary_lines = []
for speaker, profiles in speaker_profiles.items():
# 提取该说话人最近5条发言的文本片段
recent_texts = [p['text'] for p in profiles[-5:]]
texts_str = "; ".join(recent_texts)
# 统计该说话人的发言次数
speaking_count = len(profiles)
summary_lines.append(f"- 【{speaker}】(发言{speaking_count}次): {texts_str}")
return "\n".join(summary_lines)
# ========================
# 4. A2A 编排器
# ========================
class A2AOrchestrator:
"""
Agent-to-Agent 编排器
职责：
- 协调多个 Agent 的协作流程
- 处理 ASR 识别结果，依次调用 Auditor 和 Profiler
- 管理会话状态（缓冲和历史记录）
- 将最终结果推送给前端
"""
def __init__(self, client: AsyncOpenAI, redis_mgr: RedisManager):
"""
初始化编排器
Args:
client: OpenAI 异步客户端
redis_mgr: Redis 管理器实例
"""
self.redis = redis_mgr
self.auditor = AuditorAgent(client) # 审计 Agent
self.profiler = ProfilerAgent(client) # 分析 Agent
async def process_asr_fragment(self, session_id: str, fragment: str, websocket):
"""
处理 ASR 识别的文本片段（核心流程）
流程：
1. 将新片段追加到缓冲区
2. 让 Auditor 判断发言是否完成
3. 如果完成，让 Profiler 使用长期记忆识别说话人
4. 将识别结果保存到长期记忆
5. 将结果存入历史，清空缓冲
6. 推送结果到前端
Args:
session_id: 会话 ID
fragment: ASR 识别的新文本片段
websocket: 前端 WebSocket 连接（用于推送结果）
"""
# 1. 追加新片段到缓冲区
await self.redis.append_buffer(session_id, fragment)
buf = await self.redis.get_buffer(session_id)
# 2. 检查发言是否完成
if await self.auditor.check(buf):
logger.info(f"[Auditor] 发言完成检测通过 - 会话ID: {session_id}")
# 3. 获取长期记忆（说话人特征库）
speaker_profiles = await self.redis.get_speaker_profiles(session_id)
logger.debug(f"获取说话人特征库 - 会话ID: {session_id}, 说话人数量: {len(speaker_profiles)}")
# 4. 识别说话人身份（使用长期记忆）
history = await self.redis.get_history(session_id)
speaker = await self.profiler.identify(buf, history, speaker_profiles)
logger.info(f"[Profiler] 识别说话人: {speaker} - 会话ID: {session_id}")
# 5. 保存到长期记忆（更新说话人特征库）
await self.redis.save_speaker_profile(session_id, speaker, buf)
# 6. 保存到历史记录并清空缓冲
await self.redis.push_history(session_id, speaker, buf)
await self.redis.clear_buffer(session_id)
# 7. 构造结果消息
segments = [{
"id": 0,
"speaker": speaker,
"embedding": None, # 占位（预留字段）
"seek": 0,
"full_text": buf.strip()
}]
msg = {
"mode": "offline-speaker",
"type": "asr_with_speaker",
"segments": segments,
"speaker": speaker
}
# 推送结果到前端（失败时静默处理）
try:
await websocket.send(json.dumps(msg, ensure_ascii=False))
except:
pass
def build_acoustic_speaker_message(data: Dict) -> Optional[Dict]:
text = (data.get("text") or "").strip()
speaker = (data.get("spk_name") or "").strip()
if not text or not speaker or speaker.lower() == "unknown":
return None
segment = {
"id": 0,
"speaker": speaker,
"embedding": None,
"seek": 0,
"full_text": text,
"start": None,
"end": None,
}
msg = {
"mode": "offline-speaker",
"type": "asr_with_speaker",
"source": "acoustic",
"speaker": speaker,
"spk_score": data.get("spk_score"),
"text": text,
"wav_name": data.get("wav_name"),
"is_final": data.get("is_final", True),
"segments": [segment],
}
if "timestamp" in data:
msg["timestamp"] = data["timestamp"]
if "sentence_info" in data:
msg["sentence_info"] = data["sentence_info"]
return msg
MOJIBAKE_MARKERS = ("锛", "涓", "灏", "鍒", "鏄", "鎴", "浣", "鐨", "璇", "濂", "绗")
def maybe_repair_mojibake(text: str) -> str:
if not text:
return text
marker_count = sum(text.count(marker) for marker in MOJIBAKE_MARKERS)
if marker_count < 3:
return text
try:
repaired = text.encode("gbk").decode("utf-8")
except UnicodeError:
return text
repaired_marker_count = sum(repaired.count(marker) for marker in MOJIBAKE_MARKERS)
return repaired if repaired_marker_count < marker_count else text
def _extract_json_object(text: str) -> Dict:
if not text:
return {}
try:
return json.loads(text)
except json.JSONDecodeError:
pass
start = text.find("{")
end = text.rfind("}")
if start == -1 or end == -1 or end <= start:
return {}
try:
return json.loads(text[start:end + 1])
except json.JSONDecodeError:
return {}
def _safe_confidence(value, default: float = 0.0) -> float:
try:
confidence = float(value)
except (TypeError, ValueError):
confidence = default
return max(0.0, min(1.0, confidence))
class RoleIdentifier:
def __init__(self, client):
self.client = client
self.profiles: Dict[str, Dict] = {}
def _get_profile(self, speaker: str) -> Dict:
return self.profiles.setdefault(
speaker,
{"texts": [], "role": None, "confidence": 0.0, "reason": ""},
)
def _append_text(self, profile: Dict, text: str):
if not text:
return
profile["texts"].append(text)
joined = "\n".join(profile["texts"])
while len(joined) > LLM_ROLE_HISTORY_MAX_CHARS and len(profile["texts"]) > 1:
profile["texts"].pop(0)
joined = "\n".join(profile["texts"])
async def identify(self, speaker: str, text: str) -> Optional[Dict]:
clean_text = maybe_repair_mojibake(text.strip())
profile = self._get_profile(speaker)
self._append_text(profile, clean_text)
cached_role = profile.get("role")
cached_confidence = _safe_confidence(profile.get("confidence"))
if cached_role and cached_confidence >= LLM_ROLE_CACHE_CONFIDENCE:
return {
"role": cached_role,
"role_confidence": cached_confidence,
"role_reason": profile.get("reason", ""),
"role_source": "llm_text_cached",
"llm_text": clean_text,
}
if len(clean_text) < LLM_ROLE_MIN_TEXT_LEN:
return None
system_prompt = (
"你是会议/辩论转写的角色识别器。"
"只能根据文本内容、说话顺序和同一声纹说话人的历史文本判断角色。"
"不要修改 speaker 编号；speaker_1、speaker_2 是声纹身份，不是角色。"
"候选角色只能从给定列表选择。"
"只返回 JSON，不要输出解释文本。"
'JSON 格式：{"role":"候选角色之一","confidence":0.0,"reason":"一句话原因"}'
)
user_payload = {
"acoustic_speaker": speaker,
"candidate_roles": LLM_ROLE_CANDIDATES,
"speaker_history": profile["texts"][-6:],
"current_text": clean_text,
}
try:
response = await self.client.chat.completions.create(
model=LLM_MODEL,
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": json.dumps(user_payload, ensure_ascii=False)},
],
temperature=0.0,
)
content = response.choices[0].message.content or ""
parsed = _extract_json_object(content)
role = str(parsed.get("role") or "").strip()
if role not in LLM_ROLE_CANDIDATES:
role = "未知" if "未知" in LLM_ROLE_CANDIDATES else LLM_ROLE_CANDIDATES[-1]
confidence = _safe_confidence(parsed.get("confidence"))
reason = str(parsed.get("reason") or "").strip()[:160]
profile["role"] = role
profile["confidence"] = confidence
profile["reason"] = reason
return {
"role": role,
"role_confidence": confidence,
"role_reason": reason,
"role_source": "llm_text",
"llm_text": clean_text,
}
except Exception as e:
logger.error(f"LLM角色判断失败: {e}", exc_info=True)
return None
def build_role_enriched_message(speaker_msg: Dict, role_result: Dict) -> Dict:
msg = dict(speaker_msg)
msg["type"] = "asr_with_speaker_role"
msg["source"] = "acoustic+llm"
msg["role"] = role_result.get("role")
msg["role_confidence"] = role_result.get("role_confidence")
msg["role_reason"] = role_result.get("role_reason")
msg["role_source"] = role_result.get("role_source")
segments = []
for segment in speaker_msg.get("segments", []):
seg = dict(segment)
seg["role"] = msg["role"]
seg["role_confidence"] = msg["role_confidence"]
seg["role_reason"] = msg["role_reason"]
segments.append(seg)
msg["segments"] = segments
return msg
# ========================
# 5. WebSocket 处理与任务追踪
# ========================
async def forward_audio(websocket):
"""
WebSocket 连接处理函数（核心入口）
职责：
1. 接受前端 WebSocket 连接
2. 建立到 ASR 服务的下游连接
3. 双向转发消息（前端 -> ASR，ASR -> 前端）
4. 拦截 ASR 结果并触 Agent 处理
5. 确保所有后台任务完成后才退出（防止数据丢失）
Args:
websocket: 前端 WebSocket 连接对象
"""
logger.info("客户端已连接")
client = None
redis_mgr = None
role_identifier = None
use_llm_role_identification = USE_LLM_ROLE_IDENTIFICATION
if use_llm_role_identification:
if AsyncOpenAI is None:
logger.error("openai package is not installed; LLM role identification disabled")
use_llm_role_identification = False
else:
client = AsyncOpenAI(api_key=LLM_API_KEY, base_url=LLM_BASE_URL)
role_identifier = RoleIdentifier(client)
# 用于追踪后台任务，防止连接关闭时任务被强制销毁
background_tasks: Set[asyncio.Task] = set()
try:
# 连接到目标 ASR 服务
# 添加连接参数以提高鲁棒性
async with websockets.connect(
TARGET_WS_URL,
close_timeout=10, # 关闭超时
ping_interval=None,
ping_timeout=None,
max_queue=1024, # 消息队列大小
subprotocols=["binary"],
) as target_ws:
logger.info(f"已连接到 ASR 服务: {TARGET_WS_URL}")
# 发送 ASR 配置（双通道模式，PCM 音频格式）
config = {"mode": "2pass", "chunk_size": [10, 10, 10], "wav_format": "pcm"}
await target_ws.send(json.dumps(config))
logger.debug(f"发送 ASR 配置: {config}")
async def frontend_to_target():
"""
上行转发：前端音频数据 -> ASR 服务
"""
try:
async for message in websocket:
await target_ws.send(message)
except websockets.exceptions.ConnectionClosed:
logger.info("客户端连接已关闭，停止上行转发")
finally:
try:
await target_ws.close()
except Exception:
pass
async def send_role_update(speaker_msg: Dict):
if role_identifier is None:
return
try:
role_result = await role_identifier.identify(
speaker_msg.get("speaker", ""),
speaker_msg.get("text", ""),
)
if not role_result:
return
role_msg = build_role_enriched_message(speaker_msg, role_result)
await websocket.send(json.dumps(role_msg, ensure_ascii=False))
except Exception as e:
logger.error(f"发送LLM角色结果失败: {e}", exc_info=True)
async def target_to_frontend():
"""
下行转发：ASR 识别结果 -> 前端
同时拦截结果触发 Agent 处理
"""
try:
async for message in target_ws:
# 1. 透传 ASR 原始结果到前端（字幕显示）
try:
await websocket.send(message)
except websockets.exceptions.ConnectionClosed:
logger.info("客户端连接已关闭，停止下行转发")
break
# 2. 如果是文本消息，尝试触发 Agent 处理
if isinstance(message, str):
try:
data = json.loads(message)
# 只处理最终识别结果（2pass-offline 模式）
if data.get("mode") == "2pass-offline":
speaker_msg = build_acoustic_speaker_message(data)
if speaker_msg is not None:
await websocket.send(json.dumps(speaker_msg, ensure_ascii=False))
if use_llm_role_identification and role_identifier is not None:
t = asyncio.create_task(send_role_update(speaker_msg))
background_tasks.add(t)
t.add_done_callback(background_tasks.discard)
except websockets.exceptions.ConnectionClosed:
logger.info("客户端连接已关闭，停止发送增强结果")
break
except Exception as e:
logger.error(f"处理 ASR 结果时出错: {e}", exc_info=True)
except websockets.exceptions.ConnectionClosed as e:
logger.info(f"ASR 服务连接已关闭: {type(e).__name__}: {e}")
finally:
try:
await websocket.close()
except Exception:
pass
# 并发运行两个方向的数据流
await asyncio.gather(frontend_to_target(), target_to_frontend())
except websockets.exceptions.InvalidMessage as e:
logger.error(f"WebSocket 握手失败 - ASR 服务可能未正常运行或 URL 配置错误: {e}")
logger.error(f"请检查: 1) ASR 服务是否在 {TARGET_WS_URL} 正常运行")
logger.error(f" 2) ASR 服务的 URL 路径是否正确")
logger.error(f" 3) ASR 服务是否需要特定的认证或配置")
except ConnectionRefusedError:
logger.error(f"连接被拒绝 - 无法连接到 ASR 服务 {TARGET_WS_URL}")
logger.error(f"请检查 ASR 服务是否已启动")
except Exception as e:
logger.error(f"连接错误: {type(e).__name__}: {e}", exc_info=True)
finally:
# --- 优雅退出处理 ---
if background_tasks:
logger.info(f"等待 {len(background_tasks)} 个后台 Agent 任务完成...")
# 等待所有剩余任务跑完，最多等 5 秒（防止无限等待）
await asyncio.wait(background_tasks, timeout=5)
logger.debug("所有后台任务已完成或超时")
# 释放资源
if client is not None:
await client.close()
if redis_mgr is not None:
await redis_mgr.close()
logger.info("客户端已断开连接")
async def main():
"""
主函数：启动 WebSocket 服务器
"""
logger.info(f"启动 WebSocket 智能语义分析服务: ws://{LOCAL_HOST}:{LOCAL_PORT}")
logger.info(f"Agent 模型: {LLM_MODEL}")
logger.info(f"ASR 服务地址: {TARGET_WS_URL}")
logger.info(f"Redis 地址: {REDIS_URL}")
# 启动服务器并持续运行（asyncio.Future() 永不完成）
async with websockets.serve(forward_audio, LOCAL_HOST, LOCAL_PORT, ping_interval=None):
await asyncio.Future()
if __name__ == "__main__":
try:
asyncio.run(main())
except KeyboardInterrupt:
pass

271
fastapi_wss/src/agent_wss.py Normal file
View File

@ -0,0 +1,271 @@
import asyncio
import json
import os
import websockets
import redis.asyncio as redis
import re
from openai import AsyncOpenAI # ⬅️ 引入 OpenAI 异步客户端
# ========================
# 基础配置
# ========================
TARGET_WS_URL = os.getenv("TARGET_WS_URL", "ws://localhost:59805/ws/asr")
LOCAL_HOST = "0.0.0.0"
LOCAL_PORT = 10095
# Redis
REDIS_URL = os.getenv("REDIS_URL", "redis://localhost:6379/0")
REDIS_KEY_PREFIX = "meeting:"
# LLM 配置 (适配 OpenAI SDK)
# 如果是 OpenAI 官方，Base URL 默认即可；如果是 DeepSeek/阿里等，需修改 BASE_URL
LLM_API_KEY = os.getenv("LLM_API_KEY", "sk-xxxx")
LLM_BASE_URL = os.getenv("LLM_BASE_URL", "http://172.18.127.124:9997/v1")
LLM_MODEL = os.getenv("LLM_MODEL", "Qwen3-32B")
# 候选人
CANDIDATE_SPEAKERS = ["局长", "主持人", "商务专员", "市场专员"]
DEFAULT_SESSION_ID = "room_101"
# ASR 初始化配置
config = {
"chunk_size": [10, 10, 10],
"wav_name": "h5",
"is_speaking": True,
"wav_format": "pcm",
"chunk_interval": 10,
"itn": True,
"mode": "2pass",
"hotwords": "",
}
# ========================
# Redis 操作封装 (保持不变)
# ========================
async def append_to_buffer(session_id, text):
"""将新的片段追加到暂存区"""
r = redis.from_url(REDIS_URL, decode_responses=True)
key = f"{REDIS_KEY_PREFIX}buffer:{session_id}"
await r.append(key, text + " ")
current_buffer = await r.get(key)
await r.close()
return current_buffer
async def clear_buffer(session_id):
"""清空暂存区"""
r = redis.from_url(REDIS_URL, decode_responses=True)
key = f"{REDIS_KEY_PREFIX}buffer:{session_id}"
await r.delete(key)
await r.close()
async def save_history(session_id, role, content):
"""归档历史记录"""
r = redis.from_url(REDIS_URL, decode_responses=True)
key = f"{REDIS_KEY_PREFIX}history:{session_id}"
record = json.dumps({"role": role, "content": content})
await r.rpush(key, record)
await r.ltrim(key, -20, -1)
await r.close()
async def get_history(session_id):
"""获取上下文"""
r = redis.from_url(REDIS_URL, decode_responses=True)
key = f"{REDIS_KEY_PREFIX}history:{session_id}"
items = await r.lrange(key, 0, -1)
await r.close()
return [json.loads(i) for i in items]
# ========================
# 通用 LLM 调用函数 (改为 OpenAI SDK)
# ========================
async def call_llm(client: AsyncOpenAI, system_prompt: str, user_text: str):
"""
使用 OpenAI SDK 发起调用
"""
try:
response = await client.chat.completions.create(
model=LLM_MODEL,
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_text}
],
temperature=0.1,
)
content = response.choices[0].message.content
print(f"🤖 LLM 返回: {content}")
return content
except Exception as e:
print(f"❌ OpenAI API 调用异常: {e}")
return None
# ========================
# Agent A: 完结判断 (The Auditor)
# ========================
async def agent_check_completion(text_buffer, client: AsyncOpenAI):
"""
判断当前的文本累积是否已经构成了一个完整的发言回合。
"""
system_prompt = """
#ROLE
你是一个对话状态判断助手。你的任务是判断当前的发言内容是否已经结束。
#RULE
1. **强制标准**：如果内容中包含 "发言完毕"、"讲完了"、"汇报完毕" 等明确结束语，必须返回 true。
2. **语义标准**：如果内容完整且看似一段独立的话结束，返回 true；如果看似还在列举或句子未完，返回 false。
3. 如果内容仅仅是很短的一句 "嗯"、"好的"，视作未结束或无需独立归档。
4. 请将结果用 <result> </result>包裹着。
#EXAMPLES
示例1:空中侦察席报告，今日预计飞行287架航班，空气湿度适宜，安全风险较小。 <result>true</result>
示例2:控制要素席报告，我这边已经完成了所有的准备工作.<result>false<result>
示例3:下面有请局长发言 <result>false</result>
示例4:地面侦察席报告目前没有发现敌对情况.<result>false</result>
"""
result = await call_llm(client, system_prompt, f"当前累积内容：{text_buffer}")
if result:
# 更健壮的版本（容忍某些格式错误）
match = re.search(r'<result>([^<]*)(?:</result>|<result>|$)', result, re.IGNORECASE)
if match:
content = match.group(1).strip()
else:
content = False # 或者根据你的逻辑处理未匹配的情况
return content
return False
# ========================
# Agent B: 身份识别 (The Profiler)
# ========================
async def agent_identify_speaker(text_full, session_id, client: AsyncOpenAI):
"""
仅在 Agent A 判定结束时调用。根据全文和历史判断身份。
"""
history = await get_history(session_id)
history_str = "\n".join([f"{h['role']}: {h['content']}" for h in history])
system_prompt = f"""
#ROLE
你是一个会议记录专家。请根据上下文和完整的段落内容，推断当前的发言人。
#RULE
1. **强制标准**：如果内容中包含 "控制要素席"、"空中侦察席"、"海上观察席"、"联合指挥中心"等明确发言人，必须直接返回名称。
2. **语义标准**：如果内容完整且没有明确发言人，则根据语义进行判断，潜在的候选人有:{', '.join(CANDIDATE_SPEAKERS)}。
3. 提供历史上下文：{history_str}
4. 请将结果用 <result> </result>包裹着。
#EXAMPLES
示例1:空中侦察席报告，今日预计飞行287架航班，空气湿度适宜，安全风险较小。 <result>空中侦察席</result>
示例2:控制要素席报告，我这边已经完成了所有的准备工作，可以开始会议了。 <result>控制要素席<result>
示例3:下面有请局长发言 <result>主持人</result>
"""
result = await call_llm(client, system_prompt, f"完整发言内容：{text_full}")
if result:
match = re.search(r'<result>([^<]*)(?:</result>|<result>|$)', result, re.IGNORECASE)
if match:
content = match.group(1).strip()
return content
return "未知说话人"
# ========================
# 核心编排逻辑 (多 Agent 协作)
# ========================
async def orchestrate_agents(new_fragment, websocket, client: AsyncOpenAI):
session_id = DEFAULT_SESSION_ID
# 1. 存入 Buffer (累积)
current_buffer = await append_to_buffer(session_id, new_fragment)
print(f"📥 [Buffer] 长度: {len(current_buffer)} | 内容片段: {current_buffer[-20:]}")
# 2. 调用 Agent A 判断是否结束
is_finished = await agent_check_completion(current_buffer, client)
if is_finished:
print(f"🛑 [Agent A] 判定发言结束。触发身份识别...")
# 3. 调用 Agent B 识别身份
speaker = await agent_identify_speaker(current_buffer, session_id, client)
print(f"👤 [Agent B] 识别为: {speaker}")
# 4. 存入历史并清空 Buffer
await save_history(session_id, speaker, current_buffer)
await clear_buffer(session_id)
# 5. 推送最终结果给前端
msg = {
"type": "final_speech_record",
"speaker": speaker,
"content": current_buffer.strip(),
"timestamp": "now"
}
await websocket.send(json.dumps(msg, ensure_ascii=False))
else:
# 未结束，什么都不做，继续等待下一段文本
print(f"⏳ [Agent A] 判定未结束...")
# ========================
# WebSocket 主逻辑
# ========================
async def forward_audio(websocket):
print("🟢 Client Connected.")
# 初始化 OpenAI 客户端
# 注意：AsyncOpenAI 是线程/协程安全的，也可以放在全局初始化
client = AsyncOpenAI(
api_key=LLM_API_KEY,
base_url=LLM_BASE_URL
)
try:
async with websockets.connect(TARGET_WS_URL) as target_ws:
await target_ws.send(json.dumps(config))
async def frontend_to_target():
async for message in websocket:
await target_ws.send(message)
async def target_to_frontend():
async for message in target_ws:
# 1. 实时转发 (用于字幕上屏)
await websocket.send(message)
if isinstance(message, str):
try:
data = json.loads(message)
# 2. 触发 Agent 协作
if data.get("mode") == "2pass-offline":
asr_text = data.get("text", "").strip()
if asr_text:
# 异步触发，传入 client
asyncio.create_task(
orchestrate_agents(asr_text, websocket, client)
)
except json.JSONDecodeError:
pass
await asyncio.gather(frontend_to_target(), target_to_frontend())
except Exception as e:
print(f"❌ Error: {e}")
finally:
# 关闭客户端（虽然 AsyncOpenAI 通常会自动管理，但显式关闭是好习惯）
await client.close()
print("🔴 Client Disconnected.")
# ========================
# 启动
# ========================
async def main():
print(f"🚀 启动 WebSocket 智能语义分析服务: ws://{LOCAL_HOST}:{LOCAL_PORT}")
print(f"🧠 Agent 模型: {LLM_MODEL}")
# 这里的 websockets.serve 会在 main 协程内部运行，此时已有 Loop
async with websockets.serve(forward_audio, LOCAL_HOST, LOCAL_PORT):
# 创建一个永远 pending 的 Future，让服务一直运行不退出
await asyncio.Future()
if __name__ == "__main__":
try:
asyncio.run(main())
except KeyboardInterrupt:
print("\n🛑 服务已停止")

271
fastapi_wss/src/agent_wss.py.bak-202607150815 Normal file
View File

@ -0,0 +1,271 @@
import asyncio
import json
import os
import websockets
import redis.asyncio as redis
import re
from openai import AsyncOpenAI # ⬅️ 引入 OpenAI 异步客户端
# ========================
# 基础配置
# ========================
TARGET_WS_URL = os.getenv("TARGET_WS_URL", "ws://172.18.127.124:10096/")
LOCAL_HOST = "0.0.0.0"
LOCAL_PORT = 10095
# Redis
REDIS_URL = os.getenv("REDIS_URL", "redis://localhost:6379/0")
REDIS_KEY_PREFIX = "meeting:"
# LLM 配置 (适配 OpenAI SDK)
# 如果是 OpenAI 官方，Base URL 默认即可；如果是 DeepSeek/阿里等，需修改 BASE_URL
LLM_API_KEY = os.getenv("LLM_API_KEY", "sk-xxxx")
LLM_BASE_URL = os.getenv("LLM_BASE_URL", "http://172.18.127.124:9997/v1")
LLM_MODEL = os.getenv("LLM_MODEL", "Qwen3-32B")
# 候选人
CANDIDATE_SPEAKERS = ["局长", "主持人", "商务专员", "市场专员"]
DEFAULT_SESSION_ID = "room_101"
# ASR 初始化配置
config = {
"chunk_size": [10, 10, 10],
"wav_name": "h5",
"is_speaking": True,
"wav_format": "pcm",
"chunk_interval": 10,
"itn": True,
"mode": "2pass",
"hotwords": "",
}
# ========================
# Redis 操作封装 (保持不变)
# ========================
async def append_to_buffer(session_id, text):
"""将新的片段追加到暂存区"""
r = redis.from_url(REDIS_URL, decode_responses=True)
key = f"{REDIS_KEY_PREFIX}buffer:{session_id}"
await r.append(key, text + " ")
current_buffer = await r.get(key)
await r.close()
return current_buffer
async def clear_buffer(session_id):
"""清空暂存区"""
r = redis.from_url(REDIS_URL, decode_responses=True)
key = f"{REDIS_KEY_PREFIX}buffer:{session_id}"
await r.delete(key)
await r.close()
async def save_history(session_id, role, content):
"""归档历史记录"""
r = redis.from_url(REDIS_URL, decode_responses=True)
key = f"{REDIS_KEY_PREFIX}history:{session_id}"
record = json.dumps({"role": role, "content": content})
await r.rpush(key, record)
await r.ltrim(key, -20, -1)
await r.close()
async def get_history(session_id):
"""获取上下文"""
r = redis.from_url(REDIS_URL, decode_responses=True)
key = f"{REDIS_KEY_PREFIX}history:{session_id}"
items = await r.lrange(key, 0, -1)
await r.close()
return [json.loads(i) for i in items]
# ========================
# 通用 LLM 调用函数 (改为 OpenAI SDK)
# ========================
async def call_llm(client: AsyncOpenAI, system_prompt: str, user_text: str):
"""
使用 OpenAI SDK 发起调用
"""
try:
response = await client.chat.completions.create(
model=LLM_MODEL,
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_text}
],
temperature=0.1,
)
content = response.choices[0].message.content
print(f"🤖 LLM 返回: {content}")
return content
except Exception as e:
print(f"❌ OpenAI API 调用异常: {e}")
return None
# ========================
# Agent A: 完结判断 (The Auditor)
# ========================
async def agent_check_completion(text_buffer, client: AsyncOpenAI):
"""
判断当前的文本累积是否已经构成了一个完整的发言回合。
"""
system_prompt = """
#ROLE
你是一个对话状态判断助手。你的任务是判断当前的发言内容是否已经结束。
#RULE
1. **强制标准**：如果内容中包含 "发言完毕"、"讲完了"、"汇报完毕" 等明确结束语，必须返回 true。
2. **语义标准**：如果内容完整且看似一段独立的话结束，返回 true；如果看似还在列举或句子未完，返回 false。
3. 如果内容仅仅是很短的一句 "嗯"、"好的"，视作未结束或无需独立归档。
4. 请将结果用 <result> </result>包裹着。
#EXAMPLES
示例1:空中侦察席报告，今日预计飞行287架航班，空气湿度适宜，安全风险较小。 <result>true</result>
示例2:控制要素席报告，我这边已经完成了所有的准备工作.<result>false<result>
示例3:下面有请局长发言 <result>false</result>
示例4:地面侦察席报告目前没有发现敌对情况.<result>false</result>
"""
result = await call_llm(client, system_prompt, f"当前累积内容：{text_buffer}")
if result:
# 更健壮的版本（容忍某些格式错误）
match = re.search(r'<result>([^<]*)(?:</result>|<result>|$)', result, re.IGNORECASE)
if match:
content = match.group(1).strip()
else:
content = False # 或者根据你的逻辑处理未匹配的情况
return content
return False
# ========================
# Agent B: 身份识别 (The Profiler)
# ========================
async def agent_identify_speaker(text_full, session_id, client: AsyncOpenAI):
"""
仅在 Agent A 判定结束时调用。根据全文和历史判断身份。
"""
history = await get_history(session_id)
history_str = "\n".join([f"{h['role']}: {h['content']}" for h in history])
system_prompt = f"""
#ROLE
你是一个会议记录专家。请根据上下文和完整的段落内容，推断当前的发言人。
#RULE
1. **强制标准**：如果内容中包含 "控制要素席"、"空中侦察席"、"海上观察席"、"联合指挥中心"等明确发言人，必须直接返回名称。
2. **语义标准**：如果内容完整且没有明确发言人，则根据语义进行判断，潜在的候选人有:{', '.join(CANDIDATE_SPEAKERS)}。
3. 提供历史上下文：{history_str}
4. 请将结果用 <result> </result>包裹着。
#EXAMPLES
示例1:空中侦察席报告，今日预计飞行287架航班，空气湿度适宜，安全风险较小。 <result>空中侦察席</result>
示例2:控制要素席报告，我这边已经完成了所有的准备工作，可以开始会议了。 <result>控制要素席<result>
示例3:下面有请局长发言 <result>主持人</result>
"""
result = await call_llm(client, system_prompt, f"完整发言内容：{text_full}")
if result:
match = re.search(r'<result>([^<]*)(?:</result>|<result>|$)', result, re.IGNORECASE)
if match:
content = match.group(1).strip()
return content
return "未知说话人"
# ========================
# 核心编排逻辑 (多 Agent 协作)
# ========================
async def orchestrate_agents(new_fragment, websocket, client: AsyncOpenAI):
session_id = DEFAULT_SESSION_ID
# 1. 存入 Buffer (累积)
current_buffer = await append_to_buffer(session_id, new_fragment)
print(f"📥 [Buffer] 长度: {len(current_buffer)} | 内容片段: {current_buffer[-20:]}")
# 2. 调用 Agent A 判断是否结束
is_finished = await agent_check_completion(current_buffer, client)
if is_finished:
print(f"🛑 [Agent A] 判定发言结束。触发身份识别...")
# 3. 调用 Agent B 识别身份
speaker = await agent_identify_speaker(current_buffer, session_id, client)
print(f"👤 [Agent B] 识别为: {speaker}")
# 4. 存入历史并清空 Buffer
await save_history(session_id, speaker, current_buffer)
await clear_buffer(session_id)
# 5. 推送最终结果给前端
msg = {
"type": "final_speech_record",
"speaker": speaker,
"content": current_buffer.strip(),
"timestamp": "now"
}
await websocket.send(json.dumps(msg, ensure_ascii=False))
else:
# 未结束，什么都不做，继续等待下一段文本
print(f"⏳ [Agent A] 判定未结束...")
# ========================
# WebSocket 主逻辑
# ========================
async def forward_audio(websocket):
print("🟢 Client Connected.")
# 初始化 OpenAI 客户端
# 注意：AsyncOpenAI 是线程/协程安全的，也可以放在全局初始化
client = AsyncOpenAI(
api_key=LLM_API_KEY,
base_url=LLM_BASE_URL
)
try:
async with websockets.connect(TARGET_WS_URL) as target_ws:
await target_ws.send(json.dumps(config))
async def frontend_to_target():
async for message in websocket:
await target_ws.send(message)
async def target_to_frontend():
async for message in target_ws:
# 1. 实时转发 (用于字幕上屏)
await websocket.send(message)
if isinstance(message, str):
try:
data = json.loads(message)
# 2. 触发 Agent 协作
if data.get("mode") == "2pass-offline":
asr_text = data.get("text", "").strip()
if asr_text:
# 异步触发，传入 client
asyncio.create_task(
orchestrate_agents(asr_text, websocket, client)
)
except json.JSONDecodeError:
pass
await asyncio.gather(frontend_to_target(), target_to_frontend())
except Exception as e:
print(f"❌ Error: {e}")
finally:
# 关闭客户端（虽然 AsyncOpenAI 通常会自动管理，但显式关闭是好习惯）
await client.close()
print("🔴 Client Disconnected.")
# ========================
# 启动
# ========================
async def main():
print(f"🚀 启动 WebSocket 智能语义分析服务: ws://{LOCAL_HOST}:{LOCAL_PORT}")
print(f"🧠 Agent 模型: {LLM_MODEL}")
# 这里的 websockets.serve 会在 main 协程内部运行，此时已有 Loop
async with websockets.serve(forward_audio, LOCAL_HOST, LOCAL_PORT):
# 创建一个永远 pending 的 Future，让服务一直运行不退出
await asyncio.Future()
if __name__ == "__main__":
try:
asyncio.run(main())
except KeyboardInterrupt:
print("\n🛑 服务已停止")

176
fastapi_wss/src/app.py Normal file
View File

@ -0,0 +1,176 @@
import asyncio
import json
import os
import tempfile
import wave
import websockets
import aiohttp
# ========================
# 配置
# ========================
TARGET_WS_URL = os.getenv("TARGET_WS_URL", "ws://localhost:59805/ws/asr")
LOCAL_HOST = "0.0.0.0"
LOCAL_PORT = 10095
# ⚠️ 修改为你自己的 FastAPI ASR 服务地址
ASR_LOCAL_URL = os.getenv("ASR_LOCAL_URL", "http://172.18.127.124:6688/asr")
SAMPLE_RATE = 16000 # ⬅️ 关键！
SAMPLE_WIDTH = 2 # 16-bit
CHANNELS = 1
config = {
"chunk_size": [10, 10, 10],
"wav_name": "h5",
"is_speaking": True,
"wav_format": "pcm",
"chunk_interval": 10,
"itn": True,
"mode": "2pass",
"hotwords": "",
}
# ========================
# 调用本地 ASR 接口（带说话人识别）
# ========================
async def call_local_asr_with_audio(wav_path: str, session: aiohttp.ClientSession):
"""调用你自己的 /asr 接口，返回结构化结果"""
try:
with open(wav_path, "rb") as f:
data = aiohttp.FormData()
data.add_field(
"audio_file",
f,
filename=os.path.basename(wav_path),
content_type="audio/wav"
)
# 可选：传递参数（如 spk_diarization=True 已默认开启）
params = {
"spk_diarization": "true",
"spk_threshold": "0.6",
"output": "json"
}
async with session.post(ASR_LOCAL_URL, data=data, params=params, timeout=aiohttp.ClientTimeout(total=60)) as resp:
if resp.status == 200:
return await resp.json()
else:
text = await resp.text()
print(f"❌ 本地 ASR 接口错误: {resp.status} - {text}")
return None
except Exception as e:
print(f"❌ 调用本地 ASR 异常: {e}")
return None
# ========================
# 后台处理：音频 → 本地 ASR → 推送结果
# ========================
async def process_audio_with_local_asr(audio_chunks, asr_text, websocket, session):
if not audio_chunks:
return
wav_path = None
try:
# 1. 保存为 WAV
with tempfile.NamedTemporaryFile(delete=False, suffix=".wav") as tmp:
wav_path = tmp.name
with wave.open(wav_path, 'wb') as wf:
wf.setnchannels(CHANNELS)
wf.setsampwidth(SAMPLE_WIDTH)
wf.setframerate(SAMPLE_RATE)
wf.writeframes(b''.join(audio_chunks))
# 2. 调用本地 ASR 接口
asr_result = await call_local_asr_with_audio(wav_path, session)
if not asr_result or not asr_result.get("success"):
print("❌ 本地 ASR 处理失败")
return
# 3. 构建前端消息
segments = asr_result.get("segments", [])
full_text = asr_result.get("text", "")
# 推送完整结果（含说话人）
msg = {
"type": "asr_with_speaker",
"full_text": full_text,
"segments": segments, # 每段含 speaker, text, start, end 等
"original_asr_text": asr_text # 来自远程 ASR 的原始文本（可选）
}
await websocket.send(json.dumps(msg, ensure_ascii=False))
print(f"✅ 本地 ASR + 说话人识别完成，共 {len(segments)} 段")
finally:
if wav_path and os.path.exists(wav_path):
os.unlink(wav_path)
# ========================
# 主 WebSocket 转发逻辑
# ========================
async def forward_audio(websocket):
print("🟢 前端已连接。")
current_audio_chunks = []
lock = asyncio.Lock()
async with aiohttp.ClientSession() as session:
try:
async with websockets.connect(TARGET_WS_URL) as target_ws:
print(f"🔗 连接到远程 ASR: {TARGET_WS_URL}")
await target_ws.send(json.dumps(config))
async def frontend_to_target():
async for message in websocket:
if isinstance(message, bytes):
async with lock:
current_audio_chunks.append(message)
await target_ws.send(message)
else:
await target_ws.send(message)
async def target_to_frontend():
async for message in target_ws:
# 立即转发原始 ASR 消息（保持低延迟）
await websocket.send(message)
if isinstance(message, str):
try:
data = json.loads(message)
if data.get("mode") == "2pass-offline":
asr_text = data.get("text", "").strip()
print(f"🔊 触发本地 ASR 处理: '{asr_text}'")
async with lock:
audio_copy = current_audio_chunks.copy()
current_audio_chunks.clear()
# 启动后台任务：调用你自己的 /asr 接口
asyncio.create_task(
process_audio_with_local_asr(audio_copy, asr_text, websocket, session)
)
except json.JSONDecodeError:
pass
await asyncio.gather(frontend_to_target(), target_to_frontend())
except Exception as e:
print(f"❌ 连接异常: {e}")
finally:
print("🔴 前端断开连接。")
# ========================
# 启动服务
# ========================
async def main():
print(f"🚀 启动 WebSocket 中转服务: ws://{LOCAL_HOST}:{LOCAL_PORT}")
print(f"📝 本地 ASR 接口: {ASR_LOCAL_URL}")
server = await websockets.serve(forward_audio, LOCAL_HOST, LOCAL_PORT)
await server.wait_closed()
if __name__ == "__main__":
asyncio.run(main())

176
fastapi_wss/src/app.py.bak-202607150815 Normal file
View File

@ -0,0 +1,176 @@
import asyncio
import json
import os
import tempfile
import wave
import websockets
import aiohttp
# ========================
# 配置
# ========================
TARGET_WS_URL = os.getenv("TARGET_WS_URL", "ws://172.18.127.124:10096/")
LOCAL_HOST = "0.0.0.0"
LOCAL_PORT = 10095
# ⚠️ 修改为你自己的 FastAPI ASR 服务地址
ASR_LOCAL_URL = os.getenv("ASR_LOCAL_URL", "http://172.18.127.124:6688/asr")
SAMPLE_RATE = 16000 # ⬅️ 关键！
SAMPLE_WIDTH = 2 # 16-bit
CHANNELS = 1
config = {
"chunk_size": [10, 10, 10],
"wav_name": "h5",
"is_speaking": True,
"wav_format": "pcm",
"chunk_interval": 10,
"itn": True,
"mode": "2pass",
"hotwords": "",
}
# ========================
# 调用本地 ASR 接口（带说话人识别）
# ========================
async def call_local_asr_with_audio(wav_path: str, session: aiohttp.ClientSession):
"""调用你自己的 /asr 接口，返回结构化结果"""
try:
with open(wav_path, "rb") as f:
data = aiohttp.FormData()
data.add_field(
"audio_file",
f,
filename=os.path.basename(wav_path),
content_type="audio/wav"
)
# 可选：传递参数（如 spk_diarization=True 已默认开启）
params = {
"spk_diarization": "true",
"spk_threshold": "0.6",
"output": "json"
}
async with session.post(ASR_LOCAL_URL, data=data, params=params, timeout=aiohttp.ClientTimeout(total=60)) as resp:
if resp.status == 200:
return await resp.json()
else:
text = await resp.text()
print(f"❌ 本地 ASR 接口错误: {resp.status} - {text}")
return None
except Exception as e:
print(f"❌ 调用本地 ASR 异常: {e}")
return None
# ========================
# 后台处理：音频 → 本地 ASR → 推送结果
# ========================
async def process_audio_with_local_asr(audio_chunks, asr_text, websocket, session):
if not audio_chunks:
return
wav_path = None
try:
# 1. 保存为 WAV
with tempfile.NamedTemporaryFile(delete=False, suffix=".wav") as tmp:
wav_path = tmp.name
with wave.open(wav_path, 'wb') as wf:
wf.setnchannels(CHANNELS)
wf.setsampwidth(SAMPLE_WIDTH)
wf.setframerate(SAMPLE_RATE)
wf.writeframes(b''.join(audio_chunks))
# 2. 调用本地 ASR 接口
asr_result = await call_local_asr_with_audio(wav_path, session)
if not asr_result or not asr_result.get("success"):
print("❌ 本地 ASR 处理失败")
return
# 3. 构建前端消息
segments = asr_result.get("segments", [])
full_text = asr_result.get("text", "")
# 推送完整结果（含说话人）
msg = {
"type": "asr_with_speaker",
"full_text": full_text,
"segments": segments, # 每段含 speaker, text, start, end 等
"original_asr_text": asr_text # 来自远程 ASR 的原始文本（可选）
}
await websocket.send(json.dumps(msg, ensure_ascii=False))
print(f"✅ 本地 ASR + 说话人识别完成，共 {len(segments)} 段")
finally:
if wav_path and os.path.exists(wav_path):
os.unlink(wav_path)
# ========================
# 主 WebSocket 转发逻辑
# ========================
async def forward_audio(websocket):
print("🟢 前端已连接。")
current_audio_chunks = []
lock = asyncio.Lock()
async with aiohttp.ClientSession() as session:
try:
async with websockets.connect(TARGET_WS_URL) as target_ws:
print(f"🔗 连接到远程 ASR: {TARGET_WS_URL}")
await target_ws.send(json.dumps(config))
async def frontend_to_target():
async for message in websocket:
if isinstance(message, bytes):
async with lock:
current_audio_chunks.append(message)
await target_ws.send(message)
else:
await target_ws.send(message)
async def target_to_frontend():
async for message in target_ws:
# 立即转发原始 ASR 消息（保持低延迟）
await websocket.send(message)
if isinstance(message, str):
try:
data = json.loads(message)
if data.get("mode") == "2pass-offline":
asr_text = data.get("text", "").strip()
print(f"🔊 触发本地 ASR 处理: '{asr_text}'")
async with lock:
audio_copy = current_audio_chunks.copy()
current_audio_chunks.clear()
# 启动后台任务：调用你自己的 /asr 接口
asyncio.create_task(
process_audio_with_local_asr(audio_copy, asr_text, websocket, session)
)
except json.JSONDecodeError:
pass
await asyncio.gather(frontend_to_target(), target_to_frontend())
except Exception as e:
print(f"❌ 连接异常: {e}")
finally:
print("🔴 前端断开连接。")
# ========================
# 启动服务
# ========================
async def main():
print(f"🚀 启动 WebSocket 中转服务: ws://{LOCAL_HOST}:{LOCAL_PORT}")
print(f"📝 本地 ASR 接口: {ASR_LOCAL_URL}")
server = await websockets.serve(forward_audio, LOCAL_HOST, LOCAL_PORT)
await server.wait_closed()
if __name__ == "__main__":
asyncio.run(main())

317
fastapi_wss/src/speaker_det.py Normal file
View File

@ -0,0 +1,317 @@
import asyncio
import json
import os
import tempfile
import wave
import websockets
import aiohttp
import logging
from dotenv import load_dotenv
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
force=True
)
logger = logging.getLogger("app")
# ========================
# 配置
# ========================
load_dotenv()
logger.info("正在加载环境变量 ")
TARGET_WS_URL = os.getenv("TARGET_WS_URL", "ws://localhost:59805/ws/asr")
LOCAL_HOST = "0.0.0.0"
LOCAL_PORT = 10095
ASR_LOCAL_URL = os.getenv("ASR_LOCAL_URL", "http://172.18.127.124:6688/asr")
SAMPLE_RATE = 16000
SAMPLE_WIDTH = 2 # 16-bit = 2 bytes
CHANNELS = 1
# 计算每秒的字节数
BYTES_PER_SECOND = SAMPLE_RATE * SAMPLE_WIDTH * CHANNELS
# 设定最小触发时长（秒）
MIN_DURATION_THRESHOLD = 3.0
config = {
"chunk_size": [10, 10, 10],
"wav_name": "h5",
"is_speaking": True,
"wav_format": "pcm",
"chunk_interval": 10,
"itn": True,
"mode": "2pass",
"hotwords": "",
}
# ========================
# 新增：声纹会话缓存管理类 (带超时机制 + 日志)
# ========================
class SpeakerSession:
def __init__(self):
self.last_speaker = None
self.buffer = []
self.accumulated_duration = 0.0
self.lock = asyncio.Lock()
self.MAX_BUFFER_DURATION = 3.0
async def process_and_send(self, new_msg, websocket, duration=0.0, force_flush=False):
"""
duration: 当前这段消息对应的音频时长(秒)
"""
async with self.lock:
# === 1. 强制清空 (连接断开时) ===
if force_flush:
if self.buffer:
logger.info(f"🧹 [最终清空] 准备推送 {len(self.buffer)} 条缓存...")
for msg in self.buffer:
text_content = msg.get("full_text", "")
logger.info(f" └── 🚀 发送: {text_content}") # 打印发送内容
await websocket.send(json.dumps(msg, ensure_ascii=False))
self.buffer = []
self.accumulated_duration = 0.0
return
# 获取当前消息的说话人
segments = new_msg.get("segments", [])
current_speaker = segments[0].get("spk") if segments else "unknown"
# === 2. 如果是同一个人（或者刚开始） ===
if self.last_speaker is None or current_speaker == self.last_speaker:
self.buffer.append(new_msg)
self.accumulated_duration += duration
self.last_speaker = current_speaker
# 🌟 检查是否超时 20s
if self.accumulated_duration >= self.MAX_BUFFER_DURATION:
logger.info(f"⏰ [超时推送] 同人 ({current_speaker}) 积攒 {self.accumulated_duration:.2f}s >= 8s")
for msg in self.buffer:
text_content = msg.get("full_text", "")
logger.info(f" └── 🚀 发送: {text_content}") # 打印发送内容
await websocket.send(json.dumps(msg, ensure_ascii=False))
# 清空缓存，重置时长
self.buffer = []
self.accumulated_duration = 0.0
else:
# 仅缓存，不发送
text_preview = new_msg.get("full_text", "")
logger.info(f"📥 [缓存中] Spk:{current_speaker} | 文本:{text_preview} | 累计:{self.accumulated_duration:.2f}s")
# === 3. 如果换人了 ===
else:
logger.info(f"🔄 [换人推送] {self.last_speaker} -> {current_speaker}")
# A. 推送上一位说话人的所有积攒消息
for msg in self.buffer:
text_content = msg.get("full_text", "")
logger.info(f" └── 🚀 发送: {text_content}") # 打印发送内容
await websocket.send(json.dumps(msg, ensure_ascii=False))
# B. 清空并开始缓存新人的消息
self.buffer = []
self.buffer.append(new_msg)
self.accumulated_duration = duration
self.last_speaker = current_speaker
# 打印新人入队日志
new_text = new_msg.get("full_text", "")
logger.info(f"📥 [新缓存] Spk:{current_speaker} | 文本:{new_text}")
# ========================
# 调用本地 ASR 接口（带说话人识别）
# ========================
async def call_local_asr_with_audio(wav_path: str, session: aiohttp.ClientSession):
try:
with open(wav_path, "rb") as f:
data = aiohttp.FormData()
data.add_field(
"audio_file",
f,
filename=os.path.basename(wav_path),
content_type="audio/wav"
)
params = {
"spk_diarization": "true",
"spk_threshold": "0.6",
"output": "json"
}
async with session.post(ASR_LOCAL_URL, data=data, params=params, timeout=aiohttp.ClientTimeout(total=60)) as resp:
if resp.status == 200:
return await resp.json()
else:
text = await resp.text()
logger.error(f"❌ 本地 ASR 接口错误: {resp.status} - {text}")
return None
except Exception as e:
logger.error(f"❌ 调用本地 ASR 异常: {e}")
return None
# ========================
# 后台处理：音频 → 本地 ASR → Session 缓存判断
# ========================
async def process_audio_with_local_asr(audio_bytes, asr_text, websocket, session, speaker_session):
if not audio_bytes:
return
wav_path = None
try:
# 1. 计算这段音频的时长，用于 Session 判断超时
audio_duration = len(audio_bytes) / BYTES_PER_SECOND
# 2. 保存为 WAV
with tempfile.NamedTemporaryFile(delete=False, suffix=".wav") as tmp:
wav_path = tmp.name
with wave.open(wav_path, 'wb') as wf:
wf.setnchannels(CHANNELS)
wf.setsampwidth(SAMPLE_WIDTH)
wf.setframerate(SAMPLE_RATE)
wf.writeframes(audio_bytes)
# 3. 调用本地 ASR 接口
asr_result = await call_local_asr_with_audio(wav_path, session)
if not asr_result or not asr_result.get("success"):
logger.error("❌ 本地 ASR (带声纹) 处理失败")
return
# 4. 构建消息
segments = asr_result.get("segments", [])
full_text = asr_result.get("text", "")
msg = {
"mode": "offline-speaker",
"full_text": full_text,
"segments": segments,
"original_asr_text": asr_text,
"type": "asr_with_speaker",
}
# 5. 🌟 传入 duration 参数
await speaker_session.process_and_send(msg, websocket, duration=audio_duration)
except websockets.exceptions.ConnectionClosed:
logger.warning("⚠️ WebSocket 连接已关闭")
except Exception as e:
logger.error(f"❌ 后台处理异常: {e}")
finally:
if wav_path and os.path.exists(wav_path):
os.unlink(wav_path)
# ========================
# 主 WebSocket 转发逻辑
# ========================
async def forward_audio(websocket):
logger.info("🟢 前端已连接。")
current_audio_chunks = []
local_accumulated_audio = bytearray()
speaker_session = SpeakerSession()
lock = asyncio.Lock()
async with aiohttp.ClientSession() as session:
try:
async with websockets.connect(TARGET_WS_URL) as target_ws:
logger.info(f"🔗 连接到远程 ASR: {TARGET_WS_URL}")
await target_ws.send(json.dumps(config))
# ---------------------------
# 任务1: 前端 -> 远程 ASR
# ---------------------------
async def frontend_to_target():
async for message in websocket:
if isinstance(message, bytes):
async with lock:
current_audio_chunks.append(message)
await target_ws.send(message)
else:
await target_ws.send(message)
# ---------------------------
# 任务2: 远程 ASR -> 前端
# ---------------------------
async def target_to_frontend():
async for message in target_ws:
# 实时推送
await websocket.send(message)
if isinstance(message, str):
try:
data = json.loads(message)
mode = data.get("mode")
if mode == "2pass-offline":
asr_text = data.get("text", "").strip()
async with lock:
chunk_bytes = b"".join(current_audio_chunks)
current_audio_chunks.clear()
local_accumulated_audio.extend(chunk_bytes)
current_duration = len(local_accumulated_audio) / BYTES_PER_SECOND
if current_duration < MIN_DURATION_THRESHOLD:
logger.info(f"⏳ 音频拼接中: {current_duration:.2f}s")
else:
logger.info(f"⚡ 触发本地声纹识别: {current_duration:.2f}s")
audio_to_send = bytes(local_accumulated_audio)
local_accumulated_audio.clear()
asyncio.create_task(
process_audio_with_local_asr(audio_to_send, asr_text, websocket, session, speaker_session)
)
elif mode == "2pass-online":
pass
except json.JSONDecodeError:
pass
await asyncio.gather(frontend_to_target(), target_to_frontend())
except websockets.exceptions.ConnectionClosedOK:
logger.info("🔗 远程 ASR 连接正常关闭。")
except Exception as e:
logger.error(f"❌ 连接异常: {e}")
finally:
logger.info("🔴 前端断开连接，清理资源...")
if len(local_accumulated_audio) > 0:
final_duration = len(local_accumulated_audio) / BYTES_PER_SECOND
logger.info(f"⚠️ 处理断开前的剩余音频: {final_duration:.2f}s")
try:
audio_to_send = bytes(local_accumulated_audio)
local_accumulated_audio.clear()
await process_audio_with_local_asr(audio_to_send, "Final-Flush", websocket, session, speaker_session)
except Exception as e:
logger.error(f"❌ 剩余音频处理异常: {e}")
# 强制推送最后残留缓存
try:
await speaker_session.process_and_send(None, websocket, force_flush=True)
except Exception as e:
logger.error(f"❌ 最终缓存推送失败: {e}")
# ========================
# 启动服务
# ========================
async def main():
logger.info(f"🚀 启动 WebSocket 中转服务: ws://{LOCAL_HOST}:{LOCAL_PORT}")
logger.info("📝 逻辑: 声纹识别缓存，同人累积满8s或换人时推送")
server = await websockets.serve(forward_audio, LOCAL_HOST, LOCAL_PORT)
await server.wait_closed()
if __name__ == "__main__":
asyncio.run(main())

317
fastapi_wss/src/speaker_det.py.bak-202607150815 Normal file
View File

@ -0,0 +1,317 @@
import asyncio
import json
import os
import tempfile
import wave
import websockets
import aiohttp
import logging
from dotenv import load_dotenv
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
force=True
)
logger = logging.getLogger("app")
# ========================
# 配置
# ========================
load_dotenv()
logger.info("正在加载环境变量 ")
TARGET_WS_URL = os.getenv("TARGET_WS_URL", "ws://172.18.127.124:10096/")
LOCAL_HOST = "0.0.0.0"
LOCAL_PORT = 10095
ASR_LOCAL_URL = os.getenv("ASR_LOCAL_URL", "http://172.18.127.124:6688/asr")
SAMPLE_RATE = 16000
SAMPLE_WIDTH = 2 # 16-bit = 2 bytes
CHANNELS = 1
# 计算每秒的字节数
BYTES_PER_SECOND = SAMPLE_RATE * SAMPLE_WIDTH * CHANNELS
# 设定最小触发时长（秒）
MIN_DURATION_THRESHOLD = 3.0
config = {
"chunk_size": [10, 10, 10],
"wav_name": "h5",
"is_speaking": True,
"wav_format": "pcm",
"chunk_interval": 10,
"itn": True,
"mode": "2pass",
"hotwords": "",
}
# ========================
# 新增：声纹会话缓存管理类 (带超时机制 + 日志)
# ========================
class SpeakerSession:
def __init__(self):
self.last_speaker = None
self.buffer = []
self.accumulated_duration = 0.0
self.lock = asyncio.Lock()
self.MAX_BUFFER_DURATION = 3.0
async def process_and_send(self, new_msg, websocket, duration=0.0, force_flush=False):
"""
duration: 当前这段消息对应的音频时长(秒)
"""
async with self.lock:
# === 1. 强制清空 (连接断开时) ===
if force_flush:
if self.buffer:
logger.info(f"🧹 [最终清空] 准备推送 {len(self.buffer)} 条缓存...")
for msg in self.buffer:
text_content = msg.get("full_text", "")
logger.info(f" └── 🚀 发送: {text_content}") # 打印发送内容
await websocket.send(json.dumps(msg, ensure_ascii=False))
self.buffer = []
self.accumulated_duration = 0.0
return
# 获取当前消息的说话人
segments = new_msg.get("segments", [])
current_speaker = segments[0].get("spk") if segments else "unknown"
# === 2. 如果是同一个人（或者刚开始） ===
if self.last_speaker is None or current_speaker == self.last_speaker:
self.buffer.append(new_msg)
self.accumulated_duration += duration
self.last_speaker = current_speaker
# 🌟 检查是否超时 20s
if self.accumulated_duration >= self.MAX_BUFFER_DURATION:
logger.info(f"⏰ [超时推送] 同人 ({current_speaker}) 积攒 {self.accumulated_duration:.2f}s >= 8s")
for msg in self.buffer:
text_content = msg.get("full_text", "")
logger.info(f" └── 🚀 发送: {text_content}") # 打印发送内容
await websocket.send(json.dumps(msg, ensure_ascii=False))
# 清空缓存，重置时长
self.buffer = []
self.accumulated_duration = 0.0
else:
# 仅缓存，不发送
text_preview = new_msg.get("full_text", "")
logger.info(f"📥 [缓存中] Spk:{current_speaker} | 文本:{text_preview} | 累计:{self.accumulated_duration:.2f}s")
# === 3. 如果换人了 ===
else:
logger.info(f"🔄 [换人推送] {self.last_speaker} -> {current_speaker}")
# A. 推送上一位说话人的所有积攒消息
for msg in self.buffer:
text_content = msg.get("full_text", "")
logger.info(f" └── 🚀 发送: {text_content}") # 打印发送内容
await websocket.send(json.dumps(msg, ensure_ascii=False))
# B. 清空并开始缓存新人的消息
self.buffer = []
self.buffer.append(new_msg)
self.accumulated_duration = duration
self.last_speaker = current_speaker
# 打印新人入队日志
new_text = new_msg.get("full_text", "")
logger.info(f"📥 [新缓存] Spk:{current_speaker} | 文本:{new_text}")
# ========================
# 调用本地 ASR 接口（带说话人识别）
# ========================
async def call_local_asr_with_audio(wav_path: str, session: aiohttp.ClientSession):
try:
with open(wav_path, "rb") as f:
data = aiohttp.FormData()
data.add_field(
"audio_file",
f,
filename=os.path.basename(wav_path),
content_type="audio/wav"
)
params = {
"spk_diarization": "true",
"spk_threshold": "0.6",
"output": "json"
}
async with session.post(ASR_LOCAL_URL, data=data, params=params, timeout=aiohttp.ClientTimeout(total=60)) as resp:
if resp.status == 200:
return await resp.json()
else:
text = await resp.text()
logger.error(f"❌ 本地 ASR 接口错误: {resp.status} - {text}")
return None
except Exception as e:
logger.error(f"❌ 调用本地 ASR 异常: {e}")
return None
# ========================
# 后台处理：音频 → 本地 ASR → Session 缓存判断
# ========================
async def process_audio_with_local_asr(audio_bytes, asr_text, websocket, session, speaker_session):
if not audio_bytes:
return
wav_path = None
try:
# 1. 计算这段音频的时长，用于 Session 判断超时
audio_duration = len(audio_bytes) / BYTES_PER_SECOND
# 2. 保存为 WAV
with tempfile.NamedTemporaryFile(delete=False, suffix=".wav") as tmp:
wav_path = tmp.name
with wave.open(wav_path, 'wb') as wf:
wf.setnchannels(CHANNELS)
wf.setsampwidth(SAMPLE_WIDTH)
wf.setframerate(SAMPLE_RATE)
wf.writeframes(audio_bytes)
# 3. 调用本地 ASR 接口
asr_result = await call_local_asr_with_audio(wav_path, session)
if not asr_result or not asr_result.get("success"):
logger.error("❌ 本地 ASR (带声纹) 处理失败")
return
# 4. 构建消息
segments = asr_result.get("segments", [])
full_text = asr_result.get("text", "")
msg = {
"mode": "offline-speaker",
"full_text": full_text,
"segments": segments,
"original_asr_text": asr_text,
"type": "asr_with_speaker",
}
# 5. 🌟 传入 duration 参数
await speaker_session.process_and_send(msg, websocket, duration=audio_duration)
except websockets.exceptions.ConnectionClosed:
logger.warning("⚠️ WebSocket 连接已关闭")
except Exception as e:
logger.error(f"❌ 后台处理异常: {e}")
finally:
if wav_path and os.path.exists(wav_path):
os.unlink(wav_path)
# ========================
# 主 WebSocket 转发逻辑
# ========================
async def forward_audio(websocket):
logger.info("🟢 前端已连接。")
current_audio_chunks = []
local_accumulated_audio = bytearray()
speaker_session = SpeakerSession()
lock = asyncio.Lock()
async with aiohttp.ClientSession() as session:
try:
async with websockets.connect(TARGET_WS_URL) as target_ws:
logger.info(f"🔗 连接到远程 ASR: {TARGET_WS_URL}")
await target_ws.send(json.dumps(config))
# ---------------------------
# 任务1: 前端 -> 远程 ASR
# ---------------------------
async def frontend_to_target():
async for message in websocket:
if isinstance(message, bytes):
async with lock:
current_audio_chunks.append(message)
await target_ws.send(message)
else:
await target_ws.send(message)
# ---------------------------
# 任务2: 远程 ASR -> 前端
# ---------------------------
async def target_to_frontend():
async for message in target_ws:
# 实时推送
await websocket.send(message)
if isinstance(message, str):
try:
data = json.loads(message)
mode = data.get("mode")
if mode == "2pass-offline":
asr_text = data.get("text", "").strip()
async with lock:
chunk_bytes = b"".join(current_audio_chunks)
current_audio_chunks.clear()
local_accumulated_audio.extend(chunk_bytes)
current_duration = len(local_accumulated_audio) / BYTES_PER_SECOND
if current_duration < MIN_DURATION_THRESHOLD:
logger.info(f"⏳ 音频拼接中: {current_duration:.2f}s")
else:
logger.info(f"⚡ 触发本地声纹识别: {current_duration:.2f}s")
audio_to_send = bytes(local_accumulated_audio)
local_accumulated_audio.clear()
asyncio.create_task(
process_audio_with_local_asr(audio_to_send, asr_text, websocket, session, speaker_session)
)
elif mode == "2pass-online":
pass
except json.JSONDecodeError:
pass
await asyncio.gather(frontend_to_target(), target_to_frontend())
except websockets.exceptions.ConnectionClosedOK:
logger.info("🔗 远程 ASR 连接正常关闭。")
except Exception as e:
logger.error(f"❌ 连接异常: {e}")
finally:
logger.info("🔴 前端断开连接，清理资源...")
if len(local_accumulated_audio) > 0:
final_duration = len(local_accumulated_audio) / BYTES_PER_SECOND
logger.info(f"⚠️ 处理断开前的剩余音频: {final_duration:.2f}s")
try:
audio_to_send = bytes(local_accumulated_audio)
local_accumulated_audio.clear()
await process_audio_with_local_asr(audio_to_send, "Final-Flush", websocket, session, speaker_session)
except Exception as e:
logger.error(f"❌ 剩余音频处理异常: {e}")
# 强制推送最后残留缓存
try:
await speaker_session.process_and_send(None, websocket, force_flush=True)
except Exception as e:
logger.error(f"❌ 最终缓存推送失败: {e}")
# ========================
# 启动服务
# ========================
async def main():
logger.info(f"🚀 启动 WebSocket 中转服务: ws://{LOCAL_HOST}:{LOCAL_PORT}")
logger.info("📝 逻辑: 声纹识别缓存，同人累积满8s或换人时推送")
server = await websockets.serve(forward_audio, LOCAL_HOST, LOCAL_PORT)
await server.wait_closed()
if __name__ == "__main__":
asyncio.run(main())

199
fastapi_wss/src/test.py Normal file
View File

@ -0,0 +1,199 @@
import asyncio
import audioop
from array import array
import json
import sys
import time
import wave
import websockets
if hasattr(sys.stdout, "reconfigure"):
sys.stdout.reconfigure(encoding="utf-8")
if hasattr(sys.stderr, "reconfigure"):
sys.stderr.reconfigure(encoding="utf-8")
WS_URL = "ws://localhost:10095"
TARGET_SAMPLE_RATE = 16000
TARGET_SAMPLE_WIDTH = 2
CHUNK_SECONDS = 0.06
FINAL_RESULT_TIMEOUT = 60
WAV_PATH = r"E:\ZKYNLP\gonwgendoc\fastapi_wss\fastapi_wss\src\多人会议90分钟.wav"
def downmix_to_mono_pcm16(chunk: bytes, channels: int) -> bytes:
if channels == 1:
return chunk
samples = array("h")
samples.frombytes(chunk)
if sys.byteorder != "little":
samples.byteswap()
mono = array("h")
for index in range(0, len(samples), channels):
frame = samples[index:index + channels]
if len(frame) == channels:
mono.append(int(sum(frame) / channels))
if sys.byteorder != "little":
mono.byteswap()
return mono.tobytes()
def validate_wav_for_streaming(wf: wave.Wave_read):
sample_rate = wf.getframerate()
sample_width = wf.getsampwidth()
channels = wf.getnchannels()
if sample_width != TARGET_SAMPLE_WIDTH:
raise ValueError(f"WAV sample width must be {TARGET_SAMPLE_WIDTH} bytes, got {sample_width} bytes")
if channels < 1:
raise ValueError(f"WAV channels must be >= 1, got {channels}")
if sample_rate != TARGET_SAMPLE_RATE:
print(f"Input WAV is {sample_rate}Hz; resampling to {TARGET_SAMPLE_RATE}Hz before sending.")
if channels > 1:
print(f"Input WAV has {channels} channels; downmixing to mono before sending.")
source_frames_per_chunk = max(1, int(sample_rate * CHUNK_SECONDS))
return sample_rate, channels, source_frames_per_chunk
class Pcm16Mono16kConverter:
def __init__(self, source_rate: int, channels: int):
self.source_rate = source_rate
self.channels = channels
self.rate_state = None
def convert(self, chunk: bytes) -> bytes:
chunk = downmix_to_mono_pcm16(chunk, self.channels)
if self.source_rate != TARGET_SAMPLE_RATE:
chunk, self.rate_state = audioop.ratecv(
chunk,
TARGET_SAMPLE_WIDTH,
1,
self.source_rate,
TARGET_SAMPLE_RATE,
self.rate_state,
)
return chunk
def is_final_asr_message(data: dict) -> bool:
mode = data.get("mode")
if mode not in {"2pass-offline", "offline-speaker"}:
return False
return bool(data.get("is_final", True))
async def recv_loop(ws, audio_sent_event: asyncio.Event, final_result_event: asyncio.Event, stats: dict):
async for msg in ws:
print("RECV:", msg)
if not isinstance(msg, str):
continue
try:
data = json.loads(msg)
except json.JSONDecodeError:
continue
if not is_final_asr_message(data):
continue
now = time.perf_counter()
stats["last_final_time"] = now
if audio_sent_event.is_set() and "first_final_after_send_time" not in stats:
stats["first_final_after_send_time"] = now
final_result_event.set()
def print_timing_stats(stats: dict):
end_time = stats.get("first_final_after_send_time") or stats.get("end_time") or time.perf_counter()
total_elapsed = end_time - stats["start_time"]
send_elapsed = stats.get("send_end_time", end_time) - stats.get("send_start_time", stats["start_time"])
tail_elapsed = None
if "send_end_time" in stats and "first_final_after_send_time" in stats:
tail_elapsed = stats["first_final_after_send_time"] - stats["send_end_time"]
print("\n===== ASR Timing =====")
print(f"Audio duration: {stats.get('audio_duration_sec', 0.0):.2f}s")
print(f"Audio send time: {send_elapsed:.2f}s")
if tail_elapsed is not None:
print(f"Post-send ASR wait: {tail_elapsed:.2f}s")
else:
print("Post-send ASR wait: timeout/no final result")
print(f"Total elapsed: {total_elapsed:.2f}s")
if stats.get("audio_duration_sec"):
rtf = total_elapsed / stats["audio_duration_sec"]
print(f"RTF: {rtf:.3f}x")
print("======================\n")
async def main():
stats = {"start_time": time.perf_counter()}
audio_sent_event = asyncio.Event()
final_result_event = asyncio.Event()
async with websockets.connect(
WS_URL,
max_size=None,
ping_interval=None,
ping_timeout=None,
) as ws:
recv_task = asyncio.create_task(recv_loop(ws, audio_sent_event, final_result_event, stats))
init = {
"mode": "2pass",
"chunk_size": [10, 10, 10],
"chunk_interval": 10,
"wav_name": "test",
"is_speaking": True,
"wav_format": "pcm",
"itn": True
}
await ws.send(json.dumps(init, ensure_ascii=False))
connection_closed = False
with wave.open(WAV_PATH, "rb") as wf:
source_rate, channels, source_frames_per_chunk = validate_wav_for_streaming(wf)
stats["audio_duration_sec"] = wf.getnframes() / source_rate
converter = Pcm16Mono16kConverter(source_rate, channels)
stats["send_start_time"] = time.perf_counter()
while True:
chunk = wf.readframes(source_frames_per_chunk)
if not chunk:
break
chunk = converter.convert(chunk)
try:
await ws.send(chunk)
except websockets.exceptions.ConnectionClosed as e:
connection_closed = True
print(f"WebSocket closed while sending audio: {type(e).__name__}: {e}")
break
await asyncio.sleep(CHUNK_SECONDS)
if not connection_closed:
try:
await ws.send(json.dumps({"is_speaking": False}, ensure_ascii=False))
except websockets.exceptions.ConnectionClosed as e:
connection_closed = True
print(f"WebSocket closed before sending end marker: {type(e).__name__}: {e}")
stats["send_end_time"] = time.perf_counter()
audio_sent_event.set()
if connection_closed and not final_result_event.is_set():
print("Connection closed before final ASR result was received.")
stats["end_time"] = time.perf_counter()
print_timing_stats(stats)
else:
try:
await asyncio.wait_for(final_result_event.wait(), timeout=FINAL_RESULT_TIMEOUT)
except asyncio.TimeoutError:
print(f"Timed out waiting for final ASR result after {FINAL_RESULT_TIMEOUT}s.")
finally:
stats["end_time"] = time.perf_counter()
print_timing_stats(stats)
recv_task.cancel()
await asyncio.gather(recv_task, return_exceptions=True)
asyncio.run(main())

1
fastapi_wss/test.json Normal file
View File

File diff suppressed because one or more lines are too long

749
funasr_runtime/funasr_wss_server.py Normal file
View File

@ -0,0 +1,749 @@
import asyncio
import json
import websockets
import time
import numpy as np
import argparse
import ssl
import os
import wave
import functools
from concurrent.futures import ThreadPoolExecutor
from scipy.spatial.distance import cosine
import torch # 保留不影响
def to_python(obj):
"""递归地把 numpy / torch 等类型转成纯 Python，可 JSON 序列化。"""
try:
import numpy as np # noqa
import torch # noqa
except Exception:
np = None
torch = None
if np is not None and isinstance(obj, np.generic):
return obj.item()
if np is not None and isinstance(obj, np.ndarray):
return obj.tolist()
if torch is not None and isinstance(obj, torch.Tensor):
return obj.cpu().tolist()
if isinstance(obj, dict):
return {k: to_python(v) for k, v in obj.items()}
if isinstance(obj, (list, tuple)):
return [to_python(v) for v in obj]
return obj
parser = argparse.ArgumentParser()
parser.add_argument("--host", type=str, default="0.0.0.0", required=False, help="host ip")
parser.add_argument("--port", type=int, default=10095, required=False, help="grpc server port")
parser.add_argument(
"--asr_model",
type=str,
default="iic/speech_paraformer-large-contextual_asr_nat-zh-cn-16k-common-vocab8404",
help="model from modelscope",
)
parser.add_argument("--asr_model_revision", type=str, default="v2.0.4", help="")
parser.add_argument(
"--asr_model_online",
type=str,
default="iic/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-online",
help="model from modelscope",
)
parser.add_argument("--asr_model_online_revision", type=str, default="v2.0.4", help="")
parser.add_argument(
"--vad_model",
type=str,
default="iic/speech_fsmn_vad_zh-cn-16k-common-pytorch",
help="model from modelscope",
)
parser.add_argument("--vad_model_revision", type=str, default="v2.0.4", help="")
parser.add_argument(
"--punc_model",
type=str,
default="iic/punc_ct-transformer_zh-cn-common-vad_realtime-vocab272727",
help="model from modelscope",
)
parser.add_argument("--punc_model_revision", type=str, default="v2.0.4", help="")
parser.add_argument("--ngpu", type=int, default=1, help="0 for cpu, 1 for gpu")
parser.add_argument("--device", type=str, default="cuda", help="cuda, cpu")
parser.add_argument("--ncpu", type=int, default=4, help="cpu cores")
parser.add_argument(
"--certfile",
type=str,
default="../../ssl_key/server.crt",
required=False,
help="certfile for ssl",
)
parser.add_argument(
"--keyfile",
type=str,
default="../../ssl_key/server.key",
required=False,
help="keyfile for ssl",
)
# ====== 保存 2pass 离线阶段送入 ASR 的音频片段（排查 VAD 切分）======
parser.add_argument(
"--save_offline_segments",
action="store_true",
help="Save each offline (2pass) audio segment sent to offline ASR as wav for debugging VAD split.",
)
parser.add_argument(
"--save_offline_segments_dir",
type=str,
default="./offline_segments",
help="Directory to save offline wav segments when --save_offline_segments is enabled.",
)
# ====== 并发控制：核心新增 ======
parser.add_argument(
"--worker_threads",
type=int,
default=max(4, (os.cpu_count() or 4)),
help="ThreadPoolExecutor max_workers. Used to offload blocking inference so event loop won't be blocked.",
)
parser.add_argument("--concurrent_vad", type=int, default=4, help="Max concurrent VAD generate() calls.")
parser.add_argument("--concurrent_asr_online", type=int, default=4, help="Max concurrent streaming ASR generate() calls.")
parser.add_argument("--concurrent_asr_offline", type=int, default=2, help="Max concurrent offline ASR generate() calls.")
parser.add_argument("--concurrent_punc", type=int, default=1, help="Max concurrent punctuation generate() calls.")
parser.add_argument("--concurrent_sv", type=int, default=1, help="Max concurrent speaker verification generate() calls.")
parser.add_argument(
"--speaker_db_reload_sec",
type=int,
default=5,
help="Reload speaker_db.json at most once every N seconds (avoid frequent disk IO).",
)
args = parser.parse_args()
websocket_users = set()
SPEAKER_DB_PATH = os.path.join(os.path.dirname(__file__), "speaker_db.json")
def _ensure_dir(p: str):
try:
os.makedirs(p, exist_ok=True)
except Exception:
pass
def _pcm_duration_ms(pcm_bytes: bytes, fs: int, ch: int = 1, sampwidth: int = 2) -> int:
"""根据 fs/ch/sampwidth 计算 PCM 时长，避免写死 16k -> 32 bytes/ms。"""
if not pcm_bytes:
return 0
bytes_per_ms = (fs * ch * sampwidth) / 1000.0
if bytes_per_ms <= 0:
return 0
return int(len(pcm_bytes) / bytes_per_ms)
def _safe_int(v, default):
try:
return int(v)
except Exception:
return default
# ========= speaker db：加缓存，避免每段都读盘 =========
_SPEAKER_DB_CACHE = {}
_SPEAKER_DB_CACHE_TS = 0.0
def _load_speaker_db_sync():
if not os.path.exists(SPEAKER_DB_PATH):
return {}
try:
with open(SPEAKER_DB_PATH, "r", encoding="utf-8") as f:
data = json.load(f)
return data if isinstance(data, dict) else {}
except Exception:
return {}
def get_speaker_db_cached(now_ts: float, reload_sec: int):
global _SPEAKER_DB_CACHE, _SPEAKER_DB_CACHE_TS
if (now_ts - _SPEAKER_DB_CACHE_TS) >= max(1, int(reload_sec)):
_SPEAKER_DB_CACHE = _load_speaker_db_sync()
_SPEAKER_DB_CACHE_TS = now_ts
return _SPEAKER_DB_CACHE or {}
def _save_wav_sync(out_path: str, audio_bytes: bytes, fs: int, ch: int, sampwidth: int):
with wave.open(out_path, "wb") as wf:
wf.setnchannels(ch)
wf.setsampwidth(sampwidth)
wf.setframerate(fs)
wf.writeframes(audio_bytes)
def save_offline_wav_segment_sync(websocket, audio_bytes: bytes, reason: str = "offline"):
"""
保存离线阶段送入 ASR 的音频片段，方便人工试听排查 VAD 切分是否正确。
约定：audio_bytes 为 单声道 PCM16 little-endian（默认 16k）。
（注意：这是同步函数，外层会放线程池执行）
"""
if not getattr(websocket, "save_offline_segments", False):
return
if "2pass" not in (getattr(websocket, "mode", "") or ""):
return
if not audio_bytes:
return
fs = int(getattr(websocket, "audio_fs", 16000) or 16000)
ch = 1
sampwidth = 2 # int16
# int16 对齐
if len(audio_bytes) % 2 == 1:
audio_bytes = audio_bytes[:-1]
if not audio_bytes:
return
seg_idx = int(getattr(websocket, "offline_seg_idx", 0))
websocket.offline_seg_idx = seg_idx + 1
duration_ms = _pcm_duration_ms(audio_bytes, fs=fs, ch=ch, sampwidth=sampwidth)
base_dir = getattr(websocket, "offline_save_dir", args.save_offline_segments_dir)
_ensure_dir(base_dir)
wav_name = (getattr(websocket, "wav_name", "microphone") or "microphone").replace("/", "_")
ts = int(time.time() * 1000)
fname = f"{wav_name}_{ts}_seg{seg_idx:04d}_{reason}_{duration_ms}ms.wav"
out_path = os.path.join(base_dir, fname)
try:
_save_wav_sync(out_path, audio_bytes, fs=fs, ch=ch, sampwidth=sampwidth)
print(f"[SAVE_OFFLINE_SEG] {out_path} ({duration_ms} ms, {len(audio_bytes)} bytes)")
except Exception as e:
print(f"[SAVE_OFFLINE_SEG] failed: {e}")
print("model loading")
from funasr import AutoModel # noqa
# ====== 离线 ASR ======
model_asr = AutoModel(
model="paraformer-zh",
model_revision="v2.0.4",
ngpu=args.ngpu,
ncpu=args.ncpu,
device=args.device,
disable_pbar=True,
disable_log=True,
)
# streaming asr
model_asr_streaming = AutoModel(
model=args.asr_model_online,
model_revision=args.asr_model_online_revision,
ngpu=args.ngpu,
ncpu=args.ncpu,
device=args.device,
disable_pbar=True,
disable_log=True,
)
# vad
model_vad = AutoModel(
model=args.vad_model,
model_revision=args.vad_model_revision,
ngpu=args.ngpu,
ncpu=args.ncpu,
device=args.device,
disable_pbar=True,
disable_log=True,
)
# punc
if args.punc_model != "":
model_punc = AutoModel(
model=args.punc_model,
model_revision=args.punc_model_revision,
ngpu=args.ngpu,
ncpu=args.ncpu,
device=args.device,
disable_pbar=True,
disable_log=True,
)
else:
model_punc = None
# sv
model_sv = AutoModel(
model="iic/speech_campplus_sv_zh-cn_16k-common",
ngpu=args.ngpu,
device=args.device,
disable_pbar=True,
disable_log=True,
)
print("model loaded! (now supports multi-client with non-blocking inference)")
# ====== 线程池 + 并发阈值（核心）======
EXECUTOR = ThreadPoolExecutor(max_workers=int(args.worker_threads))
SEM_VAD = asyncio.Semaphore(max(1, int(args.concurrent_vad)))
SEM_ASR_ONLINE = asyncio.Semaphore(max(1, int(args.concurrent_asr_online)))
SEM_ASR_OFFLINE = asyncio.Semaphore(max(1, int(args.concurrent_asr_offline)))
SEM_PUNC = asyncio.Semaphore(max(1, int(args.concurrent_punc)))
SEM_SV = asyncio.Semaphore(max(1, int(args.concurrent_sv)))
SEM_WAV = asyncio.Semaphore(max(1, 4)) # 保存 wav 一般不需要太大
async def run_blocking(fn, *a, sem: asyncio.Semaphore | None = None, **kw):
"""
把阻塞函数丢线程池执行，避免卡 event loop。
sem 用于限流（避免 GPU / 模型被打爆）。
"""
loop = asyncio.get_running_loop()
call = functools.partial(fn, *a, **kw)
if sem is None:
return await loop.run_in_executor(EXECUTOR, call)
async with sem:
return await loop.run_in_executor(EXECUTOR, call)
def _generate_sync(model, audio_or_text, status_dict):
# 注意：status_dict 里包含 cache，会被 generate 更新
return model.generate(input=audio_or_text, **status_dict)
async def ws_reset(websocket):
print("ws reset now, total num is ", len(websocket_users))
websocket.status_dict_asr_online["cache"] = {}
websocket.status_dict_asr_online["is_final"] = True
websocket.status_dict_vad["cache"] = {}
websocket.status_dict_vad["is_final"] = True
websocket.status_dict_punc["cache"] = {}
await websocket.close()
async def clear_websocket():
for websocket in list(websocket_users):
await ws_reset(websocket)
websocket_users.clear()
async def ws_serve(websocket, path=None):
# websockets 新版本不会传 path，这里做兼容
if path is None:
path = getattr(websocket, "path", None)
frames = []
frames_asr = []
frames_asr_online = []
global websocket_users
websocket_users.add(websocket)
websocket.status_dict_asr = {} # hotword 等
websocket.status_dict_asr_online = {"cache": {}, "is_final": False}
websocket.status_dict_vad = {"cache": {}, "is_final": False}
websocket.status_dict_punc = {"cache": {}}
websocket.chunk_interval = 10
websocket.vad_pre_idx = 0
speech_start = False
speech_end_i = -1
websocket.wav_name = "microphone"
websocket.mode = "2pass"
websocket.is_speaking = True # ✅ 默认初始化，避免 AttributeError
# 保存离线片段
websocket.audio_fs = 16000
websocket.offline_seg_idx = 0
websocket.save_offline_segments = bool(args.save_offline_segments)
websocket.offline_save_dir = args.save_offline_segments_dir
if websocket.save_offline_segments:
_ensure_dir(websocket.offline_save_dir)
print(f"[SAVE_OFFLINE_SEG] enabled, dir={websocket.offline_save_dir}")
print("new user connected", flush=True)
try:
async for message in websocket:
# ========== 1) 先处理“文本配置消息” ==========
if isinstance(message, str):
try:
messagejson = json.loads(message)
except Exception as e:
print("bad json message:", e, message[:200])
continue
print("=============messagejson============", messagejson)
if "is_speaking" in messagejson:
websocket.is_speaking = bool(messagejson["is_speaking"])
websocket.status_dict_asr_online["is_final"] = (not websocket.is_speaking)
if "chunk_interval" in messagejson:
websocket.chunk_interval = _safe_int(
messagejson["chunk_interval"], websocket.chunk_interval
)
if "wav_name" in messagejson:
websocket.wav_name = messagejson.get("wav_name") or websocket.wav_name
if "chunk_size" in messagejson:
chunk_size = messagejson["chunk_size"]
if isinstance(chunk_size, str):
chunk_size = [x.strip() for x in chunk_size.split(",") if x.strip()]
websocket.status_dict_asr_online["chunk_size"] = [int(x) for x in chunk_size]
if "encoder_chunk_look_back" in messagejson:
websocket.status_dict_asr_online["encoder_chunk_look_back"] = messagejson[
"encoder_chunk_look_back"
]
if "decoder_chunk_look_back" in messagejson:
websocket.status_dict_asr_online["decoder_chunk_look_back"] = messagejson[
"decoder_chunk_look_back"
]
if "hotwords" in messagejson:
hotword_data = messagejson["hotwords"]
websocket.status_dict_asr["hotword"] = hotword_data
websocket.status_dict_asr_online["hotword"] = hotword_data
print(f"热词已更新: {hotword_data}")
if "mode" in messagejson:
websocket.mode = messagejson["mode"] or websocket.mode
if "audio_fs" in messagejson:
websocket.audio_fs = _safe_int(messagejson["audio_fs"], 16000)
continue
# ========== 2) 处理“二进制音频消息” ==========
if "chunk_size" not in websocket.status_dict_asr_online:
print("[WARN] chunk_size not set yet, skip audio frame (send config first).")
continue
try:
websocket.status_dict_vad["chunk_size"] = int(
websocket.status_dict_asr_online["chunk_size"][1] * 60 / websocket.chunk_interval
)
except Exception as e:
print("[WARN] set vad chunk_size failed:", e)
continue
pcm = message
frames.append(pcm)
duration_ms = _pcm_duration_ms(pcm, fs=websocket.audio_fs, ch=1, sampwidth=2)
websocket.vad_pre_idx += duration_ms
# online asr
frames_asr_online.append(pcm)
websocket.status_dict_asr_online["is_final"] = (speech_end_i != -1)
if (len(frames_asr_online) % websocket.chunk_interval == 0) or websocket.status_dict_asr_online["is_final"]:
if websocket.mode in ("2pass", "online"):
audio_in = b"".join(frames_asr_online)
try:
await async_asr_online(websocket, audio_in)
except Exception:
print(f"error in asr streaming, {websocket.status_dict_asr_online}")
frames_asr_online = []
if speech_start:
frames_asr.append(pcm)
# vad online
try:
speech_start_i, speech_end_i = await async_vad(websocket, pcm)
except Exception as e:
print("error in vad:", e)
speech_start_i, speech_end_i = -1, -1
if speech_start_i != -1:
speech_start = True
if duration_ms > 0:
beg_bias = (websocket.vad_pre_idx - speech_start_i) // duration_ms
else:
beg_bias = 0
frames_pre = frames[-beg_bias:] if beg_bias > 0 else []
frames_asr = []
frames_asr.extend(frames_pre)
# ========== 3) 2pass：离线阶段触发点 ==========
if (speech_end_i != -1) or (not websocket.is_speaking):
if websocket.mode in ("2pass", "offline"):
audio_in = b"".join(frames_asr)
reason = "vad_end" if speech_end_i != -1 else "not_speaking"
# 保存 wav：放线程池，避免磁盘 IO 卡 loop
if websocket.save_offline_segments and audio_in:
try:
await run_blocking(
save_offline_wav_segment_sync,
websocket,
audio_in,
reason,
sem=SEM_WAV,
)
except Exception as e:
print("[SAVE_OFFLINE_SEG] async failed:", e)
try:
await async_asr(websocket, audio_in)
except Exception as e:
print("error in asr offline:", e)
frames_asr = []
speech_start = False
frames_asr_online = []
websocket.status_dict_asr_online["cache"] = {}
if not websocket.is_speaking:
websocket.vad_pre_idx = 0
frames = []
websocket.status_dict_vad["cache"] = {}
speech_end_i = -1
else:
frames = frames[-20:]
except websockets.ConnectionClosed:
print("ConnectionClosed...", websocket_users, flush=True)
await ws_reset(websocket)
if websocket in websocket_users:
websocket_users.remove(websocket)
except websockets.InvalidState:
print("InvalidState...")
except Exception as e:
print("Exception:", e)
try:
await ws_reset(websocket)
except Exception:
pass
if websocket in websocket_users:
websocket_users.remove(websocket)
# ===================== 推理：全部改为“线程池 + 限流” =====================
async def async_vad(websocket, audio_in: bytes):
# model_vad.generate 是阻塞的，必须 offload
out = await run_blocking(_generate_sync, model_vad, audio_in, websocket.status_dict_vad, sem=SEM_VAD)
segments_result = out[0].get("value", [])
speech_start = -1
speech_end = -1
if len(segments_result) == 0 or len(segments_result) > 1:
return speech_start, speech_end
if segments_result[0][0] != -1:
speech_start = segments_result[0][0]
if segments_result[0][1] != -1:
speech_end = segments_result[0][1]
return speech_start, speech_end
def _sv_and_match_sync(audio_in: bytes, reload_sec: int):
"""
同步执行：SV embedding + speaker_db 匹配
返回 (spk_name, best_score)
"""
spk_name = "unknown"
best_score = 0.0
sv_out = model_sv.generate(input=audio_in, embedding=True)[0]
embedding = sv_out["spk_embedding"][0].cpu().numpy()
now_ts = time.time()
local_speaker_db = get_speaker_db_cached(now_ts, reload_sec=reload_sec)
if local_speaker_db:
for name, ref_embedding in local_speaker_db.items():
if ref_embedding is None:
continue
arr = np.array(ref_embedding, dtype=np.float32)
similarity = 1.0 - cosine(embedding, arr)
print("sv similarity with {}: {}".format(name, similarity))
if similarity > best_score and similarity > 0.2:
best_score = similarity
spk_name = name
return spk_name, float(best_score)
async def async_asr(websocket, audio_in: bytes):
mode = "2pass-offline" if "2pass" in (websocket.mode or "") else websocket.mode
if len(audio_in) <= 0:
message = {
"mode": mode,
"text": "",
"wav_name": websocket.wav_name,
"is_final": True,
}
await websocket.send(json.dumps(message, ensure_ascii=False))
return
# 1) ASR（阻塞，线程池执行）
rec_result_list = await run_blocking(
_generate_sync,
model_asr,
audio_in,
websocket.status_dict_asr,
sem=SEM_ASR_OFFLINE,
)
rec_result = rec_result_list[0]
print("offline_asr, raw:", rec_result)
print("offline_asr, keys:", rec_result.keys())
text = rec_result.get("text", "")
timestamp = rec_result.get("timestamp", None)
sentence_info = rec_result.get("sentence_info", None)
# 2) 声纹识别（阻塞，线程池执行）
spk_name = "unknown"
best_score = 0.0
try:
spk_name, best_score = await run_blocking(
_sv_and_match_sync,
audio_in,
int(args.speaker_db_reload_sec),
sem=SEM_SV,
)
except Exception as e:
print(f"声纹识别失败: {e}")
# 3) 标点（阻塞，线程池执行）
punc_array = None
if model_punc is not None and len(text) > 0:
try:
# punc 只对文本处理
punc_out = await run_blocking(
_generate_sync,
model_punc,
text,
websocket.status_dict_punc,
sem=SEM_PUNC,
)
punc_result = punc_out[0]
print("offline, after punc", punc_result)
if "text" in punc_result and punc_result["text"]:
text = punc_result["text"]
if "punc_array" in punc_result:
punc_array = punc_result["punc_array"]
except Exception as e:
print("punc failed:", e)
# 4) 构造最终 message
if len(text) > 0:
print("======offline final text:", text)
message = {
"mode": mode,
"spk_name": spk_name,
"spk_score": float(best_score),
"text": text,
"wav_name": websocket.wav_name,
"is_final": True,
}
if timestamp is not None:
message["timestamp"] = to_python(timestamp)
if sentence_info is not None:
message["sentence_info"] = to_python(sentence_info)
if punc_array is not None:
message["punc_array"] = to_python(punc_array)
try:
await websocket.send(json.dumps(message, ensure_ascii=False))
except Exception as e:
print("send json failed:", e)
print("message types:", {k: type(v) for k, v in message.items()})
else:
message = {
"mode": mode,
"spk_name": spk_name,
"spk_score": float(best_score),
"text": "",
"wav_name": websocket.wav_name,
"is_final": True,
}
await websocket.send(json.dumps(message, ensure_ascii=False))
async def async_asr_online(websocket, audio_in: bytes):
if len(audio_in) <= 0:
return
# streaming generate 也是阻塞：线程池执行
rec_out = await run_blocking(
_generate_sync,
model_asr_streaming,
audio_in,
websocket.status_dict_asr_online,
sem=SEM_ASR_ONLINE,
)
rec_result = rec_out[0]
print("online, ", rec_result)
# 2pass：online 只要 partial，不发 final（final 交给 offline）
if websocket.mode == "2pass" and websocket.status_dict_asr_online.get("is_final", False):
return
if rec_result.get("text"):
mode = "2pass-online" if "2pass" in (websocket.mode or "") else websocket.mode
message = {
"mode": mode,
"text": rec_result["text"],
"wav_name": websocket.wav_name,
"is_final": bool(
websocket.status_dict_asr_online.get("is_final", False) or (not websocket.is_speaking)
),
}
await websocket.send(json.dumps(message, ensure_ascii=False))
# ===================== 启动服务 =====================
async def main():
if len(args.certfile) > 0:
ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
ssl_context.load_cert_chain(args.certfile, keyfile=args.keyfile)
server = await websockets.serve(
ws_serve,
args.host,
args.port,
subprotocols=["binary"],
ping_interval=None,
ssl=ssl_context,
)
else:
server = await websockets.serve(
ws_serve,
args.host,
args.port,
subprotocols=["binary"],
ping_interval=None,
)
print(f"WS server started at ws(s)://{args.host}:{args.port}")
await server.wait_closed()
if __name__ == "__main__":
try:
asyncio.run(main())
finally:
try:
EXECUTOR.shutdown(wait=False, cancel_futures=True)
except Exception:
pass

1
funasr_runtime/requirements_server.txt Normal file
View File

@ -0,0 +1 @@
websockets

24
speakr/.dockerignore Normal file
View File

@ -0,0 +1,24 @@
.git
.codex/
.agents/
.env
.env.*
__pycache__/
*.py[cod]
*.pyo
*.swp
*.swo
.pytest_cache/
.coverage
htmlcov/
logs/
node_modules/
templates/index/node_modules/
static/vite/
instance/
uploads/
*.db
*.log
.DS_Store
.idea/
.vscode/

3
speakr/.gitattributes vendored Normal file
View File

@ -0,0 +1,3 @@
# 保证 shell 脚本在任何平台检出时都是 LF，否则 Docker 镜像内脚本无法执行
*.sh text eol=lf
docker-entrypoint.sh text eol=lf

65
speakr/.gitignore vendored Normal file
View File

@ -0,0 +1,65 @@
# Python 字节码缓存
__pycache__/
*.py[cod]
*$py.class
*.pyo
# 虚拟环境
.venv/
venv/
env/
ENV/
# 环境变量（含敏感信息，禁止提交）
.env
.env.local
.env.*.local
# 日志文件
logs/
*.log
# 编辑器 / IDE
.idea/
.vscode/
*.swp
*.swo
.DS_Store
.trae/
.codex/
# 测试 / 覆盖率
.pytest_cache/
.coverage
htmlcov/
templates/index/node_modules/
# 构建产物
dist/
build/
*.egg-info/
static/vite/
skills/
## mypy
.mypy_cache/
# data
data/
static/vite/
instance/transcriptions.db
ai-guide/
音频web转文字前端示例/
# 本地测试脚本
tests/
#本地调试文件
*.backup
# 本地数据库文件
*.db
instance/transcriptions.db
node_modules/
instance/transcriptions.db
instance/transcriptions.db

137
speakr/.gitlab-ci.yml Normal file
View File

@ -0,0 +1,137 @@
# GitLab CI/CD 配置 for speakr 项目
# 自动构建并部署到 Kubernetes
stages:
- build
- deploy
variables:
# 使用阿里云镜像仓库
IMAGE_NAME: crpi-99azmmphmxwdoi76.cn-guangzhou.personal.cr.aliyuncs.com/sea_lee/speakr
IMAGE_TAG: $CI_COMMIT_SHORT_SHA
# ============================================
# 阶段 1: 构建 Docker 镜像并推送
# ============================================
build:
stage: build
image: docker:latest
services:
- docker:dind
before_script:
# 登录阿里云镜像仓库
- echo "$ALIYUN_REGISTRY_PASSWORD" | docker login crpi-99azmmphmxwdoi76.cn-guangzhou.personal.cr.aliyuncs.com -u "$ALIYUN_REGISTRY_USER" --password-stdin
script:
- echo "=========================================="
- echo "🚀 开始构建 speakr 镜像"
- echo "=========================================="
- echo "Git 提交: $CI_COMMIT_SHORT_SHA"
- echo "分支: $CI_COMMIT_BRANCH"
- echo "提交信息: $CI_COMMIT_TITLE"
- echo "提交者: $GITLAB_USER_NAME"
- echo ""
# 显示 Dockerfile 内容
- echo "Dockerfile:"
- cat Dockerfile
- echo ""
# 构建 Docker 镜像 (使用现有的 Dockerfile)
- echo "📦 构建 Docker 镜像..."
- docker build -t $IMAGE_NAME:$IMAGE_TAG .
- echo "✓ 镜像构建完成"
- docker images | grep speakr
# 打标签为 latest
- docker tag $IMAGE_NAME:$IMAGE_TAG $IMAGE_NAME:latest
# 推送到阿里云镜像仓库
- echo ""
- echo "📤 推送镜像到阿里云..."
- docker push $IMAGE_NAME:$IMAGE_TAG
- docker push $IMAGE_NAME:latest
- echo "✓ 镜像推送完成"
- echo ""
- echo "=========================================="
- echo "✅ 构建阶段完成!"
- echo "镜像: $IMAGE_NAME:$IMAGE_TAG"
- echo "=========================================="
only:
- main
- master
- develop
# ============================================
# 阶段 2: 部署到 Kubernetes
# ============================================
deploy:
stage: deploy
image: bitnami/kubectl:latest
environment:
name: production
url: http://172.18.127.124
before_script:
# 配置 kubectl
- echo "🔧 配置 kubectl..."
- echo "$KUBECONFIG_CONTENT" | base64 -d > /tmp/kubeconfig
- export KUBECONFIG=/tmp/kubeconfig
- chmod 600 /tmp/kubeconfig
# 验证 Kubernetes 连接
- echo ""
- echo "验证 Kubernetes 连接..."
- kubectl config current-context
- kubectl version --client
- kubectl get nodes
script:
- echo "=========================================="
- echo "🚀 部署 speakr 到 Kubernetes"
- echo "=========================================="
- echo "命名空间: default"
- echo "Deployment: speakr"
- echo "镜像: $IMAGE_NAME:$IMAGE_TAG"
- echo ""
# 检查 deployment 是否存在
- |
if kubectl get deployment speakr &>/dev/null; then
echo "✓ Deployment 已存在,更新镜像..."
kubectl set image deployment/speakr speakr=$IMAGE_NAME:$IMAGE_TAG
else
echo "✗ Deployment 不存在,创建新的..."
kubectl create deployment speakr --image=$IMAGE_NAME:$IMAGE_TAG --replicas=2
fi
# 等待 Deployment rollout 完成
- echo ""
- echo "⏳ 等待部署完成..."
- kubectl rollout status deployment/speakr --timeout=10m
# 确保服务存在
- |
if ! kubectl get service speakr &>/dev/null; then
echo "创建 Service..."
kubectl expose deployment speakr --port=8899 --target-port=8899 --type=NodePort
fi
# 显示最终状态
- echo ""
- echo "=========================================="
- echo "✅ 部署完成!"
- echo "=========================================="
- kubectl get pods -l app=speakr
- echo ""
- kubectl get service speakr
- echo ""
# 获取访问地址
- |
NODEPORT=$(kubectl get service speakr -o jsonpath='{.spec.ports[0].nodePort}')
echo "🌐 访问地址:"
echo " NodePort: http://172.18.127.124:$NODEPORT"
echo " 直接访问: http://172.18.127.124:8899"
echo ""
only:
- main
- master

62
speakr/Dockerfile Normal file
View File

@ -0,0 +1,62 @@
FROM node:20-slim AS frontend-builder
WORKDIR /build/templates/index
# 安装前端依赖，并在镜像构建阶段生成 Vite 产物。
COPY templates/index/package.json templates/index/pnpm-lock.yaml ./
RUN corepack enable && \
corepack prepare pnpm@9.15.9 --activate && \
pnpm install --frozen-lockfile
COPY templates/index/ ./
RUN rm -f pnpm-workspace.yaml && pnpm build
FROM python:3.11-slim
WORKDIR /app
# 安装运行时系统依赖。
RUN apt-get update && apt-get install -y --no-install-recommends \
gcc \
ffmpeg \
&& rm -rf /var/lib/apt/lists/*
# 先安装 Python 依赖，提高 Docker 层缓存命中率。
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# 先准备 vendor 目录，便于离线依赖下载脚本写入文件。
RUN mkdir -p /app/static/vendor
# 只复制下载脚本，避免业务代码变化导致 vendor 依赖重复下载。
COPY scripts/download_offline_deps.py scripts/
RUN pip install --no-cache-dir requests && \
python scripts/download_offline_deps.py
# 复制应用代码。
COPY . .
# 使用前端构建阶段生成的产物覆盖源码中的旧产物。
COPY --from=frontend-builder /build/static/vite/index /app/static/vite/index
# 创建容器运行时数据目录。
RUN mkdir -p /data/uploads /data/instance && \
chmod 755 /data/uploads /data/instance
# 设置默认运行环境变量。
ENV FLASK_APP=src/app.py
ENV SQLALCHEMY_DATABASE_URI=sqlite:////data/instance/transcriptions.db
ENV UPLOAD_FOLDER=/data/uploads
ENV PYTHONPATH=/app
# 安装容器入口脚本。
COPY scripts/docker-entrypoint.sh /usr/local/bin/
RUN chmod +x /usr/local/bin/docker-entrypoint.sh
# 暴露 Flask/Gunicorn 服务端口。
EXPOSE 8899
# 设置入口脚本和默认启动命令。
ENTRYPOINT ["docker-entrypoint.sh"]
CMD ["gunicorn", "--workers", "3", "--threads", "8", "--bind", "0.0.0.0:8899", "--timeout", "600", "src.app:app"]

61
speakr/Dockerfile20260710 Normal file
View File

@ -0,0 +1,61 @@
FROM node:20-slim AS frontend-builder
WORKDIR /build/templates/index
# 安装前端依赖，并在镜像构建阶段生成 Vite 产物。
COPY templates/index/package.json templates/index/pnpm-lock.yaml templates/index/pnpm-workspace.yaml ./
RUN corepack enable && \
corepack prepare pnpm@9.15.9 --activate && \
pnpm install --frozen-lockfile
COPY templates/index/ ./
RUN pnpm build
FROM python:3.11-slim
WORKDIR /app
# 安装运行时系统依赖。
RUN apt-get update && apt-get install -y --no-install-recommends \
gcc \
ffmpeg \
&& rm -rf /var/lib/apt/lists/*
# 先安装 Python 依赖，提高 Docker 层缓存命中率。
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# 先准备 vendor 目录，便于离线依赖下载脚本写入文件。
RUN mkdir -p /app/static/vendor
# 只复制下载脚本，避免业务代码变化导致 vendor 依赖重复下载。
COPY scripts/download_offline_deps.py scripts/
RUN pip install --no-cache-dir requests && \
python scripts/download_offline_deps.py
# 复制应用代码。
COPY . .
# 使用前端构建阶段生成的产物覆盖源码中的旧产物。
COPY --from=frontend-builder /build/static/vite/index /app/static/vite/index
# 创建容器运行时数据目录。
RUN mkdir -p /data/uploads /data/instance && \
chmod 755 /data/uploads /data/instance
# 设置默认运行环境变量。
ENV FLASK_APP=src/app.py
ENV SQLALCHEMY_DATABASE_URI=sqlite:////data/instance/transcriptions.db
ENV UPLOAD_FOLDER=/data/uploads
ENV PYTHONPATH=/app
# 安装容器入口脚本。
COPY scripts/docker-entrypoint.sh /usr/local/bin/
RUN chmod +x /usr/local/bin/docker-entrypoint.sh
# 暴露 Flask/Gunicorn 服务端口。
EXPOSE 8899
# 设置入口脚本和默认启动命令。
ENTRYPOINT ["docker-entrypoint.sh"]
CMD ["gunicorn", "--workers", "3", "--threads", "8", "--bind", "0.0.0.0:8899", "--timeout", "600", "src.app:app"]

661
speakr/LICENSE Normal file
View File

@ -0,0 +1,661 @@
GNU AFFERO GENERAL PUBLIC LICENSE
Version 3, 19 November 2007
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The GNU Affero General Public License is a free, copyleft license for
software and other kinds of works, specifically designed to ensure
cooperation with the community in the case of network server software.
The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
our General Public Licenses are intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
Developers that use our General Public Licenses protect your rights
with two steps: (1) assert copyright on the software, and (2) offer
you this License which gives you legal permission to copy, distribute
and/or modify the software.
A secondary benefit of defending all users' freedom is that
improvements made in alternate versions of the program, if they
receive widespread use, become available for other developers to
incorporate. Many developers of free software are heartened and
encouraged by the resulting cooperation. However, in the case of
software used on network servers, this result may fail to come about.
The GNU General Public License permits making a modified version and
letting the public access it on a server without ever releasing its
source code to the public.
The GNU Affero General Public License is designed specifically to
ensure that, in such cases, the modified source code becomes available
to the community. It requires the operator of a network server to
provide the source code of the modified version running there to the
users of that server. Therefore, public use of a modified version, on
a publicly accessible server, gives the public access to the source
code of the modified version.
An older license, called the Affero General Public License and
published by Affero, was designed to accomplish similar goals. This is
a different license, not a version of the Affero GPL, but Affero has
released a new version of the Affero GPL which permits relicensing under
this license.
The precise terms and conditions for copying, distribution and
modification follow.
TERMS AND CONDITIONS
0. Definitions.
"This License" refers to version 3 of the GNU Affero General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
To "modify" a work means to copy from or adapt all or part of the work
in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
A "covered work" means either the unmodified Program or a work based
on the Program.
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through
a computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays "Appropriate Legal Notices"
to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
1. Source Code.
The "source code" for a work means the preferred form of the work
for making modifications to it. "Object code" means any non-source
form of a work.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
The "System Libraries" of an executable work include anything, other
than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
The "Corresponding Source" for a work in object code form means all
the source code needed to generate, install, and (for an executable
work) run the object code and to modify the work, including scripts to
control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
The Corresponding Source for a work in source code form is that
same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program. The output from running a
covered work is covered by this License only if the output, given its
content, constitutes a covered work. This License acknowledges your
rights of fair use or other equivalent, as provided by copyright law.
You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
Conveying under any other circumstances is permitted solely under
the conditions stated below. Sublicensing is not allowed; section 10
makes it unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article
11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice;
keep intact all notices stating that this License and any
non-permissive terms added in accord with section 7 apply to the code;
keep intact all notices of the absence of any warranty; and give all
recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey,
and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Remote Network Interaction; Use with the GNU General Public License.
Notwithstanding any other provision of this License, if you modify the
Program, your modified version must prominently offer all users
interacting with it remotely through a computer network (if your version
supports such interaction) an opportunity to receive the Corresponding
Source of your version by providing access to the Corresponding Source
from a network server at no charge, through some standard or customary
means of facilitating copying of software. This Corresponding Source
shall include the Corresponding Source for any work covered by version 3
of the GNU General Public License that is incorporated pursuant to the
following paragraph.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the work with which it is combined will remain governed by version
3 of the GNU General Public License.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU Affero General Public License from time to time. Such new versions
will be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU Affero General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU Affero General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU Affero General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
state the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published
by the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Also add information on how to contact you by electronic and paper mail.
If your software can interact with users remotely through a computer
network, you should also make sure that it provides a way for users to
get its source. For example, if your program is a web application, its
interface could display a "Source" link that leads users to an archive
of the code. There are many ways you could offer source, and different
solutions will be better for different programs; see section 13 for the
specific requirements.
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU AGPL, see
<https://www.gnu.org/licenses/>.

513
speakr/README.md Normal file
View File

@ -0,0 +1,513 @@
# Speakr - 定制版
> 基于 [murtaza-nasir/speakr](https://github.com/murtaza-nasir/speakr) 二次开发的语音转录与智能笔记平台。
## 项目简介
本项目是一个自托管的 AI 语音转录和智能笔记平台，支持：
- **语音录制与上传** - 浏览器直接录音或上传已有音频文件
- **AI 转录** - 高准确率语音转文字，支持说话人识别
- **交互式对话** - 对录音提问，获取 AI 回答
- **智能标签** - 支持带自定义 AI 提示词的标签
- **安全分享** - 生成安全链接分享录音
- **草稿功能** - 本地草稿保存与上传
- **声纹管理** - 自定义说话人声纹识别
- **标准词映射** - 自动纠正专业术语和人名
---
## 项目结构
```
speakr/
├── app.py # 入口文件（启动脚本）
├── src/
│ ├── app.py # Flask 应用主逻辑（create_app 工厂模式）
│ ├── config.py # 统一配置文件（所有环境变量集中管理）
│ ├── logging_config.py # 日志配置模块（格式化、输出目标、日志级别）
│ ├── flask_ext.py # Flask 扩展实例集合（LoginManager、Bcrypt、Limiter、CSRFProtect）
│ ├── clients/ # 外部服务客户端层
│ │ ├── __init__.py # 统一导出所有客户端和配置
│ │ ├── database.py # 数据库客户端：db 实例 + 配置 + 初始化
│ │ ├── llm_client.py # LLM 客户端：OpenAI 客户端初始化
│ │ └── voiceprint_client.py # 声纹客户端：VoiceprintClient
│ ├── api/ # API 路由层（Flask 蓝图）
│ │ ├── __init__.py # 统一蓝图注册函数 register_blueprints()
│ │ ├── template_filters.py # 模板过滤器注册（now、localdatetime）
│ │ ├── auth.py # 认证路由（登录、注册、账户管理）
│ │ ├── recording.py # 录音管理（CRUD、上传、转录、下载）
│ │ ├── share.py # 分享功能
│ │ ├── tag.py # 标签管理
│ │ ├── speaker.py # 说话人管理
│ │ ├── admin.py # 管理后台
│ │ ├── search.py # 搜索和 Inquire
│ │ ├── voiceprint.py # 声纹管理
│ │ ├── config.py # 系统配置管理
│ │ ├── hotword.py # 热词管理
│ │ ├── template.py # 转录模板
│ │ ├── event.py # 事件管理
│ │ └── main.py # 主页和杂项
│ ├── models/ # 数据模型层（每个模型独立文件）
│ │ ├── __init__.py # 统一导出所有模型
│ │ ├── user.py # 用户模型
│ │ ├── speaker.py # 说话人模型
│ │ ├── recording.py # 录音模型
│ │ ├── tag.py # 标签模型
│ │ ├── recording_tag.py # 录音-标签关联模型
│ │ ├── event.py # 事件模型
│ │ ├── share.py # 分享模型
│ │ ├── transcript_chunk.py # 转录块模型
│ │ ├── transcript_template.py # 转录模板模型
│ │ ├── inquire_session.py # 询问会话模型
│ │ ├── voiceprint.py # 声纹模型
│ │ ├── system_setting.py # 系统设置模型
│ │ ├── system_config.py # 系统配置模型
│ │ ├── hotword.py # 热词模型
│ │ ├── draft_recording.py # 草稿录音模型
│ │ ├── draft_segment.py # 草稿片段模型
│ │ └── forms.py # 表单类（注册、登录）
│ ├── services/ # 业务逻辑层
│ │ ├── __init__.py # 统一导出所有服务函数
│ │ ├── llm_service.py # LLM 服务
│ │ ├── transcription_service.py # 转录服务
│ │ ├── summary_service.py # 摘要服务
│ │ ├── embedding_service.py # 嵌入向量服务
│ │ ├── event_service.py # 事件服务
│ │ ├── document_service.py # 文档服务
│ │ ├── speaker_service.py # 说话人服务
│ │ ├── voiceprint_service.py # 声纹服务（业务逻辑）
│ │ ├── auto_process_service.py # 自动处理服务
│ │ ├── auth_service.py # 认证服务
│ │ ├── sync_service.py # 远程数据同步服务
│ │ ├── audio_chunking.py # 音频分块服务
│ │ └── file_monitor.py # 文件监控服务
│ ├── utils/ # 工具函数层
│ │ ├── __init__.py
│ │ ├── markdown.py # Markdown 转 HTML 工具
│ │ ├── datetime.py # 时区转换工具
│ │ ├── text_utils.py # 文本处理工具
│ │ ├── json_utils.py # JSON 处理工具
│ │ └── standard_word_mappings.py # 标准词映射工具
├── templates/ # Jinja2 模板
├── static/ # 静态资源（CSS/JS/图片）
├── instance/ # SQLite 数据库文件
├── uploads/ # 上传的音频文件
└── .env # 环境变量配置
```
---
## 启动方式
### 前置条件
- Python 环境（已安装 conda 环境 `speakr`）
- Nginx 反向代理（必须启动）
### 启动步骤
```bash
# 1. 进入项目根目录（speakr 2 目录）
cd "c:\Work\zdht\python-projects\speakr 2"
# 2. 启动 Nginx 反向代理（必须先启动）
# Nginx 负责将 https://localhost:8083/tool/speakr/ 请求转发到 Flask 后端
# ⚠️ 注意：不启动 Nginx 将无法访问应用
nginx # 或根据你的安装方式启动 nginx
# 3. 激活 conda 环境并启动应用
conda activate speakr
python .\speakr\app.py
```
Flask 后端默认监听 **5000 端口**，Nginx 负责将 `https://localhost:8083/tool/speakr/` 请求转发到 `http://localhost:5000`。
### 访问地址
- 主页面：`https://localhost:8083/tool/speakr/`
- 管理后台：`https://localhost:8083/tool/speakr/admin`
- 账户设置：`https://localhost:8083/tool/speakr/account`
### 常见错误
| 问题 | 原因 | 解决 |
|------|------|------|
| 页面无法访问 | Nginx 未启动 | 启动 Nginx 反向代理 |
| `ModuleNotFoundError: No module named 'xxx'` | conda 环境未激活 | 执行 `conda activate speakr` |
| 数据库文件位置错误 | 从错误目录启动 | 确保从 `speakr 2/` 目录执行 `python .\speakr\app.py` |
---
## 环境配置
编辑 `.env` 文件，主要配置项：
```ini
# ASR 服务配置
USE_ASR_ENDPOINT=true
ASR_BASE_URL=http://10.100.3.22:6688
ASR_DIARIZE=true
# 文本生成模型配置
TEXT_MODEL_BASE_URL=http://10.100.3.22:9800/v1
TEXT_MODEL_API_KEY=EMPTY
TEXT_MODEL_NAME=minimax-m2.5
# 数据库配置
SQLALCHEMY_DATABASE_URI=sqlite:///transcriptions.db
# PostgreSQL 配置示例（切换时只需修改 URI 即可）
# SQLALCHEMY_DATABASE_URI=postgresql://user:password@localhost/speakr
# DB_POOL_SIZE=10
# DB_MAX_OVERFLOW=20
# DB_POOL_TIMEOUT=30
# DB_POOL_RECYCLE=3600
# 外部访问端口（Nginx 监听端口）
EXTERNAL_PORT=8083
# 时区配置
TIMEZONE=Asia/Shanghai
```
---
## 重构计划与当前进度
### 拆分原则
1. ⭐ 标记的模块是业务特色，拆分时只改代码组织结构，不改业务逻辑
2. 拆分顺序：
- **第一步**：抽出 Models（最安全，纯数据定义）
- **第二步**：抽出 Services（把 app.py 中的业务逻辑函数提取到独立文件）
- **第三步**：抽出 API 蓝图（把路由按功能域分组注册）
- **第四步**：抽出 Utils（工具函数独立化）
- **第五步**：模板组件化（可选，优先级较低）
3. `app.py` 最终只保留：Flask 初始化、数据库初始化、蓝图注册、全局错误处理
4. 保持向后兼容：拆分过程中所有 API 路径不变，前端无需改动
### 需要特别注意的业务逻辑
| 功能 | 说明 | 拆分注意事项 |
|------|------|-------------|
| ⭐ 声纹管理 | VoiceprintClient 类、声纹注册/比对/同步 | 保持 VoiceprintClient 与外部平台的交互逻辑不变 |
| ⭐ 系统配置 | SystemConfig 动态配置读写 | 保持配置缓存和热加载机制 |
| ⭐ 草稿 API | 已有独立蓝图 draft_bp | 直接迁移，几乎不需改动 |
| ⭐ 标准词映射 | 转写后的文本纠正 | 保持映射规则和应用逻辑 |
| ⭐ 自动处理 | 定时/触发式自动转写 | 保持调度逻辑和状态管理 |
| ⭐ 收件箱 | 录音的收件箱/高亮状态 | 保持查询和状态切换逻辑 |
| ⭐ 热词 | 热词管理和应用 | 保持与转写服务的集成 |
### 当前进度
#### ✅ 已完成：Phase 1 - Models 层抽取
已将 `src/app.py` 中的 **16 个数据模型** 和 **2 个表单类** 从主文件中抽取到独立模块：
| 状态 | 任务 | 说明 |
|------|------|------|
| ✅ | `src/database.py`（已移至 clients） | 创建独立数据库实例文件，集中管理 SQLAlchemy db 对象（Phase 2.5 移至 clients/database.py） |
| ✅ | `src/models/` | 每个模型独立文件（user.py, speaker.py, recording.py 等 16 个模型） |
| ✅ | `src/models/forms.py` | 表单类独立（RegistrationForm, LoginForm, password_check） |
| ✅ | `src/utils/markdown.py` | Markdown 转 HTML 工具函数 |
| ✅ | `src/utils/datetime.py` | 时区转换工具函数 |
| ✅ | `src/app.py` | 删除模型定义，改为从 `src.models` 导入 |
**验证结果**：应用可正常启动，数据库文件路径正确（`speakr/instance/transcriptions.db`）
#### ✅ 已完成：Phase 2 - Services 层抽取
**服务文件已全部创建**（`src/services/` 目录下 11 个服务文件），app.py 中的旧函数定义已全部删除。
| 状态 | 服务文件 | 包含函数 |
|------|---------|---------|
| ✅ | `llm_service.py` | `call_llm_completion`, `clean_llm_response`, `format_transcription_for_llm`, `process_streaming_with_thinking`, `extract_thinking_content`, `preprocess_long_transcription`, `extract_corrected_text`, `format_api_error_message` |
| ✅ | `transcription_service.py` | `transcribe_audio_asr`, `transcribe_audio_task`, `transcribe_single_file`, `transcribe_with_chunking`, `extract_audio_from_video`, `convert_to_wav` |
| ✅ | `summary_service.py` | `generate_title_task`, `generate_summary_only_task` |
| ✅ | `embedding_service.py` | `get_embedding_model`, `chunk_transcription`, `generate_embeddings`, `serialize_embedding`, `deserialize_embedding`, `process_recording_chunks`, `basic_text_search_chunks`, `semantic_search_chunks` |
| ✅ | `event_service.py` | `extract_events_from_transcript`, `generate_ics_content`, `escape_ical_text` |
| ✅ | `document_service.py` | `process_markdown_to_docx`, `get_recording_display_title`, `sanitize_download_filename_component`, `build_docx_download_filename`, `set_download_filename_header` |
| ✅ | `speaker_service.py` | `update_speaker_usage`, `identify_speakers_from_text`, `identify_unidentified_speakers_from_text` |
| ✅ | `voiceprint_service.py` | `sync_voiceprint_to_platform`（VoiceprintClient 已移至 clients 层） |
| ✅ | `auto_process_service.py` | `initialize_file_monitor`, `get_file_monitor_functions` |
| ✅ | `auth_service.py` | `is_safe_url`, `auto_login` |
| ✅ | `__init__.py` | 统一导出所有服务函数 |
**已完成的工作：**
1. ✅ 删除了 app.py 中的 22 个重复函数定义，改为从服务层导入
2. ✅ 创建了 `src/utils/text_utils.py` 和 `src/utils/json_utils.py`，将工具函数移出 app.py
3. ✅ 添加了服务层和工具层的导入
4. ✅ 核心功能测试通过，应用运行正常
#### ✅ 已完成：Phase 2.5 - Clients 层创建
按照行业规范创建了 `src/clients/` 层，统一管理与外部服务的连接客户端：
| 状态 | 客户端 | 说明 |
|------|--------|------|
| ✅ | `database.py` | 数据库客户端：`db` 实例 + 配置 + `init_database()` 初始化逻辑 |
| ✅ | `llm_client.py` | LLM 客户端：OpenAI 客户端初始化（client, TEXT_MODEL_NAME, TEXT_MODEL_API_KEY 等） |
| ✅ | `voiceprint_client.py` | 声纹客户端：VoiceprintClient 类 + voiceprint_client() 延迟初始化 |
| ✅ | `__init__.py` | 统一导出所有客户端 |
**重构成果：**
- 删除了旧 `src/database.py`（空实例文件），合并到 `clients/database.py`
- 删除了 `services/voiceprint_service.py` 中的 `VoiceprintClient` 类，移到 `clients/voiceprint_client.py`
- 17 个模型文件 + 6 个服务文件的 `db` 导入统一改为 `from src.clients import db`
- 消除了层层传递 `client` 参数的问题
- 外部服务客户端与业务逻辑分离，架构更清晰
#### ✅ 已完成：Phase 2.6 - 服务层注释中文化
将所有服务层文件中的英文注释和日志消息翻译为中文，提升代码可读性：
| 文件 | 翻译内容 |
|------|---------|
| `clients/llm_client.py` | URL 注释清理逻辑、客户端初始化逻辑 |
| `clients/voiceprint_client.py` | 声纹客户端注释和日志（新建文件，全中文） |
| `clients/database.py` | 数据库配置和初始化注释（新建文件，全中文） |
| `services/llm_service.py` | 思考内容提取、流式处理、API 错误格式化、分段总结等函数的注释和日志 |
| `services/summary_service.py` | 摘要生成全流程的注释和日志（标签提示词、语言要求、系统/用户消息构建等） |
| `services/transcription_service.py` | 音频提取、ASR 转录、分块转录、Whisper 调用等全流程注释和日志 |
| `services/embedding_service.py` | 文本分块、嵌入向量生成、语义搜索等全流程注释和日志 |
| `services/voiceprint_service.py` | 声纹同步业务逻辑注释 |
#### ✅ 已完成：Phase 3 - API 蓝图抽取
将 `src/app.py` 中的约 100 个路由定义按功能域分组抽取到独立的 Flask 蓝图中，app.py 从约 10000 行精简到约 700 行：
| 状态 | 蓝图文件 | 路由数量 | 说明 |
|------|---------|---------|------|
| ✅ | `api/auth.py` | 5 | 注册、登录、登出、账户管理、修改密码 |
| ✅ | `api/recording.py` | 24 | 录音 CRUD、上传、转录、下载、状态管理、ASR 校正 |
| ✅ | `api/share.py` | 7 | 分享创建、查看、访问 |
| ✅ | `api/tag.py` | 6 | 标签 CRUD、应用到录音 |
| ✅ | `api/speaker.py` | 5 | 说话人管理、搜索 |
| ✅ | `api/admin.py` | 18 | 管理后台、统计分析、录音管理、配置管理 |
| ✅ | `api/search.py` | 7 | 搜索、Inquire 对话、收件箱 |
| ✅ | `api/voiceprint.py` | 6 | 声纹注册、管理、同步 |
| ✅ | `api/config.py` | 6 | 系统配置读写、分段服务管理 |
| ✅ | `api/hotword.py` | 2 | 热词管理 |
| ✅ | `api/template.py` | 5 | 转录模板 CRUD |
| ✅ | `api/event.py` | 3 | 事件提取、ICS 导出 |
| ✅ | `api/main.py` | 5 | 主页、摘要生成、对话 |
**架构改进：**
- 创建 `api/__init__.py` 中的 `register_blueprints()` 函数，统一管理所有蓝图注册和初始化
- 使用 `init_xxx_bp()` 延迟注入模式解决循环导入问题（limiter、csrf、bcrypt 等全局对象）
- 所有 API 路径保持不变，前端无需任何修改
- `app.py` 仅保留：Flask 初始化、配置加载、蓝图注册、全局错误处理、启动逻辑
**重构后修复的问题：**
| 问题 | 原因 | 解决方案 |
|------|------|---------|
| 重定向循环 | `main.py` 中的 `index()` 路由误加了 `@login_required` 装饰器，丢失了自动登录逻辑 | 恢复备份中的 token 验证和自动登录逻辑，移除装饰器 |
| WebSocket 连接失败 | `getUserConfig` 路由缺少 `FUNASR_WEBSOCKET_IP` 和 `SCREEN_PUSH_IP` 配置 | 在 `main.py` 的 `get_user_config()` 中补充 WebSocket 配置字段 |
| 循环导入 bcrypt | `main.py` 在模块级别导入 `bcrypt` 导致循环依赖 | 改为在 `init_main_bp()` 中延迟注入 `bcrypt_instance` |
#### ✅ 已完成：统一配置管理
创建 `src/config.py` 统一配置文件，将所有散落的环境变量读取集中管理：
| 配置域 | 包含变量 |
|--------|---------|
| 基础配置 | `SECRET_KEY`, `UPLOAD_FOLDER`, `LOG_LEVEL`, `MAX_FILE_SIZE_MB` |
| ASR 配置 | `USE_ASR_ENDPOINT`, `ASR_BASE_URL`, `ASR_DIARIZE`, `ASR_MIN_SPEAKERS`, `ASR_MAX_SPEAKERS` |
| LLM 配置 | `TEXT_MODEL_API_KEY`, `TEXT_MODEL_BASE_URL`, `TEXT_MODEL_NAME` |
| 分段配置 | `ENABLE_CHUNKING` |
| Inquire 配置 | `ENABLE_INQUIRE_MODE` |
| 数据库配置 | `SQLALCHEMY_DATABASE_URI`, `DB_POOL_SIZE`, `DB_MAX_OVERFLOW`, `DB_POOL_TIMEOUT`, `DB_POOL_RECYCLE` |
| 外部 API | `TRANSCRIPTION_BASE_URL`, `VALID_TOKEN`, `EMBEDDINGS_AVAILABLE` |
| 代理配置 | `HTTP_PROXY`, `HTTPS_PROXY` |
**重构成果：**
- `clients/llm_client.py`、`clients/voiceprint_client.py` 从 `src.config` 导入配置
- `clients/__init__.py` 统一导出所有配置项
- `app.py` 通过 `from src.clients import ...` 统一导入配置，消除散落的环境变量读取
- 提供 `validate_config()` 函数在启动时验证关键配置
#### ✅ 已完成：Phase 3.5 - app.py 清理与同步服务抽取
将 `src/app.py` 中残留的约 400 行业务逻辑彻底清理，app.py 从约 700 行精简到约 234 行：
| 状态 | 任务 | 说明 |
|------|------|------|
| ✅ | 删除重复的 preprocess 函数 | 删除了 `preprocess_long_transcription` 等 6 个重复定义的函数（已在 llm_service.py 中） |
| ✅ | 抽取远程同步服务 | 创建了 `services/sync_service.py`，将约 180 行同步逻辑从 app.py 移出 |
| ✅ | 清理无用 import | 删除了 30+ 个未使用的导入（render_template、OpenAI、httpx、subprocess、markdown 等） |
| ✅ | 注释中文化 | 将所有英文注释翻译为中文，删除无意义注释 |
| ✅ | 修复函数签名 | `start_sync_thread()` 增加 `app` 参数，调用处同步更新 |
| ✅ | 优化配置读取 | `preprocess_long_transcription` 等函数直接从 `src.config` 读取配置，不再通过参数传递 |
#### ✅ 已完成：Phase 4 - app.py 工厂模式重构
采用 Flask `create_app()` 工厂模式，将 `src/app.py` 从 234 行进一步精简到 60 行，各层职责清晰分离：
**新建模块：**
| 文件 | 职责 |
|------|------|
| `src/logging_config.py` | 日志配置模块 |
| `src/flask_ext.py` | Flask 扩展实例集合（login_manager、bcrypt、limiter、csrf） |
| `src/services/register.py` | 服务层统一初始化（音频分段、同步线程、文件监控、表结构迁移） |
| `src/api/template_filters.py` | 模板过滤器注册 |
**修改模块：**
| 文件 | 修改内容 |
|------|---------|
| `src/config.py` | 新增 `get_app_config()` 函数，一次性返回所有需写入 `app.config` 的配置项。新增配置只需改此函数，无需修改 `app.py` |
| `src/clients/__init__.py` | 集中管理 `chunking_service` 和 `EMBEDDINGS_AVAILABLE` 的初始化与导出 |
| `src/api/__init__.py` | 简化 `register_blueprints()` 函数，直接从 `src.flask_ext` 导入扩展实例 |
| `src/app.py` | 采用 `create_app()` 工厂模式，仅负责编排各层初始化顺序 |
**`app.py` 最终结构（60 行）：**
```
create_app()
├── 1. configure_logging() ← src/logging_config.py
├── 2. 创建 Flask 应用实例
├── 3. 加载应用配置 ← src/config.py 的 get_app_config()
├── 4. 配置反向代理支持
├── 5. 初始化 Flask 扩展 ← src/flask_ext.py
├── 6. 初始化数据库 ← src/clients/database.py
├── 7. 注册蓝图 ← src/api/__init__.py
├── 8. 注册模板过滤器 ← src/api/template_filters.py
└── 9. 初始化服务层 ← src/services/register.py
```
**关键设计决策：**
| 决策 | 理由 |
|------|------|
| `flask_ext.py` 独立 | Flask 扩展是全局单例，不应放在 clients 或 services 中 |
| `get_app_config()` 集中管理 | 新增配置只需改 config.py，app.py 永远不需要改动 |
| `register_blueprints` 自行读扩展 | 蓝图注册最清楚自己需要什么扩展，无需 app.py 中转 |
| `init_services` 放在最后 | 服务依赖数据库和蓝图，必须在它们之后初始化 |
---
## 开发规范
### API 接口返回格式规范
> **重要：所有新建/修改的 API 接口必须使用 `ApiResponse` 工具类统一返回格式。旧接口暂不强制迁移。**
#### 工具位置
`src/api/api_response.py`
#### 使用方式
```python
from src.api.api_response import ApiResponse
# 成功响应（返回数据）
@blueprint.route('/xxx', methods=['POST'])
def create_xxx():
result = do_something()
return ApiResponse.success({'xxx': result.to_dict()})
# 成功响应（带提示消息）
return ApiResponse.success({'xxx': result.to_dict()}, '创建成功')
# 成功响应（自定义状态码，如 201 Created）
return ApiResponse.success({'xxx': result.to_dict()}, '创建成功', status_code=201)
# 错误响应
if not xxx:
return ApiResponse.error('资源未找到', 404)
# 错误响应（带详情）
if not valid:
return ApiResponse.error('参数校验失败', 400, detail='email 格式不正确')
# 分页响应
return ApiResponse.paginated(
items=[item.to_dict() for item in items],
pagination={
'page': page,
'per_page': per_page,
'total': total,
'total_pages': total_pages,
'has_next': has_next,
'has_prev': has_prev
}
)
```
#### 响应格式约定
**成功响应结构：**
```json
{
"success": true,
"message": "操作成功（可选）",
"xxx": {...} // 业务数据直接合并到顶层
}
```
**分页响应结构：**
```json
{
"success": true,
"items": [...],
"pagination": {
"page": 1,
"total": 100,
...
}
}
```
**错误响应结构：**
```json
{
"error": "用户友好的错误提示"
}
```
#### 注意事项
1. `data` 参数必须是字典类型，列表数据请使用 `paginated()` 或直接合并到字典中
2. 前端通过 `response.ok`（HTTP 状态码）判断成功/失败，通过 `data.error` 获取错误信息
3. 旧接口暂不迁移，新接口和修改的接口必须使用 `ApiResponse`
4. 不要在 `ApiResponse` 中直接传递原始异常信息给用户（`str(e)`），应转为友好提示
---
## 技术栈
- **后端**: Python/Flask + SQLAlchemy
- **前端**: Vue.js 3 + Tailwind CSS
- **AI**: OpenAI Whisper (ASR) + Ollama/OpenRouter (LLM)
- **数据库**: SQLite（默认）/ PostgreSQL（支持环境变量切换）
- **部署**: Nginx 反向代理 + Flask 开发服务器
---
## 依赖安装
```bash
conda activate speakr
pip install -r requirements.txt
```
主要依赖：
- `flask==2.3.3`
- `flask-sqlalchemy==3.1.1`
- `flask-login==0.6.3`
- `flask-wtf==1.2.2`
- `flask-bcrypt==1.0.1`
- `openai==1.3.0`
- `markdown==3.5.1`
- `python-docx==1.1.0`
---
## 许可证
本项目基于 AGPL v3.0 开源协议。

44
speakr/Readme_deploy.md Normal file
View File

@ -0,0 +1,44 @@
# 离线部署
cd /opt/speakr
# 1. 导入镜像（无需网络）
docker load -i speakr.tar
# 2. 一次性：从镜像里取出前端构建产物和完整 vendor 离线依赖，补进宿主机代码目录
docker create --name speakr-tmp speakr:latest
docker cp speakr-tmp:/app/static/vite ./static/
docker cp speakr-tmp:/app/static/vendor ./static/
docker rm speakr-tmp
# 3. 准备 .env（按转写方式二选一，然后编辑填入密钥等配置）
cp config/env.asr.example .env # 自建 ASR 服务用这个
# cp config/env.whisper.example .env # Whisper API 用这个
cd /home/wjs/speakr/speakr
# 1. 把服务器上这份脚本的 CRLF 转成 LF，并确保可执行
sed -i 's/\r$//' scripts/docker-entrypoint.sh
chmod +x scripts/docker-entrypoint.sh
# 2. 删掉一直重启的旧容器
docker rm -f speakr
# 3. 重新启动：多加一条单文件挂载，用修好的脚本覆盖镜像里那份坏的
docker run -d \
--name speakr \
--restart unless-stopped \
-p 5000:8899 \
-v /home/wjs/speakr/speakr:/app \
-v /home/wjs/speakr/speakr/scripts/docker-entrypoint.sh:/usr/local/bin/docker-entrypoint.sh \
-v /home/wjs/speakr/uploads:/data/uploads \
-v /home/wjs/speakr/instance:/data/instance \
speakr:v2
# 4. 确认启动正常（应看到"数据库不存在，正在创建..."之类的日志）
docker logs -f speakr
# 合并压缩
cat speakr.tar.part* > speakr.tar

1
speakr/VERSION Normal file
View File

@ -0,0 +1 @@
v0.5.6

4
speakr/app.py Normal file
View File

@ -0,0 +1,4 @@
from src.app import app
if __name__ == "__main__":
app.run(debug=True, host="0.0.0.0", port=5000)

43
speakr/config/docker-compose.example.yml Normal file
View File

@ -0,0 +1,43 @@
services:
app:
image: learnedmachine/speakr:latest
container_name: speakr
restart: unless-stopped
ports:
- "8899:8899"
# --- Configuration ---
# Environment variables are loaded from the .env file.
#
# To get started:
# 1. Choose your desired transcription method.
# 2. Copy the corresponding example file to .env:
#
# For standard Whisper API:
# cp config/env.whisper.example .env
#
# For a custom ASR endpoint:
# cp config/env.asr.example .env
#
# 3. Edit the .env file to add your API keys and settings.
env_file:
- .env
environment:
# Set log level for troubleshooting
# Use ERROR for production (minimal logs)
# Use INFO for debugging issues (recommended when troubleshooting)
# Use DEBUG for detailed development logging
- LOG_LEVEL=ERROR
# --- Volume Configuration ---
# Choose ONE of the following volume configurations.
# Option 1 (Recommended): Bind mounts to local folders.
volumes:
- ./uploads:/data/uploads
- ./instance:/data/instance
# Option 2: Docker-managed volumes.
# volumes:
# - speakr-uploads:/data/uploads
# - speakr-instance:/data/instance

74
speakr/config/env.asr.example Normal file
View File

@ -0,0 +1,74 @@
# -----------------------------------------------------------------------------
# Speakr Configuration: ASR Endpoint
#
# Instructions:
# 1. Copy this file to a new file named .env
# cp env.asr.example .env
# 2. Fill in the required URLs, API keys, and settings below.
# -----------------------------------------------------------------------------
# --- Text Generation Model (for summaries, titles, etc.) ---
TEXT_MODEL_BASE_URL=https://openrouter.ai/api/v1
TEXT_MODEL_API_KEY=your_openrouter_api_key
TEXT_MODEL_NAME=openai/gpt-4o-mini
# --- Transcription Service (ASR Endpoint) ---
# Setting this to true enables all ASR features including speaker diarization
# Note: File chunking is NOT supported with ASR endpoints
USE_ASR_ENDPOINT=true
# ASR Endpoint URL
# For containers in same docker-compose: Use container name and internal port
# Example: http://whisper-asr:9000 (NOT the host port 6002 or external IP)
# For external ASR: Use http://192.168.1.100:9000 or http://asr.example.com:9000
ASR_BASE_URL=http://whisper-asr:9000
# Optional: Override default speaker settings (defaults shown)
# These are automatically set when USE_ASR_ENDPOINT=true
# ASR_DIARIZE=true # Auto-enabled with ASR (set to false to disable)
# ASR_MIN_SPEAKERS=1 # Default minimum speakers
# ASR_MAX_SPEAKERS=5 # Default maximum speakers
# --- Application Settings ---
# Set to "true" to allow user registration, "false" to disable
ALLOW_REGISTRATION=false
SUMMARY_MAX_TOKENS=8000
CHAT_MAX_TOKENS=5000
# Timezone for displaying dates and times in the UI
# Use a valid TZ database name (e.g., "America/New_York", "Europe/London", "UTC")
TIMEZONE="UTC"
# Set the logging level for the application.
# Options: DEBUG, INFO, WARNING, ERROR
LOG_LEVEL="INFO"
# --- Admin User (created on first run) ---
ADMIN_USERNAME=admin
ADMIN_EMAIL=admin@example.com
ADMIN_PASSWORD=changeme
# --- Inquire Mode (AI search across all recordings) ---
# Set to "true" to enable semantic search and chat across all recordings
# Requires additional dependencies (already included in Docker image)
ENABLE_INQUIRE_MODE=false
# --- Automated File Processing (Black Hole Directory) ---
# Set to "true" to enable automated file processing
ENABLE_AUTO_PROCESSING=false
# Processing mode: admin_only, user_directories, or single_user
AUTO_PROCESS_MODE=admin_only
# Directory to watch for new audio files
AUTO_PROCESS_WATCH_DIR=/data/auto-process
# How often to check for new files (seconds)
AUTO_PROCESS_CHECK_INTERVAL=30
# Default username for single_user mode (only used if AUTO_PROCESS_MODE=single_user)
# AUTO_PROCESS_DEFAULT_USERNAME=admin
# --- Docker Settings (rarely need to be changed) ---
SQLALCHEMY_DATABASE_URI=sqlite:////data/instance/transcriptions.db
UPLOAD_FOLDER=/data/uploads

77
speakr/config/env.whisper.example Normal file
View File

@ -0,0 +1,77 @@
# -----------------------------------------------------------------------------
# Speakr Configuration: Standard Whisper API
#
# Instructions:
# 1. Copy this file to a new file named .env
# cp env.whisper.example .env
# 2. Fill in the required API keys and settings below.
# -----------------------------------------------------------------------------
# --- Text Generation Model (for summaries, titles, etc.) ---
TEXT_MODEL_BASE_URL=https://openrouter.ai/api/v1
TEXT_MODEL_API_KEY=your_openrouter_api_key
TEXT_MODEL_NAME=openai/gpt-4o-mini
# --- Transcription Service (OpenAI Whisper API) ---
TRANSCRIPTION_BASE_URL=https://api.openai.com/v1
TRANSCRIPTION_API_KEY=your_openai_api_key
WHISPER_MODEL=whisper-1
# --- Application Settings ---
# Set to "true" to allow user registration, "false" to disable
ALLOW_REGISTRATION=false
SUMMARY_MAX_TOKENS=8000
CHAT_MAX_TOKENS=5000
# Timezone for displaying dates and times in the UI
# Use a valid TZ database name (e.g., "America/New_York", "Europe/London", "UTC")
TIMEZONE="UTC"
# Set the logging level for the application.
# Options: DEBUG, INFO, WARNING, ERROR
LOG_LEVEL="INFO"
# --- Large File Chunking ---
# Automatically splits large files to work with API limits
ENABLE_CHUNKING=true
# Choose chunking method based on your provider's limits:
# For size-based limits (e.g., OpenAI's 25MB limit):
CHUNK_LIMIT=20MB
# For duration-based limits (uncomment and use instead):
# CHUNK_LIMIT=1200s # 20 minutes
# CHUNK_LIMIT=20m # Alternative format
# Overlap between chunks (seconds)
CHUNK_OVERLAP_SECONDS=3
# --- Admin User (created on first run) ---
ADMIN_USERNAME=admin
ADMIN_EMAIL=admin@example.com
ADMIN_PASSWORD=changeme
# --- Inquire Mode (AI search across all recordings) ---
# Set to "true" to enable semantic search and chat across all recordings
# Requires additional dependencies (already included in Docker image)
ENABLE_INQUIRE_MODE=false
# --- Automated File Processing (Black Hole Directory) ---
# Set to "true" to enable automated file processing
ENABLE_AUTO_PROCESSING=false
# Processing mode: admin_only, user_directories, or single_user
AUTO_PROCESS_MODE=admin_only
# Directory to watch for new audio files
AUTO_PROCESS_WATCH_DIR=/data/auto-process
# How often to check for new files (seconds)
AUTO_PROCESS_CHECK_INTERVAL=30
# Default username for single_user mode (only used if AUTO_PROCESS_MODE=single_user)
# AUTO_PROCESS_DEFAULT_USERNAME=admin
# --- Docker Settings (rarely need to be changed) ---
SQLALCHEMY_DATABASE_URI=sqlite:////data/instance/transcriptions.db
UPLOAD_FOLDER=/data/uploads

92
speakr/deployment/setup.sh Normal file
View File

@ -0,0 +1,92 @@
#!/bin/bash
# 停止转录服务
sudo systemctl stop transcription
# 创建应用目录并设置所有者
sudo mkdir -p /opt/transcription-app
sudo chown $USER:$USER /opt/transcription-app
# 复制应用文件
cp app.py /opt/transcription-app/
cp -r templates /opt/transcription-app/
cp requirements.txt /opt/transcription-app/
cp scripts/reset_db.py /opt/transcription-app/
cp scripts/create_admin.py /opt/transcription-app/
cp .env /opt/transcription-app/ # 复制包含 API 密钥的 .env 文件
# 如果 .env 文件中不存在 SECRET_KEY，则添加
if ! grep -q "SECRET_KEY" /opt/transcription-app/.env; then
echo "SECRET_KEY=$(openssl rand -hex 32)" >> /opt/transcription-app/.env
echo "已添加 SECRET_KEY 到 .env 文件"
fi
# 创建并激活虚拟环境
python3 -m venv /opt/transcription-app/venv
source /opt/transcription-app/venv/bin/activate
# 安装依赖
cd /opt/transcription-app
pip install -r requirements.txt
# 创建上传和数据库目录并设置权限
mkdir -p /opt/transcription-app/uploads
mkdir -p /opt/transcription-app/instance
chmod 755 /opt/transcription-app/uploads
chmod 755 /opt/transcription-app/instance
# 初始化或迁移数据库
if [ ! -f /opt/transcription-app/instance/transcriptions.db ]; then
# 如果数据库不存在，从头创建
echo "数据库不存在，正在创建新数据库..."
python reset_db.py
else
# 如果数据库存在，迁移模式以保留数据
echo "数据库已存在，正在迁移模式以保留数据..."
python migrate_db.py
fi
# 设置所有文件的正确所有者
sudo chown -R $USER:$USER /opt/transcription-app
# 创建 systemd 服务文件
sudo tee /etc/systemd/system/transcription.service << EOF
[Unit]
Description=Transcription Web Application
After=network.target
[Service]
User=$USER
EnvironmentFile=/opt/transcription-app/.env
WorkingDirectory=/opt/transcription-app
Environment="PATH=/opt/transcription-app/venv/bin"
Environment="PYTHONPATH=/opt/transcription-app"
ExecStart=/opt/transcription-app/venv/bin/gunicorn --workers 3 --threads 8 --bind 0.0.0.0:8899 --timeout 600 app:app
Restart=always
RestartSec=5
[Install]
WantedBy=multi-user.target
EOF
# 重新加载 systemd 并启动服务
sudo systemctl daemon-reload
sudo systemctl restart transcription
sudo systemctl disable transcription
sudo systemctl enable transcription
# 检查服务状态
echo "正在检查服务状态..."
sleep 3
sudo systemctl status transcription
echo "安装完成！应用应在 8899 端口上运行。"
# 询问是否创建管理员账户
read -p "是否现在创建管理员账户？(y/n): " create_admin
if [[ $create_admin == "y" || $create_admin == "Y" ]]; then
echo "正在创建管理员账户..."
python create_admin.py
else
echo "你可以稍后运行以下命令来创建管理员账户：python create_admin.py"
fi

31
speakr/docs/.gitignore vendored Normal file
View File

@ -0,0 +1,31 @@
# Jekyll build output
_site/
.sass-cache/
.jekyll-cache/
.jekyll-metadata
# Bundle directory
.bundle/
vendor/
# Local development files
_config_local.yml
local-serve.sh
docker-serve.sh
simple-serve.sh
serve-local.sh
# OS files
.DS_Store
Thumbs.db
# IDE files
.vscode/
.idea/
# Ruby version files
.ruby-version
.ruby-gemset
# Documentation verification (internal use)
DOCUMENTATION_VERIFICATION_CHECKLIST.md

28
speakr/docs/Dockerfile Normal file
View File

@ -0,0 +1,28 @@
FROM python:3.11-slim
WORKDIR /app
# Install git (required for git-revision-date plugin)
RUN apt-get update && \
apt-get install -y git && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
# Install requirements
COPY requirements-docs.txt .
RUN pip install --no-cache-dir -r requirements-docs.txt
# Create docs directory structure
RUN mkdir -p docs
# Copy mkdocs config to root
COPY mkdocs.yml .
# Copy all documentation files to docs subdirectory
COPY . ./docs/
# Expose MkDocs development server port
EXPOSE 8000
# Run MkDocs server
CMD ["mkdocs", "serve", "--dev-addr=0.0.0.0:8000"]

8
speakr/docs/Gemfile.simple Normal file
View File

@ -0,0 +1,8 @@
source "https://rubygems.org"
gem "jekyll", "~> 4.3"
gem "jekyll-seo-tag"
gem "jekyll-sitemap"
gem "jekyll-feed"
gem "webrick", "~> 1.8"
gem "kramdown-parser-gfm"

33
speakr/docs/_includes/sidebar.html Normal file
View File

@ -0,0 +1,33 @@
<nav class="sidebar">
<div class="sidebar-header">
<img src="{{ '/assets/images/logo.png' | relative_url }}" alt="Speakr Logo" class="sidebar-logo">
<h2 class="sidebar-title">Speakr Docs</h2>
</div>
<div class="sidebar-content">
{% for section in site.navigation %}
<div class="sidebar-section">
<h3 class="sidebar-section-title">{{ section.title }}</h3>
<ul class="sidebar-list">
{% for item in section.items %}
<li class="sidebar-item">
<a href="{{ item.url | relative_url }}"
class="sidebar-link {% if page.url == item.url %}active{% endif %}">
{{ item.title }}
</a>
</li>
{% endfor %}
</ul>
</div>
{% endfor %}
</div>
<div class="sidebar-footer">
<a href="https://github.com/murtaza-nasir/speakr" class="github-link">
<svg class="github-icon" viewBox="0 0 16 16" width="20" height="20">
<path fill="currentColor" d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0016 8c0-4.42-3.58-8-8-8z"></path>
</svg>
GitHub
</a>
</div>
</nav>

403
speakr/docs/_layouts/default.html Normal file
View File

@ -0,0 +1,403 @@
<!DOCTYPE html>
<html lang="{{ site.lang | default: 'en-US' }}">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
{% seo %}
<link rel="stylesheet" href="{{ '/assets/css/style.css?v=' | append: site.github.build_revision | relative_url }}">
<link rel="icon" type="image/png" href="{{ '/assets/images/logo.png' | relative_url }}">
<!-- Custom fonts -->
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&display=swap" rel="stylesheet">
<style>
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
body {
font-family: 'Inter', -apple-system, BlinkMacSystemFont, "Segoe UI", Helvetica, Arial, sans-serif;
line-height: 1.6;
color: #1f2937;
font-size: 15px;
}
.docs-wrapper {
display: flex;
min-height: 100vh;
}
/* Sidebar Styles */
.sidebar {
position: fixed;
left: 0;
top: 0;
width: 280px;
height: 100vh;
background: #f9fafb;
border-right: 1px solid #e5e7eb;
overflow-y: auto;
z-index: 100;
}
.sidebar-header {
padding: 1.5rem;
background: linear-gradient(135deg, #6366f1 0%, #8b5cf6 100%);
color: white;
text-align: center;
}
.sidebar-logo {
width: 60px;
height: 60px;
border-radius: 12px;
margin-bottom: 0.5rem;
}
.sidebar-title {
font-size: 1.25rem;
font-weight: 600;
margin: 0;
}
.sidebar-content {
padding: 1rem 0;
}
.sidebar-section {
margin-bottom: 1.5rem;
}
.sidebar-section-title {
font-size: 0.75rem;
font-weight: 700;
text-transform: uppercase;
letter-spacing: 0.05em;
color: #6b7280;
padding: 0 1.5rem;
margin-bottom: 0.5rem;
}
.sidebar-list {
list-style: none;
}
.sidebar-item {
margin: 0;
}
.sidebar-link {
display: block;
padding: 0.5rem 1.5rem;
color: #4b5563;
text-decoration: none;
transition: all 0.2s ease;
font-size: 0.95rem;
}
.sidebar-link:hover {
background: #e5e7eb;
color: #1f2937;
}
.sidebar-link.active {
background: #6366f1;
color: white;
font-weight: 500;
}
.sidebar-footer {
padding: 1rem 1.5rem;
border-top: 1px solid #e5e7eb;
margin-top: auto;
}
.github-link {
display: flex;
align-items: center;
gap: 0.5rem;
color: #6b7280;
text-decoration: none;
font-size: 0.9rem;
transition: color 0.2s;
}
.github-link:hover {
color: #1f2937;
}
.github-icon {
flex-shrink: 0;
}
/* Main Content Styles */
.main-content {
flex: 1;
margin-left: 280px;
min-width: 0;
}
.content-wrapper {
max-width: 900px;
margin: 0 auto;
padding: 2rem 3rem;
}
/* Content Typography */
.content-wrapper h1 {
color: #1f2937;
font-size: 1.875rem;
font-weight: 700;
margin-bottom: 1rem;
padding-bottom: 0.5rem;
border-bottom: 2px solid #6366f1;
}
.content-wrapper h2 {
color: #1f2937;
font-size: 1.5rem;
font-weight: 600;
margin-top: 2rem;
margin-bottom: 1rem;
padding-bottom: 0.5rem;
border-bottom: 1px solid #e5e7eb;
}
.content-wrapper h3 {
color: #374151;
font-size: 1.25rem;
font-weight: 600;
margin-top: 1.5rem;
margin-bottom: 0.75rem;
}
.content-wrapper h4 {
color: #4b5563;
font-size: 1.125rem;
font-weight: 600;
margin-top: 1.25rem;
margin-bottom: 0.5rem;
}
.content-wrapper p {
margin-bottom: 1rem;
line-height: 1.6;
font-size: 0.95rem;
}
.content-wrapper img {
max-width: 100%;
height: auto;
border-radius: 12px;
box-shadow: 0 4px 20px rgba(0,0,0,0.1);
margin: 2rem 0;
}
.content-wrapper code {
background: #f3f4f6;
padding: 0.125rem 0.375rem;
border-radius: 4px;
color: #dc2626;
font-size: 0.85em;
font-family: 'Monaco', 'Courier New', monospace;
font-weight: 500;
}
.content-wrapper pre {
background: #2d3748;
color: #e2e8f0;
padding: 1rem;
border-radius: 8px;
overflow-x: auto;
margin: 1rem 0;
border: 1px solid #4a5568;
}
.content-wrapper pre code {
background: none;
color: #e2e8f0;
padding: 0;
font-weight: normal;
}
/* Override syntax highlighting for better contrast */
.content-wrapper .highlight pre {
background: #1f2937;
}
.content-wrapper .highlight code {
color: #f3f4f6;
}
.content-wrapper ul, .content-wrapper ol {
margin-bottom: 1rem;
padding-left: 2rem;
}
.content-wrapper li {
margin-bottom: 0.5rem;
font-size: 0.95rem;
}
.content-wrapper blockquote {
border-left: 4px solid #6366f1;
padding-left: 1rem;
margin: 1rem 0;
font-style: italic;
color: #6b7280;
}
.content-wrapper table {
width: 100%;
border-collapse: collapse;
margin: 1rem 0;
}
.content-wrapper th, .content-wrapper td {
padding: 0.75rem;
text-align: left;
border: 1px solid #e5e7eb;
}
.content-wrapper th {
background: #f9fafb;
font-weight: 600;
}
.content-wrapper a {
color: #4f46e5;
text-decoration: none;
font-weight: 500;
}
.content-wrapper a:hover {
text-decoration: underline;
color: #4338ca;
}
/* Ensure inline code in links is visible */
.content-wrapper a code {
color: inherit;
}
/* Override Jekyll/Rouge syntax highlighting for better contrast */
.highlighter-rouge .highlight {
background: #2d3748 !important;
}
.highlight {
background: #2d3748 !important;
color: #e2e8f0 !important;
}
.highlight pre {
background: #2d3748 !important;
color: #e2e8f0 !important;
}
/* Make all text light by default */
.highlight * {
color: #e2e8f0 !important;
}
/* Then override specific elements */
.highlight .c { color: #718096 !important; } /* Comment */
.highlight .s, .highlight .s1, .highlight .s2 { color: #68d391 !important; } /* String */
.highlight .k { color: #b794f4 !important; } /* Keyword */
.highlight .o { color: #f6ad55 !important; } /* Operator */
.highlight .m { color: #f6ad55 !important; } /* Number */
.highlight .nb { color: #63b3ed !important; } /* Builtin */
.highlight .nv { color: #fc8181 !important; } /* Variable */
/* Ensure operators like -O are visible */
.highlight .p { color: #cbd5e0 !important; } /* Punctuation */
.highlight .nt { color: #63b3ed !important; } /* Tag */
.highlight .na { color: #f6ad55 !important; } /* Attribute */
/* Mobile Responsiveness */
@media (max-width: 768px) {
.sidebar {
transform: translateX(-100%);
transition: transform 0.3s ease;
}
.sidebar.active {
transform: translateX(0);
}
.main-content {
margin-left: 0;
}
.content-wrapper {
padding: 1.5rem;
}
}
/* Home page special styles */
.home-header {
text-align: center;
padding: 3rem 0;
background: linear-gradient(135deg, #6366f1 0%, #8b5cf6 100%);
color: white;
margin: -2rem -3rem 2rem -3rem;
border-radius: 12px;
}
.project-logo {
width: 100px;
height: 100px;
margin: 0 auto 1rem;
display: block;
border-radius: 20px;
box-shadow: 0 10px 25px rgba(0,0,0,0.2);
}
</style>
</head>
<body>
<div class="docs-wrapper">
{% include sidebar.html %}
<main class="main-content">
<div class="content-wrapper">
{{ content }}
</div>
</main>
</div>
<script>
// Mobile sidebar toggle
document.addEventListener('DOMContentLoaded', function() {
const sidebar = document.querySelector('.sidebar');
const toggleBtn = document.createElement('button');
toggleBtn.innerHTML = '☰';
toggleBtn.style.cssText = 'position:fixed;top:1rem;left:1rem;z-index:200;padding:0.5rem 0.75rem;background:white;border:1px solid #e5e7eb;border-radius:8px;cursor:pointer;display:none;';
if (window.innerWidth <= 768) {
toggleBtn.style.display = 'block';
}
toggleBtn.onclick = function() {
sidebar.classList.toggle('active');
};
document.body.appendChild(toggleBtn);
window.addEventListener('resize', function() {
if (window.innerWidth <= 768) {
toggleBtn.style.display = 'block';
} else {
toggleBtn.style.display = 'none';
sidebar.classList.remove('active');
}
});
});
</script>
</body>
</html>

22
speakr/docs/_layouts/docs.html Normal file
View File

@ -0,0 +1,22 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>{% if page.title %}{{ page.title }} - {% endif %}{{ site.title }}</title>
<meta name="description" content="{% if page.description %}{{ page.description }}{% else %}{{ site.description }}{% endif %}">
<link rel="stylesheet" href="{{ '/assets/css/style.css' | relative_url }}">
<link rel="icon" type="image/png" href="{{ '/assets/images/logo.png' | relative_url }}">
</head>
<body>
<div class="docs-wrapper">
{% include sidebar.html %}
<main class="main-content">
<div class="content-wrapper">
{{ content }}
</div>
</main>
</div>
</body>
</html>

99
speakr/docs/admin-guide/index.md Normal file
View File

@ -0,0 +1,99 @@
# 管理员指南
欢迎使用 Speakr 管理员指南！作为管理员，你可以控制 Speakr 实例的核心，管理用户、监控系统健康状况以及配置 AI 行为。
## 管理控制面板
<div class="guide-cards">
<div class="guide-card">
<div class="card-icon">👥</div>
<h3>用户管理</h3>
<p>创建账户、管理权限、监控使用情况以及控制对 Speakr 实例的访问。</p>
<a href="user-management" class="card-link">管理用户 →</a>
</div>
<div class="guide-card">
<div class="card-icon">📊</div>
<h3>系统统计</h3>
<p>监控系统健康状况、跟踪使用模式，并在问题影响用户之前发现潜在问题。</p>
<a href="statistics" class="card-link">查看统计 →</a>
</div>
<div class="guide-card">
<div class="card-icon">🔧</div>
<h3>系统设置</h3>
<p>配置全局限制、超时、文件大小以及影响所有用户的系统级行为。</p>
<a href="system-settings" class="card-link">配置系统 →</a>
</div>
<div class="guide-card">
<div class="card-icon">✨</div>
<h3>默认提示词</h3>
<p>通过默认摘要提示词自定义 AI 行为，塑造内容处理方式。</p>
<a href="prompts" class="card-link">设置提示词 →</a>
</div>
<div class="guide-card">
<div class="card-icon">🔍</div>
<h3>向量存储</h3>
<p>管理语义搜索功能、监控嵌入状态以及控制 Inquire 模式。</p>
<a href="vector-store" class="card-link">管理搜索 →</a>
</div>
</div>
## 快速操作
<div class="action-cards">
<div class="action-card">
<span class="action-icon">➕</span>
<div>
<strong>添加新用户</strong>
<p>用户管理 → 添加用户按钮 → 输入详细信息 → 设置权限</p>
</div>
</div>
<div class="action-card">
<span class="action-icon">📈</span>
<div>
<strong>检查系统健康状况</strong>
<p>系统统计 → 查看指标 → 检查处理状态 → 监控存储</p>
</div>
</div>
<div class="action-card">
<span class="action-icon">⚙️</span>
<div>
<strong>更新设置</strong>
<p>系统设置 → 调整限制 → 配置超时 → 保存更改</p>
</div>
</div>
<div class="action-card">
<span class="action-icon">🔄</span>
<div>
<strong>处理嵌入</strong>
<p>向量存储 → 检查状态 → 处理待处理项 → 监控进度</p>
</div>
</div>
</div>
## 需要管理员帮助？
<div class="help-section">
<div class="help-item">
<span class="help-icon">📖</span>
<span>查看详细的<a href="../troubleshooting.md">故障排除指南</a></span>
</div>
<div class="help-item">
<span class="help-icon">🐛</span>
<span>检查 Docker 日志：<code>docker-compose logs -f app</code></span>
</div>
<div class="help-item">
<span class="help-icon">💾</span>
<span>定期备份你的数据目录</span>
</div>
</div>
---
准备好管理你的 Speakr 实例了吗？从[用户管理](user-management.md)开始 →

91
speakr/docs/admin-guide/prompts.md Normal file
View File

@ -0,0 +1,91 @@
# 默认提示词
默认提示词选项卡允许你塑造 AI 如何在你的整个 Speakr 实例中[解释和总结录音](../features.md#automatic-summarization)。在这里，你可以建立用户在没有自定义[个人提示词](../user-guide/settings.md#custom-prompts-tab)时体验的基线智能。
![默认提示词](../assets/images/screenshots/Admin default prompts.png)
## 理解提示词层级
Speakr 使用复杂的层级来确定在任何给定录音中使用哪个提示词。该系统在保持控制的同时提供灵活性，确保用户获得适当的摘要，同时在需要时允许自定义。
层级的最顶层是标签特定的提示词。当录音具有[标签](../user-guide/settings.md#tag-management-tab)且关联了提示词时，这些具有绝对优先级。在功能指南中了解[标签管理](../features.md#tagging-system)。多个标签提示词会智能地拼接，允许为专业内容类型进行复杂的提示词堆叠。
接下来是用户的个人自定义提示词，在其[账户设置](../user-guide/settings.md#custom-prompts-tab)中设置。用户还可以为摘要配置[语言偏好](../user-guide/settings.md#language-preferences)。这允许个人在不影响其他人的情况下根据自己的需求定制摘要。许多用户从不设置这个，因此你的管理员默认值更加重要。
你在此页面上配置的管理员默认提示词是大多数摘要的基础。这是新用户体验的内容，也是未自定义设置的长期用户所依赖的内容。它决定了你的 Speakr 实例的整体智能和实用性。
最后，如果所有其他方法都失败，硬编码的系统回退确保始终生成摘要。你在实践中很少看到这个，但它提供了一个安全网，确保系统永远不会无法生成输出。
## 设计有效的默认提示词
你的默认提示词不仅仅是技术指令——它是理解内容的模板。界面中显示的提示词演示了一种平衡的方法，要求关键问题、决策和待办事项。这种结构适用于商务会议，但可能不适合所有场景。
在设计提示词时考虑你的用户群体。研究机构可能会强调方法和发现。律师事务所可能会关注案件细节和判例。对于多语言支持，请参阅[语言配置](../features.md#language-support)和[语言问题故障排除](../troubleshooting.md#summary-language-doesnt-match-preference)。创意机构可能会突出概念和客户反馈。提示词应反映对用户最重要的内容。
AI 对清晰、结构化的请求响应最佳。使用项目符号或编号部分来组织输出。指定你想要的详细程度——"简要概述"与"全面分析"会产生截然不同的结果。如果某些格式至关重要，请包含示例。
请记住，此提示词适用于从五分钟检查到两小时研讨会的所有内容。设计时要考虑多样性。避免对可能不适用于所有内容的过于具体的要求。专注于提取具有普遍价值的信息，同时允许 AI 灵活地适应不同的录音类型。
## 默认提示词编辑器
大型文本区域显示你的当前默认提示词，完全支持 markdown 格式。你可以使用粗体进行强调、列表用于结构化，甚至可以使用代码块（如果需要显示示例格式）。编辑器会扩展以适应更长的提示词，尽管简洁通常会产生更好的结果。
当你点击"保存更改"按钮时，更改会立即保存。没有草稿或暂存——修改会立即影响所有新摘要。用户可以[重新生成摘要](../user-guide/transcripts.md)以将更新的提示词应用于现有录音。"重置为默认"按钮提供了安全网，如果你的自定义效果不如预期，可以恢复到原始提示词。
时间戳显示提示词的最后修改时间，有助于跟踪随时间的更改。如果多个管理员管理你的实例，这有助于协调谁更改了什么以及何时更改的。
## 理解 LLM 提示词结构
可展开的"查看完整 LLM 提示词结构"部分揭示了你的提示词如何融入发送给 AI 的完整指令。这个技术视图显示了系统提示词、你的自定义提示词以及转录文本的集成。
理解这个结构有助于你编写更好的提示词。你会看到某些指令已经由系统提示词处理，因此你不需要重复它们。你会理解你的提示词如何与转录文本交互，以及为什么某些措辞比其他措辞更有效。
这种透明性也有助于故障排除。如果摘要不符合预期，查看完整提示词结构通常会揭示原因。也许你的指令与系统指令冲突，或者你可能在请求转录文本中通常不存在的信息。
## 实用提示词策略
从经过验证的结构开始，并根据结果进行迭代。默认提示词之所以有效，是因为它请求具体、可操作的信息。关键问题提供上下文，决策记录结果，待办事项推动跟进。
在广泛部署之前，使用各种录音类型测试你的提示词。适用于正式演示的提示词可能在随意对话中失效。上传具有不同特征的测试录音并评估生成的摘要。
考虑季节性或基于项目的调整。在规划季节，你可能会强调目标和策略。在执行阶段，专注于进展和阻碍因素。随着组织需求的发展，你可以更新默认提示词。
监控用户关于摘要质量的反馈。如果用户经常编辑摘要或抱怨缺少信息，你的提示词可能需要调整。最好的提示词是用户很少需要修改的提示词。
## 高级提示词技巧
分层指令以获得细致的输出。不要只请求"待办事项"，而是指定"带责任方和提及的截止日期的待办事项"。这种精度有助于 AI 在可用时提取更有价值的信息。
使用条件语言增加灵活性。诸如"如果适用"或"当讨论时"之类的短语允许 AI 跳过不适用于每个录音的部分。这可以防止摘要中出现无关的内容。
考虑 AI 模型的优势和局限性。当前模型擅长识别主题、提取具体信息和组织内容。它们在复杂推理、数学计算和未明确陈述的信息方面存在困难。设计发挥这些优势的提示词。
在细节与可读性之间取得平衡。极其详细的提示词可能会生成用户不阅读的综合性摘要。有时简洁、聚焦的摘要比详尽的文档更好地为用户服务。
## 与用户提示词协调
你的默认提示词应补充而非与用户自定义竞争。将其设计为适用于大多数情况的坚实基础，同时鼓励高级用户根据自己的特定需求进行自定义。
向用户传达你的提示词策略。让他们知道默认提示词强调什么，以便他们可以决定自定义是否对他们有利。分享基于你默认值构建的有效用户提示词示例。
考虑为用户记录提示词最佳实践。如果某些部门需要专业摘要，提供他们可以使用的推荐提示词。这赋能了用户，同时在重要的地方保持一致性。
## 衡量提示词有效性
跟踪用户修改 AI 生成摘要的频率。频繁编辑表明你的提示词没有捕获用户需要的内容。最少的编辑表明你的提示词有效地提取了有价值的信息。
定期审查摘要样本。它们是否始终包含请求的部分？信息是否准确且相关？用户是否添加了提示词应该请求的类似信息？
在用户审查或支持交互期间收集反馈。具体询问摘要质量以及默认格式是否满足他们的需求。不自定义提示词的用户完全依赖你的默认值，因此他们的反馈至关重要。
## 常见提示词陷阱
避免过于死板的提示词，将结构强加于不兼容的内容上。并非每个录音都有"决策"或"待办事项"。强制 AI 在不存在时找到这些内容会产生无意义的填充。
不要请求 AI 无法提供的信息。要求"未说出的担忧"或"未讨论的内容"超出了转录分析的范围。AI 只能处理实际说出和记录的内容。
抵制使提示词过长的诱惑。每个指令都会增加复杂性和潜在的混淆。专注于最重要的内容，而不是试图捕获每个可能的细节。
---
下一步：[向量存储](vector-store.md) →

39
speakr/docs/admin-guide/statistics.md Normal file
View File

@ -0,0 +1,39 @@
# 系统统计
系统统计选项卡将原始数据转化为关于你的 Speakr 实例的可操作洞察。一目了然，你可以看到你正在服务多少用户、他们创建了多少录音、他们消耗了多少存储，以及一切是否处理顺畅。
![系统统计](../assets/images/screenshots/Admin stats.png)
## 关键指标概览
统计页面顶部的四个醒目卡片让你立即了解系统的规模。总用户数显示你当前的用户基础规模，帮助你了解实例的覆盖范围。总录音数揭示了系统中的累积内容，而总存储显示了实际消耗的磁盘空间。当启用 Inquire 模式时，总查询数表明用户搜索录音的活跃程度。
这些数字讲述了你的实例的健康状况和增长的故事。不断增长的用户数与按比例增长的录音数表明健康的采用率。存储增长速度超过录音数增长可能表明用户正在上传更长的文件。查询数揭示了用户是否从语义搜索功能中获得价值。
## 录音状态分布
状态分布部分将你的录音分解为四种关键状态。已完成的录音已完全处理并准备好使用——这应该是你内容的绝大多数。处理中的录音当前正在进行转录或分析。待处理的录音已排队等待处理。失败的录音遇到错误需要关注。
在健康的系统中，你会看到大部分是已完成的录音，可能随时只有少数在处理中。大量待处理的录音可能表明你的系统不堪重负或后台处理已停止。失败的录音始终值得调查——它们可能会揭示配置问题、API 问题或用户尝试上传的损坏文件。
## 存储分析
"按存储排名的顶级用户"部分揭示了谁在你的系统中消耗了最多的资源。每个用户都列出了他们的总存储消耗和录音数量，让你了解他们是拥有许多小文件还是较少的大文件。
此信息对于容量规划和用户教育非常有价值。如果一个用户消耗了不成比例的存储，你可能需要更好地了解他们的使用场景。他们是在录制数小时的会议吗？永远保留所有内容？理解数字背后的原因有助于你做出更好的策略决策。
## 理解使用模式
统计不仅仅是数字——它们是等待发现的洞察。录音的突然激增可能与项目启动、学术学期或公司计划相吻合。超过录音增长的存储增长可能表明用户正在上传更长的内容或更高质量的音频文件。
定期监控有助于你在趋势成为问题之前发现它们。如果存储每月增长 10%，你可以预测何时需要扩展容量。如果失败的录音突然激增，你可以调查是否是 API 密钥过期或服务宕机。
## 容量规划
系统统计是你的基础设施需求的水晶球。存储增长趋势告诉你何时需要更多磁盘空间。用户增长模式表明何时可能需要扩展服务器资源。处理队列揭示了当前设置是否能够处理工作负载。
主动使用这些洞察。如果你看到存储以每月 50GB 的速度增长且你有 200GB 可用空间，你知道你在需要干预之前还有大约四个月的时间。这个提前期让你可以预算升级、规划迁移或在达到临界限制之前实施保留策略。
---
下一步：[系统设置](system-settings.md) →

57
speakr/docs/admin-guide/system-settings.md Normal file
View File

@ -0,0 +1,57 @@
# 系统设置
系统设置是你配置影响 Speakr 实例中每个用户和录音的基本行为的地方。这些全局参数塑造系统的运行方式，从技术限制到面向用户的功能。
![系统设置](../assets/images/screenshots/Admin system settings.png)
## 转录文本长度限制
转录文本长度限制决定了在生成摘要或响应聊天时有多少文本会被发送给 AI。这个看似简单的数字对质量和成本都有很大影响。
当设置为"无限制"时，无论长度如何，整个转录文本都会发送给 AI。这确保了 AI 拥有完整的上下文，但对于长录音来说可能会变得昂贵。两小时的会议可能会生成 20,000 字的转录文本，消耗大量 API 令牌并可能超出 AI 模型的上下文窗口。此限制也将应用于说话人识别模态中的说话人自动检测功能。
设置字符限制（如 50,000 个字符）会为 API 消耗设置上限。系统将截断非常长的转录文本，仅将开头部分发送给 AI。这使成本可预测，但可能意味着 AI 会错过录音后面部分的重要内容。
最佳点取决于你的使用场景。对于典型的一小时以内的会议，50,000 个字符通常可以捕获所有内容。对于更长的会话，你可以增加此限制或培训用户拆分录音。监控你的 API 成本和用户反馈以找到正确的平衡。
## 最大文件大小
文件大小限制保护你的系统免受大量上传的压倒性影响，同时确保用户可以使用合理的录音。默认的 300MB 可容纳数小时的压缩音频，这涵盖了大多数使用场景。
提高此限制允许更长的录音，但需要仔细考虑。较大的文件需要更长的上传时间、消耗更多存储，并且可能在处理期间超时。你的服务器需要足够的内存来处理这些文件，并且你的存储必须能够容纳它们。网络超时、浏览器限制和用户耐心都影响着什么是实际的。
如果用户经常触及限制，请考虑他们是否真的需要这么长的单个录音。通常，将长会话拆分为逻辑片段会产生更好的结果——更容易查看、处理更快、摘要更聚焦。
## ASR 超时设置
ASR 超时决定了 Speakr 将等待高级转录服务完成工作的时间。默认的 1,800 秒（30 分钟）可处理大多数录音，但你可能需要根据你的转录服务和典型文件大小进行调整。
设置过低会导致较长的录音即使转录服务正常工作也会失败。录音似乎卡在处理中，然后最终失败，令必须重试或放弃的用户感到沮丧。设置过高会使系统资源等待可能已经失败的服务。
最佳超时取决于你的转录服务的性能和用户的录音长度。监控成功转录的处理时间，并将超时设置为舒适地高于你最长的正常处理时间。如果你经常处理数小时的录音，你可能需要 3,600 秒或更多。
## 录音免责声明
录音免责声明会在用户开始任何录音会话之前出现，非常适合法律通知、策略提醒或使用指南。这个 markdown 格式的消息确保用户在创建内容之前了解他们的责任。
组织通常将其用于合规要求——提醒用户关于同意要求、数据处理策略或适当使用指南。教育机构可能会指出录音仅用于学术目的。医疗机构可能会参考 HIPAA 合规要求。
保持免责声明简洁且相关。用户会频繁看到此消息，因此冗长的法律文本会变成被忽视的点击-through。专注于最重要的要点，并在需要时链接到详细策略。markdown 支持让你可以使用粗体文本进行强调或链接到其他资源来清晰地格式化消息。
## 系统级影响
此页面上的每个设置都会立即影响所有用户。更改在你保存后立即生效，无需系统重启或用户注销。这种即时应用意味着你应该仔细测试更改并将重大修改传达给你的用户。
刷新按钮从数据库重新加载设置，如果多个管理员可能在进行更改或你想确保你看到的是最新值，这将非常有用。界面显示每个设置的最后更新时间，帮助你跟踪随时间的更改。
## 常见问题故障排除
当录音持续失败时，请检查它们是否触及了你配置的限制。错误日志会指示文件是否过大或处理是否超时。用户可能没有意识到他们的录音超出了限制，尤其是如果他们上传现有内容而不是直接录音。
如果 API 成本意外飙升，请检查你的转录文本长度限制。如果未设置限制，单个用户上传许多长录音可能会大幅增加消耗。用户活动与系统设置的组合决定了你的实际成本。
处理积压可能表明你的超时设置过高。如果系统每次失败的转录尝试等待 30 分钟，一系列有问题的文件可能会阻塞队列数小时。在对慢速处理的耐心与服务实际宕机时快速失败之间取得平衡。
---
下一步：[默认提示词](prompts.md) →

47
speakr/docs/admin-guide/user-management.md Normal file
View File

@ -0,0 +1,47 @@
# 用户管理
用户管理选项卡是控制谁可以访问你的 Speakr 实例以及他们可以用它做什么的指挥中心。每个用户账户都从这里流转，从初始创建到最终删除，以及沿途所需的所有监控和调整。有关系统级使用模式，请参阅[系统统计](statistics.md)。
![用户管理](../assets/images/screenshots/Admin dashboard.png)
## 理解用户表格
当你打开用户管理时，你会看到一个显示系统中每个用户的综合表格。每一行都讲述了一个故事——电子邮件地址标识用户，管理员徽章显示他们的权限级别，录音数量揭示他们的活跃程度，存储度量展示他们的资源消耗。
顶部的搜索栏在你输入时会立即响应，过滤表格以帮助你快速找到特定用户。随着你的用户基础增长到超出单个屏幕可容纳的范围，这变得非常宝贵。当你在进行更改时，表格会实时更新，因此你始终看到系统的当前状态。
## 添加新用户
创建用户账户从点击右上角的"添加用户"按钮开始。出现的模态框会询问基本信息——用户名、电子邮件地址和密码。你还需要立即决定此人是否需要管理员权限，尽管你以后可以随时更改。
用户名成为他们在 Speakr 中的身份，出现在界面中并组织他们的录音。电子邮件地址具有双重用途——它是他们的登录凭证以及任何系统通信的联系方式。你设置的密码是临时的；用户应在首次登录后立即通过其[账户设置](../user-guide/settings.md)更改它。配置初始[语言偏好](../user-guide/settings.md#language-preferences)和[自定义提示词](../user-guide/settings.md#custom-prompts-tab)。
管理员权限很强大，应谨慎授予。管理员用户可以看到和修改所有[系统设置](system-settings.md)、管理其他用户（包括其他管理员）、配置[默认提示词](prompts.md)以及监控[向量存储](vector-store.md)。大多数用户永远不需要这些功能。
## 管理现有用户
每个用户行都包含操作按钮，让你对该账户拥有完全控制权。编辑按钮会打开一个模态框，你可以在其中更新他们的用户名或电子邮件地址。当人们更改姓名、切换电子邮件提供商或需要纠正初始输入错误时，这非常有用。
管理员切换可能是界面中最强大的单次点击。将用户提升为管理员授予他们可以访问和执行的所有权限。将管理员降级为普通用户会立即撤销他们的所有管理功能。系统会阻止你从自己的账户中删除管理员权限，确保你不会意外将自己锁定。
删除按钮需要仔细考虑。删除用户是永久性的，无法通过界面撤销。他们的所有录音、笔记和设置将与其账户一起删除。系统会请求确认，但一旦确认，删除立即且彻底。
## 监控使用模式
录音数量和存储列揭示了用户如何与你的 Speakr 实例交互。高录音数量可能表明严重依赖系统的重度用户，而低数量可能表明需要培训或可能根本不需要账户的用户。
存储消耗讲述了另一个重要故事。存储消耗过高的用户可能正在上传非常长的录音、无限期保留所有内容，或者可能滥用系统。你可以根据需要调整[文件大小限制](system-settings.md#maximum-file-size)并审查[分块设置](../troubleshooting.md#files-over-25mb-fail-with-openai)。此信息帮助你进行有关资源使用的知情对话并制定适当的策略。
当你定期审查这些数据时，通常会出现模式。你可能会注意到学术环境中的季节性变化、企业环境中基于项目的激增，或表明需要基础设施扩展的逐渐增长。
## 安全影响
每个用户账户都是潜在的安全向量。强密码是你的第一道防线，但仅靠它们是不够的。鼓励用户使用唯一密码、定期更改密码，并且永远不要与他人共享密码。
管理员账户需要额外的警惕。每个管理员都可以执行你可以做的所有操作，包括创建更多管理员或删除所有用户。将管理员访问限制在操作所需的绝对最小值。当某人不再需要管理员权限时，立即撤销它们。
不活动的账户构成特殊风险。它们可能拥有无人监控的弱密码或已泄露的密码。定期审核有助于你在这些漏洞成为问题之前识别并消除它们。
---
下一步：[系统统计](statistics.md) →

71
speakr/docs/admin-guide/vector-store.md Normal file
View File

@ -0,0 +1,71 @@
# 向量存储管理
向量存储选项卡控制 Inquire 模式背后的智能，这是 Speakr 的语义搜索功能，允许用户使用自然语言问题在其所有录音中查找信息。在这里，你可以监控和管理将转录文本转换为可搜索知识的嵌入系统。
![向量存储管理](../assets/images/screenshots/Admin vector store.png)
## 理解 Inquire 模式
在深入管理之前，了解你在管理什么是有价值的。Inquire 模式将每个转录文本分解为重叠的文本块，将这些块转换为称为嵌入的数学表示，并以可搜索的格式存储它们。当用户提问时，他们的查询会被转换为相同的数学格式，并与所有存储的块进行比较，以找到最相关的信息。
这种方法超越了简单的关键字匹配。系统理解"预算担忧"与"财务约束"和"成本超支"相关，即使确切的词语不同。这种语义理解使 Inquire 模式在发现用户可能无法准确记住的信息时非常强大。
## 嵌入模型
你的 Speakr 实例使用 all-MiniLM-L6-v2 模型，在界面中显著显示。该模型生成 384 维向量——想象每个文本块映射到 384 维空间中的一个点，相似含义聚集在一起。
这个特定模型是经过精心选择的。存在具有更好准确性的更大模型，但它们需要 GPU 和大量计算资源。MiniLM 模型在仅 CPU 的系统上高效运行，使高级搜索无需昂贵的基础设施即可实现。它处理文本速度快、上下文理解好，并生成不会压倒你存储的紧凑嵌入。
该模型最适合英文文本，因为这是其训练数据的主要部分。其他语言可能会以不同的成功率运行，但英文内容始终会产生最可靠的结果。这个限制是模型效率和可访问性的权衡。
## 处理状态概览
状态卡片让你立即了解向量存储的健康状况。总录音数显示你的系统中有多少音频文件，而为 Inquire 处理的数量显示有多少已转换为可搜索的嵌入。这些数字最终应该匹配，尽管随着后台处理赶上通常会存在滞后。
需要处理显示等待嵌入生成的录音。当用户上传新内容时这个数字会增加，随着后台处理器处理队列而减少。持续高数字可能表明处理已停滞或你的系统不堪重负。
总块数显示你的录音已被分解为的细粒度片段。根据转录密度，典型的一小时录音可能会生成 50-60 个块。这种分块确保即使在非常长的录音中也能找到相关片段。
嵌入状态指示器提供快速健康检查。绿色的"可用"表示一切正常运行。其他状态可能表明模型正在加载、处理正在运行或需要注意。
## 处理进度
处理进度条显示通过嵌入队列的实时进展。当达到 100% 时，所有录音都已处理并可搜索。较低的百分比表示正在进行的工作，随着录音完成条形图会填充。
这种视觉反馈帮助你一目了然地了解系统状态。卡住的进度条表明处理已停止。缓慢的进度可能表明系统资源受限。快速的进度显示一切高效运行。
## 管理处理队列
刷新状态按钮更新所有统计信息和进度指示器，用于监控活动处理或验证最近上传的内容已排队。界面不会自动刷新，因此手动刷新确保你看到的是当前信息。
当系统显示录音需要处理但进度没有推进时，可能有几个因素在起作用。后台处理器可能已停止、嵌入模型可能加载失败，或者系统资源可能已耗尽。检查你的日志以获取具体的错误消息。
处理系统被设计为具有弹性。如果特定录音的处理失败，系统会标记它并继续而不是卡住。这些失败会出现在你的日志中，可能需要手动干预来解决。
## 优化性能
处理性能在很大程度上取决于你的系统资源。嵌入模型在加载时需要大约 500MB 的 RAM，再加上处理文本的额外内存。CPU 速度直接影响嵌入生成的速度——现代多核处理器可以同时处理多个录音。
磁盘 I/O 也很重要。系统读取转录文本、处理它们并将嵌入写回数据库。快速存储，特别是 SSD，可以显著提高处理吞吐量。如果你的向量存储与转录文本在不同的磁盘上，请确保两者都有足够的性能。
网络延迟不应该影响处理，因为一切都在本地发生，但数据库性能很重要。定期数据库维护，包括索引优化和清理操作，即使你的向量存储增长也能保持查询快速。
## 常见问题故障排除
当 Inquire 模式在处理过的录音下返回较差的结果时，问题可能是查询表达而非向量存储。鼓励用户问完整的问题而不是输入关键字。"John 关于预算说了什么？"比仅仅"John 预算"效果更好。
如果处理似乎冻结了，请检查 sentence-transformers 库是否正确安装。没有它系统会优雅降级，禁用 Inquire 模式而不是崩溃，但处理不会推进。你的日志会显示嵌入模型是否成功加载。
处理期间的内存错误通常表明你的系统试图同时处理太多内容。分块系统防止单个录音压倒内存，但并行处理多个大录音可能会超出可用 RAM。
## 扩展考虑
向量存储随着你的内容可预测地增长。每个块需要大约 2KB 的存储用于其嵌入和元数据。生成 50 个块的典型一小时录音需要大约 100KB 的嵌入存储。一万小时的录音可能需要 100MB 的嵌入存储——即使在适度的系统上也是可管理的。
得益于高效索引，即使使用大型向量存储，搜索性能仍然保持快速。然而，极其大型的实例（数十万条录音）可能受益于专用向量数据库解决方案，而不是内置的 SQLite 存储。
如果你的实例增长到超出舒适的限制，请考虑归档旧录音。向量存储仅包含活动录音，因此删除过时内容可以同时改善存储和搜索性能。
---
返回[管理员指南概览](index.md) →

347
speakr/docs/assets/css/style.scss Normal file
View File

@ -0,0 +1,347 @@
---
---
@import "{{ site.theme }}";
// Custom colors
:root {
--primary-color: #6366f1;
--secondary-color: #8b5cf6;
--accent-color: #ec4899;
--success-color: #10b981;
--warning-color: #f59e0b;
--danger-color: #ef4444;
--dark-bg: #1f2937;
--light-bg: #f9fafb;
}
// Override theme colors
.page-header {
background: linear-gradient(135deg, var(--primary-color), var(--secondary-color));
padding: 3rem 1rem;
}
.site-footer {
background: var(--dark-bg);
color: #e5e7eb;
padding: 2rem 1rem;
margin-top: 3rem;
}
// Logo in header
.project-logo {
width: 60px;
height: 60px;
margin-bottom: 1rem;
border-radius: 12px;
box-shadow: 0 4px 6px rgba(0, 0, 0, 0.1);
}
.project-name {
display: flex;
align-items: center;
justify-content: center;
gap: 1rem;
}
// Navigation cards
.grid {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(250px, 1fr));
gap: 1.5rem;
margin: 2rem 0;
}
.card {
background: white;
border: 1px solid #e5e7eb;
border-radius: 12px;
padding: 1.5rem;
transition: all 0.3s ease;
box-shadow: 0 1px 3px rgba(0, 0, 0, 0.1);
}
.card:hover {
transform: translateY(-4px);
box-shadow: 0 12px 24px rgba(0, 0, 0, 0.15);
border-color: var(--primary-color);
}
.card h3 {
color: var(--primary-color);
margin-top: 0;
margin-bottom: 0.5rem;
font-size: 1.25rem;
}
.card p {
color: #6b7280;
margin-bottom: 1rem;
line-height: 1.6;
}
.card .btn {
display: inline-block;
background: linear-gradient(135deg, var(--primary-color), var(--secondary-color));
color: white;
padding: 0.5rem 1.25rem;
border-radius: 8px;
text-decoration: none;
font-weight: 500;
transition: opacity 0.3s ease;
}
.card .btn:hover {
opacity: 0.9;
}
// Feature badges
.badge {
display: inline-block;
padding: 0.25rem 0.75rem;
border-radius: 9999px;
font-size: 0.875rem;
font-weight: 500;
margin: 0 0.25rem;
}
.badge-new {
background: var(--success-color);
color: white;
}
.badge-beta {
background: var(--warning-color);
color: white;
}
// Screenshots
img[src*="screenshots"] {
border: 1px solid #e5e7eb;
border-radius: 12px;
box-shadow: 0 4px 12px rgba(0, 0, 0, 0.1);
margin: 1.5rem 0;
max-width: 100%;
height: auto;
}
// Code blocks
pre {
background: #1f2937;
color: #e5e7eb;
border-radius: 8px;
padding: 1rem;
overflow-x: auto;
margin: 1rem 0;
border: 1px solid #374151;
}
code {
background: #f3f4f6;
padding: 0.125rem 0.375rem;
border-radius: 4px;
font-size: 0.875em;
color: var(--primary-color);
}
pre code {
background: none;
padding: 0;
color: inherit;
}
// Tables
table {
width: 100%;
border-collapse: separate;
border-spacing: 0;
border: 1px solid #e5e7eb;
border-radius: 8px;
overflow: hidden;
margin: 1.5rem 0;
}
th {
background: var(--light-bg);
padding: 0.75rem;
text-align: left;
font-weight: 600;
color: #374151;
border-bottom: 2px solid #e5e7eb;
}
td {
padding: 0.75rem;
border-bottom: 1px solid #e5e7eb;
}
tr:last-child td {
border-bottom: none;
}
tr:hover {
background: #f9fafb;
}
// Alerts and callouts
.note, .warning, .tip {
padding: 1rem 1.25rem;
border-radius: 8px;
margin: 1.5rem 0;
border-left: 4px solid;
}
.note {
background: #eff6ff;
border-color: var(--primary-color);
color: #1e40af;
}
.warning {
background: #fef3c7;
border-color: var(--warning-color);
color: #92400e;
}
.tip {
background: #d1fae5;
border-color: var(--success-color);
color: #065f46;
}
// Sidebar navigation
.sidebar {
background: white;
border-radius: 12px;
padding: 1.5rem;
border: 1px solid #e5e7eb;
margin-bottom: 2rem;
}
.sidebar h3 {
color: var(--primary-color);
margin-top: 0;
margin-bottom: 1rem;
font-size: 1.125rem;
}
.sidebar ul {
list-style: none;
padding: 0;
margin: 0;
}
.sidebar li {
margin-bottom: 0.5rem;
}
.sidebar a {
color: #6b7280;
text-decoration: none;
display: block;
padding: 0.5rem 0.75rem;
border-radius: 6px;
transition: all 0.2s ease;
}
.sidebar a:hover {
background: var(--light-bg);
color: var(--primary-color);
}
.sidebar a.active {
background: linear-gradient(135deg, var(--primary-color), var(--secondary-color));
color: white;
}
// Feature list
.feature-list {
list-style: none;
padding: 0;
}
.feature-list li {
padding: 0.75rem 0;
border-bottom: 1px solid #e5e7eb;
display: flex;
align-items: flex-start;
}
.feature-list li:last-child {
border-bottom: none;
}
.feature-list li:before {
content: "✓";
color: var(--success-color);
font-weight: bold;
margin-right: 0.75rem;
font-size: 1.25rem;
}
// Responsive design
@media (max-width: 768px) {
.grid {
grid-template-columns: 1fr;
}
.page-header {
padding: 2rem 1rem;
}
.project-name {
font-size: 1.75rem;
}
}
// Smooth scrolling
html {
scroll-behavior: smooth;
}
// Better typography
.main-content {
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, "Helvetica Neue", Arial, sans-serif;
line-height: 1.7;
color: #374151;
}
h1, h2, h3, h4, h5, h6 {
font-weight: 600;
line-height: 1.3;
margin-top: 2rem;
margin-bottom: 1rem;
}
h1 {
font-size: 2.5rem;
color: var(--primary-color);
}
h2 {
font-size: 2rem;
color: #1f2937;
border-bottom: 2px solid #e5e7eb;
padding-bottom: 0.5rem;
}
h3 {
font-size: 1.5rem;
color: #374151;
}
// Animations
@keyframes fadeIn {
from {
opacity: 0;
transform: translateY(10px);
}
to {
opacity: 1;
transform: translateY(0);
}
}
.main-content > * {
animation: fadeIn 0.5s ease-out;
}

BIN
speakr/docs/assets/images/logo.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 229 KiB

BIN
speakr/docs/assets/images/main2.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 380 KiB

BIN
speakr/docs/assets/images/screenshots/Admin dashboard.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 63 KiB

BIN
speakr/docs/assets/images/screenshots/Admin default prompts.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 131 KiB

BIN
speakr/docs/assets/images/screenshots/Admin stats.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 83 KiB

BIN
speakr/docs/assets/images/screenshots/Admin system settings.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 144 KiB

BIN
speakr/docs/assets/images/screenshots/Admin vector store.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 105 KiB

BIN
speakr/docs/assets/images/screenshots/Filters.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 304 KiB

BIN
speakr/docs/assets/images/screenshots/Multilingual.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 410 KiB

BIN
speakr/docs/assets/images/screenshots/Record mic or system audio (for online meetings) and take markdown notes.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 213 KiB

BIN
speakr/docs/assets/images/screenshots/Summary View.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 327 KiB

BIN
speakr/docs/assets/images/screenshots/after recording, finalize tags and settings and notes before uploading or discard.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 155 KiB

BIN
speakr/docs/assets/images/screenshots/chat view with question.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 331 KiB

BIN
speakr/docs/assets/images/screenshots/edit recording modal.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 461 KiB

BIN
speakr/docs/assets/images/screenshots/edit recording tags modal.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 460 KiB

BIN
speakr/docs/assets/images/screenshots/edit transcript with ASR.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 156 KiB

BIN
speakr/docs/assets/images/screenshots/event extraction.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 211 KiB

BIN
speakr/docs/assets/images/screenshots/full recording reprocessing.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 523 KiB

BIN
speakr/docs/assets/images/screenshots/iinquire mode contextual searches.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 211 KiB

BIN
speakr/docs/assets/images/screenshots/inquire mode 2.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 254 KiB

BIN
speakr/docs/assets/images/screenshots/inquire mode.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 225 KiB

BIN
speakr/docs/assets/images/screenshots/live recording view with notes.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 151 KiB

BIN
speakr/docs/assets/images/screenshots/main view no diarization.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 338 KiB

BIN
speakr/docs/assets/images/screenshots/new recording view.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 173 KiB

BIN
speakr/docs/assets/images/screenshots/notes tab.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 357 KiB

BIN
speakr/docs/assets/images/screenshots/record from screen.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 85 KiB

BIN
speakr/docs/assets/images/screenshots/record from tab.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 50 KiB

BIN
speakr/docs/assets/images/screenshots/recording view with simple transcription view and chat visible.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 311 KiB

BIN
speakr/docs/assets/images/screenshots/settings Tags that allow custom prompts stackable and custom asr settings.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 82 KiB

BIN
speakr/docs/assets/images/screenshots/settings about page.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 155 KiB

BIN
speakr/docs/assets/images/screenshots/settings account info.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 124 KiB

BIN
speakr/docs/assets/images/screenshots/settings custom prompts.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 95 KiB

BIN
speakr/docs/assets/images/screenshots/settings shared transcripts.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 48 KiB

BIN
speakr/docs/assets/images/screenshots/settings speakr management.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 154 KiB

BIN
speakr/docs/assets/images/screenshots/share recording modal.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 464 KiB

BIN
speakr/docs/assets/images/screenshots/share recording.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 219 KiB

BIN
speakr/docs/assets/images/screenshots/shared recording.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 134 KiB

BIN
speakr/docs/assets/images/screenshots/shared transcript modal.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 23 KiB

BIN
speakr/docs/assets/images/screenshots/speaker id modal.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 457 KiB

BIN
speakr/docs/assets/images/screenshots/summary reprocessing modal.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 529 KiB

BIN
speakr/docs/assets/images/screenshots/tag selection and stacking.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 20 KiB

BIN
speakr/docs/assets/images/speakr-logo.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 229 KiB

89
speakr/docs/ci-cd-setup.md Normal file
View File

@ -0,0 +1,89 @@
# Speakr CI/CD 设置
本文档解释了 Speakr 项目的持续集成和持续部署（CI/CD）流水线是如何设置的。
## 自动 Docker 镜像构建与发布
Speakr 项目使用 GitHub Actions 在代码推送到仓库时自动构建 Docker 镜像并发布到 Docker Hub。
### 工作原理
1. 当代码推送到 `master` 分支或创建新标签（格式为 `v*.*.*`）时，GitHub Actions 工作流将被触发。
2. 工作流使用项目的 Dockerfile 构建 Docker 镜像。
3. 如果触发原因是推送到 `master` 分支，镜像将被标记为：
- `latest`
- 提交的短 SHA 值
- 分支名称（`master`）
4. 如果触发原因是新标签（例如 `v1.2.3`），镜像将被标记为：
- 完整版本号（`1.2.3`）
- 主.次版本号（`1.2`）
- 标签名称（`v1.2.3`）
5. 然后镜像会被推送到你账户下的 Docker Hub。
### 设置步骤
要设置自动 Docker 镜像发布，请按照以下步骤操作：
1. **创建 Docker Hub 账户**（如果你还没有的话）：
- 访问 [Docker Hub](https://hub.docker.com/) 并注册账户。
- 创建一个名为 `speakr` 的仓库。
2. **创建 Docker Hub 访问令牌**：
- 登录 Docker Hub。
- 点击右上角的用户名，选择"Account Settings"进入账户设置。
- 导航到"Security"选项卡。
- 点击"New Access Token"。
- 给令牌取个名字（例如"GitHub Actions"）并设置适当的权限（至少需要"Read & Write"）。
- 复制生成的令牌（你将无法再次看到它）。
3. **将 Docker Hub 凭据添加到 GitHub Secrets**：
- 进入你的 GitHub 仓库。
- 点击"Settings" > "Secrets and variables" > "Actions"。
- 点击"New repository secret"。
- 添加以下密钥：
- 名称：`DOCKERHUB_USERNAME`，值：你的 Docker Hub 用户名
- 名称：`DOCKERHUB_TOKEN`，值：你在上一步生成的访问令牌
4. **验证工作流**：
- 工作流文件位于 `.github/workflows/docker-publish.yml`。
- 它已经配置好可以构建并推送 Docker 镜像到 Docker Hub。
- 如果需要，你可以进一步自定义它。
5. **触发工作流**：
- 推送更改到 `master` 分支或创建新标签来触发工作流。
- 进入 GitHub 仓库的"Actions"选项卡来监控工作流进度。
### 使用已发布的 Docker 镜像
镜像发布到 Docker Hub 后，你可以在 docker-compose.yml 文件中使用它们：
```yaml
version: '3.8'
services:
app:
image: learnedmachine/speakr:latest
container_name: speakr
restart: unless-stopped
ports:
- "8899:8899"
volumes:
- ./uploads:/data/uploads
- ./instance:/data/instance
env_file:
- .env
```
### 版本控制策略
要创建新版本的应用程序：
1. 进行更改并提交到仓库。
2. 按照语义化版本控制创建新标签：
```bash
git tag v1.0.0 # 替换为适当的版本号
git push origin v1.0.0
```
3. GitHub Actions 工作流将自动构建并发布带有相应版本标签的新 Docker 镜像。
这样用户可以选择使用最新版本（`learnedmachine/speakr:latest`）或特定版本（`learnedmachine/speakr:1.0.0`）。

Some files were not shown because too many files have changed in this diff Show More