---
name: audio-transcription
description: "Transcribe local or linked audio/video files with timestamps; handle oversized files and Google Drive links."
version: 1.0.0
author: Hermes Agent
license: MIT
platforms: [linux, macos, windows]
metadata:
  hermes:
    tags: [audio, transcription, whisper, subtitles, google-drive, media]
    related_skills: [youtube-content, ocr-and-documents, google-workspace]
---

# Audio Transcription

Use when the user asks to transcribe audio/video, convert voice recordings to text, produce subtitles, or work around chat upload limits for media files.

## Core workflow

1. **Get the media file**
   - If uploaded locally, use the provided file path.
   - If the platform rejects the file because it is too large, ask for a public cloud link or for the file to be split/compressed.
   - For Google Drive links, see `references/google-drive-public-download.md`.
   - For Google Drive *folder* links, see `references/google-drive-folder-batch.md`: download the whole folder first (e.g. `gdown --folder URL -O /path/to/downloads`) and then process every media file found; users often send folders containing several recordings.
2. **Verify media metadata**
   - Use `ffprobe` to check duration, codec, channels, and size before transcribing.
   - For multiple files, loop over files and run `ffprobe` once per file; do not pass several input filenames to one `ffprobe` invocation.
   - Tell the user the approximate duration if the job may take time.
3. **Transcribe with timestamps**
   - Prefer `faster-whisper` when available for local transcription.
   - Use `scripts/transcribe_with_faster_whisper.py` to produce both `.txt` and `.srt` outputs.
   - For long recordings (roughly 20+ minutes on CPU, or any job likely to approach foreground timeout), start transcription as a tracked background process with `notify_on_complete=true` from the beginning instead of first trying a foreground 600s run. A 30-minute `.oga`/Opus voice note with `small` CPU can exceed 10 minutes wall time.
   - For several files, run them sequentially in one tracked background process with `notify_on_complete=true`, so the job can run for 10–30+ minutes without blocking the session.
   - Treat periodically written `.txt`, `.srt`, or segment JSON files as **checkpoints**, not proof of completion. A long-running transcriber may flush apparently substantial files while still processing. Mark a recording complete only after the tracked process exits successfully and the final transcript timestamp is checked against `ffprobe` duration (allowing a short trailing-silence gap).
   - When another urgent audio interrupts a sequential batch, pause/resume the tracked worker only if necessary, verify it resumed, and report the batch by exact state: complete, actively processing, or pending. Never infer completion merely from file presence or modification time.
   - Treat a multi-file batch as incomplete until every requested transcript, translation, derivative document, package, and verification step exists. Downloading media or merely starting ASR is progress, not completion.
   - Maintain an explicit checklist for long batches and update it only from verified artifacts. Before telling the user a paused or background batch “continued automatically,” inspect the process and confirm that it is actually running.
   - Avoid simultaneous CPU-heavy ASR jobs on constrained hosts. If an urgent short recording interrupts a long batch, pause the long worker, process the short item, and resume the worker with a shell `trap` so it resumes even if the short job fails. Verify both the short artifact and the resumed worker afterward.
4. **Extract the deliverable the user actually asked for**
   - If the user asks to “вычлени вопросы”, “достань вопросы”, “only questions”, or similar, transcribe first, then return only cleaned copy-ready questions unless they also asked for the full transcript.
   - Preserve the user’s language and wording style where possible, but remove filler, false starts, duplicate fragments, and transcription noise.
   - For question extraction, a useful output is: one question per line or a numbered list, no timestamps unless requested, no summary unless requested.
5. **Deliver useful artifacts**
   - On Telegram, deliver long transcript-derived analyses, summaries, and structured extractions as ordinary UTF-8 `.txt` files by default, even if Markdown was used internally. Copy the finished document into the Hermes media/cache directory, use a short mobile-friendly filename with a simple hyphen rather than typographic punctuation, verify encoding/size/line count, and send it with `MEDIA:`. Do not make `.md` the only user-facing artifact: many mobile clients do not open it conveniently. If the user says a file does not open, immediately resend the same complete content as `.txt` rather than asking them to troubleshoot.
   - For multilingual batches, preserve separate layers per recording: source-language timestamped transcript, translated clean text, and (when requested) a combined bilingual document. Never overwrite the source transcript with its translation.
   - For long transcript translations, split only at complete timestamped segments into chunks small enough to finish (start around 300–400 segments or 15–20 minutes), translate chunks in parallel, then merge mechanically. Do not begin with a whole hour-long file when worker calls are time-bounded.
   - Treat a worker timeout as an uncertain outcome, not automatic failure: inspect the requested output path. A timed-out worker may have written a complete or partial file. For translations, count timestamps and check first/last markers. For long analyses or structured extractions, inspect headings, the final paragraphs, temporary part files, and explicit continuation markers; preserve completed sections, then re-dispatch only the missing sections with a precise continuation brief. Never restart a substantial analysis merely because the worker reported `timeout`.
   - Before delivery, compare source and translation timestamp sequences positionally: same count and order, with no omissions, extras, duplicates, malformed lines, empty translations, or unexpected untranslated Latin-only segments (allow proper names, brands, and abbreviations).
   - Verify the requested subset against the downloaded folder: cloud folders may contain extra media/documents unrelated to the named files. Process the explicitly named recordings first and inventory extras separately instead of silently including or deleting them.
   - A timeout does not imply that no artifact was written. Inspect any output first: count timestamped source and translation segments, compare their ordered timestamp sequences, and identify the exact last completed timestamp. Resume only the missing suffix (with a small checked overlap if needed), rather than restarting or trusting the agent's status.
   - Before calling a translation complete, verify: source and translation timestamp counts match; timestamp sequences are identical and ordered; first and final timestamps match; every timestamp has non-empty target-language text; and no chunk-boundary segment is duplicated. A file existing, having similar line count, or reaching a late timestamp is insufficient by itself.
   - Verify the requested subset against the downloaded folder: cloud folders may contain extra media/documents unrelated to the named files. Process the explicitly named recordings first and inventory extras separately instead of silently including or deleting them.
   - For energetically expanding question sets, extract the recording’s central tensions first, then write questions that expand choice and connect inner knowing with embodied/material support. Avoid deterministic promises, false certainty, coercive yes/no framing, and advice that encourages abrupt destabilizing action. Prefer open stems such as «Каким образом…», «Что ещё возможно…», «Какой следующий шаг…».
   - Verify the requested subset against the downloaded folder: cloud folders may contain extra media/documents unrelated to the named files. Process the explicitly named recordings first and inventory extras separately instead of silently including or deleting them.
   - Return the timestamped `.txt` transcript as the main artifact when the user asked for transcription.
   - When the user explicitly asks “в txt / файлом txt”, create a clean UTF-8 `.txt` in a deliverable media/cache path (e.g. `~/.hermes/media/transcripts/...txt`) and include `MEDIA:/absolute/path.txt` in the final response; also paste a short excerpt or the full text if it is short.
   - For long transcript-derived deliverables on Telegram—deep analyses, extractions, translations, summaries, or compiled notes—default to an ordinary UTF-8 `.txt` attachment unless the user specifically requested Markdown, DOCX, PDF, or another format. Do not send `.md` as the primary user-facing artifact merely because Markdown was convenient internally.
   - Keep an internal `.md` version if useful, but create and verify the user-facing `.txt` copy separately. Prefer a simple attachment filename without typographic punctuation when compatibility is uncertain (for example, `Королевство 1 - глубокий разбор.txt` rather than an em dash). Verify encoding/type and non-zero line/byte counts before delivery.
   - Include `.srt` when subtitles may be useful.
   - On chat platforms, copy finished artifacts into an allowed Hermes media/cache directory before using `MEDIA:`; files left only in `/tmp` may not render for the user.
   - If the user says a file does not open, do not ask them to troubleshoot it: immediately resend the same content as a verified UTF-8 `.txt` from media cache with a simple filename. If that still fails or they ask for text in chat, paste it in numbered chunks with timestamp ranges and continue on “дальше/next”.
   - Warn that machine transcription can mishear names, terminology, fast speech, and low-quality sections.
5. **Offer a second pass**
   - If the user needs publication-ready material, offer cleanup, summary, chapters, quotes, posts, or show notes after the raw transcript is complete.

6. **When the user asks to extract questions from audio**
   - Transcribe first, then read the transcript and return only the questions in a clean copy-paste block unless the user asks for context.
   - Include both explicit questions and implied spoken question stems that ASR may punctuate as statements (e.g. “где…”, “что если…”, “какая…”, “что потребуется…”, “как это может быть ещё лучше”).
   - Clean obvious ASR errors and normalize phrasing, but preserve the user’s energetic style and repeated Access-like stems when they are part of the practice.
   - Do not include summaries, timestamps, explanations, or duplicates unless useful. If the user wants “все вопросы”, err on inclusion and make them copy-ready.

6. **Extract questions on request**
   - If the user asks to “вычлени/собери все вопросы” from an audio, do not deliver the full transcript unless asked. Transcribe first, then read the transcript and output only the questions.
   - Include both explicit interrogative sentences and spoken/implicit question stems that ASR may punctuate as statements (e.g. “где…”, “что потребуется…”, “какая…”, “что ещё возможно”, “как это может быть ещё лучше”).
   - For Margarita/Cosmos-style voice notes, err on inclusion: capture rhetorical questions, teaching prompts, Access-like expansion questions, and questions addressed to the listener. Remove duplicated stutters, but keep distinct variants even when they are close in meaning.
   - Clean obvious ASR errors and filler words, but preserve the user’s energetic/copywriting meaning and order of appearance.
   - Format as a plain copy-paste-ready numbered list or one fenced `text` code block when the user asks “для копирования”.
   - Do not add commentary, timestamps, summaries, or explanations unless requested.

## Supporting references

- For multi-hour multilingual batches, chunked translation, timeout recovery, and completion checks, read `references/long-multilingual-batch-verification.md`.
- For reconstructing a forwarded chat from many voice notes, text replies, duplicates, and images, read `references/chat-voice-transcript-reconstruction.md`.

## Forwarded-chat reconstruction contract

When the requested artifact is a chronological conversation rather than a standalone recording:

1. Recover and inventory the authoritative source messages before editing; do not rebuild missing long passages from a compaction summary.
2. Deduplicate only complete repeated messages, preserving natural repetition inside speech and the first chronological occurrence.
3. Keep a **complete cleaned transcript** separate from any optional abridged reading version. Never substitute a shortened document plus an external “full text” link when the user asked for the full transcript in the target document.
4. Attribute speakers from explicit addresses, reply metadata, and stable conversational roles; mark genuine uncertainty instead of guessing.
5. Preserve text replies, emoji, links, captions, and image positions as distinct chronological turns.
6. Use unique image placeholders during assembly, then replace every placeholder with an inline embedded image before completion unless the user explicitly accepts links.
7. Re-export the finished document and verify beginning/middle/end sentinels, turn counts after documented deduplication, and zero remaining media placeholders.

## Commands

Check metadata:

```bash
ffprobe -v error -show_entries format=duration,size,format_name:stream=codec_name,channels,sample_rate -of json /path/to/audio.m4a
```

Transcribe:

```bash
python SKILL_DIR/scripts/transcribe_with_faster_whisper.py /path/to/audio.m4a --language ru --model small --outdir /tmp/transcript_job
```

For long files, use a tracked background process and poll/wait until complete.

## Quality notes

- `small` with CPU `int8` is a practical default for clear speech; use a larger model only if accuracy matters more than runtime.
- For jargon-heavy energetic practices, a clean first pass may still hallucinate repeated formulas or mangle specialist terms. Re-transcribe short **overlapping chunks** around uncertain passages with `condition_on_previous_text=False`, `word_timestamps=True`, and an `initial_prompt` containing only plausible domain vocabulary. Compare both passes and mark any remaining reconstruction as uncertain rather than presenting it as verbatim.
- If the user wants functional decoding, preserve three layers separately: (1) confirmed transcript, (2) uncertain/reconstructed wording, and (3) your interpretation of the phrase’s function. Never let the interpretation silently repair the transcript.
- `vad_filter=True` usually improves long recordings by skipping silence.
- Always preserve timestamps for full transcriptions; they make later cleanup and corrections much easier.
- For terminology-heavy energetic practices, repeated ritual phrases, or mixed Russian/English formulas, treat the first transcript as a draft. Re-transcribe unclear timestamp ranges as overlapping chunks with `condition_on_previous_text=False`, a targeted `initial_prompt` containing likely terms, and word timestamps. Compare repeated occurrences inside the recording and label unresolved words instead of silently normalizing them.
- When the user says another recording/source is coming and asks for comparison, finish collecting and transcribing all promised sources before beginning phrase-by-phrase editing or approval.
- Prefer a file artifact for very long transcripts, but user preference wins: if they ask to receive the transcript directly in chat, paste it in manageable chunks instead of arguing for an attachment.
- When summarizing a completed transcript, read the full transcript back first, then produce a structured extraction of the main ideas rather than a generic summary.

## Long multilingual batches and translation verification

When a folder contains several hour-long recordings and some require translation:

1. Inventory and probe every named recording first; record language, duration, and expected deliverables.
3. Transcribe sequentially in one tracked background process, but do not describe the whole package as “processing” unless the live process is verified. On every status request, inspect actual files **and** process state. For a strategic source corpus (course recordings, resource sessions, launch studies), preserve one transcript per recording and keep source teaching, participant material, and later interpretation as separate layers; do not synthesize audience, funnel, or methodology until the user's declared collection endpoint.
3. Treat a transcript file being present as provisional while its transcriber is still running; partial checkpoints may look complete. A recording is complete only after the process reports `DONE` and the final timestamp has been checked against the media duration.
4. For full translations, do not give one worker an entire 60–80 minute transcript if the worker has a 10-minute limit. Split by complete timestamped segments into bounded chunks (typically 250–400 segments; lower the size after any timeout), translate chunks in parallel, then merge mechanically.
5. After a worker timeout, inspect whether a partial output was saved. Continue only after its last verified timestamp instead of restarting or assuming failure. Never call a timed-out translation complete merely because the path exists.
6. Verify each translated artifact one-to-one against the source:
   - identical timestamp count;
   - identical timestamp sequence;
   - same first and final timestamp;
   - no empty, duplicated, omitted, or malformed segments;
   - no English-only segments except proper names, brands, abbreviations, or explicitly marked uncertain wording.
7. Package only after all validations pass. On Telegram, deliver ordinary `.txt` files individually by default; a ZIP may be offered as an optional convenience, not as the only deliverable. If the user reports that an archive does not open, immediately resend the individual files without asking them to troubleshoot the archive.
8. State progress with precise categories: **готово**, **в работе**, **остаётся**. Do not say “готово полностью” until every promised language layer and verification step exists on disk.

For deterministic checks and assembly, prefer a short script that reads timestamped lines, compares timestamp arrays, and fails on any mismatch before copying files into the delivery directory.

## Pitfalls

- A Google Drive link that opens for the user may still be private to the agent. If direct download returns a Google sign-in page, ask the user to set sharing to “Anyone with the link / Viewer”.
- A `file:///Users/...` or other local desktop path points to the user’s computer, not the Hermes server. Do not try to transcribe it as if it were accessible; ask the user to upload the media file to chat or provide a public cloud link. If they are on Telegram and the file is large, suggest Drive/Dropbox/iCloud link, compression, or splitting.
- Chat upload limits are transport limits, not transcription limits. Use Drive/Dropbox/etc., split the audio, or compress speech to MP3 mono 64–96 kbps.
- On Telegram, a user may correctly attach an audio/video file but the gateway can still pass an empty message to the agent if the bot cannot download it. Check gateway logs for `Failed to cache audio` / `File is too big`; explain that the attachment existed but Telegram Bot API download failed, then ask for a Drive/Dropbox link, a compressed file, or split parts. Do not tell the user “there is no audio” until checking for this delivery failure when they insist it was attached.
- If the user asks to “expand permissions” for large Telegram uploads, do not look for an agent permission switch. The durable fix is a local Telegram Bot API server configured through Telegram `base_url`/`base_file_url`/`local_mode`; see `references/telegram-large-file-downloads.md`.
- Do not encode a temporary missing package as a durable limitation. If `faster-whisper` is missing, install it or choose another available transcriber.