---
name: daily-book-extraction
description: "Set up recurring daily digests from uploaded books/PDFs with accumulating tools, questions, practices, and insights."
version: 1.0.0
author: Hermes Agent
license: MIT
platforms: [linux, macos, windows]
metadata:
  hermes:
    tags: [books, pdf, daily-digest, cron, extraction, summaries, practices]
    related_skills: [ocr-and-documents, cosmos-luminosity-content]
---

# Daily Book Extraction

Use this skill when a user uploads a book/PDF/ebook and wants recurring daily summaries, chapter digests, practice extraction, or an accumulating knowledge/tool file.

## Goal

Turn a long book into a stateful daily reading/extraction pipeline that:

1. stores the source files in a stable project directory;
2. extracts text locally when needed;
3. splits the content into digestible daily fragments;
4. emits one fragment per scheduled run;
5. sends the user a clear daily digest;
6. appends tools, practices, questions, techniques, and insights to a durable Markdown file.

## Workflow

1. **Load extraction context**
   - If the source is PDF/scanned/book-like, consult `ocr-and-documents` for extraction options.
   - For user-specific style/content requirements, load the relevant content skill (for Margarita/Cosmos Luminosity, load `cosmos-luminosity-content`).

2. **Create a stable project directory and register the book**
   - For Margarita / Cosmos Luminosity, every uploaded book in any supported format must also be added to the unified Portal library register at `/home/spazio/cosmos_luminosity_encyclopedia/content/books-library.md`, with source path, extraction status, digest status, and relevant Portal links. Do this automatically rather than waiting for a separate cataloguing request.
   - Example: `/home/spazio/<project_name>/`
   - Suggested subfolders:
     - `pdf/` or `source/`
     - `text/`
     - `daily_fragments/`
     - `daily_outputs/`
   - Suggested files:
     - `README.md`
     - `manifest.json`
     - `state.json`
     - `TOOLS_AND_INSIGHTS.md` or another accumulating output file.

3. **Extract and verify text**
   - For text PDFs, use `pymupdf` / `fitz`.
   - For FB2 / `.fb2.zip`, load `ocr-and-documents` and use its `scripts/extract_fb2_chapters.py` helper to produce per-chapter `.txt` files and `manifest.json`. Inspect the manifest before scheduling: short copyright/title/dedication sections may appear as the first “chapters”; start the digest state from the first substantive section (usually chars > ~1000), not from metadata.
   - For uploaded documents copied from Hermes cache, either preserve the original basename explicitly or refer to the actual copied cache filename returned by file listing; Telegram/cache filenames may be prefixed and not match the visible upload name.
   - For uploaded ZIP ebooks, inspect the archive first; the inner `.doc` may actually be RTF (common with Royallib-style exports). If `file` reports Rich Text Format, decode as RTF rather than Word. Russian RTF may need cp1251 handling for `\\'xx` escapes.
   - For Litmir-style EPUBs, parse `content.opf` and read HTML/XHTML files in OPF spine order. Skip `cover.xhtml` / cover-only text so the first scheduled digest is substantive, not a metadata fragment. See `references/litmir-epub-daily-extraction.md`.
   - Preserve page, chapter, or paragraph markers in extracted text so daily digests can cite ranges.
   - When detecting chapter titles from PDF text, capture only the heading line (or use an explicit known title map). Do not use a multiline regex that can absorb the first sentence into the title. Inspect the generated manifest and correct malformed titles before scheduling.
   - Verify page/chapter counts when available, nonempty text, character counts, and that the first fragment can be read before scheduling.

4. **Split into daily fragments**
   - Chunk by chapter if chapters are clean.
   - Otherwise chunk by approximate character count while preserving page markers.
   - For dense Russian transcripts/books, ~12k–18k chars per fragment is usually enough for substantive extraction without overflowing the model.

5. **Create a transactional next-fragment pipeline**
   - The collector script should read `manifest.json` and `state.json` and output JSON containing:
     - `status`
     - `book`
     - `fragment_number`
     - `total_fragments`
     - source range / chapter range
     - relevant output paths
     - `fragment_text`
   - Prefer a **read-only collector**: it must not advance `state.json` merely because it emitted a fragment. A cron run may be blocked after the script executes, and pre-advancing would silently skip content.
   - Put the commit step in the cron task: only after the per-fragment report and accumulating Markdown have both been written successfully should the agent set `next_fragment = processed_fragment + 1` while preserving unrelated state fields.
   - Keep `--peek` when useful for compatibility/testing, but it should be equivalent to the normal read-only path.
   - If repairing an older pipeline that advanced state before the agent ran, reconcile `state.json` against verified output files and the accumulating Markdown before retrying. Do not infer successful delivery from the state counter alone.

6. **Install the script for Hermes cron**
   - Cron scripts must live under `~/.hermes/scripts/`.
   - When creating the cron job, pass only the relative script filename, not an absolute path.
   - Example: `script="cosmos_book_next_fragment.py"`, not `/home/.../next_fragment.py`.

7. **Mirror the accumulating Markdown into a web-readable page when useful**
   - If the user asks to “watch what is accumulating” from a phone/browser, do not build a second data store. Make the public/website Markdown page the source of truth, and symlink the book pipeline’s `TOOLS_AND_INSIGHTS.md` to that page.
   - Preserve the website page’s YAML frontmatter when appending daily sections; tell the cron prompt explicitly not to delete frontmatter.
   - Add the page to the site/wiki index if the site uses content discovery via Markdown articles.
   - Verify that the page exists and is reachable by URL before reporting success. If a public wiki/Markdown site already exists, prefer adding a new article there over creating a separate narrow one-off server.

8. **Create or update the cron job**
   - Convert the user’s timezone into a cron schedule in the scheduler timezone when needed. Check `next_run_at` after creation; if the scheduler is UTC and the user expects Asia/Singapore / Samui / Bali, 15:30 local is `30 7 * * *`, not `30 15 * * *`.
   - Interpret intervals literally. If Margarita says «раз в пять дней», use an every-five-days interval rather than a daily cron or five selected weekdays. The first digest is immediate; the next scheduled digest is five days after the successful first delivery.
   - If the user first asks to schedule before uploading the book, create a safe placeholder that asks for the source rather than inventing content. When the source arrives, extract it, create the stateful next-fragment script, then update the existing cron job to use that script (or remove/recreate if needed) so future runs have real source context.
   - If the user says “first day today” / “start today”, emit the first real digest immediately after extraction and advance the state exactly once, then ensure the recurring cron starts from the next chapter/fragment.
   - For interval requests such as “every five days,” schedule by duration (`every 5d`) unless the user names a clock time. Verify the returned `next_run_at` and report the exact date of the next digest.
   - If there are N total fragments and the first is delivered immediately, set the recurring repeat count to N−1 and initialize state to fragment 2. Do not add an extra completion-only delivery unless the user explicitly wants one.
   - Make the cron prompt self-contained: future runs do not have the current chat context.
   - Prefer direct file tools (`write_file`, `patch`) for manifests, state, README, library registration, and extracted text when practical. If a terminal command triggers an approval gate, stop and ask the user to approve that exact pending action; after approval, retry once and verify artifacts before proceeding. Do not repeatedly report that a textual “yes” should satisfy a UI approval gate.
   - Make the cron prompt self-contained: future runs do not have the current chat context.
   - Attach useful skills to the cron job.
   - Set `attach_to_session=true` for recurring digests the user may reply to.

8. **Write the cron prompt carefully**
   - Tell the future agent to:
     - summarize/extract in the user’s preferred language and style;
     - avoid long copyrighted quotations;
     - extract tools, techniques, practices, questions, ideas, and daily applications;
     - append to the accumulating Markdown file;
     - save a per-day report;
     - make the final answer the user-facing digest only, without technical logs.

## Copyright and quoting

For copyrighted books, do not reproduce long passages. The deliverable should be transformative: tools, questions, concepts, summaries, applications, and synthesis in your own words. Short phrases may be used when necessary, but the core value should be extraction and interpretation.

## Quality bar for chapter digests

- When Margarita asks to understand a book well enough to discuss it **обстоятельно**, make the digest tutorial-depth rather than a compact summary. Include: chapter map and argument sequence; definitions of central terms; assumptions and evidence; metaphors/examples and what they do; difficult passages in plain Russian; internal tensions or unsupported claims; source-author position vs your interpretation vs Cosmos Luminosity application; discussion-ready theses, counterquestions, and a concise oral retelling outline.
- For metaphysical or spiritual books, label ontology as the author’s postulate/model unless independently evidenced. Do not silently present claims such as divinity, consciousness creating reality, entities, or cosmological mechanisms as established fact.
- Write each digest for a reader who has **not read the book/chapter**: reconstruct the scene, sequence, people, places, and causality before interpreting.
- Do not drop isolated symbolic details (e.g. an animal, ritual, childhood image, place name) without explaining where they appear in the chapter and why they matter.
- If giving psychology or subtle meaning, anchor every interpretation to the chapter’s narrative evidence first, then translate it into concepts/practices.
- Label interpretive metaphors as interpretive. If you name a theme like “discipline,” “chaos,” “mother,” “initiation,” or “shadow,” make clear which concrete chapter details support it. Do not make it sound like the author/book literally centered that theme unless the source text does.
- Avoid retrofitting chapter analysis around Margarita’s previously discussed personal themes. Use her Cosmos Luminosity lens as a translation layer, but keep source-derived content and agent-derived meaning separate.
- Prefer a coherent walkthrough: what happens → professional significance → psychological mechanism → subtle/Cosmos meaning → practices/questions.
- For Margarita/Cosmos Luminosity, full-context explanation matters more than short excerpt-style bullets; she may send the analysis to clients who have not read the source.

## Action and communication discipline

- After the source arrives, produce the requested first digest in the same workflow; do not stop at identification, a promise, or a request to wait.
- Do not tell the user to approve a nonexistent UI prompt. If an internal write/command is blocked or times out, retry once through an appropriate non-destructive tool path; if still blocked, state the concrete blocker without implying that the user missed an action.
- Keep progress messages short. The first substantial user-facing result matters more than narrating internal setup.
- Never mark `first_fragment_delivered: true` before the digest has actually been delivered in the chat. Prepare/save it first, deliver it, then advance state exactly once.

## Verification checklist

Before telling the user it is set up:

- [ ] Source files copied into stable storage.
- [ ] Text extraction succeeded and page/chapter counts look plausible.
- [ ] Manifest and state files exist.
- [ ] First fragment can be emitted in `--peek` mode without advancing state.
- [ ] Cron script is copied to `~/.hermes/scripts/` and executable.
- [ ] Cron job created successfully.
- [ ] `next_run_at` matches the user’s requested local time, after timezone conversion.
- [ ] Accumulating tools/insights file exists.
- [ ] If a web mirror was requested, the Markdown source is the website article (or symlinked to it), frontmatter is preserved, the article is linked from the site index, and the URL loads from an external browser/curl check.

## References

- `references/public-drive-scanned-pdf-ingestion.md` — public Drive-folder retrieval, large-file confirmation, scan-only PDF detection, rootless Tesseract OCR, and the `OMP_THREAD_LIMIT=1` parallelism fix.
- `references/daily-pdf-book-extraction.md` — reusable layout, script skeleton, and cron prompt checklist based on a multi-PDF daily extraction setup.
- `references/web-published-markdown-mirror.md` — pattern for making the accumulating Markdown file also power a public/wiki web page via one canonical article and symlink.
- `references/royallib-rtf-zip-daily-extraction.md` — ZIP containing RTF-disguised-as-`.doc` extraction pattern, cp1251 fallback, paragraph chunking, stateful `--peek` verification, and UTC cron timezone pitfall.
- `references/modernlib-rtf-zip-daily-extraction.md` — ModernLib-style ZIP with mojibake filenames and `.doc` that is actually cp1251 RTF; includes recognition, extraction, chapter chunking, timezone confirmation, and copyright-safe daily synthesis notes.
- `references/litmir-epub-daily-extraction.md` — Litmir-style EPUB extraction via OPF spine order, cover-fragment pitfall, paragraph chunking, and Margarita timezone/confirmation nuance.
- `references/second-pass-thematic-layers.md` — extend a completed first-pass book cron into a deeper thematic/layered second pass with source excerpts, research queries, copyright-safe close reading, and updated cron prompt/toolsets.
- `references/cron-transaction-and-threat-scanner.md` — transactional state advancement, recovery after pre-agent failures, and safe diagnosis of assembled-prompt scanner false positives without disabling security.