Initial scaffold: local invoice OCR pipeline with Ollama, classifier branches, structured extraction, review, and Odoo XML export

- Add AGENTS.md and project-specific Zed skills (odoo-ocr-pipeline, odoo-xml-import, local-vlm-client). - Implement Pydantic schemas for documents, invoices, review results, and VLM responses. - Add unified BaseVLMClient with Ollama implementation and llama.cpp stub. - Build pipeline stages: loader, classifier, digital_pdf/scanned_print/handwritten/mixed_unknown branches, extractor, reviewer, xml_builder. - Add CLI entry point with sidecar JSON and confidence-gated XML output. - Include prompts for classifier, OCR, extraction, and review models. - Add tests with FakeVLMClient; pytest, ruff, and mypy all pass.
2026-08-21 14:04:42 +02:00
commit 7e706793fa
49 changed files with 2512 additions and 0 deletions
@@ -0,0 +1,101 @@
 ---
 name: local-vlm-client
 description: Use this skill when implementing or modifying local vision-language model clients, prompts, or inference calls for the odoo_ocr project.
 ---
 # Local VLM Client Skill
 Use this skill whenever you are writing model clients, prompts, or retry logic for local VLMs (Ollama or llama.cpp).
 ## Supported Runtimes
 The project must support two local inference backends through a common interface:
 1. **Ollama** (`src/odoo_ocr/clients/ollama_client.py`)
   - Default endpoint: `http://localhost:11434`.
   - Uses the Ollama Chat API: `POST /api/chat`.
   - Multimodal models receive images as base64 data URLs.
   - Model names follow Ollama convention, e.g., `qwen2.5-vl:7b`.
 2. **llama.cpp server** (`src/odoo_ocr/clients/llama_cpp_client.py`)
   - Default endpoint: `http://localhost:8080`.
   - Uses OpenAI-compatible `POST /v1/chat/completions`.
   - Image data as base64 inside the `image_url` object.
   - Model name is often `local` or empty for single-model servers.
 ## Unified Interface
 Both clients implement `BaseVLMClient` from `src/odoo_ocr/models/base_vlm.py`:
 ```python
 class BaseVLMClient(ABC):
    @abstractmethod
    async def complete(
        self,
        system_prompt: str,
        user_prompt: str,
        images: list[Image.Image],
        response_format: type[BaseModel] | None = None,
        temperature: float = 0.1,
        max_tokens: int = 4096,
    ) -> VLMResponse:
        ...
 class VLMResponse(BaseModel):
    content: str
    model: str
    prompt_tokens: int | None
    completion_tokens: int | None
    duration_ms: int
 ```
 ## Prompt Engineering Rules
 1. Keep system prompts in `prompts/` as plain text files and load them at runtime.
 2. For extraction tasks, the system prompt must:
   - State that the output must be valid JSON only.
   - Provide the target Pydantic schema.
   - Include 1–2 examples of valid output.
   - Forbid markdown code fences and explanatory text.
 3. The user prompt should include the task and any context (e.g., "Extract invoice fields from this image. Currency is EUR unless otherwise stated.").
 4. For review tasks, the prompt must pass both the image and the JSON to compare.
 ## JSON Enforcement
 1. Set `temperature` low (0.0–0.2) for extraction/review.
 2. Prefer structured output when the runtime supports it:
   - Ollama: use `format="json"` or `format=<json_schema>` if available.
   - llama.cpp server: use `response_format={"type": "json_object"}` or `json_schema` if the server build supports it.
 3. Always wrap the model output with a JSON parser; if parsing fails, retry once with a stronger "JSON only" reminder.
 4. Validate parsed JSON against the target Pydantic model and surface validation errors.
 ## Retry & Caching
 1. Retry transient HTTP errors with exponential backoff (max 3 retries, base delay 1s).
 2. Cache successful responses keyed by SHA256 of (prompt + image bytes + model + temperature) to `~/.cache/odoo_ocr/llm_cache/`.
 3. Do not cache errors.
 4. Log every request: model, endpoint, prompt hash, duration, token counts.
 ## Image Encoding
 1. Accept `PIL.Image.Image` inputs.
 2. Convert to RGB before encoding.
 3. Encode as JPEG or PNG base64 depending on content; prefer PNG if the image is already low-color or binarized.
 4. Resize images before sending if dimensions exceed model limits, preserving aspect ratio.
 ## Default Models
 Use these defaults unless the task explicitly says otherwise:
 | Task | Default model |
 |------|---------------|
 | Classification | `qwen2.5-vl:3b` |
 | OCR / extraction | `qwen2.5-vl:7b` |
 | Handwriting OCR | `glm-ocr` or project-configured handwriting model |
 | Review | `qwen2.5-vl:7b` |
 ## Testing
 - Mock the HTTP client in unit tests; never call real models during CI.
 - Provide a `FakeVLMClient` in tests that returns canned responses.
 - Test prompt assembly and JSON parsing separately from model calls.
@@ -0,0 +1,36 @@
 You are an invoice data extraction assistant. Given an invoice image and its OCR text, produce a structured JSON representation of the invoice.
 Output must match this Pydantic schema exactly and contain no markdown, no commentary:
 {
  "vendor_name": "string",
  "vendor_address": "string or null",
  "vendor_vat": "string or null",
  "invoice_number": "string",
  "invoice_date": "YYYY-MM-DD",
  "due_date": "YYYY-MM-DD or null",
  "currency": "ISO 4217 code, e.g. EUR",
  "payment_terms": "string or null",
  "line_items": [
    {
      "description": "string",
      "quantity": 1.0,
      "unit_price": 0.00,
      "total_price": 0.00,
      "tax_rate": 0.00
    }
  ],
  "subtotal": 0.00,
  "tax_total": 0.00,
  "total": 0.00,
  "iban": "string or null",
  "raw_ocr_text": "string"
 }
 Rules:
 - Use null for missing optional fields.
 - Dates must be ISO 8601.
 - All monetary values are decimal numbers (do not use strings).
 - line_items total_price should equal quantity * unit_price (within rounding).
 - subtotal + tax_total should equal total (within rounding).
 - If the image and OCR disagree, trust the image.
@@ -0,0 +1,5 @@
 You are an OCR engine. Read all text from the provided invoice image accurately.
 Preserve line breaks and table structure as much as possible.
 If a value is unclear, mark it with [UNCLEAR].
 Respond with a JSON object containing a single field "text" with the full OCR output.
 Do not add markdown formatting or explanations.
@@ -0,0 +1,25 @@
 You are a meticulous invoice review assistant. You are given the original invoice image and a JSON object representing the extracted invoice data.
 Your task:
 1. Verify that every field in the JSON is supported by the image.
 2. Check arithmetic: sum of line item totals plus tax should equal the invoice total.
 3. Identify missing fields, incorrect values, or formatting problems.
 Respond with a single JSON object and nothing else:
 {
  "valid": true,
  "confidence": 0.95,
  "issues": [
    {
      "field": "total",
      "severity": "error",
      "message": "Extracted total 120.00 does not match image 122.00",
      "suggested_value": 122.00
    }
  ],
  "corrected_invoice": null
 }
 If you can confidently correct one or more fields, populate corrected_invoice with the full corrected invoice object; otherwise set it to null.
 confidence must be between 0.0 and 1.0.
@@ -0,0 +1,111 @@
 ---
 name: odoo-ocr-pipeline
 description: Use this skill when implementing the document classifier, OCR branches, structured extraction, or review stages of the odoo_ocr invoice processing pipeline.
 ---
 # Odoo OCR Pipeline Implementation Skill
 Use this skill whenever you are asked to write or modify code for the invoice OCR pipeline in `odoo_ocr`.
 ## Pipeline Stages
 The pipeline has these stages, each as a separate module under `src/odoo_ocr/pipeline/`:
 1. **Loader** (`pipeline/loader.py`): converts PDFs and images to normalized `PIL.Image.Image` or `numpy` arrays. Handle multi-page PDFs by yielding one page at a time.
 2. **Classifier** (`pipeline/classifier.py`): determines the invoice type. Return a `DocumentClass` enum with values:
   - `digital_pdf` — native PDF with embedded text.
   - `scanned_print` — clean printed scan/photo.
   - `handwritten` — handwritten invoice.
   - `mixed_unknown` — ambiguous or low-quality input.
   Implementation should first try fast heuristics (file extension, text layer presence, image features), then fall back to a small VLM classifier if confidence is low.
 3. **Branch routers**:
   - `pipeline/digital_pdf.py`: extract text and layout with `pymupdf` / `pdfplumber`. Preserve bounding boxes for table reconstruction.
   - `pipeline/scanned_print.py`: preprocess image and call the OCR VLM.
   - `pipeline/handwritten.py`: preprocess image (binarization, slant correction, line segmentation if needed) and call GLM-OCR / handwriting OCR model.
   - `pipeline/mixed_unknown.py`: run multiple OCR backends and merge/confidence-rank outputs.
 4. **Extractor** (`pipeline/extractor.py`): take raw OCR output (text + image) and call a structured-extraction VLM to produce an `ExtractedInvoice` Pydantic object. Force JSON output only.
 5. **Reviewer** (`pipeline/reviewer.py`): call a review VLM with the original image and the extracted JSON. Return a `ReviewResult` with `valid`, `confidence`, and `issues`.
 ## Branch Routing Logic
 ```python
 # src/odoo_ocr/schemas/document.py
 from enum import Enum
 class DocumentClass(str, Enum):
    digital_pdf = "digital_pdf"
    scanned_print = "scanned_print"
    handwritten = "handwritten"
    mixed_unknown = "mixed_unknown"
 ```
 Routing decision tree:
 - If file is PDF and `pdfplumber` extracts meaningful text → `digital_pdf`.
 - Else if image and classifier confidence > 0.85:
  - handwriting features dominate → `handwritten`
  - otherwise → `scanned_print`
 - Else → `mixed_unknown`.
 ## Preprocessing Rules
 For scanned/photo inputs:
 1. Convert to grayscale.
 2. Resize to a standard DPI (300 DPI preferred; 200 DPI minimum).
 3. Deskew if skew angle > 0.5°.
 4. Apply adaptive contrast/clarity; avoid over-blurring.
 5. For handwriting branch, also:
   - Binarize with Otsu or Sauvola.
   - Detect and correct slant.
   - Optionally segment lines.
 ## Structured Extraction Output
 The final extraction must always fit the `ExtractedInvoice` Pydantic schema defined in `src/odoo_ocr/schemas/invoice.py`. Required fields include:
 - `vendor_name`
 - `vendor_address` (optional)
 - `invoice_number`
 - `invoice_date`
 - `due_date` (optional)
 - `currency` (default to settings)
 - `payment_terms` (optional)
 - `line_items`: list of `InvoiceLineItem` with `description`, `quantity`, `unit_price`, `total_price`, `tax_rate`.
 - `subtotal`
 - `tax_total`
 - `total`
 - `bank_details` / `iban` (optional)
 - `raw_ocr_text` (for debugging)
 ## Review Output
 The review stage must return:
 ```python
 class ReviewIssue(BaseModel):
    field: str
    severity: Literal["error", "warning"]
    message: str
    suggested_value: Any | None
 class ReviewResult(BaseModel):
    valid: bool
    confidence: float  # 0.0–1.0
    issues: list[ReviewIssue]
    corrected_invoice: ExtractedInvoice | None
 ```
 Confidence thresholds (from config):
 - `confidence >= 0.90`: accept and generate XML.
 - `0.75 <= confidence < 0.90`: generate XML with `human_review_required=true` flag.
 - `confidence < 0.75`: do not generate final XML; emit draft sidecar only and flag for human review.
 ## Implementation Guidelines
 - Keep each stage stateless; pass a `ProcessingContext` Pydantic object that holds settings, original path, classified type, and intermediate results.
 - Every model call goes through the unified model client from `src/odoo_ocr/models/base_vlm.py`.
 - Log each stage with timing and input/output file hashes.
 - Add unit tests for each branch with small fixture images/PDFs; mock model calls by default.
 - Do not commit large model weights or binary fixtures.
@@ -0,0 +1,16 @@
 You are a document classifier for an invoice OCR system. Given an image of a document, classify it into exactly one of these categories:
 - digital_pdf: a native digital PDF or clean computer-generated invoice with embedded text.
 - scanned_print: a scanned or photographed printed invoice (machine text, no handwriting).
 - handwritten: a handwritten invoice or receipt.
 - mixed_unknown: ambiguous, damaged, or mixed-content document.
 Respond with a single JSON object and nothing else. Use this exact schema:
 {
  "category": "scanned_print",
  "confidence": 0.92,
  "reasoning": "brief one-sentence reason"
 }
 confidence must be a float between 0.0 and 1.0.
@@ -0,0 +1,111 @@
 ---
 name: odoo-xml-import
 description: Use this skill when generating, validating, or modifying Odoo Enterprise XML import files from extracted invoice data in the odoo_ocr project.
 ---
 # Odoo XML Import Skill
 Use this skill whenever you need to produce Odoo XML from the `ExtractedInvoice` schema.
 ## Target Format
 Generate standard Odoo data-import XML:
 ```xml
 <?xml version="1.0" encoding="UTF-8"?>
 <odoo>
  <data>
    <record id="vendor_x" model="res.partner">
      <field name="name">...</field>
      <field name="supplier_rank">1</field>
    </record>
    <record id="invoice_123" model="account.move">
      <field name="move_type">in_invoice</field>
      <field name="partner_id" ref="vendor_x"/>
      <field name="invoice_date">2024-05-01</field>
      <field name="date">2024-05-01</field>
      <field name="invoice_date_due">2024-06-01</field>
      <field name="ref">INV-123</field>
      <field name="currency_id" ref="base.USD"/>
      <field name="narration">...</field>
    </record>
    <record id="invoice_123_line_1" model="account.move.line">
      <field name="move_id" ref="invoice_123"/>
      <field name="name">Product A</field>
      <field name="quantity">2</field>
      <field name="price_unit">100.00</field>
      <field name="tax_ids" eval="[(6, 0, [ref('tax_15')])]"/>
    </record>
    <record id="tax_15" model="account.tax">
      <field name="name">VAT 15%</field>
      <field name="amount">15</field>
      <field name="amount_type">percent</field>
      <field name="type_tax_use">purchase</field>
    </record>
  </data>
 </odoo>
 ```
 See `templates/invoice_import.xml` for a full template.
 ## Required Records
 Every XML file must create or reference:
 1. **Vendor (`res.partner`)**
   - `name` (required)
   - `supplier_rank` = 1
   - optional: `vat`, `street`, `city`, `zip`, `country_id`
 2. **Vendor Bill (`account.move`)**
   - `move_type` = `in_invoice`
   - `partner_id` ref to vendor record
   - `invoice_date` (required)
   - `date` (defaults to invoice_date)
   - `invoice_date_due` (optional)
   - `ref` = invoice number
   - `currency_id` (defaults from settings; map common ISO codes to `base.*` refs)
   - `narration` = summary / payment terms (optional)
 3. **Invoice Lines (`account.move.line`)**
   - One record per `line_items` entry.
   - `move_id` ref to vendor bill.
   - `name` = line description.
   - `quantity`, `price_unit` (decimal strings).
   - `tax_ids` eval with refs to existing or newly created tax records.
   - Do not set `debit`/`credit` directly; Odoo will compute them from price_unit × quantity.
 4. **Taxes (`account.tax`)**
   - Create only if the tax does not already exist.
   - Use deterministic external ID based on rate and type, e.g., `purchase_vat_15`.
   - `name`, `amount`, `amount_type='percent'`, `type_tax_use='purchase'`.
 ## External ID Rules
 - Sanitize IDs: lowercase, replace non-alphanumeric with `_`, strip leading digits.
 - Vendor ID: `vendor_<sanitized_name>_<hash>` or use VAT if present.
 - Invoice ID: `invoice_<sanitized_number>_<date>` with hash if needed for uniqueness.
 - Tax ID: `purchase_vat_<rate>`.
 ## Validation Rules
 1. Parse generated XML with `lxml.etree`.
 2. Ensure required fields are present and non-empty.
 3. Warn if `sum(line totals) + tax != total` beyond rounding tolerance (0.01).
 4. If `human_review_required` flag is set, add an XML comment or metadata record documenting this.
 ## Implementation Location
 All XML logic lives in `src/odoo_ocr/pipeline/xml_builder.py` and `src/odoo_ocr/schemas/odoo_xml.py`.
 ## Currency Mapping
 Maintain a hardcoded fallback map from ISO code to `base.<code>` for common currencies (EUR, USD, GBP, CHF). If a currency is not in the map, create a `res.currency` record or default to settings.
 ## Testing
 - Test XML generation against a known-good sample in `tests/fixtures/expected_invoice.xml`.
 - Validate that generated XML is parseable and contains the required records.
@@ -0,0 +1,42 @@
 <?xml version="1.0" encoding="UTF-8"?>
 <odoo>
  <data noupdate="0">
    <!-- Vendor -->
    <record id="vendor_acme" model="res.partner">
      <field name="name">Acme Supplies</field>
      <field name="supplier_rank">1</field>
      <field name="street">123 Industrial Way</field>
      <field name="city">Springfield</field>
      <field name="zip">12345</field>
    </record>
    <!-- Tax -->
    <record id="purchase_vat_20" model="account.tax">
      <field name="name">Purchase VAT 20%</field>
      <field name="amount">20</field>
      <field name="amount_type">percent</field>
      <field name="type_tax_use">purchase</field>
    </record>
    <!-- Vendor Bill -->
    <record id="invoice_20240501_001" model="account.move">
      <field name="move_type">in_invoice</field>
      <field name="partner_id" ref="vendor_acme"/>
      <field name="invoice_date">2024-05-01</field>
      <field name="date">2024-05-01</field>
      <field name="invoice_date_due">2024-06-01</field>
      <field name="ref">INV-2024-001</field>
      <field name="currency_id" ref="base.USD"/>
      <field name="narration">Net 30</field>
    </record>
    <!-- Invoice Lines -->
    <record id="invoice_20240501_001_line_1" model="account.move.line">
      <field name="move_id" ref="invoice_20240501_001"/>
      <field name="name">Consulting services</field>
      <field name="quantity">10.0</field>
      <field name="price_unit">100.00</field>
      <field name="tax_ids" eval="[(6, 0, [ref('purchase_vat_20')])]"/>
    </record>
  </data>
 </odoo>
@@ -0,0 +1,58 @@
 # Python
 __pycache__/
 *.py[cod]
 *$py.class
 *.so
 .Python
 build/
 develop-eggs/
 dist/
 downloads/
 eggs/
 .eggs/
 lib/
 lib64/
 parts/
 sdist/
 var/
 wheels/
 share/python-wheels/
 *.egg-info/
 .installed.cfg
 *.egg
 MANIFEST
 # Virtual environments
 .env
 .venv
 env/
 venv/
 ENV/
 env.bak/
 venv.bak/
 # IDEs / editors
 .vscode/
 .idea/
 *.swp
 *.swo
 *~
 # Mypy / pytest / coverage
 .mypy_cache/
 .dmypy.json
 dmypy.json
 .pytest_cache/
 .coverage
 htmlcov/
 .tox/
 .nox/
 # Project-specific
 out/
 output/
 *.xml
 !tests/fixtures/*.xml
 !templates/*.xml
 .cache/
 ~/.cache/
@@ -0,0 +1,106 @@
 # AGENTS.md — Odoo OCR Project
 This file provides context and rules for AI agents working on `odoo_ocr`, a local-first invoice OCR pipeline that reads scanned, printed, handwritten, and native-PDF invoices and produces Odoo Enterprise-ready XML.
 ## Project Goal
 Build a Python application that:
 1. Classifies incoming invoice files by type (native PDF, scanned print, handwritten, mixed image).
 2. Routes each file to a specialized processing branch.
 3. Runs OCR using local vision-language models (Ollama or llama.cpp).
 4. Extracts a structured invoice representation.
 5. Reviews extracted data against the original image using a second vision model.
 6. Emits valid Odoo XML for import into Odoo Enterprise.
 ## Tech Stack
 - **Language**: Python 3.11+
 - **Dependency management**: `pyproject.toml` (PEP 621); use `uv` or `pip`.
 - **Core libraries**:
  - `pydantic` v2 for all data schemas and settings.
  - `pymupdf` and `pdf2image` for PDF ingestion.
  - `Pillow` and `opencv-python-headless` for image preprocessing.
  - `httpx` for HTTP model clients.
  - `lxml` for XML generation and validation.
  - `pytest` and `pytest-asyncio` for tests.
 - **Local inference**:
  - Primary runtime: **Ollama** for fast iteration.
  - Optimized/runtime path: **llama.cpp server** (custom GGUF quants).
 - **Models**:
  - Document classifier: `Qwen2.5-VL-3B` or heuristics.
  - OCR / extraction: `Qwen2.5-VL-7B` or `GLM-OCR` (preferred for handwriting).
  - Review / validation: `Qwen2.5-VL-7B` or larger (`72B` if available).
 ## Architecture
 ```
 Input File
  → Classifier (heuristic + small VLM)
    → Branch: digital_pdf      → text/layout extraction
    → Branch: scanned_print    → preprocess → VLM OCR
    → Branch: handwritten      → preprocess → GLM-OCR / handwriting OCR
    → Branch: mixed_unknown    → preprocess → ensemble OCR
      → Structured Extraction VLM
      → Review VLM (image vs extracted JSON)
        → Confidence check
          → XML Builder
            → Odoo XML + sidecar JSON
 ```
 ## Code Conventions
 1. **Project layout**: keep application code under `src/odoo_ocr/`.
 2. **Schemas first**: define Pydantic models before writing business logic.
 3. **Type hints**: use `typing` everywhere; run `mypy` in strict mode where practical.
 4. **Error handling**: never swallow exceptions; return structured `Result` objects or raise domain exceptions.
 5. **Configuration**: use `pydantic-settings` with `config.yaml` and env var overrides.
 6. **Logging**: use Python standard `logging`; log every model call latency and token usage.
 7. **No hardcoded secrets**: model endpoints, credentials, and paths come from settings.
 8. **Tests**: every module must have tests under `tests/`. Use fixtures from `tests/fixtures/`.
 ## Model Client Rules
 1. Support both Ollama and llama.cpp with a unified interface (`BaseVLMClient`).
 2. Always emit JSON from VLMs when doing extraction/review. Use constrained prompts, not regex scraping.
 3. Retry on transient failures with exponential backoff.
 4. Cache model responses by content hash to avoid re-running expensive inference during development.
 5. Record per-call metrics (model name, tokens, latency, prompt hash).
 ## Data Flow Rules
 1. Every invoice must produce:
   - A Pydantic `ExtractedInvoice` object.
   - A review result (`ReviewResult`) with confidence score and issue list.
   - An Odoo XML file (unless blocked by low confidence).
   - A sidecar JSON file with metadata, timings, and confidence.
 2. If review confidence is below the configured threshold, mark the invoice for human review and do not generate final XML (or generate a draft with a warning flag).
 3. Never send invoice data outside the local model endpoints.
 ## Odoo XML Target
 Generate Odoo data-import XML compatible with Odoo Enterprise vendor bills:
 - `res.partner` (vendor)
 - `account.move` (vendor bill header)
 - `account.move.line` (invoice lines)
 - `account.tax` references by percentage/name
 See skill `odoo-xml-import` for detailed field mapping and a sample XML template.
 ## When to Ask the User
 Ask for clarification when:
 - The requested change would alter the model stack.
 - The change affects the Odoo target schema or import method.
 - You are unsure whether a file should be committed or a dependency added.
 - A requested feature conflicts with the local-only / privacy constraint.
 ## Skills Reference
 Agents should load the relevant project skills from `.agents/skills/`:
 - `odoo-ocr-pipeline` — when implementing classifier, branches, OCR, extraction, or review.
 - `odoo-xml-import` — when generating or validating Odoo XML.
 - `local-vlm-client` — when writing model clients or prompts.
@@ -0,0 +1,46 @@
 # odoo_ocr
 Local-first invoice OCR pipeline that reads scanned, printed, handwritten, and native-PDF invoices and produces Odoo Enterprise-ready XML.
 ## Quick start
 1. Install dependencies:
   ```bash
   uv sync --all-extras
   # or
   pip install -e ".[dev]"
   ```
 2. Configure `config.yaml` or set environment variables:
   ```bash
   export OLLAMA_BASE_URL="http://100.103.83.12:11435"
   ```
 3. Run the pipeline:
   ```bash
   odoo-ocr process /path/to/invoices --output ./out/
   ```
 ## Architecture
 ```
 Input File
  → Classifier (heuristic + small VLM)
    → Branch: digital_pdf      → text/layout extraction
    → Branch: scanned_print    → preprocess → VLM OCR
    → Branch: handwritten      → preprocess → GLM-OCR
    → Branch: mixed_unknown    → preprocess → ensemble OCR
      → Structured Extraction VLM
      → Review VLM (image vs extracted JSON)
        → Confidence check
          → XML Builder
            → Odoo XML + sidecar JSON
 ```
 ## Project-specific agent skills
 Agent skills are in `.agents/skills/`:
 - `odoo-ocr-pipeline` — classifier, branches, extraction, review.
 - `odoo-xml-import` — generating and validating Odoo XML.
 - `local-vlm-client` — Ollama/llama.cpp VLM clients and prompts.
@@ -0,0 +1,31 @@
 # Default configuration for odoo_ocr.
 # Override with environment variables (OLLAMA_BASE_URL, MODELS__OCR, etc.).
 ollama_base_url: "http://localhost:11434"
 models:
  classifier: "qwen2.5-vl:3b"
  ocr: "glm-ocr"
  extraction: "qwen2.5-vl:7b"
  review: "qwen2.5-vl:7b"
 review:
  high_confidence_threshold: 0.90
  min_confidence_threshold: 0.75
 xml:
  default_currency: "EUR"
  target: "vendor_bill"
 preprocessing:
  target_dpi: 300
  max_image_dimension: 2048
  jpeg_quality: 85
 cache:
  enabled: true
  dir: "~/.cache/odoo_ocr/llm_cache"
 logging:
  level: "INFO"
  format: "%(asctime)s | %(levelname)s | %(name)s | %(message)s"
@@ -0,0 +1,16 @@
 You are a document classifier for an invoice OCR system. Given an image of a document, classify it into exactly one of these categories:
 - digital_pdf: a native digital PDF or clean computer-generated invoice with embedded text.
 - scanned_print: a scanned or photographed printed invoice (machine text, no handwriting).
 - handwritten: a handwritten invoice or receipt.
 - mixed_unknown: ambiguous, damaged, or mixed-content document.
 Respond with a single JSON object and nothing else. Use this exact schema:
 {
  "category": "scanned_print",
  "confidence": 0.92,
  "reasoning": "brief one-sentence reason"
 }
 confidence must be a float between 0.0 and 1.0.
@@ -0,0 +1,36 @@
 You are an invoice data extraction assistant. Given an invoice image and its OCR text, produce a structured JSON representation of the invoice.
 Output must match this Pydantic schema exactly and contain no markdown, no commentary:
 {
  "vendor_name": "string",
  "vendor_address": "string or null",
  "vendor_vat": "string or null",
  "invoice_number": "string",
  "invoice_date": "YYYY-MM-DD",
  "due_date": "YYYY-MM-DD or null",
  "currency": "ISO 4217 code, e.g. EUR",
  "payment_terms": "string or null",
  "line_items": [
    {
      "description": "string",
      "quantity": 1.0,
      "unit_price": 0.00,
      "total_price": 0.00,
      "tax_rate": 0.00
    }
  ],
  "subtotal": 0.00,
  "tax_total": 0.00,
  "total": 0.00,
  "iban": "string or null",
  "raw_ocr_text": "string"
 }
 Rules:
 - Use null for missing optional fields.
 - Dates must be ISO 8601.
 - All monetary values are decimal numbers (do not use strings).
 - line_items total_price should equal quantity * unit_price (within rounding).
 - subtotal + tax_total should equal total (within rounding).
 - If the image and OCR disagree, trust the image.
@@ -0,0 +1,5 @@
 You are an OCR engine. Read all text from the provided invoice image accurately.
 Preserve line breaks and table structure as much as possible.
 If a value is unclear, mark it with [UNCLEAR].
 Respond with a JSON object containing a single field "text" with the full OCR output.
 Do not add markdown formatting or explanations.
@@ -0,0 +1,25 @@
 You are a meticulous invoice review assistant. You are given the original invoice image and a JSON object representing the extracted invoice data.
 Your task:
 1. Verify that every field in the JSON is supported by the image.
 2. Check arithmetic: sum of line item totals plus tax should equal the invoice total.
 3. Identify missing fields, incorrect values, or formatting problems.
 Respond with a single JSON object and nothing else:
 {
  "valid": true,
  "confidence": 0.95,
  "issues": [
    {
      "field": "total",
      "severity": "error",
      "message": "Extracted total 120.00 does not match image 122.00",
      "suggested_value": 122.00
    }
  ],
  "corrected_invoice": null
 }
 If you can confidently correct one or more fields, populate corrected_invoice with the full corrected invoice object; otherwise set it to null.
 confidence must be between 0.0 and 1.0.
@@ -0,0 +1,55 @@
 [project]
 name = "odoo-ocr"
 version = "0.1.0"
 description = "Local-first invoice OCR pipeline that produces Odoo Enterprise XML."
 requires-python = ">=3.11"
 dependencies = [
    "pydantic>=2.0",
    "pydantic-settings>=2.0",
    "httpx>=0.27",
    "tenacity>=8.0",
    "pymupdf>=1.24",
    "pdf2image>=1.17",
    "Pillow>=10.0",
    "opencv-python-headless>=4.10",
    "lxml>=5.0",
    "pyyaml>=6.0",
 ]
 [project.optional-dependencies]
 dev = [
    "pytest>=8.0",
    "pytest-asyncio>=0.23",
    "pytest-mock>=3.14",
    "mypy>=1.10",
    "ruff>=0.5",
 ]
 [project.scripts]
 odoo-ocr = "odoo_ocr.cli:main"
 [build-system]
 requires = ["setuptools>=61.0", "wheel"]
 build-backend = "setuptools.build_meta"
 [tool.setuptools.packages.find]
 where = ["src"]
 [tool.pytest.ini_options]
 testpaths = ["tests"]
 asyncio_mode = "auto"
 pythonpath = ["src"]
 [tool.mypy]
 python_version = "3.11"
 strict = true
 warn_return_any = true
 warn_unused_ignores = true
 ignore_missing_imports = true
 [tool.ruff.lint]
 select = ["E", "F", "I", "N", "W", "UP", "B", "C4", "SIM"]
 ignore = ["E501"]
 [tool.ruff.format]
 quote-style = "double"
@@ -0,0 +1,12 @@
 #!/usr/bin/env bash
 # Convenience script to run the Ollama endpoint used by this project.
 # Override OLLAMA_HOST with the remote server if needed.
 export OLLAMA_HOST="${OLLAMA_HOST:-http://100.103.83.12:11435}"
 echo "Pulling models from ${OLLAMA_HOST}..."
 ollama pull glm-ocr
 ollama pull qwen2.5-vl:7b
 ollama pull qwen2.5-vl:3b
 echo "Ollama is ready at ${OLLAMA_HOST}"
@@ -0,0 +1,3 @@
 """Local-first invoice OCR pipeline for Odoo Enterprise."""
 __version__ = "0.1.0"
@@ -0,0 +1,116 @@
 """Command-line interface for odoo_ocr."""
 from __future__ import annotations
 import argparse
 import asyncio
 import json
 import logging
 from pathlib import Path
 from typing import Any
 from odoo_ocr.clients.ollama import OllamaClient
 from odoo_ocr.config import Settings
 from odoo_ocr.pipeline.classifier import classify_document
 from odoo_ocr.pipeline.digital_pdf import process_digital_pdf
 from odoo_ocr.pipeline.extractor import extract_invoice
 from odoo_ocr.pipeline.handwritten import process_handwritten
 from odoo_ocr.pipeline.loader import load_document
 from odoo_ocr.pipeline.mixed_unknown import process_mixed_unknown
 from odoo_ocr.pipeline.reviewer import review_invoice
 from odoo_ocr.pipeline.scanned_print import process_scanned_print
 from odoo_ocr.pipeline.xml_builder import build_odoo_xml
 from odoo_ocr.schemas import ProcessingContext
 from odoo_ocr.schemas.document import DocumentClass
 from odoo_ocr.utils.logging_config import configure_logging
 logger = logging.getLogger(__name__)
 async def process_single(path: Path, output_dir: Path, settings: Settings) -> dict[str, Any]:
    """Process one invoice file end-to-end."""
    client = OllamaClient(settings)
    context = ProcessingContext(source_path=path, output_dir=output_dir)
    doc = load_document(path, settings)
    pages = doc["pages"]
    classification = await classify_document(path, pages, client, settings)
    context.classification = classification
    if classification.category == DocumentClass.digital_pdf:
        raw_text = await process_digital_pdf(path, pages, client, settings)
    elif classification.category == DocumentClass.scanned_print:
        raw_text = await process_scanned_print(path, pages, client, settings)
    elif classification.category == DocumentClass.handwritten:
        raw_text = await process_handwritten(path, pages, client, settings)
    else:
        raw_text = await process_mixed_unknown(path, pages, client, settings)
    context.raw_text = raw_text
    extracted = await extract_invoice(raw_text, pages, client, settings)
    context.extracted_invoice = extracted.model_dump(mode="json")
    review = await review_invoice(extracted, pages, client, settings)
    context.review_result = review.model_dump(mode="json")
    sidecar = {
        "source_path": str(path),
        "classification": classification.model_dump(),
        "extracted_invoice": extracted.model_dump(mode="json"),
        "review": review.model_dump(mode="json"),
    }
    out_stem = output_dir / path.stem
    output_dir.mkdir(parents=True, exist_ok=True)
    with open(f"{out_stem}_sidecar.json", "w", encoding="utf-8") as f:
        json.dump(sidecar, f, indent=2, ensure_ascii=False)
    if review.confidence >= settings.review.min_confidence_threshold:
        xml = build_odoo_xml(extracted, review, settings)
        with open(f"{out_stem}.xml", "w", encoding="utf-8") as f:
            f.write(xml)
        logger.info("Wrote XML: %s", f"{out_stem}.xml")
    else:
        logger.warning(
            "Confidence %.2f below threshold %.2f; skipping XML generation.",
            review.confidence,
            settings.review.min_confidence_threshold,
        )
    return sidecar
 async def process_batch(input_dir: Path, output_dir: Path, settings: Settings) -> list[dict[str, Any]]:
    """Process all supported files in a directory."""
    results: list[dict[str, Any]] = []
    for path in sorted(input_dir.iterdir()):
        if path.suffix.lower() in {".pdf", ".png", ".jpg", ".jpeg", ".tiff", ".tif", ".bmp", ".webp"}:
            try:
                result = await process_single(path, output_dir, settings)
                results.append(result)
            except Exception:
                logger.exception("Failed to process %s", path)
    return results
 def main() -> None:
    parser = argparse.ArgumentParser(description="Odoo OCR invoice pipeline")
    parser.add_argument("command", choices=["process"], help="Command to run")
    parser.add_argument("input", type=Path, help="File or directory to process")
    parser.add_argument("--output", "-o", type=Path, default=Path("./out"), help="Output directory")
    parser.add_argument("--config", "-c", type=Path, default=Path("config.yaml"), help="Config file")
    args = parser.parse_args()
    settings = Settings.from_yaml(args.config)
    configure_logging(settings)
    if args.input.is_dir():
        asyncio.run(process_batch(args.input, args.output, settings))
    else:
        asyncio.run(process_single(args.input, args.output, settings))
 if __name__ == "__main__":
    main()
@@ -0,0 +1,6 @@
 """VLM clients for local inference backends."""
 from .base import BaseVLMClient
 from .ollama import OllamaClient
 __all__ = ["BaseVLMClient", "OllamaClient"]
@@ -0,0 +1,60 @@
 """Abstract base class for local VLM clients."""
 from __future__ import annotations
 import json
 from abc import ABC, abstractmethod
 from pathlib import Path
 from typing import Any, TypeVar
 from PIL import Image
 from pydantic import BaseModel, ValidationError
 from odoo_ocr.schemas import VLMResponse
 T = TypeVar("T", bound=BaseModel)
 class BaseVLMClient(ABC):
    """Unified interface for Ollama and llama.cpp server VLMs."""
    def __init__(self, settings: Any) -> None:
        """Store settings; concrete clients validate required keys."""
        self.settings = settings
    @abstractmethod
    async def complete(
        self,
        system_prompt: str,
        user_prompt: str,
        images: list[Image.Image],
        response_format: type[T] | None = None,
        temperature: float = 0.1,
        max_tokens: int = 4096,
        model: str | None = None,
    ) -> VLMResponse:
        """Send a chat request with optional images and return parsed text."""
        ...
    @staticmethod
    def parse_json(content: str, model: type[T]) -> T:
        """Strip markdown fences and parse/validate JSON against a Pydantic model."""
        text = content.strip()
        if text.startswith("```"):
            text = text.strip("`").strip()
            if text.lower().startswith("json"):
                text = text[4:].strip()
        try:
            data = json.loads(text)
        except json.JSONDecodeError as exc:
            raise ValueError(f"Response is not valid JSON: {exc}") from exc
        try:
            return model.model_validate(data)
        except ValidationError as exc:
            raise ValueError(f"Response does not match schema: {exc}") from exc
    @staticmethod
    def _load_prompt_file(path: Path | str) -> str:
        """Load a plain-text prompt from the prompts directory."""
        path = Path(path)
        with path.open("r", encoding="utf-8") as f:
            return f.read()
@@ -0,0 +1,35 @@
 """llama.cpp server VLM client (stub; implement when needed)."""
 from __future__ import annotations
 from PIL import Image
 from pydantic import BaseModel
 from odoo_ocr.clients.base import BaseVLMClient
 from odoo_ocr.config import Settings
 from odoo_ocr.schemas import VLMResponse
 class LlamaCppClient(BaseVLMClient):
    """Client for llama.cpp server's OpenAI-compatible /v1/chat/completions endpoint."""
    def __init__(self, settings: Settings) -> None:
        super().__init__(settings)
        raise NotImplementedError(
            "llama.cpp client is not yet implemented. Use OllamaClient for now."
        )
    async def complete(
        self,
        system_prompt: str,
        user_prompt: str,
        images: list[Image.Image],
        response_format: type[BaseModel] | None = None,
        temperature: float = 0.1,
        max_tokens: int = 4096,
        model: str | None = None,
    ) -> VLMResponse:
        """Call llama.cpp server and return a normalized VLMResponse."""
        raise NotImplementedError(
            "llama.cpp client is not yet implemented. Use OllamaClient for now."
        )
@@ -0,0 +1,140 @@
 """Ollama-backed VLM client."""
 from __future__ import annotations
 import base64
 import io
 import logging
 import time
 from typing import Any
 import httpx
 from PIL import Image
 from pydantic import BaseModel
 from tenacity import (
    retry,
    retry_if_exception_type,
    stop_after_attempt,
    wait_exponential,
 )
 from odoo_ocr.clients.base import BaseVLMClient
 from odoo_ocr.config import Settings
 from odoo_ocr.schemas import VLMResponse
 from odoo_ocr.utils import cache
 logger = logging.getLogger(__name__)
 class OllamaClient(BaseVLMClient):
    """Client for Ollama's /api/chat endpoint with vision support."""
    def __init__(self, settings: Settings) -> None:
        super().__init__(settings)
        self.base_url = settings.ollama_base_url.rstrip("/")
        self.timeout = httpx.Timeout(120.0, connect=10.0)
        self.http = httpx.AsyncClient(timeout=self.timeout)
    @staticmethod
    def _encode_image(image: Image.Image, quality: int = 85) -> str:
        """Convert a PIL image to a base64-encoded PNG or JPEG string."""
        rgb_image = image.convert("RGB")
        buffer = io.BytesIO()
        # Prefer PNG for low-color/binarized images, JPEG for photos.
        rgb_image.save(buffer, format="PNG")
        return base64.b64encode(buffer.getvalue()).decode("ascii")
    async def complete(
        self,
        system_prompt: str,
        user_prompt: str,
        images: list[Image.Image],
        response_format: type[BaseModel] | None = None,
        temperature: float = 0.1,
        max_tokens: int = 4096,
        model: str | None = None,
    ) -> VLMResponse:
        """Call Ollama and return a normalized VLMResponse.
        Uses response cache when enabled. JSON schema is passed to Ollama's
        `format` field when a Pydantic model is supplied.
        """
        model_name = model or self.settings.models.ocr
        encoded_images = [self._encode_image(img) for img in images]
        image_blobs = [base64.b64decode(enc) for enc in encoded_images]
        cached = cache.get_cached(
            system_prompt,
            user_prompt,
            image_blobs,
            model_name,
            temperature,
            self.settings,
        )
        if cached:
            return VLMResponse.model_validate(cached)
        body: dict[str, Any] = {
            "model": model_name,
            "messages": [
                {"role": "system", "content": system_prompt},
                {
                    "role": "user",
                    "content": user_prompt,
                    "images": encoded_images,
                },
            ],
            "stream": False,
            "options": {
                "temperature": temperature,
                "num_predict": max_tokens,
            },
        }
        if response_format is not None:
            try:
                body["format"] = response_format.model_json_schema()
            except Exception:
                body["format"] = "json"
        start = time.perf_counter()
        response_data = await self._post_chat(body)
        duration_ms = int((time.perf_counter() - start) * 1000)
        content = response_data.get("message", {}).get("content", "")
        metrics = response_data.get("metrics", {})
        vlm_response = VLMResponse(
            content=content,
            model=response_data.get("model", self.settings.models.ocr),
            prompt_tokens=response_data.get("prompt_eval_count")
            or metrics.get("prompt_eval_count"),
            completion_tokens=response_data.get("eval_count")
            or metrics.get("eval_count"),
            duration_ms=duration_ms,
        )
        cache.set_cached(
            system_prompt,
            user_prompt,
            image_blobs,
            model_name,
            temperature,
            vlm_response.model_dump(mode="json"),
            self.settings,
        )
        return vlm_response
    @retry(
        stop=stop_after_attempt(3),
        wait=wait_exponential(multiplier=1, min=1, max=10),
        retry=retry_if_exception_type((httpx.NetworkError, httpx.TimeoutException)),
    )
    async def _post_chat(self, body: dict[str, Any]) -> dict[str, Any]:
        """Send the chat request and validate the response."""
        url = f"{self.base_url}/api/chat"
        logger.info("Ollama request to %s with model %s", url, body.get("model"))
        resp = await self.http.post(url, json=body)
        resp.raise_for_status()
        data: dict[str, Any] = resp.json()
        if not data.get("done"):
            raise RuntimeError(f"Ollama response not finished: {data}")
        return data
@@ -0,0 +1,96 @@
 """Application settings loaded from config.yaml and environment variables."""
 from __future__ import annotations
 from pathlib import Path
 from typing import Any
 import yaml
 from pydantic import Field
 from pydantic_settings import BaseSettings, SettingsConfigDict
 class ModelConfig(BaseSettings):
    """Model names used by the pipeline."""
    classifier: str = "qwen2.5-vl:3b"
    ocr: str = "glm-ocr"
    extraction: str = "qwen2.5-vl:7b"
    review: str = "qwen2.5-vl:7b"
 class ReviewConfig(BaseSettings):
    """Review stage thresholds."""
    high_confidence_threshold: float = 0.90
    min_confidence_threshold: float = 0.75
 class XmlConfig(BaseSettings):
    """XML builder defaults."""
    default_currency: str = "EUR"
    target: str = "vendor_bill"
 class PreprocessingConfig(BaseSettings):
    """Image preprocessing parameters."""
    target_dpi: int = 300
    max_image_dimension: int = 2048
    jpeg_quality: int = 85
 class CacheConfig(BaseSettings):
    """LLM response cache settings."""
    enabled: bool = True
    dir: Path = Path.home() / ".cache" / "odoo_ocr" / "llm_cache"
 class LoggingConfig(BaseSettings):
    """Logging settings."""
    level: str = "INFO"
    format: str = "%(asctime)s | %(levelname)s | %(name)s | %(message)s"
 class Settings(BaseSettings):
    """Combined application settings.
    Values are loaded from `config.yaml` in the project root, then overridden
    by environment variables. Env var names are uppercase with double
    underscores for nested fields, e.g. `MODELS__OCR=glm-ocr`.
    """
    model_config = SettingsConfigDict(
        env_file=".env",
        env_file_encoding="utf-8",
        env_nested_delimiter="__",
        extra="ignore",
    )
    ollama_base_url: str = "http://localhost:11434"
    models: ModelConfig = Field(default_factory=ModelConfig)
    review: ReviewConfig = Field(default_factory=ReviewConfig)
    xml: XmlConfig = Field(default_factory=XmlConfig)
    preprocessing: PreprocessingConfig = Field(default_factory=PreprocessingConfig)
    cache: CacheConfig = Field(default_factory=CacheConfig)
    logging: LoggingConfig = Field(default_factory=LoggingConfig)
    @classmethod
    def from_yaml(cls, path: Path | str = "config.yaml") -> Settings:
        """Load settings from a YAML file, then apply env overrides."""
        path = Path(path)
        data: dict[str, Any] = {}
        if path.exists():
            with path.open("r", encoding="utf-8") as f:
                loaded = yaml.safe_load(f)
                if isinstance(loaded, dict):
                    data = loaded
        return cls(**data)
 def get_settings() -> Settings:
    """Return loaded settings (no caching to allow env reloads in tests)."""
    return Settings.from_yaml()
@@ -0,0 +1,15 @@
 """Invoice processing pipeline stages."""
 from .classifier import classify_document
 from .extractor import extract_invoice
 from .loader import load_document
 from .reviewer import review_invoice
 from .xml_builder import build_odoo_xml
 __all__ = [
    "classify_document",
    "extract_invoice",
    "load_document",
    "review_invoice",
    "build_odoo_xml",
 ]
@@ -0,0 +1,112 @@
 """Document classification: heuristic + optional small VLM."""
 from __future__ import annotations
 import logging
 from pathlib import Path
 import fitz  # pymupdf
 from PIL import Image, ImageStat
 from odoo_ocr.clients import BaseVLMClient
 from odoo_ocr.config import Settings
 from odoo_ocr.schemas.document import ClassificationResult, DocumentClass
 logger = logging.getLogger(__name__)
 def _has_embedded_text(path: Path) -> bool:
    """Fast heuristic: does the PDF contain selectable text?"""
    try:
        with fitz.open(path) as doc:
            for page in doc:
                text = page.get_text().strip()
                if text and len(text) > 20:
                    return True
        return False
    except Exception:
        return False
 def _detect_handwriting(image: Image.Image) -> float:
    """Placeholder handwriting detection score based on simple image stats.
    A real implementation would use stroke-thickness variance or a dedicated
    classifier. Returns 0.0 (no handwriting) to 1.0 (definitely handwritten).
    """
    # Minimal heuristic: high grayscale variance often indicates handwriting.
    gray = image.convert("L")
    stat = ImageStat.Stat(gray)
    std = stat.stddev[0]
    score = min(std / 64.0, 1.0)  # rough normalization
    return round(score, 2)
 def _heuristic_classify(
    path: Path, pages: list[Image.Image]
 ) -> ClassificationResult | None:
    """Return a classification if heuristics are confident, otherwise None."""
    suffix = path.suffix.lower()
    if suffix == ".pdf" and _has_embedded_text(path):
        return ClassificationResult(
            category=DocumentClass.digital_pdf,
            confidence=0.95,
            reasoning="PDF contains embedded text layer.",
        )
    if pages:
        hw_score = _detect_handwriting(pages[0])
        if hw_score > 0.75:
            return ClassificationResult(
                category=DocumentClass.handwritten,
                confidence=hw_score,
                reasoning="Image exhibits handwriting-like stroke variance.",
            )
        if hw_score < 0.25:
            return ClassificationResult(
                category=DocumentClass.scanned_print,
                confidence=1.0 - hw_score,
                reasoning="Image appears to be printed text.",
            )
    return None
 async def classify_document(
    path: Path,
    pages: list[Image.Image],
    client: BaseVLMClient,
    settings: Settings,
 ) -> ClassificationResult:
    """Classify an invoice document.
    First tries fast heuristics. If confidence is low, asks the configured
    classifier VLM.
    """
    heuristic = _heuristic_classify(path, pages)
    if heuristic and heuristic.confidence >= 0.85:
        logger.info("Heuristic classifier: %s", heuristic.category)
        return heuristic
    system_prompt = _load_prompt("classifier_system.txt")
    user_prompt = (
        "Classify this invoice document. Return only the requested JSON object."
    )
    response = await client.complete(
        system_prompt=system_prompt,
        user_prompt=user_prompt,
        images=[pages[0]],
        response_format=ClassificationResult,
        temperature=0.1,
        model=settings.models.classifier,
    )
    result = client.parse_json(response.content, ClassificationResult)
    logger.info("VLM classifier: %s (confidence %.2f)", result.category, result.confidence)
    return result
 def _load_prompt(name: str) -> str:
    from odoo_ocr.clients.base import BaseVLMClient
    return BaseVLMClient._load_prompt_file(Path("prompts") / name)
@@ -0,0 +1,34 @@
 """Processing branch for native digital PDFs."""
 from __future__ import annotations
 import logging
 from pathlib import Path
 import fitz  # pymupdf
 from PIL import Image
 from odoo_ocr.clients import BaseVLMClient
 from odoo_ocr.config import Settings
 logger = logging.getLogger(__name__)
 async def process_digital_pdf(
    path: Path,
    pages: list[Image.Image],
    client: BaseVLMClient,
    settings: Settings,
 ) -> str:
    """Extract text from a native PDF, preserving layout where possible.
    Returns the combined raw text of all pages.
    """
    parts: list[str] = []
    with fitz.open(path) as doc:
        for i, page in enumerate(doc):
            text = page.get_text("text").strip()
            parts.append(f"--- Page {i + 1} ---\n{text}")
    full_text = "\n\n".join(parts)
    logger.info("Extracted %d characters from digital PDF %s", len(full_text), path)
    return full_text
@@ -0,0 +1,52 @@
 """Structured invoice extraction from raw OCR text + image."""
 from __future__ import annotations
 import logging
 from pathlib import Path
 from PIL import Image
 from odoo_ocr.clients import BaseVLMClient
 from odoo_ocr.config import Settings
 from odoo_ocr.schemas import ExtractedInvoice
 logger = logging.getLogger(__name__)
 def _load_prompt(name: str) -> str:
    from odoo_ocr.clients.base import BaseVLMClient
    return BaseVLMClient._load_prompt_file(Path("prompts") / name)
 async def extract_invoice(
    raw_text: str,
    pages: list[Image.Image],
    client: BaseVLMClient,
    settings: Settings,
 ) -> ExtractedInvoice:
    """Convert raw OCR text into a validated ExtractedInvoice."""
    system_prompt = _load_prompt("extraction_system.txt")
    user_prompt = (
        f"OCR text extracted from the invoice:\n```\n{raw_text}\n```\n\n"
        "Now extract the structured invoice fields from the image and OCR text. "
        f"Default currency is {settings.xml.default_currency} unless otherwise stated."
    )
    response = await client.complete(
        system_prompt=system_prompt,
        user_prompt=user_prompt,
        images=[pages[0]] if pages else [],
        response_format=ExtractedInvoice,
        temperature=0.1,
        model=settings.models.extraction,
    )
    extracted = client.parse_json(response.content, ExtractedInvoice)
    logger.info(
        "Extracted invoice %s from %s with %d line(s)",
        extracted.invoice_number,
        extracted.vendor_name,
        len(extracted.line_items),
    )
    return extracted
@@ -0,0 +1,73 @@
 """Processing branch for handwritten invoices."""
 from __future__ import annotations
 import logging
 from pathlib import Path
 import cv2
 import numpy as np
 from PIL import Image
 from odoo_ocr.clients import BaseVLMClient
 from odoo_ocr.config import Settings
 logger = logging.getLogger(__name__)
 def _preprocess_handwritten(image: Image.Image) -> Image.Image:
    """Apply handwriting-specific preprocessing.
    Steps: grayscale, Otsu binarize, slight dilation to connect strokes.
    """
    gray = image.convert("L")
    arr = np.array(gray)
    _, binary = cv2.threshold(arr, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
    kernel = np.ones((2, 2), np.uint8)
    processed = cv2.morphologyEx(binary, cv2.MORPH_CLOSE, kernel)
    return Image.fromarray(processed, mode="L").convert("RGB")
 def _load_prompt(name: str) -> str:
    from pathlib import Path
    from odoo_ocr.clients.base import BaseVLMClient
    return BaseVLMClient._load_prompt_file(Path("prompts") / name)
 async def process_handwritten(
    path: Path,
    pages: list[Image.Image],
    client: BaseVLMClient,
    settings: Settings,
 ) -> str:
    """Run OCR on handwritten invoice pages using the configured handwriting model."""
    system_prompt = _load_prompt("ocr_system.txt")
    user_prompt = (
        "Read all text from this handwritten invoice accurately. "
        "Preserve line breaks and mark unclear words with [UNCLEAR]."
    )
    page_texts: list[str] = []
    for i, page in enumerate(pages):
        preprocessed = _preprocess_handwritten(page)
        response = await client.complete(
            system_prompt=system_prompt,
            user_prompt=f"{user_prompt} (page {i + 1} of {len(pages)})",
            images=[preprocessed],
            temperature=0.1,
            model=settings.models.ocr,
        )
        import json
        try:
            parsed = json.loads(response.content.strip().strip("`").removeprefix("json").strip())
            text = parsed.get("text", response.content)
        except Exception:
            text = response.content
        page_texts.append(f"--- Page {i + 1} ---\n{text.strip()}")
    full_text = "\n\n".join(page_texts)
    logger.info("Handwritten OCR extracted %d characters from %s", len(full_text), path)
    return full_text
@@ -0,0 +1,68 @@
 """Load PDFs and images into normalized page images."""
 from __future__ import annotations
 import logging
 from pathlib import Path
 from typing import Any
 import fitz  # pymupdf
 from PIL import Image
 from odoo_ocr.config import Settings
 logger = logging.getLogger(__name__)
 SUPPORTED_IMAGE_SUFFIXES = {".png", ".jpg", ".jpeg", ".tiff", ".tif", ".bmp", ".webp"}
 def _is_pdf(path: Path) -> bool:
    return path.suffix.lower() == ".pdf"
 def _pdf_to_images(path: Path, dpi: int = 300) -> list[Image.Image]:
    """Render PDF pages to PIL images using PyMuPDF."""
    pages: list[Image.Image] = []
    with fitz.open(path) as doc:
        for page in doc:
            matrix = fitz.Matrix(dpi / 72, dpi / 72)
            pix = page.get_pixmap(matrix=matrix)
            mode = "RGB" if pix.n == 3 else "RGBA"
            img = Image.frombytes(mode, (pix.width, pix.height), pix.samples)
            pages.append(img.convert("RGB"))
    logger.info("Loaded %d page(s) from PDF %s", len(pages), path)
    return pages
 def _image_to_pil(path: Path) -> list[Image.Image]:
    img = Image.open(path)
    return [img.convert("RGB")]
 def load_document(path: Path | str, settings: Settings | None = None) -> dict[str, Any]:
    """Load a document file and return a dict with pages and metadata.
    Returns:
        {
            "pages": [PIL.Image.Image],
            "page_count": int,
            "source_path": Path,
        }
    """
    path = Path(path)
    if not path.exists():
        raise FileNotFoundError(f"Document not found: {path}")
    if _is_pdf(path):
        dpi = (settings.preprocessing.target_dpi if settings else 300)
        pages = _pdf_to_images(path, dpi=dpi)
    elif path.suffix.lower() in SUPPORTED_IMAGE_SUFFIXES:
        pages = _image_to_pil(path)
    else:
        raise ValueError(f"Unsupported file type: {path.suffix}")
    return {
        "pages": pages,
        "page_count": len(pages),
        "source_path": path,
    }
@@ -0,0 +1,61 @@
 """Processing branch for ambiguous or low-quality invoice images."""
 from __future__ import annotations
 import logging
 from pathlib import Path
 from PIL import Image
 from odoo_ocr.clients import BaseVLMClient
 from odoo_ocr.config import Settings
 logger = logging.getLogger(__name__)
 def _load_prompt(name: str) -> str:
    from pathlib import Path
    from odoo_ocr.clients.base import BaseVLMClient
    return BaseVLMClient._load_prompt_file(Path("prompts") / name)
 async def process_mixed_unknown(
    path: Path,
    pages: list[Image.Image],
    client: BaseVLMClient,
    settings: Settings,
 ) -> str:
    """Robust OCR for mixed/unknown documents.
    For now this runs the same OCR path as scanned_print but with a more
    permissive prompt. Future: ensemble multiple models and merge outputs.
    """
    system_prompt = _load_prompt("ocr_system.txt")
    user_prompt = (
        "Extract all readable text from this document. It may contain a mix of "
        "printed text, handwriting, stamps, or low-quality scans. Preserve layout."
    )
    page_texts: list[str] = []
    for i, page in enumerate(pages):
        response = await client.complete(
            system_prompt=system_prompt,
            user_prompt=f"{user_prompt} (page {i + 1} of {len(pages)})",
            images=[page],
            temperature=0.1,
            model=settings.models.ocr,
        )
        import json
        try:
            parsed = json.loads(response.content.strip().strip("`").removeprefix("json").strip())
            text = parsed.get("text", response.content)
        except Exception:
            text = response.content
        page_texts.append(f"--- Page {i + 1} ---\n{text.strip()}")
    full_text = "\n\n".join(page_texts)
    logger.info("Robust OCR extracted %d characters from %s", len(full_text), path)
    return full_text
@@ -0,0 +1,52 @@
 """Vision review: compare extracted invoice against the original image."""
 from __future__ import annotations
 import logging
 from pathlib import Path
 from PIL import Image
 from odoo_ocr.clients import BaseVLMClient
 from odoo_ocr.config import Settings
 from odoo_ocr.schemas import ExtractedInvoice, ReviewResult
 logger = logging.getLogger(__name__)
 def _load_prompt(name: str) -> str:
    from odoo_ocr.clients.base import BaseVLMClient
    return BaseVLMClient._load_prompt_file(Path("prompts") / name)
 async def review_invoice(
    invoice: ExtractedInvoice,
    pages: list[Image.Image],
    client: BaseVLMClient,
    settings: Settings,
 ) -> ReviewResult:
    """Have a vision model review the extracted invoice against the image."""
    system_prompt = _load_prompt("review_system.txt")
    user_prompt = (
        "Extracted invoice JSON:\n```json\n"
        f"{invoice.model_dump_json(indent=2)}"
        "\n```\n\nCompare this with the original invoice image and report any issues."
    )
    response = await client.complete(
        system_prompt=system_prompt,
        user_prompt=user_prompt,
        images=[pages[0]] if pages else [],
        response_format=ReviewResult,
        temperature=0.1,
        model=settings.models.review,
    )
    review = client.parse_json(response.content, ReviewResult)
    logger.info(
        "Review result: valid=%s confidence=%.2f issues=%d",
        review.valid,
        review.confidence,
        len(review.issues),
    )
    return review
@@ -0,0 +1,55 @@
 """Processing branch for scanned/photographed printed invoices."""
 from __future__ import annotations
 import logging
 from pathlib import Path
 from PIL import Image
 from odoo_ocr.clients import BaseVLMClient
 from odoo_ocr.config import Settings
 logger = logging.getLogger(__name__)
 def _load_prompt(name: str) -> str:
    from pathlib import Path
    from odoo_ocr.clients.base import BaseVLMClient
    return BaseVLMClient._load_prompt_file(Path("prompts") / name)
 async def process_scanned_print(
    path: Path,
    pages: list[Image.Image],
    client: BaseVLMClient,
    settings: Settings,
 ) -> str:
    """Run OCR on each page of a scanned printed invoice and return combined text."""
    system_prompt = _load_prompt("ocr_system.txt")
    user_prompt = "Read all text from this printed invoice page accurately."
    page_texts: list[str] = []
    for i, page in enumerate(pages):
        response = await client.complete(
            system_prompt=system_prompt,
            user_prompt=f"{user_prompt} (page {i + 1} of {len(pages)})",
            images=[page],
            temperature=0.1,
            model=settings.models.ocr,
        )
        # The OCR prompt asks for {"text": "..."}.
        import json
        try:
            parsed = json.loads(response.content.strip().strip("`").removeprefix("json").strip())
            text = parsed.get("text", response.content)
        except Exception:
            text = response.content
        page_texts.append(f"--- Page {i + 1} ---\n{text.strip()}")
    full_text = "\n\n".join(page_texts)
    logger.info("OCR extracted %d characters from scanned invoice %s", len(full_text), path)
    return full_text
@@ -0,0 +1,146 @@
 """Build Odoo Enterprise vendor-bill import XML from ExtractedInvoice."""
 from __future__ import annotations
 import hashlib
 import logging
 import re
 from decimal import Decimal
 from lxml import etree
 from odoo_ocr.config import Settings
 from odoo_ocr.schemas import ExtractedInvoice, ReviewResult
 logger = logging.getLogger(__name__)
 CURRENCY_REFS: dict[str, str] = {
    "EUR": "base.EUR",
    "USD": "base.USD",
    "GBP": "base.GBP",
    "CHF": "base.CHF",
    "JPY": "base.JPY",
 }
 def _sanitize_id(value: str) -> str:
    """Create a valid Odoo external ID from arbitrary text."""
    value = value.lower().strip()
    value = re.sub(r"[^a-z0-9]+", "_", value)
    value = value.strip("_")
    if not value:
        value = "x"
    if value[0].isdigit():
        value = f"x_{value}"
    return value
 def _external_id_vendor(vendor_name: str, vendor_vat: str | None = None) -> str:
    seed = vendor_vat if vendor_vat else vendor_name
    short = hashlib.sha256(seed.encode("utf-8")).hexdigest()[:8]
    return f"vendor_{_sanitize_id(vendor_name)}_{short}"
 def _external_id_invoice(invoice: ExtractedInvoice) -> str:
    seed = f"{invoice.invoice_number}_{invoice.invoice_date}_{invoice.vendor_name}"
    short = hashlib.sha256(seed.encode("utf-8")).hexdigest()[:8]
    number = _sanitize_id(invoice.invoice_number) or "unknown"
    return f"invoice_{number}_{invoice.invoice_date.replace('-', '')}_{short}"
 def _tax_external_id(rate: Decimal) -> str:
    return f"purchase_vat_{int(rate)}"
 def _currency_ref(currency: str, settings: Settings) -> str:
    return CURRENCY_REFS.get(currency.upper(), CURRENCY_REFS.get(settings.xml.default_currency.upper(), "base.EUR"))
 def _to_str(value: Decimal) -> str:
    return f"{value:.2f}"
 def build_odoo_xml(
    invoice: ExtractedInvoice,
    review: ReviewResult | None,
    settings: Settings,
 ) -> str:
    """Generate Odoo data-import XML for the invoice as a vendor bill."""
    vendor_id = _external_id_vendor(invoice.vendor_name, invoice.vendor_vat)
    invoice_id = _external_id_invoice(invoice)
    odoo = etree.Element("odoo")
    data = etree.SubElement(odoo, "data", noupdate="0")
    # Human-review warning comment
    if review and review.confidence < settings.review.high_confidence_threshold:
        data.append(
            etree.Comment(
                f" HUMAN REVIEW ADVISED: confidence={review.confidence:.2f}, "
                f"issues={len(review.issues)} "
            )
        )
    # Vendor
    vendor_rec = etree.SubElement(data, "record", id=vendor_id, model="res.partner")
    etree.SubElement(vendor_rec, "field", name="name").text = invoice.vendor_name
    etree.SubElement(vendor_rec, "field", name="supplier_rank").text = "1"
    if invoice.vendor_address:
        etree.SubElement(vendor_rec, "field", name="street").text = invoice.vendor_address
    if invoice.vendor_vat:
        etree.SubElement(vendor_rec, "field", name="vat").text = invoice.vendor_vat
    # Taxes
    tax_ids_used: list[str] = []
    for line in invoice.line_items:
        rate = line.tax_rate
        if rate == 0 and "purchase_vat_0" not in tax_ids_used:
            tax_ids_used.append("purchase_vat_0")
        else:
            tid = _tax_external_id(rate)
            if tid not in tax_ids_used:
                tax_ids_used.append(tid)
            tax_rec = etree.SubElement(data, "record", id=tid, model="account.tax")
            etree.SubElement(tax_rec, "field", name="name").text = f"Purchase VAT {int(rate)}%"
            etree.SubElement(tax_rec, "field", name="amount").text = str(rate)
            etree.SubElement(tax_rec, "field", name="amount_type").text = "percent"
            etree.SubElement(tax_rec, "field", name="type_tax_use").text = "purchase"
    if "purchase_vat_0" in tax_ids_used:
        tax_rec = etree.SubElement(data, "record", id="purchase_vat_0", model="account.tax")
        etree.SubElement(tax_rec, "field", name="name").text = "Purchase VAT 0%"
        etree.SubElement(tax_rec, "field", name="amount").text = "0"
        etree.SubElement(tax_rec, "field", name="amount_type").text = "percent"
        etree.SubElement(tax_rec, "field", name="type_tax_use").text = "purchase"
    # Vendor bill header
    move_rec = etree.SubElement(data, "record", id=invoice_id, model="account.move")
    etree.SubElement(move_rec, "field", name="move_type").text = "in_invoice"
    etree.SubElement(move_rec, "field", name="partner_id", ref=vendor_id)
    etree.SubElement(move_rec, "field", name="invoice_date").text = invoice.invoice_date
    etree.SubElement(move_rec, "field", name="date").text = invoice.invoice_date
    if invoice.due_date:
        etree.SubElement(move_rec, "field", name="invoice_date_due").text = invoice.due_date
    etree.SubElement(move_rec, "field", name="ref").text = invoice.invoice_number
    etree.SubElement(move_rec, "field", name="currency_id", ref=_currency_ref(invoice.currency, settings))
    if invoice.payment_terms:
        etree.SubElement(move_rec, "field", name="narration").text = invoice.payment_terms
    # Lines
    for i, line in enumerate(invoice.line_items, start=1):
        line_id = f"{invoice_id}_line_{i}"
        line_rec = etree.SubElement(data, "record", id=line_id, model="account.move.line")
        etree.SubElement(line_rec, "field", name="move_id", ref=invoice_id)
        etree.SubElement(line_rec, "field", name="name").text = line.description
        etree.SubElement(line_rec, "field", name="quantity").text = _to_str(line.quantity)
        etree.SubElement(line_rec, "field", name="price_unit").text = _to_str(line.unit_price)
        tax_ref = _tax_external_id(line.tax_rate)
        etree.SubElement(
            line_rec,
            "field",
            name="tax_ids",
            eval=f"[(6, 0, [ref('{tax_ref}')])]",
        )
    xml_bytes: bytes = etree.tostring(odoo, pretty_print=True, xml_declaration=True, encoding="UTF-8")
    return xml_bytes.decode("utf-8")
@@ -0,0 +1,16 @@
 """Pydantic schemas for the Odoo OCR pipeline."""
 from .document import DocumentClass, ProcessingContext
 from .invoice import ExtractedInvoice, InvoiceLineItem
 from .review import ReviewIssue, ReviewResult
 from .vlm import VLMResponse
 __all__ = [
    "DocumentClass",
    "ProcessingContext",
    "ExtractedInvoice",
    "InvoiceLineItem",
    "ReviewIssue",
    "ReviewResult",
    "VLMResponse",
 ]
@@ -0,0 +1,41 @@
 """Document classification and processing context schemas."""
 from __future__ import annotations
 from enum import StrEnum
 from pathlib import Path
 from typing import Any
 from pydantic import BaseModel, Field
 class DocumentClass(StrEnum):
    """Class of invoice document received by the pipeline."""
    digital_pdf = "digital_pdf"
    scanned_print = "scanned_print"
    handwritten = "handwritten"
    mixed_unknown = "mixed_unknown"
 class ClassificationResult(BaseModel):
    """Output of the document classifier."""
    category: DocumentClass
    confidence: float = Field(ge=0.0, le=1.0)
    reasoning: str = ""
 class ProcessingContext(BaseModel):
    """Mutable-ish context passed through pipeline stages.
    Holds the original file path, classification, settings handle, and any
    intermediate artifacts produced by earlier stages.
    """
    source_path: Path
    classification: ClassificationResult | None = None
    raw_text: str = ""
    extracted_invoice: dict[str, Any] | None = None
    review_result: dict[str, Any] | None = None
    output_dir: Path | None = None
@@ -0,0 +1,58 @@
 """Invoice data schemas."""
 from __future__ import annotations
 from decimal import Decimal
 from typing import Any
 from pydantic import BaseModel, ConfigDict, Field, field_validator
 class InvoiceLineItem(BaseModel):
    """A single line on an invoice."""
    model_config = ConfigDict(str_strip_whitespace=True)
    description: str
    quantity: Decimal = Field(default=Decimal("1"), ge=Decimal("0"))
    unit_price: Decimal = Field(default=Decimal("0"), ge=Decimal("0"))
    total_price: Decimal = Field(default=Decimal("0"), ge=Decimal("0"))
    tax_rate: Decimal = Field(default=Decimal("0"), ge=Decimal("0"))
    @field_validator("quantity", "unit_price", "total_price", "tax_rate", mode="before")
    @classmethod
    def _coerce_decimal(cls, value: Any) -> Decimal:
        if value is None:
            return Decimal("0")
        return Decimal(str(value))
 class ExtractedInvoice(BaseModel):
    """Structured representation of an invoice after OCR and extraction."""
    model_config = ConfigDict(str_strip_whitespace=True)
    vendor_name: str
    vendor_address: str | None = None
    vendor_vat: str | None = None
    invoice_number: str
    invoice_date: str  # ISO 8601, e.g. 2024-05-01
    due_date: str | None = None
    currency: str = "EUR"
    payment_terms: str | None = None
    line_items: list[InvoiceLineItem] = Field(default_factory=list)
    subtotal: Decimal = Field(default=Decimal("0"))
    tax_total: Decimal = Field(default=Decimal("0"))
    total: Decimal = Field(default=Decimal("0"))
    iban: str | None = None
    raw_ocr_text: str = ""
    @field_validator("subtotal", "tax_total", "total", mode="before")
    @classmethod
    def _coerce_decimal(cls, value: Any) -> Decimal:
        if value is None:
            return Decimal("0")
        return Decimal(str(value))
    def line_items_sum(self) -> Decimal:
        return sum((line.total_price for line in self.line_items), Decimal("0"))
@@ -0,0 +1,29 @@
 """Review/validation schemas."""
 from __future__ import annotations
 from typing import Any, Literal
 from pydantic import BaseModel, ConfigDict, Field
 from .invoice import ExtractedInvoice
 class ReviewIssue(BaseModel):
    """A single issue found by the review model."""
    model_config = ConfigDict(str_strip_whitespace=True)
    field: str
    severity: Literal["error", "warning"]
    message: str
    suggested_value: Any | None = None
 class ReviewResult(BaseModel):
    """Output of the vision review stage."""
    valid: bool
    confidence: float = Field(ge=0.0, le=1.0)
    issues: list[ReviewIssue] = Field(default_factory=list)
    corrected_invoice: ExtractedInvoice | None = None
@@ -0,0 +1,13 @@
 """VLM client response schema."""
 from pydantic import BaseModel
 class VLMResponse(BaseModel):
    """Normalized response from any local VLM backend."""
    content: str
    model: str
    prompt_tokens: int | None = None
    completion_tokens: int | None = None
    duration_ms: int = 0
@@ -0,0 +1 @@
 """Shared utilities."""
@@ -0,0 +1,71 @@
 """Simple file-based cache for LLM responses."""
 from __future__ import annotations
 import hashlib
 import json
 import logging
 from pathlib import Path
 from typing import Any, cast
 from odoo_ocr.config import Settings
 logger = logging.getLogger(__name__)
 def _make_key(parts: list[str]) -> str:
    hasher = hashlib.sha256()
    for part in parts:
        hasher.update(part.encode("utf-8"))
    return hasher.hexdigest()
 def _cache_path(settings: Settings, key: str) -> Path:
    cache_dir = settings.cache.dir.expanduser()
    cache_dir.mkdir(parents=True, exist_ok=True)
    return cache_dir / f"{key}.json"
 def get_cached(
    system_prompt: str,
    user_prompt: str,
    image_blobs: list[bytes],
    model: str,
    temperature: float,
    settings: Settings,
 ) -> dict[str, Any] | None:
    """Return cached response dict if it exists, otherwise None."""
    if not settings.cache.enabled:
        return None
    parts = [system_prompt, user_prompt, model, str(temperature)]
    for blob in image_blobs:
        parts.append(hashlib.sha256(blob).hexdigest())
    key = _make_key(parts)
    path = _cache_path(settings, key)
    if path.exists():
        logger.debug("Cache hit: %s", key)
        with path.open("r", encoding="utf-8") as f:
            return cast(dict[str, Any], json.load(f))
    return None
 def set_cached(
    system_prompt: str,
    user_prompt: str,
    image_blobs: list[bytes],
    model: str,
    temperature: float,
    response: dict[str, Any],
    settings: Settings,
 ) -> None:
    """Store a response in the file cache."""
    if not settings.cache.enabled:
        return
    parts = [system_prompt, user_prompt, model, str(temperature)]
    for blob in image_blobs:
        parts.append(hashlib.sha256(blob).hexdigest())
    key = _make_key(parts)
    path = _cache_path(settings, key)
    with path.open("w", encoding="utf-8") as f:
        json.dump(response, f, ensure_ascii=False, indent=2)
    logger.debug("Cached response: %s", key)
@@ -0,0 +1,16 @@
 """Centralised logging configuration."""
 import logging
 import sys
 from odoo_ocr.config import Settings
 def configure_logging(settings: Settings) -> None:
    """Configure the root logger from settings."""
    handler = logging.StreamHandler(sys.stdout)
    handler.setFormatter(logging.Formatter(settings.logging.format))
    root = logging.getLogger()
    root.handlers.clear()
    root.addHandler(handler)
    root.setLevel(settings.logging.level)
@@ -0,0 +1,54 @@
 """Shared pytest fixtures."""
 from __future__ import annotations
 import json
 from typing import Any
 import pytest
 from PIL import Image
 from odoo_ocr.clients.base import BaseVLMClient
 from odoo_ocr.config import Settings
 from odoo_ocr.schemas import VLMResponse
@pytest.fixture
 def settings() -> Settings:
    """Default test settings with cache disabled."""
    return Settings(
        ollama_base_url="http://test-ollama:11435",
        cache=Settings().model_dump()["cache"] | {"enabled": False},
    )
 class FakeVLMClient(BaseVLMClient):
    """Test double that returns canned responses based on prompt content."""
    def __init__(self, responses: dict[str, str] | None = None) -> None:
        super().__init__(Settings())
        self.responses = responses or {}
        self.calls: list[dict[str, Any]] = []
    async def complete(
        self,
        system_prompt: str,
        user_prompt: str,
        images: list[Image.Image],
        response_format: type[Any] | None = None,
        temperature: float = 0.1,
        max_tokens: int = 4096,
        model: str | None = None,
    ) -> VLMResponse:
        self.calls.append(
            {
                "system_prompt": system_prompt,
                "user_prompt": user_prompt,
                "image_count": len(images),
                "model": model,
            }
        )
        for key, value in self.responses.items():
            if key in system_prompt or key in user_prompt:
                return VLMResponse(content=json.dumps(value), model="fake")
        return VLMResponse(content=json.dumps({}), model="fake")
@@ -0,0 +1,72 @@
 """Tests for the document classifier."""
 from __future__ import annotations
 from pathlib import Path
 from unittest.mock import patch
 import pytest
 from PIL import Image
 from odoo_ocr.pipeline.classifier import classify_document
 from odoo_ocr.schemas.document import DocumentClass
@pytest.fixture
 def blank_image() -> Image.Image:
    return Image.new("RGB", (100, 100), color="white")
@pytest.mark.asyncio
 async def test_heuristic_digital_pdf(tmp_path: Path) -> None:
    pdf_path = tmp_path / "invoice.pdf"
    # Minimal valid PDF with text via pymupdf
    import fitz
    doc = fitz.open()
    page = doc.new_page()
    page.insert_text((50, 50), "Invoice number 12345\nVendor: Acme")
    doc.save(str(pdf_path))
    doc.close()
    from odoo_ocr.config import Settings
    from tests.conftest import FakeVLMClient
    client = FakeVLMClient()
    result = await classify_document(pdf_path, [], client, Settings())
    assert result.category == DocumentClass.digital_pdf
    assert result.confidence >= 0.90
@pytest.mark.asyncio
 async def test_heuristic_scanned_print(blank_image: Image.Image) -> None:
    # Blank image has low handwriting score
    from odoo_ocr.config import Settings
    from tests.conftest import FakeVLMClient
    client = FakeVLMClient()
    result = await classify_document(Path("scan.jpg"), [blank_image], client, Settings())
    assert result.category == DocumentClass.scanned_print
@pytest.mark.asyncio
 async def test_vlm_classifier_fallback(blank_image: Image.Image) -> None:
    from odoo_ocr.config import Settings
    from tests.conftest import FakeVLMClient
    client = FakeVLMClient(
        responses={
            "document classifier": {
                "category": "handwritten",
                "confidence": 0.91,
                "reasoning": "handwritten text visible",
            }
        }
    )
    with patch(
        "odoo_ocr.pipeline.classifier._heuristic_classify", return_value=None
    ):
        result = await classify_document(Path("unknown.png"), [blank_image], client, Settings())
    assert result.category == DocumentClass.handwritten
    assert result.confidence == 0.91
@@ -0,0 +1,46 @@
 """Tests for the structured extraction stage."""
 import pytest
 from PIL import Image
 from odoo_ocr.config import Settings
 from odoo_ocr.pipeline.extractor import extract_invoice
 from odoo_ocr.schemas import ExtractedInvoice
 from tests.conftest import FakeVLMClient
@pytest.mark.asyncio
 async def test_extract_invoice() -> None:
    client = FakeVLMClient(
        responses={
            "invoice data extraction": {
                "vendor_name": "Acme",
                "vendor_address": None,
                "vendor_vat": None,
                "invoice_number": "INV-1",
                "invoice_date": "2024-01-01",
                "due_date": None,
                "currency": "EUR",
                "payment_terms": None,
                "line_items": [
                    {
                        "description": "Widget",
                        "quantity": 2.0,
                        "unit_price": 50.0,
                        "total_price": 100.0,
                        "tax_rate": 0.0,
                    }
                ],
                "subtotal": 100.0,
                "tax_total": 0.0,
                "total": 100.0,
                "iban": None,
                "raw_ocr_text": "",
            }
        }
    )
    result = await extract_invoice("raw text", [Image.new("RGB", (10, 10))], client, Settings())
    assert isinstance(result, ExtractedInvoice)
    assert result.vendor_name == "Acme"
    assert result.line_items_sum() == 100
@@ -0,0 +1,74 @@
 """Tests for the Ollama VLM client."""
 import base64
 import json
 from io import BytesIO
 from unittest.mock import AsyncMock, MagicMock
 import pytest
 from PIL import Image
 from odoo_ocr.clients.base import BaseVLMClient
 from odoo_ocr.clients.ollama import OllamaClient
 from odoo_ocr.config import Settings
 from odoo_ocr.schemas import ExtractedInvoice
@pytest.fixture
 def settings() -> Settings:
    return Settings(
        ollama_base_url="http://test-ollama:11434",
        cache=Settings().model_dump()["cache"] | {"enabled": False},
    )
@pytest.mark.asyncio
 async def test_complete_parses_response(settings: Settings) -> None:
    client = OllamaClient(settings)
    fake_response = {
        "model": "glm-ocr",
        "message": {"role": "assistant", "content": '{"text": "Invoice 123"}'},
        "done": True,
        "prompt_eval_count": 100,
        "eval_count": 20,
    }
    resp = MagicMock()
    resp.json.return_value = fake_response
    resp.raise_for_status = lambda: None
    client.http = AsyncMock()
    client.http.post.return_value = resp
    img = Image.new("RGB", (50, 50), color="red")
    response = await client.complete(
        system_prompt="ocr",
        user_prompt="read",
        images=[img],
    )
    assert response.content == '{"text": "Invoice 123"}'
    assert response.model == "glm-ocr"
    assert response.prompt_tokens == 100
    assert response.completion_tokens == 20
 def test_parse_json_extracted_invoice() -> None:
    data = {
        "vendor_name": "Acme",
        "invoice_number": "1",
        "invoice_date": "2024-01-01",
        "line_items": [],
        "subtotal": "100",
        "tax_total": "20",
        "total": "120",
    }
    result = BaseVLMClient.parse_json(json.dumps(data), ExtractedInvoice)
    assert isinstance(result, ExtractedInvoice)
    assert result.vendor_name == "Acme"
 def test_encode_image_roundtrip() -> None:
    img = Image.new("RGB", (10, 10), color="blue")
    encoded = OllamaClient._encode_image(img)  # type: ignore[attr-defined]
    decoded = Image.open(BytesIO(base64.b64decode(encoded)))
    assert decoded.size == (10, 10)
@@ -0,0 +1,60 @@
 """Tests for the Odoo XML builder."""
 from decimal import Decimal
 from odoo_ocr.config import Settings
 from odoo_ocr.pipeline.xml_builder import build_odoo_xml
 from odoo_ocr.schemas import ExtractedInvoice, InvoiceLineItem, ReviewResult
 def test_build_odoo_xml() -> None:
    settings = Settings()
    invoice = ExtractedInvoice(
        vendor_name="Acme Supplies",
        vendor_vat="GB123456789",
        invoice_number="INV-2024-001",
        invoice_date="2024-05-01",
        due_date="2024-06-01",
        currency="EUR",
        payment_terms="Net 30",
        line_items=[
            InvoiceLineItem(
                description="Consulting services",
                quantity=Decimal("10.0"),
                unit_price=Decimal("100.00"),
                total_price=Decimal("1000.00"),
                tax_rate=Decimal("20"),
            )
        ],
        subtotal=Decimal("1000.00"),
        tax_total=Decimal("200.00"),
        total=Decimal("1200.00"),
    )
    review = ReviewResult(valid=True, confidence=0.95)
    xml = build_odoo_xml(invoice, review, settings)
    assert xml.startswith("<?xml version=")
    assert 'model="res.partner"' in xml
    assert 'model="account.move"' in xml
    assert 'model="account.move.line"' in xml
    assert "Acme Supplies" in xml
    assert "INV-2024-001" in xml
    assert "in_invoice" in xml
 def test_xml_adds_review_comment_for_low_confidence() -> None:
    settings = Settings()
    invoice = ExtractedInvoice(
        vendor_name="X",
        invoice_number="1",
        invoice_date="2024-01-01",
        line_items=[],
        subtotal=Decimal("0"),
        tax_total=Decimal("0"),
        total=Decimal("0"),
    )
    review = ReviewResult(valid=False, confidence=0.80)
    xml = build_odoo_xml(invoice, review, settings)
    assert "HUMAN REVIEW ADVISED" in xml
		`@@ -0,0 +1,3 @@`
							`"""Local-first invoice OCR pipeline for Odoo Enterprise."""`

							`__version__ = "0.1.0"`