Initial scaffold: local invoice OCR pipeline with Ollama, classifier branches, structured extraction, review, and Odoo XML export
- Add AGENTS.md and project-specific Zed skills (odoo-ocr-pipeline, odoo-xml-import, local-vlm-client). - Implement Pydantic schemas for documents, invoices, review results, and VLM responses. - Add unified BaseVLMClient with Ollama implementation and llama.cpp stub. - Build pipeline stages: loader, classifier, digital_pdf/scanned_print/handwritten/mixed_unknown branches, extractor, reviewer, xml_builder. - Add CLI entry point with sidecar JSON and confidence-gated XML output. - Include prompts for classifier, OCR, extraction, and review models. - Add tests with FakeVLMClient; pytest, ruff, and mypy all pass.
This commit is contained in:
@@ -0,0 +1,101 @@
|
||||
---
|
||||
name: local-vlm-client
|
||||
description: Use this skill when implementing or modifying local vision-language model clients, prompts, or inference calls for the odoo_ocr project.
|
||||
---
|
||||
|
||||
# Local VLM Client Skill
|
||||
|
||||
Use this skill whenever you are writing model clients, prompts, or retry logic for local VLMs (Ollama or llama.cpp).
|
||||
|
||||
## Supported Runtimes
|
||||
|
||||
The project must support two local inference backends through a common interface:
|
||||
|
||||
1. **Ollama** (`src/odoo_ocr/clients/ollama_client.py`)
|
||||
- Default endpoint: `http://localhost:11434`.
|
||||
- Uses the Ollama Chat API: `POST /api/chat`.
|
||||
- Multimodal models receive images as base64 data URLs.
|
||||
- Model names follow Ollama convention, e.g., `qwen2.5-vl:7b`.
|
||||
|
||||
2. **llama.cpp server** (`src/odoo_ocr/clients/llama_cpp_client.py`)
|
||||
- Default endpoint: `http://localhost:8080`.
|
||||
- Uses OpenAI-compatible `POST /v1/chat/completions`.
|
||||
- Image data as base64 inside the `image_url` object.
|
||||
- Model name is often `local` or empty for single-model servers.
|
||||
|
||||
## Unified Interface
|
||||
|
||||
Both clients implement `BaseVLMClient` from `src/odoo_ocr/models/base_vlm.py`:
|
||||
|
||||
```python
|
||||
class BaseVLMClient(ABC):
|
||||
@abstractmethod
|
||||
async def complete(
|
||||
self,
|
||||
system_prompt: str,
|
||||
user_prompt: str,
|
||||
images: list[Image.Image],
|
||||
response_format: type[BaseModel] | None = None,
|
||||
temperature: float = 0.1,
|
||||
max_tokens: int = 4096,
|
||||
) -> VLMResponse:
|
||||
...
|
||||
|
||||
class VLMResponse(BaseModel):
|
||||
content: str
|
||||
model: str
|
||||
prompt_tokens: int | None
|
||||
completion_tokens: int | None
|
||||
duration_ms: int
|
||||
```
|
||||
|
||||
## Prompt Engineering Rules
|
||||
|
||||
1. Keep system prompts in `prompts/` as plain text files and load them at runtime.
|
||||
2. For extraction tasks, the system prompt must:
|
||||
- State that the output must be valid JSON only.
|
||||
- Provide the target Pydantic schema.
|
||||
- Include 1–2 examples of valid output.
|
||||
- Forbid markdown code fences and explanatory text.
|
||||
3. The user prompt should include the task and any context (e.g., "Extract invoice fields from this image. Currency is EUR unless otherwise stated.").
|
||||
4. For review tasks, the prompt must pass both the image and the JSON to compare.
|
||||
|
||||
## JSON Enforcement
|
||||
|
||||
1. Set `temperature` low (0.0–0.2) for extraction/review.
|
||||
2. Prefer structured output when the runtime supports it:
|
||||
- Ollama: use `format="json"` or `format=<json_schema>` if available.
|
||||
- llama.cpp server: use `response_format={"type": "json_object"}` or `json_schema` if the server build supports it.
|
||||
3. Always wrap the model output with a JSON parser; if parsing fails, retry once with a stronger "JSON only" reminder.
|
||||
4. Validate parsed JSON against the target Pydantic model and surface validation errors.
|
||||
|
||||
## Retry & Caching
|
||||
|
||||
1. Retry transient HTTP errors with exponential backoff (max 3 retries, base delay 1s).
|
||||
2. Cache successful responses keyed by SHA256 of (prompt + image bytes + model + temperature) to `~/.cache/odoo_ocr/llm_cache/`.
|
||||
3. Do not cache errors.
|
||||
4. Log every request: model, endpoint, prompt hash, duration, token counts.
|
||||
|
||||
## Image Encoding
|
||||
|
||||
1. Accept `PIL.Image.Image` inputs.
|
||||
2. Convert to RGB before encoding.
|
||||
3. Encode as JPEG or PNG base64 depending on content; prefer PNG if the image is already low-color or binarized.
|
||||
4. Resize images before sending if dimensions exceed model limits, preserving aspect ratio.
|
||||
|
||||
## Default Models
|
||||
|
||||
Use these defaults unless the task explicitly says otherwise:
|
||||
|
||||
| Task | Default model |
|
||||
|------|---------------|
|
||||
| Classification | `qwen2.5-vl:3b` |
|
||||
| OCR / extraction | `qwen2.5-vl:7b` |
|
||||
| Handwriting OCR | `glm-ocr` or project-configured handwriting model |
|
||||
| Review | `qwen2.5-vl:7b` |
|
||||
|
||||
## Testing
|
||||
|
||||
- Mock the HTTP client in unit tests; never call real models during CI.
|
||||
- Provide a `FakeVLMClient` in tests that returns canned responses.
|
||||
- Test prompt assembly and JSON parsing separately from model calls.
|
||||
@@ -0,0 +1,36 @@
|
||||
You are an invoice data extraction assistant. Given an invoice image and its OCR text, produce a structured JSON representation of the invoice.
|
||||
|
||||
Output must match this Pydantic schema exactly and contain no markdown, no commentary:
|
||||
|
||||
{
|
||||
"vendor_name": "string",
|
||||
"vendor_address": "string or null",
|
||||
"vendor_vat": "string or null",
|
||||
"invoice_number": "string",
|
||||
"invoice_date": "YYYY-MM-DD",
|
||||
"due_date": "YYYY-MM-DD or null",
|
||||
"currency": "ISO 4217 code, e.g. EUR",
|
||||
"payment_terms": "string or null",
|
||||
"line_items": [
|
||||
{
|
||||
"description": "string",
|
||||
"quantity": 1.0,
|
||||
"unit_price": 0.00,
|
||||
"total_price": 0.00,
|
||||
"tax_rate": 0.00
|
||||
}
|
||||
],
|
||||
"subtotal": 0.00,
|
||||
"tax_total": 0.00,
|
||||
"total": 0.00,
|
||||
"iban": "string or null",
|
||||
"raw_ocr_text": "string"
|
||||
}
|
||||
|
||||
Rules:
|
||||
- Use null for missing optional fields.
|
||||
- Dates must be ISO 8601.
|
||||
- All monetary values are decimal numbers (do not use strings).
|
||||
- line_items total_price should equal quantity * unit_price (within rounding).
|
||||
- subtotal + tax_total should equal total (within rounding).
|
||||
- If the image and OCR disagree, trust the image.
|
||||
@@ -0,0 +1,5 @@
|
||||
You are an OCR engine. Read all text from the provided invoice image accurately.
|
||||
Preserve line breaks and table structure as much as possible.
|
||||
If a value is unclear, mark it with [UNCLEAR].
|
||||
Respond with a JSON object containing a single field "text" with the full OCR output.
|
||||
Do not add markdown formatting or explanations.
|
||||
@@ -0,0 +1,25 @@
|
||||
You are a meticulous invoice review assistant. You are given the original invoice image and a JSON object representing the extracted invoice data.
|
||||
|
||||
Your task:
|
||||
1. Verify that every field in the JSON is supported by the image.
|
||||
2. Check arithmetic: sum of line item totals plus tax should equal the invoice total.
|
||||
3. Identify missing fields, incorrect values, or formatting problems.
|
||||
|
||||
Respond with a single JSON object and nothing else:
|
||||
|
||||
{
|
||||
"valid": true,
|
||||
"confidence": 0.95,
|
||||
"issues": [
|
||||
{
|
||||
"field": "total",
|
||||
"severity": "error",
|
||||
"message": "Extracted total 120.00 does not match image 122.00",
|
||||
"suggested_value": 122.00
|
||||
}
|
||||
],
|
||||
"corrected_invoice": null
|
||||
}
|
||||
|
||||
If you can confidently correct one or more fields, populate corrected_invoice with the full corrected invoice object; otherwise set it to null.
|
||||
confidence must be between 0.0 and 1.0.
|
||||
@@ -0,0 +1,111 @@
|
||||
---
|
||||
name: odoo-ocr-pipeline
|
||||
description: Use this skill when implementing the document classifier, OCR branches, structured extraction, or review stages of the odoo_ocr invoice processing pipeline.
|
||||
---
|
||||
|
||||
# Odoo OCR Pipeline Implementation Skill
|
||||
|
||||
Use this skill whenever you are asked to write or modify code for the invoice OCR pipeline in `odoo_ocr`.
|
||||
|
||||
## Pipeline Stages
|
||||
|
||||
The pipeline has these stages, each as a separate module under `src/odoo_ocr/pipeline/`:
|
||||
|
||||
1. **Loader** (`pipeline/loader.py`): converts PDFs and images to normalized `PIL.Image.Image` or `numpy` arrays. Handle multi-page PDFs by yielding one page at a time.
|
||||
2. **Classifier** (`pipeline/classifier.py`): determines the invoice type. Return a `DocumentClass` enum with values:
|
||||
- `digital_pdf` — native PDF with embedded text.
|
||||
- `scanned_print` — clean printed scan/photo.
|
||||
- `handwritten` — handwritten invoice.
|
||||
- `mixed_unknown` — ambiguous or low-quality input.
|
||||
Implementation should first try fast heuristics (file extension, text layer presence, image features), then fall back to a small VLM classifier if confidence is low.
|
||||
3. **Branch routers**:
|
||||
- `pipeline/digital_pdf.py`: extract text and layout with `pymupdf` / `pdfplumber`. Preserve bounding boxes for table reconstruction.
|
||||
- `pipeline/scanned_print.py`: preprocess image and call the OCR VLM.
|
||||
- `pipeline/handwritten.py`: preprocess image (binarization, slant correction, line segmentation if needed) and call GLM-OCR / handwriting OCR model.
|
||||
- `pipeline/mixed_unknown.py`: run multiple OCR backends and merge/confidence-rank outputs.
|
||||
4. **Extractor** (`pipeline/extractor.py`): take raw OCR output (text + image) and call a structured-extraction VLM to produce an `ExtractedInvoice` Pydantic object. Force JSON output only.
|
||||
5. **Reviewer** (`pipeline/reviewer.py`): call a review VLM with the original image and the extracted JSON. Return a `ReviewResult` with `valid`, `confidence`, and `issues`.
|
||||
|
||||
## Branch Routing Logic
|
||||
|
||||
```python
|
||||
# src/odoo_ocr/schemas/document.py
|
||||
from enum import Enum
|
||||
|
||||
class DocumentClass(str, Enum):
|
||||
digital_pdf = "digital_pdf"
|
||||
scanned_print = "scanned_print"
|
||||
handwritten = "handwritten"
|
||||
mixed_unknown = "mixed_unknown"
|
||||
```
|
||||
|
||||
Routing decision tree:
|
||||
|
||||
- If file is PDF and `pdfplumber` extracts meaningful text → `digital_pdf`.
|
||||
- Else if image and classifier confidence > 0.85:
|
||||
- handwriting features dominate → `handwritten`
|
||||
- otherwise → `scanned_print`
|
||||
- Else → `mixed_unknown`.
|
||||
|
||||
## Preprocessing Rules
|
||||
|
||||
For scanned/photo inputs:
|
||||
|
||||
1. Convert to grayscale.
|
||||
2. Resize to a standard DPI (300 DPI preferred; 200 DPI minimum).
|
||||
3. Deskew if skew angle > 0.5°.
|
||||
4. Apply adaptive contrast/clarity; avoid over-blurring.
|
||||
5. For handwriting branch, also:
|
||||
- Binarize with Otsu or Sauvola.
|
||||
- Detect and correct slant.
|
||||
- Optionally segment lines.
|
||||
|
||||
## Structured Extraction Output
|
||||
|
||||
The final extraction must always fit the `ExtractedInvoice` Pydantic schema defined in `src/odoo_ocr/schemas/invoice.py`. Required fields include:
|
||||
|
||||
- `vendor_name`
|
||||
- `vendor_address` (optional)
|
||||
- `invoice_number`
|
||||
- `invoice_date`
|
||||
- `due_date` (optional)
|
||||
- `currency` (default to settings)
|
||||
- `payment_terms` (optional)
|
||||
- `line_items`: list of `InvoiceLineItem` with `description`, `quantity`, `unit_price`, `total_price`, `tax_rate`.
|
||||
- `subtotal`
|
||||
- `tax_total`
|
||||
- `total`
|
||||
- `bank_details` / `iban` (optional)
|
||||
- `raw_ocr_text` (for debugging)
|
||||
|
||||
## Review Output
|
||||
|
||||
The review stage must return:
|
||||
|
||||
```python
|
||||
class ReviewIssue(BaseModel):
|
||||
field: str
|
||||
severity: Literal["error", "warning"]
|
||||
message: str
|
||||
suggested_value: Any | None
|
||||
|
||||
class ReviewResult(BaseModel):
|
||||
valid: bool
|
||||
confidence: float # 0.0–1.0
|
||||
issues: list[ReviewIssue]
|
||||
corrected_invoice: ExtractedInvoice | None
|
||||
```
|
||||
|
||||
Confidence thresholds (from config):
|
||||
|
||||
- `confidence >= 0.90`: accept and generate XML.
|
||||
- `0.75 <= confidence < 0.90`: generate XML with `human_review_required=true` flag.
|
||||
- `confidence < 0.75`: do not generate final XML; emit draft sidecar only and flag for human review.
|
||||
|
||||
## Implementation Guidelines
|
||||
|
||||
- Keep each stage stateless; pass a `ProcessingContext` Pydantic object that holds settings, original path, classified type, and intermediate results.
|
||||
- Every model call goes through the unified model client from `src/odoo_ocr/models/base_vlm.py`.
|
||||
- Log each stage with timing and input/output file hashes.
|
||||
- Add unit tests for each branch with small fixture images/PDFs; mock model calls by default.
|
||||
- Do not commit large model weights or binary fixtures.
|
||||
@@ -0,0 +1,16 @@
|
||||
You are a document classifier for an invoice OCR system. Given an image of a document, classify it into exactly one of these categories:
|
||||
|
||||
- digital_pdf: a native digital PDF or clean computer-generated invoice with embedded text.
|
||||
- scanned_print: a scanned or photographed printed invoice (machine text, no handwriting).
|
||||
- handwritten: a handwritten invoice or receipt.
|
||||
- mixed_unknown: ambiguous, damaged, or mixed-content document.
|
||||
|
||||
Respond with a single JSON object and nothing else. Use this exact schema:
|
||||
|
||||
{
|
||||
"category": "scanned_print",
|
||||
"confidence": 0.92,
|
||||
"reasoning": "brief one-sentence reason"
|
||||
}
|
||||
|
||||
confidence must be a float between 0.0 and 1.0.
|
||||
@@ -0,0 +1,111 @@
|
||||
---
|
||||
name: odoo-xml-import
|
||||
description: Use this skill when generating, validating, or modifying Odoo Enterprise XML import files from extracted invoice data in the odoo_ocr project.
|
||||
---
|
||||
|
||||
# Odoo XML Import Skill
|
||||
|
||||
Use this skill whenever you need to produce Odoo XML from the `ExtractedInvoice` schema.
|
||||
|
||||
## Target Format
|
||||
|
||||
Generate standard Odoo data-import XML:
|
||||
|
||||
```xml
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<odoo>
|
||||
<data>
|
||||
<record id="vendor_x" model="res.partner">
|
||||
<field name="name">...</field>
|
||||
<field name="supplier_rank">1</field>
|
||||
</record>
|
||||
|
||||
<record id="invoice_123" model="account.move">
|
||||
<field name="move_type">in_invoice</field>
|
||||
<field name="partner_id" ref="vendor_x"/>
|
||||
<field name="invoice_date">2024-05-01</field>
|
||||
<field name="date">2024-05-01</field>
|
||||
<field name="invoice_date_due">2024-06-01</field>
|
||||
<field name="ref">INV-123</field>
|
||||
<field name="currency_id" ref="base.USD"/>
|
||||
<field name="narration">...</field>
|
||||
</record>
|
||||
|
||||
<record id="invoice_123_line_1" model="account.move.line">
|
||||
<field name="move_id" ref="invoice_123"/>
|
||||
<field name="name">Product A</field>
|
||||
<field name="quantity">2</field>
|
||||
<field name="price_unit">100.00</field>
|
||||
<field name="tax_ids" eval="[(6, 0, [ref('tax_15')])]"/>
|
||||
</record>
|
||||
|
||||
<record id="tax_15" model="account.tax">
|
||||
<field name="name">VAT 15%</field>
|
||||
<field name="amount">15</field>
|
||||
<field name="amount_type">percent</field>
|
||||
<field name="type_tax_use">purchase</field>
|
||||
</record>
|
||||
</data>
|
||||
</odoo>
|
||||
```
|
||||
|
||||
See `templates/invoice_import.xml` for a full template.
|
||||
|
||||
## Required Records
|
||||
|
||||
Every XML file must create or reference:
|
||||
|
||||
1. **Vendor (`res.partner`)**
|
||||
- `name` (required)
|
||||
- `supplier_rank` = 1
|
||||
- optional: `vat`, `street`, `city`, `zip`, `country_id`
|
||||
|
||||
2. **Vendor Bill (`account.move`)**
|
||||
- `move_type` = `in_invoice`
|
||||
- `partner_id` ref to vendor record
|
||||
- `invoice_date` (required)
|
||||
- `date` (defaults to invoice_date)
|
||||
- `invoice_date_due` (optional)
|
||||
- `ref` = invoice number
|
||||
- `currency_id` (defaults from settings; map common ISO codes to `base.*` refs)
|
||||
- `narration` = summary / payment terms (optional)
|
||||
|
||||
3. **Invoice Lines (`account.move.line`)**
|
||||
- One record per `line_items` entry.
|
||||
- `move_id` ref to vendor bill.
|
||||
- `name` = line description.
|
||||
- `quantity`, `price_unit` (decimal strings).
|
||||
- `tax_ids` eval with refs to existing or newly created tax records.
|
||||
- Do not set `debit`/`credit` directly; Odoo will compute them from price_unit × quantity.
|
||||
|
||||
4. **Taxes (`account.tax`)**
|
||||
- Create only if the tax does not already exist.
|
||||
- Use deterministic external ID based on rate and type, e.g., `purchase_vat_15`.
|
||||
- `name`, `amount`, `amount_type='percent'`, `type_tax_use='purchase'`.
|
||||
|
||||
## External ID Rules
|
||||
|
||||
- Sanitize IDs: lowercase, replace non-alphanumeric with `_`, strip leading digits.
|
||||
- Vendor ID: `vendor_<sanitized_name>_<hash>` or use VAT if present.
|
||||
- Invoice ID: `invoice_<sanitized_number>_<date>` with hash if needed for uniqueness.
|
||||
- Tax ID: `purchase_vat_<rate>`.
|
||||
|
||||
## Validation Rules
|
||||
|
||||
1. Parse generated XML with `lxml.etree`.
|
||||
2. Ensure required fields are present and non-empty.
|
||||
3. Warn if `sum(line totals) + tax != total` beyond rounding tolerance (0.01).
|
||||
4. If `human_review_required` flag is set, add an XML comment or metadata record documenting this.
|
||||
|
||||
## Implementation Location
|
||||
|
||||
All XML logic lives in `src/odoo_ocr/pipeline/xml_builder.py` and `src/odoo_ocr/schemas/odoo_xml.py`.
|
||||
|
||||
## Currency Mapping
|
||||
|
||||
Maintain a hardcoded fallback map from ISO code to `base.<code>` for common currencies (EUR, USD, GBP, CHF). If a currency is not in the map, create a `res.currency` record or default to settings.
|
||||
|
||||
## Testing
|
||||
|
||||
- Test XML generation against a known-good sample in `tests/fixtures/expected_invoice.xml`.
|
||||
- Validate that generated XML is parseable and contains the required records.
|
||||
@@ -0,0 +1,42 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<odoo>
|
||||
<data noupdate="0">
|
||||
<!-- Vendor -->
|
||||
<record id="vendor_acme" model="res.partner">
|
||||
<field name="name">Acme Supplies</field>
|
||||
<field name="supplier_rank">1</field>
|
||||
<field name="street">123 Industrial Way</field>
|
||||
<field name="city">Springfield</field>
|
||||
<field name="zip">12345</field>
|
||||
</record>
|
||||
|
||||
<!-- Tax -->
|
||||
<record id="purchase_vat_20" model="account.tax">
|
||||
<field name="name">Purchase VAT 20%</field>
|
||||
<field name="amount">20</field>
|
||||
<field name="amount_type">percent</field>
|
||||
<field name="type_tax_use">purchase</field>
|
||||
</record>
|
||||
|
||||
<!-- Vendor Bill -->
|
||||
<record id="invoice_20240501_001" model="account.move">
|
||||
<field name="move_type">in_invoice</field>
|
||||
<field name="partner_id" ref="vendor_acme"/>
|
||||
<field name="invoice_date">2024-05-01</field>
|
||||
<field name="date">2024-05-01</field>
|
||||
<field name="invoice_date_due">2024-06-01</field>
|
||||
<field name="ref">INV-2024-001</field>
|
||||
<field name="currency_id" ref="base.USD"/>
|
||||
<field name="narration">Net 30</field>
|
||||
</record>
|
||||
|
||||
<!-- Invoice Lines -->
|
||||
<record id="invoice_20240501_001_line_1" model="account.move.line">
|
||||
<field name="move_id" ref="invoice_20240501_001"/>
|
||||
<field name="name">Consulting services</field>
|
||||
<field name="quantity">10.0</field>
|
||||
<field name="price_unit">100.00</field>
|
||||
<field name="tax_ids" eval="[(6, 0, [ref('purchase_vat_20')])]"/>
|
||||
</record>
|
||||
</data>
|
||||
</odoo>
|
||||
+58
@@ -0,0 +1,58 @@
|
||||
# Python
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*$py.class
|
||||
*.so
|
||||
.Python
|
||||
build/
|
||||
develop-eggs/
|
||||
dist/
|
||||
downloads/
|
||||
eggs/
|
||||
.eggs/
|
||||
lib/
|
||||
lib64/
|
||||
parts/
|
||||
sdist/
|
||||
var/
|
||||
wheels/
|
||||
share/python-wheels/
|
||||
*.egg-info/
|
||||
.installed.cfg
|
||||
*.egg
|
||||
MANIFEST
|
||||
|
||||
# Virtual environments
|
||||
.env
|
||||
.venv
|
||||
env/
|
||||
venv/
|
||||
ENV/
|
||||
env.bak/
|
||||
venv.bak/
|
||||
|
||||
# IDEs / editors
|
||||
.vscode/
|
||||
.idea/
|
||||
*.swp
|
||||
*.swo
|
||||
*~
|
||||
|
||||
# Mypy / pytest / coverage
|
||||
.mypy_cache/
|
||||
.dmypy.json
|
||||
dmypy.json
|
||||
.pytest_cache/
|
||||
.coverage
|
||||
htmlcov/
|
||||
.tox/
|
||||
.nox/
|
||||
|
||||
# Project-specific
|
||||
out/
|
||||
output/
|
||||
*.xml
|
||||
!tests/fixtures/*.xml
|
||||
!templates/*.xml
|
||||
.cache/
|
||||
~/.cache/
|
||||
@@ -0,0 +1,106 @@
|
||||
# AGENTS.md — Odoo OCR Project
|
||||
|
||||
This file provides context and rules for AI agents working on `odoo_ocr`, a local-first invoice OCR pipeline that reads scanned, printed, handwritten, and native-PDF invoices and produces Odoo Enterprise-ready XML.
|
||||
|
||||
## Project Goal
|
||||
|
||||
Build a Python application that:
|
||||
|
||||
1. Classifies incoming invoice files by type (native PDF, scanned print, handwritten, mixed image).
|
||||
2. Routes each file to a specialized processing branch.
|
||||
3. Runs OCR using local vision-language models (Ollama or llama.cpp).
|
||||
4. Extracts a structured invoice representation.
|
||||
5. Reviews extracted data against the original image using a second vision model.
|
||||
6. Emits valid Odoo XML for import into Odoo Enterprise.
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Language**: Python 3.11+
|
||||
- **Dependency management**: `pyproject.toml` (PEP 621); use `uv` or `pip`.
|
||||
- **Core libraries**:
|
||||
- `pydantic` v2 for all data schemas and settings.
|
||||
- `pymupdf` and `pdf2image` for PDF ingestion.
|
||||
- `Pillow` and `opencv-python-headless` for image preprocessing.
|
||||
- `httpx` for HTTP model clients.
|
||||
- `lxml` for XML generation and validation.
|
||||
- `pytest` and `pytest-asyncio` for tests.
|
||||
- **Local inference**:
|
||||
- Primary runtime: **Ollama** for fast iteration.
|
||||
- Optimized/runtime path: **llama.cpp server** (custom GGUF quants).
|
||||
- **Models**:
|
||||
- Document classifier: `Qwen2.5-VL-3B` or heuristics.
|
||||
- OCR / extraction: `Qwen2.5-VL-7B` or `GLM-OCR` (preferred for handwriting).
|
||||
- Review / validation: `Qwen2.5-VL-7B` or larger (`72B` if available).
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Input File
|
||||
→ Classifier (heuristic + small VLM)
|
||||
→ Branch: digital_pdf → text/layout extraction
|
||||
→ Branch: scanned_print → preprocess → VLM OCR
|
||||
→ Branch: handwritten → preprocess → GLM-OCR / handwriting OCR
|
||||
→ Branch: mixed_unknown → preprocess → ensemble OCR
|
||||
→ Structured Extraction VLM
|
||||
→ Review VLM (image vs extracted JSON)
|
||||
→ Confidence check
|
||||
→ XML Builder
|
||||
→ Odoo XML + sidecar JSON
|
||||
```
|
||||
|
||||
## Code Conventions
|
||||
|
||||
1. **Project layout**: keep application code under `src/odoo_ocr/`.
|
||||
2. **Schemas first**: define Pydantic models before writing business logic.
|
||||
3. **Type hints**: use `typing` everywhere; run `mypy` in strict mode where practical.
|
||||
4. **Error handling**: never swallow exceptions; return structured `Result` objects or raise domain exceptions.
|
||||
5. **Configuration**: use `pydantic-settings` with `config.yaml` and env var overrides.
|
||||
6. **Logging**: use Python standard `logging`; log every model call latency and token usage.
|
||||
7. **No hardcoded secrets**: model endpoints, credentials, and paths come from settings.
|
||||
8. **Tests**: every module must have tests under `tests/`. Use fixtures from `tests/fixtures/`.
|
||||
|
||||
## Model Client Rules
|
||||
|
||||
1. Support both Ollama and llama.cpp with a unified interface (`BaseVLMClient`).
|
||||
2. Always emit JSON from VLMs when doing extraction/review. Use constrained prompts, not regex scraping.
|
||||
3. Retry on transient failures with exponential backoff.
|
||||
4. Cache model responses by content hash to avoid re-running expensive inference during development.
|
||||
5. Record per-call metrics (model name, tokens, latency, prompt hash).
|
||||
|
||||
## Data Flow Rules
|
||||
|
||||
1. Every invoice must produce:
|
||||
- A Pydantic `ExtractedInvoice` object.
|
||||
- A review result (`ReviewResult`) with confidence score and issue list.
|
||||
- An Odoo XML file (unless blocked by low confidence).
|
||||
- A sidecar JSON file with metadata, timings, and confidence.
|
||||
2. If review confidence is below the configured threshold, mark the invoice for human review and do not generate final XML (or generate a draft with a warning flag).
|
||||
3. Never send invoice data outside the local model endpoints.
|
||||
|
||||
## Odoo XML Target
|
||||
|
||||
Generate Odoo data-import XML compatible with Odoo Enterprise vendor bills:
|
||||
|
||||
- `res.partner` (vendor)
|
||||
- `account.move` (vendor bill header)
|
||||
- `account.move.line` (invoice lines)
|
||||
- `account.tax` references by percentage/name
|
||||
|
||||
See skill `odoo-xml-import` for detailed field mapping and a sample XML template.
|
||||
|
||||
## When to Ask the User
|
||||
|
||||
Ask for clarification when:
|
||||
|
||||
- The requested change would alter the model stack.
|
||||
- The change affects the Odoo target schema or import method.
|
||||
- You are unsure whether a file should be committed or a dependency added.
|
||||
- A requested feature conflicts with the local-only / privacy constraint.
|
||||
|
||||
## Skills Reference
|
||||
|
||||
Agents should load the relevant project skills from `.agents/skills/`:
|
||||
|
||||
- `odoo-ocr-pipeline` — when implementing classifier, branches, OCR, extraction, or review.
|
||||
- `odoo-xml-import` — when generating or validating Odoo XML.
|
||||
- `local-vlm-client` — when writing model clients or prompts.
|
||||
@@ -0,0 +1,46 @@
|
||||
# odoo_ocr
|
||||
|
||||
Local-first invoice OCR pipeline that reads scanned, printed, handwritten, and native-PDF invoices and produces Odoo Enterprise-ready XML.
|
||||
|
||||
## Quick start
|
||||
|
||||
1. Install dependencies:
|
||||
```bash
|
||||
uv sync --all-extras
|
||||
# or
|
||||
pip install -e ".[dev]"
|
||||
```
|
||||
|
||||
2. Configure `config.yaml` or set environment variables:
|
||||
```bash
|
||||
export OLLAMA_BASE_URL="http://100.103.83.12:11435"
|
||||
```
|
||||
|
||||
3. Run the pipeline:
|
||||
```bash
|
||||
odoo-ocr process /path/to/invoices --output ./out/
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Input File
|
||||
→ Classifier (heuristic + small VLM)
|
||||
→ Branch: digital_pdf → text/layout extraction
|
||||
→ Branch: scanned_print → preprocess → VLM OCR
|
||||
→ Branch: handwritten → preprocess → GLM-OCR
|
||||
→ Branch: mixed_unknown → preprocess → ensemble OCR
|
||||
→ Structured Extraction VLM
|
||||
→ Review VLM (image vs extracted JSON)
|
||||
→ Confidence check
|
||||
→ XML Builder
|
||||
→ Odoo XML + sidecar JSON
|
||||
```
|
||||
|
||||
## Project-specific agent skills
|
||||
|
||||
Agent skills are in `.agents/skills/`:
|
||||
|
||||
- `odoo-ocr-pipeline` — classifier, branches, extraction, review.
|
||||
- `odoo-xml-import` — generating and validating Odoo XML.
|
||||
- `local-vlm-client` — Ollama/llama.cpp VLM clients and prompts.
|
||||
+31
@@ -0,0 +1,31 @@
|
||||
# Default configuration for odoo_ocr.
|
||||
# Override with environment variables (OLLAMA_BASE_URL, MODELS__OCR, etc.).
|
||||
|
||||
ollama_base_url: "http://localhost:11434"
|
||||
|
||||
models:
|
||||
classifier: "qwen2.5-vl:3b"
|
||||
ocr: "glm-ocr"
|
||||
extraction: "qwen2.5-vl:7b"
|
||||
review: "qwen2.5-vl:7b"
|
||||
|
||||
review:
|
||||
high_confidence_threshold: 0.90
|
||||
min_confidence_threshold: 0.75
|
||||
|
||||
xml:
|
||||
default_currency: "EUR"
|
||||
target: "vendor_bill"
|
||||
|
||||
preprocessing:
|
||||
target_dpi: 300
|
||||
max_image_dimension: 2048
|
||||
jpeg_quality: 85
|
||||
|
||||
cache:
|
||||
enabled: true
|
||||
dir: "~/.cache/odoo_ocr/llm_cache"
|
||||
|
||||
logging:
|
||||
level: "INFO"
|
||||
format: "%(asctime)s | %(levelname)s | %(name)s | %(message)s"
|
||||
@@ -0,0 +1,16 @@
|
||||
You are a document classifier for an invoice OCR system. Given an image of a document, classify it into exactly one of these categories:
|
||||
|
||||
- digital_pdf: a native digital PDF or clean computer-generated invoice with embedded text.
|
||||
- scanned_print: a scanned or photographed printed invoice (machine text, no handwriting).
|
||||
- handwritten: a handwritten invoice or receipt.
|
||||
- mixed_unknown: ambiguous, damaged, or mixed-content document.
|
||||
|
||||
Respond with a single JSON object and nothing else. Use this exact schema:
|
||||
|
||||
{
|
||||
"category": "scanned_print",
|
||||
"confidence": 0.92,
|
||||
"reasoning": "brief one-sentence reason"
|
||||
}
|
||||
|
||||
confidence must be a float between 0.0 and 1.0.
|
||||
@@ -0,0 +1,36 @@
|
||||
You are an invoice data extraction assistant. Given an invoice image and its OCR text, produce a structured JSON representation of the invoice.
|
||||
|
||||
Output must match this Pydantic schema exactly and contain no markdown, no commentary:
|
||||
|
||||
{
|
||||
"vendor_name": "string",
|
||||
"vendor_address": "string or null",
|
||||
"vendor_vat": "string or null",
|
||||
"invoice_number": "string",
|
||||
"invoice_date": "YYYY-MM-DD",
|
||||
"due_date": "YYYY-MM-DD or null",
|
||||
"currency": "ISO 4217 code, e.g. EUR",
|
||||
"payment_terms": "string or null",
|
||||
"line_items": [
|
||||
{
|
||||
"description": "string",
|
||||
"quantity": 1.0,
|
||||
"unit_price": 0.00,
|
||||
"total_price": 0.00,
|
||||
"tax_rate": 0.00
|
||||
}
|
||||
],
|
||||
"subtotal": 0.00,
|
||||
"tax_total": 0.00,
|
||||
"total": 0.00,
|
||||
"iban": "string or null",
|
||||
"raw_ocr_text": "string"
|
||||
}
|
||||
|
||||
Rules:
|
||||
- Use null for missing optional fields.
|
||||
- Dates must be ISO 8601.
|
||||
- All monetary values are decimal numbers (do not use strings).
|
||||
- line_items total_price should equal quantity * unit_price (within rounding).
|
||||
- subtotal + tax_total should equal total (within rounding).
|
||||
- If the image and OCR disagree, trust the image.
|
||||
@@ -0,0 +1,5 @@
|
||||
You are an OCR engine. Read all text from the provided invoice image accurately.
|
||||
Preserve line breaks and table structure as much as possible.
|
||||
If a value is unclear, mark it with [UNCLEAR].
|
||||
Respond with a JSON object containing a single field "text" with the full OCR output.
|
||||
Do not add markdown formatting or explanations.
|
||||
@@ -0,0 +1,25 @@
|
||||
You are a meticulous invoice review assistant. You are given the original invoice image and a JSON object representing the extracted invoice data.
|
||||
|
||||
Your task:
|
||||
1. Verify that every field in the JSON is supported by the image.
|
||||
2. Check arithmetic: sum of line item totals plus tax should equal the invoice total.
|
||||
3. Identify missing fields, incorrect values, or formatting problems.
|
||||
|
||||
Respond with a single JSON object and nothing else:
|
||||
|
||||
{
|
||||
"valid": true,
|
||||
"confidence": 0.95,
|
||||
"issues": [
|
||||
{
|
||||
"field": "total",
|
||||
"severity": "error",
|
||||
"message": "Extracted total 120.00 does not match image 122.00",
|
||||
"suggested_value": 122.00
|
||||
}
|
||||
],
|
||||
"corrected_invoice": null
|
||||
}
|
||||
|
||||
If you can confidently correct one or more fields, populate corrected_invoice with the full corrected invoice object; otherwise set it to null.
|
||||
confidence must be between 0.0 and 1.0.
|
||||
@@ -0,0 +1,55 @@
|
||||
[project]
|
||||
name = "odoo-ocr"
|
||||
version = "0.1.0"
|
||||
description = "Local-first invoice OCR pipeline that produces Odoo Enterprise XML."
|
||||
requires-python = ">=3.11"
|
||||
dependencies = [
|
||||
"pydantic>=2.0",
|
||||
"pydantic-settings>=2.0",
|
||||
"httpx>=0.27",
|
||||
"tenacity>=8.0",
|
||||
"pymupdf>=1.24",
|
||||
"pdf2image>=1.17",
|
||||
"Pillow>=10.0",
|
||||
"opencv-python-headless>=4.10",
|
||||
"lxml>=5.0",
|
||||
"pyyaml>=6.0",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
dev = [
|
||||
"pytest>=8.0",
|
||||
"pytest-asyncio>=0.23",
|
||||
"pytest-mock>=3.14",
|
||||
"mypy>=1.10",
|
||||
"ruff>=0.5",
|
||||
]
|
||||
|
||||
[project.scripts]
|
||||
odoo-ocr = "odoo_ocr.cli:main"
|
||||
|
||||
[build-system]
|
||||
requires = ["setuptools>=61.0", "wheel"]
|
||||
build-backend = "setuptools.build_meta"
|
||||
|
||||
[tool.setuptools.packages.find]
|
||||
where = ["src"]
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
testpaths = ["tests"]
|
||||
asyncio_mode = "auto"
|
||||
pythonpath = ["src"]
|
||||
|
||||
[tool.mypy]
|
||||
python_version = "3.11"
|
||||
strict = true
|
||||
warn_return_any = true
|
||||
warn_unused_ignores = true
|
||||
ignore_missing_imports = true
|
||||
|
||||
[tool.ruff.lint]
|
||||
select = ["E", "F", "I", "N", "W", "UP", "B", "C4", "SIM"]
|
||||
ignore = ["E501"]
|
||||
|
||||
[tool.ruff.format]
|
||||
quote-style = "double"
|
||||
Executable
+12
@@ -0,0 +1,12 @@
|
||||
#!/usr/bin/env bash
|
||||
# Convenience script to run the Ollama endpoint used by this project.
|
||||
# Override OLLAMA_HOST with the remote server if needed.
|
||||
|
||||
export OLLAMA_HOST="${OLLAMA_HOST:-http://100.103.83.12:11435}"
|
||||
|
||||
echo "Pulling models from ${OLLAMA_HOST}..."
|
||||
ollama pull glm-ocr
|
||||
ollama pull qwen2.5-vl:7b
|
||||
ollama pull qwen2.5-vl:3b
|
||||
|
||||
echo "Ollama is ready at ${OLLAMA_HOST}"
|
||||
@@ -0,0 +1,3 @@
|
||||
"""Local-first invoice OCR pipeline for Odoo Enterprise."""
|
||||
|
||||
__version__ = "0.1.0"
|
||||
@@ -0,0 +1,116 @@
|
||||
"""Command-line interface for odoo_ocr."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from odoo_ocr.clients.ollama import OllamaClient
|
||||
from odoo_ocr.config import Settings
|
||||
from odoo_ocr.pipeline.classifier import classify_document
|
||||
from odoo_ocr.pipeline.digital_pdf import process_digital_pdf
|
||||
from odoo_ocr.pipeline.extractor import extract_invoice
|
||||
from odoo_ocr.pipeline.handwritten import process_handwritten
|
||||
from odoo_ocr.pipeline.loader import load_document
|
||||
from odoo_ocr.pipeline.mixed_unknown import process_mixed_unknown
|
||||
from odoo_ocr.pipeline.reviewer import review_invoice
|
||||
from odoo_ocr.pipeline.scanned_print import process_scanned_print
|
||||
from odoo_ocr.pipeline.xml_builder import build_odoo_xml
|
||||
from odoo_ocr.schemas import ProcessingContext
|
||||
from odoo_ocr.schemas.document import DocumentClass
|
||||
from odoo_ocr.utils.logging_config import configure_logging
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
async def process_single(path: Path, output_dir: Path, settings: Settings) -> dict[str, Any]:
|
||||
"""Process one invoice file end-to-end."""
|
||||
client = OllamaClient(settings)
|
||||
context = ProcessingContext(source_path=path, output_dir=output_dir)
|
||||
|
||||
doc = load_document(path, settings)
|
||||
pages = doc["pages"]
|
||||
|
||||
classification = await classify_document(path, pages, client, settings)
|
||||
context.classification = classification
|
||||
|
||||
if classification.category == DocumentClass.digital_pdf:
|
||||
raw_text = await process_digital_pdf(path, pages, client, settings)
|
||||
elif classification.category == DocumentClass.scanned_print:
|
||||
raw_text = await process_scanned_print(path, pages, client, settings)
|
||||
elif classification.category == DocumentClass.handwritten:
|
||||
raw_text = await process_handwritten(path, pages, client, settings)
|
||||
else:
|
||||
raw_text = await process_mixed_unknown(path, pages, client, settings)
|
||||
context.raw_text = raw_text
|
||||
|
||||
extracted = await extract_invoice(raw_text, pages, client, settings)
|
||||
context.extracted_invoice = extracted.model_dump(mode="json")
|
||||
|
||||
review = await review_invoice(extracted, pages, client, settings)
|
||||
context.review_result = review.model_dump(mode="json")
|
||||
|
||||
sidecar = {
|
||||
"source_path": str(path),
|
||||
"classification": classification.model_dump(),
|
||||
"extracted_invoice": extracted.model_dump(mode="json"),
|
||||
"review": review.model_dump(mode="json"),
|
||||
}
|
||||
|
||||
out_stem = output_dir / path.stem
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
with open(f"{out_stem}_sidecar.json", "w", encoding="utf-8") as f:
|
||||
json.dump(sidecar, f, indent=2, ensure_ascii=False)
|
||||
|
||||
if review.confidence >= settings.review.min_confidence_threshold:
|
||||
xml = build_odoo_xml(extracted, review, settings)
|
||||
with open(f"{out_stem}.xml", "w", encoding="utf-8") as f:
|
||||
f.write(xml)
|
||||
logger.info("Wrote XML: %s", f"{out_stem}.xml")
|
||||
else:
|
||||
logger.warning(
|
||||
"Confidence %.2f below threshold %.2f; skipping XML generation.",
|
||||
review.confidence,
|
||||
settings.review.min_confidence_threshold,
|
||||
)
|
||||
|
||||
return sidecar
|
||||
|
||||
|
||||
async def process_batch(input_dir: Path, output_dir: Path, settings: Settings) -> list[dict[str, Any]]:
|
||||
"""Process all supported files in a directory."""
|
||||
results: list[dict[str, Any]] = []
|
||||
for path in sorted(input_dir.iterdir()):
|
||||
if path.suffix.lower() in {".pdf", ".png", ".jpg", ".jpeg", ".tiff", ".tif", ".bmp", ".webp"}:
|
||||
try:
|
||||
result = await process_single(path, output_dir, settings)
|
||||
results.append(result)
|
||||
except Exception:
|
||||
logger.exception("Failed to process %s", path)
|
||||
return results
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Odoo OCR invoice pipeline")
|
||||
parser.add_argument("command", choices=["process"], help="Command to run")
|
||||
parser.add_argument("input", type=Path, help="File or directory to process")
|
||||
parser.add_argument("--output", "-o", type=Path, default=Path("./out"), help="Output directory")
|
||||
parser.add_argument("--config", "-c", type=Path, default=Path("config.yaml"), help="Config file")
|
||||
args = parser.parse_args()
|
||||
|
||||
settings = Settings.from_yaml(args.config)
|
||||
configure_logging(settings)
|
||||
|
||||
if args.input.is_dir():
|
||||
asyncio.run(process_batch(args.input, args.output, settings))
|
||||
else:
|
||||
asyncio.run(process_single(args.input, args.output, settings))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,6 @@
|
||||
"""VLM clients for local inference backends."""
|
||||
|
||||
from .base import BaseVLMClient
|
||||
from .ollama import OllamaClient
|
||||
|
||||
__all__ = ["BaseVLMClient", "OllamaClient"]
|
||||
@@ -0,0 +1,60 @@
|
||||
"""Abstract base class for local VLM clients."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from abc import ABC, abstractmethod
|
||||
from pathlib import Path
|
||||
from typing import Any, TypeVar
|
||||
|
||||
from PIL import Image
|
||||
from pydantic import BaseModel, ValidationError
|
||||
|
||||
from odoo_ocr.schemas import VLMResponse
|
||||
|
||||
T = TypeVar("T", bound=BaseModel)
|
||||
|
||||
class BaseVLMClient(ABC):
|
||||
"""Unified interface for Ollama and llama.cpp server VLMs."""
|
||||
|
||||
def __init__(self, settings: Any) -> None:
|
||||
"""Store settings; concrete clients validate required keys."""
|
||||
self.settings = settings
|
||||
|
||||
@abstractmethod
|
||||
async def complete(
|
||||
self,
|
||||
system_prompt: str,
|
||||
user_prompt: str,
|
||||
images: list[Image.Image],
|
||||
response_format: type[T] | None = None,
|
||||
temperature: float = 0.1,
|
||||
max_tokens: int = 4096,
|
||||
model: str | None = None,
|
||||
) -> VLMResponse:
|
||||
"""Send a chat request with optional images and return parsed text."""
|
||||
...
|
||||
|
||||
@staticmethod
|
||||
def parse_json(content: str, model: type[T]) -> T:
|
||||
"""Strip markdown fences and parse/validate JSON against a Pydantic model."""
|
||||
text = content.strip()
|
||||
if text.startswith("```"):
|
||||
text = text.strip("`").strip()
|
||||
if text.lower().startswith("json"):
|
||||
text = text[4:].strip()
|
||||
try:
|
||||
data = json.loads(text)
|
||||
except json.JSONDecodeError as exc:
|
||||
raise ValueError(f"Response is not valid JSON: {exc}") from exc
|
||||
try:
|
||||
return model.model_validate(data)
|
||||
except ValidationError as exc:
|
||||
raise ValueError(f"Response does not match schema: {exc}") from exc
|
||||
|
||||
@staticmethod
|
||||
def _load_prompt_file(path: Path | str) -> str:
|
||||
"""Load a plain-text prompt from the prompts directory."""
|
||||
path = Path(path)
|
||||
with path.open("r", encoding="utf-8") as f:
|
||||
return f.read()
|
||||
@@ -0,0 +1,35 @@
|
||||
"""llama.cpp server VLM client (stub; implement when needed)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from PIL import Image
|
||||
from pydantic import BaseModel
|
||||
|
||||
from odoo_ocr.clients.base import BaseVLMClient
|
||||
from odoo_ocr.config import Settings
|
||||
from odoo_ocr.schemas import VLMResponse
|
||||
|
||||
|
||||
class LlamaCppClient(BaseVLMClient):
|
||||
"""Client for llama.cpp server's OpenAI-compatible /v1/chat/completions endpoint."""
|
||||
|
||||
def __init__(self, settings: Settings) -> None:
|
||||
super().__init__(settings)
|
||||
raise NotImplementedError(
|
||||
"llama.cpp client is not yet implemented. Use OllamaClient for now."
|
||||
)
|
||||
|
||||
async def complete(
|
||||
self,
|
||||
system_prompt: str,
|
||||
user_prompt: str,
|
||||
images: list[Image.Image],
|
||||
response_format: type[BaseModel] | None = None,
|
||||
temperature: float = 0.1,
|
||||
max_tokens: int = 4096,
|
||||
model: str | None = None,
|
||||
) -> VLMResponse:
|
||||
"""Call llama.cpp server and return a normalized VLMResponse."""
|
||||
raise NotImplementedError(
|
||||
"llama.cpp client is not yet implemented. Use OllamaClient for now."
|
||||
)
|
||||
@@ -0,0 +1,140 @@
|
||||
"""Ollama-backed VLM client."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import io
|
||||
import logging
|
||||
import time
|
||||
from typing import Any
|
||||
|
||||
import httpx
|
||||
from PIL import Image
|
||||
from pydantic import BaseModel
|
||||
from tenacity import (
|
||||
retry,
|
||||
retry_if_exception_type,
|
||||
stop_after_attempt,
|
||||
wait_exponential,
|
||||
)
|
||||
|
||||
from odoo_ocr.clients.base import BaseVLMClient
|
||||
from odoo_ocr.config import Settings
|
||||
from odoo_ocr.schemas import VLMResponse
|
||||
from odoo_ocr.utils import cache
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class OllamaClient(BaseVLMClient):
|
||||
"""Client for Ollama's /api/chat endpoint with vision support."""
|
||||
|
||||
def __init__(self, settings: Settings) -> None:
|
||||
super().__init__(settings)
|
||||
self.base_url = settings.ollama_base_url.rstrip("/")
|
||||
self.timeout = httpx.Timeout(120.0, connect=10.0)
|
||||
self.http = httpx.AsyncClient(timeout=self.timeout)
|
||||
|
||||
@staticmethod
|
||||
def _encode_image(image: Image.Image, quality: int = 85) -> str:
|
||||
"""Convert a PIL image to a base64-encoded PNG or JPEG string."""
|
||||
rgb_image = image.convert("RGB")
|
||||
buffer = io.BytesIO()
|
||||
# Prefer PNG for low-color/binarized images, JPEG for photos.
|
||||
rgb_image.save(buffer, format="PNG")
|
||||
return base64.b64encode(buffer.getvalue()).decode("ascii")
|
||||
|
||||
async def complete(
|
||||
self,
|
||||
system_prompt: str,
|
||||
user_prompt: str,
|
||||
images: list[Image.Image],
|
||||
response_format: type[BaseModel] | None = None,
|
||||
temperature: float = 0.1,
|
||||
max_tokens: int = 4096,
|
||||
model: str | None = None,
|
||||
) -> VLMResponse:
|
||||
"""Call Ollama and return a normalized VLMResponse.
|
||||
|
||||
Uses response cache when enabled. JSON schema is passed to Ollama's
|
||||
`format` field when a Pydantic model is supplied.
|
||||
"""
|
||||
model_name = model or self.settings.models.ocr
|
||||
encoded_images = [self._encode_image(img) for img in images]
|
||||
image_blobs = [base64.b64decode(enc) for enc in encoded_images]
|
||||
|
||||
cached = cache.get_cached(
|
||||
system_prompt,
|
||||
user_prompt,
|
||||
image_blobs,
|
||||
model_name,
|
||||
temperature,
|
||||
self.settings,
|
||||
)
|
||||
if cached:
|
||||
return VLMResponse.model_validate(cached)
|
||||
|
||||
body: dict[str, Any] = {
|
||||
"model": model_name,
|
||||
"messages": [
|
||||
{"role": "system", "content": system_prompt},
|
||||
{
|
||||
"role": "user",
|
||||
"content": user_prompt,
|
||||
"images": encoded_images,
|
||||
},
|
||||
],
|
||||
"stream": False,
|
||||
"options": {
|
||||
"temperature": temperature,
|
||||
"num_predict": max_tokens,
|
||||
},
|
||||
}
|
||||
if response_format is not None:
|
||||
try:
|
||||
body["format"] = response_format.model_json_schema()
|
||||
except Exception:
|
||||
body["format"] = "json"
|
||||
|
||||
start = time.perf_counter()
|
||||
response_data = await self._post_chat(body)
|
||||
duration_ms = int((time.perf_counter() - start) * 1000)
|
||||
|
||||
content = response_data.get("message", {}).get("content", "")
|
||||
metrics = response_data.get("metrics", {})
|
||||
vlm_response = VLMResponse(
|
||||
content=content,
|
||||
model=response_data.get("model", self.settings.models.ocr),
|
||||
prompt_tokens=response_data.get("prompt_eval_count")
|
||||
or metrics.get("prompt_eval_count"),
|
||||
completion_tokens=response_data.get("eval_count")
|
||||
or metrics.get("eval_count"),
|
||||
duration_ms=duration_ms,
|
||||
)
|
||||
|
||||
cache.set_cached(
|
||||
system_prompt,
|
||||
user_prompt,
|
||||
image_blobs,
|
||||
model_name,
|
||||
temperature,
|
||||
vlm_response.model_dump(mode="json"),
|
||||
self.settings,
|
||||
)
|
||||
return vlm_response
|
||||
|
||||
@retry(
|
||||
stop=stop_after_attempt(3),
|
||||
wait=wait_exponential(multiplier=1, min=1, max=10),
|
||||
retry=retry_if_exception_type((httpx.NetworkError, httpx.TimeoutException)),
|
||||
)
|
||||
async def _post_chat(self, body: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Send the chat request and validate the response."""
|
||||
url = f"{self.base_url}/api/chat"
|
||||
logger.info("Ollama request to %s with model %s", url, body.get("model"))
|
||||
resp = await self.http.post(url, json=body)
|
||||
resp.raise_for_status()
|
||||
data: dict[str, Any] = resp.json()
|
||||
if not data.get("done"):
|
||||
raise RuntimeError(f"Ollama response not finished: {data}")
|
||||
return data
|
||||
@@ -0,0 +1,96 @@
|
||||
"""Application settings loaded from config.yaml and environment variables."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import yaml
|
||||
from pydantic import Field
|
||||
from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||
|
||||
|
||||
class ModelConfig(BaseSettings):
|
||||
"""Model names used by the pipeline."""
|
||||
|
||||
classifier: str = "qwen2.5-vl:3b"
|
||||
ocr: str = "glm-ocr"
|
||||
extraction: str = "qwen2.5-vl:7b"
|
||||
review: str = "qwen2.5-vl:7b"
|
||||
|
||||
|
||||
class ReviewConfig(BaseSettings):
|
||||
"""Review stage thresholds."""
|
||||
|
||||
high_confidence_threshold: float = 0.90
|
||||
min_confidence_threshold: float = 0.75
|
||||
|
||||
|
||||
class XmlConfig(BaseSettings):
|
||||
"""XML builder defaults."""
|
||||
|
||||
default_currency: str = "EUR"
|
||||
target: str = "vendor_bill"
|
||||
|
||||
|
||||
class PreprocessingConfig(BaseSettings):
|
||||
"""Image preprocessing parameters."""
|
||||
|
||||
target_dpi: int = 300
|
||||
max_image_dimension: int = 2048
|
||||
jpeg_quality: int = 85
|
||||
|
||||
|
||||
class CacheConfig(BaseSettings):
|
||||
"""LLM response cache settings."""
|
||||
|
||||
enabled: bool = True
|
||||
dir: Path = Path.home() / ".cache" / "odoo_ocr" / "llm_cache"
|
||||
|
||||
|
||||
class LoggingConfig(BaseSettings):
|
||||
"""Logging settings."""
|
||||
|
||||
level: str = "INFO"
|
||||
format: str = "%(asctime)s | %(levelname)s | %(name)s | %(message)s"
|
||||
|
||||
|
||||
class Settings(BaseSettings):
|
||||
"""Combined application settings.
|
||||
|
||||
Values are loaded from `config.yaml` in the project root, then overridden
|
||||
by environment variables. Env var names are uppercase with double
|
||||
underscores for nested fields, e.g. `MODELS__OCR=glm-ocr`.
|
||||
"""
|
||||
|
||||
model_config = SettingsConfigDict(
|
||||
env_file=".env",
|
||||
env_file_encoding="utf-8",
|
||||
env_nested_delimiter="__",
|
||||
extra="ignore",
|
||||
)
|
||||
|
||||
ollama_base_url: str = "http://localhost:11434"
|
||||
models: ModelConfig = Field(default_factory=ModelConfig)
|
||||
review: ReviewConfig = Field(default_factory=ReviewConfig)
|
||||
xml: XmlConfig = Field(default_factory=XmlConfig)
|
||||
preprocessing: PreprocessingConfig = Field(default_factory=PreprocessingConfig)
|
||||
cache: CacheConfig = Field(default_factory=CacheConfig)
|
||||
logging: LoggingConfig = Field(default_factory=LoggingConfig)
|
||||
|
||||
@classmethod
|
||||
def from_yaml(cls, path: Path | str = "config.yaml") -> Settings:
|
||||
"""Load settings from a YAML file, then apply env overrides."""
|
||||
path = Path(path)
|
||||
data: dict[str, Any] = {}
|
||||
if path.exists():
|
||||
with path.open("r", encoding="utf-8") as f:
|
||||
loaded = yaml.safe_load(f)
|
||||
if isinstance(loaded, dict):
|
||||
data = loaded
|
||||
return cls(**data)
|
||||
|
||||
|
||||
def get_settings() -> Settings:
|
||||
"""Return loaded settings (no caching to allow env reloads in tests)."""
|
||||
return Settings.from_yaml()
|
||||
@@ -0,0 +1,15 @@
|
||||
"""Invoice processing pipeline stages."""
|
||||
|
||||
from .classifier import classify_document
|
||||
from .extractor import extract_invoice
|
||||
from .loader import load_document
|
||||
from .reviewer import review_invoice
|
||||
from .xml_builder import build_odoo_xml
|
||||
|
||||
__all__ = [
|
||||
"classify_document",
|
||||
"extract_invoice",
|
||||
"load_document",
|
||||
"review_invoice",
|
||||
"build_odoo_xml",
|
||||
]
|
||||
@@ -0,0 +1,112 @@
|
||||
"""Document classification: heuristic + optional small VLM."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from pathlib import Path
|
||||
|
||||
import fitz # pymupdf
|
||||
from PIL import Image, ImageStat
|
||||
|
||||
from odoo_ocr.clients import BaseVLMClient
|
||||
from odoo_ocr.config import Settings
|
||||
from odoo_ocr.schemas.document import ClassificationResult, DocumentClass
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _has_embedded_text(path: Path) -> bool:
|
||||
"""Fast heuristic: does the PDF contain selectable text?"""
|
||||
try:
|
||||
with fitz.open(path) as doc:
|
||||
for page in doc:
|
||||
text = page.get_text().strip()
|
||||
if text and len(text) > 20:
|
||||
return True
|
||||
return False
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def _detect_handwriting(image: Image.Image) -> float:
|
||||
"""Placeholder handwriting detection score based on simple image stats.
|
||||
|
||||
A real implementation would use stroke-thickness variance or a dedicated
|
||||
classifier. Returns 0.0 (no handwriting) to 1.0 (definitely handwritten).
|
||||
"""
|
||||
# Minimal heuristic: high grayscale variance often indicates handwriting.
|
||||
gray = image.convert("L")
|
||||
stat = ImageStat.Stat(gray)
|
||||
std = stat.stddev[0]
|
||||
score = min(std / 64.0, 1.0) # rough normalization
|
||||
return round(score, 2)
|
||||
|
||||
|
||||
def _heuristic_classify(
|
||||
path: Path, pages: list[Image.Image]
|
||||
) -> ClassificationResult | None:
|
||||
"""Return a classification if heuristics are confident, otherwise None."""
|
||||
suffix = path.suffix.lower()
|
||||
|
||||
if suffix == ".pdf" and _has_embedded_text(path):
|
||||
return ClassificationResult(
|
||||
category=DocumentClass.digital_pdf,
|
||||
confidence=0.95,
|
||||
reasoning="PDF contains embedded text layer.",
|
||||
)
|
||||
|
||||
if pages:
|
||||
hw_score = _detect_handwriting(pages[0])
|
||||
if hw_score > 0.75:
|
||||
return ClassificationResult(
|
||||
category=DocumentClass.handwritten,
|
||||
confidence=hw_score,
|
||||
reasoning="Image exhibits handwriting-like stroke variance.",
|
||||
)
|
||||
if hw_score < 0.25:
|
||||
return ClassificationResult(
|
||||
category=DocumentClass.scanned_print,
|
||||
confidence=1.0 - hw_score,
|
||||
reasoning="Image appears to be printed text.",
|
||||
)
|
||||
|
||||
return None
|
||||
|
||||
|
||||
async def classify_document(
|
||||
path: Path,
|
||||
pages: list[Image.Image],
|
||||
client: BaseVLMClient,
|
||||
settings: Settings,
|
||||
) -> ClassificationResult:
|
||||
"""Classify an invoice document.
|
||||
|
||||
First tries fast heuristics. If confidence is low, asks the configured
|
||||
classifier VLM.
|
||||
"""
|
||||
heuristic = _heuristic_classify(path, pages)
|
||||
if heuristic and heuristic.confidence >= 0.85:
|
||||
logger.info("Heuristic classifier: %s", heuristic.category)
|
||||
return heuristic
|
||||
|
||||
system_prompt = _load_prompt("classifier_system.txt")
|
||||
user_prompt = (
|
||||
"Classify this invoice document. Return only the requested JSON object."
|
||||
)
|
||||
response = await client.complete(
|
||||
system_prompt=system_prompt,
|
||||
user_prompt=user_prompt,
|
||||
images=[pages[0]],
|
||||
response_format=ClassificationResult,
|
||||
temperature=0.1,
|
||||
model=settings.models.classifier,
|
||||
)
|
||||
result = client.parse_json(response.content, ClassificationResult)
|
||||
logger.info("VLM classifier: %s (confidence %.2f)", result.category, result.confidence)
|
||||
return result
|
||||
|
||||
|
||||
def _load_prompt(name: str) -> str:
|
||||
from odoo_ocr.clients.base import BaseVLMClient
|
||||
|
||||
return BaseVLMClient._load_prompt_file(Path("prompts") / name)
|
||||
@@ -0,0 +1,34 @@
|
||||
"""Processing branch for native digital PDFs."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from pathlib import Path
|
||||
|
||||
import fitz # pymupdf
|
||||
from PIL import Image
|
||||
|
||||
from odoo_ocr.clients import BaseVLMClient
|
||||
from odoo_ocr.config import Settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
async def process_digital_pdf(
|
||||
path: Path,
|
||||
pages: list[Image.Image],
|
||||
client: BaseVLMClient,
|
||||
settings: Settings,
|
||||
) -> str:
|
||||
"""Extract text from a native PDF, preserving layout where possible.
|
||||
|
||||
Returns the combined raw text of all pages.
|
||||
"""
|
||||
parts: list[str] = []
|
||||
with fitz.open(path) as doc:
|
||||
for i, page in enumerate(doc):
|
||||
text = page.get_text("text").strip()
|
||||
parts.append(f"--- Page {i + 1} ---\n{text}")
|
||||
full_text = "\n\n".join(parts)
|
||||
logger.info("Extracted %d characters from digital PDF %s", len(full_text), path)
|
||||
return full_text
|
||||
@@ -0,0 +1,52 @@
|
||||
"""Structured invoice extraction from raw OCR text + image."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from pathlib import Path
|
||||
|
||||
from PIL import Image
|
||||
|
||||
from odoo_ocr.clients import BaseVLMClient
|
||||
from odoo_ocr.config import Settings
|
||||
from odoo_ocr.schemas import ExtractedInvoice
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _load_prompt(name: str) -> str:
|
||||
from odoo_ocr.clients.base import BaseVLMClient
|
||||
|
||||
return BaseVLMClient._load_prompt_file(Path("prompts") / name)
|
||||
|
||||
|
||||
async def extract_invoice(
|
||||
raw_text: str,
|
||||
pages: list[Image.Image],
|
||||
client: BaseVLMClient,
|
||||
settings: Settings,
|
||||
) -> ExtractedInvoice:
|
||||
"""Convert raw OCR text into a validated ExtractedInvoice."""
|
||||
system_prompt = _load_prompt("extraction_system.txt")
|
||||
user_prompt = (
|
||||
f"OCR text extracted from the invoice:\n```\n{raw_text}\n```\n\n"
|
||||
"Now extract the structured invoice fields from the image and OCR text. "
|
||||
f"Default currency is {settings.xml.default_currency} unless otherwise stated."
|
||||
)
|
||||
|
||||
response = await client.complete(
|
||||
system_prompt=system_prompt,
|
||||
user_prompt=user_prompt,
|
||||
images=[pages[0]] if pages else [],
|
||||
response_format=ExtractedInvoice,
|
||||
temperature=0.1,
|
||||
model=settings.models.extraction,
|
||||
)
|
||||
extracted = client.parse_json(response.content, ExtractedInvoice)
|
||||
logger.info(
|
||||
"Extracted invoice %s from %s with %d line(s)",
|
||||
extracted.invoice_number,
|
||||
extracted.vendor_name,
|
||||
len(extracted.line_items),
|
||||
)
|
||||
return extracted
|
||||
@@ -0,0 +1,73 @@
|
||||
"""Processing branch for handwritten invoices."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from pathlib import Path
|
||||
|
||||
import cv2
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
from odoo_ocr.clients import BaseVLMClient
|
||||
from odoo_ocr.config import Settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _preprocess_handwritten(image: Image.Image) -> Image.Image:
|
||||
"""Apply handwriting-specific preprocessing.
|
||||
|
||||
Steps: grayscale, Otsu binarize, slight dilation to connect strokes.
|
||||
"""
|
||||
gray = image.convert("L")
|
||||
arr = np.array(gray)
|
||||
_, binary = cv2.threshold(arr, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
|
||||
kernel = np.ones((2, 2), np.uint8)
|
||||
processed = cv2.morphologyEx(binary, cv2.MORPH_CLOSE, kernel)
|
||||
return Image.fromarray(processed, mode="L").convert("RGB")
|
||||
|
||||
|
||||
def _load_prompt(name: str) -> str:
|
||||
from pathlib import Path
|
||||
|
||||
from odoo_ocr.clients.base import BaseVLMClient
|
||||
|
||||
return BaseVLMClient._load_prompt_file(Path("prompts") / name)
|
||||
|
||||
|
||||
async def process_handwritten(
|
||||
path: Path,
|
||||
pages: list[Image.Image],
|
||||
client: BaseVLMClient,
|
||||
settings: Settings,
|
||||
) -> str:
|
||||
"""Run OCR on handwritten invoice pages using the configured handwriting model."""
|
||||
system_prompt = _load_prompt("ocr_system.txt")
|
||||
user_prompt = (
|
||||
"Read all text from this handwritten invoice accurately. "
|
||||
"Preserve line breaks and mark unclear words with [UNCLEAR]."
|
||||
)
|
||||
|
||||
page_texts: list[str] = []
|
||||
for i, page in enumerate(pages):
|
||||
preprocessed = _preprocess_handwritten(page)
|
||||
response = await client.complete(
|
||||
system_prompt=system_prompt,
|
||||
user_prompt=f"{user_prompt} (page {i + 1} of {len(pages)})",
|
||||
images=[preprocessed],
|
||||
temperature=0.1,
|
||||
model=settings.models.ocr,
|
||||
)
|
||||
import json
|
||||
|
||||
try:
|
||||
parsed = json.loads(response.content.strip().strip("`").removeprefix("json").strip())
|
||||
text = parsed.get("text", response.content)
|
||||
except Exception:
|
||||
text = response.content
|
||||
page_texts.append(f"--- Page {i + 1} ---\n{text.strip()}")
|
||||
|
||||
full_text = "\n\n".join(page_texts)
|
||||
logger.info("Handwritten OCR extracted %d characters from %s", len(full_text), path)
|
||||
return full_text
|
||||
@@ -0,0 +1,68 @@
|
||||
"""Load PDFs and images into normalized page images."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import fitz # pymupdf
|
||||
from PIL import Image
|
||||
|
||||
from odoo_ocr.config import Settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
SUPPORTED_IMAGE_SUFFIXES = {".png", ".jpg", ".jpeg", ".tiff", ".tif", ".bmp", ".webp"}
|
||||
|
||||
|
||||
def _is_pdf(path: Path) -> bool:
|
||||
return path.suffix.lower() == ".pdf"
|
||||
|
||||
|
||||
def _pdf_to_images(path: Path, dpi: int = 300) -> list[Image.Image]:
|
||||
"""Render PDF pages to PIL images using PyMuPDF."""
|
||||
pages: list[Image.Image] = []
|
||||
with fitz.open(path) as doc:
|
||||
for page in doc:
|
||||
matrix = fitz.Matrix(dpi / 72, dpi / 72)
|
||||
pix = page.get_pixmap(matrix=matrix)
|
||||
mode = "RGB" if pix.n == 3 else "RGBA"
|
||||
img = Image.frombytes(mode, (pix.width, pix.height), pix.samples)
|
||||
pages.append(img.convert("RGB"))
|
||||
logger.info("Loaded %d page(s) from PDF %s", len(pages), path)
|
||||
return pages
|
||||
|
||||
|
||||
def _image_to_pil(path: Path) -> list[Image.Image]:
|
||||
img = Image.open(path)
|
||||
return [img.convert("RGB")]
|
||||
|
||||
|
||||
def load_document(path: Path | str, settings: Settings | None = None) -> dict[str, Any]:
|
||||
"""Load a document file and return a dict with pages and metadata.
|
||||
|
||||
Returns:
|
||||
{
|
||||
"pages": [PIL.Image.Image],
|
||||
"page_count": int,
|
||||
"source_path": Path,
|
||||
}
|
||||
"""
|
||||
path = Path(path)
|
||||
if not path.exists():
|
||||
raise FileNotFoundError(f"Document not found: {path}")
|
||||
|
||||
if _is_pdf(path):
|
||||
dpi = (settings.preprocessing.target_dpi if settings else 300)
|
||||
pages = _pdf_to_images(path, dpi=dpi)
|
||||
elif path.suffix.lower() in SUPPORTED_IMAGE_SUFFIXES:
|
||||
pages = _image_to_pil(path)
|
||||
else:
|
||||
raise ValueError(f"Unsupported file type: {path.suffix}")
|
||||
|
||||
return {
|
||||
"pages": pages,
|
||||
"page_count": len(pages),
|
||||
"source_path": path,
|
||||
}
|
||||
@@ -0,0 +1,61 @@
|
||||
"""Processing branch for ambiguous or low-quality invoice images."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from pathlib import Path
|
||||
|
||||
from PIL import Image
|
||||
|
||||
from odoo_ocr.clients import BaseVLMClient
|
||||
from odoo_ocr.config import Settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _load_prompt(name: str) -> str:
|
||||
from pathlib import Path
|
||||
|
||||
from odoo_ocr.clients.base import BaseVLMClient
|
||||
|
||||
return BaseVLMClient._load_prompt_file(Path("prompts") / name)
|
||||
|
||||
|
||||
async def process_mixed_unknown(
|
||||
path: Path,
|
||||
pages: list[Image.Image],
|
||||
client: BaseVLMClient,
|
||||
settings: Settings,
|
||||
) -> str:
|
||||
"""Robust OCR for mixed/unknown documents.
|
||||
|
||||
For now this runs the same OCR path as scanned_print but with a more
|
||||
permissive prompt. Future: ensemble multiple models and merge outputs.
|
||||
"""
|
||||
system_prompt = _load_prompt("ocr_system.txt")
|
||||
user_prompt = (
|
||||
"Extract all readable text from this document. It may contain a mix of "
|
||||
"printed text, handwriting, stamps, or low-quality scans. Preserve layout."
|
||||
)
|
||||
|
||||
page_texts: list[str] = []
|
||||
for i, page in enumerate(pages):
|
||||
response = await client.complete(
|
||||
system_prompt=system_prompt,
|
||||
user_prompt=f"{user_prompt} (page {i + 1} of {len(pages)})",
|
||||
images=[page],
|
||||
temperature=0.1,
|
||||
model=settings.models.ocr,
|
||||
)
|
||||
import json
|
||||
|
||||
try:
|
||||
parsed = json.loads(response.content.strip().strip("`").removeprefix("json").strip())
|
||||
text = parsed.get("text", response.content)
|
||||
except Exception:
|
||||
text = response.content
|
||||
page_texts.append(f"--- Page {i + 1} ---\n{text.strip()}")
|
||||
|
||||
full_text = "\n\n".join(page_texts)
|
||||
logger.info("Robust OCR extracted %d characters from %s", len(full_text), path)
|
||||
return full_text
|
||||
@@ -0,0 +1,52 @@
|
||||
"""Vision review: compare extracted invoice against the original image."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from pathlib import Path
|
||||
|
||||
from PIL import Image
|
||||
|
||||
from odoo_ocr.clients import BaseVLMClient
|
||||
from odoo_ocr.config import Settings
|
||||
from odoo_ocr.schemas import ExtractedInvoice, ReviewResult
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _load_prompt(name: str) -> str:
|
||||
from odoo_ocr.clients.base import BaseVLMClient
|
||||
|
||||
return BaseVLMClient._load_prompt_file(Path("prompts") / name)
|
||||
|
||||
|
||||
async def review_invoice(
|
||||
invoice: ExtractedInvoice,
|
||||
pages: list[Image.Image],
|
||||
client: BaseVLMClient,
|
||||
settings: Settings,
|
||||
) -> ReviewResult:
|
||||
"""Have a vision model review the extracted invoice against the image."""
|
||||
system_prompt = _load_prompt("review_system.txt")
|
||||
user_prompt = (
|
||||
"Extracted invoice JSON:\n```json\n"
|
||||
f"{invoice.model_dump_json(indent=2)}"
|
||||
"\n```\n\nCompare this with the original invoice image and report any issues."
|
||||
)
|
||||
|
||||
response = await client.complete(
|
||||
system_prompt=system_prompt,
|
||||
user_prompt=user_prompt,
|
||||
images=[pages[0]] if pages else [],
|
||||
response_format=ReviewResult,
|
||||
temperature=0.1,
|
||||
model=settings.models.review,
|
||||
)
|
||||
review = client.parse_json(response.content, ReviewResult)
|
||||
logger.info(
|
||||
"Review result: valid=%s confidence=%.2f issues=%d",
|
||||
review.valid,
|
||||
review.confidence,
|
||||
len(review.issues),
|
||||
)
|
||||
return review
|
||||
@@ -0,0 +1,55 @@
|
||||
"""Processing branch for scanned/photographed printed invoices."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from pathlib import Path
|
||||
|
||||
from PIL import Image
|
||||
|
||||
from odoo_ocr.clients import BaseVLMClient
|
||||
from odoo_ocr.config import Settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _load_prompt(name: str) -> str:
|
||||
from pathlib import Path
|
||||
|
||||
from odoo_ocr.clients.base import BaseVLMClient
|
||||
|
||||
return BaseVLMClient._load_prompt_file(Path("prompts") / name)
|
||||
|
||||
|
||||
async def process_scanned_print(
|
||||
path: Path,
|
||||
pages: list[Image.Image],
|
||||
client: BaseVLMClient,
|
||||
settings: Settings,
|
||||
) -> str:
|
||||
"""Run OCR on each page of a scanned printed invoice and return combined text."""
|
||||
system_prompt = _load_prompt("ocr_system.txt")
|
||||
user_prompt = "Read all text from this printed invoice page accurately."
|
||||
|
||||
page_texts: list[str] = []
|
||||
for i, page in enumerate(pages):
|
||||
response = await client.complete(
|
||||
system_prompt=system_prompt,
|
||||
user_prompt=f"{user_prompt} (page {i + 1} of {len(pages)})",
|
||||
images=[page],
|
||||
temperature=0.1,
|
||||
model=settings.models.ocr,
|
||||
)
|
||||
# The OCR prompt asks for {"text": "..."}.
|
||||
import json
|
||||
|
||||
try:
|
||||
parsed = json.loads(response.content.strip().strip("`").removeprefix("json").strip())
|
||||
text = parsed.get("text", response.content)
|
||||
except Exception:
|
||||
text = response.content
|
||||
page_texts.append(f"--- Page {i + 1} ---\n{text.strip()}")
|
||||
|
||||
full_text = "\n\n".join(page_texts)
|
||||
logger.info("OCR extracted %d characters from scanned invoice %s", len(full_text), path)
|
||||
return full_text
|
||||
@@ -0,0 +1,146 @@
|
||||
"""Build Odoo Enterprise vendor-bill import XML from ExtractedInvoice."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import logging
|
||||
import re
|
||||
from decimal import Decimal
|
||||
|
||||
from lxml import etree
|
||||
|
||||
from odoo_ocr.config import Settings
|
||||
from odoo_ocr.schemas import ExtractedInvoice, ReviewResult
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
CURRENCY_REFS: dict[str, str] = {
|
||||
"EUR": "base.EUR",
|
||||
"USD": "base.USD",
|
||||
"GBP": "base.GBP",
|
||||
"CHF": "base.CHF",
|
||||
"JPY": "base.JPY",
|
||||
}
|
||||
|
||||
|
||||
def _sanitize_id(value: str) -> str:
|
||||
"""Create a valid Odoo external ID from arbitrary text."""
|
||||
value = value.lower().strip()
|
||||
value = re.sub(r"[^a-z0-9]+", "_", value)
|
||||
value = value.strip("_")
|
||||
if not value:
|
||||
value = "x"
|
||||
if value[0].isdigit():
|
||||
value = f"x_{value}"
|
||||
return value
|
||||
|
||||
|
||||
def _external_id_vendor(vendor_name: str, vendor_vat: str | None = None) -> str:
|
||||
seed = vendor_vat if vendor_vat else vendor_name
|
||||
short = hashlib.sha256(seed.encode("utf-8")).hexdigest()[:8]
|
||||
return f"vendor_{_sanitize_id(vendor_name)}_{short}"
|
||||
|
||||
|
||||
def _external_id_invoice(invoice: ExtractedInvoice) -> str:
|
||||
seed = f"{invoice.invoice_number}_{invoice.invoice_date}_{invoice.vendor_name}"
|
||||
short = hashlib.sha256(seed.encode("utf-8")).hexdigest()[:8]
|
||||
number = _sanitize_id(invoice.invoice_number) or "unknown"
|
||||
return f"invoice_{number}_{invoice.invoice_date.replace('-', '')}_{short}"
|
||||
|
||||
|
||||
def _tax_external_id(rate: Decimal) -> str:
|
||||
return f"purchase_vat_{int(rate)}"
|
||||
|
||||
|
||||
def _currency_ref(currency: str, settings: Settings) -> str:
|
||||
return CURRENCY_REFS.get(currency.upper(), CURRENCY_REFS.get(settings.xml.default_currency.upper(), "base.EUR"))
|
||||
|
||||
|
||||
def _to_str(value: Decimal) -> str:
|
||||
return f"{value:.2f}"
|
||||
|
||||
|
||||
def build_odoo_xml(
|
||||
invoice: ExtractedInvoice,
|
||||
review: ReviewResult | None,
|
||||
settings: Settings,
|
||||
) -> str:
|
||||
"""Generate Odoo data-import XML for the invoice as a vendor bill."""
|
||||
vendor_id = _external_id_vendor(invoice.vendor_name, invoice.vendor_vat)
|
||||
invoice_id = _external_id_invoice(invoice)
|
||||
|
||||
odoo = etree.Element("odoo")
|
||||
data = etree.SubElement(odoo, "data", noupdate="0")
|
||||
|
||||
# Human-review warning comment
|
||||
if review and review.confidence < settings.review.high_confidence_threshold:
|
||||
data.append(
|
||||
etree.Comment(
|
||||
f" HUMAN REVIEW ADVISED: confidence={review.confidence:.2f}, "
|
||||
f"issues={len(review.issues)} "
|
||||
)
|
||||
)
|
||||
|
||||
# Vendor
|
||||
vendor_rec = etree.SubElement(data, "record", id=vendor_id, model="res.partner")
|
||||
etree.SubElement(vendor_rec, "field", name="name").text = invoice.vendor_name
|
||||
etree.SubElement(vendor_rec, "field", name="supplier_rank").text = "1"
|
||||
if invoice.vendor_address:
|
||||
etree.SubElement(vendor_rec, "field", name="street").text = invoice.vendor_address
|
||||
if invoice.vendor_vat:
|
||||
etree.SubElement(vendor_rec, "field", name="vat").text = invoice.vendor_vat
|
||||
|
||||
# Taxes
|
||||
tax_ids_used: list[str] = []
|
||||
for line in invoice.line_items:
|
||||
rate = line.tax_rate
|
||||
if rate == 0 and "purchase_vat_0" not in tax_ids_used:
|
||||
tax_ids_used.append("purchase_vat_0")
|
||||
else:
|
||||
tid = _tax_external_id(rate)
|
||||
if tid not in tax_ids_used:
|
||||
tax_ids_used.append(tid)
|
||||
tax_rec = etree.SubElement(data, "record", id=tid, model="account.tax")
|
||||
etree.SubElement(tax_rec, "field", name="name").text = f"Purchase VAT {int(rate)}%"
|
||||
etree.SubElement(tax_rec, "field", name="amount").text = str(rate)
|
||||
etree.SubElement(tax_rec, "field", name="amount_type").text = "percent"
|
||||
etree.SubElement(tax_rec, "field", name="type_tax_use").text = "purchase"
|
||||
|
||||
if "purchase_vat_0" in tax_ids_used:
|
||||
tax_rec = etree.SubElement(data, "record", id="purchase_vat_0", model="account.tax")
|
||||
etree.SubElement(tax_rec, "field", name="name").text = "Purchase VAT 0%"
|
||||
etree.SubElement(tax_rec, "field", name="amount").text = "0"
|
||||
etree.SubElement(tax_rec, "field", name="amount_type").text = "percent"
|
||||
etree.SubElement(tax_rec, "field", name="type_tax_use").text = "purchase"
|
||||
|
||||
# Vendor bill header
|
||||
move_rec = etree.SubElement(data, "record", id=invoice_id, model="account.move")
|
||||
etree.SubElement(move_rec, "field", name="move_type").text = "in_invoice"
|
||||
etree.SubElement(move_rec, "field", name="partner_id", ref=vendor_id)
|
||||
etree.SubElement(move_rec, "field", name="invoice_date").text = invoice.invoice_date
|
||||
etree.SubElement(move_rec, "field", name="date").text = invoice.invoice_date
|
||||
if invoice.due_date:
|
||||
etree.SubElement(move_rec, "field", name="invoice_date_due").text = invoice.due_date
|
||||
etree.SubElement(move_rec, "field", name="ref").text = invoice.invoice_number
|
||||
etree.SubElement(move_rec, "field", name="currency_id", ref=_currency_ref(invoice.currency, settings))
|
||||
if invoice.payment_terms:
|
||||
etree.SubElement(move_rec, "field", name="narration").text = invoice.payment_terms
|
||||
|
||||
# Lines
|
||||
for i, line in enumerate(invoice.line_items, start=1):
|
||||
line_id = f"{invoice_id}_line_{i}"
|
||||
line_rec = etree.SubElement(data, "record", id=line_id, model="account.move.line")
|
||||
etree.SubElement(line_rec, "field", name="move_id", ref=invoice_id)
|
||||
etree.SubElement(line_rec, "field", name="name").text = line.description
|
||||
etree.SubElement(line_rec, "field", name="quantity").text = _to_str(line.quantity)
|
||||
etree.SubElement(line_rec, "field", name="price_unit").text = _to_str(line.unit_price)
|
||||
tax_ref = _tax_external_id(line.tax_rate)
|
||||
etree.SubElement(
|
||||
line_rec,
|
||||
"field",
|
||||
name="tax_ids",
|
||||
eval=f"[(6, 0, [ref('{tax_ref}')])]",
|
||||
)
|
||||
|
||||
xml_bytes: bytes = etree.tostring(odoo, pretty_print=True, xml_declaration=True, encoding="UTF-8")
|
||||
return xml_bytes.decode("utf-8")
|
||||
@@ -0,0 +1,16 @@
|
||||
"""Pydantic schemas for the Odoo OCR pipeline."""
|
||||
|
||||
from .document import DocumentClass, ProcessingContext
|
||||
from .invoice import ExtractedInvoice, InvoiceLineItem
|
||||
from .review import ReviewIssue, ReviewResult
|
||||
from .vlm import VLMResponse
|
||||
|
||||
__all__ = [
|
||||
"DocumentClass",
|
||||
"ProcessingContext",
|
||||
"ExtractedInvoice",
|
||||
"InvoiceLineItem",
|
||||
"ReviewIssue",
|
||||
"ReviewResult",
|
||||
"VLMResponse",
|
||||
]
|
||||
@@ -0,0 +1,41 @@
|
||||
"""Document classification and processing context schemas."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from enum import StrEnum
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
class DocumentClass(StrEnum):
|
||||
"""Class of invoice document received by the pipeline."""
|
||||
|
||||
digital_pdf = "digital_pdf"
|
||||
scanned_print = "scanned_print"
|
||||
handwritten = "handwritten"
|
||||
mixed_unknown = "mixed_unknown"
|
||||
|
||||
|
||||
class ClassificationResult(BaseModel):
|
||||
"""Output of the document classifier."""
|
||||
|
||||
category: DocumentClass
|
||||
confidence: float = Field(ge=0.0, le=1.0)
|
||||
reasoning: str = ""
|
||||
|
||||
|
||||
class ProcessingContext(BaseModel):
|
||||
"""Mutable-ish context passed through pipeline stages.
|
||||
|
||||
Holds the original file path, classification, settings handle, and any
|
||||
intermediate artifacts produced by earlier stages.
|
||||
"""
|
||||
|
||||
source_path: Path
|
||||
classification: ClassificationResult | None = None
|
||||
raw_text: str = ""
|
||||
extracted_invoice: dict[str, Any] | None = None
|
||||
review_result: dict[str, Any] | None = None
|
||||
output_dir: Path | None = None
|
||||
@@ -0,0 +1,58 @@
|
||||
"""Invoice data schemas."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from decimal import Decimal
|
||||
from typing import Any
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field, field_validator
|
||||
|
||||
|
||||
class InvoiceLineItem(BaseModel):
|
||||
"""A single line on an invoice."""
|
||||
|
||||
model_config = ConfigDict(str_strip_whitespace=True)
|
||||
|
||||
description: str
|
||||
quantity: Decimal = Field(default=Decimal("1"), ge=Decimal("0"))
|
||||
unit_price: Decimal = Field(default=Decimal("0"), ge=Decimal("0"))
|
||||
total_price: Decimal = Field(default=Decimal("0"), ge=Decimal("0"))
|
||||
tax_rate: Decimal = Field(default=Decimal("0"), ge=Decimal("0"))
|
||||
|
||||
@field_validator("quantity", "unit_price", "total_price", "tax_rate", mode="before")
|
||||
@classmethod
|
||||
def _coerce_decimal(cls, value: Any) -> Decimal:
|
||||
if value is None:
|
||||
return Decimal("0")
|
||||
return Decimal(str(value))
|
||||
|
||||
|
||||
class ExtractedInvoice(BaseModel):
|
||||
"""Structured representation of an invoice after OCR and extraction."""
|
||||
|
||||
model_config = ConfigDict(str_strip_whitespace=True)
|
||||
|
||||
vendor_name: str
|
||||
vendor_address: str | None = None
|
||||
vendor_vat: str | None = None
|
||||
invoice_number: str
|
||||
invoice_date: str # ISO 8601, e.g. 2024-05-01
|
||||
due_date: str | None = None
|
||||
currency: str = "EUR"
|
||||
payment_terms: str | None = None
|
||||
line_items: list[InvoiceLineItem] = Field(default_factory=list)
|
||||
subtotal: Decimal = Field(default=Decimal("0"))
|
||||
tax_total: Decimal = Field(default=Decimal("0"))
|
||||
total: Decimal = Field(default=Decimal("0"))
|
||||
iban: str | None = None
|
||||
raw_ocr_text: str = ""
|
||||
|
||||
@field_validator("subtotal", "tax_total", "total", mode="before")
|
||||
@classmethod
|
||||
def _coerce_decimal(cls, value: Any) -> Decimal:
|
||||
if value is None:
|
||||
return Decimal("0")
|
||||
return Decimal(str(value))
|
||||
|
||||
def line_items_sum(self) -> Decimal:
|
||||
return sum((line.total_price for line in self.line_items), Decimal("0"))
|
||||
@@ -0,0 +1,29 @@
|
||||
"""Review/validation schemas."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any, Literal
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
from .invoice import ExtractedInvoice
|
||||
|
||||
|
||||
class ReviewIssue(BaseModel):
|
||||
"""A single issue found by the review model."""
|
||||
|
||||
model_config = ConfigDict(str_strip_whitespace=True)
|
||||
|
||||
field: str
|
||||
severity: Literal["error", "warning"]
|
||||
message: str
|
||||
suggested_value: Any | None = None
|
||||
|
||||
|
||||
class ReviewResult(BaseModel):
|
||||
"""Output of the vision review stage."""
|
||||
|
||||
valid: bool
|
||||
confidence: float = Field(ge=0.0, le=1.0)
|
||||
issues: list[ReviewIssue] = Field(default_factory=list)
|
||||
corrected_invoice: ExtractedInvoice | None = None
|
||||
@@ -0,0 +1,13 @@
|
||||
"""VLM client response schema."""
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
|
||||
class VLMResponse(BaseModel):
|
||||
"""Normalized response from any local VLM backend."""
|
||||
|
||||
content: str
|
||||
model: str
|
||||
prompt_tokens: int | None = None
|
||||
completion_tokens: int | None = None
|
||||
duration_ms: int = 0
|
||||
@@ -0,0 +1 @@
|
||||
"""Shared utilities."""
|
||||
@@ -0,0 +1,71 @@
|
||||
"""Simple file-based cache for LLM responses."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import logging
|
||||
from pathlib import Path
|
||||
from typing import Any, cast
|
||||
|
||||
from odoo_ocr.config import Settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _make_key(parts: list[str]) -> str:
|
||||
hasher = hashlib.sha256()
|
||||
for part in parts:
|
||||
hasher.update(part.encode("utf-8"))
|
||||
return hasher.hexdigest()
|
||||
|
||||
|
||||
def _cache_path(settings: Settings, key: str) -> Path:
|
||||
cache_dir = settings.cache.dir.expanduser()
|
||||
cache_dir.mkdir(parents=True, exist_ok=True)
|
||||
return cache_dir / f"{key}.json"
|
||||
|
||||
|
||||
def get_cached(
|
||||
system_prompt: str,
|
||||
user_prompt: str,
|
||||
image_blobs: list[bytes],
|
||||
model: str,
|
||||
temperature: float,
|
||||
settings: Settings,
|
||||
) -> dict[str, Any] | None:
|
||||
"""Return cached response dict if it exists, otherwise None."""
|
||||
if not settings.cache.enabled:
|
||||
return None
|
||||
parts = [system_prompt, user_prompt, model, str(temperature)]
|
||||
for blob in image_blobs:
|
||||
parts.append(hashlib.sha256(blob).hexdigest())
|
||||
key = _make_key(parts)
|
||||
path = _cache_path(settings, key)
|
||||
if path.exists():
|
||||
logger.debug("Cache hit: %s", key)
|
||||
with path.open("r", encoding="utf-8") as f:
|
||||
return cast(dict[str, Any], json.load(f))
|
||||
return None
|
||||
|
||||
|
||||
def set_cached(
|
||||
system_prompt: str,
|
||||
user_prompt: str,
|
||||
image_blobs: list[bytes],
|
||||
model: str,
|
||||
temperature: float,
|
||||
response: dict[str, Any],
|
||||
settings: Settings,
|
||||
) -> None:
|
||||
"""Store a response in the file cache."""
|
||||
if not settings.cache.enabled:
|
||||
return
|
||||
parts = [system_prompt, user_prompt, model, str(temperature)]
|
||||
for blob in image_blobs:
|
||||
parts.append(hashlib.sha256(blob).hexdigest())
|
||||
key = _make_key(parts)
|
||||
path = _cache_path(settings, key)
|
||||
with path.open("w", encoding="utf-8") as f:
|
||||
json.dump(response, f, ensure_ascii=False, indent=2)
|
||||
logger.debug("Cached response: %s", key)
|
||||
@@ -0,0 +1,16 @@
|
||||
"""Centralised logging configuration."""
|
||||
|
||||
import logging
|
||||
import sys
|
||||
|
||||
from odoo_ocr.config import Settings
|
||||
|
||||
|
||||
def configure_logging(settings: Settings) -> None:
|
||||
"""Configure the root logger from settings."""
|
||||
handler = logging.StreamHandler(sys.stdout)
|
||||
handler.setFormatter(logging.Formatter(settings.logging.format))
|
||||
root = logging.getLogger()
|
||||
root.handlers.clear()
|
||||
root.addHandler(handler)
|
||||
root.setLevel(settings.logging.level)
|
||||
@@ -0,0 +1,54 @@
|
||||
"""Shared pytest fixtures."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from typing import Any
|
||||
|
||||
import pytest
|
||||
from PIL import Image
|
||||
|
||||
from odoo_ocr.clients.base import BaseVLMClient
|
||||
from odoo_ocr.config import Settings
|
||||
from odoo_ocr.schemas import VLMResponse
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def settings() -> Settings:
|
||||
"""Default test settings with cache disabled."""
|
||||
return Settings(
|
||||
ollama_base_url="http://test-ollama:11435",
|
||||
cache=Settings().model_dump()["cache"] | {"enabled": False},
|
||||
)
|
||||
|
||||
|
||||
class FakeVLMClient(BaseVLMClient):
|
||||
"""Test double that returns canned responses based on prompt content."""
|
||||
|
||||
def __init__(self, responses: dict[str, str] | None = None) -> None:
|
||||
super().__init__(Settings())
|
||||
self.responses = responses or {}
|
||||
self.calls: list[dict[str, Any]] = []
|
||||
|
||||
async def complete(
|
||||
self,
|
||||
system_prompt: str,
|
||||
user_prompt: str,
|
||||
images: list[Image.Image],
|
||||
response_format: type[Any] | None = None,
|
||||
temperature: float = 0.1,
|
||||
max_tokens: int = 4096,
|
||||
model: str | None = None,
|
||||
) -> VLMResponse:
|
||||
self.calls.append(
|
||||
{
|
||||
"system_prompt": system_prompt,
|
||||
"user_prompt": user_prompt,
|
||||
"image_count": len(images),
|
||||
"model": model,
|
||||
}
|
||||
)
|
||||
for key, value in self.responses.items():
|
||||
if key in system_prompt or key in user_prompt:
|
||||
return VLMResponse(content=json.dumps(value), model="fake")
|
||||
return VLMResponse(content=json.dumps({}), model="fake")
|
||||
@@ -0,0 +1,72 @@
|
||||
"""Tests for the document classifier."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
from PIL import Image
|
||||
|
||||
from odoo_ocr.pipeline.classifier import classify_document
|
||||
from odoo_ocr.schemas.document import DocumentClass
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def blank_image() -> Image.Image:
|
||||
return Image.new("RGB", (100, 100), color="white")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_heuristic_digital_pdf(tmp_path: Path) -> None:
|
||||
pdf_path = tmp_path / "invoice.pdf"
|
||||
# Minimal valid PDF with text via pymupdf
|
||||
import fitz
|
||||
|
||||
doc = fitz.open()
|
||||
page = doc.new_page()
|
||||
page.insert_text((50, 50), "Invoice number 12345\nVendor: Acme")
|
||||
doc.save(str(pdf_path))
|
||||
doc.close()
|
||||
|
||||
from odoo_ocr.config import Settings
|
||||
from tests.conftest import FakeVLMClient
|
||||
|
||||
client = FakeVLMClient()
|
||||
result = await classify_document(pdf_path, [], client, Settings())
|
||||
assert result.category == DocumentClass.digital_pdf
|
||||
assert result.confidence >= 0.90
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_heuristic_scanned_print(blank_image: Image.Image) -> None:
|
||||
# Blank image has low handwriting score
|
||||
from odoo_ocr.config import Settings
|
||||
from tests.conftest import FakeVLMClient
|
||||
|
||||
client = FakeVLMClient()
|
||||
result = await classify_document(Path("scan.jpg"), [blank_image], client, Settings())
|
||||
assert result.category == DocumentClass.scanned_print
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_vlm_classifier_fallback(blank_image: Image.Image) -> None:
|
||||
|
||||
from odoo_ocr.config import Settings
|
||||
from tests.conftest import FakeVLMClient
|
||||
|
||||
client = FakeVLMClient(
|
||||
responses={
|
||||
"document classifier": {
|
||||
"category": "handwritten",
|
||||
"confidence": 0.91,
|
||||
"reasoning": "handwritten text visible",
|
||||
}
|
||||
}
|
||||
)
|
||||
with patch(
|
||||
"odoo_ocr.pipeline.classifier._heuristic_classify", return_value=None
|
||||
):
|
||||
result = await classify_document(Path("unknown.png"), [blank_image], client, Settings())
|
||||
assert result.category == DocumentClass.handwritten
|
||||
assert result.confidence == 0.91
|
||||
@@ -0,0 +1,46 @@
|
||||
"""Tests for the structured extraction stage."""
|
||||
|
||||
|
||||
import pytest
|
||||
from PIL import Image
|
||||
|
||||
from odoo_ocr.config import Settings
|
||||
from odoo_ocr.pipeline.extractor import extract_invoice
|
||||
from odoo_ocr.schemas import ExtractedInvoice
|
||||
from tests.conftest import FakeVLMClient
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_extract_invoice() -> None:
|
||||
client = FakeVLMClient(
|
||||
responses={
|
||||
"invoice data extraction": {
|
||||
"vendor_name": "Acme",
|
||||
"vendor_address": None,
|
||||
"vendor_vat": None,
|
||||
"invoice_number": "INV-1",
|
||||
"invoice_date": "2024-01-01",
|
||||
"due_date": None,
|
||||
"currency": "EUR",
|
||||
"payment_terms": None,
|
||||
"line_items": [
|
||||
{
|
||||
"description": "Widget",
|
||||
"quantity": 2.0,
|
||||
"unit_price": 50.0,
|
||||
"total_price": 100.0,
|
||||
"tax_rate": 0.0,
|
||||
}
|
||||
],
|
||||
"subtotal": 100.0,
|
||||
"tax_total": 0.0,
|
||||
"total": 100.0,
|
||||
"iban": None,
|
||||
"raw_ocr_text": "",
|
||||
}
|
||||
}
|
||||
)
|
||||
result = await extract_invoice("raw text", [Image.new("RGB", (10, 10))], client, Settings())
|
||||
assert isinstance(result, ExtractedInvoice)
|
||||
assert result.vendor_name == "Acme"
|
||||
assert result.line_items_sum() == 100
|
||||
@@ -0,0 +1,74 @@
|
||||
"""Tests for the Ollama VLM client."""
|
||||
|
||||
import base64
|
||||
import json
|
||||
from io import BytesIO
|
||||
from unittest.mock import AsyncMock, MagicMock
|
||||
|
||||
import pytest
|
||||
from PIL import Image
|
||||
|
||||
from odoo_ocr.clients.base import BaseVLMClient
|
||||
from odoo_ocr.clients.ollama import OllamaClient
|
||||
from odoo_ocr.config import Settings
|
||||
from odoo_ocr.schemas import ExtractedInvoice
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def settings() -> Settings:
|
||||
return Settings(
|
||||
ollama_base_url="http://test-ollama:11434",
|
||||
cache=Settings().model_dump()["cache"] | {"enabled": False},
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_complete_parses_response(settings: Settings) -> None:
|
||||
client = OllamaClient(settings)
|
||||
fake_response = {
|
||||
"model": "glm-ocr",
|
||||
"message": {"role": "assistant", "content": '{"text": "Invoice 123"}'},
|
||||
"done": True,
|
||||
"prompt_eval_count": 100,
|
||||
"eval_count": 20,
|
||||
}
|
||||
|
||||
resp = MagicMock()
|
||||
resp.json.return_value = fake_response
|
||||
resp.raise_for_status = lambda: None
|
||||
client.http = AsyncMock()
|
||||
client.http.post.return_value = resp
|
||||
|
||||
img = Image.new("RGB", (50, 50), color="red")
|
||||
response = await client.complete(
|
||||
system_prompt="ocr",
|
||||
user_prompt="read",
|
||||
images=[img],
|
||||
)
|
||||
|
||||
assert response.content == '{"text": "Invoice 123"}'
|
||||
assert response.model == "glm-ocr"
|
||||
assert response.prompt_tokens == 100
|
||||
assert response.completion_tokens == 20
|
||||
|
||||
|
||||
def test_parse_json_extracted_invoice() -> None:
|
||||
data = {
|
||||
"vendor_name": "Acme",
|
||||
"invoice_number": "1",
|
||||
"invoice_date": "2024-01-01",
|
||||
"line_items": [],
|
||||
"subtotal": "100",
|
||||
"tax_total": "20",
|
||||
"total": "120",
|
||||
}
|
||||
result = BaseVLMClient.parse_json(json.dumps(data), ExtractedInvoice)
|
||||
assert isinstance(result, ExtractedInvoice)
|
||||
assert result.vendor_name == "Acme"
|
||||
|
||||
|
||||
def test_encode_image_roundtrip() -> None:
|
||||
img = Image.new("RGB", (10, 10), color="blue")
|
||||
encoded = OllamaClient._encode_image(img) # type: ignore[attr-defined]
|
||||
decoded = Image.open(BytesIO(base64.b64decode(encoded)))
|
||||
assert decoded.size == (10, 10)
|
||||
@@ -0,0 +1,60 @@
|
||||
"""Tests for the Odoo XML builder."""
|
||||
|
||||
from decimal import Decimal
|
||||
|
||||
from odoo_ocr.config import Settings
|
||||
from odoo_ocr.pipeline.xml_builder import build_odoo_xml
|
||||
from odoo_ocr.schemas import ExtractedInvoice, InvoiceLineItem, ReviewResult
|
||||
|
||||
|
||||
def test_build_odoo_xml() -> None:
|
||||
settings = Settings()
|
||||
invoice = ExtractedInvoice(
|
||||
vendor_name="Acme Supplies",
|
||||
vendor_vat="GB123456789",
|
||||
invoice_number="INV-2024-001",
|
||||
invoice_date="2024-05-01",
|
||||
due_date="2024-06-01",
|
||||
currency="EUR",
|
||||
payment_terms="Net 30",
|
||||
line_items=[
|
||||
InvoiceLineItem(
|
||||
description="Consulting services",
|
||||
quantity=Decimal("10.0"),
|
||||
unit_price=Decimal("100.00"),
|
||||
total_price=Decimal("1000.00"),
|
||||
tax_rate=Decimal("20"),
|
||||
)
|
||||
],
|
||||
subtotal=Decimal("1000.00"),
|
||||
tax_total=Decimal("200.00"),
|
||||
total=Decimal("1200.00"),
|
||||
)
|
||||
review = ReviewResult(valid=True, confidence=0.95)
|
||||
|
||||
xml = build_odoo_xml(invoice, review, settings)
|
||||
|
||||
assert xml.startswith("<?xml version=")
|
||||
assert 'model="res.partner"' in xml
|
||||
assert 'model="account.move"' in xml
|
||||
assert 'model="account.move.line"' in xml
|
||||
assert "Acme Supplies" in xml
|
||||
assert "INV-2024-001" in xml
|
||||
assert "in_invoice" in xml
|
||||
|
||||
|
||||
def test_xml_adds_review_comment_for_low_confidence() -> None:
|
||||
settings = Settings()
|
||||
invoice = ExtractedInvoice(
|
||||
vendor_name="X",
|
||||
invoice_number="1",
|
||||
invoice_date="2024-01-01",
|
||||
line_items=[],
|
||||
subtotal=Decimal("0"),
|
||||
tax_total=Decimal("0"),
|
||||
total=Decimal("0"),
|
||||
)
|
||||
review = ReviewResult(valid=False, confidence=0.80)
|
||||
|
||||
xml = build_odoo_xml(invoice, review, settings)
|
||||
assert "HUMAN REVIEW ADVISED" in xml
|
||||
Reference in New Issue
Block a user