Commit Graph

78 Commits

Author SHA1 Message Date
fegger 106abfa718 Integrate Obsidian metadataCache across extraction and tooling
Replaces regex-based parsing of frontmatter and headings with
Obsidian's metadataCache where available, falling back to regex
when the cache is unavailable. Propagates App dependency through
constructors to enable cache access.

Key changes:
- ContentExtractor accepts optional cache for frontmatter/headings
- ToolExecutor uses cache for section replacement and frontmatter
- NoteContextBuilder resolves titles/tags from cache
- VaultVectorStore passes cache through indexing pipeline
- AutoTagger checks cache for existing tags instead of content
- Mock updated with metadataCache stubs for tests
2026-05-20 20:36:47 +02:00
fegger 3c7c4d58bb Add hybrid search with filtering and recency boost
Introduces SearchOptions interface with folder/tag filters, exact phrase
matching via quoted queries, and optional recency boosting with configurable
half-life. Replaces pure semantic or keyword search with a combined scoring
model: keyword scores are blended with semantic results, then adjusted by
filters and recency. Adds mtime to vault index entries and expands test
coverage for the new options.
2026-05-20 19:52:49 +02:00
fegger 3abacb5d6e Add NoteContextBuilder for enhanced chat context gathering
Introduces NoteContextBuilder to extract explicit wikilink mentions,
detect scope intent, and gather contextual note data including
backlinks, outlinks, and related notes. Integrates into ChatView
and adds comprehensive unit tests.

Also includes minor type fixes: removes unnecessary `as` cast in
action-preview-builder, fixes non-null assertion in tool-executor,
and cleans up unused import in auto-organizer. Simplifies auto-tag
command callback by removing redundant async/await.
2026-05-20 19:17:42 +02:00
fegger 64ee7763fe Add workflow command support to chat view
Integrate WorkflowEngine into ChatView to handle `/workflow` commands.
When a message starts with `/workflow`, the workflow engine generates
and executes a multi-step plan instead of the standard chat flow.
Results are formatted with step-by-step status and output display.

Includes workflow result formatting, assistant message updates, and
short-term context tracking. Tests verify command parsing and engine
invocation.
2026-05-20 18:58:21 +02:00
fegger 158d5f68e6 Add action preview builder for write tool confirmation
Introduces `ActionPreviewBuilder` to generate before/after previews for
destructive operations. Write tools are now deferred with apply/cancel
UI instead of executing immediately. Includes `ProposedAction` type,
CSS for diff views, and state management in `ChatView`.
2026-05-20 18:36:00 +02:00
fegger 95a6954b50 Add vault note management tools
Expand the tool executor with create_note, append_to_note, replace_note_section,
update_frontmatter, rename_note, move_note, delete_note, and insert_link tools.
Rename create_file to create_note for consistency and add helper methods for
common file operations. Update tests to cover the new tools and renamed
functionality.
2026-05-20 18:19:09 +02:00
fegger 8e338afeac feat: increase vault context limits and make them configurable
- Replace hardcoded 2000-char context limit with configurable maxContextLength
  setting (default: 8000 characters)
- Increase vaultSearchLimit default from 3 to 5 notes
- Add 'Max Context Length' setting to plugin settings UI
- Update README with new defaults
- Update chat-view tests for new setting
2026-05-20 01:20:49 +02:00
fegger f09e134948 fix: include tags in vault context sent to AI and update system prompt
- chat-view.ts: include 'Tags: ...' prefix in vault context when tags exist
- chat-view.ts: update system prompt to instruct AI to pay attention to tags
  when organizing or categorizing notes
2026-05-20 01:13:40 +02:00
fegger a8d8936b11 feat: add auto-tag and auto-link note organization features
- Add AutoTagger: scans untagged notes, generates AI tags via Ollama,
  and applies them to frontmatter
- Add AutoLinker: finds semantically related notes via vault search
  and appends a 'Related Notes' section with wiki-links
- Add settings UI for both features with configurable thresholds,
  prompt templates, and limits
- Add commands: 'Auto-Tag Untagged Notes' and 'Auto-Link Related Notes'
- Add auto-organizer.test.ts with 17 tests covering tagging,
  linking, frontmatter manipulation, and filtering
2026-05-20 00:12:30 +02:00
fegger cb621c83b5 fix: process embeddings sequentially, shorten prompts, fix tests
- main.ts: process files sequentially (not Promise.all) to avoid concurrent
  embedding requests hammering Ollama; batch size reduced to 1
- vectorization.ts: shorten embedding prompts from 1000 to 500 chars,
  limit headings to 5, remove frontmatter from prompt to stay well within
  embedding model context window
- Update vectorization and indexing-pipeline tests for new prompt format
2026-05-19 23:51:29 +02:00
fegger cc97d77810 fix: use collection deletion instead of reset, add embedding retry, slow down indexing
- semantic-cache.clearCache: delete collection instead of client.reset()
  to avoid 403 Forbidden on newer ChromaDB versions
- vault-vector-store.clearIndex: same collection deletion approach
- ContentVectorizer: add 3-attempt retry with exponential backoff (1s, 2s, 4s)
- main.ts: reduce indexing batch size from 5 to 2, increase delay from 100ms to 500ms
- Update semantic-cache tests for deleteCollection
2026-05-19 23:41:05 +02:00
fegger ce109cf6fb fix: prevent race conditions during vault index rebuild and rate-limit embeddings
- Add AbortController to cancel ongoing indexing before clear/rebuild
- Track currentIndexingPromise to await cancellation before clearing collection
- Batch background indexing (5 files at a time) with 100ms delays between batches
- Skip incremental event-based indexing while a full rebuild is in progress
- Cancel indexing on plugin unload
- Fix nginx CORS headers on OPTIONS preflight responses
2026-05-19 23:29:34 +02:00
fegger fae74ade95 feat: add semantic/RAG vault indexing with automatic background sync
- Add VaultVectorStore backed by ChromaDB for vector-based vault search
- Integrate existing ContentVectorizer/IndexingPipeline for embeddings
- Update VaultIndexer to prefer semantic search with keyword fallback
- Background indexing on plugin load + incremental sync via vault events
- Add vault index settings, commands, and UI controls
- Add tests for VaultVectorStore
- Update README with RAG setup instructions
2026-05-19 23:00:04 +02:00
fegger 4f3472a49c fix: move all CORS headers into nginx location block
Server-level add_header does not reliably attach to proxied upstream
responses. Moving all CORS directives inside location / ensures they
are present on every response ChromaDB sends back, including 4xx/5xx.

Also removes the redundant CHROMA_SERVER_CORS_ALLOW_ORIGINS from the
chroma service so only Nginx manages CORS, avoiding conflicting
duplicate headers.
2026-05-19 22:21:44 +02:00
fegger b62ad15be8 fix: move CORS OPTIONS handling into nginx location block
Nginx does not allow add_header inside if blocks at the server level.
Moving the OPTIONS preflight response into the location / block fixes the
emerg directive error and allows nginx to start correctly.
2026-05-19 22:15:04 +02:00
fegger 76887ea8ca feat: add Nginx reverse proxy for robust CORS handling
Replace direct host port mapping with an Nginx reverse proxy that properly
handles CORS for any origin, including Obsidian's app://obsidian.md.

- docker-compose.yml: Add nginx service on port 8666, route to internal
  chroma:8000. Remove CORS env var from ChromaDB (handled by proxy now).

- nginx.conf (new): Minimal alpine config with permissive CORS headers,
  preflight OPTIONS handling, and streaming support (proxy_buffering off).

This fixes cross-origin access from remote Obsidian clients connecting over
Tailscale or VPN.
2026-05-19 22:08:11 +02:00
fegger 1ccd637149 fix: sanitize ChromaDB URL and update docker-compose CORS config
- src/semantic-cache.ts: Add robust URL sanitization in initialize().
  Trim whitespace and reject malformed URLs (e.g. empty host like
  'http://:8666') before passing to ChromaClient.

- src/main.ts: Validate ChromaDB URL in the settings tab onChange.
  Fall back to 'http://localhost:8000' if the value is empty or lacks
  '://'.

- docker-compose.yml: Add CHROMA_SERVER_CORS_ALLOW_ORIGINS=["*"] to
  allow cross-origin requests from Obsidian's app://obsidian.md origin.
2026-05-19 21:59:35 +02:00
fegger 6c438f7a4d fix: remove artificial MAX_STREAM_CHUNKS limit that cut off long responses
The plugin hard-capped streaming responses at 1000 chunks. For large
models like qwen2.5:32b generating detailed answers, this limit was
easily exceeded, causing the response to stop mid-sentence.

The Ollama stream already terminates naturally when the model sends the
final done signal, so the artificial chunk limit served no purpose.

- src/chat-view.ts: Remove chunkCount tracking and MAX_STREAM_CHUNKS
  constant. Let the stream run until Ollama signals completion.
2026-05-19 21:24:38 +02:00
fegger 70bf963f28 feat: add thinking indicator while model is generating
- src/types.ts: Add isThinking flag to ChatMessage to track transient
  'model is working' state.

- src/chat-view.ts: Set isThinking: true on the assistant placeholder
  message when user sends input. Clear it when the first stream chunk
  arrives or on error. Update render() to show a spinner + 'Thinking…'
  text while isThinking is active.

- styles.css: Add ollama-thinking-indicator class with a CSS spinner
  animation and muted italic text styling.
2026-05-19 21:21:47 +02:00
fegger 810676ff21 feat: add Ollama icon and modern chat UI styling
- src/chat-view.ts: Add getIcon() returning 'bot' for the view tab icon.
  Improve render() with role-specific CSS classes (user vs assistant) and
  message header structure for better styling hooks.

- src/main.ts: Add ribbon icon ('bot') in the left sidebar that opens the
  chat view with a single click.

- styles.css (new): Modern chat UI with message bubbles, distinct user and
  assistant themes using Obsidian CSS variables, sticky input bar, styled
  send button with accent color, and emoji role indicators.

- install.sh: Copy styles.css into the plugin directory and verify its
  presence during installation.

- README.md: Include styles.css in manual install instructions.

- __mocks__/obsidian.ts: Add addRibbonIcon() mock for test compatibility.

- tests/chat-view.test.ts: Add getIcon() assertion.
2026-05-19 21:18:56 +02:00
fegger 138890b9d2 feat: improve UX for Ollama 404 errors (missing model)
- src/ollama-client.ts: Detect HTTP 404 on /api/chat and throw a descriptive
  ApiError with the model name and the exact ollama pull command needed.

- src/error-handler.ts: For API_ERROR type, return the error message directly
  instead of prefixing with 'API error: ', so the user-friendly 404 message
  is shown cleanly in the Obsidian notice.

- tests/ollama-client.test.ts: Update 404 assertions to match the new
  descriptive error message.
2026-05-19 21:13:08 +02:00
fegger 1ed2e39c3d fix: resolve ESLint errors and align chromadb types with bundled client
- src/semantic-cache.ts: Replace require('chromadb') with static import and
  use proper ChromaClient/Collection types instead of any. Fix camelCase
  API parameters (queryEmbeddings, nResults) and wrap single embedding into
  Embedding[] for upsert. Fix clearCache to call client.reset() instead of
  collection.reset() (matches actual chromadb API).

- src/workflow-engine/workflow-engine.ts: Fix unnecessary escapes in regex,
  remove redundant 'as unknown' assertion, handle never type in template
  literal, and add type annotations to replace callback to satisfy
  no-unsafe-argument and no-base-to-string rules.

- tests/semantic-cache.test.ts: Update mocks to include client.reset() and
  adjust clearCache assertions to match new implementation.
2026-05-19 20:44:27 +02:00
fegger 97cc4ed5fe fix: bundle chromadb and fix fetch invocation in Electron
- Bundle chromadb into main.js via esbuild instead of externalizing it.
  Obsidian's renderer cannot resolve bare require('chromadb') against a
  plugin-local node_modules. By bundling, the client library is inlined and
  works out of the box with just manifest.json + main.js.

- Remove eager cache initialization from OllamaClient constructor to avoid
  unhandled promise rejections when chromadb/ChromaDB is unavailable.

- Fix 'Failed to execute fetch on Window: Illegal invocation' by wrapping
  the default fetch fallback in an arrow function:
    this.fetchFn = fetchFn ?? ((url, init) => fetch(url, init));
  This preserves the window binding when fetch is called later.

- Update install.sh and README to remove the obsolete node_modules/chromadb
  copy step.

- Update ollama-client-cache tests to reflect that cache initialization is
  no longer eager.
2026-05-19 20:38:50 +02:00
fegger 26e178fa96 Fix Electron resolution for optional chromadb dependency
Replace dynamic ESM import with require() for chromadb to ensure
Electron can resolve the package against the plugin's node_modules.
Also wrap default fetch fallback in an arrow function to avoid
potential strict mode issues with global fetch.
2026-05-19 20:19:28 +02:00
fegger 3342d7d955 fix: use npm install (not --production) so build tools are available
The install script ran npm install --production, which skips devDependencies.
Since typescript, esbuild, and other build tools live in devDependencies,
npm run build would fail with 'tsc: command not found'.

Switched to plain npm install so all dependencies are available at build
time.  Only the necessary runtime dep (chromadb) is copied to the plugin
folder — obsidian stays out since it's a type stub.
2026-05-19 19:01:45 +02:00
fegger 9367811c5a fix: switch to esbuild bundling — single main.js at plugin root
The root cause: Obsidian's plugin loader expects main.js at the plugin root
alongside manifest.json.  The previous 'dist/' output + shim approach caused
'Cannot find module ./dist/main.js' because dist/ was either missing or not
resolved correctly in Obsidian's module loader.

Changes:
- build: use esbuild to bundle all source into a single main.js (61KB)
  tsc --noEmit for type checking; esbuild for the actual bundle
- main.js: no longer a shim — it's the fully bundled plugin
- package.json: added esbuild as devDependency; obsidian moved to devDeps
- install.sh: remove dist/ copy step, add cleanup of old dist/ from vault
- README.md: updated manual install steps to reflect bundling
2026-05-19 18:23:24 +02:00
fegger 4ea2734ade fix: move obsidian to devDependencies, improve shim export
- package.json: move obsidian from dependencies to devDependencies
  (it is a type stub — having it in dependencies risks shadowing
  Obsidian's built-in API if node_modules is present in the plugin folder)
- main.js: explicitly extract default export from dist/main.js so the
  shim works with both plugin.default and direct-export loader patterns
2026-05-19 18:21:17 +02:00
fegger 44cff07ea1 fix: add main.js shim at plugin root for Obsidian loader
Obsidian's plugin loader expects main.js at the plugin root alongside
manifest.json — it does not resolve subdirectory paths in the 'main'
field.  Added a thin CommonJS shim that re-exports from dist/main.js.

- main.js: new entry shim (module.exports = require('./dist/main.js'))
- manifest.json: 'main' changed from 'dist/main.js' to 'main.js'
- install.sh: copy main.js shim to plugin folder, verify its presence
- README.md: add main.js to manual install instructions
2026-05-19 18:12:14 +02:00
fegger d31c989423 fix: cherry-pick improvements from feature/semantic_caching
- semantic-cache.ts: use upsert instead of add to avoid duplicate-ID errors
- semantic-cache.ts: add crypto.randomUUID() fallback for constrained runtimes
- ollama-client.ts: JSON.stringify non-string Ollama errors to avoid [object Object]
- tests: update semantic-cache mocks and assertions for upsert
2026-05-19 18:02:23 +02:00
fegger 378642152e fix: plugin cannot be installed — manifest path, deps, and install script
- manifest.json: fix 'main' from 'src/main.js' to 'dist/main.js'
  (TypeScript compiles to dist/, not src/ — Obsidian could not find the entry point)
- manifest.json: set isDesktopOnly to true (plugin requires a local Ollama server)
- package.json: remove unused node-fetch dependency (ESM-only, conflicts with CommonJS build)
- src/semantic-cache.ts: use dynamic import for chromadb instead of top-level import
  (prevents plugin load crash when chromadb is not installed; cache defaults to disabled)
- install.sh: new script that installs deps, builds, and copies the plugin into a vault
- README.md: add quick install, manual install, and expanded troubleshooting sections
2026-05-19 17:39:10 +02:00
fegger 26d86d01db Update main entry point to dist/main.js 2026-05-19 17:35:47 +02:00
fegger 2d78882594 chore: move compiled output to dist
Update the plugin entrypoint and TypeScript output directory to use dist instead of writing generated JavaScript into src.

Remove previously checked-in compiled source files and add coverage for the Ollama client and tool executor.
2026-05-19 17:30:05 +02:00
fegger 2e26c72c0c Add workflow engine for multi-step chained reasoning 2026-05-08 19:44:16 +02:00
fegger 52b4a6c021 Add tests for conversation state, graph view, tool executor, and vectorization 2026-05-08 18:21:20 +02:00
fegger 4a66b46844 Add dependency graph view and refine chat message type 2026-05-08 12:33:45 +02:00
fegger 771db09d24 Refactor chat view and add conversation state management
Introduce ConversationStateManager to handle short, medium, and long-term
context for improved conversation flow. Update ChatView to use this manager
and refactor input handling to accept values directly for better testability.

Update OllamaClient with non-streaming chat support and improved error
handling for malformed chunks. Enhance vault indexer with caching, better
scoring, and stop word filtering. Refactor main plugin entry point and
semantic cache initialization for robustness.
2026-05-08 11:52:31 +02:00
fegger b37ca2c178 Add cache interface and semantic cache tests 2026-05-08 02:20:06 +02:00
fegger 6bc1133a81 Refactor type annotations and error handling in core modules
Standardize error message extraction to handle non-Error objects.
Add explicit return types and interface definitions to VaultIndexer.
Update SemanticCacheService to use any types where necessary.
Rename SemanticCache to SemanticCacheService and adjust imports.
2026-05-08 02:19:44 +02:00
fegger fff98d1a2e Merge branch 'modularize-indexing-pipeline' 2026-05-07 23:45:31 +02:00
fegger 6423e2aa0d Clean up settings display and remove validation logic
Remove unused imports and simplify Ollama client stream handling

Add configurable ChromaDB host and UUID fallback

Refactor vault indexer to use tokenized content and improve caching
2026-05-07 23:23:32 +02:00
fegger 1985f849f4 Add semantic caching support with ChromaDB integration
Implement semantic caching for Ollama chat responses using ChromaDB to store and retrieve embeddings. The cache can be
enabled/disabled in settings and includes configurable similarity threshold, embedding model, and ChromaDB URL. Added
cache clear functionality and error handling for cache operations.
2026-05-07 23:03:58 +02:00
fegger 951c3bbc92 Fix variable naming consistency for chroma URL configuration
Update chromaUrl to chromaURL throughout the codebase to ensure consistent naming convention for the Chroma database URL
configuration parameter. This change affects the semantic cache service implementation and related tests.

The change updates the configuration property name from `chromaUrl` to `chromaURL` in:
- SemanticCacheService class
- Test files (chat-view.test.ts, ollama-client-cache.test.ts, semantic-cache.test.ts)

This maintains consistency with other URL configuration parameters in the codebase and improves code readability.
2026-05-07 22:49:07 +02:00
fegger d4a0919764 Fix merge conflicts and apply all semantic cache fixes 2026-05-07 22:49:07 +02:00
fegger e6d791a655 Update README.md 2026-05-07 22:49:01 +02:00
fegger ae16396a7a Integrate semantic cache with ChromaDB URL 2026-05-07 22:47:32 +02:00
fegger 179a58b95b Add semantic cache support using ChromaDB 2026-05-07 22:47:01 +02:00
fegger d37b9f23bd Add semantic caching and indexing pipeline
This commit adds semantic caching functionality to speed up repeated queries and implements a complete indexing pipeline
for processing vault files. The changes include:

- Added semantic cache service using ChromaDB for storing and retrieving cached responses
- Implemented indexing pipeline with extraction, normalization, and vectorization steps
- Added cache configuration settings to the plugin
- Updated Ollama client to support cache integration
- Added tests for all new indexing components
- Extended vault indexer with indexing pipeline support
2026-05-07 22:23:35 +02:00
fegger 47ecd3f803 feat: Implement modular indexing pipeline with extraction, normalization, and vectorization components 2026-05-07 22:13:43 +02:00
fegger f3a10a4b01 Merge pull request 'Refactor retry logic and extract scoring weights' (#1) from agent/review into main
Reviewed-on: http://100.103.83.12:3003/fegger/obsidian_ollama/pulls/1
2026-05-07 20:56:27 +02:00
fegger b37aaeb4d4 Add semantic cache support using ChromaDB 2026-05-07 20:53:28 +02:00