Add semantic cache support using ChromaDB #3
@@ -5,10 +5,11 @@ A plugin that integrates Ollama with Obsidian to create a chat interface that ca
|
||||
## Features
|
||||
|
||||
- Chat with Ollama models directly in Obsidian
|
||||
- Vault context search - the assistant can reference your notes
|
||||
- Tool integration - create files based on chat responses
|
||||
- Vault context search — the assistant can reference your notes
|
||||
- Tool integration — create files based on chat responses
|
||||
- Streaming responses
|
||||
- Customizable model and URL settings
|
||||
- Semantic response cache — repeated or similar queries are answered instantly without hitting the model (requires ChromaDB)
|
||||
- Customisable model, URL, and cache settings
|
||||
|
||||
## Installation
|
||||
|
||||
@@ -17,48 +18,87 @@ A plugin that integrates Ollama with Obsidian to create a chat interface that ca
|
||||
|
||||
## Setup
|
||||
|
||||
1. **Install Ollama**: Follow the instructions at [ollama.ai](https://ollama.ai) to install Ollama
|
||||
2. **Start Ollama service**: `ollama serve`
|
||||
3. **Pull a model**: `ollama pull llama3` (or any other model you prefer)
|
||||
### Required
|
||||
|
||||
1. **Install Ollama**: Follow the instructions at [ollama.ai](https://ollama.ai)
|
||||
2. **Start Ollama**: `ollama serve`
|
||||
3. **Pull a chat model**: `ollama pull llama3` (or any other model you prefer)
|
||||
|
||||
### Optional — Semantic Cache
|
||||
|
||||
The semantic cache stores responses in a local [ChromaDB](https://www.trychroma.com) vector database. When you ask a question that is semantically similar to one already cached, the stored answer is returned immediately instead of calling the model.
|
||||
|
||||
1. **Install ChromaDB**:
|
||||
```bash
|
||||
pip install chromadb
|
||||
```
|
||||
2. **Start ChromaDB**:
|
||||
```bash
|
||||
chroma run --host localhost --port 8000
|
||||
```
|
||||
3. **Pull an embedding model** (used to generate vectors for cache lookups):
|
||||
```bash
|
||||
ollama pull nomic-embed-text
|
||||
```
|
||||
4. Enable the cache in the plugin settings and configure the ChromaDB URL.
|
||||
|
||||
## Configuration
|
||||
|
||||
1. Open the plugin settings via Obsidian's settings panel
|
||||
2. Configure the Ollama URL (default: `http://localhost:11434`)
|
||||
3. Configure the model name (default: `llama3`)
|
||||
4. Restart the plugin if needed
|
||||
Open **Settings → Ollama Chat** to configure the plugin.
|
||||
|
||||
| Setting | Default | Description |
|
||||
|---------|---------|-------------|
|
||||
| Ollama URL | `http://localhost:11434` | Base URL of your Ollama instance |
|
||||
| Model | `llama3` | Model used for chat responses |
|
||||
| Enable Semantic Cache | Off | Cache responses for fast repeated queries |
|
||||
| ChromaDB URL | `http://localhost:8000` | URL of your running ChromaDB instance |
|
||||
| Cache Embedding Model | `nomic-embed-text` | Ollama model used to generate cache embeddings |
|
||||
| Cache Similarity Threshold | `0.85` | Minimum cosine similarity (0–1) for a cache hit — higher values require closer matches |
|
||||
| Clear Semantic Cache | — | Button to wipe all cached responses from ChromaDB |
|
||||
|
||||
## Usage
|
||||
|
||||
1. Click the ribbon icon to open the chat view
|
||||
2. Type your message in the input box
|
||||
3. Press Enter or click Send to send your message
|
||||
4. Click the "New Chat" button to start a fresh conversation
|
||||
3. Press **Enter** or click **Send** to send your message
|
||||
4. Press **Shift+Enter** to insert a line break
|
||||
5. Click **New Chat** to start a fresh conversation
|
||||
|
||||
## Semantic Cache Behaviour
|
||||
|
||||
- The cache is **bypassed** when tool calls are involved (e.g. file creation), since those requests have side effects.
|
||||
- Responses are stored against the last user message in the conversation. If a new query is sufficiently similar (above the configured threshold), the cached response is returned.
|
||||
- Re-asking the same question updates the existing cache entry rather than creating a duplicate.
|
||||
- Use the **Clear Semantic Cache** button in settings to remove all stored responses (for example after switching embedding models).
|
||||
|
||||
## Supported Models
|
||||
|
||||
Any model supported by Ollama should work, including:
|
||||
|
||||
- llama3
|
||||
- llama2
|
||||
- mistral
|
||||
- codellama
|
||||
- etc.
|
||||
- `llama3`
|
||||
- `llama2`
|
||||
- `mistral`
|
||||
- `codellama`
|
||||
- and many more — see [ollama.com/library](https://ollama.com/library)
|
||||
|
||||
## Development
|
||||
|
||||
To build from source:
|
||||
|
||||
```bash
|
||||
npm install
|
||||
npm run build
|
||||
npm test
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- **Connection issues**: Ensure Ollama is running and accessible at the configured URL
|
||||
- **Model not found**: Make sure you've pulled the model (`ollama pull <modelname>`)
|
||||
- **Permission issues**: Check that your Obsidian vault has proper write permissions
|
||||
| Symptom | Likely cause | Fix |
|
||||
|---------|-------------|-----|
|
||||
| Cannot connect to Ollama | Ollama is not running | Run `ollama serve` |
|
||||
| Model not found | Model not pulled | Run `ollama pull <model>` |
|
||||
| Semantic cache unavailable (notice shown) | ChromaDB is not running, or the ChromaDB URL is wrong | Start ChromaDB (`chroma run`) and verify the URL in settings |
|
||||
| Cache always misses | Similarity threshold is too high, or the embedding model was changed | Lower the threshold or click **Clear Semantic Cache** and let the cache rebuild |
|
||||
| Slow first response after enabling cache | Embedding model not yet pulled | Run `ollama pull nomic-embed-text` (or the model you configured) |
|
||||
| Permission issues | Vault write permissions | Check that your Obsidian vault has proper write permissions |
|
||||
|
||||
## License
|
||||
|
||||
|
||||
Reference in New Issue
Block a user