Add semantic cache support using ChromaDB #3
@@ -5,10 +5,11 @@ A plugin that integrates Ollama with Obsidian to create a chat interface that ca
|
|||||||
## Features
|
## Features
|
||||||
|
|
||||||
- Chat with Ollama models directly in Obsidian
|
- Chat with Ollama models directly in Obsidian
|
||||||
- Vault context search - the assistant can reference your notes
|
- Vault context search — the assistant can reference your notes
|
||||||
- Tool integration - create files based on chat responses
|
- Tool integration — create files based on chat responses
|
||||||
- Streaming responses
|
- Streaming responses
|
||||||
- Customizable model and URL settings
|
- Semantic response cache — repeated or similar queries are answered instantly without hitting the model (requires ChromaDB)
|
||||||
|
- Customisable model, URL, and cache settings
|
||||||
|
|
||||||
## Installation
|
## Installation
|
||||||
|
|
||||||
@@ -17,48 +18,87 @@ A plugin that integrates Ollama with Obsidian to create a chat interface that ca
|
|||||||
|
|
||||||
## Setup
|
## Setup
|
||||||
|
|
||||||
1. **Install Ollama**: Follow the instructions at [ollama.ai](https://ollama.ai) to install Ollama
|
### Required
|
||||||
2. **Start Ollama service**: `ollama serve`
|
|
||||||
3. **Pull a model**: `ollama pull llama3` (or any other model you prefer)
|
1. **Install Ollama**: Follow the instructions at [ollama.ai](https://ollama.ai)
|
||||||
|
2. **Start Ollama**: `ollama serve`
|
||||||
|
3. **Pull a chat model**: `ollama pull llama3` (or any other model you prefer)
|
||||||
|
|
||||||
|
### Optional — Semantic Cache
|
||||||
|
|
||||||
|
The semantic cache stores responses in a local [ChromaDB](https://www.trychroma.com) vector database. When you ask a question that is semantically similar to one already cached, the stored answer is returned immediately instead of calling the model.
|
||||||
|
|
||||||
|
1. **Install ChromaDB**:
|
||||||
|
```bash
|
||||||
|
pip install chromadb
|
||||||
|
```
|
||||||
|
2. **Start ChromaDB**:
|
||||||
|
```bash
|
||||||
|
chroma run --host localhost --port 8000
|
||||||
|
```
|
||||||
|
3. **Pull an embedding model** (used to generate vectors for cache lookups):
|
||||||
|
```bash
|
||||||
|
ollama pull nomic-embed-text
|
||||||
|
```
|
||||||
|
4. Enable the cache in the plugin settings and configure the ChromaDB URL.
|
||||||
|
|
||||||
## Configuration
|
## Configuration
|
||||||
|
|
||||||
1. Open the plugin settings via Obsidian's settings panel
|
Open **Settings → Ollama Chat** to configure the plugin.
|
||||||
2. Configure the Ollama URL (default: `http://localhost:11434`)
|
|
||||||
3. Configure the model name (default: `llama3`)
|
| Setting | Default | Description |
|
||||||
4. Restart the plugin if needed
|
|---------|---------|-------------|
|
||||||
|
| Ollama URL | `http://localhost:11434` | Base URL of your Ollama instance |
|
||||||
|
| Model | `llama3` | Model used for chat responses |
|
||||||
|
| Enable Semantic Cache | Off | Cache responses for fast repeated queries |
|
||||||
|
| ChromaDB URL | `http://localhost:8000` | URL of your running ChromaDB instance |
|
||||||
|
| Cache Embedding Model | `nomic-embed-text` | Ollama model used to generate cache embeddings |
|
||||||
|
| Cache Similarity Threshold | `0.85` | Minimum cosine similarity (0–1) for a cache hit — higher values require closer matches |
|
||||||
|
| Clear Semantic Cache | — | Button to wipe all cached responses from ChromaDB |
|
||||||
|
|
||||||
## Usage
|
## Usage
|
||||||
|
|
||||||
1. Click the ribbon icon to open the chat view
|
1. Click the ribbon icon to open the chat view
|
||||||
2. Type your message in the input box
|
2. Type your message in the input box
|
||||||
3. Press Enter or click Send to send your message
|
3. Press **Enter** or click **Send** to send your message
|
||||||
4. Click the "New Chat" button to start a fresh conversation
|
4. Press **Shift+Enter** to insert a line break
|
||||||
|
5. Click **New Chat** to start a fresh conversation
|
||||||
|
|
||||||
|
## Semantic Cache Behaviour
|
||||||
|
|
||||||
|
- The cache is **bypassed** when tool calls are involved (e.g. file creation), since those requests have side effects.
|
||||||
|
- Responses are stored against the last user message in the conversation. If a new query is sufficiently similar (above the configured threshold), the cached response is returned.
|
||||||
|
- Re-asking the same question updates the existing cache entry rather than creating a duplicate.
|
||||||
|
- Use the **Clear Semantic Cache** button in settings to remove all stored responses (for example after switching embedding models).
|
||||||
|
|
||||||
## Supported Models
|
## Supported Models
|
||||||
|
|
||||||
Any model supported by Ollama should work, including:
|
Any model supported by Ollama should work, including:
|
||||||
|
|
||||||
- llama3
|
- `llama3`
|
||||||
- llama2
|
- `llama2`
|
||||||
- mistral
|
- `mistral`
|
||||||
- codellama
|
- `codellama`
|
||||||
- etc.
|
- and many more — see [ollama.com/library](https://ollama.com/library)
|
||||||
|
|
||||||
## Development
|
## Development
|
||||||
|
|
||||||
To build from source:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
npm install
|
npm install
|
||||||
npm run build
|
npm run build
|
||||||
|
npm test
|
||||||
```
|
```
|
||||||
|
|
||||||
## Troubleshooting
|
## Troubleshooting
|
||||||
|
|
||||||
- **Connection issues**: Ensure Ollama is running and accessible at the configured URL
|
| Symptom | Likely cause | Fix |
|
||||||
- **Model not found**: Make sure you've pulled the model (`ollama pull <modelname>`)
|
|---------|-------------|-----|
|
||||||
- **Permission issues**: Check that your Obsidian vault has proper write permissions
|
| Cannot connect to Ollama | Ollama is not running | Run `ollama serve` |
|
||||||
|
| Model not found | Model not pulled | Run `ollama pull <model>` |
|
||||||
|
| Semantic cache unavailable (notice shown) | ChromaDB is not running, or the ChromaDB URL is wrong | Start ChromaDB (`chroma run`) and verify the URL in settings |
|
||||||
|
| Cache always misses | Similarity threshold is too high, or the embedding model was changed | Lower the threshold or click **Clear Semantic Cache** and let the cache rebuild |
|
||||||
|
| Slow first response after enabling cache | Embedding model not yet pulled | Run `ollama pull nomic-embed-text` (or the model you configured) |
|
||||||
|
| Permission issues | Vault write permissions | Check that your Obsidian vault has proper write permissions |
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user