Improve VLM client lifecycle, retries, metrics, and cache keys
- Adds abstract aclose() and shared load_prompt() helper to BaseVLMClient. - OllamaClient now logs latency, tokens, and prompt hash per call. - Retries cover HTTP 429 and 5xx in addition to network/timeout errors. - Cache key now includes response_format and max_tokens. - Warns when JSON schema generation falls back to plain 'json' format. - Adds FakeVLMClient.aclose() and cache unit tests.
This commit is contained in:
@@ -52,3 +52,7 @@ class FakeVLMClient(BaseVLMClient):
|
||||
if key in system_prompt or key in user_prompt:
|
||||
return VLMResponse(content=json.dumps(value), model="fake")
|
||||
return VLMResponse(content=json.dumps({}), model="fake")
|
||||
|
||||
async def aclose(self) -> None:
|
||||
"""The fake client holds no resources."""
|
||||
return None
|
||||
|
||||
Reference in New Issue
Block a user