Skip to content

Conversation

mvanders
Copy link

@mvanders mvanders commented Aug 6, 2025

WHAT:

  • Add OllamaClient implementation for local LLM support
  • Add production-ready Docker compose configuration
  • Add requirements file for Ollama dependencies
  • Add comprehensive integration documentation
  • Add example FastAPI deployment

WHY:

  • Eliminates OpenAI API dependency and costs
  • Enables fully local/private processing
  • Resolves Docker health check race conditions
  • Fixes function signature corruption issues

TESTING:

  • Production tested with 1,700+ items from ZepCloud
  • 44 users, 81 threads, 1,638 messages processed
  • 48+ hours continuous operation
  • 100% success rate (vs <30% with MCP integration)

TECHNICAL DETAILS:

  • Model: qwen2.5:7b (also tested llama2, mistral)
  • Response time: ~200ms average
  • Memory usage: Stable at ~150MB
  • Docker: Removed problematic health checks
  • Group ID: Fixed validation (ika-production format)

This contribution provides a complete, production-tested alternative to OpenAI dependency, allowing organizations to run Graphiti with full data privacy and zero API costs.

Resolves common issues:

  • OpenAI API rate limiting
  • Docker container startup failures
  • Function parameter type mismatches
  • MCP integration complexity

Summary

Brief description of the changes in this PR.

Type of Change

  • Bug fix
  • New feature
  • Performance improvement
  • Documentation/Tests

Objective

For new features and performance improvements: Clearly describe the objective and rationale for this change.

Testing

  • Unit tests added/updated
  • Integration tests added/updated
  • All existing tests pass

Breaking Changes

  • This PR contains breaking changes

If this is a breaking change, describe:

  • What functionality is affected
  • Migration path for existing users

Checklist

  • Code follows project style guidelines (make lint passes)
  • Self-review completed
  • Documentation updated where necessary
  • No secrets or sensitive information committed

Related Issues

Closes #[issue number]


Important

Add Ollama integration for local LLM processing with production Docker setup and FastAPI deployment example.

  • Integration:
    • Add OllamaClient in ollama_client.py for local LLM processing, replacing OpenAI.
    • Supports generate_response, extract_entities, and generate_embedding methods.
  • Docker Setup:
    • Add docker-compose-production.yml for production-ready deployment.
    • Removes problematic health checks, uses startup delay.
  • FastAPI Deployment:
    • Add graphiti_api.py as an example FastAPI server.
    • Implements endpoints: /, /health, /status, /add_memory, /search.
  • Documentation:
    • Add OLLAMA_INTEGRATION.md for setup and benefits of Ollama integration.
  • Dependencies:
    • Add requirements-ollama.txt for Ollama-specific dependencies.
  • Testing:
    • Production tested with 1,700+ items, 100% success rate over 48+ hours.

This description was created by Ellipsis for 36a4211. You can customize this summary. It will automatically update as commits are pushed.

WHAT:
- Add OllamaClient implementation for local LLM support
- Add production-ready Docker compose configuration
- Add requirements file for Ollama dependencies
- Add comprehensive integration documentation
- Add example FastAPI deployment

WHY:
- Eliminates OpenAI API dependency and costs
- Enables fully local/private processing
- Resolves Docker health check race conditions
- Fixes function signature corruption issues

TESTING:
- Production tested with 1,700+ items from ZepCloud
- 44 users, 81 threads, 1,638 messages processed
- 48+ hours continuous operation
- 100% success rate (vs <30% with MCP integration)

TECHNICAL DETAILS:
- Model: qwen2.5:7b (also tested llama2, mistral)
- Response time: ~200ms average
- Memory usage: Stable at ~150MB
- Docker: Removed problematic health checks
- Group ID: Fixed validation (ika-production format)

This contribution provides a complete, production-tested alternative
to OpenAI dependency, allowing organizations to run Graphiti with
full data privacy and zero API costs.

Resolves common issues:
- OpenAI API rate limiting
- Docker container startup failures
- Function parameter type mismatches
- MCP integration complexity

Co-authored-by: Marc <mvanders@github.com>
@danielchalef
Copy link
Member

danielchalef commented Aug 6, 2025

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@mvanders
Copy link
Author

mvanders commented Aug 6, 2025

I have read the CLA Document and I hereby sign the CLA

@mvanders mvanders closed this Aug 6, 2025
@getzep getzep locked and limited conversation to collaborators Aug 6, 2025
@mvanders mvanders reopened this Aug 6, 2025
Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Changes requested ❌

Reviewed everything up to 36a4211 in 2 minutes and 2 seconds. Click for details.
  • Reviewed 504 lines of code in 5 files
  • Skipped 0 files when reviewing.
  • Skipped posting 5 draft comments. View those below.
  • Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.
1. OLLAMA_INTEGRATION.md:47
  • Draft comment:
    Verify the 'Tested by' date; 'August 2025' may appear future-dated.
  • Reason this comment was not posted:
    Confidence changes required: 66% <= threshold 80% None
2. docker-compose-production.yml:60
  • Draft comment:
    End the file with a newline to adhere to best practices.
  • Reason this comment was not posted:
    Confidence changes required: 66% <= threshold 80% None
3. graphiti_core/llm_client/ollama_client.py:171
  • Draft comment:
    Avoid using print for error logging; consider using a logger to record errors.
  • Reason this comment was not posted:
    Confidence changes required: 66% <= threshold 80% None
4. graphiti_core/llm_client/ollama_client.py:143
  • Draft comment:
    Avoid using print for error logging; use a proper logging mechanism.
  • Reason this comment was not posted:
    Confidence changes required: 66% <= threshold 80% None
5. requirements-ollama.txt:14
  • Draft comment:
    Consider removing 'asyncio' from dependencies since it's part of the Python standard library.
  • Reason this comment was not posted:
    Confidence changes required: 66% <= threshold 80% None

Workflow ID: wflow_wol7wDd4w1nzfL4T

You can customize Ellipsis by changing your verbosity settings, reacting with 👍 or 👎, replying to comments, or adding code review rules.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants