feat(backend): Extract backend copilot/chat enhancements from hackathon

[Swiftyos]

This PR extracts backend changes from the hackathon/copilot branch, adding enhanced chat capabilities, agent management tools, store embeddings, and hybrid search functionality.

Changes 🏗️

Chat Features:

• Added chat database layer (db.py) for conversation and message persistence
• Extended chat models with new types and response structures
• New onboarding system prompt for guided user experiences
• Enhanced chat routes with additional endpoints
• Expanded chat service with more capabilities

Chat Agent Tools:

• agent_output.py - Handle agent execution outputs
• create_agent.py - Tool for creating new agents via chat
• edit_agent.py - Tool for modifying existing agents
• find_library_agent.py - Search and discover library agents
• Enhanced run_agent.py with additional functionality
• New models.py for shared tool types

Store Enhancements:

• embeddings.py - Vector embeddings support for semantic search
• hybrid_search.py - Combined keyword and semantic search
• backfill_embeddings.py - Utility for backfilling existing data
• Updated store database operations

Admin:

• Enhanced store admin routes

Data Layer:

• New understanding.py module for agent understanding/context

Database Migrations:

• add_chat_tables - Chat conversation and message tables
• add_store_embeddings - Embeddings storage for store items
• enhance_search - Search index improvements

Checklist 📋

For code changes:

• I have clearly listed my changes in the PR description
• I have made a test plan
• I have tested my changes according to the test plan:
  • Chat endpoints respond correctly
  • Agent tools (create/edit/find/run) function properly
  • Store embeddings and hybrid search work
  • Database migrations apply cleanly

For configuration changes:

• .env.default is updated or already compatible with my changes
• docker-compose.yml is updated or already compatible with my changes
• I have included a list of my configuration changes in the PR description (under Changes)

Summary by CodeRabbit

Release Notes

• New Features
  • Added persistent chat session management with session history and listing.
  • Introduced onboarding flow for new users to capture business context.
  • Launched tools for agent creation, editing, discovery, and library search.
  • Added documentation search and business understanding tracking capabilities.
  • Enabled automatic session title generation from first user message.
  • Introduced streaming chat with context support and agent output retrieval.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}