Design feedback on a provider-agnostic multi-agent framework in Python

[fa8i]

Hi everyone,

I’m looking for architectural feedback on a Python project I’ve been building around multi-agent AI systems.

While working with agent-based workflows, I often ran into issues when trying to scale prototypes into more structured setups: tight coupling to specific LLM providers, ad-hoc orchestration logic, and difficulty composing agents in a clean way. As an experiment, I started building a small framework to explore alternative design choices.

The current design focuses on:

• provider-agnostic LLM clients
• explicit agent and tool abstractions
• simple composition patterns (pipelines, teams, hierarchical agents)
• pluggable memory backends

At this stage, I’m less interested in promotion and more in understanding whether these abstractions make sense from a software design perspective.

In particular, I’d appreciate feedback on:

• whether this feels over-engineered or under-specified
• what abstractions you would avoid or simplify
• how you usually approach orchestration and memory in multi-agent systems

If useful, the code is here:
https://github.com/fa8i/Agentify

Thanks for any thoughts or discussion!

[krishparmar22242]

This is a fantastic project and a great example of a "developer-first" approach to the multi-agent problem. After looking through the repository, it is clear that you have put significant thought into avoiding the "bloat" that often comes with larger frameworks like LangChain.

1. Over-engineered vs. Under-specified
   The framework currently sits in a "Sweet Spot." It is not over-engineered because you’ve avoided the complex "black-box" wrappers seen in larger frameworks; your use of standard Python protocols makes the logic easy to follow. However, it feels slightly under-specified regarding "State Handoff." While you have pipelines, the protocol for how one agent's structured output becomes another agent's validated input could be more explicit. Adding a shared "State" or "Context" object that travels with the execution flow would make the transition from prototype to production much more robust.

2. Abstractions to Avoid or Simplify
   I would suggest being cautious with the "Hierarchical Agent" as a base abstraction. If an agent is responsible for both "doing work" and "managing other agents," the class becomes bloated and difficult to unit test. I would recommend simplifying the Agent to be a pure "executor" and keeping the "Hierarchy/Orchestration" logic strictly within a Coordinator or Team class. This separation of concerns makes it much easier to debug why a specific agent in a complex chain is failing.

3. Approach to Orchestration and Memory
   Orchestration: In my experience, the most scalable approach is a State Machine or a Directed Acyclic Graph (DAG). Instead of agents calling each other, a central orchestrator manages the "Turn Taking." This prevents infinite loops and makes it easier to implement "Human-in-the-loop" checkpoints, where a person can approve an agent's plan before it executes.
   Memory: For large-scale systems, I recommend a Layered Memory approach.
   Short-term: A sliding window of the last 5-10 messages (Buffer).
   Long-term: Vector database for "searchable" history.
   Semantic: A "Summarizer" layer that creates a condensed narrative of the conversation so far. This keeps the token count low while preserving the "Soul" of the interaction.
   Overall Verdict
   You have built a very coherent system that respects the developer's intelligence. The "Provider-Agnostic" layer is the project's strongest asset. By refining the "State Management" between agents, you'll have a very competitive alternative to the heavier frameworks currently on the market.

If this architectural feedback helps you refine the roadmap for Agentify, please consider marking this as the accepted answer so I can keep supporting the community!

[fa8i]

Hi Krish,

Thank you for the feedback! It’s really encouraging to hear that the "developer-first" approach and the effort to avoid framework bloat are coming through clearly.

I wanted to touch on a few of your specific suggestions, as they align very closely with where I want to take the project (and where some of it actually is today!):

1. Hierarchical Abstractions: You raised a great point about keeping agents as "pure executors" and moving management logic to a Coordinator. This aligns perfectly with the implementation of HierarchicalTeam. It serves as a separate coordinator class that configures the connectivity between agents (registering children as tools), rather than a "SuperAgent" class that muddies the inheritance. This ensures the agents themselves remain simple and focused.

2. State Handoff: You hit the nail on the head here. Currently, the framework relies heavily on string-based handoffs (Pipeline input/output). I completely agree that for production use cases, a structured "State" or "Context" object that travels with the flow is necessary to handle validations and richer data types. This is definitely a priority for the next iteration.

3. Orchestration & Memory: Your suggestion for a Layered Memory approach is spot on. We currently support Short-term buffering via policies and Long-term searchable history via ElasticsearchStore, but we are missing that middle "Semantic/Summarizer" layer to keep the context "soulful" without blowing up token limits. Also, moving towards a DAG-based orchestrator for better "Turn Taking" control and Human-in-the-loop checks is a next step that I’m very keen to explore.

Thank you again for the high-quality review. It’s exactly the kind of "battle-tested" advice that helps steer a project in the right direction!

Best, fa8i

[Liwyd]

Hi Fa8i,

Thanks for sharing this — really enjoyed reading through the README and skimming the code. You’ve clearly put thought into the pain points (tight coupling, ad-hoc orchestration, composition friction), and it shows. As someone who’s built a few multi-agent prototypes that quickly turned into spaghetti, I totally feel the motivation here.

A few thoughts — mostly high-level, design-oriented — in no particular order:

---

✅ What feels right

• Provider-agnostic LLM interface: This is table stakes, but you’ve done it cleanly — BaseLLMClient with async methods and unified config (e.g., LLMConfig) avoids the usual trap of “just pass **kwargs everywhere”. Good call isolating model-specific quirks into adapters (e.g., OpenAILLMClient, OllamaLLMClient).

• Explicit separation of Agent / Tool / Memory / Orchestrator: Even if some abstractions evolve, having them named and distinct from day one prevents the “agent does everything” blob. I especially like how Tool isn’t just a function, but a first-class object with metadata (name, description) — that’s essential for self-discovery/tool calling workflows.

• Memory as a pluggable strategy, not a side effect: So many projects bake memory into the agent class or rely on global state. Your BaseMemory + concrete backends (InMemory, Redis) + optional per-agent override is a solid pattern. Bonus points for ConversationMemory already thinking about turn structure.

---

Where I’d probe deeper

1. Agent ≠ Runnable?
   Right now, Agent is mostly a config/identity wrapper, while run() lives in orchestrators (e.g., PipelineOrchestrator.run_agent). I wonder if Agent should itself be a minimal Runnable (e.g., async def invoke(self, input: Message) -> Message) — even if it delegates to tools/memory internally. That way, agents could be composed outside orchestrators too (e.g., as LangChain-style chains or LlamaIndex nodes). Might increase flexibility without much cost.

2. Orchestrator granularity
   PipelineOrchestrator, TeamOrchestrator, HierarchicalOrchestrator are clear conceptually — but I suspect real-world usage will blur these lines (e.g., a “team” inside a pipeline step). Have you considered a composable orchestrator model? Like:

   pipeline = Pipeline(
       step1=Agent(...),
       step2=Team(members=[a, b], strategy=Debate()),
       step3=SubAgentTree(root=manager, children=[dev, reviewer])
   )

   Not necessarily suggesting this exact API, but just flagging that rigid categories might constrain later. (Your current CustomOrchestrator is a good escape hatch, though!)

3. Tool binding & lifecycle
   Tools are registered per-agent, which is fine — but how do you handle:

   • Tools that need async setup/teardown (e.g., DB connections)?
   • Tools that require auth context (e.g., per-user API keys)?
   • Tools that produce side effects (e.g., send email) — do you want explicit “effect” tracking or sandboxing?
     This might be out of scope for v0, but it’s where many abstractions crack under pressure.

---

How I’ve approached this (for comparison)

In past projects, I’ve leaned heavily on:

• Message-passing as the core protocol (you’re doing this well with Message and roles).
• State machines for orchestration (e.g., SupervisorAgent emits route_to: agent_name, and a router drives next step). Less declarative than pipelines, but more debuggable.
• Memory versioning / snapshots — especially when agents backtrack or retry. Redis + msgpack worked, but I wish I’d built a MemorySession abstraction earlier.

One anti-pattern I fell into: trying to make agents too autonomous early on (e.g., letting them call other agents directly). Ended up with cyclic dependencies and race conditions. Your explicit orchestrator layer helps avoid that — smart.

---

Over-engineered or under-specified?

Honestly? Neither.
It’s at that sweet spot: small enough to evolve rapidly, but with enough structure to prevent collapse at ~5 agents. The abstractions feel minimal but intentional — not YAGNI, not cargo-culted.

If anything, I’d simplify just one thing for now:
→ Consider merging AgentConfig and Agent (or making Agent immutable + constructed via config). Right now there’s a bit of duplication (e.g., name, role live in both Agent and its config). Probably not worth a refactor yet — just something to watch as the API stabilizes.

---

Really excited to see where this goes. If you end up writing a short design rationale (even just a gist) — e.g., “why not use LangChain/Marvin/crewAI?” — I’d love to read it. Happy to test-drive a prototype or give feedback on a specific module if helpful.

Cheers,
Ali

[krishparmar22242]

If this architectural feedback helps you refine the roadmap for Agentify, please consider marking this as the accepted answer so I can keep supporting the community!

[fa8i]

Hi Liwyd,

Thank you so much for the thoughtful feedback! I appreciate you taking the time to dive into the codebase and share your experience with multi-agent systems.

I wanted to clarify a few points you raised, as some of the features you mentioned are actually already built into the core design:

1. Agent as a Runnable You mentioned that Agent appeared to be mostly a config wrapper, but BaseAgent is actually designed as a fully standalone Runnable. It implements comprehensive run() and arun() methods that handle the entire execution loop including memory interaction, LLM calls, and tool execution independently. The orchestrators (like Pipeline or Team) delegate to these agents rather than driving their internal state.

2. Orchestrator Composability Regarding the orchestrator structure, SequentialPipeline is already fully composable. It allows you to pass not just BaseAgent instances, but also Team objects or even other SequentialPipeline instances as steps. The API supports the exact nested structure you suggested (e.g., a Pipeline containing a Team) out of the box.

3. Tool Lifecycle This is a spot-on observation. You are absolutely right that the current Tool implementation is lightweight and lacks explicit lifecycle hooks for things like async setup/teardown or context injection. This is a very valid point and something I plan to address to support more complex integrations.

4. Agent vs. AgentConfig Currently, BaseAgent accesses attributes like name directly from self.config to avoid state duplication, but I agree that keeping the API surface clean is important. I’ll keep an eye on this as the project evolves.

Thanks again for the excellent insights, especially regarding the tool lifecycle patterns. I’m always open to more feedback!

Best, fa8i

[vikassinghz]

Great question! Your provider-agnostic framework is a clever approach. Here are my thoughts:

Strengths:

1. Clean Abstraction: The separation of LLM clients from agent logic is excellent. It avoids vendor lock-in and makes testing easier.

2. Composition Over Inheritance: Your pipelines and team abstractions allow flexible agent composition without inheritance bloat.

3. State Management: Explicit state objects are better than implicit context passing through kwargs.

Suggestions:

1. Protocol for Coordination: Consider a formal message protocol between agents. Maybe a TypedDict or Pydantic model defining message structure would make multi-agent communication more reliable as complexity grows.

2. Memory Scaling: For production, you might want to explore vector databases (like Pinecone) alongside your current memory approach for semantic search capabilities.

3. Error Handling & Retry Logic: Add a robust retry mechanism with exponential backoff for LLM calls, since they can be flaky.

4. Monitoring: Consider adding structured logging with timestamps and token counts to track performance across different providers.

Overall: Your design respects developer ergonomics while maintaining flexibility. This is a solid foundation for a production multi-agent system!