From Idea to Co-Creation
Research 2026

From Idea to Co-Creation

Planner-Actor-Critic Framework for Agent-Augmented 3D Modeling
A multi-agent 3D modeling framework where planner, actor, and critic agents iteratively reason, execute, evaluate, and improve Blender outputs with human-in-the-loop supervision.

Abstract

We present a framework that extends the Actor-Critic architecture to creative 3D modeling through multi-agent self-reflection and human-in-the-loop supervision. While existing approaches rely on single-prompt agents that directly execute modeling commands via tools like Blender MCP, our approach introduces a Planner–Actor–Critic architecture.

In this design, the Planner coordinates modeling steps, the Actor executes them, and the Critic provides iterative feedback, while human users act as supervisors and advisors throughout the process. Through systematic comparison between single-prompt modeling and our reflective multi-agent approach, we demonstrate improvements in geometric accuracy, aesthetic quality, and task completion rates across diverse 3D modeling scenarios.

Demonstration of the Planner–Actor–Critic architecture in action within Blender.

Key Contributions

  • Multi-Agent Architecture: A novel framework that mimics human design iterations using specialized roles (Planner, Actor, Critic).
  • Geometric Accuracy: Significant reduction in modeling errors through critic-guided self-reflection.
  • Human-in-the-Loop: Real-time synchronization allowing human supervisors to guide and advise agents during the modeling process.
  • Blender Integration: Full modeling flexibility leveraging Blender's API and the Model Context Protocol (MCP).
Agent Co-Creation Diagram
The collaborative loop between human and AI agents

1. Introduction

Recent advances in large language models (LLMs) have demonstrated impressive capabilities across various domains, from natural language processing to creative content generation. However, a significant gap exists between AI's ability to generate unstructured content (text, images) and its capacity to produce structured, engineered outputs such as functional 3D models or CAD designs. While generative AI excels at direct output production, professional engineering workflows require an iterative process involving ideation, prototyping, evaluation, and refinement—steps that demand both creativity and technical precision.

This paper introduces a Planner–Actor–Critic (PAC) framework that bridges this gap by enabling AI agents to participate in engineering design processes through structured reasoning and tool use. Our approach draws inspiration from reinforcement learning's actor-critic architecture while adapting it for creative engineering tasks where "rewards" are not numeric scores but qualitative assessments of design coherence, functionality, and aesthetic quality.

2. Framework Architecture

The Planner–Actor–Critic (PAC) framework orchestrates three specialized AI agents to enable collaborative 3D modeling. Each agent operates through a distinct LLM instance with specialized prompts and tool access:

  1. The Planner Agent generates high-level strategies through chain-of-thought reasoning, decomposing complex design objectives into sequences of specific modeling operations.
  2. The Actor Agent translates these plans into executable actions within the modeling environment, utilizing the Model Context Protocol (MCP) to operate modeling tools.
  3. The Critic Agent evaluates intermediate results, ensures logical consistency across operations, and provides feedback to guide subsequent actions, creating a self-correcting design loop.
System Architecture Detail
Detailed view of the Planner, Actor, and Critic orchestration

3. Results and Evaluation

Our evaluation reveals that critic-guided reflection, combined with human supervisory input, increases the complexity and quality of the results compared to direct single-prompt execution. This work establishes that structured agent self-reflection produces higher-quality 3D models while maintaining efficient workflow integration.

Co-creation workflow

Human review, planning, execution, critique, and refinement across PAC iterations

Human review of the generated plan

Human review actions

3.1 Quantitative Metrics

We evaluated the framework's performance across 30 design tasks of varying complexity, measuring:

  • Success Rate: Simple tasks achieved 93% success, while complex tasks (16+ operations) reached 47%, significantly outperforming single-prompt baselines.
  • Iteration Counts: 77% of tasks reached success within 3 iterations through the Critic-guided loop.
  • Reasoning Transparency: User studies confirmed that the explicit planning phase provides interpretable strategies that users can review and modify.
Performance Metrics
Comparison of success rates and iteration efficiency

4. Conclusion

This research demonstrates a viable path toward AI-augmented engineering design that augments rather than replaces human creativity. By positioning AI as a collaborative partner that proposes strategies and automates routine operations while maintaining human oversight, the framework offers a pragmatic model for human-AI co-creation in professional contexts.