Abstract

Pillzumi tries to introduce a fresh take on AI agent design by focusing on story-driven development instead of relying on static, hard-coded lore and experiences. By using an autonomous storytelling framework, Pillzumi allows AI agents to evolve dynamically, much like how humans change over time. We delve into the architecture and methods used to create AI agents that adjust their context based on experiences within an autonomously generated narrative. We also explore ways to visualize agent memories and discuss how we preserve aesthetics through collaboration with artists. Our goal is to show how AI deliver value in projects by acting as a control layer unlocking new functionalities, more so than just mimic human behavior.

Introduction

Most AI agents today are built on static contexts and fixed knowledge bases, which leads to rigid behaviors that don't adapt over time in their personality. Pillzumi aims to change that by creating AI agents that evolve through an autonomously generated story. Centered around the tale of pills escaping their pharmacy, Pillzumi demonstrates that agent-based projects should be led by stories, allowing for dynamic changes in behavior and context.

Also, by finding novel ways to visualize and contribute to agents' memories, we move beyond traditional character files and opaque models. We focus on aesthetics to ensure that despite the layers of automation, the output remains engaging and artistically rich.

The Landscape of Current Projects

In the realm of innovative projects, there are two predominant types: art-based projects and AI agent-based projects. Art-based projects are incredibly aesthetic, with cohesive narratives and strong visual identities that foster tight-knit communities. The vibes and aesthetics are what hold these communities together.

On the other hand, the AI agent ecosystem is highly fragmented. This fragmentation I believe is largely due to the prevalence of AI slop, as projects attempt to push Large Language Models to their limits, primarily in social contexts. The current incentive structure focuses on extending as far as possible without necessarily considering the importance of aesthetics or cohesive storytelling.

As the initial excitement and hype around AI agents begin to wane, I believe that projects which deliver on aesthetics and maintain a cohesive narrative will stand out. By combining the strengths of art-based projects with advanced AI functionalities, these projects can offer more engaging and meaningful experiences. Pillzumi aims to bridge this gap by integrating a strong aesthetic focus with dynamic, story-driven AI agents.

The Pillzumi Approach / Architecture Overview

Close up of story → season → story line architecture. accessible here: https://www.tldraw.com/s/v2_c_F77I8FZ5pIvk1iW7FxfrP?d=v916.3258.7055.3839.Y-dNOof7cNlcDnZw3N9PO

Pillzumi's architecture revolves around an autonomous storytelling framework that lets AI agents evolve naturally. The story kicks off with general lore for the first season and an opening episode that includes foundational memories. From there, all stories are generated autonomously. The first episode acts as the only seed we need. Instead of seeding context by some random selection mechanism, which is the common approach for social agents, the context never remains the same, due to evolving agent memories and experiences developed through the story, and the evolving story line developed from the narrator agent, and consultant agents.

The narrator agent plays a key role by looking at the last episode, active character arcs, plot threads, the world state, relationships between agents, and events, and relevant memories through semantic search. Based on this, the narrator generates three different episode plots. These plots are reviewed by three consultant agents, who decide by majority vote which plot becomes the next episode.

Once an episode plot is selected, we plan out the scenes and create summaries for each one based on the previous scenes. New characters are given profile pictures and seeded with memories and lore that fit the story. As the story progresses, these characters evolve, updating their memories and traits to reflect their experiences, so they're not static agents.

Draft scenes are sent to the consultant agents for review, where they suggest changes to ensure the story flows well. For dialogue, the narrator generates lines and describes the goals between agent exchanges based on the story summary and scene details. Each agent gets the story summary, scene summary, and preceding dialogue, allowing them to generate contextualized lines for the script.

At the end of each episode, we review the scene summaries and update the agents' memories. Resolved or inactive memories are archived, and new memories from the episode are added. The consultant agents decide if the season concludes based on the generated story and season summary. If it's time for a new season, the narrator generates new options, and the cycle continues.