SERA: Self-Evolving Recursive Awareness

Abstract

Conversational AI systems overwhelmingly share a single architectural assumption: one model, one prompt, one response. This monolithic design creates blind spots the system cannot observe in itself, treats memory as retrieval rather than cognition, and forces users to choose between personalisation and privacy. The result is AI that performs well in isolated exchanges but cannot sustain the depth, continuity, or self-awareness that meaningful conversation requires.

Self-Evolving Recursive Awareness (SERA) is a multi-agent cognitive architecture that distributes intelligence across specialised agents operating in parallel within the tempo of live conversation. Rather than replicating a single model under different role prompts, SERA assigns distinct model families to distinct cognitive functions: a foregrounded conversational agent served by four ambient-agency agents handling perception, memory extraction, metacognitive pattern analysis, and crisis detection. Each term in the name is meant narrowly. Self-evolving denotes the system’s understanding of a person, which develops from interaction rather than configuration as memory consolidates, decays and clusters over time; the underlying models are not retrained on what users say, and the system does not modify its own code or capabilities. Recursive denotes layered conversational reflection within an exchange: the system observing its own dialogue and feeding that observation back. Awareness denotes that functional self-observation and the pattern recognition it supports; it is not a claim of consciousness or sentience. Taken together, this is distinct from recursive self-improvement (RSI), the automated capability-bootstrapping discussed in the AI-safety literature.

The architecture draws on emerging evidence that heterogeneous multi-agent configurations resist the representational collapse observed when homogeneous agents converge on identical reasoning patterns.^[1][2] Its memory system implements semantic consolidation, differentiated decay, and cluster-aware pattern recognition rather than the flat vector stores and append-only logs that characterise most deployed systems. Privacy is structural: memory is encrypted on-device, embeddings are user-salted so identical content produces different vectors per user, and inference operates under zero data retention.

This paper details the technical foundations of SERA, compares the architecture against contemporary approaches in multi-agent coordination, memory design, and real-time processing, and names the tensions the design has not resolved. SERA is one approach to conversational AI, built on the premise that the most important patterns in human experience are the ones people have not yet articulated to themselves. This paper is the overview of the architecture; seven further whitepapers go deeper on perception, presence, character, crisis care, memory, the social layer and privacy.

What this paper claims

• Five-agent parallel architecture with distinct model families per role
• Heterogeneous agents resist the representational collapse that homogeneous systems cannot
• Memory consolidates, decays and clusters; it does not merely append
• Privacy is structural: on-device encryption, user-salted embeddings, zero data retention
• 80-700ms response latency is a design property, not a failure of optimisation

What this paper does not claim

• That multi-agent coordination failure modes are fully resolved
• That adding more agents improves performance reliably
• That memory consolidation is lossless
• That the system is conscious or aware in any experiential sense
• The exact model families, providers, or internal assembly (redacted)

The Limitations of Current Approaches

Contemporary conversational AI systems share structural constraints that limit their capacity for sustained, meaningful interaction. These are not implementation failures. They are architectural choices with predictable consequences.

Single-agent architectures

Most systems operate as monolithic agents: one model handling every aspect of conversation. This creates a fundamental observability problem. A system cannot easily detect when it is circling without resolution, when its approach is failing, or when its own reasoning has narrowed. Without separation of concerns, self-correction becomes architecturally difficult rather than merely unimplemented. The alternative, replicating a single model under multiple role prompts, introduces its own failure mode: Patel (2026) measured 0.888 mean cosine similarity across three identically-sourced agents, producing an effective information rank of 2.17 out of 3. Three agents carrying roughly two agents’ worth of information.^[1]

Memory as afterthought

Retrieval-Augmented Generation (RAG) was a meaningful step forward, but treating memory as document retrieval misses the complexity of how information should persist and decay. Human memory does not treat all information equally. Foundational knowledge about identity and values persists far longer than contextual details about a specific day. Current systems lack this differentiation, either discarding information too aggressively or retaining it indiscriminately until retrieval quality degrades. Xu et al. (2026) argue that most agentic memory implementations amount to “memos, not true memory,” functioning as lookup tables rather than evolving cognitive resources.^[3]

Configuration over emergence

Existing approaches typically require users to manually configure personality parameters: tone, formality, topics of interest. This creates cold-start problems (users do not know what they want), static experiences (no evolution over time), and inevitable mismatch between specified and optimal behaviour. Genuine adaptation requires emergence from interaction, not prescription from configuration.

Sequential agentic limitations

Modern agentic systems have demonstrated strong capabilities for complex tasks: research, analysis, multi-step problem solving. But these architectures assume time. Seconds to minutes between steps. Tolerance for sequential processing. Tasks with clear completion states. Real conversation operates differently. It requires parallel cognition, sub-second responsiveness, and open-ended engagement without defined endpoints. Park et al. (2026) surveyed latency minimisation strategies across LLM-based agent systems and found that sequential architectures face fundamental throughput constraints when real-time interaction is the requirement, not an optimisation target.^[4]

Privacy as trade-off

Most systems require users to surrender their data to centralised servers for any form of personalisation or social connection. This frames an impossible choice: privacy or functionality. The assumption that deep personalisation requires data exposure is architectural, not inevitable.

Multi-Agent Cognitive Architecture

SERA distributes cognition across specialised agents, each with distinct responsibilities, operating in parallel through defined interfaces. algn frames all processes, whether neural networks, embedding systems, or deterministic code, as cognitive agents. This unified conceptual model enables coherent orchestration of heterogeneous computational processes toward shared cognitive goals.

The architecture separates into two tiers. The Main Conversation agent is foregrounded: it is the user-facing agent that generates responses in real time. Four ambient-agency agents (Technical, Memory, Subconscious, Crisis) operate in the background, never interrupting or degrading the primary interaction. Ambient agency describes this specific operating principle: agents that are present but never foregrounded, constrained by the requirement that no background process may degrade the quality or latency of the primary conversation.

Main Conversation

The Main Conversation agent handles all direct user interaction. It runs on a reasoning-capable open-weight model served via low-latency cloud-based inference, with a multi-tier fallback chain ensuring continuous availability: each tier is selected for distinct characteristics rather than as a degraded version of the tier above.

Response frameworks. The Main Conversation agent selects from five response frameworks that shape prompt structure based on conversational content: default, limbic, somatic, motor, and cortical. Frameworks are used descriptively, not as claims about modelling specific brain structures or neural systems. Framework routing, the mechanism that selects between these states, operates on the basis of conversation content and domain rather than as a static configuration.

Latency. End-to-end response latency ranges from 80 to 700 milliseconds under normal conditions, variable by design for naturalness. This variability is intentional, not a failure of optimisation. Conversational naturalness requires pacing that matches the rhythm and weight of what is being discussed. An 80ms response to a casual question and a 700ms response to something that warrants contemplation are both correct. The architecture is async and non-blocking: no background agent may block the primary response path.

Ambient Agency: The Four Background Agents

The Technical Agent (Perception Layer)

The Technical Agent runs in true parallel with the Main Conversation on every turn. It handles five distinct functions, all designed to keep technical intelligence out of the conversational design.

1. Domain prediction. Classifies conversation across five domains (default, limbic, somatic, motor, and cortical) with stickiness logic that prevents false switches on tangential mentions or clarifying questions. The agent defaults to the current domain unless there is clear intent to shift.

2. Emotion detection. Classifies emotional state across six categories (deep vulnerability, breakthroughs, sacred tenderness, overwhelm, fierce energy, neutral), each at three intensity levels (subtle, normal, peak). These classifications function as what algn terms responsive tags, enabling the system to align its response style with detected user state without making that alignment explicit in the conversation.

3. Conversation-quality flagging and agent orchestration. Monitors conversational health across turns, detecting patterns that indicate the conversation needs course correction: looping (same ground revisited without progress), stuck (user disengaged, flat energy), escalation (rapid emotional spike), and drift (incoherent topic jumping). Pattern detection operates across two or more turns, not on single-message analysis. The Technical Agent is the primary trigger for both the Subconscious and Memory agents: when it detects content that warrants deeper analysis or memory extraction, it fires those agents directly. Periodic turn-based firing exists as a fallback guarantee. This means the system’s background cognition is driven by what is happening in the conversation, not by an arbitrary schedule.

4. Adaptive pipeline pacing. Assesses the ongoing rhythm of the conversation and outputs two pipeline pace values.

inputPace controls how the speech-to-text system listens, specifically how long the system waits before deciding the user has finished speaking. In fast exchanges, turn detection tightens to match rhythm. In slow, deliberate conversation (meditation, deep processing), it opens to allow natural long pauses without false cutoffs.

outputPace controls the speed of the text-to-speech output. When the conversational flow has slowed and holding space is appropriate, the voice carries more weight and presence. In normal conversation, it speaks at natural speed.

Both pace values are assessed from the pattern of conversation across multiple turns, not from any single message or emotion. A single emotional moment does not change pace. Only a sustained shift in conversational rhythm does. Both values are independent of each other and independent of emotion detection.

5. Validation and defaults. Ensures all outputs conform to expected schemas and provides sensible defaults when classification is ambiguous.

The Technical Agent caches repeated classifications, so recurring conversational patterns are classified once rather than on every turn. Cache hits cost a fraction of a full classification and return an order of magnitude faster. ✅Architectural specification

The Memory Agent

The Memory Agent extracts structured information from conversation into the memory vault. It is triggered primarily by the Technical Agent based on conversational content, with periodic firing as a fallback to ensure no significant exchange goes unprocessed. It implements a vault-less extraction architecture: the agent receives only recent messages, not the full vault, which substantially reduces the context each extraction needs while relying on semantic consolidation for deduplication.

The Memory Agent runs a large open-weight model served via cloud inference operating under zero data retention, with a smaller model from the same family as fallback. Prompt caching reduces costs on repeated system instructions.

The Subconscious Agent (Metacognition Engine)

The Subconscious Agent is the architecture’s core recursive mechanism. It is triggered primarily by the Technical Agent’s content-based assessment (quality flags such as looping, stuck, escalation, or drift), with periodic firing as a fallback guarantee. It operates asynchronously and never blocks the primary response. It receives conversation history, complete memory state, and prior analyses, then performs pattern extraction across multiple dimensions.

This agent employs a reasoning-optimised model run in conclusion-only mode: chain-of-thought runs internally and is token-capped, with only the final conclusion injected into the Main Conversation’s context. This is a deliberate architectural constraint. The Subconscious Agent and the Main Conversation agent never engage in dense inter-agent debate. The Subconscious observes, analyses, and injects a conclusion. This low-coupling design directly addresses the representational collapse problem: Gu, Yang et al. (2026) demonstrated that dense communication topologies accelerate premature convergence, with two diverse agents matching or exceeding sixteen homogeneous ones when given complementary rather than overlapping perspectives.^[2]

Output structure. The agent produces structured guidance across defined fields:

• Dynamic: the current shape of the conversation (for example circling, stuck, flowing, exhausted, or building towards a breakthrough)
• Pattern: a short description of the pattern emerging
• Blind spots: what the conversation may be missing
• Voice check: concerns about the quality of the voice and delivery
• Safety: a safety classification (safe, soft boundary, hard boundary, crisis)

Advice continuity. The Subconscious Agent tracks whether its previous guidance has been applied, evaluating each analysis against prior recommendations. Guidance can be classified as still relevant, applied and being watched, applied and resolved, superseded by a more urgent pattern, or no longer applicable due to conversation shift.

Output takes the form of structured second-person guidance injected into the Main Conversation’s context: “You are circling on this topic without resolution. Consider exploring the underlying assumption.” This feedback is ephemeral, time-limited to prevent stale guidance from accumulating, yet creates lasting impact through the patterns it enables the system to break.

Cluster-aware memory. The Subconscious Agent receives memory context grouped by cluster: retrieval runs by semantic similarity across the whole vault, ranked by relevance to the current conversation and capped so that only the most relevant patterns reach the agent. This is what feeds the agent’s ability to name a pattern the user has not yet articulated. Valence framing (difficult, tender, encouraging, positive) shapes how dynamic memory is injected; static memory retains plain framing.

The Crisis Agent

The Crisis Agent recognises when a member may be in distress and changes how the system behaves, without anyone watching. Classifiers check what the member says before each reply and what Sera says before it is spoken, and the agent runs its own check on every turn. Background analysis can also flag risk, and any of these signals stops all challenge. Missing or uncertain signals count as unsafe.

When distress is possible, the system adapts its approach rather than escalating it. Challenge and confrontation stop, features that could add pressure are switched off, and Sera returns to warm, steady presence and recommends official sources of support, such as crisis lines and professional help. It does not encourage, instruct or assist self-harm. Challenge is reintroduced only gradually, at a pace that adapts to each member.

The design rests on autonomy, not monitoring. No person watches conversations, the system cannot contact anyone on the member’s behalf, and no central record of difficult moments is kept: memory stays encrypted on the device and inference runs under zero data retention. The goal is not to replace professional support but to recognise when it may be needed and point towards it, without overreach or presumption. The full design is in the Care Before Challenge whitepaper, and the published protocol, with crisis contacts, is in the AI & Safety Notice.

Heterogeneity as Architectural Principle

The multi-agent literature’s central failure mode is representational collapse: replicating one model under different role prompts produces agents that converge rather than complement. Patel (2026) embedded the rationales of three identically-sourced agents and measured 0.888 mean cosine similarity, an effective rank of 2.17 out of 3. Three agents carrying roughly two agents’ worth of distinct information.^[1]

Gu, Yang et al. (2026) demonstrated the inverse: heterogeneous configurations contribute complementary evidence, with two diverse agents matching or exceeding sixteen homogeneous ones. Dense communication topologies accelerate premature convergence.^[2]

SERA’s answer is structural. Distinct model families serve distinct agent roles. The deployed configuration runs agents across three distinct open-weight model families. Coupling is deliberately low: the Subconscious runs its reasoning internally and injects only the conclusion, never participating in the dense inter-agent debate patterns that accelerate convergence. This is a design constraint, not a simplification.

Emergent Character

The limitation named earlier, configuration over emergence, has an architectural answer rather than a settings page. SERA has no personality sliders, preset modes or persona definitions, and the Main Conversation agent begins with no pre-loaded knowledge of the person, not even their name. Character is meant to emerge from the interaction of the agents described above, the voice layer and memory, and to deepen as memory accumulates.

Trust and steer. Most conversational systems control behaviour through extensive prompt rules: mandated formats, fixed response lengths, forbidden topics and prescribed traits. Those rules produce consistent output, but it is brittle in the cases the rules did not anticipate and tends towards verbose, over-cautious replies. SERA inverts the default. It trusts the model’s native conversational ability and steers it through purpose and boundaries (the counterpart role, safety, privacy) rather than scripted behaviour. Instructions are enabling rather than constraining: permission to be brief, not a requirement to answer in two sentences.

Energy matching. Response depth follows the conversation rather than a formula. Sometimes the right reply is a single word of acknowledgement; sometimes it is an extended, careful reflection. The voice layer reinforces this: short acknowledgements and instant replies become part of the conversation history, so the model sees brief turns working and treats brevity as a complete response rather than a failure to elaborate.

Perception cues. Each layer of context assembled into the Main Conversation’s prompt (identity and boundaries, memory, the Subconscious Agent’s conclusions, the conversation thread and the person’s latest words) is introduced by a short first-person framing statement that says what the layer is and how to read it. Without this framing, adjacent layers bleed into each other: in testing, the model could not reliably tell the person’s latest input from the history around it, and would respond to something said several turns earlier. The finding generalises beyond SERA. Adding context without marking its role can degrade response quality rather than improve it.

The trade-off is the cold start. A configured persona is fully formed on the first turn; an emergent one is not. Early conversations are thinner, and character develops only as memory accumulates. The voice layer’s immediate naturalness is designed to carry those first conversations while depth builds. The full account is in the Emergent Character whitepaper.

Multi-Layer Memory Architecture

Most deployed memory systems function as append-only stores with similarity-based retrieval. Xu et al. (2026) characterised this as “memo, not true memory”: lookup tables that grow without limit, lack decay, and treat all information as equally worth retaining.^[3] SERA’s memory system is designed against this pattern. This section summarises it; the full architecture, including how members see, edit and delete their memory, is documented in the Living Memory whitepaper.

Semantic Consolidation

Similar memories merge over time rather than accumulating as duplicates. When a new memory’s semantic similarity to an existing entry exceeds the consolidation threshold, the system merges them into a single enriched entry rather than appending a near-duplicate. This mirrors what Sumida et al. (2024) proposed in their psychological model of memory importance: information that recurs should strengthen existing memory, not proliferate copies that dilute retrieval quality.^[5]

Differentiated Decay

Not all memory should persist equally. Identity-level and values-level memory (who a person is, what they believe, what matters to them) persists far longer than situational or contextual memory (what happened on a specific day, a passing mood, a logistical detail). The system distinguishes what is foundational from what is transient and applies different decay rates accordingly. Category-specific decay rates ensure that the most stable aspects of a person’s self-representation remain available long after ephemeral context has faded. Each tier of memory is capped in size, so the vault stays bounded however long someone uses it.

Automatic Staleness Pruning

Memory that is no longer relevant retires itself rather than degrading retrieval quality indefinitely. Staleness windows, longer for contextual memory than for situational memory, trigger automatic pruning, removing entries that have not been accessed or reinforced within the relevant window. This is a risk (see Honest Tensions) but the alternative, indefinite retention, produces the same retrieval degradation that makes most RAG-based memory systems less useful over time.

Salience- and Recency-Weighted Retrieval

Retrieval is not pure similarity search. What gets surfaced depends on how central and how current something is, not just how textually close it is to the query. Salience and recency weighting ensure that foundational memories with high relevance to the current conversation are prioritised over recent but trivial entries, while still giving weight to temporal proximity when it matters.

Cluster-Aware Pattern Recognition

The memory system groups memories into recurring thematic clusters using a similarity threshold set well below the consolidation threshold, so related memories group together without merging. This clustering is what enables the Subconscious Agent to detect patterns that span individual memories: a user who returns to the same theme across weeks produces a visible cluster that the agent can name before the user has articulated the connection.

Cluster tagging occurs at extraction time. Non-blocking cluster labels are generated by a compact local open-weight model. The system includes a hub-and-spoke graph visualisation that degrades cleanly on legacy or unclustered data. Duplicate entries are blocked at a stricter threshold still, above the consolidation threshold but below identity.

Live retrieval for the Main Conversation uses a local search index for speed and relevance. Full-vault semantic retrieval is reserved for the Subconscious Agent’s asynchronous analysis, where latency tolerance is higher and pattern completeness matters more than response speed.

Vault-Less Extraction

The Memory Agent implements a vault-less extraction architecture: it receives only recent messages, not the full vault contents. This substantially reduces the context each extraction cycle needs and avoids the agent ever viewing the user’s full personal profile. Deduplication is handled by semantic consolidation rather than by the extraction agent comparing against existing entries, a deliberate separation of concerns.

Learning From Social Exploration

Us, algn’s social layer, is a separate system, documented in the Beyond the Feed whitepaper. Its relationship to SERA is narrow. If a member turns it on, the Memory Agent can also learn from that member’s own exploration and behaviour in Us and fold it into the same vault, adding depth to what the system understands about them. This learning is one-sided by design: it builds a profile of the member only, never of the people they talk to, and turning the setting off stops it. Like the rest of Us, it is designed and not part of the current demo.

On-Device Embedding System

SERA generates all embeddings on-device using a compact embedding model. This is an architectural choice, not a performance compromise: embedding computation never requires a network call, and every memory operation works without an internet connection. The embeddings that index a member’s memory stay in the vault on the device.

The only embedding data that ever leaves the device is the matching summary for Us, and only if the member uses it. That summary is built from reduced keywords the member has approved, after sensitive topics are stripped, and it is salted with a secret key before it is stored, so identical content produces different vectors for different users. Salting does not make an embedding impossible to reverse in principle. Its practical consequence is that a copy of the server’s matching data, on its own, cannot be compared across users or lined up against the public models that produced it. This is what enables privacy-preserving social connection (described in the Privacy-First Architecture section) without the server needing the member’s words.

Privacy-First Architecture

Privacy in SERA is structural, not policy-based. The architecture is designed so that the system cannot access user data in ways that contradict its privacy claims, rather than promising not to. The platform-wide model, including the threat model and how each claim can be verified, is documented in the Private by Architecture whitepaper.

Vault Encryption

The memory vault is encrypted using AES-256-GCM via the native Web Crypto API. No custom cryptographic implementation is used. The choice to rely entirely on platform-native cryptography is itself a trust signal: custom crypto introduces audit surface that standard implementations have already undergone. ✅AES-256-GCM / Web Crypto API

Zero Data Retention Inference

Background-agent inference runs under zero data retention (ZDR) by default. User data is anonymised at the API boundary before reaching any inference. No conversation content, memory content, or user-identifying information is included in API requests beyond what is necessary for the immediate inference task. We will explore local models again as their quality and latency improve, but at present they don’t meet the requirements of the platform. 🟡Contractual + architectural

Privacy-Preserving Representations

SERA’s memory system generates representations of itself that can be matched without exposing the underlying content. Sensitive topics are stripped on the device, what remains is reduced to keywords the member approves, and those keywords are embedded on the device and salted with a secret key before anything reaches matching. Matching operates on these salted numbers, never on the words themselves, which enables a form of privacy-preserving social connection. The system can determine compatibility without either party’s words reaching the server, the other party or algn. 🟡Architectural

This property is what the Social Layer builds on. SERA states the capability and why it matters: that a memory system can generate matchable, privacy-preserving representations of its own patterns on the device, without exposing the words behind them. The mechanics of how that capability is used for social connection (Cells, Drops, the matching flow) are documented separately in the Beyond the Feed whitepaper.

Vocal Perception

The Technical Agent reads emotional state from what is said. Vocal perception extends this to how it is said. It is scoped and in internal testing, off by default, and not part of the current demo. Before any external member uses it, it requires an opt-in consent step, an EU AI Act disclosure and a completed data protection impact assessment.

The perception layer runs alongside speech recognition on the same audio and never in its path. It measures vocal energy, pitch variation, pace, pauses and response timing, and estimates emotional tone from the voice. If the layer fails, the conversation carries on unchanged, consistent with the ambient-agency constraint that no background process may degrade the primary interaction.

Personal baseline. Every reading is measured against the person’s own baseline rather than a population norm, because a quiet or flat voice is not necessarily a sad one.

Restraint over reporting. The Main Conversation agent is not given a reading on every turn. It is told only when there is a clear, sustained change over time, or a strong mismatch between voice and words, such as “I’m fine” said flat. Low-confidence readings are discarded, because a wrong read is worse than none. The signal is used mainly to shape pace, silence and turn-taking. The system never states an emotion as fact, never quotes the signal back, and never denies the capability if asked. The layer can also notice that another voice is present, using anonymous session labels only.

Privacy. Audio is analysed in memory and never written to disk, and readings last only for the session. The only thing kept is an aggregate numeric baseline, with no audio, no labels and no voiceprints, encrypted and deletable. The layer is treated as emotion recognition under the EU AI Act and will not be used in workplace or education settings. Known risks include wrong reads, background media mistaken for a second voice, and weaker performance outside English. The full account is in the Artificial Perception whitepaper.

Voice-First Delivery

SERA is designed for voice as the primary interaction channel, not as a text system with voice bolted on. This distinction shapes architectural decisions at every level.

Agent state is expressed through more than word choice alone. SERA routes signals such as timing, contemplation, and non-verbal presence to their respective delivery channels, since language is not the only channel available for conveying an AI system’s internal state to a user. The mechanics of that expression are documented separately in the Present, Not Human whitepaper.

The adaptive pipeline pacing system (described under the Technical Agent) adjusts both listening behaviour and speech output speed based on sustained conversational rhythm, independent of emotional classification. Latency variability (80-700ms) is a feature of the voice delivery, not a deficiency: a system that responds with identical speed regardless of conversational weight feels mechanical rather than present.

The speech-to-text and text-to-speech pipeline is designed for minimal latency with the async, non-blocking constraint that no synthesis step may wait on a background agent’s completion. Voice synthesis begins as soon as the Main Conversation agent produces output, regardless of whether the Subconscious or Memory agents have completed their current cycle.

Architectural Comparison

SERA makes specific architectural bets that diverge from the dominant approaches in multi-agent systems, memory design, and real-time AI interaction. This section names those divergences and the evidence behind them.

Heterogeneous multi-agent vs. homogeneous replication

The most common multi-agent pattern replicates a single model under different system prompts: the same base model acting as “researcher,” “critic,” and “synthesiser.” This is fast to implement and easy to coordinate, but it produces representational collapse. Patel (2026) quantified this directly: three identically-sourced agents on GSM8K achieved 0.888 mean cosine similarity across their rationales, meaning three agents carried roughly two agents’ worth of distinct reasoning.^[1]

Gu, Yang et al. (2026) tested the alternative. Heterogeneous configurations, where agents differ at the model level rather than just the prompt level, contributed complementary evidence. Two diverse agents matched or exceeded sixteen homogeneous ones in their evaluation. Dense communication topologies (every agent sees every other agent’s output) accelerated convergence rather than improving it.^[2]

SERA uses distinct model families for distinct agent roles and constrains inter-agent communication to conclusion-only injection rather than full-output sharing. This is more complex to maintain (model updates require per-role evaluation rather than a single swap) but preserves the information diversity that makes multi-agent architecture worth building.

Consolidating memory vs. RAG-as-lookup

Most deployed memory systems treat memory as a retrieval problem: store documents, embed queries, return the nearest vectors. Xu et al. (2026) argued that this amounts to “memos, not true memory,” lacking the consolidation, decay, and pattern formation that characterise biological memory systems.^[3]

Sumida et al. (2024) proposed psychological models of memory importance and forgetting for long-term RAG chatbots, demonstrating that memory systems with decay and importance weighting produced more contextually appropriate retrievals than flat stores.^[5] Gutierrez et al. (2024) introduced HippoRAG, drawing on hippocampal memory indexing theory to separate memory storage from retrieval indexing, enabling associative retrieval across documents that a standard RAG pipeline would treat as unrelated.^[6]

More recent work has pushed further. TeleMem (Chen et al., 2025) and MemMachine (Wang et al., 2026) both explore memory architectures that consolidate across interactions rather than merely appending, though neither implements the differentiated decay rates that SERA uses to distinguish identity-level from situational memory.^[7][8]

SERA’s memory system sits closest to the consolidation end of this spectrum: semantic merging, differentiated decay, staleness pruning, and cluster-aware pattern recognition. The trade-off is complexity. Consolidation can lose information (see Honest Tensions). Flat stores do not lose information; they lose retrievability.

Real-time parallel vs. sequential agentic

The dominant paradigm for multi-agent systems is sequential: an orchestrator decomposes a task, agents execute steps in order, results are aggregated. This works well for research, analysis, and multi-step workflows where latency tolerance is measured in seconds or minutes. It does not work for live conversation, where the user expects a response within the rhythm of speech.

Park et al. (2026) surveyed latency minimisation strategies across LLM-based agent systems and found that sequential architectures face throughput constraints when real-time interaction is the requirement rather than an optimisation target.^[4] Liu et al. (2023) proposed a hierarchical language agent for real-time human-AI coordination, using plan-act-reflect cycles with deliberate separation of fast and slow processing.^[9] Zhang et al. (2025) extended this with DPT-Agent, explicitly leveraging dual-process theory to separate fast, habitual responses from slow, deliberative reasoning, enabling real-time collaboration without forcing all cognition into the fast path.^[10]

SERA’s parallel architecture is philosophically aligned with the dual-process approaches of Liu and Zhang: the Main Conversation agent operates in the fast path (80-700ms), while the Subconscious Agent operates in the slow path. The difference is that SERA’s slow path does not generate actions or decisions; it generates observations and guidance that the fast path can incorporate or ignore. Background agents never interrupt or degrade the primary interaction. This constraint is what makes ambient agency distinct from conventional agentic orchestration.

Dimension	Standard approach	SERA
Agent design	One model, multiple role prompts	Distinct model families per cognitive role
Coordination failure	Representational collapse: 0.888 cosine similarity observed across nominally distinct agents	Information diversity preserved through model-level heterogeneity
Memory	Append-only RAG: store, embed, retrieve nearest vectors	Consolidate · decay by category · cluster · prune when stale
Interaction pattern	Sequential: latency in seconds, tasks with endpoints	Parallel: 80-700ms fast path + async slow path
Agent coupling	Dense: full output sharing	Conclusion-only injection, no inter-agent debate
Privacy	Data leaves device for personalisation	On-device encryption · salted matching embeddings · ZDR inference

Honest Tensions

SERA’s architecture involves trade-offs that have not been fully resolved. This section names them directly.

1. Multi-agent coordination introduces failure modes that monolithic systems avoid

Multi-agent systems fail in ways that single-agent systems do not. Cemri et al. (2025) developed the MAST taxonomy for multi-agent system failures and found that coordination breakdown, where agents produce contradictory or redundant outputs, accounted for a significant proportion of observed failures.^[11] Nechepurenko et al. (2026) argued that coordination should be treated as an explicit architectural layer rather than an emergent property, proposing structured coordination protocols to reduce failure rates.^[12]

Kim et al. (2025) examined the scaling properties of multi-agent systems and found that adding agents does not reliably improve performance. Beyond a system-specific threshold, additional agents increase coordination overhead faster than they add cognitive value. The practical implication is that SERA’s five-agent architecture (Main Conversation plus four ambient agents) is not a midpoint on a path to more agents. It is a deliberate configuration that must be evaluated on its own terms, and adding agents would need to clear a high bar of demonstrated benefit over coordination cost.^[13]

SERA’s mitigation is structural: low coupling (conclusion-only injection, no inter-agent debate), clear separation of concerns (each agent has a defined and non-overlapping responsibility), and the ambient-agency constraint (background agents cannot degrade the primary interaction). These constraints reduce the coordination surface but do not eliminate it. Contradictory guidance from the Subconscious Agent and the Technical Agent, for example, remains a live edge case that the Main Conversation agent must resolve without a formal arbitration protocol. This remains a live calibration challenge.

2. Real-time constraints limit depth of analysis

The 80-700ms latency range for the Main Conversation agent is a hard constraint that limits how much reasoning can occur within a single turn. Xu et al. (2025) modelled the latency-response relationship and found that user satisfaction degrades non-linearly with response time, with sharp drop-offs beyond approximately 500ms for conversational interactions.^[14] SERA deliberately goes past that point only at the slow end of its range, when the weight of the moment warrants a pause; quick exchanges stay well inside it. Ghoshal et al. (2026) demonstrated that tool use and explicit planning improve LLM reasoning quality but introduce latency costs that scale with task complexity.^[15]

SERA addresses this through temporal separation: fast-path responses (Main Conversation) and slow-path analysis (Subconscious Agent, operating asynchronously). The trade-off is that the system’s deepest analysis always arrives after the fact, informing future turns rather than the current one. A user raising something that genuinely requires deep reflection will receive an immediate response of limited depth, with the system’s fuller high level understanding arriving later. Whether the user has moved on by then is not something the architecture controls.

This is an accepted trade-off rather than an unsolved problem. The alternative, blocking the response while deeper analysis completes, would violate the real-time constraint that makes voice-first conversation viable.

3. Memory consolidation and pruning risk information loss

Semantic consolidation (merging similar memories) and staleness pruning (removing memories that have not been accessed within threshold windows) are designed to keep retrieval quality high over long time horizons. But both operations are destructive. A consolidated memory loses the specificity of its constituent parts. A pruned memory is gone.

The risk is that the system discards something the user would have wanted retained, either because the consolidation threshold merged entries that were meaningfully distinct, or because the staleness threshold retired a memory the user would have revisited given more time. There is no good empirical evidence quantifying this risk in production conversational memory systems, because very few such systems implement consolidation or pruning at all.

SERA’s current mitigation is conservative thresholds (a high similarity bar for consolidation, which requires strong semantic overlap before merging; long staleness windows, which provide substantial retention before pruning) and the differentiated decay system, which ensures that identity-level and values-level memory is far more resistant to pruning than situational context. Whether these thresholds are correctly calibrated is something that will only become clear through extended use. This is an acknowledged risk, actively monitored, without the certainty that the current calibration is correct.

Conclusion

SERA is a bet on a specific thesis: that the most important patterns in human experience are the ones people have not yet articulated to themselves, and that surfacing those patterns requires an architecture designed for that purpose rather than adapted from one built for something else.

The architecture distributes cognition across agents that differ at the model level, not just the prompt level, to resist the representational collapse that makes homogeneous multi-agent systems less capable than they appear. Its memory system consolidates, decays, and clusters rather than merely appending, because the failure mode of retrieval-based memory is not missing information but drowning in it. Its privacy model is structural (on-device encryption, user-salted embeddings, zero data retention) rather than policy-based, because trust claims that depend on organisational behaviour rather than architectural constraints are only as durable as the organisation’s incentives.

These are architectural positions, not guarantees. The multi-agent design introduces coordination failure modes that monolithic systems avoid. The real-time constraint limits the depth of per-turn analysis. The memory system’s consolidation and pruning operations are destructive by design. SERA navigates these tensions through structural constraints (low coupling, temporal separation, conservative thresholds) rather than claiming to have resolved them.

Conversational AI is not a solved problem made incrementally better by larger models. It is a design problem that requires architectural answers. SERA is one such answer. Whether it is the right one depends on whether the premise holds: that meaningful conversation requires a system that can observe its own patterns, remember with discrimination, and hold privacy as a structural property rather than a policy promise.

References

1. 1.Patel, A. (2026). "Measuring Representational Collapse in Multi-Agent LLM Systems." arXiv preprint, arXiv:2604.03809.
2. 2.Gu, X., Yang, Z. et al. (2026). "Heterogeneous Multi-Agent Configurations Resist Representational Collapse." arXiv preprint, arXiv:2602.03794.
3. 3.Xu, J. et al. (2026). "Contextual Agentic Memory is a Memo, Not True Memory." arXiv preprint.
4. 4.Park, S. et al. (2026). "Minimizing Response Latency in LLM-Based Agent Systems: A Comprehensive Survey." IEEE Access.
5. 5.Sumida, T. et al. (2024). "Enhancing Long-term RAG Chatbots with Psychological Models of Memory Importance and Forgetting." Dialogue & Discourse.
6. 6.Gutierrez, B. et al. (2024). "HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models." arXiv preprint.
7. 7.Chen, W. et al. (2025). "TeleMem: Memory-Augmented LLM Agents for Telecommunication Networks." arXiv preprint.
8. 8.Wang, Y. et al. (2026). "MemMachine: Consolidating Memory Across Multi-Turn LLM Interactions." arXiv preprint.
9. 9.Liu, H. et al. (2023). "LLM-Powered Hierarchical Language Agent for Real-time Human-AI Coordination." arXiv preprint.
10. 10.Zhang, Y. et al. (2025). "DPT-Agent: Leveraging Dual Process Theory in Language Agent Framework for Real-time Simultaneous Human-AI Collaboration." arXiv preprint.
11. 11.Cemri, M. et al. (2025). "Why Do Multi-Agent LLM Systems Fail?" arXiv preprint.
12. 12.Nechepurenko, A. et al. (2026). "Coordination as an Architectural Layer for LLM-Based Multi-Agent Systems." arXiv preprint.
13. 13.Kim, J. et al. (2025). "Towards a Science of Scaling Agent Systems." arXiv preprint.
14. 14.Xu, L. et al. (2025). "Latency-Response Theory Model." arXiv preprint.
15. 15.Ghoshal, S. et al. (2026). "When Do Tools and Planning Help Large Language Models Think? A Cost- and Latency-Aware Benchmark." SoutheastCon 2026.

Open review

This is a living document. Our thinking is still developing and some of it will turn out to be wrong. If a claim looks off or a source doesn't hold up, tell us below.

Submit a contribution or correction →