This post is Part 1 of the 'AI Agent Engineering' series. Moving beyond single-shot ChatCompletion API calls, it explores agent runtimes and system architectures that autonomously update state and execute tools to fulfill user goals.
1. Prompt Calling vs AI Agent Runtime
The fundamental distinction between simple LLM-integrated applications and true AI Agents lies in an Autonomous Control Loop and Persistent State.
- subgraph
- User Input
- LLM Prompt API
- Static Response
- end
- User Goal
- Observe: State & Context Perception
- Plan: Reasoning & Task Decomposition
- Act: Tool Execution & Environment State Change
- Goal Completion Verification
- Final Answer / Safe Exit
Paradigm Shift: Where traditional LLM development focused on prompt engineering for a single response, Agent Engineering shifts focus toward system architecture—how an agent autonomously detects failure, adapts variables, and manages control loops until the goal is achieved.
Architecture Comparison
| Dimension | Single-Shot LLM Prompt | AI Agent Runtime (Agent Loop) |
|---|---|---|
| Control Flow | Unidirectional Pipeline (Input → Prompt → LLM → Output) | Dynamic Cyclic Control Loop (Observe → Plan → Act → Verify) |
| State Management | Stateless (No state persistence across calls) | Stateful (Continual update of memory, trace log, environmental feedback) |
| Tool Integration | Single function call or none | Multi-tool chaining with dynamic re-planning based on tool outputs |
| Fault Tolerance | Single failure causes full request collapse | Self-healing fallback pathways upon tool execution errors |
| Complex Task Execution | Precision degrades rapidly as context fills | Hierarchical processing via Task Decomposition |
| Cost & Latency | Predictable (Single call token & latency) | Variable depending on loop iteration count and execution trajectory |
2. Observe-Plan-Act Paradigm and Task Decomposition
Attempting to solve complex, multi-step requests with a single LLM prompt drastically lowers reasoning precision. Agent architectures mitigate this by breaking goals into multi-stage subtasks via Task Decomposition.
Observe-Plan-Act State Machine
- Idle
- Observe
- state
- ReadEnvironment
- UpdateState
- Plan
- AnalyzeGoal
- DecomposeSubtasks
- SelectNextAction
- Act
- InvokeTool
- CaptureResult
- Evaluate
- RePlan
- GoalAchieved
- BoundaryExceeded
- SafeExit
Task Structuring via Pydantic: Forcing Strict JSON Schemas or Pydantic models instead of free-form text output allows Python runtimes to safely parse and track subtask states.
from typing import List, Optionalfrom pydantic import BaseModel, Field class SubTask(BaseModel): id: int description: str tool_name: Optional[str] = Field(default=None, description="Bound tool name required for execution") status: str = Field(default="pending", description="pending | in_progress | completed | failed") class ExecutionPlan(BaseModel): goal: str subtasks: List[SubTask] reasoning: strTask Decomposition Tree
Hierarchical task decomposition breaks high-level objectives down into actionable tool calls and state transitions.
1) Plan
The model analyzes subtask dependencies to achieve the target goal and constructs a structured ExecutionPlan.
2) Observe & Act
Subtasks are executed sequentially (Act), and outputs or API errors are appended to the state history (Observe). If unexpected outputs or errors occur, the agent initiates dynamic re-planning.
3. Defensive Reasoning Boundaries and Exit Conditions
In production runtimes, one of the most critical failure modes is the Infinite Agent Loop—where a model endlessly retries an unachievable goal. Runtimes must enforce strict Reasoning Boundaries.
Infinite Loop Defense: Without explicit hard limits and exit conditions, broken tool loops lead to runaway token costs and server resource exhaustion.
Essential 3-Tier Defensive Boundaries
| Defensive Boundary | Threshold Example | Trigger Condition | Fallback Action | Safety Impact |
|---|---|---|---|---|
| 1. Max Iteration Limit | 10 iterations | Measure cumulative loop counter per goal | Force loop exit and return graceful summary based on current observations | Prevents infinite loop billing and protects server resources |
| 2. Context & Token Budget | 80% Context Window | Real-time calculation of prompt tokens + observation payload | Trigger context compression or halt after extracting core state | Prevents token overflow errors and caps cost per session |
| 3. Repeated Tool Failure | 3 consecutive failures (Same tool + args) | Hash comparison of consecutive tool execution logs | Block tool invocation, force alternative path, or request HITL approval | Prevents blind retries during external API outages |
Reasoning Boundary Exit Sequence: Boundary conditions are validated at every loop entry point, blocking LLM calls and routing execution to a safe exit pathway when limits are breached.
Reasoning Boundary Exit Sequence Diagram
- User / System
- Agent Runtime Loop
- Reasoning Boundary Guard
- Safe Exit / Fallback / HITL
- Graceful System Exit
- LLM Reasoner
- Tool / External API
Python Control Loop Implementation Reference
# Agent runtime loop exit control reference (Python concept)MAX_ITERATIONS = 10 def run_agent_loop(goal: str, state: AgentState) -> AgentResult: iteration = 0 while iteration < MAX_ITERATIONS: iteration += 1 # 1. Observe state & construct prompt context context = state.get_observation_context() # 2. LLM reasoning (Determine next action) action = llm_reason(goal, context) # 3. Verify final answer exit condition if action.is_final_answer: return AgentResult(success=True, output=action.final_output) # 4. Execute action & check failure counter result = execute_action(action) state.update(action, result) if state.consecutive_tool_failures >= 3: # Defensive reasoning boundary exit return AgentResult(success=False, output="Safe exit due to repeated tool failures") return AgentResult(success=False, output=f"Exceeded maximum iteration limit ({MAX_ITERATIONS})")4. Production Agent Architecture Checklist
When designing a production-grade agent baseline, verify the following core checklist:
Omitting any of these checklist items increases the risk of infinite loops, token overflow, or tool deadlocks in production environments.
| Domain | Inspection Item | Priority | Verification Method |
|---|---|---|---|
| Loop Safety | Are hard limits for Max Iteration, Execution Timeout, and Token Budget active? | CRITICAL | Runtime Middleware / Guard Loop |
| Schema Validation | Are Task Decomposition and Action Outputs validated with Pydantic / Zod schemas? | HIGH | LLM Structured Outputs / Guardrails |
| Self-Healing | Do fallback pathways exist when repeated tool failures occur? | HIGH | Retry Counter & Alternate Prompt Path |
| State Isolation | Is raw LLM reasoning text isolated from execution context storage? | MEDIUM | Clean State Architecture / Memory Store |
5. Next Episode Teaser
In Episode 02 Agent Memory and State Management, we explore short-term and long-term memory architectures and graph state schema management in depth.
댓글
GitHub 계정으로 로그인하면 댓글을 남길 수 있습니다. 댓글은 GitHub Discussions를 통해 운영됩니다.