This post is Part 1 of the 'AI Agent Engineering' series. Moving beyond single-shot ChatCompletion API calls, it explores agent runtimes and system architectures that autonomously update state and execute tools to fulfill user goals.
1. Prompt Calling vs AI Agent Runtime
This article describes an AI agent as a runtime that maintains state and runs a control loop.
Read the diagram as text
The stateless path goes from User Input to LLM Prompt API to Static Response. The separate agent path goes from User Goal to Observe for state and context perception, Plan for reasoning and task decomposition, Act for tool execution and environment state change, then Goal Completion Verification. Not achieved returns to Observe for re-planning. Achieved or boundary exceeded leads to Final Answer or Safe Exit.
Agent engineering also covers what happens after a response: detecting failure, revising the plan, choosing the next tool call, and deciding when to stop.
Architecture Comparison
| Dimension | Single-Shot LLM Prompt | AI Agent Runtime (Agent Loop) |
|---|---|---|
| Control Flow | Unidirectional Pipeline (Input → Prompt → LLM → Output) | Dynamic Cyclic Control Loop (Observe → Plan → Act → Verify) |
| State Management | Stateless (No state persistence across calls) | Stateful (Continual update of memory, trace log, environmental feedback) |
| Tool Integration | Single function call or none | Multi-tool chaining with dynamic re-planning based on tool outputs |
| Fault Tolerance | Single failure causes full request collapse | Self-healing fallback pathways upon tool execution errors |
| Complex Task Execution | Precision degrades rapidly as context fills | Hierarchical processing via Task Decomposition |
| Cost & Latency | Predictable (Single call token & latency) | Variable depending on loop iteration count and execution trajectory |
2. Observe-Plan-Act Paradigm and Task Decomposition
Attempting to solve complex, multi-step requests with a single LLM prompt drastically lowers reasoning precision. Agent architectures mitigate this by breaking goals into multi-stage subtasks via Task Decomposition.
Observe-Plan-Act State Machine
Observe, Plan, and Act are composite states with internal steps. Each entry point below belongs to its own enclosing state.
| Composite state | Internal source | Internal destination | Operation |
|---|---|---|---|
| Observe | Entry | ReadEnvironment | Start reading the environment |
| Observe | ReadEnvironment | UpdateState | Sync Tool Results and Feedback |
| Plan | Entry | AnalyzeGoal | Start goal analysis |
| Plan | AnalyzeGoal | DecomposeSubtasks | Dependency Graph and Subtask Breakdown |
| Plan | DecomposeSubtasks | SelectNextAction | Bind Tools and Determine Priority |
| Act | Entry | InvokeTool | Start the tool call |
| Act | InvokeTool | CaptureResult | Capture Tool Output and Errors |
The outer transitions are separate. Evaluate is a decision point with four possible outcomes.
| Current state | Condition or operation | Next state |
|---|---|---|
| Start | Goal Received | Idle |
| Idle | Initial State and Environment Context Collection | Observe |
| Observe | Observation Context Ready | Plan |
| Plan | Execution Ready | Act |
| Act | Deliver Feedback | Evaluate |
| Evaluate | Success: Proceed to Next Subtask | Observe |
| Evaluate | Tool Failure or Exception: Re-plan | RePlan |
| Evaluate | All Subtasks Completed: Goal Met | GoalAchieved |
| Evaluate | Loop or Resource Limit Exceeded: Exit Boundary | BoundaryExceeded |
| RePlan | Revise Plan and Adjust Strategy | Plan |
| GoalAchieved | Return Final Answer: Success Exit | End |
| BoundaryExceeded | Fallback or Human-In-The-Loop: Safe Exit | SafeExit |
| SafeExit | Graceful System Shutdown | End |
Task Structuring via Pydantic: Forcing Strict JSON Schemas or Pydantic models instead of free-form text output allows Python runtimes to safely parse and track subtask states.
from typing import List, Optionalfrom pydantic import BaseModel, Field class SubTask(BaseModel): id: int description: str tool_name: Optional[str] = Field(default=None, description="Bound tool name required for execution") status: str = Field(default="pending", description="pending | in_progress | completed | failed") class ExecutionPlan(BaseModel): goal: str subtasks: List[SubTask] reasoning: strTask Decomposition Tree
Hierarchical task decomposition breaks high-level objectives down into actionable tool calls and state transitions.
The top-level goal is “Fetch web traffic logs and generate abnormal transaction report.” Its task decomposition contains three tasks. Each task leads to its own tool and status.
| Task under the goal | Work | Connected tool | Status after the tool |
|---|---|---|---|
| Task 1 | Query API log data | fetch_api_logs(date='2026-07-21') | completed |
| Task 2 | Detect anomalies with ensemble detector | detect_anomalies(df, threshold=3.0) | in_progress |
| Task 3 | Compile summary markdown report | generate_markdown_report(summary) | pending |
Task 1's completed status feeds its observation to Task 2 and meets that dependency. The dependency from Task 2's status to Task 3 is still pending. Do not read an in-progress task as completed.
1) Plan
The model analyzes subtask dependencies to achieve the target goal and constructs a structured ExecutionPlan.
2) Observe & Act
Subtasks are executed sequentially (Act), and outputs or API errors are appended to the state history (Observe). If unexpected outputs or errors occur, the agent initiates dynamic re-planning.
3. Defensive Reasoning Boundaries and Exit Conditions
In production runtimes, one of the most critical failure modes is the Infinite Agent Loop—where a model endlessly retries an unachievable goal. Runtimes must enforce strict Reasoning Boundaries.
Infinite Loop Defense: Without explicit hard limits and exit conditions, broken tool loops lead to runaway token costs and server resource exhaustion.
Defensive Boundaries to Check
| Defensive Boundary | Threshold Example | Trigger Condition | Fallback Action | Safety Impact |
|---|---|---|---|---|
| 1. Max Iteration Limit | 10 iterations | Measure cumulative loop counter per goal | Force loop exit and return graceful summary based on current observations | Prevents infinite loop billing and protects server resources |
| 2. Context & Token Budget | 80% Context Window | Real-time calculation of prompt tokens + observation payload | Trigger context compression or halt after extracting core state | Prevents token overflow errors and caps cost per session |
| 3. Repeated Tool Failure | 3 consecutive failures (Same tool + args) | Hash comparison of consecutive tool execution logs | Block tool invocation, force alternative path, or request HITL approval | Prevents blind retries during external API outages |
Reasoning Boundary Exit Sequence: Boundary conditions are validated at every loop entry point, blocking LLM calls and routing execution to a safe exit pathway when limits are breached.
Reasoning Boundary Exit Sequence Diagram
Read the diagram as text
The user or system submits a goal to the Agent Runtime Loop. The Reasoning Boundary Guard validates loop state and resources. Exceeding the max loop or token limit goes to Safe Exit, Fallback, or HITL, then returns a graceful failure response and exits. Normal execution allows the LLM Reasoner to call a Tool or External API, whose execution result and state update return to Runtime.
Python Control Loop Implementation Reference
# Agent runtime loop exit control reference (Python concept)MAX_ITERATIONS = 10 def run_agent_loop(goal: str, state: AgentState) -> AgentResult: iteration = 0 while iteration < MAX_ITERATIONS: iteration += 1 # 1. Observe state & construct prompt context context = state.get_observation_context() # 2. LLM reasoning (Determine next action) action = llm_reason(goal, context) # 3. Verify final answer exit condition if action.is_final_answer: return AgentResult(success=True, output=action.final_output) # 4. Execute action & check failure counter result = execute_action(action) state.update(action, result) if state.consecutive_tool_failures >= 3: # Defensive reasoning boundary exit return AgentResult(success=False, output="Safe exit due to repeated tool failures") return AgentResult(success=False, output=f"Exceeded maximum iteration limit ({MAX_ITERATIONS})")4. Production Agent Architecture Checklist
When designing a production-grade agent baseline, verify the following core checklist:
Omitting any of these checklist items increases the risk of infinite loops, token overflow, or tool deadlocks in production environments.
| Domain | Inspection Item | Priority | Verification Method |
|---|---|---|---|
| Loop Safety | Are hard limits for Max Iteration, Execution Timeout, and Token Budget active? | CRITICAL | Runtime Middleware / Guard Loop |
| Schema Validation | Are Task Decomposition and Action Outputs validated with Pydantic / Zod schemas? | HIGH | LLM Structured Outputs / Guardrails |
| Self-Healing | Do fallback pathways exist when repeated tool failures occur? | HIGH | Retry Counter & Alternate Prompt Path |
| State Isolation | Is raw LLM reasoning text isolated from execution context storage? | MEDIUM | Clean State Architecture / Memory Store |
5. Next Episode Teaser
In Episode 02 Agent Memory and State Management, we explore short-term and long-term memory architectures and graph state schema management in depth.
댓글
GitHub 계정으로 로그인하면 댓글을 남길 수 있습니다. 댓글은 GitHub Discussions를 통해 운영됩니다.