Function Calling 설계

Where to draw the line between LLM and internal API

In this article, we outline what boundaries should be drawn when connecting LLM and internal API using function calling.

function calling allows models to interact with external systems. In addition to answering user questions, you can also query data in the database, create tickets, search documents, and run workflows.

But an important question arises here.

How much should we leave to the model? Which API will be exposed as a tool? Who will do the permission checking? How do you bounce back when you fail?

Analysis base date: 2026-05-12 Practice environment: OpenAI API, FastAPI, PostgreSQL Key references: OpenAI function calling Docs, Structured Outputs Docs, AWS Idempotency

Key takeaways

• function calling should be understood as a method in which the model structures the call intent and delivers it to the backend, rather than directly executing an internal API.
• The backend tool executor must have actual execution permission.
• Read-only tools and mutating tools must be separated.
• State change tools may require idempotency, audit logs, and human approval.
• The tool schema should not expose the entire internal API, but only provide the minimum interface required for the model.

1. How to understand function calling

function calling is dangerous when understood as the ability of a model to “directly execute” a function. More precisely, it is a function that suggests in a structured form which function the model should call with which arguments.

The actual execution must be done by the backend.

1# This is an example.2User request3→ LLM decides tool call4→ Backend validates tool call5→ Permission check6→ Execute internal API7→ Return tool result to model8→ Generate final response

The model is the decision assistant, and the backend is the executor.

2. What is Tool Boundary?

Tool Boundary is the boundary of functions exposed to the model.

Bad example:

1# This is an example.2execute_sql(query: string)

This tool is so powerful. Models can generate arbitrary SQL, and permissions and safety are difficult to control.

Good example:

1# This is an example.2search_documents(query: string, scope: enum, limit: integer)3get_document_summary(document_id: string)4create_support_ticket(title: string, description: string, priority: enum)

The actions required of the model should be limited to small tools.

3. Separation of Read Tool and Write Tool

For initial function calling, it is safe to start with a read-only tool.

4. Tool schema design

Example document search tool:

1// This is an example JSON structure.2{3  "name": "search_documents",4  "description": "Search for relevant content in documents accessible to the user.",5  "parameters": {6    "type": "object",7    "properties": {8      "query": {9        "type": "string",10        "description": "Natural language query to search"11      },12      "scope": {13        "type": "string",14        "enum": ["official_docs", "papers", "tech_blogs"]15      },16      "limit": {17        "type": "integer",18        "minimum": 1,19        "maximum": 1020      }21    },22    "required": ["query", "scope", "limit"],23    "additionalProperties": false24  }25}

The design points are as follows:

1# This is an example.2[ ] Limit the scope with enum.3[ ] limit Sets the maximum value.4[ ] Internal ID or permission conditions are determined by the server.5[ ] The user's original text input is not passed on to the internal API as is.

5. Authorization and authentication

Permission checking should be done by the server, not the model.

1# This is an example.2User A asks a question3to model suggests calling search_documents4to server checks user A's document permissions5to Search only authorized scopes6to return result

It is not enough to tell the model, “Retrieve only documents accessible to this user.” The actual filter must be enforced on the backend.

6. Pre-execution Verification and Human Approval

State change tools require verification before execution.

1# This is an example.2Create LLM tool call3→ schema validation4→ business rule validation5→ permission check6→ risk classification7→ human approval if needed8→ execute

For example, sending emails, creating tickets, or changing customer status may require human approval.

1// This is an example JSON structure.2{3  "tool": "create_support_ticket",4  "arguments": {5    "title": "Redis cache issue",6    "description": "Users reported Redis cache miss issues.",7    "priority": "medium"8  },9  "requires_approval": true10}

7. Audit logs and reproducibility

function calling must leave an audit log.

However, sensitive information must be masked.

8. Failure and rollback

Tool execution may fail.

It is safer not to expose state change tools unless there is a rollback plan.

9. Practical checklist

1# This is an example.2[ ] Are the read tool and write tool separated?3[ ] Does the tool schema follow the principle of least privilege?4[ ] Are there input restrictions such as enum, max limit, etc.?5[ ] Is permission checking enforced on the server?6[ ] Does the status change tool have an idempotency key?7[ ] Is there human approval for hazardous work?8[ ] Are all tool calls left in the audit log?9[ ] Is there an explainable response to the user in case of failure?

Failure case: When the internal API is exposed as a tool

A common mistake seen in function calling is exposing an already existing backend API almost as a tool. For example, if the model can call endpoints such asPOST /tickets,PATCH /users/:id, andDELETE /documents/:id, the implementation will be faster. However, these APIs are designed with the assumption that they are explicitly called by a person or server. If you give it to the model as is, the scope of arguments, permissions, approval, and retry implications are too broad.

In particular, the write API is dangerous without idempotency. If the model sees a timeout and tries the same tool call again, but the first request was actually successful, two tickets may be created. For tasks that require confirmation from the user, simply writing “call with caution” in the tool description is not enough. The server must check the approval state before final execution.

Implementation example: Creating a command API for a model

If you have a narrow command surrounding the internal API, the boundary becomes clearer.

1type CreateTicketCommand = {2  kind: "create_support_ticket";3  idempotencyKey: string;4  title: string;5  summary: string;6  priority: "low" | "normal" | "high";7  requiresHumanApproval: true;8};

This command is intentionally smaller than the internal ticket API. Values ​​such as assignee, internal label, and billing field are filled in by the server, making the model uneven. The model structures the proposal, “I need to create a ticket,” and the backend checks permissions and approval and calls the internal API.

Separate design interpretation from formal facts

What the official documentation provides is that “the model can make structured tool calls.” However, “which internal APIs will be exposed as tools,” “which tasks will be sent after approval,” and “which fields will be calculated by the server” are product design interpretations. If you mix the two, it's easy to mistake tool support for operational safety.

With this separation, function calling can be treated as a backend boundary rather than a model feature.

Practical example: Creating a customer support ticket

Let's say your customer support chatbot needs to view the conversation and create a ticket. A bad design would be to have the model populate every field in the internal ticket API. A good design is for the model to structure only the “ticket creation intent” and the server to populate the user information, organization, SLA, responsible queue, and approval status.

Failure case is retry after timeout. The model calledcreate_support_ticketand the provider response was delayed. The user makes the request again, and the model calls the same thing one more time. If you do not have an idempotency key, you will have two tickets. If there is no approval status, the user may say "just create a draft" but an actual ticket will be sent.

Therefore, it is safer to divide the write tool into three steps. First, the model creates a command draft, second, the server attaches validation and idempotency keys, and third, the user or policy approves and executes the internal API. In this structure, the result of function calling is not a side effect, but an intermediate product that turns side effect candidates into verifiable commands.

10. Q&A

Q1. Can I expose the internal API as a tool?

Not recommended. Tools for models should have a smaller and more limited interface than the internal API.

Q2. What if I select the wrong tool for the model?

The server must verify and reject the tool call. Model judgments are suggestions only.

Q3. Is function calling the same as Agent?

They can be components of an Agent, but they are not the same. function calling is an interface that structures external function calls.

11. References and uncertainty

References

• OpenAI function calling:https://platform.openai.com/docs/guides/function-calling
• OpenAI Structured Outputs:https://platform.openai.com/docs/guides/structured-outputs
• AWS Builders Library — Idempotent APIs:https://aws.amazon.com/builders-library/making-retries-safe-with-idempotent-APIs/

uncertainty

• function calling's detailed API and support model may vary depending on the time.
• Actual tool approval criteria must be aligned with the organization's security and operational policies.