Palantir Foundry AIP Speedrun: Complete End-to-End AI Workflow Master Guide

When introducing AI Agents and Large Language Models (LLMs) to enterprise environments, the biggest challenge is "how to process massive volumes of unstructured documents (PDFs, policy books, academic papers) into trusted knowledge and connect them to operational systems in real time."

Relying solely on basic RAG (Retrieval-Augmented Generation)—storing document chunks in a Vector DB and performing similarity search—exposes fundamental limitations like hallucinations and context disconnection when documents are complex or require precise multi-entity relational reasoning.

Palantir's "Speedrun: Your First AIP Workflow" course connects PDF information extraction, Ontology, an AIP Chatbot, and an interactive application. This article also explains related data, logic, and interface design examples.

The pipeline, Ontology, AIP Logic, and UI sections below illustrate how the components can connect.

---

Official learning scope and supplementary design examples

The official Palantir AIP getting-started guide describes Speedrun: Your First AIP Workflow as a 60–90 minute hands-on course. Its documented sequence is PDF information extraction → Ontology integration → an AIP Chatbot using the knowledge graph → an interactive application. The AIP Chatbot Studio documentation and the Workshop widget documentation are independent product references.

The nine-module structure below is this article's explanatory breakdown, not a verified reproduction of the current official course's number of lessons or UI sequence. Chunk/Entity datasets, Vertex, joins, and AIP Logic are architecture examples; their inclusion does not prove that the official course runs those exact steps.

Conceptual flow from PDFs to an app, rather than a Foundry UI screenshot.

This is an architecture and learning-path reference. Follow the official course for hands-on procedures, and check detailed menus and features against your account permissions and version.

1. End-to-End Pipeline Architecture (E2E Architecture)

The Palantir AIP architecture ingests unstructured PDF media, converts data into semantic Ontology objects step-by-step, and enables real-time reasoning and visualization through AIP Logic and Workshop UI.

Read the connections by layer: data processing, ontology, then application use. Each table row is a connection from its source to its destination. Rows with the same source are branches, rather than consecutive execution steps.

Data Processing & LLM Batch Extraction has these inputs and outputs:

In the Palantir Semantic Layer, datasets and the join table map to ontology object and link types:

The Analytics & AIP Reasoning Engine connects each of the three ontology components to both consumers. Both consumers feed the Workshop UI.

---

2. Module Configuration (Modules 1–9)

Module 1. Course Introduction (Use Case & Goals)

1.1 Use Case Background & Objectives

• Primary Goal: Build a "Digital Summer Intern" app capable of extracting key entities and relationships from enterprise libraries (PDF policy books, research papers, patents) and answering user queries with explicit citations.
• Design intent: Connect Entities and Chunks to supply relationship context and sources. Extraction and reasoning errors remain possible, so verify that answers match their cited evidence.

1.2 Prerequisites

• A Foundry account with access to the required data, Ontology, and AIP features. Check official guidance for enrollment and usage terms.

---

Module 2. Setting Up Your Project and Folder

Organize and isolate data resources and Ontology definitions inside Foundry for governance and security.

1. Create a Foundry Learning Project:

• Navigate to Projects & Files in the Foundry header and create AIP Speedrun - Learning Project.
• Configure who can access the learning data. Separate folders alone do not enforce access isolation.

2. Create Training Folder:

• Create Training_PDF_AIP_Workflow inside the project.
• Structure subfolders: 01_Ingestion, 02_Pipeline, 03_Ontology, 04_AIP_Logic, 05_Workshop_App.

---

Module 3. Uploading the Data (Raw Data Ingestion)

Ingest unstructured PDF documents into the Foundry platform infrastructure.

1. Create Media Set:

• Inside 01_Ingestion, click New -> Media Set.
• Upload target PDF file sets (e.g., aerospace engineering technical manuals or legal casebooks).

2. Index Media Set Files:

• Connect the PDF media set as pipeline input. Uploading and text extraction are separate steps. Select raw text extraction or OCR to match the documents.

3. [Optional] Deploy via Marketplace:

• Deploy pre-cleaned reference dataset templates directly from Marketplace for accelerated walkthroughs.

---

Module 4. Transforming Data & LLM Extraction Pipeline

Convert unstructured PDFs into 3 structured datasets (Chunks, Entities, Join Table) using Pipeline Builder.

Copy1       [Raw Media Set PDF]2               │3      ┌────────┴────────┐4      ▼                 ▼5[Text Chunking]  [Embedding Vector]6      │                 │7      ▼                 │8[Chunk Dataset] ◄───────┘9      │10      ▼11[LLM Batch Entity Extraction]12      │13      ├─────────────────┐14      ▼                 ▼15[Entities Dataset]   [Join Table (Chunk-Entity M:N)]

Step 4.1. Text Chunking

• Split raw PDF text streams into uniform semantic context blocks (Text Chunks).
• Principle: Each chunk contains a complete "Single Point of Information" aligned with paragraph/sentence boundaries.
• Primary Key (chunk_id): Assign a unique Hash PK combining document title and text contents.

Step 4.2. Vector Embeddings

• Run embedding models over chunk text to attach vector dimension fields (embedding_vector) to the dataset schema.

Step 4.3. LLM Batch Entity Extraction

• Attach an LLM Batch Node inside Pipeline Builder and define extraction prompt specs:

  Copy1Prompt: |2  Analyze the following text chunk and extract core entities (people, organizations, technical terms, products, locations).3  Output valid JSON array with fields: entity_name, entity_type, description.

• Entity Resolution: Standardize variations (e.g., "Palantir Technologies", "Palantir Corp", "Palantir Inc") to canonical entity_id keys and deduplicate.

Step 4.4. Join Table (M:N Relationship Table)

• Construct the Chunk_Entity_Join dataset to represent relationships between Chunks and Entities.
• Schema: join_id, chunk_id, entity_id, extraction_confidence.

Step 4.5. Deploy Pipeline

• Execute the Pipeline Builder graph to generate the 3 backing datasets.

---

Module 5. Configuring Ontology (Ontology Object Mapping)

Map structured datasets to the Foundry Semantic Layer (Ontology) to construct business-centric object models.

5.1 Chunk Object Type

• Object Type Name: Chunk
• Backing Dataset: Chunks Dataset
• Primary Key: chunk_id (String)
• Properties: text_content (String), source_doc (String), embedding_vector (Vector)

5.2 Entity Object Type

• Object Type Name: Entity
• Backing Dataset: Entities Dataset
• Primary Key: entity_id (String)
• Properties: entity_name (String), entity_type (String), description (String)

5.3 Object Link Types (M:N Relation Binding)

• Link Type Name: ChunkToEntity / EntityToChunk
• Relation Pattern: Many-to-Many (M:N) via Chunk_Entity_Join
• Effect: Allows dynamic graph traversal between any Entity and all Chunk objects where it appears.

---

Module 6. Exploring Object Relations (Vertex Knowledge Graph)

1. Open Vertex Explorer and add Entity and Chunk object types to the visual canvas.
2. Graph Traversal: Select a central node (e.g., "Palantir AIP") and click Expand Links to reveal connected Chunks and linked Entities.
3. Save Vertex Template: Save visual layout as Knowledge_Graph_Explorer_Template for dynamic embedding inside Workshop apps.

---

Module 7. Creating an AIP Logic Function

Configure application logic in AIP Logic. The inputs, outputs, and prompt below are conceptual examples. TypeScript Functions are a separate implementation path.

7.1 AIP Logic Input/Output

• Inputs: user_query (String), selected_entities (List of Entity)
• Outputs: response_text (String), referenced_chunks (List of Chunk)

7.2 Grounded Prompt & Guardrails

Copy1System Prompt: |2  You are an expert AI knowledge agent powered by Palantir AIP.3  Answer user questions ({user_query}) strictly using the provided Knowledge Graph Context objects.4 5  [Context Objects]6  - Related Entities: {selected_entities}7  - Grounded Chunks: {retrieved_chunks_text}8 9  [Rules]10  1. Never extrapolate beyond provided Context objects (Strict Anti-Hallucination).11  2. Append explicit chunk citations at sentence ends in format [Chunk: {chunk_id}].12  3. If context is insufficient, state "The provided knowledge base does not contain enough information to answer this query."

7.3 Function Publish

• Run unit tests and execute Publish Version to deploy production artifacts.

---

Module 8. Building an App (Workshop Interactive Application)

Use Palantir Workshop to construct a responsive, reactive web application for enterprise users.

Copy1┌─────────────────────────────────────────────────────────────────────────┐2│  [Header Bar]  AIP Speedrun Knowledge Explorer App                      │3├───────────────────────────────┬─────────────────────────────────────────┤4│ [Search & Input Panel]        │ [Vertex Graph Component]                │5│ > Search Entity: "AIP"        │                                         │6│ > User Question: "AIP Specs"  │    (Entity: AIP) ─── (Chunk #102)       │7│                               │          │                 │            │8│ [Object List Panel]           │    (Chunk #105) ─── (Entity: Ontology)  │9│ - Entity: Palantir AIP        │                                         │10│ - Chunk #102: "AIP is..."     ├─────────────────────────────────────────┤11│ - Chunk #105: "Ontology..."   │ [AIP Response Panel]                    │12│                               │ > "Palantir AIP integrates..."          │13│ [Button: Run AIP Logic]       │   [Source: Chunk #102, #105]            │14└───────────────────────────────┴─────────────────────────────────────────┘

8.1 Workshop Layout Design

1. Left Navigation Panel: Entity search box and object list tables.
2. Top-Right Main Canvas: Embedded Vertex Graph Widget bound to selected entities.
3. Bottom-Right Response Panel: AIP Response Card with query input, AIP Logic Function call button, and deep-linked citation cards.

8.2 Event & State Binding

• On Search Submit -> Filter Entity Objects -> Update Vertex Graph Selection -> Invoke AIP Logic Function -> Render Response & Citation Links.

---

Module 9. Conclusion & Advanced Patterns

• Learning objective: Understand how PDF information becomes usable through Ontology, a chatbot, and an app. The supplementary design above is not a claim that all those components are built in the official course's 60–90 minutes.
• Production Scale-Up Principles:

1. RBAC & Governance: Enforce object-level security (Markings & Role Access) to trim graph nodes based on user permissions.
2. Incremental Data Pipelines: Use incremental Spark processing for new PDF ingestions without re-running full historical extractions.

---

3. Comparison: Basic RAG vs GraphRAG vs Palantir Ontology AIP

---

Conclusion & Engineering Takeaways

Follow how each module output becomes the next module input: extracted entities become Ontology objects, feed search and logic functions, and appear in the Workshop interface.

1. Unstructured Data to Semantic Assets: Converts raw PDFs into first-class Chunk and Entity Ontology objects.
2. Contextual Accuracy via Knowledge Graphs: Feeds rich structural context into LLM prompts using Vertex Knowledge Graphs.
3. Controlled AI Workflows: Integrates AIP Logic and Workshop UI to deliver secure, cited, reactive enterprise AI applications.