palantir-cost-tuning

name: palantir-cost-tuning description: 'Optimize Palantir Foundry costs through compute tuning, incremental builds, and usage monitoring.

Use when analyzing Foundry compute costs, reducing API usage,

or implementing cost monitoring for Foundry workloads.

Trigger with phrases like "palantir cost", "foundry billing",

"reduce foundry costs", "foundry pricing", "foundry expensive".

' allowed-tools: Read, Grep version: 2.0.0 license: MIT author: Jeremy Longshore jeremy@intentsolutions.io tags:

• saas
• palantir
• foundry
• cost
• optimization compatibility: Designed for Claude Code, also compatible with Codex and OpenClaw

Palantir Cost Tuning

Overview

Optimize Foundry compute and API costs through incremental transforms, right-sized Spark profiles, efficient pagination, and usage monitoring.

Prerequisites

• Active Foundry enrollment with build history
• Access to Foundry resource usage metrics
• Understanding of transform build patterns

Instructions

Step 1: Cost Drivers in Foundry

Step 2: Convert Full Rebuilds to Incremental

from transforms.api import transform_df, Input, Output, incremental

# BEFORE: Full rebuild every run (expensive for large datasets)
@transform_df(Output("/out"), data=Input("/in"))
def expensive(data):
    return data.filter(data.status == "active")

# AFTER: Only processes new/changed rows
@incremental()
@transform_df(Output("/out"), data=Input("/in"))
def cheap(data):
    return data.filter(data.status == "active")

Step 3: Right-Size Spark Profiles

from transforms.api import configure

# DON'T: Default profile for everything
# DO: Match profile to actual data size

# Small data (< 1GB) — use lightweight transforms (no Spark)
from transforms.api import transform_polars
@transform_polars(Output("/out"), data=Input("/small_table"))
def small_job(data):
    return data.filter(data["status"] == "active")

# Medium data (1-50GB) — default profile is fine
@transform_df(Output("/out"), data=Input("/medium_table"))
def medium_job(data):
    return data.select("id", "name")

# Large data (50GB+) — explicit large profile
@configure(profile=["DRIVER_MEMORY_LARGE"])
@transform_df(Output("/out"), data=Input("/big_table"))
def large_job(data):
    return data.groupBy("region").count()

Step 4: Replace Polling with Webhooks

# EXPENSIVE: Polling every 30 seconds
import time
while True:
    result = client.ontologies.OntologyObject.list(
        ontology="co", object_type="Order", page_size=100,
    )
    process_new_orders(result.data)
    time.sleep(30)  # 2,880 API calls/day!

# CHEAP: Webhook-driven (0 polling API calls)
# Register webhook for ontology.object.created events
# See palantir-webhooks-events skill

Step 5: Monitor Usage

def log_api_usage(response):
    """Log rate limit headers to track usage patterns."""
    remaining = response.headers.get("X-RateLimit-Remaining", "?")
    limit = response.headers.get("X-RateLimit-Limit", "?")
    print(f"API usage: {remaining}/{limit} remaining")

Output

• Incremental transforms reducing rebuild compute by 90%+
• Right-sized Spark profiles matching actual data volumes
• Webhook-driven architecture eliminating polling costs
• Usage monitoring for ongoing optimization

Error Handling

Resources

• Incremental Transforms
• Transform Polars
• @configure Profiles

Next Steps

For reference architecture, see palantir-reference-architecture.