feast-architecture

name: feast-architecture description: Internals of the Feast codebase — how each component works, where the key abstractions live, and the data flow through the system. Use when asked how feast apply works, how the registry stores data, how materialization moves data, how get_online_features retrieves features, how the feature server works, how the Kubernetes operator manages deployments, or when navigating the codebase to understand where to make a change. license: Apache-2.0 compatibility: Works with Claude Code, OpenAI Codex, and any Agent Skills compatible tool. metadata: author: feast-dev version: "1.0"

Feast Architecture Internals

Full Component Map

┌─────────────────────────────────────────────────────────────────┐
│                     Feast Deployment Modes                       │
│                                                                   │
│  Local / Python SDK          Kubernetes (feast-operator)          │
│  ─────────────────           ─────────────────────────────────   │
│  feature_store.yaml    ←──   FeatureStore CR (CRD)               │
│        │                            │                             │
│        ▼                            ▼                             │
│  FeatureStore (Python)      Operator deploys services:            │
│    ├── Registry               - feature-server (Go or Python)     │
│    ├── Provider               - offline-store-server              │
│    │   ├── OnlineStore        - registry-server                   │
│    │   └── OfflineStore     + manages feature_store.yaml config   │
│    └── FeatureServer                                              │
│        (Python FastAPI or                                         │
│         Go gRPC/HTTP)                                             │
└─────────────────────────────────────────────────────────────────┘

Python SDK Core

FeatureStore — the orchestrator

File: sdk/python/feast/feature_store.py

FeatureStore is the single entry point for all operations. It never reads/writes data directly — it delegates to the registry (for metadata) and the provider (for infrastructure and data movement).

FeatureStore(repo_path=".")   # loads feature_store.yaml
store.apply(objects)           # register feature definitions
store.materialize(...)         # offline → online
store.get_online_features(...) # serve
store.get_historical_features(...) # training data

File: sdk/python/feast/repo_config.py — parses feature_store.yaml into typed RepoConfig. All component classes (online store, offline store, registry) are loaded dynamically from the type: string via repo_config.ONLINE_STORE_TYPE_MAP / OFFLINE_STORE_TYPE_MAP.

Registry

Purpose: Metadata store — persists definitions of entities, feature views, data sources, feature services, permissions.

Backends and their files:

How the proto/file backend works:

1. All metadata is serialized into one Registry protobuf (protos/feast/core/Registry.proto)
2. The proto blob is stored at the configured registry: path
3. In-memory cached_registry_proto is refreshed on a TTL (default 10s)
4. Writes re-serialize and overwrite the full blob — no partial updates

Key pattern — apply:

# Python object → proto → stored in registry blob
registry.apply_feature_view(feature_view, project)
# → feature_view.to_proto()
# → upserts into cached_registry_proto.feature_views
# → registry_store.update_registry_proto(proto)

Supporting files:

• infra/registry/base_registry.py — abstract interface
• infra/registry/proto_registry_utils.py — proto serialization helpers
• infra/registry/caching_registry.py — adds TTL caching on top of any backend

Provider

Purpose: Infrastructure lifecycle — creates/updates/tears down online store tables. Also dispatches online_write_batch and get_historical_features.

File: sdk/python/feast/infra/provider.py

Built-in providers (set via provider: in feature_store.yaml):

• local — SQLite online store, file offline (dev default)
• gcp — Datastore/Bigtable online, BigQuery offline
• aws — DynamoDB online, Redshift offline

Custom providers extend Provider and override update_infra / teardown_infra.

Online Store

Purpose: Low-latency feature serving. Stores the latest feature values per entity key.

Interface: sdk/python/feast/infra/online_stores/online_store.py Implementations: sdk/python/feast/infra/online_stores/ (redis, dynamodb, sqlite, bigtable, postgres, snowflake, …)

Key methods:

• online_write_batch — write entity→feature values
• online_read — read by entity keys
• update — provision/deprovision tables on feast apply
• teardown — clean up on feast teardown

Offline Store

Purpose: Historical feature retrieval and training data generation (point-in-time joins).

Interface: sdk/python/feast/infra/offline_stores/offline_store.py Implementations: sdk/python/feast/infra/offline_stores/ (bigquery, snowflake, redshift, duckdb, file, …)

Returns a RetrievalJob (lazy) — no data moves until .to_df() or .to_arrow() is called.

PIT join logic (shared): sdk/python/feast/infra/offline_stores/offline_utils.py

Key methods to implement for a new backend:

class MyOfflineStore(OfflineStore):
    def get_historical_features(self, config, feature_views, feature_refs,
                                entity_df, registry, project, ...) -> RetrievalJob: ...
    def pull_latest_from_table_or_query(self, config, data_source,
                                        join_key_columns, feature_name_columns,
                                        timestamp_field, created_timestamp_column,
                                        start_date, end_date) -> RetrievalJob: ...
    def pull_all_from_table_or_query(self, config, data_source, join_key_columns,
                                     feature_name_columns, timestamp_field,
                                     start_date, end_date) -> RetrievalJob: ...
    def write_logged_features(self, config, data, source, logging_config,
                              registry) -> None: ...  # optional

Config class: subclass FeastConfigBaseModel with a type Literal (short alias + full dotted path). Register in OFFLINE_STORE_TYPE_MAP in sdk/python/feast/repo_config.py.

Data source: each offline store backend pairs with a DataSource subclass (e.g. BigQuerySource, FileSource). Add it to sdk/python/feast/data_sources/ and register in DATA_SOURCE_CLASS_FOR_TYPE.

Data Flows

feast apply

feast apply  (CLI → repo_operations.py)
  ├── Parse Python files → collect FeastObjects
  ├── store.apply(objects)
  │     ├── diff against registry (diff/registry_diff.py)
  │     ├── update registry metadata for changed objects
  │     └── provider.update_infra(tables_to_keep, tables_to_delete)
  │           └── online_store.update(...)  ← create/drop tables
  └── Write updated registry to storage

feast materialize

store.materialize(start_date, end_date)
  ├── Load feature views from registry
  ├── For each feature view:
  │     ├── offline_store.pull_latest_from_table_or_query(...)
  │     │     └── Returns RetrievalJob (lazy)
  │     ├── job.to_arrow()  ← executes query, fetches Arrow table
  │     └── provider.online_write_batch(...)
  │           └── online_store.online_write_batch(config, table, data, progress)
  └── Update last_updated_timestamp in registry

get_online_features

store.get_online_features(features, entity_rows)
  ├── Resolve feature refs → FeatureViews from registry
  ├── online_store.online_read(config, table, entity_rows, requested_features)
  │     └── Deserialize ValueProto → Python dict
  ├── Apply OnDemandFeatureView transformations (if any)
  └── Return OnlineFeaturesResponse

get_historical_features

store.get_historical_features(entity_df, features)
  ├── Resolve feature refs → FeatureViews from registry
  ├── offline_store.get_historical_features(config, feature_views, entity_df)
  │     └── Point-in-time join:
  │           for each entity row, find latest values where
  │           event_timestamp ≤ entity_df.event_timestamp
  │           (prevents data leakage in training)
  └── Returns RetrievalJob → .to_df() / .to_arrow()

Feature Servers

Python Feature Server (FastAPI)

File: sdk/python/feast/feature_server.py

A FastAPI app that wraps FeatureStore. Started with feast serve.

Endpoints:

• POST /get-online-features — online feature retrieval
• POST /push — push features to online/offline store
• POST /materialize — trigger materialization
• GET /health — health check

The app loads feature_store.yaml at startup, creates a FeatureStore, and periodically refreshes the registry in the background (async timer).

Go Feature Server

Directory: go/ Entry point: go/main.go

A high-performance alternative to the Python feature server, written in Go. Supports HTTP, HTTPS, and gRPC transports:

go run go/main.go -type http  -port 6566
go run go/main.go -type grpc  -port 6566

Key packages:

• go/internal/feast/ — Go port of FeatureStore (reads feature_store.yaml, calls online store)
• go/internal/feast/server/ — HTTP and gRPC server implementations
• go/internal/feast/server/logging/ — feature logging to offline store

The Go server reads the registry directly (proto file or remote) and calls the online store. It does not support feast apply or materialization — those remain Python-only.

Feast Operator (Kubernetes)

Directory: infra/feast-operator/ Language: Go (controller-runtime / kubebuilder)

The operator manages the full lifecycle of a Feast deployment on Kubernetes via a FeatureStore Custom Resource Definition (CRD).

CRD: FeatureStore

API version: feast.dev/v1 File: infra/feast-operator/api/v1/featurestore_types.go

apiVersion: feast.dev/v1
kind: FeatureStore
metadata:
  name: my-feast
spec:
  feastProjectName: my_project
  services:
    offlineStore:
      persistence:
        file:
          type: dask
    onlineStore:
      persistence:
        store:
          type: redis
          secretRef:
            name: redis-credentials
    registry:
      local:
        persistence:
          file:
            path: /data/registry.db

What the operator manages

Reconcile loop

File: infra/feast-operator/internal/controller/featurestore_controller.go

The FeatureStoreReconciler.Reconcile method runs on every CR change:

1. Fetches the FeatureStore CR
2. Calls deployFeast() → creates/updates Deployments, Services, ConfigMaps
3. Updates CR status conditions (OfflineStore, OnlineStore, Registry ready conditions)
4. Watches owned resources; re-reconciles on any change

Supporting services logic: infra/feast-operator/internal/controller/services/

Serialization Layer

All persistent metadata and the feature server wire format use Protocol Buffers.

Python object (FeatureView, Entity, ...)
    ├── .to_proto()   → Protobuf message → stored in registry or sent over gRPC
    └── .from_proto() ← Protobuf message

Proto definitions:
  protos/feast/core/    # registry objects (FeatureView, Entity, DataSource, …)
  protos/feast/serving/ # serving API (GetOnlineFeaturesRequest/Response)
  protos/feast/types/   # Value, EntityKey, Field

When adding a new field to a Feast object:

1. Update the .proto file
2. Run make compile-protos-python (and make compile-protos-go if applicable)
3. Update .to_proto() and .from_proto() in the Python class

Key Files Quick Reference

Official Architecture Documentation

The docs/ directory contains user-facing architecture documentation that complements this skill: