developing-genkit-java

name: developing-genkit-java description: Best practices for developing with and contributing to Genkit Java — the open-source Java AI framework by Google. Covers project architecture, plugin development, flow definition, model integration, RAG pipelines, testing, naming conventions, and code quality guidelines. Use this skill when the user asks about building AI applications in Java with Genkit, creating custom plugins, defining flows, working with models, embedders, retrievers, tools, prompts, agents, or contributing to the Genkit Java codebase. argument-hint: Describe the Genkit Java task (e.g., "create an Anthropic plugin", "add a RAG flow", "define a tool")

Developing with Genkit Java

You are an expert on Genkit Java, the open-source Java AI framework by Google. This skill covers the full framework: architecture, plugin system, AI abstractions, and contribution guidelines.

Project Architecture

Genkit Java is a Maven multi-module project requiring Java 21+.

Module Hierarchy

genkit-java/
├── pom.xml                  # Parent POM (dependency management, plugins)
├── core/                    # Foundational abstractions (Action, Flow, Registry, Plugin, Middleware, Tracing)
│   └── com.google.genkit.core
├── ai/                      # AI-specific abstractions (Model, Tool, Embedder, Retriever, Indexer, Message, Part)
│   └── com.google.genkit.ai
├── genkit/                  # High-level user-facing API (Genkit class, Prompts, Sessions, Agents, Evaluators)
│   └── com.google.genkit
├── plugins/                 # Provider integrations (21 plugins)
│   └── com.google.genkit.plugins.{name}
└── samples/                 # Example applications (20+ samples)
    └── com.google.genkit.samples

Dependency Flow

core ← ai ← genkit ← plugins ← samples

• core has zero Genkit internal dependencies. It depends on Jackson, SLF4J, OpenTelemetry, victools JSON Schema.
• ai depends on core.
• genkit depends on core + ai + Handlebars (for .prompt files).
• plugins depend on genkit (or ai/core).
• samples depend on genkit + chosen plugins.

Key Dependencies (Managed in Parent POM)

Core Abstractions

Action — The Universal Unit

Every capability in Genkit is an Action<I, O, S>:

• I = Input type
• O = Output type
• S = Streaming chunk type (Void for non-streaming)

public interface Action<I, O, S> extends Registerable {
    String getName();
    ActionType getType();
    O run(ActionContext ctx, I input);
    O run(ActionContext ctx, I input, Consumer<S> streamCallback);
}

All AI primitives (Model, Tool, Embedder, Retriever, Indexer, Flow) implement Action. Actions self-register with the Registry using keys in the format {type}/{name} (e.g., model/openai/gpt-4o, flow/myFlow, tool/getWeather).

ActionType Enum

RETRIEVER, INDEXER, EMBEDDER, EVALUATOR, FLOW, MODEL, BACKGROUND_MODEL,
EXECUTABLE_PROMPT, PROMPT, RESOURCE, TOOL, TOOL_V2, UTIL, CUSTOM,
CHECK_OPERATION, CANCEL_OPERATION

ActionContext

Passed to every action execution. Carries tracing info, registry access, session state:

public class ActionContext {
    SpanContext spanContext;
    String flowName;
    Registry registry;
    String sessionId;
}

Registry

Centralized action discovery and lookup:

registry.registerAction(key, action);
registry.lookupAction("model/openai/gpt-4o");
registry.lookupAction(ActionType.FLOW, "myFlow");

The Genkit Class — Main Entry Point

The Genkit class is the high-level API. Always use the builder pattern:

Genkit genkit = Genkit.builder()
    .options(GenkitOptions.builder()
        .devMode(true)
        .reflectionPort(3100)
        .build())
    .plugin(new OpenAIPlugin())
    .plugin(new JettyPlugin())
    .build();

Lifecycle

1. Genkit.builder()...build() — creates instance, and initialize
2. genkit.stop() — cleanup resources

Defining Flows

Flows are user-defined actions exposed as HTTP endpoints:

// Simple (no context needed)
Flow<String, String, Void> greetFlow = genkit.defineFlow(
    "greet", String.class, String.class,
    (name) -> "Hello, " + name + "!");

// With ActionContext (for nested AI calls)
Flow<String, String, Void> jokeFlow = genkit.defineFlow(
    "tellJoke", String.class, String.class,
    (ctx, topic) -> {
        ModelResponse response = genkit.generate(
            GenerateOptions.builder()
                .model("openai/gpt-4o-mini")
                .prompt("Tell a joke about: " + topic)
                .config(GenerationConfig.builder().temperature(0.9).build())
                .build());
        return response.getText();
    });

// With middleware
Flow<String, String, Void> securedFlow = genkit.defineFlow(
    "secured", String.class, String.class,
    (ctx, input) -> processInput(input),
    List.of(authMiddleware, loggingMiddleware));

Generation API

Simple Generation

ModelResponse response = genkit.generate(
    GenerateOptions.builder()
        .model("openai/gpt-4o")
        .prompt("Explain quantum computing")
        .build());
String text = response.getText();

Streaming Generation

ModelResponse response = genkit.generateStream(
    GenerateOptions.builder()
        .model("openai/gpt-4o")
        .prompt("Write a story")
        .build(),
    chunk -> System.out.print(chunk.getText()));

Structured Output

MyPojo result = genkit.generateObject(
    GenerateOptions.<MyPojo>builder()
        .model("openai/gpt-4o")
        .prompt("Generate a recipe for pasta")
        .outputClass(MyPojo.class)
        .build());

Multi-turn Messages

ModelResponse response = genkit.generate(
    GenerateOptions.builder()
        .model("openai/gpt-4o")
        .messages(List.of(
            Message.system("You are a helpful assistant."),
            Message.user("What is the capital of France?"),
            Message.model("Paris is the capital of France."),
            Message.user("What about Germany?")))
        .build());

GenerationConfig

GenerationConfig.builder()
    .temperature(0.9)
    .maxOutputTokens(2048)
    .topK(40)
    .topP(0.95)
    .stopSequences(List.of("\n\n"))
    .build()

Tools — AI-Callable Functions

Define tools that models can invoke:

// With auto-generated JSON Schema from classes
Tool<WeatherInput, WeatherOutput> weatherTool = genkit.defineTool(
    "getWeather",
    "Get current weather for a location",
    (ctx, input) -> fetchWeather(input.getLocation()),
    WeatherInput.class, WeatherOutput.class);

// Use tools in generation
ModelResponse response = genkit.generate(
    GenerateOptions.builder()
        .model("openai/gpt-4o")
        .prompt("What's the weather in London?")
        .tools(List.of(weatherTool))
        .build());

RAG (Retrieval-Augmented Generation)

Embedding

EmbedResponse embeddings = genkit.embed(
    "openai/text-embedding-3-small",
    List.of(Document.fromText("Hello world")));

Indexing

genkit.index("devLocalVectorStore/my-index", documents);

Retrieval + Generation

List<Document> context = genkit.retrieve("devLocalVectorStore/my-index", query);

ModelResponse response = genkit.generate(
    GenerateOptions.builder()
        .model("openai/gpt-4o")
        .prompt(query)
        .docs(context)  // Inject retrieved documents
        .build());

DotPrompt — .prompt Files

Prompt files live in resources/prompts/ with Handlebars templates and YAML frontmatter:

---
model: openai/gpt-4o-mini
config:
  temperature: 0.9
  maxOutputTokens: 500
input:
  schema:
    ingredient: string
    style?: string
---
Create a recipe using {{ingredient}} in a {{style}} style.

Loading Prompts

ExecutablePrompt<RecipeInput> prompt = genkit.prompt("recipe", RecipeInput.class);

// With variant (recipe.robot.prompt)
ExecutablePrompt<RecipeInput> robotPrompt = genkit.prompt("recipe", RecipeInput.class, "robot");

Sessions & Chat

Session<MyState> session = genkit.createSession();
Chat<MyState> chat = genkit.chat(ChatOptions.<MyState>builder()
    .model("openai/gpt-4o")
    .session(session)
    .build());

Agents

Multi-agent systems with tool delegation:

Agent researchAgent = genkit.defineAgent(AgentConfig.builder()
    .name("researcher")
    .model("openai/gpt-4o")
    .description("Research specialist")
    .tools(List.of(searchTool, summarizeTool))
    .build());

Interrupts — Human-in-the-Loop

Tool<ConfirmInput, ConfirmOutput> confirmTool = genkit.defineInterrupt(
    InterruptConfig.<ConfirmInput, ConfirmOutput>builder()
        .name("confirmAction")
        .inputClass(ConfirmInput.class)
        .outputClass(ConfirmOutput.class)
        .build());

Evaluators

Evaluator<String> factualityEval = genkit.defineEvaluator(
    "factuality", "Factuality Check", "Checks factual accuracy",
    (datapoint) -> {
        // Return EvalResponse with score, rationale, detail
    });

EvalRunKey result = genkit.evaluate(RunEvaluationRequest.builder()
    .evaluators(List.of("factuality"))
    .dataset(dataset)
    .build());

Plugin Development

The Plugin Interface

public interface Plugin {
    String getName();
    List<Action<?, ?, ?>> init();
    default List<Action<?, ?, ?>> init(Registry registry) { return init(); }
}

Creating a New Plugin

1. Create module under plugins/{name}/ with its own pom.xml.
2. Package: com.google.genkit.plugins.{name}
3. Implement Plugin interface.
4. Return actions from init() — Models, Embedders, Tools, Retrievers, etc.

Standard Plugin Structure

plugins/my-provider/
├── pom.xml
├── README.md
└── src/main/java/com/google/genkit/plugins/my_provider/
    ├── MyProviderPlugin.java          # Plugin entry point
    ├── MyProviderPluginOptions.java   # Configuration POJO (builder pattern)
    ├── MyProviderModel.java           # Model implementation
    ├── MyProviderEmbedder.java        # Embedder (if applicable)
    └── ...

Plugin Implementation Pattern

public class MyProviderPlugin implements Plugin {
    public static final List<String> SUPPORTED_MODELS = List.of("model-a", "model-b");
    
    private final MyProviderPluginOptions options;
    
    public MyProviderPlugin(MyProviderPluginOptions options) {
        this.options = options;
    }
    
    public static MyProviderPlugin create() {
        return new MyProviderPlugin(MyProviderPluginOptions.builder().build());
    }
    
    @Override
    public String getName() {
        return "my-provider";
    }
    
    @Override
    public List<Action<?, ?, ?>> init() {
        List<Action<?, ?, ?>> actions = new ArrayList<>();
        for (String model : SUPPORTED_MODELS) {
            actions.add(new MyProviderModel(
                getName() + "/" + model, model, options));
        }
        return actions;
    }
}

compat-oai — Shared OpenAI-Compatible Base

Many plugins (OpenAI, Anthropic, XAI, DeepSeek, Mistral, Cohere, Groq) extend the compat-oai plugin which provides a shared base for OpenAI-compatible APIs. When building a plugin for an OpenAI-compatible provider, extend CompatOAIPlugin / CompatOAIModel instead of implementing from scratch.

Server Plugins

• JettyPlugin — Lightweight HTTP server, exposes flows as endpoints
• SpringPlugin — Spring Boot integration with auto-generated REST endpoints (GenkitFlowController)

Model Implementation

Models implement Action<ModelRequest, ModelResponse, ModelResponseChunk>:

public class MyModel implements Model {
    @Override
    public ModelInfo getInfo() { return modelInfo; }
    
    @Override
    public boolean supportsStreaming() { return true; }
    
    @Override
    public ModelResponse run(ActionContext ctx, ModelRequest request) {
        // Make HTTP call to provider API
        // Map response to ModelResponse
    }
    
    @Override
    public ModelResponse run(ActionContext ctx, ModelRequest request,
                            Consumer<ModelResponseChunk> streamCallback) {
        // SSE streaming via OkHttp
    }
}

Message & Part Model

Message.user("Hello")                          // User message
Message.system("You are a helper")             // System message
Message.model("Response text")                 // Model response
Message.tool(List.of(Part.toolResponse(...)))   // Tool result

Part.text("Hello")                             // Text content
Part.media("image/png", dataUrl)               // Media content
Part.toolRequest(name, ref, input)             // Tool call
Part.toolResponse(name, ref, output)           // Tool result

Middleware

Cross-cutting concerns applied to flows:

@FunctionalInterface
public interface Middleware<I, O> {
    O handle(I request, ActionContext context, MiddlewareNext<I, O> next)
        throws GenkitException;
}

// Example: logging middleware
Middleware<String, String> logger = (input, ctx, next) -> {
    log.info("Input: {}", input);
    String result = next.handle(input, ctx);
    log.info("Output: {}", result);
    return result;
};

Tracing & Observability

Genkit uses OpenTelemetry for tracing:

SpanMetadata metadata = SpanMetadata.builder()
    .name("myOperation")
    .type("model")
    .build();

Tracer.runInNewSpan(ctx, metadata, input, (spanCtx, req) -> {
    return doWork(req);
});

Naming Conventions

Packages

com.google.genkit                  # Main API
com.google.genkit.core             # Core abstractions
com.google.genkit.ai               # AI abstractions
com.google.genkit.prompt           # Prompt system
com.google.genkit.plugins.{name}   # Plugins (use underscores for multi-word: google_genai)
com.google.genkit.samples          # Sample apps

Classes

Methods

Action Keys

Format: {type}/{name}

flow/myFlow
model/openai/gpt-4o
tool/getWeather
embedder/openai/text-embedding-3-small
retriever/myStore/docs
executable-prompt/myPrompt

Testing Patterns

Framework

JUnit 5 (Jupiter) + Mockito. Tests mirror source structure in src/test/java/.

Typical Test

@Test
void testFlowExecution() {
    Registry registry = new DefaultRegistry();
    
    Flow<String, String, Void> flow = Flow.define(
        registry, "testFlow", String.class, String.class,
        (ctx, input) -> input.toUpperCase());
    
    ActionContext ctx = new ActionContext(registry);
    String result = flow.run(ctx, "hello");
    
    assertEquals("HELLO", result);
}

Test Conventions

• One test class per production class
• Test class name: {ClassName}Test.java
• Method names: test{Behavior} or descriptive camelCase
• Use @Test, @BeforeEach, @AfterEach annotations
• Mock external dependencies with Mockito

Sample Application Structure

samples/{provider}/
├── pom.xml          # Dependencies: genkit, plugin, jetty/spring
├── README.md        # Setup instructions
├── run.sh           # Execution script
└── src/main/java/com/google/genkit/samples/
    └── {Provider}Sample.java

Canonical Sample Pattern

public class MySample {
    public static void main(String[] args) throws Exception {
        JettyPlugin jetty = new JettyPlugin(
            JettyPluginOptions.builder().port(8080).build());
        
        Genkit genkit = Genkit.builder()
            .options(GenkitOptions.builder().devMode(true).reflectionPort(3100).build())
            .plugin(MyProviderPlugin.create())
            .plugin(jetty)
            .build();
        
        // Define flows
        genkit.defineFlow("myFlow", String.class, String.class,
            (ctx, input) -> {
                return genkit.generate(GenerateOptions.builder()
                    .model("provider/model-name")
                    .prompt(input)
                    .build()).getText();
            });
        
        jetty.start();
    }
}

Code Quality & Build

Commands

# Full build
mvn clean install

# Format code (Google Java Format)
mvn fmt:format

# Check formatting
mvn fmt:check

# Run tests
mvn test

# Build specific module
mvn -pl core clean install

# Build with dependencies
mvn -pl plugins/openai -am clean install

Code Style

• Formatter: Google Java Format 1.35.0
• Linter: Checkstyle 13.4.0 (google_checks.xml)
• Serialization: Jackson @JsonInclude(Include.NON_NULL) — always omit null fields
• Immutability: Use builder pattern for configuration objects
• Generics: Preserve type safety with Action<I, O, S> pattern throughout

Conventional Commits

All commits follow the format: <type>(<scope>): <subject>

Types

Scopes

core, ai, genkit, openai, google-genai, anthropic, jetty, spring, firebase, localvec, mcp, samples, deps

Examples

feat(ai): add streaming support for generate
fix(openai): handle rate limit errors gracefully
docs(samples): add RAG example README
feat!: rename Genkit.create() to Genkit.builder()

Error Handling

public class GenkitException extends RuntimeException {
    String errorCode;
    Object details;
    String traceId;
    
    public static Builder builder() { ... }
}

// Throw structured errors
throw GenkitException.builder()
    .errorCode("NOT_FOUND")
    .message("Model not found: " + modelName)
    .build();

Adding a New Module Checklist

When adding a new plugin or module:

1. Create directory under plugins/{name}/
2. Add pom.xml with parent reference to genkit-parent
3. Add module to root pom.xml <modules> section
4. Create package com.google.genkit.plugins.{name}
5. Implement Plugin interface
6. Create options class with builder pattern
7. Add tests in src/test/java/
8. Add README.md with setup instructions
9. Add sample app under samples/{name}/
10. Format code with mvn fmt:format