update-terraform-fields

name: update-terraform-fields description: Guides modifying, patching, and updating resources and fields managed by the Terraform/legacy controller, including patching the local/vendored Terraform Google Beta provider.

KCC Update Terraform Fields (Agentic-Friendly Guide)

This skill provides step-by-step instructions for an AI agent to add or update fields or resources managed by the Terraform/legacy controller (distinguished by the label cnrm.cloud.google.com/tf2crd: "true" or the static mappings). It also covers patching the local/vendored copy of the Terraform Google Beta Provider (TPG Beta).

1. Pre-requisites & Codebase Discovery

Before making any changes, the agent must inspect the repository to locate the resource files:

Identify Kind and API Group:
- Determine the Kubernetes resource Kind (e.g., SpannerDatabase) and the api group (e.g., spanner.cnrm.cloud.google.com).
Find the Terraform Resource Name:
- Locate the service mapping file in config/servicemappings/<service>.yaml (e.g., config/servicemappings/spanner.yaml).
- Find the mapping block matching the Kind to extract the underlying Terraform resource name (e.g., google_spanner_database).
Verify the Controller Mode:
- Check if the resource is managed by the Terraform controller by verifying that config/servicemappings/<service>.yaml contains the kind.
- Also, search for the resource's Group and Kind in pkg/controller/resourceconfig/static_config.go to ensure DefaultController is set to k8s.ReconcilerTypeTerraform (or similar for legacy) and is NOT set to k8s.ReconcilerTypeDirect. Note that static_config.go is auto-generated by dev/tasks/generate_static_config.py and should not be edited manually.
Identify Go Types Existence:
- Run a search (e.g., find_by_name) for apis/<service>/<version>/<kind>_types.go.
- Agentic Pitfall: Some files omit the service prefix in their filename (e.g., healthcheck_types.go instead of computehealthcheck_types.go). If find_by_name returns 0 results, verify by running grep_search across apis/<service>/ for type <Kind> struct or WithKind("<Kind>").
- If it exists: Go to Section 3 (Case B).
- If it does NOT exist: Go to Section 3 (Case A).

2. Modifying the Vendored Terraform Provider (TPG Beta)

If the field or bug fix is not yet present in the vendored copy of TPG Beta under third_party/github.com/hashicorp/terraform-provider-google-beta/, patch the provider locally. Refer to .gemini/skills/update-tf-provider-for-resource/SKILL.md for detailed guidelines on comparing, aligning, and backporting upstream changes while preserving local patches.

Steps to Patch TPG Beta:

Locate the Provider Code:
- Locate the schema and flattener/expander files under third_party/github.com/hashicorp/terraform-provider-google-beta/google-beta/services/<service>/.
- Common files are named resource_<terraform_resource_name>.go or node_config.go.
Implement Changes:
- Add/update the field in the schema map (Schema: map[string]*schema.Schema).
- Ensure the field type (TypeSchema), description, optional/required attributes, and ForceNew properties match the GCP API behavior.
- Update the corresponding expander (which maps KRM/Terraform values to GCP SDK structures) and flattener (which maps GCP responses back to Terraform schema maps).
Handle Optional/ForceNew Block Flattening (Critical Pitfall):
- For optional blocks/structs that are immutable (ForceNew: true), the flattener MUST explicitly return an empty slice or nil if all attributes inside the block are empty. Otherwise, Terraform calculates a diff (from 0 blocks to 1 block containing default values) and fails the update.
- Example:
```
func flattenMyConfig(c *service.MyConfig) []map[string]interface{} {
    if c == nil || c.SomeAttribute == "" {
        return nil // Or return []map[string]interface{}{}
    }
    ...
}
```

Verify Compilation:

Run the compiler check locally:

go vet ./third_party/github.com/hashicorp/terraform-provider-google-beta/...

Stage Provider Changes:
- Stage the modified third-party files. You can commit them together with KCC changes as the repository no longer enforces subtree change isolation:
```
git add third_party/
```

3. Updating KCC CRD Definitions

Case A: The resource is NOT yet migrated to Go types in `apis/`

KCC dynamically generates the CRD directly from the Terraform schema:

Modify Service Mapping:
- Open config/servicemappings/<service>.yaml.
- Locate the resource definition block.
- Ensure the field is not listed in ignoredFields or ignoredOutputOnlySpecFields.
- Resource References: If the field points to another KCC resource, define it under resourceReferences:
```
resourceReferences:
  - tfField: encryption.default_kms_key_name
    key: kmsKeyRef
    gvk:
      kind: KMSCryptoKey
      version: v1beta1
      group: kms.cnrm.cloud.google.com
    targetField: self_link
```
- Observed Fields: If the field is output-only and should be updated in the resource status, specify it in observedFields.
Regenerate CRD Manifests:
- Run:
```
make manifests
```
- Run git diff config/crds/resources/ to verify that the new field is properly generated in the CRD schema.
Update Go Client:
- Run:
```
make generate-go-client
```
- Verify if any generated files under pkg/clients/generated/ are modified using git status or git diff. If there are modifications, stage and commit them.
Regenerate Resource Reference Docs (CRITICAL):
- Run:
```
make resource-docs
```
- Verify if any documentation or generated files under scripts/generate-google3-docs/resource-reference/generated/ are modified using git status or git diff. If there are modifications, stage and commit them.

Case B: The resource HAS Go types in `apis/`

If a <kind>_types.go file exists under apis/<service>/<version>/:

Update Go Types:
- Open apis/<service>/<version>/<kind>_types.go.
- Locate the <Kind>Spec struct (or the nested block struct).
- Add the field using the correct Go type and tag. Use camelCase for the json tag:
```
// +optional
MyNewField *string `json:"myNewField,omitempty"`
```
- Ensure you use appropriate kubebuilder annotations (e.g. // +optional, // +kubebuilder:validation:Enum).
- Resource References: If the new field is a resource reference (e.g., kmsKeyRef of type v1alpha1.ResourceRef), ensure the struct field is defined correctly.
Update Service Mapping (For Resource References):
- CRITICAL: The TF controller relies on service mappings to translate reference fields and map them to the underlying Terraform schema.
- Even though the resource has Go types, if you are adding a resource reference, you MUST also configure the reference under resourceReferences in config/servicemappings/<service>.yaml:
```
resourceReferences:
  - tfField: encryption.default_kms_key_name
    key: kmsKeyRef
    gvk:
      kind: KMSCryptoKey
      version: v1beta1
      group: kms.cnrm.cloud.google.com
    targetField: self_link
```
Regenerate CRD & Mappers:
- Run:
```
dev/tasks/generate-types-and-mappers
```
- Verify the generated files (_types.go tags, auto-generated mappers, and config/crds/resources/) via git diff.
Update Go Client:
- Run:
```
make generate-go-client
```
- Verify if any generated files under pkg/clients/generated/ are modified using git status or git diff. If there are modifications, stage and commit them.
Regenerate Resource Reference Docs (CRITICAL):
- Run:
```
make resource-docs
```
- Verify if any documentation or generated files under scripts/generate-google3-docs/resource-reference/generated/ are modified using git status or git diff. If there are modifications, stage and commit them.

4. Verification & E2E Testing

Once definitions are updated, the agent must verify correctness of the field using the fixture testing framework.

Check Copyright Headers:

If you created any new files, make sure they have the copyright header:

// Copyright 2026 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
...

Proceed to Test Skill:
- Refer to .gemini/skills/test-terraform-fields/SKILL.md to set up E2E tests, record golden files, check against MockGCP, and run linters.
Field Ownership Conflicts with oneOf:
- Gotcha: Some legacy resources are exempted from Server-Side Apply (SSA) for object creation in ratcheting.go. During fixture tests, the object is created using a legacy Create call, but updates are always applied via Apply (SSA). If your update switches between choices in a oneOf field (e.g., from rrdatas to rrdatasRefs), the legacy field is not owned by the SSA field manager and remains in the live object. This triggers a validation failure: "spec" must validate one and only one schema (oneOf). Found 2 valid alternatives.
- Solution: Enable Server-Side Apply for the resource by removing it from the exempted list in ratcheting.go. This ensures both creation and updates use SSA, and unreferenced fields under a oneOf choice are correctly cleaned up.
Local CI/CD Presubmit Verification (CRITICAL):
- To ensure that generated PRs do not fail CI/CD validation pipelines (validate-generated-files and validations), you MUST run the resource docs generation and the local presubmit verifications before submitting:
```
make resource-docs
dev/ci/presubmits/validate-generated-files
scripts/validate-prereqs.sh
```
- If these scripts or commands generate any updates to auto-generated mappers, CRDs, static configs, documentation, or GitHub Actions workflows, verify with git status and stage and commit them:
```
git add -A
git commit -m "chore: ensure clean generated state for CI/CD presubmits"
```