test-and-fix-connector

name: test-and-fix-connector description: "Run the AdmeOsduLakeflowConnect pytest suite, diagnose failures, and fix the connector or simulator until everything passes. Branches on mode={simulate|live}."

Test and Fix the ADME OSDU Connector

Goal

Validate AdmeOsduLakeflowConnect by running its pytest suite and applying minimal fixes until everything passes.

0. Rules (both modes)

• Run pytest synchronously — never in background. Never use sleep, poll loops, or tail.
• Do NOT run exploratory scripts before the test file exists.
• Read implementation source before diagnosing failures.

1. Setup (skip if test file already exists)

Verify these files exist:

• connector/lakeflow/interface.py
• connector/lakeflow/adme_osdu.py
• connector/simulator/corpus/wellbore.json
• connector/simulator/corpus/reservoir.json
• connector/simulator/corpus/rock_and_fluid.json
• connector/simulator/corpus/legal_tags.json
• connector/simulator/corpus/entitlements.json
• tests/unit/adme_osdu/test_adme_osdu_lakeflow_connect.py

SIMULATE MODE

S1. Install dependencies

pip install -e ".[dev]" -q 2>/dev/null || pip install -r requirements-dev.txt -q

S2. Run simulate-mode tests

pytest tests/unit/adme_osdu/ -v

No CONNECTOR_TEST_MODE env var. No credentials. No network.

S3. Diagnose failures

• Schema mismatch — connector returns field not in schema, or schema field missing from record
• Corpus bug — primary key is None, modifyTime missing, JSON malformed
• Routing bug — search handler picks wrong corpus file (check kind field routing in conftest.py)
• Pagination loop — read_terminates fails because offset never stabilises; check _read_domain returns start_offset unchanged when results is empty

Fix the connector or corpus as needed. Iterate until all tests pass.

LIVE MODE

L1. Credentials

Place tests/unit/adme_osdu/dev_config.json (gitignored):

{
  "base_url": "https://YOUR-ADME.energy.azure.com",
  "data_partition_id": "YOUR-PARTITION",
  "access_token": "YOUR-BEARER-TOKEN"
}

Or set CONNECTOR_TEST_CONFIG_JSON env var.

L2. Run live tests

CONNECTOR_TEST_MODE=live pytest tests/unit/adme_osdu/ -v

L3. Seed corpus from live responses

After a successful live run, update corpus files with real data:

• Copy representative records from the live responses into connector/simulator/corpus/*.json
• Keep 3–5 records per domain; real data improves simulate-mode realism

L4. Final simulate pass

pytest tests/unit/adme_osdu/ -v

Must still be green.

Debugging Hangs (live mode)

If a test hangs on a domain table:

1. Set a tight page_size — edit conf/domains/wellbore.yaml and set page_size: 5
2. Enable debug logging: pytest ... -s --log-cli-level=DEBUG
3. If search still hangs, check incremental_filter_template — an unbounded query: "" on large partitions can time out

Merge note

After fixing the connector, no merge script is needed (this is a standalone repo, not the community connector monorepo).