autofish-control

name: autofish-control description: Control Android devices with Autofish via the af CLI. Use this skill when you need reliable observe-act-verify loops, ref-based tapping, overlay management, screenshots, and recovery steps.

Autofish Control Skill

Use deterministic control with evidence at every step.

When To Use

• You need to navigate Android UI through Autofish (af).
• You need stable replay using refs (@nK) instead of coordinates.
• You need explicit verification after each action.

Setup

af config set remote.url "http://<IP>:<PORT>"
af config set remote.token "<token>"
af config set memory.db "$HOME/.config/af/af.db"
af config set output.default "text"
af config set artifacts.dir "$HOME/.config/af/artifacts"

Copy IP, PORT, and token from the Autofish Android app. For USB port forwarding, use the forwarded local address, for example http://127.0.0.1:<PORT>.

If the device is connected with adb and the App is new enough to write a connection hint, prefer:

af connect usb --device <ADB_SERIAL>

af connect usb writes the forwarded remote.url and connection.* USB metadata, verifies af health, and does not write remote.token. Keep the token copied from the app for observe, act, verify, and recover.

Environment variables also work: AF_URL, AF_TOKEN, AF_DB, AF_OUTPUT, AF_ARTIFACT_DIR.

• af health only requires URL.
• observe, act, verify, recover require URL + token.
• af memory ... is local-only and requires memory enabled.
• config list and config get remote.token redact tokens as <redacted>.
• Use --output json when parsing results in scripts.
• Check af --help first if the installed CLI may be older than this skill.
• If config/env is unavailable, put remote flags after the command group: af --session <session> observe --url <url> --token <token> page.

Choose one session per task:

SESSION="task-name"
af --session "$SESSION" observe page --field screen --field refs --max-rows 80

Do not reuse the default default session across unrelated tasks.

Workflow

Memory session rules:

• memory context and memory save use the global --session.
• memory log and memory stats filter with --for-session <session>.
• memory search and memory experience query cross-session history; do not treat global --session as a filter for them.

1. Baseline:

• af --session <session> observe page --field screen --field refs --max-rows 80
• af --session <session> memory context
• af memory search --app <current-app>
• af memory experience --app <current-app> --activity <current-activity> --page-fp "<page-fingerprint>"

2. One-step execution:

• Run exactly one action command.
• Run af --session <session> observe page (add --field refs if needed).
• Verify expected state.

3. Continue only from fresh observations. Do not run blind action chains.

4. After solving a non-trivial problem:

• af --session <session> memory save --app <pkg> --topic "<category>/<name>" --content "<what you learned>"

Memory Loop

For experience learning, keep this exact order:

1. observe the starting page.
2. Run one act.
3. observe again.
4. Run one verify.

act and recover invalidate the cached page fingerprint. A transition is recorded only when a fresh successful observation happens after the action and before verification.

observe page

Always returns: topActivity, mode, hasWebView, nodeReliability.

Use the returned screen.artifact saved path when full page rows are needed. observe screenshot and overlay commands are diagnostics only; they do not refresh memory context.

topActivity is null when the service cannot confirm a stable value. Re-observe before acting.

Tap Priority

1. af --session <session> act tap --by ref --value @nK
2. af --session <session> act tap --by text|desc|resid --value "<value>" [--exact-match]
3. af --session <session> act tap --xy X,Y

Other actions exist for non-tap flows: swipe, text, launch, stop, key, back, home.

Refs

• Refresh refs on dynamic pages: af --session <session> observe page --field refs.
• If ref tap fails with stale/unobserved errors, re-observe once and retry once.
• Do not assume alias index stability after UI updates.
• Do not auto-pick among ambiguous matching candidates.

Verify Priority

1. af --session <session> verify top-activity --expected "<activity>" --mode contains
2. af --session <session> verify text-contains --text "<text>"
3. af --session <session> verify node-exists --by text|desc|resource_id|class --value "<value>" [--exact-match]

top-activity --mode is contains or equals. text-contains is case-insensitive by default; use --case-sensitive only when exact case matters.

On WebView-heavy pages (hasWebView=true or nodeReliability=low), prefer text-contains. Do not rely on node/ref structure inside WebView content.

Recovery

When state is uncertain:

1. af --session <session> observe page --max-rows 120
2. af memory experience --app <pkg> --activity <act> --failure-cause <cause>
3. Choose one recovery:

• af --session <session> recover back --times 1 for modal/back-stack drift.
• af --session <session> recover home for lost navigation context.
• af --session <session> recover relaunch --package <pkg> for app reset.

4. af memory log --for-session <session> --status failed --limit 5
5. Re-run baseline.

Diagnostics

af --session <session> observe overlay set --enable --mark-scope all --refresh on --refresh-interval-ms 800
af --session <session> observe screenshot --annotate --hide-overlay --max-marks 120 --mark-scope interactive
af --session <session> observe overlay set --disable --mark-scope all --refresh off

Artifacts still write to disk when memory is disabled, but DB artifact ids may be absent.

Memory Topic Conventions

Guidelines

• Favor refs over coordinates.
• Keep --session stable and unique per task.
• Each action must be followed by observation, then verification.
• Use af memory experience before acting on unfamiliar pages.
• Use af --session <session> memory context to inspect the current cached page fingerprint.
• Use af memory log --for-session <session> and af memory stats --for-session <session> to inspect failures and session quality.