By name: lux description: Control the desktop using AI vision (computer-use). Use for GUI automation, clicking buttons, typing in applications, and interacting with desktop software.
LUX Computer-Use CLI
The lux CLI enables AI-powered desktop control. It takes screenshots, sends them to a vision AI, and executes the returned actions (clicks, typing, etc.) on your desktop.
This is for GUI automation - use it when you need to interact with graphical applications that can't be controlled via command line.
Quick Start
# Start a session with a goal
lux start "Open Firefox and navigate to github.com"
# Run until the task is complete (or limit reached)
lux run --max-steps 10
# Or run step-by-step for more control
lux step # Take screenshot, get actions, execute
lux step # Repeat until done
lux end # End session when finished
Commands
lux start <task>
Start a new computer-use session with a goal description.
lux start "Click the Settings icon and enable dark mode"
lux start "Open the terminal and run htop"
lux start "Fill out the login form with username 'test'"
Options:
--model <model>- AI model:auto(default),lux-actor-1(fast), orlux-thinker-1(complex tasks)--max-steps <n>- Maximum steps (default: 20 for actor, 100 for thinker)
Model selection in auto mode (the default):
- Actor is picked for short, direct tasks like "click X", "open Y", "type Z"
- Thinker is picked for complex/vague/multi-step goals like "research X and compare Y"
- You can always override with
--model lux-actor-1or--model lux-thinker-1
lux step
Execute one step: takes a screenshot, sends it to the AI, and executes the returned actions.
lux step
This is useful when you want to observe each step or intervene between actions.
lux run
Run steps continuously until the task is complete or the limit is reached.
lux run # Run up to 20 steps
lux run --max-steps 10 # Run up to 10 steps
lux end
End the current session early.
lux end
lux status
Show your usage and limits.
lux status
Action Types
The AI returns actions that are executed on your desktop:
| Action | Description | Example Argument |
|---|---|---|
click |
Single left click | "512, 384" (normalized 0-1000) |
left_double |
Double click | "512, 384" |
right_single |
Right click | "512, 384" |
drag |
Click and drag | "100, 100, 500, 500" (start to end) |
type |
Type text | "Hello World" |
hotkey |
Press key combination | "ctrl+c", "alt+tab" |
scroll |
Scroll in direction | "up", "down", "left", "right" |
wait |
Pause execution | "1000" (milliseconds) |
finish |
Task completed | - |
call_user |
Need human help | "Please enter password" |
Best Practices
1. Write Clear Task Descriptions
Be specific about what you want to accomplish:
# Good - specific and actionable
lux start "Click the blue 'Submit' button in the form"
# Bad - vague
lux start "Submit the form"
2. Use Step-by-Step for Debugging
When things don't work as expected, use lux step to see what the AI is doing:
lux start "Navigate to settings"
lux step # See what it clicks
lux step # Continue manually
3. End Sessions When Done
Always end sessions to free up resources:
lux end
4. Check Status for Limits
Monitor your usage:
lux status
Examples
Open a Browser and Navigate
lux start "Open Firefox and go to https://github.com"
lux run --max-steps 5
Fill Out a Form
lux start "Fill in the username field with 'testuser' and click Login"
lux run
Use Desktop Applications
lux start "Open the file manager and navigate to Documents folder"
lux run --max-steps 10
Complex Multi-Step Tasks
For complex tasks, use lux-thinker-1 model:
lux start "Open VS Code, create a new file called test.py, and write a hello world program" --model lux-thinker-1
lux run --max-steps 20
Limitations
- Rate Limited: 20 sessions/day, 50 steps/session (configurable by platform)
- Screenshot-Based: AI only sees what's on screen
- Timing: UI may need time to respond between actions
- No Authentication: Can't access protected content without credentials already entered
Troubleshooting
"No active session"
Start a session first:
lux start "your task here"
"Session not found or expired"
Sessions expire after 30 minutes of inactivity. Start a new one:
lux start "your task here"
"Step limit reached"
You've hit the per-session limit. End and start a new session:
lux end
lux start "continue from where I left off"
Actions not working
The AI might misidentify UI elements. Try being more specific:
lux start "Click the green button labeled 'Save' in the bottom right corner"
Security Notes
- Screenshots are sent to backend over encrypted connection
- LUX API key is stored on backend, never on your machine
- Sessions are tied to your agent identity via JWT
- All usage is logged for rate limiting and auditing