CLI Reference¶
Every command in the synthesize CLI, organized by group. Run synthesize --help or synthesize <command> --help for full option details.
Core Workflow¶
generate — Turn .synth files into test code¶
synthesize generate <file.synth> # Robot Framework (default)
synthesize generate <file.synth> --format <fmt> # robot | playwright | cypress | selenium
synthesize generate <file.synth> -o ./generated # Custom output directory
synthesize generate <file.synth> --knowledge # Use the project knowledge store
synthesize generate <file.synth> --resolve-data -e staging # Inject data profiles
synthesize generate <file.synth> --docker # Generate and run in Docker
run — Generate and execute¶
synthesize run <file.synth> # Generate + execute
synthesize run <file.synth> --dry-run # Generate + validate only
synthesize run a.synth b.synth c.synth # Multiple files
# Self-healing modes
synthesize run <file.synth> --heal auto # Fix failures automatically
synthesize run <file.synth> --heal supervised # Propose fixes for approval
synthesize run <file.synth> --heal aggressive # Broader fix strategies
synthesize run <file.synth> --heal off
synthesize run <file.synth> --heal auto --max-retries 3
# Smart selection
synthesize run <file.synth> --smart # Risk-based test selection
synthesize run <file.synth> --smart --git-diff HEAD~1
synthesize run <file.synth> --smoke # Top 10% by risk
synthesize run <file.synth> --target "login" # Filter by pattern
# Gates and extras
synthesize run <file.synth> --review # Quality review gate
synthesize run <file.synth> --resolve-data -e staging
synthesize run <file.synth> --notify
review — Quality-check generated tests¶
synthesize review output/tests.robot
synthesize review output/tests.robot --strict relaxed|standard|strict|paranoid
synthesize review output/tests.robot --fix # Auto-fix issues
synthesize review output/tests.robot --format json
synthesize review output/tests.robot -o ./reports
chat — Conversational test authoring¶
synthesize chat # Interactive session
synthesize chat --session my-session # Resume a saved session
synthesize chat --no-knowledge # Skip the knowledge store
synthesize chat --no-stream
init — Create a project¶
Authentication¶
synthesize auth login # Browser-based device login
synthesize auth login --token synth_KEY # Headless / CI
synthesize auth status # Email, plan, usage, key prefix
synthesize auth logout
Importing & Spec-Based Generation¶
import — Migrate existing tests to .synth¶
synthesize import existing.robot
synthesize import existing.robot --ai-enrich # AI-enhance descriptions
synthesize import --bulk ./robot-tests --pattern "*.robot"
synthesize import existing.robot -o ./imported
synthesize import existing.robot --merge # Merge into existing .synth
spec — Generate from OpenAPI and other specs¶
synthesize spec generate openapi.yaml
synthesize spec generate openapi.yaml --negative # Include negative tests
synthesize spec generate openapi.yaml --constraints # Constraint-violation tests
synthesize spec generate openapi.yaml --ai-enrich
synthesize spec generate --bulk ./specs --pattern "*.yaml"
synthesize spec generate openapi.yaml --merge
analyze — Scan source code, generate tests¶
synthesize analyze scan ./src -o ./generated-synth # Generate .synth from code
synthesize analyze detect ./src # Detect application flows
synthesize analyze components ./src # List components
synthesize analyze scan ./src --format extended
Knowledge & Data¶
index — Project knowledge store¶
synthesize index ./src # Index project files
synthesize index ./src --force # Re-index
synthesize index --clear
data — Test data profiles¶
synthesize data create <name> --type user|token|config|record
synthesize data create <name> --type user --ai # AI-generated values
synthesize data create <name> --type user -e staging # Per-environment
synthesize data list [-e staging]
synthesize data show <name> [-e staging]
synthesize data generate <file.synth> --save # Derive profiles from tests
synthesize data resolve <file.synth> [-e staging] # Resolve placeholders
synthesize data delete <name> [-e staging]
registry — Shared keywords, locators, endpoints¶
synthesize registry --list
synthesize registry --search "login"
synthesize registry --type keyword|locator|endpoint
Development Loops¶
watch — TDD mode¶
synthesize watch <file.synth> # Auto-run on save
synthesize watch <file.synth> --heal auto
synthesize watch <file.synth> --review
synthesize watch <file.synth> --debounce 2000
synthesize watch <file.synth> --clear # Clear screen between runs
Requires the watch extra: pip install "synthesize-ai[watch]".
dashboard — Local results dashboard¶
synthesize dashboard # Generate static dashboard
synthesize dashboard -o ./dashboard
synthesize dashboard --serve --port 8080 # Live server
synthesize dashboard --no-open
Analytics & Quality¶
trends — Suite-level metrics over time¶
synthesize trends
synthesize trends --metric pass_rate|fail_rate|heal_rate|flaky
synthesize trends --metric duration --test "Test Name"
synthesize trends --window 7
synthesize trends --format json
flaky — Flaky test detection & quarantine¶
synthesize flaky
synthesize flaky --window 14 --threshold 0.3
synthesize flaky --quarantine "Test Name"
synthesize flaky --list-quarantined
synthesize flaky --unquarantine "Test Name"
synthesize flaky --format json
analytics — Deeper insights¶
synthesize analytics insights [--days 14]
synthesize analytics flaky-tests [--min-runs 5 --min-rate 0.2]
synthesize analytics flake "Test Name" [--days 30]
synthesize analytics regressions [--threshold 15.0]
synthesize analytics trend "Test Name" [--days 60]
coverage — Map tests to source code¶
synthesize coverage scan <file.synth> -s ./src
synthesize coverage gaps <file.synth> -s ./src
synthesize coverage list [--file "src/auth.py"] [--test "Login Test"]
synthesize coverage report [--format json]
Cost Management¶
synthesize cost report [--days 7] [--format json] # LLM spend analysis
synthesize cost stats # Cache hit rates
synthesize cost providers [--format json] # Per-provider breakdown
synthesize cost clear-cache [--days 30]
Security & Compliance¶
security¶
synthesize security scan ./src # Full scan
synthesize security secrets ./src # Secrets detection only
synthesize security scan ./src -e ".py" -e ".env" # Specific extensions
synthesize security auth --target <url> -e /api/login -e /api/admin
synthesize security report [--format json]
compliance — Requirement traceability¶
synthesize compliance -r requirements.yaml -t <file.synth>
synthesize compliance -r requirements.yaml -t <file.synth> --report --format html|json|md|all
synthesize compliance -r requirements.yaml -t <file.synth> --save-baseline
synthesize compliance -r requirements.yaml -t <file.synth> --baseline ./baseline.json
synthesize compliance -r requirements.yaml -t <file.synth> --gate # Exit non-zero on failure
CI/CD¶
synthesize ci context # Detect CI environment
synthesize ci init --platform github|gitlab|jenkins
synthesize ci run <file.synth> --junit --artifacts --gate --heal auto
synthesize ci run <file.synth> --comment # Post PR comment
synthesize ci run <file.synth> --notify
Execution Environments¶
docker¶
synthesize docker status
synthesize docker build [-t my-tests:latest]
synthesize docker run output/tests.robot [-s selenium-chrome] [--timeout 120] [--no-cleanup]
synthesize docker compose output/tests.robot [-s selenium-chrome -s postgres]
synthesize docker cleanup [--images --force]
mock — Local mock API server¶
synthesize mock fixture create <name> -m GET -p /api/users -s 200 -b '{"users": []}'
synthesize mock fixture list
synthesize mock fixture delete <name> --force
synthesize mock start -p 8888 [--foreground] [-f ./fixtures/]
synthesize mock status
synthesize mock stop
Performance Testing¶
synthesize performance run <url> -p smoke|load|stress|spike|soak
synthesize performance run <url> --users 50 --requests 100 --duration 30
synthesize performance run <url> --method POST --header "Content-Type:application/json" --body '{"key":"value"}'
synthesize performance run <url> -p load --save-baseline
synthesize performance run <url> -p load --check-baseline
synthesize performance profiles
synthesize performance trend --profile load --url <url>
synthesize performance baseline list [--profile smoke]
synthesize performance baseline delete <id>
Visual Regression¶
synthesize visual baseline "Suite" "page-name" ./screenshots/page.png
synthesize visual compare "Suite" ./screenshots/ -t 0.95
synthesize visual list [-s "Suite"] [-S pending|approved]
synthesize visual approve "Suite" ["page-name"] [--by "name"]
synthesize visual delete "Suite" ["page-name"] [--force]
Requires the visual extra: pip install "synthesize-ai[visual]".
Team Collaboration¶
Collaboration commands sync with a Synthesize team server and require an account.
synthesize collaborate library [-q "login"] [-d "auth"] [-t "security"]
synthesize collaborate share "Keyword Name" -d "Description" -t "tags"
synthesize collaborate standards output/tests.robot
synthesize collaborate sync
# Review workflow
synthesize collaborate review submit output/tests.robot -r "reviewer@example.com"
synthesize collaborate review list --status pending
synthesize collaborate review show <id>
synthesize collaborate review approve <id> -f "Feedback"
synthesize collaborate review reject <id> -r "Reason"
Promotion Pipelines¶
Gate-based promotion of test suites across environments (dev → staging → prod).
synthesize promote init
synthesize promote pipeline [--name production]
synthesize promote run <file.synth> --from dev --to staging
synthesize promote run <file.synth> --from staging --to prod --requested-by "name"
synthesize promote status [<request-id>]
synthesize promote approve <request-id>
synthesize promote rollback <request-id>
Plugins¶
synthesize plugin list
synthesize plugin info <name>
synthesize plugin formats
synthesize plugin install <git-url | local-path | registry-name>
synthesize plugin uninstall <name>
synthesize plugin enable <name>
synthesize plugin disable <name>
synthesize plugin create <name> [-o ./plugins]
See Plugins for the development guide.