Skip to content

The .synth Format

.synth files are test specifications written in natural language. Synthesize interprets them and generates executable test code for your chosen framework.

Basic YAML Format

suite: User Authentication
context: Web application, login page at /auth/login

tests:
  - Successful login redirects to dashboard
  - Bad password shows error without revealing if username exists
  - Account locks after 5 failed attempts

Three parts:

  • suite — a name for the group of tests
  • context — what the AI needs to know about your application: what it is, where things live, relevant domain details. Better context produces better tests.
  • tests — plain-English statements of what should be true

Extended Format (with metadata)

Add per-test names, intent, and priority when you need finer control:

suite: Data Migration Validation
context: >
  MySQL database, compliance_records table.
  Migrated from Elasticsearch; source counts available via API.

tests:
  - name: Record count matches source
    intent: Verify total record count in MySQL matches ES source
    priority: critical

  - name: No orphaned junction records
    intent: Check every junction record has a valid parent
    priority: high

Plain Text Format

For quick tests, skip YAML entirely — one test per line:

Test that the login page loads in under 2 seconds
Test that the search returns results for "compliance audit"
Test that the export button generates a valid CSV

Environment Blocks

Target APIs, databases, or specific environments:

suite: User API
context: REST API for user management
environment:
  type: api
  base_url: ${API_URL}

tests:
  - GET /users returns 200 with user list
  - POST /users creates new user
  - DELETE /users/:id returns 404 for non-existent user

Environment variables (${VAR}) resolve at generation time. Combine with data profiles for per-environment values.

Compliance Tags

Map tests to regulatory requirements for traceability reporting:

suite: HIPAA Compliance Validation
context: Patient portal, PHI data handling
compliance: [HIPAA-164.312(a)(1), HIPAA-164.312(e)(1)]

tests:
  - PHI is encrypted in transit
  - Access logs record all PHI access
  - Session timeout enforces 15-minute limit

Run synthesize compliance -r requirements.yaml -t your.synth --report to generate audit-ready traceability reports. See the CLI Reference.

Writing Tips

Give real context. "Web app" generates generic tests. "E-commerce checkout at /checkout, Stripe payment form, guest checkout allowed" generates tests that match your application.

One assertion per test line. "User can login and sees the dashboard and the welcome banner shows their name" is three tests. Write them as three lines.

Name concrete values when they matter. "Session expires after 30 minutes" beats "session expires eventually."

Let the AI handle mechanics. Don't write selectors, waits, or setup steps — describe outcomes. Self-healing keeps the generated mechanics working as your app changes.