Skip to content

Plugins

Synthesize generates tests for six frameworks out of the box — Robot Framework, Playwright, Cypress, Selenium, Cucumber, and pytest. Plugins add more.

Managing Plugins

synthesize plugin list                # What's installed and enabled
synthesize plugin formats             # Formats available to `generate --format`
synthesize plugin info <name>         # Details: version, provides, dependencies
synthesize plugin install <source>    # git URL, local path, or registry name
synthesize plugin uninstall <name>
synthesize plugin enable <name>
synthesize plugin disable <name>

Install sources:

synthesize plugin install https://github.com/someone/synth-plugin-testcafe.git
synthesize plugin install ./my-local-plugin
synthesize plugin install testcafe        # From the plugin registry

Building a Plugin

Scaffold

synthesize plugin create my_framework -o ./plugins

This creates three files:

my_framework/
├── plugin.py        # The Plugin class — generation logic
├── plugin.yaml      # Metadata manifest
└── pyproject.toml   # Makes it pip-installable with the right entry point

The Plugin Class

A minimal plugin defines four identity properties and two prompt methods:

class Plugin(FrameworkPlugin):
    @property
    def name(self) -> str:
        return "my_framework"       # API/CLI identifier

    @property
    def label(self) -> str:
        return "My Framework"       # Display name

    @property
    def language(self) -> str:
        return "python"             # Editor syntax highlighting

    @property
    def extension(self) -> str:
        return ".py"                # Generated file extension

    def system_prompt(self) -> str:
        return (
            "You are an expert test engineer. Generate complete, "
            "executable test files.\n\n"
            "Rules:\n"
            "- Output ONLY the test file content. No markdown fences."
        )

    def generation_prompt(self, synth_content: str) -> str:
        return (
            "Generate a test file from this specification:\n\n"
            f"{synth_content}\n\n"
            "Output ONLY the test file content, nothing else."
        )

The two prompt methods are the heart of a plugin: system_prompt() sets the rules and persona for the AI, and generation_prompt() receives the parsed .synth content and asks for output. Prompt-writing tips:

  • Always end with "Output ONLY the test file content, no markdown fences" — this keeps generated files clean.
  • Encode your framework's conventions as explicit rules (naming, markers, assertion style).
  • Keep the system prompt stable; put per-request detail in the generation prompt.

The Manifest (plugin.yaml)

name: my_framework
label: My Framework
version: "1.0.0"
author: "Your Name"
description: "My Framework tests generated by Synthesize."
language: python
extension: ".py"
execution_mode: docker    # native | docker | export-only
can_execute: true
can_review: false
can_heal: false
tags:
  - custom

Optional Capabilities

Beyond generation, plugins can implement:

  • Execution — run generated tests and report pass/fail counts (enables run for your format)
  • Healing — classify failures and propose fixes (enables self-healing)
  • Configuration — declare settings users can adjust
  • Zip export — provide requirements.txt/package.json and a README when users download generated tests

Each is a small set of additional methods on the same class. Start with generation only — can_execute: false makes your plugin export-only, which is a perfectly good v1.

Distribution

The scaffolded pyproject.toml registers your plugin via a Python entry point, so distribution is just packaging:

# Local development install
pip install -e ./my_framework

# Or publish to PyPI and let users:
pip install synth-plugin-my-framework

Users can also install straight from your repo:

synthesize plugin install https://github.com/you/my_framework.git

CLI Plugins vs. Studio Plugins

  • Directory plugins live in ~/.synth/plugins/ with a plugin.yaml manifest and provide generators to the local CLI.
  • Framework plugins register via the entry-point mechanism above and integrate with Synthesize Studio's full generate/run/heal loop, including the web UI.

synthesize plugin list shows both kinds. On a standalone CLI install, Studio-hosted framework features simply don't appear — that's expected.

Publishing Checklist

  • [ ] name, label, language, extension set
  • [ ] system_prompt() and generation_prompt() produce clean, fence-free output
  • [ ] plugin.yaml metadata complete (author, description, tags)
  • [ ] Tested locally: synthesize plugin install ./my_framework && synthesize generate demo.synth --format my_framework
  • [ ] Published as a pip package or public git repo