RF-39249 Add AI Test Generation Command#500

sebaherrera07 · 2026-02-03T11:50:51Z

Tested manually ☑️

Introduces AI-powered test generation to the Rainforest CLI, allowing users to create tests using natural language prompts.

New Commands

rainforest generate - Generate a new test from a natural language prompt

rainforest generate "Log in with valid credentials" --title "Log In Flow"  --start-uri /login --platform windows11_chrome

Key Features

Natural language prompt input
Support for --start-uri or --url (mutually exclusive)
Support for --title
Optional --environment-id flags
Login via --credentials (free-form string passed to AI) or --login-snippet-id (mutually exclusive)
Single platform support via --platform (validated with [a-z0-9_] pattern)

Implementation Highlights

Added CreateTestWithAI() API method
Comprehensive validation (mutual exclusivity, required fields, platform format)
Single browser enforcement (API limitation)
Backend handles defaults for platform and title when not specified

Version: 3.8.0

marvin-rfbot · 2026-02-03T11:50:54Z

https://rainforest.atlassian.net/browse/RF-39249

sebaherrera07 · 2026-02-11T17:58:41Z

/reviewme @ubergeek42

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

ubergeek42

I think I have a bunch of questions/concerns about RFML/download/export, because I don't think that's a thing for automated tests.

ubergeek42 · 2026-02-11T18:48:07Z

+Download the generated test as an RFML file after creation.
+
+```bash
+rainforest generate "Search for a product" --start-uri /search --download
+```


I don't think this makes any sense. RFML is for plain english tests, which test generation doesn't produce.

ubergeek42 · 2026-02-11T18:50:21Z

+				},
+				cli.StringFlag{
+					Name:  "credentials",
+					Usage: "Credentials information to pass to the AI for test generation. Mutually exclusive with --login-snippet-id.",


This help text seems like it could be better. What is the expected format for this? Is it just an opaque string that gets passed into some AI model under the hood, so you can write whatever you want here? Or do they need to be username: whatever; password: something_else?

ubergeek42 · 2026-02-11T18:50:34Z

+				cli.BoolFlag{
+					Name:  "download, d",
+					Usage: "Download the generated test as an RFML file after creation.",
+				},


Likewise, I don't think this makes sense.

ubergeek42 · 2026-02-11T18:52:19Z

+	if len(browserNames) == 0 {
+		browserName = "windows11_chrome"


Does the backend require us to pass a value for platform? If we leave it blank/unset/null in the api request, will it default to something sane on the backend?

I'd prefer not to set a default here if the backend will already do so, as it makes changing the default later on more annoying (two places instead of one)

ubergeek42 · 2026-02-11T18:54:22Z

+// parseAndValidateBrowser parses the platform flag for AI test generation.
+// Returns the browsers array with a single browser name.
+func parseAndValidateBrowser(c cliContext) ([]string, error) {
+	browserNames := c.StringSlice("platform")


I'm a little curious why we're doing any processing on this to turn it into a list, just to say "hey, we don't support this being a list". Shouldn't it just be that we only accept a single platform, and let the backend validate it as a valid platform or not. And if we want to get fancier, just say that anything that contains characters other than [a-z0-9_] isn't valid.

ubergeek42 · 2026-02-11T18:56:29Z

+
+	title := c.String("title")
+	if title == "" {
+		// Use first 50 characters (runes) of prompt as default title


Out of scope, but I wonder if we should just send in an empty title, and have the backend generate one based on what actually got created. Similar to how chatgpt/gemini/etc give a name to your chat after your initial message that "summarizes" what it's about.

But similar to the above, I don't like setting a default/passing it in if we can avoid it and let the backend choose a sane default. I don't know what the backend behavior currently is, so maybe this is fine for now.

ubergeek42 · 2026-02-11T20:25:32Z

+	log.Printf("Successfully created test #%d: %s", response.TestID, response.Title)
+	log.Printf("RFML ID: %s", response.RFMLID)
+	log.Printf("State: %s", response.State)


Seems like this needs some note about how the test is technically created, but is still processing/generating? And RFMLID should go away.

sebaherrera07 · 2026-02-12T18:55:22Z

/reviewme @ubergeek42

Add support for generating tests using AI through the CLI. The new `generate` command (with `gen` and `ai` aliases) accepts a natural language prompt and creates a test via the Rainforest API. Features: - Generate tests from natural language prompts - Support for --start-uri or --url to specify the starting point - Optional --title flag for custom test name - Optional --environment-id flag for test environment - Support for authentication via --credentials or --login-snippet-id (mutually exclusive); credentials is a free-form string passed to the AI model - Platform selection with --platform (single value, validated with [a-z0-9_] pattern, backend provides default if omitted) - Comprehensive validation and error handling Example usage: rainforest generate "Log in with valid credentials" --start-uri /login rainforest generate "Add item to cart" --url https://example.com

Update version constant and add CHANGELOG entry for the new `generate` command feature.

marvin-rfbot Bot assigned sebaherrera07 Feb 3, 2026

sebaherrera07 requested a review from Copilot February 3, 2026 12:00

Copilot started reviewing on behalf of sebaherrera07 February 3, 2026 12:01 View session