CLI
Integrate with CI/CD using BugBug Command Line Interface
You can operate BugBug via the Command Line Interface (CLI). This empowers you to integrate with any continuous integration (CI) or continuous deployment (CD) pipelines or build system hooks.
Install via NPM
Open your terminal. First of all, you need Node.js installed on your machine and npm installed.
You need Node.js version 20 or newer
After you have NodeJS and npm installed, simply run:
Remember that you need to have admin user permissions on NodeJS execution.
Get your API token
You need to take the API token of the project you want to run with CLI. You will find that in the BugBug web app in the Integrations tab from the side menu:
Then configure CLI with the project's API token:
Run tests from the terminal
On BugBug npm's page, you find the available commands. You can also just strike bugbug help to see what you can do and how.
Example: list suites within the connected project:
To run a test via the CLI, you need to find its ID. It's easy, just go to your test, expand 3 dots, and select Run via CLI or webhook.
Simply copy the command for running, open the terminal, paste, and run. The command looks as follows:
Find your suite ID
You can run the whole suite as well by going to Suites the tab, expanding the details, and selecting Run via CLI or webhook. The command is the same, but SUITE_ID is different. That's how we recognize you want to run the whole suite!
Run tests from your build pipeline
Update your CI/CD build scripts to see test results directly in your build management tool (for example, in Bitbucket)
Here's an example of what you can add to your build script:
You can also override individual variables from the command line with --variable variableName="customVariableValue". This allows you to run different test data combinations across environments; for example, you can set a different user password in prod and another in staging.
If you're ambitious
Command-line variable overrides allow you to test various combinations of test data. You could create a for each type of script and execute a suite with all the combinations.
When you run your pipeline, BugBug tests will be triggered, and your build will succeed only if all tests pass.
Also read: our advanced guide to automation testing for startups
Available commands list
Here is a list of the commands that may be in use with different options and/or flags:
bugbug help <option>
configremoteversion
N/A
Show help menu for a specific command.
bugbug config <option>
set-token <token>
N/A
You can use this option for the command to set a valid token from your web app project settings
bugbug remote <option>
list test
--no-waitExit immediately without waiting for the result--no-progressDon't show progress spinner--debugShow more data (like raw API response)
Returns a list of all existing tests
list suite
--no-waitExit immediately without waiting for the result--no-progressDon't show progress spinner--debugShow more data (like raw API response)
Returns a list of all existing test suites
list profile
--no-waitExit immediately without waiting for the result--no-progressDon't show progress spinner--debugShow more data (like raw API response)
Returns a list of all existing test profiles
run test <test-id>
--no-waitExit immediately without waiting for the result--no-progressExit immediately without waiting for the result--debugShow more data (like raw API response)--with-detailsShow result with details--profile"<profile-name>"Run with a specific, existing profile--variable"<variable-name>"Override default variable during a single run--reporterThe name of the reporter to use (default: "inline"). Instead of "inline" you can also set "junit" for the report to be exported to an XML file--output-pathThe path to save the test report. Relative to the current working directory
Runs a specific test based on its ID
stop test <test-run-id>
--no-progressExit immediately without waiting for the result--debugShow more data (like raw API response)--result-timeout <int>Modify the default result waiting time (minutes, default: 60)
Stop the test run based on its ID
run suite <suite-id>
--no-waitExit immediately without waiting for the result--no-progressExit immediately without waiting for the result--debugShow more data (like raw API response)--with-detailsShow result with details--profile"<profile-name>"Run with a specific, existing profile--variable"<variable-name>"Override default variable during a single run--reporterThe name of the reporter to use (default: "inline"). Instead of "inline" you can also set "junit" for the report to be exported to an XML file--output-pathThe path to save the test report. Relative to the current working directory
Runs a specific test suite based on its ID
stop suite <suite-run-id>
--no-progressExit immediately without waiting for the result--debugShow more data (like raw API response)--result-timeout <int>Modify the default result waiting time (minutes, default: 60)
Stop the test suite run based on its ID
status test <test-id>
--no-progressExit immediately without waiting for the result--debugShow more data (like raw API response)
Display the test's status based on a run ID
status suite <suite-run-id>
--no-progressExit immediately without waiting for the result--debugShow more data (like raw API response)
Display the test suite's status based on a run ID
result test <test-run-id>
--no-progressExit immediately without waiting for the result--debugShow more data (like raw API response)--with-detailsShow results with details
Display the test result based on a run ID
result suite <test-run-id>
--no-progressExit immediately without waiting for the result--debugShow more data (like raw API response)--with-detailsShow results with details
Display the test suite's result based on a run ID
Examples of usage
Check the status of a test run by its ID:
Output:
Generate a report of a test run by its ID:
By default "reporter" option is set to "inline".
Output:
Generate a report of a test run by its ID that's exported to a junit format (XML file):
Output:
The XML file is automatically exported to your project's main directory:
Execute a test suite by its ID:
Output:
Check test suite run's status:
Output:
Last updated
Was this helpful?
