New Features

UI

• Introduced a Glimmer component for managing pipeline secrets in the new UI to enhance user experience and functionality. See PR.
• Introduced a new Glimmer component to manage API tokens, allowing users to create and manage them more efficiently. See PR.
• PR Desync: a feature in the UI to detect and notify users of desynced pull requests in the database due to prolonged pipeline inactivity, providing a convenient resync button. See PR.
• Support for reverting overridden secrets and improved ambiguity in secret deletion messaging in the new UI. See PR.
• Introduced a new navigation bar for switching between secrets and tokens in the new UI. See PR.
• Default status sorting added to the event jobs table, with sorting based on build color and then alphabetically by stage and job name. As depicted below, the failed build is sorted at the top. See PR.

API

• Added feature to trigger jobs when a pull_request closed webhook event is received in Screwdriver. See PR and guide. 
• Added feature to set event status when a build is updated, tracking overall execution status for an event. See PR.
• Added a DELETING state to the pipeline to prevent updates during pipeline deletion, addressing remaining pipeline data post-deletion issue. See PR.
• Implemented a feature flag to control log payload logging, aiding log volume management. See PR.
• Pipeline admin user list now updated based on SD user permissions table. See PR.

Bug Fixes

UI

• Implemented a fix to allow navbar to be displayed when the hamburger menu is expanded on small devices. See PR.
• Fixes issues with edit secret modal in new landing page by setting secret value properly for API request and returning full response when “allow in pr” field is changed. See PR.
• Ensures that the workflow graph maintains the toggle state of downstream triggers when switching between different pipeline events. See PR.
• Event rail reloading logic in the new pipeline landing page was adjusted to align with the vertical collection component. See PR.
• Fixes an issue where cacheKey was incorrectly removed during data reloading in the pipeline workflow, ensuring the latest commit SHA is correctly reset. See PR.
• Pipeline header updating issue resolved by adding necessary tracking to update properly when a pipeline changes. See PR.
• Pipeline root directory now included and displayed alongside branch name in new UI. See PR. 

API

• Added a check for adminUserIds for backward compatibility in situations where old pipelines have a null adminUserIds column due to inability of MySQL 5 version to set default values in TEXT fields. See PR.
• Fix for triggering the next build on the child event when some builds are restarted before all builds are completed. See PR. 

Compatibility List

In order to have these improvements, you will need these minimum versions:

• API - v8.0.26
• UI - v1.0.1272
• Store - v7.0.0
• Queue-Service - v5.0.1
• Launcher - v6.0.221
• Build Cluster Worker - v5.0.1

Contributors

Thanks to the following contributors for making this feature possible. 

• Akinori
• Keisuke
• Ming
• Pritam
• Sagar
• Vonny
• Yuki
• Yuta
• Jithin

Questions and Suggestions

We’d love to hear from you. If you have any questions, please feel free to reach out here. You can also visit us on Github and Slack.

Author

Vonny Jap, Senior Manager, Software Dev Engineering, Yahoo

The Screwdriver team is pleased to announce our newest release which brings in new features and bug fixes across various components.

New Features

UI

• The new Pipeline landing page is now generally available. Once users opt in, they will be redirected to the new experience.

• The expand and collapse feature has been added to pipeline stages. The stage’s completion status is now reflected in its border.

• The display of multiple banners has been modified from a stacked format to a paginated layout

API

• Relax permission check to allow admins from other SCMs to update pipeline
• Added APIs to create and delete buildCluster annotations, enabling Screwdriver admins to move jobs to specific clusters
• New endpoint to update buildcluster of a pipeline
• Improve pipeline searches by adding the ‘scmUri’ query parameter
• Introduced a 'not equal’ filter for pipeline events creator in the pipeline events list API
• Changes to banner to include scope and scopeId

Bug Fixes

UI

• Pipeline header updating issue resolved by adding necessary tracking to the pipeline header component, allowing proper updates when a pipeline changes, see PR
• Pipeline root directory now included and displayed with branch name in new landing page, with additional CSS styling cleanup, see PR
• Corrected the aggressive logic in the new landing page that was preventing jobs included in a stage from being started/restarted, now allowing these jobs to be started/restarted as needed, see PR
• Fixed missing stages border when accessing pipeline from Collection by directly requesting stage data from the backend, see PR
• Enhanced UI consistency by applying standardized CSS styling to the modal button within the build detail page, aligning it with the appearance of the workflow graph tooltip start modal, see PR
• Ensured the start all children pipelines API call requests the entire error object, allowing more detailed analysis of the API call and returned status code, see PR
• Fix: Remote trigger’s pipeline name now uses 'remoteName’ instead of 'displayName’ to avoid confusion. See PR
• Fix: Build status determination was updated by relaxing the build meta warning object check and included warning statuses in the event card warning count. See PR
• Fix: The edges from stages to upstreams are now properly drawn by providing a solution for insufficient horizontal spacing between stages. See PR
• Fix: The post body for PR events was improved by correctly including the job prefix in the `startFrom` value when restarting a PR job. See PR
• Fix: Column name in event job display changed from 'HISTORY’ to 'STATUS’ for accuracy in new landing page. See PR
• Fix: Long branch names on the search page are now truncated. See PR
• Fix: API fetch updated to ignore scheduler events for configured pipeline. See PR
• Fix: The new landing page now correctly reloads the workflow graph when starting a new event. See PR
• Fix: Overflow handling for long messages added to prevent text overflow in event card. See PR
• Fix: Pipeline data removed from route models due to direct injection into components. See PR
• Fix: The maximum node depth issue was fixed, addressing problems with 0-based counted node depth and empty arrays for pipelines with detached jobs only. See PR
• Fix: The creation of a sparse array with empty slots which caused downstream calculations to break was fixed. See PR
• Fix: Parameters drop down issue resolved in new landing page by adjusting dropdown spacing and removing additional scroll logic. See PR
• Fix: Build warning determination logic was fixed to correctly display warning status. See PR
• Fix: Node depth calculation in workflow graph was improved by introducing 'Topological sort’ for more accurate depth assignments. See PR
• Fix: An issue preventing the use of anchor tags within the navigation banner was fixed. See PR
• Fix: Job restart is now disabled for jobs with no builds in the new landing page. See PR

API

• Upgrade of Node.js v22 and dependencies
• Addressed a bug in build triggering in certain cases of restarts, ensuring the next build is correctly triggered from the latest child event, see PR
• Fixed an issue where an external trigger couldn’t trigger a pipeline stage due to a misinterpretation of the event associated with the pipeline the job belongs to, see PR
• Avoided creation of empty events when changed file names match the source directory, see PR
• ParentBuildId now set just before execution to ensure correct metadata capture from parent builds, especially when they finish simultaneously, see PR
• Converted the 'isActive’ parameter from string to boolean for MySQL on banner list endpoint to ensure proper banner list retrieval, see PR
• Included a webhook executor to the admin group before retrieving the pipeline token. This resolved the 'Pipeline has no admin’ error during the webhook process, see PR
• The metadata from the job that directly precedes the virtual job is now prioritized over the event metadata, see PR
• Proper handling of virtual jobs in remote triggers, see PR
• Fixed duplicate build issue by preventing the build status from updating to RUNNING if a build with the same ID is already running, see PR
• Virtual builds now marked as completed when triggered by webhook, resolving a regression introduced in a previous update, see PR

Compatibility List

In order to have these improvements, you will need these minimum versions:

• API - v8.0.16
• UI - v1.0.1241
• Store - v7.0.0
• Queue-Service - v5.0.1
• Launcher - v6.0.221
• Build Cluster Worker - v5.0.1

Contributors

Thanks to the following contributors for making this feature possible. 

• Akinori
• Keisuke
• Ming
• Pritam
• Sagar
• Vonny
• Yuki
• Yuta
• Teppei Minegishi

Questions and Suggestions

We’d love to hear from you. If you have any questions, please feel free to reach out here. You can also visit us on Github and Slack.

Author

Vonny Jap, Senior Manager, Software Dev Engineering, Yahoo

We’re thankful for you! The Screwdriver team is pleased to announce our newest release which brings in new features and bug fixes across various components.

New Features

• UI
• Virtual job improvements - You can now see a different icon for virtual jobs in the workflow graph. They will also have a `virtual` tag in the pipeline Options tab. The virtual job build page will no longer show logs or warnings. You can read more about using `screwdriver.cd/virtualJob` annotation in the Screwdriver documentation.

• Option to download directory artifacts as zipped file - You can now download an artifact directory for a build as a zipped file. Navigate to a folder under Artifacts and click the existing “Download” button. Max size 2GB.

• Jobs list view has stages column and ability to select number of historical builds shown - You can now see which stage each job belongs to and sort by stage. You can also select to show up to 30 historical builds on the same page.

Keep reading

Spoooky season is here! The Screwdriver team is pleased to announce our newest release which brings in new features and bug fixes across various components.

New Features

• UI
• Show remote trigger’s displayName as title - You can now see the remote trigger’s pipeline name and branch by hovering on its node in the workflow graph.

• Option to download all artifacts as zipped file - You can now download all artifacts for a build as a zipped file. Navigate to root artifact folder and you should see Download All button.

Keep reading

The end of summer is approaching! The Screwdriver team is pleased to announce our newest release which brings in new features and bug fixes across various components.

New Features

• UI
• Allow starting/restarting job from middle of stage - You can now start/restart the event from a job that is in the middle of a stage. Caveats: Stage setup will always rerun and you cannot rerun stage teardown directly.

• Allow starting/restarting detached stage - You can also start/restart a detached stage. Click on the three dots on the top right corner of the stage to see the prompt.

Keep reading

It’s summer! The Screwdriver team is pleased to announce our newest release which brings in new features and bug fixes across various components.

New Features

• UI
• Display admins on pipeline options page

• API
• Skip execution of virtual jobs - A job can be marked as virtual by adding the annotation screwdriver.cd/virtualJob: true. These jobs will not be executed while processing the workflow for an event, but will proceed with SUCCESS status. These jobs will skip queueing and execution and builds of its downstream jobs will be immediately created to continue the processing of the workflow.
• Get repository data from scmRepo without GitHub access - Before this change, extra calls were made to GitHub to fetch SCM repository information. After this change, the API now will pass scmRepo info directly in order to avoid extra GitHub calls. This has been implemented for webhooks only so far.
• Pipeline template enhancements: Can customize image, settings, environment, and requires for jobs that already exist in pipeline template, Add new jobs that are not part of pipeline template in user yaml, Flatten shared steps to jobs in pipeline template, Pipeline template validator shows extended configs for job template, Pipeline template validator shows workflow graph
• Store
• Replace jwt expiration with a config and environment variable - The jwt expiration is currently 13h and is written directly in the source code. This makes it difficult for SD administrators to change to arbitrary values to suit their use cases. Now Screwdriver cluster admins can set the JWT expiration length using JWT_MAX_AGE.
• Add handling for the Expect header at onPreAuth - To reduce unnecessary retries when uploading large files, Screwdriver will use the Expect: 100-continue header. The content length can be checked before the body is uploaded. Thus, the HTTP client (store-cli) can receive a 413 error earlier without unnecessary retries.

Keep reading

Features

Pipeline Stages

The Stages feature in Screwdriver lets users group jobs with similar objectives, simplifying workflow organization. Jobs within a stage can run sequentially or in parallel based on specific triggers and can include setup and teardown tasks. This categorizes and organizes jobs, managing complex workflows efficiently. For more details, visit the Stages Documentation.

Pipeline Templates

Pipeline Templates in Screwdriver let users define reusable pipeline configurations with predefined jobs, steps, and container images. Users can find, use, and version existing templates, or create new ones in sd-template.yaml. Templates can be tested, validated, tagged, and published through the Screwdriver pipeline, streamlining setup and ensuring consistency. For more details, visit the Pipeline Templates Documentation.

Add the builds endpoint for efficient build retrieval per pipeline.

This update introduces the GET /pipelines/:id/builds endpoint, allowing users to fetch the latest pipeline builds with parameters like groupEventId and latest. This reduces API calls and improves build retrieval efficiency.

Add filter to Jobs Endpoint for PR Job Retrieval in a Pipeline

This PR introduces a filter for the GET /pipelines/:id/jobs endpoint to fetch only pull request (PR) jobs by adding a type=pr query parameter. This change aims to reduce the number of API calls needed to gather information about PR jobs, enhancing efficiency.

Add SHA filter for Pipeline Events

This change introduces a new feature that adds a SHA filter for pipeline events in Screwdriver. This enhancement allows users to filter pipeline events based on specific commit SHAs, providing more granular control and visibility into pipeline executions tied to particular commits.

Fixes

Return 403/404 Errors for Mismatched SCM Host

This PR updates the API to return appropriate error codes when attempting to stop events with a mismatched SCM host. Instead of a generic 500 error, it will now return a 403 error for public pipelines or a 404 error for private pipelines, aligning with the intended security and access control measures.

Subscribed SCM Notifications to Respect Event Types and Allow PR Jobs of Upstream Pipelines

This pull request fixes issues with SCM notifications, ensuring they respect event types and allow PR jobs from upstream pipelines. The correction involves preserving the original SHA of the upstream pipeline and matching event types between the upstream repository and downstream subscriptions, improving functionality and reliability.

Improve Visibility of Build Parameters in Modal

This PR addresses an issue with build parameters being hidden when displayed in a dropdown. The update ensures the screen scrolls automatically to make the entire dropdown visible. 

Indicate Pipeline Execution by Pipeline Token

The implementation aims to make it clear when a pipeline is executed using a pipeline token. Currently, it appears as if a pipeline admin started it, causing confusion. The improvement will explicitly indicate execution by a pipeline token, enhancing transparency and user understanding.

Refactor Build Triggers for Improved Readability and Maintainability

This issue highlights the need to refactor trigger processing in the Screwdriver build system due to code complexity, which hampers debugging and maintenance. The proposed changes involve creating a dedicated directory for triggers, splitting processes into separate files, and simplifying complex functions. The goal is to improve readability, maintainability, testing ease, and the reliability of build triggers.

Correct Template Name Formatting on Usage Page

This PR corrects the template name format on the usage page of the Screwdriver CD UI. It changes the display from <name>/<namespace> to <namespace>/<name> to ensure the correct naming convention is used.

Improve Notification Handling for Build Status Transitions

This PR fixes an issue in the Screwdriver CD UI to prevent unnecessary notifications when transitioning between builds via the COMMIT drop-down. Previously, notifications were triggered if the statuses of the builds being navigated to differed. The update ensures that these extra notifications are avoided, improving user experience.

These pull requests (PRs) include several small fixes and improvements to the Screwdriver UI.

PR #1030: Fixed height on PR list element

PR #1026: Fixed background-color for profile icon when focused

PR #1025: Reverted logic for combining graph data

PR #1024: Fixed background color on hover for profile icon

PR #1023: Updated pipeline events page style

PR #1022: Updated nav banner style

PR #1020: Set application navbar to a constant height

PR #1064: Fixed the job list view that was missing in Safari browser

Compatibility List

In order to have these improvements, you will need these minimum versions as listed below. 

API - v7.0.140

UI - v1.0.947

Store - v5.0.10

Queue-Service - v3.0.10

Launcher - v6.0.200

Build Cluster Worker - v3.1.1

Author

Vonny Jap, Sr Mgr, Software Dev Engineering, Yahoo

Fixes

Update Node Start and Restart Logic in Screwdriver UI

This implementation aims to enhance the tooltip functionality in the Screwdriver UI by updating the logic to depend solely on workflow graph information for determining job start and restart capabilities. This approach addresses issues caused by missing data from the API, ensuring tooltips are correctly generated based on the current view and associated workflow graph details.

Fix Padding on Pipeline Routes to Remove Whitespace

This pull request resolves an issue with the left and right padding on pipeline routes in the Screwdriver UI. The excessive padding was causing unwanted whitespace, which affected the overall layout and user experience.

Standardize Navbar Height for Consistent Layout

This update addresses the issue of the navbar’s height changing based on window size, which, while minor, can cause inconsistencies in the layout. By setting the navbar to a constant height, we eliminate these variations, simplifying height calculations for other components and ensuring a more consistent user experience.

Features

Implement Browser Notifications

The implementation includes enhancements to the notification system through the browsers to ensure timely and relevant updates for users. This feature aims to provide a more intuitive and informative user experience by optimizing the delivery and content of notifications based on user actions and system events.

Include job template information on the build detail page

In Screwdriver, jobs can implement existing templates. This pull request modifies the build detail page to display information about the template used by each build, along with a link to view details about the template itself.

Author

Vonny Jap, Sr Mgr, Software Dev Engineering, Yahoo

We are currently catching up with retro updates, so we kindly ask for your patience as we return to our normal rhythm.

• Fixes

Fix PR Build Configuration to Ignore Periodic Triggers

This pull request addresses an issue where pull request (PR) jobs were inadvertently triggering periodic builds. The fix involves adding a check to ensure that PR jobs do not start periodic builds. This change prevents unnecessary builds and ensures that the CI/CD pipeline operates as intended without running redundant periodic jobs for PRs.

Fix PR Graph Display for Non-ChainPR Configurations

This pull request addresses an issue where the pipeline graph for pull requests is not displayed correctly when the chainPR setting is set to false. The update ensures that the pipeline graph is rendered accurately for these configurations, and also makes the graph nodes fully clickable with accessible dropdown menu options. This fix improves the usability and accuracy of the PR view in the Screwdriver UI.

Fix PR Graph Rendering Issue When Switching to PR Tab

This pull request fixes a bug that affects the rendering of the PR graph when switching from the Events view to the Pull Request view in the Screwdriver UI. The issue occurs particularly with pipelines containing extensive build data and the chainPR setting enabled. The fix ensures that the showDownstreamTriggers field is correctly set to false when the Pull Request view is rendered, allowing for proper display of the pipeline graphs.

• Features

Add Resource Usage Information in the Build Launcher

This pull request adds resource usage information to the metrics pushed to Prometheus. The objective is to capture and report the amount of resources (such as CPU and memory) used by builds, providing better insights into resource consumption.

Enhance Validator Page with Toggle for Split and Spread Views

This pull request enhances the validator page’s user interface by adding a button to toggle between split view and spread view. This feature addresses the issue of narrow display areas for the validator input form, workflow graph, and job details, particularly for users with vertical monitors. The spread view option allows for a more expansive and user-friendly layout, improving usability and accessibility.

Author
Vonny Jap, Sr Mgr, Software Dev Engineering, Yahoo

Features

Automatic Pipeline Data Refresh Post-Synchronization in Screwdriver UI

It introduces a feature that automatically reloads pipeline data after the “sync pipeline” button is clicked, ensuring that the UI reflects the latest pipeline information without requiring a manual page refresh.

Add no-image-pull Option to Skip Docker Image Pull in sd-local

This introduces a new build option called no-image-pull. This option allows users to skip the Docker image pull step when running builds, which can significantly reduce build times when the same image is used repeatedly.

Fixes

Automatic Loading and Start/Stop Toggle for PR Builds in Screwdriver UI

This update fixes a bug where pull request (PR) builds were not automatically loading when the “start” button was clicked. It also ensures that PR builds load automatically, and the button changes to “stop” while the builds are running.

Fix breakage on Options Page with Long Job Names in Screwdriver UI

This fixes the UI issue where long job names caused screen breakage on the Options page. The update truncates long job names with an ellipsis and adds a tooltip to display the full name on hover. Additionally, it includes functionality to expand and collapse long messages for better readability without breaking the layout.

Author

Vonny Jap, Sr Mgr, Software Dev Engineering, Yahoo