<!-- The first section "Release notes" is required if you want to have your release post blog MR auto generated. Currently in BETA, details on the **release post item generator** can be found in the handbook: https://about.gitlab.com/handbook/marketing/blog/release-posts/#release-post-item-generator and this video: https://www.youtube.com/watch?v=rfn9ebgTwKg. The next four sections: "Problem to solve", "Intended users", "User experience goal", and "Proposal", are strongly recommended in your first draft, while the rest of the sections can be filled out during the problem validation or breakdown phase. However, keep in mind that providing complete and relevant information early helps our product team validate the problem and start working on a solution. --> ### Release notes <!-- What is the problem and solution you're proposing? This content sets the overall vision for the feature and serves as the release notes that will populate in various places, including the [release post blog](https://about.gitlab.com/releases/categories/releases/) and [Gitlab project releases](https://gitlab.com/gitlab-org/gitlab/-/releases). " --> ## Problem to solve Invalid [environment configurations](https://docs.gitlab.com/ee/ci/yaml/#environment) can interrupt [the environment update process](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/services/deployments/update_environment_service.rb) after a deployment succeeds. For example, you're seeing a successful deployment in the following screenshot, where actually it failed to update the environment status. https://gitlab.com/shinya.maeda/pipeline-playground/-/jobs/2364758866  The deployment job should update [the environment URL](https://gitlab.com/shinya.maeda/pipeline-playground/-/environments/2235830) to `www.google.com`, but since it's malformed URL, the system can't update it. ### Customer impact This silent failure often gives support engineers and customers having hard time to debug the problems. https://gitlab.com/gitlab-org/gitlab/-/issues/332374#note_594418867 > We've found the problem: the environment tier and the deployment-related merge request metrics are not going to be updated due to the missing `http` or `https` URL prefix under the `environment.url` YAML key. The validation on the `Environment` record will silently fail here: https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/services/deployments/update_environment_service.rb#L34 ### Errors on SaaS You can see how often this environment update failure happens on SaaS: - https://sentry.gitlab.net/gitlab/gitlabcom/issues/2678585/?query=EnvironmentUpdateFailure - https://log.gprd.gitlab.net/goto/f08bab70-c220-11ec-b73f-692cc1ae8214 At the moment, roughly 4,000 of deployments encounter the failure every day. There is no feedback feature to let users/customers be aware of this problem. Here is the frequency per error message: https://log.gprd.gitlab.net/goto/b4f34b10-c223-11ec-afaf-2bca15dfbf33  We can see that 100% of error messages are related to the [`environment.url` keyword](https://docs.gitlab.com/ee/ci/yaml/#environmenturl). ### Intended users * [Rachel (Release Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#rachel-release-manager) * [Priyanka (Platform Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#priyanka-platform-engineer) <!-- Who will use this feature? If known, include any of the following: types of users (e.g. Developer), personas, or specific company roles (e.g. Release Manager). It's okay to write "Unknown" and fill this field in later. Personas are described at https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/ * [Cameron (Compliance Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#cameron-compliance-manager) * [Parker (Product Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#parker-product-manager) * [Delaney (Development Team Lead)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#delaney-development-team-lead) * [Presley (Product Designer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#presley-product-designer) * [Sasha (Software Developer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sasha-software-developer) * [Devon (DevOps Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#devon-devops-engineer) * [Sidney (Systems Administrator)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sidney-systems-administrator) * [Sam (Security Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sam-security-analyst) * [Rachel (Release Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#rachel-release-manager) * [Alex (Security Operations Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#alex-security-operations-engineer) * [Simone (Software Engineer in Test)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#simone-software-engineer-in-test) * [Allison (Application Ops)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#allison-application-ops) * [Priyanka (Platform Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#priyanka-platform-engineer) * [Dana (Data Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#dana-data-analyst) * [Eddie (Content Editor)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#eddie-content-editor) --> ### Metrics Tracking the absolute number of failed pipelines due to invalid environments. <!-- How are you going to track usage of this feature? Think about user behavior and their interaction with the product. What indicates someone is getting value from it? Create tracking issue using the Snowplow event tracking template. See https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Snowplow%20event%20tracking.md --> ### User experience goal The user should be able to see _why_ their environment failed to update ## Proposal: Soft validation on External URL https://gitlab.com/gitlab-org/gitlab/-/issues/337417#note_922931598 - Since `external_url` is used for that users accessing to the website, and it's not used for internal server request, we can persist an URL without `AddressableUrlValidator`. - Since we expose the `external_url` as a button in some pages (environment page, MR page, etc), we sanitize the URL not to include javascript code. <details> <summary> Previous proposal (turned down due to drawbacks) </summary> ## Proposal: Validate Environment URL at pipeline creation When a new pipeline is created, we additionally validate the Environment URL on each job. We expand the Environment URL (e.g. `url: appname-$CI_COMMIT_REF_SLUG`) based on the [CI/CD variables](https://docs.gitlab.com/ee/ci/variables/). If it's invalid, we mark the job as `failed` status similar to what we did in [the previous issue](https://gitlab.com/gitlab-org/gitlab/-/issues/21182). A few notes: - Since the job is failed, users almost 100% notice that something went wrong. - Easy to implement. The weight would be 1-2. - This is a breaking change that could disturb user's CI/CD workflow. (This might not even a con at %"15.0", because we allow breaking change at major update.) We should communicate with affected customers in advance to mitigate the impact. - There is an edge case that users can [set dynamic environment URLs after a job finishes](https://docs.gitlab.com/ee/ci/environments/#set-dynamic-environment-urls-after-a-job-finishes). We can't detect this error by this approach. ### UI/UX Surface the following error message to the pipeline job page if an environment update is failed:  **SSOT ~"UI text"*** > This job could not be executed because it would update the environment with an invalid URL. [Learn More.](https://docs.gitlab.com/ee/ci/yaml/index.html#environmenturl) Documentation link: https://docs.gitlab.com/ee/ci/yaml/index.html#environmenturl Similarly, it might be the case the pipeline fails because both the name and URL are invalid. In that case, the message can refer both, and the documentation link take the user to the parent section:  **SSOT ~"UI text"*** > This job could not be executed because it would update the environment with an invalid URL and name. [Learn more.](https://docs.gitlab.com/ee/ci/yaml/index.html#environment) Documentation link: https://docs.gitlab.com/ee/ci/yaml/index.html#environment <!-- How are we going to solve the problem? Try to include the user journey! https://about.gitlab.com/handbook/journeys/#user-journey --> </details> ### Further details <!-- Include use cases, benefits, goals, or any other details that will help us understand the problem better. --> ### Permissions and Security <!-- What permissions are required to perform the described actions? Are they consistent with the existing permissions as documented for users, groups, and projects as appropriate? Is the proposed behavior consistent between the UI, API, and other access methods (e.g. email replies)? Consider adding checkboxes and expectations of users with certain levels of membership https://docs.gitlab.com/ee/user/permissions.html * [ ] Add expected impact to members with no access (0) * [ ] Add expected impact to Guest (10) members * [ ] Add expected impact to Reporter (20) members * [ ] Add expected impact to Developer (30) members * [ ] Add expected impact to Maintainer (40) members * [ ] Add expected impact to Owner (50) members --> ### Documentation No expected documentation change, other than pointing to https://docs.gitlab.com/ee/ci/yaml/index.html#environmenturl <!-- See the Feature Change Documentation Workflow https://docs.gitlab.com/ee/development/documentation/workflow.html#for-a-product-change * Add all known Documentation Requirements in this section. See https://docs.gitlab.com/ee/development/documentation/workflow.html * If this feature requires changing permissions, update the permissions document. See https://docs.gitlab.com/ee/user/permissions.html --> ### Availability & Testing <!-- This section needs to be retained and filled in during the workflow planning breakdown phase of this feature proposal, if not earlier. What risks does this change pose to our availability? How might it affect the quality of the product? What additional test coverage or changes to tests will be needed? Will it require cross-browser testing? Please list the test areas (unit, integration and end-to-end) that needs to be added or updated to ensure that this feature will work as intended. Please use the list below as guidance. * Unit test changes * Integration test changes * End-to-end test change See the test engineering planning process and reach out to your counterpart Software Engineer in Test for assistance: https://about.gitlab.com/handbook/engineering/quality/test-engineering/#test-planning --> ### Available Tier ~"GitLab Free" <!-- This section should be used for setting the appropriate tier that this feature will belong to. Pricing can be found here: https://about.gitlab.com/pricing/ * Free * Premium/Silver * Ultimate/Gold --> ### What does success look like, and how can we measure that? Customers are able to fix this problem after their first encounter with the error message, and the absolute number of failed pipelines due to invalid environments drops. <!-- Define both the success metrics and acceptance criteria. Note that success metrics indicate the desired business outcomes, while acceptance criteria indicate when the solution is working correctly. If there is no way to measure success, link to an issue that will implement a way to measure this. Create tracking issue using the Snowplow event tracking template. See https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Snowplow%20event%20tracking.md --> ### What is the type of buyer? <!-- What is the buyer persona for this feature? See https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/buyer-persona/ In which enterprise tier should this feature go? See https://about.gitlab.com/handbook/product/pricing/#three-tiers --> - [Casey - the Release and Change Management Director](https://about.gitlab.com/handbook/marketing/strategic-marketing/roles-personas/buyer-persona/#casey---the-release-and-change-management-director) - [Dakota - the Application Development Director](https://about.gitlab.com/handbook/marketing/strategic-marketing/roles-personas/buyer-persona/#dakota---the-application-development-director) - [Kennedy - the Infrastructure Engineering Director](https://about.gitlab.com/handbook/marketing/strategic-marketing/roles-personas/buyer-persona/#kennedy---the-infrastructure-engineering-director) ### Is this a cross-stage feature? <!-- Communicate if this change will affect multiple Stage Groups or product areas. We recommend always start with the assumption that a feature request will have an impact into another Group. Loop in the most relevant PM and Product Designer from that Group to provide strategic support to help align the Group's broader plan and vision, as well as to avoid UX and technical debt. https://about.gitlab.com/handbook/product/#cross-stage-features --> ### Links / references - This is a follow-up of https://gitlab.com/gitlab-org/gitlab/-/issues/21182 <!-- Label reminders - you should have one of each of the following labels. Use the following resources to find the appropriate labels: - https://gitlab.com/gitlab-org/gitlab/-/labels - https://about.gitlab.com/handbook/product/categories/features/ -->