Skip to content

Refine onboarding verification branch coverage#11328

Merged
zachlloyd merged 2 commits into
masterfrom
zach/onboarding-skill-refinement
May 20, 2026
Merged

Refine onboarding verification branch coverage#11328
zachlloyd merged 2 commits into
masterfrom
zach/onboarding-skill-refinement

Conversation

@zachlloyd
Copy link
Copy Markdown
Contributor

Summary

  • expand the onboarding verification skill into a bounded multi-wave orchestration flow
  • require baseline logged-in/logged-out explorers to propose concrete follow-up coverage branches
  • add screenshot-backed reporting for suspicious UI states, rough edges, and error conditions

Validation

  • python3 structural sanity check for required skill sections
  • direct cloud verification run launched from the updated workflow

Conversation: https://staging.warp.dev/conversation/862a4563-4a63-4dfd-b5d8-aea28cbb5ac1

Co-Authored-By: Oz oz-agent@warp.dev

Co-Authored-By: Oz <oz-agent@warp.dev>
@cla-bot cla-bot Bot added the cla-signed label May 19, 2026
@oz-for-oss
Copy link
Copy Markdown
Contributor

oz-for-oss Bot commented May 19, 2026

@zachlloyd

I'm starting a first review of this pull request.

You can view the conversation on Warp.

I completed the review and no human review was requested for this pull request.

Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

Copy link
Copy Markdown
Contributor

@oz-for-oss oz-for-oss Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overview

This PR expands the onboarding verification skill from two baseline logged-out/logged-in explorers into a bounded follow-up wave for branch coverage, with additional screenshot-backed reporting for suspicious UI states.

Concerns

  • The second-wave run_agents example omits the matching logged-in/logged-out flow prompt, which can cause follow-up agents to miss required auth handling, artifact directory, and flow-specific behavior.

Verdict

Found: 0 critical, 1 important, 0 suggestions

Request changes

Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

Comment on lines +60 to +63
- name: "warp-onboarding-followup-theme-choice"
prompt: the follow-up flow prompt below plus one synthesized branch assignment
- name: "warp-onboarding-followup-model-choice"
prompt: the follow-up flow prompt below plus one synthesized branch assignment
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ [IMPORTANT] This example omits the matching logged-out/logged-in flow prompt, despite step 8 requiring it. Copying this shape would leave follow-up agents without required auth handling, artifact directory, skip/login behavior, and flow-specific terminal verification.

Suggested change
- name: "warp-onboarding-followup-theme-choice"
prompt: the follow-up flow prompt below plus one synthesized branch assignment
- name: "warp-onboarding-followup-model-choice"
prompt: the follow-up flow prompt below plus one synthesized branch assignment
- name: "warp-onboarding-followup-theme-choice"
prompt: the follow-up flow prompt below, the logged-out flow prompt below, and one synthesized logged-out branch assignment
- name: "warp-onboarding-followup-model-choice"
prompt: the follow-up flow prompt below, the logged-in flow prompt below, and one synthesized logged-in branch assignment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oz's feedback here seems legit given step 8 above.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@zachlloyd zachlloyd requested a review from captainsafia May 19, 2026 20:21
Copy link
Copy Markdown
Contributor

@captainsafia captainsafia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good but I think Oz's feedback is good to address here.

Comment on lines +60 to +63
- name: "warp-onboarding-followup-theme-choice"
prompt: the follow-up flow prompt below plus one synthesized branch assignment
- name: "warp-onboarding-followup-model-choice"
prompt: the follow-up flow prompt below plus one synthesized branch assignment
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oz's feedback here seems legit given step 8 above.

Co-Authored-By: Oz <oz-agent@warp.dev>
@zachlloyd zachlloyd merged commit 340a329 into master May 20, 2026
26 checks passed
@zachlloyd zachlloyd deleted the zach/onboarding-skill-refinement branch May 20, 2026 00:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants