Add client support for single transaction reward claiming#9801

rickyrombo · 2024-09-19T22:45:12Z

Description

Adds a reward manager lookup table to bring down the size of reward claim transactions
- Adds a script, manageRewardsLookupTable.ts, that handles creating and updating that lookup table
- Adds the initialized lookup table to the genesis of the local dev solana-test-validator
Adds support to SDK for using hash IDs in specifiers and shortened challenge IDs
- ⚠️ This SDK change needs to release immediately after the backend changes but not before. It is a breaking change.
Adds back arm64 build for solana-test-validator-build
Updates challenge claiming to submit all attestations in one transaction, and optionally evaluate them as well if it thinks it can fit them. ⚠️ did some rough math here... it's potentially possible I'm a bit off on my calculations, we'll want to test more... but ultimately this shouldn't matter after the backend goes out.
Updates challenge claiming client to not have off-by-one error by updating the @audius/spl program to take into account minVotes when decoding account data
Fixes custom errors in SDK

How Has This Been Tested?

Tested against both current staging (without backend changes) and local dev (with backend changes) to ensure that both continue to work

ROLLOUT PLAN

Release this PR to web/mobile and let soak
Set the release with this PR in it as the minimum version and force upgrade
Release Anti Abuse Oracle allowing both types of IDs
Release DN changes that migrates challenge IDs and specifiers
- Immediately publish new SDK
Cleanup:
- Remove old IDs from optimizely config
- Remove old IDs from client config
- Remove old IDs from SDK
- Remove check for TX size in SDK

changeset-bot · 2024-09-19T22:45:17Z

🦋 Changeset detected

Latest commit: 712b737

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages

Name	Type
@audius/sdk	Major
@audius/spl	Major
@audius/sp-actions	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

gitguardian · 2024-09-19T22:45:17Z

⚠️ GitGuardian has uncovered 17 secrets following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secrets in your pull request

GitGuardian id	GitGuardian status	Secret	Commit	Filename
1606950	Triggered	Generic High Entropy Secret	`0b36a1a`	packages/libs/src/sdk/config/production.ts	View secret
1606950	Triggered	Generic High Entropy Secret	`0b36a1a`	packages/libs/src/sdk/config/production.ts	View secret
1606950	Triggered	Generic High Entropy Secret	`0b36a1a`	packages/libs/src/sdk/scripts/generateServicesConfig.ts	View secret
1606950	Triggered	Generic High Entropy Secret	`0b36a1a`	packages/libs/src/sdk/scripts/generateServicesConfig.ts	View secret
4714545	Triggered	Generic High Entropy Secret	`0b36a1a`	packages/libs/src/sdk/config/development.ts	View secret
4714545	Triggered	Generic High Entropy Secret	`0b36a1a`	packages/libs/src/sdk/scripts/generateServicesConfig.ts	View secret
4714545	Triggered	Generic High Entropy Secret	`0b36a1a`	packages/libs/src/sdk/config/development.ts	View secret
7150224	Triggered	Generic High Entropy Secret	`0b36a1a`	packages/libs/src/sdk/scripts/generateServicesConfig.ts	View secret
7150224	Triggered	Generic High Entropy Secret	`0b36a1a`	packages/libs/src/sdk/scripts/generateServicesConfig.ts	View secret
7150224	Triggered	Generic High Entropy Secret	`0b36a1a`	packages/libs/src/sdk/config/staging.ts	View secret
7150224	Triggered	Generic High Entropy Secret	`0b36a1a`	packages/libs/src/sdk/config/staging.ts	View secret
10622416	Triggered	Generic High Entropy Secret	`0b36a1a`	packages/libs/src/sdk/config/production.ts	View secret
10622416	Triggered	Generic High Entropy Secret	`0b36a1a`	packages/libs/src/sdk/scripts/generateServicesConfig.ts	View secret
10622416	Triggered	Generic High Entropy Secret	`0b36a1a`	packages/libs/src/sdk/scripts/generateServicesConfig.ts	View secret
10622416	Triggered	Generic High Entropy Secret	`0b36a1a`	packages/libs/src/sdk/config/staging.ts	View secret
10622417	Triggered	Generic High Entropy Secret	`0b36a1a`	packages/libs/src/sdk/scripts/generateServicesConfig.ts	View secret
10622417	Triggered	Generic High Entropy Secret	`0b36a1a`	packages/libs/src/sdk/config/development.ts	View secret

🛠 Guidelines to remediate hardcoded secrets

Understand the implications of revoking this secret by investigating where it is used in your code.
Replace and store your secrets safely. Learn here the best practices.
Revoke and rotate these secrets.
If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider

following these best practices for managing and storing secrets including API keys and other credentials
install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.

^{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.}

audius-infra · 2024-09-19T23:09:07Z

Preview this change https://demo.audius.co/mjp-single-transaction-rewards

packages/libs/src/sdk/api/challenges/ChallengesApi.ts

dharit-tan · 2024-09-20T18:29:29Z

packages/libs/src/sdk/scripts/manageRewardsLookupTable.ts

+    AddressLookupTableProgram.createLookupTable({
+      authority,
+      payer,
+      recentSlot: slot - 1


why the -1?

Note that using the most recent slot sometimes results in an error after sending the transaction. To avoid this, you can use a slot that is one slot prior the most recent one (e.g. recentSlot: slot - 1). However, if you still encounter an error when sending the transaction, you can try resending the transaction.

https://solana.com/developers/courses/program-optimization/lookup-tables#create-a-lookup-table

In the future, maybe add a comment for things like this? It's going to be cryptic looking at this code later on. :-)

dharit-tan · 2024-09-20T18:32:18Z

packages/libs/src/sdk/services/Solana/programs/RewardManagerClient/RewardManagerClient.ts

-    return RewardManagerProgram.decodeAttestationsAccountData(accountInfo.data)
+    const rewardManagerState = await this.getRewardManagerState()
+    // Min Votes = discovery votes, +1 for AAO oracle
+    const maxAttestations = rewardManagerState.minVotes + 1


nit: could also pull this out somewhere.. this is probably fine tho

dharit-tan · 2024-09-20T18:33:02Z

packages/spl/src/reward-manager/RewardManagerProgram.ts

-        'messages'
-      )
-    ])
+    attestationsAccountData: (maxAttestations: number) =>


dharit-tan · 2024-09-20T18:34:06Z

solana-programs/README.md

 ./scripts/init-dev-fixtures.sh
 ```

+NOTE: The Reward Manager Lookup Table is not included in this script. To recreate the Reward Manager lookup table, see packages/libs/src/sdk/scripts/manageRewardsLookupTable.ts. You'll then have to export it to a file using the same commands found in the init-dev-fixtures.sh script


could we get an example of how to call the script somewhere? would be useful to see examples of both create and update.

dharit-tan

godlike

sliptype

Awesome work! this all looks good to me although I don't have the most context on rewards

sliptype · 2024-09-20T19:55:44Z

packages/spl/src/reward-manager/RewardManagerProgram.ts

+            blob(45),
+            ethAddress('operator')
+          ]),
+          maxAttestations,


audius-infra · 2024-09-20T23:37:48Z

Preview this change https://demo.audius.co/mjp-single-transaction-rewards

[e28b82b] Add client support for single transaction reward claiming (#9801) Marcus Pasell [1696b04] Version Packages (#9772) github-actions[bot] [1c2edae] Reapply libs split (#9817) Sebastian Klingler [c9f1142] [C-5026] Optimistic UI tweaks (web only for now) (#9816) JD Francis [7c4b196] Don't show empty chat prompt for blasts (#9815) Reed [7ef38cd] [QA-1732] Undo test code to fix unread pill check for notifications (#9814) Kyle Shanks [ad274d8] [PAY-3214] Add CreateBlast CTA to followers and supporters user lists (web) (#9804) Andrew Mendelsohn [3f862a8] UDF for chat_allowed (#9789) Steve Perkins [fa1da71] revert max width on chats (#9806) Andrew Mendelsohn [6074a68] [PAY-3216] Update upload share buttons (#9805) Andrew Mendelsohn [b9a8c0b] Revert libs split (#9802) Sebastian Klingler [8c80712] Fix merge issue (#9800) Sebastian Klingler [0ea3a11] [QA-1731] Fix key filter label (#9798) Kyle Shanks [bc28909] Separate libs from main sdk bundle (#9791) Sebastian Klingler [4d8ad95] Add comment email and push (#9784) Dylan Jeffers [734f766] [QA-1717] Fix empty unfurl messages for tracks and playlists (#9795) Kyle Shanks [2faf2e0] Add CHAT routes to static_routes (#9796) Reed [8b28228] [C-4969] Comment tombstones (#9790) JD Francis [e613a0b] [PAY-3215] ChatBlastCTA in own followers and supporters screens (#9793) Andrew Mendelsohn [123403b] [PAY-3211] Chat blast audience display (mobile) (#9792) Andrew Mendelsohn [d93fcab] [PAY-3210, PAY-3346] Chat blast audience tile (web) (#9778) Andrew Mendelsohn [32f758d] [QA-1730] Add sort back to the search query (#9787) Kyle Shanks [2359e1a] Chat fix quicktype (#9782) Steve Perkins [9be9c3a] [PAY-3209] Chat blast mobile header (#9777) Reed [9f05224] Use Harmony Artwork for ChatBlastHeader (#9785) Reed [2d838c5] [C-5024] Add comment notification (#9781) Dylan Jeffers [a6850d4] Handle dupe comment thread indexing (#9574) Isaac Solo [c27a040] [C-4826] (web) Comment action confirmation modals & toasts (#9766) JD Francis [079cd5d] Remove direct only and add noQuote error (#9780) Raymond Jacobson [68c4084] Fix usdc-usdc purchase (#9779) Raymond Jacobson [839e1e8] Fix harmony tests (#9775) Sebastian Klingler [93c216b] [PAY-3367] Mobile chat blast audience/content selection (#9683) Reed [c6bb074] [PAY-3208] Web chat blast header (#9767) Reed [bfd9c38] Version Packages (#8915) github-actions[bot] [afc5416] [C-4970, C-4825] Add comment reports (#9680) Dylan Jeffers [f38fb31] Improve mobile profile spacing (#9758) Dylan Jeffers [74ed02a] Fix typecheck (#9768) Sebastian Klingler [38a5f29] [C-1601] Fix verified badge, Improve text-link (#9757) Dylan Jeffers [b6b0827] [C-5014] Remove search input auto-correct (#9756) Dylan Jeffers [d7c8a4c] [C-4973] Add edit comment e2e (#9679) Dylan Jeffers [5c839b9] Harmonize web challenge UI (#9762) Reed [94f614d] Fix composer input error and styling for DMs (#9765) Kyle Shanks [99fb10f] [C-4971] Add comment pin e2e (#9678) Dylan Jeffers [59c2d6e] [C-4988] Update mobile TrackScreen layout for remixes + comments (#9657) Sebastian Klingler [66b1b2c] [C-4943] Add comment sort (#9672) Dylan Jeffers [be35c44] Fixing lint from bad merge (#9764) Kyle Shanks [85be4f0] [QA-1326] Update challenge button labels to match spec (#9743) Reed [8e4f02b] [C-4871] Add timestamp highlighting for composer input (#9761) Kyle Shanks [0a67586] Add tab completion for compose UI autocomplete (#9682) Kyle Shanks [85b27c4] [C-5020] Search filter fixes for FilterButton (#9763) Sebastian Klingler [921f74d] Revert "[PAY-3388] Migrate feed to SDK (#9593)" (#9754) Randy Schott [fc257e1] Fix challenge summary table (#9755) Reed [ab45d0e] [QA-1713] Add filter input to feature flag modal (#9740) Dylan Jeffers

* origin/main: (161 commits) Audius Client (Web and Mobile) v1.5.100 Add client support for single transaction reward claiming (#9801) Enable audiusd workflow in main release (#9820) Version Packages (#9772) Add audius-d to monorepo (#9808) push cmds for core (#9807) Reapply libs split (#9817) [C-5026] Optimistic UI tweaks (web only for now) (#9816) Reactions also unset is_hidden for blast sender (#9813) Don't show empty chat prompt for blasts (#9815) [QA-1732] Undo test code to fix unread pill check for notifications (#9814) [PAY-3214] Add CreateBlast CTA to followers and supporters user lists (web) (#9804) [PAY-3426] Set unread count when seeding chats with blasts (#9812) [C-5033] Prevent notification when artist comments (#9799) Add back /d route, aka /uptime (#9809) UDF for chat_allowed (#9789) Audius Protocol v0.6.195 revert max width on chats (#9806) [PAY-3216] Update upload share buttons (#9805) Revert libs split (#9802) ...

[1e9eb11] worker to fix truncated qm audio cids (#9810) Steve Perkins [e28b82b] Add client support for single transaction reward claiming (#9801) Marcus Pasell [b0252db] Add audius-d to monorepo (#9808) Danny [1c2edae] Reapply libs split (#9817) Sebastian Klingler [6455bae] Reactions also unset is_hidden for blast sender (#9813) Reed [07c0e5a] [PAY-3426] Set unread count when seeding chats with blasts (#9812) Reed [075ea1a] [C-5033] Prevent notification when artist comments (#9799) Dylan Jeffers [d2b6cb4] Add back /d route, aka /uptime (#9809) endline [3f862a8] UDF for chat_allowed (#9789) Steve Perkins

schottra

Just leaving some post-merge comments. Don't mind me.

schottra · 2024-09-24T15:03:59Z

packages/libs/src/sdk/api/challenges/ChallengesApi.ts

      case ChallengeId.SEND_FIRST_TIP:
      case ChallengeId.TRACK_UPLOADS:
-        return `${args.userId}`
+        return `${encodeHashId(args.userId)}`


Just curious:
IIRC, encodeHashId is not guaranteed to produce output and can return null if the input is not a valid ID. What seems like it would generate some pretty subtle nested failures if it were to happen here. Any thoughts on using a zod parser or something that will throw if the input is invalid so that we get a clearer error in those cases?

schottra · 2024-09-24T15:10:47Z

packages/libs/src/sdk/scripts/manageRewardsLookupTable.ts

+    AddressLookupTableProgram.createLookupTable({
+      authority,
+      payer,
+      recentSlot: slot - 1


In the future, maybe add a comment for things like this? It's going to be cryptic looking at this code later on. :-)

packages/libs/src/sdk/services/Solana/programs/ClaimableTokensClient/ClaimableTokensClient.ts

rickyrombo added 6 commits September 12, 2024 01:20

[QA-1543] Claim rewards in a single transaction

c534151

Fix count for attestations decoding

d1cf11a

this naming hurts me

2f1e3af

Update client to support shortened ChallengeId and specifiers

0b36a1a

fix custom errors

4e52c3f

typo

640c7da

rickyrombo requested review from dharit-tan, isaacsolo, schottra and sliptype September 19, 2024 22:45

pull-request-size bot added the size/XL label Sep 19, 2024

add changeset

406a29e

rickyrombo assigned raymondjacobson Sep 19, 2024

rickyrombo mentioned this pull request Sep 19, 2024

Update DN to use shortened rewards challenge IDs and specifiers #9803

Merged

dharit-tan reviewed Sep 20, 2024

View reviewed changes

packages/libs/src/sdk/api/challenges/ChallengesApi.ts Outdated Show resolved Hide resolved

dharit-tan reviewed Sep 20, 2024

View reviewed changes

dharit-tan approved these changes Sep 20, 2024

View reviewed changes

sliptype approved these changes Sep 20, 2024

View reviewed changes

packages/spl/src/reward-manager/RewardManagerProgram.ts

blob(45),

ethAddress('operator')

]),

maxAttestations,

Copy link

Contributor

sliptype Sep 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙌

rickyrombo added 2 commits September 20, 2024 16:14

use solana constant

dc72a0e

mobile challenge config

712b737

pull-request-size bot added size/XXL and removed size/XL labels Sep 20, 2024

rickyrombo merged commit e28b82b into main Sep 21, 2024

rickyrombo deleted the mjp-single-transaction-rewards branch September 21, 2024 05:30

rickyrombo mentioned this pull request Sep 24, 2024

Add new shortened Challenge IDs to valid ID list on mobile #9841

Merged

schottra reviewed Sep 24, 2024

View reviewed changes

Conversation

rickyrombo commented Sep 19, 2024

Description

How Has This Been Tested?

ROLLOUT PLAN

Uh oh!

changeset-bot bot commented Sep 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

gitguardian bot commented Sep 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ GitGuardian has uncovered 17 secrets following the scan of your pull request.

Uh oh!

audius-infra commented Sep 19, 2024

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dharit-tan left a comment

Choose a reason for hiding this comment

Uh oh!

sliptype left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

audius-infra commented Sep 20, 2024

Uh oh!

schottra left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

changeset-bot bot commented Sep 19, 2024 •

edited

Loading

gitguardian bot commented Sep 19, 2024 •

edited

Loading