[PAY-3319][PAY-3399] Update Discovery to support new purchase transaction details #9586

rickyrombo · 2024-08-29T22:57:38Z

Adds a new column to usdc_purchases table that is a direct clone of the "extended" splits object (the one returned from tracks/<id>/access-info )
Writes to that new column on successful purchases, using the splits gathered from the price history table
Migrates existing data to that column, forming the new splits model
⚠️ Removes legacy purchase validation in indexing. Verified in prod there are no price history records with legacy splits that are still active (there's a handful of them but they are no longer usdc gated)
⚠️ Removes user bank purchase indexing and tests. Using forced app update to ensure users are using payment router now

NOTE: The splits in the usdc_purchases table does NOT tell you how much each individual was paid, just how much they were owed. "Pay Extra" is separated in the receipts. When doing credit splits we'll probably want to revisit this.

…e()" This reverts commit 42c7d03.

changeset-bot · 2024-08-29T22:57:42Z

🦋 Changeset detected

Latest commit: d78919c

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages

Name	Type
@audius/sdk	Minor
@audius/sp-actions	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

rickyrombo · 2024-08-30T10:13:39Z

packages/discovery-provider/src/api/v1/models/access_gate.py

    "extended_payment_split",
    payment_split,
    {
+        "user_id": fields.Integer(required=False),


network splits don't have a userId, so make this optional

raymondjacobson

lgtm!

raymondjacobson · 2024-08-30T19:13:11Z

packages/discovery-provider/plugins/notifications/src/types/dn.ts

  signature: string
  slot: number
  updated_at?: Date
+  splits: any


can this be typed?

This in theory should be autogenerated from sql-ts which seems to put any for JSONB types. I didn't know how to regenerate it so I just based it on stream_conditions etc

oh lmao.... rip yuck ew. keep it autogenerated 🤷

raymondjacobson · 2024-08-30T19:13:51Z

packages/discovery-provider/ddl/migrations/0093_splits_in_purchases.sql

+        user_payout_wallet_history
+        JOIN usdc_purchases ON seller_user_id = user_id
+    WHERE
+        block_timestamp < usdc_purchases.created_at


should this be <=? maybe in practice it doesn't matter

Yeah I don't think this matters. Also I'd argue the price change being processed strictly before the purchase makes more sense anyway

schottra

No major notes here. Looks good.

I do wonder if we shouldn't just implement the ability to handle credit splits now vs later. But I guess we can just do another migration then.

schottra · 2024-08-30T20:33:18Z

packages/discovery-provider/src/api/v1/models/access_gate.py

    "extended_payment_split",
    payment_split,
    {
+        "user_id": fields.Integer(required=False),


schottra · 2024-08-30T20:35:37Z

packages/libs/src/sdk/api/generated/full/models/Purchase.ts

        'createdAt': json['created_at'],
        'updatedAt': json['updated_at'],
        'access': json['access'],
+        'splits': ((json['splits'] as Array<any>).map(PurchaseSplitFromJSON)),


Whoa that's a weird one.

schottra · 2024-08-30T20:37:41Z

packages/discovery-provider/src/api/v1/models/users.py

 )

+split = ns.model(
+    "purchase_split",


I may be confused, but if we're storing the percentage in the DB, why are we not returning it in this model?

Yeah these percentages are another thing I'd want to address. I purposely left them out because they're not useful right now in this flow anyway, and they can be misleading. For example, now with the network split, "100%" doesn't really look like "100%"

I think it's worth a conversation about whether we ultimately have "nested" percentages, like:

"splits": [ { "label": "users", "percentage": 90, "splits": [ { "user_id": 1, "percentage": 100, ... }, ] }, { "label": "network", "percentage": 100, ... } ]

or do we recalculate percentages after adding the network cut?

I should write up more details on this...

but yeah i think it's out of scope for right now - we just want to see amounts for the receipts

and to your point, it probably shouldn't be in the DB either then, but I didn't feel like adding yet another split format would be beneficial.

packages/discovery-provider/ddl/migrations/0093_splits_in_purchases.sql

packages/discovery-provider/integration_tests/gated_content/test_access.py

packages/discovery-provider/src/tasks/index_user_bank.py

dharit-tan · 2024-08-30T20:49:44Z

packages/libs/src/sdk/api/generated/default/models/ExtendedPaymentSplit.ts

    return {

-        'userId': json['user_id'],
+        'userId': !exists(json, 'user_id') ? undefined : json['user_id'],


neat, maybe this is a pattern we can learn from instead of 'user_id' in json

rickyrombo · 2024-08-30T21:48:18Z

I do wonder if we shouldn't just implement the ability to handle credit splits now vs later. But I guess we can just do another migration then.

@schottra I had the same concern... but yeah the migration would be the same sort of thing anyways... so I think it's fine for now.

[58bdb34] [PAY-3324] Purchase/Sales details w/ take rate (#9602) Marcus Pasell [ebf9040] Support negative fractional numbers in FixedDecimal, default USDC formatting (#9600) Marcus Pasell [fa2ac4c] [C-4953] Mobile comments editing + replying (#9584) Sebastian Klingler [ed697c2] New audius-query hook for paginated queries (#9592) JD Francis [847c075] [PAY-3319][PAY-3399] Update Discovery to support new purchase transaction details (#9586) Marcus Pasell [760bfee] Center long track/collection titles on mobile (#9598) Reed [c18d431] [C-4799] Add mobile-web comments (#9578) Dylan Jeffers [74ab87e] [PAY-3388] Add user feed V1 endpoint (#9583) Randy Schott [f9ae82a] [QA-1510] Simplify hover gradient styles coming from harmony (#9590) Raymond Jacobson [86343d1] Fix search filters (#9595) Sebastian Klingler [398190f] [QA-1498] Reland fix to QA-1498 (#9582) Raymond Jacobson [0b92de2] Revert "Enabling blocking users from commenting (#9469)" (#9552) Isaac Solo [27b6332] [QA-1531][QA-1535] Fix mobile track page issues (#9589) Raymond Jacobson [7649131] Remove quality toggle for mobile downloads (#9587) Reed [92737e3] [PAY-3372] Add sales aggregate endpoint (#9585) Andrew Mendelsohn [ef67166] [QA-1536] Remove download all buttons (#9581) Reed [83bb4bb] Set default comment entity_type in indexing (#9580) Dylan Jeffers [331111e] Update dapp-store build artifacts audius-infra [d08cce0] Fix mobile composer input newlines and unfurl overflow (#9575) Raymond Jacobson [7eb83d7] Fix broken offline mode (#9571) Raymond Jacobson [906f90a] [PAY-3377] Migrate remaining apiClient endpoints to SDK (#9559) Randy Schott [c908aef] Fix mobile comment overflow menu lint (#9577) Dylan Jeffers [0188a6a] [QA-875][QA-850] Fix lineup pagination when dupes are present (#9572) Raymond Jacobson [c83ca67] [PAY-3394] Maintain chat list sort order on blast (#9568) Reed [7470a55] [PAY-3397] Fix mobile layout with UserGeneratedText in DMs (#9570) Raymond Jacobson [2c39178] [PAY-3374] Fix selection colors for composer (#9565) Raymond Jacobson [78a028e] Update comment-section-api (#9551) Dylan Jeffers [fedd220] [PAY-3384] Implement web pay with anything (#9279) Raymond Jacobson [51e911f] [C-4981] Comments desktop tweaks (designer QA) (#9548) JD Francis [fa43c8d] [PAY-3152] Fix Accounts You Manage modal showing after switching accounts (#9558) Randy Schott [39c7764] [PAY-3371] Add endpoint to get tracks by user that have been remixed (#9554) Andrew Mendelsohn [39a1ec1] [QA-1526] Hotfix pagination issues (#9557) JD Francis [cb1446c] Fix types on harmony virtualized (#9553) Raymond Jacobson [c4965de] [C-4955] Comment action drawer (#9534) Sebastian Klingler [713712e] [PAY-2919] Migrate playlists endpoints from apiClient -> sdk (#9535) Randy Schott [f8c197f] [QA-1016] Fix playlist creation from track added time (#9529) Raymond Jacobson [ba1aaed] Fix trending reward notifs (#9532) Isaac Solo [303614d] Add virtualized filter button, Fix option select (#9543) Dylan Jeffers [97ac58e] Bump mobile apps to 1.1.112 Dylan Jeffers [c10b064] [PAY-3348] Remove dead withdrawals code (#9542) Marcus Pasell

[b87f574] Fix Splits Migration: Stage has users w/o indexed user banks (#9604) Marcus Pasell [847c075] [PAY-3319][PAY-3399] Update Discovery to support new purchase transaction details (#9586) Marcus Pasell [74ab87e] [PAY-3388] Add user feed V1 endpoint (#9583) Randy Schott [5c520c1] [QA-1533] Filter out deactivated users from UserList queries (#9596) Reed [c52e0cb] Filter genre metrics endpoint by allowlist (#9591) Randy Schott [0b92de2] Revert "Enabling blocking users from commenting (#9469)" (#9552) Isaac Solo

rickyrombo added 7 commits August 28, 2024 19:23

Add splits to purchases

8736157

Generate SDK

af5aad2

Rename to get_purchase_gate to disambiguate extend_access_gate()

42c7d03

Generate SDK (userId is optional now)

18c5819

Revert "Rename to get_purchase_gate to disambiguate extend_access_gat…

9c7468d

…e()" This reverts commit 42c7d03.

userId as optional number

607241d

remove hashid'ing purchase splits to be consistent

e044e3c

pull-request-size bot added the size/L label Aug 29, 2024

rickyrombo added 4 commits August 29, 2024 15:59

idempotency on the update

0ba4dc8

correct type to integer, since no longer hash IDing it

aa666d4

remove typo

fdc3368

Remove user bank purchase indexing and fix tests

14003a0

pull-request-size bot added size/XXL and removed size/L labels Aug 30, 2024

rickyrombo added 3 commits August 29, 2024 21:39

fix notifs tests

1a3a3da

it's gotta be stringified I guess

eca21dc

Generate SDK

a7ca952

rickyrombo requested review from dharit-tan, raymondjacobson and schottra August 30, 2024 10:03

rickyrombo added 3 commits August 30, 2024 03:10

update purchasesplit add payoutwallet, optional userid

0c8635f

changeset

d53a3e0

Update changeset

d78919c

rickyrombo commented Aug 30, 2024

View reviewed changes

raymondjacobson approved these changes Aug 30, 2024

View reviewed changes

raymondjacobson reviewed Aug 30, 2024

View reviewed changes

schottra approved these changes Aug 30, 2024

View reviewed changes

dharit-tan reviewed Aug 30, 2024

View reviewed changes

packages/discovery-provider/ddl/migrations/0093_splits_in_purchases.sql Show resolved Hide resolved

dharit-tan reviewed Aug 30, 2024

View reviewed changes

packages/discovery-provider/integration_tests/gated_content/test_access.py Show resolved Hide resolved

dharit-tan reviewed Aug 30, 2024

View reviewed changes

packages/discovery-provider/src/tasks/index_user_bank.py Show resolved Hide resolved

dharit-tan reviewed Aug 30, 2024

View reviewed changes

dharit-tan approved these changes Aug 30, 2024

View reviewed changes

rickyrombo merged commit 847c075 into main Aug 30, 2024

rickyrombo deleted the mjp-network-split-backend branch August 30, 2024 22:05

rickyrombo mentioned this pull request Aug 31, 2024

Fix Splits Migration: Stage has users w/o indexed user banks #9604

Merged

[PAY-3319][PAY-3399] Update Discovery to support new purchase transaction details #9586

[PAY-3319][PAY-3399] Update Discovery to support new purchase transaction details #9586

Uh oh!

Conversation

rickyrombo commented Aug 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

changeset-bot bot commented Aug 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

raymondjacobson left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

schottra left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rickyrombo commented Aug 30, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

rickyrombo commented Aug 29, 2024 •

edited

Loading

changeset-bot bot commented Aug 29, 2024 •

edited

Loading