Skip to content

RyanRana/market-dog

Repository files navigation

MarketDog

A chat-driven autonomous sell agent. You text it on imessage what you want to sell (with photos) and any conditions you have, then it researches the market, lists your items on the best platforms, handles everything, and doesn't text you until a deal is closed.

User flow

Chat: "Sell my ASUS ROG Flow laptop, minor scratch on the lid" + 📎 photo
        │
        ▼
Agent runs Nimble web search + Senso AI synthesis
        │
        ▼
Chat shows: { Min $X · Suggested $Y · Max $Z }
            + link to cited.md (the why)
            + draft title
        │
        ▼
Agent: "Where do you want to list it?"
        [Craigslist]  [eBay]  [Facebook Marketplace]
        │
        ▼  user clicks
Visible Chromium opens, navigates to marketplace login
        │
        ▼
Agent: 📸 screenshot of login page
       "Log in in the browser, then tap Continue"
       [ I'm done — take it over ]
        │
        ▼  user logs in, clicks Continue
Agent takes over: fills form, attaches photos, parks before final submit
       (Craigslist requires email confirmation, eBay's wizard wants category, etc.)

Quick start

npm install
npx playwright install chromium

cp .env.example .env
# fill in NIMBLE_API_KEY and SENSO_API_KEY
# (no marketplace credentials needed — you log in manually in the visible browser)

npm run dev
# → http://localhost:3000  (chat interface)

Optional: iMessage bridge

# In .env, also set:
#   ANTHROPIC_API_KEY
#   SPECTRUM_PROJECT_ID / SPECTRUM_PROJECT_SECRET   (from https://app.photon.codes)
#   AGENT_GREET_PHONE                                (your phone, E.164)
#   MARKETDOG_URL=http://localhost:3000

npm run dev      # web UI + rails  (terminal 1)
npm run agent    # iMessage bridge (terminal 2 — uses Bun)

Text the agent: "sell my Sony WH-1000XM5 headphones, like new".

Architecture

src/
  app/
    page.tsx                              # chat UI (single page)
    cited/[id]/route.ts                   # serves saved cited.md
    api/
      sell/
        prepare/route.ts                  # POST multipart: item + photos → research + draft
        publish/route.ts                  # POST: kick off browser with chosen marketplace
        resume/route.ts                   # POST: user clicked Continue after logging in
        listings/route.ts                 # GET all
        listings/[id]/route.ts            # GET one
        events/[id]/route.ts              # GET (SSE): live browser progress
      uploads/[listingId]/[filename]/route.ts  # serves uploaded photos
  lib/
    types.ts                              # shared types
    research.ts                           # Nimble + Senso pipeline
    draft.ts                              # title + description templates
    cited.ts                              # builds the pricing justification (cited.md)
    store.ts                              # JSON persistence (data/listings.json)
    photos.ts                             # photo storage to data/uploads/
    events.ts                             # in-memory event bus (SSE-safe)
    awaitUser.ts                          # pause/resume coordination for manual-login
    browser/
      session.ts                          # headed Chromium launch
      types.ts                            # MarketplaceAdapter + PublishContext interfaces
      publish.ts                          # orchestrator: launch → adapter → cleanup
      marketplace.ts                      # adapter registry
      adapters/{craigslist,ebay,facebook-marketplace}.ts
agent/
  spectrum-bridge.ts                      # iMessage bridge (Bun, Claude tool-use)
data/
  listings.json                           # persisted state (gitignored)
  uploads/<listingId>/photo-N.png         # uploaded photos

How the browser automation works

Each adapter is tiny — its only job is deterministic navigation + manual-login pauses. The actual form-filling is handed to a vision agent that screenshots the page, decides what to click/type with Claude, and drives Playwright via a small set of semantic tools.

// Inside an adapter
await page.goto(loginUrl);
await ctx.awaitUser("log in manually");
await page.goto(postUrl);
await ctx.awaitUser("pick category if needed");

// Delegates to the vision agent — Claude sees the page and acts
await ctx.agentBrowse(`Fill this listing form with:
  Title: ${listing.title}
  Price: ${listing.suggested_price}
  Description: ${listing.description}
  Photos: ${listing.photos.length}
Do NOT click Publish/Submit/Post. Call done when filled.`);

The agent's tool set: click_by_text, click_by_role, fill_form (fills every visible field in one call), fill_by_label (single-field retry), select_option, press_key, scroll, upload_photos, wait, ask_user, done. Selectors are semantic (text + ARIA role), not CSS — they survive marketplace UI tweaks. fill_form returns per-field success/failure so the agent can retry just the fields that didn't take. If the agent gets stuck or hits a field it can't fill (condition, zip code, category specifics), it calls ask_user and the chat surfaces the question with a Continue button.

The agent drives the whole posting — including the final Publish/Post button. The only hard ban is money movement: the SYSTEM prompt blocks clicking "Pay", "Boost", "Promote", "Upgrade to Premium", or filling billing fields. The listing's own price field (the user's asking price) is allowed. If a payment screen is the only way forward, the agent calls ask_user and stops.

The orchestrator confirms "actually live" via URL change: if the page is still on the same form path after the agent calls done, status flips to failed (needs human takeover) rather than published.

Adding a new marketplace

Add src/lib/browser/adapters/<name>.ts:

export const myMarketAdapter: MarketplaceAdapter = {
  name: "mymarket",
  async publish(listing, ctx) {
    await ctx.page.goto("https://...");
    await ctx.awaitUser("log in");
    // ...any deterministic navigation...
    await ctx.agentBrowse(`Fill this form: ${describe(listing)}`);
    return { marketplace_url: ctx.page.url() };
  },
};

Register it in src/lib/browser/marketplace.ts and add the name to MarketplaceName in src/lib/types.ts.

What's intentionally NOT here

  • No payment — sell flow is free anyway; buy flow is not in scope.
  • No autonomous final submit — agent always stops before posting becomes irreversible; the human clicks the last button.
  • No credentials in env — manual login in the visible browser. The agent never sees your password.
  • No lead-chat negotiation — described in the original spec; not built.
  • No buy pipeline — scaffolded by the adapter pattern but not implemented.

About

autonomous sales agent in imessage.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages