PicAppear coming to NEAR

Inspiration

As folks, who co-founded PizzaDAO and other global community organizations, we realized that event ticketing protocols, and the digital memorabilia services like POAP have a long way to go to satisfy vibrant communities with different artistic values (care about the art), and most importantly who care about experiences. In particular, we became more aware that

1) these digital collectible services focus too much on the JPEG, and not other media that can be more immersive.

"What if you can attend a music concert, and your POAP is not just a JPEG, but perhaps a short music recording .. or better yet your favorite artist dropping you a personal message". This sounds amazing, but there isn't actually a simple way to create such digital memorabilia and most importantly there isn't a way to experience

"What if you can arrive to a particular location find that there's a collectible at a particular relocation (geo-tagging NFTs)".

2) The second part we became aware is that we need not only nee better ways to mint, create and experience these digital collectibles. But, that the experience must be seamless and more decentralized. (POAP is not decentralized).

With this in mind, this app started off as a web-based project at ETHNewYork. Where we were focused on building a new protocol for minting NFTs that represent events. We began building a web frontend and targeted deploying on BOS (blockchain operating system). But, soon realized that traditional web technologies were too constraining for our "how we want users to experience digital assets". Hence, now it's a fully mobile application, that will be getting new features that are only possible on mobile.

What it does

1 -- Wallet-less Experience

It allows you to login with Google OAuth. Behind the scenes we started implemented a scheme similar to ZK Login, where we leverage the OpenID protocol and the ISS and SSU identifiers alongside a salt, to then derive a unique identifier that connects to a particular wallet (behind the scenes).

2 -- Meta Transactions / Gas Relayer Functionality

We allow users to mint NFTs without even knowing they are doing it on a blockchain. We leverage @microchignu's implementation for writing our own gas relayers.

Shout out to Mintbase and Banyan for all their support on this!

3 -- Immersive Multi-Peer Experiences

Mint your NFTs, but also get to interact with them in real-space, and better yet!! Get your friends involved by having their PiAppear app tether to yours.

4 -- Create NFTs leveraging AI

We used our own SDK (OpenAIKit) to allow folks to create images and also text for NFTs.

How we built it

We used MintBase for indexer (big shout out). We leveraged @microchipgnu's relayer implementation and extended to allow user to create NEAR accounts, and mint NFTs without paying for gas. We researched the ZK Login paper and documents to implement Google OAuth on for account registration We used ARKit for AR And we modified our working UI for this hackathon -- but, most of the work on the UI took place getting the multi-peer AR working, and also 3D modeling a NEARCON logo (for demo).

We implemented a no wallet

Challenges we ran into

Lack of documentation on indexers, also existing indexers missing metadata. Mintbase has the best one so far -- shout out to Banyan for helping us get in touch. More tutorials and docs on meta-transactions and gas relayer -- shout out to Mintbase team. Mobile integration was pain!! Not all OAuth providers support OpenID standard 100% and our services had issues.