FTM Speech Tools

Speech synthesis tools for DeSci dapps and Conversational interface
Adjusting Rate and Pitch of Speech Tools
Internationalization in key languages for better dapp support
Sleep Mode

Inspiration

The inspiration of the project came from our previous work 14 years back when we were developing a language learning and a vocabulary project for OLPC Laptops. We came across TTS (text to speech) engine libraries that enabled us to develop an activity for the OLPC's Sugar environment: http://activities.sugarlabs.org/en-US/sugar/addon/4234. During the early days of Post PC era, we developed multiple language learning projects on iphones, but were always short of integrating speech synthesis and a proper P2P storage mechanism: https://www.youtube.com/@seetateam/videos. With advent of blockchain and its eco-system, we can now develop and integrate a decentralize developer toolkit for DeSci eco-system

What it does

FTM Speech Tools are developer tools that enables smart contract designers, developers and community managers to integrate speech synthesis, conversational interface and internationalization for their DeSci dapps using Fantom, Covalent APIs, AWS, Chainlink and Filecoin.

We are also developing an analytics toolkit that enables data tabulation, organization, validation and visualization of data using an OSS spreadsheet. Our toolkit enables a speech interface for dapp developers to integrate it within their solution as a low code framework to empower Web3 users via speech synthesis and internationalization.

How we built it

We developed the speech synthesis toolkit using open source TTS (text to speech) engines, interface using JS frameworks and are developing on-chain integrations for DeSci dapps. We also integrated an analytics toolkit built using an OSS spreadsheet for data tabulation, organization, collaboration of data.

We are utilizing Fantom, Covalent APIs and AWS, Chainlink, Tableland and Filecoin in dapp modules.

Covalent NFT Dashboard: https://github.com/seetadev/Speech-Tools/tree/main/Covalent-NFT-Dashboard

We built the developer tooling for alarm and task notifiers, reference notes dapp using Tableland and OSS JS frameworks. This dapp serves as a notes section for the conversational interface for understanding smart contracts. We are interacting with off-chain data sources like Wordnet dictionary and fetching keywords using Chainlink.

Challenges we ran into

AWS deployment: We were using PV (paravirtual) based classic instance. We were required to convert PV (paravirtual) instance to an HVM (Hardware Virtual Machine) and convert it from a classic instance to a VPC instance. We faced a number of challenges in converting our PV instance to an HVM instance as our instance was not reachable via ssh. We arrived at a good conclusion on the issue by doing the following: Created an ami of the current classic instance. Launch a new instance from this ami after shifting to vpc as the base with new VPC security groups and updated volume. The newly launched instance is working and we are able to ssh and has the code base. We purchased the developer support plan using AWS credits and have being shared key pointers on converting our VPC-based PV instance to a VPC-based HVM instance. This challenge would not have been solved without the support of AWS team.
Deployment challenges and learning the steps for deploying at FTM testnet and mainnet.

Accomplishments that we're proud of

Ability to support internationalization of speech synthesis tool.
Ability to adjust the pitch and rate of sound for better clarity on certain words.
We are proud of encouraging the development of a decentralized eco-system based on the foundation of sharing of ideas and active collaboration among the community members using Web3 powered platform using Fantom, Covalent NFT Dashboard and AWS.
Ability to utilize the potential of Tableland and Chainlink to scale the Speech tools with powerful features like reference notes, alarm notifier and also interact with off-chain data sources like Wordnet dictionaries.

What we learned

We can utilize SocialCalc, Machine Learning Models coupled with decentralized Fantom blockchain and Covalent API endpoints for developing a FTM Speech tools platform, which could be utilized for collaboration. We also witnessed the great ecosystem available to developers to learn and contribute in the Fantom, Covalent and AWS eco-system.

Lessons in reference to using Chainlink:

Chainlink enables to interact securely with and obtain from off-chain sources like Wordnet dictionary. We can use Chainlink to extend our project to interact with key dictionary providers and help the web3 user understand the context better.

Some of the learning lessons in reference to unlocking the potential of FVM and Tableland:

FVM enables us to use composable blocks exposed through standardized interfaces, enabling the construction of automated Web3 solutions.
Requirement of an intermediate burner wallet to move filecoin from F1 (exchange address) to F4 address (Filecoin Mainnet address).

Lessons in reference to using Tableland:

Great tool for MySQL developers interesting in moving from a LAMP architecture to Web3 stack.

What's next for Speech Tools

Extending the developer toolit as an API or low code framework in Fantom and Chainlink boilerplate examples.
Decentralize key components, bottlenecks in our existing solution: We were able to improve the storage of off-chain data;
Production Launch: Enable Fantom Mainnet transactions