Sharkroom

homepage
sharkman
GIF
Project Demo

Inspiration

In this COVID-19 pandemic, teaching on a Work from Home basis has become a great mess, especially for toddlers. We just wanted to create something which eases the process & helps brought a smile to everyone's face and one of the best ways to do that is by making things more intuitive & fun! Not only kids but adults are attracted by fun stuff & our project Sharkroom helps people achieve that!

To be more precise, we got our inspiration from Apple's Animoji™ ✨

What it does

Sharkroom is based on Tensorflow.js, where the main goal is to provide a next-level teaching experience between teachers and children specifically toddlers. In this COVID-19 pandemic, teaching on a Work from Home basis has become a great mess, especially for toddlers. Here project Sharkroom comes to the rescue. Sharkroom provides p2p video conferencing via WEBRTC between 'Sharkman' and 'n' number of persons with accurate live captioning & transcribe feature! Sharkroom is easy to use & requires no custom application/software installation. The entire application runs completely on the client-side & is 100% GDPR compliant, so that there's no privacy leak. In short, it's a portal where we're gonna use the shark faces instead of our faces with live captioning using facemesh.js. The model is encapsulated inside posenet.js skeleton.

It has the following features:-

Attractive, fun, intuitive & playable! :)
It is supported across all updated browsers.
WEBRTC p2p real-time video chat room (Shark & User).
Provides 80% accurate live captioning for free in realtime.
Embedded Dark mode feature!
Download captions as .txt.
& some easter eggs 😉

How we built it

We used the following tech stacks for building our project,

Challenges we ran into

Sharkroom is running on Tensorflow.js and facemesh.js encapsulated inside Posenet model. Configuring the model for rendering inside the canvas via paper was a really tough job. Moreover, training the skeleton model and configuring the skeleton.js with the 'sharkman' svg also took a lot of time for us. Also, it was a bit difficult for us to collaborate in a virtual setting but we somehow managed to finish the project on time.

Accomplishments that we're proud of

We are proud of finishing the project on time which seemed like a tough task as we started working on it quite late due to other commitments and were also able to add most of the features that we envisioned for the app during ideation.

What we learned

A lot of things, both summed up in technical & non-technical sides. For the technical side, we got to learn so much about configuring posenet models and training them on GCP (our system couldn't handle that much of processing :p) and also gained some UI/UX skills. The SVG was created in Adobe Illustrator, which isn't an easy tool to start editing with, but yeah it was a great experience working overnight & watching MLH's mini-events! Not to mention, Stackoverflow was the gem for us while we're troubleshooting some complicated issues late-night.

Apart from these, we also had to do research on the current education system, children's likes & privacy policies.

What's next for Sharkroom

We're going to go through many changes & planning to add the following updates to the project in the future,

UI improvement
Add on-screen canvas video screen-recording & screen-capturing feature. Although for now, we have avoided that because of GDPR privacy protocols.
Add more (animal) characters (Currently on Sharkman v1.0)
Custom Background set feature.
Multilanguage real-time translation support!
Direct interaction with Zoom, Google Meet & similar software.

Sharkroom can be further developed to add more features that attract the heart of fellow toddlers and small children. This would include some research work which we are planning to undergo soon!