There was no shortage of creative entries in Neosensory’s second developers contest “Expand your Senses” hosted in cooperation with Edge Impulse and Hackster, ranging from a device helping firefighters move through smoke to a gadget letting you feel your internet connection. The judges were especially impressed with SensiGlove, a device allowing wearers of prosthetics to feel sensation, and crowned the entry by Jay Desai and Ryan Peng the winner.
The team
Desai proved formal qualifications are less important than a willingness to learn when it comes to software development: The biology major learned most of his skills on Youtube and Udemy.
“I’ve really, always just been interested in creating solutions for problems and software development is a great way to go about doing that,” he said.
Desai, a rising senior at Emory University, met his teammate Ryan Peng at a virtual hackathon when both were looking for a partner. Since then, the two have been entering competitions together. Their contest experience – they enter one contest per month on average – came in handy when looking for a winning project idea.
“I have a decent idea as to what kind of ideas tend to stand out, and what kind of ideas are pretty overused,” Desai said. “When you come up with a creative idea that stands out, there’s a much higher chance of success.”
Developing SensiGlove for prosthetics
One day, as Desai was pondering how prosthetics work, he realized while advanced artificial limbs were becoming cheaper, many lacked the ability to transfer sensation to the wearer.
One important factor was accessibility. Desai and Peng wanted every person who already owned a prosthetic arm to be able to use their invention, which meant creating a prosthetic from scratch was off the table. The two eventually landed on the idea of a glove that could simply be worn over any artificial arm. This also helped keep the cost down and make SensiGlove affordable.
The device consists of a fabric glove, conductive plates embedded on the thumb, fingers, and palm connected to a series of GPIO pins on an Adafruit nrF52840 microcontroller that contain built-in capacitive touch sensors, and a Neosensory Buzz wristband. Buzz is a haptic-feedback device with four vibrating motors that allows the wearer to feel data input as vibrations. The conductive plates are grouped into thumb; index and middle fingers; ring and pinky fingers; and palm. Each group corresponds to one of Buzz’ motors and vibrates when touched by human skin.
In addition, the capacitive touch reading ranges can also be modified to ranges corresponding to other common materials (besides human skin) to enable the user to make distinctions between touching different materials through differing haptic feedback mechanisms. By using Edge Impulse’s machine learning platform, Desai and Peng also developed a neural network that allows the user to even perceive different touch patterns, such as tapping, through haptic feedback.
Overcoming challenges
Turning SensiGlove into reality proved difficult at times since neither Desai nor Peng had much experience with hardware development. In the end, persistence and research paid off.
“It was kind of tough, like understanding how all that works. But I think that the Neosensory SDK was pretty user friendly. So that was able to help us in terms of the programming aspect of things,” Desai said. “And then the design aspect was also a little challenging, because we never really worked with hardware before. But YouTube is a great resource, and there’s a lot of resources online to jumpstart and accelerate your projects.”
Dr. Scott Novich, co-founder and CTO of Neosensory, was impressed by how well rounded Desai and Peng’s entry was.
“Jay’s team nailed it across the board with a clear useful concept, good execution, and fantastic demonstration!” Novich said.
A second entry
But Desai and Peng weren’t content entering just one project into the Expand your Senses contest, so they submitted a second entry: EchoSense. The device consists of a glove embedded with an ultrasonic sensor, a microcontroller, and a Neosensory Buzz. The ultrasonic sensor sends out a sound wave, which bounces back carrying information about obstacles in the room. This information is then sent to Buzz, which presents it to the wearer in the form of vibrations.
“So, we can have people that are blind gauge distance to various obstacles in their environment without actually having to contact it,” Desai explained. “In terms of balancing the work, you just have to have good time management skills. And we split our time between these two projects, such that we had time for both.”
Other projects using Buzz
Desai and Peng have also used Buzz in another contest, MakeUofT 2021, where the pair’s project won second place overall. OptiLink, a sight-assistance device, used a total of three Buzz bands. Desai said he likes how easy to use Buzz is, and it also helps his submissions stand out.
“When you see a hackathon project these days, it’s usually something a lot more basic, or something that you’ve probably seen before, but like a spin to it,” he explained. “But with the Buzz, it’s like a completely different concept. You rarely see that kind of device in a hackathon submission. So I think integrating the Buzz in your submission will help your submission stand out. And that’s why I’m a huge fan of it.”
Novich enjoys seeing the many creative uses other developers find for Buzz.
“It’s been way more fun than I ever imagined seeing developers put our technology to work,” he said. “We’ve been working on the core technology for years and have always known there’s a huge space of possibilities we haven’t even begun to think about.”
Read more about SensiGlove’s conception here and follow Jay Desai on Devpost here.