Babies have one primary way of communication: crying. Deaf and hard-of-hearing parents have many ways of knowing exactly what their children need even without hearing them, such as interpreting body language to determine their needs and emotions or devices that alert parents to crying noises by giving visual alerts.
Software engineer Mithun Das wanted to create another option for peace of mind and set out to build BabyConnect, a project that won both the Audience and Judges’ Favorite awards in Neosensory’s Feel the Future contest.
The idea behind BabyConnect
BabyConnect is a tablet-only app that allows young children and other non-verbal people to send messages such as “help”, “hunger”, “yes”, and “no” to another person. When the child presses the symbol corresponding to their need, the recipient receives the message in the form of haptic feedback. In order to achieve this, Das relies on Neosensory Buzz, a haptic-feedback wristband featuring four motors. Each symbol is turned into a unique vibrational pattern and sent to Buzz via Bluetooth. With practice, the user can learn to distinguish each pattern and message without thinking about it.
Das had the idea for BabyConnect while he was researching Augmentative and Alternative Communication (AAC) — any kind of communication without the use of speech. This can include everyday communication like gestures and facial expressions done without the need for tools, also called unaided systems. Aided systems, on the other hand, can take the form of basic systems like pen and paper or high-tech systems such as speech-generating devices or communication through computers.
He then set out to create an app aimed at parents that would combine AAC and haptic feedback.
Creating Baby Connect
Inspired by a movie in which a character communicates in morse code with a light bulb, Das decided to use morse code as a basis for his vibrational patterns. Buzz’ four motors allowed him to modify the code and make it more efficient.
“One difference from Morse code is that, in Morse code, signals are sequential such as DOT DOT DOT ( S) or DASH DASH DASH ( O) but here, as we have Buzz with 4 motors which can vibrate independently, we have 4 channels. So “LIKE” is represented by DOT, DOT, DASH ( motor 1, 2 and 3 vibrating simultaneously),” he says.
In order to speed up the learning process, Das developed a training program on the app. Users are shown three symbols (e.g. “no”, “hungry”, and “help”) and feel a vibration matching one of the symbols. They then have to choose the correct symbol and can work their way through several levels.
BabyConnect also features voice transcription, so older kids can verbalize their needs instead of pressing a symbol.
As an added safety feature, BabyConnect monitors indoor air quality (IAQ), temperature, humidity, and ambient light. Users can check the app or press Buzz’ power button to feel the data via vibrations.
What’s next
Das is currently working on expanding BabyConnect’s functions by mapping different types of baby cries: hungry, fussy, and pain. This would allow parents to immediately feel their child’s need by vibrations alone. In order to achieve this, Das analyzed recordings of babies crying. He was able to detect differences between cris from hunger and pain, but other distinctions are more difficult.
“When a baby is in pain, they cry continuously with a break just to breathe. But when they are hungry, you will notice some gaps in between. But it’s very hard to separate hungry vs fussy. At present my model is able to classify noise vs baby cry with around 85% accuracy, but it’s not quite good in separating hungry vs pain vs fussy,” Das says.
To collect more data, Das plans on starting a campaign that would allow others to contribute recordings. You can follow him on hackster.io and Twitter.