Relloquent Project - Using Intel RealSense Technology for Alternative Communication

1 0
  • 0 Collaborators

Relloquent is a face-controlled communication system designed for people who cannot use speech to communicate effectively in all situations. ...learn more

RealSense™

Overview / Usage

==How does Relloquent work?

Use a "half-smile" to move to the next cardboard, a kiss to select the option, an "eyebrows up" to exit from the cardboard. When an option is selected, the system speaks out the option. You can choose up to 7 facial expressions. You might need to choose at least 3 expressions.

Video 1: https://youtu.be/mEqEbgy5NtI
Video 2: https://youtu.be/IEtzM3x1Xto
Video 3: https://youtu.be/YexMO3pd8Zw
Video 4: https://youtu.be/Rn3Y2mugFU0

*All videos show the single way cursor movement version. We can set up other expressions to move the cursor, not only to the right, but also to the left, down and up.

==What is Relloquent?

The Relloquent is the first ACC (Augmentative and alternative communication) system to use Intel RealSense technology. Relloquent is a face-controlled communication system designed for people who cannot use speech to communicate effectively in all situations.

Relloquent uses an Intel RealSense Camera to capture very small face expressions of the user that are used to choose options on a communication board. These options are organized int major groups: I am, I want, I want to talk to, etc, to make the selection easier.

When the user chooses the option, the system speaks out the specific sentence. There also an option to type on an on-screen keyboard.

==How does Relloquent work?

Use a "half-smile" to move to the next board, a kiss to select the option, an "eyebrows up" to exit from the board. When an option is selected, the system speaks out the option. You can choose up to 7 facial expressions. You might need to choose at least 3 expressions.

==How will Relloquent work?

We are building up a dictionary of combinations of expressions that will allow the person to communicate faster using gestures.

Something like morse code, but with 8 different symbols instead of 2.

==What is AAC?

Augmentative and alternative communication (AAC) encompasses all other forms of communication than oral speech that can be used to express thoughts, needs, wants, and ideas. Gestures, symbols, write and even facial expressions are examples of these forms.

== Whom AAC systems are designed for?

Adults or children who cannot use speech to communicate.

==Purpose:

What we are proposing is using facial expressions to express more than the facial expressions could do. For example, we can use a "eyebrows up" to express surprise or a smile to express happiness. However we could also use a combination of these facial expressions to call a doctor or tell others we are in pain or hungry.

==Relloquent vs. others ACC systems

Most of the AAC systems use eye tracking to help the user to choose options from a communication board. However, there are many people who cannot speak but are able to perform facial expressions and are forced to use the eye tracking to communicate, leaving facial expressions just to demonstrate intensities on their emotions or to guide the caregiver to choose an option on a manual communication board.

The issue on using eye-tracking approach is that you might not have many options at the same time on the virtual communication board due the lack of precision.

Relloquent is the first system to take full advantage of facial expressions transforming this potential nonverbal communication in high significance verbal communication.

The human face may be able to perform various facial expressions that when combined could verbalize the most complex sentences.

Comments (0)