UR’s Human-Computer Interaction research group (HCI) developed a wearable technology that delivers real-time feedback on speaking ability. The technology, named “Rhema,” makes use of the heads-up display capabilities of Google Glass to notify a speaker if they need to adjust their pace or volume.
Assistant Professor of Computer Science Ehsan Hoque is the senior author of the paper on Rhema. The project was a joint effort between Hoque, graduate student M. Iftekhar Tanveer and undergraduate student Emy Lin. The three researchers published their paper on March 31 in Atlanta at the Association for Computer Machinery’s Intelligent User Interfaces conference. “The talk was well received,” Hoque said. “Everyone liked the idea [of Rhema].”
Rhema works by picking up vocal cues from a user and sending information to a remote server to be analyzed. This is returned as feedback to the speaker via a short message on the user’s screen.
The message format was determined to be the least distracting to users in an exhaustive series of tests, delivering feedback by color, graphs and words. According to Hoque, fellow UR computer science faculty member Randal Nelson preferred the more qualitative, or graphed, feedback when he assisted in testing Rhema. “A lot of others,” Hoque added, “liked the [less qualitative] feedback,” ultimately deciding which delivery method would be most appropriate for Rhema.
Currently, Rhema’s range of analysis is confined to volume and speed. Hoque says that this range could be expanded to include feedback on pauses, rhythm, intonation and pitch, among others. Too many instructions at once can overwhelm a speaker, so any extension of Rhema’s capabilities will have to be mindful of the speaker’s capacity for feedback. “One possibility is for the speaker to customize the feedback,” Hoque said. “[Then] they can target the things they know they want to improve.”
According to Hoque, Rhema could be expanded for use on platforms other than Glass. A smartphone implementation could provide the same “listening” capabilities as the Glass version but provide feedback by vibrating in a user’s pocket. As smart-devices become more ubiquitous, Hoque anticipates seeing Rhema’s ubiquity increase as well.
Rhema was tested in-person by over 30 native English speakers and online by 10 Mechanical Turk workers. Mechanical Turk is an Amazon.com, Inc. service for outsourcing simple tasks. The consensus was that Rhema’s feedback is more helpful than no feedback at all and more helpful than continuous feedback. Even Hoque has been known to use Rhema, citing modest success from the real-time feedback in improving his lectures. “Just the reminder is very useful,” Hoque says, citing Rhema’s usefulness, even for experienced speakers.
While useful feedback was delivered to him during lecture, Hoque says that he does not plan to use Rhema every day. However, he does plan to wear it occasionally just “to show how it works.”
Future plans include using Rhema to help improve those with social difficulties or to train employees in the service industry, where interfacing with customers is important.
Ransom is a member of
the class of 2017.
