Tuesday, August 25, 2015

Blindshell: A Simple Phone Interface For Blind Users

a blind person using the blindshell interface on his phone


VoiceOver on iOS and TalkBack on Android have definitely made it possible for blind users to use smartphones on a daily basis. As the user navigates around on the interface, the phone talks back to the user, letting them know the apps they have access to, and also helps them with menu options and actions that can be performed on the apps. However, for some users, utilities like VoiceOver or TalkBack may be a little overwhelming - their needs may be as simple as calling, texting or just reading a book. They may desire to have just the most basic actions at their fingertips, and having to train themselves to use native phone accessibility utilities like VoiceOver may be a little too much to handle.

photo of blindshell interfaceThat’s where Blindshell can be of help. Essentially an app, Blindshell replaces the regular interface on an Android phone with a minimal interface that brings basic and most essential functions like calling, texting, contacts, alarm, notes, calendar, color recognition, and money recognition to the user. There is also a voice recorder available, and any text uploaded to the phone is read back to the user. Thousands of books can also be downloaded and read back to the user by the Blindshell via Bookshare.

The interface of the phone is very minimal and simple. The phone works with six gestures which can be performed anywhere on the screen. The six gestures are used to navigate through menu options, confirm a choice, start/stop the screen reader, go back to the previous screen etc.

This video shows what using Blindshell is like.


Many blind users in the Czech Republic, where Blindshell is based, have found success with the app. Blind users, who damaged or lost their phones with physical buttons, had to get smartphones because of older phones' apparent scarcity, and with minimal training, got used to the Blindshell interface. Some users also use the voice recorder in the phone for recreational purposes, like singing and recording their music on the phone!

Blindshell's long term goal is to ultimately expand into India, China , and Brazil. Currently, the company focusses on finding professional partners all over the globe but would like to help thousands of blind users through their app.

A trial for Blindshell is available on the Google Play store, and the full unlocked version is priced at $40.

Website: http://blindshell.com

Sunday, August 16, 2015

Autcraft: Minecraft For Autistic Children



In the last few years, Minecraft has taken the world by storm.  We all perhaps know that it is a video game, but we don't know much beyond that. Almost all kids, on the other hand, not only know what it is (a video game!), but also know how to maneuver that game very well, and pick up all sorts of tips & tricks and shortcuts pretty quickly, which they use to build their world and fight monsters.

But what exactly is Minecraft, and why is it such a popular game?

First of all, unlike other video games, Minecraft does not have set objectives. It relies heavily on exploring surroundings around you, creative building, resource gathering, and survival. As a player, you look at resources around you (trees for wood, for example), and using those resources, you build a world. You can either build your world by yourself or collaborate with others. The creative freedom that this game brings is just amazing.

However, every rose has its thorn, and Minecraft is no exception to this idiom. Usually kids progress from playing solo to online servers, where they can play with thousands of other players from around the world, and that's where the trouble starts. Online gaming is notorious for bullying - rogue members are known to tease and cause embarrassment to others, disrupting  what's otherwise a fun environment for many. This discourages many from returning to the online community of gaming. This is not good for kids with autism, who are especially vulnerable to bullying.  

To combat the bullying, provide a safe haven for kids on the spectrum, and to give them an environment where they can be creative without being judged and harassed by peers, Stuart Duncan has started a new server called "Autcraft" for kids with Autism. In order to join,  a member has to register on the Autcraft website, and await a response. Once registered, kids can play with others without the fear of being judged and bullied. Once they get comfortable in the environment, they share their interests with others and start making new friends, and they learn how to interact with others better. 

Autcraft has been very popular with kids with Autism who have a naturally "engineering" brain. Their knack for being detail oriented and ability to be analytical and logical really helps them flourish in Autcraft. On top of that, it gives them a fantasy world where they can let their imagination run wild, especially when they are playing with so many other similar people from around the world. After playing Autcraft, kids with autism have shown tremendous progress in reading and writing skills. They also have more self confidence and make newer friends easily.

Through Autcraft, kids develop their Minecraft skills so well that other kids come up to them to talk about Minecraft. Contributing something valuable to a social conversation boosts their self confidence. Autcraft has given kids on the spectrum new ways to make friends and be social. 

Watch this video to see how Stuart explains what Autcraft is, how it works, and the benefits it brings to kids on the spectrum.


Stuart Duncan, who has an autistic son, is not just the creator and moderator of Autcraft. He also provides ad hoc counselling to players, and maintains a blog separately where he writes about Autism from a parent's perspective.

Autcraft website
Stuart Duncan's Blog

Source: news.com.au
Photo source: Savvy Advocate Mom




Sunday, August 9, 2015

Simple Audiobook Player For Seniors And Blind People


Books are an integral part of our lives! Doesn't matter if we are voracious or occasional readers - we always appreciate the entertainment and knowledge books impart. However, in case of blind people or sighted people losing their partial or full vision, physical books can seldom be an option. That's where audiobooks come into the picture, but they come with their own challenges. 

This post has been written by Marcin Simonides who has created a very simple, minimalist interface for playing audiobooks for blind and/or elderly people.

When eyesight deteriorates, either due to illness or advanced age, many people turn to audiobooks as an alternative to reading. With today's technology there is an abundance of services and devices one can use.

For many seniors, however, mastering the technology can be an additional obstacle. Most audio players are designed either to be small, which requires that buttons are tiny and close together, or big with touch screens which is susceptible to accidental touch. Either way it is easy to inadvertently press the wrong button, especially if fingers are no longer as flexible and precise as they used to be. Add to this features like shuffle or repeat that sometimes get enabled by mistake and a simple task of playing your favourite book feels like entering a minefield.

I have not felt this myself (yet). But I have observed my grandmother struggle with a number of audio players.

Over the past few years we have tested many devices. The best one was a second-hand CD player with MP3 support. Sadly, it did not store the last played position and playback started from the beginning after it has been turned off. And it still had too many buttons.

An ideal audiobook player for my grandmother would have as little features as necessary, namely play and pause, and it would be easy to operate with imprecise gestures. How hard can it be? :) I've decided to try.

I'm not good at building stuff but I'm a software developer so I have decided to take an off-the-shelf device, an Android tablet, and achieve my goal with a custom app.

I came up with something like this:


The app plays audio files but also does three things that are geared towards visually imparied and elderly:

- it reads the book titles when the screen is enabled and when browsing,
- it has large START and STOP buttons in contrast colors,
- stops playback when placed with the screen downward on a level surface.

And, last but not least, the app can be installed in such a way that the user cannot leave it. This hides all the complexity of the underlying system and other applications - the device serves one purpose and one purpose only.

For the time being it is still an experiment although my grandmother is already using it.
The application is available for download on my website (it will also be available in Google Play within a few weeks) so you can test it yourself and give me your feedback.


Similar work

Obviously I'm not the first one to recognize the problem. During my research I have found two other project with similar goals but different solutions. Both are custom made devices based on the Raspberry PI platform.

https://gist.github.com/wkjagt/814b3f62ea03c7b1a765


Marcin Simondes is a software developer and lives in Wrocław, Poland.




Tuesday, July 28, 2015

[VIDEO] Assistive Technology For Quadriplegic Student: Conversation With Francis Feoko

photo of francis feoko talking to venkat rao on skype

I recently got a chance to talk with Francis Feoko, a C5 quadriplegic, who is a sophomore in the Bay Area in California. Francis wanted to share with Assistive Technology Blog's readers how, being a college student, he takes care of his home and school work and how technology has made him more confident and self sufficient in a classroom setting. He wants other students with disabilities to know that there are options available out there that can make working on classwork and homework  a breeze. Technology has boosted Francis' self esteem and motivated him to do things on his own.

What's great about Francis' setup is not just his device of choice, which is iPad 2, but also his mindset and willingness to explore assistive technology devices for himself, which is really important. There are so many devices available now, but which one suits us best in different circumstances? Francis also reminds us that "what works best for me may not work for others". 

Francis uses different apps for different classes - For his math class, he uses Notes Plus, an app that gives him lots of flexibility in terms of drawing graphs and using various shapes and symbols for his math homework. Since he is C5 quadriplegic, he cannot use his fingers, so he uses his pinky finger's knuckle to write his equations. (he has used a stylus in the past but has gotten used to his pinky for writing now) Once done, he likes to email his homework to himself so he could review, reorganize and make changes as needed, before submitting it to his teacher. Francis used to have a notetaker who would take notes for him in class, but now he just uses the iPad to take notes himself.

Notability is another app he uses for "visual learning". Notability lets Francis copy and paste images from the internet, so if he is learning something and wants to use pictures to help him understand the concepts better, he uses Notability that lets him add and organize pictures into his notes to make them more visual.

Since he is a computer science student, Francis has to write code for his C++ or HTML homework. In order to do that, he remote logs in to his laptop computer through his iPad using an app called Splashtop. Sometimes it is not easy for him to take his laptop to school, but thanks to Splashtop, he can access his laptop even when he doesn't physically have it with him. 

Francis also uses Read&Write Gold to have his notes read back to him, although later in the conversation he suggests that many iOS apps now have the ability to read notes back to the user. One advice he has for other students with disabilities is to explore assistive technology options before signing up for a class. Some classes may require special accessibility options that may not be available at school.

Francis also feels like more people with disabilities should be employed in the technology industry since they are daily users and understand what exactly needs to be developed to take accessibility to the next level. He works part time at his school's Disability Resource Center where he works with their Assistive Technology Specialist who is on top of things, and knows about all the newer apps that are being released. Francis is also  a client of Department of Rehabilitation and encourages others to be one too. Through the department, he gets in touch with other experts who help him with other accessibility needs. For example, the experts at his local department of rehabilitation office made a front mount for his backpack so he has easy  access to his things all the time.

Here is a video of me chatting with Francis. What was supposed to be an interview turned out to be a casual and candid conversation where Francis shares his experiences and insights about his disability and how technology has been helping him be more and more self sufficient. During this conversation, he also mentions his love for the Assistive Technology Blog and how he has been sharing posts from this blog with his friends and teachers in his native country Fiji!

Thanks for a great conversation, Francis!



Sunday, July 19, 2015

Self-Driving Cars Open Up New Opportunities for the Disabled

google self driving car prototype


Transportation is one of the main obstacles that those with disabilities face. Though public transport tries to be accommodating, there are still several obstacles that the disabled have to navigate in order to get to a specific location -- walking from train or bus stops, shelling out money for cab rides, relying on friends and family for transportation.  It can be a full-time job. 

But there is good news for those who need to get where they’re going without having to dodge traffic or tip a driver.  It’s the self-driving car, and it’s coming soon. 

Google has recently launched its Self-Driving Car Project, and is currently testing its cars on open roads and public streets in California.  It has even managed to traverse Lombard Street in San Francisco, famous for its sharp hairpin turns.  Google has two different models -- an older generation modified Lexus with an onboard computer and sensors, as well as a second generation driverless pod, without a steering wheel or pedals. 

Both models have chauffeured a variety of disabled passengers, including visually impaired riders such as Steve Mahan, executive director of the Santa Clara Valley Blind Center. “I miss driving,” he said. “My experience with Google has been terrific, and I want it to happen. Everyone in the blind community wants it to happen.” 

How does it work? Google designed their vehicle with “push-button” technology, and strives to benefit the disabled population by making it “easier, safer and more enjoyable to get around.” The car is designed to replicate human response. A rooftop array of sensors detect its location, surroundings, stationary and moving obstacles such as pedestrians, bicycles, and other vehicles. They navigate and obey traffic rules through sophisticated algorithms and controllers that eliminate any need for driver input through steering wheels or brakes.

google self driving car prototype on the roadThis innovation in technology is not limited to the Google car. Companies including Audi, BMW, Ford and Mercedes are all currently working to develop their own driverless, or driver-assisted vehicles. Uber has even expressed an interest in releasing a fleet of driverless vehicles. In addition to the benefits to the disabled population, there are environmental benefits, as well - according to Ohio Gas and Berkley Labs, autonomous cars could cut fuel usage by up to 90%. 

According to the New York Times, since the second-generation Google driverless vehicle is designed to run without driver intervention, it would be appropriate for people physically unable to operate a car. Google said that “the potential of a self-driver to help those with disabilities could be realized only if the human operator were taken out of the equation."

Of course, there are concerns as to how this new technology will impact the marketplace. 

Insurance companies will have to modify their liability policies as self-driving cars increasingly eliminate driver error as the cause of vehicular fatalities. At the same time, the automotive business model will change from individual ownership to usage-as-needed. Also, the disabled who already operate specially modified vehicles worry that self-driving cars might threaten their funding.  The costs of the self-driving car are estimated to add $7,000 to $10,000 to the normal sticker price of a new vehicle.  It is still unclear whether or not government assistance will incorporate the self-driving car into its current funding programs.

There is still much testing to be done.  For instance, the vehicles are not yet able to navigate unforeseen obstacles such as construction zones, unexpected weather or disabled traffic lights. 

But, the fact remains that these efforts to bring the self-driving, or driver-assisted cars to the public will greatly benefit those with limited ability.  Whether it’s shopping, doctor’s appointments, getting to work or visiting family, Google is committed to making life easier, more convenient, and safer for those with disabilities.

Want to see what it's like to ride in a self driving car?



This blog post was written by Emma Bailey. Emma is  a blogger in the greater Chicago area with a keen interest in technology, astronomy, and anthropology. Emma's regular contributions to the Assistive Technology Blog are appreciated to a wonderful degree!


Friday, July 17, 2015

Digital Animation To Help Kids With Autism Develop Social Skills

a child working with invirtua's digital avatart


Gary Jesch, owner of Invirtua and a digital puppeteer who has been in the live animation industry for 20 years, was inspired to see what he could do for those on the autism spectrum.

Eye contact, social skills & social interaction, and difficulties with attention and concentration are some of the many challenges children with autism face on a regular basis. Going to therapy sessions to develop or enhance these skills is not unusual for these children. However, sessions with an adult therapist can be dull, mundane and not fun. Therapists most likely know that too, but how would they make their therapy sessions more interesting for the children and make sure they grasp everything they are being taught? Another challenge for the therapists is that there are a number of established and emerging interventions, and they would want to stick to them without bringing in another medium that would water down or compromise the details of interventions.

This is where Invirtua 3D Digital Puppeteer's digital avatars come into the picture. Instead of doing a direct one on one session with the child, a therapist can introduce an animated avatar that helps energize the session. The therapist is still involved. They are still driving the session - they are the ones who control the avatars and use them as a medium to interact with the children. And for the children, the session get more enjoyable and fun!

This is how the setup works - the therapists client room will have a  monitor, webcam and speakers
photo of gary jesch, owner of invirtua at teh autism society national conference in denver
that are connected to a computer in a different corner of the room or in a different room altogether. This computer run the software that has all the animated avatars. The avatar's facial expressions, emotions and body movement are controlled by a tablet and joystick connected to the computer. The joystick controls the eyes, and tablet controls the position of the avatar on the screen and various emotions that the therapist wants the avatar to display. The trackpad also has vertical sliders for expressions - the higher you go on the slider, the greater that expression is displayed by the avatar. (think of smiling as an expression) While controlling the avatar, the therapist also speaks into a microphone to be the voice of the avatar. Essentially, everything being done on the computer is being reflected on the monitor in the client room - the child sees and hears the avatar, observes all the positive behavior (body language, facial expressions) that the avatar is demonstrating, and learns from it without having to deal with the stress of being in a therapy session with another adult. Therapy now becomes fun and something to look forward to! Through these avatars, therapists can also be comical and goofy to be entertaining to the child! It is quite easy for the child as a client of the therapist to control the avatar themselves and they find that idea very motivating, as Invirtua discovered in their pilot program. Children love the idea of being in control and being the “avateer".

One thing nice about these avatars is that they intersect and integrate well with therapeutic approach. They don't add anything overbearing to the session nor do they try to change any of the therapy processes - all they do is bring in a mechanism that helps children grasp knowledge without any unnecessary stress.

The complete system consisting of a high end computer, software, tablet and joystick can be purchased for $2,790 from Invirtua's website. To put things in perspective, a similar setup would have cost around $30,000 two years ago. The low cost of this system also makes it possible for parents to buy and install it at home, and start using it with their child while they are waiting to be aligned with a therapist. Sadly, it can take anywhere from 6 - 12 months before a child can meet their therapist.

Invirtua would keep developing and enhancing this technology. In the future, we may see it on Xbox, virtual reality devices like Oculus and even augmented reality (AR) devices like Microsoft's Hololens.


Invirtua 3D Digital Puppeteer system is available to purchase through Invirtua's website. It takes 2-3 weeks to deliver and is shipped worldwide.

Gary introduced this system at the Autism Society National Conference in Denver last week. In the video below, he talks in detail about his inspiration behind these avatars, how this system works, shows us why and how a therapist would use it, and talks about pricing and future plans.


Adam Mercier, a visitor at the conference who happened to be at Gary's booth,  tried it out while I was waiting to talk to Gary. Adam has Asperger's, and after trying out the avatars briefly, was quite excited and wished he had access to such a system while growing up in the '90s.





Sunday, June 28, 2015

This Is How You Do Makeup When You Are Blind

photo of lucy edwards

Lucy Edwards is a 19 year old who lost vision in her right eye when she was 11 and left eye two years ago because of a rare genetic disorder called Incontinentia Pigmenti. Lucy has a great fashion sense and is very much into fashion, make up and being a social butterfly. Not being able to do her makeup anymore after her vision loss was somewhat of a setback. However, to stay positive and  to continue to do what she loved, she learned to do make up without a mirror!

She didn't stop there though - along with her boyfriend, she recently started a youtube channel called yesterday's wishes where she shows both blind and sighted people how to apply make up in a very detailed manner, gives her opinions about  make up products, and shares several tips and tricks that are especially helpful for blind people.





But why do blind people care about their looks?

This is something she addresses very confidently - it's about looking good and feeling good about yourself. It feels nice when someone gives you a compliment - "you look nice", "you look lovely". Being sighted or not doesn't really matter. It doesn't matter either whether you can see yourself in all the makeup. What matters is the feel good factor that wearing makeup, being fashionable, and receiving compliments from people, and being confident in social surroundings brings.

Besides giving makeup tutorials, Lucy also answers various questions related to blind culture and dismisses several myths and misconceptions about blind people in her Q&A sessions.

"Are most blind people totally blind?"
"All blind people read braille"
"Do all blind people go to special schools"
"Blind people have no social life"

These are some of the various questions she answers providing her own perspective in a very nice and humble way.


She also has a blog where she posts make up tips, product reviews and transcripts for her videos.

Her videos are very well made and edited. If you know anyone with vision impairment wanting makeup skills, send them her way!

Source: Buzzfeed
Youtube channel: Yesterday's Wishes
Blog: https://wishesforyesterday.wordpress.com/



Thursday, June 18, 2015

Gesture Vocalizer: Device That Converts Sign Language To Speech & Text


photo of students who developed digital vocalizer

Along the lines of MotionSavvy and UNI, a group of engineering students from India have created a prototype that translates sign language into audio and text so that speech impaired people could communicate with those who don't know sign language.

Unlike MotionSavvy, which is essentially just a tablet, this prototype consists of a glove with sensors and accelerometer attached to a "control section", and lcd screen with audio. The sensors on the glove detect the way fingers are being  bent and the accelerometer detects the tilting and movement of hand. Once those gestures are captured by the glove, they are sent to the control section which has a database of sign language gestures. The captured movements are matched against the stored database gestures and the pre-recorded audio  equivalent is spoken and text for those signs is displayed on the screen.

There is no news on when this device will hit the market and how much it will cost, but the good thing about this is that students everywhere are taking initiative to change the way we interact with people with impairments and disabilities! It is nice to see a shift in the way we are thinking about bridging all sorts of communication gaps and overcoming limitations!

Source: Inquisitr

[Thanks for sharing, Mayank!]

Sunday, June 14, 2015

Sesame Enable: Phone That Tracks Head Movement

image explaining how sesame phone tracks head movement and enables users to interact with their phone

 There are lots of people with paralysis, spinal cord injuries, cerebral palsy, and other disabilities that make them have limited or no movement in their hands. What that means is that communicating using a phone for them can be challenging, and in quite many cases, not possible, since a touch screen on a smart phone requires physical effort in terms of screen swiping, and clicking buttons and icons.

Sesame Enable recently launched a new technology keeping such people in mind. Sesame Enable's technology, built using the Google Nexus 5 (Android) phone, enables the user to use just their head to interact with the phone. The head tracking interface is turned on when the words "Open Sesame" are said out loud, and the front facing camera enables an on screen cursor, which starts following the user's head movement. Even the slightest of head movements is tracked with great accuracy to open apps, read emails, browse the internet, play games, etc. Different actions like click and swipe can be performed by hovering over an icon, which brings up another dialog with those options. The sensitivity of the cursor movement can also be adjusted in the settings area.
Watch the video below to see how well Sesame Phones work. They have a ton of tutorial videos on their website as well.

This technology has been designed keeping in mind people with ALS, Cerebral Palsy, Muscular Dystrophy, Multiple Sclerosis, Spinal Cord Injuries and other similar disabilities.

The Sesame Phone can be ordered from their Indiegogo page for $700. The phone is shipped worldwide and can take from 1-2 weeks to 2 months depending on where you live.

Website: sesame-enable.com

Source: CBS News

Tuesday, June 2, 2015

Lechal Footwear: Shoes And Insoles For Blind Walkers

photo of red lechal shoe


If you walk a lot to get from one place to another, you probably appreciate a good pair of shoes that make your walk comfortable. What if that same shoe could tell you where to go?

lechal phone app showing steps taken, distance walked, calories burned etc.
Lechal shoes and insoles, that connect to a phone app via bluetooth do exactly that. All one has to do is enter their destination in the phone app and get going. While walking, the shoes slightly vibrate whenever the person is approaching a turn. The right shoe would vibrate for a right turn, and the left shoe for a left turn. There is no need to listen to the smartphone for directions. It sits in the pocket while the person walks conveniently to their destination!

Lechal was specially designed keeping blind and visually impaired people in mind. With so many other features added to it, it's become a shoe with sighted people can wear.

Lechal not only provides directions via haptic feedback but also counts steps taken, total distance walked, calories burnt etc. So a blind person, who wants to track all those metrics, doesn't really need another smart device or app - Lechal will do it for them!

Here are a couple of videos that show how Lechal footwear works.




Lechal can be preordered on the company's website. There is no information on pricing yet but other sources on the internet suggest that the shoes will be priced close to $100. The insoles would most likely cost less than that.

Lechal website

[Thanks for sharing, Abhinav and Swetha!]