My thoughts about how as a VoiceOver user, Face ID although accessible is not very efficient

Posted on September 13, 2018
On January 9, 2007 Steve Jobs introduced the first iPhone, but I wasn’t sucked into his reality distortion field quite yet. My friend Nathan Klapoetke was ecstatic about it though and told me later that day when reinstalling Windows for me that he couldn’t wait to buy one; that was a very long 6 months for Nathan.

I, and much of the blind community were frustrated though, and felt left out. Some of us new that the touch screen, just a piece of glass for us then, would be the future and that soon no companies would make phones with number pads anymore. Then, at WWDC 2009, Steve Jobs introduced iOS 3 and mentioned VoiceOver during the keynote; the blind could use modern phones again. Those 2 years were hard though. Many of us in the blind community had Nokia phones, and some were playing with Windows phones, but they weren’t changing our lives anywhere near as much as the iPhone would later. This is how I feel current things are with the iPhone X models starting last year.

Apple is obsessed with Face ID and is making it their future, but I’m feeling a bit left out again.

Yes Face ID is accessible, I played some with an iPhone X last fall but as more people are recently realizing, accessible does not always mean efficient. I found it quite finicky when setting it up, and because I wear prosthetic eyes, when they’re out, I appear as a different person. There is an article written specifically for a blind person setting up Face ID, but note that they tell you to put your face in the frame, but don’t actually explain how to do it when unable to see the frame. Then there’s the trick of figuring out how far away to hold the phone from your face. I also have no idea how to move my face around in a circle while simultaneously staring at one spot. Ok, I understand the concept, but can’t physically do it. It took me about 15 minutes to get Face ID setup the first time. Another problem is how attention mode is disabled by default when VoiceOver is enabled. I understand that Face ID would work less with it on for blind people who are unable to visually focus, but that’s a potential security hole. A blind person could have their phone next to them on their desk and another person could quietly walk by pick up the phone pan it by the blind person’s face, and they’re in.

Beyond setting it up and all the problems of having eyes that don’t work, there is the inconveniences of work flow. My phone is often on my belt, and most people blind or sighted keep their phones in a pocket. If you’re blind, and have headphones, why would you ever want to  take your phone off your belt, or out of your pocket to use it? Taking your phone out just to authenticate gets annoying real fast, it also may require the person to stop what else they were doing. I’m rarely if ever looking at my phone when I use it, often my face is completely in a different direction.

I could go on a rant, oh wait I already am. I could be cynical, flame Apple, or just give up and switch to Android, and some might; but from my experience where although it took 2 years, Apple did bring VoiceOver to the iPhone in 2009. Here are some thoughts.

Things I could do today to unlock my phone, I have a long complicated password, I really don’t want to enter it every time. I could use the Bluetooth keyboard I already carry with me. I could plug in a series 4 Yubico key when I’m home or not around other people or in a situation where having something rigid plugged into the phone has a low probability of being bumped or damaged. These are only hacks though, I’m really hoping Apple can come up with an awesome solution again.

The iPhone can already unlock the Apple watch, and the watch can unlock my mack; I really hope that my Apple watch could unlock my iPhone too. Just having the phone unlock if my watch is on would definitely not be secure at all, but with the new Apple watch series 4 having some capacitance in the digital crown, having to touch the crown to unlock the phone could be a start. Putting Touch ID into a future model of the watch crown would be awesome.

I already wrote about how there are solutions that let me use my wired headphones even with no headphone jack, I know there are solutions for Touch ID equivalents that don’t include Face ID too. Whether Apple implements any of them is still a question, but I really hope they will realize how visual only options inconvenient a non-trivial segment of their market, and give us an alternative.

Advertisements

How to play one of my more favorite games, Quixo, completely wood and low tech remotely with friends who live farther away

since 2014 one of the highlights of my summers has been a day at the Bristol WI Renaissance fair. Last year I found out they had a  game store , and was introduced to the game Quixo. Quixo is some times explained as Tic Tac Toe on steroids, but that’s superficial at best. The game has 5 rows of 5 cubes, starting out all facing up blank. Players either 2 to a game, or 2 teams pick either x or o and begin. The wooden set of Quixo have the letters engraved into the cubes so it is totally accessible literally out of the box; the travel version, however, uses plastic cubes and on those pieces the letters are not tactile. Players actually move the pieces around and whole rows or columns of pieces in a move, it’s lots of fun.

I’ve enjoyed it and even gave a few to family for Christmas last year. The problem though is Quixo seems to be rarely known about, so most my friends who play it thus far don’t live near enough to play in person very often, so I’ve been thinking about how to play remotely. I thought about how people play chess using algebraic notation, and went from there.

It took me a while, but then I finally realized that if 2 people played a game of Quixo and thought of the board as a1 being in the lower left, e1 lower right, a5 upper-left, and e5 upper-right, and communicated moves as something like a1 to a5; it would work. It would be the same as if player 1 sat at the board and made a move, then got up and player 2 sat in the exact same chair and made a move. This would allow people to play by text or some kind of voice call.

My friend Andrew Hanson was over last weekend and he said when people played remotely they didn’t even have to say a1 is lower-left corner, that no matter which corner a1 was in if the player always referred to that square as a1 it would work, though the boards would look different. . Trying to rotate spacial frames of reference around in my mind caused a meltdown, so I’ll just prefer to always call a1 lower-left and leave it at that.

I still think that both players need to refer to the squares with the same notation in a game, especially when played on the same board, or at best it will soon become very complicated.

At any rate, Quixo can now easily be played remotely. Try it out, it can be lots of fun.

How for me, bone conduction headphones by Aftershokz is one of those technologies that is not just nice to have but a huge game changer for blind people

Posted on July 17 2018
Shortly after Christmas when i. Was 5, my sister Andrea introduced me to headphones with one of those single ear plugs from the 1970s, and showed me how I could plug it into my radio and listen to it. After a few minutes of private listening I couldn’t understand at all why she or anyone else around me couldn’t hear it. That just blew my 5-year-old mind, and it hasn’t been the same since.

As I got older, headphones became more a part of my workflow. I knew there are good speakers out there, I’m just a headphone guy at ear. Part of this came from. Using screen readers on first computers and later phones, besides using headphones for privacy, I’m sure people around me appreciate not being annoyed by it. I even used headphones on the bus, at work, and even when I took classes in college. There was still one area where headphones couldn’t help me though, when traveling alone using the white cane. I began to use GPS apps on my phone, but all the headphones I knew of still blocked some of the sounds from my environment, thus I didn’t feel safe using them when walking, and when traffic was loud I couldn’t hear the GPS info on the phone. Then I learned about bone conduction.

My friend Hai Nguyen Ly told me about Aftershokz and their line of bone conducting headphones, and how they rested on the face using transducers to convey sound through the cheek bone thus leaving the ear completely uncovered and blocking none of a person’s natural hearing. I couldn’t afford them then, so Hai sent me an old pair he was no longer using. Like Andrea introducing me to headphones so long ago, Hai improved my life again.

The first time I used the Aftershokz psychologically I wasn’t quite sure that it really wasn’t blocking my hearing, but it didn’t take long for me to realize that they really weren’t. I could hear traffic just fine, and the GPS info from my phone was always hearable even in the loudest truck or bus roared by. Now 5 years later after buying 2 more pair of Aftershokz headphones, I still use them pretty much every day. I wear them all the time when I’m in public, they work great at meetings and conferences. Even when I’m not needing to hear traffic when cane traveling, they still let me have the ability to use my phone without interrupting anyone around me but still be able to hear what they’re saying. Yes, trying to understand both audio streams might not work as well as I’d like, but sighted people get distracted too.

Some of you reading this might be wondering, ok but why does this matter? In my last post I talked about how for most of us most of the time technology is a nice convenience, but for those with disabilities, technology can be a huge life changer; this is definitely one of them.
Especially for blind users of screen readers. Bone conduction headphones allow them to get the info they need in real time while still having full access to their environment through their primary sense. Bone conduction technology may have been initially invented for the military, but now thankfully it is now also being used to help humans also be more human.

Technology for most is a nice thing to have, but for those with disabilities, how huge of a game changer technology is in improving their lives cannot be exaggerated

Posted on July 6, 2018

As I wrote about in my last post, technology in its most basic definition is an innovation which makes some task easier, and for most of us that is most of the time, the case; but for people with disabilities, it is way more than that.
Technology helps us get places faster and safer. Technology has made communication more possible and convenient today than was ever thought to ever be possible even a short time ago. Technology is nice, cool, fun, entertaining, amazing, and many more words beyond this sentence; but for those with disabilities, technology is way more than any of those accolades, technology is life changing; sometimes in unimaginable ways. I think sometimes even in some ways that can only be realized not in blog posts or videos, but in first hand experiences.

One of my favorite podcasts is the Mac Power Users, and several episodes ago David Sparks, and Katie Floyd were talking about one of my least favorite forms of technology, the pdf file format. If pdfs have done anything good for me though it is that many more people in recent years are aware of and use optical character recognition (OCR)
During the episode David talked about how he’d OCR scanned a bunch of documents and that enabled him to find a phrase he heard and needed to refer to during a court case. OCR is great even if you’re sighted because it enables you to very quickly search documents and even automate tasks like organizing and processing them. It wasn’t that many years ago that OCR was thought of as unnecessary, slow, not worth it, and often avoided; but for blind people optical character recognition is one of the most enabling technologies to come out since the invention of braille itself.
Yes braille is awesome and crucial to the education and development of a blind person’s intellect, but only about 1% of all the books in the world are ever commercially produced in braille. Audio books, and text to speech together with OCR have made many more books available to blind readers, but nothing will replace braille for things like mathematics and program code. I know some blind geeks will flame me for this  and say they don’t need braille at all and they write program code all day; but I know from years of personal experience that no matter how good they are at hearing text to speech spell out arcane function name spellings and all types of punctuation, that using an exorbitantly expensive refreshable  braille display would significantly increase their efficiency; a whole other topic for another post another day.

Beyond all of that, OCR means that if there isn’t any e-book available for a title, a blind person can probably buy a print copy, and after scanning it in, have a copy they can read in braille or text to speech relatively quickly.

The smartphone is also something that along with the internet and OCR is a close second to braille in how huge of a life changer inventions can be for the improvements of the lives of blind people.
When the iPhone was first announced in 2007 I was seriously frustrated thinking that a touch screen would never be accessible, and also knowing that touch screens were the future. No one outside of Apple saw VoiceOver coming to the iPhone in June 2009 but that along with Talkback on Android some time later may be the largest improvement to the blind world by technology in the last 20 years.
Yes I can call people on my iPhone, or text them; I can also play games or listen to music, but that is only the beginning. There are GPS apps that not only tell me how to get to places, but also tell me landmarks and street names as I travel, almost like a sighted person looking around and telling me in my ear what they see passing by; , a true form of augmented reality and it’s not even visual. Some blind people use their phones to read small documents or food labels on the fly, and if it can’t read the text maybe a barcode scan instead.
There are still talking devices made specifically for the blind, like for example, scales for weighing, thermometers for body temperature, cooking, or outside. There are talking glucose monitors and other things not mentioned here. These devices are often significantly more expensive than their mainstream counterparts, and before smartphones they were the only options blind people had. Some still feel more comfortable with them than trying to pair a smartphone to a more modern device, but that’s just another way technology is improving our world. A talking smartphone, plus an accessible app, plus a mainstream bluetooth device; means an often very accessible and usable device that has more features than the blind-specific devices also out there, and they’re also devices many people have, sighted and blind. If I buy a bluetooth scale and don’t understand how to add my weight to the health app, I can ask anyone who has the scale, not only the few blind people I might know who bought the blind-specific one.

Yes there are other disabilities that being blind, like people in wheelchairs, or people with cognitive disabilities, and those are just as important to realize, and they have been just as impacted by technology, just take a moment to remember what Stephen Hawking could do. Helen Keller if still alive today would probably be amazed at what more she could have done.

Please think about instead of just how cool your app is or what fun it can be, but more of how can You improve someone’s life even or more especially if they perceive information and interact differently with interfaces than You do. If you are a  developer or designer reading this post, please take a moment and step outside the box that is your subset of reality, and not only imagine how you could make the world, including the lives of those who think differently better, , but then actually do it.

Some reflections on how technology has progressed as I flew home from North Carolina to Wisconsin last Friday

Posted on June 25 2018

Just a reflection on how technology has advanced over a time period I can remember.I can remember in high school wishing I could type some of my homework on my  30 minute  bus rides each way, and although until 11th grade I had a very nice electronic typewriter, it wasn’t portable.   Then when I went to California in 11th grade there was no way to even think about taking my Apple IIe computer along.Now, as I type this post I’m at  35000 feet traveling between  490-520 miles per hour currently over Kentucky, (I love running GPS on my phone)   flying home to Madison WI from Charlotte NC. Yes I flew to California, my first time in an airplane,  back in high school, but back then when on a trip I pretty much lost contact with family and friends “gone for a week talk to you when I get back” I’d have to say. Thankfully I can now say that time is over.The basic definition of technology is to make doing some task easier, but I’d say think deeper, think more  human. Technology are innovations to make our lives better beyond just doing tasks; technology when used properly can let us be more human. In this case technology let me communicate with family and friends while I was working in Dayton Ohio this week. It also lets me put down thoughts on an airplane. Ye we still have pen and paper, but my handwriting is so bad I can’t even read it; oh wait, I can’t read anyone’s handwriting   for other reasons, some people can actually read mine, although they cringe and complain about it.I’m not saying technology is amaZing, because that implies it can’t be understood, but I do think we shouldn’t take technology as much for granted, yes that includes me too some times. Beyond using it, and enjoying its benefits, I think more regularly appreciating how it has and will continue to transform who we are and what we can do. RecogniZing this mind set more may also help us shape how and what we do with technology available to us. If we consider our human side more while geeking out with our technical side, just imagine how you, I, or together we can make the world better. No this post won’t end any wars, but maybe thinking more like this we could avoid a few more  fights, or discuss and debate more  and argue less. Maybe with all this technology at my fingertips I could understand someone different than me a little more.; and don’t forget friendships, try to reconnect a few of those that have faded too.  GPS update, the plane is  now over the south western corner of OHio.No I can’t post this to the blog from this airplane, no wifi on this flight,  and some might find that frustrating; despite the fact that if they could tell the pilgrims from 1620, or even Immigrants from only 150 years ago that you could get from London to New York in about 6 hours they would have never believed it.  but I’m totally happy with how far things have progressed as much as they have, posting it when I get home, or maybe not even until tomorrow is just fine by me.

How I discovered that the audio in Live Photos can help blind people identify and organize them

Posted on May 17, 2018

When Apple announced live photos along with their iPhone 6s in 2015 almost everyone I know or read thought they were nothing more than a stupid gimmick, and promptly turned them off. It took 2 plus years before advantages of Live Photos started to show up as mentioned by Allison Sheridan recently on her blog, as well as a post last year on How to Geek; but if you’re blind, you’ve had something cool since day one.

When everyone else thought Live Photos were stupid and just a silly way to waste space, I immediately realized that they brought accessibility to photos in an interesting way. With 3 seconds of audio, someone could easily provide an audio label for those pictures. If a blind person went on vacation and wanted a few pictures for their sighted family and friends, they and/or someone with them could add audio like “Uncle John and Grandma on the beach” or “Julie standing near the Eiffel Tower at night”. The possibilities are endless. Then, when a blind person scrolls through their library and opens one of their Live photos they hear the audio and can rename them for faster browsing in the future.
A sighted person could even take live photos on their phone adding audio tags, and then send them to a blind friend or family member.

I could even see a future version of iOS offer to transcribe the audio in Live photos and use that text to rename the file, that would just be cool and make my love of efficiency side all warm and happy.

Just another reminder that when a feature seems silly or useless, maybe it helps someone else in huge unimagined ways.

Instructions on how to get your Aftershokz Trex headset to work with Bluetooth Multipoint

Posted on November 9, 2017

About a year ago, I wrote how useful the Apple watch was for me, but one problem still was unless I used my mess of cables hack I couldn’t get VoiceOver from both my iPhone and watch simultaneously.
The Aftershokz headphone company released their Trekz-Titanium model in early 2016 and many people in the blind community were excited because they claimed the titaniums had Bluetooth Multipoint. They did, but after 1-2 hours of trying to get it working I gave up in frustration.

Then the Apple AirPods came out and I was hoping they could do the trick, but no, the user has to switch between devices; they won’t do it automatically.
Then my friend Hai Nguyen Ly who introduced me to bone conduction headphones 4+ years ago said in passing last month, that he’d gotten them working in Multipoint with both his iPhone and Apple watch, so I decided to reexamine the challenge, and this time was successful within about 30 minutes.

Here are the steps to do it, this works for both the Aftershokz Trex Titanium, and Trekz-air models; hopefully they make sense.

1. first you have to reset the headset, Turn off the headset before beginning to reset it.

2. Enter pairing mode by turning on the headset and holding the volume up button for 5-7 seconds.
You will hear the Audrey Says™ voice say “Welcome to Trex Titanium, and then shortly after, “pairing”. The LED will flash red and blue.
Audrey will say Trex-air if you have that model instead.

3. Then press and hold all 3 buttons on the headset for 3-5 seconds. You will hear a low pitched double beep, or feel vibrations.

4. Turn the headset back off.

5. Enter pairing mode again by pressing and holding the volume up-power on button. Audrey will first say “Welcome to Trex Titanium” and then “pairing.” The LED will flash red and blue.

6. Continue to press-hold the volume up button while then simultaneously also pressing and holding the triangular multi-function button on the left. after about 3 more seconds Audrey will say Multipoint enabled.

7. In your first device’s Bluetooth settings, select your Trekz model. Audrey will say “Connected.”

8. Turn the headset off.

9. Reenter pairing mode again by pressing and holding the volume up-power on button. Audrey will first say “Welcome to Trex Titanium” and then “pairing.” The LED will flash red and blue.
10. In your second device’s Bluetooth settings, select your Trekz model. Audrey will say “Connected” or “Device 2 Connected.”
11. Turn the headset off.

The next time you turn your Trex headset on it will connect to both devices. It works pretty well, though here are some things I’ve noticed.

If I move out of range of one of the connected devices and then move back into range, the device doesn’t always reconnect. Turning the headset off and back on reconnects both again.

I said Multipoint lets you connect 2 devices simultaneously but that doesn’t mean you can hear audio from both simultaneously. only one at a time. This means if I’m playing a podcast on my iPhone, I won’t hear anything from my Apple watch; that has already bit me a few times. if I pause the podcast on the phone, audio from the watch will start playing in about 2 seconds.
Beyond that, using Multipoint is still quite useful. I can use either device in a meeting, concert, or at church. I can also use either device while traveling in loud situations like around heavy traffic. I can also use the watch in situations where the watch’s built-in speaker would be too quiet to hear. Even with the limitations I’ve mentioned , I think you’ll still find using your Aftershokz with Multipoint a productivity boost.
Oh, my mess of cables hack is still useful if I want to hear more than 2 devices; and with that solution, the audio really is simultaneous.