Accessibility of FTelnet and canvas

[nromey]

Hi Rick,

I am blind and many of my users would need to use FTelnet in a web browser either on a phone or a browser with a screen reader. Canvas tends to not be accessible, when I use JAWS or NVDA, I get a message which says I need to use a browser that supports canvas. In other words, it's telling us that you use Canvas. I think that output can happen if we were to output text to aria or an aria live region, but I'm not sure where I'd do this i.e. take the content out of canvas and place it in an aria region. You could default to the normal mode and, since you're using javascript, I can help you get code to detect a screen reader and in an application window and then load the accessible page. I don't mind helping to make changes, but I'm unsure how the code is formed. I'm also not as familiar with js as I should be. Can you assist/can we have a look at making this a reality? HTML 5 is great for accessibility, but canvas is about the only thing in HTML 5 that totally breaks things / makes it so it's not renderable.

[rickparrish]

Extracting the text from the canvas can be done by hooking some of the Crt functions. For example I've hooked into the ClrScr and Write methods, and am outputting the printable characters to a textarea on this new Demo page:

https://www.ftelnet.ca/demos/accessible-demo/

Does that help at all? I tried installing NVDA since it's free, and it does read out the contents of the textarea when I move my mouse around it. But then I find I'm unable to type in fTelnet, and I'm guessing that's because NVDA doesn't see any elements focused so it thinks I'm trying to hit shortcut keys, and not type text. So I wasn't able to test beyond my login screen.

[nromey]

Hey Rick. This could work, but let’s see if we can make it better. You’ve got a page outputting to a text box. To be able to type into it, you can press capslock plus space, and then you’re in what’s called application mode. If we had an accessible page, we could set this automatically. Once in application mode we can send all the content of the text to the screen reader using aria—I need to do some research to see how to actually tell you how to do this. Sites like Gmail, google docs, etc. send text to screen readres this way. I look at the workflow as follows: 1. User clicks connect. On my BBS, I could default to the accessible version or I could use Javascript to detect a screen reader 2. The connect button fires off your handy little page—I’d leave the text box in case folks would want to navigate it for scrollback. Once you hit connect, the page would turn to application mode automatically—this is also done with ari) and an aria live region could be set up with the bounds of the box. I think you could set up the box and also send the crt output to the live arix region as well because then, you wouldn’t get screen reader speaking, for instance whenever the edit box is updated/ned text comes in. 3. And, that’s all you’d have to do. With JAWS, JVDA, or even Voiceover on a pone, you’d be able to hear what’s coming in, I hink. Like I said, let me do some research on what aria we’d need to do. Assuming your box is easily findable in the source of the page, I can help you implement an even better way. Remember, if you want to type stuff using NVDA, go out of browse mode by pressing the NVDA kwy which is usually capslock+space. Now all text is sent out like NVDA wasn’t running. I’ll get back to you when I figure things out. Thanks so much for helping with this, I didn’t want to invent the wheel here, and so I thought I’d ask first. This is a marvelous start. Noel From: rickparrish ***@***.***> Sent: Monday, October 2, 2023 11:54 AM To: rickparrish/fTelnet ***@***.***> Cc: nromey ***@***.***>; Author ***@***.***> Subject: Re: [rickparrish/fTelnet] Accessibility of FTelnet and canvas (Issue #22) Extracting the text from the canvas can be done by hooking some of the Crt functions. For example I've hooked into the ClrScr and Write methods, and am outputting the printable characters to a textarea on this new Demo page: https://www.ftelnet.ca/demos/accessible-demo/ Does that help at all? I tried installing NVDA since it's free, and it does read out the contents of the textarea when I move my mouse around it. But then I find I'm unable to type in fTelnet, and I'm guessing that's because NVDA doesn't see any elements focused so it thinks I'm trying to hit shortcut keys, and not type text. So I wasn't able to test beyond my login screen. — Reply to this email directly, view it on GitHub <#22 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFB7LSKQQSDXJ6EDCZAVFOTX5LWRDAVCNFSM6AAAAAA5L2SBNWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONBTGM4TCMJYGM> . You are receiving this because you authored the thread. <https://github.com/notifications/beacon/AFB7LSPF425JGK6HST7IKGTX5LWRDA5CNFSM6AAAAAA5L2SBNWWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTTH5IE46.gif> Message ID: ***@***.*** ***@***.***> >

[nromey]

Hey Rick. Check this page out. I thin kit’s what we need. I’m still looking at what aria attribute for the page we’d set to automatically go into application of non-browse mode. Get back to you on that, but this explains the aria live attribute really well. From: rickparrish ***@***.***> Sent: Monday, October 2, 2023 11:54 AM To: rickparrish/fTelnet ***@***.***> Cc: nromey ***@***.***>; Author ***@***.***> Subject: Re: [rickparrish/fTelnet] Accessibility of FTelnet and canvas (Issue #22) Extracting the text from the canvas can be done by hooking some of the Crt functions. For example I've hooked into the ClrScr and Write methods, and am outputting the printable characters to a textarea on this new Demo page: https://www.ftelnet.ca/demos/accessible-demo/ Does that help at all? I tried installing NVDA since it's free, and it does read out the contents of the textarea when I move my mouse around it. But then I find I'm unable to type in fTelnet, and I'm guessing that's because NVDA doesn't see any elements focused so it thinks I'm trying to hit shortcut keys, and not type text. So I wasn't able to test beyond my login screen. — Reply to this email directly, view it on GitHub <#22 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFB7LSKQQSDXJ6EDCZAVFOTX5LWRDAVCNFSM6AAAAAA5L2SBNWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONBTGM4TCMJYGM> . You are receiving this because you authored the thread. <https://github.com/notifications/beacon/AFB7LSPF425JGK6HST7IKGTX5LWRDA5CNFSM6AAAAAA5L2SBNWWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTTH5IE46.gif> Message ID: ***@***.*** ***@***.***> >

[rickparrish]

Your latest message didn't contain a link. But I did a bit of googling, and swapped out the textarea element for a div element with aria-live="assertive" set, and now NVDA reads out the incoming text.

I also hooked the Connect function to try setting role="application" on the body element, but it seems like adding that dynamically via javascript doesn't work as I still need to manually enter application mode. But at least now I know there's a keyboard shortcut for that, so thanks! I'll take another look at setting application mode via javascript in the morning.

And just to be clear, hooking the various functions is just a quick and dirty way to see what works. Once things are working properly, I'll update fTelnet itself to natively support detecting and supporting screen readers.

[nromey]

You’re amazing! Thanks so much for helping. Since my BBS is tailored toward blind and visually impaired individuals, I just wanted a way for young folks and/or people without terminal emulators can use it. Also, I like the fact that you support ANSI music. I wish I could help you with coding, but I looked at the codebase and I can’t follow it. I really need to work on learning javascript as I know it could help me more effectively customize Synchronet. It’s all going to be about just getting familiar with syntax—if you’ve learned one language … kind of thing. Also, I tested the accessible ftelnnet page with JAWS and it works too. I haven’t tried it via mobile. Please let me know if I can help you with any of the code. I may just need guidance on where to start. Thanks again. Noel From: rickparrish ***@***.***> Sent: Monday, October 2, 2023 10:25 PM To: rickparrish/fTelnet ***@***.***> Cc: nromey ***@***.***>; Author ***@***.***> Subject: Re: [rickparrish/fTelnet] Accessibility of FTelnet and canvas (Issue #22) Your latest message didn't contain a link. But I did a bit of googling, and swapped out the textarea element for a div element with aria-live="assertive" set, and now NVDA reads out the incoming text. I also hooked the Connect function to try setting role="application" on the body element, but it seems like adding that dynamically via javascript doesn't work as I still need to manually enter application mode. But at least now I know there's a keyboard shortcut for that, so thanks! I'll take another look at setting application mode via javascript in the morning. And just to be clear, hooking the various functions is just a quick and dirty way to see what works. Once things are working properly, I'll update fTelnet itself to natively support detecting and supporting screen readers. — Reply to this email directly, view it on GitHub <#22 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFB7LSKWZOQ7VGN6K5IC2RDX5OAQHAVCNFSM6AAAAAA5L2SBNWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONBUGEYDIMJWGU> . You are receiving this because you authored the thread. <https://github.com/notifications/beacon/AFB7LSOCMYEHZFJNRYAAZRDX5OAQHA5CNFSM6AAAAAA5L2SBNWWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTTH6TVOK.gif> Message ID: ***@***.*** ***@***.***> >

[rickparrish]

Hey Noel,

I've been playing around with this a bit more, and it looks like I can set the Canvas element to use aria-live="true" and update its innerText when new text is written to the canvas. This has the benefit of not needing an extra div, which means it should be safe to enable by default (users without a screen reader running won't notice any difference).

I also have it auto-entering application mode on connect, and then exiting on disconnect. I'm doing this via an extra element. On connect I focus it, and then on disconnect I blur it. Any idea if this is the best way to go, or if there are other ways to enable application mode programmatically? I tried updating the body tag to have role="application" on connect, but that change didn't seem to be recognized by the screen reader so it stayed in browse mode.

These latest changes are only in my local copy of fTelnet for now, so unfortunately nothing new for you to test yet.

[rickparrish]

Just updated the demo page here:
https://www.ftelnet.ca/demos/accessible-demo/

As I mentioned in the last message, this uses the canvas element as an aria-live="assertive" element for the screen reader (is "assertive" or "polite" the better option? And is there value in allowing the sysop to choose which to use?)

It also includes a textbox with role="application" that gets focused and blurred on connect and disconnect, so the screen reader should automatically enter and exit application mode when needed.

Please let me know how it works for you, especially with JAWS since I only have NVDA to test with here.

Thanks,
Rick

[nromey]

Hey Rick, Here’s what I find. 1. Ftelnet does read as it’s being output. The problem, and I’m not sure if it’s how you’re updating the canvas element or if there’s something else going on, but the text repeats the whole screen when I type a key. I tried with JAWS and with NVDA and received the same result. I think this could be something we can change in the aria role. I’m looking now, and I can’t type a link in. hold on. It looks like we might want to set aria-live = “polite” aria-relevant= “additions” If you only update things that have changed, then the screen reader should just speak that content. I’m wondering though if the canvas text is sent / updated every time a key is pressed. If this is the case, then I’m not sure how we get around it. Aria-live=”polite” tells the system to only speak when the system is idle, there is an aria-atomic property which defaults to false, which we probably don’t need to assign. It may actually be better, though I know you didn’t want to do this, to do a div. The scrollback seems to be viewable now—I’m not getting the error I usually get when canvas isn’t available, so you could make a div and make it aria-hidden=true” which I think will hide the thing visually, but still work for a screen reader. This would mean you don’t need to do anything that’s screen reader specific, it’d just work. I’m really just spit balling here as I haven’t played with this stuff. That’s funny because I advise and consult about this stuff all the time at work, so this is probably good practice for me to understand how it works. I’m working on researching the application role and a good way to work it. I also have some friend that might help us out, but let me see what I can find there. Recommend checking out the aria role examples to see how it works—I can’t post a link here—but mozilla has some good examples on how polite works and the other things you can set. Man, I need to play with aria more, it looks way powerful. I’ll get back to you when I know something. Thanks so much for playing with this. We’ll get it. NER From: rickparrish ***@***.***> Sent: Thursday, October 5, 2023 7:01 PM To: rickparrish/fTelnet ***@***.***> Cc: nromey ***@***.***>; Author ***@***.***> Subject: Re: [rickparrish/fTelnet] Accessibility of FTelnet and canvas (Issue #22) Just updated the demo page here: https://www.ftelnet.ca/demos/accessible-demo/ As I mentioned in the last message, this uses the canvas element as an aria-live="assertive" element for the screen reader (is "assertive" or "polite" the better option? And is there value in allowing the sysop to choose which to use?) It also includes a textbox with role="application" that gets focused and blurred on connect and disconnect, so the screen reader should automatically enter and exit application mode when needed. Please let me know how it works for you, especially with JAWS since I only have NVDA to test with here. Thanks, Rick — Reply to this email directly, view it on GitHub <#22 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFB7LSMHKWNDG6SHEO6EGC3X55C4ZAVCNFSM6AAAAAA5L2SBNWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONBZHAYTAOJZHA> . You are receiving this because you authored the thread. <https://github.com/notifications/beacon/AFB7LSN34N3GSGVNVMKZHBDX55C4ZA5CNFSM6AAAAAA5L2SBNWWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTTIJP7TM.gif> Message ID: ***@***.*** ***@***.***> >

[rickparrish]

Hey Noel,

I've switched from assertive to polite. I've also tweaked how the text is added to the canvas live region -- originally it was setting the innerText of the canvas, so yeah hitting a key would overwrite the innerText with the same text as before but with your new keypress added to the end, and then seeing the text changed must make the screen reader start over (even though 99% of the text is the same)

Now it appends child divs to the canvas element, so the behaviour I'm seeing is that if NVDA gets halfway through reading the screen when I hit a key, it stops reading and only announces the new keypress I just made, then stops talking. Is that the desired behaviour?

I can switch to a div if needed, I was just hoping to keep things backwards compatible so screen reading could be on by default. As far as I know any standard method used to hide a div visually will also hide it from a screen reader, so non-standard methods like position: absolute; left: -9999px; top: -9999px to position it outside the viewable area of the screen would need to be used, but I'm not 100% sure whether that's safe or if it may mess up the display for some users.

Thanks,
Rick

[nromey]

Yes, this should be the behavior. I’ll need to test it—I’ll do that later today, but what will happen is that say you try to login to your demo page. If you type a letter, it would say nothing, but when you press enter, it’d say something like password rather than reading the whole screen. Give me an hour or so and I’ll try this. These changes may actually help others who do not use screen readers—I’ve found that adding these things i.e. tweeking your canvas element to only update when it needs to, can help with low bandwidth settings. Sure, you’re not sending lots and lots of data, but still. I’ll get back to ya. From: rickparrish ***@***.***> Sent: Friday, October 6, 2023 7:24 AM To: rickparrish/fTelnet ***@***.***> Cc: nromey ***@***.***>; Author ***@***.***> Subject: Re: [rickparrish/fTelnet] Accessibility of FTelnet and canvas (Issue #22) Hey Noel, I've switched from assertive to polite. I've also tweaked how the text is added to the canvas live region -- originally it was setting the innerText of the canvas, so yeah hitting a key would overwrite the innerText with the same text as before but with your new keypress added to the end, and then seeing the text changed must make the screen reader start over (even though 99% of the text is the same) Now it appends child divs to the canvas element, so the behaviour I'm seeing is that if NVDA gets halfway through reading the screen when I hit a key, it stops reading and only announces the new keypress I just made, then stops talking. Is that the desired behaviour? I can switch to a div if needed, I was just hoping to keep things backwards compatible so screen reading could be on by default. As far as I know any standard method used to hide a div visually will also hide it from a screen reader, so non-standard methods like position: absolute; left: -9999px; top: -9999px to position it outside the viewable area of the screen would need to be used, but I'm not 100% sure whether that's safe or if it may mess up the display for some users. Thanks, Rick — Reply to this email directly, view it on GitHub <#22 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFB7LSJIXHFEHPTLRPPFJSTX57Z7JAVCNFSM6AAAAAA5L2SBNWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONJQGU2TSOJRG4> . You are receiving this because you authored the thread. <https://github.com/notifications/beacon/AFB7LSI2HRYHT4XMWF2P3DDX57Z7JA5CNFSM6AAAAAA5L2SBNWWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTTIK5WK2.gif> Message ID: ***@***.*** ***@***.***> >

[nromey]

Ok Rickk. Well done on the text being only read. It looks like there’s a keyboard trap. I’m going to send one of my developer friends over to your demo to see if they have any idea. Basically, it forces application mode, but when you’re there, it keeps you there, even if you press capslock+space. You’d want to do this if you wanted to review what’s entered. I’m starting to wonder if it would be best to add a note that tells screen reader users to “Please exit browse or forms mode to review the scrollback. You can do this notification by including a 1 pixel graphic and using an alt text for that pixel and it shouldn’t affect how the page displays. I’m not sure how else to work with application mode/allowing the user to exit that part of the page. Application mode is great, but if you can’t exit it, you’re in trouble. If the text box continues to be an edit field, the user could focus it, and press enter to activate it. From: rickparrish ***@***.***> Sent: Friday, October 6, 2023 7:24 AM To: rickparrish/fTelnet ***@***.***> Cc: nromey ***@***.***>; Author ***@***.***> Subject: Re: [rickparrish/fTelnet] Accessibility of FTelnet and canvas (Issue #22) Hey Noel, I've switched from assertive to polite. I've also tweaked how the text is added to the canvas live region -- originally it was setting the innerText of the canvas, so yeah hitting a key would overwrite the innerText with the same text as before but with your new keypress added to the end, and then seeing the text changed must make the screen reader start over (even though 99% of the text is the same) Now it appends child divs to the canvas element, so the behaviour I'm seeing is that if NVDA gets halfway through reading the screen when I hit a key, it stops reading and only announces the new keypress I just made, then stops talking. Is that the desired behaviour? I can switch to a div if needed, I was just hoping to keep things backwards compatible so screen reading could be on by default. As far as I know any standard method used to hide a div visually will also hide it from a screen reader, so non-standard methods like position: absolute; left: -9999px; top: -9999px to position it outside the viewable area of the screen would need to be used, but I'm not 100% sure whether that's safe or if it may mess up the display for some users. Thanks, Rick — Reply to this email directly, view it on GitHub <#22 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFB7LSJIXHFEHPTLRPPFJSTX57Z7JAVCNFSM6AAAAAA5L2SBNWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONJQGU2TSOJRG4> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[nromey]

I was able to use the mouse to exit the application, but that’s the only way I could exit it. Reading the text in “developer.mozilla.org”, first: You want to probably use a DescribedBy aria element to the application. You might also want to include a keypress event in Javascript to change the focus properly using aria.activedescentant to keep focus. It’s probably best to set application on your canvas only, not the body. If you can figure out how to use aria.activedescendant to set focus outside after a keypress—I’m not sure if escape would be helpful as some systems use escape while connected, so maybe shift-escape if it can be scripted—I have no idea. If you can fix that, and use the aria.describedy on the canvas element to tell screen readers what to do will help tremendously. I don’t think you need to use a pixel and associated alt text, that’s old style. DescribedBy is where it’s at I’ve heard. From: rickparrish ***@***.***> Sent: Friday, October 6, 2023 7:24 AM To: rickparrish/fTelnet ***@***.***> Cc: nromey ***@***.***>; Author ***@***.***> Subject: Re: [rickparrish/fTelnet] Accessibility of FTelnet and canvas (Issue #22) Hey Noel, I've switched from assertive to polite. I've also tweaked how the text is added to the canvas live region -- originally it was setting the innerText of the canvas, so yeah hitting a key would overwrite the innerText with the same text as before but with your new keypress added to the end, and then seeing the text changed must make the screen reader start over (even though 99% of the text is the same) Now it appends child divs to the canvas element, so the behaviour I'm seeing is that if NVDA gets halfway through reading the screen when I hit a key, it stops reading and only announces the new keypress I just made, then stops talking. Is that the desired behaviour? I can switch to a div if needed, I was just hoping to keep things backwards compatible so screen reading could be on by default. As far as I know any standard method used to hide a div visually will also hide it from a screen reader, so non-standard methods like position: absolute; left: -9999px; top: -9999px to position it outside the viewable area of the screen would need to be used, but I'm not 100% sure whether that's safe or if it may mess up the display for some users. Thanks, Rick — Reply to this email directly, view it on GitHub <#22 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFB7LSJIXHFEHPTLRPPFJSTX57Z7JAVCNFSM6AAAAAA5L2SBNWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONJQGU2TSOJRG4> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[rickparrish]

For the keyboard trap, what's happening is I added a textbox with the role set to application, and then on connect the textbox is focused to enter application mode, and on disconnect it is blurred to exit application mode. It didn't occur to me that the keyboard shortcut to exit application mode wouldn't work while the textbox is focused, so that's not good!

I can definitely add a keyboard shortcut to exit application mode, but then users will need to know what the shortcut is, and so maybe another option would be to not add the textbox, and not auto-enter application mode? Then users could use their screen reader's shortcut keys to enter/exit application mode as needed.

[nromey]

Yeah I think that that might work better. In case you wanted to do a keyboard shortcut some day, you would just need to set an aria.describedby role on the body. When the web page gets focused, it’ll speak some instructional junk at you. This is a good way to hide instructions away. Sighted users won’t know it exists. Let’s do this: Allow users to enter the application with the screen reader, but we use a described by on the field so that when the user arrows, or goes to it via tab, instructions get read. “Enter application or forms mode to type to the terminal” Or something like that. You could also add a describedby to the body, I think, but it might be better to do a described by on the element in question for clarity. What do you think? From: rickparrish ***@***.***> Sent: Saturday, October 7, 2023 5:32 PM To: rickparrish/fTelnet ***@***.***> Cc: nromey ***@***.***>; Author ***@***.***> Subject: Re: [rickparrish/fTelnet] Accessibility of FTelnet and canvas (Issue #22) For the keyboard trap, what's happening is I added a textbox with the role set to application, and then on connect the textbox is focused to enter application mode, and on disconnect it is blurred to exit application mode. It didn't occur to me that the keyboard shortcut to exit application mode wouldn't work while the textbox is focused, so that's not good! I can definitely add a keyboard shortcut to exit application mode, but then users will need to know what the shortcut is, and so maybe another option would be to not add the textbox, and not auto-enter application mode? Then users could use their screen reader's shortcut keys to enter/exit application mode as needed. — Reply to this email directly, view it on GitHub <#22 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFB7LSNJ76GC7WQ4N3K77DLX6HJ63AVCNFSM6AAAAAA5L2SBNWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONJRHA2DINJXHA> . You are receiving this because you authored the thread. <https://github.com/notifications/beacon/AFB7LSPWEKAWBJTNKRLZFQDX6HJ63A5CNFSM6AAAAAA5L2SBNWWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTTINMDOE.gif> Message ID: ***@***.*** ***@***.***> >

[rickparrish]

I haven't had much luck with the describedby setup, it seems to be very hit and miss for me. So sometimes it works and reads the message about needing to enter application mode to interact with the terminal, but other times it seems to completely ignore it.

I wonder if maybe when the client connects it could add the text to an aria-live region set to assertive, so that text would be read first. I'll try that next and see how it goes.