Questions & Answers:

 

Why is the transcription often not accurate?

1) The quality of the input. If the articulation of the speaker is not precise enough, the transcribing servers might have difficulties. Or a strong dialects, accents. Or too fast. Or the volume is just too low.

2) Background noise. The servers will try to extract speech from any surrounding sounds but might fail in precisely doing so. Other people talking in at close range or music is deadly for speech recognition. Real hearing people can easily distinguish speech from noise and music but speech recognition computers can not (yet).

3) Bad internet connection. If the network connection is slow, the sound of the speech is not sent fast enough to the servers so the servers only process parts of the speech resulting in incomplete transcriptions.

4) Small language. Simply put, Apple puts in more money, energy and training into bigger languages like English followed bij Spanish and Chinese. Smaller languages are less important to Apple.

5) Interpretation. Nowadays transcribing speech is more than just converting sounds. The servers try to analyse the meaning of the spoken sentence and reply with their understanding. That might be wrong. It can even happen that first you see the right word(s) but then it suddenly changes to a wrong interpretation. Silly but unavoidable.

 

Why does the transcription often stalls for a moment?

1) Bad internet connection.

2) Apple's servers overloaded.

3) Apple limits continuous speech-to-text streams to close to a minute. That means that the app will close and restart the stream after ~ 50 seconds. This will most likely cause a hickup in the transcription.

 

Why do i see sometimes nothing or really wrong words appearing?

Besides possible problems with the above mentioned issues, you could have selected the wrong language. The app and the servers do not have automatic language recognition so you have to select the right language to transcribe. Also nowadays people often use foreign words so the servers might get confused. For example talking in Dutch to an English listening computer will give unexpected results.

 

Why don't I see interpunction at all or capitals at the beginning of e new sentence?

Unfortunately Apple's speech recognition (and by the way most others as well) are still not clever enough to really understand what is said or who said what. That simply means that the machines have no clue of where to put a full stop and start a new sentence. Let alone comma's, question marks or exclamation marks. However, it is expected that in the near future this will automatically improve even without updating the ToggleSub app. The software and Artificial Intelligence on Apple Speech Recognition servers will definitely improve over time.

Only when a significant pause is recognised or a continuous speech stream had to be restarted after ~50 seconds, the app will clear the subs and start a new line with a capital.

 

I can't access other app windows with the mouse in the area the subtitles are displayed.

That's right and by choice. To display the subtitles always on top, the overlay is, besides some Apple features like the Dock, the top-most window. Since it is transparent, it looks like you can acces the windows underneath but you can not. And there is where the Toggle comes into play. CTL-ALT-CMD-S will switch the window from on to off, do what you want to do and switch it on again. It's highly recommend to switch it off when you are not transcribing at all. Not only you won't have issue accessing anything else on the screen but you also avoid useless network traffic

To get used to it, in the preferences System Tab, you can switch on a border which will light up whenever the mouse enters the subtitle area. After a while and getting more experienced with the app, you will probably switch it off again.

 

What is that percentage popping up every now and then?

Apple's transcription servers irregularly send a "confidence" percentage. It's a indication of the confidence they have transcribing the speech to readable text. However, it is therefore more an indication of the quality of the input and connection since the servers are actually pretty good with continuous, clearly articulated, undisturbed speech. A green color means pretty good, a red color pretty bad. If it is always red (or yellow) you should look after the quality of the input.

The issue is a bit that this confidence percentage is only send by the servers when there is a pause, a probable end of speech. However for example in discussion programs or interviews, people tend to keep talking at a high rate so when there is finally a sort of pause, the confidence percentage hardly reflects reality.

When it's getting a bit annoying, you can switch it off in the app's Preferences just like the volume bar.

We don’t collect, store or share your data, configuration or actions.

• ToggleSub does not read anything from your machine except the sounds you want to transscribe
• ToggleSub does not store or distribute any of the sounds or transcriptions longer than the app is alive
• ToggleSub does not write anything to your disks except for you preferences
• ToggleSub does not keep anything in memory except its preferences and status as long as the app is active
• ToggleSub does not monitor your keyboard actions except the ^ ⌥ ⌘ (CTL-ALT-CMD) combinations to toggle
• ToggleSub does not connect to your local network nor the internet

 

Keyboard shortcuts

 

Press CTL-ALT-CMD (⌃⌥⌘) simultaneously

+ S = Toggles the subtitle overlay on or off

+ UP = Increases the size of the subtitle font

+ DOWN = Decreases the size of the font

Background and use.

The app was initially created as an aid for people with hearing disabilities but proved to be very useful for hearing people as well.

For hearing disabled people, It allows you to keep looking at the screen while having a conversation where other people's speech is picked up by a microphone connected to your Mac.

For all, so bad/non-hearing and hearing people, with an audio loopback driver (like BlackHole (1) or LoopBack (2)) selected, video applications such as Facetime or Quicktime or podcasts, movies and live TV via a web browser can be subtitled live without the need for physical speakers to be switched on and maintain silence in the room.

Working.

Audio picked up by a microphone or driver is broken into chunks and sent of the internet to Apple's speech services. On some powerful and properly configured Macs the conversion might be performed locally. The speech is converted into readable text and projected as an overlay on top of all other windows, usually even if they are in fullscreen mode. However, some other apps may forcefully grab full focus of a screen pushing the subtitles to an invisible background.

A built-in LED bar can be used to visualize whether there is sound on the input channel at all. Just next to it a percentage of "confidence" pops up sometimes. Be aware that this is not the accuracy of the transcription but the confidence level of the machine transcribing. It can happen that the output, the transcription is 100% correct but the confidence surprisingly low and vice versa. It's more an indicator of the quality of the input which the machine is trying to understand.

Currently the user still needs to select the language the spoken text is in. The app can not detect the language automatically (yet). In the preferences this language can be set next to a number of other settings like the view characteristics of the subtitles.

Be aware that when the subtitles are turned on, the overlay blocks the area on the screen for interaction with other apps. You can either turn the subtitles off (CTL-ALT-CMD-S) temporarily of just grab the subtitle area with your mouse and drag it to anywhere else on the screen. Since it is in fact a window with invisible borders, you could still resize the window by grabbing its sides. In the preferences-system there is check to turn the window borders on in the same color the subtitle text is. Useful for learning how to use the app but not recommended for experienced users.

And finally, it is highly recommended to switch the subs off (CTL-ALT-CMD-S) when you don't need them. Since it is an overlapping window, handling of content of underlying windows can be more complicated or even impossible. The app "lives" in the top bar of your screen so it always listens to your toggle.

What if it does not work out of the box?

The first thing you need to check is the input source found in you Mac's preferences. For example, if you want to grab speech around you, the input source should be a microphone. If you are watching a podcast or movie, the input source should be a driver which feeds sounds to the input. Make sure that with some 3rd party drivers you have to feed the output not to the speakers but to the driver as well. You could also use real audio so sound from the speakers you have is picked up by the Mac's microphone. However, this can cause unwanted audio feedback loops. So testing what are the most convenient and practical use cases, is not a waste of time.

Some defaults.

The defaults are set to your own preference. However there are some recommendations:

• Alignment LEFT. Why? Because it is more stable and therefore less distracting. Remember that in movie complete sentences are displayed at once and even just before the speaker starts. In that case a CENTER alignment is the better choice. However with real time subtitling we display word for word to follow the speed of the speaker. Waiting for a sentence to complete would be much too slow.
• Color GREEN. Why? Because green is avoided on TV and movies. "Green-screens" are used to create artificial backgrounds so exposing a solid green color would make things complicated for the movie editor. Of course you can make a different choice.
• Border OFF. Why? You can move the subtitles to anywhere and any size on your screen but sometime it is handy to see what you are doing. More experienced users know where the corners of the subs are but to learn what is most useful for you, switching the border on temporarily to move or resize the subs can be convenient.

Privacy and security.

This app does not store audio or transcribed text for more than a few seconds. However, unless a powerful Mac transcribes speech locally, pieces of audio are sent to Apple's servers to convert speech to text. Just like that also happens for Siri. The app has no means to avoid or manage that. It does not use or track any sound or transcription.

The only data that is stored are the app defaults in the preferences.

Keep in mind that, for example, if you watch a video stream with a virtual loopback microphone and with physical speakers turned off, someone else in the room may be looking over your shoulder and reading the closed captions on your screen.

Disclaimer.

Transcribing spoken text is a difficult task and is certainly not 100% error free. It is best performed by powerful machines of big service providers like Apple and Google. Nevertheless the results may vary depending on:

• The quality of the input like the loudness, accent and articulation of the speaker and the background noise;
• The speed and quality of the local network and internet;
• The language. English is by far the language which is transcribed best. Simply because the majority of research and development in speech recognition is done in and with the English language. The “smaller” languages like Dutch get less attention so the quality of the transcription will be a lot less as a result. It makes the need for excellent input quality even more important.

That said, the subtitles you see can never be better than given the quality of the input and what Apple's services can provide. This app can't do anything to improve that.

However, results are expected to get better over time, even already in the near future. Since the rapid rise of AI applications, speech recognition has gained momentum.

• we are not connected to or endorsing other software developers like from BlackHole or LoopBack. They are just examples of loopback drivers which may  or may not work on your specific Mac. We are never responsible for the working or performance of these drivers. However, using a driver like these would make this app a lot  more versatile.

——————

(as per October 2024)

(1) BlackHole from https://existential.audio/blackhole/

(2) LoopBack from https://rogueamoeba.com/