Over the years, we have seen Apple's unwavering commitment to ensuring the accessibility of your devices to all users. Apple introduced several new accessibility features in iOS 17, including Point and Speak, a feature built into the Magnifier app.
The new "Detection Mode" Helps people with visual impairments navigate and interact with their physical environment. Here's everything you need to know about How to use the Point and Talk feature on iPhone and iPad.
Point and talk is a new accessibility feature built into Magnifier detection mode, which uses the iPhone or iPad sensors to detect text, people, doors and nearby scenes and provide users with sound, speech or haptic feedback.
Combines on-device machine learning and device camera input and the LiDAR scanner to detect the position of your finger and read the text labels pointed to by your finger within the camera view.
Prerequisites for using Point and Talk on iPhone and iPad
To use Point and Talk in the Magnifier app, your device must meet the following conditions:
- An iPhone or iPad with iOS 17 or iPadOS 17 or later.
- A device with a LiDAR sensor. LiDAR sensors are currently available on the following iPhone and iPad models: iPhone 15 Pro and 15 Pro Max, iPhone 14 Pro and 14 Pro Max, iPhone 13 Pro and 13 Pro Max, iPhone 12 Pro and 12 Pro Max, iPad Pro ( 2020- 2022)
Also, to use Point and Talk in the Magnifier app, you must first enable detection mode, since Magnifier's main controls are for zooming in on things in the environment.
How to Enable Discovery Mode in the Magnifier App on iPhone
This is how you can enable discovery mode on iPhone
- First Open the Magnifying Glass app.
- Now tap the gear icon > Settings. You will be directed to the Customize Controls screen.
- Locate the detection mode. If a plus (+) icon precedes it, drag it to the parent or child controls to display the feature in the magnifying glass. If you can't add it, consider removing an existing control before replacing it with discovery mode.
- Touch Done. If you've placed it in Primary Controls, it should be at the top level of the Magnifying Glass icons. If you put it under Secondary Control, you can find its icon on the second layer. The detection mode icon looks like a square that looks like four brackets.
Once you have secured everything above, you can proceed to use this new feature.
How to Use Point and Talk Magnifier in iOS 17 on iPhone
- First open the Magnifier app on your iPhone or iPad.
- Select the detection mode icon (square shaped).
- Now tap the point and talk icon (the index finger icon). The icon will highlight yellow once selected.
- Place your finger under the camera and wait for your device to announce that your finger has been detected. You will also see a notification detected by hand on the screen.
- Keep your camera parallel to the surface or object being scanned. By default, Point and Speak It will detect the text under your finger. To change this, select Point and talk > tap Settings at the top left of the screen.
- Select Point and Talk. Under Location, tap Under or Over. Toggle auto flashlight if you want the magnifier to automatically turn on the flashlight for 10 seconds when the environment is too dark to detect text.
You can also select which feedback you want to get each time you use this feature. Under Detection Feedback, turn on any of the following:
- Sound: You will hear a clicking sound every time it detects something
- Tags: Your device will display the text it was able to recognize
- Speech: Your device will describe/read the text it is pointing at
- Haptics: You will feel a haptic response every time you identify a text
By default, when you point to text on your iPhone or iPad, it will close with a yellow border. However, you can modify the color of this border or remove it completely if you prefer. Here's how you can do it:
- To remove the border: Turn off the Border option.
- To change the border color: Press Color. A range of color options will appear, allowing you to choose your preferred border color. You can tap the Grid, Spectrum, or Sliders tab to explore and select the ideal color that suits your preferences.
Benefits of using this option in Lupa
- Many people with low vision retain the ability to perceive contours and distinguish high contrasts, allowing them to recognize buttons and objects in their environment, especially when they are marked with significant contrast. However, they may lack the visual acuity to decipher labels, particularly those with small fonts or close spacing. In such cases, Point and Talk can be immensely valuable to those who require help interpreting labels printed on various objects.
- Also may be beneficial for people whose eyes tire quickly. With this you can read even the smallest labels before displaying the detected text in a larger font and even narrate it for you, so you don't have to strain your eyes to read tiny text.
Conclusion
You should know that Magnifier is an application integrated into the iPhone and iPad. If you can't find it, you may have deleted it from your iPhone when you bought it, thinking you wouldn't use it. Now go to the App Store to download it again.
Vision is a precious gift, allowing us to appreciate our surroundings and interact with the world. Unfortunately, many everyday objects have labels printed in small fonts, and for those experiencing a loss of visual clarity, this can make interacting with such objects difficult.
Fortunately, innovative features like this can transform your iPhone or iPad into a revolutionary magnifying tool that not only enlarges labels, but also reads them aloud. If you already used it, let us know in the comments.