[RNMobile] Fix dictation regression on iOS

[SiobhyB]

Proposed fix for wordpress-mobile/gutenberg-mobile#5165.

Related PRs:

• Gutenberg Mobile: Fix dictation regression on iOS wordpress-mobile/gutenberg-mobile#5612
• WordPress iOS: TESTING ONLY: DO NOT MERGE: Fix dictation issue on iOS wordpress-mobile/WordPress-iOS#20669

What?

The changes in this PR address an issue with dictation not working as expected on devices running iOS 16 or later.

Why?

Following improvements made to dictation in iOS 16‌, a previous hack that avoided an empty string being propagated with dictation, stopped working.

Now, with devices that are running iOS 16 or later, text added with iOS dictation is lost when tapping into the editor or typing while the dictation is still recording. This causes unexpected content loss, and could be viewed as an accessibility issue for those who rely on dictation to use the editor.

How?

By removing the previous dictation-related hacks, we fallback to the default dictation system, which works without causing any content loss for later iOS versions. The reasoning behind following this approach is detailed here.

In addition, it's still necessary to remove the obj symbol that's added due to an RN bug. This is done directly within the textViewDidChange() function.

Testing Instructions

An installable build is available at wordpress-mobile/WordPress-iOS#20669 for testing on a physical device.

The following steps need to be tested with both iOS 16 or later as well as on an earlier version of iOS:

Test 1: Ensure new blocks don't replace old ones ⤵️

1. Navigate to My Site → Posts and create a new post.
2. Type some text into a paragraph block.
3. Tap the inserter in the editor's toolbar.
4. Select a new block to add to the editor.
5. Confirm that the new block is added beneath the currently select block, and that it doesn't replace it.

Based on testing steps from this PR: #49154

Test 2: Verify there is no content loss while dictating ⤵️

1. Navigate to My Site → Posts and create a new post.
2. Activate the iOS dictation feature and dictate some text.
3. Notice the text added to the editor.
4. If the keyboard remains visible while dictating, tap another block on the editor or begin typing onto the keyboard. Alternatively, if the keyboard is not visible while dictating, tap the language pop up.
5. Verify that taking an action while dictation is still running doesn't lead to content loss

Based on bug testing steps from this issue: wordpress-mobile/gutenberg-mobile#5165

Test 3: Verify no obj symbol is added after dictation ⤵️

1. Navigate to My Site → Posts and create a new post.
2. Tap on the Post Title.
3. Start dictating with voice.
4. When finished, press Enter.
5. Confirm that there is no obj symbol appended to the post title.

Based on testing steps from this PR: wordpress-mobile/gutenberg-mobile#1969

Test 4: Verify block expands with content ⤵️

1. Navigate to My Site → Posts and create a new post.
2. Tap on "Start writing..." to focus the default paragraph block.
3. Say "I am writing a new block with my voice"
4. Switch to keyboard input and type some text
5. Confirm that the paragraph box expands to a second line

Only happens with an external keyboard (only applicable to iPad or Simulator with physical keyboard input)

Test 5: Verify text position is retained after dictating in middle of paragraph ⤵️

1. Navigate to My Site → Posts and create a new post.
2. Tap on "Start writing..." to focus the default paragraph block.
3. Activate the iOS dictation feature and dictate "Mary had a little lamb its fleece was white as snow"
4. Place the cursor after "little" then deactivate iOS dictation
5. Activate the iOS dictation feature and dictate "Old MacDonald had a Farm"
6. Tap below the block to start a new paragraph
7. Confirm that the ordering of the words is "Mary had a little old MacDonald had a Farm lamb it's fleece was white as snow" not "Mary had a little lamb it's fleece was white as snow old MacDonald had a Farm"

Based on this PR: wordpress-mobile/gutenberg-mobile#1969

In addition, verify that there are no failed tests across the Gutenberg, Gutenberg Mobile, and WordPress iOS PRs signalling unwanted side effects.

Screenshots or screencast

The following screencast shows dictation working, even after tapping the language pop up and resuming dictation (previously this led to content loss):

dictation.mov

[github-actions]

Flaky tests detected in ff64c55.
Some tests passed with failed attempts. The failures may not be related to this commit but are still reported for visibility. See the documentation for more information.

🔍 Workflow run URL: https://github.com/WordPress/gutenberg/actions/runs/5132777788
📝 Reported issues:

• [Flaky Test] shouldn't conflict with server-side autosave #35176 in specs/editor/various/autosave.test.js

[twstokes]

👋 Thanks @SiobhyB for continuing to tackle this! 🙇

The change you've proposed here looks good, but I wondered why we had this dictation override in the first place. From what I can tell, the history is:

1. An issue was discovered in 2019 on iOS 12.1.4
2. This PR was created to resolved it
3. This follow-up fix related to the OBJ character and string positioning was created
4. In September of 2022 this issue appeared on iOS 16
5. We tried a fix here to work around the iOS 16 issue
6. We reverted the fix due to newly added blocks being replaced

😅

This made me wonder if we can frame the problem this way:

1. We have code that works around an issue found on iOS 12
2. We have code that addresses the OBJ character and string position, possibly discovered on iOS 13 (judging by the date)
3. We have another issue that appeared in iOS 16
4. We're not sure if items 1 and 2 apply to iOS 14-16 (side note: iOS 13 is still the minimum platform for RNTAztecView and I think we should bring that up to 14)

From here I created five tests:

Test 1

• Based on testing steps from this PR: [RNMobile] Prevent blocks being un-wantingly replaced with newly added blocks #49154

1. Navigate to My Site → Posts and create a new post.
2. Type some text into a paragraph block.
3. Tap the inserter in the editor's toolbar.
4. Select a new block to add to the editor.
5. Confirm that the new block is added beneath the currently select block, and that it doesn't replace it.

Test 2

• Based on testing steps from this PR: [RNMobile] Prevent blocks being un-wantingly replaced with newly added blocks #49154
• Based on bug testing steps from this issue: Text added with iOS dictation disappears wordpress-mobile/gutenberg-mobile#5165

1. Navigate to My Site → Posts and create a new post.
2. Tap on "Start writing..." to focus the default paragraph block.
3. Activate the iOS dictation feature and dictate some text.
4. Notice the text added to the editor.
5. Tap another block on the editor or begin typing onto the keyboard to confirm there is no content loss.
6. Experiment with adding various text and attempting to break dictation.

Test 3

• Based on testing steps from this PR: [iOS] Fix [Obj] character after dictation wordpress-mobile/gutenberg-mobile#1969

1. Navigate to My Site → Posts and create a new post.
2. Tap on the Post Title.
3. Start dictating with voice.
4. When finished, press Enter.
5. Confirm that there is no [UNDEFINED] appended to the post title.

Test 4

• Only happens with an external keyboard (only applicable to iPad or Simulator with physical keyboard input)

1. Navigate to My Site → Posts and create a new post.
2. Tap on "Start writing..." to focus the default paragraph block.
3. Say "I am writing a new block with my voice"
4. Switch to keyboard input and type some text
5. Confirm that the paragraph box expands to a second line

Test 5

• Based on this PR

1. Navigate to My Site → Posts and create a new post.
2. Tap on "Start writing..." to focus the default paragraph block.
3. Activate the iOS dictation feature and dictate "Mary had a little lamb its fleece was what as snow"
4. Place the cursor after "little" then deactivate iOS dictation
5. Activate the iOS dictation feature and dictate "Old MacDonald had a Farm"
6. Tap below the block to start a new paragraph
7. Confirm that the ordering of the words is "Mary had a little old MacDonald had a Farm lamb it's fleece was white as snow" not "Mary had a little lamb it's fleece was white as snow old MacDonald had a Farm"

Results

trunk

	iOS 15.7.1	iOS / iPadOS 16.4.1
Test 1	✅	❌
Test 2	✅	❌
Test 3	✅	❌
Test 4	✅	❌
Test 5	⚠️ (flaky)	❌

gutenberg/test-dictation-fix

	iOS 15.7.1	iOS / iPadOS 16.4.1
Test 1	✅	✅
Test 2	✅	✅
Test 3	✅	✅
Test 4	✅	✅
Test 5	❌	❌

Dictation overrides removed

For this test I removed all workarounds:

• Reverting the code changed here (as well as the subsequent fixes mentioned above)
• ⚠️ Only tested on iOS 15 and 16
• ⚠️ Behavior change: A space is added when dictation is disabled, but that appears to be normal iOS behavior (confirmed in the Notes app). However, this could lead to extraneous spaces so we'd need to consider adding trimming logic.

	iOS 15.7.1	iOS / iPadOS 16.4.1
Test 1	✅	✅
Test 2	✅	✅
Test 3	✅	✅
Test 4	✅	✅
Test 5	✅	✅

Bonus, this feature seemed to be enabled:

Thoughts

• I'm fine to proceed with the original change if we can fix the test 5 regression.
• It'd be great if we could remove the workarounds at some point if we determine for sure that they're not needed.
• It's possible I've missed a critical test, so I'd love another set of 👁️s. I didn't test the language pop-up mentioned at the top of this PR.
• We should consider dropping platforms older than iOS 14 (stats 1, stats 2)

I'd love to hear your thoughts on this @SiobhyB. Thanks!

[SiobhyB]

@twstokes, thanks so much for the thorough and thoughtful testing, I truly appreciate it. 🙇‍♀️ I had overlooked the fifth test case.

I followed the same tests that you outlined using an iPhone XR (iOS 16.4.1) and came up with the following results:

Testing	trunk	dictation-regression-ios	overrides removed
Test 1	✅	✅	✅
Test 2	❌	✅	✅
Test 3	✅	✅	❌
Test 4	❌	✅	✅
Test 5	✅	❌	✅

With all of the overrides removed, I'm unfortunately able to consistently replicate the issue with an added obj symbol after dictating title text. The symbol can only be viewed from the post list, after dictating title text and exiting the editor.

I then tested removing the isInsertingDictationResult flag and only keeping the following override:

public override func insertDictationResult(_ dictationResult: [UIDictationPhrase]) {
        let objectPlaceholder = "\u{FFFC}"
        let dictationText = dictationResult.reduce("") { $0 + $1.text }
        self.text = self.text?.replacingOccurrences(of: objectPlaceholder, with: dictationText)
    }

However, that brings us back to the issue with content loss. 🤔 I'll explore possible other ways around the obj bug tomorrow.

[twstokes]

I'm unfortunately able to consistently replicate the issue with an added obj symbol after dictating title text. The symbol can only be viewed from the post list, after dictating title text and exiting the editor.

Ah! Thanks for this - I didn't know about the post list part. 🙇 Yep, Test 3 now fails for me as well. 😅

override func insertDictationResult may very well be a necessity, then. It's possible we could do the find / replace in textViewDidChange but that doesn't some nearly as performant since it would happen on every call.

Behavior change: A space is added when dictation is disabled, but that appears to be normal iOS behavior (confirmed in the Notes app). However, this could lead to extraneous spaces so we'd need to consider adding trimming logic.

Knowing that the OBJ character is causing this I don't think this is "normal" anymore.

[SiobhyB]

@twstokes, I've updated the approach to adding dictating text to the following, which is working for all tests cases in my testing:

gutenberg/packages/react-native-aztec/ios/RNTAztecView/RCTAztecView.swift

Lines 367 to 369 in 316276d

	if let textRange = self.selectedTextRange {
	self.replace(textRange, withText: dictationText)
	}

Looking forward to hearing whether this also works for and whether you have any thoughts!

[SiobhyB]

I just found an issue with obj continuing to be added following the latest proposed fix, so am setting this back to draft while I address that. cc: @twstokes

[twstokes]

👋 Hey @SiobhyB, I wanted to recap some thoughts on my side from our discussion today:

1. Here's an active RN issue to track.
2. An approach to consider is to remove all of our override code (identified in this comment) and try a find / replace for the OBJ character here.

Two potential drawbacks of this approach that I can identify are:

1. A performance penalty of the find / replace on every character insertion (in reality it may not be much).
2. Removing the overrides may cause issues on iOS versions older than 15 (iOS 13 is still the minimum platform, but 13 and 14's market share should be extremely low).

[SiobhyB]

@twstokes, thank you for discovering that RN issue! A recent internal post by @guarani made me aware that the WordPress app dropped support for iOS 13 ~8 months ago. In addition, the post confirmed that iOS 14 app usage was low (~1% of all sessions) and that we would not be locking users out of using the feature by dropping support:

However, it’s important to consider that anyone running iOS 14 can upgrade to iOS 15 (check [iOS 14 supported models](https://href.li/?https://support.apple.com/en-au/guide/iphone/iphe3fa5df43/14.0/ios/14.0) and [iOS 15 supported models](https://href.li/?https://support.apple.com/en-au/guide/iphone/iphe3fa5df43/15.0/ios/15.0)), so users are not left without options.

With that in mind, as well as the good efforts we've gone to trying to support the older versions, I agree that we should remove the dictation hacks and replace the obj symbol elsewhere in the code. I'll implement that change shortly 🙇‍♀️

[SiobhyB]

Without this code, the cursor would always be repositioned (along with the dictated text) after dictating in the middle of a paragraph. I experimented with a few different ways to repositioning the cursor, and this is the one that seemed to work the best.

I've been able to reproduce one issue with this approach:

1. Begin dictating text into a paragraph block.
2. Without ending dictation, move the cursor into the middle of the text and begin dictating.
3. Notice that the cursor can appear out of place when the text moves onto a new line.
4. The cursor will correct itself as you continue dictating, and the text will remain in the correct position.

Note, this doesn't happen if you end dictation and then re-start it to add text into the middle of the text. I'd consider this somewhat of an edge case, which corrects itself and doesn't impact the actual text entry.

In the interest of shipping a timely fix, and if there aren't any obvious other solutions, I'd like to propose that we go with this one for now and address improving the cursor positioning in a separate PR.

[twstokes]

That sounds reasonable to me @SiobhyB. 👍

[SiobhyB]

@twstokes, this is ready for review again when you have the chance 🙇‍♀️

[twstokes]

LGTM @SiobhyB - thanks for your work on this issue. I made one note about test 5, but I don't think it should block this PR.

I tested both iOS 15 and 16. 🚀

[twstokes]

When running test 5 on iOS 16 I seemed to encounter an off-by-one where the cursor advanced its position when I disabled dictation. I'm not super familiar with the textView cursor position logic, but I wonder if when we calculate the offset under the condition that textView.selectedTextRange?.start isn't nil, and subtracted one from the resulting integer, if that would resolve the problem.

Any thoughts on that @SiobhyB? I'm unsure of the pitfalls of that logic when the integer is 0. IMO we shouldn't let it hold up this PR, so I'm OK with noting it for a subsequent fix.

[twstokes]

Dropping a note about this fix that was merged in the RN codebase: facebook/react-native#37188.

[SiobhyB]

@twstokes, thank you so much for all of your help on this PR. 🙇‍♀️ I'll go ahead to merge as-is, so that the main fix is with users for the next 22.6 release, but have created an issue outlining the cursor issues at #51227. I've assigned myself and will work on it during HACK week next week.

Dropping a note about this fix that was merged in the RN codebase: facebook/react-native#37188.

Awesome! Looking over the guidance around when fixes get included in RN releases, it seems this might be included in the 0.72.0 release. I'll set myself a reminder to keep checking back in on the status over the next couple of months, so we can remove the workaround all together at some point.