1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141
|
// Copyright 2024 The Chromium Authors
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
#ifndef CHROME_RENDERER_ACCESSIBILITY_READ_ANYTHING_READ_ALOUD_TRAVERSAL_UTILS_H_
#define CHROME_RENDERER_ACCESSIBILITY_READ_ANYTHING_READ_ALOUD_TRAVERSAL_UTILS_H_
#include <string>
#include "ui/accessibility/ax_enums.mojom.h"
#include "ui/accessibility/ax_node_position.h"
// Utilities for traversing the accessibility tree for Read Aloud.
// TODO(crbug.com/346612365): Rename to util instead of utils.
// A current segment of text that will be consumed by Read Aloud.
struct ReadAloudTextSegment {
// The AXNodeID associated with this particular text segment.
ui::AXNodeID id;
// The starting index for the text with the node of the given id.
int text_start;
// The ending index for the text with the node of the given id.
int text_end;
};
namespace a11y {
// During text traversal when adding new text to the current speech segment,
// this is used to indicate the next traversal steps.
enum class TraversalState {
// The end of the current granularity segment.
EndOfSegment = 0,
// Traversal should continue to the next valid AXNode.
ContinueToNextNode = 1,
// Traversal should continue with text within the current node.
ContinueInCurrentNode = 2,
};
// A representation of multiple ReadAloudTextSegments that are processed
// by Read Aloud at a single moment. For example, when using sentence
// granularity, the list of ReadAloudTextSegments in a
// ReadAloudCurrentGranularity will include all ReadAloudTextSegments
// necessary to represent a single sentence.
struct ReadAloudCurrentGranularity {
ReadAloudCurrentGranularity();
ReadAloudCurrentGranularity(const ReadAloudCurrentGranularity& other);
~ReadAloudCurrentGranularity();
// Adds a node containing text to the current granularity.
void AddText(ui::AXNodeID id,
int text_start,
int text_end,
const std::u16string& text);
// For a given start..end range within `text`, returns a list of nodes and
// offsets corresponding to that range.
std::vector<ReadAloudTextSegment> GetSegmentsForRange(int start_index,
int end_index);
// Calculate placeholder phrase boundaries from the text (3 words per phrase).
void CalculatePlaceholderPhrases();
// Calculate the phrase_boundaries index corresponding to a text index.
int GetPhraseIndex(int index) {
return std::upper_bound(phrase_boundaries.begin(), phrase_boundaries.end(),
index) -
phrase_boundaries.begin() - 1;
}
// All of the ReadAloudTextSegments in the current granularity.
std::map<ui::AXNodeID, ReadAloudTextSegment> segments;
// Because GetCurrentText returns a vector of node ids to be used by
// TypeScript also store the node ids as a vector for easier retrieval.
std::vector<ui::AXNodeID> node_ids;
// Map of text start and end indices of text to a specific AXNodeID.
// The text for a given segment may span multiple AXNodes, such as
// Node 1: This is a
// Node 2: link
// Node 3: in a separate node.
// which is presented as a single segment when using sentence granularity.
// However, when we get word callbacks, we get them in terms of the text
// index across the entire segment of text, not by node. Therefore, this
// mapping helps us better parse callbacks for different types of
// granularity highlighting.
// TODO(b/40927698): Investigate using this to replace
// highlightedNodeToOffsetInParent in app.ts
std::map<std::pair<int, int>, ui::AXNodeID> index_map;
// The human readable text represented by this segment of node ids. This
// is stored separately for easier retrieval for non-sentence granularity
// highlighting.
std::u16string text;
// Tokens for phrase breaking. Usually a word, but can also be punctuations or
// numbers.
std::vector<std::u16string_view> tokens;
// Boundary indices for phrases. Starts at 0.
std::vector<int> phrase_boundaries;
// Whether phrases has been calculated based on the dependency heads generated
// by the model.
bool are_phrases_calculated = false;
};
} // namespace a11y
// Returns the index of the next sentence of the given text, such that the
// next sentence is equivalent to text.substr(0, <returned_index>).
int GetNextSentence(const std::u16string& text);
// Returns the index of the next word of the given text, such that the
// next word is equivalent to text.substr(0, <returned_index>).
int GetNextWord(const std::u16string& text);
// Returns true if both positions are non-null and equal.
bool ArePositionsEqual(const ui::AXNodePosition::AXPositionInstance& position,
const ui::AXNodePosition::AXPositionInstance& other);
// Returns the correct anchor node from an AXPositionInstance that should be
// used by Read Aloud. AXPosition can sometimes return leaf nodes that don't
// actually correspond to the AXNodes we're using in Reading Mode, so we need
// to get a parent node from the AXPosition's returned anchor.
ui::AXNode* GetAnchorNode(
const ui::AXNodePosition::AXPositionInstance& position);
// Returns if the given character can be considered opening puncutation.
// This is used to ensure we're not reading out opening punctuation
// as a separate segment.
bool IsOpeningPunctuation(char& c);
// Returns whether we should split the current utterance at a paragraph
// boundary.
bool ShouldSplitAtParagraph(
const ui::AXNodePosition::AXPositionInstance& position,
const a11y::ReadAloudCurrentGranularity current_granularity);
#endif // CHROME_RENDERER_ACCESSIBILITY_READ_ANYTHING_READ_ALOUD_TRAVERSAL_UTILS_H_
|