`TreeCursor::goto_previous_sibling` fails at child index 256, 512, 768, ...
AI Policy
- I have read the AI Policy and this issue complies with it.
Problem
cursor.goto_previous_sibling() fails when the child_index % 256 == 0. I imagine that the reason this went undetected in practice is that it is rare for internal child indices to be so large, due to the use of hidden nodes. It is observable when an ERROR node ends up with many children, or by having many extra nodes (like comments).
Bug found by automatically comparing behavior with my experimental alternative representation for Tree-sitter trees (written by AI of course). Test and fix written by codex+astra.
Steps to reproduce
Regression test and fix: https://github.com/tree-sitter/tree-sitter/compare/master...mgsloan:tree-sitter:fix-previous-sibling-index-underflow?expand=1
git fetch mgsloan fix-previous-sibling-index-underflow
git switch --detach 267f79926
cargo test -p tree-sitter-cli --lib test_tree_cursor_previous_sibling_with_many_commentsOr just copy paste the regression test:
#[test]
fn test_tree_cursor_previous_sibling_with_many_comments() {
let mut parser = Parser::new();
parser.set_language(&get_language("rust")).unwrap();
let text = "// comment\n".repeat(300);
let tree = parser.parse(&text, None).unwrap();
let root = tree.root_node();
assert!(!root.has_error());
let mut cursor = tree.walk();
let children: Vec<_> = root.children(&mut cursor).collect();
assert_eq!(children.len(), 300);
assert!(children.iter().all(|node| node.is_extra()));
cursor.reset(root);
assert!(cursor.goto_last_child());
for index in (0..children.len()).rev() {
assert_eq!(cursor.node(), children[index]);
assert_eq!(cursor.goto_previous_sibling(), index > 0, "child {index}");
}
}Observe that the fix works
git switch fix-previous-sibling-index-underflow
cargo test -p tree-sitter-cli --lib test_tree_cursor_previous_sibling_with_many_comments--- a/lib/src/tree_cursor.c
+++ b/lib/src/tree_cursor.c
@@ -124,7 +124,7 @@ static inline bool ts_tree_cursor_child_iterator_previous(
) {
// this is mostly a reverse `ts_tree_cursor_child_iterator_next` taking into
// account unsigned underflow
- if (!self->parent.ptr || (int8_t)self->child_index == -1) return false;
+ if (!self->parent.ptr || self->child_index == UINT32_MAX) return false;
const Subtree *child = &ts_subtree_children(self->parent)[self->child_index];
*result = (TreeCursorEntry) {
.subtree = child,Expected behavior
Should move to previous sibling, even when index % 256 == 0
Tree-sitter version (tree-sitter --version)
HEAD. Has probably been a bug for years
Operating system/version
Debian
Source: tree-sitter/tree-sitter