Invalid language object with tree-sitter v0.21 - caused by NAPI?

[rien]

I tried updating to tree-sitter 0.21 for our project https://github.com/dodona-edu/dolos/tree/main/parsers which builds a collection of parsers to be used in our tool. We do this by importing each parser as a submodule and using a simple binding.gyp including the gyp-files of each parser.

This worked fine with the node package of tree-sitter 0.20.6. With the latest release 0.21.0, building using node-gyp succeeds, but actually using the parser fails.

Consider the following snippet:

import { java } from "@dodona/dolos-parsers";
import { default as Parser } from "tree-sitter";

const parser = new Parser();
parser.setLanguage(java);
// throws Uncaught TypeError: Invalid language object
//    at Parser.setLanguage (/home/rien/repos/dolos/node_modules/tree-sitter/index.js:338:17)

Which is thrown on line 338 in index.js, calling the setLanguage function from the native tree-sitter_runtime binding:

node-tree-sitter/index.js

Lines 329 to 345 in 2bdc76a

	/*
	* Parser
	*/
	const {parse, setLanguage} = Parser.prototype;
	const languageSymbol = Symbol('parser.language');
	Parser.prototype.setLanguage = function(language) {
	if (this instanceof Parser && setLanguage) {
	setLanguage.call(this, language);
	}
	this[languageSymbol] = language;
	if (!language.nodeSubclasses) {
	initializeLanguageNodeClasses(language)
	}
	return this;
	};

I noticed in the commits between v0.20.6...v0.21.0 that the bindings were changed to NAPI instead of Node. I guess this might be caused by this refactor?

Does anything need to be changed to the parsers themselves to work with NAPI? Is this something that I could fix in my binding.gyp or will this require changes in the parsers?

[gustavotoyota]

Same here. All languages I've tested (JS, TS and TSX) failed to load with "Invalid language object". Downgrading to 0.20.6 solved the problem.

[verhovsky]

With node-tree-sitter 0.21.0, the bindings switch to NAPI instead of NaN #190, so you need to change the binding.cc file of all the languages you want to use with node-tree-sitter 0.21.0+ to be a NAPI binding.cc file. If you have just a normal binding.cc right now then you can update your tree-sitter CLI (npm install -g tree-sitter-cli or brew upgrade tree-sitter on macOS) then go to that language's repo/directory and regenerate it like this

tree-sitter generate

If you've made changes to your language's binding.cc file previously, you will have to port those changes to NAPI after re-generating the file. This command also now generates more bindings for other programming languages, by the way, which you may or may not want to keep. This has been done for some languages in the tree-sitter org, you can see how it was done for tree-sitter-python or for tree-sitter-bash. Like I said, it's just re-running tree-sitter generate and re-compiling.

By the way, with 0.21 there's also a handful of backwards incompatible changes to make the Node and Wasm bindings interchangeable (crazy, but they weren't before), namely in Node SyntaxNode.hasChanges, .hasError and .isMissing become properties instead of methods. See tree-sitter/tree-sitter#3103

[segevfiner]

I started creating a module that should allow loading older grammars using the new node-tree-sitter as a stop-gap solution until they all get updated https://github.com/segevfiner/node-tree-sitter-compat, haven't published it to npm just yet, but I likely will.

[segevfiner]

Published https://www.npmjs.com/package/tree-sitter-compat