Skip to content

Releases

v2.1.0

Choose a tag to compare

@github-actions github-actions released this 28 Oct 05:19

Added

  • Add Version::CRuby4_0 to the Rust bindings. Version::CRuby3_5 remains available but is deprecated in favor of the new name; both select Ruby 4.0 syntax.
  • Let embedders set the allocation size for the first block allocated in an arena via PM_ARENA_INITIAL_SIZE.
  • Let embedders set the minimum initial size for the constant pool via PM_CONSTANT_POOL_MIN_CAPACITY.

v2.0.0

Choose a tag to compare

@github-actions github-actions released this 21 Oct 14:48

Highlights

  • Significant performance and memory usage improvements.
  • Precompiled platform-specific variants on rubygems.org.
  • Many documentation improvements, including a new playground.
  • Prism now ships accurate and complete RBS and RBI definitions.
  • The license is now owned by the Ruby developers.
  • Prism now ships an official C++ interface.

Added

  • Support for parsing Ruby 4.1 syntax:
    • Add support for &nil to declare that a method accepts no block.
    • Thoroughly implement void value expression check.
    • Allow trailing commas in method signatures like def foo(a,); end.
    • Reject END { break } and END { next }.
    • a /b/ no longer warns about ambiguous /.
  • version: "nearest" is now supported. It behaves similarly to version: "current" but will not raise when the version is unsupported, instead falling back to the nearest syntax version prism currently supports.
  • raise_error was added as a valid option. Prism will raise a SyntaxError when given syntax-invalid code. When true is given, prism decides the formatting (tty?, NO_COLOR, etc.). If you need a specific style, :plain, :style, and :color are also supported values.
  • Added Prism::ParseResult#continuable to check if the parsed source can become valid by appending more code.
  • Prism.find takes a Method, UnboundMethod, Proc, or Thread::Backtrace::Location and returns the associated prism AST node.
  • ForwardingSuperNode now has keyword_loc.
  • Allow pattern matching with prism nodes to match on location slices, like node in Prism::StringNode[content: "foo"].
  • Prism.parse_file and similar now accept anything responding to to_path (like Pathname) directly.
  • attach_comments was added as a valid option. Each comment is attached to the node that ends latest on the comment's line before it, otherwise to the innermost node that contains it (or to that node's statements, if only whitespace and comments come between the comment and the statements), otherwise to the top-level statements. The attached comments are available through Node#comments and through pm_node_comments_size and pm_node_comment in the C API, as well as pm_parser_comment_attachment, pm_parser_comment_ranges_size, and pm_parser_comment_range, which expose the attachments grouped by node. When the option is set, the comments on the parse result are sorted by start offset.
  • StringQuery#keyword? was added to check if the string query matches a keyword.
  • Source#checksum was added to retrieve the checksum of the source content.

Changed

Note that for this major version, almost every change listed below is breaking.

  • Prism now defaults to version: "nearest" instead of version: "latest", so code is parsed with the syntax of the running Ruby (or the closest version prism supports). Pass version: "latest" to keep the previous behavior.
  • MissingNode has been replaced by ErrorRecoveryNode. ErrorRecoveryNode wraps unexpected nodes, so when walking syntax-invalid code you will now only need to handle this one specific node type.
  • Some fields were renamed for consistency. Previous names are aliased and will continue to work in the Ruby layer.
    • InNode#{in,then}_loc -> InNode#{in,then}_keyword_loc
    • MatchPredicateNode#operator_loc -> MatchPredicateNode#keyword_loc
    • UnlessNode#keyword_loc -> UnlessNode#unless_keyword_loc
    • UntilNode#{keyword,closing}_loc -> UntilNode#{until,end}_keyword_loc
    • WhenNode#keyword_loc -> WhenNode#when_keyword_loc
    • WhileNode#{keyword,closing}_loc -> WhileNode#{while,end}_keyword_loc
  • Optional locations like StringNode#closing_loc in 'foo that were empty for syntax-invalid code will now be nil instead.
  • Prism.lex now returns an array of tokens instead of an array of [token, internal_parse_state]. Previous access like token[0] or token.first to extract the actual token will continue to work. Additionally the following tokens were changed:
    • PARENTHESIS_LEFT was split up into PARENTHESIS_LEFT and PARENTHESIS_LEFT_GROUPING.
    • BRACE_LEFT was split up into BRACE_LEFT, BRACE_LEFT_ARGUMENT, and BRACE_LEFT_HASH.
    • BACKTICK was split up into BACKTICK and XSTRING_BACKTICK.
    • NEWLINE was split up into NEWLINE and NEWLINE_TERMINATOR.
    • WORDS_SEP was split up into WORDS_SEP and WORDS_SEP_IMPLICIT.

Ruby

  • Require arguments for Prism::Source.for. The second and third positional parameters are no longer optional.
  • Comments now live on the source. Prism::Source#comments returns them and Result#comments delegates to it, so ParseResult.new, LexResult.new, and ParseLexResult.new no longer take a comments argument.
  • Comment attachment now happens during parsing with the attach_comments option instead of afterward with ParseResult#attach_comments!, which was removed. Node#comments now returns the comments attached to the node by that option. Location#comments, Location#leading_comments, Location#trailing_comments, Node#leading_comments, Node#trailing_comments, and the comment fields of Prism::Relocation were removed.

C

  • Most structs are now opaque (except for nodes), and many internal functions have been hidden.
  • Locations are now represented using slices (pairs of 32-bit integers for start offset and length).
  • Most allocations are now performed through an arena.
  • You may now build side tables of comment attachments through the attach_comments option.
  • newline_list was renamed to list_offset_list.
  • Visitor callbacks passed to pm_visit_node and pm_visit_child_nodes now return a pm_visit_result_t instead of a bool. Returning PM_VISIT_DESCEND or PM_VISIT_SKIP replaces returning true or false, and returning PM_VISIT_STOP ends the walk. Both functions now return false if the walk was stopped.
  • pm_string_t is now a view of bytes owned by someone else. It no longer has a type field, pm_string_owned_init was removed, and pm_string_constant_init was renamed to pm_string_init, which takes the bytes as const uint8_t *.
  • pm_options_free no longer frees the names of scope locals. The parser copies them when it is created, so they only need to remain valid until then.

Rust

  • Locations are now represented using slices (pairs of 32-bit integers for start offset and length).
  • Most allocations are now performed through an arena.
  • newline_list was renamed to list_offset_list.
  • Diagnostic::message now returns &[u8] instead of &str.
  • StringQuery is now exposed.

JavaScript

  • The parse method now only accepts Uint8Array, instead of forcing a transcoding to UTF-16.

Removed

  • Removed Prism::Pack.

v2.0.0.rc2

v2.0.0.rc2 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 20 Oct 01:47

Highlights

  • Significant performance and memory usage improvements.
  • Precompiled platform-specific variants on rubygems.org.
  • Many documentation improvements, including a new playground.
  • Prism now ships accurate and complete RBS and RBI definitions.
  • The license is now owned by the Ruby developers.

Added

  • Support for parsing Ruby 4.1 syntax:
    • Add support for &nil to declare that a method accepts no block.
    • Thoroughly implement void value expression check.
    • Allow trailing commas in method signatures like def foo(a,); end.
    • Reject END { break } and END { next }.
    • a /b/ no longer warns about ambiguous /.
  • version: "nearest" is now supported. It behaves similarly to version: "current" but will not raise when the version is unsupported, instead falling back to the nearest syntax version prism currently supports.
  • raise_error was added as a valid option. Prism will raise a SyntaxError when given syntax-invalid code. When true is given, prism decides the formatting (tty?, NO_COLOR, etc.). If you need a specific style, :plain, :style, and :color are also supported values.
  • Added Prism::ParseResult#continuable to check if the parsed source can become valid by appending more code.
  • Prism.find takes a Method, UnboundMethod, Proc, or Thread::Backtrace::Location and returns the associated prism AST node.
  • ForwardingSuperNode now has keyword_loc.
  • Allow pattern matching with prism nodes to match on location slices, like node in Prism::StringNode[content: "foo"].
  • Prism.parse_file and similar now accept anything responding to to_path (like Pathname) directly.
  • attach_comments was added as a valid option. Each comment is attached to the node that ends latest on the comment's line before it, otherwise to the innermost node that contains it (or to that node's statements, if only whitespace and comments come between the comment and the statements), otherwise to the top-level statements. The attached comments are available through Node#comments and through pm_node_comments_size and pm_node_comment in the C API, as well as pm_parser_comment_attachment, pm_parser_comment_ranges_size, and pm_parser_comment_range, which expose the attachments grouped by node. When the option is set, the comments on the parse result are sorted by start offset.
  • StringQuery#keyword? was added to check if the string query matches a keyword.
  • Source#checksum was added to retrieve the checksum of the source content.

Changed

Note that for this major version, almost every change listed below is breaking.

  • Prism now defaults to version: "nearest" instead of version: "latest", so code is parsed with the syntax of the running Ruby (or the closest version prism supports). Pass version: "latest" to keep the previous behavior.
  • MissingNode has been replaced by ErrorRecoveryNode. ErrorRecoveryNode wraps unexpected nodes, so when walking syntax-invalid code you will now only need to handle this one specific node type.
  • Some fields were renamed for consistency. Previous names are aliased and will continue to work in the Ruby layer.
    • InNode#{in,then}_loc -> InNode#{in,then}_keyword_loc
    • MatchPredicateNode#operator_loc -> MatchPredicateNode#keyword_loc
    • UnlessNode#keyword_loc -> UnlessNode#unless_keyword_loc
    • UntilNode#{keyword,closing}_loc -> UntilNode#{until,end}_keyword_loc
    • WhenNode#keyword_loc -> WhenNode#when_keyword_loc
    • WhileNode#{keyword,closing}_loc -> WhileNode#{while,end}_keyword_loc
  • Optional locations like StringNode#closing_loc in 'foo that were empty for syntax-invalid code will now be nil instead.
  • Prism.lex now returns an array of tokens instead of an array of [token, internal_parse_state]. Previous access like token[0] or token.first to extract the actual token will continue to work. Additionally the following tokens were changed:
    • PARENTHESIS_LEFT was split up into PARENTHESIS_LEFT and PARENTHESIS_LEFT_GROUPING.
    • BRACE_LEFT was split up into BRACE_LEFT, BRACE_LEFT_ARGUMENT, and BRACE_LEFT_HASH.
    • BACKTICK was split up into BACKTICK and XSTRING_BACKTICK.
    • NEWLINE was split up into NEWLINE and NEWLINE_TERMINATOR.
    • WORDS_SEP was split up into WORDS_SEP and WORDS_SEP_IMPLICIT.

Ruby

  • Require arguments for Prism::Source.for. The second and third positional parameters are no longer optional.
  • Comments now live on the source. Prism::Source#comments returns them and Result#comments delegates to it, so ParseResult.new, LexResult.new, and ParseLexResult.new no longer take a comments argument.
  • Comment attachment now happens during parsing with the attach_comments option instead of afterward with ParseResult#attach_comments!, which was removed. Node#comments now returns the comments attached to the node by that option. Location#comments, Location#leading_comments, Location#trailing_comments, Node#leading_comments, Node#trailing_comments, and the comment fields of Prism::Relocation were removed.

C

  • Most structs are now opaque (except for nodes), and many internal functions have been hidden.
  • Locations are now represented using slices (pairs of 32-bit integers for start offset and length).
  • Most allocations are now performed through an arena.
  • You may now build side tables of comment attachments through the attach_comments option.
  • newline_list was renamed to list_offset_list.

Rust

  • Locations are now represented using slices (pairs of 32-bit integers for start offset and length).
  • Most allocations are now performed through an arena.
  • newline_list was renamed to list_offset_list.
  • Diagnostic::message now returns &[u8] instead of &str.
  • StringQuery is now exposed.

JavaScript

  • The parse method now only accepts Uint8Array, instead of forcing a transcoding to UTF-16.

Removed

  • Removed Prism::Pack.

v2.0.0.rc1

v2.0.0.rc1 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 15 Oct 16:04
184add3

Highlights

  • Significant performance and memory usage improvements.
  • Precompiled platform-specific variants on rubygems.org.
  • Many documentation improvements, including a new playground.
  • Prism now ships accurate and complete RBS and RBI definitions.

Added

  • Support for parsing Ruby 4.1 syntax:
    • Add support for &nil to declare that a method accepts no block.
    • Thoroughly implement void value expression check.
    • Allow trailing commas in method signatures like def foo(a,); end.
    • Reject END { break } and END { next }.
    • a /b/ no longer warns about ambiguous /.
  • version: "nearest" is now supported. It behaves similarly to version: "current" but will not raise when the version is unsupported, instead falling back to the nearest syntax version prism currently supports.
  • raise_error was added as a valid option. Prism will raise a SyntaxError when given syntax-invalid code. When true is given, prism decides the formatting (tty?, NO_COLOR, etc.). If you need a specific style, :plain, :style, and :color are also supported values.
  • Added Prism::ParseResult#continuable to check if the parsed source can become valid by appending more code.
  • Prism.find takes a Method, UnboundMethod, Proc, or Thread::Backtrace::Location and returns the associated prism AST node.
  • ForwardingSuperNode now has keyword_loc.
  • Allow pattern matching with prism nodes to match on location slices, like node in Prism::StringNode[content: "foo"].
  • Prism.parse_file and similar now accept anything responding to to_path (like Pathname) directly.
  • attach_comments was added as a valid option. Each comment is attached to the node that ends latest on the comment's line before it, otherwise to the innermost node that contains it (or to that node's statements, if only whitespace and comments come between the comment and the statements), otherwise to the top-level statements. The attached comments are available through Node#comments and through pm_node_comments_size and pm_node_comment in the C API, as well as pm_parser_comment_attachment, pm_parser_comment_ranges_size, and pm_parser_comment_range, which expose the attachments grouped by node. When the option is set, the comments on the parse result are sorted by start offset.

Changed

Note that for this major version, almost every change listed below is breaking.

  • Prism now defaults to version: "nearest" instead of version: "latest", so code is parsed with the syntax of the running Ruby (or the closest version prism supports). Pass version: "latest" to keep the previous behavior.
  • MissingNode has been replaced by ErrorRecoveryNode. ErrorRecoveryNode wraps unexpected nodes, so when walking syntax-invalid code you will now only need to handle this one specific node type.
  • Some fields were renamed for consistency. Previous names are aliased and will continue to work in the Ruby layer.
    • InNode#{in,then}_loc -> InNode#{in,then}_keyword_loc
    • MatchPredicateNode#operator_loc -> MatchPredicateNode#keyword_loc
    • UnlessNode#keyword_loc -> UnlessNode#unless_keyword_loc
    • UntilNode#{keyword,closing}_loc -> UntilNode#{until,end}_keyword_loc
    • WhenNode#keyword_loc -> WhenNode#when_keyword_loc
    • WhileNode#{keyword,closing}_loc -> WhileNode#{while,end}_keyword_loc
  • Optional locations like StringNode#closing_loc in 'foo that were empty for syntax-invalid code will now be nil instead.
  • Prism.lex now returns an array of tokens instead of an array of [token, internal_parse_state]. Previous access like token[0] or token.first to extract the actual token will continue to work. Additionally the following tokens were changed:
    • PARENTHESIS_LEFT was split up into PARENTHESIS_LEFT and PARENTHESIS_LEFT_GROUPING.
    • BRACE_LEFT was split up into BRACE_LEFT, BRACE_LEFT_ARGUMENT, and BRACE_LEFT_HASH.
    • BACKTICK was split up into BACKTICK and XSTRING_BACKTICK.
    • NEWLINE was split up into NEWLINE and NEWLINE_TERMINATOR.
    • WORDS_SEP was split up into WORDS_SEP and WORDS_SEP_IMPLICIT.

Ruby

  • Require arguments for Prism::Source.for. The second and third positional parameters are no longer optional.
  • Comments now live on the source. Prism::Source#comments returns them and Result#comments delegates to it, so ParseResult.new, LexResult.new, and ParseLexResult.new no longer take a comments argument.
  • Comment attachment now happens during parsing with the attach_comments option instead of afterward with ParseResult#attach_comments!, which was removed. Node#comments now returns the comments attached to the node by that option. Location#comments, Location#leading_comments, Location#trailing_comments, Node#leading_comments, Node#trailing_comments, and the comment fields of Prism::Relocation were removed.

C

  • Most structs are now opaque (except for nodes), and many internal functions have been hidden.
  • Locations are now represented using slices (pairs of 32-bit integers for start offset and length).
  • Most allocations are now performed through an arena.
  • You may now build side tables of comment attachments through the attach_comments option.
  • newline_list was renamed to list_offset_list.

Rust

  • Locations are now represented using slices (pairs of 32-bit integers for start offset and length).
  • Most allocations are now performed through an arena.
  • newline_list was renamed to list_offset_list.
  • Diagnostic::message now returns &[u8] instead of &str.

JavaScript

  • The parse method now only accepts Uint8Array, instead of forcing a transcoding to UTF-16.

Removed

  • Removed Prism::Pack.

v1.8.1

Choose a tag to compare

@Earlopain Earlopain released this 17 Mar 10:16

Changes

  • Fix not binding power in endless methods.
  • Correctly handle and? and similar on Ruby 4.0.
  • Fix error message for block/lambda with ... argument.
  • Fix in handling.

v1.9.0

Choose a tag to compare

@github-actions github-actions released this 28 Jan 10:56
c0e3781

Added

  • Lots of work on the Ripper translation layer to make it more compatible and efficient.
  • Alias Prism::Node#breadth_first_search to Prism::Node#find.
  • Add Prism::Node#breadth_first_search_all/Prism::Node#find_all for finding all nodes matching a condition.

Changed

  • Fixed location of opening tokens when invalid syntax is parsed.
  • Fix RBI for parsing options.

v1.8.0

Choose a tag to compare

@k0kubun k0kubun released this 13 Jan 01:56
90c0578

Added

  • Optimize ruby visitor.
  • Report unterminated construct errors at opening token.

Changed

  • Correctly expose ripper state.
  • Use one file for versioned parser classes.
  • Fix denominator of rational float literal.
  • Decouple ripper translator from ripper library.
  • Sync Prism::Translation::ParserCurrent with Ruby 4.0.

v1.7.0

Choose a tag to compare

@kddnewton kddnewton released this 18 Dec 15:58
df55403

Added

  • Support 4.1 as a version option.
  • Add equal_loc to CallNode.
  • Add len()/is_empty() to ConstantList and NodeList in the Rust API.

Changed

  • Rename version 3.5 to version 4.0.
  • Fix compiling the gem from source on Windows.
  • Fix parsing of unary method calls like 42.~@.
  • Reject def f a, (b) = 1.
  • Reject endless method as a block parameter default.
  • Reject variable capture in alternative pattern.
  • Many fixes in regards to memory safety, found through fuzzing.
  • Many fixes to better handle invalid syntax, also found through fuzzing.
  • Fix the ruby version used by the ripper translator.
  • Fix ruby_parser translation comment processing.

v1.6.0

Choose a tag to compare

@kddnewton kddnewton released this 16 Oct 12:44
2924f8f

Added

  • Add support for passing "current" as the version option to Prism.* APIs.

Changed

  • Remove a compiler warning for a missing unsigned cast for a shift value.

v1.5.2

Choose a tag to compare

@kddnewton kddnewton released this 09 Oct 13:33
5446f7b

Changed

  • Fix character literal forced encoding when a unicode escape sequence is used.
  • Reject 1 if foo = bar baz.
  • Clear static literal flag on interpolated strings.
  • Reject optional argument/endless method definition ambiguity.