Repository navigation
Releases
Releases · ruby/prism
v2.1.0
Added
- Add
Version::CRuby4_0to the Rust bindings.Version::CRuby3_5remains available but is deprecated in favor of the new name; both select Ruby 4.0 syntax. - Let embedders set the allocation size for the first block allocated in an arena via
PM_ARENA_INITIAL_SIZE. - Let embedders set the minimum initial size for the constant pool via
PM_CONSTANT_POOL_MIN_CAPACITY.
v2.0.0
Highlights
- Significant performance and memory usage improvements.
- Precompiled platform-specific variants on rubygems.org.
- Many documentation improvements, including a new playground.
- Prism now ships accurate and complete RBS and RBI definitions.
- The license is now owned by the Ruby developers.
- Prism now ships an official C++ interface.
Added
- Support for parsing Ruby 4.1 syntax:
- Add support for
&nilto declare that a method accepts no block. - Thoroughly implement void value expression check.
- Allow trailing commas in method signatures like
def foo(a,); end. - Reject
END { break }andEND { next }. a /b/no longer warns about ambiguous/.
- Add support for
version: "nearest"is now supported. It behaves similarly toversion: "current"but will not raise when the version is unsupported, instead falling back to the nearest syntax version prism currently supports.raise_errorwas added as a valid option. Prism will raise aSyntaxErrorwhen given syntax-invalid code. Whentrueis given, prism decides the formatting (tty?,NO_COLOR, etc.). If you need a specific style,:plain,:style, and:colorare also supported values.- Added
Prism::ParseResult#continuableto check if the parsed source can become valid by appending more code. Prism.findtakes aMethod,UnboundMethod,Proc, orThread::Backtrace::Locationand returns the associated prism AST node.ForwardingSuperNodenow haskeyword_loc.- Allow pattern matching with prism nodes to match on location slices, like
node in Prism::StringNode[content: "foo"]. Prism.parse_fileand similar now accept anything responding toto_path(likePathname) directly.attach_commentswas added as a valid option. Each comment is attached to the node that ends latest on the comment's line before it, otherwise to the innermost node that contains it (or to that node's statements, if only whitespace and comments come between the comment and the statements), otherwise to the top-level statements. The attached comments are available throughNode#commentsand throughpm_node_comments_sizeandpm_node_commentin the C API, as well aspm_parser_comment_attachment,pm_parser_comment_ranges_size, andpm_parser_comment_range, which expose the attachments grouped by node. When the option is set, the comments on the parse result are sorted by start offset.StringQuery#keyword?was added to check if the string query matches a keyword.Source#checksumwas added to retrieve the checksum of the source content.
Changed
Note that for this major version, almost every change listed below is breaking.
- Prism now defaults to
version: "nearest"instead ofversion: "latest", so code is parsed with the syntax of the running Ruby (or the closest version prism supports). Passversion: "latest"to keep the previous behavior. MissingNodehas been replaced byErrorRecoveryNode.ErrorRecoveryNodewraps unexpected nodes, so when walking syntax-invalid code you will now only need to handle this one specific node type.- Some fields were renamed for consistency. Previous names are aliased and will continue to work in the Ruby layer.
InNode#{in,then}_loc->InNode#{in,then}_keyword_locMatchPredicateNode#operator_loc->MatchPredicateNode#keyword_locUnlessNode#keyword_loc->UnlessNode#unless_keyword_locUntilNode#{keyword,closing}_loc->UntilNode#{until,end}_keyword_locWhenNode#keyword_loc->WhenNode#when_keyword_locWhileNode#{keyword,closing}_loc->WhileNode#{while,end}_keyword_loc
- Optional locations like
StringNode#closing_locin'foothat were empty for syntax-invalid code will now benilinstead. Prism.lexnow returns an array of tokens instead of an array of[token, internal_parse_state]. Previous access liketoken[0]ortoken.firstto extract the actual token will continue to work. Additionally the following tokens were changed:PARENTHESIS_LEFTwas split up intoPARENTHESIS_LEFTandPARENTHESIS_LEFT_GROUPING.BRACE_LEFTwas split up intoBRACE_LEFT,BRACE_LEFT_ARGUMENT, andBRACE_LEFT_HASH.BACKTICKwas split up intoBACKTICKandXSTRING_BACKTICK.NEWLINEwas split up intoNEWLINEandNEWLINE_TERMINATOR.WORDS_SEPwas split up intoWORDS_SEPandWORDS_SEP_IMPLICIT.
Ruby
- Require arguments for
Prism::Source.for. The second and third positional parameters are no longer optional. - Comments now live on the source.
Prism::Source#commentsreturns them andResult#commentsdelegates to it, soParseResult.new,LexResult.new, andParseLexResult.newno longer take acommentsargument. - Comment attachment now happens during parsing with the
attach_commentsoption instead of afterward withParseResult#attach_comments!, which was removed.Node#commentsnow returns the comments attached to the node by that option.Location#comments,Location#leading_comments,Location#trailing_comments,Node#leading_comments,Node#trailing_comments, and the comment fields ofPrism::Relocationwere removed.
C
- Most structs are now opaque (except for nodes), and many internal functions have been hidden.
- Locations are now represented using slices (pairs of 32-bit integers for start offset and length).
- Most allocations are now performed through an arena.
- You may now build side tables of comment attachments through the
attach_commentsoption. newline_listwas renamed tolist_offset_list.- Visitor callbacks passed to
pm_visit_nodeandpm_visit_child_nodesnow return apm_visit_result_tinstead of abool. ReturningPM_VISIT_DESCENDorPM_VISIT_SKIPreplaces returningtrueorfalse, and returningPM_VISIT_STOPends the walk. Both functions now returnfalseif the walk was stopped. pm_string_tis now a view of bytes owned by someone else. It no longer has atypefield,pm_string_owned_initwas removed, andpm_string_constant_initwas renamed topm_string_init, which takes the bytes asconst uint8_t *.pm_options_freeno longer frees the names of scope locals. The parser copies them when it is created, so they only need to remain valid until then.
Rust
- Locations are now represented using slices (pairs of 32-bit integers for start offset and length).
- Most allocations are now performed through an arena.
newline_listwas renamed tolist_offset_list.Diagnostic::messagenow returns&[u8]instead of&str.StringQueryis now exposed.
JavaScript
- The parse method now only accepts
Uint8Array, instead of forcing a transcoding to UTF-16.
Removed
- Removed
Prism::Pack.
v2.0.0.rc2
Highlights
- Significant performance and memory usage improvements.
- Precompiled platform-specific variants on rubygems.org.
- Many documentation improvements, including a new playground.
- Prism now ships accurate and complete RBS and RBI definitions.
- The license is now owned by the Ruby developers.
Added
- Support for parsing Ruby 4.1 syntax:
- Add support for
&nilto declare that a method accepts no block. - Thoroughly implement void value expression check.
- Allow trailing commas in method signatures like
def foo(a,); end. - Reject
END { break }andEND { next }. a /b/no longer warns about ambiguous/.
- Add support for
version: "nearest"is now supported. It behaves similarly toversion: "current"but will not raise when the version is unsupported, instead falling back to the nearest syntax version prism currently supports.raise_errorwas added as a valid option. Prism will raise aSyntaxErrorwhen given syntax-invalid code. Whentrueis given, prism decides the formatting (tty?,NO_COLOR, etc.). If you need a specific style,:plain,:style, and:colorare also supported values.- Added
Prism::ParseResult#continuableto check if the parsed source can become valid by appending more code. Prism.findtakes aMethod,UnboundMethod,Proc, orThread::Backtrace::Locationand returns the associated prism AST node.ForwardingSuperNodenow haskeyword_loc.- Allow pattern matching with prism nodes to match on location slices, like
node in Prism::StringNode[content: "foo"]. Prism.parse_fileand similar now accept anything responding toto_path(likePathname) directly.attach_commentswas added as a valid option. Each comment is attached to the node that ends latest on the comment's line before it, otherwise to the innermost node that contains it (or to that node's statements, if only whitespace and comments come between the comment and the statements), otherwise to the top-level statements. The attached comments are available throughNode#commentsand throughpm_node_comments_sizeandpm_node_commentin the C API, as well aspm_parser_comment_attachment,pm_parser_comment_ranges_size, andpm_parser_comment_range, which expose the attachments grouped by node. When the option is set, the comments on the parse result are sorted by start offset.StringQuery#keyword?was added to check if the string query matches a keyword.Source#checksumwas added to retrieve the checksum of the source content.
Changed
Note that for this major version, almost every change listed below is breaking.
- Prism now defaults to
version: "nearest"instead ofversion: "latest", so code is parsed with the syntax of the running Ruby (or the closest version prism supports). Passversion: "latest"to keep the previous behavior. MissingNodehas been replaced byErrorRecoveryNode.ErrorRecoveryNodewraps unexpected nodes, so when walking syntax-invalid code you will now only need to handle this one specific node type.- Some fields were renamed for consistency. Previous names are aliased and will continue to work in the Ruby layer.
InNode#{in,then}_loc->InNode#{in,then}_keyword_locMatchPredicateNode#operator_loc->MatchPredicateNode#keyword_locUnlessNode#keyword_loc->UnlessNode#unless_keyword_locUntilNode#{keyword,closing}_loc->UntilNode#{until,end}_keyword_locWhenNode#keyword_loc->WhenNode#when_keyword_locWhileNode#{keyword,closing}_loc->WhileNode#{while,end}_keyword_loc
- Optional locations like
StringNode#closing_locin'foothat were empty for syntax-invalid code will now benilinstead. Prism.lexnow returns an array of tokens instead of an array of[token, internal_parse_state]. Previous access liketoken[0]ortoken.firstto extract the actual token will continue to work. Additionally the following tokens were changed:PARENTHESIS_LEFTwas split up intoPARENTHESIS_LEFTandPARENTHESIS_LEFT_GROUPING.BRACE_LEFTwas split up intoBRACE_LEFT,BRACE_LEFT_ARGUMENT, andBRACE_LEFT_HASH.BACKTICKwas split up intoBACKTICKandXSTRING_BACKTICK.NEWLINEwas split up intoNEWLINEandNEWLINE_TERMINATOR.WORDS_SEPwas split up intoWORDS_SEPandWORDS_SEP_IMPLICIT.
Ruby
- Require arguments for
Prism::Source.for. The second and third positional parameters are no longer optional. - Comments now live on the source.
Prism::Source#commentsreturns them andResult#commentsdelegates to it, soParseResult.new,LexResult.new, andParseLexResult.newno longer take acommentsargument. - Comment attachment now happens during parsing with the
attach_commentsoption instead of afterward withParseResult#attach_comments!, which was removed.Node#commentsnow returns the comments attached to the node by that option.Location#comments,Location#leading_comments,Location#trailing_comments,Node#leading_comments,Node#trailing_comments, and the comment fields ofPrism::Relocationwere removed.
C
- Most structs are now opaque (except for nodes), and many internal functions have been hidden.
- Locations are now represented using slices (pairs of 32-bit integers for start offset and length).
- Most allocations are now performed through an arena.
- You may now build side tables of comment attachments through the
attach_commentsoption. newline_listwas renamed tolist_offset_list.
Rust
- Locations are now represented using slices (pairs of 32-bit integers for start offset and length).
- Most allocations are now performed through an arena.
newline_listwas renamed tolist_offset_list.Diagnostic::messagenow returns&[u8]instead of&str.StringQueryis now exposed.
JavaScript
- The parse method now only accepts
Uint8Array, instead of forcing a transcoding to UTF-16.
Removed
- Removed
Prism::Pack.
v2.0.0.rc1
Highlights
- Significant performance and memory usage improvements.
- Precompiled platform-specific variants on rubygems.org.
- Many documentation improvements, including a new playground.
- Prism now ships accurate and complete RBS and RBI definitions.
Added
- Support for parsing Ruby 4.1 syntax:
- Add support for
&nilto declare that a method accepts no block. - Thoroughly implement void value expression check.
- Allow trailing commas in method signatures like
def foo(a,); end. - Reject
END { break }andEND { next }. a /b/no longer warns about ambiguous/.
- Add support for
version: "nearest"is now supported. It behaves similarly toversion: "current"but will not raise when the version is unsupported, instead falling back to the nearest syntax version prism currently supports.raise_errorwas added as a valid option. Prism will raise aSyntaxErrorwhen given syntax-invalid code. Whentrueis given, prism decides the formatting (tty?,NO_COLOR, etc.). If you need a specific style,:plain,:style, and:colorare also supported values.- Added
Prism::ParseResult#continuableto check if the parsed source can become valid by appending more code. Prism.findtakes aMethod,UnboundMethod,Proc, orThread::Backtrace::Locationand returns the associated prism AST node.ForwardingSuperNodenow haskeyword_loc.- Allow pattern matching with prism nodes to match on location slices, like
node in Prism::StringNode[content: "foo"]. Prism.parse_fileand similar now accept anything responding toto_path(likePathname) directly.attach_commentswas added as a valid option. Each comment is attached to the node that ends latest on the comment's line before it, otherwise to the innermost node that contains it (or to that node's statements, if only whitespace and comments come between the comment and the statements), otherwise to the top-level statements. The attached comments are available throughNode#commentsand throughpm_node_comments_sizeandpm_node_commentin the C API, as well aspm_parser_comment_attachment,pm_parser_comment_ranges_size, andpm_parser_comment_range, which expose the attachments grouped by node. When the option is set, the comments on the parse result are sorted by start offset.
Changed
Note that for this major version, almost every change listed below is breaking.
- Prism now defaults to
version: "nearest"instead ofversion: "latest", so code is parsed with the syntax of the running Ruby (or the closest version prism supports). Passversion: "latest"to keep the previous behavior. MissingNodehas been replaced byErrorRecoveryNode.ErrorRecoveryNodewraps unexpected nodes, so when walking syntax-invalid code you will now only need to handle this one specific node type.- Some fields were renamed for consistency. Previous names are aliased and will continue to work in the Ruby layer.
InNode#{in,then}_loc->InNode#{in,then}_keyword_locMatchPredicateNode#operator_loc->MatchPredicateNode#keyword_locUnlessNode#keyword_loc->UnlessNode#unless_keyword_locUntilNode#{keyword,closing}_loc->UntilNode#{until,end}_keyword_locWhenNode#keyword_loc->WhenNode#when_keyword_locWhileNode#{keyword,closing}_loc->WhileNode#{while,end}_keyword_loc
- Optional locations like
StringNode#closing_locin'foothat were empty for syntax-invalid code will now benilinstead. Prism.lexnow returns an array of tokens instead of an array of[token, internal_parse_state]. Previous access liketoken[0]ortoken.firstto extract the actual token will continue to work. Additionally the following tokens were changed:PARENTHESIS_LEFTwas split up intoPARENTHESIS_LEFTandPARENTHESIS_LEFT_GROUPING.BRACE_LEFTwas split up intoBRACE_LEFT,BRACE_LEFT_ARGUMENT, andBRACE_LEFT_HASH.BACKTICKwas split up intoBACKTICKandXSTRING_BACKTICK.NEWLINEwas split up intoNEWLINEandNEWLINE_TERMINATOR.WORDS_SEPwas split up intoWORDS_SEPandWORDS_SEP_IMPLICIT.
Ruby
- Require arguments for
Prism::Source.for. The second and third positional parameters are no longer optional. - Comments now live on the source.
Prism::Source#commentsreturns them andResult#commentsdelegates to it, soParseResult.new,LexResult.new, andParseLexResult.newno longer take acommentsargument. - Comment attachment now happens during parsing with the
attach_commentsoption instead of afterward withParseResult#attach_comments!, which was removed.Node#commentsnow returns the comments attached to the node by that option.Location#comments,Location#leading_comments,Location#trailing_comments,Node#leading_comments,Node#trailing_comments, and the comment fields ofPrism::Relocationwere removed.
C
- Most structs are now opaque (except for nodes), and many internal functions have been hidden.
- Locations are now represented using slices (pairs of 32-bit integers for start offset and length).
- Most allocations are now performed through an arena.
- You may now build side tables of comment attachments through the
attach_commentsoption. newline_listwas renamed tolist_offset_list.
Rust
- Locations are now represented using slices (pairs of 32-bit integers for start offset and length).
- Most allocations are now performed through an arena.
newline_listwas renamed tolist_offset_list.Diagnostic::messagenow returns&[u8]instead of&str.
JavaScript
- The parse method now only accepts
Uint8Array, instead of forcing a transcoding to UTF-16.
Removed
- Removed
Prism::Pack.
v1.8.1
v1.9.0
Added
- Lots of work on the Ripper translation layer to make it more compatible and efficient.
- Alias
Prism::Node#breadth_first_searchtoPrism::Node#find. - Add
Prism::Node#breadth_first_search_all/Prism::Node#find_allfor finding all nodes matching a condition.
Changed
- Fixed location of opening tokens when invalid syntax is parsed.
- Fix RBI for parsing options.
v1.8.0
Added
- Optimize ruby visitor.
- Report unterminated construct errors at opening token.
Changed
- Correctly expose ripper state.
- Use one file for versioned parser classes.
- Fix denominator of rational float literal.
- Decouple ripper translator from ripper library.
- Sync Prism::Translation::ParserCurrent with Ruby 4.0.
v1.7.0
Added
- Support
4.1as a version option. - Add
equal_loctoCallNode. - Add
len()/is_empty()toConstantListandNodeListin the Rust API.
Changed
- Rename version
3.5to version4.0. - Fix compiling the gem from source on Windows.
- Fix parsing of unary method calls like
42.~@. - Reject
def f a, (b) = 1. - Reject endless method as a block parameter default.
- Reject variable capture in alternative pattern.
- Many fixes in regards to memory safety, found through fuzzing.
- Many fixes to better handle invalid syntax, also found through fuzzing.
- Fix the ruby version used by the
rippertranslator. - Fix
ruby_parsertranslation comment processing.