Skip to content

Migrate Python to use next-gen scope handlers for text fragment extractors - #1862

Merged
pokey merged 9 commits into
cursorless-dev:mainfrom
saidelike:migrate-python-scope-string
Sep 9, 2023
Merged

pokey merged 9 commits into
cursorless-dev:mainfrom
saidelike:migrate-python-scope-string

Conversation

@saidelike

@saidelike saidelike commented Sep 7, 2023 •

Copy link
Copy Markdown
Collaborator

Checklist

  • I have added tests
  • [/] I have updated the docs and cheatsheet
  • [/] I have not broken the cheatsheet
  • merge migrate Python scope 'comment' #1858 first as seems required
  • @pokey to fix a bug in the "processSurroundingPairCore" function atm that makes it not working atm (see commit message). It will be handled in a different commit before we can take this PR into account
  • record a test for the fix by @pokey by saying "cursorless record" and using "change string"

This PR also includes a few additional unit tests for Python strings.

Cedric Halbronn added 2 commits September 7, 2023 12:19
It is not fully working yet due to a bug in the original parser in processSurroundingPairCore() where we get returned a text based instead of tree based element that will be dealt by Pokey

Current limitation:

'''abc''' with cursor on b and "take string" gives us 'abc' (one quote included only)
'''abc''' with cursor on first quote and "take string" gives us '' (first and second quote)
@saidelike
saidelike requested a review from pokey as a code owner September 7, 2023 11:28
@pokey pokey changed the title Migrate python scope string Migrate Python to use next-gen scope handlers for text fragment extractors Sep 7, 2023
@pokey pokey mentioned this pull request Sep 7, 2023
3 tasks
@pokey
pokey force-pushed the migrate-python-scope-string branch from cfeb2a3 to 5565b66 Compare September 7, 2023 12:38
@pokey pokey mentioned this pull request Sep 7, 2023
2 tasks done
@saidelike

Copy link
Copy Markdown
Collaborator Author

I have confirmed that with this and #1863 that I can take string air and it works as expected (eg on triple quote strings)

github-merge-queue Bot pushed a commit that referenced this pull request Sep 8, 2023
The fallback logic for determining whether to use text-based surrounding
pairs or parse-tree-based was broken for text fragment extractors based
on next-gen scope handlers (ie using the new `@textFragment` tag in a
query file).

This breakage meant that in a string like `"""hello"""` in Python, the
surrounding pair finder fell back to text-based, which resulted in it
thinking `"hello"` was a string, rather than `"""hello"""`, as would
happen if it fell back to parse-tree-based

This PR fixes the issue by unifying the fallback logic between legacy
and next-gen text fragment extractors, so that they don't fall out of
sync again

See also
#1812 (comment);
arguably that will make this PR irrelevant, but until then, it's better
to have string not be broken when we migrate a language to use
`@textFragment`

Note that the tests in this PR don't actually test the code yet, because
Python is still using legacy text fragment extractors. The tests will
start biting when this PR is merged into main and then merged into
#1862

- This PR is required by
#1862

## Checklist

- [ ] I have added
[tests](https://www.cursorless.org/docs/contributing/test-case-recorder/)
- [ ] I have updated the
[docs](https://github.com/cursorless-dev/cursorless/tree/main/docs) and
[cheatsheet](https://github.com/cursorless-dev/cursorless/tree/main/cursorless-talon/src/cheatsheet)
- [ ] I have not broken the cheatsheet

---------

Co-authored-by: Andreas Arvidsson <andreas.arvidsson87@gmail.com>
@pokey pokey added lang-python Issues related to Python programming language support scope-migration Migrating scopes to next-gen scope implementation labels Sep 9, 2023

@pokey pokey left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! I removed some unnecessary parens in d303419 fwiw

@pokey
pokey enabled auto-merge September 9, 2023 08:53
@pokey
pokey added this pull request to the merge queue Sep 9, 2023
Merged via the queue into cursorless-dev:main with commit 0b50c4a Sep 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lang-python Issues related to Python programming language support scope-migration Migrating scopes to next-gen scope implementation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants