Repository navigation
Conversation
|
|
@Guts Please help review, thanks! |
Guts
left a comment
There was a problem hiding this comment.
Pfiou huge delay here, sorry!
Thanks for the PR, stripping internal comments from feed descriptions is a useful feature. The regex approach works for the simple cases, but there are two edge cases in the truncation branch (chars_count > 0), because the markdown is cut before rendering and the comments are only stripped afterwards.
1. Comment leaks when truncation falls inside it. An unclosed <!-- is escaped by Python-Markdown to <!--, so the regex no longer matches:
md = "Début. <!-- note interne très longue qui sera coupée -->\n\nFin."
# chars_count=25 -> '<p>Début. <!-- note inter...</p>'The internal note ends up in the feed, which is what this PR aims to prevent.
2. Comments eat the character length. With an 80-character comment at the top of a page and chars_count=50, the output is only <!-- xxxx... and contains no visible text.
Both would be fixed by stripping comments from in_page.markdown before computing lengths and cutting. Please make sure the excerpt delimiter (<!-- more --> by default) is still located first. Comments inside fenced code blocks would then be stripped too, so that case should be handled or documented.
Other points:
- tests are missing
strip_html_comments()is called in six places. Computing the result in the branches and stripping once would be more concise. The regex could also be compiled once at module level.
Can you also rebase please?
a4b21c1 to
aabb23c
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #421 +/- ##
==========================================
+ Coverage 83.58% 84.34% +0.75%
==========================================
Files 11 11
Lines 786 792 +6
Branches 133 132 -1
==========================================
+ Hits 657 668 +11
+ Misses 84 81 -3
+ Partials 45 43 -2
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
Guts
left a comment
There was a problem hiding this comment.
Thanks for the update, the truncation fixes and the tests look good.
One remaining case: the compliance warning is evaluated before comments are stripped, so a page whose excerpt is empty after stripping now returns "" silently.
For example, a page containing only <!-- TODO -->:
- On
main, the raw comment is returned as the description. - With this PR,
""is returned and nothing is logged, so the item ends up without a description (non-compliant feed) and the user gets no hint.
Suggested fix: run the empty check after stripping. For example:
page_markdown = self.strip_html_comments(page_markdown)
if not page_markdown.strip():
logger.warning(...) # same message as today
return ""This would need care for the case where the delimiter is at position 0, to avoid changing today's behavior there.
A test for this case (page with only comments, chars_count=160, asserting "" and the warning via assertLogs) would also be welcome.
Add automatic HTML comment removal from all feed item descriptions to ensure clean content in RSS and JSON feeds. Comments are stripped from all content sources including markdown, page.meta descriptions, and full page content. - Add strip_html_comments() static method to Util class - Update get_description_or_abstract() to strip comments in all code paths - Handle both inline and multiline HTML comments - Preserve content structure while removing comment artifacts This prevents HTML comments (including TODO notes, analytics codes, and other internal annotations) from appearing in published feeds. Co-Authored-By: Claude <<EMAIL_ADDRESS>>
Locate excerpt delimiters before removing comments, then measure and truncate the cleaned Markdown. Consolidate final description cleanup and compile the comment pattern once. Add regressions and document comment examples in code blocks. Co-authored-by: Codex <codex@openai.com>
Check the cleaned Markdown excerpt for empty content before rendering it, while preserving silent empty excerpts when the delimiter is at position zero. Cover comment-only excerpts, surrounding whitespace, and delimiter handling with log assertions. AI-Disclosure-Level: 7 (human specced, bots coded; pending human review) AI-Model: GPT-6 (Codex) Co-authored-by: Codex <codex@openai.com>
aabb23c to
e321b91
Compare
|
|
@Guts Could you please take another look since previous comments have been addressed? |



Add automatic HTML comment removal from all feed item descriptions to ensure clean content in RSS and JSON feeds. Comments are stripped from all content sources including markdown, page.meta descriptions, and full page content.
This prevents HTML comments (including TODO notes, analytics codes, and other internal annotations) from appearing in published feeds.