documents: drawn text reads back from its end, decoded as written
Each piece records how its source decodes: Markdown's escapes and references for text, HTML's for an HTML block's runs, none for code spans, cut as CommonMark cuts their end spaces. Matching runs from the piece's end, so quote marks and indents at a line's start are never taken for text, and an alert's text needs no moved position: alert() takes no source again. Lines split on LF, CRLF and CR alike; LineMap matches each drawn line to the written line it ends once, so an HTML block maps in linear time and a tab cut to spaces lands on the tab. A write-back refuses the press when the pressed block's kind, text or drawing changed. The hostile HTML test measures growth: a fixed 2 s budget failed under the full suite's load on the old code too. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
ZacharyZhang-NYandClaude Opus 5.5 committed
1 parent
83d747d87e
commit
d4c614cb7f
10 files changed
+536
-221
No files matched your search
@@ -13,7 +13,7 @@ Ely GPUI Component. A component library for GPUI, in light and dark.
|
||||
- Web views: `wry` 0.57 (Apache-2.0 OR MIT, default features off, macOS only) lays a child WKWebView over the window.
|
||||
- Terminal: `alacritty_terminal` 0.26 (Apache-2.0, default features off, native only) for the grid, its parser and the pseudo-terminal; `futures` carries its events. `vte` 0.15 (Apache-2.0 OR MIT), the parser it re-exports, reads `AnsiText`'s codes alone, so colored output builds for wasm32.
|
||||
- Diffs: `similar` 3.2 (Apache-2.0) for line and word diffs and three-way merges; its `unicode` feature splits words at punctuation.
|
||||
- Markdown: `markdown` 1.0.0 (markdown-rs, MIT) reads GitHub's flavor and math into a syntax tree with byte positions for `MarkdownRenderer`, `MarkdownEditor`, the highlights, block ranges and outline. `MarkdownRenderer` draws display math through `typography::Latex`, a ```` ```mermaid ```` block through `documents::Diagram`, and pictures from the owner's folder (`base`) or `data:` addresses (`base64` 0.22, MIT OR Apache-2.0); web pictures stay unloaded. Inline math shows as TeX: gpui text carries no inline pictures. Inline it reads as Typora's defaults: `:shortcode:` turns to its emoji as GitHub reads it (the `emojis` crate's names, GitHub's today, so `:cricket:` is the insect; read after escapes and references, so `\:smile:` turns too; inside an autolink or `<code>` it stays text; within one DOM text node, which filtered tags do not split, the first `:name:`-shaped run decides for all: blank text or a space before it turns every shortcode, else none, so `x:smile: :smile:` stays), inline HTML's `u`, `b`/`strong`, `i`/`em`, `s`/`del`/`strike`, `code`/`kbd`, `mark`, `br` and `a href` draw as what they mean GFM's filtered tags (`script`, `style`, `title`, `textarea`, `iframe`, `xmp`, `noembed`, `noframes`, `plaintext`) show as text, and other tags, hyphenated names too, vanish, leaving their text; a tag's name runs to white space or `/`, as HTML5 reads it; attribute values split on ASCII whitespace alone and decode as HTML decodes them; each Markdown container is the element GitHub writes (`em`, `strong`, `del`, `a`), on one stack with the tags and repaired as HTML5 repairs: an end tag closes the innermost element of its name and all above it, of which only formatting elements (`a`, `b`, `code`, `em`, `i`, `s`, `strike`, `strong`, `u`) reopen, a new `a` ends the open one and an `a` without `href` links nothing; other tags are not tracked, and `</br>` breaks as `<br>` does; and a footnote reference is its number in superscript, numbered by first reference, as its definition is; one `~` still strikes. At the root, blocks read as GitHub shows them: a quote whose first line is plain text reading `[!NOTE]`, `[!TIP]`, `[!IMPORTANT]`, `[!WARNING]` or `[!CAUTION]` in any case, after escapes and references, spaces and tabs around it, with more after it, is an alert under its title (Important in the palette's violet, `syntax.keyword`, read at 4.5:1); a paragraph that is `[toc]` alone, as Typora reads it, lists every heading by depth, and a press on one reveals it as a link press does (`MarkdownDocument`'s `reveal`), the heading found where `CodeEditor` anchors carried it through each edit since the read, gone if an edit or an undo took it, and a secondary press acts on press, since a block writing back moves the list; front matter shows in a box without its fences; an HTML block shows its text with the inline tags above, white space collapsed and owed as a browser lays it, block elements breaking lines and closing what opened inside them, HTML5's start tags ending an open `p`, `fieldset` and `menu` too, and an `li` start an `li`, a `dd` or `dt` start either, as HTML5 ends them, comments dropped, the filtered tags shown as text, and an unclosed tag dropping the rest; a rule is a 2px `border_strong` line; bullets go •, ◦, ▪ by list depth, ordered lists counted. Exports read a tree as the drawing does through `MarkdownReader` (`block` gives its `MarkdownBlock`s, `inline` its text, `note` and `label` its footnotes, `headings` the `[toc]` list), one fresh stack per paragraph, heading or cell. `MarkdownDocument` holds a whole file in one `CodeEditor` (text and undo, colored by `CodeEditor::highlighter` with `markdown_highlights`): visual draws the file's top-level blocks in a gpui `list`, a pressed block opening as its source in a `TextInput` whose typing is the document's text at once and whose write-back is one undo step; source is the editor; preview draws without presses. A written-back block, a mode switch or a new text is read at once up to 128 KiB, longer ones on the background executor; typing in source or split reads 300 ms after the last key, off the main thread (markdown-rs grows with the square of a paragraph's lines: 40,000 lines take 0.9 s). Texts past `LARGEST_READ` (2 MiB) are never read: no blocks, no outline. A new read keeps the list's blocks alike at both ends. One caret serves every mode: the editor's selection, which a written-back block moves; `set_caret` and `set_top_line` set caret and view apart; a top line asked (numbered, so a late frame callback finishes only its own ask; new text places it again, clamped to the text as it then is) waits for the read, lands on its block, then a frame later at its share of the block's height; `reveal` is such an ask with the caret at its line; a switch between source and the list keeps the top line, one between visual and preview keeps the list's scroll as it is. A block opens only when the read is of the text as it is now. A pressed block keeps its identitLine truncated
|
||||
- Markdown: `markdown` 1.0.0 (markdown-rs, MIT) reads GitHub's flavor and math into a syntax tree with byte positions for `MarkdownRenderer`, `MarkdownEditor`, the highlights, block ranges and outline. `MarkdownRenderer` draws display math through `typography::Latex`, a ```` ```mermaid ```` block through `documents::Diagram`, and pictures from the owner's folder (`base`) or `data:` addresses (`base64` 0.22, MIT OR Apache-2.0); web pictures stay unloaded. Inline math shows as TeX: gpui text carries no inline pictures. Inline it reads as Typora's defaults: `:shortcode:` turns to its emoji as GitHub reads it (the `emojis` crate's names, GitHub's today, so `:cricket:` is the insect; read after escapes and references, so `\:smile:` turns too; inside an autolink or `<code>` it stays text; within one DOM text node, which filtered tags do not split, the first `:name:`-shaped run decides for all: blank text or a space before it turns every shortcode, else none, so `x:smile: :smile:` stays), inline HTML's `u`, `b`/`strong`, `i`/`em`, `s`/`del`/`strike`, `code`/`kbd`, `mark`, `br` and `a href` draw as what they mean GFM's filtered tags (`script`, `style`, `title`, `textarea`, `iframe`, `xmp`, `noembed`, `noframes`, `plaintext`) show as text, and other tags, hyphenated names too, vanish, leaving their text; a tag's name runs to white space or `/`, as HTML5 reads it; attribute values split on ASCII whitespace alone and decode as HTML decodes them; each Markdown container is the element GitHub writes (`em`, `strong`, `del`, `a`), on one stack with the tags and repaired as HTML5 repairs: an end tag closes the innermost element of its name and all above it, of which only formatting elements (`a`, `b`, `code`, `em`, `i`, `s`, `strike`, `strong`, `u`) reopen, a new `a` ends the open one and an `a` without `href` links nothing; other tags are not tracked, and `</br>` breaks as `<br>` does; and a footnote reference is its number in superscript, numbered by first reference, as its definition is; one `~` still strikes. At the root, blocks read as GitHub shows them: a quote whose first line is plain text reading `[!NOTE]`, `[!TIP]`, `[!IMPORTANT]`, `[!WARNING]` or `[!CAUTION]` in any case, after escapes and references, spaces and tabs around it, with more after it, is an alert under its title (Important in the palette's violet, `syntax.keyword`, read at 4.5:1); a paragraph that is `[toc]` alone, as Typora reads it, lists every heading by depth, and a press on one reveals it as a link press does (`MarkdownDocument`'s `reveal`), the heading found where `CodeEditor` anchors carried it through each edit since the read, gone if an edit or an undo took it, and a secondary press acts on press, since a block writing back moves the list; front matter shows in a box without its fences; an HTML block shows its text with the inline tags above, white space collapsed and owed as a browser lays it, block elements breaking lines and closing what opened inside them, HTML5's start tags ending an open `p`, `fieldset` and `menu` too, and an `li` start an `li`, a `dd` or `dt` start either, as HTML5 ends them, comments dropped, the filtered tags shown as text, and an unclosed tag dropping the rest; a rule is a 2px `border_strong` line; bullets go •, ◦, ▪ by list depth, ordered lists counted. Exports read a tree as the drawing does through `MarkdownReader` (`block` gives its `MarkdownBlock`s, `inline` its text, `note` and `label` its footnotes, `headings` the `[toc]` list), one fresh stack per paragraph, heading or cell. `MarkdownDocument` holds a whole file in one `CodeEditor` (text and undo, colored by `CodeEditor::highlighter` with `markdown_highlights`): visual draws the file's top-level blocks in a gpui `list`, a pressed block opening as its source in a `TextInput` whose typing is the document's text at once and whose write-back is one undo step; source is the editor; preview draws without presses. A written-back block, a mode switch or a new text is read at once up to 128 KiB, longer ones on the background executor; typing in source or split reads 300 ms after the last key, off the main thread (markdown-rs grows with the square of a paragraph's lines: 40,000 lines take 0.9 s). Texts past `LARGEST_READ` (2 MiB) are never read: no blocks, no outline. A new read keeps the list's blocks alike at both ends. One caret serves every mode: the editor's selection, which a written-back block moves; `set_caret` and `set_top_line` set caret and view apart; a top line asked (numbered, so a late frame callback finishes only its own ask; new text places it again, clamped to the text as it then is) waits for the read, lands on its block, then a frame later at its share of the block's height; `reveal` is such an ask with the caret at its line; a switch between source and the list keeps the top line, one between visual and preview keeps the list's scroll as it is. A block opens only when the read is of the text as it is now. A pressed block keeps its identitLine truncated
|
||||
- Pictures: `image` 0.25 (MIT OR Apache-2.0), the crate gpui decodes with, default features off; it turns decoded frames and rims map tiles.
|
||||
- Math and diagrams: RaTeX 0.1.14 (`ratex-parser`, `ratex-layout`, `ratex-svg` with `embed-fonts`, `ratex-types`; MIT, KaTeX's fonts OFL) typesets `Latex` to SVG with outlined glyphs; `mermaid-rs-renderer` 0.3.1 (MIT, default features off) draws `Diagram`. Both go through gpui's `SvgRenderer` at the window's scale, loaded as gpui assets off the main thread; `usvg` 0.46, gpui's own, reads a diagram's size. A RaTeX or Mermaid panic on parsed input becomes the element's error. Each element holds its picture through `primitives::held`: a new source, or the element leaving the window, removes the old asset from gpui's cache and drops its image from the atlas after the frame.
|
||||
- Maps: Natural Earth's 110m countries (public domain). `assets/maps/countries.geojson`, trimmed by `scripts/countries.py` to an ISO code, a name and positions to 0.01°, backs `maps::WorldMap`; the gallery's tiles are cut from the same file, z0 to z3, by `scripts/tiles.py` with Pillow.
|
||||
|
||||
@@ -3,11 +3,10 @@ use std::rc::Rc;
|
||||
use gpui::{Context, MouseButton, Window};
|
||||
use markdown::mdast;
|
||||
|
||||
use super::{DocumentEvent, MarkdownDocument, edit::Landing, edit::moved};
|
||||
use super::{DocumentEvent, MarkdownDocument, edit::Landing, edit::moved, parse::Block};
|
||||
use crate::documents::{
|
||||
render::{OnPoint, Place, Pointed},
|
||||
tree::lines_at,
|
||||
tree::{alert, is_toc},
|
||||
tree::{Node, alert, is_toc, lines_at},
|
||||
};
|
||||
|
||||
/// A top-level block's kind.
|
||||
@@ -36,7 +35,7 @@ impl TargetBlock {
|
||||
mdast::Node::Paragraph(_) if is_toc(node, source) => Self::Toc,
|
||||
mdast::Node::Paragraph(_) => Self::Paragraph,
|
||||
mdast::Node::Heading(heading) => Self::Heading(heading.depth),
|
||||
mdast::Node::Blockquote(quote) if alert(quote, source).is_some() => Self::Alert,
|
||||
mdast::Node::Blockquote(quote) if alert(quote).is_some() => Self::Alert,
|
||||
mdast::Node::Blockquote(_) => Self::Quote,
|
||||
mdast::Node::List(_) => Self::List,
|
||||
mdast::Node::Code(_) => Self::Code,
|
||||
@@ -95,6 +94,12 @@ impl MarkdownDocument {
|
||||
})
|
||||
}
|
||||
|
||||
/// A block's kind, text and what it draws.
|
||||
fn drawn(&self, block: &Block) -> (TargetBlock, String, Option<Vec<Node>>) {
|
||||
let text = self.parsed.text[block.range.clone()].to_string();
|
||||
(block.kind, text, block.nodes.clone())
|
||||
}
|
||||
|
||||
/// Caret to `at`, else block `ix`; the owner hears.
|
||||
pub(super) fn point(
|
||||
&mut self,
|
||||
@@ -109,7 +114,7 @@ impl MarkdownDocument {
|
||||
block => block.flatten(),
|
||||
};
|
||||
let kind = block.map(|block| block.kind);
|
||||
let pressed = block.map(|block| self.parsed.text[block.range.clone()].to_string());
|
||||
let pressed = block.map(|block| self.drawn(block));
|
||||
let at = at.or(block.map(|block| block.range.start));
|
||||
let written = self.commit(cx);
|
||||
if written.is_some() {
|
||||
@@ -128,7 +133,7 @@ impl MarkdownDocument {
|
||||
// A write-back may change the pressed block.
|
||||
let now = (self.parsed.blocks.iter())
|
||||
.find(|block| block.range.start <= at && at <= block.range.end)
|
||||
.map(|block| self.parsed.text[block.range.clone()].to_string());
|
||||
.map(|block| self.drawn(block));
|
||||
if written.is_some() && pressed.is_some() && now != pressed {
|
||||
return log::warn!("markdown document: the write-back changed the pressed block");
|
||||
}
|
||||
|
||||
@@ -371,6 +371,7 @@ mod links;
|
||||
mod look;
|
||||
mod place;
|
||||
mod point;
|
||||
mod reading;
|
||||
mod toc;
|
||||
mod tools;
|
||||
|
||||
|
||||
@@ -6,11 +6,7 @@ use gpui::{Entity, Modifiers, MouseButton, Pixels, TestAppContext, VisualTestCon
|
||||
|
||||
use super::{open, type_in};
|
||||
use crate::{
|
||||
documents::{
|
||||
DocumentEvent, DocumentTarget, MarkdownDocument, MarkdownMode, TargetBlock,
|
||||
document::parse::parsed,
|
||||
tree::{Inline, Node},
|
||||
},
|
||||
documents::{DocumentEvent, DocumentTarget, MarkdownDocument, MarkdownMode, TargetBlock},
|
||||
theme::{ActiveTheme, TextSize},
|
||||
};
|
||||
|
||||
@@ -191,42 +187,6 @@ fn press_lines(k: usize, line: usize, button: MouseButton, cx: &mut VisualTestCo
|
||||
press_at(x, y, button, cx);
|
||||
}
|
||||
|
||||
/// The source byte under drawn `shown` in block 0.
|
||||
fn read_back(source: &str, shown: &str) -> Option<usize> {
|
||||
fn find<'a>(nodes: &'a [Node], shown: &str) -> Option<&'a Inline> {
|
||||
nodes.iter().find_map(|node| match node {
|
||||
Node::Paragraph(inline) | Node::Heading(_, inline) if inline.text.contains(shown) => {
|
||||
Some(inline)
|
||||
}
|
||||
Node::Quote(body) | Node::Alert { body, .. } | Node::Footnote { body, .. } => {
|
||||
find(body, shown)
|
||||
}
|
||||
_ => None,
|
||||
})
|
||||
}
|
||||
let parsed = parsed(source);
|
||||
let block = &parsed.blocks[0];
|
||||
let inline = find(block.nodes.as_ref().expect("drawn"), shown).expect("shown");
|
||||
let at = inline.text.find(shown).expect("shown");
|
||||
let written = inline.source_at(&source[block.range.start..], at)?;
|
||||
Some(block.range.start + written)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn quote_marks_escapes_trims_and_alert_markers_read_back() {
|
||||
let cases = [
|
||||
("> alpha\n> beta", "eta"),
|
||||
("> [!NOTE]\n> alpha beta", "beta"),
|
||||
("a \\*star\\* z", "*star"),
|
||||
("a ` xy ` z", "xy"),
|
||||
("<div>alpha beta</div>\n", "eta"),
|
||||
("> <div>\n> alpha beta</div>\n", "beta"),
|
||||
];
|
||||
for (source, shown) in cases {
|
||||
assert_eq!(read_back(source, shown), source.find(shown), "{source:?}");
|
||||
}
|
||||
}
|
||||
|
||||
#[gpui::test]
|
||||
fn a_secondary_press_in_a_quote_lands_past_its_mark(cx: &mut TestAppContext) {
|
||||
let source = "> alpha\n> beta";
|
||||
@@ -328,3 +288,104 @@ fn a_write_back_that_changes_the_pressed_block_points_nowhere(cx: &mut TestAppCo
|
||||
);
|
||||
assert_ne!(caret(&document, cx), written.find("eta").unwrap());
|
||||
}
|
||||
|
||||
#[gpui::test]
|
||||
fn crlf_lines_press_where_they_were_written(cx: &mut TestAppContext) {
|
||||
let source = "```
|
||||
alpha
|
||||
beta
|
||||
```
|
||||
";
|
||||
let (document, cx) = open(source, MarkdownMode::Visual, cx);
|
||||
let events = heard(&document, cx);
|
||||
press_lines(2, 0, MouseButton::Right, cx);
|
||||
let at = source.find("pha").unwrap();
|
||||
let expected = DocumentTarget {
|
||||
in_code: true,
|
||||
..target(at, TargetBlock::Code)
|
||||
};
|
||||
assert_eq!(*events.borrow(), [expected]);
|
||||
press_lines(2, 0, MouseButton::Left, cx);
|
||||
assert_eq!(opened_caret(&document, cx), at);
|
||||
}
|
||||
|
||||
#[gpui::test]
|
||||
fn crlf_front_matter_presses_where_written(cx: &mut TestAppContext) {
|
||||
let source = "---
|
||||
title: value
|
||||
author: person
|
||||
---
|
||||
";
|
||||
let (document, cx) = open(source, MarkdownMode::Visual, cx);
|
||||
let events = heard(&document, cx);
|
||||
press_lines(7, 0, MouseButton::Right, cx);
|
||||
let at = source.find("value").unwrap();
|
||||
assert_eq!(*events.borrow(), [target(at, TargetBlock::FrontMatter)]);
|
||||
press_lines(7, 0, MouseButton::Left, cx);
|
||||
assert_eq!(opened_caret(&document, cx), at);
|
||||
}
|
||||
|
||||
#[gpui::test]
|
||||
fn a_crlf_definition_presses_where_written(cx: &mut TestAppContext) {
|
||||
let source = "[ref]: https://ely.dev
|
||||
\"title\"
|
||||
";
|
||||
let (document, cx) = open(source, MarkdownMode::Visual, cx);
|
||||
let events = heard(&document, cx);
|
||||
press_on(0, 10, MouseButton::Right, cx);
|
||||
assert_eq!(*events.borrow(), [target(10, TargetBlock::Definition)]);
|
||||
}
|
||||
|
||||
#[gpui::test]
|
||||
fn a_changed_definition_points_nowhere(cx: &mut TestAppContext) {
|
||||
let source = "[ref]: https://old.dev
|
||||
|
||||
Go [there][ref]
|
||||
";
|
||||
let (document, cx) = open(source, MarkdownMode::Visual, cx);
|
||||
let events = heard(&document, cx);
|
||||
type_in(&document, Some(0), Some("[ref]: https://new.dev"), cx);
|
||||
press_on(1, 4, MouseButton::Right, cx);
|
||||
assert_eq!(
|
||||
super::text(&document, cx),
|
||||
source.replace("old.dev", "new.dev")
|
||||
);
|
||||
assert!(
|
||||
events.borrow().is_empty(),
|
||||
"the link changed under the press"
|
||||
);
|
||||
}
|
||||
|
||||
#[gpui::test]
|
||||
fn a_quoted_greater_than_presses_on_itself(cx: &mut TestAppContext) {
|
||||
let source = "> <div>
|
||||
> >alpha beta</div>";
|
||||
let (document, cx) = open(source, MarkdownMode::Visual, cx);
|
||||
let events = heard(&document, cx);
|
||||
let bounds = cx.debug_bounds("markdown-block-0").expect("drawn");
|
||||
let (body, _) = sizes(cx);
|
||||
let x = bounds.left() + px(31.0) + body * 0.6 * 0.25;
|
||||
press_at(x, bounds.top() + px(10.0), MouseButton::Right, cx);
|
||||
let at = source.find(">alpha").unwrap();
|
||||
assert_eq!(*events.borrow(), [target(at, TargetBlock::Quote)]);
|
||||
}
|
||||
|
||||
#[gpui::test]
|
||||
fn an_escaped_backslash_presses_on_itself(cx: &mut TestAppContext) {
|
||||
let source = r"a \\ z";
|
||||
let (document, cx) = open(source, MarkdownMode::Visual, cx);
|
||||
let events = heard(&document, cx);
|
||||
press_on(0, 2, MouseButton::Right, cx);
|
||||
let at = source.find(r"\ z").unwrap();
|
||||
assert_eq!(*events.borrow(), [target(at, TargetBlock::Paragraph)]);
|
||||
}
|
||||
|
||||
#[gpui::test]
|
||||
fn an_html_space_moved_by_a_tag_presses_on_its_word(cx: &mut TestAppContext) {
|
||||
let source = "<div>alpha <b> beta</b></div>";
|
||||
let (document, cx) = open(source, MarkdownMode::Visual, cx);
|
||||
let events = heard(&document, cx);
|
||||
press_on(0, 8, MouseButton::Right, cx);
|
||||
let at = source.find("eta").unwrap();
|
||||
assert_eq!(*events.borrow(), [target(at, TargetBlock::Html)]);
|
||||
}
|
||||
@@ -0,0 +1,123 @@
|
||||
//! Drawn text reads back to its source byte (R4.5).
|
||||
|
||||
use crate::documents::{
|
||||
TargetBlock,
|
||||
document::parse::parsed,
|
||||
tree::{Inline, Node, lines_at},
|
||||
};
|
||||
|
||||
/// The source byte under drawn `shown` in block 0.
|
||||
fn read_back(source: &str, shown: &str) -> Option<usize> {
|
||||
fn find<'a>(nodes: &'a [Node], shown: &str) -> Option<&'a Inline> {
|
||||
nodes.iter().find_map(|node| match node {
|
||||
Node::Paragraph(inline) | Node::Heading(_, inline) if inline.text.contains(shown) => {
|
||||
Some(inline)
|
||||
}
|
||||
Node::Quote(body) | Node::Alert { body, .. } | Node::Footnote { body, .. } => {
|
||||
find(body, shown)
|
||||
}
|
||||
_ => None,
|
||||
})
|
||||
}
|
||||
let parsed = parsed(source);
|
||||
let block = &parsed.blocks[0];
|
||||
let inline = find(block.nodes.as_ref().expect("drawn"), shown).expect("shown");
|
||||
let at = inline.text.find(shown).expect("shown");
|
||||
let written = inline.source_at(&source[block.range.start..], at)?;
|
||||
Some(block.range.start + written)
|
||||
}
|
||||
|
||||
/// The source byte under drawn `word` of block 0's code.
|
||||
fn code_back(source: &str, word: &str) -> Option<usize> {
|
||||
fn code(nodes: &[Node]) -> (&str, usize) {
|
||||
match &nodes[0] {
|
||||
Node::Code { text, from, .. } => (text, *from),
|
||||
Node::Quote(body) => code(body),
|
||||
other => panic!("not code: {other:?}"),
|
||||
}
|
||||
}
|
||||
let parsed = parsed(source);
|
||||
let block = &parsed.blocks[0];
|
||||
let (drawn, from) = code(block.nodes.as_ref().expect("drawn"));
|
||||
let at = drawn.find(word).expect("drawn");
|
||||
let written = lines_at(&source[block.range.start..], from, drawn, at)?;
|
||||
Some(block.range.start + written)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn text_reads_back_past_marks_escapes_trims_and_markers() {
|
||||
let cases = [
|
||||
("> alpha\n> beta", "eta"),
|
||||
("> [!NOTE]\n> alpha beta", "beta"),
|
||||
("> [!NOTE] \n> alpha beta", "beta"),
|
||||
("> [!NOTE]\\\n> alpha beta", "beta"),
|
||||
("> [!NOTE]\n> \\*beta\\*", "*beta"),
|
||||
("> [!NOTE] alpha beta", "beta"),
|
||||
("a \\*star\\* z", "*star"),
|
||||
("a \\\\ z", "\\ z"),
|
||||
("a ` xy ` z", "xy"),
|
||||
("a ` xy ` z", " xy "),
|
||||
("a ` >xy ` z", ">xy"),
|
||||
("a `\\\\xy` z", "xy"),
|
||||
("<div>alpha beta</div>\n", "eta"),
|
||||
("> <div>\n> alpha beta</div>\n", "beta"),
|
||||
("> <div>\n> >alpha beta</div>", ">alpha"),
|
||||
("> <div>\n> >alpha beta</div>", "beta"),
|
||||
("<div>alpha <b>beta</b> gamma</div>", "gamma"),
|
||||
("<div>alpha <b>beta</b> gamma</div>", "beta"),
|
||||
("<div>alpha <b> beta</b></div>", "eta"),
|
||||
("<div>one</div>\n<div>alpha beta</div>", "beta"),
|
||||
];
|
||||
for (source, shown) in cases {
|
||||
assert_eq!(read_back(source, shown), source.find(shown), "{source:?}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_reference_reads_back_to_its_ampersand() {
|
||||
let source = "a & b";
|
||||
assert_eq!(read_back(source, "& b"), source.find('&'));
|
||||
assert_eq!(read_back(source, "b"), source.find('b'));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn text_after_an_emoji_reads_back_and_before_it_lands_at_its_start() {
|
||||
let source = "ab :smile: y";
|
||||
assert_eq!(read_back(source, "y"), source.find('y'));
|
||||
assert_eq!(read_back(source, "b"), Some(0));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn code_reads_back_line_for_line_whatever_its_line_ends() {
|
||||
let cases = [
|
||||
"```\nalpha\nbeta\n```\n",
|
||||
"```\r\nalpha\r\nbeta\r\n```\r\n",
|
||||
"```\ralpha\rbeta\r```\r",
|
||||
" alpha\n beta\n",
|
||||
"> ```\n> alpha\n> beta\n> ```\n",
|
||||
];
|
||||
for source in cases {
|
||||
for word in ["pha", "eta"] {
|
||||
assert_eq!(code_back(source, word), source.find(word), "{source:?}");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn code_cut_from_a_tab_reads_back_past_it() {
|
||||
for spaces in 0..=3 {
|
||||
for prefix in ["\t", " \t", " \t", " \t", "\t\t"] {
|
||||
let indent = " ".repeat(spaces);
|
||||
let source = format!("{indent}```\n{indent}alpha\n{prefix}beta\n{indent}```\n");
|
||||
assert_eq!(code_back(&source, "eta"), source.find("eta"), "{source:?}");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_decoded_line_end_reads_without_a_written_one() {
|
||||
let quote = parsed("> alpha beta");
|
||||
assert_eq!(quote.blocks[0].kind, TargetBlock::Quote);
|
||||
let alert = parsed("> [!NOTE] alpha beta");
|
||||
assert_eq!(alert.blocks[0].kind, TargetBlock::Alert);
|
||||
}
|
||||
@@ -69,7 +69,7 @@ impl Reader<'_> {
|
||||
let mdast::Node::Blockquote(quote) = node else {
|
||||
return None;
|
||||
};
|
||||
let (kind, parts) = alert(quote, self.source)?;
|
||||
let (kind, parts) = alert(quote)?;
|
||||
let mut body = Vec::new();
|
||||
self.blocks(&parts, &mut body);
|
||||
Some(Node::Alert { kind, body })
|
||||
@@ -77,11 +77,11 @@ impl Reader<'_> {
|
||||
}
|
||||
|
||||
/// A quote's alert kind and its blocks after the marker.
|
||||
pub fn alert(quote: &mdast::Blockquote, source: &str) -> Option<(AlertKind, Vec<mdast::Node>)> {
|
||||
pub fn alert(quote: &mdast::Blockquote) -> Option<(AlertKind, Vec<mdast::Node>)> {
|
||||
let [mdast::Node::Paragraph(paragraph), rest @ ..] = "e.children[..] else {
|
||||
return None;
|
||||
};
|
||||
let (kind, after) = marker(¶graph.children, source)?;
|
||||
let (kind, after) = marker(¶graph.children)?;
|
||||
if after.is_empty() && rest.is_empty() {
|
||||
return None;
|
||||
}
|
||||
@@ -101,7 +101,7 @@ pub fn is_toc(node: &mdast::Node, source: &str) -> bool {
|
||||
}
|
||||
|
||||
/// A plain first line naming a kind, and what follows.
|
||||
fn marker(children: &[mdast::Node], source: &str) -> Option<(AlertKind, Vec<mdast::Node>)> {
|
||||
fn marker(children: &[mdast::Node]) -> Option<(AlertKind, Vec<mdast::Node>)> {
|
||||
let mut line = String::new();
|
||||
for (at, child) in children.iter().enumerate() {
|
||||
let head = match child {
|
||||
@@ -116,7 +116,6 @@ fn marker(children: &[mdast::Node], source: &str) -> Option<(AlertKind, Vec<mdas
|
||||
let after = after.strip_prefix("\r\n").unwrap_or(&after[1..]);
|
||||
let mut kept = text.clone();
|
||||
kept.value = after.to_string();
|
||||
past_line(&mut kept, source);
|
||||
(!after.is_empty()).then_some(mdast::Node::Text(kept))
|
||||
}
|
||||
},
|
||||
@@ -129,25 +128,6 @@ fn marker(children: &[mdast::Node], source: &str) -> Option<(AlertKind, Vec<mdas
|
||||
Some((AlertKind::marked(&line)?, Vec::new()))
|
||||
}
|
||||
|
||||
/// `text` starts past its first line, as its value does.
|
||||
fn past_line(text: &mut mdast::Text, source: &str) {
|
||||
let start = &mut text
|
||||
.position
|
||||
.as_mut()
|
||||
.expect("parsed text has a place")
|
||||
.start;
|
||||
let written = &source[start.offset..];
|
||||
let end = written.find(['\n', '\r']).expect("its line end is written");
|
||||
let width = if written[end..].starts_with("\r\n") {
|
||||
2
|
||||
} else {
|
||||
1
|
||||
};
|
||||
start.offset += end + width;
|
||||
start.line += 1;
|
||||
start.column = 1;
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
@@ -300,7 +300,7 @@ fn decoded(text: &str, attribute: bool) -> String {
|
||||
const LONGEST: usize = 32;
|
||||
|
||||
/// A reference at `text`'s start, as HTML reads it.
|
||||
fn reference(text: &str, attribute: bool) -> Option<(String, usize)> {
|
||||
pub(super) fn reference(text: &str, attribute: bool) -> Option<(String, usize)> {
|
||||
let body = text.strip_prefix('&')?;
|
||||
if let Some(number) = body.strip_prefix('#') {
|
||||
let (digits, radix) = match number.strip_prefix(['x', 'X']) {
|
||||
|
||||
@@ -1,15 +1,24 @@
|
||||
use std::ops::Range;
|
||||
|
||||
use super::{Inline, Run, html, sources::Follow};
|
||||
use super::{Escapes, Inline, Run, html, sources::LineMap};
|
||||
|
||||
/// An HTML block's text, as a browser shows it.
|
||||
pub(super) fn html_block_text(written: &str, source: &str, span: Range<usize>) -> Inline {
|
||||
let (mut run, mut follow) = (Run::default(), Follow::new(written, source, span.clone()));
|
||||
let (mut run, map) = (Run::default(), LineMap::new(source, span.start, written));
|
||||
let mut rest = written;
|
||||
// Each text run keeps where it was written.
|
||||
let mut words = |run: &mut Run, rest: &str, text: &str| {
|
||||
if let Some(at) = follow.to(written.len() - rest.len()) {
|
||||
run.text.sources.push((run.text.text.len(), at..span.end));
|
||||
let words = |run: &mut Run, rest: &str, text: &str| {
|
||||
let from = written.len() - rest.len();
|
||||
let at = |value| map.as_ref()?.at(value);
|
||||
// Blank text draws nothing where it stands.
|
||||
if text.chars().all(html::spacing) {
|
||||
return run.words(text);
|
||||
}
|
||||
match at(from).zip(at(from + text.len())) {
|
||||
Some((start, end)) => {
|
||||
(run.text.sources).push((run.text.text.len(), start..end, Escapes::Html))
|
||||
}
|
||||
None => log::warn!("markdown: an HTML block's text lost its source"),
|
||||
}
|
||||
run.words(text);
|
||||
};
|
||||
@@ -278,23 +287,32 @@ mod tests {
|
||||
|
||||
#[test]
|
||||
fn hostile_html_reads_in_linear_time() {
|
||||
let started = web_time::Instant::now();
|
||||
for source in [
|
||||
"<".repeat(1 << 17),
|
||||
"<a".repeat(1 << 16),
|
||||
"<!--".repeat(1 << 16),
|
||||
"<b>".repeat(1 << 16),
|
||||
"<b>".repeat(1 << 15) + &"</i>".repeat(1 << 15),
|
||||
"<script></script>".repeat(1 << 13),
|
||||
"<div>a ".repeat(1 << 16),
|
||||
"<b> ".repeat(1 << 16),
|
||||
"<li><b>x ".repeat(1 << 15),
|
||||
"<p><i>x ".repeat(1 << 15),
|
||||
] {
|
||||
let block = format!("<div>\n{source}");
|
||||
parse(&block);
|
||||
}
|
||||
let spent = started.elapsed();
|
||||
assert!(spent.as_secs() < 2, "took {spent:?}");
|
||||
let sources = |n: usize| {
|
||||
[
|
||||
"<".repeat(2 * n),
|
||||
"<a".repeat(n),
|
||||
"<!--".repeat(n),
|
||||
"<b>".repeat(n),
|
||||
"<b>".repeat(n / 2) + &"</i>".repeat(n / 2),
|
||||
"<script></script>".repeat(n / 8),
|
||||
"<div>a ".repeat(n),
|
||||
"<b> ".repeat(n),
|
||||
"<li><b>x ".repeat(n / 2),
|
||||
"<p><i>x ".repeat(n / 2),
|
||||
]
|
||||
};
|
||||
let time = |n| {
|
||||
let started = web_time::Instant::now();
|
||||
for source in sources(n) {
|
||||
parse(&format!(
|
||||
"<div>
|
||||
{source}"
|
||||
));
|
||||
}
|
||||
started.elapsed()
|
||||
};
|
||||
let (small, large) = (time(1 << 14), time(1 << 16));
|
||||
// Four times the input, at most eight the time.
|
||||
assert!(large < small * 8, "{small:?} then {large:?}");
|
||||
}
|
||||
}
|
||||
@@ -13,6 +13,7 @@ mod stack;
|
||||
|
||||
pub use blocks::{AlertKind, DocumentHeading, alert, is_toc};
|
||||
use html_block::html_block_text;
|
||||
pub use sources::Escapes;
|
||||
pub(crate) use sources::lines_at;
|
||||
use stack::close;
|
||||
|
||||
@@ -38,7 +39,7 @@ pub struct Inline {
|
||||
pub styles: Vec<(Range<usize>, Style)>,
|
||||
pub links: Vec<(Range<usize>, String)>,
|
||||
/// Where each piece was written: drawn start, source span.
|
||||
pub sources: Vec<(usize, Range<usize>)>,
|
||||
pub sources: Vec<(usize, Range<usize>, Escapes)>,
|
||||
}
|
||||
|
||||
/// A block of a markdown document.
|
||||
@@ -342,8 +343,9 @@ impl<'a> Reader<'a> {
|
||||
if !matches!(node, mdast::Node::Text(_) | mdast::Node::Html(_)) {
|
||||
run.shortcodes.split();
|
||||
}
|
||||
if let Some(written) = sources::written(node, self.source) {
|
||||
run.text.sources.push((run.text.text.len(), written));
|
||||
if let Some((written, escapes)) = sources::written(node, self.source) {
|
||||
let at = run.text.text.len();
|
||||
run.text.sources.push((at, written, escapes));
|
||||
}
|
||||
let text = &mut run.text;
|
||||
let (children, (name, style), url) = match node {
|
||||
|
||||
+247
-122
@@ -1,11 +1,21 @@
|
||||
use std::{iter::Peekable, ops::Range, str::CharIndices};
|
||||
use std::ops::Range;
|
||||
|
||||
use markdown::mdast::Node;
|
||||
use markdown::{decode_named, mdast::Node};
|
||||
|
||||
use super::{Inline, Node as Drawn, span};
|
||||
use super::{Inline, Node as Drawn, html, span};
|
||||
|
||||
/// A leaf's written span; code and math inside their marks.
|
||||
pub(super) fn written(node: &Node, source: &str) -> Option<Range<usize>> {
|
||||
/// How a piece's source turns into its text.
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub enum Escapes {
|
||||
Plain,
|
||||
/// Backslash escapes and references, as Markdown reads them.
|
||||
Markdown,
|
||||
/// References, as HTML reads them.
|
||||
Html,
|
||||
}
|
||||
|
||||
/// A leaf's written span and how it reads.
|
||||
pub(super) fn written(node: &Node, source: &str) -> Option<(Range<usize>, Escapes)> {
|
||||
let whole = match node {
|
||||
Node::Text(_)
|
||||
| Node::Break(_)
|
||||
@@ -17,14 +27,31 @@ pub(super) fn written(node: &Node, source: &str) -> Option<Range<usize>> {
|
||||
| Node::InlineMath(_) => span(node),
|
||||
_ => return None,
|
||||
};
|
||||
let marks = match node {
|
||||
Node::InlineCode(_) | Node::InlineMath(_) => {
|
||||
let text = &source[whole.clone()];
|
||||
text.len() - text.trim_start_matches(['`', '$']).len()
|
||||
}
|
||||
_ => 0,
|
||||
Some(match node {
|
||||
Node::Text(_) => (whole, Escapes::Markdown),
|
||||
Node::InlineCode(_) => (inside(source, whole, '`'), Escapes::Plain),
|
||||
Node::InlineMath(_) => (inside(source, whole, '$'), Escapes::Plain),
|
||||
_ => (whole, Escapes::Plain),
|
||||
})
|
||||
}
|
||||
|
||||
/// A code span's text, one end space cut each side.
|
||||
fn inside(source: &str, whole: Range<usize>, mark: char) -> Range<usize> {
|
||||
let written = &source[whole.clone()];
|
||||
let open = written.len() - written.trim_start_matches(mark).len();
|
||||
let close = written.len() - written.trim_end_matches(mark).len();
|
||||
let text = &written[open..written.len().saturating_sub(close).max(open)];
|
||||
let blank = |ch: char| matches!(ch, ' ' | '\n' | '\r');
|
||||
let cut = text.starts_with(blank) && text.ends_with(blank) && !text.chars().all(blank);
|
||||
let width = |crlf: bool| if crlf { 2 } else { 1 };
|
||||
let (head, tail) = match cut {
|
||||
true => (
|
||||
width(text.starts_with("\r\n")),
|
||||
width(text.ends_with("\r\n")),
|
||||
),
|
||||
false => (0, 0),
|
||||
};
|
||||
Some(whole.start + marks..whole.end)
|
||||
whole.start + open + head..whole.start + open + text.len() - tail
|
||||
}
|
||||
|
||||
/// Where a block's text lines start: past a fence line.
|
||||
@@ -36,27 +63,93 @@ pub(super) fn body(source: &str, node: &Node) -> usize {
|
||||
.iter()
|
||||
.any(|fence| written.starts_with(fence));
|
||||
match fenced {
|
||||
true => written
|
||||
.find('\n')
|
||||
.map_or(whole.end, |end| whole.start + end + 1),
|
||||
true => lines(written)
|
||||
.nth(1)
|
||||
.map_or(whole.end, |line| whole.start + line.start),
|
||||
false => whole.start,
|
||||
}
|
||||
}
|
||||
|
||||
/// `text`'s lines without their endings, split as CommonMark splits.
|
||||
fn lines(text: &str) -> impl Iterator<Item = Range<usize>> + '_ {
|
||||
let mut next = Some(0);
|
||||
std::iter::from_fn(move || {
|
||||
let start = next?;
|
||||
let rest = &text[start..];
|
||||
let Some(end) = rest.find(['\n', '\r']) else {
|
||||
next = None;
|
||||
return Some(start..text.len());
|
||||
};
|
||||
let width = if rest[end..].starts_with("\r\n") {
|
||||
2
|
||||
} else {
|
||||
1
|
||||
};
|
||||
next = Some(start + end + width);
|
||||
Some(start..start + end)
|
||||
})
|
||||
}
|
||||
|
||||
/// Source byte under drawn byte `at`; lines start at `from`.
|
||||
pub(crate) fn lines_at(source: &str, from: usize, drawn: &str, at: usize) -> Option<usize> {
|
||||
let line = drawn[..at].matches('\n').count();
|
||||
let column = at - drawn[..at].rfind('\n').map_or(0, |end| end + 1);
|
||||
let shown = drawn.split('\n').nth(line)?;
|
||||
let mut start = from;
|
||||
for _ in 0..line {
|
||||
start += source.get(start..)?.find('\n')? + 1;
|
||||
LineMap::new(source, from, drawn)?.at(at)
|
||||
}
|
||||
|
||||
/// Each drawn line set against the written line it ends.
|
||||
pub(super) struct LineMap(Vec<Line>);
|
||||
|
||||
struct Line {
|
||||
drawn: Range<usize>,
|
||||
/// Where its written line starts, and its shared end.
|
||||
written: usize,
|
||||
common: usize,
|
||||
kept: usize,
|
||||
/// The char before the shared end, as a cut tab.
|
||||
before: usize,
|
||||
}
|
||||
|
||||
impl LineMap {
|
||||
pub(super) fn new(source: &str, from: usize, drawn: &str) -> Option<Self> {
|
||||
let written = source.get(from..)?;
|
||||
let lines = lines(drawn).zip(lines(written)).map(|(line, place)| {
|
||||
let (shown, text) = (&drawn[line.clone()], &written[place.clone()]);
|
||||
let kept: usize = match text.ends_with(shown) {
|
||||
true => shown.len(),
|
||||
false => (shown.chars().rev())
|
||||
.zip(text.chars().rev())
|
||||
.take_while(|(a, b)| a == b)
|
||||
.map(|(a, _)| a.len_utf8())
|
||||
.sum(),
|
||||
};
|
||||
let common = text.len() - kept;
|
||||
let before =
|
||||
(text[..common].chars().next_back()).map_or(0, |ch| common - ch.len_utf8());
|
||||
Line {
|
||||
drawn: line,
|
||||
written: from + place.start,
|
||||
common,
|
||||
kept,
|
||||
before,
|
||||
}
|
||||
});
|
||||
Some(Self(lines.collect()))
|
||||
}
|
||||
|
||||
/// Source byte under drawn byte `at`.
|
||||
pub(super) fn at(&self, at: usize) -> Option<usize> {
|
||||
let line = self
|
||||
.0
|
||||
.get(self.0.partition_point(|line| line.drawn.end < at))?;
|
||||
let column = at.saturating_sub(line.drawn.start);
|
||||
let shared = line.drawn.len() - line.kept;
|
||||
Some(
|
||||
line.written
|
||||
+ match column.checked_sub(shared) {
|
||||
Some(into) => line.common + into,
|
||||
None => line.before,
|
||||
},
|
||||
)
|
||||
}
|
||||
let written = source[start..].split('\n').next()?.trim_end_matches('\r');
|
||||
// Each drawn line ends its written one.
|
||||
written
|
||||
.ends_with(shown)
|
||||
.then(|| start + written.len() - shown.len() + column)
|
||||
}
|
||||
|
||||
impl Drawn {
|
||||
@@ -64,7 +157,7 @@ impl Drawn {
|
||||
pub(crate) fn rebase(&mut self, start: usize) {
|
||||
match self {
|
||||
Drawn::Paragraph(inline) | Drawn::Heading(_, inline) => {
|
||||
for (_, span) in &mut inline.sources {
|
||||
for (_, span, _) in &mut inline.sources {
|
||||
*span = span.start - start..span.end - start;
|
||||
}
|
||||
}
|
||||
@@ -90,109 +183,140 @@ impl Drawn {
|
||||
impl Inline {
|
||||
/// Source byte under drawn byte `at`, in some piece.
|
||||
pub(crate) fn source_at(&self, source: &str, at: usize) -> Option<usize> {
|
||||
let piece = self.sources.partition_point(|(start, _)| *start <= at);
|
||||
let (start, span) = self.sources.get(piece.checked_sub(1)?)?;
|
||||
let piece = self.sources.partition_point(|(start, ..)| *start <= at);
|
||||
let (start, span, escapes) = self.sources.get(piece.checked_sub(1)?)?;
|
||||
let end = self
|
||||
.sources
|
||||
.get(piece)
|
||||
.map_or(self.text.len(), |(next, _)| *next);
|
||||
.map_or(self.text.len(), |(next, ..)| *next);
|
||||
let drawn = self.text.get(*start..end.max(*start))?;
|
||||
let written = source.get(span.clone())?;
|
||||
// Unreadable text lands at its start.
|
||||
Some(span.start + align(drawn, written, at - start).unwrap_or(0))
|
||||
Some(span.start + align(drawn, written, *escapes, at - start).unwrap_or(0))
|
||||
}
|
||||
}
|
||||
|
||||
/// Where drawn byte `at` was written, past marks.
|
||||
fn align(drawn: &str, written: &str, at: usize) -> Option<usize> {
|
||||
let mut reading = Reading::new(written, 0);
|
||||
for (byte, shown) in drawn.char_indices() {
|
||||
let found = reading.take(shown)?;
|
||||
if byte >= at {
|
||||
/// A source char as read, and where it was written.
|
||||
struct Read {
|
||||
ch: char,
|
||||
at: usize,
|
||||
/// Written as itself, not escaped or referenced.
|
||||
raw: bool,
|
||||
/// A container mark or indent at a line's start.
|
||||
lead: bool,
|
||||
}
|
||||
|
||||
impl Read {
|
||||
fn ends_line(&self) -> bool {
|
||||
self.raw && matches!(self.ch, '\n' | '\r')
|
||||
}
|
||||
}
|
||||
|
||||
/// `written`'s chars, escapes and references read.
|
||||
fn read(written: &str, escapes: Escapes) -> Vec<Read> {
|
||||
let mut out = Vec::with_capacity(written.len());
|
||||
let (mut at, mut line_start) = (0, false);
|
||||
while let Some(ch) = written[at..].chars().next() {
|
||||
let rest = &written[at..];
|
||||
let decoded = match (escapes, ch) {
|
||||
(Escapes::Markdown, '\\') => (rest[1..].chars().next())
|
||||
.filter(char::is_ascii_punctuation)
|
||||
.map(|punct| (punct.to_string(), 2, at + 1)),
|
||||
(Escapes::Markdown, '&') => reference(rest).map(|(value, used)| (value, used, at)),
|
||||
(Escapes::Html, '&') => {
|
||||
html::reference(rest, false).map(|(value, used)| (value, used, at))
|
||||
}
|
||||
_ => None,
|
||||
};
|
||||
if let Some((value, used, place)) = decoded {
|
||||
out.extend(value.chars().map(|ch| Read {
|
||||
ch,
|
||||
at: place,
|
||||
raw: false,
|
||||
lead: false,
|
||||
}));
|
||||
(at, line_start) = (at + used, false);
|
||||
continue;
|
||||
}
|
||||
let ends = matches!(ch, '\n' | '\r');
|
||||
let lead = line_start && !ends && matches!(ch, ' ' | '\t' | '>');
|
||||
out.push(Read {
|
||||
ch,
|
||||
at,
|
||||
raw: true,
|
||||
lead,
|
||||
});
|
||||
(at, line_start) = (at + ch.len_utf8(), ends || lead);
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// A reference at `text`'s start, as Markdown reads it.
|
||||
fn reference(text: &str) -> Option<(String, usize)> {
|
||||
let body = text.strip_prefix('&')?;
|
||||
let (end, _) = body.char_indices().take(34).find(|(_, ch)| *ch == ';')?;
|
||||
let name = &body[..end];
|
||||
let code = |digits: &str, radix: u32, most: usize| {
|
||||
let fits =
|
||||
(1..=most).contains(&digits.len()) && digits.chars().all(|ch| ch.is_digit(radix));
|
||||
let code = u32::from_str_radix(digits, radix).ok().filter(|_| fits)?;
|
||||
let ch = char::from_u32(code).filter(|ch| *ch != '\0');
|
||||
Some(ch.unwrap_or(char::REPLACEMENT_CHARACTER).to_string())
|
||||
};
|
||||
let value = match name.strip_prefix('#') {
|
||||
Some(hex) if hex.starts_with(['x', 'X']) => code(&hex[1..], 16, 6)?,
|
||||
Some(digits) => code(digits, 10, 7)?,
|
||||
None if name.chars().all(|ch| ch.is_ascii_alphanumeric()) => decode_named(name, true)?,
|
||||
None => return None,
|
||||
};
|
||||
Some((value, end + 2))
|
||||
}
|
||||
|
||||
/// Where drawn byte `at` was written, matched from the end.
|
||||
fn align(drawn: &str, written: &str, escapes: Escapes, at: usize) -> Option<usize> {
|
||||
if at >= drawn.len() {
|
||||
return Some(written.len());
|
||||
}
|
||||
let source = read(written, escapes);
|
||||
let mut j = source.len();
|
||||
let mut shown = drawn.char_indices().rev().peekable();
|
||||
while let Some((byte, d)) = shown.next() {
|
||||
let found = loop {
|
||||
let s = source[..j].last()?;
|
||||
// A join: a line's end and the marks after it.
|
||||
if d.is_whitespace() && (s.ends_line() || s.lead) {
|
||||
while source[..j].last().is_some_and(|s| s.lead) {
|
||||
j -= 1;
|
||||
}
|
||||
j -= 1;
|
||||
let crlf = source[j].ch == '\n' && j > 0;
|
||||
if crlf && source[j - 1].ends_line() && source[j - 1].ch == '\r' {
|
||||
j -= 1;
|
||||
}
|
||||
if d == '\n' && shown.peek().is_some_and(|(_, ch)| *ch == '\r') {
|
||||
let (cr, _) = shown.next().expect("peeked");
|
||||
if cr == at {
|
||||
return Some(source[j].at);
|
||||
}
|
||||
}
|
||||
break source[j].at;
|
||||
}
|
||||
if d == s.ch {
|
||||
j -= 1;
|
||||
break s.at;
|
||||
}
|
||||
// White space, line ends and marks the text dropped.
|
||||
if s.raw && (matches!(s.ch, ' ' | '\t') || s.ends_line() || s.lead) {
|
||||
j -= 1;
|
||||
continue;
|
||||
}
|
||||
return None;
|
||||
};
|
||||
if byte == at {
|
||||
return Some(found);
|
||||
}
|
||||
}
|
||||
Some(reading.end)
|
||||
}
|
||||
|
||||
/// Finds drawn chars, front to back, in written text.
|
||||
struct Reading<'a> {
|
||||
written: Peekable<CharIndices<'a>>,
|
||||
base: usize,
|
||||
/// Past the last char found.
|
||||
end: usize,
|
||||
}
|
||||
|
||||
impl<'a> Reading<'a> {
|
||||
fn new(written: &'a str, base: usize) -> Self {
|
||||
Self {
|
||||
written: written.char_indices().peekable(),
|
||||
base,
|
||||
end: base,
|
||||
}
|
||||
}
|
||||
|
||||
/// Where `shown` was written; true if a char holds it.
|
||||
fn at(&mut self, shown: char) -> Option<(usize, bool)> {
|
||||
loop {
|
||||
let Some(&(byte, written)) = self.written.peek() else {
|
||||
return shown.is_whitespace().then_some((self.end, false));
|
||||
};
|
||||
let byte = self.base + byte;
|
||||
match (shown, written) {
|
||||
(a, b) if a == b => return Some((byte, true)),
|
||||
(a, b) if a.is_whitespace() => return Some((byte, b.is_whitespace())),
|
||||
// Joins, indents, escapes and quote marks.
|
||||
(_, ' ' | '\t' | '\r' | '\n' | '\\' | '>') => {
|
||||
self.written.next();
|
||||
}
|
||||
_ => return None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn take(&mut self, shown: char) -> Option<usize> {
|
||||
let (byte, held) = self.at(shown)?;
|
||||
if held {
|
||||
let (_, written) = self.written.next().expect("a held char");
|
||||
self.end = byte + written.len_utf8();
|
||||
}
|
||||
Some(byte)
|
||||
}
|
||||
}
|
||||
|
||||
/// A node's value bytes mapped to source, front to back.
|
||||
pub(super) struct Follow<'a> {
|
||||
value: &'a str,
|
||||
done: usize,
|
||||
reading: Option<Reading<'a>>,
|
||||
}
|
||||
|
||||
impl<'a> Follow<'a> {
|
||||
pub(super) fn new(value: &'a str, source: &'a str, span: Range<usize>) -> Self {
|
||||
let reading = Reading::new(&source[span.clone()], span.start);
|
||||
Self {
|
||||
value,
|
||||
done: 0,
|
||||
reading: Some(reading),
|
||||
}
|
||||
}
|
||||
|
||||
/// Where value byte `at` was written.
|
||||
pub(super) fn to(&mut self, at: usize) -> Option<usize> {
|
||||
let reading = self.reading.as_mut()?;
|
||||
let lost = self.value[self.done..at]
|
||||
.chars()
|
||||
.any(|shown| reading.take(shown).is_none());
|
||||
self.done = at;
|
||||
if lost {
|
||||
log::warn!("markdown: a block's text lost its source");
|
||||
self.reading = None;
|
||||
return None;
|
||||
}
|
||||
let next = self.value[at..].chars().next()?;
|
||||
self.reading.as_mut()?.at(next).map(|(byte, _)| byte)
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
@@ -226,10 +350,11 @@ mod tests {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn text_that_does_not_read_back_lands_at_its_start() {
|
||||
fn references_read_back_and_emoji_end_the_reading() {
|
||||
let source = "a & b";
|
||||
assert_eq!(under(source, "b"), 0, "the reference ends the reading");
|
||||
let source = "x :smile: y";
|
||||
assert_eq!(under(source, "y"), 0);
|
||||
assert_eq!(under(source, "b"), source.find('b').unwrap());
|
||||
let source = "ab :smile: y";
|
||||
assert_eq!(under(source, "y"), source.find('y').unwrap());
|
||||
assert_eq!(under(source, "b"), 0, "before the emoji: its start");
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user