C12.6: IMAP sync

This commit is contained in:
雷电芽衣 committed 2026-10-19 21:21:27 -04:00
1 parent 1c225f6812
commit 890bd80d4d
32 files changed
+2343 -276

No files matched your search

Generated
+2
View File
@@ -2130,6 +2130,7 @@ dependencies = [
"ely-ui",
"embed-resource",
"gpui-pre",
"jiff",
"log",
"tempfile",
"tokio",
@@ -2145,6 +2146,7 @@ dependencies = [
"getrandom 0.4.3",
"html5ever",
"imap-proto",
"jiff",
"keyring",
"log",
"mail-builder",
+1 -1
View File
@@ -320,6 +320,6 @@ cargo run -p ely-doc 启动一个应用(ElyDoc 先 cargo build -p ely
- 2026-10-18:Word 公式(R8.2,Gitea #9)。附件 math.docx 的六类公式逐类核对(本机无 Word,以文件所写为准):键盘输入、OMML(内核转为公式对象)、MathType(OLE,显示随带的 WMF 预览)、上下标、图片原本就对;差别在两处根因。其一,运行的 `w:position`(升降)原先只给文字,图片与 OLE 公式所在的运行失去它(25 张图片与一个 ½ 浮在行上):补丁 `0049-writer-inline-objects-move-with-their-run.patch` 把它作为固定升降写到对象的锚字符上(按锚字符自己的字号换算百分比,不扩展到其后的文字),版面让“作为字符”的对象随锚字符的固定升降移动(vertAlign 的自动上下标不动对象,同原样);各导出本就写出锚字符的升降,DOCX 往返仍为 `w:position`。其二,EQ 域原先只认 `\o`、`\s\do` 与注音,其余(如分数 `\f`)整个消失:补丁 `0050-writer-eq-field-parser.patch`(读法)与 `0051-writer-eq-fields-as-formulas.patch`(导入)把 EQ 域转为公式对象:`\f` 分数、`\r` 根式、`\s` 上下标、`\b` 括号、`\l` 列表、`\a` 阵列、`\i` 积分与求和求积;`\x` 与 `\o` 只留内容(无框、无叠印),`\d` 只前移,`\ai`、`\di` 只是行距(不升降),开关不分大小写;每个命令字符保留其运行的上下标(Word 照此画),同一段上下标里字样变了仍是一个上下标;斜体字母写成标识符(StarMath 与 OMML 都画成斜体,DOCX 往返不丢),斜体的数字与符号写成 `ital` 文字,存为 DOCX 后成正体(OMML 无此写法,已知限制);没有底的上下标以空文字为底(空的底在 OMML 里是占位符,重开成 `<?>`);公式取域的字号;读不通的命令以其域代码显示为文字,不消失。只有整个域是 `\s\do N(文字)` 时照原样成为下移的文字,其余都经读法(原先小写的 `\s\do` 读到第一个括号即止,其后丢失)。`client/tests/word.rs` 以 `tests/fixtures/office/word.docx`(由 `.work/r8/gen/eq.py` 生成,不入库)核对公式文本(含开头的空格、大写开关、读不通的域、字样变化的下标、`\ai`、`\di`、小写 `\s\do` 的尾部与嵌套)、DOCX 往返后的斜体、下标与无占位符、锚字符的升降与图块里图片低于文字(去掉版面的升降即失败)。审查三轮:第 3 轮的四处已修并以上述测试核对,未再审查(每条至多三轮)。仍与 Word 不同:编辑区里公式与图片四周有内核的辅助框线(`ShowBoundaries`,视图辅助,不印出);DOCX 另存后重开行距变宽,因内核导出改写文档默认字体与段距,与公式无关。
- 2026-10-18:工作表过多时标签条(R8.3,Gitea #8)。组件库 `navigation::Tabs` 横排时可横向滚动(滚轮同样),标签可窄于自然宽度;所选标签在所选或标签条宽度变了时滚入视口,在绘制时以刚排好的宽度判定(原先只在首帧滚一次,其后右栏打开、标签条变窄,所选标签被挤出视口);手动滚动保留到二者之一再变;横排的标签至多与标签条同宽,名字过长以省略号结尾(组件库 e56a343、3a68d1b、7d704d9,同批带入上游列外壳拖动的四个提交)。ElyTable 的标签条随之收缩,添加按钮留在其旁;标签条的状态按工作簿各记一份(元素 id 带标签号),切到另一工作簿时其所选标签照样滚入。`navigation/tests/tabs.rs` 以 30 个标签核对:首帧滚到所选、滚轮回到第一个、新选的滚入、变窄后所选仍在末端(去掉按宽度判定即失败)、长名的所选标签整个在标签条内(去掉宽度上限即失败);ElyTable 的 `tests/strip.rs` 以 `sheets::strip` 画两个各有 30 张表、都选第 30 张的工作簿,滚回开头后来回切换,所选标签每次都在标签条内(两个工作簿共用一个 id 即失败)。Windows 真窗口(`.work/r8/strip3.png`):30 张表、第 30 张为当前,1100 宽时标签条末端为它。
- 2026-10-18:ElyMark 公式(R8.4,用户定两项都做)。行内公式照块公式由 RaTeX 排版(原只显示着色的 TeX 原文):gpui 的文本里放不进图片,组件库 `render/math_text.rs` 把每个行内公式换成等宽的不断行空格(U+00A0,按字体量出个数),排好的公式居中画在其上,行高随最高的公式撑开;空格串与词一样整个折到下一行,画在其所在行(折行处的序号 gpui 读作上一行末);按下与右键所在的字节经换算读回公式的原文;排不出的公式照旧显示 TeX 原文,其后以危险色写出 RaTeX 给的原因与位置,记警告(不含公式)。`\(…\)` 读作行内公式,`\[…\]` 读作块公式(ChatGPT 等常写的分隔符;GitHub 不认,Typora 默认不认):解析前逐字节把成对的分隔符换成 `$`(长度不变,位置照旧对得上)。成对只在同一段落、标题或单元格里(跨列表项、CR 空行不成对);不跨代码、公式、HTML 标签、链接地址与引用标签、图片、脚注引用,`<code>`、`<kbd>` 之间(含嵌套;交叉闭合照可视的读法关闭与重开)的文字照原文;`\\(` 不换。公式里与段落里另有的 `$`(含 `\$`)解析时暂写成文本里没有的控制字节,解析后还原,不与换上的 `$` 配对;相邻的两个公式之间暂插一个空格,解析后去掉,仍是两个公式。公式节点的范围是整对分隔符,值是写下的 TeX 原文。独占一段的 `\[…\]` 为块公式(段首可有一至三个空格)。组件库公开 `documents::markdown_tree`,ElyMark 的 HTML、PDF、纯文本导出都经它解析(`markdown_options` 不再公开),纯文本里独占一段的公式照块公式写其 TeX。同时查出(组件库 e095f06):RaTeX 的 SVG 以 pt 定尺寸,usvg 按 px 量墨迹,两者差 4/3,每个公式的画面右侧与下方多出约四分之一的空白(块公式因此偏左)、导出的公式比正文大 4/3;现生成时即把根改为 px。`markdown/tex/tests.rs` 与 `tree/tests.rs` 核对分隔符的各种情形(相邻、`\$`、链接地址、`<code>`、跨块),`math_text.rs` 的窗口测试核对公式排进并读回原文、折到第二行的公式画在第二行首(改回按起点定位即失败)、排不出的公式写出原因(组件库 a20e712、5fba7b3、de961eb),`math/tests.rs` 核对画面与公式同宽(改回 pt 即失败),ElyMark 的 `visual.rs` 核对导出与纯文本。Windows 真窗口(`.work/r8/mark/visual2.png`):`$…$`、`\(…\)` 排在行内,`\[…\]` 居中成块,行内代码与转义的 `\\(` 照原文。
- 2026-10-19(用户定):ElyMail 的后端改为 Rust crate,取代 MailCore 2(其 Windows 构建是 VS 2013 工程加 2015 年前后的预编译依赖,POP 无 OAuth2):mail-send、mail-parser、mail-builder,在应用进程的后台任务里运行,不另起 worker;MailCore 2 与 libetpan 的取码、构建与探测随之删去。正文以 html5ever 解析、净化后按块的子集由 GPUI 绘制,不当作完整页面;写信用 Markdown 文档组件,发出纯文本与 HTML。Gmail 与 Microsoft 走 OAuth(授权码加 PKCE,本机回环),客户端 ID 在构建时注入:`ELY_GOOGLE_CLIENT_ID`、`ELY_GOOGLE_CLIENT_SECRET`、`ELY_MICROSOFT_CLIENT_ID`,本机写在项目根的 `oauth.env`(不入库),CI 为仓库机密;缺 ID 的构建里该登录写明不可用,不退回别的方式。iCloud 不给第三方 OAuth,走应用专用密码向导(`imap.mail.me.com:993`、`smtp.mail.me.com:587`,无 POP3)。注册由用户做(Google Desktop 客户端,Entra 公共客户端,委托权限 IMAP、POP、SMTP 与 offline_access)。POP3(C12.1 审查后)由引擎自写,不用 async-pop:它留着点填充、丢末尾 CRLF、吞掉登录后的连接错误、不答 XOAUTH2 的错误挑战、在 trace 日志里写出密码。IMAP(第 2 轮审查后)同样自写,以 imap-proto 解析回复,不用 async-imap:它的 LIST、FETCH 等流式命令不看结尾的 NO 与 BAD(拒绝成了空结果),登录与命令中途的 `* BYE` 被丢掉(只剩断线),trace 日志写出密码与邮件。SMTP 的事务(MAIL、RCPT、DATA)也由引擎自走:mail-send 不给首行的点加点;正文有 8 位字节即带 `BODY=8BITMIME`,服务器不支持即拒绝。读信按 MIME 树与所在容器(C12.2),嵌套(多部分与内嵌邮件都算)深于 64 层即报错,超深的解析结果以迭代拆开再丢(直接丢弃会递归、栈溢出);mail-parser 开 `full_encoding`;SMTPUTF8 按每一层的头块判定。本地存储(C12.3):rusqlite 自带的 SQLite,`mail.db` 加内容目录(原文按 SHA-256),FTS5(trigram)可由索引与原文重建;`user_version` 记结构版本,升级前备份,失败即只读;原文文件在删去或替换邮件的事务提交前删去,删不掉即回滚。只有 HTML 的邮件经 html5ever 的树构建器取文字入索引,元素深过 512 层即停读(树构建器的开放元素栈扫描随深度平方增长)。凭据(C12.4):密码与刷新令牌只进系统凭据库(keyring),被拒即只留在内存到退出;OAuth 换令牌与刷新经 ureq,回跳在 127.0.0.1 随机端口;Google 授权带 `access_type=offline` 与 `prompt=consent`(每次同意都发刷新令牌),Microsoft 另要 `profile`(无 `email` 时以 `preferred_username` 为地址),换码没有刷新令牌即报错;客户端 ID 由 `crates/ely-mail-engine/build.rs` 取环境变量或 `oauth.env`,该文件不存在即未配置,读不出或写错即构建失败。ureq 在 trace 级逐字节记下请求与回复(令牌、客户端密钥),`ely_platform::logging` 在任何 `ELY_LOG` 下都不记 `ureq`、`ureq_proto` 低于 info 的日志。账号(C12.5):`ely_mail_engine::Account` 存在 accounts 表的 settings 列(JSON),凭据库名 `account:地址`;删账号先删凭据再删本地数据,本地删除失败即放回凭据,凭据库删不掉时内存里的凭据照留,不动服务端;地址在库里唯一,不分 ASCII 大小写(`Error::Taken`)。应用的库读写在后台、锁后进行,网络任务在应用自带的两线程 tokio 运行时上,结果交回 gpui。iCloud 收发都以完整地址为用户名。删去原文先改名为 `.散列.gone` 留在原处,事务提交后才删,失败即改回;打开库时按库里是否仍有邮件指向它放回或删去(C12.5 第 1 轮后)。对话框的请求(检查、登录、保存)在整个窗口里编号,回来时只作用于仍在等它的那个对话框;保存的结果照样更新账号列表,失败而对话框已换即以提示写明。端口记下上次自动填入的值,与之不同的即为键入,换安全方式不覆盖。
- 2026-10-19(用户定):ElyMail 的后端改为 Rust crate,取代 MailCore 2(其 Windows 构建是 VS 2013 工程加 2015 年前后的预编译依赖,POP 无 OAuth2):mail-send、mail-parser、mail-builder,在应用进程的后台任务里运行,不另起 worker;MailCore 2 与 libetpan 的取码、构建与探测随之删去。正文以 html5ever 解析、净化后按块的子集由 GPUI 绘制,不当作完整页面;写信用 Markdown 文档组件,发出纯文本与 HTML。Gmail 与 Microsoft 走 OAuth(授权码加 PKCE,本机回环),客户端 ID 在构建时注入:`ELY_GOOGLE_CLIENT_ID`、`ELY_GOOGLE_CLIENT_SECRET`、`ELY_MICROSOFT_CLIENT_ID`,本机写在项目根的 `oauth.env`(不入库),CI 为仓库机密;缺 ID 的构建里该登录写明不可用,不退回别的方式。iCloud 不给第三方 OAuth,走应用专用密码向导(`imap.mail.me.com:993`、`smtp.mail.me.com:587`,无 POP3)。注册由用户做(Google Desktop 客户端,Entra 公共客户端,委托权限 IMAP、POP、SMTP 与 offline_access)。POP3(C12.1 审查后)由引擎自写,不用 async-pop:它留着点填充、丢末尾 CRLF、吞掉登录后的连接错误、不答 XOAUTH2 的错误挑战、在 trace 日志里写出密码。IMAP(第 2 轮审查后)同样自写,以 imap-proto 解析回复,不用 async-imap:它的 LIST、FETCH 等流式命令不看结尾的 NO 与 BAD(拒绝成了空结果),登录与命令中途的 `* BYE` 被丢掉(只剩断线),trace 日志写出密码与邮件。SMTP 的事务(MAIL、RCPT、DATA)也由引擎自走:mail-send 不给首行的点加点;正文有 8 位字节即带 `BODY=8BITMIME`,服务器不支持即拒绝。读信按 MIME 树与所在容器(C12.2),嵌套(多部分与内嵌邮件都算)深于 64 层即报错,超深的解析结果以迭代拆开再丢(直接丢弃会递归、栈溢出);mail-parser 开 `full_encoding`;SMTPUTF8 按每一层的头块判定。本地存储(C12.3):rusqlite 自带的 SQLite,`mail.db` 加内容目录(原文按 SHA-256),FTS5(trigram)可由索引与原文重建;`user_version` 记结构版本,升级前备份,失败即只读;原文文件在删去或替换邮件的事务提交前删去,删不掉即回滚。只有 HTML 的邮件经 html5ever 的树构建器取文字入索引,元素深过 512 层即停读(树构建器的开放元素栈扫描随深度平方增长)。凭据(C12.4):密码与刷新令牌只进系统凭据库(keyring),被拒即只留在内存到退出;OAuth 换令牌与刷新经 ureq,回跳在 127.0.0.1 随机端口;Google 授权带 `access_type=offline` 与 `prompt=consent`(每次同意都发刷新令牌),Microsoft 另要 `profile`(无 `email` 时以 `preferred_username` 为地址),换码没有刷新令牌即报错;客户端 ID 由 `crates/ely-mail-engine/build.rs` 取环境变量或 `oauth.env`,该文件不存在即未配置,读不出或写错即构建失败。ureq 在 trace 级逐字节记下请求与回复(令牌、客户端密钥),`ely_platform::logging` 在任何 `ELY_LOG` 下都不记 `ureq`、`ureq_proto` 低于 info 的日志。账号(C12.5):`ely_mail_engine::Account` 存在 accounts 表的 settings 列(JSON),凭据库名 `account:地址`;删账号先删凭据再删本地数据,本地删除失败即放回凭据,凭据库删不掉时内存里的凭据照留,不动服务端;地址在库里唯一,不分 ASCII 大小写(`Error::Taken`)。应用的库读写在后台、锁后进行,网络任务在应用自带的两线程 tokio 运行时上,结果交回 gpui。iCloud 收发都以完整地址为用户名。删去原文先改名为 `.散列.gone` 留在原处,事务提交后才删,失败即改回;打开库时按库里是否仍有邮件指向它放回或删去(C12.5 第 1 轮后)。对话框的请求(检查、登录、保存)在整个窗口里编号,回来时只作用于仍在等它的那个对话框;保存的结果照样更新账号列表,失败而对话框已换即以提示写明。端口记下上次自动填入的值,与之不同的即为键入,换安全方式不覆盖。IMAP 同步(C12.6):文件夹按 LIST 存为邮箱(角色按 RFC 6154,INBOX 按名、库里不分大小写;有 LIST-EXTENDED 与 SPECIAL-USE 即以 `RETURN (SPECIAL-USE)` 取角色),不再列出的连同邮件删去;EXAMINE 缺 UIDVALIDITY 或 EXISTS 即报错;UIDVALIDITY 变了即清空重建;同一 UID 的 FETCH 回复合并;邮件头全取,正文只取 90 天内(`sync::BODY_DAYS`):先读 BODYSTRUCTURE,无附件即取整封,有附件只取头与正文部分、按原结构重组,附件内容为空、部分号记在 `missing`,附件按需(C12.17),处置为 attachment 的多部分整个按需,related 里不看处置(RFC 2387);新列由结构 v2 升级加上,UIDVALIDITY 不同于所记时只删别的 ULine truncated
- 2026-10-19(用户定):代码与发布从 Gitea 搬到 GitHub 组织 ELY-Open-Source(`ELY-Open-Source/ElyOffice`;组件库 `ELY-Open-Source/Ely-GPUI-Components`,Cargo 的 git 依赖与 `fetch-references.sh` 随之改址,钉的提交不变;Kigi CLI 同在此组织)。历史、标签与发布已迁过,Gitea 停用,GitHub 是唯一的家。发布由 `.github/workflows/release.yml` 在推送 `v*` 标签时运行(只此一种触发,拉取请求到不了自托管机器):macOS 在 `[self-hosted, macos-signing]`(Mac mini 的图形会话,Developer ID 签名并公证),Windows 与 Linux(WSL)在 `[self-hosted, ely-windows]`;各机的常驻克隆 `$HOME/ely-ci/ElyOffice` 从 GitHub 拉取。`scripts/ci.sh` 经 GitHub API 按标签取发布、没有就建(`GITHUB_TOKEN`,`contents: write`),同名文件先删再传:重跑只换本平台的文件。更新检查读 `api.github.com/repos/ELY-Open-Source/ElyOffice/releases/latest` 的 `<slug>.json`,清单的下载地址为 `github.com/ELY-Open-Source/ElyOffice/releases/download/v<版本>/<文件>`;0.1.8 及以前装好的应用仍查 Gitea,需手动装一次新版。官网 worker 读 GitHub API(可选机密 `GITHUB_TOKEN` 避开共享出口地址的限流),下载与源码链接指向 GitHub(GitHub 标志);macOS 版已公证,去掉“未经公证”的提示。发布视频的源文件与成片里仍是 Gitea 地址,未改未重渲染。
- 2026-10-19(用户定):自有代码由 AGPL-3.0-or-later 改为 MIT(`LICENSE`,Copyright (c) 2026 Yang Zhang;工作区 `Cargo.toml`、Linux 包的许可字段、`docs/licenses.md`、官网)。第三方照旧:LibreOffice 与 `patches/libreoffice` 为 MPL-2.0 等,Git 为 GPL-2.0,MuPDF 为 AGPL-3.0;ElyPDF 链接 MuPDF,其发行包整体仍须按 AGPL 履行,Linux 包的许可字段据此写为 `MIT AND <引擎许可>`。
+1
View File
@@ -16,6 +16,7 @@ ely-mail-engine.workspace = true
ely-platform.workspace = true
ely-ui.workspace = true
gpui.workspace = true
jiff.workspace = true
log.workspace = true
tokio = { workspace = true, features = ["rt-multi-thread"] }
+1
View File
@@ -252,6 +252,7 @@ impl Mail {
accounts.push((id, account));
}
mail.picked = Some(id);
mail.sync(id, cx);
if mail.awaiting(asked).is_some() {
mail.adding = None;
}
+11 -2
View File
@@ -13,7 +13,7 @@ use ely_ui::Root;
use gpui::{App, AppContext as _, Context, Entity, FocusHandle, SharedString, WeakEntity, Window};
use tokio::runtime::Runtime;
use crate::{adding::Adding, removing::Removing, side::Side};
use crate::{adding::Adding, removing::Removing, side::Side, syncing::Syncs};
/// What the engine runs on and keeps to.
pub struct Parts {
@@ -43,6 +43,7 @@ pub struct Mail {
pub picked: Option<AccountId>,
pub adding: Option<Adding>,
pub removing: Option<Removing>,
pub syncs: Syncs,
/// Numbers every request across dialogs.
pub asked: u64,
pub focus: FocusHandle,
@@ -100,6 +101,7 @@ impl Mail {
picked: None,
adding: None,
removing: None,
syncs: Syncs::new(),
asked: 0,
focus: cx.focus_handle(),
root: WeakEntity::new_invalid(),
@@ -113,7 +115,8 @@ impl Mail {
self.with_store(
cx,
|store| store.accounts(),
|mail, read, _| {
|mail, read, cx| {
let first = mail.accounts.is_none();
if let Ok(accounts) = &read {
let still = |id| accounts.iter().any(|(each, _)| *each == id);
mail.picked = mail
@@ -122,6 +125,12 @@ impl Mail {
.or_else(|| accounts.first().map(|(id, _)| *id));
}
mail.accounts = Some(read);
if first {
let ids: Vec<_> = mail.accounts().iter().map(|(id, _)| *id).collect();
for id in ids {
mail.sync(id, cx);
}
}
},
);
}
+1
View File
@@ -9,6 +9,7 @@ mod mail;
mod menu;
mod removing;
mod side;
mod syncing;
mod view;
#[cfg(test)]
+5 -1
View File
@@ -55,6 +55,7 @@ impl Mail {
return log::info!("mail: the account to remove is already gone");
};
let (id, name, vault) = (removing.id, account.secret_name(), self.vault.clone());
self.stop_sync(id);
let address = removing.address;
self.with_store(
cx,
@@ -80,7 +81,10 @@ impl Mail {
cx,
);
}
Err(error) => mail.failed("mail.remove.failed", &error, cx),
Err(error) => {
mail.failed("mail.remove.failed", &error, cx);
mail.sync(id, cx);
}
}
mail.read_accounts(cx);
},
+93
View File
@@ -0,0 +1,93 @@
//! IMAP accounts sync on open and once added.
use std::{
collections::HashMap,
time::{SystemTime, UNIX_EPOCH},
};
use ely_mail_engine::{AccountId, Protocol, Synced, sync};
use gpui::Context;
use tokio::task::AbortHandle;
use crate::mail::Mail;
/// One account's sync as the window shows it.
#[derive(Default)]
pub struct Syncing {
pub task: Option<AbortHandle>,
pub synced: Option<Synced>,
pub failed: Option<String>,
}
pub type Syncs = HashMap<AccountId, Syncing>;
impl Mail {
pub fn sync(&mut self, id: AccountId, cx: &mut Context<Self>) {
let Some((_, account)) = self.accounts().iter().find(|(each, _)| *each == id) else {
return log::info!("mail: sync asked of a gone account");
};
if account.protocol != Protocol::Imap {
return;
}
if self.syncs.get(&id).is_some_and(|each| each.task.is_some()) {
return log::info!("mail: account {} is already syncing", id.0);
}
let Ok(store) = self.store.clone() else {
return log::error!("mail: no store to sync with");
};
let tls = match self.tls.clone() {
Ok(tls) => tls,
Err(error) => {
log::warn!("mail: account {} cannot sync without TLS", id.0);
self.syncs.entry(id).or_default().failed = Some(error);
return self.read_synced(id, cx);
}
};
let (account, vault) = (account.clone(), self.vault.clone());
let work = async move {
let now = SystemTime::now()
.duration_since(UNIX_EPOCH)
.map_err(|err| ely_mail_engine::Error::Store(format!("the clock: {err}")))?
.as_secs() as i64;
let secret = sync::secret(&account, &vault).await?;
sync::imap(&account, id, secret, &tls, &store, now).await
};
let task = self.on_runtime(cx, work, move |mail, done, cx| {
let failed = match done {
Ok(Ok(_)) => None,
Ok(Err(error)) => Some(error.to_string()),
Err(error) => Some(error),
};
if failed.is_some() {
log::warn!("mail: account {} did not sync", id.0);
}
let Some(syncing) = mail.syncs.get_mut(&id) else {
return log::info!("mail: a sync ended for a removed account");
};
syncing.task = None;
syncing.failed = failed;
mail.read_synced(id, cx);
});
self.syncs.entry(id).or_default().task = Some(task);
self.read_synced(id, cx);
}
/// Stops the account's sync and forgets it.
pub fn stop_sync(&mut self, id: AccountId) {
if let Some(task) = self.syncs.remove(&id).and_then(|each| each.task) {
task.abort();
}
}
fn read_synced(&mut self, id: AccountId, cx: &mut Context<Self>) {
self.with_store(
cx,
move |store| store.synced(id),
move |mail, read, cx| match (read, mail.syncs.get_mut(&id)) {
(Ok(synced), Some(syncing)) => syncing.synced = Some(synced),
(Err(error), Some(_)) => mail.failed("mail.sync.unread", &error, cx),
(_, None) => {}
},
);
}
}
+20 -6
View File
@@ -10,7 +10,9 @@ use std::{
time::{Duration, Instant},
};
use ely_mail_engine::{Account, Auth, Backend, Checked, Protocol, Security, Server, Store, Vault};
use ely_mail_engine::{
Account, Auth, Backend, Checked, Protocol, Security, Server, Store, TlsConnector, Vault,
};
use gpui::{App, Entity, TestAppContext, VisualTestContext};
use crate::{
@@ -20,6 +22,7 @@ use crate::{
};
mod flows;
mod syncs;
/// A credential store that keeps or refuses.
#[derive(Clone, Default)]
@@ -69,6 +72,16 @@ pub(crate) fn window<'a>(
cx: &'a mut TestAppContext,
dir: &Path,
keys: Keys,
) -> (Entity<Mail>, &'a mut VisualTestContext) {
let tls = ely_mail_engine::system_tls().map_err(|err| err.to_string());
window_with(cx, dir, keys, tls)
}
pub(crate) fn window_with<'a>(
cx: &'a mut TestAppContext,
dir: &Path,
keys: Keys,
tls: Result<TlsConnector, String>,
) -> (Entity<Mail>, &'a mut VisualTestContext) {
cx.executor().allow_parking();
cx.update(ely_gpui_component::init_for_tests);
@@ -89,7 +102,7 @@ pub(crate) fn window<'a>(
let parts = Parts {
store: Store::open(dir).map_err(|err| err.to_string()),
vault: Vault::new(Box::new(keys)),
tls: ely_mail_engine::system_tls().map_err(|err| err.to_string()),
tls,
};
let (mail, cx) = cx.add_window_view(move |_, cx| Mail::new(parts, cx));
settle(cx, |app| mail.read(app).accounts.is_some());
@@ -102,9 +115,10 @@ pub(crate) fn closed_port() -> u16 {
listener.local_addr().unwrap().port()
}
/// On this machine's discard port, so syncs end at once.
pub(crate) fn account(address: &str) -> Account {
let server = |host: &str, port| Server {
host: host.into(),
let server = |port| Server {
host: "127.0.0.1".into(),
port,
security: Security::Tls,
};
@@ -113,8 +127,8 @@ pub(crate) fn account(address: &str) -> Account {
user: address.into(),
auth: Auth::Password,
protocol: Protocol::Imap,
incoming: server("imap.example.com", 993),
outgoing: server("smtp.example.com", 465),
incoming: server(9),
outgoing: server(9),
}
}
+90
View File
@@ -0,0 +1,90 @@
use ely_mail_engine::{Protocol, Store};
use gpui::TestAppContext;
use super::{Keys, account, closed_port, settle, window, window_with};
#[test]
fn an_imap_account_syncs_on_open_and_says_why_it_failed() {
let dir = tempfile::tempdir().unwrap();
let store = Store::open(dir.path()).unwrap();
let mut ann = account("ann@example.com");
ann.incoming.port = closed_port();
let ann = store.add_account(&ann).unwrap();
let pop = ely_mail_engine::Account {
protocol: Protocol::Pop,
..account("bo@example.com")
};
let bo = store.add_account(&pop).unwrap();
drop(store);
let keys = Keys::default();
keys.kept
.lock()
.unwrap()
.insert("account:ann@example.com".into(), "pw".into());
let mut cx = TestAppContext::single();
let (mail, cx) = window(&mut cx, dir.path(), keys);
settle(cx, |app| {
mail.read(app)
.syncs
.get(&ann)
.is_some_and(|each| each.task.is_none() && each.synced.is_some())
});
cx.update(|_, app| {
let mail = mail.read(app);
let syncing = &mail.syncs[&ann];
let failed = syncing.failed.as_deref().unwrap();
assert!(failed.starts_with("cannot connect"), "{failed}");
assert_eq!(syncing.synced.unwrap().at, None);
assert!(!mail.syncs.contains_key(&bo), "POP3 is collected apart");
});
}
#[test]
fn an_account_without_its_secret_says_so() {
let dir = tempfile::tempdir().unwrap();
let ann = Store::open(dir.path())
.unwrap()
.add_account(&account("ann@example.com"))
.unwrap();
let mut cx = TestAppContext::single();
let (mail, cx) = window(&mut cx, dir.path(), Keys::default());
settle(cx, |app| {
mail.read(app)
.syncs
.get(&ann)
.is_some_and(|each| each.failed.is_some())
});
cx.update(|_, app| {
let failed = mail.read(app).syncs[&ann].failed.clone().unwrap();
assert_eq!(
failed,
"credential store failed: no saved secret for account:ann@example.com"
);
});
}
#[test]
fn a_missing_tls_is_the_sync_failure() {
let dir = tempfile::tempdir().unwrap();
let ann = Store::open(dir.path())
.unwrap()
.add_account(&account("ann@example.com"))
.unwrap();
let mut cx = TestAppContext::single();
let tls = Err("platform trust store unavailable".to_owned());
let (mail, cx) = window_with(&mut cx, dir.path(), Keys::default(), tls);
settle(cx, |app| {
mail.read(app)
.syncs
.get(&ann)
.is_some_and(|each| each.synced.is_some())
});
cx.update(|_, app| {
let syncing = &mail.read(app).syncs[&ann];
assert_eq!(
syncing.failed.as_deref(),
Some("platform trust store unavailable")
);
assert!(syncing.task.is_none());
});
}
+59 -2
View File
@@ -8,7 +8,7 @@ use ely_gpui_component::{
primitives::{IconName, Severity},
typography::format,
};
use ely_mail_engine::{Account, Auth, Protocol, Security, Server};
use ely_mail_engine::{Account, AccountId, Auth, Protocol, Security, Server, sync};
use ely_platform::AppId;
use ely_ui::Shell;
use gpui::{
@@ -77,7 +77,7 @@ impl Mail {
Some(Err(detail)) => return error("mail.store.unread", detail, cx),
Some(Ok(accounts)) => accounts,
};
let Some((_, account)) = accounts.iter().find(|(id, _)| Some(*id) == self.picked) else {
let Some((id, account)) = accounts.iter().find(|(id, _)| Some(*id) == self.picked) else {
return EmptyState::new("mail-empty", IconName::Mail, text(cx, "mail.empty", &[]))
.action(self.add_button(cx))
.into_any_element();
@@ -97,6 +97,7 @@ impl Mail {
server(&account.outgoing, cx),
)
.text(label("mail.account.auth"), auth(account.auth, cx));
let (details, failed) = self.sync_rows(*id, details, cx);
let remove = Button::new("mail-remove", label("mail.remove.open"))
.on_click(|_, window, cx| window.dispatch_action(Box::new(menu::RemoveAccount), cx));
div()
@@ -105,11 +106,53 @@ impl Mail {
.flex_col()
.gap_4()
.max_w_128()
.children(failed)
.child(details)
.child(div().child(remove))
.into_any_element()
}
/// When it last synced, what is here, why it failed.
fn sync_rows(
&self,
id: AccountId,
details: DescriptionList,
cx: &App,
) -> (DescriptionList, Option<Banner>) {
let Some(syncing) = self.syncs.get(&id) else {
return (details, None);
};
let label = |key: &str| text(cx, key, &[]);
let last = match (&syncing.task, syncing.synced.and_then(|synced| synced.at)) {
(Some(_), _) => label("mail.sync.running"),
(None, Some(at)) => local_time(at).into(),
(None, None) => label("mail.sync.never"),
};
let days = sync::BODY_DAYS.to_string();
let mut details = details.text(label("mail.sync.last"), last).text(
label("mail.sync.range"),
text(cx, "mail.sync.range_value", &[("days", &days)]),
);
if let Some(synced) = syncing.synced {
for (key, n) in [
("mail.sync.uncached", synced.uncached),
("mail.sync.partial", synced.partial),
] {
let value = format!("{key}_value");
let words = ely_ui::i18n::counted(cx, &value, n, &[("n", &n.to_string())]);
details = details.text(label(key), words);
}
}
let failed = syncing.failed.as_ref().map(|error| {
Banner::new(
"mail-sync-failed",
Severity::Danger,
text(cx, "mail.sync.failed", &[("error", error)]),
)
});
(details, failed)
}
fn remove_dialog(&self, cx: &mut Context<Self>) -> Option<AnyElement> {
let removing = self.removing.as_ref()?;
let this = cx.weak_entity();
@@ -184,3 +227,17 @@ impl Render for Mail {
.children(self.remove_dialog(cx))
}
}
/// Local time to the minute.
fn local_time(at: i64) -> String {
match jiff::Timestamp::from_second(at) {
Ok(at) => at
.to_zoned(jiff::tz::TimeZone::system())
.strftime("%Y-%m-%d %H:%M")
.to_string(),
Err(error) => {
log::error!("mail: a sync time out of range: {error}");
at.to_string()
}
}
}
+1
View File
@@ -12,6 +12,7 @@ form_urlencoded = "1"
getrandom.workspace = true
html5ever.workspace = true
imap-proto.workspace = true
jiff.workspace = true
log.workspace = true
mail-builder.workspace = true
mail-parser.workspace = true
+363 -230
View File
@@ -1,29 +1,90 @@
use base64::{Engine, engine::general_purpose::STANDARD};
use imap_proto::{AttributeValue, MailboxDatum, Response, Status};
use tokio::io::{AsyncRead, AsyncWrite};
use std::collections::HashMap;
use imap_proto::{AttributeValue, Capability, MailboxDatum, NameAttribute, Response, ResponseCode};
use tokio_rustls::TlsConnector;
use crate::{
Error, Result,
net::{self, Login, Secret, Security, Server, Stream, within},
wire::{Wire, words},
};
mod conn;
mod parts;
use conn::{Arg, Conn};
pub use parts::Node;
/// A logged-in IMAP session over TLS.
pub struct Imap {
conn: Conn<Stream>,
}
/// One command argument.
enum Arg<'a> {
/// Written as is.
Atom(&'a str),
/// Quoted, or a literal when quotes cannot carry it.
Text(&'a str),
/// What a folder is for (RFC 6154; INBOX by name).
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Role {
Inbox,
Sent,
Drafts,
Trash,
Junk,
Archive,
All,
Flagged,
}
/// OK, or the words of NO or BAD.
type Outcome = std::result::Result<(), String>;
impl Role {
pub fn name(self) -> &'static str {
match self {
Role::Inbox => "inbox",
Role::Sent => "sent",
Role::Drafts => "drafts",
Role::Trash => "trash",
Role::Junk => "junk",
Role::Archive => "archive",
Role::All => "all",
Role::Flagged => "flagged",
}
}
pub fn from_name(name: &str) -> Option<Self> {
[
Role::Inbox,
Role::Sent,
Role::Drafts,
Role::Trash,
Role::Junk,
Role::Archive,
Role::All,
Role::Flagged,
]
.into_iter()
.find(|role| role.name() == name)
}
}
/// One listed folder.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct Folder {
pub name: String,
pub role: Option<Role>,
/// False for `\Noselect` and `\NonExistent`.
pub selectable: bool,
}
/// A folder opened read-only.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct Opened {
pub uidvalidity: u32,
pub exists: u32,
}
/// One fetched message: header, text or whole.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct Got {
pub uid: u32,
pub flags: Vec<String>,
pub bytes: Vec<u8>,
}
impl Imap {
pub async fn connect(server: &Server, login: &Login, tls: &TlsConnector) -> Result<Self> {
@@ -57,52 +118,168 @@ impl Imap {
Ok(Self { conn })
}
/// Every mailbox name the server lists.
pub async fn mailboxes(&mut self) -> Result<Vec<String>> {
let mut names = Vec::new();
let args = [Arg::Text(""), Arg::Atom("*")];
/// Every folder the server lists, with its role.
pub async fn folders(&mut self) -> Result<Vec<Folder>> {
let can = self.capabilities().await?;
let has = |name: &str| can.iter().any(|it| it.eq_ignore_ascii_case(name));
// RFC 6154 §2: roles may come only on request.
let extended = has("LIST-EXTENDED") && has("SPECIAL-USE");
let mut folders = Vec::new();
let all = [
Arg::Text(""),
Arg::Atom("*"),
Arg::Atom("RETURN (SPECIAL-USE)"),
];
let args = if extended { &all[..] } else { &all[..2] };
within(
"listing mailboxes",
self.conn.run("LIST", &args, |response, raw| {
self.conn.run("LIST", args, |response, raw| {
if let Response::MailboxData(MailboxDatum::List(list)) = response {
names.push(mailbox_name(&list.name, raw));
let name = mailbox_name(&list.name, raw);
folders.push(folder(name, &list.name_attributes));
}
}),
)
.await??
.map_err(Error::Server)?;
Ok(names)
Ok(folders)
}
async fn capabilities(&mut self) -> Result<Vec<String>> {
let mut can = Vec::new();
within(
"reading capabilities",
self.conn.run("CAPABILITY", &[], |response, _| {
if let Response::Capabilities(list) = response {
can.extend(list.iter().filter_map(|it| match it {
Capability::Atom(atom) => Some(atom.to_string()),
_ => None,
}));
}
}),
)
.await??
.map_err(Error::Server)?;
Ok(can)
}
pub async fn examine(&mut self, mailbox: &str) -> Result<Opened> {
let (mut uidvalidity, mut exists) = (None, None);
within(
"opening the mailbox",
self.conn.run(
"EXAMINE",
&[Arg::Text(mailbox)],
|response, _| match response {
Response::Data { outcome, .. } => {
if let Some(ResponseCode::UidValidity(value)) = &outcome.code {
uidvalidity = Some(*value);
}
}
Response::MailboxData(MailboxDatum::Exists(count)) => exists = Some(*count),
_ => {}
},
),
)
.await??
.map_err(Error::Server)?;
let uidvalidity = uidvalidity
.ok_or_else(|| Error::Protocol("the mailbox opened without a UIDVALIDITY".into()))?;
// RFC 3501 §6.3.1: EXISTS is required.
let exists =
exists.ok_or_else(|| Error::Protocol("the mailbox opened without EXISTS".into()))?;
Ok(Opened {
uidvalidity,
exists,
})
}
/// Every UID in the open mailbox with its flags.
pub async fn flags(&mut self) -> Result<Vec<(u32, Vec<String>)>> {
let got = self
.uid_fetch("1:*", "(UID FLAGS)", "reading flags")
.await?;
Ok(got.into_iter().map(|(uid, it)| (uid, it.flags)).collect())
}
/// Headers of `uids`; a UID gone meanwhile is absent.
pub async fn headers(&mut self, uids: &[u32]) -> Result<Vec<Got>> {
let what = "(UID FLAGS BODY.PEEK[HEADER])";
let got = self.uid_fetch(&set(uids), what, "fetching headers").await?;
asked(got, uids, "HEADER")
}
/// Whole messages of `uids`; a UID gone meanwhile is absent.
pub async fn bodies(&mut self, uids: &[u32]) -> Result<Vec<Got>> {
let what = "(UID FLAGS BODY.PEEK[])";
let got = self
.uid_fetch(&set(uids), what, "fetching messages")
.await?;
asked(got, uids, "")
}
/// Each UID's MIME tree; a UID gone meanwhile is absent.
pub async fn structures(&mut self, uids: &[u32]) -> Result<Vec<(u32, Node)>> {
let what = "(UID BODYSTRUCTURE)";
let mut got = self
.uid_fetch(&set(uids), what, "reading structures")
.await?;
let mut out = Vec::new();
for &uid in uids {
let Some(it) = got.remove(&uid) else {
continue;
};
let node = it
.structure
.ok_or_else(|| Error::Protocol(format!("UID {uid} came without a structure")))?;
out.push((uid, node?));
}
Ok(out)
}
/// The message without its left-out parts; `None` once gone.
pub async fn text(&mut self, uid: u32, node: &Node) -> Result<Option<Got>> {
let peeks: Vec<String> = node
.sections()
.iter()
.map(|section| format!("BODY.PEEK[{section}]"))
.collect();
let what = format!("(UID FLAGS {})", peeks.join(" "));
let mut got = self
.uid_fetch(&uid.to_string(), &what, "fetching message text")
.await?;
let Some(it) = got.remove(&uid) else {
return Ok(None);
};
Ok(Some(Got {
uid,
bytes: node.build(&it.sections)?,
flags: it.flags,
}))
}
/// UIDs of messages that arrived on or after `day` (`19-Jul-2026`).
pub async fn since(&mut self, day: &str) -> Result<Vec<u32>> {
let mut uids = Vec::new();
let args = [Arg::Atom("SINCE"), Arg::Atom(day)];
within(
"searching by date",
self.conn.run("UID SEARCH", &args, |response, _| {
if let Response::MailboxData(MailboxDatum::Search(found)) = response {
uids.extend_from_slice(found);
}
}),
)
.await??
.map_err(Error::Server)?;
uids.sort_unstable();
Ok(uids)
}
/// The raw message; `None` once the UID is gone.
pub async fn fetch(&mut self, mailbox: &str, uid: u32) -> Result<Option<Vec<u8>>> {
let examine = [Arg::Text(mailbox)];
within(
"opening the mailbox",
self.conn.run("EXAMINE", &examine, |_, _| {}),
)
.await??
.map_err(Error::Server)?;
let number = uid.to_string();
let args = [Arg::Atom(&number), Arg::Atom("BODY.PEEK[]")];
let mut found = None;
within(
"fetching the message",
self.conn.run("UID FETCH", &args, |response, _| {
if let Response::Fetch(_, values) = response
&& values.contains(&AttributeValue::Uid(uid))
{
found = Some(body(values));
}
}),
)
.await??
.map_err(Error::Server)?;
match found {
None => Ok(None),
Some(None) => Err(Error::Protocol(format!("UID {uid} came without a body"))),
Some(Some(body)) => Ok(Some(body)),
}
self.examine(mailbox).await?;
Ok(self.bodies(&[uid]).await?.pop().map(|got| got.bytes))
}
pub async fn logout(mut self) -> Result<()> {
@@ -110,6 +287,140 @@ impl Imap {
.await??
.map_err(Error::Server)
}
/// Every FETCH for a UID, merged; flags may come apart.
async fn uid_fetch(
&mut self,
uids: &str,
what: &str,
step: &'static str,
) -> Result<HashMap<u32, Fetch>> {
let mut got: HashMap<u32, Fetch> = HashMap::new();
let args = [Arg::Atom(uids), Arg::Atom(what)];
within(
step,
self.conn.run("UID FETCH", &args, |response, _| {
if let Response::Fetch(_, values) = response {
merge(&mut got, values);
}
}),
)
.await??
.map_err(Error::Server)?;
Ok(got)
}
}
#[derive(Default)]
struct Fetch {
flags: Vec<String>,
sections: HashMap<String, Vec<u8>>,
structure: Option<Result<Node>>,
}
/// Unsolicited FETCH responses without a UID are skipped.
fn merge(got: &mut HashMap<u32, Fetch>, values: &[AttributeValue]) {
let Some(uid) = values.iter().find_map(|value| match value {
AttributeValue::Uid(uid) => Some(*uid),
_ => None,
}) else {
return;
};
let it = got.entry(uid).or_default();
for value in values {
match value {
AttributeValue::Flags(flags) => {
// \Recent belongs to one session.
it.flags = flags
.iter()
.filter(|flag| !flag.eq_ignore_ascii_case(r"\Recent"))
.map(|flag| flag.to_string())
.collect();
it.flags.sort();
}
AttributeValue::BodySection {
section,
data: Some(data),
..
} => {
it.sections
.insert(parts::section_key(section), data.to_vec());
}
AttributeValue::BodyStructure(structure) => {
it.structure = Some(Node::read(structure));
}
_ => {}
}
}
}
/// The asked UIDs in order; each needs `section`.
fn asked(mut got: HashMap<u32, Fetch>, uids: &[u32], section: &str) -> Result<Vec<Got>> {
let mut out = Vec::new();
for &uid in uids {
let Some(mut it) = got.remove(&uid) else {
continue;
};
let bytes = it
.sections
.remove(section)
.ok_or_else(|| Error::Protocol(format!("UID {uid} came without {section:?}")))?;
out.push(Got {
uid,
flags: it.flags,
bytes,
});
}
Ok(out)
}
/// `1:3,7` from sorted UIDs.
fn set(uids: &[u32]) -> String {
let mut out = String::new();
let mut at = 0;
while at < uids.len() {
let start = uids[at];
let mut end = start;
while at + 1 < uids.len() && uids[at + 1] == end + 1 {
at += 1;
end = uids[at];
}
if !out.is_empty() {
out.push(',');
}
match start == end {
true => out.push_str(&start.to_string()),
false => out.push_str(&format!("{start}:{end}")),
}
at += 1;
}
out
}
fn folder(name: String, attributes: &[NameAttribute]) -> Folder {
let mut role = name.eq_ignore_ascii_case("INBOX").then_some(Role::Inbox);
let mut selectable = true;
for attribute in attributes {
match attribute {
NameAttribute::NoSelect => selectable = false,
NameAttribute::Extension(other) if other.eq_ignore_ascii_case("\\NonExistent") => {
selectable = false
}
NameAttribute::Sent => role = Some(Role::Sent),
NameAttribute::Drafts => role = Some(Role::Drafts),
NameAttribute::Trash => role = Some(Role::Trash),
NameAttribute::Junk => role = Some(Role::Junk),
NameAttribute::Archive => role = Some(Role::Archive),
NameAttribute::All => role = Some(Role::All),
NameAttribute::Flagged => role = Some(Role::Flagged),
_ => {}
}
}
Folder {
name,
role,
selectable,
}
}
/// imap-proto keeps quoted escapes; a literal is as sent.
@@ -137,189 +448,11 @@ fn mailbox_name(name: &str, raw: &[u8]) -> String {
plain
}
fn body(values: &[AttributeValue]) -> Option<Vec<u8>> {
values.iter().find_map(|value| match value {
AttributeValue::BodySection {
section: None,
data: Some(data),
..
} => Some(data.to_vec()),
_ => None,
})
}
struct Conn<S> {
wire: Wire<S>,
tags: u32,
}
impl<S: AsyncRead + AsyncWrite + Unpin> Conn<S> {
fn new(io: S) -> Self {
Self {
wire: Wire::new(io),
tags: 0,
}
}
/// `* BYE` ends with the server's words.
async fn greeting(&mut self) -> Result<()> {
self.wire
.next(|response, raw| match response {
Response::Data {
status: Status::Ok, ..
} => Ok(()),
Response::Data {
status: Status::Bye,
..
} => Err(Error::Server(words(raw))),
Response::Data { status, .. } => Err(Error::Protocol(format!(
"the greeting was {status:?}, not OK"
))),
_ => Err(Error::Protocol("the first line was not a greeting".into())),
})
.await?
}
fn tag(&mut self) -> String {
self.tags += 1;
format!("A{}", self.tags)
}
/// Sends a command; untagged responses go to `each`.
async fn run(
&mut self,
verb: &str,
args: &[Arg<'_>],
mut each: impl FnMut(&Response, &[u8]),
) -> Result<Outcome> {
let tag = self.tag();
log::debug!("mail: IMAP {tag} {verb}");
let mut line = format!("{tag} {verb}").into_bytes();
for arg in args {
line.push(b' ');
match arg {
Arg::Atom(atom) => line.extend_from_slice(atom.as_bytes()),
Arg::Text(text) => match quoted(text)? {
Some(quoted) => line.extend_from_slice(quoted.as_bytes()),
None => {
line.extend_from_slice(format!("{{{}}}\r\n", text.len()).as_bytes());
self.wire.write(&line).await?;
line.clear();
if let Err(words) = self.proceed(&tag, &mut each).await? {
return Ok(Err(words));
}
line.extend_from_slice(text.as_bytes());
}
},
}
}
line.extend_from_slice(b"\r\n");
self.wire.write(&line).await?;
loop {
match self.step(&tag, &mut each).await? {
Step::Done(outcome) => return Ok(outcome),
Step::Untagged => {}
Step::Continue => {
return Err(Error::Protocol(
"the server asked for more than was sent".into(),
));
}
}
}
}
/// Waits for `+` before a literal.
async fn proceed(
&mut self,
tag: &str,
each: &mut impl FnMut(&Response, &[u8]),
) -> Result<Outcome> {
loop {
match self.step(tag, each).await? {
Step::Continue => return Ok(Ok(())),
Step::Done(Ok(())) => {
return Err(Error::Protocol(
"the command ended before its literal".into(),
));
}
Step::Done(refused) => return Ok(refused),
Step::Untagged => {}
}
}
}
/// Answers the first challenge; empty after, so a refusal ends.
async fn xoauth2(&mut self, initial: &str) -> Result<Outcome> {
let tag = self.tag();
self.wire
.write(format!("{tag} AUTHENTICATE XOAUTH2\r\n").as_bytes())
.await?;
let mut answer = Some(STANDARD.encode(initial));
loop {
match self.step(&tag, &mut |_, _| {}).await? {
Step::Continue => {
let line = answer.take().unwrap_or_default() + "\r\n";
self.wire.write(line.as_bytes()).await?;
}
Step::Done(outcome) => return Ok(outcome),
Step::Untagged => {}
}
}
}
async fn step(&mut self, tag: &str, each: &mut impl FnMut(&Response, &[u8])) -> Result<Step> {
self.wire
.next(|response, raw| match response {
Response::Continue(_) => Ok(Step::Continue),
Response::Done {
tag: done, status, ..
} => {
if done.0 != tag {
return Err(Error::Protocol(format!(
"a reply for {} came while waiting for {tag}",
done.0
)));
}
match status {
Status::Ok => Ok(Step::Done(Ok(()))),
Status::No | Status::Bad => Ok(Step::Done(Err(words(raw)))),
other => Err(Error::Protocol(format!("{tag} ended with {other:?}"))),
}
}
untagged => {
each(untagged, raw);
Ok(Step::Untagged)
}
})
.await?
#[cfg(test)]
mod tests {
#[test]
fn a_uid_set_joins_runs() {
assert_eq!(super::set(&[1, 2, 3, 7, 9, 10]), "1:3,7,9:10");
assert_eq!(super::set(&[5]), "5");
}
}
enum Step {
Continue,
Untagged,
Done(Outcome),
}
/// `None` when only a literal can carry the text.
fn quoted(text: &str) -> Result<Option<String>> {
if text.contains('\0') {
return Err(Error::Invalid("IMAP cannot carry a NUL byte".into()));
}
if !text
.bytes()
.all(|byte| byte.is_ascii() && byte != b'\r' && byte != b'\n')
{
return Ok(None);
}
let mut quoted = String::with_capacity(text.len() + 2);
quoted.push('"');
for char in text.chars() {
if char == '"' || char == '\\' {
quoted.push('\\');
}
quoted.push(char);
}
quoted.push('"');
Ok(Some(quoted))
}
+197
View File
@@ -0,0 +1,197 @@
//! Tagged commands, literals and replies on one stream.
use base64::{Engine, engine::general_purpose::STANDARD};
use imap_proto::{Response, Status};
use tokio::io::{AsyncRead, AsyncWrite};
use crate::{
Error, Result,
wire::{Wire, words},
};
/// One command argument.
pub(super) enum Arg<'a> {
/// Written as is.
Atom(&'a str),
/// Quoted, or a literal when quotes cannot carry it.
Text(&'a str),
}
/// OK, or the words of NO or BAD.
pub(super) type Outcome = std::result::Result<(), String>;
pub(super) struct Conn<S> {
pub(super) wire: Wire<S>,
tags: u32,
}
impl<S: AsyncRead + AsyncWrite + Unpin> Conn<S> {
pub(super) fn new(io: S) -> Self {
Self {
wire: Wire::new(io),
tags: 0,
}
}
/// `* BYE` ends with the server's words.
pub(super) async fn greeting(&mut self) -> Result<()> {
self.wire
.next(|response, raw| match response {
Response::Data {
status: Status::Ok, ..
} => Ok(()),
Response::Data {
status: Status::Bye,
..
} => Err(Error::Server(words(raw))),
Response::Data { status, .. } => Err(Error::Protocol(format!(
"the greeting was {status:?}, not OK"
))),
_ => Err(Error::Protocol("the first line was not a greeting".into())),
})
.await?
}
fn tag(&mut self) -> String {
self.tags += 1;
format!("A{}", self.tags)
}
/// Sends a command; untagged responses go to `each`.
pub(super) async fn run(
&mut self,
verb: &str,
args: &[Arg<'_>],
mut each: impl FnMut(&Response, &[u8]),
) -> Result<Outcome> {
let tag = self.tag();
log::debug!("mail: IMAP {tag} {verb}");
let mut line = format!("{tag} {verb}").into_bytes();
for arg in args {
line.push(b' ');
match arg {
Arg::Atom(atom) => line.extend_from_slice(atom.as_bytes()),
Arg::Text(text) => match quoted(text)? {
Some(quoted) => line.extend_from_slice(quoted.as_bytes()),
None => {
line.extend_from_slice(format!("{{{}}}\r\n", text.len()).as_bytes());
self.wire.write(&line).await?;
line.clear();
if let Err(words) = self.proceed(&tag, &mut each).await? {
return Ok(Err(words));
}
line.extend_from_slice(text.as_bytes());
}
},
}
}
line.extend_from_slice(b"\r\n");
self.wire.write(&line).await?;
loop {
match self.step(&tag, &mut each).await? {
Step::Done(outcome) => return Ok(outcome),
Step::Untagged => {}
Step::Continue => {
return Err(Error::Protocol(
"the server asked for more than was sent".into(),
));
}
}
}
}
/// Waits for `+` before a literal.
async fn proceed(
&mut self,
tag: &str,
each: &mut impl FnMut(&Response, &[u8]),
) -> Result<Outcome> {
loop {
match self.step(tag, each).await? {
Step::Continue => return Ok(Ok(())),
Step::Done(Ok(())) => {
return Err(Error::Protocol(
"the command ended before its literal".into(),
));
}
Step::Done(refused) => return Ok(refused),
Step::Untagged => {}
}
}
}
/// Answers the first challenge; empty after, so a refusal ends.
pub(super) async fn xoauth2(&mut self, initial: &str) -> Result<Outcome> {
let tag = self.tag();
self.wire
.write(format!("{tag} AUTHENTICATE XOAUTH2\r\n").as_bytes())
.await?;
let mut answer = Some(STANDARD.encode(initial));
loop {
match self.step(&tag, &mut |_, _| {}).await? {
Step::Continue => {
let line = answer.take().unwrap_or_default() + "\r\n";
self.wire.write(line.as_bytes()).await?;
}
Step::Done(outcome) => return Ok(outcome),
Step::Untagged => {}
}
}
}
async fn step(&mut self, tag: &str, each: &mut impl FnMut(&Response, &[u8])) -> Result<Step> {
self.wire
.next(|response, raw| match response {
Response::Continue(_) => Ok(Step::Continue),
Response::Done {
tag: done, status, ..
} => {
if done.0 != tag {
return Err(Error::Protocol(format!(
"a reply for {} came while waiting for {tag}",
done.0
)));
}
match status {
Status::Ok => Ok(Step::Done(Ok(()))),
Status::No | Status::Bad => Ok(Step::Done(Err(words(raw)))),
other => Err(Error::Protocol(format!("{tag} ended with {other:?}"))),
}
}
untagged => {
each(untagged, raw);
Ok(Step::Untagged)
}
})
.await?
}
}
enum Step {
Continue,
Untagged,
Done(Outcome),
}
/// `None` when only a literal can carry the text.
fn quoted(text: &str) -> Result<Option<String>> {
if text.contains('\0') {
return Err(Error::Invalid("IMAP cannot carry a NUL byte".into()));
}
if !text
.bytes()
.all(|byte| byte.is_ascii() && byte != b'\r' && byte != b'\n')
{
return Ok(None);
}
let mut quoted = String::with_capacity(text.len() + 2);
quoted.push('"');
for char in text.chars() {
if char == '"' || char == '\\' {
quoted.push('\\');
}
quoted.push(char);
}
quoted.push('"');
Ok(Some(quoted))
}
+290
View File
@@ -0,0 +1,290 @@
//! Text without attachments, from BODYSTRUCTURE (§12.3).
use std::collections::HashMap;
use imap_proto::{BodyParams, BodyStructure, MessageSection, SectionPath};
use crate::{Error, Result};
/// A message's MIME tree; paths are IMAP part numbers.
#[derive(Clone, Debug, PartialEq, Eq)]
pub enum Node {
/// `body` parts come down; the rest wait for a request.
Leaf { path: Vec<u32>, body: bool },
Multi {
path: Vec<u32>,
boundary: String,
children: Vec<Node>,
},
}
impl Node {
/// The reader's rules: `mime/walk.rs`.
pub(super) fn read(structure: &BodyStructure) -> Result<Self> {
match structure {
BodyStructure::Multipart { .. } => body(structure, Vec::new()),
single => Ok(Node::Leaf {
path: vec![1],
body: !attachment(single) && shown(single),
}),
}
}
/// True when nothing is left out.
pub fn complete(&self) -> bool {
match self {
Node::Leaf { body, .. } => *body,
Node::Multi { children, .. } => children.iter().all(Node::complete),
}
}
/// Part numbers left out, as `2` or `1.3`.
pub fn missing(&self) -> Vec<String> {
let mut out = Vec::new();
self.walk(&mut |node| {
if let Node::Leaf { path, body: false } = node {
out.push(number(path));
}
});
out
}
/// The FETCH sections that rebuild it.
pub fn sections(&self) -> Vec<String> {
let mut out = vec!["HEADER".to_owned()];
if let Node::Leaf { body, .. } = self {
if *body {
out.push("1".into());
}
return out;
}
self.walk(&mut |node| match node {
Node::Leaf { path, body } => {
out.push(mime(path));
if *body {
out.push(number(path));
}
}
Node::Multi { path, .. } if !path.is_empty() => out.push(mime(path)),
Node::Multi { .. } => {}
});
out
}
/// The message with left-out parts empty.
pub fn build(&self, got: &HashMap<String, Vec<u8>>) -> Result<Vec<u8>> {
let mut out = header(take(got, "HEADER")?);
match self {
Node::Leaf { body: true, .. } => out.extend_from_slice(take(got, "1")?),
Node::Leaf { body: false, .. } => {}
multi => content(multi, got, &mut out)?,
}
Ok(out)
}
fn walk(&self, each: &mut impl FnMut(&Node)) {
each(self);
if let Node::Multi { children, .. } = self {
for child in children {
child.walk(each);
}
}
}
}
/// The key a FETCH section comes back under.
pub(super) fn section_key(section: &Option<SectionPath>) -> String {
let named = |part: &MessageSection| match part {
MessageSection::Header => "HEADER",
MessageSection::Mime => "MIME",
MessageSection::Text => "TEXT",
};
match section {
None => String::new(),
Some(SectionPath::Full(part)) => named(part).to_owned(),
Some(SectionPath::Part(path, None)) => number(path),
Some(SectionPath::Part(path, Some(part))) => format!("{}.{}", number(path), named(part)),
}
}
/// RFC 2183 §2.8, §2.9: the whole entity waits.
fn body(structure: &BodyStructure, path: Vec<u32>) -> Result<Node> {
if attachment(structure) {
return Ok(Node::Leaf { path, body: false });
}
inside(structure, path)
}
fn inside(structure: &BodyStructure, path: Vec<u32>) -> Result<Node> {
let BodyStructure::Multipart { common, bodies, .. } = structure else {
return Ok(Node::Leaf {
body: shown(structure),
path,
});
};
let boundary = param(&common.ty.params, "boundary")
.ok_or_else(|| Error::Protocol("a multipart came without a boundary".into()))?;
let related = common.ty.subtype.eq_ignore_ascii_case("related");
let root = related.then(|| root(&common.ty.params, bodies));
let children = bodies
.iter()
.zip(1..)
.enumerate()
.map(|(at, (child, number))| {
let mut next = path.clone();
next.push(number);
match root {
None => body(child, next),
Some(Root::At(root)) if root == at => inside(child, next),
Some(Root::Unseen) if multipart(child) => inside(child, next),
// RFC 2387 §4, §6.1: one entity, no disposition.
Some(_) => Ok(Node::Leaf {
body: id(child).is_some(),
path: next,
}),
}
})
.collect::<Result<_>>()?;
Ok(Node::Multi {
path,
boundary: boundary.to_owned(),
children,
})
}
/// The related root, or one hidden in a multipart.
#[derive(Clone, Copy)]
enum Root {
At(usize),
Unseen,
}
/// RFC 2387: `start` names the root, else the first.
fn root(params: &BodyParams, bodies: &[BodyStructure]) -> Root {
let Some(start) = param(params, "start") else {
return Root::At(0);
};
if let Some(at) = bodies
.iter()
.position(|child| id(child).is_some_and(|id| bare(id) == bare(start)))
{
return Root::At(at);
}
if bodies.iter().any(multipart) {
return Root::Unseen;
}
log::warn!("mail: related start names no part; first part is the root");
Root::At(0)
}
fn multipart(structure: &BodyStructure) -> bool {
matches!(structure, BodyStructure::Multipart { .. })
}
/// Plain and HTML text, and `cid:` targets.
fn shown(structure: &BodyStructure) -> bool {
match structure {
BodyStructure::Text { common, .. }
if ["plain", "html"]
.iter()
.any(|kind| common.ty.subtype.eq_ignore_ascii_case(kind)) =>
{
true
}
BodyStructure::Text { .. } | BodyStructure::Basic { .. } => id(structure).is_some(),
BodyStructure::Message { .. } | BodyStructure::Multipart { .. } => false,
}
}
/// RFC 2183 §2.8: unknown dispositions are attachments.
fn attachment(structure: &BodyStructure) -> bool {
let common = match structure {
BodyStructure::Basic { common, .. }
| BodyStructure::Text { common, .. }
| BodyStructure::Message { common, .. }
| BodyStructure::Multipart { common, .. } => common,
};
common
.disposition
.as_ref()
.is_some_and(|it| !it.ty.eq_ignore_ascii_case("inline"))
}
fn id<'a>(structure: &'a BodyStructure) -> Option<&'a str> {
match structure {
BodyStructure::Basic { other, .. }
| BodyStructure::Text { other, .. }
| BodyStructure::Message { other, .. } => other.id.as_deref(),
BodyStructure::Multipart { .. } => None,
}
}
fn bare(id: &str) -> &str {
id.trim().trim_start_matches('<').trim_end_matches('>')
}
fn param<'a>(params: &'a BodyParams, name: &str) -> Option<&'a str> {
params
.iter()
.flatten()
.find(|(key, _)| key.eq_ignore_ascii_case(name))
.map(|(_, value)| value.as_ref())
}
fn content(node: &Node, got: &HashMap<String, Vec<u8>>, out: &mut Vec<u8>) -> Result<()> {
match node {
Node::Leaf { path, body: true } => out.extend_from_slice(take(got, &number(path))?),
Node::Leaf { body: false, .. } => {}
Node::Multi {
boundary, children, ..
} => {
for child in children {
let path = match child {
Node::Leaf { path, .. } | Node::Multi { path, .. } => path,
};
out.extend_from_slice(format!("--{boundary}\r\n").as_bytes());
out.extend(header(take(got, &mime(path))?));
content(child, got, out)?;
out.extend_from_slice(b"\r\n");
}
out.extend_from_slice(format!("--{boundary}--\r\n").as_bytes());
}
}
Ok(())
}
/// A header block ends with a blank line.
fn header(bytes: &[u8]) -> Vec<u8> {
let mut out = bytes.to_vec();
if out.is_empty() {
return b"\r\n".to_vec();
}
while !out.ends_with(b"\r\n\r\n") {
out.extend_from_slice(b"\r\n");
}
out
}
fn take<'a>(got: &'a HashMap<String, Vec<u8>>, key: &str) -> Result<&'a [u8]> {
got.get(key)
.map(Vec::as_slice)
.ok_or_else(|| Error::Protocol(format!("section {key} did not come")))
}
/// A whole message's body is `TEXT`.
fn number(path: &[u32]) -> String {
if path.is_empty() {
return "TEXT".into();
}
path.iter()
.map(u32::to_string)
.collect::<Vec<_>>()
.join(".")
}
fn mime(path: &[u32]) -> String {
format!("{}.MIME", number(path))
}
#[cfg(test)]
mod tests;
@@ -0,0 +1,151 @@
use std::collections::HashMap;
use imap_proto::{AttributeValue, Response};
use super::Node;
/// The node of a `* 1 FETCH (BODYSTRUCTURE …)` line.
fn node(structure: &str) -> Node {
let line = format!("* 1 FETCH (BODYSTRUCTURE {structure})\r\n");
let (_, response) = Response::parse(line.as_bytes()).unwrap();
let Response::Fetch(_, attributes) = response else {
panic!("not a FETCH: {line}");
};
let [AttributeValue::BodyStructure(structure)] = attributes.as_slice() else {
panic!("no structure: {line}");
};
Node::read(structure).unwrap()
}
/// Reads the rebuilt message; sections not given come empty.
fn rebuilt(node: &Node, given: &[(&str, &str)]) -> crate::Mail {
let mut got: HashMap<String, Vec<u8>> = given
.iter()
.map(|(key, text)| ((*key).to_owned(), text.replace('\n', "\r\n").into_bytes()))
.collect();
for section in node.sections() {
got.entry(section).or_default();
}
crate::read(&node.build(&got).unwrap()).unwrap()
}
const TEXT: &str = "(\"TEXT\" \"PLAIN\" (\"CHARSET\" \"us-ascii\") NIL NIL \"7BIT\" 6 1)";
const HTML: &str = "(\"TEXT\" \"HTML\" (\"CHARSET\" \"us-ascii\") NIL NIL \"7BIT\" 9 1)";
#[test]
fn an_attached_multipart_waits_whole() {
let structure = format!(
"({TEXT}({TEXT}{HTML} \"ALTERNATIVE\" (\"BOUNDARY\" \"b2\") \
(\"ATTACHMENT\" (\"FILENAME\" \"note\")) NIL NIL) \"MIXED\" (\"BOUNDARY\" \"b1\") NIL NIL NIL)"
);
let node = node(&structure);
assert!(!node.complete());
assert_eq!(node.missing(), ["2"]);
assert_eq!(node.sections(), ["HEADER", "1.MIME", "1", "2.MIME"]);
let mail = rebuilt(
&node,
&[
(
"HEADER",
"MIME-Version: 1.0\nContent-Type: multipart/mixed; boundary=\"b1\"\n\n",
),
("1.MIME", "Content-Type: text/plain\n\n"),
("1", "hello\n"),
(
"2.MIME",
"Content-Type: multipart/alternative; boundary=\"b2\"\n\
Content-Disposition: attachment; filename=\"note\"\n\n",
),
],
);
assert_eq!(mail.text.as_deref().map(str::trim_end), Some("hello"));
assert_eq!(mail.attachments.len(), 1);
assert_eq!(mail.attachments[0].content_type, "multipart/alternative");
}
#[test]
fn a_related_image_marked_attachment_comes_down() {
let structure = format!(
"({HTML}(\"IMAGE\" \"PNG\" NIL \"<logo>\" NIL \"BASE64\" 12 NIL \
(\"ATTACHMENT\" (\"FILENAME\" \"logo.png\")) NIL NIL) \
\"RELATED\" (\"BOUNDARY\" \"r1\") NIL NIL NIL)"
);
let node = node(&structure);
assert!(node.complete());
assert!(node.missing().is_empty());
let mail = rebuilt(
&node,
&[
(
"HEADER",
"MIME-Version: 1.0\nContent-Type: multipart/related; boundary=\"r1\"\n\n",
),
("1.MIME", "Content-Type: text/html\n\n"),
("1", "<img src=\"cid:logo\">\n"),
(
"2.MIME",
"Content-Type: image/png\nContent-ID: <logo>\n\
Content-Disposition: attachment; filename=\"logo.png\"\n\
Content-Transfer-Encoding: base64\n\n",
),
("2", "iVBORw0KGgo=\n"),
],
);
assert_eq!(mail.inline.len(), 1);
assert_eq!(mail.inline[0].bytes, b"\x89PNG\r\n\x1a\n");
}
#[test]
fn a_related_start_root_ignores_its_disposition() {
let structure = "((\"IMAGE\" \"PNG\" NIL NIL NIL \"BASE64\" 8 NIL NIL NIL NIL)\
(\"TEXT\" \"HTML\" NIL \"<root>\" NIL \"7BIT\" 9 1 NIL (\"ATTACHMENT\" NIL) NIL NIL) \
\"RELATED\" (\"BOUNDARY\" \"r1\" \"START\" \"<root>\") NIL NIL NIL)";
let node = node(structure);
assert_eq!(node.missing(), ["1"]);
}
#[test]
fn an_attached_message_root_waits_whole() {
let structure =
format!("({TEXT}{TEXT} \"MIXED\" (\"BOUNDARY\" \"b1\") (\"ATTACHMENT\" NIL) NIL NIL)");
let node = node(&structure);
assert_eq!(node.missing(), ["TEXT"]);
assert_eq!(node.sections(), ["HEADER"]);
}
#[test]
fn a_related_start_may_name_a_multipart() {
let structure = format!(
"((\"IMAGE\" \"PNG\" NIL \"<logo>\" NIL \"BASE64\" 12 NIL NIL NIL NIL)\
({TEXT}{HTML} \"ALTERNATIVE\" (\"BOUNDARY\" \"b2\") NIL NIL NIL) \
\"RELATED\" (\"BOUNDARY\" \"r1\" \"START\" \"<root>\") NIL NIL NIL)"
);
let node = node(&structure);
assert!(node.complete());
let mail = rebuilt(
&node,
&[
(
"HEADER",
"MIME-Version: 1.0\nContent-Type: multipart/related; boundary=\"r1\"; \
start=\"<root>\"\n\n",
),
(
"1.MIME",
"Content-Type: image/png\nContent-ID: <logo>\n\
Content-Transfer-Encoding: base64\n\n",
),
("1", "iVBORw0KGgo=\n"),
(
"2.MIME",
"Content-Type: multipart/alternative; boundary=\"b2\"\nContent-ID: <root>\n\n",
),
("2.1.MIME", "Content-Type: text/plain\n\n"),
("2.1", "hello\n"),
("2.2.MIME", "Content-Type: text/html\n\n"),
("2.2", "<p>hello</p>\n"),
],
);
assert_eq!(mail.text.as_deref().map(str::trim_end), Some("hello"));
assert!(mail.html.is_some());
}
+5 -2
View File
@@ -9,16 +9,19 @@ pub mod oauth;
mod pop;
mod smtp;
mod store;
pub mod sync;
mod vault;
mod wire;
pub use account::{Account, Auth, Checked, Protocol};
pub use error::{Error, Result};
pub use imap::Imap;
pub use imap::{Folder, Got, Imap, Opened, Role};
pub use mime::{Built, Draft, Mail, Mailbox, Part, build, read};
pub use net::{Login, Secret, Security, Server, system_tls};
pub use pop::Pop;
pub use smtp::Smtp;
pub use store::{AccountId, Cached, Fetched, Key, MailboxId, MessageId, Sizes, Store, Stored};
pub use store::{
AccountId, Cached, Fetched, Held, Key, MailboxId, MessageId, Sizes, Store, Stored, Synced,
};
pub use tokio_rustls::TlsConnector;
pub use vault::{Backend, Kept, Keyring, Vault};
+11
View File
@@ -12,6 +12,7 @@ use crate::{Error, Result};
mod accounts;
mod aside;
mod folders;
mod html;
mod messages;
mod schema;
@@ -20,6 +21,7 @@ mod search;
mod tests;
pub use accounts::Cached;
pub use folders::{Held, Synced};
pub use messages::{Fetched, Key, Stored};
const DB: &str = "mail.db";
@@ -68,6 +70,7 @@ impl Store {
/// The mailbox by name, made on first use.
pub fn mailbox(&self, account: AccountId, name: &str) -> Result<MailboxId> {
let name = mailbox_name(name);
let find = "SELECT id FROM mailboxes WHERE account = ?1 AND name = ?2";
if let Some(id) = self
.conn
@@ -215,3 +218,11 @@ fn folder_size(folder: &Path) -> Result<u64> {
fn disk(step: &str, err: io::Error) -> Error {
Error::Store(format!("{step}: {err}"))
}
/// RFC 3501 §5.1: INBOX in any case is INBOX.
fn mailbox_name(name: &str) -> &str {
match name.eq_ignore_ascii_case("INBOX") {
true => "INBOX",
false => name,
}
}
+219
View File
@@ -0,0 +1,219 @@
//! IMAP folders as mailboxes, and how far they are synced.
use std::collections::HashMap;
use rusqlite::{Transaction, params};
use super::{AccountId, MailboxId, MessageId, Store, aside::Aside, mailbox_name};
use crate::{Error, Result, Role};
/// A local IMAP message as sync compares it.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct Held {
pub id: MessageId,
/// The body is here, perhaps without attachments.
pub body: bool,
/// Sorted.
pub flags: Vec<String>,
}
/// An account's sync, as the window shows it.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct Synced {
/// Unix seconds of the last full sync.
pub at: Option<i64>,
pub messages: u64,
/// Messages with only their header here.
pub uncached: u64,
/// Messages whose attachments wait for a request.
pub partial: u64,
}
impl Store {
/// Keeps the listed folders; the rest go with their mail.
pub fn set_folders(
&mut self,
account: AccountId,
folders: &[(&str, Option<Role>)],
) -> Result<Vec<MailboxId>> {
self.writable()?;
let held: Vec<(i64, String)> = self
.conn
.prepare("SELECT id, name FROM mailboxes WHERE account = ?1")?
.query_map([account.0], |row| Ok((row.get(0)?, row.get(1)?)))?
.collect::<rusqlite::Result<_>>()?;
for (id, name) in held {
if !folders
.iter()
.any(|(listed, _)| mailbox_name(listed) == name)
{
self.without_mail(MailboxId(id), None, |tx| {
tx.execute("DELETE FROM mailboxes WHERE id = ?1", [id])
})?;
log::info!("mail: mailbox {id} is no longer listed; removed");
}
}
folders
.iter()
.map(|(name, role)| {
let id = self.mailbox(account, name)?;
self.conn.execute(
"UPDATE mailboxes SET role = ?1 WHERE id = ?2",
params![role.map(Role::name), id.0],
)?;
Ok(id)
})
.collect()
}
/// The account's mailboxes in name order.
pub fn folders(&self, account: AccountId) -> Result<Vec<(MailboxId, String, Option<Role>)>> {
let mut query = self
.conn
.prepare("SELECT id, name, role FROM mailboxes WHERE account = ?1 ORDER BY name")?;
let rows = query.query_map([account.0], |row| {
Ok((
MailboxId(row.get(0)?),
row.get(1)?,
row.get::<_, Option<String>>(2)?,
))
})?;
rows.map(|row| {
let (id, name, role) = row?;
let role = role
.map(|role| {
Role::from_name(&role)
.ok_or_else(|| Error::Store(format!("an unknown folder role {role:?}")))
})
.transpose()?;
Ok((id, name, role))
})
.collect()
}
/// Held messages by UID; mail under another UIDVALIDITY goes.
pub fn open_mailbox(
&mut self,
mailbox: MailboxId,
uidvalidity: u32,
) -> Result<HashMap<u32, Held>> {
self.writable()?;
let known: Option<u32> = self.conn.query_row(
"SELECT uidvalidity FROM mailboxes WHERE id = ?1",
[mailbox.0],
|row| row.get(0),
)?;
if known != Some(uidvalidity) {
if known.is_some() {
log::warn!("mail: mailbox {} has a new UIDVALIDITY; rebuilt", mailbox.0);
}
self.without_mail(mailbox, Some(uidvalidity), |tx| {
tx.execute(
"UPDATE mailboxes SET uidvalidity = ?1 WHERE id = ?2",
params![uidvalidity, mailbox.0],
)
})?;
}
let mut held: HashMap<u32, Held> = HashMap::new();
let mut query = self.conn.prepare(
"SELECT id, uid, content IS NOT NULL FROM messages
WHERE mailbox = ?1 AND uidvalidity = ?2",
)?;
let rows = query.query_map(params![mailbox.0, uidvalidity], |row| {
Ok((row.get::<_, u32>(1)?, MessageId(row.get(0)?), row.get(2)?))
})?;
for row in rows {
let (uid, id, body) = row?;
held.insert(
uid,
Held {
id,
body,
flags: Vec::new(),
},
);
}
let mut query = self.conn.prepare(
"SELECT messages.uid, flags.flag FROM flags JOIN messages ON flags.message = messages.id
WHERE messages.mailbox = ?1 AND messages.uidvalidity = ?2 ORDER BY flags.flag",
)?;
let rows = query.query_map(params![mailbox.0, uidvalidity], |row| {
Ok((row.get::<_, u32>(0)?, row.get::<_, String>(1)?))
})?;
for row in rows {
let (uid, flag) = row?;
held.get_mut(&uid)
.ok_or_else(|| Error::Store(format!("a flag for no message {uid}")))?
.flags
.push(flag);
}
Ok(held)
}
pub fn set_synced(&self, account: AccountId, at: i64) -> Result<()> {
self.writable()?;
let changed = self.conn.execute(
"UPDATE accounts SET synced = ?1 WHERE id = ?2",
params![at, account.0],
)?;
if changed != 1 {
return Err(Error::Store(format!("no account {}", account.0)));
}
Ok(())
}
pub fn synced(&self, account: AccountId) -> Result<Synced> {
let at = self.conn.query_row(
"SELECT synced FROM accounts WHERE id = ?1",
[account.0],
|row| row.get(0),
)?;
let (messages, cached, partial): (i64, i64, i64) = self.conn.query_row(
"SELECT count(*), count(content), count(CASE WHEN missing <> '[]' THEN 1 END)
FROM messages JOIN mailboxes ON messages.mailbox = mailboxes.id
WHERE mailboxes.account = ?1",
[account.0],
|row| Ok((row.get(0)?, row.get(1)?, row.get(2)?)),
)?;
let count =
|n: i64| u64::try_from(n).map_err(|_| Error::Store(format!("a negative count: {n}")));
Ok(Synced {
at,
messages: count(messages)?,
uncached: count(messages - cached)?,
partial: count(partial)?,
})
}
/// Deletes the mailbox's messages not under `keep`, with `change`.
fn without_mail(
&mut self,
mailbox: MailboxId,
keep: Option<u32>,
change: impl FnOnce(&Transaction) -> rusqlite::Result<usize>,
) -> Result<()> {
let hashes: Vec<String> = self
.conn
.prepare(&format!(
"SELECT DISTINCT content FROM messages WHERE {STALE} AND content IS NOT NULL"
))?
.query_map(params![mailbox.0, keep], |row| row.get(0))?
.collect::<rusqlite::Result<_>>()?;
let tx = self.conn.transaction()?;
let mut aside = Aside::new(&self.dir);
let done = (|| -> Result<()> {
tx.execute(
&format!("DELETE FROM messages WHERE {STALE}"),
params![mailbox.0, keep],
)?;
change(&tx)?;
for hash in &hashes {
aside.drop_unused(&tx, hash)?;
}
Ok(tx.commit()?)
})();
aside.settle(done)
}
}
const STALE: &str = "mailbox = ?1 AND (?2 IS NULL OR uidvalidity IS NOT ?2)";
+26 -12
View File
@@ -22,6 +22,11 @@ pub enum Key {
/// What a sync brought down.
pub enum Fetched<'a> {
Headers(&'a [u8]),
/// Left-out parts are empty and named by part number.
Text {
bytes: &'a [u8],
missing: &'a [String],
},
Whole(&'a [u8]),
}
@@ -40,13 +45,15 @@ pub struct Stored {
pub to: Vec<Mailbox>,
pub cc: Vec<Mailbox>,
pub date: Option<i64>,
/// None until the whole message is here.
/// None until the body is here.
pub attachments: Option<u32>,
pub whole: bool,
pub body: bool,
/// Part numbers not downloaded; their content is empty.
pub missing: Vec<String>,
}
const COLUMNS: &str = "id, mailbox, uidvalidity, uid, uidl, message_id, in_reply_to, refs, \
subject, sender, recipients, copies, date, attachments, content";
subject, sender, recipients, copies, date, attachments, content, missing";
impl Store {
pub fn find(&self, mailbox: MailboxId, key: &Key) -> Result<Option<MessageId>> {
@@ -75,7 +82,7 @@ impl Store {
) -> Result<MessageId> {
self.writable()?;
let existing = self.find(mailbox, key)?;
let bytes = match (fetched, existing) {
let (bytes, missing): (&[u8], &[String]) = match (fetched, existing) {
(Fetched::Headers(_), Some(id)) => {
self.set_flags(id, flags)?;
return Ok(id);
@@ -84,11 +91,13 @@ impl Store {
let mail = read(bytes)?;
return self.write(mailbox, key, flags, &mail, None, None);
}
(Fetched::Whole(bytes), _) => bytes,
(Fetched::Text { bytes, missing }, _) => (bytes, missing),
(Fetched::Whole(bytes), _) => (bytes, &[][..]),
};
let mail = read(bytes)?;
let (content, made) = self.keep(bytes)?;
let written = self.write(mailbox, key, flags, &mail, Some(&content), existing);
let body = Some((content.as_str(), missing));
let written = self.write(mailbox, key, flags, &mail, body, existing);
if written.is_err() && made {
let mut aside = Aside::new(&self.dir);
let gone = aside.drop_unused(&self.conn, &content);
@@ -105,9 +114,11 @@ impl Store {
key: &Key,
flags: &[&str],
mail: &Mail,
content: Option<&str>,
body: Option<(&str, &[String])>,
existing: Option<MessageId>,
) -> Result<MessageId> {
let content = body.map(|(content, _)| content);
let missing = body.map_or(&[][..], |(_, missing)| missing);
let (uidvalidity, uid, uidl) = match key {
Key::Imap { uidvalidity, uid } => (Some(*uidvalidity), Some(*uid), None),
Key::Pop { uidl } => (None, None, Some(uidl.as_str())),
@@ -119,6 +130,7 @@ impl Store {
serde_json::to_string(&mail.from)?,
serde_json::to_string(&mail.to)?,
serde_json::to_string(&mail.cc)?,
serde_json::to_string(missing)?,
];
let mut values: Vec<&dyn ToSql> = vec![
&mail.message_id,
@@ -131,6 +143,7 @@ impl Store {
&mail.date,
&attachments,
&content,
&json[5],
];
let tx = self.conn.transaction()?;
let mut aside = Aside::new(&self.dir);
@@ -147,7 +160,7 @@ impl Store {
tx.execute(
"UPDATE messages SET message_id = ?1, in_reply_to = ?2, refs = ?3,
subject = ?4, sender = ?5, recipients = ?6, copies = ?7, date = ?8,
attachments = ?9, content = ?10 WHERE id = ?11",
attachments = ?9, content = ?10, missing = ?11 WHERE id = ?12",
values.as_slice(),
)?;
*id
@@ -156,9 +169,9 @@ impl Store {
values.extend::<[&dyn ToSql; 4]>([&mailbox.0, &uidvalidity, &uid, &uidl]);
tx.execute(
"INSERT INTO messages (message_id, in_reply_to, refs, subject, sender,
recipients, copies, date, attachments, content, mailbox, uidvalidity,
uid, uidl) VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12,
?13, ?14)",
recipients, copies, date, attachments, content, missing, mailbox,
uidvalidity, uid, uidl) VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10,
?11, ?12, ?13, ?14, ?15)",
values.as_slice(),
)?;
MessageId(tx.last_insert_rowid())
@@ -295,7 +308,8 @@ fn row(row: &Row) -> Result<Stored> {
cc: serde_json::from_str(&json(11)?)?,
date: row.get(12)?,
attachments: row.get(13)?,
whole: row.get::<_, Option<String>>(14)?.is_some(),
body: row.get::<_, Option<String>>(14)?.is_some(),
missing: serde_json::from_str(&json(15)?)?,
})
}
+9 -1
View File
@@ -1,5 +1,5 @@
/// Step `n` upgrades version `n`.
pub(super) const STEPS: &[&str] = &[V1];
pub(super) const STEPS: &[&str] = &[V1, V2];
const V1: &str = "
CREATE TABLE accounts (
@@ -50,3 +50,11 @@ CREATE TRIGGER messages_gone AFTER DELETE ON messages BEGIN
DELETE FROM search WHERE rowid = old.id;
END;
";
/// IMAP sync (C12.6).
const V2: &str = "
ALTER TABLE accounts ADD COLUMN synced INTEGER;
ALTER TABLE mailboxes ADD COLUMN role TEXT;
ALTER TABLE mailboxes ADD COLUMN uidvalidity INTEGER;
ALTER TABLE messages ADD COLUMN missing TEXT NOT NULL DEFAULT '[]';
";
+43 -1
View File
@@ -2,8 +2,9 @@ use std::fs;
use rusqlite::Connection;
use super::folders::Synced;
use super::{BACKUPS, DB, open, schema::STEPS};
use crate::{Account, Auth, Error, Protocol, Security, Server};
use crate::{Account, Auth, Error, Protocol, Role, Security, Server};
fn account(address: &str) -> Account {
let server = Server {
@@ -110,3 +111,44 @@ fn no_backup_means_no_upgrade() {
drop(store);
assert_eq!(version(&dir.path().join(DB)), 1);
}
#[test]
fn a_v1_store_gains_sync_and_keeps_its_mail() {
let dir = tempfile::tempdir().unwrap();
let old = open(dir.path(), &[STEPS[0]]).unwrap();
let ann = old.add_account(&account("ann@example.com")).unwrap();
old.conn
.execute_batch(&format!(
"INSERT INTO mailboxes (account, name) VALUES ({}, 'INBOX');
INSERT INTO messages (mailbox, uidvalidity, uid, in_reply_to, refs,
sender, recipients, copies, content) VALUES
(1, 7, 1, '[]', '[]', '[]', '[]', '[]', '{}'),
(1, 6, 2, '[]', '[]', '[]', '[]', '[]', NULL);",
ann.0,
"a".repeat(64)
))
.unwrap();
drop(old);
let mut store = open(dir.path(), STEPS).unwrap();
assert_eq!(store.read_only(), None);
assert_eq!(version(&dir.path().join(DB)), STEPS.len() as i64);
assert_eq!(backups(dir.path()).len(), 1);
let synced = Synced {
at: None,
messages: 2,
uncached: 1,
partial: 0,
};
assert_eq!(store.synced(ann).unwrap(), synced);
store
.set_folders(ann, &[("INBOX", Some(Role::Inbox))])
.unwrap();
let [(inbox, name, role)] = store.folders(ann).unwrap().try_into().unwrap();
assert_eq!((name.as_str(), role), ("INBOX", Some(Role::Inbox)));
let held = store.open_mailbox(inbox, 7).unwrap();
assert_eq!(held.keys().copied().collect::<Vec<_>>(), [1]);
assert!(held[&1].body);
store.set_synced(ann, 9).unwrap();
assert_eq!(store.synced(ann).unwrap().at, Some(9));
assert_eq!(store.synced(ann).unwrap().messages, 1);
}
+227
View File
@@ -0,0 +1,227 @@
//! IMAP sync: folders, flags, deletions and the 90-day window (§12.3).
use std::{
collections::{HashMap, HashSet},
sync::{Mutex, MutexGuard},
};
use crate::{
Account, AccountId, Error, Fetched, Got, Held, Imap, Key, MailboxId, Result, Secret, Store,
TlsConnector, Vault, oauth,
};
/// Bodies come down for mail this recent.
pub const BODY_DAYS: i64 = 90;
/// UIDs per FETCH.
const BATCH: usize = 100;
/// What one sync changed.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
pub struct Report {
pub folders: usize,
pub headers: usize,
pub bodies: usize,
pub removed: usize,
}
/// The password, or an access token from the refresh token.
pub async fn secret(account: &Account, vault: &Vault) -> Result<Secret> {
let name = account.secret_name();
let kept = vault
.get(&name)?
.ok_or_else(|| Error::Vault(format!("no saved secret for {name}")))?;
let Some(provider) = account.auth.provider() else {
return Ok(Secret::Password(kept));
};
let tokens = oauth::refresh(&provider?, &kept).await?;
if let Some(next) = &tokens.refresh {
vault.put(&name, next);
}
Ok(Secret::Token(tokens.access))
}
/// One pass over every folder; `now` is Unix seconds.
pub async fn imap(
account: &Account,
id: AccountId,
secret: Secret,
tls: &TlsConnector,
store: &Mutex<Store>,
now: i64,
) -> Result<Report> {
let mut imap = Imap::connect(&account.incoming, &account.login(secret), tls).await?;
let folders: Vec<_> = imap
.folders()
.await?
.into_iter()
.filter(|folder| folder.selectable)
.collect();
let named: Vec<_> = folders
.iter()
.map(|folder| (folder.name.as_str(), folder.role))
.collect();
let boxes = lock(store)?.set_folders(id, &named)?;
let day = since(now)?;
let mut report = Report {
folders: folders.len(),
..Report::default()
};
for (folder, mailbox) in folders.iter().zip(boxes) {
one(&mut imap, store, mailbox, &folder.name, &day, &mut report).await?;
}
imap.logout().await?;
lock(store)?.set_synced(id, now)?;
log::info!(
"mail: account {} synced: {} folders, {} headers, {} bodies, {} removed",
id.0,
report.folders,
report.headers,
report.bodies,
report.removed
);
Ok(report)
}
async fn one(
imap: &mut Imap,
store: &Mutex<Store>,
mailbox: MailboxId,
name: &str,
day: &str,
report: &mut Report,
) -> Result<()> {
let opened = imap.examine(name).await?;
let held = lock(store)?.open_mailbox(mailbox, opened.uidvalidity)?;
if opened.exists == 0 {
return remove(store, held.values(), report);
}
let server: HashMap<u32, Vec<String>> = imap.flags().await?.into_iter().collect();
let gone = held
.iter()
.filter(|(uid, _)| !server.contains_key(uid))
.map(|(_, held)| held);
remove(store, gone, report)?;
for (uid, held) in &held {
match server.get(uid) {
Some(flags) if *flags != held.flags => {
lock(store)?.set_flags(held.id, &strs(flags))?;
}
_ => {}
}
}
let recent: HashSet<u32> = imap.since(day).await?.into_iter().collect();
let mut headers = Vec::new();
let mut bodies = Vec::new();
for uid in server.keys() {
match (held.get(uid), recent.contains(uid)) {
(None | Some(Held { body: false, .. }), true) => bodies.push(*uid),
(None, false) => headers.push(*uid),
_ => {}
}
}
headers.sort_unstable();
bodies.sort_unstable();
let key = |uid| Key::Imap {
uidvalidity: opened.uidvalidity,
uid,
};
for batch in headers.chunks(BATCH) {
for got in imap.headers(batch).await? {
put(
store,
mailbox,
key(got.uid),
&got,
Fetched::Headers(&got.bytes),
)?;
report.headers += 1;
}
}
for batch in bodies.chunks(BATCH) {
let (whole, partial): (Vec<_>, Vec<_>) = imap
.structures(batch)
.await?
.into_iter()
.partition(|(_, node)| node.complete());
let whole: Vec<u32> = whole.into_iter().map(|(uid, _)| uid).collect();
if !whole.is_empty() {
for got in imap.bodies(&whole).await? {
put(
store,
mailbox,
key(got.uid),
&got,
Fetched::Whole(&got.bytes),
)?;
report.bodies += 1;
}
}
// §12.3: attachments wait for a request.
for (uid, node) in partial {
let Some(got) = imap.text(uid, &node).await? else {
continue;
};
let missing = node.missing();
let text = Fetched::Text {
bytes: &got.bytes,
missing: &missing,
};
put(store, mailbox, key(uid), &got, text)?;
report.bodies += 1;
}
}
Ok(())
}
fn put(
store: &Mutex<Store>,
mailbox: MailboxId,
key: Key,
got: &Got,
fetched: Fetched,
) -> Result<()> {
lock(store)?.put(mailbox, &key, &strs(&got.flags), fetched)?;
Ok(())
}
fn remove<'a>(
store: &Mutex<Store>,
gone: impl Iterator<Item = &'a Held>,
report: &mut Report,
) -> Result<()> {
for held in gone {
lock(store)?.remove(held.id)?;
report.removed += 1;
}
Ok(())
}
fn strs(flags: &[String]) -> Vec<&str> {
flags.iter().map(String::as_str).collect()
}
/// `SINCE` takes a date in English: `19-Jul-2026`.
fn since(now: i64) -> Result<String> {
let start = jiff::Timestamp::from_second(now - BODY_DAYS * 86_400)
.map_err(|err| Error::Store(format!("a sync time: {err}")))?;
Ok(start
.to_zoned(jiff::tz::TimeZone::UTC)
.strftime("%-d-%b-%Y")
.to_string())
}
fn lock(store: &Mutex<Store>) -> Result<MutexGuard<'_, Store>> {
store
.lock()
.map_err(|_| Error::Store("a store task panicked".into()))
}
#[cfg(test)]
mod tests {
#[test]
fn the_window_starts_ninety_days_back() {
// 2026-10-19 12:00 UTC
assert_eq!(super::since(1_792_411_200).unwrap(), "21-Jul-2026");
}
}
+37 -13
View File
@@ -1,7 +1,7 @@
mod fake;
use base64::{Engine, engine::general_purpose::STANDARD};
use ely_mail_engine::{Error, Imap, Secret, Security, system_tls};
use ely_mail_engine::{Error, Imap, Role, Secret, Security, system_tls};
use fake::{certs, expect, login, send, serve};
use tokio::task::JoinHandle;
@@ -19,13 +19,15 @@ async fn a_password_login_lists_and_fetches() {
send("* OK ready\n"),
expect("{tag} LOGIN \"ann@example.com\" \"pw 1\""),
send("{tag} OK done\n"),
expect("{tag} CAPABILITY"),
send("* CAPABILITY IMAP4rev1\n{tag} OK caps\n"),
expect("{tag} LIST \"\" *"),
send("* LIST (\\HasNoChildren) \"/\" INBOX\n"),
send("* LIST (\\HasNoChildren \\Sent) \"/\" \"Sent Mail\"\n"),
send("{tag} OK listed\n"),
expect("{tag} EXAMINE \"INBOX\""),
send("* 1 EXISTS\n* OK [UIDVALIDITY 7] v\n{tag} OK [READ-ONLY] in\n"),
expect("{tag} UID FETCH 42 BODY.PEEK[]"),
expect("{tag} UID FETCH 42 (UID FLAGS BODY.PEEK[])"),
send(&format!(
"* 1 FETCH (UID 42 BODY[] {{{}}}\n{lines})\n{{tag}} OK fetched\n",
body.len()
@@ -42,7 +44,15 @@ async fn a_password_login_lists_and_fetches() {
)
.await
.unwrap();
assert_eq!(imap.mailboxes().await.unwrap(), ["INBOX", "Sent Mail"]);
let folders = imap.folders().await.unwrap();
let names: Vec<_> = folders.iter().map(|f| (f.name.as_str(), f.role)).collect();
assert_eq!(
names,
[
("INBOX", Some(Role::Inbox)),
("Sent Mail", Some(Role::Sent))
]
);
let fetched = imap.fetch("INBOX", 42).await.unwrap().unwrap();
assert_eq!(String::from_utf8(fetched).unwrap(), body);
imap.logout().await.unwrap();
@@ -151,8 +161,8 @@ async fn a_missing_uid_is_none() {
expect("{tag} LOGIN \"ann@example.com\" \"pw\""),
send("{tag} OK done\n"),
expect("{tag} EXAMINE \"INBOX\""),
send("{tag} OK in\n"),
expect("{tag} UID FETCH 9 BODY.PEEK[]"),
send("* 1 EXISTS\n* OK [UIDVALIDITY 7] v\n{tag} OK in\n"),
expect("{tag} UID FETCH 9 (UID FLAGS BODY.PEEK[])"),
send("{tag} OK nothing\n"),
],
)
@@ -178,6 +188,8 @@ async fn an_unreadable_reply_is_a_protocol_error() {
send("* OK ready\n"),
expect("{tag} LOGIN \"ann@example.com\" \"pw\""),
send("{tag} OK done\n"),
expect("{tag} CAPABILITY"),
send("* CAPABILITY IMAP4rev1\n{tag} OK caps\n"),
expect("{tag} LIST \"\" *"),
send("* LIST (((\n"),
],
@@ -190,7 +202,7 @@ async fn an_unreadable_reply_is_a_protocol_error() {
)
.await
.unwrap();
let err = imap.mailboxes().await.unwrap_err();
let err = imap.folders().await.unwrap_err();
assert!(matches!(err, Error::Protocol(_)), "{err}");
script.await.unwrap().unwrap();
}
@@ -257,18 +269,26 @@ async fn logged_in(steps: Vec<fake::Step>) -> (Imap, JoinHandle<Result<(), Strin
#[tokio::test]
async fn listed_names_lose_quote_escapes_and_open_as_listed() {
let (mut imap, script) = logged_in(vec![
expect("{tag} CAPABILITY"),
send("* CAPABILITY IMAP4rev1\n{tag} OK caps\n"),
expect("{tag} LIST \"\" *"),
send("* LIST () \"/\" \"a\\\"b\"\n"),
send("* LIST () \"/\" \"c\\\\d\"\n"),
send("* LIST () \"/\" {3}\ne\\\"\n"),
send("{tag} OK listed\n"),
expect("{tag} EXAMINE \"a\\\"b\""),
send("{tag} OK in\n"),
expect("{tag} UID FETCH 9 BODY.PEEK[]"),
send("* 1 EXISTS\n* OK [UIDVALIDITY 7] v\n{tag} OK in\n"),
expect("{tag} UID FETCH 9 (UID FLAGS BODY.PEEK[])"),
send("{tag} OK nothing\n"),
])
.await;
let names = imap.mailboxes().await.unwrap();
let names: Vec<_> = imap
.folders()
.await
.unwrap()
.into_iter()
.map(|f| f.name)
.collect();
assert_eq!(names, ["a\"b", "c\\d", "e\\\""]);
assert_eq!(imap.fetch(&names[0], 9).await.unwrap(), None);
script.await.unwrap().unwrap();
@@ -277,12 +297,14 @@ async fn listed_names_lose_quote_escapes_and_open_as_listed() {
#[tokio::test]
async fn a_refused_list_keeps_the_server_words() {
let (mut imap, script) = logged_in(vec![
expect("{tag} CAPABILITY"),
send("* CAPABILITY IMAP4rev1\n{tag} OK caps\n"),
expect("{tag} LIST \"\" *"),
send("* LIST () \"/\" INBOX\n"),
send("{tag} NO [UNAVAILABLE] Mailbox listing temporarily unavailable\n"),
])
.await;
let err = imap.mailboxes().await.unwrap_err();
let err = imap.folders().await.unwrap_err();
assert!(
matches!(&err, Error::Server(text) if text.contains("listing temporarily unavailable")),
"{err}"
@@ -294,8 +316,8 @@ async fn a_refused_list_keeps_the_server_words() {
async fn a_refused_fetch_is_not_a_missing_uid() {
let (mut imap, script) = logged_in(vec![
expect("{tag} EXAMINE \"INBOX\""),
send("{tag} OK in\n"),
expect("{tag} UID FETCH 9 BODY.PEEK[]"),
send("* 1 EXISTS\n* OK [UIDVALIDITY 7] v\n{tag} OK in\n"),
expect("{tag} UID FETCH 9 (UID FLAGS BODY.PEEK[])"),
send("{tag} NO [UNAVAILABLE] Message store temporarily unavailable\n"),
])
.await;
@@ -335,11 +357,13 @@ async fn a_bye_after_login_keeps_the_server_words() {
script.await.unwrap().unwrap();
let (mut imap, script) = logged_in(vec![
expect("{tag} CAPABILITY"),
send("* CAPABILITY IMAP4rev1\n{tag} OK caps\n"),
expect("{tag} LIST \"\" *"),
send("* BYE Shutting down\n"),
])
.await;
let err = imap.mailboxes().await.unwrap_err();
let err = imap.folders().await.unwrap_err();
assert!(
matches!(&err, Error::Server(text) if text.contains("Shutting down")),
"{err}"
+2 -2
View File
@@ -27,8 +27,8 @@ async fn trace_logs_keep_no_password_token_or_mail() {
expect(&format!("{{tag}} LOGIN \"ann@example.com\" \"{PASSWORD}\"")),
send("{tag} OK in\n"),
expect("{tag} EXAMINE \"INBOX\""),
send("{tag} OK done\n"),
expect("{tag} UID FETCH 7 BODY.PEEK[]"),
send("* 1 EXISTS\n* OK [UIDVALIDITY 7] v\n{tag} OK done\n"),
expect("{tag} UID FETCH 7 (UID FLAGS BODY.PEEK[])"),
send(&format!(
"* 1 FETCH (UID 7 BODY[] {{{}}}\n{message})\n{{tag}} OK fetched\n",
message.replace('\n', "\r\n").len()
+2 -2
View File
@@ -71,7 +71,7 @@ fn a_whole_message_is_indexed_and_kept_by_its_hash() {
assert_eq!(stored.message_id.as_deref(), Some("m1@example.com"));
assert_eq!(stored.flags, ["\\Seen"]);
assert_eq!(stored.attachments, Some(1));
assert!(stored.whole);
assert!(stored.body);
assert_eq!(store.original(id).unwrap().as_deref(), Some(raw.as_slice()));
assert_eq!(std::fs::read(original_path(dir.path(), &raw)).unwrap(), raw);
assert_eq!(found(dir.path(), "numbers"), [id.0]);
@@ -92,7 +92,7 @@ fn headers_first_then_the_whole_message() {
.unwrap();
let stored = store.get(id).unwrap();
assert_eq!(stored.subject.as_deref(), Some("Café report"));
assert_eq!((stored.whole, stored.attachments), (false, None));
assert_eq!((stored.body, stored.attachments), (false, None));
assert_eq!(store.original(id).unwrap(), None);
assert_eq!(found(dir.path(), "report"), [id.0]);
assert!(found(dir.path(), "numbers").is_empty());
+409
View File
@@ -0,0 +1,409 @@
mod fake;
use std::sync::Mutex;
use ely_mail_engine::{
Account, AccountId, Error, Key, MailboxId, Role, Secret, Server, Store, sync,
};
use fake::{account, certs, expect, send, serve};
use tokio::task::JoinHandle;
/// 2026-10-19 12:00 UTC; bodies from 21 July.
const NOW: i64 = 1_792_411_200;
const SINCE: &str = "{tag} UID SEARCH SINCE 21-Jul-2026";
const OLD: &str = "Subject: Old\nFrom: a@example.com\n\n";
const NEW: &str = "Subject: New\nFrom: b@example.com\n\nfresh\n";
/// A FETCH line carrying `text` as a literal.
fn literal(uid: u32, flags: &str, section: &str, text: &str) -> String {
let crlf = text.replace('\n', "\r\n");
format!(
"* {uid} FETCH (UID {uid} FLAGS ({flags}) {section} {{{}}}\n{text})\n",
crlf.len()
)
}
/// A one-part text message's structure, and the tag.
fn plain(uid: u32) -> String {
format!(
"* {uid} FETCH (UID {uid} BODYSTRUCTURE (\"TEXT\" \"PLAIN\" NIL NIL NIL \"7BIT\" 7 1))\n\
{{tag}} OK structure\n"
)
}
async fn server(steps: Vec<fake::Step>) -> (Account, JoinHandle<Result<(), String>>, fake::Certs) {
let certs = certs();
let mut script = vec![
send("* OK ready\n"),
expect("{tag} LOGIN \"ann@example.com\" \"pw\""),
send("{tag} OK done\n"),
];
script.extend(steps);
script.push(expect("{tag} LOGOUT"));
script.push(send("* BYE\n{tag} OK bye\n"));
let (incoming, task) = serve(&certs, ely_mail_engine::Security::Tls, script).await;
let account = Account {
incoming: Server { ..incoming },
..account("ann@example.com")
};
(account, task, certs)
}
async fn run(
store: &Mutex<Store>,
id: AccountId,
steps: Vec<fake::Step>,
) -> Result<sync::Report, Error> {
let (account, task, certs) = server(steps).await;
let secret = Secret::Password("pw".into());
let done = sync::imap(&account, id, secret, &certs.trusted, store, NOW).await;
if done.is_ok() {
task.await.unwrap().unwrap();
}
done
}
fn first_sync() -> Vec<fake::Step> {
vec![
expect("{tag} CAPABILITY"),
send("* CAPABILITY IMAP4rev1\n{tag} OK caps\n"),
expect("{tag} LIST \"\" *"),
send("* LIST (\\HasNoChildren) \"/\" INBOX\n"),
send("* LIST (\\HasNoChildren \\Sent) \"/\" Sent\n"),
send("* LIST (\\Noselect) \"/\" Archive\n"),
send("{tag} OK listed\n"),
expect("{tag} EXAMINE \"INBOX\""),
send("* 2 EXISTS\n* OK [UIDVALIDITY 7] v\n{tag} OK in\n"),
expect("{tag} UID FETCH 1:* (UID FLAGS)"),
send("* 1 FETCH (UID 1 FLAGS (\\Seen))\n* 2 FETCH (UID 2 FLAGS (\\Recent))\n"),
send("{tag} OK flags\n"),
expect(SINCE),
send("* SEARCH 2\n{tag} OK found\n"),
expect("{tag} UID FETCH 1 (UID FLAGS BODY.PEEK[HEADER])"),
send(&literal(1, "\\Seen", "BODY[HEADER]", OLD)),
send("{tag} OK headers\n"),
expect("{tag} UID FETCH 2 (UID BODYSTRUCTURE)"),
send(&plain(2)),
expect("{tag} UID FETCH 2 (UID FLAGS BODY.PEEK[])"),
send(&literal(2, "\\Recent", "BODY[]", NEW)),
send("{tag} OK bodies\n"),
expect("{tag} EXAMINE \"Sent\""),
send("* 0 EXISTS\n* OK [UIDVALIDITY 3] v\n{tag} OK in\n"),
]
}
fn key(uidvalidity: u32, uid: u32) -> Key {
Key::Imap { uidvalidity, uid }
}
fn inbox(store: &Store, id: AccountId) -> MailboxId {
store.folders(id).unwrap()[0].0
}
fn opened() -> (tempfile::TempDir, Mutex<Store>, AccountId) {
let dir = tempfile::tempdir().unwrap();
let store = Store::open(dir.path()).unwrap();
let id = store.add_account(&account("ann@example.com")).unwrap();
(dir, Mutex::new(store), id)
}
#[tokio::test]
async fn a_first_sync_takes_headers_and_recent_bodies() {
let (_dir, store, id) = opened();
let report = run(&store, id, first_sync()).await.unwrap();
assert_eq!(
report,
sync::Report {
folders: 2,
headers: 1,
bodies: 1,
removed: 0
}
);
let store = store.lock().unwrap();
let folders: Vec<_> = store
.folders(id)
.unwrap()
.into_iter()
.map(|(_, name, role)| (name, role))
.collect();
assert_eq!(
folders,
[
("INBOX".to_owned(), Some(Role::Inbox)),
("Sent".to_owned(), Some(Role::Sent))
]
);
let inbox = inbox(&store, id);
let old = store
.get(store.find(inbox, &key(7, 1)).unwrap().unwrap())
.unwrap();
assert_eq!((old.body, old.flags), (false, vec!["\\Seen".to_owned()]));
assert_eq!(old.subject.as_deref(), Some("Old"));
let new = store.find(inbox, &key(7, 2)).unwrap().unwrap();
assert_eq!(
store.original(new).unwrap().unwrap(),
NEW.replace('\n', "\r\n").as_bytes()
);
assert!(
store.get(new).unwrap().flags.is_empty(),
"\\Recent is not kept"
);
let synced = store.synced(id).unwrap();
assert_eq!(
(synced.at, synced.messages, synced.uncached),
(Some(NOW), 2, 1)
);
}
#[tokio::test]
async fn deletions_flags_and_dropped_folders_follow_the_server() {
let (_dir, store, id) = opened();
run(&store, id, first_sync()).await.unwrap();
let report = run(
&store,
id,
vec![
expect("{tag} CAPABILITY"),
send("* CAPABILITY IMAP4rev1\n{tag} OK caps\n"),
expect("{tag} LIST \"\" *"),
send("* LIST () \"/\" INBOX\n{tag} OK listed\n"),
expect("{tag} EXAMINE \"INBOX\""),
send("* 1 EXISTS\n* OK [UIDVALIDITY 7] v\n{tag} OK in\n"),
expect("{tag} UID FETCH 1:* (UID FLAGS)"),
send("* 1 FETCH (UID 1 FLAGS (\\Seen \\Flagged))\n{tag} OK flags\n"),
expect(SINCE),
send("* SEARCH\n{tag} OK found\n"),
],
)
.await
.unwrap();
assert_eq!(report.removed, 1);
let store = store.lock().unwrap();
assert_eq!(store.folders(id).unwrap().len(), 1);
let inbox = inbox(&store, id);
assert_eq!(store.find(inbox, &key(7, 2)).unwrap(), None);
let old = store
.get(store.find(inbox, &key(7, 1)).unwrap().unwrap())
.unwrap();
assert_eq!(old.flags, ["\\Flagged", "\\Seen"]);
assert_eq!(store.sizes().unwrap().originals, 0);
}
#[tokio::test]
async fn a_new_uidvalidity_rebuilds_the_mailbox() {
let (_dir, store, id) = opened();
run(&store, id, first_sync()).await.unwrap();
run(
&store,
id,
vec![
expect("{tag} CAPABILITY"),
send("* CAPABILITY IMAP4rev1\n{tag} OK caps\n"),
expect("{tag} LIST \"\" *"),
send("* LIST () \"/\" INBOX\n{tag} OK listed\n"),
expect("{tag} EXAMINE \"INBOX\""),
send("* 1 EXISTS\n* OK [UIDVALIDITY 8] v\n{tag} OK in\n"),
expect("{tag} UID FETCH 1:* (UID FLAGS)"),
send("* 1 FETCH (UID 1 FLAGS ())\n{tag} OK flags\n"),
expect(SINCE),
send("* SEARCH 1\n{tag} OK found\n"),
expect("{tag} UID FETCH 1 (UID BODYSTRUCTURE)"),
send(&plain(1)),
expect("{tag} UID FETCH 1 (UID FLAGS BODY.PEEK[])"),
send(&literal(1, "", "BODY[]", NEW)),
send("{tag} OK bodies\n"),
],
)
.await
.unwrap();
let store = store.lock().unwrap();
let inbox = inbox(&store, id);
assert_eq!(store.find(inbox, &key(7, 1)).unwrap(), None);
let fresh = store.find(inbox, &key(8, 1)).unwrap().unwrap();
assert_eq!(store.get(fresh).unwrap().subject.as_deref(), Some("New"));
assert_eq!(store.synced(id).unwrap().messages, 1);
}
#[tokio::test]
async fn a_failed_sync_keeps_the_last_time() {
let (_dir, store, id) = opened();
let err = run(
&store,
id,
vec![
expect("{tag} CAPABILITY"),
send("* CAPABILITY IMAP4rev1\n{tag} OK caps\n"),
expect("{tag} LIST \"\" *"),
send("{tag} NO [UNAVAILABLE] Try again later\n"),
],
)
.await
.unwrap_err();
assert!(
matches!(&err, Error::Server(text) if text.contains("Try again later")),
"{err}"
);
assert_eq!(store.lock().unwrap().synced(id).unwrap().at, None);
}
/// INBOX: UID 1 old, UID 2 recent; up to SEARCH.
fn inbox_of_two() -> Vec<fake::Step> {
vec![
expect("{tag} CAPABILITY"),
send("* CAPABILITY IMAP4rev1\n{tag} OK caps\n"),
expect("{tag} LIST \"\" *"),
send("* LIST () \"/\" INBOX\n{tag} OK listed\n"),
expect("{tag} EXAMINE \"INBOX\""),
send("* 2 EXISTS\n* OK [UIDVALIDITY 7] v\n{tag} OK in\n"),
expect("{tag} UID FETCH 1:* (UID FLAGS)"),
send("* 1 FETCH (UID 1 FLAGS (\\Seen))\n* 2 FETCH (UID 2 FLAGS ())\n"),
send("{tag} OK flags\n"),
expect(SINCE),
send("* SEARCH 2\n{tag} OK found\n"),
]
}
#[tokio::test]
async fn an_examine_without_exists_does_not_delete_mail() {
let (_dir, store, id) = opened();
run(&store, id, first_sync()).await.unwrap();
let err = run(
&store,
id,
vec![
expect("{tag} CAPABILITY"),
send("* CAPABILITY IMAP4rev1\n{tag} OK caps\n"),
expect("{tag} LIST \"\" *"),
send("* LIST () \"/\" INBOX\n* LIST () \"/\" Sent\n{tag} OK listed\n"),
expect("{tag} EXAMINE \"INBOX\""),
send("* OK [UIDVALIDITY 7] v\n{tag} OK in\n"),
],
)
.await
.unwrap_err();
assert!(
matches!(&err, Error::Protocol(text) if text.contains("EXISTS")),
"{err}"
);
let store = store.lock().unwrap();
let synced = store.synced(id).unwrap();
assert_eq!((synced.at, synced.messages), (Some(NOW), 2));
}
#[tokio::test]
async fn concurrent_flags_do_not_make_a_body_fetch_fail() {
let (_dir, store, id) = opened();
let mut steps = first_sync();
let body = steps
.iter()
.position(|step| matches!(step, fake::Step::Send(text) if text.contains("BODY[]")))
.unwrap();
steps.insert(body + 1, send("* 2 FETCH (UID 2 FLAGS (\\Seen))\n"));
run(&store, id, steps).await.unwrap();
let store = store.lock().unwrap();
let new = store.find(inbox(&store, id), &key(7, 2)).unwrap().unwrap();
let new = store.get(new).unwrap();
assert_eq!((new.body, new.flags), (true, vec!["\\Seen".to_owned()]));
}
#[tokio::test]
async fn inbox_casing_does_not_destroy_cached_mail() {
let (_dir, store, id) = opened();
run(&store, id, first_sync()).await.unwrap();
let mut store = store.lock().unwrap();
let kept = store
.set_folders(id, &[("inbox", Some(Role::Inbox))])
.unwrap();
let folders = store.folders(id).unwrap();
assert_eq!(folders.len(), 1);
assert_eq!((kept[0], folders[0].1.as_str()), (folders[0].0, "INBOX"));
let new = store.find(kept[0], &key(7, 2)).unwrap().unwrap();
assert_eq!(
store.original(new).unwrap().unwrap(),
NEW.replace('\n', "\r\n").as_bytes()
);
}
#[tokio::test]
async fn roles_are_asked_for_when_the_server_can_list_them() {
let (_dir, store, id) = opened();
run(
&store,
id,
vec![
expect("{tag} CAPABILITY"),
send("* CAPABILITY IMAP4rev1 LIST-EXTENDED SPECIAL-USE\n{tag} OK caps\n"),
expect("{tag} LIST \"\" * RETURN (SPECIAL-USE)"),
send("* LIST (\\Sent) \"/\" Outbox\n{tag} OK listed\n"),
expect("{tag} EXAMINE \"Outbox\""),
send("* 0 EXISTS\n* OK [UIDVALIDITY 3] v\n{tag} OK in\n"),
],
)
.await
.unwrap();
let folders = store.lock().unwrap().folders(id).unwrap();
assert_eq!(folders[0].2, Some(Role::Sent));
}
/// One FETCH line carrying each section as a literal.
fn sections(uid: u32, parts: &[(&str, &str)]) -> String {
let mut line = format!("* {uid} FETCH (UID {uid} FLAGS ()");
for (name, text) in parts {
let crlf = text.replace('\n', "\r\n");
line.push_str(&format!(" BODY[{name}] {{{}}}\n{text}", crlf.len()));
}
line + ")\n{tag} OK text\n"
}
#[tokio::test]
async fn a_recent_body_sync_leaves_attachments_on_the_server() {
let (_dir, store, id) = opened();
let header = "Subject: Report\nFrom: b@example.com\nMIME-Version: 1.0\n\
Content-Type: multipart/mixed; boundary=\"b1\"\n\n";
let structure = "* 2 FETCH (UID 2 BODYSTRUCTURE ((\"TEXT\" \"PLAIN\" (\"CHARSET\" \"us-ascii\") \
NIL NIL \"7BIT\" 6 1)(\"APPLICATION\" \"PDF\" (\"NAME\" \"report.pdf\") NIL NIL \
\"BASE64\" 12 NIL (\"ATTACHMENT\" (\"FILENAME\" \"report.pdf\")) NIL NIL) \
\"MIXED\" (\"BOUNDARY\" \"b1\") NIL NIL NIL))\n{tag} OK structure\n";
let mut steps = inbox_of_two();
steps.extend([
expect("{tag} UID FETCH 1 (UID FLAGS BODY.PEEK[HEADER])"),
send(&literal(1, "\\Seen", "BODY[HEADER]", OLD)),
send("{tag} OK headers\n"),
expect("{tag} UID FETCH 2 (UID BODYSTRUCTURE)"),
send(structure),
expect(
"{tag} UID FETCH 2 (UID FLAGS BODY.PEEK[HEADER] BODY.PEEK[1.MIME] BODY.PEEK[1] \
BODY.PEEK[2.MIME])",
),
send(&sections(
2,
&[
("HEADER", header),
("1.MIME", "Content-Type: text/plain; charset=us-ascii\n\n"),
("1", "hello\n"),
(
"2.MIME",
"Content-Type: application/pdf; name=\"report.pdf\"\n\
Content-Disposition: attachment; filename=\"report.pdf\"\n\
Content-Transfer-Encoding: base64\n\n",
),
],
)),
]);
run(&store, id, steps).await.unwrap();
let store = store.lock().unwrap();
let report = store.find(inbox(&store, id), &key(7, 2)).unwrap().unwrap();
let stored = store.get(report).unwrap();
assert_eq!(
(stored.body, stored.missing.clone()),
(true, vec!["2".to_owned()])
);
assert_eq!(stored.attachments, Some(1));
let mail = ely_mail_engine::read(&store.original(report).unwrap().unwrap()).unwrap();
assert_eq!(mail.text.as_deref().map(str::trim_end), Some("hello"));
assert_eq!(mail.attachments[0].name.as_deref(), Some("report.pdf"));
assert!(mail.attachments[0].bytes.is_empty());
assert_eq!(store.synced(id).unwrap().partial, 1);
}
+41
View File
@@ -111,6 +111,34 @@ pub(super) const EN: &[(&str, &str)] = &[
("mail.remove.go", "Remove Account"),
("mail.remove.done", "{address} removed."),
("mail.remove.failed", "The account was not removed: {error}"),
("mail.sync.last", "Last synced"),
("mail.sync.running", "Syncing…"),
("mail.sync.never", "Not yet"),
("mail.sync.range", "Downloaded"),
(
"mail.sync.range_value",
"Headers of all mail; text from the last {days} days; attachments when asked",
),
("mail.sync.uncached", "Not downloaded"),
(
"mail.sync.uncached_value",
"{n} messages have only their headers here",
),
(
"mail.sync.uncached_value.one",
"1 message has only its headers here",
),
("mail.sync.partial", "Attachments"),
(
"mail.sync.partial_value",
"{n} messages have attachments not downloaded",
),
(
"mail.sync.partial_value.one",
"1 message has attachments not downloaded",
),
("mail.sync.failed", "The last sync failed: {error}"),
("mail.sync.unread", "The sync state could not be read: {error}"),
];
pub(super) const ZH_CN: &[(&str, &str)] = &[
@@ -212,4 +240,17 @@ pub(super) const ZH_CN: &[(&str, &str)] = &[
("mail.remove.go", "删除账号"),
("mail.remove.done", "已删除 {address}。"),
("mail.remove.failed", "账号没有删除:{error}"),
("mail.sync.last", "上次同步"),
("mail.sync.running", "正在同步…"),
("mail.sync.never", "尚未同步"),
("mail.sync.range", "已下载"),
("mail.sync.range_value", "全部邮件的邮件头;最近 {days} 天的正文;附件按需下载"),
("mail.sync.uncached", "未下载"),
("mail.sync.uncached_value", "{n} 封邮件只有邮件头"),
("mail.sync.uncached_value.one", "1 封邮件只有邮件头"),
("mail.sync.partial", "附件"),
("mail.sync.partial_value", "{n} 封邮件的附件未下载"),
("mail.sync.partial_value.one", "1 封邮件的附件未下载"),
("mail.sync.failed", "上次同步失败:{error}"),
("mail.sync.unread", "读不出同步状态:{error}"),
];
+1 -1
View File
@@ -139,7 +139,7 @@
- [x] C12.3 本地存储(§12.3、§18.1、§18.3):SQLite 索引(账号、邮箱、消息身份为账号加邮箱加 UIDVALIDITY 加 UID、标记、头字段)、原文按内容存进内容目录、FTS5 全文索引(可重建)、结构版本与升级前备份、升级失败只读;各类大小可查。已做:`Store`(rusqlite,自带 SQLite 3.53.2,FTS5 与 dbstat)在给定文件夹开 `mail.db`(WAL,外键开);表 accounts、mailboxes(账号加名字唯一)、messages(身份为邮箱加 UIDVALIDITY 加 UID,POP3 为邮箱加 UIDL,两者恰居其一,各有部分唯一索引;头字段:Message-ID、In-Reply-To、References、主题、发件人、收件人、抄送(JSON)、日期;附件数在整封到达前为空)、flags(每标记一行);原文按 SHA-256 存在 `content/前两位/散列`(暂存后改名,同内容只存一份,读时核对散列,不符即报“已损坏”,最后一封引用它的邮件删去时删文件);同一身份再来头部只改标记,整封到达即补原文、附件数与正文索引;FTS5 表 `search`(主题、人、正文、附件名,无内容副本,trigram 分词,删邮件的触发器同删索引行),HTML 独有的正文以 html5ever 分词器取文字(略去 script 与 style,标签处断词);`rebuild_search` 由索引行与原文重建;`PRAGMA user_version` 记结构版本,升级前以 `VACUUM INTO` 备份到 `backups/`,各步在一个事务里,备份或升级失败即以只读重开、写入报 `ReadOnly` 并带原因,更新的版本写的库同样只读;`sizes` 给索引、全文索引(dbstat)与原文三类字节数。测试:结构版本 5 个(新库、升级先备份、失败回滚且只读、更新的版本只读、无备份不升级),存储 6 个(整封、先头后整封、UIDVALIDITY 与 POP 身份、共用原文、损坏原文、HTML 正文与重建),去掉各自的规则即失败;引擎共 72 个测试。第 1 轮审查后:重建先删去整张全文表再按库里记的定义重建(损坏的段删不掉);存原文时已有的文件先核对散列,坏了即以本次收到的字节改写;写库失败时删去本次新建且无人引用的原文;`sizes` 的索引按 `mail.db`、`-wal`、`-shm` 的实际字节减去全文页;HTML 取文字照树构建器给分词器的状态(script 为脚本数据,style、iframe、noembed、noframes 为原始文本,title 与 textarea 为 RCDATA,xmp 与 plaintext 照显示,自闭合在 HTML 元素上无效;SVG 与 MathML 里自闭合有效、script 与 style 的文字略去);给不存在的邮件设标记即报错。`tests/store_repair.rs` 6 个,去掉各自的修复即失败;引擎共 78 个测试。第 2 轮审查后:HTML 取文字改走 html5ever 的整个树构建器(`store/html.rs`,自建的节点表作 `TreeSink`,脚本关闭),命名空间、整合点与插入模式全由它定,不再自仿分词器状态;不画的元素照 HTML 渲染规则(script、style、title、head、iframe、template 等)略去;树构建器每个开始标签都扫一遍开放元素栈,嵌套越深越慢(实测 release:十万层 div 28 秒),故按 4 KB 分块喂入,元素深过 512 层(同 Blink 的上限)即停,已读的文字照常入索引,警告写明哪封邮件;暂存原文写入或改名失败时删去暂存文件,删不掉即连同原因报错。`tests/store_repair.rs` 共 8 个;去掉清理、不画元素、MathML 整合点、深度上限即各自失败。第 3 轮审查后(最后一轮,修完不再审查):删除与替换邮件时,原文文件在同一事务提交之前删去,删不掉即整步回滚,邮件与原文都留着、可重试(原先先提交再删文件,失败后失去引用,文件永久残留);提交在删文件之后失败时,下次清理遇文件已不在只记警告;两处超长注释缩短。`tests/store_release.rs`(Windows,以只读共享占住文件)2 个,删去与替换各一,改回先提交即失败;引擎共 82 个测试
- [x] C12.4 凭据与 OAuth(MAIL-02,§18.2):密码与刷新令牌只进系统凭据库(不可用即只留到本次退出并写明);Gmail 与 Microsoft 的授权码加 PKCE、本机回环、换令牌与刷新;客户端 ID 编进构建,缺 ID 即写明不可用;刷新被拒转为“需重新登录”;替身授权端点测试。已做:`Vault`(keyring:钥匙串、Windows 凭据、Secret Service;拒绝时只在内存,`Kept::Session` 带原因);`oauth::SignIn` 在 127.0.0.1 随机端口等回跳(Google 写 127.0.0.1,Microsoft 写 localhost),state 与 S256 PKCE 各 32 随机字节,回跳的 state 不符或带 error 即报错,其余请求答 404;换令牌与刷新经 ureq(后台线程,30 秒),刷新答 invalid_grant 或 interaction_required 即 `SignInAgain`,其余为 `SignIn`;邮箱取 ID 令牌的 email,Microsoft 无则 preferred_username;客户端 ID 由 build.rs 读环境变量或项目根 `oauth.env`(坏行、未知键即构建失败),缺即 `Unavailable`;`tests/oauth.rs` 以替身令牌端点与替身浏览器核对,`tests/vault.rs` 以替身凭据库核对。审查三轮:第 1 轮四处(Google 授权带 `access_type=offline` 与 `prompt=consent`、换码无刷新令牌即报错;Microsoft 另要 `profile`;`oauth.env` 读不出即构建失败、不存在为未配置;超长注释)、第 2 轮一处(ureq 在 trace 级写出令牌与客户端密钥,`ely_platform::logging` 不记 `ureq`、`ureq_proto` 低于 info 的日志;`tests/private_oauth_logs.rs` 以真日志器在 trace 下核对,改回即失败)已修,第 3 轮通过
- [x] C12.5 账号(MAIL-01、02):ElyMail 主窗口与添加账号对话框:Gmail、Microsoft、iCloud(应用专用密码向导)、其他 IMAP/SMTP、POP3/SMTP;服务器、端口、TLS 方式、认证方式;连通性检查逐项报结果;多账号;删除账号先写明删去的本地缓存、草稿与凭据,不动服务端。已做:引擎 `Account`(地址、用户名、`Auth` 为密码或 Google、Microsoft 的 OAuth,`Protocol` 为 IMAP 或 POP3,收发两个 `Server`),以 JSON 存在 accounts 表的 settings 列(结构 v1 未发布,直接改,不加升级步);`valid` 照 SMTP 的 Mailbox 核对地址,用户名不空不带换行,主机不空不带空白,端口非零;`check` 同时登录收发两个服务器再退出,各自报结果;`Store::cached` 给该账号的邮件数与只属它的原文字节,`remove_account` 在一个事务里删账号(邮箱、邮件、标记随外键删去)与只属它的原文,原文在提交前删、删不掉即回滚,与别的账号共用的原文留着;凭据库名为 `account:地址`。应用:主窗口左栏列出账号(地址与协议、收信主机),中间为所选账号的地址、用户名、收发服务器、登录方式与“删除账号…”,无账号时为空状态与“添加账号…”;“账号”菜单两项。添加对话框分 Gmail、Microsoft、iCloud、IMAP、POP3:Gmail 与 Microsoft 填可选的地址作登录提示,“以 Google/Microsoft 登录”开浏览器,回来后以 ID 令牌的地址与访问令牌检查两台服务器,构建缺客户端 ID 即提示条写明并禁用按钮;iCloud 写明生成应用专用密码的步骤并可开 Apple 账户,服务器固定(imap.mail.me.com:993 TLS,smtp.mail.me.com:587 STARTTLS,两者用户名都用完整地址,Apple 写明可用);IMAP 与 POP3 填地址、登录方式(密码或两种 OAuth)、用户名、密码、收发主机、端口与加密,改加密时端口随之换成惯用的,手填的端口不动。“检查”逐项报收信与发信服务器(连不上、TLS 失败、拒绝登录及服务器原话等,按界面语言),通过才可“添加账号”,之后改任一字段即回到编辑;后到的检查与登录按编号丢弃,关对话框即中止网络任务;同一地址(不分大小写)不重复添加。保存先写库再存凭据,凭据库拒绝即提示只保留到退出。删除先在后台量出缓存,对话框写明删去的邮件数与大小、草稿与保存的密码或登录、服务器上的邮件不动;先删凭据,删不掉即不删账号并报错。库与网络都在后台(库在锁后,网络在两线程的 tokio 运行时),提示文字不进日志(可能含地址)。测试:引擎 `tests/accounts.rs` 6 个(往返、地址唯一、删去时留下共用原文与量缓存、两台服务器都登录、各自报结果、不合的设置),应用窗口测试 8 个(缺项逐个写明、端口随加密、检查逐项报连不上且改字段回到编辑、iCloud 的服务器、缺客户端 ID、凭据库拒绝仍保存并留在本次、保存与重开读回再删去且凭据随之删、凭据删不掉即留账号),去掉各自的规则即失败;真窗口冒烟(`ELY_SMOKE=1`)画出一帧。未验证:真实的 Gmail、Microsoft、iCloud 与其他服务器(须用户的客户端 ID 与账号)。第 1 轮审查后:删去或替换的原文先改名留在原处,提交后才删、失败即改回,打开库时收拾中断留下的(引擎 `tests/aside.rs`:第二个原文删不掉时两个都还读得到,重开后放回或删去);删除确认按钮带着所删的账号(原先关闭回调先清掉状态,按了什么也不删),测试改按真的按钮;端口以上次自动填入的值判断是否键入(原先把惯用端口当未键入,IMAP 收信填 995 会被冲掉);请求在整个窗口编号,保存的结果只作用于仍在等它的对话框(原先会关掉或改写之后新开的对话框),测试 `a_late_save…`、`a_late_failed_save…`;去掉各修复即失败。第 2 轮审查后:本地删除失败即把读出的凭据放回(凭据库拒绝时只放回本次并写明),账号仍可登录(`a_failed_removal_puts_the_secret_back`:原文的停放处被占,删除失败后凭据、账号与原文都在);地址唯一由库保证(`address` 不分 ASCII 大小写,`add_account` 遇冲突报 `Error::Taken`,对话框写明已添加),两个在途的保存不再都成功(`two_saves_in_flight_add_an_address_once`);左栏账号列表改用组件库 `SelectableList`(一个 Tab 停靠点,上下键选账号,`the_keyboard_picks_another_account`);去掉各修复即失败。第 3 轮审查后:凭据库删不掉时内存里的凭据照留(`Vault::remove` 先删凭据库,成功才清内存),凭据库不可用时添加的账号删除失败后仍能取回凭据(`a_Line truncated
- [ ] C12.6 IMAP 同步(MAIL-14,§12.3,QA-18):文件夹列表与特殊用途;邮件头全部、正文只取 90 天内;UIDVALIDITY 变化即重建该邮箱;服务端删除与标记变化随之;后台任务执行,界面显示最后同步时间、下载范围、未缓存的数量与失败原因
- [x] C12.6 IMAP 同步(MAIL-14,§12.3,QA-18):文件夹列表与特殊用途;邮件头全部、正文只取 90 天内;UIDVALIDITY 变化即重建该邮箱;服务端删除与标记变化随之;后台任务执行,界面显示最后同步时间、下载范围、未缓存的数量与失败原因。已做:引擎 `Imap::folders` 先问 CAPABILITY,有 LIST-EXTENDED 与 SPECIAL-USE 即以 `LIST "" * RETURN (SPECIAL-USE)` 取角色,否则照常 LIST(INBOX 按名,其余按 RFC 6154 的属性;`\Noselect`、`\NonExistent` 不可选);`examine` 读 UIDVALIDITY 与 EXISTS(缺任一即报错,不当作空邮箱);`flags`、`headers`、`bodies`、`structures`、`text` 按 UID 取,同一 UID 的多条 FETCH 回复合并(别的客户端改标记时插来的标记行照收),命令结束后才查所要的段(`\Recent` 属本次会话,不存);`since` 以 UID SEARCH SINCE 找 90 天内的邮件。`sync::imap` 一轮:列出的可选文件夹存为邮箱(带角色),不再列出的连同邮件删去,库里 INBOX 不分大小写(`Store::mailbox`、`set_folders`);每个邮箱 UIDVALIDITY 变了即清空重建;服务端已无的 UID 删去,标记不同即改;更早的邮件只取邮件头;90 天内的先读 BODYSTRUCTURE,无附件即取整封,有附件只取邮件头、各部分的 MIME 头与正文部分(文字,related 里的内嵌图片),按原结构重组,附件内容为空,部分号记在 `missing`(`Fetched::Text`、`Stored::missing`),附件按需(C12.17);已有邮件头的在进入 90 天时补正文;每次 FETCH 至多 100 个 UID;任何一步失败即停下报错,上次同步时间不变,成功才记下;`sync::secret` 取密码,OAuth 以刷新令牌换访问令牌(新的刷新令牌随即存下)。应用:窗口打开时与添加账号后各同步一次 IMAP 账号(POP3 随 C12.7),在应用的 tokio 运行时上;删除账号先中止其同步;账号详情写明上次同步时间(或正在同步、尚未同步)、下载范围、只有邮件头的邮件数与附件未下载的邮件数,失败以提示条写出原因。测试:引擎 `tests/sync.rs` 9 个(首次同步取邮件头与近期正文并记角色与时间;删除、标记与不再列出的文件夹随服务端;新 UIDVALIDITY 重建;失败不记时间;缺 EXISTS 即失败且邮件留着;取信中插来的标记行;INBOX 换大小写不删邮件;扩展 LIST 取角色;有附件的只存正文、附件为空且记部分号),应用 `tests/syncs.rs` 3 个(打开即同步并写明连不上、POP3 不同步;凭据库里没有即写明);去掉各规则即失败。未验证:真实服务器;令牌失效(`SignInAgain`)后重新登录的界面未做,现只写明原因。第 2 轮审查后:结构 v1 照已提交的原样,同步所需的列由 v2 升级加上(`schema::V2`:accounts.synced、mailboxes.role 与 uidvalidity、messages.missing),已存的 v1 库升级后邮件照留并能同步(`a_v1_store_gains_sync_and_keeps_its_mail`);处置为 attachment 的多部分整个是一个按需的实体,部分号记在 `missing`,不再逐个子部分判(`an_attached_multipart_waits_whole`);related 里不看处置:根照常读,带 Content-ID 的其余部分即正文资源,标为 attachment 的内嵌图片照样取下(`a_related_image_marked_attachment_comes_down`、`a_related_start_root_ignores_its_disposition`);本机 TLS 初始化失败即为该账号的同步失败,提示条写出原因(`a_missing_tls_is_the_sync_failure`);部分选择单测在 `imap/parts/tests.rs`(5 个,以真实的 BODYSTRUCTURE 解析后重组再经 MIME 读回);去掉各修复即失败。第 3 轮审查后:邮箱的 UIDVALIDITY 不同于所记(含 v2 升级后尚未记下)时只删去别的 UIDVALIDITY 下的邮件,同一 UIDVALIDITY 的照留,升级后的首次同步不再删去已存的正文(`a_v1_store_gains_sync_and_keeps_its_mail` 接着打开邮箱);related 的 `start` 在 BODYSTRUCTURE 里找不到(多部分不带 Content-ID)而有多部分子项时,各多部分子项照正文读,根的文字与 HTML 照样取下(`a_related_start_may_name_a_multipart`);去掉各修复即失败;三轮已满,未再审查
- [ ] C12.7 POP3 收取(MAIL-01,§12.3):按 UIDL 去重下载到本地收件箱,“保留服务端副本”默认开,不做伪文件夹同步
- [ ] C12.8 同步控制(MAIL-14):手动刷新、运行期间定时同步、暂停、按文件夹完整离线下载
- [ ] C12.9 邮箱视图(MAIL-03):账号与文件夹、邮件列表、阅读区三栏;统一收件箱与账号视图;未读、标记、带附件筛选;列表只画可见行,10 万封滚动不掉帧
+11
View File
@@ -0,0 +1,11 @@
FAIL
1. crates/ely-mail-engine/src/imap.rs:139 — Missing `EXISTS` silently becomes an empty mailbox. If `EXAMINE` returns UIDVALIDITY and tagged OK but omits the required message count, `sync::one` deletes every cached message and records a successful sync. The independent probe returned `Ok(Report { removed: 1, .. })`, zero cached messages and a new last-sync time. Keep the count optional until validated and return a protocol error before changing local data when it is absent. Add a regression asserting both failure and cache preservation. Evidence: `.work/C12.6-r1-probes.log`, `an_examine_without_exists_does_not_delete_mail`; [RFC 3501 §6.3.1–6.3.2](https://www.rfc-editor.org/rfc/rfc3501.html#section-6.3.1).
2. crates/ely-mail-engine/src/imap.rs:285 — A concurrent flag notification makes a valid header/body fetch fail. Another client can change an asked UID's flags while it is being downloaded; a valid body response followed by `* 1 FETCH (UID 1 FLAGS (\Seen))` is rejected as `UID 1 came without a body`, aborting synchronization and discarding the fetched batch. Unsolicited FETCH responses can include UID. Accumulate attributes by UID, process flag updates separately, and validate the requested section after the command completes rather than requiring it in every response. Evidence: `.work/C12.6-r1-probes.log`, `concurrent_flags_do_not_make_a_body_fetch_fail`; [RFC 3501 §7](https://www.rfc-editor.org/rfc/rfc3501.html#section-7).
3. crates/ely-mail-engine/src/store/folders.rs:43 — Folder reconciliation treats INBOX's case variants as different identities. After caching `INBOX`, a subsequent LIST spelling it `inbox` deletes its messages and originals even with unchanged UIDVALIDITY and UIDs. Messages older than 90 days then return only as headers, losing their offline bodies. The probe's previously cached MessageId became absent after this case change. Normalize INBOX consistently for reconciliation and mailbox lookup while preserving other mailbox names' case distinctions, and test retention of the original bytes. Evidence: `.work/C12.6-r1-probes.log`, `inbox_casing_does_not_destroy_cached_mail`; [RFC 3501 §5.1](https://www.rfc-editor.org/rfc/rfc3501.html#section-5.1).
4. crates/ely-mail-engine/src/imap.rs:123 — Special-use discovery relies entirely on optional attributes in ordinary LIST responses. A server may advertise `LIST-EXTENDED SPECIAL-USE` and return its Sent/Drafts/Trash roles only when requested with `RETURN (SPECIAL-USE)`. This implementation never discovers capabilities or requests those attributes, so such folders are stored with `role = None`; the probe returned no Sent role despite the advertised capability. Discover the supported capabilities and request special-use attributes when available, retaining ordinary LIST for servers without the extension. Add coverage for a server that omits roles from ordinary LIST. Evidence: `.work/C12.6-r1-probes.log`, `roles_are_discovered_when_only_extended_list_carries_them`; [RFC 6154 §2](https://www.rfc-editor.org/rfc/rfc6154.html#section-2).
5. crates/ely-mail-engine/src/imap.rs:186 — Default body synchronization downloads attachments without a request. `BODY.PEEK[]` retrieves the entire MIME message, and `sync.rs:148` persists it as `Fetched::Whole`. With the existing multipart sample classified as recent, the independent sync probe cached `report.pdf` and all nine attachment bytes automatically. Larger attachments are likewise transferred and stored, contrary to PRD §12.3's attachment-on-demand default. Fetch and cache the message's body sections separately from attachment payloads; retrieve attachment bytes only on request or explicit complete offline retention, and test that default synchronization leaves those payloads uncached. Evidence: `.work/C12.6-r1-probes.log`, `a_default_recent_body_sync_does_not_fetch_attachments`.
+9
View File
@@ -0,0 +1,9 @@
FAIL
1. crates/ely-mail-engine/src/store/schema.rs:2 — The schema changes overwrite V1 without adding a migration. A database created by the completed C12.3/C12.5 implementation already has `user_version = 1`, so opening it skips upgrading and leaves `synced`, `role`, `uidvalidity` and `missing` absent. The independent probe opened the exact HEAD schema successfully, then `Store::synced` failed with `no such column: synced`; folder synchronization and message reads likewise reference absent columns. Preserve the existing V1 and append a versioned migration, with a regression that reopens a populated previous-version database and verifies its data and synchronization. Evidence: `.work/C12.6-r2-probes.log`, `probe_an_existing_v1_store_is_upgraded`.
2. crates/ely-mail-engine/src/imap/parts.rs:110 — Multipart attachment dispositions are ignored before descending into their children, leaving round 1 defect 5 incompletely fixed. For a recent multipart/mixed message containing an attached multipart/alternative whose child is text/plain, every leaf is classified as body content, `Node::complete()` returns true, and sync downloads and stores the whole attachment through `BODY.PEEK[]`. The independent probe cached `nested.mime`, including its private attachment payload, without a request. Classify an attached multipart as one omitted entity before descending, retain its part number in `missing`, and cover multipart attachments in the synchronization regression. This matches the existing MIME reader and [RFC 2183 §2.9](https://www.rfc-editor.org/rfc/rfc2183.html#section-2.9). Evidence: `.work/C12.6-r2-probes.log`, `probe_multipart_attachments_stay_on_demand`.
3. crates/ely-mail-engine/src/imap/parts.rs:142 — The attachment check overrides multipart/related semantics and omits inline body content. A recent HTML message with a CID PNG marked `Content-Disposition: attachment` is valid and is classified as an inline image by the existing MIME reader, but synchronization never requests its payload. The probe completed synchronization with empty image bytes instead of the PNG bytes; the message is then marked body-cached and subsequent default syncs skip it. Apply related-container semantics before disposition checks when selecting body parts, and test related roots and CID images carrying attachment dispositions. [RFC 2387 §4](https://www.rfc-editor.org/rfc/rfc2387.html#section-4) requires ignoring the disposition type within multipart/related. Evidence: `.work/C12.6-r2-probes.log`, `probe_related_images_marked_attachment_are_cached`.
4. apps/ely-mail/src/syncing.rs:35 — A TLS initialization failure is discarded instead of appearing as a synchronization failure. If the local store opens but platform TLS initialization returns an error, startup synchronization returns after a generic log message, creates no `Syncing` entry, and the account view displays neither synchronization status nor the actual failure reason required by §12.3. The GPUI probe injected `platform trust store unavailable` and confirmed that the account had no failure state. Preserve the original TLS error in the account's synchronization state and render it through the existing failure banner; add startup coverage with failed TLS initialization. Evidence: `.work/C12.6-r2-app-probes.log`, `probe_tls_initialization_failure_is_shown_as_a_sync_failure`.
+5
View File
@@ -0,0 +1,5 @@
FAIL
1. crates/ely-mail-engine/src/store/folders.rs:106 — The first sync deletes an upgraded cache even when UIDVALIDITY has not changed. V2 leaves `mailboxes.uidvalidity` NULL, so `open_mailbox` calls `without_mail` and deletes every existing message and original before comparing their stored UIDVALIDITY with the server's. The independent probe upgraded a populated v1 database, confirmed its cached body survived opening, then called the same `open_mailbox(INBOX, 7)` used by sync: the message with UIDVALIDITY 7 disappeared and the message count became zero. For mail older than 90 days, synchronization restores only headers, losing its offline body. Reconcile existing message identities with the reported UIDVALIDITY when initializing the new column; preserve matching cached messages. Extend the migration regression through the first synchronization, including an older cached body. Evidence: `.work/C12.6-r3/probes.log`, `probe_a_migrated_body_survives_its_first_examine`.
2. crates/ely-mail-engine/src/imap/parts.rs:159 — Related-root selection cannot resolve a multipart child named by `start`. It uses `id(child)`, which always returns None for multipart nodes, then falls back to the first child and marks the actual root as an omitted attachment. A valid multipart/related message with a CID image first and a multipart/alternative root second (`Content-ID: <root>`, `start="<root>"`) loses both its plain and HTML bodies. The independent reconstruction probe read `hello` from the full message, but the selected sections omitted part 2 and the cached message had neither text nor HTML. Sync nevertheless stores it as body-cached, so later default syncs skip repairing it. Resolve related roots using their MIME headers when BODYSTRUCTURE cannot supply Content-ID, and cover a non-first multipart root through reconstruction and synchronization. Evidence: `.work/C12.6-r3/probes.log`, `probe_related_multipart_start_root_is_cached`.