Keep every file inside the rules the repo already wrote down
The 500-line rule and the one-line-comment rule were enforced only from
memory, so they drifted. bench/filesize_check.py now checks the budget on
every tracked source file, and it passes.
Split, with behaviour proven unchanged by comparing each moved function body
against HEAD byte for byte (53 training bodies, 21 benchmark bodies, 0
differences):
db.py 503 -> 396 store_search.py 105
bench/run.py 609 -> 351 query_set.py 80, report.py 211
benchmarks.py 619 -> 424 benchmark_scoring.py 262
training.py 1630 -> 94 training_{common,encode,metrics,datasets,lora,retrieval}
TrainingService stays as a thin facade for the CLI; the facade's parameter
lists are pinned against the module functions by inspect, which is what
catches a leaked self.
Deleted the eight top-level functions no file called, after fixing the scan
that had reported 201 false positives by skipping the defining file. The
v1/v2/v3 sweep found nothing to delete and says so: every remaining version
tag is an on-disk format marker, a migration step, a fixture name, or a
third-party model name.
Collapsed 39 multi-line comment blocks (86 lines) to one line each, keeping
the measured facts and error strings that justify the code.
269 tests pass; ruff clean; mypy 24 errors in 9 files against a 24-in-6
baseline, net zero; pyright 4, the same four HEAD measures.
This commit is contained in:
1 parent
56b94cd58e
commit
9cb46c18d3
47 files changed
+2678
-2272
No files matched your search
@@ -12,15 +12,13 @@ import re
|
||||
import sqlite3
|
||||
from unicodedata import normalize
|
||||
|
||||
# Han, its extension A block, the compatibility ideographs, kana, and Hangul syllables:
|
||||
# scripts written without spaces. One source of truth for both the tokenizer and the query.
|
||||
# Han, its extension A block, compatibility ideographs, kana and Hangul syllables: scripts written without spaces, shared by tokenizer and query.
|
||||
_BLOCKS = ((0x3400, 0x4DBF), (0x4E00, 0x9FFF), (0xF900, 0xFAFF), (0x3040, 0x30FF), (0xAC00, 0xD7AF))
|
||||
_CJK_CLASS = "".join(f"{chr(start)}-{chr(end)}" for start, end in _BLOCKS)
|
||||
_CJK_CHARS = frozenset("".join("".join(chr(code) for code in range(start, end + 1)) for start, end in _BLOCKS))
|
||||
|
||||
_CAMEL_BOUNDARY = re.compile(r"(?<=[a-z0-9])(?=[A-Z])|(?<=[A-Z])(?=[A-Z][a-z])|(?<=[A-Za-z])(?=\d)")
|
||||
# Everything a reader would not type as a word: symbols, punctuation, and the underscore
|
||||
# that joins identifiers. `[^[^\W\s]` looks equivalent but deletes CJK characters on 3.14.
|
||||
# What a reader would not type as a word; `[^[^\W\s]` looks equivalent but deletes CJK characters on 3.14.
|
||||
_SEPARATORS = re.compile(r"[^\w\s]|_")
|
||||
_CJK_RUN = re.compile(f"[{_CJK_CLASS}]+")
|
||||
_SPACES = re.compile(r"\s+")
|
||||
|
||||
Reference in new issue
Block a user