vesper (n.): the evening star.
When Gemma 4 came out, it felt like LLMs could finally run on your own computer and still do a decent job. Around the same time Apple kept pushing back the smarter Siri it had promised with Apple Intelligence, again and again, so I figured I’d try building a Siri replacement on a local model. Vesper is built on that premise: it reads your mail, notes and calendar, so it should work offline. These are things we write for ourselves. If we wouldn’t show our notes to another person, we shouldn’t send them to a cloud model either (and yet plenty of people talk to ChatGPT about their love lives?).
Here are the problems I ran into while building it.
One model can’t be both fast and right
I tested two local models with English and Chinese prompts:
| Model | Size | Routing accuracy | Avg latency |
|---|---|---|---|
qwen3:8b |
5.2 GB | 95% (37 / 39) | 9,990 ms |
llama3.2:3b |
2.0 GB | 79% (31 / 39) | 613 ms |
Ten seconds is too slow for a menubar tool, but the 3B model’s mistakes were the dangerous kind: wrong tool, lost reminder text, garbled Chinese titles. They weren’t random though. Five of the seven failures I traced came from Chinese prompts or long note bodies, so instead of picking one model I split along that line:
- Deterministic fast paths: regex rules for the most common commands (“remind me to … tomorrow at 9”, “show me my recent notes”, “take a note: …”), in English and Chinese, with no LLM call at all. One of the 3B model’s short-English misses is now handled at this layer too.
- Small model: short English prompts (80 characters or fewer) go to the 3B model.
- Reliable model: anything with Chinese, or longer than that, goes to the 8B model, which is exactly where the 3B model broke.
Every rule is checked against a 48-case English/Chinese eval set that measures fast-path hit rate, LLM calls per prompt and latency by language. Each turn also writes a redacted trace (tool, confidence, latency, never the content of a note or email), so a misroute can be debugged without putting private data in a log.
AppleScript is the real bottleneck
With the model sorted, the slowest part was talking to the Mac’s own apps. Adding one reminder took 27 seconds. I timed each property read after the write to find out where it went:
Read after make new reminder |
Time |
|---|---|
id |
4 s |
name |
2 s |
remind me date |
17 s |
creation date |
16 s |
Reminders.app blocks while it syncs a new item to iCloud, and Vesper was reading back values it had just written. Now it reads only the id and rebuilds the rest from its own input: 27 s → 5.6 s from the CLI.
That finding shaped the other tools too:
Mail is cache-first: queries hit a local SQLite FTS5 index, and a background sync runs only when the question is about recent mail and the cache is older than that window. Calendar gives each calendar its own timeout, and a slow one is skipped and mentioned in the reply. Reminders lists are read with one batched script per list, which holds up with 100+ items.
In the end Vesper never became something I use every day. I’m not much of a planner: I barely use Notes, I don’t look at my calendar, and my inbox is ads and USC newsletters. A month later, after WWDC, the Siri beta got a bit better, and Vesper turned out to be like the evening star it’s named after: the first to show up at dusk, and nobody cared, lol.
vesper（名詞）：昏星，黃昏時最先亮起的那顆星。
當時我看到Gemma 4的推出，掀起一種LLM可以跑在自己電腦上而且還可以有不錯表現的感覺，加上當時apple intelligence說好的更聰明的siri一次又一次的延期，我就想可以試試用本地模型做個替代Siri的工具。Vesper 就是以此為前提設計的：它會讀你的郵件、備忘錄和行事曆，所以應該要能離線運作，畢竟這些都是我們給自己看的，如果我們不會把備忘錄給別人看，也不應該發送給雲端模型（但居然有許多人會跟chatgpt聊感情事？）
以下是我在開發時遇到的問題
一個模型無法同時又快又準
我用中英文 prompt測了兩個本地模型：
| 模型 | 大小 | 路由準確率 | 平均延遲 |
|---|---|---|---|
qwen3:8b |
5.2 GB | 95%（37 / 39） | 9,990 ms |
llama3.2:3b |
2.0 GB | 79%（31 / 39） | 613 ms |
十秒對選單列工具來說太慢，但 3B 模型出錯的方式很危險：選錯工具、把提醒事項的內容弄丟、把中文標題變成亂碼。不過這些錯誤並非隨機：我追查的七個失敗案例中，有五個來自中文 prompt 或較長的筆記內容。所以我沒有二選一，而是沿著這條界線分流：
- 確定性的 fast path：用 regex 規則處理最常見的指令（「remind me to … tomorrow at 9」、「show me my recent notes」、「take a note: …」），中英文都支援，完全不呼叫 LLM。3B 模型答錯的其中一題短英文指令，現在也在這一層就處理掉了。
- 小模型：80 個字元以內的英文短指令交給 3B 模型。
- 可靠的模型：含中文或超過這個長度的輸入交給 8B 模型，剛好就是 3B 模型會出錯的那些情況。
每條規則都用一份 48 題的中英文評測集驗證，量 fast path 命中率、每題的 LLM 呼叫次數，以及分語言的延遲。每一輪對話也會寫下一筆去識別化的 trace（工具、信心分數、延遲，絕不包含筆記或郵件內容），出現路由錯誤時可以除錯，又不會把私人資料寫進 log。
真正的瓶頸是 AppleScript
模型的問題解決之後，最慢的變成和 Mac 內建 app 溝通的部分。新增一則提醒事項要 27 秒。我逐一量測寫入後讀取每個屬性的時間，才找到時間花在哪裡：
make new reminder 之後讀取 |
時間 |
|---|---|
id |
4 秒 |
name |
2 秒 |
remind me date |
17 秒 |
creation date |
16 秒 |
本來提醒事項 app 在把新項目同步到 iCloud 時會卡住，而 Vesper 把自己剛寫進去的值再讀回來。現在它只讀 id，其餘欄位用自己的輸入重建：CLI 上從 27 秒降到 5.6 秒。
從這個發現也決定了其他工具的做法：
郵件採快取優先：查詢直接打本地的 SQLite FTS5 索引，只有在問的是近期郵件、而快取比那個時間範圍還舊時，才在背景同步；行事曆每個日曆各自設 timeout，慢的日曆會被略過，並在回覆中提及；提醒事項清單每個清單用一支批次腳本讀取，項目超過 100 筆也撐得住。
這個project最後沒有變成一個我每天在用的工具，原因是我實在很P，不太用備忘錄也沒有在看行事曆，平常的郵件也都是一些廣告或是USC寄來的news letter。隔了一個月Apple WWDC後siri AI在beta版本變得好用一些了，vesper 也就像黃昏最先亮的那顆星，沒人在乎了哈哈
