Salastil/nobilis
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 9 Wiki Activity

Compare commits

base: Salastil/nobilis:protocol/twitch
Salastil/nobilis:master Salastil/nobilis:development Salastil/nobilis:protocol/telegram Salastil/nobilis:protocol/twitch
Salastil/nobilis:v1.3.3 Salastil/nobilis:v1.3.2 Salastil/nobilis:v1.3.1 Salastil/nobilis:v1.2.1 Salastil/nobilis:v1.2.0 Salastil/nobilis:v1.1.0 Salastil/nobilis:v1.0.0 Salastil/nobilis:v1.0.0-rc.5 Salastil/nobilis:v1.0.0-rc.4 Salastil/nobilis:v1.0.0-rc.3 Salastil/nobilis:v1.0.0-rc.2 Salastil/nobilis:v1.0.0-rc.1
...
compare: Salastil/nobilis:v1.3.3
Salastil/nobilis:development Salastil/nobilis:master Salastil/nobilis:protocol/telegram Salastil/nobilis:protocol/twitch
Salastil/nobilis:v1.3.3 Salastil/nobilis:v1.3.2 Salastil/nobilis:v1.3.1 Salastil/nobilis:v1.2.1 Salastil/nobilis:v1.2.0 Salastil/nobilis:v1.1.0 Salastil/nobilis:v1.0.0 Salastil/nobilis:v1.0.0-rc.5 Salastil/nobilis:v1.0.0-rc.4 Salastil/nobilis:v1.0.0-rc.3 Salastil/nobilis:v1.0.0-rc.2 Salastil/nobilis:v1.0.0-rc.1
62 Commits
protocol/twitch ... v1.3.3
Author SHA1 Message Date
SalastilandClaude Sonnet 5.5 9382bc04ea Merge development into master: 1.3.3 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-23 16:26:18 -04:00
SalastilandClaude Sonnet 5.5 5983d71baf 1.3.3 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-23 16:26:14 -04:00
SalastilandClaude Sonnet 5.5 b1862bb1f2 Discord (#338): the green new-member badge closes a server's box for the member's first week 
A member who joined a server less than seven days ago wears Discord's green badge, and (presumed, from a
flagging in a server where the account had spoken from the official apps) what is said from another client in
that time is treated as spam. Every channel of such a server is read-only with a banner saying so and when it
ends; it opens by itself at the week's end. The owner is exempt; what the server's own rules ask comes first.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-23 12:27:22 -04:00
SalastilandClaude Sonnet 5.5 64a13fa9fc Telegram links: t.me and tg:// links open the chat (joined if need be) and the post (#26) 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-23 11:56:23 -04:00
SalastilandClaude Sonnet 5.5 c9c98dec3a Telegram folders: the refusal for an empty folder is worded as the library words it (seen on the real account) 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-23 11:27:06 -04:00
SalastilandClaude Sonnet 5.5 430e89fdd3 Telegram: stickers, bot commands in groups, topic unread counts, QR sign-in, chat folders (#26) 
- stickers: the account's recent, favourite and installed packs are listed (a few packs at a time, kept
  15 minutes) and one is sent as its document, in a topic too; pictures are the preview Telegram ships
  with each, fetched when drawn
- bot commands in groups: the commands of every bot in a group, read with the group (basic chat or
  supergroup), shared by its topics; a command two bots have is offered once per bot as cmd@bot
- topic unread counts: a topic's own unread and mention counts seed the window's count (Buffer
  unreadSeed/mentionsSeed, set_buffer_unread); reading a topic reads up to the newest message Telegram
  says it holds, and a topic read on another device clears here
- QR sign-in: exportLoginToken shown as a tg://login QR, renewed as Telegram says (updateLoginToken, with
  a 15 s ask-again as a fallback), DC migration through importLoginToken, then the cloud password if
  asked; addTelegramAccountQr and a telegramLoginQr event
- folders: the account's dialog filters are read after the conversations (what each holds by its rules is
  worked out from what is known of every chat), told to the window as telegramFolders, kept current from
  updateDialogFilter*, and edited from the window: create (with its first chat: Telegram keeps no empty
  folder), rename, delete, reorder, and file a chat into/out of the folder it is drawn under, taking it out of
  a rule-held folder as an exception

All unit-tested; none of it run against Telegram yet.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-23 11:19:39 -04:00
SalastilandClaude Sonnet 5.5 35573351a3 Discord (#338 H1-H12): no requests after a 401 or three 403s on a route, a captcha closes its place, heartbeats carry the sequence, the server's own rules checked before a send (verification, onboarding, timeout, slowmode), DMs need five of the account's own messages, empty channels locked, history asked once, the official wire shape and two-step uploads, one send at a time a channel 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-22 22:44:11 -04:00
SalastilandClaude Sonnet 5.5 26389385bc Discord (#338): READY with deduplicated users put back and declared, the gateway stream compressed, no conversation-list request when READY names them, acks while the newest line is read at the bottom 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-22 21:55:57 -04:00
SalastilandClaude Sonnet 5.5 cb01043f2c Discord first-word lock: a system message about the account (its own join) is not the account speaking 
The join message of a server is authored by the account and counted as its first word, so a server was unlocked the moment it was joined. Only an ordinary message or a reply counts, live and in search.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-22 20:41:05 -04:00
SalastilandClaude Sonnet 5.5 ec34657d92 Merge development into master: 1.3.2 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-21 23:36:05 -04:00
SalastilandClaude Sonnet 5.5 e9ce99fb42 1.3.2 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-21 23:36:03 -04:00
SalastilandClaude Sonnet 5.5 7943767c47 Discord: safeguards after Dissent's (#338): paced reads that obey rate limits, capabilities in IDENTIFY, no history fetched until a conversation is opened, one member-list subscription a connection, own membership from READY, acks only of what is read, text checked before it is sent 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-21 23:23:16 -04:00
SalastilandClaude Sonnet 5.5 99253cbd5b Merge development into master: 1.3.1 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-21 21:03:46 -04:00
SalastilandClaude Sonnet 5.5 04e4e85f97 1.3.1 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-21 21:03:44 -04:00
SalastilandClaude Sonnet 5.5 b9a79e4b9c Discord: no first word from this client (a server the account has not spoken in, a conversation with nothing in it, are locked), a nonce and allowed_mentions on every message, files checked against the upload limit, a pause between a message and a file 
The account's own messages since it joined are looked for in what is stored and then in Discord's search (one request a server, kept); the messages this client sent do not count. The lock is refused in the daemon and shown by the window as the conversation's read-only reason.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-21 20:39:40 -04:00
SalastilandClaude Sonnet 5.5 264ee7a1f5 Merge development into master: 1.3.0, Telegram 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-21 19:03:03 -04:00
SalastilandClaude Sonnet 5.5 6345ac4df6 1.3.0 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-21 19:02:54 -04:00
SalastilandClaude Sonnet 5.5 3bf68e747a Discord: send the install id, locale and timezone with API requests 
X-Installation-ID (random, made once, kept in the data dir), X-Discord-Locale,
X-Discord-Timezone and Accept-Language on the API client; system_locale in
IDENTIFY. The user agent stays moho/<version> (nobilis); no super properties.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-21 18:49:18 -04:00
SalastilandClaude Sonnet 5.5 e8b25d99bf Discord: resume across restarts, ask only about DMs that moved on, throttle typing, skip repeat acks 
A restart within 150 s of the last heartbeat ack resumes the saved session
(READY and GUILD_CREATE kept in discord-sessions/, replayed through the live
handlers) instead of logging in afresh; a refused resume or a 4003/4007/4009
close discards it. DM catch-up compares last_message_id with what is stored
and skips the rest; typing posts are at most one per 8 s per channel; an ack
is not repeated for the same newest message.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-21 18:19:19 -04:00
SalastilandClaude Sonnet 5.5 58bf5ac83a Discord: Connect anyway lets a held login through (still counted) 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-20 23:52:42 -04:00
SalastilandClaude Sonnet 5.5 8a7d73346e Discord: a limit on fresh logins per account (3 an hour, 8 a day, kept across restarts); the connection is held and the window says why 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-20 23:41:56 -04:00
SalastilandClaude Sonnet 5.5 54f3e3f697 Telegram: the GIF picker - the account's saved GIFs and Telegram's GIF search, sent as the files they are 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-20 18:41:45 -04:00
SalastilandClaude Sonnet 5.5 931cd458af Merge development into master: Telegram 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-19 22:47:35 -04:00
SalastilandClaude Sonnet 5.5 ad597ea2d2 Telegram: tests use made-up ids 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-19 22:47:29 -04:00
SalastilandClaude Sonnet 5.5 e4c4fd756f Telegram: a channel's post carries the channel's picture, and a picture or video is not also named in words 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-18 18:20:32 -04:00
SalastilandClaude Sonnet 5.5 7f7f25761c Telegram: each forum topic is a conversation of its own under its group's rail entry 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-18 18:13:29 -04:00
SalastilandClaude Sonnet 5.5 50ca7b4255 Telegram: each group is its own entry on the rail, with its name and picture 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-18 18:00:56 -04:00
SalastilandClaude Sonnet 5.5 55016757d3 Discord: the platform's own account carries the OFFICIAL mark 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-18 17:30:53 -04:00
SalastilandClaude Sonnet 5.5 5d17b2397a Telegram accounts are Telegram, not IRC: the service with no prefix is IRC and a prefix not on the list fell into it 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-18 14:15:20 -04:00
SalastilandClaude Sonnet 5.5 eae7add558 Telegram: a bot chat offers only the bot's own commands 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-18 14:06:14 -04:00
SalastilandClaude Sonnet 5.5 8e5e641446 Telegram: a bot's command menu - the Menu button and the commands offered while a slash is typed 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-18 13:58:48 -04:00
SalastilandClaude Sonnet 5.5 a79749f8d7 Telegram: say what markup a message carried when it gave no buttons 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-18 13:53:35 -04:00
SalastilandClaude Sonnet 5.5 13b0a5c6af Telegram: a bot's inline keyboard - callback buttons answered with the bot's notice, links, edits that change or remove them 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-18 13:40:17 -04:00
SalastilandClaude Sonnet 5.5 dba773160f Discord: bots and applications carry Discord's APP mark (verified ones with a check), in history and live 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-18 00:01:47 -04:00
SalastilandClaude Sonnet 5.5 41c2780aaa Discord: messages made with layout components (containers, sections, text, pictures) are drawn as cards with their buttons 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-17 23:53:46 -04:00
SalastilandClaude Sonnet 5.5 63093e1f1a Discord: the server's rules dialog shows the rules and any questions, and sends the answers 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-16 20:28:43 -04:00
SalastilandClaude Sonnet 5.5 65635276f3 Telegram: a forwarded message says where it came from, and a post in an open channel is a click away 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-16 20:07:35 -04:00
SalastilandClaude Sonnet 5.5 5094a9a2ec Telegram: an answer to a post in another channel is drawn as Telegram draws it and goes to the original 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-16 19:45:55 -04:00
SalastilandClaude Sonnet 5.5 7a4b0130a0 Telegram: a horn before broadcast channels and a robot before bots in the list 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-16 19:36:54 -04:00
SalastilandClaude Sonnet 5.5 ee0e0f921b Telegram: bots have a heading of their own 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-16 19:32:07 -04:00
SalastilandClaude Sonnet 5.5 a6f35525d0 Telegram: an article's cover is found among the page's own picture, and a missing picture keeps its caption 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-16 15:27:28 -04:00
SalastilandClaude Sonnet 5.5 b1d1a53c2f Telegram: Instant View articles drawn as a document of their own; link cards say so 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-16 15:22:23 -04:00
SalastilandClaude Sonnet 5.5 15d9058604 Telegram: link previews (what Instant View pages carry) as cards with their picture 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-16 15:09:43 -04:00
SalastilandClaude Sonnet 5.5 b4ddb310b6 Telegram: a file not fetched yet keeps its place in a message (it was being dropped as swept) 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-16 10:03:45 -04:00
SalastilandClaude Sonnet 5.5 d368413241 Telegram: each download has its own half-written file; a restore waits for a preview already on its way 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-16 09:55:50 -04:00
SalastilandClaude Sonnet 5.5 5037705cc4 Telegram: previews are fetched as messages are recorded 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-16 09:47:29 -04:00
SalastilandClaude Sonnet 5.5 f725dcc2da Telegram: pictures no longer wait behind whole-file downloads; failures to fetch are logged 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-16 09:38:01 -04:00
SalastilandClaude Sonnet 5.5 4643cb082e Telegram: a message kept without its media gets it when read again 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-16 09:30:15 -04:00
SalastilandClaude Sonnet 5.5 11ce3e974b Telegram: messages from a person in a private chat are recorded (short updates name no one) 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-16 09:23:12 -04:00
SalastilandClaude Sonnet 5.5 0f5003c061 Telegram: start polls, forum messages are not replies to their topic 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-16 00:23:49 -04:00
SalastilandClaude Sonnet 5.5 ca47ffeae7 Telegram: join and leave, admin actions on members, mute synced with the account 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-16 00:17:18 -04:00
SalastilandClaude Sonnet 5.5 6a8c35fe14 Telegram: polls, thread replies and channel comments 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-16 00:08:09 -04:00
SalastilandClaude Sonnet 5.5 584e3b8744 Telegram: reactions, pins, members, profile cards, server search, jumping to old messages 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-15 23:59:03 -04:00
Salastil 84258f4381 Telegram: pictures, video, files, voice and stickers - fetched when looked at - and sending files, albums and voice messages 2026-10-15 23:53:15 -04:00
SalastilandClaude Sonnet 5.5 3fc8cd1a1d Telegram: send, reply, edit, delete, typing both ways, read positions, formatting as entities 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-15 23:46:34 -04:00
SalastilandClaude Sonnet 5.5 22f838417f Telegram: a conversation the account cannot write in shows why in place of the message box 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-15 23:36:21 -04:00
SalastilandClaude Sonnet 5.5 d027d2e39f Telegram: conversations as buffers, history on opening and scrolling back, live messages, edits and deletions 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-15 23:27:29 -04:00
SalastilandClaude Sonnet 5.5 537ee520cd Telegram: accounts, sessions in files of their own, sign-in by phone code and cloud password, a connection that is kept and reconnected 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-15 23:17:27 -04:00
SalastilandClaude Sonnet 5.5 5ab8eafceb Merge development into master: 1.2.1, Steam identity 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-15 17:14:18 -04:00
SalastilandClaude Sonnet 5.5 e4a16278ab 1.2.1 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-15 17:14:12 -04:00
SalastilandClaude Sonnet 5.5 f3de870d49 Steam: sign in as a desktop client with a stable machine id and a logon id of this install's own, and wait longer before reconnecting 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-15 01:01:33 -04:00
SalastilandClaude Sonnet 5.5 2355fcb6ac Merge development into master: 1.2.0, Twitch 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-14 21:54:56 -04:00
71 changed files with 13244 additions and 299 deletions
Expand all files Collapse all files

No files matched your search

Cargo.lock
Generated
+662 -20
View File
@@ -841,6 +841,16 @@ dependencies = [
"futures",
]
[[package]]
name = "core-foundation"
version = "0.9.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "91e195e091a93c46f7102ec7818a2aa394e1e1771c3ab4825963fa03e45afb8f"
dependencies = [
"core-foundation-sys",
"libc",
]
[[package]]
name = "core-foundation"
version = "0.10.1"
@@ -944,13 +954,28 @@ dependencies = [
[[package]]
name = "crc32fast"
version = "1.5.0"
version = "1.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9481c1c90cbf2ac953f07c8d4a58aa3945c425b7185c9154d67a65e4230da511"
checksum = "01a7799fd6b852db0e61728dde9a204c423b44d689dbd432522543614b490e78"
dependencies = [
"cfg-if",
]
[[package]]
name = "critical-section"
version = "1.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "790eea4361631c5e7d22598ecd5723ff611904e3344ce8720784c93e3d83d40b"
[[package]]
name = "crossbeam-channel"
version = "0.5.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "98b0cc327b5bc766e7fda9c9260cc0fa81b43a8e240440422dff70788e3f9ef1"
dependencies = [
"crossbeam-utils",
]
[[package]]
name = "crossbeam-deque"
version = "0.8.7"
@@ -997,6 +1022,19 @@ dependencies = [
"zeroize",
]
[[package]]
name = "crypto-bigint"
version = "0.7.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1a52aa3fcda4e6302a9f48734f234d35d4721b96f8fe07d073f07ce9df4f0271"
dependencies = [
"cpubits",
"ctutils",
"num-traits",
"rand_core 0.10.1",
"serdect",
]
[[package]]
name = "crypto-common"
version = "0.1.7"
@@ -1019,6 +1057,16 @@ dependencies = [
"rand_core 0.10.1",
]
[[package]]
name = "crypto-primes"
version = "0.7.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3633a51a39c69ebbaa4feaa694bd83d241e4093901c84a0963b19d9bb3f0cf8f"
dependencies = [
"crypto-bigint 0.7.5",
"rand_core 0.10.1",
]
[[package]]
name = "ctr"
version = "0.9.2"
@@ -1274,6 +1322,37 @@ version = "3.3.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a9244323129647178bf41ac861a2cdb9d9c81b9b09d3d0d1de9cd302b33b8a1d"
[[package]]
name = "defmt"
version = "1.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e4a0795ac740cb54d7b3775c9c17345f6aaecf884c01733b39d6017738e0e3e1"
dependencies = [
"bitflags 1.3.2",
"defmt-macros",
]
[[package]]
name = "defmt-macros"
version = "1.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "840bb056bd99c495c9188198a258ab787c79f306d864db3e6e8451442d6d8366"
dependencies = [
"defmt-parser",
"proc-macro2",
"quote",
"syn 3.0.3",
]
[[package]]
name = "defmt-parser"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "71958f69fb2de856c5dd51ac9377e986ca1aa5919f931bf5395a807277f889a0"
dependencies = [
"thiserror 2.0.21",
]
[[package]]
name = "der"
version = "0.7.10"
@@ -1565,7 +1644,7 @@ dependencies = [
"ed25519 3.0.0",
"rand_core 0.10.1",
"serde",
"sha2 0.11.0",
"sha2 0.11.1",
"signature 3.0.0",
"subtle",
"zeroize",
@@ -1596,7 +1675,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b5e6043086bf7973472e0c7dff2142ea0b680d30e18d9cc40f267efbf222bd47"
dependencies = [
"base16ct 0.2.0",
"crypto-bigint",
"crypto-bigint 0.5.5",
"digest 0.10.7",
"ff",
"generic-array",
@@ -1630,7 +1709,7 @@ version = "3.1.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1bf1fa3f06bbff1ea5b1a9c7b14aa992a39657db60a2759457328d7e058f49ee"
dependencies = [
"num-bigint",
"num-bigint 0.4.8",
"num-traits",
"proc-macro2",
"quote",
@@ -1813,12 +1892,13 @@ checksum = "26b73573e6edcd2af0cdf47bd6cb58f0b3839491263c314eaad1ccf24430e1de"
[[package]]
name = "flate2"
version = "1.1.9"
version = "1.1.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "843fba2746e448b37e26a819579957415c8cef339bf08564fe8b7ddbd959573c"
checksum = "6e634e2e0ebac1ee034020da1ca582e17ffe4e0f5e985823721e168928136dcb"
dependencies = [
"crc32fast",
"miniz_oxide",
"miniz_oxide 0.9.1",
"zlib-rs",
]
[[package]]
@@ -2101,6 +2181,22 @@ dependencies = [
"weezl",
]
[[package]]
name = "glass_pumpkin"
version = "2.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6ff9ef01d1ba41822349a2a9970e706dc6035637957cdddecf32482a8a90e86e"
dependencies = [
"crypto-bigint 0.7.5",
"crypto-primes",
"getrandom 0.4.3",
"num-bigint 0.5.1",
"num-integer",
"num-traits",
"rand_core 0.10.1",
"thiserror 2.0.21",
]
[[package]]
name = "glob-match"
version = "0.2.1"
@@ -2119,6 +2215,121 @@ dependencies = [
"wasm-bindgen",
]
[[package]]
name = "grammers-client"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "08072ddb6eeaca38870a0401d7a25ade7a541a5a3b81ada6877b981fe44606f5"
dependencies = [
"futures-util",
"grammers-crypto",
"grammers-mtsender",
"grammers-session",
"grammers-tl-types",
"jiff",
"md5",
"mime_guess",
"pin-project-lite",
"tokio",
"tracing",
]
[[package]]
name = "grammers-crypto"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c0154344dab25d8365408df205e4440b432cae02ef9dae2df478434be77c7075"
dependencies = [
"aes 0.9.3",
"ctr 0.10.1",
"getrandom 0.4.3",
"glass_pumpkin",
"hmac 0.13.0",
"num-bigint 0.5.1",
"num-traits",
"pbkdf2 0.13.0",
"sha1 0.11.0",
"sha2 0.11.1",
]
[[package]]
name = "grammers-mtproto"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ecead7953adb1c9ad6a2bdcea9f9931414be54246bdb4c722e990f7ade1193d3"
dependencies = [
"bytes",
"crc32fast",
"flate2",
"getrandom 0.4.3",
"grammers-crypto",
"grammers-tl-types",
"num-bigint 0.5.1",
"sha1 0.11.0",
"tracing",
]
[[package]]
name = "grammers-mtsender"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b009d7d0f571cabac15e1ac7f588fefb764f23aba13bf8b9063a0a32a036075d"
dependencies = [
"bytes",
"grammers-crypto",
"grammers-mtproto",
"grammers-session",
"grammers-tl-types",
"hickory-resolver",
"locate-locale",
"os_info",
"tokio",
"tokio-socks",
"tracing",
"url",
]
[[package]]
name = "grammers-session"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3f34260cb63ecf5901e735a5f088ba6cb837d954649c223d5a53e7e980468ae0"
dependencies = [
"grammers-tl-types",
"serde",
"serde_with",
"tokio",
"tracing",
]
[[package]]
name = "grammers-tl-gen"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bf49f6769cffae7ea376669f3073c70eeb16ccc38f00ff019394069f3d401a4c"
dependencies = [
"grammers-tl-parser",
]
[[package]]
name = "grammers-tl-parser"
version = "1.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "646fc5eeb27461ffcc94fcf9fda3ffd9b402a20250447ae4a65c39c1a905e381"
dependencies = [
"crc32fast",
]
[[package]]
name = "grammers-tl-types"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "069e64c424bde5764391bc72f07085a12177d7053c8a0da355425527fd7c221f"
dependencies = [
"grammers-tl-gen",
"grammers-tl-parser",
]
[[package]]
name = "group"
version = "0.13.0"
@@ -2210,7 +2421,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "543f93241d32b3f00569201bfce9d7a93c92c6421b23c77864ac929dc947b9fc"
dependencies = [
"hax-lib-macros",
"num-bigint",
"num-bigint 0.4.8",
"num-traits",
]
@@ -2264,6 +2475,76 @@ version = "0.3.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7ebdb29d2ea9ed0083cd8cece49bbd968021bd99b0849edb4a9a7ee0fdf6a4e0"
[[package]]
name = "hickory-net"
version = "0.26.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c480823ed7c2c5d0f09c41020cb6b7c28029ce60ec42dc942158dcf22f8e0a4d"
dependencies = [
"async-trait",
"cfg-if",
"data-encoding",
"futures-channel",
"futures-io",
"futures-util",
"hickory-proto",
"idna",
"ipnet",
"jni",
"rand 0.10.2",
"thiserror 2.0.21",
"tinyvec",
"tokio",
"tracing",
"url",
]
[[package]]
name = "hickory-proto"
version = "0.26.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "12b92608f679a6fa515dd1d15c1ff89443026e391200a2c840c7afcba482893d"
dependencies = [
"data-encoding",
"idna",
"ipnet",
"jni",
"once_cell",
"prefix-trie",
"rand 0.10.2",
"ring",
"thiserror 2.0.21",
"tinyvec",
"tracing",
"url",
]
[[package]]
name = "hickory-resolver"
version = "0.26.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3f3da5255c95d5a716857d54b5b8f4e8d67c3484d3beaaaae2ce25063b3ba981"
dependencies = [
"cfg-if",
"futures-util",
"hickory-net",
"hickory-proto",
"ipconfig",
"ipnet",
"jni",
"moka",
"ndk-context",
"once_cell",
"parking_lot 0.12.5",
"rand 0.10.2",
"resolv-conf",
"smallvec",
"system-configuration",
"thiserror 2.0.21",
"tokio",
"tracing",
]
[[package]]
name = "hkdf"
version = "0.12.4"
@@ -2318,7 +2599,7 @@ dependencies = [
"hkdf 0.13.0",
"hybrid-array",
"rand_core 0.10.1",
"sha2 0.11.0",
"sha2 0.11.1",
"subtle",
"x25519-dalek 3.0.0",
"zeroize",
@@ -2802,11 +3083,27 @@ dependencies = [
"rustversion",
]
[[package]]
name = "ipconfig"
version = "0.3.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4d40460c0ce33d6ce4b0630ad68ff63d6661961c48b6dba35e5a4d81cfb48222"
dependencies = [
"socket2",
"widestring",
"windows-registry",
"windows-result",
"windows-sys 0.61.2",
]
[[package]]
name = "ipnet"
version = "2.12.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6a756c3fac73139e83f14c2d742155dd2b78d3ee56597b419a0579b7bdd6dd78"
dependencies = [
"serde",
]
[[package]]
name = "iprange"
@@ -2876,6 +3173,60 @@ version = "1.0.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682"
[[package]]
name = "jiff"
version = "0.2.38"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b2b005715dcbeb0089a3c0dab99f2ff1cc3b2525323552703d648585d342a383"
dependencies = [
"defmt",
"jiff-core",
"jiff-static",
"jiff-tzdb-platform",
"log",
"portable-atomic",
"portable-atomic-util",
"serde_core",
"windows-link",
]
[[package]]
name = "jiff-core"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5e52fe76043ccecc9005d2305ebaadf7d7fc0cc89ca6baa10a94d6bc68c7128c"
dependencies = [
"defmt",
"log",
]
[[package]]
name = "jiff-static"
version = "0.2.38"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2cc9817253cf7c7ee4684451bd327e88d6f3658014e54a29198625590650695c"
dependencies = [
"jiff-core",
"proc-macro2",
"quote",
"syn 2.0.119",
]
[[package]]
name = "jiff-tzdb"
version = "0.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fa8377070c6bae868759445e5a77f66d84f0b72f3a054bfb00e6d038b8282da7"
[[package]]
name = "jiff-tzdb-platform"
version = "0.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "875a5a69ac2bab1a891711cf5eccbec1ce0341ea805560dcd90b7a2e925132e8"
dependencies = [
"jiff-tzdb",
]
[[package]]
name = "jni"
version = "0.22.4"
@@ -3332,6 +3683,15 @@ version = "0.8.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "92daf443525c4cce67b150400bc2316076100ce0b3686209eb8cf3c31612e6f0"
[[package]]
name = "locate-locale"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2835eaaed39a92511442aff277d4dca3d7674ca058df3bc45170661c2ccb4619"
dependencies = [
"winapi",
]
[[package]]
name = "lock_api"
version = "0.4.14"
@@ -3485,7 +3845,7 @@ dependencies = [
"ruma",
"serde",
"serde_json",
"sha2 0.11.0",
"sha2 0.11.1",
"subtle",
"thiserror 2.0.21",
"time",
@@ -3557,11 +3917,17 @@ dependencies = [
"serde",
"serde_bytes",
"serde_json",
"sha2 0.11.0",
"sha2 0.11.1",
"thiserror 2.0.21",
"zeroize",
]
[[package]]
name = "md5"
version = "0.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7ebb8d8732c6a6df3d8f032a82911cfc747e00efb95cc46e8d0acd5b5b88570c"
[[package]]
name = "memchr"
version = "2.8.3"
@@ -3621,6 +3987,16 @@ dependencies = [
"simd-adler32",
]
[[package]]
name = "miniz_oxide"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b63fbc4a50860e98e7b2aa7804ded1db5cbc3aff9193adaff57a6931bf7c4b4c"
dependencies = [
"adler2",
"simd-adler32",
]
[[package]]
name = "mio"
version = "1.2.2"
@@ -3633,6 +4009,23 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "moka"
version = "0.12.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4293f18e7567a1caf3c584855554377025c65e0aa445344d04171f5ad63d19b9"
dependencies = [
"crossbeam-channel",
"crossbeam-epoch",
"crossbeam-utils",
"equivalent",
"parking_lot 0.12.5",
"portable-atomic",
"smallvec",
"tagptr",
"uuid",
]
[[package]]
name = "moxcms"
version = "0.8.1"
@@ -3684,9 +4077,21 @@ version = "1.0.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "650eef8c711430f1a879fdd01d4745a7deea475becfb90269c06775983bbf086"
[[package]]
name = "nix"
version = "0.31.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cf20d2fde8ff38632c426f1165ed7436270b44f199fc55284c38276f9db47c3d"
dependencies = [
"bitflags 2.13.2",
"cfg-if",
"cfg_aliases",
"libc",
]
[[package]]
name = "nobilis"
version = "1.2.0"
version = "1.3.3"
dependencies = [
"aes-gcm",
"anyhow",
@@ -3703,11 +4108,15 @@ dependencies = [
"flate2",
"fs2",
"futures",
"grammers-client",
"grammers-mtsender",
"grammers-session",
"hex",
"hmac 0.13.0",
"http-body-util",
"hyper",
"hyper-util",
"iana-time-zone",
"image",
"imagesize",
"irc",
@@ -3818,6 +4227,17 @@ dependencies = [
"num-traits",
]
[[package]]
name = "num-bigint"
version = "0.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "93e7820bc0a80a0238e650327316f929ba18d5be054b647490a3a6a339f3e7c0"
dependencies = [
"num-integer",
"num-traits",
"rand_core 0.10.1",
]
[[package]]
name = "num-bigint-dig"
version = "0.8.6"
@@ -3947,6 +4367,17 @@ dependencies = [
"objc2-foundation",
]
[[package]]
name = "objc2-cloud-kit"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "73ad74d880bb43877038da939b7427bba67e9dd42004a18b809ba7d87cee241c"
dependencies = [
"bitflags 2.13.2",
"objc2",
"objc2-foundation",
]
[[package]]
name = "objc2-core-audio"
version = "0.3.2"
@@ -3970,6 +4401,16 @@ dependencies = [
"objc2",
]
[[package]]
name = "objc2-core-data"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0b402a653efbb5e82ce4df10683b6b28027616a2715e90009947d50b8dd298fa"
dependencies = [
"objc2",
"objc2-foundation",
]
[[package]]
name = "objc2-core-foundation"
version = "0.3.2"
@@ -3983,6 +4424,51 @@ dependencies = [
"objc2",
]
[[package]]
name = "objc2-core-graphics"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e022c9d066895efa1345f8e33e584b9f958da2fd4cd116792e15e07e4720a807"
dependencies = [
"bitflags 2.13.2",
"dispatch2",
"objc2",
"objc2-core-foundation",
"objc2-io-surface",
]
[[package]]
name = "objc2-core-image"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e5d563b38d2b97209f8e861173de434bd0214cf020e3423a52624cd1d989f006"
dependencies = [
"objc2",
"objc2-foundation",
]
[[package]]
name = "objc2-core-location"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ca347214e24bc973fc025fd0d36ebb179ff30536ed1f80252706db19ee452009"
dependencies = [
"objc2",
"objc2-foundation",
]
[[package]]
name = "objc2-core-text"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0cde0dfb48d25d2b4862161a4d5fcc0e3c24367869ad306b0c9ec0073bfed92d"
dependencies = [
"bitflags 2.13.2",
"objc2",
"objc2-core-foundation",
"objc2-core-graphics",
]
[[package]]
name = "objc2-encode"
version = "4.1.0"
@@ -4012,11 +4498,69 @@ dependencies = [
"objc2-core-foundation",
]
[[package]]
name = "objc2-io-surface"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "180788110936d59bab6bd83b6060ffdfffb3b922ba1396b312ae795e1de9d81d"
dependencies = [
"bitflags 2.13.2",
"objc2",
"objc2-core-foundation",
]
[[package]]
name = "objc2-quartz-core"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "96c1358452b371bf9f104e21ec536d37a650eb10f7ee379fff67d2e08d537f1f"
dependencies = [
"bitflags 2.13.2",
"objc2",
"objc2-core-foundation",
"objc2-foundation",
]
[[package]]
name = "objc2-ui-kit"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d87d638e33c06f577498cbcc50491496a3ed4246998a7fbba7ccb98b1e7eab22"
dependencies = [
"bitflags 2.13.2",
"block2",
"objc2",
"objc2-cloud-kit",
"objc2-core-data",
"objc2-core-foundation",
"objc2-core-graphics",
"objc2-core-image",
"objc2-core-location",
"objc2-core-text",
"objc2-foundation",
"objc2-quartz-core",
"objc2-user-notifications",
]
[[package]]
name = "objc2-user-notifications"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9df9128cbbfef73cda168416ccf7f837b62737d748333bfe9ab71c245d76613e"
dependencies = [
"objc2",
"objc2-foundation",
]
[[package]]
name = "once_cell"
version = "1.21.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50"
dependencies = [
"critical-section",
"portable-atomic",
]
[[package]]
name = "oneshot-fused-workaround"
@@ -4143,6 +4687,21 @@ dependencies = [
"num-traits",
]
[[package]]
name = "os_info"
version = "3.15.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9cf20a545b305cf1da722b236b5155c9bb35f1d5ceb28c048bd96ca842f41b5b"
dependencies = [
"android_system_properties",
"log",
"nix",
"objc2",
"objc2-foundation",
"objc2-ui-kit",
"windows-sys 0.61.2",
]
[[package]]
name = "os_str_bytes"
version = "6.6.1"
@@ -4284,6 +4843,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "112d82ceb8c5bf524d9af484d4e4970c9fd5a0cc15ba14ad93dccd28873b0629"
dependencies = [
"digest 0.11.3",
"hmac 0.13.0",
]
[[package]]
@@ -4452,7 +5012,7 @@ dependencies = [
"crc32fast",
"fdeflate",
"flate2",
"miniz_oxide",
"miniz_oxide 0.8.9",
]
[[package]]
@@ -4499,6 +5059,21 @@ dependencies = [
"universal-hash 0.6.1",
]
[[package]]
name = "portable-atomic"
version = "1.15.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "05c8b63e8d9609db387f0324918f81d68fe27748f084ef092fb35954d0539a85"
[[package]]
name = "portable-atomic-util"
version = "0.2.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "10ab3eb7f3becc3a1cbc4f2c6f20267996cfc1a6467a873763411b136a122715"
dependencies = [
"portable-atomic",
]
[[package]]
name = "postage"
version = "0.5.0"
@@ -4544,6 +5119,17 @@ version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "925383efa346730478fb4838dbe9137d2a47675ad789c546d150a6e1dd4ab31c"
[[package]]
name = "prefix-trie"
version = "0.8.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4cf6e3177f0684016a5c209b00882e15f8bdd3f3bb48f0491df10cd102d0c6e7"
dependencies = [
"either",
"ipnet",
"num-traits",
]
[[package]]
name = "primeorder"
version = "0.13.6"
@@ -5044,6 +5630,12 @@ dependencies = [
"rand_core 0.10.1",
]
[[package]]
name = "resolv-conf"
version = "0.7.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e061d1b48cb8d38042de4ae0a7a6401009d6143dc80d2e2d6f31f0bdd6470c7"
[[package]]
name = "retry-error"
version = "0.14.0"
@@ -5512,7 +6104,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b7f4bc775c73d9a02cde8bf7b2ec4c9d12743edf609006c7facc23998404cd1d"
dependencies = [
"bitflags 2.13.2",
"core-foundation",
"core-foundation 0.10.1",
"core-foundation-sys",
"libc",
"security-framework-sys",
@@ -5738,9 +6330,9 @@ dependencies = [
[[package]]
name = "sha2"
version = "0.11.0"
version = "0.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "446ba717509524cb3f22f17ecc096f10f4822d76ab5c0b9822c5f9c284e825f4"
checksum = "47d7069beb7d6ac7b9acd1039986e73443f24234f41074da099d6f994ac9ad19"
dependencies = [
"cfg-if",
"cpufeatures 0.3.0",
@@ -6270,6 +6862,33 @@ dependencies = [
"windows",
]
[[package]]
name = "system-configuration"
version = "0.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a13f3d0daba03132c0aa9767f98351b3488edc2c100cda2d2ec2b04f3d8d3c8b"
dependencies = [
"bitflags 2.13.2",
"core-foundation 0.9.4",
"system-configuration-sys",
]
[[package]]
name = "system-configuration-sys"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8e1d1b10ced5ca923a1fcb8d03e96b8d3268065d724548c0211415ff6ac6bac4"
dependencies = [
"core-foundation-sys",
"libc",
]
[[package]]
name = "tagptr"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7b2093cf4c8eb1e67749a6762251bc9cd836b6fc171623bd0a9d324d37af2417"
[[package]]
name = "tap"
version = "1.0.1"
@@ -6428,9 +7047,9 @@ dependencies = [
[[package]]
name = "tokio"
version = "1.53.1"
version = "1.53.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "202caea871b69668250d242070849eb495be178ed697a3e98aebce5bc81a0bed"
checksum = "ce3335fa71841cda333a58d7615b03901380ecf09d59b3296d21f8bbac0dde4e"
dependencies = [
"bytes",
"libc",
@@ -8009,7 +8628,7 @@ dependencies = [
"serde",
"serde_bytes",
"serde_json",
"sha2 0.11.0",
"sha2 0.11.1",
"subtle",
"thiserror 2.0.21",
"x25519-dalek 3.0.0",
@@ -8219,6 +8838,12 @@ dependencies = [
"safe_arch",
]
[[package]]
name = "widestring"
version = "1.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "72069c3113ab32ab29e5584db3c6ec55d416895e60715417b5b883a357c3e471"
[[package]]
name = "wildmatch"
version = "2.6.1"
@@ -8339,6 +8964,17 @@ dependencies = [
"windows-link",
]
[[package]]
name = "windows-registry"
version = "0.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "02752bf7fbdcce7f2a27a742f798510f3e5ad88dbe84871e5168e2120c3d5720"
dependencies = [
"windows-link",
"windows-result",
"windows-strings",
]
[[package]]
name = "windows-result"
version = "0.4.1"
@@ -8784,6 +9420,12 @@ dependencies = [
"syn 2.0.119",
]
[[package]]
name = "zlib-rs"
version = "0.6.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b268e58e7c693d7c271f93ffc4ba3b380412554231c85bf61ca7af91042a4112"
[[package]]
name = "zmij"
version = "1.0.23"
Cargo.toml
+8 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "nobilis"
version = "1.2.0"
version = "1.3.3"
edition = "2021"
description = "Chat daemon: IRC, Discord, Sneedchat and Matrix behind one unified JSON model over a Unix socket"
license = "GPL-3.0-or-later"
@@ -59,6 +59,12 @@ qrcode = { version = "0.14", features = ["image"] }
image = { version = "0.25", default-features = false, features = ["png", "gif"] }
chrono = { version = "0.4", default-features = false, features = ["std"] }
# Telegram backend (MTProto, pure Rust). `grammers-session` without its default `sqlite-storage` (libsql would sit
# beside rusqlite and matrix-sdk-sqlite): the session is serialized into the account instead - see backend/telegram.
grammers-client = { version = "=0.11.0", default-features = false, features = ["proxy", "fs"] }
grammers-mtsender = { version = "=0.11.0", default-features = false, features = ["proxy"] }
grammers-session = { version = "=0.11.0", default-features = false, features = ["serde"] }
# Sneedchat backend (SneedChat over embedded Tor)
arti-client = { version = "0.47", default-features = false, features = ["tokio", "rustls", "onion-service-client", "compression"] }
tor-rtcompat = { version = "0.47", features = ["tokio", "rustls"] }
@@ -121,6 +127,7 @@ sonora = "0.2"
# Soundboard sounds are MP3 or Ogg Vorbis, decoded here to be mixed into a call.
symphonia = { version = "0.5", default-features = false, features = ["mp3", "ogg", "vorbis"] }
imagesize = "0.13"
iana-time-zone = "0.1.65"
# Two calls, both in main.rs: see keep_the_heap_honest() there for why a chat
# daemon has anything to say to its allocator.
src/accounts.rs
+222 -6
View File
@@ -390,9 +390,8 @@ pub struct SteamAccountConfig {
/// sign-in so Steam Guard does not ask for a code every time.
#[serde(default)]
pub guard_data: Option<String>,
/// Which kind of session this account signed in as. Always `chat`, the
/// mobile app's, which lives beside the Steam launcher; kept so files
/// written when the desktop kind could still be chosen read the same.
/// Which kind of session this account signed in as. Unused now: moho signs in as a desktop client of its own
/// (see backend/steam/auth.rs); kept so files written when it was chosen read the same.
#[serde(default)]
pub session: String,
/// Whether this account's connections go through Tor or the SOCKS5 proxy
@@ -490,6 +489,54 @@ impl TwitchAccountConfig {
}
}
/// A Telegram account: a phone number's, signed in over MTProto (see backend/telegram).
///
/// Telegram gives the owner of a phone number one `api_id` and `api_hash`, and a client sends them with every
/// connection. Moho carries none of its own: the ones the person typed into the sign-in form are kept here, with
/// the account, and are as private as the session is. The session itself (the authorization key Telegram issued,
/// and what a client caches to catch up on updates) is large and changes often, so it is a file of its own in the
/// daemon's data directory (see backend/telegram/session.rs), not a field of this one.
#[derive(Serialize, Deserialize, Clone, Debug, Default)]
pub struct TelegramAccountConfig {
#[serde(default)]
pub api_id: i32,
#[serde(default)]
pub api_hash: String,
/// The number the account was signed in with, in international form (`+15551234567`).
#[serde(default)]
pub phone: String,
/// Telegram's number for the account, which survives the number or the name being changed. The account's id here
/// is built from it.
#[serde(default)]
pub user_id: i64,
/// The `@name`, if the account has one. Telegram's, not ours.
#[serde(default)]
pub username: Option<String>,
/// The account's own name in Telegram (first and last), for the sidebar.
#[serde(default)]
pub name: String,
/// What to call this account here. Local: Telegram is never told about it.
#[serde(default)]
pub display_name: Option<String>,
/// Whether this account's connections go through Tor or the SOCKS5 proxy set in the network settings.
#[serde(default)]
pub use_tor: bool,
/// Strict routing: everything the window loads for this account's conversations goes through the route too.
#[serde(default)]
pub strict_route: bool,
}
impl TelegramAccountConfig {
pub fn account_id(&self) -> String {
format!("telegram:{}", self.user_id)
}
/// Whether there is a sign-in behind this account: one that has been through is what has an id.
pub fn is_signed_in(&self) -> bool {
self.user_id != 0 && self.api_id != 0 && !self.api_hash.is_empty()
}
}
#[derive(Serialize, Deserialize, Clone, Debug, Default)]
struct AccountsFile {
#[serde(default, rename = "account")]
@@ -510,6 +557,8 @@ struct AccountsFile {
steam: Vec<SteamAccountConfig>,
#[serde(default, rename = "twitch_account")]
twitch: Vec<TwitchAccountConfig>,
#[serde(default, rename = "telegram_account")]
telegram: Vec<TelegramAccountConfig>,
}
/// Account/credential persistence at ~/.config/nobilis/accounts.toml,
@@ -529,6 +578,8 @@ pub struct AccountStore {
steam: Mutex<HashMap<String, SteamAccountConfig>>,
/// Read by `persist` the same way, for the same reason.
twitch: Mutex<HashMap<String, TwitchAccountConfig>>,
/// And this one.
telegram: Mutex<HashMap<String, TelegramAccountConfig>>,
}
impl AccountStore {
@@ -539,7 +590,7 @@ impl AccountStore {
}
pub fn open(path: PathBuf) -> Result<Self> {
let (irc, discord, sneedchat, matrix, kick, steam, twitch) = if path.exists() {
let (irc, discord, sneedchat, matrix, kick, steam, twitch, telegram) = if path.exists() {
let text = std::fs::read_to_string(&path)
.with_context(|| format!("reading {}", path.display()))?;
let file: AccountsFile = toml::from_str(&text)
@@ -551,9 +602,10 @@ impl AccountStore {
let kick = file.kick.into_iter().map(|a| (a.account_id(), a)).collect();
let steam = file.steam.into_iter().map(|a| (a.account_id(), a)).collect();
let twitch = file.twitch.into_iter().map(|a| (a.account_id(), a)).collect();
(irc, discord, sneedchat, matrix, kick, steam, twitch)
let telegram = file.telegram.into_iter().map(|a| (a.account_id(), a)).collect();
(irc, discord, sneedchat, matrix, kick, steam, twitch, telegram)
} else {
(HashMap::new(), HashMap::new(), HashMap::new(), HashMap::new(), HashMap::new(), HashMap::new(), HashMap::new())
(HashMap::new(), HashMap::new(), HashMap::new(), HashMap::new(), HashMap::new(), HashMap::new(), HashMap::new(), HashMap::new())
};
Ok(Self {
path,
@@ -564,6 +616,7 @@ impl AccountStore {
kick: Mutex::new(kick),
steam: Mutex::new(steam),
twitch: Mutex::new(twitch),
telegram: Mutex::new(telegram),
})
}
@@ -586,6 +639,7 @@ impl AccountStore {
kick: kick.values().cloned().collect(),
steam: self.steam.lock().unwrap().values().cloned().collect(),
twitch: self.twitch.lock().unwrap().values().cloned().collect(),
telegram: self.telegram.lock().unwrap().values().cloned().collect(),
};
let text = toml::to_string_pretty(&file)?;
// Atomic write-temp-then-rename, same spirit as libpurple's periodic
@@ -823,6 +877,48 @@ impl AccountStore {
Ok(true)
}
pub fn get_telegram(&self, account_id: &str) -> Option<TelegramAccountConfig> {
self.telegram.lock().unwrap().get(account_id).cloned()
}
pub fn all_telegram(&self) -> Vec<TelegramAccountConfig> {
self.telegram.lock().unwrap().values().cloned().collect()
}
/// Upserts: signing in again as the same account is how a session Telegram has ended gets replaced. What the
/// person set up on the account - its route, its name here - stays.
pub fn add_telegram(&self, mut config: TelegramAccountConfig) -> Result<TelegramAccountConfig> {
let id = config.account_id();
{
let mut telegram = self.telegram.lock().unwrap();
if let Some(old) = telegram.get(&id) {
config.strict_route |= old.strict_route;
config.use_tor |= old.use_tor;
if config.display_name.is_none() {
config.display_name = old.display_name.clone();
}
}
telegram.insert(id, config.clone());
}
self.persist(&self.irc.lock().unwrap(), &self.discord.lock().unwrap(), &self.sneedchat.lock().unwrap(), &self.matrix.lock().unwrap(), &self.kick.lock().unwrap())?;
Ok(config)
}
/// Changes one Telegram account, and writes the file. `false` when there is no such account.
///
/// The lock is let go before the write: `persist` takes it again.
pub fn mutate_telegram(&self, account_id: &str, change: impl FnOnce(&mut TelegramAccountConfig)) -> Result<bool> {
{
let mut telegram = self.telegram.lock().unwrap();
match telegram.get_mut(account_id) {
None => return Ok(false),
Some(account) => change(account),
}
}
self.persist(&self.irc.lock().unwrap(), &self.discord.lock().unwrap(), &self.sneedchat.lock().unwrap(), &self.matrix.lock().unwrap(), &self.kick.lock().unwrap())?;
Ok(true)
}
/// Upserts, like the others: signing in to Kick again with the same
/// username is how a rejected token gets replaced.
pub fn add_kick(&self, mut config: KickAccountConfig) -> Result<KickAccountConfig> {
@@ -965,6 +1061,10 @@ impl AccountStore {
a.use_tor = routed;
a.strict_route = strict;
true
} else if let Some(a) = self.telegram.lock().unwrap().get_mut(account_id) {
a.use_tor = routed;
a.strict_route = strict;
true
} else {
false
};
@@ -995,6 +1095,9 @@ impl AccountStore {
if let Some(a) = self.get_twitch(account_id) {
return pick(a.use_tor, a.strict_route);
}
if let Some(a) = self.get_telegram(account_id) {
return pick(a.use_tor, a.strict_route);
}
self.get_kick(account_id).and_then(|a| pick(a.use_tor, a.strict_route))
}
@@ -1082,6 +1185,13 @@ impl AccountStore {
return Ok(true);
}
}
{
let removed = self.telegram.lock().unwrap().remove(account_id).is_some();
if removed {
self.persist(&self.irc.lock().unwrap(), &self.discord.lock().unwrap(), &self.sneedchat.lock().unwrap(), &self.matrix.lock().unwrap(), &self.kick.lock().unwrap())?;
return Ok(true);
}
}
let mut kick = self.kick.lock().unwrap();
if kick.remove(account_id).is_some() {
self.persist(&self.irc.lock().unwrap(), &self.discord.lock().unwrap(), &self.sneedchat.lock().unwrap(), &self.matrix.lock().unwrap(), &kick)?;
@@ -1246,6 +1356,9 @@ impl AccountStore {
if self.mutate_twitch(account_id, |a| a.display_name = value.clone())? {
return Ok(true);
}
if self.mutate_telegram(account_id, |a| a.display_name = value.clone())? {
return Ok(true);
}
let mut kick = self.kick.lock().unwrap();
match kick.get_mut(account_id) {
None => Ok(false),
@@ -1282,6 +1395,9 @@ impl AccountStore {
if let Some(a) = self.twitch.lock().unwrap().get(account_id) {
return named(&a.display_name);
}
if let Some(a) = self.telegram.lock().unwrap().get(account_id) {
return named(&a.display_name);
}
self.kick.lock().unwrap().get(account_id).and_then(|a| named(&a.display_name))
}
@@ -1627,6 +1743,50 @@ pub fn twitch_account_to_json(a: &TwitchAccountConfig, state: &str) -> Account {
}
}
/// Telegram has none of IRC's concepts either. `has_password` says there is a sign-in behind the account, which is
/// what "signed in" means for one that never keeps a password.
pub fn telegram_account_to_json(a: &TelegramAccountConfig, state: &str) -> Account {
Account {
id: a.account_id(),
service: "telegram".to_string(),
status: "online".to_string(),
status_text: String::new(),
auto_idle: false,
display_name: a
.display_name
.clone()
.filter(|s| !s.is_empty())
.or_else(|| Some(a.name.clone()).filter(|s| !s.is_empty()))
.or_else(|| a.username.clone().filter(|s| !s.is_empty()))
.unwrap_or_else(|| a.phone.clone()),
state: state.to_string(),
autojoin: String::new(),
has_nickserv_password: false,
sasl_enabled: false,
sasl_username: String::new(),
allow_plaintext_sasl: false,
reconnect_for_sasl: false,
sasl_mechanism: String::new(),
has_sasl_certificate: false,
current_nick: String::new(),
highlight_keywords: Vec::new(),
ssl: true,
has_password: a.is_signed_in(),
avatar_url: None,
sneedchat_rooms: Vec::new(),
tor_mode: None,
tor_proxy: None,
use_tor: a.use_tor,
route_level: route_level(a.use_tor, a.strict_route),
has_key_backup: false,
rtc_focus_url: None,
sliding_sync: false,
dehydration: false,
web_session: false,
moderation: false,
}
}
/// Matrix has none of IRC's autojoin/SASL/NickServ concepts either - same
/// zero-value convention as discord_account_to_json/sneedchat_account_to_json.
/// `has_key_backup` comes from the caller (Runtime::list_accounts) rather
@@ -1881,3 +2041,59 @@ mod twitch_account_tests {
assert!(reopened.all_twitch().is_empty());
}
}
#[cfg(test)]
mod telegram_account_tests {
use super::*;
fn store(name: &str) -> (AccountStore, PathBuf) {
let path = std::env::temp_dir().join(format!("nobilis-telegram-accounts-{name}-{}.toml", std::process::id()));
let _ = std::fs::remove_file(&path);
(AccountStore::open(path.clone()).unwrap(), path)
}
fn account(user_id: i64) -> TelegramAccountConfig {
TelegramAccountConfig { api_id: 12345, api_hash: "abcdef".into(), phone: "+15551234567".into(), user_id, name: "Some One".into(), username: Some("someone".into()), ..Default::default() }
}
#[test]
fn a_telegram_account_survives_a_restart_without_its_session_in_the_file() {
let (s, path) = store("restart");
s.add_telegram(account(777)).unwrap();
let text = std::fs::read_to_string(&path).unwrap();
assert!(text.contains("[[telegram_account]]"), "{text}");
assert!(!text.contains("session"), "the session is a file of its own, not a field here: {text}");
let reopened = AccountStore::open(path.clone()).unwrap();
let got = reopened.get_telegram("telegram:777").expect("found under the id built from Telegram's number");
assert_eq!((got.api_id, got.api_hash.as_str(), got.phone.as_str(), got.username.as_deref()), (12345, "abcdef", "+15551234567", Some("someone")));
assert!(got.is_signed_in());
std::fs::remove_file(path).ok();
}
#[test]
fn signing_in_again_keeps_what_the_person_set_up_and_removing_takes_only_that_account() {
let (s, path) = store("again");
s.add_telegram(account(1)).unwrap();
s.add_telegram(account(2)).unwrap();
s.set_route_level("telegram:1", "strict").unwrap();
s.set_display_name("telegram:1", "Me on Telegram").unwrap();
s.add_telegram(TelegramAccountConfig { name: "Renamed".into(), ..account(1) }).unwrap();
let got = s.get_telegram("telegram:1").unwrap();
assert_eq!((got.name.as_str(), got.display_name.as_deref(), got.strict_route), ("Renamed", Some("Me on Telegram"), true));
assert_eq!(s.route_level_of("telegram:1").as_deref(), Some("strict"));
assert_eq!(s.all_telegram().len(), 2);
assert!(s.remove("telegram:1").unwrap());
assert!(!s.remove("telegram:1").unwrap());
assert!(s.get_telegram("telegram:2").is_some());
assert!(AccountStore::open(path.clone()).unwrap().get_telegram("telegram:1").is_none(), "the file forgot too");
std::fs::remove_file(path).ok();
}
#[test]
fn the_listed_account_is_named_for_the_person_and_says_it_is_signed_in() {
let listed = telegram_account_to_json(&account(5), "connected");
assert_eq!((listed.service.as_str(), listed.id.as_str(), listed.display_name.as_str(), listed.has_password), ("telegram", "telegram:5", "Some One", true));
let unnamed = telegram_account_to_json(&TelegramAccountConfig { name: String::new(), username: None, ..account(5) }, "connected");
assert_eq!(unnamed.display_name, "+15551234567");
}
}
src/backend/discord/commands.rs
+2 -2
View File
@@ -31,7 +31,7 @@ pub async fn list_commands(state: &AppState, account_id: &str, buffer_id: &str,
.get(format!("{API_BASE}/channels/{channel_id}/application-commands/search"))
.query(&params)
.header("Authorization", &cfg.token)
.send()
.send_paced()
.await
.context("reading the commands")?;
if !resp.status().is_success() {
@@ -243,7 +243,7 @@ pub async fn use_component(
let resp = http_client_for(&cfg.token)
.get(format!("{API_BASE}/channels/{channel_id}/messages/{message_id}"))
.header("Authorization", &cfg.token)
.send()
.send_paced()
.await
.context("finding out whose button that is")?;
if !resp.status().is_success() {
src/backend/discord/events.rs
+3 -3
View File
@@ -135,7 +135,7 @@ pub async fn list(state: &AppState, account_id: &str, guild_id: &str) -> Result<
.get(format!("{API_BASE}/guilds/{guild_id}/scheduled-events"))
.query(&[("with_user_count", "true")])
.header("Authorization", &cfg.token)
.send()
.send_paced()
.await
.context("reading the events")?;
if resp.status().is_success() {
@@ -153,7 +153,7 @@ pub async fn list(state: &AppState, account_id: &str, guild_id: &str) -> Result<
.get(format!("{API_BASE}/guilds/{guild_id}/scheduled-events/{id}/users"))
.query(&[("limit", "100")])
.header("Authorization", &cfg.token)
.send()
.send_paced()
.await
{
if let Ok(users) = resp.json::<Value>().await {
@@ -380,7 +380,7 @@ pub async fn share_targets(state: &AppState, account_id: &str, guild_id: &str) -
.get(format!("{API_BASE}/users/{user_id}/profile"))
.query(&[("with_mutual_guilds", "true")])
.header("Authorization", &token)
.send()
.send_paced()
.await
{
if let Ok(profile) = resp.json::<Value>().await {
src/backend/discord/forums.rs
+2 -2
View File
@@ -125,7 +125,7 @@ async fn search_page(token: &str, channel_id: &str, sort_by: &str, archived: boo
.get(format!("{API_BASE}/channels/{channel_id}/threads/search"))
.query(&query)
.header("Authorization", token)
.send()
.send_paced()
.await
.context("reading the forum")?;
if !resp.status().is_success() {
@@ -216,7 +216,7 @@ pub async fn list_forum_posts(state: &AppState, account_id: &str, buffer_id: &st
.get(format!("{API_BASE}/channels/{id}/messages"))
.query(&[("limit", "1"), ("after", "0")])
.header("Authorization", token)
.send()
.send_paced()
.await
.ok()?;
if !resp.status().is_success() {
src/backend/discord/gateway.rs
+431 -48
View File
@@ -7,7 +7,134 @@
use super::*;
pub(super) const GATEWAY_URL: &str = "wss://gateway.discord.gg/?v=10&encoding=json";
pub(super) const GATEWAY_URL: &str = "wss://gateway.discord.gg/?v=10&encoding=json&compress=zlib-stream";
/// The same address asking for the stream compressed, as Discord's web client and Dissent do: what a session was last
/// given to resume at (kept on disk) may not say so, and a gateway that is not asked to compress does not.
fn compressed(url: &str) -> String {
if url.contains("compress=zlib-stream") {
url.to_string()
} else if url.contains('?') {
format!("{url}&compress=zlib-stream")
} else {
format!("{url}?compress=zlib-stream")
}
}
/// Discord's `zlib-stream`: every message is a piece of one zlib stream that lasts as long as the connection, each
/// ending in the four bytes of a sync flush (`00 00 FF FF`) - a message may arrive in several frames, and is whole when
/// the last of them ends so. Inflating needs the whole stream so far, not the frame alone.
pub(super) struct Inflater {
inflate: flate2::Decompress,
pending: Vec<u8>,
}
impl Inflater {
const SYNC_FLUSH: [u8; 4] = [0x00, 0x00, 0xff, 0xff];
pub(super) fn new() -> Self {
Inflater { inflate: flate2::Decompress::new(true), pending: Vec::new() }
}
/// Takes a frame; hands back the message it completes, if it does.
pub(super) fn push(&mut self, frame: &[u8]) -> Result<Option<String>> {
self.pending.extend_from_slice(frame);
if !self.pending.ends_with(&Self::SYNC_FLUSH) {
return Ok(None);
}
let input = std::mem::take(&mut self.pending);
let mut out: Vec<u8> = Vec::with_capacity(input.len() * 4);
let start = self.inflate.total_in();
loop {
let consumed = (self.inflate.total_in() - start) as usize;
if out.capacity() == out.len() {
out.reserve(input.len().max(4096));
}
let (before_in, before_out) = (self.inflate.total_in(), out.len());
self.inflate.decompress_vec(&input[consumed..], &mut out, flate2::FlushDecompress::Sync).context("inflating the gateway's stream")?;
let all_in = (self.inflate.total_in() - start) as usize >= input.len();
// Done when everything given has been taken and the output stopped for want of input, not of room: while
// the buffer is full there may be more to come out of what was taken.
if all_in && out.len() < out.capacity() {
break;
}
if self.inflate.total_in() == before_in && out.len() == before_out {
bail!("the gateway's stream stopped making sense");
}
}
Ok(Some(String::from_utf8(out).context("the gateway's message was not text")?))
}
}
#[cfg(test)]
mod heartbeat_tests {
use super::heartbeat_message;
#[test]
fn a_heartbeat_says_how_far_the_connection_has_got() {
assert_eq!(heartbeat_message(0), r#"{"d":null,"op":1}"#);
assert_eq!(heartbeat_message(42), r#"{"d":42,"op":1}"#);
}
}
#[cfg(test)]
mod inflate_tests {
use super::*;
/// A zlib stream the way Discord makes one: one compressor, each message flushed (sync) at its end.
fn frames(messages: &[&str]) -> Vec<Vec<u8>> {
let mut compress = flate2::Compress::new(flate2::Compression::default(), true);
messages
.iter()
.map(|m| {
let mut out = Vec::with_capacity(m.len() + 64);
let start = compress.total_in();
while ((compress.total_in() - start) as usize) < m.len() {
let done = (compress.total_in() - start) as usize;
compress.compress_vec(&m.as_bytes()[done..], &mut out, flate2::FlushCompress::None).unwrap();
if out.capacity() == out.len() {
out.reserve(64);
}
}
loop {
compress.compress_vec(&[], &mut out, flate2::FlushCompress::Sync).unwrap();
if out.ends_with(&Inflater::SYNC_FLUSH) {
break;
}
out.reserve(64);
}
out
})
.collect()
}
#[test]
fn messages_of_one_stream_are_read_in_turn() {
let big = format!(r#"{{"op":0,"t":"READY","d":{{"name":"{}"}}}}"#, "x".repeat(50_000));
let sent = [r#"{"op":10,"d":{"heartbeat_interval":41250}}"#, big.as_str(), r#"{"op":11}"#];
let mut inflater = Inflater::new();
for (frame, want) in frames(&sent).iter().zip(sent) {
assert_eq!(inflater.push(frame).unwrap().as_deref(), Some(want));
}
}
#[test]
fn a_message_in_two_frames_is_whole_when_the_second_arrives() {
let sent = [r#"{"op":0,"t":"MESSAGE_CREATE","d":{"content":"hello hello hello hello"}}"#];
let all = &frames(&sent)[0];
let (first, second) = all.split_at(all.len() / 2);
let mut inflater = Inflater::new();
assert_eq!(inflater.push(first).unwrap(), None, "not yet");
assert_eq!(inflater.push(second).unwrap().as_deref(), Some(sent[0]));
}
#[test]
fn the_address_asks_for_the_stream_compressed_once() {
assert_eq!(compressed("wss://x/?v=10&encoding=json"), "wss://x/?v=10&encoding=json&compress=zlib-stream");
assert_eq!(compressed(GATEWAY_URL), GATEWAY_URL);
assert_eq!(compressed("wss://x"), "wss://x?compress=zlib-stream");
}
}
/// Discord's gateway close code for "the token in your IDENTIFY payload is
/// invalid/revoked" - documented at discord.com/developers/docs/topics/
@@ -59,6 +186,22 @@ impl std::fmt::Display for GatewayRateLimited {
impl std::error::Error for GatewayRateLimited {}
/// Close codes that say the session is gone and cannot be resumed: not authenticated, an invalid sequence number, a
/// session that timed out. The next connection logs in afresh.
const SESSION_LOST_CODES: [u16; 3] = [4003, 4007, 4009];
/// The gateway closed the connection and ended the session with it.
#[derive(Debug)]
pub(super) struct SessionLost(u16);
impl std::fmt::Display for SessionLost {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "the gateway ended the session (close code {})", self.0)
}
}
impl std::error::Error for SessionLost {}
/// Discord's op 7: move to another server, resuming where this left off.
/// Its own type so the retry loop can tell it from a failure.
#[derive(Debug)]
@@ -73,12 +216,25 @@ impl std::fmt::Display for ReconnectRequested {
impl std::error::Error for ReconnectRequested {}
pub(super) async fn next_json<S>(stream: &mut S) -> Result<Value>
where
S: futures::Stream<Item = std::result::Result<WsMessage, tokio_tungstenite::tungstenite::Error>> + Unpin,
{
next_json_inflating(stream, None).await
}
/// `next_json` for a connection that was asked to compress its stream: binary frames are pieces of it.
pub(super) async fn next_json_inflating<S>(stream: &mut S, mut inflater: Option<&mut Inflater>) -> Result<Value>
where
S: futures::Stream<Item = std::result::Result<WsMessage, tokio_tungstenite::tungstenite::Error>> + Unpin,
{
loop {
match stream.next().await {
Some(Ok(WsMessage::Text(text))) => return serde_json::from_str(&text).context("parsing gateway JSON"),
Some(Ok(WsMessage::Binary(frame))) if inflater.is_some() => {
if let Some(text) = inflater.as_mut().expect("checked").push(&frame)? {
return serde_json::from_str(&text).context("parsing gateway JSON");
}
}
Some(Ok(WsMessage::Close(frame))) => {
if frame.as_ref().is_some_and(|f| u16::from(f.code) == CLOSE_CODE_AUTH_FAILED) {
return Err(GatewayAuthFailed.into());
@@ -86,6 +242,9 @@ where
if frame.as_ref().is_some_and(|f| u16::from(f.code) == CLOSE_CODE_RATE_LIMITED) {
return Err(GatewayRateLimited.into());
}
if let Some(code) = frame.as_ref().map(|f| u16::from(f.code)).filter(|c| SESSION_LOST_CODES.contains(c)) {
return Err(SessionLost(code).into());
}
bail!("connection closed: {frame:?}")
}
Some(Ok(_)) => continue,
@@ -132,7 +291,7 @@ pub fn spawn(state: AppState, config: DiscordAccountConfig) {
state.runtime.insert_task_handle(&account_id, join_handle.abort_handle());
}
pub(super) const RECONNECT_INITIAL_DELAY: Duration = Duration::from_secs(3);
pub(super) const RECONNECT_INITIAL_DELAY: Duration = Duration::from_secs(4);
pub(super) const RECONNECT_MAX_DELAY: Duration = Duration::from_secs(60);
@@ -173,6 +332,11 @@ pub(super) struct GatewaySession {
channel_map: HashMap<String, (String, String)>,
/// Each guild's payload as GUILD_CREATE delivered it.
guild_context: HashMap<String, Value>,
/// The session as kept on disk, for a restart to resume.
persist: super::session_store::SessionFile,
/// What a resumed session is not sent again - READY and the GUILD_CREATEs saved from the last run - to be handled
/// as though they had just arrived, before anything is read from the socket.
pending: std::collections::VecDeque<Value>,
}
/// Enough to pick a session back up: what it was called, where to reconnect,
@@ -183,20 +347,100 @@ pub(super) struct Resume {
seq: u64,
}
/// A heartbeat: its data is the number of the last dispatch received, and null before there has been one.
pub(super) fn heartbeat_message(seq: u64) -> String {
json!({ "op": 1, "d": if seq == 0 { Value::Null } else { Value::from(seq) } }).to_string()
}
/// The gateway capabilities this client declares in IDENTIFY, as a user client does (Dissent's `ningen` sends the same
/// six of the seven it knows): notes fetched lazily, read states and guild settings in their versioned form, user
/// objects deduplicated, guild experiments as populations, and read states for things that are not channels. Each
/// changes only the shape of a field READY carries and this client either ignores (`read_state`, notes, experiments),
/// reads in both shapes (`user_guild_settings`, see `mutes::note_settings`) or has put back (the deduplicated users,
/// see `ready_shape`).
///
/// Not declared: PRIORITIZED_READY_PAYLOAD (32). It splits what READY carries in two, the second part (presences, the
/// rest of each guild's members, who is in which voice channel) arriving as READY_SUPPLEMENTAL after the guilds are
/// registered, and every one of those is read at the moment of registering here. It buys the client a faster first
/// event; the identity it adds is the one bit.
pub(super) const CAPABILITIES: u32 = (1 << 0) | (1 << 2) | (1 << 3) | (1 << 4) | (1 << 6) | (1 << 7);
/// What IDENTIFY says this client is: moho, on Linux, in this machine's language.
fn identify_properties() -> Value {
let mut properties = json!({ "os": "linux", "browser": "nobilis", "device": "nobilis" });
if let Some(locale) = super::install_headers::locale() {
properties["system_locale"] = Value::from(locale);
}
properties
}
/// Conversations asked about this run, found or not: one that Discord will not describe is not asked about again for
/// every message in it.
static ASKED_ABOUT: std::sync::Mutex<Option<std::collections::HashSet<String>>> = std::sync::Mutex::new(None);
/// Looks up a direct message that READY did not list - the one request it costs is made when somebody speaks in it,
/// which is the moment it matters - and registers it like any other.
async fn unknown_dm(state: &AppState, config: &DiscordAccountConfig, channel_id: &str, channel_map: &mut HashMap<String, (String, String)>) {
let first = ASKED_ABOUT.lock().unwrap_or_else(|e| e.into_inner()).get_or_insert_with(Default::default).insert(format!("{}:{channel_id}", config.user_id));
if !first {
return;
}
let resp = http_client_for(&config.token).get(format!("{API_BASE}/channels/{channel_id}")).header("Authorization", &config.token).send_paced().await;
let Ok(resp) = resp else { return };
if !resp.status().is_success() {
return;
}
let Ok(channel) = resp.json::<Value>().await else { return };
// 1 is a direct message and 3 a group.
if matches!(channel["type"].as_i64(), Some(1) | Some(3)) {
register_dm_channel(state, &config.account_id(), &channel, channel_map, &HashMap::new());
}
}
pub(super) async fn run_gateway_with_retry(state: &AppState, config: &DiscordAccountConfig, account_id: &str) {
let mut delay = RECONNECT_INITIAL_DELAY;
let mut session = GatewaySession::default();
let mut session = GatewaySession {
persist: super::session_store::SessionFile::new(account_id, &config.user_id, &config.token),
..Default::default()
};
// A restart within a few minutes of the last session picks it up, as Discord's own client does, rather than
// logging in afresh: see session_store.
if let Some((position, replay)) = session.persist.load(&config.token, chrono::Utc::now().timestamp()) {
tracing::info!("discord[{account_id}]: resuming the session from before the restart");
session.resume = Some(Resume { session_id: position.session_id, url: position.url, seq: position.seq });
session.pending = replay.into();
}
// A login Discord has already refused is not tried again: the person signs in afresh, which is a new token.
if token_is_halted(&config.token) {
state.runtime.set_conn_state(state, account_id, ConnState::AuthFailed, Some("Discord has refused this account's login - sign in again from Accounts"));
return;
}
state.runtime.set_conn_state(state, account_id, ConnState::Connecting, None);
loop {
// A connection that will log in afresh (not pick a session back up) is counted, and held when the account has
// made too many lately: see identify_budget.
if session.resume.is_none() {
hold_for_login_budget(state, account_id).await;
}
let result = std::panic::AssertUnwindSafe(run_gateway(state, config, &mut session)).catch_unwind().await;
let requested_reconnect =
matches!(&result, Ok(Err(e)) if e.downcast_ref::<ReconnectRequested>().is_some());
let rate_limited = matches!(&result, Ok(Err(e)) if e.downcast_ref::<GatewayRateLimited>().is_some());
// A session Discord has ended is not resumed, here or after a restart: the next connection is a login.
if matches!(&result, Ok(Err(e)) if e.downcast_ref::<SessionLost>().is_some()) {
session.resume = None;
session.persist.clear();
session.pending.clear();
session.channel_map.clear();
session.guild_context.clear();
}
let detail = match result {
Ok(Ok(())) => "gateway session ended".to_string(),
Ok(Err(e)) => {
if e.downcast_ref::<GatewayAuthFailed>().is_some() {
tracing::warn!("discord[{account_id}]: {e} - giving up, needs a fresh login");
// Every other request with it stops too: see http::Safety.
halt_token(&config.token);
state.runtime.set_conn_state(state, account_id, ConnState::AuthFailed, Some(&e.to_string()));
return;
}
@@ -229,13 +473,42 @@ pub(super) async fn run_gateway_with_retry(state: &AppState, config: &DiscordAcc
}
}
/// Waits until the account may make a fresh login, saying in the window why it is waiting. The person may let it
/// through sooner ("Connect anyway"); that login is still counted.
async fn hold_for_login_budget(state: &AppState, account_id: &str) {
loop {
match super::identify_budget::ask(account_id) {
Ok(()) => return,
Err(wait) => {
let detail = super::identify_budget::explain(wait);
tracing::warn!("discord[{account_id}]: {detail}");
state.runtime.set_conn_state(state, account_id, ConnState::Connecting, None);
state.runtime.report_retry(state, account_id, &detail, Duration::from_secs(wait.max(1) as u64));
let pass = super::identify_budget::begin_hold(account_id);
let let_through = tokio::select! {
_ = tokio::time::sleep(Duration::from_secs(wait.max(1) as u64)) => false,
_ = pass.notified() => true,
};
super::identify_budget::end_hold(account_id);
if let_through {
tracing::warn!("discord[{account_id}]: connecting past the login limit, as asked");
super::identify_budget::record(account_id);
return;
}
}
}
}
}
pub(super) async fn run_gateway(state: &AppState, config: &DiscordAccountConfig, session: &mut GatewaySession) -> Result<()> {
// Disjoint borrows, so the resume state can be updated while the maps
// are being passed around mutably.
let GatewaySession { resume, channel_map, guild_context } = session;
let GatewaySession { resume, channel_map, guild_context, persist, pending } = session;
// Discord asks that a resume go to the url it handed out with the
// session rather than to the front door.
let connect_url = resume.as_ref().map(|r| r.url.clone()).unwrap_or_else(|| GATEWAY_URL.to_string());
let connect_url = compressed(&resume.as_ref().map(|r| r.url.clone()).unwrap_or_else(|| GATEWAY_URL.to_string()));
// One stream for the whole connection, and a new one for the next: the compressor at the far end starts afresh.
let mut inflater = Inflater::new();
let (ws, _resp) = tokio::time::timeout(CONNECT_TIMEOUT, crate::net::route::websocket(&config.token, connect_url.as_str()))
.await
.map_err(|_| anyhow!("timed out connecting to Discord's gateway"))?
@@ -259,7 +532,7 @@ pub(super) async fn run_gateway(state: &AppState, config: &DiscordAccountConfig,
}
let _forwarder_guard = AbortOnDrop(forwarder);
let hello = next_json(&mut stream).await.context("waiting for gateway hello")?;
let hello = next_json_inflating(&mut stream, Some(&mut inflater)).await.context("waiting for gateway hello")?;
let heartbeat_interval = hello["d"]["heartbeat_interval"].as_u64().unwrap_or(41_250).max(1);
// Discord's own documented client contract: if an ack (op 11) hasn't
@@ -274,6 +547,10 @@ pub(super) async fn run_gateway(state: &AppState, config: &DiscordAccountConfig,
// false-positive as a missed ack.
let ack_pending = Arc::new(AtomicBool::new(false));
let stale_notify = Arc::new(Notify::new());
// The number of the last dispatch received, which a heartbeat says: it tells Discord how far this connection has
// got, as it does for every client. (A resume starts from where the saved session had got to.)
let last_seq = Arc::new(std::sync::atomic::AtomicU64::new(resume.as_ref().map(|r| r.seq).unwrap_or(0)));
let hb_seq = last_seq.clone();
let hb_tx = out_tx.clone();
let hb_ack_pending = ack_pending.clone();
let hb_stale_notify = stale_notify.clone();
@@ -286,7 +563,7 @@ pub(super) async fn run_gateway(state: &AppState, config: &DiscordAccountConfig,
hb_stale_notify.notify_one();
break;
}
if hb_tx.send(json!({ "op": 1, "d": Value::Null }).to_string()).is_err() {
if hb_tx.send(heartbeat_message(hb_seq.load(Ordering::SeqCst))).is_err() {
break;
}
}
@@ -299,6 +576,8 @@ pub(super) async fn run_gateway(state: &AppState, config: &DiscordAccountConfig,
// Exposed so setAccountStatus can push a presence update onto this
// connection rather than waiting for a reconnect to carry it.
let account_id = config.account_id();
// A connection starts with no subscriptions.
super::presence::forget_member_subscriptions(&account_id);
state.runtime.set_discord_gateway_sender(&account_id, out_tx.clone());
struct ClearSenderOnDrop<'a>(&'a AppState, String);
impl Drop for ClearSenderOnDrop<'_> {
@@ -319,7 +598,8 @@ pub(super) async fn run_gateway(state: &AppState, config: &DiscordAccountConfig,
"op": 2,
"d": {
"token": config.token,
"properties": { "os": "linux", "browser": "nobilis", "device": "nobilis" },
"properties": identify_properties(),
"capabilities": CAPABILITIES,
"compress": false,
"large_threshold": 50,
// Carried in IDENTIFY as well as pushed live, so a status set
@@ -347,17 +627,30 @@ pub(super) async fn run_gateway(state: &AppState, config: &DiscordAccountConfig,
let mut tally_since = std::time::Instant::now();
loop {
let msg = tokio::select! {
result = next_json(&mut stream) => result?,
_ = stale_notify.notified() => bail!("no heartbeat ack received - connection is zombied"),
// What a resume does not send comes first, from the last run; `replaying` says so to the handlers.
let (msg, replaying) = match pending.front().cloned() {
Some(saved) => {
pending.pop_front();
(saved, true)
}
None => {
let msg = tokio::select! {
result = next_json_inflating(&mut stream, Some(&mut inflater)) => result?,
_ = stale_notify.notified() => bail!("no heartbeat ack received - connection is zombied"),
};
(msg, false)
}
};
let op = msg.get("op").and_then(|v| v.as_i64()).unwrap_or(-1);
// Every dispatch is numbered, and a resume says how far it got.
// Tracked before the match so it counts events this client chose
// not to handle - the server's count includes them either way, and
// resuming from a lower number would replay what was already seen.
if let (Some(seq), Some(r)) = (msg.get("s").and_then(|v| v.as_u64()), resume.as_mut()) {
r.seq = seq;
if let Some(seq) = msg.get("s").and_then(|v| v.as_u64()) {
last_seq.store(seq, Ordering::SeqCst);
if let Some(r) = resume.as_mut() {
r.seq = seq;
}
}
match op {
0 => {
@@ -386,6 +679,15 @@ pub(super) async fn run_gateway(state: &AppState, config: &DiscordAccountConfig,
tracing::info!("discord[{account_id}]: resumed");
}
"READY" => {
// A new login has missed what the last one did not: what was asked about is asked afresh.
if !replaying {
super::history::forget_asked(&account_id);
}
// Kept as it came (it is what a resume replays), and read with its users put back: see
// ready_shape. A READY that was not dehydrated is read as it is.
let raw = d;
let hydrated = super::ready_shape::rehydrate(raw);
let d = hydrated.as_ref().unwrap_or(raw);
let username = d["user"]["global_name"]
.as_str()
.filter(|s| !s.is_empty())
@@ -393,6 +695,8 @@ pub(super) async fn run_gateway(state: &AppState, config: &DiscordAccountConfig,
.unwrap_or("me")
.to_string();
state.runtime.set_own_identity(&account_id, &username);
// What the account can say of itself to a server's rules: a verified email or phone, how old.
super::send_guard::note_account(&account_id, &d["user"]);
// Nitro, which is what lets this account's emoji be
// used away from the guild they belong to. Read here
// because READY is the only place the payload carries
@@ -411,14 +715,19 @@ pub(super) async fn run_gateway(state: &AppState, config: &DiscordAccountConfig,
// Also what an interaction has to name - see
// Runtime::set_discord_gateway_session.
state.runtime.set_discord_gateway_session(&account_id, session_id);
*resume = Some(Resume {
let next = Resume {
session_id: session_id.to_string(),
url: d["resume_gateway_url"]
.as_str()
.map(|u| format!("{u}/?v=10&encoding=json"))
.map(|u| format!("{u}/?v=10&encoding=json&compress=zlib-stream"))
.unwrap_or_else(|| GATEWAY_URL.to_string()),
seq: msg.get("s").and_then(|v| v.as_u64()).unwrap_or(0),
});
};
// A new session replaces what was kept; the saved one handled again is that same one.
if !replaying {
persist.ready(&next.session_id, &next.url, next.seq, raw.clone());
}
*resume = Some(next);
}
if let Some(hash) = d["user"]["avatar"].as_str() {
let url = format!("https://cdn.discordapp.com/avatars/{}/{hash}.png", config.user_id);
@@ -475,8 +784,13 @@ pub(super) async fn run_gateway(state: &AppState, config: &DiscordAccountConfig,
}
let mut new_dm_buffers: Vec<(String, String)> = Vec::new();
// What Discord says each conversation's newest message
// is, so catching up asks only about the ones that
// have something newer than is stored.
let mut newest_in: HashMap<String, Option<String>> = HashMap::new();
if let Some(dms) = d["private_channels"].as_array() {
for ch in dms {
note_newest(&mut newest_in, ch);
if let Some(pair) = register_dm_channel(state, &account_id, ch, channel_map, &presences) {
new_dm_buffers.push(pair);
}
@@ -492,15 +806,11 @@ pub(super) async fn run_gateway(state: &AppState, config: &DiscordAccountConfig,
update_presence_in_rosters(state, user_id, p["status"].as_str().unwrap_or("offline"), Some(activity));
}
}
// Conversations are the ones people notice a gap in,
// and there are few enough of them to ask about
// directly. Channels wait until opened - a few hundred
// requests on every connect would be rate-limited and
// mostly wasted.
spawn_dm_catch_up(state.clone(), config.clone(), channel_map.clone());
state.runtime.set_conn_state(state, &account_id, ConnState::Connected, None);
tracing::info!("discord[{account_id}]: ready as {username}");
// Saved ready is not yet accepted by Discord: RESUMED says when it is.
if !replaying {
state.runtime.set_conn_state(state, &account_id, ConnState::Connected, None);
}
tracing::info!("discord[{account_id}]: ready as {username}{}", if replaying { " (from the last run)" } else { "" });
// For an account in relatively few guilds, READY's own
// `guilds` array already carries full guild objects
@@ -515,8 +825,11 @@ pub(super) async fn run_gateway(state: &AppState, config: &DiscordAccountConfig,
// harmlessly on a guild it's already seen via either
// path (channel_map dedup).
if let Some(guilds) = d["guilds"].as_array() {
for g in guilds {
register_guild_channels(state, config, g, channel_map).await;
// Our own membership of each, from READY's list of them, so that no guild is asked about.
let merged = d["merged_members"].as_array();
for (i, g) in guilds.iter().enumerate() {
let with_me = guild_with_own_member(g, merged.and_then(|m| m.get(i)), &config.user_id);
register_guild_channels(state, config, with_me.as_ref().unwrap_or(g), channel_map).await;
}
}
@@ -526,30 +839,67 @@ pub(super) async fn run_gateway(state: &AppState, config: &DiscordAccountConfig,
// traffic for channel ids that never appeared there) -
// fetch the full list over REST as a follow-up rather
// than trusting the gateway snapshot alone.
match http_client_for(&config.token).get(format!("{API_BASE}/users/@me/channels")).header("Authorization", &config.token).send().await {
Ok(resp) => {
let body = resp.text().await.unwrap_or_default();
match serde_json::from_str::<Vec<Value>>(&body) {
Ok(channels) => {
for ch in &channels {
if let Some(pair) = register_dm_channel(state, &account_id, ch, channel_map, &presences) {
new_dm_buffers.push(pair);
}
//
// A session picked up again after a restart already
// has that list, kept with the rest of it - asking
// again is a request for nothing.
let listed: Option<Vec<Value>> = if replaying {
d["moho_dms"].as_array().cloned()
} else if d["private_channels"].as_array().is_some_and(|l| !l.is_empty()) {
// READY names the account's conversations, as it does for Discord's own clients, and
// any it leaves out is found when somebody speaks in it (see `unknown_dm`): no request
// for a list that was just sent.
None
} else {
match http_client_for(&config.token).get(format!("{API_BASE}/users/@me/channels")).header("Authorization", &config.token).send_paced().await {
Ok(resp) => {
let body = resp.text().await.unwrap_or_default();
match serde_json::from_str::<Vec<Value>>(&body) {
Ok(channels) => Some(channels),
Err(e) => {
tracing::warn!("discord[{account_id}]: parsing /users/@me/channels: {e}");
None
}
}
Err(e) => tracing::warn!("discord[{account_id}]: parsing /users/@me/channels: {e}"),
}
Err(e) => {
tracing::warn!("discord[{account_id}]: fetching /users/@me/channels: {e}");
None
}
}
Err(e) => tracing::warn!("discord[{account_id}]: fetching /users/@me/channels: {e}"),
};
for ch in listed.iter().flatten() {
note_newest(&mut newest_in, ch);
if let Some(pair) = register_dm_channel(state, &account_id, ch, channel_map, &presences) {
new_dm_buffers.push(pair);
}
}
if !replaying {
persist.dms(listed.unwrap_or_default());
}
// The direct messages now exist, so the settings
// read at the top of READY have somewhere to land.
mutes::apply(state, &account_id);
spawn_backfill(state.clone(), config.token.clone(), config.user_id.clone(), config.display_name.clone(), new_dm_buffers);
// Conversations are the ones people notice a gap in,
// and there are few enough of them to ask about
// directly - and only the ones Discord says have
// moved on. Channels wait until opened: a few hundred
// requests on every connect would be rate-limited and
// mostly wasted.
// (A resumed session is replayed what it missed.)
if !replaying {
spawn_dm_catch_up(state.clone(), config.clone(), channel_map.clone(), newest_in);
}
// Nor is any conversation's history fetched here: it is read when it is opened (see getBacklog).
let _ = new_dm_buffers;
}
"GUILD_CREATE" => {
if let (false, Some(id)) = (replaying, d["id"].as_str()) {
persist.guild(id, d);
}
register_guild_channels(state, config, d, channel_map).await;
// The channels this guild's settings were about now
// exist. READY carried the mutes before any of them
@@ -640,6 +990,10 @@ pub(super) async fn run_gateway(state: &AppState, config: &DiscordAccountConfig,
if let Some(topic) = d.get("topic") {
state.runtime.set_buffer_topic(state, &crate::model::buffer_id(&account_id, &old_name), topic.as_str().unwrap_or_default());
}
// And so is a channel's slowmode.
if let Some(seconds) = d["rate_limit_per_user"].as_u64() {
super::send_guard::set_slowmode(&crate::model::buffer_id(&account_id, &old_name), seconds);
}
continue;
}
state.runtime.remove_buffer(state, &crate::model::buffer_id(&account_id, &old_name));
@@ -716,6 +1070,10 @@ pub(super) async fn run_gateway(state: &AppState, config: &DiscordAccountConfig,
// own state rather than needing to guess.
"MESSAGE_ACK" => {
let Some(channel_id) = d["channel_id"].as_str() else { continue };
// Remembered, so what is read here to the same point is not told to Discord a second time.
if let Some(message_id) = d["message_id"].as_str() {
super::send::acked_elsewhere(channel_id, message_id);
}
let Some((name, _)) = channel_map.get(channel_id) else { continue };
state.events.emit(
"bufferRead",
@@ -801,6 +1159,8 @@ pub(super) async fn run_gateway(state: &AppState, config: &DiscordAccountConfig,
if d["user"]["id"].as_str() != Some(config.user_id.as_str()) {
continue;
}
// What the server's rules ask of the account: pending, flags, roles, a timeout.
super::send_guard::note_member(state, &account_id, guild_id, Some(d));
if d["pending"].as_bool().unwrap_or(false) {
continue;
}
@@ -819,6 +1179,12 @@ pub(super) async fn run_gateway(state: &AppState, config: &DiscordAccountConfig,
}
"GUILD_DELETE" => {
if let Some(id) = d["id"].as_str() {
persist.guild_gone(id);
if d["unavailable"].as_bool() != Some(true) {
super::send_guard::forget_guild(&account_id, id);
}
}
let Some(guild_id) = d["id"].as_str() else { continue };
if d["unavailable"].as_bool() == Some(true) {
continue;
@@ -834,10 +1200,15 @@ pub(super) async fn run_gateway(state: &AppState, config: &DiscordAccountConfig,
}
"MESSAGE_CREATE" => {
let channel_id = d["channel_id"].as_str().unwrap_or_default();
// A conversation READY did not name: asked about, once, when it first speaks.
if !channel_map.contains_key(channel_id) && d["guild_id"].is_null() {
unknown_dm(state, config, channel_id, channel_map).await;
}
let Some((buffer_name, kind)) = channel_map.get(channel_id).cloned() else {
tracing::info!("discord[{account_id}]: a message in {channel_id} was dropped - that channel is not one this connection knows ({} are)", channel_map.len());
continue;
};
super::send_guard::heard(state, config, d, &model::buffer_id(&account_id, &buffer_name), kind == "dm");
let author = &d["author"];
let from = author["global_name"]
.as_str()
@@ -862,7 +1233,7 @@ pub(super) async fn run_gateway(state: &AppState, config: &DiscordAccountConfig,
let real_msg_id = d["id"].as_str().map(|s| s.to_string());
let avatar_url = author_avatar_url(author);
if author["system"].as_bool() == Some(true) && kind == "dm" {
state.runtime.set_buffer_read_only(state, &model::buffer_id(&account_id, &buffer_name), Some(super::people::OFFICIAL_ONLY));
super::send_guard::set_permission(state, &account_id, &model::buffer_id(&account_id, &buffer_name), Some(super::people::OFFICIAL_ONLY));
}
// Kept against their id as well as put on the message.
// A direct call's voice states carry no member object,
@@ -891,7 +1262,9 @@ pub(super) async fn run_gateway(state: &AppState, config: &DiscordAccountConfig,
// The author's id travels with the message. It is what a profile
// lookup asks Discord about - a display name is not something
// the API accepts - and what a moderation action would need.
state.runtime.record_message(state, &account_id, &buffer_name, &kind, &from, &body, false, "chat", reply_to, real_msg_id.clone(), is_mention, avatar_url, embeds, attachments, author["id"].as_str().map(str::to_string));
let badges = author_badges(author);
let style = if badges.is_empty() { None } else { Some(model::SenderStyle { color: None, badges }) };
state.runtime.record_message_at(state, &account_id, &buffer_name, &kind, &from, &body, false, "chat", reply_to, real_msg_id.clone(), is_mention, avatar_url, embeds, attachments, author["id"].as_str().map(str::to_string), None, None, style);
// Who wrote a forwarded message, which the snapshot
// does not say. A read of the original, so it happens
// after the message is already on screen rather than
@@ -1131,12 +1504,10 @@ pub(super) async fn run_gateway(state: &AppState, config: &DiscordAccountConfig,
mutes::note_settings(state, &account_id, d);
}
"GUILD_MEMBER_ADD" | "GUILD_MEMBER_REMOVE" => {
let Some(guild_id) = d["guild_id"].as_str() else { continue };
if let Some(buffer_id) = state.runtime.discord_member_list_target(&account_id, guild_id) {
request_member_list(state, &buffer_id);
}
}
// Not a reason to ask for the list again: the list is subscribed to, and Discord sends what changes in
// it. Asking at every join and leave of a busy server was a request for what was already coming.
"GUILD_MEMBER_ADD" | "GUILD_MEMBER_REMOVE" => {}
// A purge. Without this every message in it stayed on
// screen, since the per-message deletes are not sent.
@@ -1351,6 +1722,10 @@ pub(super) async fn run_gateway(state: &AppState, config: &DiscordAccountConfig,
}
}
}
// Discord asking for a heartbeat now: answered at once, as its own clients answer.
1 => {
let _ = out_tx.send(heartbeat_message(last_seq.load(Ordering::SeqCst)));
}
7 => return Err(ReconnectRequested.into()),
// The resume was refused, or the session is gone. Everything
// carried across goes with it: the next connection gets a fresh
@@ -1358,6 +1733,8 @@ pub(super) async fn run_gateway(state: &AppState, config: &DiscordAccountConfig,
// holding buffers for channels this account may no longer be in.
9 => {
*resume = None;
persist.clear();
pending.clear();
channel_map.clear();
guild_context.clear();
bail!("session invalidated by gateway")
@@ -1366,7 +1743,13 @@ pub(super) async fn run_gateway(state: &AppState, config: &DiscordAccountConfig,
// sending the *next* one, so a real ack landing between ticks
// is exactly what keeps this connection from ever being
// declared zombied in the first place.
11 => ack_pending.store(false, Ordering::SeqCst),
11 => {
ack_pending.store(false, Ordering::SeqCst);
// Alive as of now: what a restart soon enough will resume.
if let Some(r) = resume.as_ref() {
persist.acked(r.seq);
}
}
_ => {}
}
}
src/backend/discord/gifpicker.rs
+1 -1
View File
@@ -87,7 +87,7 @@ pub async fn list_gifs(state: &AppState, query: &str) -> Result<Value> {
let resp = http_client_for(&config.token)
.get(url)
.header("Authorization", &config.token)
.send()
.send_paced()
.await
.context("asking Discord for GIFs")?;
if !resp.status().is_success() {
src/backend/discord/guilds.rs
+126 -22
View File
@@ -96,7 +96,7 @@ pub async fn member_verification(state: &AppState, account_id: &str, guild_id: &
let resp = http_client_for(&cfg.token)
.get(format!("{API_BASE}/guilds/{guild_id}/member-verification"))
.header("Authorization", &cfg.token)
.send()
.send_paced()
.await
.context("asking Discord for this server's rules")?;
let status = resp.status();
@@ -107,6 +107,27 @@ pub async fn member_verification(state: &AppState, account_id: &str, guild_id: &
serde_json::from_str(&text).context("parsing this server's rules")
}
/// The form's fields with the answers filled in: the rules are agreed to, and every other field takes what the person
/// gave for it (their text, or the number of the choice), by its place in the form. A field nobody answered is sent
/// without a response, which Discord accepts only where the server did not require one.
pub fn responses_for(form: &Value, answers: &[Value]) -> Vec<Value> {
form["form_fields"]
.as_array()
.into_iter()
.flatten()
.enumerate()
.map(|(i, f)| {
let mut field = f.clone();
if f["field_type"] == "TERMS" {
field["response"] = Value::Bool(true);
} else if let Some(answer) = answers.get(i).filter(|a| !a.is_null()) {
field["response"] = answer.clone();
}
field
})
.collect()
}
/// Agrees to them, which is what lifts the gate.
///
/// The form is fetched again rather than taken from the caller: what is
@@ -114,20 +135,11 @@ pub async fn member_verification(state: &AppState, account_id: &str, guild_id: &
/// captured when the dialog opened may not still be it. Every field is
/// returned as sent with its response set - Discord rejects a form that has
/// been reshaped, and it is the server's form, not ours to edit.
pub async fn accept_member_verification(state: &AppState, account_id: &str, guild_id: &str) -> Result<()> {
pub async fn accept_member_verification(state: &AppState, account_id: &str, guild_id: &str, answers: &[Value]) -> Result<()> {
let cfg = state.accounts.get_discord(account_id).context("account not connected")?;
let form = member_verification(state, account_id, guild_id).await?;
let version = form["version"].clone();
let fields: Vec<Value> = form["form_fields"]
.as_array()
.into_iter()
.flatten()
.map(|f| {
let mut field = f.clone();
field["response"] = Value::Bool(true);
field
})
.collect();
let fields = responses_for(&form, answers);
let resp = send_write(
http_client_for(&cfg.token)
@@ -189,7 +201,7 @@ pub(super) async fn resync_guild_once(state: &AppState, config: &DiscordAccountC
http_client_for(&config.token)
.get(format!("{API_BASE}/{path}"))
.header("Authorization", &config.token)
.send()
.send_paced()
.await
.ok()?
.json::<Value>()
@@ -231,6 +243,32 @@ pub(super) async fn resync_guild_once(state: &AppState, config: &DiscordAccountC
}
}
/// This account's own member object out of READY's `merged_members`, which lists, for each guild in the order READY
/// names them, the account's own membership of it (with a `user_id` where a member elsewhere has a `user`). Handed to
/// `register_guild_channels` inside the guild's own payload, where it is looked for first, so that a guild whose
/// payload leaves the account out costs no request of its own - one request a guild, at every fresh login, was the
/// alternative.
pub(super) fn guild_with_own_member(guild: &Value, merged: Option<&Value>, user_id: &str) -> Option<Value> {
let has = |list: &Value| list.as_array().is_some_and(|l| l.iter().any(|m| m["user"]["id"].as_str() == Some(user_id)));
if has(&guild["members"]) {
return None;
}
let mut me = merged?
.as_array()?
.iter()
.find(|m| m["user_id"].as_str() == Some(user_id) || m["user"]["id"].as_str() == Some(user_id))?
.clone();
if me["user"]["id"].is_null() {
me["user"] = json!({ "id": user_id });
}
let mut guild = guild.clone();
match guild["members"].as_array_mut() {
Some(list) => list.push(me),
None => guild["members"] = json!([me]),
}
Some(guild)
}
/// This account's own membership of a guild.
///
/// From the payload when it is there, and asked for when it is not. The
@@ -257,7 +295,7 @@ pub(super) async fn own_member(config: &DiscordAccountConfig, guild: &Value) ->
let resp = http_client_for(&config.token)
.get(format!("{API_BASE}/users/@me/guilds/{guild_id}/member"))
.header("Authorization", &config.token)
.send()
.send_paced()
.await
.ok()?;
if !resp.status().is_success() {
@@ -353,6 +391,7 @@ pub(super) fn can_view_channel(is_owner: bool, guild_id: &str, roles: &[Value],
/// written by almost nobody - rules, announcements - and showing a message box
/// that every send is refused from is a worse answer than saying so.
#[allow(clippy::too_many_arguments)]
#[cfg(test)]
pub(super) fn can_send_in_channel(is_owner: bool, guild_id: &str, roles: &[Value], member_role_ids: &[String], user_id: &str, channel: &Value) -> bool {
channel_permissions(is_owner, guild_id, roles, member_role_ids, user_id, channel) & PERM_SEND_MESSAGES != 0
}
@@ -382,6 +421,11 @@ pub(super) async fn register_guild_channels(state: &AppState, config: &DiscordAc
// the way the roles-only version could.
let me = own_member(config, guild).await;
let own_roles = member_role_ids(me.as_ref());
// What the server asks of its members, and what the account is to it: see rules.
super::send_guard::note_guild(&config.account_id(), guild, &config.user_id);
super::send_guard::note_member(state, &config.account_id(), guild_id, me.as_ref());
// The account may not have spoken here: see send_guard. Begun before any channel is registered, so they open locked.
super::send_guard::open_guild(state, config, guild);
// Kept where every path that registers a guild passes, rather than in the
@@ -537,15 +581,20 @@ pub(super) async fn register_guild_channels(state: &AppState, config: &DiscordAc
// Said on the conversation, so the window can put a banner where the
// message box would be. Re-asked every time the guild is read, since
// permissions change under a channel that is already registered.
let read_only = (!can_send_in_channel(is_owner, guild_id, &roles, &member_role_ids, &config.user_id, ch)).then_some(NO_SEND_PERMISSION);
let permissions = channel_permissions(is_owner, guild_id, &roles, &member_role_ids, &config.user_id, ch);
let read_only = (permissions & PERM_SEND_MESSAGES == 0).then_some(NO_SEND_PERMISSION);
if channel_map.contains_key(channel_id) {
state.runtime.set_buffer_read_only(state, &crate::model::buffer_id(&account_id, &name), read_only);
let existing = crate::model::buffer_id(&account_id, &name);
super::send_guard::note_channel(state, &existing, ch, permissions, is_owner);
super::send_guard::set_permission(state, &account_id, &existing, read_only);
continue;
}
let buf = state.runtime.ensure_buffer(state, &account_id, &name, "channel");
state.runtime.set_buffer_read_only(state, &buf.id, read_only);
// The guild is known before the reason is worked out: the lock is the guild's.
state.runtime.set_discord_channel(state, &buf.id, channel_id);
state.runtime.set_discord_guild(&buf.id, guild_id);
super::send_guard::note_channel(state, &buf.id, ch, permissions, is_owner);
super::send_guard::set_permission(state, &account_id, &buf.id, read_only);
state.runtime.set_buffer_group(state, &buf.id, &group_id);
// Where Discord itself puts this channel. Uncategorised channels sit
@@ -692,7 +741,9 @@ pub(super) async fn register_guild_channels(state: &AppState, config: &DiscordAc
},
);
spawn_backfill(state.clone(), config.token.clone(), config.user_id.clone(), config.display_name.clone(), new_channels);
// No history is fetched for these: a channel is read when it is opened (see getBacklog), which is the only time
// anyone looks at it. Fetching the latest of every channel of every server when it is first seen was a request a
// channel, at the moment of a fresh login, for conversations nobody was about to read.
visible
}
@@ -770,7 +821,7 @@ pub async fn list_threads(state: &AppState, account_id: &str, buffer_id: &str) -
requests.push(format!("{API_BASE}/channels/{channel_id}/threads/archived/public?limit=25"));
for url in requests {
let resp = match http_client_for(&cfg.token).get(&url).header("Authorization", &cfg.token).send().await {
let resp = match http_client_for(&cfg.token).get(&url).header("Authorization", &cfg.token).send_paced().await {
Ok(resp) if resp.status().is_success() => resp,
// A channel with no archive, or one this account may not read the
// archive of, is not a failure worth refusing the whole list for.
@@ -941,7 +992,7 @@ pub async fn moderate_member(
if let Some(reason) = reason.filter(|r| !r.trim().is_empty()) {
request = request.header("X-Audit-Log-Reason", reason);
}
let resp = request.send().await.context(doing)?;
let resp = request.send_paced().await.context(doing)?;
if !resp.status().is_success() {
let status = resp.status();
let text = resp.text().await.unwrap_or_default();
@@ -965,7 +1016,7 @@ pub async fn set_member_role(
let http = http_client_for(&cfg.token);
let request = if give { http.put(url).json(&json!({})) } else { http.delete(url) };
let doing = if give { "giving them the role" } else { "taking the role away" };
let resp = request.header("Authorization", &cfg.token).send().await.context(doing)?;
let resp = request.header("Authorization", &cfg.token).send_paced().await.context(doing)?;
if !resp.status().is_success() {
let status = resp.status();
let text = resp.text().await.unwrap_or_default();
@@ -999,7 +1050,7 @@ pub async fn open_thread(state: &AppState, account_id: &str, buffer_id: &str, th
let resp = http
.get(format!("{API_BASE}/channels/{thread_id}"))
.header("Authorization", &cfg.token)
.send()
.send_paced()
.await
.context("opening the thread")?;
if !resp.status().is_success() {
@@ -1217,3 +1268,56 @@ mod system_dm_tests {
assert!(!is_system_dm(&json!({ "recipients": [{ "id": "1", "system": true }, { "id": "2" }] })));
}
}
#[cfg(test)]
mod verification_tests {
use super::*;
#[test]
fn the_rules_are_agreed_to_and_other_fields_take_the_persons_answers() {
let form = json!({
"version": "2026-01-01",
"form_fields": [
{ "field_type": "TERMS", "label": "Read and agree to the server rules", "values": ["Be kind"], "required": true },
{ "field_type": "TEXT_INPUT", "label": "Why are you joining?", "required": true },
{ "field_type": "MULTIPLE_CHOICE", "label": "Pick one", "values": ["a", "b"], "required": false },
]
});
let sent = responses_for(&form, &[Value::Null, json!("to talk"), json!(1)]);
assert_eq!(sent[0]["response"], true);
assert_eq!(sent[1]["response"], "to talk");
assert_eq!(sent[2]["response"], 1);
assert_eq!(sent[0]["values"], json!(["Be kind"]), "the server's own form is not reshaped");
// Nothing answered: no response key at all, not a made-up one.
let bare = responses_for(&form, &[]);
assert!(bare[1].get("response").is_none() && bare[2].get("response").is_none());
assert_eq!(bare[0]["response"], true);
}
}
#[cfg(test)]
mod merged_member_tests {
use super::*;
#[test]
fn a_guild_that_leaves_the_account_out_is_given_the_membership_ready_listed() {
let guild = json!({ "id": "g", "members": [{ "user": { "id": "other" } }] });
let merged = json!([{ "user_id": "me", "roles": ["r1"], "pending": true }, { "user_id": "x" }]);
let with = guild_with_own_member(&guild, Some(&merged), "me").expect("added");
let mine = with["members"].as_array().unwrap().iter().find(|m| m["user"]["id"] == "me").expect("the member is there");
assert_eq!(mine["roles"], json!(["r1"]));
assert!(member_is_pending(Some(mine)), "what the account's membership says survives");
assert_eq!(with["members"].as_array().unwrap().len(), 2, "the others stay");
// And it is what `own_member` would find first, without a request.
assert!(guild_with_own_member(&with, Some(&merged), "me").is_none());
}
#[test]
fn nothing_is_made_up_when_there_is_nothing_to_add() {
let guild = json!({ "id": "g" });
assert!(guild_with_own_member(&guild, None, "me").is_none());
assert!(guild_with_own_member(&guild, Some(&json!([{ "user_id": "someone-else" }])), "me").is_none());
let with = guild_with_own_member(&guild, Some(&json!([{ "user_id": "me" }])), "me").unwrap();
assert_eq!(with["members"][0]["user"]["id"], "me");
}
}
src/backend/discord/history.rs
+217 -28
View File
@@ -28,26 +28,20 @@ impl Drop for HistoryFetch<'_> {
}
}
/// Fires off history backfill for a batch of newly-registered buffers as a
/// background task, sequentially with a small delay between requests - a
/// guild can easily have 30+ channels (confirmed live), and firing that
/// many REST requests at once risks Discord's rate limiter; a one-time
/// startup cost taking a few extra seconds in the background is a fine
/// trade for not tripping it.
pub(super) fn spawn_backfill(state: AppState, token: String, user_id: String, display_name: Option<String>, targets: Vec<(String, String)>) {
if targets.is_empty() {
return;
}
tokio::spawn(async move {
for (buffer_id, channel_id) in targets {
backfill_channel_history(&state, &token, &user_id, display_name.as_deref(), &buffer_id, &channel_id).await;
tokio::time::sleep(Duration::from_millis(250)).await;
}
});
/// How many messages a conversation is first opened with. Few: the rest is a scroll away (`extend_history`), and a
/// client that asks for fifty of every conversation it is shown is asking for what nobody reads.
const FIRST_PAGE: &str = "30";
/// The time a message was sent, from its id: a snowflake's top bits are milliseconds since Discord's epoch. How a
/// conversation's last activity is known without fetching anything of it.
pub(super) fn snowflake_secs(id: &str) -> Option<i64> {
const DISCORD_EPOCH_MS: u64 = 1_420_070_400_000;
let id: u64 = id.parse().ok()?;
Some(((id >> 22) + DISCORD_EPOCH_MS) as i64 / 1000)
}
/// Backfills up to 50 recent messages for a channel/DM with no scrollback
/// yet. Unlike IRC (which has no server-side history at all - nobilis only
/// Backfills the most recent messages for a channel/DM with no scrollback
/// yet - when it is first opened, and not before. Unlike IRC (which has no server-side history at all - nobilis only
/// ever knows what it's personally seen live), Discord actually retains
/// and exposes history, so starting every buffer blank here would look
/// broken compared to Discord's own client. Seed-once, not sync: a buffer
@@ -61,7 +55,7 @@ pub(super) fn spawn_backfill(state: AppState, token: String, user_id: String, di
/// "notification" event, neither of which should fire for messages from
/// potentially weeks ago just because this is the first time nobilis has
/// seen the channel.
pub(super) async fn backfill_channel_history(state: &AppState, token: &str, user_id: &str, own_display_name: Option<&str>, buffer_id: &str, channel_id: &str) {
pub async fn backfill_channel_history(state: &AppState, token: &str, user_id: &str, own_display_name: Option<&str>, buffer_id: &str, channel_id: &str) {
match state.store.has_messages(buffer_id) {
Ok(true) => return,
Ok(false) => {}
@@ -73,9 +67,9 @@ pub(super) async fn backfill_channel_history(state: &AppState, token: &str, user
let resp = match http_client_for(token)
.get(format!("{API_BASE}/channels/{channel_id}/messages"))
.query(&[("limit", "50")])
.query(&[("limit", FIRST_PAGE)])
.header("Authorization", token)
.send()
.send_paced()
.await
{
Ok(r) => r,
@@ -127,7 +121,7 @@ pub async fn refill_history(state: &AppState, buffer_id: &str, limit: u32) -> Re
.get(format!("{API_BASE}/channels/{channel_id}/messages"))
.query(&[("limit", limit.clamp(1, 100).to_string().as_str())])
.header("Authorization", &config.token)
.send()
.send_paced()
.await
.context("re-reading history")?;
if resp.status() == reqwest::StatusCode::TOO_MANY_REQUESTS {
@@ -164,12 +158,99 @@ pub async fn refill_history(state: &AppState, buffer_id: &str, limit: u32) -> Re
Ok(missing.len())
}
/// Asks every open conversation what it missed, one at a time.
/// What has been asked of Discord about each conversation's history this connection, so it is not asked again for what
/// it has already said. A conversation is caught up once in a while, not at every opening of it (its messages arrive
/// live); older history is asked for until its start is reached and never after; a failed ask waits before the next.
#[derive(Default)]
pub struct Asked {
caught_up: HashMap<String, std::time::Instant>,
exhausted: std::collections::HashSet<String>,
failed: HashMap<String, std::time::Instant>,
}
/// How long after a catch-up another is not made for the same conversation.
const CATCH_UP_EVERY: Duration = Duration::from_secs(10 * 60);
/// How long after a failed request for older history the next one waits.
const FAILED_WAIT: Duration = Duration::from_secs(60);
impl Asked {
pub fn catch_up_due(&mut self, buffer: &str, now: std::time::Instant) -> bool {
if self.caught_up.get(buffer).is_some_and(|at| now.duration_since(*at) < CATCH_UP_EVERY) {
return false;
}
self.caught_up.insert(buffer.to_string(), now);
true
}
/// The catch-up did not come to anything: the next opening asks again.
pub fn catch_up_failed(&mut self, buffer: &str) {
self.caught_up.remove(buffer);
}
pub fn older_due(&mut self, buffer: &str, now: std::time::Instant) -> bool {
!self.exhausted.contains(buffer) && !self.failed.get(buffer).is_some_and(|at| now.duration_since(*at) < FAILED_WAIT)
}
/// Asked for `limit` and got `got`: fewer than asked for is the start of the conversation.
pub fn older_answered(&mut self, buffer: &str, got: usize, limit: usize) {
self.failed.remove(buffer);
if got < limit {
self.exhausted.insert(buffer.to_string());
}
}
pub fn older_failed(&mut self, buffer: &str, now: std::time::Instant) {
self.failed.insert(buffer.to_string(), now);
}
/// A new login has missed what the last one did not: everything of the account's is asked about afresh.
pub fn forget_account(&mut self, account: &str) {
let prefix = format!("{account}|");
self.caught_up.retain(|b, _| !b.starts_with(&prefix));
self.failed.retain(|b, _| !b.starts_with(&prefix));
}
}
static ASKED: std::sync::Mutex<Option<Asked>> = std::sync::Mutex::new(None);
fn asked<R>(f: impl FnOnce(&mut Asked) -> R) -> R {
f(ASKED.lock().unwrap_or_else(|e| e.into_inner()).get_or_insert_with(Asked::default))
}
pub(super) fn forget_asked(account: &str) {
asked(|a| a.forget_account(account));
}
/// Notes what a channel object says its newest message is: `None` for a
/// conversation with no messages, nothing at all when it does not say.
pub(super) fn note_newest(into: &mut HashMap<String, Option<String>>, channel: &Value) {
let (Some(id), Some(last)) = (channel["id"].as_str(), channel.get("last_message_id")) else { return };
into.insert(id.to_string(), last.as_str().map(String::from));
}
/// Whether Discord's newest message is newer than the newest stored. Ids are
/// snowflakes, which grow with time; one that will not parse is taken as
/// newer, since asking is the safe answer.
fn has_newer(theirs: Option<&str>, ours: Option<&str>) -> bool {
let Some(theirs) = theirs else { return false };
let Some(ours) = ours else { return true };
match (theirs.parse::<u64>(), ours.parse::<u64>()) {
(Ok(t), Ok(o)) => t > o,
_ => true,
}
}
/// Asks the open conversations that moved on what they missed, one at a time.
///
/// Spaced out deliberately: this runs right after connecting, alongside
/// everything else a fresh connection does, and a burst of requests is the
/// quickest way to be rate-limited by a service that has just let us in.
pub(super) fn spawn_dm_catch_up(state: AppState, config: DiscordAccountConfig, channels: HashMap<String, (String, String)>) {
pub(super) fn spawn_dm_catch_up(
state: AppState,
config: DiscordAccountConfig,
channels: HashMap<String, (String, String)>,
newest_in: HashMap<String, Option<String>>,
) {
tokio::spawn(async move {
let account_id = config.account_id();
let dms: Vec<(String, String)> = channels
@@ -177,6 +258,14 @@ pub(super) fn spawn_dm_catch_up(state: AppState, config: DiscordAccountConfig, c
.filter(|(_, (_, kind))| kind == "dm")
.filter_map(|(channel_id, (name, _))| {
let buffer = state.runtime.list_buffers().into_iter().find(|b| b.account_id == account_id && &b.name == name)?;
// Discord's word for the newest message, when it gave one:
// nothing newer than what is stored means nothing missed.
if let Some(theirs) = newest_in.get(channel_id) {
let ours = state.store.newest_msg_id(&buffer.id).ok().flatten();
if !has_newer(theirs.as_deref(), ours.as_deref()) {
return None;
}
}
Some((buffer.id, channel_id.clone()))
})
.collect();
@@ -206,8 +295,12 @@ pub async fn catch_up_channel(
buffer_id: &str,
channel_id: &str,
) {
if !asked(|a| a.catch_up_due(buffer_id, std::time::Instant::now())) {
return;
}
let Some(_fetch) = HistoryFetch::begin(state, buffer_id) else {
tracing::info!("discord: catching up {buffer_id} skipped - a fetch for it is already under way");
asked(|a| a.catch_up_failed(buffer_id));
return;
};
let result: Result<()> = async {
@@ -218,7 +311,7 @@ pub async fn catch_up_channel(
.get(format!("{API_BASE}/channels/{channel_id}/messages"))
.query(&[("limit", "50"), ("after", after_id.as_str())])
.header("Authorization", token)
.send()
.send_paced()
.await
.context("fetching missed messages")?;
if resp.status().is_success() {
@@ -229,16 +322,22 @@ pub async fn catch_up_channel(
store_history_messages(state, buffer_id, &messages, user_id, own_display_name);
} else {
tracing::info!("discord: catching up {buffer_id}: Discord answered {}", resp.status());
asked(|a| a.catch_up_failed(buffer_id));
}
Ok(())
}
.await;
if let Err(e) = result {
tracing::warn!("discord: catching up {buffer_id}: {e}");
asked(|a| a.catch_up_failed(buffer_id));
}
}
pub async fn extend_history(state: &AppState, token: &str, user_id: &str, own_display_name: Option<&str>, buffer_id: &str, channel_id: &str) {
// The start of a conversation is asked for once: after it is reached, nothing is older.
if !asked(|a| a.older_due(buffer_id, std::time::Instant::now())) {
return;
}
let Some(_fetch) = HistoryFetch::begin(state, buffer_id) else { return };
let result: Result<()> = async {
let Some(before_id) = state.store.oldest_msg_id(buffer_id)? else { return Ok(()) };
@@ -246,18 +345,22 @@ pub async fn extend_history(state: &AppState, token: &str, user_id: &str, own_di
.get(format!("{API_BASE}/channels/{channel_id}/messages"))
.query(&[("limit", "50"), ("before", before_id.as_str())])
.header("Authorization", token)
.send()
.send_paced()
.await
.context("fetching more history")?;
if resp.status().is_success() {
let messages: Vec<Value> = resp.json().await.context("parsing more history")?;
asked(|a| a.older_answered(buffer_id, messages.len(), 50));
store_history_messages(state, buffer_id, &messages, user_id, own_display_name);
} else {
asked(|a| a.older_failed(buffer_id, std::time::Instant::now()));
}
Ok(())
}
.await;
if let Err(e) = result {
tracing::warn!("discord: extending history for {buffer_id}: {e}");
asked(|a| a.older_failed(buffer_id, std::time::Instant::now()));
}
}
@@ -279,7 +382,7 @@ pub async fn load_newer(state: &AppState, account_id: &str, buffer_id: &str, mes
.get(format!("{API_BASE}/channels/{channel_id}/messages"))
.query(&[("limit", "50"), ("after", message_id)])
.header("Authorization", &cfg.token)
.send()
.send_paced()
.await
.context("reading the rest of the channel")?;
if !resp.status().is_success() {
@@ -299,7 +402,7 @@ pub async fn load_context(state: &AppState, account_id: &str, buffer_id: &str, m
.get(format!("{API_BASE}/channels/{channel_id}/messages"))
.query(&[("limit", "50"), ("around", message_id)])
.header("Authorization", &cfg.token)
.send()
.send_paced()
.await
.context("reading that part of the channel")?;
if !resp.status().is_success() {
@@ -318,3 +421,89 @@ pub async fn load_context(state: &AppState, account_id: &str, buffer_id: &str, m
.map(|dt| dt.timestamp())
.context("Discord did not return that message")
}
#[cfg(test)]
mod catch_up_tests {
use super::*;
use serde_json::json;
/// Discord's own documentation's example: 175928847299117063 was made at 2016-04-30 11:18:25.796 UTC.
#[test]
fn the_time_of_a_message_is_read_from_its_id() {
assert_eq!(snowflake_secs("175928847299117063"), Some(1_462_015_105));
assert_eq!(snowflake_secs("not a number"), None);
// What `search::snowflake_at` makes of a time reads back as that time.
assert_eq!(snowflake_secs(&super::search::snowflake_at(1_700_000_000)), Some(1_700_000_000));
}
#[test]
fn a_conversation_is_asked_about_only_when_discord_has_a_newer_message() {
assert!(has_newer(Some("200"), Some("100")));
assert!(!has_newer(Some("100"), Some("100")));
assert!(!has_newer(Some("99"), Some("100")));
// Nothing stored, but something said: a first sight is the backfill's, yet asking is harmless.
assert!(has_newer(Some("5"), None));
// An empty conversation has nothing to miss.
assert!(!has_newer(None, Some("100")));
// Ids that are not numbers are not compared.
assert!(has_newer(Some("abc"), Some("100")));
// Snowflakes outgrow 32 bits.
assert!(has_newer(Some("1500000000000000001"), Some("1500000000000000000")));
}
#[test]
fn what_a_channel_says_of_its_newest_message_is_noted_as_it_says_it() {
let mut into = HashMap::new();
note_newest(&mut into, &json!({ "id": "1", "last_message_id": "10" }));
note_newest(&mut into, &json!({ "id": "2", "last_message_id": null }));
note_newest(&mut into, &json!({ "id": "3" }));
assert_eq!(into.get("1"), Some(&Some("10".to_string())));
assert_eq!(into.get("2"), Some(&None));
assert!(!into.contains_key("3"));
}
}
#[cfg(test)]
mod asked_tests {
use super::*;
use std::time::Instant;
#[test]
fn a_conversation_is_caught_up_once_in_a_while_and_again_after_a_failure() {
let mut a = Asked::default();
let t = Instant::now();
assert!(a.catch_up_due("b", t));
assert!(!a.catch_up_due("b", t + Duration::from_secs(60)), "opened again a minute later");
assert!(a.catch_up_due("other", t));
assert!(a.catch_up_due("b", t + Duration::from_secs(601)), "ten minutes on");
a.catch_up_failed("b");
assert!(a.catch_up_due("b", t + Duration::from_secs(602)), "a failed one is asked again");
}
#[test]
fn older_history_is_asked_for_until_the_start_is_reached_and_not_after() {
let mut a = Asked::default();
let t = Instant::now();
assert!(a.older_due("b", t));
a.older_answered("b", 50, 50);
assert!(a.older_due("b", t), "a full page means more");
a.older_answered("b", 12, 50);
assert!(!a.older_due("b", t + Duration::from_secs(9999)), "the start was reached");
assert!(a.older_due("c", t));
a.older_failed("c", t);
assert!(!a.older_due("c", t + Duration::from_secs(30)));
assert!(a.older_due("c", t + Duration::from_secs(61)));
}
#[test]
fn a_new_login_asks_about_its_own_conversations_afresh() {
let mut a = Asked::default();
let t = Instant::now();
a.catch_up_due("discord:1|chat", t);
a.catch_up_due("discord:2|chat", t);
a.forget_account("discord:1");
assert!(a.catch_up_due("discord:1|chat", t));
assert!(!a.catch_up_due("discord:2|chat", t));
}
}
src/backend/discord/http.rs
+332 -8
View File
@@ -54,21 +54,249 @@ async fn pace() {
}
}
/// Where every read waits its turn: when Discord has said how long to hold off - a 429, or a bucket that ran out - no
/// request of any kind goes out before then.
static HOLD_UNTIL: std::sync::Mutex<Option<std::time::Instant>> = std::sync::Mutex::new(None);
/// What is added to a bucket's reset time, as in arikawa's limiter: Discord's own clock and ours do not agree to the
/// millisecond, and arriving on the boundary is arriving early.
const BUCKET_SLACK: Duration = Duration::from_millis(250);
/// How many times a read that was rate limited is made again, waiting as long as it was told to.
const READ_RETRIES: usize = 2;
fn hold_for(wait: Duration) {
let until = std::time::Instant::now() + wait;
let mut held = HOLD_UNTIL.lock().unwrap_or_else(|e| e.into_inner());
if held.is_none_or(|at| at < until) {
*held = Some(until);
}
}
async fn wait_out_hold() {
loop {
let wait = {
let held = HOLD_UNTIL.lock().unwrap_or_else(|e| e.into_inner());
held.map(|at| at.saturating_duration_since(std::time::Instant::now())).unwrap_or_default()
};
if wait.is_zero() {
return;
}
tokio::time::sleep(wait).await;
}
}
/// Notes what a response says about the bucket it counted against: with none left, nothing goes out until it resets.
fn note_bucket(resp: &reqwest::Response) {
let header = |name: &str| resp.headers().get(name).and_then(|v| v.to_str().ok());
if header("x-ratelimit-remaining") == Some("0") {
if let Some(reset) = header("x-ratelimit-reset-after").and_then(|v| v.parse::<f64>().ok()) {
hold_for(Duration::from_secs_f64(reset.clamp(0.0, 30.0)) + BUCKET_SLACK);
}
}
}
/// What this client has learned about where it is no longer welcome, and so does not ask again: a token Discord has
/// refused (every request stops until the person signs in again - a revoked login that keeps asking is a client
/// hammering a dead door), and a route that keeps answering 403 (three times in five minutes closes it for five, so a
/// channel that cannot be read is not asked about at every opening).
#[derive(Default)]
pub struct Safety {
halted: std::collections::HashSet<String>,
circuits: HashMap<(String, String, String), Circuit>,
}
struct Circuit {
refusals: u8,
last: std::time::Instant,
open_until: Option<std::time::Instant>,
}
/// Why a request was not made.
#[derive(Debug, PartialEq, Eq)]
pub enum Blocked {
/// The token was refused (401): nothing more is sent with it.
Refused,
/// The route was refused (403) repeatedly: not for this long.
Closed(Duration),
}
impl std::fmt::Display for Blocked {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Blocked::Refused => write!(f, "Discord has refused this account's login, so nothing more is sent for it - sign in again from Accounts"),
Blocked::Closed(wait) => write!(f, "Discord has refused this three times; not asking again for {} minutes", wait.as_secs().div_ceil(60).max(1)),
}
}
}
impl std::error::Error for Blocked {}
const CIRCUIT_REFUSALS: u8 = 3;
const CIRCUIT_WINDOW: Duration = Duration::from_secs(5 * 60);
const CIRCUIT_OPEN_FOR: Duration = Duration::from_secs(5 * 60);
/// Routes remembered: they include ids, so a few refusals on many messages must not be kept all run.
const MAX_CIRCUITS: usize = 512;
impl Safety {
pub fn preflight_at(&mut self, token: &str, method: &str, path: &str, now: std::time::Instant) -> Result<(), Blocked> {
if self.halted.contains(token) {
return Err(Blocked::Refused);
}
self.circuits.retain(|_, c| match c.open_until {
Some(until) => until > now,
None => now.saturating_duration_since(c.last) < CIRCUIT_WINDOW,
});
let key = (token.to_string(), method.to_string(), path.to_string());
match self.circuits.get(&key).and_then(|c| c.open_until) {
Some(until) => Err(Blocked::Closed(until.saturating_duration_since(now))),
None => Ok(()),
}
}
pub fn record_at(&mut self, token: &str, method: &str, path: &str, status: u16, now: std::time::Instant) {
let key = (token.to_string(), method.to_string(), path.to_string());
if status == 401 {
self.halted.insert(token.to_string());
return;
}
if status != 403 {
self.circuits.remove(&key);
return;
}
if !self.circuits.contains_key(&key) && self.circuits.len() >= MAX_CIRCUITS {
if let Some(oldest) = self.circuits.iter().min_by_key(|(_, c)| c.last).map(|(k, _)| k.clone()) {
self.circuits.remove(&oldest);
}
}
let circuit = self.circuits.entry(key).or_insert(Circuit { refusals: 0, last: now, open_until: None });
if now.saturating_duration_since(circuit.last) >= CIRCUIT_WINDOW {
circuit.refusals = 0;
circuit.open_until = None;
}
circuit.refusals = circuit.refusals.saturating_add(1);
circuit.last = now;
if circuit.refusals >= CIRCUIT_REFUSALS {
circuit.open_until = Some(now + CIRCUIT_OPEN_FOR);
}
}
pub fn halt(&mut self, token: &str) {
self.halted.insert(token.to_string());
}
pub fn is_halted(&self, token: &str) -> bool {
self.halted.contains(token)
}
}
static SAFETY: std::sync::Mutex<Option<Safety>> = std::sync::Mutex::new(None);
fn safety<R>(f: impl FnOnce(&mut Safety) -> R) -> R {
let mut slot = SAFETY.lock().unwrap_or_else(|e| e.into_inner());
f(slot.get_or_insert_with(Safety::default))
}
/// Nothing more is sent with this token: Discord has said it is no good.
pub(super) fn halt_token(token: &str) {
safety(|s| s.halt(token));
}
pub(super) fn token_is_halted(token: &str) -> bool {
safety(|s| s.is_halted(token))
}
/// Which token, method and path a request is for, where it can be told without sending it.
fn inspect(request: &reqwest::RequestBuilder) -> Option<(String, String, String)> {
let built = request.try_clone()?.build().ok()?;
let token = built.headers().get("authorization")?.to_str().ok()?.to_string();
Some((token, built.method().as_str().to_string(), built.url().path().to_string()))
}
/// Asked before a request is sent, told after: see `Safety`.
fn preflight(who: &Option<(String, String, String)>) -> Result<()> {
if let Some((token, method, path)) = who {
safety(|s| s.preflight_at(token, method, path, std::time::Instant::now())).map_err(anyhow::Error::from)?;
}
Ok(())
}
fn note_status(who: &Option<(String, String, String)>, status: reqwest::StatusCode) {
if let Some((token, method, path)) = who {
if status == reqwest::StatusCode::UNAUTHORIZED {
tracing::warn!("discord: a request was refused as unauthorised: nothing more is sent with this login");
}
safety(|s| s.record_at(token, method, path, status.as_u16(), std::time::Instant::now()));
}
}
/// Whether a 429's body is a channel's slowmode (code 20016): the wait is the person's, and sending again when it is
/// over is not for this client to do on their behalf.
fn is_slowmode(body: &[u8]) -> bool {
serde_json::from_slice::<Value>(body).ok().and_then(|v| v["code"].as_u64()) == Some(20016)
}
/// A response put back together once its body has been read, so a caller that reads it again finds it whole.
fn rebuilt(status: reqwest::StatusCode, headers: reqwest::header::HeaderMap, body: bytes::Bytes) -> reqwest::Response {
let mut response = tokio_tungstenite::tungstenite::http::Response::new(body);
*response.status_mut() = status;
*response.headers_mut() = headers;
reqwest::Response::from(response)
}
/// Sending that behaves as a client of an API with rate limits should: waits out any hold Discord has put on this
/// client, notes how much of a bucket is left and waits for it to refill rather than running into the limit, and when
/// it is told to slow down anyway waits as long as it was told and asks again - instead of reading a refusal as an
/// answer. Reads were sent plain, so a 429 was a silent "nothing found" and the next request followed straight on.
pub(super) trait SendPaced {
fn send_paced(self) -> impl std::future::Future<Output = Result<reqwest::Response>> + Send;
}
impl SendPaced for reqwest::RequestBuilder {
async fn send_paced(self) -> Result<reqwest::Response> {
let who = inspect(&self);
preflight(&who)?;
wait_out_hold().await;
let mut pending = Some(self);
for attempt in 0..=READ_RETRIES {
let Some(current) = pending.take() else { break };
// A body that cannot be copied is sent once.
let again = current.try_clone();
let resp = current.send().await.context("talking to Discord")?;
note_bucket(&resp);
note_status(&who, resp.status());
if resp.status() != reqwest::StatusCode::TOO_MANY_REQUESTS || attempt == READ_RETRIES {
return Ok(resp);
}
let Some(again) = again else { return Ok(resp) };
let pause = retry_after(&resp);
tracing::debug!("discord: rate limited on a read, waiting {}ms", pause.as_millis());
hold_for(pause);
wait_out_hold().await;
pending = Some(again);
}
unreachable!("the last attempt returns")
}
}
/// Like `send_write`, for a request whose body cannot be copied - a file
/// streamed from disk - and so is built again for every try.
pub(super) async fn send_write_rebuilt<F>(mut build: F) -> Result<reqwest::Response>
where
F: FnMut() -> Result<reqwest::RequestBuilder>,
{
wait_out_hold().await;
pace().await;
for attempt in 0..=RATE_LIMIT_RETRIES {
let resp = build()?.send().await.context("talking to Discord")?;
note_bucket(&resp);
if resp.status() != reqwest::StatusCode::TOO_MANY_REQUESTS || attempt == RATE_LIMIT_RETRIES {
return Ok(resp);
}
let pause = retry_after(&resp);
tracing::debug!("discord: rate limited, waiting {}ms", pause.as_millis());
tokio::time::sleep(pause).await;
hold_for(pause);
wait_out_hold().await;
}
bail!("Discord kept rate-limiting that request")
}
@@ -85,6 +313,9 @@ where
/// history quickly would be a slow client, and what Discord judges an account
/// on is what it sends.
pub(super) async fn send_write(request: reqwest::RequestBuilder) -> Result<reqwest::Response> {
let who = inspect(&request);
preflight(&who)?;
wait_out_hold().await;
pace().await;
let mut pending = Some(request);
@@ -94,13 +325,23 @@ pub(super) async fn send_write(request: reqwest::RequestBuilder) -> Result<reqwe
// that cannot be cloned is one this cannot retry, and is sent once.
let again = current.try_clone();
let resp = current.send().await.context("talking to Discord")?;
note_bucket(&resp);
note_status(&who, resp.status());
if resp.status() != reqwest::StatusCode::TOO_MANY_REQUESTS || attempt == RATE_LIMIT_RETRIES {
return Ok(resp);
}
let Some(again) = again else { return Ok(resp) };
let pause = retry_after(&resp);
// A channel's slowmode is answered with a 429 too. It is the person's wait, and a message they have not seen
// refused is not one to send again for them when it ends.
let (status, headers) = (resp.status(), resp.headers().clone());
let body = resp.bytes().await.context("reading Discord's answer")?;
if is_slowmode(&body) {
return Ok(rebuilt(status, headers, body));
}
let pause = pause_for(headers.get("retry-after").and_then(|v| v.to_str().ok()));
let Some(again) = again else { return Ok(rebuilt(status, headers, body)) };
tracing::debug!("discord: rate limited, waiting {}ms", pause.as_millis());
tokio::time::sleep(pause).await;
hold_for(pause);
wait_out_hold().await;
pending = Some(again);
}
bail!("Discord kept rate-limiting that request")
@@ -125,7 +366,13 @@ fn pause_for(seconds: Option<&str>) -> Duration {
/// The client for requests made under `key` - the account's token, or its
/// id - routed through Tor or the configured proxy when that account is.
pub(super) fn http_client_for(key: &str) -> reqwest::Client {
discord_client(crate::net::route::router().routed(key))
discord_client(crate::net::route::router().routed(key), true)
}
/// The client for a request that is not for Discord's API but is made on an account's behalf - an upload to the storage
/// address Discord hands out - routed as the account is, and carrying none of the headers that belong to the API.
pub(super) fn bare_client_for(key: &str) -> reqwest::Client {
discord_client(crate::net::route::router().routed(key), false)
}
/// The client for requests that belong to no account yet - the sign-in - or
@@ -133,14 +380,17 @@ pub(super) fn http_client_for(key: &str) -> reqwest::Client {
/// when the add form's switch says so, or when any Discord account is.
pub(super) fn anonymous_client() -> reqwest::Client {
let router = crate::net::route::router();
discord_client(router.routed(&crate::net::route::pending_key("discord")) || router.wanted(SHARED_KEY))
discord_client(router.routed(&crate::net::route::pending_key("discord")) || router.wanted(SHARED_KEY), false)
}
/// Marked while any Discord account is routed.
pub(super) const SHARED_KEY: &str = "discord:any-routed";
fn discord_client(routed: bool) -> reqwest::Client {
crate::net::route::router().client_if("discord", routed, |builder| {
/// `api` clients say which install, language and timezone they are (see `install_headers`); the others - the CDN,
/// the sign-in before there is an account - do not, since nothing there asks.
fn discord_client(routed: bool, api: bool) -> reqwest::Client {
crate::net::route::router().client_if(if api { "discord-api" } else { "discord" }, routed, |builder| {
let builder = if api { builder.default_headers(super::install_headers::headers().clone()) } else { builder };
// Pinned to HTTP/1.1. Enabling reqwest's http2 feature (which the
// file-upload path needs - see upload::http_client) would otherwise
// let every client here negotiate h2 as a side effect, changing the
@@ -282,4 +532,78 @@ mod error_tests {
let text = discord_error_text(reqwest::StatusCode::BAD_GATEWAY, junk, "adding a friend");
assert!(text.contains("502"), "got {text}");
}
/// What a client of an API with rate limits does: told to slow down, it waits as long as it was told and asks
/// again; told a bucket is empty, it lets it refill before anything else goes out. Asked of a local listener.
#[tokio::test]
async fn a_read_waits_out_a_rate_limit_and_a_bucket_that_has_run_dry() {
use tokio::io::{AsyncReadExt, AsyncWriteExt};
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let url = format!("http://{}/", listener.local_addr().unwrap());
// 1: refused with a one-second wait. 2: answered, saying the bucket is empty for one second. 3: answered.
let server = tokio::spawn(async move {
let mut seen = Vec::new();
for reply in [
"HTTP/1.1 429 Too Many Requests\r\nRetry-After: 1\r\nContent-Length: 0\r\nConnection: close\r\n\r\n",
"HTTP/1.1 200 OK\r\nX-RateLimit-Remaining: 0\r\nX-RateLimit-Reset-After: 1\r\nContent-Length: 0\r\nConnection: close\r\n\r\n",
"HTTP/1.1 200 OK\r\nContent-Length: 0\r\nConnection: close\r\n\r\n",
] {
let (mut socket, _) = listener.accept().await.unwrap();
let mut buf = vec![0u8; 4096];
let _ = socket.read(&mut buf).await.unwrap();
seen.push(std::time::Instant::now());
socket.write_all(reply.as_bytes()).await.unwrap();
}
seen
});
let client = reqwest::Client::new();
let started = std::time::Instant::now();
let first = client.get(&url).send_paced().await.unwrap();
assert_eq!(first.status(), 200, "the refusal was waited out and the read made again");
assert!(started.elapsed() >= Duration::from_millis(950), "it waited the second it was told to: {:?}", started.elapsed());
let second_started = std::time::Instant::now();
let second = client.get(&url).send_paced().await.unwrap();
assert_eq!(second.status(), 200);
assert!(second_started.elapsed() >= Duration::from_millis(950), "the empty bucket was let refill first: {:?}", second_started.elapsed());
let seen = server.await.unwrap();
assert_eq!(seen.len(), 3);
}
#[test]
fn a_refused_login_stops_everything_and_repeated_403s_close_their_route() {
use std::time::Instant;
let mut s = Safety::default();
let t = Instant::now();
let at = |secs: u64| t + Duration::from_secs(secs);
assert!(s.preflight_at("tok", "GET", "/channels/1/messages", t).is_ok());
// Two refusals are not three.
s.record_at("tok", "GET", "/channels/1/messages", 403, at(1));
s.record_at("tok", "GET", "/channels/1/messages", 403, at(2));
assert!(s.preflight_at("tok", "GET", "/channels/1/messages", at(3)).is_ok());
s.record_at("tok", "GET", "/channels/1/messages", 403, at(4));
let closed = s.preflight_at("tok", "GET", "/channels/1/messages", at(5)).unwrap_err();
assert!(matches!(closed, Blocked::Closed(w) if w > Duration::from_secs(290)));
// Other routes, other methods and other accounts are unaffected.
assert!(s.preflight_at("tok", "GET", "/channels/2/messages", at(5)).is_ok());
assert!(s.preflight_at("tok", "POST", "/channels/1/messages", at(5)).is_ok());
assert!(s.preflight_at("other", "GET", "/channels/1/messages", at(5)).is_ok());
// It opens again after the five minutes, and a success clears the count.
assert!(s.preflight_at("tok", "GET", "/channels/1/messages", at(4 + 301)).is_ok());
s.record_at("tok", "GET", "/channels/9/messages", 403, at(10));
s.record_at("tok", "GET", "/channels/9/messages", 200, at(11));
s.record_at("tok", "GET", "/channels/9/messages", 403, at(12));
s.record_at("tok", "GET", "/channels/9/messages", 403, at(13));
assert!(s.preflight_at("tok", "GET", "/channels/9/messages", at(14)).is_ok(), "403, 200, 403, 403 is not three in a row");
// A 401 ends all of it for that token, for good.
s.record_at("tok", "GET", "/users/@me", 401, at(20));
assert_eq!(s.preflight_at("tok", "POST", "/anything", at(21)), Err(Blocked::Refused));
assert!(s.preflight_at("fresh", "POST", "/anything", at(21)).is_ok(), "a new login is a new token");
}
#[test]
fn a_slowmode_answer_is_told_from_any_other_429() {
assert!(is_slowmode(br#"{"message":"The resource is being rate limited.","retry_after":3.5,"code":20016}"#));
assert!(!is_slowmode(br#"{"message":"You are being rate limited.","retry_after":1.2,"global":false}"#));
assert!(!is_slowmode(b"not json"));
}
}
src/backend/discord/identify_budget.rs
+179
View File
@@ -0,0 +1,179 @@
//! How often an account may log in to Discord afresh.
//!
//! A Discord client keeps one session open for days and, when the connection drops, *resumes* it. A fresh login (an
//! IDENTIFY) is what happens when a client starts or its session is lost, and a client that does it many times a day
//! looks like automation to Discord's spam detection. Restarting moho is exactly that, so the number of fresh logins an
//! account makes is kept - across restarts, in a file - and when it is over the limit the connection is held until it
//! is under again, with the window told why. A resume costs nothing and is never held.
use serde::{Deserialize, Serialize};
use std::collections::HashMap;
use std::sync::Mutex;
/// Fresh logins an account may make in any hour.
pub const PER_HOUR: usize = 3;
/// And in any day.
pub const PER_DAY: usize = 8;
const HOUR: i64 = 3600;
const DAY: i64 = 24 * HOUR;
/// Whether one more fresh login may be made at `now` (Unix seconds), given the times of the earlier ones. Answers
/// the number of seconds to wait when not. The list is kept as it should be saved: only what still counts.
pub fn admit(history: &mut Vec<i64>, now: i64) -> Result<(), i64> {
history.retain(|t| now - t < DAY);
history.sort_unstable();
let in_hour: Vec<i64> = history.iter().copied().filter(|t| now - t < HOUR).collect();
let mut wait = 0;
// Over the hour's limit: when the one that makes room ages out.
if in_hour.len() >= PER_HOUR {
wait = wait.max(in_hour[in_hour.len() - PER_HOUR] + HOUR - now);
}
// Over the day's.
if history.len() >= PER_DAY {
wait = wait.max(history[history.len() - PER_DAY] + DAY - now);
}
if wait > 0 {
return Err(wait.max(1));
}
history.push(now);
Ok(())
}
#[derive(Default, Serialize, Deserialize)]
struct Saved {
/// Account id -> when its fresh logins were made.
accounts: HashMap<String, Vec<i64>>,
}
fn file() -> std::path::PathBuf {
crate::default_data_dir().join("discord-logins.json")
}
fn held() -> &'static Mutex<Option<Saved>> {
static H: Mutex<Option<Saved>> = Mutex::new(None);
&H
}
/// Asks to make a fresh login now. `Err` carries how many seconds to wait first; `Ok` means it has been counted.
pub fn ask(account_id: &str) -> Result<(), i64> {
let now = chrono::Utc::now().timestamp();
let mut slot = held().lock().unwrap_or_else(|e| e.into_inner());
let saved = slot.get_or_insert_with(|| std::fs::read_to_string(file()).ok().and_then(|t| serde_json::from_str(&t).ok()).unwrap_or_default());
let history = saved.accounts.entry(account_id.to_string()).or_default();
let answer = admit(history, now);
if answer.is_ok() {
if let Ok(text) = serde_json::to_string(saved) {
if let Some(dir) = file().parent() {
let _ = std::fs::create_dir_all(dir);
}
let _ = std::fs::write(file(), text);
}
}
answer
}
/// The accounts whose login is being held now, each with the way to let it through.
type Held = Mutex<HashMap<String, std::sync::Arc<tokio::sync::Notify>>>;
fn waiting() -> &'static Held {
static W: std::sync::OnceLock<Held> = std::sync::OnceLock::new();
W.get_or_init(Default::default)
}
/// Marks an account as being held, answering what to wait on besides the clock.
pub fn begin_hold(account_id: &str) -> std::sync::Arc<tokio::sync::Notify> {
waiting().lock().unwrap_or_else(|e| e.into_inner()).entry(account_id.to_string()).or_default().clone()
}
/// The hold is over, by the clock or by the person.
pub fn end_hold(account_id: &str) {
waiting().lock().unwrap_or_else(|e| e.into_inner()).remove(account_id);
}
/// Lets a held login through now. `false` when nothing was being held, so that pressing the button twice, or after the
/// hold ended, cannot leave a pass waiting for some later hold.
pub fn connect_anyway(account_id: &str) -> bool {
match waiting().lock().unwrap_or_else(|e| e.into_inner()).get(account_id) {
Some(note) => {
note.notify_one();
true
}
None => false,
}
}
/// Counts a login that was let through past the limit: it still counts towards the next ones.
pub fn record(account_id: &str) {
let now = chrono::Utc::now().timestamp();
let mut slot = held().lock().unwrap_or_else(|e| e.into_inner());
let saved = slot.get_or_insert_with(|| std::fs::read_to_string(file()).ok().and_then(|t| serde_json::from_str(&t).ok()).unwrap_or_default());
let history = saved.accounts.entry(account_id.to_string()).or_default();
history.retain(|t| now - t < DAY);
history.push(now);
if let Ok(text) = serde_json::to_string(saved) {
let _ = std::fs::write(file(), text);
}
}
/// What the window says while a login is held.
pub fn explain(wait: i64) -> String {
let at = chrono::Local::now() + chrono::Duration::seconds(wait);
format!(
"Holding the Discord connection: this account has already logged in {PER_HOUR} times in the last hour (or {PER_DAY} in a day), and many fresh logins can get an account flagged as spam. It will connect at {}.",
at.format("%H:%M")
)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_few_logins_an_hour_are_let_through_and_the_next_waits_for_the_oldest_to_age_out() {
let mut history = Vec::new();
let start = 1_000_000;
assert_eq!(admit(&mut history, start), Ok(()));
assert_eq!(admit(&mut history, start + 60), Ok(()));
assert_eq!(admit(&mut history, start + 120), Ok(()));
// The fourth, ten minutes in: held until the first is an hour old.
assert_eq!(admit(&mut history, start + 600), Err(HOUR - 600));
assert_eq!(history.len(), 3, "a held login is not counted");
// An hour after the first, there is room.
assert_eq!(admit(&mut history, start + HOUR), Ok(()));
}
#[test]
fn the_day_has_its_own_limit_however_the_hours_fall() {
let mut history = Vec::new();
let start = 5_000_000;
// Three an hour for three hours is nine; the eighth is held until the first is a day old.
let mut made = 0;
for hour in 0..3 {
for n in 0..3 {
if admit(&mut history, start + hour * HOUR + n).is_ok() {
made += 1;
}
}
}
assert_eq!(made, PER_DAY);
let wait = admit(&mut history, start + 3 * HOUR).unwrap_err();
assert_eq!(wait, DAY - 3 * HOUR);
}
#[test]
fn a_pass_is_only_given_to_a_login_that_is_being_held() {
assert!(!connect_anyway("discord:test-never-held"), "nothing is held, so nothing is let through");
let _pass = begin_hold("discord:test-held");
assert!(connect_anyway("discord:test-held"));
end_hold("discord:test-held");
assert!(!connect_anyway("discord:test-held"), "after the hold ends there is nothing to let through");
}
#[test]
fn old_logins_are_forgotten() {
let mut history = vec![1, 2, 3, 4, 5];
assert_eq!(admit(&mut history, 10 * DAY), Ok(()));
assert_eq!(history, vec![10 * DAY]);
}
}
src/backend/discord/install_headers.rs
+136
View File
@@ -0,0 +1,136 @@
//! What a Discord client tells the API about the install it runs on: which install, in what language, in what
//! timezone.
//!
//! Discord's own clients send these with every request (`X-Installation-ID`, `X-Discord-Locale`,
//! `X-Discord-Timezone`), so an API that sees a long-lived account from one install, in one place, with one
//! language, sees what it expects. They say nothing about *what* is connecting - that stays `moho/<version>
//! (nobilis)` - and the install id is moho's own, made here once and kept: it is not a copy of anything Discord
//! issued.
use reqwest::header::{HeaderMap, HeaderName, HeaderValue};
use std::sync::OnceLock;
/// The id of this install: random, made the first time it is needed, and kept in the data directory so that every
/// run - and every account on it - is the same install.
pub fn installation_id() -> String {
let file = crate::default_data_dir().join("discord-installation-id");
if let Some(id) = std::fs::read_to_string(&file).ok().map(|t| t.trim().to_string()).filter(|t| well_formed(t)) {
return id;
}
let bytes: [u8; 16] = rand::random();
let id: String = bytes.iter().map(|b| format!("{b:02x}")).collect();
if let Some(dir) = file.parent() {
let _ = std::fs::create_dir_all(dir);
}
let _ = std::fs::write(&file, &id);
id
}
fn well_formed(id: &str) -> bool {
id.len() == 32 && id.bytes().all(|b| b.is_ascii_hexdigit())
}
/// The language this machine is set to, as Discord writes it (`en-US`): from the environment's usual variables, in
/// their usual order of precedence. `None` when it says nothing, or says "C".
pub fn locale() -> Option<String> {
let raw = ["LC_ALL", "LC_MESSAGES", "LANG"].iter().filter_map(|v| std::env::var(v).ok()).find(|v| !v.is_empty())?;
locale_from(&raw)
}
fn locale_from(raw: &str) -> Option<String> {
// `en_US.UTF-8@euro` -> `en-US`
let tag = raw.split(['.', '@']).next()?.replace('_', "-");
let mut parts = tag.split('-');
let language = parts.next().filter(|l| (2..=3).contains(&l.len()) && l.bytes().all(|b| b.is_ascii_alphabetic()))?;
if language.eq_ignore_ascii_case("C") || language.eq_ignore_ascii_case("POSIX") {
return None;
}
let language = language.to_ascii_lowercase();
match parts.next().filter(|r| r.len() == 2 && r.bytes().all(|b| b.is_ascii_alphabetic())) {
Some(region) => Some(format!("{language}-{}", region.to_ascii_uppercase())),
None => Some(language),
}
}
/// The IANA name of this machine's timezone (`America/New_York`), when it can be told.
pub fn timezone() -> Option<String> {
iana_time_zone::get_timezone().ok().filter(|t| !t.is_empty())
}
/// The headers every API request carries.
pub fn headers() -> &'static HeaderMap {
static HEADERS: OnceLock<HeaderMap> = OnceLock::new();
HEADERS.get_or_init(|| {
let mut pairs: Vec<(&'static str, String)> = vec![("x-installation-id", installation_id())];
if let Some(locale) = locale() {
let language = locale.split('-').next().unwrap_or("en").to_string();
pairs.push(("accept-language", format!("{locale},{language};q=0.9")));
pairs.push(("x-discord-locale", locale));
}
if let Some(zone) = timezone() {
pairs.push(("x-discord-timezone", zone));
}
let mut map = HeaderMap::new();
for (name, value) in pairs {
if let Ok(value) = HeaderValue::from_str(&value) {
map.insert(HeaderName::from_static(name), value);
}
}
map
})
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_locale_is_written_the_way_discord_writes_it() {
assert_eq!(locale_from("en_US.UTF-8").as_deref(), Some("en-US"));
assert_eq!(locale_from("de_DE@euro").as_deref(), Some("de-DE"));
assert_eq!(locale_from("fr").as_deref(), Some("fr"));
assert_eq!(locale_from("pt-br").as_deref(), Some("pt-BR"));
}
#[test]
fn a_machine_with_no_language_set_sends_none() {
assert_eq!(locale_from("C"), None);
assert_eq!(locale_from("POSIX"), None);
assert_eq!(locale_from("C.UTF-8"), None);
assert_eq!(locale_from(""), None);
assert_eq!(locale_from("12345"), None);
}
#[test]
fn an_installation_id_is_recognised_only_when_whole() {
assert!(well_formed("0123456789abcdef0123456789abcdef"));
assert!(!well_formed("0123"));
assert!(!well_formed("zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz"));
}
/// What actually leaves: the API client carries the headers and still says it is moho, and the client for the CDN
/// and sign-in carries none of them. Asked of a local listener, so nothing reaches Discord.
#[tokio::test]
async fn the_api_client_sends_them_and_the_other_does_not() {
use tokio::io::{AsyncReadExt, AsyncWriteExt};
async fn request_seen_by_a_listener(client: reqwest::Client) -> String {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let url = format!("http://{}/", listener.local_addr().unwrap());
let server = tokio::spawn(async move {
let (mut socket, _) = listener.accept().await.unwrap();
let mut buf = vec![0u8; 8192];
let n = socket.read(&mut buf).await.unwrap();
socket.write_all(b"HTTP/1.1 200 OK\r\nContent-Length: 0\r\nConnection: close\r\n\r\n").await.unwrap();
String::from_utf8_lossy(&buf[..n]).to_lowercase()
});
let _ = client.get(url).send().await;
server.await.unwrap()
}
let api = request_seen_by_a_listener(crate::backend::discord::http::http_client_for("a-test-token")).await;
assert!(api.contains("x-installation-id: "), "{api}");
assert!(api.contains("user-agent: moho/"), "{api}");
assert!(!api.contains("x-super-properties"), "{api}");
let other = request_seen_by_a_listener(crate::backend::discord::http::anonymous_client()).await;
assert!(!other.contains("x-installation-id"), "{other}");
}
}
src/backend/discord/media.rs
+2 -2
View File
@@ -249,7 +249,7 @@ pub(super) async fn run_resign(state: &AppState, buffer_id: &str, mut stale: Vec
.get(format!("{API_BASE}/channels/{channel_id}/messages"))
.query(&[("limit", "50"), ("around", anchor.as_str())])
.header("Authorization", &config.token)
.send()
.send_paced()
.await
.context("fetching a page to re-sign")?;
if !resp.status().is_success() {
@@ -358,7 +358,7 @@ pub async fn refresh_attachments(state: &AppState, buffer_id: &str, message_id:
.get(format!("{API_BASE}/channels/{channel_id}/messages"))
.query(&[("limit", "1"), ("around", message_id)])
.header("Authorization", &config.token)
.send()
.send_paced()
.await
.context("re-fetching the message")?;
if !resp.status().is_success() {
src/backend/discord/messages.rs
+312 -64
View File
@@ -23,7 +23,7 @@ pub async fn list_pinned(state: &AppState, account_id: &str, buffer_id: &str) ->
let resp = http_client_for(&cfg.token)
.get(format!("{API_BASE}/channels/{channel_id}/pins"))
.header("Authorization", &cfg.token)
.send()
.send_paced()
.await
.context("reading the pins")?;
if !resp.status().is_success() {
@@ -96,7 +96,7 @@ pub async fn set_pinned(state: &AppState, account_id: &str, buffer_id: &str, mes
let http = http_client_for(&cfg.token);
let request = if pinned { http.put(url) } else { http.delete(url) };
let doing = if pinned { "pinning the message" } else { "unpinning the message" };
let resp = request.header("Authorization", &cfg.token).send().await.context(doing)?;
let resp = request.header("Authorization", &cfg.token).send_paced().await.context(doing)?;
if !resp.status().is_success() {
let status = resp.status();
let text = resp.text().await.unwrap_or_default();
@@ -601,7 +601,7 @@ pub async fn name_forward(state: &AppState, account_id: &str, buffer_id: &str, m
let resp = http_client_for(&cfg.token)
.get(format!("{API_BASE}/channels/{source_channel}/messages?limit=1&around={source_message}"))
.header("Authorization", &cfg.token)
.send()
.send_paced()
.await;
// A source this account cannot see is the ordinary case for a forward
// out of somebody else's server, and not worth a word to anybody.
@@ -703,6 +703,12 @@ pub(super) fn extract_attachments(d: &Value) -> Vec<Attachment> {
/// represented by extract_body's own media-URL handling above, so
/// including it here too would just render an empty box under the media.
pub(super) fn extract_embeds(d: &Value) -> Vec<Embed> {
let mut out = ordinary_embeds(d);
out.extend(layout_cards(d));
out
}
fn ordinary_embeds(d: &Value) -> Vec<Embed> {
let Some(embeds) = d["embeds"].as_array() else { return Vec::new() };
embeds
.iter()
@@ -910,6 +916,7 @@ pub(super) fn extract_reply(d: &Value) -> Option<ReplyPreview> {
thread: false,
forwarded: true,
thread_root: None,
chat: None,
});
}
let referenced = &d["referenced_message"];
@@ -921,7 +928,7 @@ pub(super) fn extract_reply(d: &Value) -> Option<ReplyPreview> {
let author = &referenced["author"];
let from = author["global_name"].as_str().filter(|s| !s.is_empty()).or_else(|| author["username"].as_str()).unwrap_or("unknown").to_string();
let body = extract_body(referenced).unwrap_or_default();
Some(ReplyPreview { id: reply_id.to_string(), from, body, thread: false, forwarded: false, thread_root: None })
Some(ReplyPreview { id: reply_id.to_string(), from, body, thread: false, forwarded: false, thread_root: None, chat: None })
}
/// Shared by both the initial backfill and extend_history - Discord
@@ -957,9 +964,17 @@ pub(super) fn store_history_messages(state: &AppState, buffer_id: &str, messages
.and_then(|s| chrono::DateTime::parse_from_rfc3339(s).ok())
.map(|dt| dt.timestamp())
.unwrap_or(0);
if let Err(e) = state.store.append_message(buffer_id, msg_id, from, &body, ts, false, false, "chat", reply_to.as_ref(), &reactions, is_own, avatar_url.as_deref(), &embeds, &attachments, author["id"].as_str(), None, &state.runtime.buffer_kind_of(buffer_id), None, &[]) {
tracing::warn!("discord: storing history message: {e}");
continue;
let badges = author_badges(author);
match state.store.append_message(buffer_id, msg_id, from, &body, ts, false, false, "chat", reply_to.as_ref(), &reactions, is_own, avatar_url.as_deref(), &embeds, &attachments, author["id"].as_str(), None, &state.runtime.buffer_kind_of(buffer_id), None, &badges) {
Err(e) => {
tracing::warn!("discord: storing history message: {e}");
continue;
}
// Read again, a message kept before bots were marked gets its mark.
Ok(false) if !badges.is_empty() => {
let _ = state.store.set_badges(buffer_id, msg_id, &badges);
}
Ok(_) => {}
}
note_components(state, buffer_id, msg_id, msg);
// A poll in the history may still be running - Discord's last hours
@@ -1012,6 +1027,26 @@ pub(super) fn extract_reactions(msg: &Value) -> Vec<Reaction> {
.collect()
}
/// What Discord marks an author as, beside their name: "APP" for a bot or an application, with a check when Discord has
/// verified it. Read off the author the message carries, so it is there for history and for live messages alike.
pub(super) fn author_badges(author: &Value) -> Vec<crate::backend::kick::api::Badge> {
// Bit 16 of the public flags: a verified bot.
const VERIFIED_BOT: u64 = 1 << 16;
// Discord's own account: what it sends is the platform speaking, and says so with the OFFICIAL mark.
if author["system"].as_bool() == Some(true) {
return vec![crate::backend::kick::api::Badge { kind: "official".to_string(), text: "Official Discord Message".to_string(), ..Default::default() }];
}
if author["bot"].as_bool() != Some(true) {
return Vec::new();
}
let verified = author["public_flags"].as_u64().unwrap_or(0) & VERIFIED_BOT != 0;
vec![crate::backend::kick::api::Badge {
kind: if verified { "verified-app" } else { "app" }.to_string(),
text: if verified { "Verified App" } else { "App" }.to_string(),
..Default::default()
}]
}
/// The buttons and menus on a message, in this client's own shape.
///
/// Discord lays them out in rows of up to five; the row is kept so a client
@@ -1021,65 +1056,190 @@ pub(super) fn extract_reactions(msg: &Value) -> Vec<Reaction> {
pub(super) fn extract_components(msg: &Value) -> Vec<model::Component> {
let Some(rows) = msg["components"].as_array() else { return Vec::new() };
let mut out = Vec::new();
for (row_index, row) in rows.iter().enumerate() {
// A row is itself a component (type 1) holding the real ones. A
// message whose top level is already a control is not a shape Discord
// sends, but reading it either way costs nothing.
let children = row["components"].as_array().cloned().unwrap_or_else(|| vec![row.clone()]);
for child in children {
let kind = match child["type"].as_i64() {
Some(2) => "button",
// 3 is the plain string select; 5 through 8 are the ones that
// pick users, roles, mentionables and channels. They are all
// menus, and all of them answer the same way.
Some(3) | Some(5) | Some(6) | Some(7) | Some(8) => "select",
_ => continue,
};
let style = child["style"].as_i64().map(|s| match s {
1 => "primary",
2 => "secondary",
3 => "success",
4 => "danger",
5 => "link",
_ => "secondary",
});
let options = child["options"]
.as_array()
.map(|opts| {
opts.iter()
.filter_map(|o| {
Some(model::ComponentOption {
value: o["value"].as_str()?.to_string(),
label: o["label"].as_str().unwrap_or_default().to_string(),
description: o["description"].as_str().map(str::to_string),
})
})
.collect()
})
.unwrap_or_default();
out.push(model::Component {
kind: kind.to_string(),
custom_id: child["custom_id"].as_str().map(str::to_string),
label: child["label"].as_str().filter(|l| !l.is_empty()).map(str::to_string),
style: style.map(str::to_string),
url: child["url"].as_str().map(str::to_string),
disabled: child["disabled"].as_bool().unwrap_or(false),
// Written the way this client writes emoji everywhere else, so
// a button's picture renders like one in a message does.
emoji: match (child["emoji"]["name"].as_str(), child["emoji"]["id"].as_str()) {
(Some(name), Some(id)) => Some(format!("<:{name}:{id}>")),
(Some(name), None) => Some(name.to_string()),
_ => None,
},
options,
placeholder: child["placeholder"].as_str().map(str::to_string),
row: row_index as i64,
});
}
let mut row_index = 0i64;
for row in rows {
controls_in(row, &mut row_index, &mut out);
}
out
}
/// The buttons and menus inside one component, however deep the layout puts them.
///
/// An ordinary message has rows (type 1) of controls. A message made with Discord's newer layout components wraps
/// them: a container (17) holds rows, sections (9) - whose accessory may itself be a button - and text, and a control
/// may sit anywhere in it. Each row, and each section's accessory, is one line of controls in the order met.
fn controls_in(node: &Value, row_index: &mut i64, out: &mut Vec<model::Component>) {
match node["type"].as_i64() {
// An action row: its children are the controls.
Some(1) => {
for child in node["components"].as_array().into_iter().flatten() {
push_control(child, *row_index, out);
}
*row_index += 1;
}
// A container, or anything else that holds components: look inside.
Some(17) => {
for child in node["components"].as_array().into_iter().flatten() {
controls_in(child, row_index, out);
}
}
// A section's accessory is a thumbnail or a button; the text beside it holds no controls.
Some(9) => {
if node["accessory"]["type"].as_i64() == Some(2) {
push_control(&node["accessory"], *row_index, out);
*row_index += 1;
}
}
// A control with no row around it is not a shape Discord sends, but reading it costs nothing.
_ => push_control(node, *row_index, out),
}
}
fn push_control(child: &Value, row_index: i64, out: &mut Vec<model::Component>) {
let kind = match child["type"].as_i64() {
Some(2) => "button",
// 3 is the plain string select; 5 through 8 are the ones that
// pick users, roles, mentionables and channels. They are all
// menus, and all of them answer the same way.
Some(3) | Some(5) | Some(6) | Some(7) | Some(8) => "select",
_ => return,
};
let style = child["style"].as_i64().map(|s| match s {
1 => "primary",
2 => "secondary",
3 => "success",
4 => "danger",
5 => "link",
_ => "secondary",
});
let options = child["options"]
.as_array()
.map(|opts| {
opts.iter()
.filter_map(|o| {
Some(model::ComponentOption {
value: o["value"].as_str()?.to_string(),
label: o["label"].as_str().unwrap_or_default().to_string(),
description: o["description"].as_str().map(str::to_string),
})
})
.collect()
})
.unwrap_or_default();
out.push(model::Component {
kind: kind.to_string(),
custom_id: child["custom_id"].as_str().map(str::to_string),
label: child["label"].as_str().filter(|l| !l.is_empty()).map(str::to_string),
style: style.map(str::to_string),
url: child["url"].as_str().map(str::to_string),
disabled: child["disabled"].as_bool().unwrap_or(false),
// Written the way this client writes emoji everywhere else, so
// a button's picture renders like one in a message does.
emoji: match (child["emoji"]["name"].as_str(), child["emoji"]["id"].as_str()) {
(Some(name), Some(id)) => Some(format!("<:{name}:{id}>")),
(Some(name), None) => Some(name.to_string()),
_ => None,
},
options,
placeholder: child["placeholder"].as_str().map(str::to_string),
row: row_index,
});
}
/// What a message made with Discord's layout components says, as cards.
///
/// Those messages carry no text of their own: a container (type 17) holds text displays (10), sections (9) pairing text
/// with a picture or a button, galleries of pictures (12), files (13) and separators (14). Each container becomes one
/// card - its text as the description, a leading heading line as the title, its first picture as the card's picture and
/// its accent colour - and text or pictures outside any container become one more. The buttons in them are read apart
/// (see `controls_in`).
fn layout_cards(d: &Value) -> Vec<Embed> {
let Some(top) = d["components"].as_array() else { return Vec::new() };
let mut cards = Vec::new();
let mut loose = Card::default();
for node in top {
if node["type"].as_i64() == Some(17) {
let mut card = Card::default();
card.take(node);
if let Some(embed) = card.into_embed(node["accent_color"].as_i64()) {
cards.push(embed);
}
} else {
loose.take(node);
}
}
if let Some(embed) = loose.into_embed(None) {
cards.insert(0, embed);
}
cards
}
#[derive(Default)]
struct Card {
texts: Vec<String>,
picture: Option<String>,
}
impl Card {
fn take(&mut self, node: &Value) {
match node["type"].as_i64() {
Some(10) => {
if let Some(text) = node["content"].as_str().filter(|t| !t.trim().is_empty()) {
self.texts.push(text.to_string());
}
}
// A thumbnail, or a gallery of pictures: the first is the card's.
Some(11) => self.picture_of(&node["media"]),
Some(12) => {
for item in node["items"].as_array().into_iter().flatten() {
self.picture_of(&item["media"]);
}
}
Some(13) => {
if let Some(url) = node["file"]["url"].as_str() {
self.texts.push(format!("[{}]({url})", node["name"].as_str().unwrap_or("Attached file")));
}
}
// A section: its text, and its accessory when that is a picture.
Some(9) | Some(17) => {
for child in node["components"].as_array().into_iter().flatten() {
self.take(child);
}
if node["accessory"]["type"].as_i64() == Some(11) {
self.picture_of(&node["accessory"]["media"]);
}
}
_ => {}
}
}
fn picture_of(&mut self, media: &Value) {
if self.picture.is_none() {
self.picture = media["url"].as_str().map(str::to_string);
}
}
fn into_embed(self, accent: Option<i64>) -> Option<Embed> {
if self.texts.is_empty() && self.picture.is_none() {
return None;
}
let joined = self.texts.join("\n\n");
// A first line that is a markdown heading is what the card is called.
let (title, description) = match joined.split_once('\n') {
Some((first, rest)) if first.starts_with('#') => (Some(first.trim_start_matches('#').trim().to_string()), rest.trim().to_string()),
None if joined.starts_with('#') => (Some(joined.trim_start_matches('#').trim().to_string()), String::new()),
_ => (None, joined),
};
Some(Embed {
title: title.filter(|t| !t.is_empty()),
description: Some(description).filter(|d| !d.is_empty()),
color: accent,
image_url: self.picture,
..Default::default()
})
}
}
/// Stores a message's buttons and tells the client, if it has any.
///
/// Separate from the message itself because they arrive with it and are
@@ -1520,7 +1680,7 @@ pub async fn list_reactors(state: &AppState, account_id: &str, buffer_id: &str,
let resp = http_client_for(&cfg.token)
.get(url)
.header("Authorization", &cfg.token)
.send()
.send_paced()
.await
.context("reading who reacted")?;
if !resp.status().is_success() {
@@ -1583,7 +1743,7 @@ pub async fn reread_embeds(state: &AppState, buffer_id: &str) -> Result<usize> {
.get(format!("{API_BASE}/channels/{channel_id}/messages"))
.query(&[("limit", "50")])
.header("Authorization", &config.token)
.send()
.send_paced()
.await
.context("re-reading the conversation")?;
if !resp.status().is_success() {
@@ -1725,3 +1885,91 @@ mod gif_tests {
assert_eq!(extract_body(&quiet), None);
}
}
#[cfg(test)]
mod layout_tests {
use super::{extract_components, extract_embeds};
use serde_json::json;
/// What a bot's message built with Discord's layout components looks like: no text of its own, a container with
/// a section (text and a picture) and a row holding one button.
fn honeypot() -> serde_json::Value {
json!({
"content": "",
"flags": 32768,
"components": [{
"type": 17,
"accent_color": 0x5865f2,
"components": [
{ "type": 9,
"components": [{ "type": 10, "content": "## DO NOT SEND MESSAGES IN THIS CHANNEL\nThis channel is used to catch spam bots. Any messages sent here will result in **an immediate ban**." }],
"accessory": { "type": 11, "media": { "url": "https://cdn.discordapp.com/attachments/1/2/honeypot.png" } } },
{ "type": 14, "divider": true },
{ "type": 1, "components": [{ "type": 2, "style": 2, "label": "Bans: 19", "custom_id": "bans", "disabled": true, "emoji": { "name": "🍯" } }] }
]
}]
})
}
#[test]
fn a_container_is_one_card_with_a_title_its_text_and_its_picture() {
let cards = extract_embeds(&honeypot());
assert_eq!(cards.len(), 1);
assert_eq!(cards[0].title.as_deref(), Some("DO NOT SEND MESSAGES IN THIS CHANNEL"));
assert!(cards[0].description.as_deref().unwrap().starts_with("This channel is used to catch spam bots."));
assert_eq!(cards[0].image_url.as_deref(), Some("https://cdn.discordapp.com/attachments/1/2/honeypot.png"));
assert_eq!(cards[0].color, Some(0x5865f2));
}
#[test]
fn the_buttons_inside_a_container_are_found() {
let controls = extract_components(&honeypot());
assert_eq!(controls.len(), 1);
assert_eq!((controls[0].label.as_deref(), controls[0].kind.as_str(), controls[0].disabled), (Some("Bans: 19"), "button", true));
assert_eq!(controls[0].emoji.as_deref(), Some("🍯"));
}
#[test]
fn a_sections_button_accessory_is_a_control_and_its_picture_accessory_is_not() {
let msg = json!({ "components": [{ "type": 17, "components": [
{ "type": 9, "components": [{ "type": 10, "content": "Pick" }], "accessory": { "type": 2, "style": 1, "label": "Go", "custom_id": "go" } },
] }] });
let controls = extract_components(&msg);
assert_eq!(controls.len(), 1);
assert_eq!(controls[0].label.as_deref(), Some("Go"));
assert_eq!(extract_embeds(&msg)[0].description.as_deref(), Some("Pick"));
}
#[test]
fn an_ordinary_message_has_no_layout_card() {
let msg = json!({ "content": "hi", "components": [{ "type": 1, "components": [{ "type": 2, "label": "x", "custom_id": "x", "style": 1 }] }] });
assert!(extract_embeds(&msg).is_empty());
assert_eq!(extract_components(&msg).len(), 1);
}
}
#[cfg(test)]
mod app_badge_tests {
use super::author_badges;
use serde_json::json;
#[test]
fn a_bot_is_an_app_and_a_verified_one_says_so() {
let plain = author_badges(&json!({ "id": "1", "bot": true, "public_flags": 0 }));
assert_eq!((plain[0].kind.as_str(), plain[0].text.as_str()), ("app", "App"));
let verified = author_badges(&json!({ "id": "1", "bot": true, "public_flags": 65536 }));
assert_eq!((verified[0].kind.as_str(), verified[0].text.as_str()), ("verified-app", "Verified App"));
}
#[test]
fn discords_own_account_is_official_even_though_it_is_not_an_app() {
let official = author_badges(&json!({ "id": "643945264868098049", "system": true, "bot": true }));
assert_eq!((official[0].kind.as_str(), official[0].text.as_str()), ("official", "Official Discord Message"));
}
#[test]
fn a_person_has_no_badge() {
assert!(author_badges(&json!({ "id": "2", "username": "someone" })).is_empty());
assert!(author_badges(&json!({ "id": "2", "bot": false })).is_empty());
}
}
src/backend/discord/mod.rs
+6
View File
@@ -49,10 +49,16 @@ pub mod voice;
pub mod calls;
pub mod commands;
pub mod gateway;
pub mod identify_budget;
pub mod dave;
pub mod golive;
pub mod guilds;
pub mod history;
pub mod install_headers;
pub mod ready_shape;
pub mod rules;
pub mod send_guard;
pub mod session_store;
pub mod http;
pub mod login;
pub mod media;
src/backend/discord/people.rs
+11 -4
View File
@@ -27,7 +27,9 @@ pub(super) fn register_dm_channel(
state.runtime.set_discord_channel(state, &buf.id, channel_id);
// Discord's own notices come from an account flagged `system`, and nothing
// sent back to it goes anywhere.
state.runtime.set_buffer_read_only(state, &buf.id, is_system_dm(ch).then_some(OFFICIAL_ONLY));
super::send_guard::set_permission(state, account_id, &buf.id, is_system_dm(ch).then_some(OFFICIAL_ONLY));
// A conversation with nothing in it yet is not started from here: see send_guard.
super::send_guard::note_dm(state, account_id, &buf.id, ch);
// Its notices were saved before their cards could be read whole, and saved
// history is not fetched again, so they are read once a run, a little after
// connecting - one request, for the latest page.
@@ -40,6 +42,11 @@ pub(super) fn register_dm_channel(
}
});
}
// When it was last spoken in, from the id of its last message - which is all that ordering the list by activity
// needs, so nothing is fetched for it.
if let Some(at) = ch["last_message_id"].as_str().and_then(super::history::snowflake_secs) {
state.runtime.raise_buffer_activity(state, &buf.id, at);
}
if let Some(avatar) = dm_avatar_url(ch) {
state.runtime.set_buffer_avatar(state, &buf.id, &avatar);
}
@@ -166,7 +173,7 @@ pub async fn profile(state: &AppState, account_id: &str, buffer_id: &str, user_i
}
}
let response = http_client_for(&config.token).get(&url).header("Authorization", &config.token).send().await;
let response = http_client_for(&config.token).get(&url).header("Authorization", &config.token).send_paced().await;
let Ok(body) = response else { return profile };
let Ok(body) = body.json::<Value>().await else { return profile };
@@ -295,7 +302,7 @@ pub async fn adopt_dm_channel(state: &AppState, account_id: &str, channel_id: &s
let resp = http_client_for(&config.token)
.get(format!("{API_BASE}/channels/{channel_id}"))
.header("Authorization", &config.token)
.send()
.send_paced()
.await
.context("asking about that conversation")?;
if !resp.status().is_success() {
@@ -379,7 +386,7 @@ pub async fn answer_friend_request(state: &AppState, account_id: &str, user_id:
let url = format!("{API_BASE}/users/@me/relationships/{user_id}");
let request = if accept { http.put(url).json(&json!({})) } else { http.delete(url) };
let doing = if accept { "accepting the friend request" } else { "declining the friend request" };
let resp = request.header("Authorization", &cfg.token).send().await.context(doing)?;
let resp = request.header("Authorization", &cfg.token).send_paced().await.context(doing)?;
if !resp.status().is_success() {
let status = resp.status();
let text = resp.text().await.unwrap_or_default();
src/backend/discord/presence.rs
+73 -1
View File
@@ -117,6 +117,46 @@ fn settings_body(status: &str, text: Option<&str>) -> serde_json::Value {
/// does when you open a channel, which is also why the roster only exists for
/// channels somebody is actually looking at.
pub fn request_member_list(state: &AppState, buffer_id: &str) -> bool {
subscribe_member_list(state, buffer_id, false)
}
/// The window the list is kept in no longer matches what Discord holds, so it is asked for again - but not more often
/// than every few seconds, which is the most a roster that keeps falling out of step deserves.
pub(super) fn resync_member_list(state: &AppState, buffer_id: &str) -> bool {
subscribe_member_list(state, buffer_id, true)
}
/// What each server has been subscribed to this session: the channel, and when.
type Subscribed = HashMap<(String, String), (String, std::time::Instant)>;
static SUBSCRIBED: std::sync::Mutex<Option<Subscribed>> = std::sync::Mutex::new(None);
/// The least time between two subscriptions to one server's list that are not a change of channel.
const RESYNC_EVERY: std::time::Duration = std::time::Duration::from_secs(10);
/// Whether a subscription to this channel's list is to be sent: not if it already was, this session, and not a
/// repeat of one sent a moment ago. A connection is subscribed once - as Dissent's is, once a server - and a new one
/// starts again (`forget_member_subscriptions`).
fn subscription_due(account: &str, guild: &str, channel: &str, resync: bool, now: std::time::Instant) -> bool {
let mut slot = SUBSCRIBED.lock().unwrap_or_else(|e| e.into_inner());
let map = slot.get_or_insert_with(HashMap::new);
let key = (account.to_string(), guild.to_string());
match map.get(&key) {
Some((was, at)) if was == channel && (!resync || now.duration_since(*at) < RESYNC_EVERY) => false,
_ => {
map.insert(key, (channel.to_string(), now));
true
}
}
}
/// A new connection holds none of what the last one subscribed to.
pub(super) fn forget_member_subscriptions(account: &str) {
if let Some(map) = SUBSCRIBED.lock().unwrap_or_else(|e| e.into_inner()).as_mut() {
map.retain(|(a, _), _| a != account);
}
}
fn subscribe_member_list(state: &AppState, buffer_id: &str, resync: bool) -> bool {
let Some(buffer) = state.runtime.get_buffer(buffer_id) else { return false };
let account_id = buffer.account_id;
let Some(sender) = state.runtime.discord_gateway_sender(&account_id) else { return false };
@@ -131,6 +171,11 @@ pub fn request_member_list(state: &AppState, buffer_id: &str) -> bool {
// channel, so remember which buffer this was for.
state.runtime.set_discord_member_list_target(&account_id, &guild_id, buffer_id);
// Asked for once. Whoever asked is told it is being sent: the roster arrives, or is already arriving.
if !subscription_due(&account_id, &guild_id, &channel_id, resync, std::time::Instant::now()) {
return true;
}
// Ranges are 100-member windows; one covers any channel we would show.
sender
.send(
@@ -370,7 +415,7 @@ pub(super) fn update_member_list(state: &AppState, buffer_id: &str, d: &Value) {
Applied::Stale => {
tracing::debug!("discord: member window for {buffer_id} is out of step; resyncing");
state.runtime.set_discord_member_window(buffer_id, Vec::new());
request_member_list(state, buffer_id);
resync_member_list(state, buffer_id);
return;
}
}
@@ -629,3 +674,30 @@ mod status_tests {
assert_eq!(settings_body("idle", None)["status"], "idle");
}
}
#[cfg(test)]
mod subscription_tests {
use super::*;
use std::time::{Duration, Instant};
#[test]
fn a_list_is_subscribed_to_once_a_connection_and_again_for_another_channel() {
let t = Instant::now();
assert!(subscription_due("a1", "g", "c1", false, t));
assert!(!subscription_due("a1", "g", "c1", false, t + Duration::from_secs(1)), "the same again is nothing");
assert!(!subscription_due("a1", "g", "c1", false, t + Duration::from_secs(600)), "however long after");
assert!(subscription_due("a1", "g", "c2", false, t + Duration::from_secs(2)), "another channel is a change");
assert!(subscription_due("a1", "g2", "c1", false, t), "another server is its own");
forget_member_subscriptions("a1");
assert!(subscription_due("a1", "g", "c2", false, t), "a new connection starts again");
}
#[test]
fn a_resync_is_not_sent_again_within_a_few_seconds() {
let t = Instant::now();
assert!(subscription_due("a2", "g", "c1", false, t));
assert!(!subscription_due("a2", "g", "c1", true, t + Duration::from_secs(3)));
assert!(subscription_due("a2", "g", "c1", true, t + Duration::from_secs(11)));
assert!(!subscription_due("a2", "g", "c1", true, t + Duration::from_secs(12)));
}
}
src/backend/discord/ready_shape.rs
+152
View File
@@ -0,0 +1,152 @@
//! READY as a client that asked for DEDUPE_USER_OBJECTS receives it, put back as every reader here expects it.
//!
//! With that capability Discord "dehydrates" READY (<https://docs.discord.food/topics/gateway-events#ready>): every
//! user object is moved into one `users` table and left behind as an id - `user_id` in members, presences and
//! relationships, `recipient_ids` in private channels - each guild's `members` is collapsed into READY's
//! `merged_members` (one list a guild, in the order of `guilds`), and `presences`, with each guild's, becomes
//! `merged_presences`. Everything that reads READY - the friends list, the names of conversations, who is where in a
//! voice channel - was written against the objects. So they are put back first, here, once, and nothing else has to
//! know the shape changed. A READY without a `users` table is already whole and comes back unchanged.
use serde_json::{json, Map, Value};
use std::collections::HashMap;
/// READY with its users put back, or `None` when there is nothing to put back.
pub fn rehydrate(ready: &Value) -> Option<Value> {
let users: HashMap<String, Value> = ready["users"]
.as_array()?
.iter()
.filter_map(|u| Some((u["id"].as_str()?.to_string(), u.clone())))
.collect();
let mut out = ready.clone();
// The members the guilds were relieved of, back where they were: in the guild, by its place in the list.
if let Some(merged) = ready["merged_members"].as_array() {
if let Some(guilds) = out["guilds"].as_array_mut() {
for (guild, members) in guilds.iter_mut().zip(merged) {
if guild["members"].is_null() && members.is_array() {
guild["members"] = members.clone();
}
}
}
}
// And the presences: the friends' as READY's own `presences`, each guild's with the guild.
if let Some(merged) = ready["merged_presences"].as_object() {
if out["presences"].is_null() {
out["presences"] = merged.get("friends").cloned().unwrap_or_else(|| json!([]));
}
if let (Some(per_guild), Some(guilds)) = (merged.get("guilds").and_then(Value::as_array), out["guilds"].as_array_mut()) {
for (guild, presences) in guilds.iter_mut().zip(per_guild) {
if guild["presences"].is_null() && presences.is_array() {
guild["presences"] = presences.clone();
}
}
}
}
restore(&mut out, &users);
Some(out)
}
/// Gives every object that names a user by id the user itself, and every conversation that lists its recipients by id
/// the recipients, wherever in the payload they are.
fn restore(value: &mut Value, users: &HashMap<String, Value>) {
match value {
Value::Object(object) => {
fill(object, users);
for child in object.values_mut() {
restore(child, users);
}
}
Value::Array(list) => {
for child in list {
restore(child, users);
}
}
_ => {}
}
}
fn fill(object: &mut Map<String, Value>, users: &HashMap<String, Value>) {
if !object.contains_key("user") {
if let Some(user) = object.get("user_id").and_then(Value::as_str).and_then(|id| users.get(id)) {
object.insert("user".to_string(), user.clone());
}
}
if !object.contains_key("recipients") {
if let Some(ids) = object.get("recipient_ids").and_then(Value::as_array) {
let recipients: Vec<Value> = ids.iter().filter_map(|id| users.get(id.as_str()?).cloned()).collect();
object.insert("recipients".to_string(), Value::Array(recipients));
}
}
}
#[cfg(test)]
mod tests {
use super::*;
/// READY as the documentation describes it for a client with DEDUPE_USER_OBJECTS: users in a table, ids in their
/// place, members and presences merged.
fn dehydrated() -> Value {
json!({
"user": { "id": "1", "username": "me" },
"users": [
{ "id": "1", "username": "me" },
{ "id": "2", "username": "ada", "global_name": "Ada" },
{ "id": "3", "username": "bob" }
],
"relationships": [{ "id": "2", "type": 1, "user_id": "2" }, { "id": "3", "type": 3, "user_id": "3" }],
"private_channels": [{ "id": "20", "type": 1, "recipient_ids": ["2"], "last_message_id": null }, { "id": "21", "type": 3, "recipient_ids": ["2", "3"] }],
"merged_presences": { "friends": [{ "user_id": "2", "status": "online", "activities": [] }], "guilds": [[{ "user_id": "3", "status": "idle" }]] },
"guilds": [{ "id": "g", "voice_states": [{ "user_id": "3", "channel_id": "v" }] }],
"merged_members": [[{ "user_id": "1", "roles": ["r"], "pending": false }, { "user_id": "3", "roles": [] }]]
})
}
#[test]
fn a_dehydrated_ready_is_made_whole_again() {
let whole = rehydrate(&dehydrated()).expect("there was a users table");
// Friends and their requests, by the user object every reader looks for.
assert_eq!(whole["relationships"][0]["user"]["username"], "ada");
assert_eq!(whole["relationships"][1]["user"]["id"], "3");
// Conversations with the people in them.
assert_eq!(whole["private_channels"][0]["recipients"][0]["global_name"], "Ada");
assert_eq!(whole["private_channels"][1]["recipients"].as_array().unwrap().len(), 2);
// Presences where READY used to carry them.
assert_eq!(whole["presences"][0]["user"]["id"], "2");
assert_eq!(whole["presences"][0]["status"], "online");
assert_eq!(whole["guilds"][0]["presences"][0]["user"]["username"], "bob");
// The guild's members, and our own among them as `own_member` looks for it.
let members = whole["guilds"][0]["members"].as_array().unwrap();
assert_eq!(members.len(), 2);
assert_eq!(members[0]["user"]["id"], "1");
assert_eq!(members[0]["roles"][0], "r");
// Voice states, which name their user the same way.
assert_eq!(whole["guilds"][0]["voice_states"][0]["user"]["username"], "bob");
}
#[test]
fn a_ready_that_was_never_dehydrated_is_left_as_it_was() {
let whole = json!({ "user": { "id": "1" }, "relationships": [{ "id": "2", "user": { "id": "2" } }] });
assert!(rehydrate(&whole).is_none());
}
#[test]
fn what_is_already_there_is_not_overwritten_and_the_unknown_is_left_alone() {
let mut ready = dehydrated();
ready["relationships"][0]["user"] = json!({ "id": "2", "username": "already" });
ready["private_channels"][0]["recipient_ids"] = json!(["2", "999"]);
ready["guilds"][0]["members"] = json!([{ "user": { "id": "1" } }]);
let whole = rehydrate(&ready).unwrap();
assert_eq!(whole["relationships"][0]["user"]["username"], "already");
assert_eq!(whole["private_channels"][0]["recipients"].as_array().unwrap().len(), 1, "an id nobody told us about is not made up");
assert_eq!(whole["guilds"][0]["members"].as_array().unwrap().len(), 1, "a guild's own members are not replaced");
}
#[test]
fn the_whole_thing_can_be_done_twice() {
let once = rehydrate(&dehydrated()).unwrap();
let twice = rehydrate(&once).unwrap();
assert_eq!(once, twice);
}
}
src/backend/discord/rules.rs
+329
View File
@@ -0,0 +1,329 @@
//! What Discord's own rules say a message needs before it is worth sending - checked here, from what login and the
//! guild already told us, so a message that is bound to be turned down is not sent at all. A refused send is one more
//! request from an account that is being watched, and says nothing that a sentence cannot say sooner.
//!
//! The server rules are Discord's documented ones (and Concord's reading of them, `discord/verification.rs`): a
//! server's verification level asks for a verified email (low), an account older than five minutes (medium), a member
//! of ten minutes' standing (high) or a verified phone (very high); membership screening and onboarding must be done;
//! and a member who is timed out may not speak. The owner, anybody with every permission, anybody with a role (for
//! the levels) and anybody the server exempts (`BYPASSES_VERIFICATION`) are not held to the levels.
use std::time::{Duration, Instant};
/// Member flags (the `flags` of a guild member).
const FLAG_COMPLETED_ONBOARDING: u64 = 1 << 1;
const FLAG_BYPASSES_VERIFICATION: u64 = 1 << 2;
/// How long a member of a server wears Discord's green "new member" badge by their name: the first week.
pub const NEW_MEMBER_SECS: i64 = 7 * 24 * 60 * 60;
const ACCOUNT_AGE_SECS: i64 = 5 * 60;
const MEMBER_AGE_SECS: i64 = 10 * 60;
/// What a server asks of everyone, as its payload says.
#[derive(Debug, Clone, Default)]
pub struct GuildFacts {
/// 0 none, 1 low, 2 medium, 3 high, 4 very high.
pub verification: u8,
pub community: bool,
pub owner: bool,
}
/// This account's membership of it.
#[derive(Debug, Clone, Default)]
pub struct MemberFacts {
pub pending: Option<bool>,
pub flags: Option<u64>,
pub has_roles: bool,
/// When it joined, Unix seconds.
pub joined_at: Option<i64>,
/// Until when it may not speak (a timeout), Unix seconds.
pub timed_out_until: Option<i64>,
}
/// And the account itself.
#[derive(Debug, Clone, Default)]
pub struct AccountFacts {
pub email_verified: Option<bool>,
pub phone_verified: Option<bool>,
/// When it was made, Unix seconds (from its id).
pub created_at: i64,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum Restriction {
Screening,
Onboarding,
EmailRequired,
PhoneRequired,
AccountTooNew(i64),
MemberTooNew(i64),
TimedOut(i64),
/// Wearing the green new-member badge: for as many more seconds.
NewMember(i64),
}
impl Restriction {
/// The sentence a refused send gives.
pub fn sentence(&self) -> String {
match self {
Restriction::Screening => "This server asks you to accept its rules first. Do that from the notice above the conversation, or in Discord's own app.".into(),
Restriction::Onboarding => "This server asks you to finish its onboarding first - do that in Discord's own app.".into(),
Restriction::EmailRequired => "This server needs a verified email on your Discord account before you can talk here.".into(),
Restriction::PhoneRequired => "This server needs a verified phone number on your Discord account before you can talk here.".into(),
Restriction::AccountTooNew(s) => format!("This server asks for an account more than five minutes old: {}.", wait(*s)),
Restriction::MemberTooNew(s) => format!("This server asks you to have been a member for ten minutes before you talk: {}.", wait(*s)),
Restriction::TimedOut(s) => format!("You are timed out in this server: {}.", wait(*s)),
Restriction::NewMember(s) => format!(
"You joined this server recently, so Discord shows the green new-member badge by your name here, for about {} more. Messages from another client while it shows are treated as spam: use Discord's own app until it is gone.",
long_wait(*s)
),
}
}
/// Whether it is the kind that stays until the person does something, and so is worth a lock over the box. The
/// others pass by themselves, and are said when a send is tried. (Screening has a notice of its own.)
pub fn banner(&self) -> Option<&'static str> {
match self {
Restriction::Onboarding => Some("This server asks you to finish its onboarding first - do that in Discord's own app."),
Restriction::EmailRequired => Some("This server needs a verified email on your Discord account before you can talk here."),
Restriction::PhoneRequired => Some("This server needs a verified phone number on your Discord account before you can talk here."),
Restriction::NewMember(_) => Some("You are a new member of this server: Discord shows a green badge by your name for the first week, and treats messages from another client as spam while it shows. Use Discord's own app until it is gone; this opens again then."),
_ => None,
}
}
}
/// A wait counted in days or hours, for one that is long.
fn long_wait(seconds: i64) -> String {
let hours = (seconds.max(0) + 3599) / 3600;
if hours >= 48 {
format!("{} days", (hours + 23) / 24)
} else if hours > 1 {
format!("{hours} hours")
} else {
"an hour".to_string()
}
}
fn wait(seconds: i64) -> String {
if seconds >= 120 {
format!("wait {} more minutes", (seconds + 59) / 60)
} else {
format!("wait {} more seconds", seconds.max(1))
}
}
/// Whether the account may speak in a server now, and if not why. `full` is every permission (the owner, an
/// administrator); `onboarding_enabled` is what the server's onboarding says of itself, where it was asked.
pub fn participation(
guild: &GuildFacts,
member: Option<&MemberFacts>,
account: &AccountFacts,
full: bool,
onboarding_enabled: Option<bool>,
now: i64,
) -> Option<Restriction> {
if guild.owner {
return None;
}
// Not known is not blocked: a check that stops everything for want of data is a lock nobody can open.
let member = member?;
if let Some(until) = member.timed_out_until.filter(|u| *u > now) {
return Some(Restriction::TimedOut(until - now));
}
if member.pending == Some(true) {
return Some(Restriction::Screening);
}
if full {
return None;
}
let flags = member.flags.unwrap_or(0);
if flags & FLAG_BYPASSES_VERIFICATION != 0 {
return None;
}
if onboarding_enabled == Some(true) && flags & FLAG_COMPLETED_ONBOARDING == 0 {
return Some(Restriction::Onboarding);
}
if guild.verification == 0 || member.has_roles || account.phone_verified == Some(true) {
return None;
}
if guild.verification >= 4 {
return (account.phone_verified == Some(false)).then_some(Restriction::PhoneRequired);
}
if account.email_verified == Some(false) {
return Some(Restriction::EmailRequired);
}
if guild.verification >= 2 {
let age = now - account.created_at;
if account.created_at > 0 && age < ACCOUNT_AGE_SECS {
return Some(Restriction::AccountTooNew(ACCOUNT_AGE_SECS - age));
}
}
if guild.verification >= 3 {
if let Some(joined) = member.joined_at {
let age = now - joined;
if age < MEMBER_AGE_SECS {
return Some(Restriction::MemberTooNew(MEMBER_AGE_SECS - age));
}
}
}
None
}
/// Whether the account still wears the green new-member badge: it joined the server in the last week. Discord treats
/// what a person says from another client in that time as spam, so it is a reason not to speak from here, whatever
/// else the server allows. The owner has no such badge; `joined` is when the account joined if the member does not say.
pub fn new_member(guild: &GuildFacts, member: Option<&MemberFacts>, joined: Option<i64>, now: i64) -> Option<Restriction> {
if guild.owner {
return None;
}
let joined = member.and_then(|m| m.joined_at).or(joined)?;
let age = now - joined;
// A join time in the future is a clock that is wrong, and not a reason to lock anything.
(0..NEW_MEMBER_SECS).contains(&age).then_some(Restriction::NewMember(NEW_MEMBER_SECS - age))
}
/// How long until a channel's slowmode lets the account speak again: `seconds` is the channel's setting, `last` the
/// account's last message there, `bypass` the permissions that are not held to it (manage messages, manage channel).
pub fn slowmode_remaining(seconds: u64, last: Option<Instant>, bypass: bool, now: Instant) -> Option<Duration> {
if seconds == 0 || bypass {
return None;
}
let ends = last? + Duration::from_secs(seconds);
(ends > now).then(|| ends - now)
}
/// Own messages (not sent from here) a direct message needs before it counts as a conversation that is begun.
pub const ESTABLISHED_DM_MESSAGES: usize = 5;
#[cfg(test)]
mod tests {
use super::*;
fn guild(level: u8) -> GuildFacts {
GuildFacts { verification: level, community: false, owner: false }
}
fn account() -> AccountFacts {
AccountFacts { email_verified: Some(true), phone_verified: Some(false), created_at: 1_000_000 }
}
const NOW: i64 = 2_000_000;
#[test]
fn an_open_server_asks_nothing_of_a_member_in_good_standing() {
let member = MemberFacts { pending: Some(false), flags: Some(0), ..Default::default() };
assert_eq!(participation(&guild(0), Some(&member), &account(), false, None, NOW), None);
}
#[test]
fn each_level_asks_for_what_it_says() {
let member = MemberFacts { pending: Some(false), flags: Some(0), joined_at: Some(NOW - 60), ..Default::default() };
let mut a = account();
a.email_verified = Some(false);
assert_eq!(participation(&guild(1), Some(&member), &a, false, None, NOW), Some(Restriction::EmailRequired));
a.email_verified = Some(true);
assert_eq!(participation(&guild(1), Some(&member), &a, false, None, NOW), None);
// Medium: the account must be over five minutes old.
a.created_at = NOW - 100;
assert_eq!(participation(&guild(2), Some(&member), &a, false, None, NOW), Some(Restriction::AccountTooNew(200)));
a.created_at = 1_000_000;
assert_eq!(participation(&guild(2), Some(&member), &a, false, None, NOW), None);
// High: and the membership over ten minutes.
assert_eq!(participation(&guild(3), Some(&member), &a, false, None, NOW), Some(Restriction::MemberTooNew(540)));
let old = MemberFacts { joined_at: Some(NOW - 700), ..member.clone() };
assert_eq!(participation(&guild(3), Some(&old), &a, false, None, NOW), None);
// Very high: a phone, and nothing else stands in for it but a role or an exemption.
assert_eq!(participation(&guild(4), Some(&old), &a, false, None, NOW), Some(Restriction::PhoneRequired));
a.phone_verified = Some(true);
assert_eq!(participation(&guild(4), Some(&old), &a, false, None, NOW), None);
}
#[test]
fn the_owner_a_role_and_an_exemption_are_not_held_to_the_levels() {
let mut a = account();
a.phone_verified = Some(false);
let plain = MemberFacts { pending: Some(false), flags: Some(0), joined_at: Some(NOW - 5), ..Default::default() };
assert!(participation(&guild(4), Some(&plain), &a, false, None, NOW).is_some());
let owner = GuildFacts { owner: true, ..guild(4) };
assert_eq!(participation(&owner, Some(&plain), &a, false, None, NOW), None);
let roled = MemberFacts { has_roles: true, ..plain.clone() };
assert_eq!(participation(&guild(4), Some(&roled), &a, false, None, NOW), None);
let exempt = MemberFacts { flags: Some(1 << 2), ..plain.clone() };
assert_eq!(participation(&guild(4), Some(&exempt), &a, false, None, NOW), None);
assert_eq!(participation(&guild(4), Some(&plain), &a, true, None, NOW), None, "every permission");
}
#[test]
fn screening_onboarding_and_a_timeout_come_before_the_levels() {
let a = account();
let pending = MemberFacts { pending: Some(true), ..Default::default() };
assert_eq!(participation(&guild(0), Some(&pending), &a, false, None, NOW), Some(Restriction::Screening));
let member = MemberFacts { pending: Some(false), flags: Some(0), ..Default::default() };
assert_eq!(participation(&guild(0), Some(&member), &a, false, Some(true), NOW), Some(Restriction::Onboarding));
assert_eq!(participation(&guild(0), Some(&member), &a, false, Some(false), NOW), None);
let done = MemberFacts { flags: Some(1 << 1), ..member.clone() };
assert_eq!(participation(&guild(0), Some(&done), &a, false, Some(true), NOW), None);
let timed = MemberFacts { timed_out_until: Some(NOW + 300), ..member };
assert_eq!(participation(&guild(0), Some(&timed), &a, true, None, NOW), Some(Restriction::TimedOut(300)), "even with every permission");
let over = MemberFacts { timed_out_until: Some(NOW - 5), ..timed };
assert_eq!(participation(&guild(0), Some(&over), &a, false, None, NOW), None);
}
#[test]
fn what_is_not_known_is_not_a_block() {
assert_eq!(participation(&guild(4), None, &account(), false, None, NOW), None);
let unknown = AccountFacts { email_verified: None, phone_verified: None, created_at: 0 };
let member = MemberFacts { pending: Some(false), flags: Some(0), joined_at: Some(NOW - 5), ..Default::default() };
assert_eq!(participation(&guild(4), Some(&member), &unknown, false, None, NOW), None);
assert_eq!(participation(&guild(1), Some(&member), &unknown, false, None, NOW), None);
}
#[test]
fn a_member_of_under_a_week_wears_the_badge_until_the_week_is_up() {
let day = 24 * 60 * 60;
let member = |ago: i64| MemberFacts { joined_at: Some(NOW - ago), ..Default::default() };
assert_eq!(new_member(&guild(0), Some(&member(60)), None, NOW), Some(Restriction::NewMember(7 * day - 60)));
assert_eq!(new_member(&guild(0), Some(&member(6 * day)), None, NOW), Some(Restriction::NewMember(day)));
assert_eq!(new_member(&guild(0), Some(&member(7 * day)), None, NOW), None, "gone at a week");
assert_eq!(new_member(&guild(0), Some(&member(90 * day)), None, NOW), None);
}
#[test]
fn the_badge_is_not_worn_by_the_owner_or_by_a_member_nothing_is_known_of() {
let fresh = MemberFacts { joined_at: Some(NOW - 5), ..Default::default() };
assert_eq!(new_member(&GuildFacts { owner: true, ..guild(0) }, Some(&fresh), None, NOW), None);
assert_eq!(new_member(&guild(0), None, None, NOW), None, "not known is not blocked");
assert_eq!(new_member(&guild(0), Some(&MemberFacts::default()), None, NOW), None);
// The server's own join time stands in for a member object that does not carry one.
assert!(new_member(&guild(0), Some(&MemberFacts::default()), Some(NOW - 5), NOW).is_some());
assert_eq!(new_member(&guild(0), Some(&MemberFacts { joined_at: Some(NOW + 500), ..Default::default() }), None, NOW), None, "a join in the future is a wrong clock");
}
#[test]
fn the_badge_says_how_long_in_days_and_is_a_lock_over_the_box() {
let day = 24 * 60 * 60;
assert!(Restriction::NewMember(3 * day).sentence().contains("3 days"));
assert!(Restriction::NewMember(5 * 3600).sentence().contains("5 hours"));
assert!(Restriction::NewMember(600).sentence().contains("an hour"));
assert!(Restriction::NewMember(day).banner().is_some(), "it lasts days, so it is a lock and not a thing said when a send is tried");
}
#[test]
fn slowmode_lets_the_account_speak_when_it_is_over() {
let t = Instant::now();
let s = Duration::from_secs;
assert_eq!(slowmode_remaining(0, Some(t), false, t), None);
assert_eq!(slowmode_remaining(10, None, false, t), None, "nothing said yet");
assert_eq!(slowmode_remaining(10, Some(t), false, t + s(3)), Some(s(7)));
assert_eq!(slowmode_remaining(10, Some(t), false, t + s(10)), None);
assert_eq!(slowmode_remaining(10, Some(t), true, t + s(3)), None, "manage messages is not held to it");
}
#[test]
fn the_sentences_say_how_long_to_wait() {
assert!(Restriction::AccountTooNew(200).sentence().contains("4 more minutes"));
assert!(Restriction::MemberTooNew(30).sentence().contains("30 more seconds"));
assert!(Restriction::TimedOut(1).sentence().contains("1 more seconds") || Restriction::TimedOut(1).sentence().contains("wait"));
assert!(Restriction::Onboarding.banner().is_some() && Restriction::AccountTooNew(5).banner().is_none());
}
}
src/backend/discord/search.rs
+1 -1
View File
@@ -103,7 +103,7 @@ pub async fn search_messages(
.get(url)
.query(&query)
.header("Authorization", &cfg.token)
.send()
.send_paced()
.await
.context("searching")?;
// Discord answers a search of a server it is still indexing with 202 and
src/backend/discord/send.rs
+433 -44
View File
@@ -35,6 +35,33 @@ pub(super) fn put_reply(payload: &mut Value, reply_to_id: Option<&str>, ping_aut
}
}
/// Says where the message answered is, as Discord's clients do: the channel, and the server where there is one. The id
/// alone is enough for the API, and is not what its own clients send.
pub(super) fn put_reply_place(payload: &mut Value, channel_id: &str, guild_id: Option<&str>) {
if payload["message_reference"].is_object() {
payload["message_reference"]["channel_id"] = Value::from(channel_id);
if let Some(guild) = guild_id {
payload["message_reference"]["guild_id"] = Value::from(guild);
}
}
}
/// Said when Discord refuses a message that creates one: a captcha closes the place for a while (moho does not answer
/// those, and trying again is what escalates), a channel's slowmode says how long is left, anything else is as
/// `refusal_text` has it.
pub(super) fn refused(state: &AppState, buffer_id: &str, status: reqwest::StatusCode, text: &str) -> String {
let parsed: Value = serde_json::from_str(text).unwrap_or(Value::Null);
if parsed.get("captcha_key").is_some() {
super::send_guard::hold_captcha(state, buffer_id);
return "Discord asked for a captcha before this message. moho does not answer Discord's captchas - trying again is what gets an account blocked - so this place is closed to it for a while: say something here from Discord's own app.".to_string();
}
if parsed["code"].as_u64() == Some(20016) {
let seconds = parsed["retry_after"].as_f64().unwrap_or(0.0).ceil() as u64;
return format!("This channel has slowmode on: wait {} more seconds.", seconds.max(1));
}
refusal_text(status, text)
}
/// REST message send - Discord's gateway is receive-only from the client's
/// perspective for user accounts; sending is always a plain HTTP POST.
/// Turns the names somebody typed into the mentions Discord understands.
@@ -220,6 +247,8 @@ pub async fn forward_message(
.runtime
.get_discord_channel(to_buffer)
.ok_or_else(|| anyhow!("there is no Discord conversation to send it to"))?;
super::send_guard::ensure_may_speak(state, to_buffer).await?;
let _turn = super::send_guard::channel_turn(&target_channel).await;
let mut reference = json!({
// 1 is Discord's own number for a forward, as against 0 for a reply.
"type": 1,
@@ -235,15 +264,21 @@ pub async fn forward_message(
http_client_for(token)
.post(format!("{API_BASE}/channels/{target_channel}/messages"))
.header("Authorization", token)
.json(&json!({ "content": "", "message_reference": reference })),
.json(&{
let mut payload = json!({ "content": "", "message_reference": reference });
super::send_guard::stamp(&mut payload);
payload
}),
)
.await
.context("forwarding the message")?;
if !resp.status().is_success() {
let status = resp.status();
let text = resp.text().await.unwrap_or_default();
bail!("{}", discord_error_text(status, &text, "forwarding that message"));
bail!("{}", refused(state, to_buffer, status, &text));
}
super::send_guard::record_own_send(to_buffer);
super::send_guard::note_sent(resp).await;
Ok(())
}
@@ -252,10 +287,18 @@ pub async fn send_message(state: &AppState, buffer_id: &str, token: &str, body:
.runtime
.get_discord_channel(buffer_id)
.ok_or_else(|| anyhow!("no known Discord channel for this buffer"))?;
// Nothing is said anywhere the account has not spoken before: see send_guard.
super::send_guard::ensure_may_speak(state, buffer_id).await?;
let account_id = state.runtime.get_buffer(buffer_id).map(|b| b.account_id).unwrap_or_default();
let content = resolve_outgoing_mentions(body, mention_candidates(state, &account_id, buffer_id));
super::send_guard::check_text(&content, state.runtime.discord_premium(&account_id), false)?;
let mut payload = json!({ "content": content });
put_reply(&mut payload, reply_to_id, reply_ping);
put_reply_place(&mut payload, &channel_id, state.runtime.get_discord_guild(buffer_id).as_deref());
super::send_guard::stamp(&mut payload);
// One send at a time to a channel, and each a moment after the last.
let _turn = super::send_guard::channel_turn(&channel_id).await;
super::send_guard::take_turn(&account_id, false).await;
let resp = send_write(
http_client_for(token)
.post(format!("{API_BASE}/channels/{channel_id}/messages"))
@@ -267,8 +310,10 @@ pub async fn send_message(state: &AppState, buffer_id: &str, token: &str, body:
if !resp.status().is_success() {
let status = resp.status();
let text = resp.text().await.unwrap_or_default();
bail!("{}", refusal_text(status, &text));
bail!("{}", refused(state, buffer_id, status, &text));
}
super::send_guard::record_own_send(buffer_id);
super::send_guard::note_sent(resp).await;
Ok(())
}
@@ -371,6 +416,8 @@ pub async fn send_attachments(
.runtime
.get_discord_channel(buffer_id)
.ok_or_else(|| anyhow!("no known Discord channel for this buffer"))?;
super::send_guard::ensure_may_speak(state, buffer_id).await?;
let _turn = super::send_guard::channel_turn(&channel_id).await;
if let Some(p) = progress {
p.at(Phase::Preparing, 0, Meter::HOST);
}
@@ -385,44 +432,197 @@ pub async fn send_attachments(
total += len;
files.push((path.clone(), name, len));
}
// What Discord would refuse is refused here, before any of it is sent.
super::send_guard::ensure_fits(state, buffer_id, &files)?;
// A caption is a message like any other, so a name typed in one is a
// mention like any other.
let account_id = state.runtime.get_buffer(buffer_id).map(|b| b.account_id).unwrap_or_default();
let content = resolve_outgoing_mentions(body, mention_candidates(state, &account_id, buffer_id));
super::send_guard::check_text(&content, state.runtime.discord_premium(&account_id), true)?;
let mut payload = json!({ "content": content });
put_reply(&mut payload, reply_to_id, reply_ping);
put_reply_place(&mut payload, &channel_id, state.runtime.get_discord_guild(buffer_id).as_deref());
super::send_guard::stamp(&mut payload);
// A file does not follow straight on from a message.
super::send_guard::take_turn(&account_id, true).await;
if let Some(p) = progress {
p.sent(0, total, Meter::HOST);
}
let client = http_client_for(token);
let url = format!("{API_BASE}/channels/{channel_id}/messages");
let resp = send_write_rebuilt(|| {
// Built afresh for every try: a streamed body is used up by sending
// it, so a rate-limited request has to open its files again.
let meter = std::sync::Arc::new(Meter {
sent: std::sync::atomic::AtomicU64::new(0),
total,
last: std::sync::Mutex::new((std::time::Instant::now(), 0)),
progress: progress.cloned(),
gap: Meter::GAP,
});
let mut form = reqwest::multipart::Form::new().text("payload_json", payload.to_string());
for (i, (path, name, len)) in files.iter().enumerate() {
form = form.part(format!("files[{i}]"), counted_part(path, name, *len, meter.clone())?);
let meter = std::sync::Arc::new(Meter {
sent: std::sync::atomic::AtomicU64::new(0),
total,
last: std::sync::Mutex::new((std::time::Instant::now(), 0)),
progress: progress.cloned(),
gap: Meter::GAP,
});
let uploads: Vec<Upload> = files.iter().map(|(path, name, len)| Upload { path: path.clone(), name: name.clone(), len: *len, voice: None }).collect();
let resp = match upload_in_steps(state, token, &channel_id, &uploads, &mut payload, &meter).await? {
Some(resp) => resp,
// Where the way Discord's own apps send files is not there, the one bots use is.
None => {
let client = http_client_for(token);
let url = format!("{API_BASE}/channels/{channel_id}/messages");
for key in ["attachments", "type", "sticker_ids", "channel_id"] {
if let Some(object) = payload.as_object_mut() {
object.remove(key);
}
}
send_write_rebuilt(|| {
// Built afresh for every try: a streamed body is used up by sending it, so a rate-limited request has
// to open its files again.
let meter = std::sync::Arc::new(Meter {
sent: std::sync::atomic::AtomicU64::new(0),
total,
last: std::sync::Mutex::new((std::time::Instant::now(), 0)),
progress: progress.cloned(),
gap: Meter::GAP,
});
let mut form = reqwest::multipart::Form::new().text("payload_json", payload.to_string());
for (i, (path, name, len)) in files.iter().enumerate() {
form = form.part(format!("files[{i}]"), counted_part(path, name, *len, meter.clone())?);
}
Ok(client.post(&url).header("Authorization", token).multipart(form))
})
.await
.context("uploading Discord attachment")?
}
Ok(client.post(&url).header("Authorization", token).multipart(form))
})
.await
.context("uploading Discord attachment")?;
};
if !resp.status().is_success() {
let status = resp.status();
let text = resp.text().await.unwrap_or_default();
bail!("{}", refusal_text(status, &text));
bail!("{}", refused(state, buffer_id, status, &text));
}
super::send_guard::record_own_send(buffer_id);
super::send_guard::note_sent(resp).await;
Ok(())
}
/// A file on its way, and what it is when it is a voice message.
struct Upload {
path: String,
name: String,
len: u64,
/// A voice message's duration and waveform.
voice: Option<(f64, String)>,
}
/// What Discord answered to a request for somewhere to put the files: for each, by its place in the request, where to
/// send its bytes and the name to give the message. None when the answer is not that.
fn parse_slots(body: &Value, count: usize) -> Option<Vec<(String, String)>> {
let list = body["attachments"].as_array()?;
let mut slots: Vec<Option<(String, String)>> = vec![None; count];
for slot in list {
let id = slot["id"].as_u64().or_else(|| slot["id"].as_str().and_then(|s| s.parse().ok()))? as usize;
let url = slot["upload_url"].as_str().filter(|u| u.starts_with("https://"))?;
let name = slot["upload_filename"].as_str().filter(|n| !n.is_empty())?;
*slots.get_mut(id)? = Some((url.to_string(), name.to_string()));
}
slots.into_iter().collect()
}
/// The `attachments` a message names once its files are uploaded.
fn attachments_named(uploads: &[Upload], slots: &[(String, String)]) -> Value {
Value::Array(
uploads
.iter()
.zip(slots)
.enumerate()
.map(|(i, (upload, (_, uploaded)))| {
let mut entry = json!({ "id": i.to_string(), "filename": upload.name, "uploaded_filename": uploaded });
if let Some((duration, waveform)) = &upload.voice {
entry["duration_secs"] = json!(duration);
entry["waveform"] = json!(waveform);
}
entry
})
.collect(),
)
}
/// The body of an upload: the file, read from disk as it goes, counted by `meter`.
fn counted_body(path: &str, meter: std::sync::Arc<Meter>) -> Result<reqwest::Body> {
let file = std::fs::File::open(path).with_context(|| format!("reading {path}"))?;
let stream = tokio_util::io::ReaderStream::with_capacity(tokio::fs::File::from_std(file), 64 * 1024);
let counted = futures::StreamExt::map(stream, move |chunk| {
if let Ok(bytes) = &chunk {
meter.add(bytes.len());
}
chunk
});
Ok(reqwest::Body::wrap_stream(counted))
}
/// Sends files the way Discord's own apps do: ask where to put them (a file too big is refused here, before a byte
/// goes), put each there, then send the message with the files named - not the way bots do, with the bytes inside the
/// message. Returns the answer to the message, or `None` when Discord has no such way to offer (the endpoint is
/// missing, or answers something that is not slots) and the old one is to be used.
async fn upload_in_steps(
state: &AppState,
token: &str,
channel_id: &str,
uploads: &[Upload],
payload: &mut Value,
meter: &std::sync::Arc<Meter>,
) -> Result<Option<reqwest::Response>> {
let _ = state;
let request = json!({
"files": uploads.iter().enumerate().map(|(i, u)| json!({ "filename": u.name, "file_size": u.len, "id": i.to_string() })).collect::<Vec<_>>(),
});
let resp = send_write(
http_client_for(token).post(format!("{API_BASE}/channels/{channel_id}/attachments")).header("Authorization", token).json(&request),
)
.await
.context("asking Discord where to put the files")?;
let status = resp.status();
if matches!(status.as_u16(), 404 | 405 | 501) {
return Ok(None);
}
if status.as_u16() == 413 {
bail!("Discord says a file is too big to send here");
}
if !status.is_success() {
let text = resp.text().await.unwrap_or_default();
bail!("{}", discord_error_text(status, &text, "asking where to put the files"));
}
let body: Value = resp.json().await.context("reading where Discord said to put the files")?;
let Some(slots) = parse_slots(&body, uploads.len()) else { return Ok(None) };
// The bytes go to the address Discord gave, which is another host: none of Discord's headers go with them.
let storage = bare_client_for(token);
for (upload, (url, _)) in uploads.iter().zip(&slots) {
let put = storage
.put(url)
.header(reqwest::header::CONTENT_TYPE, "application/octet-stream")
.header(reqwest::header::CONTENT_LENGTH, upload.len)
.body(counted_body(&upload.path, meter.clone())?)
.send()
.await
.with_context(|| format!("uploading {}", upload.name))?;
if !put.status().is_success() {
bail!("Discord's storage refused {} ({})", upload.name, put.status());
}
}
payload["attachments"] = attachments_named(uploads, &slots);
payload["type"] = json!(0);
payload["sticker_ids"] = json!([]);
payload["channel_id"] = json!(channel_id);
// A message that names its files is not sent as a bare one.
if let Some(object) = payload.as_object_mut() {
object.remove("tts");
if object.get("flags") == Some(&json!(0)) {
object.remove("flags");
}
}
let sent = send_write(
http_client_for(token).post(format!("{API_BASE}/channels/{channel_id}/messages")).header("Authorization", token).json(&*payload),
)
.await
.context("sending the message with the files")?;
Ok(Some(sent))
}
/// Sending a recording as a voice message rather than as a sound file.
///
/// Three things make Discord treat it as one, and all three are required: the
@@ -452,6 +652,7 @@ pub async fn send_voice_message(
.runtime
.get_discord_channel(buffer_id)
.ok_or_else(|| anyhow!("no known Discord channel for this buffer"))?;
super::send_guard::ensure_may_speak(state, buffer_id).await?;
let bytes = tokio::fs::read(file_path)
.await
.with_context(|| format!("reading {}", file_path.display()))?;
@@ -459,7 +660,7 @@ pub async fn send_voice_message(
// than on this, but a file that arrives anywhere else should say what it
// is - and an unnamed part is rejected outright.
let file_name = "voice-message.ogg";
let payload = json!({
let mut payload = json!({
"content": "",
"flags": IS_VOICE_MESSAGE,
"attachments": [{
@@ -469,28 +670,55 @@ pub async fn send_voice_message(
"waveform": waveform,
}],
});
let form = reqwest::multipart::Form::new()
.text("payload_json", payload.to_string())
.part(
"files[0]",
reqwest::multipart::Part::bytes(bytes)
.file_name(file_name)
.mime_str("audio/ogg")
.context("audio/ogg is a valid mime type")?,
);
let resp = send_write(
http_client_for(token)
.post(format!("{API_BASE}/channels/{channel_id}/messages"))
.header("Authorization", token)
.multipart(form),
)
.await
.context("uploading the voice message")?;
super::send_guard::stamp(&mut payload);
let _turn = super::send_guard::channel_turn(&channel_id).await;
let account_id = state.runtime.get_buffer(buffer_id).map(|b| b.account_id).unwrap_or_default();
super::send_guard::take_turn(&account_id, true).await;
// Sent the way Discord's own apps send a recording, with the old way (the bytes inside the message) where that is
// not there.
let meter = std::sync::Arc::new(Meter {
sent: std::sync::atomic::AtomicU64::new(0),
total: bytes.len() as u64,
last: std::sync::Mutex::new((std::time::Instant::now(), 0)),
progress: None,
gap: Meter::GAP,
});
let upload = Upload { path: file_path.to_string_lossy().into_owned(), name: file_name.to_string(), len: bytes.len() as u64, voice: Some((duration_secs, waveform.to_string())) };
let resp = match upload_in_steps(state, token, &channel_id, std::slice::from_ref(&upload), &mut payload, &meter).await? {
Some(resp) => resp,
None => {
for key in ["type", "sticker_ids", "channel_id"] {
if let Some(object) = payload.as_object_mut() {
object.remove(key);
}
}
payload["attachments"] = json!([{ "id": "0", "filename": file_name, "duration_secs": duration_secs, "waveform": waveform }]);
let form = reqwest::multipart::Form::new()
.text("payload_json", payload.to_string())
.part(
"files[0]",
reqwest::multipart::Part::bytes(bytes)
.file_name(file_name)
.mime_str("audio/ogg")
.context("audio/ogg is a valid mime type")?,
);
send_write(
http_client_for(token)
.post(format!("{API_BASE}/channels/{channel_id}/messages"))
.header("Authorization", token)
.multipart(form),
)
.await
.context("uploading the voice message")?
}
};
if !resp.status().is_success() {
let status = resp.status();
let text = resp.text().await.unwrap_or_default();
bail!("{}", refusal_text(status, &text));
bail!("{}", refused(state, buffer_id, status, &text));
}
super::send_guard::record_own_send(buffer_id);
super::send_guard::note_sent(resp).await;
Ok(())
}
@@ -795,6 +1023,12 @@ pub async fn send_typing(state: &AppState, buffer_id: &str, token: &str) -> Resu
.runtime
.get_discord_channel(buffer_id)
.ok_or_else(|| anyhow!("no known Discord channel for this buffer"))?;
// The window repeats its notice every few seconds for every service, and
// a Discord indicator stands for ten, so most of those would only be
// noise on the wire: one a channel per `TYPING_EVERY` is plenty.
if !typing_due(&channel_id) {
return Ok(());
}
send_write(
http_client_for(token)
.post(format!("{API_BASE}/channels/{channel_id}/typing"))
@@ -806,6 +1040,24 @@ pub async fn send_typing(state: &AppState, buffer_id: &str, token: &str) -> Resu
Ok(())
}
/// Shortest gap between two typing notices to the same channel. Short of the
/// ten seconds an indicator lasts, so it never lapses mid-sentence.
const TYPING_EVERY: std::time::Duration = std::time::Duration::from_secs(8);
/// Whether a typing notice to this channel is due, noting it if so.
fn typing_due(channel_id: &str) -> bool {
static LAST: std::sync::Mutex<Option<HashMap<String, std::time::Instant>>> = std::sync::Mutex::new(None);
let mut last = LAST.lock().unwrap();
let last = last.get_or_insert_with(HashMap::new);
let now = std::time::Instant::now();
if last.get(channel_id).is_some_and(|at| now.duration_since(*at) < TYPING_EVERY) {
return false;
}
last.retain(|_, at| now.duration_since(*at) < TYPING_EVERY);
last.insert(channel_id.to_string(), now);
true
}
/// Tells Discord this conversation has been read up to its newest message.
///
/// Unread was purely local before this, so reading a channel here did not
@@ -823,16 +1075,27 @@ pub async fn ack_read(state: &AppState, buffer_id: &str, token: &str) -> Result<
.runtime
.get_discord_channel(buffer_id)
.ok_or_else(|| anyhow!("no known Discord channel for this buffer"))?;
let Some(msg_id) = state.store.newest_msg_id(buffer_id)? else { return Ok(()) };
let Some((msg_id, own)) = state.store.newest_msg(buffer_id)? else { return Ok(()) };
// Nothing to tell Discord about a conversation whose last word is the account's own: sending a message reads
// everything before it, on every device, and Discord's own clients do not ack what they sent.
if own {
return Ok(());
}
// Reading a channel again with nothing new in it says nothing new.
if !ack_due(&channel_id, &msg_id) {
return Ok(());
}
let resp = send_write(
http_client_for(token)
.post(format!("{API_BASE}/channels/{channel_id}/messages/{msg_id}/ack"))
.header("Authorization", token)
.json(&json!({ "token": serde_json::Value::Null }))
.json(&json!({ "token": serde_json::Value::Null, "last_viewed": last_viewed_day() }))
)
.await
.inspect_err(|_| forget_ack(&channel_id))
.context("acking Discord read state")?;
if !resp.status().is_success() {
forget_ack(&channel_id);
let status = resp.status();
let text = resp.text().await.unwrap_or_default();
bail!("{}", refusal_text(status, &text));
@@ -840,6 +1103,55 @@ pub async fn ack_read(state: &AppState, buffer_id: &str, token: &str) -> Result<
Ok(())
}
/// The day, counted from Discord's epoch (2015-01-01) and rounded up, that a read mark says the channel was last
/// looked at: Discord's own clients send it with every ack.
pub(super) fn last_viewed_day() -> i64 {
last_viewed_day_at(chrono::Utc::now().timestamp())
}
pub(super) fn last_viewed_day_at(unix_secs: i64) -> i64 {
const DISCORD_EPOCH_SECS: i64 = 1_420_070_400;
(unix_secs - DISCORD_EPOCH_SECS + 86_399) / 86_400
}
/// The newest message each channel was last acked up to, this run.
static ACKED: std::sync::Mutex<Option<HashMap<String, String>>> = std::sync::Mutex::new(None);
/// Whether acking `msg_id` in this channel would say anything new, noting it
/// if so.
fn ack_due(channel_id: &str, msg_id: &str) -> bool {
let mut acked = ACKED.lock().unwrap();
let acked = acked.get_or_insert_with(HashMap::new);
if acked.get(channel_id).is_some_and(|id| id == msg_id) {
return false;
}
acked.insert(channel_id.to_string(), msg_id.to_string());
true
}
/// Another client of the account read this far: no need to say it again from here.
pub(super) fn acked_elsewhere(channel_id: &str, msg_id: &str) {
let mut acked = ACKED.lock().unwrap();
// Never backwards: an ack from a device that is behind does not make this one's later ack redundant.
let newer = match acked.get_or_insert_with(HashMap::new).get(channel_id) {
Some(known) => match (known.parse::<u64>(), msg_id.parse::<u64>()) {
(Ok(k), Ok(m)) => m > k,
_ => true,
},
None => true,
};
if newer {
acked.get_or_insert_with(HashMap::new).insert(channel_id.to_string(), msg_id.to_string());
}
}
/// A refused ack must be tried again, not remembered as done.
fn forget_ack(channel_id: &str) {
if let Some(acked) = ACKED.lock().unwrap().as_mut() {
acked.remove(channel_id);
}
}
pub async fn delete_message(state: &AppState, buffer_id: &str, token: &str, msg_id: &str) -> Result<()> {
let channel_id = state
.runtime
@@ -949,3 +1261,80 @@ mod refusal_tests {
assert_eq!(describe_seconds(45), "45 seconds");
}
}
#[cfg(test)]
mod ack_tests {
use super::*;
#[test]
fn an_ack_made_elsewhere_is_not_made_again_and_never_moves_backwards() {
// Own ids for this test: the table is shared by the process.
let channel = "ack-test-channel";
acked_elsewhere(channel, "200");
assert!(!ack_due(channel, "200"), "read this far on another device");
assert!(ack_due(channel, "300"), "something newer is still to be told");
acked_elsewhere(channel, "250");
assert!(!ack_due(channel, "300"), "an older ack from elsewhere does not undo what was told");
}
}
#[cfg(test)]
mod wire_tests {
use super::*;
#[test]
fn a_reply_says_where_the_message_it_answers_is() {
let mut payload = json!({ "content": "hi" });
put_reply(&mut payload, Some("42"), true);
put_reply_place(&mut payload, "7", Some("9"));
assert_eq!(payload["message_reference"], json!({ "message_id": "42", "channel_id": "7", "guild_id": "9" }));
let mut dm = json!({ "content": "hi" });
put_reply(&mut dm, Some("42"), true);
put_reply_place(&mut dm, "7", None);
assert_eq!(dm["message_reference"], json!({ "message_id": "42", "channel_id": "7" }));
// Not a reply: nothing is added.
let mut plain = json!({ "content": "hi" });
put_reply_place(&mut plain, "7", Some("9"));
assert!(plain.get("message_reference").is_none());
}
#[test]
fn a_read_mark_counts_days_from_discords_epoch_rounded_up() {
assert_eq!(last_viewed_day_at(1_420_070_400), 0);
assert_eq!(last_viewed_day_at(1_420_070_401), 1);
// 2026-10-22 00:00 UTC is 4,191 days after 2015-01-01.
assert_eq!(last_viewed_day_at(1_420_070_400 + 4_191 * 86_400), 4_191);
}
#[test]
fn where_to_put_the_files_is_read_from_the_answer() {
let body = json!({ "attachments": [
{ "id": 1, "upload_url": "https://storage.example/b?sig=2", "upload_filename": "u/b.png" },
{ "id": "0", "upload_url": "https://storage.example/a?sig=1", "upload_filename": "u/a.png" }
] });
let slots = parse_slots(&body, 2).unwrap();
assert_eq!(slots[0], ("https://storage.example/a?sig=1".to_string(), "u/a.png".to_string()));
assert_eq!(slots[1].1, "u/b.png");
// Anything that is not a slot for every file is not an answer to use.
assert!(parse_slots(&body, 3).is_none(), "one missing");
assert!(parse_slots(&json!({ "attachments": [{ "id": 0, "upload_url": "http://insecure/x", "upload_filename": "n" }] }), 1).is_none(), "plain http");
assert!(parse_slots(&json!({ "message": "no" }), 1).is_none());
assert!(parse_slots(&json!({ "attachments": [{ "id": 5, "upload_url": "https://x/y", "upload_filename": "n" }] }), 1).is_none(), "an id out of range");
}
#[test]
fn the_message_names_each_file_and_a_voice_message_says_how_long_it_is() {
let uploads = vec![
Upload { path: "/a".into(), name: "a.png".into(), len: 3, voice: None },
Upload { path: "/v".into(), name: "voice-message.ogg".into(), len: 9, voice: Some((2.5, "AAAA".into())) },
];
let slots = vec![("https://s/1".to_string(), "u/a.png".to_string()), ("https://s/2".to_string(), "u/v.ogg".to_string())];
let named = attachments_named(&uploads, &slots);
assert_eq!(named[0], json!({ "id": "0", "filename": "a.png", "uploaded_filename": "u/a.png" }));
assert_eq!(named[1]["id"], "1");
assert_eq!(named[1]["uploaded_filename"], "u/v.ogg");
assert_eq!(named[1]["duration_secs"], json!(2.5));
assert_eq!(named[1]["waveform"], "AAAA");
}
}
src/backend/discord/send_guard.rs
+1249
View File
File diff suppressed because it is too large. Load diff
src/backend/discord/session_store.rs
+304
View File
@@ -0,0 +1,304 @@
//! A gateway session kept on disk, so a restart can pick it up instead of logging in afresh.
//!
//! Discord keeps a dropped session for a few minutes and replays what it missed to a client that resumes it; its own
//! client resumes only when the last heartbeat was acknowledged under three minutes ago, and logs in afresh otherwise.
//! A restart of moho is a dropped connection like any other, so it does the same - for which the session has to
//! survive the process: what to resume (`Position`), and what a resume does *not* send again, because Discord sends
//! READY and GUILD_CREATE only to a fresh login (`State`). The state is replayed through the very code that handles
//! them live, so a resumed start ends up with the same friends, servers, mutes and plan as a fresh one.
//!
//! Two files per account, because the position changes every heartbeat and the state hardly ever.
use serde::{Deserialize, Serialize};
use serde_json::Value;
use sha2::{Digest, Sha256};
use std::collections::HashMap;
use std::path::PathBuf;
/// How old the last acknowledged heartbeat may be for a session to be worth resuming. Discord's client uses three
/// minutes; a little under, since connecting takes time and a refused resume costs a fresh login.
pub const FRESH_FOR: i64 = 150;
/// Past this the state is not kept (and so nothing is resumed): a few huge servers are not worth a file that size
/// rewritten on every change.
const MAX_STATE_BYTES: usize = 24 * 1024 * 1024;
#[derive(Serialize, Deserialize, Clone)]
pub struct Position {
pub user_id: String,
/// Which token this session belongs to, as a hash - the token itself is not written here.
pub token: String,
pub session_id: String,
pub url: String,
pub seq: u64,
/// When a heartbeat was last acknowledged, Unix seconds.
pub acked_at: i64,
}
#[derive(Serialize, Deserialize, Default)]
pub struct State {
pub session_id: String,
/// READY's payload, with the direct messages found over REST added as `moho_dms`.
pub ready: Value,
/// Each server's GUILD_CREATE payload, by id.
pub guilds: HashMap<String, Value>,
}
/// One account's session files, and what is known of them in memory.
#[derive(Default)]
pub struct SessionFile {
/// Where the files go; the daemon's data directory unless a test says otherwise.
root: Option<PathBuf>,
account: String,
user_id: String,
token: String,
position: Option<Position>,
state: Option<State>,
state_dirty: bool,
}
pub fn token_hash(token: &str) -> String {
Sha256::digest(token.as_bytes()).iter().take(12).map(|b| format!("{b:02x}")).collect()
}
fn default_dir() -> PathBuf {
crate::default_data_dir().join("discord-sessions")
}
/// Whether a saved position can still be resumed at `now`, by this token.
pub fn resumable(position: &Position, user_id: &str, token: &str, now: i64) -> bool {
position.user_id == user_id && position.token == token_hash(token) && now - position.acked_at < FRESH_FOR && position.acked_at <= now + 5
}
impl SessionFile {
fn dir(&self) -> PathBuf {
self.root.clone().unwrap_or_else(default_dir)
}
fn path(&self, what: &str) -> PathBuf {
let safe: String = self.account.chars().map(|c| if c.is_ascii_alphanumeric() || c == '-' { c } else { '_' }).collect();
self.dir().join(format!("{safe}.{what}.json"))
}
pub fn new(account: &str, user_id: &str, token: &str) -> Self {
SessionFile { account: account.to_string(), user_id: user_id.to_string(), token: token_hash(token), ..Default::default() }
}
/// The saved session as READY and GUILD_CREATE dispatches in the order they were first received, with the
/// position to resume from; `None` when there is nothing fresh enough to resume.
pub fn load(&mut self, token: &str, now: i64) -> Option<(Position, Vec<Value>)> {
let position: Position = serde_json::from_str(&std::fs::read_to_string(self.path("pos")).ok()?).ok()?;
if !resumable(&position, &self.user_id, token, now) {
return None;
}
let state: State = serde_json::from_str(&std::fs::read_to_string(self.path("state")).ok()?).ok()?;
if state.session_id != position.session_id || state.ready.is_null() {
return None;
}
let mut replay = vec![dispatch("READY", position.seq, state.ready.clone())];
replay.extend(state.guilds.values().map(|g| dispatch("GUILD_CREATE", position.seq, g.clone())));
self.position = Some(position.clone());
self.state = Some(state);
Some((position, replay))
}
/// A fresh READY: a new session, with nothing of the last one worth keeping.
pub fn ready(&mut self, session_id: &str, url: &str, seq: u64, ready: Value) {
self.position = Some(Position {
user_id: self.user_id.clone(),
token: self.token.clone(),
session_id: session_id.to_string(),
url: url.to_string(),
seq,
acked_at: chrono::Utc::now().timestamp(),
});
self.state = Some(State { session_id: session_id.to_string(), ready, guilds: HashMap::new() });
self.state_dirty = true;
}
/// The direct messages listed over REST at READY, kept with it: a resume does not list them again.
pub fn dms(&mut self, list: Vec<Value>) {
if let Some(state) = self.state.as_mut() {
state.ready["moho_dms"] = Value::Array(list);
self.state_dirty = true;
}
}
pub fn guild(&mut self, id: &str, payload: &Value) {
if let Some(state) = self.state.as_mut() {
state.guilds.insert(id.to_string(), payload.clone());
self.state_dirty = true;
}
}
pub fn guild_gone(&mut self, id: &str) {
if let Some(state) = self.state.as_mut() {
if state.guilds.remove(id).is_some() {
self.state_dirty = true;
}
}
}
/// A heartbeat was acknowledged: the session is alive as of now, and has seen events up to `seq`.
pub fn acked(&mut self, seq: u64) {
let Some(position) = self.position.as_mut() else { return };
position.seq = seq;
position.acked_at = chrono::Utc::now().timestamp();
self.write();
}
/// The session is gone (refused, or ended by Discord): nothing is left to resume.
pub fn clear(&mut self) {
self.position = None;
self.state = None;
self.state_dirty = false;
let _ = std::fs::remove_file(self.path("pos"));
let _ = std::fs::remove_file(self.path("state"));
}
fn write(&mut self) {
let (Some(position), Some(state)) = (self.position.as_ref(), self.state.as_ref()) else { return };
let _ = std::fs::create_dir_all(self.dir());
if self.state_dirty {
match serde_json::to_string(state) {
Ok(text) if text.len() <= MAX_STATE_BYTES => {
if write_private(&self.path("state"), &text) {
self.state_dirty = false;
}
}
_ => {
// Too big to keep: this session cannot be resumed after a restart, which is the cost of that
// many servers.
let _ = std::fs::remove_file(self.path("pos"));
self.position = None;
self.state = None;
return;
}
}
}
if let Ok(text) = serde_json::to_string(position) {
write_private(&self.path("pos"), &text);
}
}
}
/// Removes whatever is kept for an account that has been taken away.
pub fn forget(account: &str) {
let mut file = SessionFile { account: account.to_string(), ..Default::default() };
file.clear();
}
fn dispatch(kind: &str, seq: u64, d: Value) -> Value {
serde_json::json!({ "op": 0, "t": kind, "s": seq, "d": d, "moho_replay": true })
}
/// Written to a temporary file and renamed, so a restart in the middle of it never reads half a file; readable by
/// this user alone, as the session is as good as a token for as long as it lives.
fn write_private(path: &std::path::Path, text: &str) -> bool {
let tmp = path.with_extension("tmp");
let ok = (|| -> std::io::Result<()> {
#[cfg(unix)]
{
use std::io::Write;
use std::os::unix::fs::OpenOptionsExt;
let mut f = std::fs::OpenOptions::new().write(true).create(true).truncate(true).mode(0o600).open(&tmp)?;
f.write_all(text.as_bytes())?;
}
#[cfg(not(unix))]
std::fs::write(&tmp, text)?;
std::fs::rename(&tmp, path)
})();
ok.is_ok()
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
/// A directory of its own, gone when the test is.
struct Scratch(PathBuf);
impl Scratch {
fn new(name: &str) -> Self {
let dir = std::env::temp_dir().join(format!("nobilis-session-store-{name}-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
Scratch(dir)
}
fn path(&self) -> &std::path::Path {
&self.0
}
}
impl Drop for Scratch {
fn drop(&mut self) {
let _ = std::fs::remove_dir_all(&self.0);
}
}
fn in_dir(dir: &std::path::Path) -> SessionFile {
SessionFile { root: Some(dir.to_path_buf()), ..SessionFile::new("discord:7", "7", "tok") }
}
#[test]
fn a_saved_session_comes_back_as_the_dispatches_it_was_made_of() {
let dir = Scratch::new("comes_back");
let mut first = in_dir(dir.path());
first.ready("s", "wss://x", 3, json!({ "session_id": "s", "guilds": [] }));
first.dms(vec![json!({ "id": "9" })]);
first.guild("g1", &json!({ "id": "g1" }));
first.acked(42);
let (position, replay) = in_dir(dir.path()).load("tok", chrono::Utc::now().timestamp()).expect("fresh enough");
assert_eq!((position.session_id.as_str(), position.seq), ("s", 42));
// READY first, then the servers, each marked as saved and numbered where the session had got to.
assert_eq!(replay[0]["t"], "READY");
assert_eq!(replay[0]["d"]["moho_dms"][0]["id"], "9");
assert_eq!(replay[1]["t"], "GUILD_CREATE");
assert!(replay.iter().all(|m| m["s"] == 42 && m["moho_replay"] == true));
}
#[test]
fn a_session_cleared_or_of_another_token_is_not_resumed() {
let dir = Scratch::new("cleared");
let mut file = in_dir(dir.path());
file.ready("s", "wss://x", 1, json!({}));
file.acked(2);
let now = chrono::Utc::now().timestamp();
assert!(in_dir(dir.path()).load("other", now).is_none());
file.clear();
assert!(in_dir(dir.path()).load("tok", now).is_none());
}
#[test]
fn a_session_that_was_never_acknowledged_is_not_saved() {
let dir = Scratch::new("never_acked");
let mut file = in_dir(dir.path());
file.ready("s", "wss://x", 1, json!({}));
assert!(in_dir(dir.path()).load("tok", chrono::Utc::now().timestamp()).is_none());
}
fn pos(acked_at: i64) -> Position {
Position { user_id: "7".into(), token: token_hash("tok"), session_id: "s".into(), url: "wss://x".into(), seq: 3, acked_at }
}
#[test]
fn a_session_is_resumed_only_while_its_last_ack_is_fresh() {
assert!(resumable(&pos(1000), "7", "tok", 1000 + FRESH_FOR - 1));
assert!(!resumable(&pos(1000), "7", "tok", 1000 + FRESH_FOR));
assert!(!resumable(&pos(1000), "7", "tok", 1000 + 3600));
}
#[test]
fn a_session_belongs_to_one_account_and_one_token() {
assert!(!resumable(&pos(1000), "8", "tok", 1010));
assert!(!resumable(&pos(1000), "7", "other", 1010));
// A clock set back is not a fresh session.
assert!(!resumable(&pos(5000), "7", "tok", 1000));
}
#[test]
fn the_token_itself_is_not_what_is_written() {
assert!(!token_hash("tok").contains("tok"));
assert_eq!(token_hash("tok"), token_hash("tok"));
assert_ne!(token_hash("tok"), token_hash("tok2"));
}
}
src/backend/discord/soundboard.rs
+2 -2
View File
@@ -95,7 +95,7 @@ async fn default_sounds(token: &str) -> Vec<Sound> {
let resp = http_client_for(token)
.get(format!("{API_BASE}/soundboard-default-sounds"))
.header("Authorization", token)
.send()
.send_paced()
.await
.ok()?;
resp.status().is_success().then_some(())?;
@@ -229,7 +229,7 @@ async fn fetch(token: &str, id: &str) -> Result<Vec<u8>> {
}
let resp = http_client_for(token)
.get(format!("https://cdn.discordapp.com/soundboard-sounds/{id}"))
.send()
.send_paced()
.await
.context("fetching the sound")?;
if !resp.status().is_success() {
src/backend/discord/stickers.rs
+12 -4
View File
@@ -74,7 +74,7 @@ pub async fn lottie_file(token: &str, id: &str) -> Result<String> {
if !tokio::fs::try_exists(&path).await.unwrap_or(false) {
let resp = http_client_for(token)
.get(lottie_url(id))
.send()
.send_paced()
.await
.context("fetching the sticker")?;
if !resp.status().is_success() {
@@ -157,7 +157,7 @@ async fn standard_packs(token: &str) -> Vec<(String, Vec<Sticker>)> {
let resp = http_client_for(token)
.get(format!("{API_BASE}/sticker-packs"))
.header("Authorization", token)
.send()
.send_paced()
.await
.ok()?;
if !resp.status().is_success() {
@@ -241,19 +241,27 @@ fn guild_list(state: &AppState, account_id: &str, buffer_id: &str) -> Vec<Value>
/// Sends a sticker as a message of its own.
pub async fn send(state: &AppState, buffer_id: &str, token: &str, sticker_id: &str) -> Result<()> {
let channel_id = state.runtime.get_discord_channel(buffer_id).ok_or_else(|| anyhow!("no known Discord channel for this buffer"))?;
super::send_guard::ensure_may_speak(state, buffer_id).await?;
let _turn = super::send_guard::channel_turn(&channel_id).await;
let resp = send_write(
http_client_for(token)
.post(format!("{API_BASE}/channels/{channel_id}/messages"))
.header("Authorization", token)
.json(&json!({ "sticker_ids": [sticker_id] })),
.json(&{
let mut payload = json!({ "sticker_ids": [sticker_id] });
super::send_guard::stamp(&mut payload);
payload
}),
)
.await
.context("sending a Discord sticker")?;
if !resp.status().is_success() {
let status = resp.status();
let text = resp.text().await.unwrap_or_default();
bail!("Discord API error {status}: {text}");
bail!("{}", super::send::refused(state, buffer_id, status, &text));
}
super::send_guard::record_own_send(buffer_id);
super::send_guard::note_sent(resp).await;
Ok(())
}
src/backend/kick/chat.rs
+1
View File
@@ -589,6 +589,7 @@ pub(super) fn reply_preview(metadata: Option<&ReplyMetadata>, thread_parent: Opt
thread: false,
forwarded: false,
thread_root: Some(thread_root),
chat: None,
})
}
src/backend/matrix/timeline.rs
+1 -1
View File
@@ -24,7 +24,7 @@ pub(super) fn relation_preview(state: &AppState, buffer_id: &str, content: &Valu
let in_thread = thread.is_some();
let target = thread.or_else(|| protocol::reply_target(content))?;
match state.store.get_message(buffer_id, target) {
Ok(Some(m)) => Some(crate::model::ReplyPreview { id: target.to_string(), from: m.from, body: m.body, thread: in_thread, forwarded: false, thread_root: None }),
Ok(Some(m)) => Some(crate::model::ReplyPreview { id: target.to_string(), from: m.from, body: m.body, thread: in_thread, forwarded: false, thread_root: None, chat: None }),
_ => Some(crate::model::ReplyPreview { id: target.to_string(), thread: in_thread, ..Default::default() }),
}
}
src/backend/mod.rs
+1
View File
@@ -16,4 +16,5 @@ pub mod kick;
pub mod matrix;
pub mod sneedchat;
pub mod steam;
pub mod telegram;
pub mod twitch;
src/backend/steam/auth.rs
+94 -14
View File
@@ -27,19 +27,75 @@ use base64::Engine;
use rsa::{BigUint, Pkcs1v15Encrypt, RsaPublicKey};
use std::time::Duration;
/// What moho signs in as: the Steam mobile app's chat session.
/// What moho signs in as: a desktop Steam client, the way every other third-party client does.
///
/// This is part of the token Steam issues and decides how Steam treats the
/// connection that uses it. The desktop client's session is one per account,
/// so signing in as it would sign the Steam launcher out (and the reverse).
/// The mobile chat session is chat and friends only, and lives beside the
/// launcher. It is the only kind moho offers.
/// This is part of the token Steam issues and decides how Steam treats the connection that uses it. It was the
/// mobile app's chat session (platform 3, app type 2, website "Mobile") until Steam raised an account alert for an
/// unexpected device: a "phone" that reports a Linux desktop's operating system, has no machine id and logs on again
/// every time its connection is closed does not look like any real device. SteamKit2, ArchiSteamFarm, node-steam-user,
/// node-steam-session and steam-vent all sign in as the desktop client (platform 1, website "Client"), and so does this.
///
/// `EAuthTokenPlatformType` 3 is the mobile app.
const PLATFORM_TYPE: i32 = 3;
/// `EAuthTokenAppType` 2 is the mobile chat app.
const APP_TYPE: i32 = 2;
const WEBSITE_ID: &str = "Mobile";
/// `EAuthTokenPlatformType` 1 is the desktop client. It has no app type.
const PLATFORM_TYPE: i32 = 1;
const WEBSITE_ID: &str = "Client";
/// `EGamingDeviceType` 1 is an ordinary PC, as node-steam-session sends for the desktop client.
const GAMING_DEVICE_TYPE: u32 = 1;
/// This install's own random number, made once and kept next to the daemon's other files, from which its machine
/// id and logon id are drawn. One per install and not per account: a machine is one machine however many accounts
/// sign in on it, and two installs (a test window, a second computer) are two.
pub fn install_id() -> [u8; 16] {
// A test never writes to the real data directory.
#[cfg(test)]
return [7u8; 16];
#[cfg(not(test))]
{
static ID: std::sync::OnceLock<[u8; 16]> = std::sync::OnceLock::new();
*ID.get_or_init(|| {
let path = crate::default_data_dir().join("steam-install-id");
if let Some(read) = std::fs::read_to_string(&path).ok().and_then(|t| hex::decode(t.trim()).ok()).and_then(|b| <[u8; 16]>::try_from(b).ok()) {
return read;
}
let made: [u8; 16] = rand::random();
let _ = std::fs::create_dir_all(path.parent().unwrap_or(std::path::Path::new(".")));
let _ = std::fs::write(&path, hex::encode(made));
made
})
}
}
/// The machine id Steam's client sends: a binary key-value object of three SHA-1 hashes (named BB3, FF2 and 3B3 in the
/// client), here of strings made from the install's number, so it is the same each time and different for each install.
pub fn machine_id_from(install: &[u8; 16]) -> Vec<u8> {
use sha1::{Digest, Sha1};
let mut out = vec![0u8];
out.extend_from_slice(b"MessageObject\0");
for name in ["BB3", "FF2", "3B3"] {
let hash = hex::encode(Sha1::digest(format!("moho {name} {}", hex::encode(install)).as_bytes()));
out.push(1);
out.extend_from_slice(name.as_bytes());
out.push(0);
out.extend_from_slice(hash.as_bytes());
out.push(0);
}
out.extend_from_slice(&[8, 8]);
out
}
pub fn machine_id() -> Vec<u8> {
machine_id_from(&install_id())
}
/// What tells this install's sessions from another's on the same account: Steam ends a session when a later one with
/// the same logon id arrives, and lets those with different ones live side by side. The desktop client sends its own
/// private address here; this sends a number of its own, never zero (which is "none").
pub fn logon_id_from(install: &[u8; 16]) -> u32 {
(u32::from_le_bytes([install[0], install[1], install[2], install[3]]) | 1) ^ 0xBAAD_F00D
}
pub fn logon_id() -> u32 {
logon_id_from(&install_id())
}
/// `EOSType` as the connection reports it: Linux 6.x is -184 and Windows 10
/// is 16, sent as the unsigned number the field is.
@@ -155,7 +211,9 @@ fn device_details() -> DeviceDetails {
device_friendly_name: Some(device_name()),
platform_type: Some(PLATFORM_TYPE),
os_type: Some(os_type() as i32),
app_type: Some(APP_TYPE),
gaming_device_type: Some(GAMING_DEVICE_TYPE),
// Sent with a password sign-in, as the desktop client's is.
machine_id: Some(machine_id()),
..DeviceDetails::default()
}
}
@@ -433,8 +491,11 @@ mod tests {
assert_eq!(begin_req.encryption_timestamp, Some(1790000005000));
assert_eq!(begin_req.persistence, Some(1));
assert_eq!(begin_req.guard_data.as_deref(), Some("old-guard"));
assert_eq!(begin_req.platform_type, Some(3), "it signs in as the mobile app, never as the desktop client");
assert_eq!(begin_req.device_details.unwrap().app_type, Some(2));
assert_eq!(begin_req.platform_type, Some(1), "it signs in as a desktop client, like every other third-party client");
assert_eq!(begin_req.website_id.as_deref(), Some("Client"));
let details = begin_req.device_details.unwrap();
assert_eq!((details.platform_type, details.app_type, details.gaming_device_type), (Some(1), None, Some(1)));
assert_eq!(details.machine_id.as_deref(), Some(machine_id().as_slice()), "and with its install's machine id");
assert!(!begin_req.encrypted_password.unwrap().contains("hunter2"));
let code_req = UpdateAuthWithGuardCodeRequest::decode(seen[2].body.as_slice()).unwrap();
@@ -519,4 +580,23 @@ mod tests {
assert_eq!(steamid_of_token("not a token"), None);
assert_eq!(steamid_of_token("a.b.c"), None);
}
#[test]
fn the_machine_id_is_a_key_value_object_of_three_hashes_that_is_the_same_for_one_install_and_not_for_another() {
let one = machine_id_from(&[1u8; 16]);
assert_eq!(one, machine_id_from(&[1u8; 16]));
assert_ne!(one, machine_id_from(&[2u8; 16]));
// 0x00 "MessageObject" 0x00, then three strings named BB3, FF2 and 3B3 of forty hex digits, then 0x08 0x08.
assert!(one.starts_with(b"\0MessageObject\0\x01BB3\0"));
assert!(one.ends_with(&[0, 8, 8]));
let text = String::from_utf8_lossy(&one).to_string();
for name in ["BB3", "FF2", "3B3"] {
assert!(text.contains(name));
}
assert_eq!(one.len(), 1 + 14 + 3 * (1 + 3 + 1 + 40 + 1) + 2);
assert_ne!(logon_id_from(&[1u8; 16]), 0);
assert_ne!(logon_id_from(&[1u8; 16]), logon_id_from(&[2u8; 16]));
// Never the same as the zero a client with no id of its own sends.
assert_ne!(logon_id_from(&[0xFF; 16]), 0);
}
}
src/backend/steam/connect.rs
+4 -3
View File
@@ -39,8 +39,8 @@ pub const PROTOCOL_VERSION: u32 = 65581;
/// How long Steam has to answer a sign-in.
const LOGON_TIMEOUT: Duration = Duration::from_secs(30);
const RECONNECT_INITIAL_DELAY: Duration = Duration::from_secs(3);
const RECONNECT_MAX_DELAY: Duration = Duration::from_secs(60);
const RECONNECT_INITIAL_DELAY: Duration = Duration::from_secs(10);
const RECONNECT_MAX_DELAY: Duration = Duration::from_secs(300);
/// How long to leave an account alone after Steam says it is being asked too
/// often. Seconds would be asking again; this is the length of time that the
@@ -124,10 +124,11 @@ pub fn logon_message(config: &SteamAccountConfig, language: &str) -> ClientLogon
should_remember_password: Some(true),
// Where the machine is on its own network is nothing Steam needs to
// be told; the field exists and asks to be filled.
obfuscated_private_ip: Some(IpAddress { ip: Some(ip_address::Ip::V4(0)) }),
obfuscated_private_ip: Some(IpAddress { ip: Some(ip_address::Ip::V4(auth::logon_id())) }),
chat_mode: Some(2),
account_name: Some(config.username.clone()),
machine_name: Some(auth::device_name()),
machine_id: Some(auth::machine_id()),
supports_rate_limit_response: Some(true),
// The refresh token, which is what this field takes whatever its name.
access_token: Some(config.refresh_token.clone()),
src/backend/steam/login.rs
+1 -1
View File
@@ -305,7 +305,7 @@ pub async fn flow(
display_name: None,
refresh_token: tokens.refresh_token,
guard_data: tokens.guard_data,
// Always the mobile chat session; kept so older files still read.
// The kind is no longer chosen; kept so older files still read.
session: "chat".to_string(),
use_tor: crate::net::route::router().wanted(&key),
strict_route: false,
src/backend/telegram/admin.rs
+46
View File
@@ -0,0 +1,46 @@
//! What an administrator may do to a member: remove, ban, silence.
//!
//! The menu beside the member list asks what this account may do here, and offers only that. Telegram has the last
//! word: an administrator who lacks the particular right is refused ("Only an administrator may do that here").
use super::send::{explain, peer_for};
use crate::model::Buffer;
use crate::state::AppState;
use grammers_session::types::{PeerId, PeerKind};
use grammers_session::Session;
use serde_json::{json, Value};
use std::time::Duration;
/// How long a mute lasts, as everywhere else in moho.
const MUTE: Duration = Duration::from_secs(10 * 60);
/// What this account may do to others in a conversation, in the shape the member menu reads.
pub async fn permissions(state: &AppState, buffer: &Buffer) -> Result<Value, String> {
let none = json!({ "canRedactOthers": false, "canKick": false, "canBan": false, "canMute": false });
if buffer.kind == "dm" {
return Ok(none);
}
let (handle, peer) = peer_for(state, buffer).await?;
let me = handle.client.get_me().await.map_err(|e| explain(&e))?;
let Ok(Some(me)) = me.to_ref().await else { return Ok(none) };
let Ok(found) = handle.client.get_permissions(peer, me).await else { return Ok(none) };
let may = found.is_creator() || found.is_admin();
// A small group can only have a member removed; bans and silences are for supergroups and channels.
let supergroup = peer.id.kind() == PeerKind::Channel;
Ok(json!({ "canRedactOthers": may, "canKick": may, "canBan": may && supergroup, "canMute": may && supergroup }))
}
/// Removes, bans or silences one member.
pub async fn moderate(state: &AppState, buffer: &Buffer, user_id: &str, action: &str) -> Result<(), String> {
let id: i64 = user_id.parse().map_err(|_| "that member cannot be told apart from the others".to_string())?;
let (handle, peer) = peer_for(state, buffer).await?;
let user = handle.session.peer_ref(PeerId::user_unchecked(id)).await.ok().flatten().ok_or_else(|| "Telegram has not told this account about that member yet".to_string())?;
let supergroup = peer.id.kind() == PeerKind::Channel;
match action {
"kick" => handle.client.kick_participant(peer, user).await.map_err(|e| explain(&e)),
"ban" if supergroup => handle.client.set_banned_rights(peer, user).view_messages(false).await.map_err(|e| explain(&e)),
"mute" if supergroup => handle.client.set_banned_rights(peer, user).send_messages(false).duration(MUTE).await.map_err(|e| explain(&e)),
"ban" | "mute" => Err("Telegram only lets a supergroup or a channel ban or silence somebody".to_string()),
_ => Err("Telegram has no such action".to_string()),
}
}
src/backend/telegram/bots.rs
+210
View File
@@ -0,0 +1,210 @@
//! What a bot offers: the commands in its menu.
//!
//! Telegram keeps a bot's list of commands (`/help`, `/weblink`, ...) with the bot, and its own apps show it as a
//! Menu button beside the message box and as suggestions while typing a slash. It is read once from the bot's profile
//! and kept for a while, since it changes when the bot's author changes it and not otherwise.
//!
//! In a group it is the commands of every bot that is in the group, which Telegram lists with the group itself (and
//! scopes: a bot can offer other commands to a group's administrators than to its members). Two bots can both have a
//! `/help`, and a group says which one is meant by writing the bot's name after it, `/help@somebot`: a command that
//! more than one bot has is offered once for each bot, in that form.
use super::peers;
use super::send::{explain, peer_for};
use crate::model::Buffer;
use crate::state::AppState;
use grammers_client::tl;
use std::collections::HashMap;
use std::sync::Mutex;
use std::time::{Duration, Instant};
/// How long a bot's list is kept before it is asked for again.
const KEEP: Duration = Duration::from_secs(15 * 60);
/// One command in a bot's menu.
#[derive(Clone, Debug, PartialEq)]
pub struct Command {
pub name: String,
pub description: String,
/// Whose it is, where that is not the conversation itself: a bot in a group, as `@name`.
pub source: Option<String>,
}
/// When each conversation's list was read, and what it said.
type Kept = Mutex<HashMap<String, (Instant, Vec<Command>)>>;
fn kept() -> &'static Kept {
static K: std::sync::OnceLock<Kept> = std::sync::OnceLock::new();
K.get_or_init(Default::default)
}
/// The commands a bot's profile lists.
fn from_info(info: &tl::enums::BotInfo) -> Vec<Command> {
let tl::enums::BotInfo::Info(info) = info;
info.commands
.iter()
.flatten()
.map(|c| {
let tl::enums::BotCommand::Command(c) = c;
Command { name: c.command.trim_start_matches('/').to_string(), description: c.description.clone(), source: None }
})
.filter(|c| !c.name.is_empty())
.collect()
}
/// The commands of a bot's conversation, or of the bots in a group; nothing for any other conversation.
pub async fn commands(state: &AppState, buffer: &Buffer) -> Result<Vec<Command>, String> {
let with_bot = buffer.kind == "dm" && buffer.name_icon.as_deref() == Some("smart_toy");
// A channel's posts are not commands, and a person is not a bot. A group (or a topic of one) has bots.
let in_group = buffer.kind != "dm" && buffer.category.as_deref() != Some(peers::CHANNELS);
if !with_bot && !in_group {
return Ok(Vec::new());
}
// A group's topics are one group: they share a list.
let key = format!("{}|{}", buffer.account_id, buffer.remote_id.as_deref().map(|r| r.split(':').next().unwrap_or(r)).unwrap_or(&buffer.id));
if let Some((when, list)) = kept().lock().unwrap_or_else(|e| e.into_inner()).get(&key) {
if when.elapsed() < KEEP {
return Ok(list.clone());
}
}
let (handle, peer) = peer_for(state, buffer).await?;
let list = if with_bot {
let answer = handle.client.invoke(&tl::functions::users::GetFullUser { id: peer.into() }).await.map_err(|e| explain(&e))?;
let tl::enums::users::UserFull::Full(full) = answer;
let tl::enums::UserFull::Full(user) = full.full_user;
user.bot_info.as_ref().map(from_info).unwrap_or_default()
} else {
let (infos, users) = if peer.id.kind() == grammers_session::types::PeerKind::Channel {
let answer = handle.client.invoke(&tl::functions::channels::GetFullChannel { channel: peer.into() }).await.map_err(|e| explain(&e))?;
let tl::enums::messages::ChatFull::Full(full) = answer;
let tl::enums::ChatFull::ChannelFull(group) = full.full_chat else { return Ok(Vec::new()) };
(group.bot_info, full.users)
} else {
let answer = handle.client.invoke(&tl::functions::messages::GetFullChat { chat_id: peer.id.bare_id_unchecked() }).await.map_err(|e| explain(&e))?;
let tl::enums::messages::ChatFull::Full(full) = answer;
let tl::enums::ChatFull::Full(group) = full.full_chat else { return Ok(Vec::new()) };
(group.bot_info.unwrap_or_default(), full.users)
};
merge(&infos, &users)
};
kept().lock().unwrap_or_else(|e| e.into_inner()).insert(key, (Instant::now(), list.clone()));
Ok(list)
}
/// The @name a user is reached by.
fn username_of(user: &tl::enums::User) -> Option<String> {
let tl::enums::User::User(user) = user else { return None };
user.username.clone().or_else(|| user.usernames.as_ref()?.iter().find_map(|u| {
let tl::enums::Username::Username(u) = u;
u.active.then(|| u.username.clone())
}))
}
/// The commands of the bots in a group, each named for its bot when more than one bot has it.
fn merge(infos: &[tl::enums::BotInfo], users: &[tl::enums::User]) -> Vec<Command> {
let mut per_bot: Vec<(String, Vec<Command>)> = Vec::new();
for info in infos {
let tl::enums::BotInfo::Info(raw) = info;
let Some(id) = raw.user_id else { continue };
let Some(name) = users.iter().find(|u| matches!(u, tl::enums::User::User(u) if u.id == id)).and_then(username_of) else { continue };
per_bot.push((name, from_info(info)));
}
label(&per_bot)
}
/// Each bot's commands as the group's menu, a name that two bots share written with the bot's after it.
fn label(per_bot: &[(String, Vec<Command>)]) -> Vec<Command> {
let mut taken: HashMap<&str, usize> = HashMap::new();
for (_, list) in per_bot {
for c in list {
*taken.entry(c.name.as_str()).or_default() += 1;
}
}
let mut out = Vec::new();
for (bot, list) in per_bot {
for c in list {
let name = if taken.get(c.name.as_str()).copied().unwrap_or(0) > 1 { format!("{}@{bot}", c.name) } else { c.name.clone() };
out.push(Command { name, description: c.description.clone(), source: Some(format!("@{bot}")) });
}
}
out
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_bots_menu_is_read_from_its_profile() {
let info = tl::enums::BotInfo::Info(tl::types::BotInfo {
has_preview_medias: false,
user_id: Some(1),
description: None,
description_photo: None,
description_document: None,
commands: Some(vec![
tl::enums::BotCommand::Command(tl::types::BotCommand { ephemeral: false, command: "help".into(), description: "Show help for bot commands".into() }),
tl::enums::BotCommand::Command(tl::types::BotCommand { ephemeral: false, command: "/weblink".into(), description: "Get available link to ZLibrary".into() }),
tl::enums::BotCommand::Command(tl::types::BotCommand { ephemeral: false, command: "".into(), description: "nothing".into() }),
]),
menu_button: None,
privacy_policy_url: None,
app_settings: None,
verifier_settings: None,
});
assert_eq!(
from_info(&info),
vec![
Command { name: "help".into(), description: "Show help for bot commands".into(), source: None },
Command { name: "weblink".into(), description: "Get available link to ZLibrary".into(), source: None },
]
);
}
fn cmd(name: &str, description: &str) -> Command {
Command { name: name.into(), description: description.into(), source: None }
}
#[test]
fn a_group_offers_each_bots_commands_and_names_the_bot_only_where_two_share_one() {
let per_bot = vec![
("rose_bot".to_string(), vec![cmd("help", "Rose help"), cmd("warn", "Warn a user")]),
("music_bot".to_string(), vec![cmd("help", "Music help"), cmd("play", "Play a song")]),
];
let got: Vec<(String, Option<String>)> = label(&per_bot).into_iter().map(|c| (c.name, c.source)).collect();
assert_eq!(
got,
vec![
("help@rose_bot".to_string(), Some("@rose_bot".to_string())),
("warn".to_string(), Some("@rose_bot".to_string())),
("help@music_bot".to_string(), Some("@music_bot".to_string())),
("play".to_string(), Some("@music_bot".to_string())),
]
);
}
#[test]
fn a_lone_bots_commands_are_not_named_for_it() {
let got = label(&[("only_bot".to_string(), vec![cmd("start", "Begin")])]);
assert_eq!(got.iter().map(|c| c.name.as_str()).collect::<Vec<_>>(), vec!["start"]);
assert!(label(&[]).is_empty());
}
#[test]
fn a_bot_the_group_does_not_name_cannot_be_written_to_and_is_left_out() {
let info = tl::enums::BotInfo::Info(tl::types::BotInfo {
has_preview_medias: false,
user_id: Some(3),
description: None,
description_photo: None,
description_document: None,
commands: Some(vec![tl::enums::BotCommand::Command(tl::types::BotCommand { ephemeral: false, command: "ghost".into(), description: "Nobody".into() })]),
menu_button: None,
privacy_policy_url: None,
app_settings: None,
verifier_settings: None,
});
let users = vec![tl::enums::User::Empty(tl::types::UserEmpty { id: 3 })];
assert!(merge(&[info], &users).is_empty());
}
}
src/backend/telegram/connect.rs
+299
View File
@@ -0,0 +1,299 @@
//! An account's connection: open the session it saved, stay connected, and reconnect when Telegram or the network
//! lets go - without ever signing in afresh to get over a hiccup (see mod.rs on why).
//!
//! Modelled on Steam's loop (`steam/connect.rs run_with_retry`): a wait that grows while a connection falls over at
//! once and starts again after one that held, a long wait when Telegram says to slow down, and a stop - not a retry
//! - when Telegram has ended the session, because the same key can never work again.
use super::link::{self, Link};
use super::messages::{self, Own};
use super::peers;
use super::Handle;
use super::session::{self, FileSession};
use crate::accounts::TelegramAccountConfig;
use crate::runtime::ConnState;
use crate::state::AppState;
use anyhow::{Context, Result};
use grammers_client::InvocationError;
use grammers_mtsender::UpdatesConfiguration;
use std::sync::Arc;
use std::time::Duration;
const RECONNECT_INITIAL_DELAY: Duration = Duration::from_secs(10);
const RECONNECT_MAX_DELAY: Duration = Duration::from_secs(300);
/// How often a changed session is written.
const SAVE_EVERY: Duration = Duration::from_secs(5);
/// Ends the account's session at Telegram, so removing it here does not leave a session in the account's list of
/// devices, and takes the saved session away. Best effort: the saved one is gone either way.
pub async fn sign_out(state: &AppState, config: &TelegramAccountConfig) {
let path = session::path_for(config.user_id);
if let Ok(session) = FileSession::open(path.clone()) {
let session = Arc::new(session);
if session.has_key() {
// The account was removed first, so its own key is the only route it has left: not Tor unless it was.
match link::open(state, &config.account_id(), config.api_id, session.clone()).await {
Ok(link) => {
if let Err(e) = link.client.sign_out().await {
tracing::info!("telegram[{}]: signing out at Telegram: {e}", config.account_id());
}
link.close().await;
}
Err(e) => tracing::info!("telegram[{}]: could not reach Telegram to sign out: {e:#}", config.account_id()),
}
}
session.delete();
} else {
let _ = std::fs::remove_file(path);
}
}
/// How a connection ended.
#[derive(Debug, PartialEq)]
pub enum Ending {
/// Telegram ended the session (or it is gone): sign in again.
SignInAgain(String),
/// Telegram asked for a pause, or the network is down: try again after this.
Later(Duration, String),
/// The connection ended; try again after the usual wait.
Closed,
}
/// What an error from Telegram means for the connection.
pub fn classify(error: &InvocationError) -> Ending {
match error {
InvocationError::Rpc(rpc) => {
let name = rpc.name.as_str();
if rpc.code == 401 || matches!(name, "AUTH_KEY_UNREGISTERED" | "SESSION_REVOKED" | "SESSION_EXPIRED" | "USER_DEACTIVATED" | "USER_DEACTIVATED_BAN" | "AUTH_KEY_DUPLICATED") {
Ending::SignInAgain(super::login::explain(name))
} else if name == "FLOOD_WAIT" || name == "FLOOD_PREMIUM_WAIT" {
// The number of seconds Telegram asked for, and a little more.
Ending::Later(Duration::from_secs(u64::from(rpc.value.unwrap_or(60)) + 5), format!("Telegram asked for a pause of {} seconds", rpc.value.unwrap_or(60)))
} else {
Ending::Closed
}
}
_ => Ending::Closed,
}
}
/// Starts (or restarts) an account's connection.
pub fn spawn(state: AppState, config: TelegramAccountConfig) {
let account_id = config.account_id();
crate::net::route::router().mark(&account_id, config.use_tor);
state.runtime.reset_connection(&account_id);
let handle = tokio::spawn({
let account_id = account_id.clone();
let state = state.clone();
async move {
run_with_retry(&state, &account_id).await;
state.runtime.remove_task_handle(&account_id);
}
});
state.runtime.insert_task_handle(&account_id, handle.abort_handle());
}
async fn run_with_retry(state: &AppState, account_id: &str) {
let mut delay = RECONNECT_INITIAL_DELAY;
state.runtime.set_conn_state(state, account_id, ConnState::Connecting, None);
loop {
// The account as it is now: removed while this slept, it is not reconnected.
let Some(config) = state.accounts.get_telegram(account_id) else { return };
let (detail, wait) = match run(state, &config, account_id).await {
Ok(Ending::SignInAgain(why)) => {
tracing::info!("telegram[{account_id}]: stopped: {why}");
state.runtime.set_conn_state(state, account_id, ConnState::AuthFailed, Some(&format!("{why}. Sign in to Telegram again from Accounts.")));
return;
}
Ok(Ending::Later(wait, why)) => (why, Some(wait)),
Ok(Ending::Closed) => ("connection ended".to_string(), None),
Err(e) => {
tracing::warn!("telegram[{account_id}]: {e:#}");
(format!("{e:#}"), None)
}
};
state.runtime.set_conn_state(state, account_id, ConnState::Connecting, None);
let wait = wait.unwrap_or_else(|| state.runtime.next_retry_delay(account_id, &mut delay, RECONNECT_INITIAL_DELAY, RECONNECT_MAX_DELAY));
tracing::info!("telegram[{account_id}]: not connected ({detail}); trying again in {}s", wait.as_secs());
state.runtime.report_retry(state, account_id, &detail, wait);
tokio::time::sleep(wait).await;
}
}
/// One go at being connected.
async fn run(state: &AppState, config: &TelegramAccountConfig, account_id: &str) -> Result<Ending> {
let session = Arc::new(FileSession::open(session::path_for(config.user_id))?);
if !session.has_key() {
return Ok(Ending::SignInAgain("this account has no saved session".to_string()));
}
let mut link = link::open(state, account_id, config.api_id, session).await?;
let ending = stay(state, config, account_id, &mut link).await;
state.runtime.clear_telegram_handle(account_id);
link.close().await;
ending
}
async fn stay(state: &AppState, config: &TelegramAccountConfig, account_id: &str, link: &mut Link) -> Result<Ending> {
match link.client.is_authorized().await {
Ok(true) => {}
Ok(false) => return Ok(Ending::SignInAgain("Telegram no longer accepts this session".to_string())),
Err(e) => return Ok(classify(&e)).and_then(|ending| if ending == Ending::Closed { Err(anyhow::Error::from(e)).context("asking Telegram whether the session is good") } else { Ok(ending) }),
}
// The name and number the account has now, which are Telegram's to change.
let mut own = Own { name: if config.name.is_empty() { config.phone.clone() } else { config.name.clone() }, user_id: config.user_id };
if let Ok(me) = link.client.get_me().await {
let (name, username) = (me.full_name(), me.username().map(str::to_string));
if !name.is_empty() {
own.name = name.clone();
}
let _ = state.accounts.mutate_telegram(account_id, |a| {
a.name = name;
a.username = username;
});
}
state.runtime.set_own_identity(account_id, &own.name);
let handle = Handle { client: link.client.clone(), session: link.session.clone(), own: own.clone(), names: Default::default() };
state.runtime.set_telegram_handle(account_id, handle.clone());
state.runtime.set_conn_state(state, account_id, ConnState::Connected, None);
tracing::info!("telegram[{account_id}]: connected as {} ({})", config.user_id, config.username.as_deref().unwrap_or("no username"));
// The conversations, in the background: the connection is up whether or not this succeeds.
tokio::spawn({
let (state, account_id, handle) = (state.clone(), account_id.to_string(), handle.clone());
async move {
if let Err(e) = super::dialogs::sync(&state, &account_id, &handle).await {
tracing::warn!("telegram[{account_id}]: reading the conversations: {e:#}");
}
// After the conversations: what a folder holds by its rules is worked out from them.
if let Err(e) = super::folders::sync(&state, &account_id, &handle).await {
tracing::warn!("telegram[{account_id}]: reading the folders: {e:#}");
}
}
});
let updates = link.updates.take().context("the updates were already taken")?;
let mut stream = link.client.stream_updates(updates, UpdatesConfiguration { catch_up: true }).await.map_err(|e| anyhow::anyhow!("starting to read updates: {e}"))?;
let mut save = tokio::time::interval(SAVE_EVERY);
loop {
tokio::select! {
_ = save.tick() => {
if let Err(e) = link.session.save_if_dirty() {
tracing::warn!("telegram[{account_id}]: saving the session: {e:#}");
}
}
update = stream.next() => match update {
Ok(update) => handle_update(state, account_id, &handle, update).await,
Err(e) => {
let _ = stream.sync_update_state().await;
return match classify(&e) {
Ending::Closed => Err(anyhow::Error::from(e)).context("reading updates"),
other => Ok(other),
};
}
},
}
}
}
/// What Telegram sent unasked: a message, an edit, something deleted.
async fn handle_update(state: &AppState, account_id: &str, handle: &Handle, update: grammers_client::update::Update) {
use grammers_client::update::Update;
match update {
Update::NewMessage(message) => {
let recorded = match message.peer() {
Some(_) => messages::record_named(state, account_id, &handle.own, &message, Some(&handle.names)),
// A short update from a person names no one: asked of Telegram, who is the other side.
None => {
let (state, account_id, handle) = (state.clone(), account_id.to_string(), handle.clone());
tokio::spawn(async move { record_unnamed(&state, &account_id, &handle, &message).await });
return;
}
};
tracing::debug!("telegram[{account_id}]: new message in {:?}: recorded {recorded}", message.peer_id());
}
Update::Raw(raw) => super::updates::raw(state, account_id, handle, &raw.raw),
Update::MessageEdited(message) => {
let peer = match message.peer() {
Some(peer) => Some(peer.clone()),
None => resolve(handle, &message).await,
};
if let Some(peer) = peer {
let inner: &grammers_client::message::Message = &message;
let buffer = super::topics::buffer_of_message(state, account_id, &peer, &inner.raw);
let body = messages::body_of(&message);
let attachments = buffer.remote_id.as_deref().map(|remote| super::media::attachments_of(handle.own.user_id, remote, &message)).unwrap_or_default();
state.runtime.update_message(state, &buffer.id, &message.id().to_string(), &body, &[], &attachments);
// A bot changes or removes its buttons by editing the message they are under.
super::keyboards::note(state, &buffer.id, &message.id().to_string(), &inner.raw, true);
}
}
Update::MessageDeleted(deleted) => {
for id in deleted.messages() {
forget_message(state, account_id, deleted.channel_id(), *id);
}
}
_ => {}
}
}
/// The conversation a message from a short update belongs to, from what the session knows of the person.
async fn resolve(handle: &Handle, message: &grammers_client::message::Message) -> Option<grammers_client::peer::Peer> {
let peer_ref = message.peer_ref().await.ok().flatten()?;
handle.client.resolve_peer(peer_ref).await.ok()
}
async fn record_unnamed(state: &AppState, account_id: &str, handle: &Handle, message: &grammers_client::message::Message) {
match resolve(handle, message).await {
Some(peer) => {
messages::record_in(state, account_id, &handle.own, message, &peer, None, Some(&handle.names));
}
None => tracing::warn!("telegram[{account_id}]: a message in {:?} from somebody this account has no record of was not shown", message.peer_id()),
}
}
/// Removes a message Telegram says is gone. A channel's deletion names the channel; any other names only the number,
/// which is unique across the account's people and small groups, so the conversations are looked through.
fn forget_message(state: &AppState, account_id: &str, channel: Option<i64>, id: i32) {
let msg_id = id.to_string();
let candidates: Vec<crate::model::Buffer> = state
.runtime
.list_buffers()
.into_iter()
.filter(|b| b.account_id == account_id)
.filter(|b| match (channel, b.remote_id.as_deref()) {
// A group with topics keeps each topic's conversation under the group's name and a colon.
(Some(c), Some(remote)) => {
let channel = peers::remote_id(grammers_session::types::PeerId::channel_unchecked(c));
remote == channel || remote.strip_prefix(channel.as_str()).is_some_and(|rest| rest.starts_with(':'))
}
(Some(_), None) => false,
(None, Some(remote)) => !remote.starts_with("-100"),
(None, None) => false,
})
.collect();
for buffer in candidates {
if state.store.get_message(&buffer.id, &msg_id).ok().flatten().is_some() {
state.runtime.delete_message(state, &buffer.id, &msg_id);
return;
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use grammers_mtsender::RpcError;
fn rpc(code: i32, name: &str, value: Option<u32>) -> InvocationError {
InvocationError::Rpc(RpcError { code, name: name.to_string(), value, caused_by: None })
}
#[test]
fn a_session_telegram_ended_stops_the_connection_and_a_flood_wait_waits_as_long_as_asked() {
assert!(matches!(classify(&rpc(401, "AUTH_KEY_UNREGISTERED", None)), Ending::SignInAgain(_)));
assert!(matches!(classify(&rpc(401, "SESSION_REVOKED", None)), Ending::SignInAgain(_)));
assert!(matches!(classify(&rpc(420, "FLOOD_WAIT", Some(120))), Ending::Later(d, _) if d == Duration::from_secs(125)));
assert_eq!(classify(&rpc(500, "INTERDC_CALL_ERROR", None)), Ending::Closed);
assert_eq!(classify(&InvocationError::Io(std::io::Error::other("reset"))), Ending::Closed);
}
}
src/backend/telegram/dialogs.rs
+93
View File
@@ -0,0 +1,93 @@
//! The conversations an account has: asked for when the connection comes up, and each one's last message with it.
//!
//! Telegram's dialog list answers with every conversation and, in the same response, its last message, so one
//! request gives the list its names, kinds and recent activity without opening any of them. History proper is read
//! when a conversation is opened (see `history`).
use super::media;
use super::messages;
use super::Handle;
use super::peers;
use crate::state::AppState;
use anyhow::Result;
use grammers_client::tl;
/// How many conversations are read on connect. Telegram pages them a hundred at a time.
const LIMIT: usize = 200;
/// What the account's folders look at in a conversation, noted for working out what each folder holds.
fn note_for_folders(account_id: &str, dialog: &grammers_client::peer::Dialog) {
use super::folders::{Facts, Kind};
use grammers_client::peer::Peer;
let (kind, contact) = match dialog.peer() {
Peer::User(user) if user.is_bot() => (Kind::Bot, false),
Peer::User(user) => (Kind::Person, user.contact()),
Peer::Group(_) | Peer::Community(_) => (Kind::Group, false),
Peer::Channel(_) => (Kind::Broadcast, false),
};
let tl::enums::Dialog::Dialog(d) = &dialog.raw else { return };
let facts = Facts {
kind,
contact,
muted: super::notify::silent(&d.notify_settings, chrono::Utc::now().timestamp()),
unread: d.unread_count > 0 || d.unread_mark,
// Telegram keeps its archive as the folder numbered 1.
archived: d.folder_id == Some(1),
};
super::folders::note(account_id, peers::remote_id(dialog.peer().id()), facts);
}
/// Makes a buffer of each of the account's conversations, newest activity first, and records the last message of each.
pub async fn sync(state: &AppState, account_id: &str, handle: &Handle) -> Result<usize> {
let mut dialogs = handle.client.iter_dialogs();
let mut count = 0;
while count < LIMIT {
let Some(dialog) = dialogs.next().await.map_err(|e| anyhow::anyhow!("{e}"))? else { break };
note_for_folders(account_id, &dialog);
// A group with Topics is a conversation for each topic, found by asking the group.
if super::topics::is_forum(dialog.peer()) {
peers::rail_for(state, account_id, dialog.peer(), count as i64);
if let Err(e) = super::topics::sync(state, account_id, handle, dialog.peer()).await {
tracing::warn!("telegram[{account_id}]: reading a group's topics: {e:#}");
}
if let Some(last) = &dialog.last_message {
messages::record_named(state, account_id, &handle.own, last, Some(&handle.names));
}
count += 1;
continue;
}
let buffer = peers::buffer_for(state, account_id, dialog.peer());
// Filed again each time: a bot read before bots had a heading of their own was filed with the people.
// A group is its own entry on the rail and has no heading in the account's list.
if peers::is_group(dialog.peer()) {
peers::file_in_rail(state, account_id, &buffer, dialog.peer(), count as i64);
} else {
state.runtime.set_buffer_category(state, &buffer.id, Some(peers::category_of(dialog.peer())), buffer.position);
}
state.runtime.set_buffer_name_icon(state, &buffer.id, peers::name_icon_of(dialog.peer()));
if let grammers_client::peer::Peer::User(user) = dialog.peer() {
handle.names.lock().unwrap_or_else(|e| e.into_inner()).insert(user.id().bare_id_unchecked(), peers::title_of(dialog.peer()));
}
if let Some(last) = &dialog.last_message {
messages::record_named(state, account_id, &handle.own, last, Some(&handle.names));
}
// Silenced on the account, perhaps on the phone.
if let tl::enums::Dialog::Dialog(d) = &dialog.raw {
state.runtime.set_silenced(state, &buffer.id, super::notify::silent(&d.notify_settings, chrono::Utc::now().timestamp()));
}
// Nothing unread there, as Telegram counts it: the window need not call it unread either.
if matches!(&dialog.raw, tl::enums::Dialog::Dialog(d) if d.unread_count == 0) {
state.events.emit("bufferRead", serde_json::json!({ "accountId": account_id, "bufferId": buffer.id }));
}
if let Some(avatar) = media::avatar_url(handle.own.user_id, dialog.peer()) {
state.runtime.set_buffer_avatar(state, &buffer.id, &avatar);
}
// Whether the account may write there, read off the dialog's own copy of the chat (which carries its rights):
// where it may not, the window shows why in place of the message box.
let now = chrono::Utc::now().timestamp();
state.runtime.set_buffer_read_only(state, &buffer.id, peers::read_only_for(dialog.peer(), now));
count += 1;
}
tracing::info!("telegram[{account_id}]: {count} conversations");
Ok(count)
}
src/backend/telegram/folders.rs
+734
View File
@@ -0,0 +1,734 @@
//! Chat folders: the tabs Telegram lets an account sort its chats into, kept in step with moho's categories.
//!
//! A folder is a name and a rule for what is in it: chats named one by one, and kinds of chat (contacts, groups,
//! channels, bots) with some taken back out (muted, read, archived, or named one by one). The window draws each folder
//! as a category on the account's page, and a change on either side is made on the other:
//!
//! - a folder made, renamed, deleted or moved in order on Telegram turns up as the same change to the category
//! (`telegramFolders`, said whenever the account's folders are read: on connect and on every
//! `updateDialogFilter*` Telegram sends);
//! - a category made, renamed, deleted or moved in order here is made on Telegram, and a conversation filed under a
//! category goes into that folder.
//!
//! A chat is in one category here and can be in several folders there. It is drawn under the first folder (in
//! Telegram's order) that holds it, and filing it somewhere else takes it out of that one and puts it in the new one;
//! the folders it is in besides are left as they are. Taking it out of a folder it is in by a rule - a chat in
//! "Groups" because it is a group - is done the way Telegram's own apps do it, by adding it to the folder's
//! exceptions.
//!
//! What Telegram does not allow is also not allowed here, and said in its words: a folder needs something in it, so
//! a category is made on Telegram when its first conversation is filed into it, and the last one cannot be taken
//! out. Groups are not conversations on the account's page (each is an entry on the rail), so they are not shown
//! under a category, but a folder that holds them still does, and an edit made here never touches them.
//!
//! Which chats a rule holds is worked out from what is known of each chat when the conversations are read - whether
//! it is a contact, muted, unread or archived - so a folder defined by "unread" is as current as that read.
use super::send::{explain, peer_for};
use super::{peers, Handle};
use crate::model::Buffer;
use crate::state::AppState;
use anyhow::{anyhow, Result};
use grammers_client::tl;
use grammers_session::types::PeerId;
use serde_json::{json, Value};
use std::collections::{HashMap, HashSet};
use std::sync::Mutex;
/// The first number a folder made by hand may have: 0 is the list of every chat.
const FIRST_ID: i32 = 2;
/// What kind of chat a conversation is, as a folder's rules tell them apart.
#[derive(Clone, Copy, Debug, PartialEq)]
pub enum Kind {
Person,
Bot,
Group,
Broadcast,
}
/// What a folder's rules look at in a chat.
#[derive(Clone, Copy, Debug, PartialEq)]
pub struct Facts {
pub kind: Kind,
pub contact: bool,
pub muted: bool,
pub unread: bool,
pub archived: bool,
}
/// A folder's rules, read from Telegram's description of it.
#[derive(Clone, Debug, Default, PartialEq)]
pub struct Rules {
contacts: bool,
non_contacts: bool,
groups: bool,
broadcasts: bool,
bots: bool,
exclude_muted: bool,
exclude_read: bool,
exclude_archived: bool,
/// Chats named one by one, pinned ones among them, as the ids this client calls them by.
include: Vec<String>,
exclude: Vec<String>,
}
impl Rules {
/// Whether a chat is in by the kinds of chat the folder takes (and not taken back out), leaving the chats named
/// one by one aside.
fn by_kind(&self, facts: &Facts) -> bool {
let wanted = match facts.kind {
Kind::Person => (self.contacts && facts.contact) || (self.non_contacts && !facts.contact),
Kind::Bot => self.bots,
Kind::Group => self.groups,
Kind::Broadcast => self.broadcasts,
};
wanted && !(self.exclude_muted && facts.muted) && !(self.exclude_read && !facts.unread) && !(self.exclude_archived && facts.archived)
}
/// Whether the folder holds a chat.
pub fn holds(&self, remote: &str, facts: Option<&Facts>) -> bool {
if self.include.iter().any(|r| r == remote) {
return true;
}
if self.exclude.iter().any(|r| r == remote) {
return false;
}
facts.is_some_and(|f| self.by_kind(f))
}
}
/// A folder as Telegram holds it, with what is needed to change it.
#[derive(Clone, Debug)]
pub struct Folder {
pub id: i32,
pub title: String,
pub emoticon: Option<String>,
pub rules: Rules,
/// Shared folders (a link anyone can join by) have no rules and no exceptions.
pub shared: bool,
raw: tl::enums::DialogFilter,
}
/// The chat an input peer names, as this client calls it; `None` for none.
pub fn remote_of_input(peer: &tl::enums::InputPeer, own: i64) -> Option<String> {
use tl::enums::InputPeer as P;
let id = match peer {
P::PeerSelf => PeerId::user_unchecked(own),
P::User(u) => PeerId::user_unchecked(u.user_id),
P::UserFromMessage(u) => PeerId::user_unchecked(u.user_id),
P::Chat(c) => PeerId::chat_unchecked(c.chat_id),
P::Channel(c) => PeerId::channel_unchecked(c.channel_id),
P::ChannelFromMessage(c) => PeerId::channel_unchecked(c.channel_id),
P::Empty => return None,
};
Some(peers::remote_id(id))
}
fn text_of(title: &tl::enums::TextWithEntities) -> String {
let tl::enums::TextWithEntities::Entities(t) = title;
t.text.clone()
}
fn remotes(list: &[tl::enums::InputPeer], own: i64) -> Vec<String> {
list.iter().filter_map(|p| remote_of_input(p, own)).collect()
}
/// A folder read from Telegram's description of it; `None` for the entry that stands for every chat.
pub fn folder_of(raw: &tl::enums::DialogFilter, own: i64) -> Option<Folder> {
match raw {
tl::enums::DialogFilter::Default => None,
tl::enums::DialogFilter::Filter(f) => {
let mut include = remotes(&f.include_peers, own);
for pinned in remotes(&f.pinned_peers, own) {
if !include.contains(&pinned) {
include.push(pinned);
}
}
Some(Folder {
id: f.id,
title: text_of(&f.title),
emoticon: f.emoticon.clone(),
rules: Rules {
contacts: f.contacts,
non_contacts: f.non_contacts,
groups: f.groups,
broadcasts: f.broadcasts,
bots: f.bots,
exclude_muted: f.exclude_muted,
exclude_read: f.exclude_read,
exclude_archived: f.exclude_archived,
include,
exclude: remotes(&f.exclude_peers, own),
},
shared: false,
raw: raw.clone(),
})
}
tl::enums::DialogFilter::Chatlist(f) => {
let mut include = remotes(&f.include_peers, own);
for pinned in remotes(&f.pinned_peers, own) {
if !include.contains(&pinned) {
include.push(pinned);
}
}
Some(Folder { id: f.id, title: text_of(&f.title), emoticon: f.emoticon.clone(), rules: Rules { include, ..Default::default() }, shared: true, raw: raw.clone() })
}
}
}
/// The conversations a folder holds, by the ids this client calls them by: every one it names, and every one of the
/// chats whose facts are known that its rules take.
pub fn members(folder: &Folder, facts: &HashMap<String, Facts>) -> Vec<String> {
let mut out: Vec<String> = folder.rules.include.clone();
let mut seen: HashSet<&str> = folder.rules.include.iter().map(String::as_str).collect();
let mut found: Vec<&String> = facts.keys().collect();
found.sort();
for remote in found {
if !seen.contains(remote.as_str()) && folder.rules.holds(remote, facts.get(remote)) {
seen.insert(remote);
out.push(remote.clone());
}
}
out
}
/// The first folder, in the order given, that holds a chat: the one it is drawn under.
pub fn home_of(folders: &[Folder], facts: &HashMap<String, Facts>, remote: &str) -> Option<i32> {
folders.iter().find(|f| f.rules.holds(remote, facts.get(remote))).map(|f| f.id)
}
/// The folder with a chat put in it.
pub fn with_peer(raw: &tl::enums::DialogFilter, peer: &tl::enums::InputPeer, own: i64) -> Option<tl::enums::DialogFilter> {
let remote = remote_of_input(peer, own)?;
let mut edited = raw.clone();
match &mut edited {
tl::enums::DialogFilter::Default => return None,
tl::enums::DialogFilter::Filter(f) => {
f.exclude_peers.retain(|p| remote_of_input(p, own).as_deref() != Some(remote.as_str()));
if !f.include_peers.iter().chain(&f.pinned_peers).any(|p| remote_of_input(p, own).as_deref() == Some(remote.as_str())) {
f.include_peers.push(peer.clone());
}
}
tl::enums::DialogFilter::Chatlist(f) => {
if !f.include_peers.iter().chain(&f.pinned_peers).any(|p| remote_of_input(p, own).as_deref() == Some(remote.as_str())) {
f.include_peers.push(peer.clone());
}
}
}
Some(edited)
}
/// The folder with a chat taken out: no longer named, and - if its rules would still take it - made an exception.
pub fn without_peer(raw: &tl::enums::DialogFilter, peer: &tl::enums::InputPeer, own: i64, facts: Option<&Facts>) -> Option<tl::enums::DialogFilter> {
let remote = remote_of_input(peer, own)?;
let named = |p: &tl::enums::InputPeer| remote_of_input(p, own).as_deref() == Some(remote.as_str());
let mut edited = raw.clone();
match &mut edited {
tl::enums::DialogFilter::Default => return None,
tl::enums::DialogFilter::Filter(f) => {
f.include_peers.retain(|p| !named(p));
f.pinned_peers.retain(|p| !named(p));
let kept = folder_of(&tl::enums::DialogFilter::Filter(f.clone()), own)?;
if facts.is_some_and(|facts| kept.rules.by_kind(facts)) && !f.exclude_peers.iter().any(named) {
f.exclude_peers.push(peer.clone());
}
}
tl::enums::DialogFilter::Chatlist(f) => {
f.include_peers.retain(|p| !named(p));
f.pinned_peers.retain(|p| !named(p));
}
}
Some(edited)
}
/// A folder with another name.
pub fn renamed(raw: &tl::enums::DialogFilter, name: &str) -> Option<tl::enums::DialogFilter> {
let title = tl::enums::TextWithEntities::Entities(tl::types::TextWithEntities { text: name.to_string(), entities: Vec::new() });
let mut edited = raw.clone();
match &mut edited {
tl::enums::DialogFilter::Default => return None,
tl::enums::DialogFilter::Filter(f) => f.title = title,
tl::enums::DialogFilter::Chatlist(f) => f.title = title,
}
Some(edited)
}
/// A folder that holds one chat and nothing else: how a category becomes a folder.
pub fn starting_with(id: i32, name: &str, peer: &tl::enums::InputPeer) -> tl::enums::DialogFilter {
tl::enums::DialogFilter::Filter(tl::types::DialogFilter {
contacts: false,
non_contacts: false,
groups: false,
broadcasts: false,
bots: false,
exclude_muted: false,
exclude_read: false,
exclude_archived: false,
title_noanimate: false,
id,
title: tl::enums::TextWithEntities::Entities(tl::types::TextWithEntities { text: name.to_string(), entities: Vec::new() }),
emoticon: None,
color: None,
pinned_peers: Vec::new(),
include_peers: vec![peer.clone()],
exclude_peers: Vec::new(),
})
}
/// The number for a folder made now: the next one after those in use.
pub fn next_id(folders: &[tl::enums::DialogFilter]) -> i32 {
folders
.iter()
.map(|f| match f {
tl::enums::DialogFilter::Default => 0,
tl::enums::DialogFilter::Filter(f) => f.id,
tl::enums::DialogFilter::Chatlist(f) => f.id,
})
.max()
.map_or(FIRST_ID, |m| (m + 1).max(FIRST_ID))
}
fn id_of(raw: &tl::enums::DialogFilter) -> i32 {
match raw {
tl::enums::DialogFilter::Default => 0,
tl::enums::DialogFilter::Filter(f) => f.id,
tl::enums::DialogFilter::Chatlist(f) => f.id,
}
}
// What is known of each account's chats and folders, kept for the window to ask for and for the edits below.
type FactsByAccount = Mutex<HashMap<String, HashMap<String, Facts>>>;
fn all_facts() -> &'static FactsByAccount {
static F: std::sync::OnceLock<FactsByAccount> = std::sync::OnceLock::new();
F.get_or_init(Default::default)
}
type Held = Mutex<HashMap<String, Vec<Value>>>;
fn held() -> &'static Held {
static H: std::sync::OnceLock<Held> = std::sync::OnceLock::new();
H.get_or_init(Default::default)
}
/// Notes what a folder's rules look at in a chat, as the conversations are read.
pub fn note(account_id: &str, remote: String, facts: Facts) {
all_facts().lock().unwrap_or_else(|e| e.into_inner()).entry(account_id.to_string()).or_default().insert(remote, facts);
}
fn facts_of(account_id: &str) -> HashMap<String, Facts> {
all_facts().lock().unwrap_or_else(|e| e.into_inner()).get(account_id).cloned().unwrap_or_default()
}
/// Forgets an account's folders when it goes.
pub fn forget(account_id: &str) {
all_facts().lock().unwrap_or_else(|e| e.into_inner()).remove(account_id);
held().lock().unwrap_or_else(|e| e.into_inner()).remove(account_id);
}
/// The folders as the window is told of them: in Telegram's order, each with the conversations it holds.
pub fn summary(folders: &[Folder], facts: &HashMap<String, Facts>) -> Vec<Value> {
folders
.iter()
.map(|f| json!({ "id": f.id, "name": f.title, "emoticon": f.emoticon, "members": members(f, facts), "shared": f.shared }))
.collect()
}
/// What the window last heard of an account's folders.
pub fn current(account_id: &str) -> Value {
Value::Array(held().lock().unwrap_or_else(|e| e.into_inner()).get(account_id).cloned().unwrap_or_default())
}
async fn read(handle: &Handle) -> Result<Vec<tl::enums::DialogFilter>> {
let answer = handle.client.invoke(&tl::functions::messages::GetDialogFilters {}).await.map_err(|e| anyhow!("{}", explain(&e)))?;
let tl::enums::messages::DialogFilters::Filters(found) = answer;
Ok(found.filters)
}
/// Reads the account's folders and tells the window, if they are not what it was last told.
pub async fn sync(state: &AppState, account_id: &str, handle: &Handle) -> Result<Vec<Value>> {
let raws = read(handle).await?;
let folders: Vec<Folder> = raws.iter().filter_map(|r| folder_of(r, handle.own.user_id)).collect();
let list = summary(&folders, &facts_of(account_id));
let changed = {
let mut all = held().lock().unwrap_or_else(|e| e.into_inner());
let before = all.insert(account_id.to_string(), list.clone());
before.as_ref() != Some(&list)
};
if changed {
state.events.emit("telegramFolders", json!({ "accountId": account_id, "folders": list }));
}
Ok(list)
}
/// Telegram says the account's folders changed, on this client or another: read them again.
pub fn changed(state: &AppState, account_id: &str, handle: &Handle) {
let (state, account_id, handle) = (state.clone(), account_id.to_string(), handle.clone());
tokio::spawn(async move {
if let Err(e) = sync(&state, &account_id, &handle).await {
tracing::debug!("telegram[{account_id}]: reading the folders: {e:#}");
}
});
}
fn handle_for(state: &AppState, account_id: &str) -> Result<Handle, String> {
state.runtime.telegram_handle(account_id).ok_or_else(|| "that Telegram account is not connected".to_string())
}
async fn save(handle: &Handle, id: i32, filter: Option<tl::enums::DialogFilter>) -> Result<(), String> {
handle.client.invoke(&tl::functions::messages::UpdateDialogFilter { id, filter }).await.map_err(|e| folder_words(&explain(&e)))?;
Ok(())
}
/// Telegram's refusals about folders, said so a person can act on them.
fn folder_words(raw: &str) -> String {
// Telegram's codes arrive as FILTER_INCLUDE_EMPTY or, once worded for a person, as "filter include empty".
let upper = raw.to_uppercase().replace(' ', "_");
for (code, words) in [
("FILTER_INCLUDE_EMPTY", "Telegram does not keep an empty folder: it needs at least one chat in it. Delete the folder instead"),
("FILTER_TITLE_EMPTY", "A folder needs a name"),
("DIALOG_FILTERS_TOO_MUCH", "This account already has as many folders as Telegram allows"),
("FILTER_NOT_SUPPORTED", "Telegram does not allow that change to this folder"),
("CHATLISTS_TOO_MUCH", "Telegram will not share any more folders from this account"),
("PEERS_LIST_EMPTY", "Telegram does not keep an empty folder: it needs at least one chat in it. Delete the folder instead"),
] {
if upper.contains(code) {
return words.to_string();
}
}
raw.to_string()
}
/// A folder by number, in the order Telegram has them.
fn find(folders: &[Folder], id: i32) -> Result<&Folder, String> {
folders.iter().find(|f| f.id == id).ok_or_else(|| "Telegram has no such folder any more".to_string())
}
/// Makes a folder from a conversation: how a category made here comes to exist on Telegram.
pub async fn create(state: &AppState, buffer: &Buffer, name: &str) -> Result<(), String> {
let name = name.trim();
if name.is_empty() {
return Err("A folder needs a name".to_string());
}
let (handle, peer) = peer_for(state, buffer).await?;
let raws = read(&handle).await.map_err(|e| e.to_string())?;
let filter = starting_with(next_id(&raws), name, &peer.into());
save(&handle, id_of(&filter), Some(filter)).await?;
sync(state, &buffer.account_id, &handle).await.map_err(|e| e.to_string())?;
Ok(())
}
/// Gives a folder another name.
pub async fn rename(state: &AppState, account_id: &str, id: i32, name: &str) -> Result<(), String> {
let name = name.trim();
if name.is_empty() {
return Err("A folder needs a name".to_string());
}
let handle = handle_for(state, account_id)?;
let raws = read(&handle).await.map_err(|e| e.to_string())?;
let raw = raws.iter().find(|r| id_of(r) == id && id != 0).ok_or("Telegram has no such folder any more")?;
save(&handle, id, renamed(raw, name)).await?;
sync(state, account_id, &handle).await.map_err(|e| e.to_string())?;
Ok(())
}
/// Deletes a folder. The chats in it stay where they are; only the tab goes.
pub async fn delete(state: &AppState, account_id: &str, id: i32) -> Result<(), String> {
let handle = handle_for(state, account_id)?;
if id == 0 {
return Err("The list of every chat cannot be deleted".to_string());
}
save(&handle, id, None).await?;
sync(state, account_id, &handle).await.map_err(|e| e.to_string())?;
Ok(())
}
/// Puts the account's folders in the order given. Folders not named keep their places after those that are.
pub async fn reorder(state: &AppState, account_id: &str, order: &[i32]) -> Result<(), String> {
let handle = handle_for(state, account_id)?;
let raws = read(&handle).await.map_err(|e| e.to_string())?;
let all: Vec<i32> = raws.iter().map(id_of).collect();
let wanted = ordered(&all, order);
if wanted == all {
return Ok(());
}
handle.client.invoke(&tl::functions::messages::UpdateDialogFiltersOrder { order: wanted }).await.map_err(|e| folder_words(&explain(&e)))?;
sync(state, account_id, &handle).await.map_err(|e| e.to_string())?;
Ok(())
}
/// The folders' numbers in the order asked for. Those named take the places of those named, in the order named; any
/// not named - the entry for every chat, which is not drawn and so cannot have been moved - stay where they were.
pub fn ordered(all: &[i32], asked: &[i32]) -> Vec<i32> {
let mut named: Vec<i32> = Vec::new();
for id in asked {
if all.contains(id) && !named.contains(id) {
named.push(*id);
}
}
let mut fill = named.clone().into_iter();
let mut out = all.to_vec();
for slot in out.iter_mut().filter(|id| named.contains(id)) {
*slot = fill.next().unwrap_or(*slot);
}
out
}
/// Files a conversation under a folder (`to`), or under none: out of the one it is drawn under and into the new one.
pub async fn file(state: &AppState, buffer: &Buffer, to: Option<i32>) -> Result<(), String> {
let (handle, peer) = peer_for(state, buffer).await?;
let own = handle.own.user_id;
let input: tl::enums::InputPeer = peer.into();
let remote = remote_of_input(&input, own).ok_or("that is not a chat Telegram can file")?;
let raws = read(&handle).await.map_err(|e| e.to_string())?;
let folders: Vec<Folder> = raws.iter().filter_map(|r| folder_of(r, own)).collect();
let facts = facts_of(&buffer.account_id);
let from = home_of(&folders, &facts, &remote);
if from == to {
return Ok(());
}
// Put in first: if Telegram refuses to take it out of the old one, it is at least where it was asked to go.
if let Some(id) = to {
let folder = find(&folders, id)?;
let edited = with_peer(&folder.raw, &input, own).ok_or("Telegram has no such folder any more")?;
save(&handle, id, Some(edited)).await?;
}
if let Some(id) = from {
let folder = find(&folders, id)?;
let edited = without_peer(&folder.raw, &input, own, facts.get(&remote)).ok_or("Telegram has no such folder any more")?;
if let Err(e) = save(&handle, id, Some(edited)).await {
let _ = sync(state, &buffer.account_id, &handle).await;
return Err(format!("It is in the new folder, but could not be taken out of \"{}\": {e}", folder.title));
}
}
sync(state, &buffer.account_id, &handle).await.map_err(|e| e.to_string())?;
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
const OWN: i64 = 1;
fn user(id: i64) -> tl::enums::InputPeer {
tl::enums::InputPeer::User(tl::types::InputPeerUser { user_id: id, access_hash: 7 })
}
fn channel(id: i64) -> tl::enums::InputPeer {
tl::enums::InputPeer::Channel(tl::types::InputPeerChannel { channel_id: id, access_hash: 7 })
}
fn filter(id: i32, name: &str, edit: impl FnOnce(&mut tl::types::DialogFilter)) -> tl::enums::DialogFilter {
let tl::enums::DialogFilter::Filter(mut f) = starting_with(id, name, &user(100)) else { unreachable!() };
f.include_peers.clear();
edit(&mut f);
tl::enums::DialogFilter::Filter(f)
}
/// A channel's id as this client writes it.
fn ch(id: i64) -> String {
peers::remote_id(PeerId::channel_unchecked(id))
}
fn facts(kind: Kind) -> Facts {
Facts { kind, contact: false, muted: false, unread: true, archived: false }
}
#[test]
fn a_chat_is_called_by_the_id_this_client_gives_it() {
assert_eq!(remote_of_input(&user(100), OWN).as_deref(), Some("100"));
assert_eq!(remote_of_input(&channel(555), OWN), Some(ch(555)));
assert!(ch(555).starts_with("-100") && ch(555).ends_with("555"));
assert_eq!(remote_of_input(&tl::enums::InputPeer::Chat(tl::types::InputPeerChat { chat_id: 9 }), OWN).as_deref(), Some("-9"));
assert_eq!(remote_of_input(&tl::enums::InputPeer::PeerSelf, OWN).as_deref(), Some("1"), "saved messages are the account's own chat");
assert_eq!(remote_of_input(&tl::enums::InputPeer::Empty, OWN), None);
}
#[test]
fn a_folder_holds_the_chats_it_names_and_the_kinds_it_takes_less_the_exceptions() {
let raw = filter(2, "Work", |f| {
f.bots = true;
f.groups = true;
f.exclude_muted = true;
f.include_peers = vec![user(100)];
f.pinned_peers = vec![user(101)];
f.exclude_peers = vec![user(200)];
});
let folder = folder_of(&raw, OWN).unwrap();
assert_eq!((folder.id, folder.title.as_str(), folder.shared), (2, "Work", false));
let mut known = HashMap::new();
known.insert("100".to_string(), facts(Kind::Person));
known.insert("200".to_string(), facts(Kind::Bot));
known.insert("201".to_string(), facts(Kind::Bot));
known.insert(ch(300), facts(Kind::Group));
known.insert(ch(301), Facts { muted: true, ..facts(Kind::Group) });
known.insert(ch(302), facts(Kind::Broadcast));
known.insert("102".to_string(), facts(Kind::Person));
// Named ones first (pinned among them, though Telegram keeps them apart), then the kinds, in a steady order.
assert_eq!(members(&folder, &known), vec!["100".to_string(), "101".to_string(), ch(300), "201".to_string()]);
}
#[test]
fn contacts_and_everyone_else_are_told_apart() {
let raw = filter(2, "People", |f| f.contacts = true);
let folder = folder_of(&raw, OWN).unwrap();
let friend = Facts { contact: true, ..facts(Kind::Person) };
assert!(folder.rules.holds("5", Some(&friend)));
assert!(!folder.rules.holds("6", Some(&facts(Kind::Person))));
assert!(!folder.rules.holds("7", None), "a chat nothing is known of is held only if it is named");
let others = folder_of(&filter(3, "Strangers", |f| f.non_contacts = true), OWN).unwrap();
assert!(others.rules.holds("6", Some(&facts(Kind::Person))));
assert!(!others.rules.holds("5", Some(&friend)));
}
#[test]
fn read_and_archived_chats_can_be_left_out_of_a_folder() {
let raw = filter(2, "Unread", |f| {
f.groups = true;
f.exclude_read = true;
f.exclude_archived = true;
});
let folder = folder_of(&raw, OWN).unwrap();
assert!(folder.rules.holds("-1", Some(&facts(Kind::Group))));
assert!(!folder.rules.holds("-1", Some(&Facts { unread: false, ..facts(Kind::Group) })));
assert!(!folder.rules.holds("-1", Some(&Facts { archived: true, ..facts(Kind::Group) })));
}
#[test]
fn the_entry_for_every_chat_is_not_a_folder() {
assert!(folder_of(&tl::enums::DialogFilter::Default, OWN).is_none());
}
#[test]
fn a_chat_is_drawn_under_the_first_folder_that_holds_it() {
let a = folder_of(&filter(2, "Groups", |f| f.groups = true), OWN).unwrap();
let b = folder_of(&filter(3, "Mine", |f| f.include_peers = vec![channel(300)]), OWN).unwrap();
let mut known = HashMap::new();
known.insert(ch(300), facts(Kind::Group));
known.insert("5".to_string(), facts(Kind::Person));
assert_eq!(home_of(&[a.clone(), b.clone()], &known, &ch(300)), Some(2));
assert_eq!(home_of(&[b, a], &known, &ch(300)), Some(3));
assert_eq!(home_of(&[], &known, "5"), None);
}
#[test]
fn putting_a_chat_in_names_it_once_and_lifts_an_exception() {
let raw = filter(2, "Work", |f| f.exclude_peers = vec![user(100)]);
let put = with_peer(&raw, &user(100), OWN).unwrap();
let tl::enums::DialogFilter::Filter(f) = &put else { unreachable!() };
assert_eq!(f.include_peers.len(), 1);
assert!(f.exclude_peers.is_empty());
let again = with_peer(&put, &user(100), OWN).unwrap();
let tl::enums::DialogFilter::Filter(f) = &again else { unreachable!() };
assert_eq!(f.include_peers.len(), 1, "a chat already named is not named twice");
}
#[test]
fn taking_a_chat_out_of_a_named_list_is_taking_it_off_the_list() {
let raw = filter(2, "Work", |f| {
f.include_peers = vec![user(100), user(101)];
f.pinned_peers = vec![user(100)];
});
let out = without_peer(&raw, &user(100), OWN, Some(&facts(Kind::Person))).unwrap();
let tl::enums::DialogFilter::Filter(f) = &out else { unreachable!() };
assert_eq!(remotes(&f.include_peers, OWN), vec!["101"]);
assert!(f.pinned_peers.is_empty());
assert!(f.exclude_peers.is_empty(), "nothing else held it, so there is no exception to make");
}
#[test]
fn taking_a_chat_out_of_a_folder_that_holds_it_by_a_rule_makes_it_an_exception() {
let raw = filter(2, "Groups", |f| f.groups = true);
let out = without_peer(&raw, &channel(300), OWN, Some(&facts(Kind::Group))).unwrap();
let tl::enums::DialogFilter::Filter(f) = &out else { unreachable!() };
assert_eq!(remotes(&f.exclude_peers, OWN), vec![ch(300)]);
let folder = folder_of(&out, OWN).unwrap();
assert!(!folder.rules.holds(&ch(300), Some(&facts(Kind::Group))));
assert!(folder.rules.holds(&ch(301), Some(&facts(Kind::Group))), "the others are still held");
// And not twice.
let again = without_peer(&out, &channel(300), OWN, Some(&facts(Kind::Group))).unwrap();
let tl::enums::DialogFilter::Filter(f) = &again else { unreachable!() };
assert_eq!(f.exclude_peers.len(), 1);
}
#[test]
fn a_shared_folder_has_no_exceptions_to_make() {
let raw = tl::enums::DialogFilter::Chatlist(tl::types::DialogFilterChatlist {
has_my_invites: false,
title_noanimate: false,
id: 4,
title: tl::enums::TextWithEntities::Entities(tl::types::TextWithEntities { text: "Shared".into(), entities: vec![] }),
emoticon: None,
color: None,
pinned_peers: vec![],
include_peers: vec![user(100), user(101)],
});
let folder = folder_of(&raw, OWN).unwrap();
assert!(folder.shared);
let out = without_peer(&raw, &user(100), OWN, Some(&facts(Kind::Person))).unwrap();
assert_eq!(folder_of(&out, OWN).unwrap().rules.include, vec!["101"]);
assert_eq!(folder_of(&renamed(&out, "Team").unwrap(), OWN).unwrap().title, "Team");
}
#[test]
fn renaming_keeps_everything_but_the_name() {
let raw = filter(2, "Work", |f| {
f.bots = true;
f.include_peers = vec![user(100)];
f.emoticon = Some("💼".into());
});
let renamed = renamed(&raw, "Job").unwrap();
let (before, after) = (folder_of(&raw, OWN).unwrap(), folder_of(&renamed, OWN).unwrap());
assert_eq!(after.title, "Job");
assert_eq!((before.rules, before.emoticon, before.id), (after.rules, after.emoticon, after.id));
assert!(super::renamed(&tl::enums::DialogFilter::Default, "x").is_none());
}
#[test]
fn a_new_folder_has_the_next_number_after_those_in_use_and_never_below_two() {
assert_eq!(next_id(&[]), 2);
assert_eq!(next_id(&[tl::enums::DialogFilter::Default]), 2);
assert_eq!(next_id(&[tl::enums::DialogFilter::Default, filter(2, "A", |_| {}), filter(7, "B", |_| {})]), 8);
}
#[test]
fn a_new_folder_starts_with_the_one_chat_it_was_made_for() {
let made = folder_of(&starting_with(3, "Friends", &user(100)), OWN).unwrap();
assert_eq!((made.id, made.title.as_str()), (3, "Friends"));
assert_eq!(made.rules.include, vec!["100"]);
assert!(!made.rules.holds("101", Some(&facts(Kind::Person))), "no kind of chat is taken until asked for");
}
#[test]
fn folders_are_put_in_the_order_asked_for_and_the_entry_for_every_chat_keeps_its_place() {
assert_eq!(ordered(&[0, 2, 3, 4], &[4, 2, 3]), vec![0, 4, 2, 3]);
assert_eq!(ordered(&[2, 3, 0, 4], &[4, 3]), vec![2, 4, 0, 3], "the two named swap places, the rest stay");
assert_eq!(ordered(&[0, 2, 3], &[9, 3, 3, 2]), vec![0, 3, 2], "numbers that are not folders, and repeats, are dropped");
assert_eq!(ordered(&[0, 2, 3], &[]), vec![0, 2, 3]);
}
#[test]
fn the_summary_is_in_telegrams_order_with_each_folders_conversations() {
let a = folder_of(&filter(3, "Second", |f| f.include_peers = vec![user(100)]), OWN).unwrap();
let b = folder_of(&filter(2, "First", |f| f.bots = true), OWN).unwrap();
let mut known = HashMap::new();
known.insert("200".to_string(), facts(Kind::Bot));
let out = summary(&[a, b], &known);
assert_eq!(out[0]["id"], 3);
assert_eq!(out[0]["members"], json!(["100"]));
assert_eq!(out[1]["name"], "First");
assert_eq!(out[1]["members"], json!(["200"]));
}
#[test]
fn telegrams_refusals_about_folders_are_said_in_words() {
assert!(folder_words("rpc error 400: FILTER_INCLUDE_EMPTY").contains("empty folder"));
assert!(folder_words("DIALOG_FILTERS_TOO_MUCH").contains("as many folders"));
// As the library words it after `explain`: seen from the real account.
assert!(folder_words("Telegram refused: filter include empty").contains("empty folder"));
assert_eq!(folder_words("SOMETHING_ELSE"), "SOMETHING_ELSE");
}
}
src/backend/telegram/gifs.rs
+197
View File
@@ -0,0 +1,197 @@
//! GIFs: the account's saved ones, and a search through Telegram's own GIF bot.
//!
//! Telegram does not send a GIF as a link: a GIF is a short silent video that lives on Telegram, and sending one is
//! sending that file. The account keeps a list of saved GIFs, and the search is the inline bot `@gif`, which answers
//! with files of the same kind. Both are listed as the picker's cells; the pictures and the videos are named in the
//! cache and fetched when the picker draws them, from the files remembered here (a file is fetched with its access
//! key and a reference that goes stale, so what was listed is what can be fetched).
use super::media::{download_thumb, download_to, extension_of, MEDIA};
use super::messages;
use super::send::{explain, peer_for};
use super::Handle;
use crate::model::Buffer;
use crate::state::AppState;
use anyhow::{anyhow, Context, Result};
use grammers_client::media::Document;
use grammers_client::message::InputMessage;
use grammers_client::tl;
use serde_json::{json, Value};
use std::collections::HashMap;
use std::path::{Path, PathBuf};
use std::sync::Mutex;
/// How many GIFs a list shows.
const LIMIT: usize = 100;
/// The documents that were listed, by (account, id): what the picker names is what can be fetched or sent.
type Known = Mutex<HashMap<(i64, i64), tl::types::Document>>;
fn known() -> &'static Known {
static K: std::sync::OnceLock<Known> = std::sync::OnceLock::new();
K.get_or_init(Default::default)
}
/// A GIF's cached file: `tgif_<account>_<id>_<thumb|full>.<ext>`.
fn name(account: i64, id: i64, thumb: bool, ext: &str) -> String {
format!("tgif_{account}_{id}_{}.{ext}", if thumb { "thumb" } else { "full" })
}
/// What a `tgif_` name says: the account, the document and which of the two files it is.
fn parse_name(file: &str) -> Option<(i64, i64, bool)> {
let rest = file.strip_prefix("tgif_")?;
let stem = rest.rsplit_once('.')?.0;
let mut parts = stem.split('_');
let account = parts.next()?.parse().ok()?;
let id = parts.next()?.parse().ok()?;
let thumb = match parts.next()? {
"thumb" => true,
"full" => false,
_ => return None,
};
parts.next().is_none().then_some((account, id, thumb))
}
fn dimensions(doc: &tl::types::Document) -> (i32, i32) {
doc.attributes
.iter()
.find_map(|a| match a {
tl::enums::DocumentAttribute::Video(v) => Some((v.w, v.h)),
tl::enums::DocumentAttribute::ImageSize(s) => Some((s.w, s.h)),
_ => None,
})
.unwrap_or((0, 0))
}
fn title_of(doc: &tl::types::Document) -> String {
doc.attributes
.iter()
.find_map(|a| match a {
tl::enums::DocumentAttribute::Filename(f) => Some(f.file_name.clone()),
_ => None,
})
.unwrap_or_else(|| "GIF".to_string())
}
pub(super) fn wrapper(doc: &tl::types::Document) -> Document {
Document::from_raw_media(tl::types::MessageMediaDocument {
nopremium: false,
spoiler: false,
video: true,
round: false,
voice: false,
document: Some(tl::enums::Document::Document(doc.clone())),
alt_documents: None,
video_cover: None,
video_timestamp: None,
ttl_seconds: None,
})
}
/// A listed document as the picker's cell: its files named in the cache, fetched when drawn.
fn entry(account: i64, doc: &tl::types::Document) -> Value {
let (w, h) = dimensions(doc);
let full = format!("file://{}", crate::media_cache::dir(MEDIA).join(name(account, doc.id, false, &extension_of(None, Some(&doc.mime_type)))).display());
let thumb = format!("file://{}", crate::media_cache::dir(MEDIA).join(name(account, doc.id, true, "jpg")).display());
json!({ "id": doc.id.to_string(), "title": title_of(doc), "url": "", "src": full, "gifSrc": full, "preview": thumb, "width": w, "height": h })
}
fn remember(account: i64, docs: Vec<tl::types::Document>) -> Vec<Value> {
let mut out = Vec::new();
let mut held = known().lock().unwrap_or_else(|e| e.into_inner());
for doc in docs.into_iter().take(LIMIT) {
out.push(entry(account, &doc));
held.insert((account, doc.id), doc);
}
out
}
fn documents(list: Vec<tl::enums::Document>) -> Vec<tl::types::Document> {
list.into_iter()
.filter_map(|d| match d {
tl::enums::Document::Document(d) => Some(d),
_ => None,
})
.collect()
}
/// What the picker shows: the saved GIFs when nothing is typed, else what Telegram's GIF search finds.
pub async fn list(state: &AppState, buffer: &Buffer, query: &str) -> Result<Value, String> {
let (handle, peer) = peer_for(state, buffer).await?;
let account = handle.own.user_id;
let docs = if query.trim().is_empty() {
let answer = handle.client.invoke(&tl::functions::messages::GetSavedGifs { hash: 0 }).await.map_err(|e| explain(&e))?;
match answer {
tl::enums::messages::SavedGifs::Gifs(g) => documents(g.gifs),
tl::enums::messages::SavedGifs::NotModified => Vec::new(),
}
} else {
search(&handle, peer, query.trim()).await.map_err(|e| e.to_string())?
};
Ok(json!({ "gifs": remember(account, docs), "categories": [] }))
}
async fn search(handle: &Handle, peer: grammers_session::types::PeerRef, query: &str) -> Result<Vec<tl::types::Document>> {
let bot = handle.client.resolve_username("gif").await.map_err(|e| anyhow!("{}", explain(&e)))?.ok_or_else(|| anyhow!("Telegram's GIF search is not available"))?;
let bot_ref = bot.to_ref().await.ok().flatten().ok_or_else(|| anyhow!("Telegram's GIF search is not available"))?;
let answer = handle
.client
.invoke(&tl::functions::messages::GetInlineBotResults { bot: bot_ref.into(), peer: peer.into(), geo_point: None, query: query.to_string(), offset: String::new() })
.await
.map_err(|e| anyhow!("{}", explain(&e)))?;
let tl::enums::messages::BotResults::Results(found) = answer;
Ok(found
.results
.into_iter()
.filter_map(|r| match r {
tl::enums::BotInlineResult::BotInlineMediaResult(m) => m.document,
_ => None,
})
.filter_map(|d| match d {
tl::enums::Document::Document(d) => Some(d),
_ => None,
})
.collect())
}
/// Sends a GIF that was listed, to a conversation.
pub async fn send(state: &AppState, buffer: &Buffer, id: &str) -> Result<(), String> {
let id: i64 = id.parse().map_err(|_| "that is not a GIF".to_string())?;
let (handle, peer) = peer_for(state, buffer).await?;
let doc = known().lock().unwrap_or_else(|e| e.into_inner()).get(&(handle.own.user_id, id)).cloned().ok_or_else(|| "That GIF is no longer listed: open the picker again".to_string())?;
let media = wrapper(&doc).to_raw_input_media();
// In a topic of a group the GIF answers the topic's first message, which is how it is sent there.
let reply = super::topics::topic_of_buffer(buffer);
let sent = handle.client.send_message(peer, InputMessage::new().media(tl::enums::InputMedia::Document(media)).reply_to(reply)).await.map_err(|e| explain(&e))?;
messages::record(state, &buffer.account_id, &handle.own, &sent);
Ok(())
}
/// Fetches one of a listed GIF's files again, for the window asking for it.
pub async fn restore(state: &AppState, file: &str, path: &Path) -> Result<Option<PathBuf>> {
let Some((account, id, thumb)) = parse_name(file) else { return Ok(None) };
let handle = state.runtime.telegram_handle(&format!("telegram:{account}")).context("that Telegram account is not connected")?;
let doc = known().lock().unwrap_or_else(|e| e.into_inner()).get(&(account, id)).cloned().context("that GIF is no longer listed")?;
let wrapped = wrapper(&doc);
if thumb {
download_thumb(&handle.client, &wrapped, path).await?;
} else {
download_to(&handle.client, &wrapped, path, false).await?;
}
Ok(Some(path.to_path_buf()))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_gifs_files_are_named_and_read_back() {
let video = name(1234567890, 5_000_000_000_001, false, "mp4");
assert_eq!(video, "tgif_1234567890_5000000000001_full.mp4");
assert_eq!(parse_name(&video), Some((1234567890, 5_000_000_000_001, false)));
assert_eq!(parse_name(&name(1, 7, true, "jpg")), Some((1, 7, true)));
assert_eq!(parse_name("tg_1_n5_2_full.jpg"), None);
assert_eq!(parse_name("tgif_1_2_other.jpg"), None);
}
}
src/backend/telegram/history.rs
+152
View File
@@ -0,0 +1,152 @@
//! A conversation's history, read when it is opened and again when it is scrolled back.
//!
//! Telegram keeps every message, so opening a conversation asks for its recent end, and scrolling to the top asks for
//! the part before. What is stored is returned by `getBacklog` either way, so a failure to reach Telegram leaves what
//! there is rather than an error.
use super::peers;
use super::topics;
use crate::model::Buffer;
use crate::state::AppState;
use anyhow::{anyhow, Result};
use grammers_session::Session;
/// How many messages a page asks for.
const PAGE: usize = 50;
/// A page of one topic's messages, as the messages themselves. Telegram lists a topic's messages as raw ones, so the
/// ids are taken from that answer and the messages asked for by id: the library hands those over whole.
async fn topic_page(
handle: &super::Handle,
peer: grammers_session::types::PeerRef,
topic: i32,
offset_id: i32,
offset_date: i32,
add_offset: i32,
limit: i32,
) -> Result<Vec<grammers_client::message::Message>> {
use grammers_client::tl;
let answer = handle
.client
.invoke(&tl::functions::messages::GetReplies { peer: peer.into(), msg_id: topic, offset_id, offset_date, add_offset, limit, max_id: 0, min_id: 0, hash: 0 })
.await
.map_err(|e| anyhow!("{}", super::send::explain(&e)))?;
let listed = match answer {
tl::enums::messages::Messages::Messages(m) => m.messages,
tl::enums::messages::Messages::Slice(m) => m.messages,
tl::enums::messages::Messages::ChannelMessages(m) => m.messages,
tl::enums::messages::Messages::NotModified(_) => Vec::new(),
};
let mut ids: Vec<i32> = listed
.iter()
.filter_map(|m| match m {
tl::enums::Message::Message(m) => Some(m.id),
tl::enums::Message::Service(m) => Some(m.id),
tl::enums::Message::Empty(_) => None,
})
.collect();
ids.sort_unstable();
ids.dedup();
if ids.is_empty() {
return Ok(Vec::new());
}
let found = handle.client.get_messages_by_id(peer, &ids).await.map_err(|e| anyhow!("{}", super::send::explain(&e)))?;
Ok(found.into_iter().flatten().collect())
}
async fn read(state: &AppState, buffer: &Buffer, before: Option<i64>) -> Result<usize> {
let handle = state.runtime.telegram_handle(&buffer.account_id).ok_or_else(|| anyhow!("this Telegram account is not connected"))?;
let peer = buffer.remote_id.as_deref().and_then(peers::peer_id_of).ok_or_else(|| anyhow!("this is not a Telegram conversation"))?;
let peer = handle.session.peer_ref(peer).await.ok().flatten().ok_or_else(|| anyhow!("Telegram has not told this account about that conversation yet"))?;
if let Some(topic) = topics::topic_of_buffer(buffer) {
let date = before.filter(|t| *t > 0).map(|t| t.clamp(0, i64::from(i32::MAX)) as i32).unwrap_or(0);
let got = topic_page(&handle, peer, topic, 0, date, 0, PAGE as i32).await?;
let count = got.len();
for message in &got {
super::messages::record(state, &buffer.account_id, &handle.own, message);
}
return Ok(count);
}
let mut iter = handle.client.iter_messages(peer).limit(PAGE);
if let Some(ts) = before.filter(|t| *t > 0) {
iter = iter.offset_date(ts.clamp(0, i64::from(i32::MAX)) as i32);
}
let mut got = Vec::new();
while let Some(message) = iter.next().await.map_err(|e| anyhow!("{e}"))? {
got.push(message);
}
// Oldest first, so a reply finds the message it answers already stored.
got.reverse();
let count = got.len();
for message in &got {
super::messages::record(state, &buffer.account_id, &handle.own, message);
}
Ok(count)
}
/// Opening a conversation: its recent end.
pub async fn open(state: &AppState, buffer: &Buffer) -> Result<usize> {
read(state, buffer, None).await
}
/// Scrolling back: what came before `before` (a Unix time).
pub async fn older(state: &AppState, buffer: &Buffer, before: i64) -> Result<usize> {
read(state, buffer, Some(before)).await
}
/// Reads the messages around one, so a jump to an old message (a pin, a reply, a search result) arrives with what was
/// said before and after it. Returns the message's time.
pub async fn context(state: &AppState, buffer: &Buffer, msg_id: &str) -> Result<i64> {
let id: i32 = msg_id.parse().map_err(|_| anyhow!("that is not a Telegram message"))?;
let handle = state.runtime.telegram_handle(&buffer.account_id).ok_or_else(|| anyhow!("this Telegram account is not connected"))?;
let peer = buffer.remote_id.as_deref().and_then(peers::peer_id_of).ok_or_else(|| anyhow!("this is not a Telegram conversation"))?;
let peer = handle.session.peer_ref(peer).await.ok().flatten().ok_or_else(|| anyhow!("Telegram has not told this account about that conversation yet"))?;
if let Some(topic) = topics::topic_of_buffer(buffer) {
let mut around = topic_page(&handle, peer, topic, id.saturating_add(1), 0, 0, 26).await?;
around.extend(topic_page(&handle, peer, topic, id, 0, -25, 25).await?);
around.sort_by_key(|m| m.id());
around.dedup_by_key(|m| m.id());
for message in &around {
super::messages::record(state, &buffer.account_id, &handle.own, message);
}
return around.iter().find(|m| m.id() == id).map(|m| m.date().as_second()).ok_or_else(|| anyhow!("that message is no longer there"));
}
let mut around = Vec::new();
// Itself and what came before it, newest first...
let mut older = handle.client.iter_messages(peer).offset_id(id.saturating_add(1)).limit(26);
while let Some(message) = older.next().await.map_err(|e| anyhow!("{e}"))? {
around.push(message);
}
// ...and what came after, oldest first.
let mut newer = handle.client.iter_messages(peer).reverse(true).offset_id(id).limit(25);
while let Some(message) = newer.next().await.map_err(|e| anyhow!("{e}"))? {
around.push(message);
}
around.sort_by_key(|m| m.id());
for message in &around {
super::messages::record(state, &buffer.account_id, &handle.own, message);
}
around.iter().find(|m| m.id() == id).map(|m| m.date().as_second()).ok_or_else(|| anyhow!("that message is no longer there"))
}
/// Reads on from a message towards the present, for a reader who arrived in the middle of a conversation.
pub async fn newer(state: &AppState, buffer: &Buffer, msg_id: &str) -> Result<usize> {
let id: i32 = msg_id.parse().map_err(|_| anyhow!("that is not a Telegram message"))?;
let handle = state.runtime.telegram_handle(&buffer.account_id).ok_or_else(|| anyhow!("this Telegram account is not connected"))?;
let peer = buffer.remote_id.as_deref().and_then(peers::peer_id_of).ok_or_else(|| anyhow!("this is not a Telegram conversation"))?;
let peer = handle.session.peer_ref(peer).await.ok().flatten().ok_or_else(|| anyhow!("Telegram has not told this account about that conversation yet"))?;
if let Some(topic) = topics::topic_of_buffer(buffer) {
let got = topic_page(&handle, peer, topic, id, 0, -(PAGE as i32), PAGE as i32).await?;
for message in got.iter().filter(|m| m.id() > id) {
super::messages::record(state, &buffer.account_id, &handle.own, message);
}
return Ok(got.iter().filter(|m| m.id() > id).count());
}
let mut iter = handle.client.iter_messages(peer).reverse(true).offset_id(id).limit(PAGE);
let mut count = 0;
while let Some(message) = iter.next().await.map_err(|e| anyhow!("{e}"))? {
super::messages::record(state, &buffer.account_id, &handle.own, &message);
count += 1;
}
Ok(count)
}
src/backend/telegram/instant.rs
+447
View File
@@ -0,0 +1,447 @@
//! Instant View: the article a link's preview carries, drawn as a page of its own.
//!
//! A message that links to a page Telegram has made an Instant View of carries the page's blocks (headings,
//! paragraphs with their styling, pictures, lists, quotes, tables) as a tree. The window opens it in a window of its
//! own, so this turns the tree into one self-contained HTML document: everything in it is escaped here, pictures are
//! fetched into the cache and named by their local file, and links are limited to the kinds a page may reasonably have.
//! What it cannot draw (embedded players, maps, other channels' posts) is left out, with the page's own link on the
//! card to open it in a browser instead.
use super::media::{download_thumb, download_to, media_name, pick, Variant, MEDIA};
use super::send::{explain, peer_for};
use crate::model::Buffer;
use crate::state::AppState;
use grammers_client::media::{Document, Media, Photo};
use grammers_client::tl;
use std::collections::HashMap;
use std::path::PathBuf;
use tl::enums::{PageBlock as B, RichText as R};
/// A finished article: its title, for the window, and the document.
pub struct Article {
pub title: String,
pub url: String,
pub html: String,
}
/// What a page needs fetched: a picture, or a video's poster.
#[derive(Clone, Copy, PartialEq, Eq, Hash, Debug)]
enum Want {
Photo(i64),
Poster(i64),
}
struct Cx {
account: i64,
remote: String,
message: i32,
photos: HashMap<i64, tl::enums::Photo>,
documents: HashMap<i64, tl::enums::Document>,
wanted: Vec<Want>,
}
fn escape(text: &str) -> String {
let mut out = String::with_capacity(text.len());
for c in text.chars() {
match c {
'&' => out.push_str("&amp;"),
'<' => out.push_str("&lt;"),
'>' => out.push_str("&gt;"),
'"' => out.push_str("&quot;"),
'\'' => out.push_str("&#39;"),
c => out.push(c),
}
}
out
}
/// An address a page's link may point to: web pages, mail, phones and Telegram's own, or an anchor in the page.
fn safe_url(url: &str) -> String {
let lower = url.trim().to_ascii_lowercase();
let ok = ["http://", "https://", "mailto:", "tel:", "tg://", "#"].iter().any(|p| lower.starts_with(p));
if ok { escape(url.trim()) } else { "#".to_string() }
}
fn percent(path: &str) -> String {
let mut out = String::new();
for b in path.bytes() {
if b.is_ascii_alphanumeric() || b"-_.~".contains(&b) {
out.push(b as char);
} else {
out.push_str(&format!("%{b:02X}"));
}
}
out
}
impl Cx {
fn path(&self, want: Want) -> PathBuf {
let (kind, id) = match want {
Want::Photo(id) => ("iv", id),
Want::Poster(id) => ("ivp", id),
};
let name = media_name(self.account, &self.remote, self.message, Variant::Full, "jpg").replace("_full.jpg", &format!("_{kind}{id}.jpg"));
crate::media_cache::dir(MEDIA).join(name)
}
/// The address of a picture the page uses, once it is fetched; `None` when the page does not carry it.
fn picture(&mut self, want: Want) -> Option<String> {
let known = match want {
Want::Photo(id) => self.photos.contains_key(&id),
Want::Poster(id) => self.documents.contains_key(&id),
};
if !known {
tracing::info!("telegram: an article uses {want:?}, which its page does not carry ({} pictures, {} files)", self.photos.len(), self.documents.len());
return None;
}
if !self.wanted.contains(&want) {
self.wanted.push(want);
}
Some(format!("moho-media://file/?p={}", percent(&self.path(want).display().to_string())))
}
}
fn rich(t: &R, cx: &mut Cx) -> String {
let wrap = |tag: &str, inner: &R, cx: &mut Cx| format!("<{tag}>{}</{tag}>", rich(inner, cx));
match t {
R::TextEmpty => String::new(),
R::TextPlain(p) => escape(&p.text).replace('\n', "<br>"),
R::TextBold(b) => wrap("b", &b.text, cx),
R::TextItalic(i) => wrap("i", &i.text, cx),
R::TextUnderline(u) => wrap("u", &u.text, cx),
R::TextStrike(s) => wrap("s", &s.text, cx),
R::TextFixed(f) => wrap("code", &f.text, cx),
R::TextSubscript(s) => wrap("sub", &s.text, cx),
R::TextSuperscript(s) => wrap("sup", &s.text, cx),
R::TextMarked(m) => wrap("mark", &m.text, cx),
R::TextUrl(u) => format!("<a href=\"{}\">{}</a>", safe_url(&u.url), rich(&u.text, cx)),
R::TextEmail(e) => format!("<a href=\"mailto:{}\">{}</a>", safe_url(&e.email).trim_start_matches("mailto:"), rich(&e.text, cx)),
R::TextPhone(p) => format!("<a href=\"tel:{}\">{}</a>", escape(&p.phone), rich(&p.text, cx)),
R::TextConcat(c) => c.texts.iter().map(|t| rich(t, cx)).collect(),
R::TextAnchor(a) => format!("<span id=\"{}\">{}</span>", escape(&a.name), rich(&a.text, cx)),
R::TextSpoiler(s) => rich(&s.text, cx),
R::TextMention(m) => rich(&m.text, cx),
R::TextHashtag(m) => rich(&m.text, cx),
R::TextBotCommand(m) => rich(&m.text, cx),
R::TextCashtag(m) => rich(&m.text, cx),
R::TextBankCard(m) => rich(&m.text, cx),
R::TextAutoUrl(m) => rich(&m.text, cx),
R::TextAutoEmail(m) => rich(&m.text, cx),
R::TextAutoPhone(m) => rich(&m.text, cx),
_ => String::new(),
}
}
fn plain(t: &R) -> String {
let mut cx = Cx { account: 0, remote: String::new(), message: 0, photos: HashMap::new(), documents: HashMap::new(), wanted: Vec::new() };
let html = rich(t, &mut cx);
// Only for a title: the tags taken out again.
let mut out = String::new();
let mut inside = false;
for c in html.chars() {
match c {
'<' => inside = true,
'>' => inside = false,
c if !inside => out.push(c),
_ => {}
}
}
out.replace("&amp;", "&").replace("&lt;", "<").replace("&gt;", ">").replace("&quot;", "\"").replace("&#39;", "'").trim().to_string()
}
fn is_empty(t: &R) -> bool {
matches!(t, R::TextEmpty) || plain(t).is_empty()
}
fn caption(c: &tl::enums::PageCaption, cx: &mut Cx) -> String {
let tl::enums::PageCaption::Caption(c) = c;
let (text, credit) = (rich(&c.text, cx), rich(&c.credit, cx));
if text.is_empty() && credit.is_empty() {
return String::new();
}
format!("<figcaption>{text}{}</figcaption>", if credit.is_empty() { String::new() } else { format!("<cite>{credit}</cite>") })
}
fn blocks(list: &[B], cx: &mut Cx) -> String {
list.iter().map(|b| block(b, cx)).collect()
}
fn heading(tag: &str, t: &R, cx: &mut Cx) -> String {
format!("<{tag}>{}</{tag}>", rich(t, cx))
}
fn block(b: &B, cx: &mut Cx) -> String {
match b {
B::Title(t) => heading("h1", &t.text, cx),
B::Subtitle(t) => format!("<p class=\"subtitle\">{}</p>", rich(&t.text, cx)),
B::Kicker(t) => format!("<p class=\"kicker\">{}</p>", rich(&t.text, cx)),
B::AuthorDate(a) => {
let who = rich(&a.author, cx);
let when = chrono::DateTime::from_timestamp(i64::from(a.published_date), 0).filter(|_| a.published_date > 0).map(|d| d.format("%B %-d, %Y").to_string());
let parts: Vec<String> = [Some(who).filter(|w| !w.is_empty()), when].into_iter().flatten().collect();
if parts.is_empty() { String::new() } else { format!("<p class=\"byline\">{}</p>", parts.join(" · ")) }
}
B::Header(h) => heading("h2", &h.text, cx),
B::Heading1(h) => heading("h2", &h.text, cx),
B::Heading2(h) => heading("h3", &h.text, cx),
B::Heading3(h) => heading("h4", &h.text, cx),
B::Heading4(h) => heading("h5", &h.text, cx),
B::Heading5(h) => heading("h6", &h.text, cx),
B::Heading6(h) => heading("h6", &h.text, cx),
B::Subheader(t) => heading("h3", &t.text, cx),
B::Paragraph(p) => {
let inner = rich(&p.text, cx);
if inner.is_empty() { String::new() } else { format!("<p>{inner}</p>") }
}
B::Preformatted(p) => format!("<pre><code>{}</code></pre>", rich(&p.text, cx)),
B::Footer(f) => format!("<p class=\"footer\">{}</p>", rich(&f.text, cx)),
B::Divider => "<hr>".to_string(),
B::Anchor(a) => format!("<a id=\"{}\"></a>", escape(&a.name)),
B::List(l) => {
let items: String = l
.items
.iter()
.map(|i| match i {
tl::enums::PageListItem::Text(t) => format!("<li>{}</li>", rich(&t.text, cx)),
tl::enums::PageListItem::Blocks(b) => format!("<li>{}</li>", blocks(&b.blocks, cx)),
})
.collect();
format!("<ul>{items}</ul>")
}
B::OrderedList(l) => {
let items: String = l
.items
.iter()
.map(|i| match i {
tl::enums::PageListOrderedItem::Text(t) => format!("<li>{}</li>", rich(&t.text, cx)),
tl::enums::PageListOrderedItem::Blocks(b) => format!("<li>{}</li>", blocks(&b.blocks, cx)),
})
.collect();
let start = l.start.map(|s| format!(" start=\"{s}\"")).unwrap_or_default();
format!("<ol{start}{}>{items}</ol>", if l.reversed { " reversed" } else { "" })
}
B::Blockquote(q) => {
let who = rich(&q.caption, cx);
format!("<blockquote><p>{}</p>{}</blockquote>", rich(&q.text, cx), if who.is_empty() { String::new() } else { format!("<cite>{who}</cite>") })
}
B::BlockquoteBlocks(q) => format!("<blockquote>{}</blockquote>", blocks(&q.blocks, cx)),
B::Pullquote(q) => {
let who = rich(&q.caption, cx);
format!("<blockquote class=\"pull\"><p>{}</p>{}</blockquote>", rich(&q.text, cx), if who.is_empty() { String::new() } else { format!("<cite>{who}</cite>") })
}
B::Photo(p) => {
// A picture the page does not carry still has its words.
let img = match cx.picture(Want::Photo(p.photo_id)) {
Some(src) => {
let img = format!("<img src=\"{src}\" alt=\"\">");
match &p.url {
Some(url) => format!("<a href=\"{}\">{img}</a>", safe_url(url)),
None => img,
}
}
None => String::new(),
};
format!("<figure>{img}{}</figure>", caption(&p.caption, cx))
}
B::Video(v) => {
let poster = cx.picture(Want::Poster(v.video_id)).map(|src| format!("<img src=\"{src}\" alt=\"\">")).unwrap_or_default();
format!("<figure class=\"video\">{poster}<span class=\"note\">Video: open the link above to watch it</span>{}</figure>", caption(&v.caption, cx))
}
B::Cover(c) => format!("<div class=\"cover\">{}</div>", block(&c.cover, cx)),
B::Collage(c) => format!("<figure class=\"collage\"><div>{}</div>{}</figure>", blocks(&c.items, cx), caption(&c.caption, cx)),
B::Slideshow(c) => format!("<figure class=\"collage\"><div>{}</div>{}</figure>", blocks(&c.items, cx), caption(&c.caption, cx)),
B::Embed(e) => {
let poster = e.poster_photo_id.and_then(|id| cx.picture(Want::Photo(id))).map(|src| format!("<img src=\"{src}\" alt=\"\">")).unwrap_or_default();
let link = e.url.as_deref().map(|u| format!("<a class=\"note\" href=\"{}\">Open the embedded content</a>", safe_url(u))).unwrap_or_default();
format!("<figure>{poster}{link}{}</figure>", caption(&e.caption, cx))
}
B::Table(t) => {
let rows: String = t
.rows
.iter()
.map(|row| {
let tl::enums::PageTableRow::Row(row) = row;
let cells: String = row
.cells
.iter()
.map(|cell| {
let tl::enums::PageTableCell::Cell(c) = cell;
let tag = if c.header { "th" } else { "td" };
let span = format!("{}{}", c.colspan.map(|n| format!(" colspan=\"{n}\"")).unwrap_or_default(), c.rowspan.map(|n| format!(" rowspan=\"{n}\"")).unwrap_or_default());
let align = if c.align_center { " class=\"c\"" } else if c.align_right { " class=\"r\"" } else { "" };
format!("<{tag}{span}{align}>{}</{tag}>", c.text.as_ref().map(|t| rich(t, cx)).unwrap_or_default())
})
.collect();
format!("<tr>{cells}</tr>")
})
.collect();
let title = rich(&t.title, cx);
format!("<div class=\"table\"><table>{}{rows}</table></div>", if title.is_empty() { String::new() } else { format!("<caption>{title}</caption>") })
}
B::Details(d) => format!("<details{}><summary>{}</summary>{}</details>", if d.open { " open" } else { "" }, rich(&d.title, cx), blocks(&d.blocks, cx)),
B::Math(m) => format!("<pre><code>{}</code></pre>", escape(&m.source)),
_ => String::new(),
}
}
const STYLE: &str = "
:root{color-scheme:light dark;--bg:#fff;--fg:#1c2733;--muted:#6b7a89;--line:#d8dee4;--link:#2b7bd3;--soft:#f2f5f8}
@media (prefers-color-scheme:dark){:root{--bg:#0e1621;--fg:#e4e9ee;--muted:#8a99a8;--line:#26323f;--link:#6ab3f3;--soft:#17212b}}
html,body{margin:0;background:var(--bg);color:var(--fg)}
body{font:17px/1.65 system-ui,-apple-system,'Segoe UI','Noto Sans',sans-serif}
main{max-width:740px;margin:0 auto;padding:0 20px 80px}
a{color:var(--link);text-decoration:none}a:hover{text-decoration:underline}
h1{font-size:2rem;line-height:1.2;margin:.8em 0 .4em}h2{font-size:1.5rem;margin:1.4em 0 .4em}h3{font-size:1.25rem;margin:1.3em 0 .3em}h4,h5,h6{margin:1.2em 0 .3em}
p{margin:.9em 0}.kicker{color:var(--muted);text-transform:uppercase;letter-spacing:.06em;font-size:.8rem;margin:1.6em 0 0}
.subtitle{color:var(--muted);font-size:1.15rem}.byline,.footer{color:var(--muted);font-size:.9rem}
figure{margin:1.4em -20px}figure img,.cover img{display:block;width:100%;height:auto}figcaption{padding:.5em 20px 0;color:var(--muted);font-size:.9rem}figcaption cite{display:block;font-style:normal;opacity:.8}
.cover{margin:0 -20px}.cover figure{margin:0}.cover figcaption{padding:.7em 20px;text-transform:uppercase;letter-spacing:.04em;font-size:.8rem}.cover figcaption cite{margin:.6em -20px -.7em;padding:.6em 20px;background:var(--soft);color:var(--fg);font-size:1rem;font-weight:600;text-transform:none;letter-spacing:0;opacity:1}.collage div{display:grid;grid-template-columns:repeat(auto-fit,minmax(220px,1fr));gap:4px}.collage figure{margin:0}
.note{display:block;padding:.6em 20px;color:var(--muted);font-size:.9rem}
blockquote{margin:1.2em 0;padding:.1em 1em;border-left:3px solid var(--link);color:var(--fg)}blockquote.pull{font-size:1.25rem;border-left:0;text-align:center;font-style:italic}cite{display:block;color:var(--muted);font-style:normal;font-size:.9rem}
pre{background:var(--soft);padding:12px 14px;overflow:auto;border-radius:6px;font-size:.9rem}code{font-family:ui-monospace,Menlo,Consolas,monospace}mark{background:#ffe08a55;color:inherit}
hr{border:0;border-top:1px solid var(--line);margin:2em 0}ul,ol{padding-left:1.4em}li{margin:.3em 0}
.table{overflow:auto;margin:1.2em 0}table{border-collapse:collapse;width:100%}td,th{border:1px solid var(--line);padding:6px 10px;vertical-align:top}th{background:var(--soft)}.c{text-align:center}.r{text-align:right}caption{caption-side:top;text-align:left;font-weight:600;padding-bottom:.4em}
details{margin:1em 0;border:1px solid var(--line);border-radius:6px;padding:.4em .9em}summary{cursor:pointer;font-weight:600}
";
/// The article a message's link preview carries, with its pictures fetched.
pub async fn article(state: &AppState, buffer: &Buffer, msg_id: &str) -> Result<Article, String> {
let id: i32 = msg_id.parse().map_err(|_| "that message has no article".to_string())?;
let (handle, peer) = peer_for(state, buffer).await?;
let found = handle.client.get_messages_by_id(peer, &[id]).await.map_err(|e| explain(&e))?;
let Some(Some(message)) = found.into_iter().next() else { return Err("that message is no longer there".to_string()) };
let Some(Media::WebPage(media)) = message.media() else { return Err("that message links to no page".to_string()) };
let tl::enums::WebPage::Page(page) = &media.raw.webpage else { return Err("Telegram has no preview of that page".to_string()) };
let url = page.url.clone();
let title = page.title.clone().unwrap_or_default();
let (preview_photo, preview_document) = (page.photo.clone(), page.document.clone());
let mut cached = page.cached_page.clone();
// A message may carry only the start of the page: the whole of it is asked for.
if matches!(&cached, Some(tl::enums::Page::Page(p)) if p.part) {
if let Ok(tl::enums::messages::WebPage::Page(full)) = handle.client.invoke(&tl::functions::messages::GetWebPage { url: url.clone(), hash: 0 }).await {
if let tl::enums::WebPage::Page(whole) = full.webpage {
cached = whole.cached_page.or(cached);
}
}
}
let Some(tl::enums::Page::Page(doc)) = cached else { return Err("Telegram has no Instant View of that page".to_string()) };
let remote = buffer.remote_id.clone().unwrap_or_default();
let mut cx = Cx {
account: handle.own.user_id,
remote,
message: id,
// The page's own picture (the one its preview shows) is often the cover and is not listed again with the rest.
photos: doc.photos.iter().chain(preview_photo.iter()).filter_map(|p| match p { tl::enums::Photo::Photo(x) => Some((x.id, p.clone())), _ => None }).collect(),
documents: doc.documents.iter().chain(preview_document.iter()).filter_map(|d| match d { tl::enums::Document::Document(x) => Some((x.id, d.clone())), _ => None }).collect(),
wanted: Vec::new(),
};
let body = blocks(&doc.blocks, &mut cx);
// The page's own title, from its first heading when the preview gave none.
let title = if title.is_empty() { doc.blocks.iter().find_map(|b| if let B::Title(t) = b { Some(plain(&t.text)).filter(|t| !t.is_empty()) } else { None }).unwrap_or_default() } else { title };
let has_title = doc.blocks.iter().any(|b| matches!(b, B::Title(t) if !is_empty(&t.text)));
let heading = if has_title || title.is_empty() { String::new() } else { format!("<h1>{}</h1>", escape(&title)) };
// The pictures, fetched together; one that will not come leaves a gap where it was.
let mut jobs = tokio::task::JoinSet::new();
for want in cx.wanted.clone() {
let path = cx.path(want);
if path.exists() {
continue;
}
let client = handle.client.clone();
let (photo, document) = match want {
Want::Photo(id) => (cx.photos.get(&id).cloned(), None),
Want::Poster(id) => (None, cx.documents.get(&id).cloned()),
};
jobs.spawn(async move {
let done = if let Some(photo) = photo {
let photo = Photo::from_raw(photo);
match pick(&photo.thumbs(), 1280) {
Some(size) => download_to(&client, size, &path, false).await,
None => download_to(&client, &photo, &path, false).await,
}
} else if let Some(document) = document {
let document = Document::from_raw_media(tl::types::MessageMediaDocument {
nopremium: false,
spoiler: false,
video: true,
round: false,
voice: false,
document: Some(document),
alt_documents: None,
video_cover: None,
video_timestamp: None,
ttl_seconds: None,
});
download_thumb(&client, &document, &path).await
} else {
Ok(())
};
if let Err(e) = done {
tracing::debug!("telegram: a picture of an article: {e:#}");
}
});
}
while jobs.join_next().await.is_some() {}
let rtl = if doc.rtl { " dir=\"rtl\"" } else { "" };
let html = format!(
"<!doctype html><html lang=\"en\"{rtl}><head><meta charset=\"utf-8\"><meta name=\"viewport\" content=\"width=device-width,initial-scale=1\">\
<meta http-equiv=\"Content-Security-Policy\" content=\"default-src 'none'; img-src moho-media: data:; style-src 'unsafe-inline'\">\
<title>{}</title><style>{STYLE}</style></head><body><main>{heading}{body}<p class=\"footer\"><a href=\"{}\">Open the original page</a></p></main></body></html>",
escape(&title),
safe_url(&url)
);
Ok(Article { title, url, html })
}
#[cfg(test)]
mod tests {
use super::*;
fn cx() -> Cx {
Cx { account: 1, remote: "-100123".into(), message: 7, photos: HashMap::new(), documents: HashMap::new(), wanted: Vec::new() }
}
fn plain_text(t: &str) -> R {
R::TextPlain(tl::types::TextPlain { text: t.into() })
}
#[test]
fn text_is_escaped_and_styled() {
let mut c = cx();
let t = R::TextConcat(tl::types::TextConcat {
texts: vec![plain_text("a <b> & "), R::TextBold(Box::new(tl::types::TextBold { text: plain_text("bold") }))],
});
assert_eq!(rich(&t, &mut c), "a &lt;b&gt; &amp; <b>bold</b>");
}
#[test]
fn a_link_may_only_go_where_a_page_would() {
assert_eq!(safe_url("https://example.com/a?b=1&c=2"), "https://example.com/a?b=1&amp;c=2");
assert_eq!(safe_url("javascript:alert(1)"), "#");
assert_eq!(safe_url("data:text/html,x"), "#");
assert_eq!(safe_url("#section"), "#section");
let mut c = cx();
let link = R::TextUrl(Box::new(tl::types::TextUrl { text: plain_text("x"), url: "javascript:alert(1)".into(), webpage_id: 0 }));
assert_eq!(rich(&link, &mut c), "<a href=\"#\">x</a>");
}
#[test]
fn a_picture_the_page_lacks_keeps_its_words_and_one_it_has_is_named_by_its_file() {
let mut c = cx();
let photo = B::Photo(tl::types::PageBlockPhoto {
spoiler: false,
photo_id: 9,
caption: tl::enums::PageCaption::Caption(tl::types::PageCaption { text: plain_text("a caption"), credit: R::TextEmpty }),
url: None,
webpage_id: None,
});
let without = block(&photo, &mut c);
assert!(!without.contains("<img") && without.contains("a caption"), "{without}");
c.photos.insert(9, tl::enums::Photo::Empty(tl::types::PhotoEmpty { id: 9 }));
let html = block(&photo, &mut c);
assert!(html.contains("moho-media://file/?p=") && html.contains("_iv9.jpg") && html.contains("a caption"), "{html}");
assert_eq!(c.wanted, vec![Want::Photo(9)]);
}
}
src/backend/telegram/keyboards.rs
+162
View File
@@ -0,0 +1,162 @@
//! A bot's inline keyboard: the buttons under one of its messages.
//!
//! Telegram sends them as rows of buttons with the message. Most are "callback" buttons, which tell the bot something
//! when pressed and are answered with a short notice (or nothing, the bot editing its message instead); the rest are
//! links. They are drawn with the same shape Discord's buttons use, and an edit to the message that changes or removes
//! them says so.
use super::send::{explain, peer_for};
use crate::model::{Buffer, Component};
use crate::state::AppState;
use grammers_client::tl;
use serde_json::json;
use tl::enums::InlineButtonType as T;
fn hex(data: &[u8]) -> String {
data.iter().map(|b| format!("{b:02x}")).collect()
}
fn unhex(text: &str) -> Option<Vec<u8>> {
if !text.len().is_multiple_of(2) {
return None;
}
(0..text.len()).step_by(2).map(|i| u8::from_str_radix(text.get(i..i + 2)?, 16).ok()).collect()
}
/// Only an address a window may open in a browser.
fn web(url: &str) -> Option<String> {
let lower = url.trim().to_ascii_lowercase();
(lower.starts_with("https://") || lower.starts_with("http://")).then(|| url.trim().to_string())
}
/// The buttons of an inline keyboard, in rows, as this client draws buttons. Nothing for any other kind of markup.
pub fn components_of(markup: &tl::enums::ReplyMarkup) -> Vec<Component> {
let tl::enums::ReplyMarkup::ReplyInlineMarkup(inline) = markup else { return Vec::new() };
let mut out = Vec::new();
for (row_index, row) in inline.rows.iter().enumerate() {
let tl::enums::KeyboardInlineButtonRow::Row(row) = row;
for button in &row.buttons {
let tl::enums::KeyboardInlineButton::Button(button) = button;
let style = button.style.as_ref().map(|s| {
let tl::enums::KeyboardButtonStyle::Style(s) = s;
if s.bg_danger {
"danger"
} else if s.bg_success {
"success"
} else if s.bg_primary {
"primary"
} else {
"secondary"
}
});
let make = |custom_id: Option<String>, style: Option<&str>, url: Option<String>, disabled: bool| Component {
kind: "button".to_string(),
custom_id,
label: Some(button.text.clone()).filter(|t| !t.is_empty()),
style: style.map(str::to_string),
url,
disabled,
emoji: None,
options: Vec::new(),
placeholder: None,
row: row_index as i64,
};
match &button.r#type {
T::Callback(c) => out.push(make(Some(hex(&c.data)), style.or(Some("secondary")), None, c.requires_password)),
T::Url(u) => out.extend(web(&u.url).map(|url| make(None, Some("link"), Some(url), false))),
T::UrlAuth(u) => out.extend(web(&u.url).map(|url| make(None, Some("link"), Some(url), false))),
T::WebView(w) => out.extend(web(&w.url).map(|url| make(None, Some("link"), Some(url), false))),
T::Disabled => out.push(make(None, Some("secondary"), None, true)),
// Switching to inline mode, games, payments, profile links and copy buttons are not done here.
_ => {}
}
}
}
out
}
/// Keeps the buttons on a message and says so. With none, an edit that took them away takes them off the row too.
pub fn note(state: &AppState, buffer_id: &str, msg_id: &str, raw: &tl::enums::Message, edited: bool) {
let tl::enums::Message::Message(m) = raw else { return };
let components = m.reply_markup.as_ref().map(components_of).unwrap_or_default();
// Said when a message has markup that gave no buttons, so what the bot sent is on record rather than guessed at.
if let Some(markup) = m.reply_markup.as_ref().filter(|_| components.is_empty()) {
let text = format!("{markup:?}");
tracing::info!("telegram: message {msg_id} carries markup with no buttons this client draws: {}", text.chars().take(400).collect::<String>());
}
if components.is_empty() && !edited {
return;
}
let kept = if components.is_empty() { state.store.clear_components(buffer_id, msg_id) } else { state.store.set_components(buffer_id, msg_id, &components) };
match kept {
Ok(()) => state.events.emit("messageComponents", json!({ "bufferId": buffer_id, "messageId": msg_id, "components": components })),
Err(e) => tracing::debug!("telegram: keeping the buttons on {msg_id}: {e:#}"),
}
}
/// What the bot said back to a press: a notice to show (as a pop-up alert when it asked for one), and an address to open.
pub async fn press(state: &AppState, buffer: &Buffer, msg_id: &str, custom_id: &str) -> Result<serde_json::Value, String> {
let id: i32 = msg_id.parse().map_err(|_| "that message has no buttons".to_string())?;
let data = unhex(custom_id).ok_or_else(|| "that button is not one this client can press".to_string())?;
let (handle, peer) = peer_for(state, buffer).await?;
let answer = handle
.client
.invoke(&tl::functions::messages::GetBotCallbackAnswer { game: false, peer: peer.into(), msg_id: id, data: Some(data), password: None })
.await
.map_err(|e| match &e {
grammers_client::InvocationError::Rpc(r) if r.name == "BOT_RESPONSE_TIMEOUT" => "The bot did not answer".to_string(),
grammers_client::InvocationError::Rpc(r) if r.name == "DATA_INVALID" || r.name == "BUTTON_DATA_INVALID" => "That button is no longer valid".to_string(),
_ => explain(&e),
})?;
let tl::enums::messages::BotCallbackAnswer::Answer(a) = answer;
Ok(json!({ "message": a.message, "alert": a.alert, "url": a.url.as_deref().and_then(web) }))
}
#[cfg(test)]
mod tests {
use super::*;
fn button(text: &str, kind: T) -> tl::enums::KeyboardInlineButton {
tl::enums::KeyboardInlineButton::Button(tl::types::KeyboardInlineButton { style: None, text: text.into(), r#type: kind })
}
fn keyboard(rows: Vec<Vec<tl::enums::KeyboardInlineButton>>) -> tl::enums::ReplyMarkup {
tl::enums::ReplyMarkup::ReplyInlineMarkup(tl::types::ReplyInlineMarkup {
force_reply: false,
rows: rows.into_iter().map(|buttons| tl::enums::KeyboardInlineButtonRow::Row(tl::types::KeyboardInlineButtonRow { buttons })).collect(),
})
}
#[test]
fn data_survives_the_trip_through_a_button() {
assert_eq!(unhex(&hex(&[0, 1, 0xab, 0xff])), Some(vec![0, 1, 0xab, 0xff]));
assert_eq!(unhex("abc"), None);
assert_eq!(unhex("zz"), None);
}
#[test]
fn rows_keep_their_places_and_links_are_links() {
let markup = keyboard(vec![
vec![button("Next", T::Callback(tl::types::InlineButtonTypeCallback { requires_password: false, data: b"next:2".to_vec() })), button("Close", T::Callback(tl::types::InlineButtonTypeCallback { requires_password: false, data: b"x".to_vec() }))],
vec![button("Open site", T::Url(tl::types::InlineButtonTypeUrl { url: "https://example.com/a".into() }))],
]);
let got = components_of(&markup);
assert_eq!(got.iter().map(|c| (c.label.as_deref().unwrap(), c.row)).collect::<Vec<_>>(), vec![("Next", 0), ("Close", 0), ("Open site", 1)]);
assert_eq!(got[0].custom_id.as_deref(), Some(hex(b"next:2").as_str()));
assert_eq!(got[2].url.as_deref(), Some("https://example.com/a"));
assert_eq!(got[2].style.as_deref(), Some("link"));
}
#[test]
fn what_cannot_be_opened_or_pressed_is_left_out() {
let markup = keyboard(vec![vec![
button("App", T::Url(tl::types::InlineButtonTypeUrl { url: "tg://resolve?domain=x".into() })),
button("Pay", T::Buy),
button("Locked", T::Callback(tl::types::InlineButtonTypeCallback { requires_password: true, data: b"x".to_vec() })),
]]);
let got = components_of(&markup);
assert_eq!(got.len(), 1, "only the callback is kept, and it cannot be pressed");
assert!(got[0].disabled);
assert!(components_of(&tl::enums::ReplyMarkup::ReplyKeyboardHide(tl::types::ReplyKeyboardHide { selective: false })).is_empty());
}
}
src/backend/telegram/link.rs
+49
View File
@@ -0,0 +1,49 @@
//! One connection to Telegram: the library's sender pool, its client and the stream of what Telegram sends.
//!
//! Everything that talks to Telegram - a sign-in, a connected account - opens one of these and closes it when it
//! is done. The pool connects to each datacenter on demand, and a closed one drops its connections gracefully.
use super::session::FileSession;
use crate::state::AppState;
use anyhow::{Context, Result};
use grammers_client::Client;
use grammers_mtsender::{SenderPool, SenderPoolFatHandle};
use grammers_session::updates::UpdatesLike;
use tokio::sync::mpsc::Receiver;
use std::sync::Arc;
pub struct Link {
pub client: Client,
pub session: Arc<FileSession>,
/// What Telegram sends unasked, to be handed to `Client::stream_updates` once (taken by whoever reads it).
pub updates: Option<Receiver<UpdatesLike>>,
handle: SenderPoolFatHandle,
runner: tokio::task::JoinHandle<()>,
}
/// Opens a connection under `key` (the account's id, or a sign-in's pending key), by the account's route if it has
/// one: Tor or the proxy, as a SOCKS5 address the library connects through.
pub async fn open(state: &AppState, key: &str, api_id: i32, session: Arc<FileSession>) -> Result<Link> {
let router = crate::net::route::router();
let proxy = if router.routed(key) {
let (host, port) = router.ready(|msg| state.runtime.report_progress(state, key, msg)).await.context("starting Tor or reaching the proxy")?;
Some(format!("socks5://{host}:{port}"))
} else {
None
};
let SenderPool { runner, handle, updates } = SenderPool::with_configuration(session.clone(), api_id, super::connection_params(proxy.as_deref()));
let client = Client::new(handle.clone());
let runner = tokio::spawn(runner.run());
Ok(Link { client, session, updates: Some(updates), handle, runner })
}
impl Link {
/// Closes the connections, and writes the session if anything changed.
pub async fn close(self) {
self.handle.quit();
let _ = self.runner.await;
if let Err(e) = self.session.save_if_dirty() {
tracing::warn!("telegram: saving the session: {e:#}");
}
}
}
src/backend/telegram/links.rs
+271
View File
@@ -0,0 +1,271 @@
//! Telegram links: what a `t.me` address or a `tg://` link points at, and taking the window there.
//!
//! A link names a public chat (`t.me/disclosetv`, `tg://resolve?domain=disclosetv`), a message in one
//! (`t.me/disclosetv/21953`, `...&post=21953`), a message in a chat the account is already in
//! (`t.me/c/1234567/89`, `tg://privatepost?channel=1234567&post=89`) or a private invite (`t.me/+hash`,
//! `tg://join?invite=hash`); in a group with topics the topic comes before the message
//! (`t.me/group/5/21953`). Opening one makes the conversation if the account has none (a chat it is not in is
//! joined, which is what a link to a channel asks for) and answers where it is, and which message to go to.
//!
//! The web page Telegram shows for a post has a "View in Telegram" button that opens `tg://resolve?domain=..&post=..`,
//! which is how a click in a browser comes here once moho is the handler for `tg:` links.
use super::peers;
use super::send::explain;
use super::{dialogs, topics, Handle};
use crate::model::Buffer;
use crate::state::AppState;
use grammers_client::peer::Peer;
use grammers_client::tl;
use grammers_session::types::PeerId;
use grammers_session::Session;
use serde_json::{json, Value};
/// What a link points at.
#[derive(Debug, PartialEq)]
pub enum Link {
/// A chat named by its public @name.
Public { name: String, topic: Option<i32>, post: Option<i32> },
/// A chat named by its number, which only an account that is in it can open.
Private { channel: i64, topic: Option<i32>, post: Option<i32> },
/// A private invite.
Invite(String),
}
fn number(text: &str) -> Option<i32> {
text.parse::<i32>().ok().filter(|n| *n > 0)
}
fn name_ok(name: &str) -> bool {
(5..=32).contains(&name.len()) && name.chars().next().is_some_and(|c| c.is_ascii_alphabetic()) && name.chars().all(|c| c.is_ascii_alphanumeric() || c == '_')
}
fn invite_ok(hash: &str) -> bool {
!hash.is_empty() && hash.chars().all(|c| c.is_ascii_alphanumeric() || c == '_' || c == '-')
}
/// Reads a `t.me` / `telegram.me` / `telegram.dog` address or a `tg://` link; `None` for anything else, including the
/// pages on those hosts that are not a chat (sticker packs, proxies, the web preview).
pub fn parse(input: &str) -> Option<Link> {
let url = url::Url::parse(input.trim()).ok()?;
let query = |key: &str| url.query_pairs().find(|(k, _)| k == key).map(|(_, v)| v.into_owned());
match url.scheme() {
"tg" => match url.host_str()? {
"resolve" => {
let name = query("domain")?;
if name == "telegrampassport" || !name_ok(&name) {
return None;
}
let topic = query("thread").or_else(|| query("topic")).and_then(|t| number(&t));
Some(Link::Public { name, topic, post: query("post").and_then(|p| number(&p)) })
}
"privatepost" => {
let channel: i64 = query("channel")?.parse().ok().filter(|c| *c > 0)?;
Some(Link::Private { channel, topic: query("thread").or_else(|| query("topic")).and_then(|t| number(&t)), post: query("post").and_then(|p| number(&p)) })
}
"join" => Some(Link::Invite(query("invite").filter(|h| invite_ok(h))?)),
_ => None,
},
"http" | "https" => {
if !matches!(url.host_str()?.trim_start_matches("www."), "t.me" | "telegram.me" | "telegram.dog") {
return None;
}
let parts: Vec<&str> = url.path_segments()?.filter(|p| !p.is_empty()).collect();
let first = *parts.first()?;
if let Some(hash) = first.strip_prefix('+') {
return invite_ok(hash).then(|| Link::Invite(hash.to_string()));
}
match first {
"joinchat" => parts.get(1).filter(|h| invite_ok(h)).map(|h| Link::Invite((*h).to_string())),
"c" => {
let channel: i64 = parts.get(1)?.parse().ok().filter(|c| *c > 0)?;
let (topic, post) = match (parts.get(2).and_then(|p| number(p)), parts.get(3).and_then(|p| number(p))) {
(Some(topic), Some(post)) => (Some(topic), Some(post)),
(Some(post), None) => (None, Some(post)),
_ => (None, None),
};
Some(Link::Private { channel, topic, post })
}
// Pages that are about something other than a chat.
"s" | "addstickers" | "addemoji" | "setlanguage" | "proxy" | "socks" | "share" | "login" | "iv" | "addtheme" | "bg" | "boost" | "m" | "addlist" | "invoice" | "giftcode" | "contact" => None,
name if name_ok(name) => {
let (topic, post) = match (parts.get(1).and_then(|p| number(p)), parts.get(2).and_then(|p| number(p))) {
(Some(topic), Some(post)) => (Some(topic), Some(post)),
(Some(post), None) => (None, Some(post)),
_ => (None, None),
};
Some(Link::Public { name: name.to_string(), topic, post })
}
_ => None,
}
}
_ => None,
}
}
/// Where a link led.
pub struct Opened {
pub account_id: String,
pub buffer: Buffer,
pub message_id: Option<String>,
}
/// The first connected Telegram account, when the window did not say which.
fn account_for(state: &AppState, wanted: Option<&str>) -> Result<(String, Handle), String> {
if let Some(id) = wanted {
return state.runtime.telegram_handle(id).map(|h| (id.to_string(), h)).ok_or_else(|| "that Telegram account is not connected".to_string());
}
state
.accounts
.all_telegram()
.into_iter()
.find_map(|a| {
let id = a.account_id();
state.runtime.telegram_handle(&id).map(|h| (id, h))
})
.ok_or_else(|| "Add a Telegram account, and wait for it to connect, to open that link".to_string())
}
/// Whether the chat is one this account has a conversation for.
fn known(state: &AppState, account_id: &str, peer: &Peer) -> bool {
let remote = peers::remote_id(peer.id());
state.runtime.list_buffers().iter().any(|b| b.account_id == account_id && b.remote_id.as_deref().is_some_and(|r| r == remote || r.strip_prefix(remote.as_str()).is_some_and(|rest| rest.starts_with(':'))))
}
fn chats_of(updates: &tl::enums::Updates) -> Vec<tl::enums::Chat> {
match updates {
tl::enums::Updates::Updates(u) => u.chats.clone(),
tl::enums::Updates::Combined(u) => u.chats.clone(),
_ => Vec::new(),
}
}
/// Joins a chat by its invite, and says which chat it was.
async fn join_invite(state: &AppState, account_id: &str, handle: &Handle, hash: &str) -> Result<Peer, String> {
let checked = handle.client.invoke(&tl::functions::messages::CheckChatInvite { hash: hash.to_string() }).await.map_err(|e| invite_words(&e))?;
// Already in it: the invite names a chat the account has.
if let tl::enums::ChatInvite::Already(already) = checked {
return Ok(Peer::from_raw(&handle.client, already.chat));
}
let answer = handle.client.invoke(&tl::functions::messages::ImportChatInvite { hash: hash.to_string() }).await.map_err(|e| invite_words(&e))?;
let tl::enums::messages::ChatInviteJoinResult::Ok(joined) = answer else {
return Err("That chat has to be joined from Telegram's own app first".to_string());
};
let chat = chats_of(&joined.updates).into_iter().next().ok_or("Telegram did not say which chat that was")?;
let peer = Peer::from_raw(&handle.client, chat);
let _ = dialogs::sync(state, account_id, handle).await;
Ok(peer)
}
fn invite_words(e: &grammers_client::InvocationError) -> String {
match e {
grammers_client::InvocationError::Rpc(r) if r.name == "INVITE_HASH_EXPIRED" || r.name == "INVITE_HASH_INVALID" => "That invite link is no longer valid".to_string(),
grammers_client::InvocationError::Rpc(r) if r.name == "INVITE_REQUEST_SENT" => "Your request to join was sent: an administrator has to accept it".to_string(),
other => explain(other),
}
}
/// The conversation a message of a chat is in: the chat's own, or in a group with topics the topic's.
fn buffer_in(state: &AppState, account_id: &str, peer: &Peer, topic: Option<i32>, post: Option<i32>) -> Buffer {
if topics::is_forum(peer) {
let remote = peers::remote_id(peer.id());
return match (topic, post) {
(Some(topic), _) => topics::buffer_for_topic(state, account_id, peer, topic, None),
(None, Some(post)) => topics::holding(state, account_id, &remote, post).unwrap_or_else(|| topics::buffer_for_topic(state, account_id, peer, topics::GENERAL, None)),
(None, None) => topics::buffer_for_topic(state, account_id, peer, topics::GENERAL, None),
};
}
peers::buffer_for(state, account_id, peer)
}
/// Opens a link: finds or makes the conversation it points into, joining the chat if the account is not in it.
pub async fn open(state: &AppState, account: Option<&str>, input: &str) -> Result<Opened, String> {
let link = parse(input).ok_or_else(|| "That is not a Telegram link moho can open".to_string())?;
let (account_id, handle) = account_for(state, account)?;
let (peer, topic, post) = match link {
Link::Invite(hash) => (join_invite(state, &account_id, &handle, &hash).await?, None, None),
Link::Public { name, topic, post } => {
let peer = handle.client.resolve_username(&name).await.map_err(|e| explain(&e))?.ok_or_else(|| format!("Nobody on Telegram is called @{name}"))?;
if !matches!(peer, Peer::User(_)) && !known(state, &account_id, &peer) {
// A link to a channel asks to be in it.
let peer_ref = peer.to_ref().await.ok().flatten().ok_or("Telegram did not say how to reach that chat")?;
handle.client.join_chat(peer_ref).await.map_err(|e| invite_words(&e))?;
let _ = dialogs::sync(state, &account_id, &handle).await;
}
(peer, topic, post)
}
Link::Private { channel, topic, post } => {
let peer_ref = handle.session.peer_ref(PeerId::channel_unchecked(channel)).await.ok().flatten().ok_or("This account is not in that chat, so a link by its number cannot open it")?;
(handle.client.resolve_peer(peer_ref).await.map_err(|e| explain(&e))?, topic, post)
}
};
let buffer = buffer_in(state, &account_id, &peer, topic, post);
Ok(Opened { account_id, buffer, message_id: post.map(|p| p.to_string()) })
}
/// What the window is told.
pub fn answer(opened: &Opened) -> Value {
json!({ "accountId": opened.account_id, "bufferId": opened.buffer.id, "messageId": opened.message_id })
}
#[cfg(test)]
mod tests {
use super::*;
fn public(name: &str, topic: Option<i32>, post: Option<i32>) -> Option<Link> {
Some(Link::Public { name: name.into(), topic, post })
}
#[test]
fn a_web_link_to_a_post_names_the_channel_and_the_post() {
assert_eq!(parse("https://t.me/disclosetv/21953"), public("disclosetv", None, Some(21953)));
assert_eq!(parse("https://t.me/disclosetv/21953?single"), public("disclosetv", None, Some(21953)));
assert_eq!(parse("http://telegram.me/disclosetv"), public("disclosetv", None, None));
assert_eq!(parse("https://www.t.me/disclosetv/5"), public("disclosetv", None, Some(5)));
}
#[test]
fn in_a_group_with_topics_the_topic_comes_before_the_message() {
assert_eq!(parse("https://t.me/somegroup/7/21953"), public("somegroup", Some(7), Some(21953)));
assert_eq!(parse("tg://resolve?domain=somegroup&post=21953&thread=7"), public("somegroup", Some(7), Some(21953)));
}
#[test]
fn the_app_link_a_browsers_view_in_telegram_button_opens_is_read() {
assert_eq!(parse("tg://resolve?domain=disclosetv&post=21953"), public("disclosetv", None, Some(21953)));
assert_eq!(parse("tg://resolve?domain=disclosetv"), public("disclosetv", None, None));
// A bot started with a parameter is the bot's chat.
assert_eq!(parse("tg://resolve?domain=zlibrary_bot&start=abc"), public("zlibrary_bot", None, None));
assert_eq!(parse("tg://resolve?domain=telegrampassport&bot_id=1"), None);
}
#[test]
fn a_chat_named_by_its_number_is_private() {
assert_eq!(parse("https://t.me/c/1234567/89"), Some(Link::Private { channel: 1234567, topic: None, post: Some(89) }));
assert_eq!(parse("https://t.me/c/1234567/5/89"), Some(Link::Private { channel: 1234567, topic: Some(5), post: Some(89) }));
assert_eq!(parse("tg://privatepost?channel=1234567&post=89"), Some(Link::Private { channel: 1234567, topic: None, post: Some(89) }));
assert_eq!(parse("https://t.me/c/notanumber/89"), None);
}
#[test]
fn invites_are_read_in_every_form() {
let invite = |h: &str| Some(Link::Invite(h.to_string()));
assert_eq!(parse("https://t.me/+AbC_d-123"), invite("AbC_d-123"));
assert_eq!(parse("https://t.me/joinchat/AbC_d-123"), invite("AbC_d-123"));
assert_eq!(parse("tg://join?invite=AbC_d-123"), invite("AbC_d-123"));
assert_eq!(parse("tg://join?invite="), None);
}
#[test]
fn pages_that_are_not_a_chat_and_other_hosts_are_left_alone() {
assert_eq!(parse("https://t.me/addstickers/SomePack"), None);
assert_eq!(parse("https://t.me/s/disclosetv"), None);
assert_eq!(parse("https://t.me/proxy?server=x&port=1"), None);
assert_eq!(parse("https://t.me/"), None);
assert_eq!(parse("https://t.me/ab"), None, "too short to be a name");
assert_eq!(parse("https://example.com/disclosetv/1"), None);
assert_eq!(parse("https://t.me.evil.example/disclosetv/1"), None);
assert_eq!(parse("irc://irc.libera.chat/moho"), None);
assert_eq!(parse("not a link"), None);
}
}
src/backend/telegram/login.rs
+508
View File
@@ -0,0 +1,508 @@
//! Adding an account: the sign-in a person takes part in.
//!
//! The person types an `api_id`, an `api_hash` and a phone number. Telegram sends a code to the account's other
//! sessions (or by SMS), the person types it, and if the account has a cloud password that is asked for next.
//! Neither the code nor the password is kept: what the account keeps is the session Telegram issued.
//!
//! Like Steam's, it is a conversation that can take minutes, so it runs as its own task and talks to the window
//! through events:
//!
//! - `telegramLoginStatus` - what it is doing, for a line of text
//! - `telegramLoginNeeds` - what Telegram is asking for: `step` is `code` or `password` (with the password's hint),
//! and `error` says why the last answer was refused
//! - `telegramLoginQr` - a picture to scan, for a sign-in by QR code (below)
//! - `telegramLoginResult` - done, with the account, or the reason it failed
//!
//! and the window answers with `submitTelegramCode` and `submitTelegramPassword`.
//!
//! # By QR code
//!
//! The other way in is to scan a code with Telegram on a phone that is already signed in (Settings, Devices, Link
//! Desktop Device): no number is typed and no code is sent. Telegram gives a login token that lasts about half a
//! minute; it is shown as a QR code and asked for again when it runs out, and Telegram says when it was scanned
//! (`updateLoginToken`; the token is asked for again every so often too, in case that update does not reach a
//! connection that has no account yet). The scanned token comes back as the account, or - when the account lives
//! in another datacenter - as a pointer to it, where the token is handed in and the key made there becomes the
//! session's home. An account with a cloud password still asks for it, after the scan.
use super::link::{self, Link};
use super::session::{self, FileSession};
use crate::accounts::TelegramAccountConfig;
use crate::state::AppState;
use anyhow::{anyhow, bail, Context, Result};
use grammers_client::SignInError;
use grammers_client::tl;
use grammers_session::Session;
use serde_json::json;
use std::collections::HashMap;
use std::sync::{Arc, Mutex};
use std::time::Duration;
use tokio::sync::mpsc;
/// How long a sign-in may wait for a person. Telegram's own code is good for a few minutes, and a form left open
/// overnight should not leave a task.
const PATIENCE: Duration = Duration::from_secs(5 * 60);
/// What the window can hand to a sign-in that is waiting.
enum Answer {
Code(String),
Password(String),
}
/// The sign-ins waiting for an answer, by id, and the way to give each one.
fn waiting() -> &'static Mutex<HashMap<String, mpsc::UnboundedSender<Answer>>> {
static WAITING: std::sync::OnceLock<Mutex<HashMap<String, mpsc::UnboundedSender<Answer>>>> = std::sync::OnceLock::new();
WAITING.get_or_init(Default::default)
}
fn give(login_id: &str, answer: Answer) -> Result<()> {
let sender = waiting().lock().unwrap().get(login_id).cloned().ok_or_else(|| anyhow!("that sign-in is not waiting for anything any more"))?;
sender.send(answer).map_err(|_| anyhow!("that sign-in has ended"))
}
/// Gives a sign-in the code Telegram sent.
pub fn submit_code(login_id: &str, code: &str) -> Result<()> {
// Telegram's own client shows the code as "12345"; a pasted one may carry spaces or a dash.
let code: String = code.chars().filter(|c| c.is_ascii_digit()).collect();
if code.is_empty() {
bail!("type the code first");
}
give(login_id, Answer::Code(code))
}
/// Gives a sign-in the account's cloud password.
pub fn submit_password(login_id: &str, password: &str) -> Result<()> {
if password.is_empty() {
bail!("type the password first");
}
give(login_id, Answer::Password(password.to_string()))
}
/// Stops a sign-in that is waiting, because the form was closed.
pub fn cancel(login_id: &str) {
waiting().lock().unwrap().remove(login_id);
}
/// A phone number as Telegram wants it: digits with a `+` in front.
pub fn normalize_phone(typed: &str) -> Option<String> {
let digits: String = typed.chars().filter(|c| c.is_ascii_digit()).collect();
// The shortest numbers in use are about seven digits and the longest fifteen.
(7..=15).contains(&digits.len()).then(|| format!("+{digits}"))
}
/// How the account is being signed in.
enum Method {
/// A phone number, then the code sent to it.
Phone(String),
/// A QR code scanned from a phone that is already signed in.
Qr,
}
/// Starts a sign-in by phone number. Answers at once; the rest is events.
pub fn start(state: AppState, login_id: String, api_id: i32, api_hash: String, phone: String) {
begin(state, login_id, api_id, api_hash, Method::Phone(phone));
}
/// Starts a sign-in by QR code. Answers at once; the rest is events.
pub fn start_qr(state: AppState, login_id: String, api_id: i32, api_hash: String) {
begin(state, login_id, api_id, api_hash, Method::Qr);
}
fn begin(state: AppState, login_id: String, api_id: i32, api_hash: String, method: Method) {
let (tx, rx) = mpsc::unbounded_channel();
waiting().lock().unwrap().insert(login_id.clone(), tx);
tokio::spawn(async move {
let outcome = flow(&state, &login_id, api_id, &api_hash, method, rx).await;
waiting().lock().unwrap().remove(&login_id);
match outcome {
Ok(Some(saved)) => {
let account = crate::accounts::telegram_account_to_json(&saved, "connecting");
super::connect::spawn(state.clone(), saved);
state.events.emit("telegramLoginResult", json!({ "loginId": login_id, "success": true, "account": account }));
}
// The form was closed; nobody is waiting for an answer.
Ok(None) => {}
Err(e) => {
let error = format!("{e:#}");
tracing::info!("telegram: sign-in {login_id}: {error}");
state.events.emit("telegramLoginResult", json!({ "loginId": login_id, "success": false, "error": error }));
}
}
});
}
fn status(state: &AppState, login_id: &str, text: &str) {
state.events.emit("telegramLoginStatus", json!({ "loginId": login_id, "text": text }));
}
fn needs(state: &AppState, login_id: &str, step: &str, hint: Option<&str>, error: Option<&str>) {
state.events.emit("telegramLoginNeeds", json!({ "loginId": login_id, "step": step, "hint": hint, "error": error }));
}
/// What was typed, or `None` when the form was closed or nobody answered in time.
async fn answer(rx: &mut mpsc::UnboundedReceiver<Answer>) -> Option<Answer> {
tokio::time::timeout(PATIENCE, rx.recv()).await.ok().flatten()
}
async fn flow(state: &AppState, login_id: &str, api_id: i32, api_hash: &str, method: Method, mut rx: mpsc::UnboundedReceiver<Answer>) -> Result<Option<TelegramAccountConfig>> {
if api_id <= 0 || api_hash.trim().is_empty() {
bail!("an api_id and an api_hash are needed: get them at my.telegram.org/apps");
}
let typed_phone = match &method {
Method::Phone(phone) => Some(normalize_phone(phone).ok_or_else(|| anyhow!("that does not look like a phone number; type it with its country code"))?),
Method::Qr => None,
};
status(state, login_id, "Connecting to Telegram");
let session = Arc::new(FileSession::fresh(session::path_for_login(login_id)));
let mut link = link::open(state, &crate::net::route::pending_key("telegram"), api_id, session.clone()).await?;
let outcome = match &typed_phone {
Some(phone) => signed_in(state, login_id, &link, api_hash, phone, &mut rx).await,
None => scanned(state, login_id, &mut link, api_id, api_hash, &mut rx).await,
};
let user = match outcome {
Ok(Some(user)) => user,
other => {
link.close().await;
session.delete();
return other.map(|_| None);
}
};
let config = TelegramAccountConfig {
api_id,
api_hash: api_hash.trim().to_string(),
// What was typed, or - signed in by QR code, where nothing was - what Telegram says the number is.
phone: typed_phone.or_else(|| user.phone().map(|p| format!("+{}", p.trim_start_matches('+')))).unwrap_or_default(),
user_id: user.id().bare_id_unchecked(),
username: user.username().map(str::to_string),
name: user.full_name(),
use_tor: crate::net::route::router().wanted(&crate::net::route::pending_key("telegram")),
..Default::default()
};
// The session is this account's now: under its own name, and written before the connection that made it closes.
session.move_to(session::path_for(config.user_id)).context("saving the session")?;
link.close().await;
let saved = state.accounts.add_telegram(config)?;
Ok(Some(saved))
}
/// The conversation with Telegram, up to being signed in.
async fn signed_in(state: &AppState, login_id: &str, link: &Link, api_hash: &str, phone: &str, rx: &mut mpsc::UnboundedReceiver<Answer>) -> Result<Option<grammers_client::peer::User>> {
let client = &link.client;
if client.is_authorized().await.context("asking Telegram whether this session is signed in")? {
bail!("this session is already signed in");
}
status(state, login_id, "Asking Telegram to send a code");
let token = client.request_login_code(phone, api_hash).await.map_err(|e| anyhow!("{}", explain(&e.to_string())))?;
needs(state, login_id, "code", None, None);
loop {
let Some(given) = answer(rx).await else { return Ok(None) };
match given {
Answer::Code(code) => match client.sign_in(&token, &code).await {
Ok(user) => return Ok(Some(user)),
Err(SignInError::PasswordRequired(password)) => {
let hint = password.hint().map(str::to_string);
needs(state, login_id, "password", hint.as_deref(), None);
return ask_password(state, login_id, client, password, rx).await;
}
Err(SignInError::InvalidCode) => needs(state, login_id, "code", None, Some("That code is not right. Check it and try again")),
Err(SignInError::SignUpRequired) => bail!("this number has no Telegram account. Make one with Telegram's own app first"),
Err(e) => bail!("{}", explain(&e.to_string())),
},
Answer::Password(_) => needs(state, login_id, "code", None, Some("Telegram has not asked for a password")),
}
}
}
/// How long the QR code is offered before the sign-in gives up: a form left open should not keep a task.
const QR_PATIENCE: Duration = Duration::from_secs(10 * 60);
/// How often the token is asked for again while nothing has been heard of a scan.
const QR_POLL: Duration = Duration::from_secs(15);
/// What asking for the login token came to.
#[derive(Debug)]
enum Exported {
/// A token to show, and when it stops being good (seconds since the epoch, Telegram's clock).
Token { token: Vec<u8>, expires: i32 },
/// Scanned and approved: the account.
Signed(Box<tl::types::auth::Authorization>),
/// Scanned, but the account has a cloud password.
NeedsPassword,
}
/// The link a phone's camera opens in Telegram.
pub fn qr_address(token: &[u8]) -> String {
use base64::Engine;
format!("tg://login?token={}", base64::engine::general_purpose::URL_SAFE_NO_PAD.encode(token))
}
fn qr_path(login_id: &str) -> std::path::PathBuf {
crate::media_cache::transient_dir().join(format!("moho-telegram-qr-{}.png", login_id.replace(|c: char| !c.is_ascii_alphanumeric(), "-")))
}
/// Asks Telegram for the login token, and follows it to the account's datacenter if it was scanned.
async fn export_token(client: &grammers_client::Client, session: &FileSession, api_id: i32, api_hash: &str) -> Result<Exported> {
let asked = client.invoke(&tl::functions::auth::ExportLoginToken { api_id, api_hash: api_hash.to_string(), except_ids: Vec::new() }).await;
let answer = match asked {
Ok(answer) => answer,
Err(e) if e.is("SESSION_PASSWORD_NEEDED") => return Ok(Exported::NeedsPassword),
Err(e) => bail!("{}", explain(&e.to_string())),
};
match answer {
tl::enums::auth::LoginToken::Token(t) => Ok(Exported::Token { token: t.token, expires: t.expires }),
tl::enums::auth::LoginToken::Success(s) => signed(s.authorization),
// The account is kept in another datacenter: the token is handed in there, and the key made there is the
// session's from then on.
tl::enums::auth::LoginToken::MigrateTo(m) => {
let imported = client.invoke_in_dc(m.dc_id, &tl::functions::auth::ImportLoginToken { token: m.token }).await;
match imported {
Ok(tl::enums::auth::LoginToken::Success(s)) => {
session.set_home_dc_id(m.dc_id).await.map_err(|e| anyhow!("{e}"))?;
signed(s.authorization)
}
Ok(_) => bail!("Telegram did not finish the sign-in in the account's datacenter"),
Err(e) if e.is("SESSION_PASSWORD_NEEDED") => {
session.set_home_dc_id(m.dc_id).await.map_err(|e| anyhow!("{e}"))?;
Ok(Exported::NeedsPassword)
}
Err(e) => bail!("{}", explain(&e.to_string())),
}
}
}
}
fn signed(authorization: tl::enums::auth::Authorization) -> Result<Exported> {
match authorization {
tl::enums::auth::Authorization::Authorization(a) => Ok(Exported::Signed(Box::new(a))),
tl::enums::auth::Authorization::SignUpRequired(_) => bail!("this number has no Telegram account. Make one with Telegram's own app first"),
}
}
/// What the library does when it signs in by code, which it does not do for a login token: the account's own entry
/// in the session, and where its updates stand.
async fn settle(client: &grammers_client::Client, session: &FileSession, authorization: tl::types::auth::Authorization) -> Result<grammers_client::peer::User> {
use grammers_session::types::{PeerInfo, UpdateState, UpdatesState};
let update_state = client.invoke(&tl::functions::updates::GetState {}).await.ok();
let user = grammers_client::peer::User::from_raw(client, authorization.user);
let auth = user.to_ref().await.map_err(|e| anyhow!("{e}"))?.context("Telegram did not say who signed in")?.auth;
session
.cache_peer(PeerInfo::User { id: user.id().bare_id_unchecked(), auth: Some(auth), bot: Some(user.is_bot()), is_self: Some(true) })
.await
.map_err(|e| anyhow!("{e}"))?;
if let Some(tl::enums::updates::State::State(state)) = update_state {
session
.set_update_state(UpdateState::All(UpdatesState { pts: state.pts, qts: state.qts, date: state.date, seq: state.seq, channels: Vec::new() }))
.await
.map_err(|e| anyhow!("{e}"))?;
}
Ok(user)
}
/// The conversation with Telegram, up to being signed in by QR code.
async fn scanned(state: &AppState, login_id: &str, link: &mut Link, api_id: i32, api_hash: &str, rx: &mut mpsc::UnboundedReceiver<Answer>) -> Result<Option<grammers_client::peer::User>> {
let client = link.client.clone();
let session = link.session.clone();
if client.is_authorized().await.context("asking Telegram whether this session is signed in")? {
bail!("this session is already signed in");
}
// Telegram says a token was scanned with an update; a connection that cannot read it is covered by asking again.
let mut stream = match link.updates.take() {
Some(updates) => client.stream_updates(updates, grammers_mtsender::UpdatesConfiguration { catch_up: false }).await.ok(),
None => None,
};
let began = tokio::time::Instant::now();
let mut shown: Vec<u8> = Vec::new();
status(state, login_id, "Asking Telegram for a code to scan");
loop {
if began.elapsed() > QR_PATIENCE {
bail!("The code was not scanned in time. Start again to get a new one");
}
match export_token(&client, &session, api_id, api_hash).await? {
Exported::Signed(authorization) => return Ok(Some(settle(&client, &session, *authorization).await?)),
Exported::NeedsPassword => return ask_cloud_password(state, login_id, &client, rx).await,
Exported::Token { token, expires } => {
if token != shown {
let path = qr_path(login_id);
super::super::steam::login::write_qr(&qr_address(&token), &path)?;
state.events.emit("telegramLoginQr", json!({ "loginId": login_id, "path": path.to_string_lossy(), "expires": expires }));
shown = token;
}
// Until it is scanned, it runs out, or it is time to ask again.
let now = chrono::Utc::now().timestamp();
let until = Duration::from_secs((i64::from(expires) - now - 2).clamp(1, QR_POLL.as_secs() as i64) as u64);
let heard = async {
match stream.as_mut() {
Some(stream) => loop {
match stream.next().await {
Ok(grammers_client::update::Update::Raw(raw)) if matches!(raw.raw, tl::enums::Update::LoginToken) => return,
Ok(_) => {}
// Not readable here: the poll is what notices a scan.
Err(_) => return std::future::pending().await,
}
},
None => std::future::pending().await,
}
};
tokio::select! {
_ = tokio::time::sleep(until) => {}
_ = heard => {}
given = rx.recv() => {
// The form was closed.
if given.is_none() {
return Ok(None);
}
}
}
}
}
}
}
/// Asks for the cloud password of an account that was scanned in, and signs it in.
async fn ask_cloud_password(state: &AppState, login_id: &str, client: &grammers_client::Client, rx: &mut mpsc::UnboundedReceiver<Answer>) -> Result<Option<grammers_client::peer::User>> {
let password: tl::types::account::Password = client.invoke(&tl::functions::account::GetPassword {}).await.map_err(|e| anyhow!("{}", explain(&e.to_string())))?.into();
let token = grammers_client::client::PasswordToken::new(password);
needs(state, login_id, "password", token.hint(), None);
ask_password(state, login_id, client, token, rx).await
}
async fn ask_password(state: &AppState, login_id: &str, client: &grammers_client::Client, mut token: grammers_client::client::PasswordToken, rx: &mut mpsc::UnboundedReceiver<Answer>) -> Result<Option<grammers_client::peer::User>> {
loop {
let Some(given) = answer(rx).await else { return Ok(None) };
let Answer::Password(password) = given else { continue };
match client.check_password(token, password.as_bytes()).await {
Ok(user) => return Ok(Some(user)),
Err(SignInError::InvalidPassword(again)) => {
needs(state, login_id, "password", again.hint(), Some("That password is not right"));
token = again;
}
Err(e) => bail!("{}", explain(&e.to_string())),
}
}
}
/// Telegram's own wording for what went wrong, said so a person can act on it.
pub fn explain(raw: &str) -> String {
let upper = raw.to_uppercase();
for (code, words) in [
("API_ID_INVALID", "Telegram does not accept that api_id and api_hash pair. Check them at my.telegram.org/apps"),
("PHONE_NUMBER_INVALID", "Telegram says that is not a valid phone number"),
("PHONE_NUMBER_BANNED", "Telegram has banned that phone number"),
("PHONE_CODE_EXPIRED", "That code has expired. Start again to get a new one"),
("PHONE_CODE_INVALID", "That code is not right"),
("PASSWORD_HASH_INVALID", "That password is not right"),
("FLOOD_WAIT", "Telegram says too many attempts were made. Wait before trying again"),
("SESSION_PASSWORD_NEEDED", "This account has a cloud password"),
("AUTH_KEY_UNREGISTERED", "Telegram has ended this session"),
("USER_DEACTIVATED", "Telegram says this account has been deleted"),
] {
if upper.contains(code) {
return words.to_string();
}
}
raw.to_string()
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_phone_number_is_made_the_way_telegram_wants_it() {
assert_eq!(normalize_phone("+1 (555) 123-4567").as_deref(), Some("+15551234567"));
assert_eq!(normalize_phone("44 20 7946 0958").as_deref(), Some("+442079460958"));
assert_eq!(normalize_phone("12345"), None);
assert_eq!(normalize_phone("not a number"), None);
}
#[test]
fn answers_for_a_sign_in_that_is_not_waiting_are_refused_in_words() {
assert!(submit_code("telegram-none", "12345").unwrap_err().to_string().contains("not waiting"));
assert!(submit_code("telegram-none", "abc").unwrap_err().to_string().contains("type the code"));
let (tx, mut rx) = mpsc::unbounded_channel();
waiting().lock().unwrap().insert("telegram-t".into(), tx);
submit_code("telegram-t", "1 2-3 4 5").unwrap();
submit_password("telegram-t", "hunter2").unwrap();
assert!(matches!(rx.try_recv(), Ok(Answer::Code(c)) if c == "12345"), "spaces and dashes in a pasted code are dropped");
assert!(matches!(rx.try_recv(), Ok(Answer::Password(p)) if p == "hunter2"));
cancel("telegram-t");
assert!(submit_code("telegram-t", "12345").is_err());
}
#[test]
fn a_login_token_is_offered_as_telegrams_link_with_the_token_in_base64url() {
// The alphabet is the URL-safe one, and there is no padding: a phone reads the link as it is written.
assert_eq!(qr_address(&[0xfb, 0xff]), "tg://login?token=-_8");
assert_eq!(qr_address(&[1, 2, 3, 4]), "tg://login?token=AQIDBA");
assert!(qr_address(&[0; 32]).starts_with("tg://login?token="));
}
#[test]
fn a_scanned_codes_picture_is_written_where_the_window_can_read_it() {
let path = std::env::temp_dir().join(format!("moho-telegram-qr-test-{}.png", std::process::id()));
super::super::super::steam::login::write_qr(&qr_address(&[7; 32]), &path).unwrap();
let bytes = std::fs::read(&path).unwrap();
std::fs::remove_file(&path).ok();
assert_eq!(&bytes[..4], &[0x89, b'P', b'N', b'G']);
}
#[test]
fn a_sign_in_by_qr_is_waited_on_like_one_by_phone_and_given_up_by_closing_the_form() {
let (tx, mut rx) = mpsc::unbounded_channel();
waiting().lock().unwrap().insert("telegram-qr".into(), tx);
cancel("telegram-qr");
assert!(rx.try_recv().is_err());
assert!(submit_password("telegram-qr", "x").is_err(), "nothing is waiting for a password once the form is closed");
}
#[test]
fn telegrams_refusals_are_said_so_a_person_can_act_on_them() {
assert!(explain("rpc error 400: API_ID_INVALID").contains("my.telegram.org"));
assert!(explain("PHONE_CODE_EXPIRED").contains("expired"));
assert_eq!(explain("something unforeseen"), "something unforeseen");
}
/// Against Telegram's own test servers, which have no real accounts: a number `99966XYYYY` signs in with the code
/// `XXXXX` (its digit X, five times). Needs the person's `api_id` and `api_hash` (the test servers take a real pair):
///
/// `TELEGRAM_LIVE_API_ID=... TELEGRAM_LIVE_API_HASH=... TELEGRAM_LIVE_PHONE=+99966XYYYY cargo test --release live_ -- --ignored --nocapture`
#[tokio::test]
#[ignore]
async fn live_a_test_number_signs_in_and_out_on_telegrams_test_servers() {
use grammers_session::types::DcOption;
use grammers_session::Session;
let _ = rustls::crypto::ring::default_provider().install_default();
let api_id: i32 = std::env::var("TELEGRAM_LIVE_API_ID").unwrap().parse().unwrap();
let api_hash = std::env::var("TELEGRAM_LIVE_API_HASH").unwrap();
let phone = normalize_phone(&std::env::var("TELEGRAM_LIVE_PHONE").unwrap()).unwrap();
assert!(phone.starts_with("+99966"), "only a test number, which has no real account behind it");
let code: String = std::iter::repeat_n(phone.chars().nth(6).unwrap(), 5).collect();
let state = crate::backend::kick::testkit::simulated_daemon("telegram-live");
let path = std::env::temp_dir().join(format!("nobilis-telegram-live-{}.session", std::process::id()));
let session = Arc::new(FileSession::fresh(path.clone()));
// Telegram's test datacenter 2 stands where the real one does.
let mut dc: DcOption = session.dc_option(2).unwrap().unwrap();
dc.ipv4 = "149.154.167.40:443".parse().unwrap();
dc.ipv6 = "[::ffff:149.154.167.40]:443".parse().unwrap();
session.set_dc_option(&dc).await.unwrap();
session.set_home_dc_id(2).await.unwrap();
let link = link::open(&state, "telegram:live-test", api_id, session.clone()).await.unwrap();
assert!(!link.client.is_authorized().await.unwrap());
let token = link.client.request_login_code(&phone, &api_hash).await.unwrap();
let user = match link.client.sign_in(&token, &code).await {
Ok(user) => user,
Err(e) => panic!("{}", explain(&e.to_string())),
};
println!("signed in as {} ({})", user.full_name(), user.id().bare_id_unchecked());
assert!(link.client.is_authorized().await.unwrap());
assert!(session.has_key());
session.save().unwrap();
let reopened = FileSession::open(path.clone()).unwrap();
assert!(reopened.has_key(), "the saved session still has its key");
link.client.sign_out().await.unwrap();
link.close().await;
std::fs::remove_file(path).ok();
}
}
src/backend/telegram/markup.rs
+284
View File
@@ -0,0 +1,284 @@
//! What the message box holds, as Telegram reads it: plain text and a list of entities.
//!
//! The window writes formatting as Markdown, the way Discord reads it (`**bold**`, `*italic*`, `__underline__`,
//! `~~strike~~`, `||spoiler||`, `` `code` ``, fenced code, `> quote`, `[text](url)`), and Telegram wants the text with
//! the marks taken out and a list saying where each kind starts and how long it runs, counted in UTF-16 code units.
//! A mark with no partner is the character it looks like.
use grammers_client::tl;
#[derive(Debug, PartialEq, Clone, Copy)]
enum Kind {
Bold,
Italic,
Underline,
Strike,
Spoiler,
Code,
Pre,
Quote,
}
#[derive(Debug, PartialEq)]
struct Span {
kind: Kind,
offset: usize,
length: usize,
/// For a link, where it goes.
url: Option<String>,
/// For a fenced block, its language.
language: Option<String>,
}
#[derive(Default)]
struct Out {
text: String,
units: usize,
spans: Vec<Span>,
}
impl Out {
fn push(&mut self, c: char) {
self.text.push(c);
self.units += c.len_utf16();
}
fn push_str(&mut self, s: &str) {
for c in s.chars() {
self.push(c);
}
}
}
/// The text with its marks taken out, and the entities that say what they marked.
pub fn parse(source: &str) -> (String, Vec<tl::enums::MessageEntity>) {
let mut out = Out::default();
let lines: Vec<&str> = source.split('\n').collect();
let mut i = 0;
let mut first = true;
while i < lines.len() {
if !first {
out.push('\n');
}
first = false;
let line = lines[i];
if let Some(fence) = line.strip_prefix("```") {
// A fenced block runs to the next fence, or to the end.
if let Some(end) = lines[i + 1..].iter().position(|l| l.trim_end() == "```") {
let language = fence.trim().to_string();
let body = lines[i + 1..i + 1 + end].join("\n");
let offset = out.units;
out.push_str(&body);
out.spans.push(Span { kind: Kind::Pre, offset, length: out.units - offset, url: None, language: (!language.is_empty()).then_some(language) });
i += end + 2;
continue;
}
}
if line.starts_with("> ") || line == ">" {
// Consecutive quoted lines are one quote.
let start = out.units;
let mut quoted = vec![line.strip_prefix("> ").unwrap_or("")];
while i + 1 < lines.len() && (lines[i + 1].starts_with("> ") || lines[i + 1] == ">") {
i += 1;
quoted.push(lines[i].strip_prefix("> ").unwrap_or(""));
}
for (n, q) in quoted.iter().enumerate() {
if n > 0 {
out.push('\n');
}
inline(&q.chars().collect::<Vec<_>>(), &mut out);
}
out.spans.push(Span { kind: Kind::Quote, offset: start, length: out.units - start, url: None, language: None });
i += 1;
continue;
}
inline(&line.chars().collect::<Vec<_>>(), &mut out);
i += 1;
}
let entities = out.spans.iter().filter(|s| s.length > 0).map(entity).collect();
(out.text, entities)
}
fn entity(s: &Span) -> tl::enums::MessageEntity {
use tl::enums::MessageEntity as E;
use tl::types as t;
let (offset, length) = (s.offset as i32, s.length as i32);
if let Some(url) = &s.url {
return E::TextUrl(t::MessageEntityTextUrl { offset, length, url: url.clone() });
}
match s.kind {
Kind::Bold => E::Bold(t::MessageEntityBold { offset, length }),
Kind::Italic => E::Italic(t::MessageEntityItalic { offset, length }),
Kind::Underline => E::Underline(t::MessageEntityUnderline { offset, length }),
Kind::Strike => E::Strike(t::MessageEntityStrike { offset, length }),
Kind::Spoiler => E::Spoiler(t::MessageEntitySpoiler { offset, length }),
Kind::Code => E::Code(t::MessageEntityCode { offset, length }),
Kind::Pre => E::Pre(t::MessageEntityPre { offset, length, language: s.language.clone().unwrap_or_default() }),
Kind::Quote => E::Blockquote(t::MessageEntityBlockquote { collapsed: false, offset, length }),
}
}
fn find(chars: &[char], from: usize, marker: &[char]) -> Option<usize> {
(from..chars.len().saturating_sub(marker.len() - 1)).find(|&j| chars[j..].starts_with(marker))
}
/// A single `*` or `_` only opens at the start of a word and closes at the end of one, so `snake_case_name` and `2*3*4`
/// stay as written.
fn word_edge(chars: &[char], at: usize, opening: bool) -> bool {
let alnum = |c: char| c.is_alphanumeric();
if opening {
(at == 0 || !alnum(chars[at - 1])) && chars.get(at + 1).is_some_and(|c| !c.is_whitespace())
} else {
at > 0 && !chars[at - 1].is_whitespace() && chars.get(at + 1).is_none_or(|c| !alnum(*c))
}
}
fn inline(chars: &[char], out: &mut Out) {
let mut i = 0;
while i < chars.len() {
let c = chars[i];
if c == '\\' && i + 1 < chars.len() && "\\*_~|`[]()>".contains(chars[i + 1]) {
out.push(chars[i + 1]);
i += 2;
continue;
}
// Code takes its contents as they are.
if c == '`' {
if let Some(end) = find(chars, i + 1, &['`']).filter(|&e| e > i + 1) {
let offset = out.units;
out.push_str(&chars[i + 1..end].iter().collect::<String>());
out.spans.push(Span { kind: Kind::Code, offset, length: out.units - offset, url: None, language: None });
i = end + 1;
continue;
}
}
// [text](url)
if c == '[' {
if let Some(close) = find(chars, i + 1, &[']']) {
if chars.get(close + 1) == Some(&'(') {
if let Some(end) = find(chars, close + 2, &[')']) {
let url: String = chars[close + 2..end].iter().collect();
if ["http://", "https://", "tg://", "mailto:"].iter().any(|p| url.starts_with(p)) && close > i + 1 {
let offset = out.units;
inline(&chars[i + 1..close], out);
out.spans.push(Span { kind: Kind::Bold, offset, length: out.units - offset, url: Some(url), language: None });
i = end + 1;
continue;
}
}
}
}
}
let two = |m: &str| m.chars().collect::<Vec<_>>();
let pairs: [(&str, Kind); 4] = [("**", Kind::Bold), ("__", Kind::Underline), ("~~", Kind::Strike), ("||", Kind::Spoiler)];
let mut done = false;
for (marker, kind) in pairs {
let m = two(marker);
if chars[i..].starts_with(&m) {
if let Some(end) = find(chars, i + 2, &m).filter(|&e| e > i + 2) {
let offset = out.units;
inline(&chars[i + 2..end], out);
out.spans.push(Span { kind, offset, length: out.units - offset, url: None, language: None });
i = end + 2;
done = true;
break;
}
}
}
if done {
continue;
}
if (c == '*' || c == '_') && word_edge(chars, i, true) {
// The closing one: the next of the same mark that ends a word.
let mut j = i + 1;
let mut found = None;
while let Some(e) = find(chars, j, &[c]) {
if e > i + 1 && word_edge(chars, e, false) {
found = Some(e);
break;
}
j = e + 1;
}
if let Some(end) = found {
let offset = out.units;
inline(&chars[i + 1..end], out);
out.spans.push(Span { kind: Kind::Italic, offset, length: out.units - offset, url: None, language: None });
i = end + 1;
continue;
}
}
out.push(c);
i += 1;
}
}
#[cfg(test)]
mod tests {
use super::*;
use tl::enums::MessageEntity as E;
fn kinds(entities: &[E]) -> Vec<(&'static str, i32, i32)> {
entities
.iter()
.map(|e| match e {
E::Bold(x) => ("bold", x.offset, x.length),
E::Italic(x) => ("italic", x.offset, x.length),
E::Underline(x) => ("underline", x.offset, x.length),
E::Strike(x) => ("strike", x.offset, x.length),
E::Spoiler(x) => ("spoiler", x.offset, x.length),
E::Code(x) => ("code", x.offset, x.length),
E::Pre(x) => ("pre", x.offset, x.length),
E::Blockquote(x) => ("quote", x.offset, x.length),
E::TextUrl(x) => ("link", x.offset, x.length),
_ => ("other", 0, 0),
})
.collect()
}
#[test]
fn marks_are_taken_out_and_the_entities_say_where_they_were() {
let (text, entities) = parse("a **bold** and *italic* and ~~gone~~ and ||secret|| and __under__");
assert_eq!(text, "a bold and italic and gone and secret and under");
let got = kinds(&entities);
assert!(got.contains(&("bold", 2, 4)) && got.contains(&("italic", 11, 6)) && got.contains(&("strike", 22, 4)) && got.contains(&("spoiler", 31, 6)) && got.contains(&("underline", 42, 5)), "{got:?}");
}
#[test]
fn marks_nest_and_a_mark_with_no_partner_is_just_the_character() {
let (text, entities) = parse("**a *b* c**");
assert_eq!(text, "a b c");
let got = kinds(&entities);
assert!(got.contains(&("bold", 0, 5)) && got.contains(&("italic", 2, 1)), "{got:?}");
let (text, entities) = parse("2 * 3 and snake_case_name and **unclosed");
assert_eq!(text, "2 * 3 and snake_case_name and **unclosed");
assert!(entities.is_empty());
}
#[test]
fn code_is_taken_as_written_and_a_fence_is_a_block_with_its_language() {
let (text, entities) = parse("see `**not bold**` ok");
assert_eq!(text, "see **not bold** ok");
assert_eq!(kinds(&entities), vec![("code", 4, 12)]);
let (text, entities) = parse("```rust\nlet x = 1;\n```\nafter");
assert_eq!(text, "let x = 1;\nafter");
assert!(matches!(&entities[0], E::Pre(p) if p.language == "rust" && p.offset == 0 && p.length == 10));
}
#[test]
fn a_link_and_a_quote_and_an_escape() {
let (text, entities) = parse("[here](https://example.com/a?b=1) \\*not\\*");
assert_eq!(text, "here *not*");
assert!(matches!(&entities[0], E::TextUrl(u) if u.url == "https://example.com/a?b=1" && u.offset == 0 && u.length == 4));
let (text, entities) = parse("> one\n> two\nthree");
assert_eq!(text, "one\ntwo\nthree");
assert_eq!(kinds(&entities), vec![("quote", 0, 7)]);
// A link to somewhere that is not a web address is left as it was typed.
assert_eq!(parse("[x](javascript:alert(1))").0, "[x](javascript:alert(1))");
}
#[test]
fn offsets_count_utf16_units_so_an_emoji_before_a_mark_moves_it_by_two() {
let (text, entities) = parse("😀 **hi**");
assert_eq!(text, "😀 hi");
assert_eq!(kinds(&entities), vec![("bold", 3, 2)]);
}
}
src/backend/telegram/media.rs
+490
View File
@@ -0,0 +1,490 @@
//! Pictures, video, files, voice and stickers: what a message carries, and fetching it.
//!
//! Nothing is fetched until it is looked at. A message is recorded with attachments that name files in the daemon's
//! caches (`telegram-media`, `telegram-avatars`) - files that need not exist yet - and when the window asks for one
//! that is not there, the "restore a cached picture" route it already has for every other service asks the daemon,
//! which finds the file's message again (a file reference expires, so the one remembered would not do), downloads it
//! and answers. So opening a channel of fifty posts fetches the pictures on screen, not fifty videos.
//!
//! A cached file is named for what it is: `tg_<account>_<conversation>_<message>_<thumb|full>.<ext>` for a message's
//! media, `tg_<account>_<person>_<photo>.jpg` for a profile picture.
use super::peers;
use super::Handle;
use crate::model::{Attachment, Embed};
use crate::state::AppState;
use anyhow::{anyhow, bail, Context, Result};
use grammers_client::media::{Document, Downloadable, Media, PhotoSize};
use grammers_client::message::Message;
use grammers_client::tl;
use grammers_session::types::PeerId;
use grammers_session::Session;
use std::path::{Path, PathBuf};
pub const MEDIA: &str = "telegram-media";
pub const AVATARS: &str = "telegram-avatars";
/// How many downloads run at once: enough for a screenful of pictures, few enough not to look like a scraper.
const AT_ONCE: usize = 4;
/// How many whole files (videos, documents) are fetched at once. They take minutes, and pictures must not wait behind
/// them: those have places of their own.
const BIG_AT_ONCE: usize = 2;
fn permits(big: bool) -> &'static tokio::sync::Semaphore {
static P: std::sync::OnceLock<tokio::sync::Semaphore> = std::sync::OnceLock::new();
static BIG: std::sync::OnceLock<tokio::sync::Semaphore> = std::sync::OnceLock::new();
if big {
BIG.get_or_init(|| tokio::sync::Semaphore::new(BIG_AT_ONCE))
} else {
P.get_or_init(|| tokio::sync::Semaphore::new(AT_ONCE))
}
}
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
pub enum Variant {
/// A small picture to draw in the list.
Thumb,
/// The picture at the size the viewer shows, or the file itself.
Full,
}
impl Variant {
fn word(self) -> &'static str {
match self {
Variant::Thumb => "thumb",
Variant::Full => "full",
}
}
}
/// A conversation's id as a piece of a file name: a group's `-` would be taken for a separator.
fn slug(remote: &str) -> String {
remote.replace('-', "n")
}
fn unslug(text: &str) -> String {
match text.strip_prefix('n') {
Some(rest) => format!("-{rest}"),
None => text.to_string(),
}
}
pub fn media_name(account: i64, remote: &str, message: i32, variant: Variant, ext: &str) -> String {
format!("tg_{account}_{}_{message}_{}.{ext}", slug(remote), variant.word())
}
pub fn avatar_name(account: i64, remote: &str, photo: i64) -> String {
format!("tg_{account}_{}_{photo}.jpg", slug(remote))
}
/// What a cached media file's name says: the account, the conversation, the message and which version of it.
pub fn parse_media_name(name: &str) -> Option<(i64, String, i32, Variant)> {
let stem = name.rsplit_once('.')?.0;
let mut parts = stem.strip_prefix("tg_")?.split('_');
let account = parts.next()?.parse().ok()?;
let remote = unslug(parts.next()?);
let message = parts.next()?.parse().ok()?;
let variant = match parts.next()? {
"thumb" => Variant::Thumb,
"full" => Variant::Full,
_ => return None,
};
parts.next().is_none().then_some((account, remote, message, variant))
}
/// What a cached avatar's name says: the account, the person or chat, and which picture of theirs.
pub fn parse_avatar_name(name: &str) -> Option<(i64, String, i64)> {
let stem = name.strip_suffix(".jpg")?;
let mut parts = stem.strip_prefix("tg_")?.split('_');
let account = parts.next()?.parse().ok()?;
let remote = unslug(parts.next()?);
let photo = parts.next()?.parse().ok()?;
parts.next().is_none().then_some((account, remote, photo))
}
fn local(dir: &str, name: &str) -> String {
format!("file://{}", crate::media_cache::dir(dir).join(name).display())
}
/// The file's extension, from its name or else its type.
pub fn extension_of(name: Option<&str>, mime: Option<&str>) -> String {
if let Some(ext) = name.and_then(|n| n.rsplit_once('.')).map(|(_, e)| e.to_ascii_lowercase()).filter(|e| !e.is_empty() && e.len() <= 8 && e.chars().all(|c| c.is_ascii_alphanumeric())) {
return ext;
}
match mime.unwrap_or("") {
"image/jpeg" => "jpg",
"image/png" => "png",
"image/webp" => "webp",
"image/gif" => "gif",
"video/mp4" => "mp4",
"video/webm" => "webm",
"video/quicktime" => "mov",
"audio/ogg" | "audio/opus" => "ogg",
"audio/mpeg" => "mp3",
"audio/mp4" | "audio/m4a" => "m4a",
"application/pdf" => "pdf",
"application/zip" => "zip",
"application/x-tgsticker" => "tgs",
"text/plain" => "txt",
_ => "bin",
}
.to_string()
}
/// The broad shape of a document, which decides how the window draws it.
fn kind_of_mime(mime: &str) -> &'static str {
if mime.starts_with("image/") && mime != "image/gif" {
"image"
} else if mime.starts_with("video/") || mime == "image/gif" {
"video"
} else if mime.starts_with("audio/") {
"audio"
} else {
"file"
}
}
fn dimensions(size: &PhotoSize) -> Option<(i32, i32)> {
match size {
PhotoSize::Size(s) => Some((s.width, s.height)),
PhotoSize::Cached(s) => Some((s.width, s.height)),
PhotoSize::Progressive(s) => Some((s.width, s.height)),
_ => None,
}
}
/// The version of a photo to draw in the list (about 320 wide) and the one to show in the viewer (about 1280).
pub(super) fn pick(sizes: &[PhotoSize], target: i32) -> Option<&PhotoSize> {
let sized: Vec<(&PhotoSize, i32)> = sizes.iter().filter_map(|s| dimensions(s).map(|(w, h)| (s, w.max(h)))).collect();
choose(&sized, target).copied()
}
/// The smallest that reaches the target, else the largest there is.
fn choose<T>(items: &[(T, i32)], target: i32) -> Option<&T> {
let mut by_size: Vec<&(T, i32)> = items.iter().collect();
by_size.sort_by_key(|(_, longest)| *longest);
by_size.iter().find(|(_, longest)| *longest >= target).or(by_size.last()).map(|(item, _)| item)
}
/// The width a link card's picture is fetched at: it is drawn larger than a list thumbnail.
const CARD: i32 = 800;
/// The picture of the web page a message links to, if the page has one.
fn page_photo(media: &Media) -> Option<grammers_client::media::Photo> {
let Media::WebPage(page) = media else { return None };
let tl::enums::WebPage::Page(page) = &page.raw.webpage else { return None };
page.photo.clone().map(grammers_client::media::Photo::from_raw)
}
/// The link preview a message carries (what Telegram calls a web page, with or without an Instant View): a card with
/// the site, title, description and picture. Its picture is named in the cache and fetched like any preview.
pub fn embeds_of(account: i64, remote: &str, message: &Message) -> Vec<Embed> {
let Some(media) = message.media() else { return Vec::new() };
let Media::WebPage(page) = &media else { return Vec::new() };
let tl::enums::WebPage::Page(page) = &page.raw.webpage else { return Vec::new() };
let has_picture = page_photo(&media).is_some();
if page.title.is_none() && page.description.is_none() && !has_picture {
return Vec::new();
}
let non_empty = |t: &Option<String>| t.clone().filter(|t| !t.trim().is_empty());
vec![Embed {
title: non_empty(&page.title),
description: non_empty(&page.description),
url: Some(page.url.clone()),
provider: non_empty(&page.site_name),
author: non_empty(&page.author),
image_url: has_picture.then(|| local(MEDIA, &media_name(account, remote, message.id(), Variant::Thumb, "jpg"))),
// A page Telegram has made an article of opens as one: the card says so, as Telegram's own does.
kind: page.cached_page.is_some().then(|| "instant".to_string()),
cta: page.cached_page.is_some().then(|| crate::model::EmbedLink { label: "Instant View".to_string(), url: None }),
..Default::default()
}]
}
/// What the message carries, as attachments naming files in the cache. Nothing is downloaded here.
pub fn attachments_of(account: i64, remote: &str, message: &Message) -> Vec<Attachment> {
let id = message.id();
let Some(media) = message.media() else { return Vec::new() };
let thumb = |ext: &str| Some(local(MEDIA, &media_name(account, remote, id, Variant::Thumb, ext)));
let full = |ext: &str| Some(local(MEDIA, &media_name(account, remote, id, Variant::Full, ext)));
match media {
Media::Photo(photo) => {
let sizes = photo.thumbs();
let view = pick(&sizes, 1280).and_then(dimensions);
vec![Attachment {
kind: "image".into(),
mimetype: Some("image/jpeg".into()),
filename: Some(if photo.is_spoiler() { "SPOILER_photo.jpg".into() } else { "photo.jpg".into() }),
size: photo.size().map(|s| s as u64),
width: view.map(|(w, _)| w as u32),
height: view.map(|(_, h)| h as u32),
path: full("jpg"),
thumbnail_path: thumb("jpg"),
..Default::default()
}]
}
Media::Sticker(sticker) => {
let document = sticker.document.clone();
let mime = document.mime_type().unwrap_or("image/webp").to_string();
let still = mime == "image/webp";
let (w, h) = document.resolution().unwrap_or((512, 512));
vec![Attachment {
kind: "image".into(),
mimetype: Some(if still { mime.clone() } else { "image/webp".into() }),
filename: Some(format!("sticker{}", if sticker.emoji().is_empty() { String::new() } else { format!(" {}", sticker.emoji()) })),
size: document.size().map(|s| s as u64),
width: Some(w as u32),
height: Some(h as u32),
// An animated or video sticker is drawn as the still picture Telegram sends with it.
path: if still { full(&extension_of(document.name(), Some(&mime))) } else { thumb("jpg") },
thumbnail_path: thumb("jpg"),
..Default::default()
}]
}
Media::Document(document) => {
let mime = document.mime_type().unwrap_or("application/octet-stream").to_string();
let name = document.name().map(str::to_string);
let ext = extension_of(name.as_deref(), Some(&mime));
let voice = is_voice(&document);
let mut kind = kind_of_mime(&mime);
if document.is_animated() && kind == "image" {
kind = "video";
}
let (w, h) = document.resolution().map(|(w, h)| (Some(w as u32), Some(h as u32))).unwrap_or((None, None));
vec![Attachment {
kind: kind.into(),
mimetype: Some(mime),
filename: Some(match (&name, document.is_spoiler()) {
(Some(n), true) => format!("SPOILER_{n}"),
(Some(n), false) => n.clone(),
(None, _) => format!("file.{ext}"),
}),
size: document.size().map(|s| s as u64),
width: w,
height: h,
duration_secs: document.duration(),
voice,
path: full(&ext),
thumbnail_path: (!document.thumbs().is_empty() && matches!(kind, "video" | "file" | "image")).then(|| local(MEDIA, &media_name(account, remote, id, Variant::Thumb, "jpg"))),
..Default::default()
}]
}
_ => Vec::new(),
}
}
/// Whether a recording was spoken into the app rather than attached as a file.
fn is_voice(document: &Document) -> bool {
match document.raw.document.as_ref() {
Some(tl::enums::Document::Document(d)) => d.attributes.iter().any(|a| matches!(a, tl::enums::DocumentAttribute::Audio(audio) if audio.voice)),
_ => false,
}
}
/// Downloads to a file, whole and renamed into place so a half-written one is never taken for a picture.
pub(super) async fn download_to<D: Downloadable>(client: &grammers_client::Client, what: &D, path: &Path, big: bool) -> Result<()> {
let _permit = permits(big).acquire().await?;
if let Some(parent) = path.parent() {
tokio::fs::create_dir_all(parent).await?;
}
// Its own name for the half-written file: two fetches of one picture (a preview being fetched as the message was
// recorded, and the window asking for it) must not write into the same one.
static SEQUENCE: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(0);
let part = path.with_extension(format!("{}.part", SEQUENCE.fetch_add(1, std::sync::atomic::Ordering::Relaxed)));
if let Err(e) = client.download_media(what, &part).await {
let _ = tokio::fs::remove_file(&part).await;
return Err(anyhow!("{}", super::send::explain(&e)));
}
tokio::fs::rename(&part, path).await.with_context(|| format!("writing {}", path.display()))?;
Ok(())
}
/// Fetches a message's media again and writes the version the name asks for.
async fn fetch(handle: &Handle, remote: &str, message: i32, variant: Variant, path: &Path) -> Result<()> {
let id = peers::peer_id_of(remote).ok_or_else(|| anyhow!("not a Telegram conversation"))?;
let peer = handle.session.peer_ref(id).await.ok().flatten().ok_or_else(|| anyhow!("Telegram has not told this account about that conversation yet"))?;
let found = handle.client.get_messages_by_id(peer, &[message]).await.map_err(|e| anyhow!("{}", super::send::explain(&e)))?;
let Some(Some(message)) = found.into_iter().next() else { bail!("that message is no longer there") };
match message.media().ok_or_else(|| anyhow!("that message has no media"))? {
Media::Photo(photo) => {
let sizes = photo.thumbs();
let wanted = match variant {
Variant::Thumb => pick(&sizes, 320),
Variant::Full => pick(&sizes, 1280),
};
match wanted {
Some(size) => download_to(&handle.client, size, path, false).await,
None => download_to(&handle.client, &photo, path, false).await,
}
}
Media::Sticker(sticker) => match variant {
Variant::Full => download_to(&handle.client, &sticker.document, path, false).await,
Variant::Thumb => download_thumb(&handle.client, &sticker.document, path).await,
},
Media::Document(document) => match variant {
Variant::Full => download_to(&handle.client, &document, path, true).await,
Variant::Thumb => download_thumb(&handle.client, &document, path).await,
},
media @ Media::WebPage(_) => {
let photo = page_photo(&media).ok_or_else(|| anyhow!("that link has no picture"))?;
match pick(&photo.thumbs(), CARD) {
Some(size) => download_to(&handle.client, size, path, false).await,
None => download_to(&handle.client, &photo, path, false).await,
}
}
_ => bail!("that kind of media cannot be fetched"),
}
}
pub(super) async fn download_thumb(client: &grammers_client::Client, document: &Document, path: &Path) -> Result<()> {
let sizes = document.thumbs();
let size = pick(&sizes, 320).or(sizes.first()).ok_or_else(|| anyhow!("Telegram sent no preview of that"))?;
// A tiny inline preview carries its own bytes; the rest are downloaded.
if let Some(bytes) = Downloadable::to_data(size) {
tokio::fs::write(path, bytes).await?;
return Ok(());
}
download_to(client, size, path, false).await
}
/// The previews being fetched now, so a message read twice at once fetches its preview once.
fn fetching() -> &'static std::sync::Mutex<std::collections::HashSet<PathBuf>> {
static F: std::sync::OnceLock<std::sync::Mutex<std::collections::HashSet<PathBuf>>> = std::sync::OnceLock::new();
F.get_or_init(Default::default)
}
/// Fetches the small preview of a message's picture or video in the background, as the message is recorded.
///
/// Previews are a few kilobytes and are what the list draws, so they are not left to be asked for one by one when the
/// window finds them missing (which asks Telegram for the message again each time). The message is in hand here, so
/// nothing more is asked. Whole files are still fetched only when looked at.
pub fn prefetch_preview(state: &AppState, account_id: &str, account: i64, remote: &str, message: &Message) {
let Some(media) = message.media() else { return };
let path = crate::media_cache::dir(MEDIA).join(media_name(account, remote, message.id(), Variant::Thumb, "jpg"));
if path.exists() || !matches!(media, Media::Photo(_) | Media::Document(_) | Media::Sticker(_) | Media::WebPage(_)) {
return;
}
if !fetching().lock().unwrap_or_else(|e| e.into_inner()).insert(path.clone()) {
return;
}
let Some(handle) = state.runtime.telegram_handle(account_id) else {
fetching().lock().unwrap_or_else(|e| e.into_inner()).remove(&path);
return;
};
tokio::spawn(async move {
let done = match &media {
Media::Photo(photo) => match pick(&photo.thumbs(), 320) {
Some(size) => download_to(&handle.client, size, &path, false).await,
None => download_to(&handle.client, photo, &path, false).await,
},
Media::Document(document) => download_thumb(&handle.client, document, &path).await,
Media::Sticker(sticker) => download_thumb(&handle.client, &sticker.document, &path).await,
Media::WebPage(_) => match page_photo(&media) {
Some(photo) => match pick(&photo.thumbs(), CARD) {
Some(size) => download_to(&handle.client, size, &path, false).await,
None => download_to(&handle.client, &photo, &path, false).await,
},
None => Ok(()),
},
_ => Ok(()),
};
if let Err(e) = done {
tracing::debug!("telegram: preview {}: {e:#}", path.display());
}
fetching().lock().unwrap_or_else(|e| e.into_inner()).remove(&path);
});
}
/// Fetches a cached Telegram file again, after it expired, was cleared, or was named before it was ever fetched.
pub async fn restore_cached(state: &AppState, path: &Path) -> Result<PathBuf> {
let name = path.file_name().and_then(|n| n.to_str()).context("not a cached file")?;
let parent = path.parent().and_then(|p| p.file_name()).and_then(|n| n.to_str()).unwrap_or("");
if parent == AVATARS {
let (account, peer, photo) = parse_avatar_name(name).context("not a cached Telegram picture")?;
let handle = state.runtime.telegram_handle(&format!("telegram:{account}")).context("that Telegram account is not connected")?;
fetch_avatar(&handle, &peer, photo, path).await?;
return Ok(path.to_path_buf());
}
// A GIF the picker listed is fetched from what the picker remembered of it.
if name.starts_with("tgif_") {
return super::gifs::restore(state, name, path).await?.context("not a cached Telegram GIF");
}
// So is a sticker's picture.
if name.starts_with("tstk_") {
return super::stickers::restore(state, name, path).await?.context("not a cached Telegram sticker");
}
let (account, remote, message, variant) = parse_media_name(name).context("not a cached Telegram file")?;
// A preview already on its way is waited for, not fetched a second time.
for _ in 0..120 {
if path.exists() {
return Ok(path.to_path_buf());
}
if !fetching().lock().unwrap_or_else(|e| e.into_inner()).contains(path) {
break;
}
tokio::time::sleep(std::time::Duration::from_millis(250)).await;
}
let handle = state.runtime.telegram_handle(&format!("telegram:{account}")).context("that Telegram account is not connected")?;
fetch(&handle, &remote, message, variant, path).await?;
Ok(path.to_path_buf())
}
/// The file an avatar is cached as, for a person or chat with a profile photo; `None` for one without.
pub fn avatar_url(account: i64, peer: &grammers_client::peer::Peer) -> Option<String> {
use grammers_client::peer::Peer;
let photo_id = match peer {
Peer::User(u) => u.photo().map(|p| p.photo_id),
Peer::Group(g) => g.photo().map(|p| p.photo_id),
Peer::Channel(c) => c.photo().map(|p| p.photo_id),
Peer::Community(_) => None,
}?;
Some(local(AVATARS, &avatar_name(account, &peers::remote_id(peer.id()), photo_id)))
}
async fn fetch_avatar(handle: &Handle, remote: &str, photo_id: i64, path: &Path) -> Result<()> {
let id: PeerId = peers::peer_id_of(remote).ok_or_else(|| anyhow!("not a Telegram conversation"))?;
let peer = handle.session.peer_ref(id).await.ok().flatten().ok_or_else(|| anyhow!("Telegram has not told this account about that picture's owner yet"))?;
let what = grammers_client::media::ChatPhoto { raw: tl::types::InputPeerPhotoFileLocation { big: false, peer: peer.into(), photo_id }.into() };
download_to(&handle.client, &what, path, false).await
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_cached_files_name_says_what_it_is_and_is_read_back() {
let name = media_name(1234567890, "-1001234567890", 21910, Variant::Full, "mp4");
assert_eq!(name, "tg_1234567890_n1001234567890_21910_full.mp4");
assert_eq!(parse_media_name(&name), Some((1234567890, "-1001234567890".to_string(), 21910, Variant::Full)));
let person = media_name(1, "6576171495", 7, Variant::Thumb, "jpg");
assert_eq!(parse_media_name(&person), Some((1, "6576171495".to_string(), 7, Variant::Thumb)));
assert_eq!(parse_media_name("not_ours.jpg"), None);
assert_eq!(parse_media_name("tg_1_n5_2_other.jpg"), None);
let avatar = avatar_name(1, "-100123", 99);
assert_eq!(parse_avatar_name(&avatar), Some((1, "-100123".to_string(), 99)));
assert_eq!(parse_avatar_name("tg_1_n5_2_full.jpg"), None, "a media file is not an avatar");
}
#[test]
fn a_files_extension_comes_from_its_name_else_its_type() {
assert_eq!(extension_of(Some("Report.PDF"), None), "pdf");
assert_eq!(extension_of(None, Some("video/mp4")), "mp4");
assert_eq!(extension_of(Some("noextension"), Some("audio/ogg")), "ogg");
assert_eq!(extension_of(Some("weird.name with spaces"), None), "bin");
assert_eq!(kind_of_mime("image/webp"), "image");
assert_eq!(kind_of_mime("image/gif"), "video");
assert_eq!(kind_of_mime("application/pdf"), "file");
}
#[test]
fn the_size_to_draw_is_the_smallest_that_reaches_the_target_else_the_largest() {
let sizes = vec![("s", 90), ("y", 1280), ("m", 320), ("x", 800)];
assert_eq!(choose(&sizes, 320), Some(&"m"));
assert_eq!(choose(&sizes, 1280), Some(&"y"));
assert_eq!(choose(&sizes, 5000), Some(&"y"));
assert_eq!(choose(&sizes, 100), Some(&"m"));
assert_eq!(choose::<&str>(&[], 320), None);
}
}
src/backend/telegram/members.rs
+59
View File
@@ -0,0 +1,59 @@
//! Who is in a group: the list beside the conversation.
//!
//! Telegram will list the members of a group or a supergroup (a channel's subscribers are for its administrators
//! only), two hundred at a time. The first page is read when somebody opens the conversation, with the group's own
//! count, and the owner and administrators carry a mark in front of their name.
use super::media;
use super::messages;
use super::send::{explain, peer_for};
use crate::model::Buffer;
use crate::state::AppState;
use grammers_client::peer::Role;
use serde_json::{json, Value};
/// How many members are read: Telegram's own page.
const LIMIT: usize = 200;
/// What stands in front of an owner's or an administrator's name.
const OWNER: &str = "👑";
const ADMIN: &str = "@";
/// Reads a group's members and says so, as the list and as how many there are in all.
pub async fn refresh(state: &AppState, buffer: &Buffer) -> Result<usize, String> {
let (handle, peer) = peer_for(state, buffer).await?;
// A person's conversation, a channel somebody only follows, or the account's own: nobody to list.
if buffer.kind == "dm" || buffer.category.as_deref() == Some(super::peers::CHANNELS) {
return Ok(0);
}
let mut iter = handle.client.iter_participants(peer);
let total = iter.total().await.unwrap_or(0);
let mut members: Vec<Value> = Vec::new();
while members.len() < LIMIT {
let Some(found) = iter.next().await.map_err(|e| explain(&e))? else { break };
let (prefix, banned) = match &found.role {
Role::Creator(_) => (OWNER, false),
Role::Admin(_) => (ADMIN, false),
Role::Banned(_) | Role::Left(_) => ("", true),
_ => ("", false),
};
if banned {
continue;
}
let peer = grammers_client::peer::Peer::User(found.user.clone());
handle.names.lock().unwrap_or_else(|e| e.into_inner()).insert(found.user.id().bare_id_unchecked(), messages::name_of(&peer));
members.push(json!({
"nick": messages::name_of(&peer),
"userId": found.user.id().bare_id_unchecked().to_string(),
"prefix": prefix,
"bot": found.user.is_bot(),
"avatarUrl": media::avatar_url(handle.own.user_id, &peer),
}));
}
let count = members.len();
let list = Value::Array(members);
state.runtime.set_presence(&buffer.id, list.clone());
state.runtime.set_presence_total(&buffer.id, total.max(count));
state.events.emit("presenceChange", json!({ "bufferId": buffer.id, "members": list, "total": total.max(count) }));
Ok(count)
}
src/backend/telegram/membership.rs
+134
View File
@@ -0,0 +1,134 @@
//! Joining and leaving: a public chat by its @name or t.me link, a private one by its invite link, and leaving what
//! the account is in.
//!
//! A chat that was joined arrives as a dialog, so the list is read again afterwards and the window opens the new
//! conversation as it does for any service.
use super::dialogs;
use super::peers;
use super::send::{explain, peer_for};
use crate::model::Buffer;
use crate::state::AppState;
use grammers_client::peer::Peer;
use grammers_client::tl;
use grammers_session::types::PeerKind;
/// What a person typed to name somewhere to join.
#[derive(Debug, PartialEq)]
pub enum Target {
/// A public name: a person, a group or a channel.
Username(String),
/// The secret part of a private invite link.
Invite(String),
}
/// Reads `@name`, `name`, `t.me/name`, `https://t.me/+hash`, `t.me/joinchat/hash` and `tg://join?invite=hash`.
pub fn parse_target(input: &str) -> Option<Target> {
let text = input.trim();
if let Some(rest) = text.strip_prefix("tg://join?invite=") {
return clean_invite(rest).map(Target::Invite);
}
let rest = text.strip_prefix("https://").or_else(|| text.strip_prefix("http://")).unwrap_or(text);
let path = ["t.me/", "telegram.me/", "telegram.dog/"].iter().find_map(|host| rest.strip_prefix(host));
let Some(path) = path else {
let name = text.strip_prefix('@').unwrap_or(text);
return valid_username(name).then(|| Target::Username(name.to_string()));
};
let path = path.split(['?', '#']).next().unwrap_or("");
let mut parts = path.split('/').filter(|p| !p.is_empty());
let first = parts.next()?;
if first == "joinchat" {
return clean_invite(parts.next()?).map(Target::Invite);
}
if let Some(hash) = first.strip_prefix('+') {
return clean_invite(hash).map(Target::Invite);
}
valid_username(first).then(|| Target::Username(first.to_string()))
}
fn clean_invite(hash: &str) -> Option<String> {
let hash = hash.trim();
(!hash.is_empty() && hash.chars().all(|c| c.is_ascii_alphanumeric() || c == '_' || c == '-')).then(|| hash.to_string())
}
/// Telegram's rule: 5 to 32 letters, digits and underscores, starting with a letter.
fn valid_username(name: &str) -> bool {
(5..=32).contains(&name.len()) && name.chars().next().is_some_and(|c| c.is_ascii_alphabetic()) && name.chars().all(|c| c.is_ascii_alphanumeric() || c == '_')
}
/// Joins what was named, and makes its conversation.
pub async fn join(state: &AppState, account_id: &str, input: &str) -> Result<(), String> {
let target = parse_target(input).ok_or_else(|| format!("\"{}\" is not a Telegram @name or t.me link", input.trim()))?;
let handle = state.runtime.telegram_handle(account_id).ok_or_else(|| "that Telegram account is not connected".to_string())?;
match target {
Target::Invite(hash) => {
handle.client.invoke(&tl::functions::messages::ImportChatInvite { hash }).await.map_err(|e| match &e {
grammers_client::InvocationError::Rpc(r) if r.name == "INVITE_HASH_EXPIRED" || r.name == "INVITE_HASH_INVALID" => "That invite link is no longer valid".to_string(),
grammers_client::InvocationError::Rpc(r) if r.name == "INVITE_REQUEST_SENT" => "Your request to join was sent: an administrator has to accept it".to_string(),
_ => explain(&e),
})?;
}
Target::Username(name) => {
let peer = handle.client.resolve_username(&name).await.map_err(|e| explain(&e))?.ok_or_else(|| format!("Nobody on Telegram is called @{name}"))?;
match &peer {
// A person: the conversation with them is all there is to open.
Peer::User(_) => {
let buffer = peers::buffer_for(state, account_id, &peer);
if let Some(avatar) = super::media::avatar_url(handle.own.user_id, &peer) {
state.runtime.set_buffer_avatar(state, &buffer.id, &avatar);
}
return Ok(());
}
_ => {
let peer_ref = peer.to_ref().await.ok().flatten().ok_or_else(|| "Telegram did not say how to reach that chat".to_string())?;
handle.client.join_chat(peer_ref).await.map_err(|e| match &e {
grammers_client::InvocationError::Rpc(r) if r.name == "INVITE_REQUEST_SENT" => "Your request to join was sent: an administrator has to accept it".to_string(),
_ => explain(&e),
})?;
}
}
}
}
dialogs::sync(state, account_id, &handle).await.map_err(|e| e.to_string())?;
Ok(())
}
/// Leaves a conversation: a channel or supergroup is left, a small group is left, a private chat is cleared.
pub async fn leave(state: &AppState, buffer: &Buffer) -> Result<(), String> {
let (handle, peer) = peer_for(state, buffer).await?;
match peer.id.kind() {
PeerKind::Channel => {
handle.client.invoke(&tl::functions::channels::LeaveChannel { channel: peer.into() }).await.map_err(|e| explain(&e))?;
}
PeerKind::Chat => {
handle.client.invoke(&tl::functions::messages::DeleteChatUser { revoke_history: false, chat_id: peer.into(), user_id: tl::enums::InputUser::UserSelf }).await.map_err(|e| explain(&e))?;
}
_ => {
// History is deleted from this account's side only; the other person keeps theirs.
handle.client.invoke(&tl::functions::messages::DeleteHistory { just_clear: false, revoke: false, peer: peer.into(), max_id: 0, min_date: None, max_date: None }).await.map_err(|e| explain(&e))?;
}
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn names_and_links_are_read() {
let user = |n: &str| Some(Target::Username(n.to_string()));
let invite = |h: &str| Some(Target::Invite(h.to_string()));
assert_eq!(parse_target("@abcd"), None, "too short for Telegram, so not a name");
assert_eq!(parse_target("@disclosetv"), user("disclosetv"));
assert_eq!(parse_target("disclosetv"), user("disclosetv"));
assert_eq!(parse_target("https://t.me/disclosetv"), user("disclosetv"));
assert_eq!(parse_target("t.me/disclosetv/1234?single"), user("disclosetv"));
assert_eq!(parse_target("https://t.me/+AbC_d-123"), invite("AbC_d-123"));
assert_eq!(parse_target("https://t.me/joinchat/AbC_d-123"), invite("AbC_d-123"));
assert_eq!(parse_target("tg://join?invite=AbC_d-123"), invite("AbC_d-123"));
assert_eq!(parse_target("two words"), None);
assert_eq!(parse_target("1startsdigit"), None);
assert_eq!(parse_target(""), None);
}
}
src/backend/telegram/messages.rs
+487
View File
@@ -0,0 +1,487 @@
//! A Telegram message as moho's: who said it, what it says, what it answers, what came with it.
//!
//! Telegram marks up a message with entities - a kind, an offset and a length, counted in UTF-16 code units - which
//! become the message's `html` (bold, italic, underline, strike, code, a quote, a link, a spoiler) beside its plain
//! `body`. Media is not drawn until Round 4; a message that carries some says what it is in words, after its caption.
use super::media;
use super::peers;
use crate::model::ReplyPreview;
use crate::state::AppState;
use grammers_client::message::Message;
use grammers_client::peer::Peer;
use grammers_client::tl;
/// What the connection knows about the account itself: its name here, which a message it sent is attributed to.
#[derive(Clone)]
pub struct Own {
pub name: String,
pub user_id: i64,
}
fn escape(text: &str) -> String {
text.replace('&', "&amp;").replace('<', "&lt;").replace('>', "&gt;").replace('"', "&quot;")
}
/// A span to mark: where it starts and ends in UTF-16 units, and the tags that open and close it.
struct Span {
from: usize,
to: usize,
open: String,
close: &'static str,
}
fn span_of(entity: &tl::enums::MessageEntity, text16: &[u16]) -> Option<Span> {
use tl::enums::MessageEntity as E;
let (offset, length, open, close): (i32, i32, String, &'static str) = match entity {
E::Bold(e) => (e.offset, e.length, "<b>".into(), "</b>"),
E::Italic(e) => (e.offset, e.length, "<i>".into(), "</i>"),
E::Underline(e) => (e.offset, e.length, "<u>".into(), "</u>"),
E::Strike(e) => (e.offset, e.length, "<s>".into(), "</s>"),
E::Code(e) => (e.offset, e.length, "<code>".into(), "</code>"),
E::Pre(e) => (e.offset, e.length, "<pre>".into(), "</pre>"),
E::Blockquote(e) => (e.offset, e.length, "<blockquote>".into(), "</blockquote>"),
E::Spoiler(e) => (e.offset, e.length, "<span class=\"spoiler\">".into(), "</span>"),
E::TextUrl(e) => (e.offset, e.length, format!("<a href=\"{}\">", escape(&e.url)), "</a>"),
E::Url(e) => {
// The link is its own text.
let (from, to) = (e.offset.max(0) as usize, ((e.offset + e.length).max(0) as usize).min(text16.len()));
let target = String::from_utf16_lossy(text16.get(from..to)?);
let target = if target.contains("://") { target } else { format!("https://{target}") };
(e.offset, e.length, format!("<a href=\"{}\">", escape(&target)), "</a>")
}
_ => return None,
};
let from = offset.max(0) as usize;
let to = ((offset + length).max(0) as usize).min(text16.len());
(from < to).then_some(Span { from, to, open, close })
}
/// The message's text with its entities as html, or `None` when it has none worth drawing.
pub fn entities_html(text: &str, entities: &[tl::enums::MessageEntity]) -> Option<String> {
let text16: Vec<u16> = text.encode_utf16().collect();
let mut spans: Vec<Span> = entities.iter().filter_map(|e| span_of(e, &text16)).collect();
if spans.is_empty() {
return None;
}
// Outer spans open first: by where they start, the longer before the shorter.
spans.sort_by(|a, b| a.from.cmp(&b.from).then(b.to.cmp(&a.to)));
let mut out = String::new();
let mut open: Vec<&Span> = Vec::new();
let mut next = 0;
let mut position = 0;
let mut run = String::new();
let flush = |run: &mut String, out: &mut String| {
if !run.is_empty() {
out.push_str(&escape(run).replace('\n', "<br>"));
run.clear();
}
};
let mut units = text16.iter().copied().peekable();
while position <= text16.len() {
// What ends here closes first, innermost first.
while let Some(last) = open.last() {
if last.to <= position {
flush(&mut run, &mut out);
out.push_str(last.close);
open.pop();
} else {
break;
}
}
while next < spans.len() && spans[next].from <= position {
flush(&mut run, &mut out);
out.push_str(&spans[next].open);
open.push(&spans[next]);
next += 1;
}
let Some(unit) = units.next() else { break };
// A character outside the basic plane is two units: take both together.
let mut pair = vec![unit];
if (0xD800..0xDC00).contains(&unit) {
if let Some(low) = units.next() {
pair.push(low);
position += 1;
}
}
run.push_str(&String::from_utf16_lossy(&pair));
position += 1;
}
flush(&mut run, &mut out);
while let Some(last) = open.pop() {
out.push_str(last.close);
}
Some(out)
}
/// What a message with media says it carries, in words.
pub fn media_label(media: &tl::enums::MessageMedia) -> Option<String> {
use tl::enums::MessageMedia as M;
Some(match media {
M::Empty | M::Unsupported => return None,
// A link's card is the link already in the text.
M::WebPage(_) => return None,
M::Photo(_) => "[Photo]".to_string(),
M::Geo(_) | M::GeoLive(_) => "[Location]".to_string(),
M::Venue(v) => format!("[Place: {}]", v.title),
M::Contact(c) => format!("[Contact: {} {}]", c.first_name, c.last_name).trim_end().replace(" ]", "]"),
M::Poll(p) => format!("[Poll: {}]", poll_question(&p.poll)),
M::Dice(d) => format!("[{} {}]", d.emoticon, d.value),
M::Document(d) => match &d.document {
Some(tl::enums::Document::Document(doc)) => document_label(doc),
_ => "[File]".to_string(),
},
_ => "[Attachment]".to_string(),
})
}
fn poll_question(poll: &tl::enums::Poll) -> String {
match poll {
tl::enums::Poll::Poll(p) => match &p.question {
tl::enums::TextWithEntities::Entities(t) => t.text.clone(),
},
}
}
fn document_label(doc: &tl::types::Document) -> String {
use tl::enums::DocumentAttribute as A;
let name = doc.attributes.iter().find_map(|a| if let A::Filename(f) = a { Some(f.file_name.clone()) } else { None });
for attribute in &doc.attributes {
match attribute {
A::Sticker(s) => return format!("[Sticker {}]", s.alt).replace("[Sticker ]", "[Sticker]"),
A::Audio(a) if a.voice => return "[Voice message]".to_string(),
A::Audio(a) => return format!("[Audio: {}]", a.title.clone().or(name.clone()).unwrap_or_else(|| "track".to_string())),
A::Video(v) if v.round_message => return "[Video message]".to_string(),
A::Video(_) => return format!("[Video{}]", name.map(|n| format!(": {n}")).unwrap_or_default()),
A::Animated => return "[GIF]".to_string(),
_ => {}
}
}
format!("[File{}]", name.map(|n| format!(": {n}")).unwrap_or_default())
}
/// What a service message says (somebody joined, the title changed), or `None` for those with nothing to say.
pub fn service_text(who: &str, action: &tl::enums::MessageAction) -> Option<String> {
use tl::enums::MessageAction as A;
Some(match action {
A::ChatCreate(a) => format!("{who} created the group \"{}\"", a.title),
A::ChatEditTitle(a) => format!("{who} changed the title to \"{}\"", a.title),
A::ChatEditPhoto(_) => format!("{who} changed the group photo"),
A::ChatDeletePhoto => format!("{who} removed the group photo"),
A::ChatAddUser(a) => format!("{who} added {} {}", a.users.len(), if a.users.len() == 1 { "person" } else { "people" }),
A::ChatDeleteUser(_) => format!("{who} removed somebody from the group"),
A::ChatJoinedByLink(_) => format!("{who} joined by invite link"),
A::ChatJoinedByRequest => format!("{who} joined after being approved"),
A::ChannelCreate(a) => format!("The channel \"{}\" was created", a.title),
A::PinMessage => format!("{who} pinned a message"),
A::HistoryClear => "The history was cleared".to_string(),
A::ScreenshotTaken => format!("{who} took a screenshot"),
A::ContactSignUp => format!("{who} joined Telegram"),
A::SetMessagesTtl(a) => if a.period == 0 { "Messages no longer disappear".to_string() } else { format!("Messages now disappear after {} seconds", a.period) },
A::PhoneCall(a) => format!("{who} {} call", if a.video { "had a video" } else { "had a voice" }),
A::TopicCreate(a) => format!("{who} created the topic \"{}\"", a.title),
A::Empty => return None,
_ => format!("{who} did something Telegram describes in a way moho does not show yet"),
})
}
/// A person's name, for a line that says who did something.
pub fn name_of(peer: &Peer) -> String {
peers::title_of(peer)
}
/// Everything about a message that is words: the caption or text and what came with it. Where it was forwarded from is
/// said above it (see `forward_preview`), not in it.
pub fn body_of(message: &Message) -> String {
let mut parts: Vec<String> = Vec::new();
let text = message.text();
if !text.is_empty() {
parts.push(text.to_string());
}
if let tl::enums::Message::Message(m) = &message.raw {
// A picture, a video, a file, a voice message or a sticker is drawn from its attachment: saying it again in
// words (with the file's name) is noise. Everything else with no picture of its own keeps its label.
let drawn = matches!(m.media, Some(tl::enums::MessageMedia::Photo(_)) | Some(tl::enums::MessageMedia::Document(_)));
if let Some(label) = m.media.as_ref().filter(|_| !drawn).and_then(media_label) {
parts.push(label);
}
}
parts.join("\n")
}
/// Records a message in its conversation, as it arrived live or as history. `false` when it was already there.
pub fn record(state: &AppState, account_id: &str, own: &Own, message: &Message) -> bool {
record_named(state, account_id, own, message, None)
}
/// The same, remembering who the sender is for `names` when it is given.
pub fn record_named(state: &AppState, account_id: &str, own: &Own, message: &Message, names: Option<&std::sync::Mutex<std::collections::HashMap<i64, String>>>) -> bool {
let Some(peer) = message.peer() else { return false };
record_in(state, account_id, own, message, peer, message.sender(), names)
}
/// The same for a message whose conversation the caller found. A message a person sent in a private chat arrives as
/// a short update that names no one, so the library cannot say who it is from or where it is: the caller asks
/// Telegram, and the other person in a private chat is the sender of everything not sent by the account.
pub fn record_in(state: &AppState, account_id: &str, own: &Own, message: &Message, peer: &Peer, sender: Option<&Peer>, names: Option<&std::sync::Mutex<std::collections::HashMap<i64, String>>>) -> bool {
let buffer = super::topics::buffer_of_message(state, account_id, peer, &message.raw);
let outgoing = message.outgoing();
let sender = sender.or_else(|| (!outgoing && matches!(peer, Peer::User(_))).then_some(peer));
if let (Some(names), Some(Peer::User(user))) = (names, sender) {
names.lock().unwrap_or_else(|e| e.into_inner()).insert(user.id().bare_id_unchecked(), name_of(&Peer::User(user.clone())));
}
let from = if outgoing {
own.name.clone()
} else {
sender.map(name_of).unwrap_or_else(|| peers::title_of(peer))
};
let sender_id = if outgoing { Some(own.user_id.to_string()) } else { message.sender_id().map(peers::remote_id) };
let ts = message.date().as_second();
let id = message.id().to_string();
if let Some(action) = message.action() {
let Some(text) = service_text(&from, action) else { return false };
return state.runtime.record_message_at(state, account_id, &buffer.name, &buffer.kind, "Telegram", &text, false, "system", None, Some(id), false, None, Vec::new(), Vec::new(), None, Some(ts), None, None);
}
let topic_only = matches!(&message.raw, tl::enums::Message::Message(m) if m.reply_to.as_ref().is_some_and(is_topic_root));
let answered = if topic_only { None } else { reply_preview(state, account_id, &buffer, message) };
let forward = message.forward_header().map(|h| forward_preview(state, account_id, &h));
// One slot above a message: an answer wins, and says it was forwarded in its text.
let (reply, forward_line) = match (answered, forward) {
(Some(answer), Some(forwarded)) => (Some(answer), Some(format!("↪ Forwarded from {}", forwarded.from))),
(Some(answer), None) => (Some(answer), None),
(None, forwarded) => (forwarded, None),
};
let ask_for_reply = reply.as_ref().filter(|r| r.body.is_empty() && !r.forwarded).map(|r| (r.id.clone(), r.chat.clone()));
let has_forward_line = forward_line.is_some();
let body = match forward_line {
Some(line) => format!("{line}\n{}", body_of(message)),
None => body_of(message),
};
let html = message.fmt_entities().and_then(|e| entities_html(message.text(), e)).filter(|h| !h.is_empty());
// The first line of a forward is words of our own, so the html (made from the text alone) would lose it: a
// forwarded message is shown as plain text.
let html = if has_forward_line { None } else { html };
let attachments = buffer.remote_id.as_deref().map(|remote| media::attachments_of(own.user_id, remote, message)).unwrap_or_default();
// A post in a channel has no sender but the channel, whose picture it is.
let avatar = if outgoing { None } else { sender.or(Some(peer)).and_then(|p| media::avatar_url(own.user_id, p)) };
if let Some(remote) = buffer.remote_id.as_deref() {
media::prefetch_preview(state, account_id, own.user_id, remote, message);
}
let again = attachments.clone();
let reply_target = ask_for_reply.map(|(reply_id, chat)| {
// Where the answered message is: the other conversation if it is one, else this.
let remote = chat.and_then(|c| state.runtime.get_buffer(&c)).and_then(|b| b.remote_id).or_else(|| buffer.remote_id.clone());
(remote, reply_id)
});
let cards = buffer.remote_id.as_deref().map(|remote| media::embeds_of(own.user_id, remote, message)).unwrap_or_default();
let cards_again = cards.clone();
let recorded = state.runtime.record_message_at(state, account_id, &buffer.name, &buffer.kind, &from, &body, false, "chat", reply, Some(id.clone()), message.mentioned(), avatar, cards, attachments, sender_id, Some(ts), html, None);
if !recorded && !cards_again.is_empty() {
if let Ok(Some(stored)) = state.store.get_message(&buffer.id, &id) {
if stored.embeds != cards_again && state.store.set_message_embeds(&buffer.id, &id, &cards_again).unwrap_or(false) {
state.events.emit("messageUpdated", serde_json::json!({ "bufferId": buffer.id, "id": id, "body": stored.body, "edited": false, "embeds": cards_again, "attachments": stored.attachments }));
}
}
}
// Read again, a message kept with a label for its picture in its text (earlier builds wrote "[Photo]") loses it.
if !recorded {
if let Ok(Some(stored)) = state.store.get_message(&buffer.id, &id) {
let clean = body.trim_end();
if stored.body != body && stored.body.starts_with(clean) && stored.body[clean.len()..].trim_start().starts_with('[') && !again.is_empty() {
state.runtime.update_message_body_only(state, &buffer.id, &id, &body);
}
}
}
// Read again, a message already kept without its picture or file gets it: earlier builds kept the text alone.
if !recorded && !again.is_empty() {
if let Ok(Some(stored)) = state.store.get_message(&buffer.id, &id) {
if stored.attachments.is_empty() && state.store.update_message_attachments(&buffer.id, &id, &again).unwrap_or(false) {
state.events.emit("messageUpdated", serde_json::json!({ "bufferId": buffer.id, "id": id, "body": stored.body, "edited": false, "embeds": stored.embeds, "attachments": again }));
}
}
}
// The buttons a bot put under it.
super::keyboards::note(state, &buffer.id, &id, &message.raw, false);
// What a reply answered, when it was not in hand: asked of Telegram, and the row is filled in when it comes.
if let Some((Some(remote), reply_id)) = reply_target {
fill_reply_later(state, account_id, &buffer.id, &id, &remote, &reply_id);
}
// A poll that is still open is the conversation's card.
if let tl::enums::Message::Message(raw) = &message.raw {
if let Some(tl::enums::MessageMedia::Poll(p)) = &raw.media {
let (tl::enums::Poll::Poll(poll), tl::enums::PollResults::Results(results)) = (&p.poll, &p.results);
super::polls::announce(state, &buffer.id, message.id(), poll, results);
}
}
// What people have reacted with, which a message arrives carrying.
if let tl::enums::Message::Message(m) = &message.raw {
// How many have answered it, or commented under it if it is a channel's post.
if let Some(tl::enums::MessageReplies::Replies(r)) = &m.replies {
if r.replies > 0 {
state.events.emit("threadCount", serde_json::json!({ "bufferId": buffer.id, "id": id, "count": r.replies }));
}
}
if let Some(reactions) = &m.reactions {
let list = super::reactions::from_tl(reactions);
if !list.is_empty() {
super::reactions::apply(state, &buffer.id, &id, &list);
}
}
}
recorded
}
/// Where a forwarded message came from, as the line above it: the name of the source and, when it was a post in a
/// channel this account has open, where to find it.
fn forward_preview(state: &AppState, account_id: &str, header: &tl::enums::MessageFwdHeader) -> ReplyPreview {
let tl::enums::MessageFwdHeader::Header(h) = header;
let source = h.from_id.clone().map(|p| peers::remote_id(grammers_session::types::PeerId::from(p)));
let origin = source.as_ref().and_then(|remote| state.runtime.list_buffers().into_iter().find(|b| b.account_id == account_id && b.remote_id.as_deref() == Some(remote.as_str())));
let from = origin.as_ref().map(|b| b.name.clone()).or_else(|| h.from_name.clone()).or_else(|| h.post_author.clone()).unwrap_or_else(|| "somebody".to_string());
// Only a post in a channel can be gone to: a person's forward has no message of theirs to point at.
let (chat, id) = match (&origin, h.channel_post) {
(Some(b), Some(post)) if b.kind == "channel" => (Some(b.id.clone()), post.to_string()),
_ => (None, String::new()),
};
ReplyPreview { id, from, body: String::new(), thread: false, forwarded: true, thread_root: None, chat }
}
/// What a message answers, as the quote above it: the words quoted (or the message answered), who said them, and - when
/// it was said in another conversation - where, so the reader can go there.
fn reply_preview(state: &AppState, account_id: &str, buffer: &crate::model::Buffer, message: &Message) -> Option<ReplyPreview> {
let reply_id = message.reply_to_message_id()?;
let header = match &message.raw {
tl::enums::Message::Message(m) => match &m.reply_to {
Some(tl::enums::MessageReplyHeader::Header(h)) => Some(h.clone()),
_ => None,
},
_ => None,
};
let elsewhere = header
.as_ref()
.and_then(|h| h.reply_to_peer_id.clone())
.map(|p| peers::remote_id(grammers_session::types::PeerId::from(p)))
.filter(|remote| buffer.remote_id.as_deref() != Some(remote.as_str()));
let other = elsewhere.as_ref().and_then(|remote| state.runtime.list_buffers().into_iter().find(|b| b.account_id == account_id && b.remote_id.as_deref() == Some(remote.as_str())));
let source = other.as_ref().map(|b| b.id.clone()).unwrap_or_else(|| buffer.id.clone());
// A message in a conversation this account has not got open is not one to look for here.
let stored = if elsewhere.is_some() && other.is_none() { None } else { state.store.get_message(&source, &reply_id.to_string()).ok().flatten() };
let quote = header.as_ref().and_then(|h| h.quote_text.clone()).filter(|t| !t.trim().is_empty());
let named = header.as_ref().and_then(|h| h.reply_from.as_ref()).and_then(|f| {
let tl::enums::MessageFwdHeader::Header(f) = f;
f.from_name.clone().or_else(|| f.post_author.clone())
});
let shown_media = header.as_ref().and_then(|h| h.reply_media.as_ref()).and_then(media_label);
let from = other.as_ref().map(|b| b.name.clone()).or(named).or_else(|| stored.as_ref().map(|m| m.from.clone())).unwrap_or_default();
let body = quote.or_else(|| stored.map(|m| if m.body.trim().is_empty() { attachment_words(&m.attachments) } else { m.body })).filter(|b| !b.is_empty()).or(shown_media).unwrap_or_default();
Some(ReplyPreview { id: reply_id.to_string(), from, body, thread: false, forwarded: false, thread_root: None, chat: other.map(|b| b.id) })
}
/// What a message with only a picture or a file says in a quote, from what it carries.
fn attachment_words(attachments: &[crate::model::Attachment]) -> String {
let Some(first) = attachments.first() else { return String::new() };
match (first.kind.as_str(), first.voice) {
(_, true) => "Voice message".to_string(),
("image", _) => "Photo".to_string(),
("video", _) => "Video".to_string(),
("audio", _) => "Audio".to_string(),
_ => first.filename.clone().unwrap_or_else(|| "File".to_string()),
}
}
/// Asks Telegram what a reply answered and fills the quote in.
fn fill_reply_later(state: &AppState, account_id: &str, buffer_id: &str, msg_id: &str, remote: &str, reply_id: &str) {
let (Some(handle), Some(id), Ok(reply)) = (state.runtime.telegram_handle(account_id), peers::peer_id_of(remote), reply_id.parse::<i32>()) else { return };
let (state, buffer_id, msg_id) = (state.clone(), buffer_id.to_string(), msg_id.to_string());
tokio::spawn(async move {
use grammers_session::Session;
let Some(peer) = handle.session.peer_ref(id).await.ok().flatten() else { return };
let Ok(found) = handle.client.get_messages_by_id(peer, &[reply]).await else { return };
let Some(Some(original)) = found.into_iter().next() else { return };
let body = body_of(&original);
let from = original.sender().map(name_of).unwrap_or_else(|| original.peer().map(peers::title_of).unwrap_or_default());
if !body.is_empty() {
state.runtime.fill_reply_preview(&state, &buffer_id, &msg_id, &from, &body);
}
});
}
/// In a group with topics every message "replies" to its topic's first message; that is where it is, not who it
/// answers. A real reply names a message and the topic it is in as well.
fn is_topic_root(header: &tl::enums::MessageReplyHeader) -> bool {
matches!(header, tl::enums::MessageReplyHeader::Header(h) if h.forum_topic && h.reply_to_top_id.is_none())
}
#[cfg(test)]
mod tests {
use super::*;
use tl::types as t;
fn bold(offset: i32, length: i32) -> tl::enums::MessageEntity {
tl::enums::MessageEntity::Bold(t::MessageEntityBold { offset, length })
}
fn italic(offset: i32, length: i32) -> tl::enums::MessageEntity {
tl::enums::MessageEntity::Italic(t::MessageEntityItalic { offset, length })
}
fn header(forum_topic: bool, msg: i32, top: Option<i32>) -> tl::enums::MessageReplyHeader {
tl::enums::MessageReplyHeader::Header(t::MessageReplyHeader {
reply_to_scheduled: false,
forum_topic,
quote: false,
reply_to_ephemeral: false,
reply_to_msg_id: Some(msg),
reply_to_peer_id: None,
reply_from: None,
reply_media: None,
reply_to_top_id: top,
quote_text: None,
quote_entities: None,
quote_offset: None,
todo_item_id: None,
poll_option: None,
})
}
#[test]
fn a_message_in_a_topic_is_not_a_reply_to_the_topic() {
assert!(is_topic_root(&header(true, 10, None)));
assert!(!is_topic_root(&header(true, 55, Some(10))), "answering a message inside the topic");
assert!(!is_topic_root(&header(false, 10, None)), "an ordinary reply");
}
#[test]
fn entities_become_html_and_nest() {
assert_eq!(entities_html("hello world", &[bold(0, 5)]).unwrap(), "<b>hello</b> world");
assert_eq!(entities_html("a bc d", &[bold(2, 2), italic(2, 1)]).unwrap(), "a <b><i>b</i>c</b> d");
assert_eq!(entities_html("plain", &[]), None);
// The text is escaped, and a line break is one.
assert_eq!(entities_html("a < b\nc", &[bold(0, 1)]).unwrap(), "<b>a</b> &lt; b<br>c");
}
#[test]
fn offsets_are_in_utf16_units_so_an_emoji_before_the_text_moves_it_by_two() {
// "😀 hi": the emoji is two units, the space one, so "hi" starts at 3.
assert_eq!(entities_html("😀 hi", &[bold(3, 2)]).unwrap(), "😀 <b>hi</b>");
// A span that runs past the end is cut at it rather than panicking.
assert_eq!(entities_html("ab", &[bold(1, 50)]).unwrap(), "a<b>b</b>");
}
#[test]
fn a_link_is_its_own_text_or_its_url_and_a_bare_domain_gets_a_scheme() {
let url = tl::enums::MessageEntity::Url(t::MessageEntityUrl { offset: 4, length: 11 });
assert_eq!(entities_html("see example.com now", &[url]).unwrap(), "see <a href=\"https://example.com\">example.com</a> now");
let text_url = tl::enums::MessageEntity::TextUrl(t::MessageEntityTextUrl { offset: 0, length: 4, url: "https://a.example/?q=1&r=2".into() });
assert_eq!(entities_html("here", &[text_url]).unwrap(), "<a href=\"https://a.example/?q=1&amp;r=2\">here</a>");
}
#[test]
fn service_messages_say_who_did_what() {
let title = tl::enums::MessageAction::ChatEditTitle(t::MessageActionChatEditTitle { title: "Lunch".into() });
assert_eq!(service_text("Ann", &title).as_deref(), Some("Ann changed the title to \"Lunch\""));
assert_eq!(service_text("Ann", &tl::enums::MessageAction::PinMessage).as_deref(), Some("Ann pinned a message"));
assert_eq!(service_text("Ann", &tl::enums::MessageAction::Empty), None);
}
}
src/backend/telegram/mod.rs
+129
View File
@@ -0,0 +1,129 @@
//! Telegram, over MTProto, in pure Rust (the `grammers` crates).
//!
//! # How this folder is arranged
//!
//! Built in rounds, as Twitch was (see Gitea issue #26 for the checklist and what has been verified):
//!
//! - this file - the identity moho gives Telegram, and what every other file shares
//! - `session` - what the library remembers between runs, in a private file of its own per account
//! - `link` - one connection to Telegram: the sender pool, the client and the stream of updates
//! - `links` - what a t.me or tg:// link points at, and opening it
//! - `login` - adding an account: phone, code, cloud password
//! - `connect` - an account's connection, kept up and reconnected
//! - `dialogs` - the account's conversations, read when it connects
//! - `history` - a conversation's history, read when it is opened and scrolled back
//! - `markup` - what the message box holds, as plain text and entities
//! - `updates` - what Telegram sends that is not a message: typing, reads on other apps
//! - `send` - saying things: send, answer, edit, delete, typing, read
//! - `media` - pictures, video, files and voice: what a message carries, fetched when it is looked at
//! - `reactions` - the emoji on a message, and the account's own
//! - `pins` - a conversation's pinned messages
//! - `members` - who is in a group
//! - `profile` - the card for a person
//! - `search` - searching on Telegram's side
//! - `polls` - a poll as the conversation's card, voted in from it
//! - `threads` - replies in a thread, and a channel post's comments
//! - `membership` - joining by @name or link, leaving
//! - `admin` - what an administrator may do to a member
//! - `notify` - muting a chat on the account
//! - `instant` - an Instant View article as a page of its own
//! - `keyboards` - a bot's inline buttons
//! - `bots` - a bot's command menu
//! - `topics` - a group with Topics as several conversations
//! - `folders` - the account's chat folders, kept in step with moho's categories
//! - `gifs` - saved GIFs and the GIF search
//! - `stickers` - the account's sticker packs, and sending one
//! - `peers` - a conversation as a buffer: its name, its kind, its heading in the list
//! - `messages` - a message as moho's, with its entities as html
//!
//! # The api_id
//!
//! Telegram gives one `api_id` and `api_hash` to a phone number's owner, and a client sends them with every
//! connection. Moho does not embed one: each account carries its own, typed into the sign-in form, and they are
//! kept with the session like a password. Nothing here logs them.
//!
//! # What moho tells Telegram about itself
//!
//! Telegram lists every session of an account (Settings, Devices) with the app and device each one reported, and
//! treats a stream of unfamiliar sessions as a risk. So the identity is set once, is the same on every connection,
//! and says what this is: the device is "moho" on this operating system, the version is moho's own. A session is
//! saved and reused; nothing here signs in afresh to get over a hiccup.
pub mod admin;
pub mod bots;
pub mod connect;
pub mod dialogs;
pub mod folders;
pub mod gifs;
pub mod history;
pub mod instant;
pub mod keyboards;
pub mod link;
pub mod links;
pub mod login;
pub mod markup;
pub mod media;
pub mod members;
pub mod membership;
pub mod messages;
pub mod notify;
pub mod peers;
pub mod pins;
pub mod polls;
pub mod profile;
pub mod reactions;
pub mod search;
pub mod send;
pub mod stickers;
pub mod threads;
pub mod topics;
pub mod updates;
pub mod session;
use grammers_mtsender::ConnectionParams;
use std::sync::Arc;
/// What the rest of the daemon needs to act on a connected account: its client, its session (to turn a conversation
/// into something Telegram will accept, an id with the authority to use it) and its own name.
#[derive(Clone)]
pub struct Handle {
pub client: grammers_client::Client,
pub session: Arc<session::FileSession>,
pub own: messages::Own,
/// What people are called, by Telegram's number for them, so a typing notice can say who. Filled as people are
/// met in the conversation list and in messages.
pub names: Arc<std::sync::Mutex<std::collections::HashMap<i64, String>>>,
}
/// The name Telegram shows for this app's sessions: `moho` and the kind of machine it runs on.
pub fn device_model() -> String {
format!("moho ({} {})", std::env::consts::OS, std::env::consts::ARCH)
}
/// How every connection of an account introduces itself.
///
/// `proxy` is the SOCKS5 address of the account's route (Tor or the proxy in the network settings), if it has one.
pub fn connection_params(proxy: Option<&str>) -> ConnectionParams {
ConnectionParams {
device_model: device_model(),
// Moho's own version, which is the daemon's.
app_version: env!("CARGO_PKG_VERSION").to_string(),
proxy_url: proxy.map(str::to_string),
..ConnectionParams::default()
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn the_identity_says_what_this_is_and_is_the_same_each_time() {
let a = connection_params(None);
let b = connection_params(None);
assert!(a.device_model.starts_with("moho ("), "{}", a.device_model);
assert_eq!((a.device_model.as_str(), a.app_version.as_str()), (b.device_model.as_str(), b.app_version.as_str()));
assert_eq!(a.app_version, env!("CARGO_PKG_VERSION"));
assert!(!a.system_version.is_empty() && !a.lang_code.is_empty());
}
}
src/backend/telegram/notify.rs
+77
View File
@@ -0,0 +1,77 @@
//! Muting a conversation on the account itself.
//!
//! Telegram keeps "mute until" per chat with the account, so a chat silenced on the phone is silent here and the
//! other way round. It is read off each dialog and from the update Telegram sends when it changes, and written when
//! the window asks.
use super::peers;
use super::send::{explain, peer_for};
use crate::model::Buffer;
use crate::state::AppState;
use grammers_client::tl;
use grammers_session::types::PeerId;
/// "Forever", as Telegram's own apps write it.
const FOREVER: i32 = i32::MAX;
/// Whether notification settings say the chat is silent at `now`.
pub fn silent(settings: &tl::enums::PeerNotifySettings, now: i64) -> bool {
let tl::enums::PeerNotifySettings::Settings(s) = settings;
s.mute_until.is_some_and(|until| i64::from(until) > now)
}
/// A chat's settings changed (on another of the account's apps, or here): the buffer follows.
pub fn changed(state: &AppState, account_id: &str, update: &tl::types::UpdateNotifySettings) {
let tl::enums::NotifyPeer::Peer(p) = &update.peer else { return };
let remote = peers::remote_id(PeerId::from(p.peer.clone()));
let Some(buffer) = state.runtime.list_buffers().into_iter().find(|b| b.account_id == account_id && b.remote_id.as_deref() == Some(remote.as_str())) else { return };
state.runtime.set_silenced(state, &buffer.id, silent(&update.notify_settings, chrono::Utc::now().timestamp()));
}
/// Mutes a chat, or takes the mute off.
pub async fn set_muted(state: &AppState, buffer: &Buffer, muted: bool) -> Result<(), String> {
let (handle, peer) = peer_for(state, buffer).await?;
let settings = tl::types::InputPeerNotifySettings {
show_previews: None,
silent: None,
mute_until: Some(if muted { FOREVER } else { 0 }),
sound: None,
stories_muted: None,
stories_hide_sender: None,
stories_sound: None,
};
let peer = tl::enums::InputNotifyPeer::Peer(tl::types::InputNotifyPeer { peer: peer.into() });
handle.client.invoke(&tl::functions::account::UpdateNotifySettings { peer, settings: settings.into() }).await.map_err(|e| explain(&e))?;
state.runtime.set_silenced(state, &buffer.id, muted);
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
fn settings(mute_until: Option<i32>) -> tl::enums::PeerNotifySettings {
tl::enums::PeerNotifySettings::Settings(tl::types::PeerNotifySettings {
show_previews: None,
silent: None,
mute_until,
ios_sound: None,
android_sound: None,
other_sound: None,
stories_muted: None,
stories_hide_sender: None,
stories_ios_sound: None,
stories_android_sound: None,
stories_other_sound: None,
})
}
#[test]
fn a_chat_is_silent_until_its_mute_runs_out() {
assert!(silent(&settings(Some(FOREVER)), 1_790_000_000));
assert!(silent(&settings(Some(1_790_000_100)), 1_790_000_000));
assert!(!silent(&settings(Some(1_789_999_000)), 1_790_000_000));
assert!(!silent(&settings(Some(0)), 1_790_000_000));
assert!(!silent(&settings(None), 1_790_000_000));
}
}
src/backend/telegram/peers.rs
+312
View File
@@ -0,0 +1,312 @@
//! A Telegram conversation as a buffer: its name, its kind, its heading in the list.
//!
//! People, groups, supergroups, channels and bots are all "peers". A person (or a bot) is a direct message; a group or
//! a channel is a channel. The list is headed Direct Messages, Groups and Channels, as Twitch's is headed Channels and
//! Direct Messages (see `runtime.rs fixed_category`).
//!
//! The buffer is named for the conversation, and the buffer's name is what its history is filed under, so it must not
//! change when somebody renames themselves: a buffer is found by the peer's own id (Telegram's "bot API dialog id",
//! kept as the buffer's `remote_id`), and keeps the name it was first given. Two people with the same name are told
//! apart by their `@username`, or failing that by the end of their number.
use crate::model::Buffer;
use crate::state::AppState;
use grammers_client::peer::Peer;
use grammers_client::tl;
use grammers_session::types::PeerId;
pub const DIRECT_MESSAGES: &str = "Direct Messages";
pub const BOTS: &str = "Bots";
pub const GROUPS: &str = "Groups";
pub const CHANNELS: &str = "Channels";
/// The id Telegram's own tools (and the bot API) call a peer by: a person's number, minus a group's, and a
/// supergroup's or channel's with `-100` in front.
pub fn remote_id(peer: PeerId) -> String {
peer.bot_api_dialog_id().map(|id| id.to_string()).unwrap_or_default()
}
/// The peer a buffer's `remote_id` names.
pub fn peer_id_of(remote_id: &str) -> Option<PeerId> {
// A topic's conversation names its group first and the topic after a colon.
PeerId::from_bot_api_dialog_id(remote_id.split(':').next()?.parse().ok()?)
}
pub fn kind_of(peer: &Peer) -> &'static str {
match peer {
Peer::User(_) => "dm",
_ => "channel",
}
}
pub fn category_of(peer: &Peer) -> &'static str {
match peer {
// A bot is a conversation with a program, not a person: its own heading (it is still a direct chat).
Peer::User(user) if user.is_bot() => BOTS,
Peer::User(_) => DIRECT_MESSAGES,
Peer::Channel(_) => CHANNELS,
_ => GROUPS,
}
}
/// Whether a conversation is a group (small, or a supergroup): each has an entry of its own on the rail, as a server does.
pub fn is_group(peer: &Peer) -> bool {
matches!(peer, Peer::Group(_))
}
/// The rail entry of a group.
pub fn rail_id(account_id: &str, remote: &str) -> String {
format!("{account_id}|tggroup:{remote}")
}
/// Gives a group its place on the rail, with its own name and picture. Answers its rail id; nothing for anything that
/// is not a group.
pub fn rail_for(state: &AppState, account_id: &str, peer: &Peer, position: i64) -> Option<String> {
if !is_group(peer) {
return None;
}
let remote = remote_id(peer.id());
let rail = rail_id(account_id, &remote);
let account: i64 = account_id.strip_prefix("telegram:").and_then(|a| a.parse().ok()).unwrap_or(0);
state.runtime.upsert_buffer_group(
state,
crate::model::BufferGroup {
id: rail.clone(),
account_id: account_id.to_string(),
service: "telegram".to_string(),
kind: "guild".to_string(),
name: title_of(peer),
icon_url: super::media::avatar_url(account, peer),
position,
pending: false,
},
);
Some(rail)
}
/// Gives a group its place on the rail and files its conversation under it. Nothing for anything that is not a group:
/// people, bots and channels stay in the account's own list.
pub fn file_in_rail(state: &AppState, account_id: &str, buffer: &Buffer, peer: &Peer, position: i64) {
let Some(rail) = rail_for(state, account_id, peer, position) else { return };
state.runtime.set_buffer_group(state, &buffer.id, &rail);
state.runtime.set_buffer_category(state, &buffer.id, None, 0);
}
/// The glyph in front of a conversation's name: a horn for a broadcast channel, a robot for a bot.
pub fn name_icon_of(peer: &Peer) -> Option<&'static str> {
match peer {
Peer::Channel(channel) if channel.raw.broadcast => Some("campaign"),
Peer::User(user) if user.is_bot() => Some("smart_toy"),
_ => None,
}
}
/// What the conversation is called: the person's name, the group's title, or "Saved Messages" for the account's own.
pub fn title_of(peer: &Peer) -> String {
match peer {
Peer::User(user) if user.is_self() => "Saved Messages".to_string(),
Peer::User(user) => {
let name = user.full_name();
if !name.trim().is_empty() {
name
} else if let Some(username) = user.username() {
username.to_string()
} else {
"Deleted account".to_string()
}
}
other => other.name().filter(|n| !n.trim().is_empty()).unwrap_or("Unnamed chat").to_string(),
}
}
/// A name for a buffer that no other conversation of the account has.
pub fn unique_name(taken: &[(String, Option<String>)], remote_id: &str, title: &str, username: Option<&str>) -> String {
let used = |name: &str| taken.iter().any(|(n, r)| n == name && r.as_deref() != Some(remote_id));
if !used(title) {
return title.to_string();
}
if let Some(username) = username.filter(|u| !u.is_empty()) {
let with = format!("{title} (@{username})");
if !used(&with) {
return with;
}
}
let digits: String = remote_id.chars().filter(|c| c.is_ascii_digit()).collect();
let tail = &digits[digits.len().saturating_sub(4)..];
format!("{title} ·{tail}")
}
/// The conversation's buffer, made if it is not there yet.
pub fn buffer_for(state: &AppState, account_id: &str, peer: &Peer) -> Buffer {
let remote = remote_id(peer.id());
let mine: Vec<Buffer> = state.runtime.list_buffers().into_iter().filter(|b| b.account_id == account_id).collect();
if let Some(found) = mine.iter().find(|b| b.remote_id.as_deref() == Some(remote.as_str())) {
return found.clone();
}
let taken: Vec<(String, Option<String>)> = mine.iter().map(|b| (b.name.clone(), b.remote_id.clone())).collect();
let name = unique_name(&taken, &remote, &title_of(peer), peer.username());
let buffer = state.runtime.ensure_buffer(state, account_id, &name, kind_of(peer));
state.runtime.set_buffer_remote_id(state, &buffer.id, &remote);
state.runtime.set_buffer_category(state, &buffer.id, Some(category_of(peer)), 0);
state.runtime.set_buffer_name_icon(state, &buffer.id, name_icon_of(peer));
file_in_rail(state, account_id, &buffer, peer, 1000);
state.runtime.get_buffer(&buffer.id).unwrap_or(buffer)
}
/// What the window says in place of the message box where the account may not write: the words every service uses.
pub const NO_SEND_PERMISSION: &str = "You do not have permission to send messages in this channel.";
/// What decides whether an account may write in a group or a channel, read off what Telegram says about it.
#[derive(Default, Clone, Debug, PartialEq)]
pub struct Rights {
/// A broadcast channel, where only those allowed to post may write.
pub broadcast: bool,
pub creator: bool,
/// An administrator (of any kind) of this chat.
pub admin: bool,
/// An administrator who may post in a broadcast channel.
pub may_post: bool,
/// The account has left, or was removed.
pub gone: bool,
/// The chat was upgraded to a supergroup and no longer takes messages.
pub deactivated: bool,
/// This account in particular may not send.
pub banned_here: bool,
/// Nobody but administrators may send.
pub members_banned: bool,
}
/// Why the account cannot write in a chat, or `None` when it can.
pub fn read_only_reason(r: &Rights) -> Option<&'static str> {
if r.gone {
return Some("You are not a member of this chat.");
}
if r.deactivated {
return Some("This group was upgraded and no longer takes messages.");
}
let boss = r.creator || r.admin;
if r.broadcast {
return (!(r.creator || r.may_post)).then_some(NO_SEND_PERMISSION);
}
if r.banned_here || (r.members_banned && !boss) {
return Some(NO_SEND_PERMISSION);
}
None
}
fn sending_banned(rights: &Option<tl::enums::ChatBannedRights>, now: i64) -> bool {
match rights {
Some(tl::enums::ChatBannedRights::Rights(b)) => b.send_messages && (b.until_date == 0 || i64::from(b.until_date) > now),
None => false,
}
}
fn is_admin(rights: &Option<tl::enums::ChatAdminRights>) -> bool {
rights.is_some()
}
fn may_post(rights: &Option<tl::enums::ChatAdminRights>) -> bool {
matches!(rights, Some(tl::enums::ChatAdminRights::Rights(a)) if a.post_messages)
}
fn rights_of_channel(c: &tl::types::Channel, now: i64) -> Rights {
Rights {
broadcast: c.broadcast,
creator: c.creator,
admin: is_admin(&c.admin_rights),
may_post: may_post(&c.admin_rights),
gone: c.left,
deactivated: false,
banned_here: sending_banned(&c.banned_rights, now),
members_banned: sending_banned(&c.default_banned_rights, now),
}
}
/// What the account may do in a conversation. Only meaningful for a peer read from the dialog list: one that arrived
/// with a message may be a "min" copy without the account's rights in it.
pub fn rights_of(peer: &Peer, now: i64) -> Option<Rights> {
match peer {
Peer::Channel(channel) => Some(rights_of_channel(&channel.raw, now)),
Peer::Group(group) => Some(match &group.raw {
tl::enums::Chat::Chat(c) => Rights {
creator: c.creator,
admin: is_admin(&c.admin_rights),
gone: c.left,
deactivated: c.deactivated,
members_banned: sending_banned(&c.default_banned_rights, now),
..Default::default()
},
tl::enums::Chat::Channel(c) => rights_of_channel(c, now),
// A chat that is gone, or that this account cannot see.
_ => Rights { gone: true, ..Default::default() },
}),
Peer::User(user) if user.id().bare_id_unchecked() == 777000 => None,
_ => None,
}
}
/// The reason the window gives for a message box it does not show, for a conversation read from the dialog list.
pub fn read_only_for(peer: &Peer, now: i64) -> Option<&'static str> {
if let Peer::User(user) = peer {
// Telegram's own service account sends its login codes and takes no reply.
return (user.id().bare_id_unchecked() == 777000).then_some("Telegram does not take replies here.");
}
rights_of(peer, now).and_then(|r| read_only_reason(&r))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn two_people_with_one_name_are_told_apart_and_a_rename_never_makes_a_second_buffer() {
let taken = vec![("Anna".to_string(), Some("111".to_string()))];
assert_eq!(unique_name(&taken, "222", "Anna", None), "Anna ·222");
assert_eq!(unique_name(&taken, "222", "Anna", Some("anna_k")), "Anna (@anna_k)");
// The same person asking again is not a clash with themselves.
assert_eq!(unique_name(&taken, "111", "Anna", None), "Anna");
assert_eq!(unique_name(&taken, "333", "Boris", None), "Boris");
assert_eq!(unique_name(&taken, "123456789", "Anna", None), "Anna ·6789");
}
#[test]
fn a_broadcast_channel_takes_messages_only_from_those_allowed_to_post() {
let feed = Rights { broadcast: true, ..Default::default() };
assert_eq!(read_only_reason(&feed), Some(NO_SEND_PERMISSION));
assert_eq!(read_only_reason(&Rights { admin: true, ..feed.clone() }), Some(NO_SEND_PERMISSION), "an administrator who may not post is a reader here");
assert_eq!(read_only_reason(&Rights { admin: true, may_post: true, ..feed.clone() }), None);
assert_eq!(read_only_reason(&Rights { creator: true, ..feed }), None);
}
#[test]
fn a_group_takes_messages_unless_this_account_or_every_member_is_banned_from_sending() {
assert_eq!(read_only_reason(&Rights::default()), None);
assert_eq!(read_only_reason(&Rights { banned_here: true, ..Default::default() }), Some(NO_SEND_PERMISSION));
assert_eq!(read_only_reason(&Rights { members_banned: true, ..Default::default() }), Some(NO_SEND_PERMISSION));
assert_eq!(read_only_reason(&Rights { members_banned: true, admin: true, ..Default::default() }), None, "administrators may still send where members may not");
assert!(read_only_reason(&Rights { gone: true, ..Default::default() }).unwrap().contains("not a member"));
assert!(read_only_reason(&Rights { deactivated: true, ..Default::default() }).unwrap().contains("upgraded"));
}
#[test]
fn a_ban_on_sending_that_has_run_out_is_not_one() {
use tl::types::ChatBannedRights as B;
let banned = |until| Some(tl::enums::ChatBannedRights::Rights(B { send_messages: true, until_date: until, view_messages: false, send_media: false, send_stickers: false, send_gifs: false, send_games: false, send_inline: false, embed_links: false, send_polls: false, change_info: false, invite_users: false, pin_messages: false, manage_topics: false, send_photos: false, send_videos: false, send_roundvideos: false, send_audios: false, send_voices: false, send_docs: false, send_plain: false, edit_rank: false, manage_linked_peers: false, send_reactions: false }));
assert!(sending_banned(&banned(0), 1000), "no end is for ever");
assert!(sending_banned(&banned(2000), 1000));
assert!(!sending_banned(&banned(500), 1000));
assert!(!sending_banned(&None, 1000));
}
#[test]
fn an_id_goes_to_the_buffer_and_back() {
let user = PeerId::user_unchecked(1234567890);
assert_eq!(remote_id(user), "1234567890");
assert_eq!(peer_id_of("1234567890"), Some(user));
let channel = PeerId::channel_unchecked(1234567890);
assert_eq!(remote_id(channel), "-1001234567890");
assert_eq!(peer_id_of("-1001234567890"), Some(channel));
assert_eq!(peer_id_of("not a number"), None);
}
}
src/backend/telegram/pins.rs
+49
View File
@@ -0,0 +1,49 @@
//! A conversation's pinned messages: listed, pinned and unpinned.
//!
//! Telegram keeps the list on its side, so it is asked for when somebody opens the pins and again when an update says
//! the pins changed. The answer has the shape the other services' pins have, so the window's banner and list work
//! the same.
use super::messages;
use super::send::{explain, peer_for};
use crate::model::Buffer;
use crate::state::AppState;
use grammers_client::tl;
use serde_json::{json, Value};
/// How many pins are read: a conversation with more has more than anybody is going to scroll.
const LIMIT: usize = 50;
pub async fn list(state: &AppState, buffer: &Buffer) -> Result<Value, String> {
let (handle, peer) = peer_for(state, buffer).await?;
let mut found = handle.client.search_messages(peer).filter(tl::enums::MessagesFilter::InputMessagesFilterPinned);
let mut pinned: Vec<Value> = Vec::new();
while pinned.len() < LIMIT {
let Some(message) = found.next().await.map_err(|e| explain(&e))? else { break };
let from = if message.outgoing() { handle.own.name.clone() } else { message.sender().map(messages::name_of).unwrap_or_else(|| buffer.name.clone()) };
let body = messages::body_of(&message);
pinned.push(json!({
"id": message.id().to_string(),
"bufferId": buffer.id,
"from": from,
"body": if body.trim().is_empty() { "(no text)".to_string() } else { body },
"ts": message.date().as_second(),
"kind": "chat",
}));
}
let ids: Vec<String> = pinned.iter().filter_map(|m| m["id"].as_str().map(str::to_string)).collect();
// Said as well as answered, so the header's count is right in every window watching this conversation.
state.events.emit("pinnedMessages", json!({ "bufferId": buffer.id, "pinned": ids }));
Ok(json!({ "bufferId": buffer.id, "pinned": pinned }))
}
pub async fn set(state: &AppState, buffer: &Buffer, msg_id: &str, pinned: bool) -> Result<(), String> {
let id: i32 = msg_id.parse().map_err(|_| "that message cannot be pinned".to_string())?;
let (handle, peer) = peer_for(state, buffer).await?;
if pinned {
handle.client.pin_message(peer, id).await.map_err(|e| explain(&e))?;
} else {
handle.client.unpin_message(peer, id).await.map_err(|e| explain(&e))?;
}
Ok(())
}
src/backend/telegram/polls.rs
+280
View File
@@ -0,0 +1,280 @@
//! Polls: a question with answers, shown as the card over the conversation and voted in from it.
//!
//! A poll is part of a message. When one that is still open is read (live or from history) it is announced as the
//! conversation's poll card; Telegram sends an update whenever the tally moves, and the card is drawn again. Only the
//! newest open poll has the card (a conversation has one slot for it); a closed poll stays in the log as its label.
use super::send::{explain, peer_for};
use crate::model::Buffer;
use crate::state::AppState;
use grammers_client::tl;
use serde_json::{json, Value};
use std::collections::HashMap;
use std::sync::Mutex;
/// Which message each poll is, by Telegram's id for the poll, for an update that names only the poll.
fn registry() -> &'static Mutex<HashMap<i64, (String, i32)>> {
static R: std::sync::OnceLock<Mutex<HashMap<i64, (String, i32)>>> = std::sync::OnceLock::new();
R.get_or_init(Default::default)
}
fn text_of(t: &tl::enums::TextWithEntities) -> String {
let tl::enums::TextWithEntities::Entities(t) = t;
t.text.clone()
}
/// An answer's id as text: Telegram names an answer by a few bytes.
pub fn option_id(bytes: &[u8]) -> String {
hex::encode(bytes)
}
/// The card for a poll, from its definition and its tally.
pub fn card(buffer_id: &str, msg_id: i32, poll: &tl::types::Poll, results: &tl::types::PollResults) -> Value {
let tally: HashMap<Vec<u8>, (i32, bool)> = results
.results
.iter()
.flatten()
.map(|r| {
let tl::enums::PollAnswerVoters::Voters(v) = r;
(v.option.clone(), (v.voters.unwrap_or(0), v.chosen))
})
.collect();
let mut voted: Option<String> = None;
let options: Vec<Value> = poll
.answers
.iter()
.map(|a| {
let tl::enums::PollAnswer::Answer(a) = a else { return json!({}) };
let (votes, chosen) = tally.get(&a.option).copied().unwrap_or((0, false));
if chosen {
voted = Some(option_id(&a.option));
}
json!({ "id": option_id(&a.option), "label": text_of(&a.text), "votes": votes })
})
.collect();
json!({
"bufferId": buffer_id,
"kind": "poll",
"id": msg_id.to_string(),
"title": text_of(&poll.question),
"options": options,
// A poll has no clock the card could count: it is open until it is closed.
"open": !poll.closed,
"duration": 0,
"remaining": 0,
"resultDisplayDuration": 0,
"hasVoted": voted.is_some(),
"votedOptionId": voted,
"noEnd": true,
})
}
/// Shows a poll as the conversation's card, and remembers which message it is.
pub fn announce(state: &AppState, buffer_id: &str, msg_id: i32, poll: &tl::types::Poll, results: &tl::types::PollResults) {
registry().lock().unwrap_or_else(|e| e.into_inner()).insert(poll.id, (buffer_id.to_string(), msg_id));
if poll.closed {
return;
}
let card = card(buffer_id, msg_id, poll, results);
state.runtime.set_live_card(buffer_id, "poll", card.clone());
let ts = chrono::Utc::now().timestamp();
if let Err(e) = state.store.record_live_card(buffer_id, "poll", &msg_id.to_string(), card["title"].as_str().unwrap_or(""), &card.to_string(), ts) {
tracing::debug!("telegram: keeping the poll: {e:#}");
}
state.events.emit("pollCard", json!({ "bufferId": buffer_id, "kind": "poll", "poll": card }));
}
/// The tally of a poll moved.
pub fn update(state: &AppState, update: &tl::types::UpdateMessagePoll) {
let Some((buffer_id, msg_id)) = registry().lock().unwrap_or_else(|e| e.into_inner()).get(&update.poll_id).cloned() else { return };
let tl::enums::PollResults::Results(results) = &update.results;
// A poll that closed is drawn as it ended; one whose definition came with the update is redrawn from it.
match &update.poll {
Some(tl::enums::Poll::Poll(poll)) => announce_any(state, &buffer_id, msg_id, poll, results),
None => {
if let Some(mut live) = state.runtime.live_card(&buffer_id, "poll").filter(|c| c["id"].as_str() == Some(&msg_id.to_string())) {
let mut chosen_id: Option<String> = None;
if let Some(options) = live["options"].as_array_mut() {
for option in options.iter_mut() {
let id = option["id"].as_str().unwrap_or("").to_string();
let found = results.results.iter().flatten().find_map(|r| {
let tl::enums::PollAnswerVoters::Voters(v) = r;
(option_id(&v.option) == id).then_some((v.voters.unwrap_or(0), v.chosen))
});
if let Some((votes, chosen)) = found {
option["votes"] = json!(votes);
if chosen {
chosen_id = Some(id);
}
}
}
}
if let Some(id) = chosen_id {
live["votedOptionId"] = json!(id);
live["hasVoted"] = json!(true);
}
state.runtime.set_live_card(&buffer_id, "poll", live.clone());
state.events.emit("pollCard", json!({ "bufferId": buffer_id, "kind": "poll", "poll": live }));
}
}
}
}
/// Redraws a poll that may have closed: the card says so rather than vanishing.
fn announce_any(state: &AppState, buffer_id: &str, msg_id: i32, poll: &tl::types::Poll, results: &tl::types::PollResults) {
let card = card(buffer_id, msg_id, poll, results);
state.runtime.set_live_card(buffer_id, "poll", card.clone());
state.events.emit("pollCard", json!({ "bufferId": buffer_id, "kind": "poll", "poll": card }));
}
/// Votes for an answer in the poll the conversation's card shows.
pub async fn vote(state: &AppState, buffer: &Buffer, answer: &str) -> Result<(), String> {
let card = state.runtime.live_card(&buffer.id, "poll").ok_or_else(|| "there is no poll open here".to_string())?;
let msg_id: i32 = card["id"].as_str().and_then(|i| i.parse().ok()).ok_or_else(|| "that poll cannot be voted in".to_string())?;
let option = hex::decode(answer).map_err(|_| "that is not one of the poll's answers".to_string())?;
let (handle, peer) = peer_for(state, buffer).await?;
handle.client.invoke(&tl::functions::messages::SendVote { peer: peer.into(), msg_id, options: vec![option] }).await.map_err(|e| explain(&e))?;
// Said at once; Telegram's own update with the new tally follows.
let mut live = card;
live["hasVoted"] = json!(true);
live["votedOptionId"] = json!(answer);
if let Some(options) = live["options"].as_array_mut() {
for o in options.iter_mut() {
if o["id"].as_str() == Some(answer) {
o["votes"] = json!(o["votes"].as_i64().unwrap_or(0) + 1);
}
}
}
state.runtime.set_live_card(&buffer.id, "poll", live.clone());
state.events.emit("pollCard", json!({ "bufferId": buffer.id, "kind": "poll", "poll": live }));
Ok(())
}
/// Telegram's limits on a poll: a question of 255 characters, answers of 100, two to ten of them.
const QUESTION_MAX: usize = 255;
const ANSWER_MAX: usize = 100;
const ANSWERS: std::ops::RangeInclusive<usize> = 2..=10;
/// What a poll somebody is starting is, checked and as Telegram's own type. `disclosed` is whether the running
/// count shows before it closes.
pub fn build(question: &str, answers: &[String], disclosed: bool) -> Result<tl::types::Poll, String> {
let question = question.trim();
let answers: Vec<&str> = answers.iter().map(|a| a.trim()).filter(|a| !a.is_empty()).collect();
if question.is_empty() {
return Err("A poll needs a question".to_string());
}
if question.chars().count() > QUESTION_MAX {
return Err(format!("The question can be at most {QUESTION_MAX} characters"));
}
if !ANSWERS.contains(&answers.len()) {
return Err("Telegram polls have between 2 and 10 answers".to_string());
}
if answers.iter().any(|a| a.chars().count() > ANSWER_MAX) {
return Err(format!("An answer can be at most {ANSWER_MAX} characters"));
}
let plain = |t: &str| tl::enums::TextWithEntities::Entities(tl::types::TextWithEntities { text: t.to_string(), entities: Vec::new() });
Ok(tl::types::Poll {
id: 0,
closed: false,
// Anonymous, as Telegram's own apps start one.
public_voters: false,
multiple_choice: false,
quiz: false,
open_answers: false,
revoting_disabled: false,
shuffle_answers: false,
hide_results_until_close: !disclosed,
creator: false,
subscribers_only: false,
question: plain(question),
answers: answers.iter().enumerate().map(|(i, a)| tl::enums::PollAnswer::Answer(tl::types::PollAnswer { text: plain(a), option: vec![i as u8], media: None, added_by: None, date: None })).collect(),
close_period: None,
close_date: None,
hash: 0,
countries_iso2: None,
})
}
/// Starts a poll in a conversation.
pub async fn start(state: &AppState, buffer: &Buffer, question: &str, answers: &[String], disclosed: bool) -> Result<(), String> {
let poll = build(question, answers, disclosed)?;
let (handle, peer) = peer_for(state, buffer).await?;
let media = tl::types::InputMediaPoll { poll: tl::enums::Poll::Poll(poll), correct_answers: None, attached_media: None, solution: None, solution_entities: None, solution_media: None };
let input = grammers_client::message::InputMessage::new().media(tl::enums::InputMedia::Poll(Box::new(media)));
let sent = handle.client.send_message(peer, input).await.map_err(|e| explain(&e))?;
super::messages::record(state, &buffer.account_id, &handle.own, &sent);
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
fn text(s: &str) -> tl::enums::TextWithEntities {
tl::enums::TextWithEntities::Entities(tl::types::TextWithEntities { text: s.into(), entities: vec![] })
}
fn answer(label: &str, option: &[u8]) -> tl::enums::PollAnswer {
tl::enums::PollAnswer::Answer(tl::types::PollAnswer { text: text(label), option: option.to_vec(), media: None, added_by: None, date: None })
}
fn poll(closed: bool) -> tl::types::Poll {
tl::types::Poll {
id: 77,
closed,
public_voters: false,
multiple_choice: false,
quiz: false,
open_answers: false,
revoting_disabled: false,
shuffle_answers: false,
hide_results_until_close: false,
creator: false,
subscribers_only: false,
question: text("Lunch?"),
answers: vec![answer("Pizza", b"0"), answer("Soup", b"1")],
close_period: None,
close_date: None,
hash: 0,
countries_iso2: None,
}
}
fn voters(option: &[u8], n: i32, chosen: bool) -> tl::enums::PollAnswerVoters {
tl::enums::PollAnswerVoters::Voters(tl::types::PollAnswerVoters { chosen, correct: false, option: option.to_vec(), voters: Some(n), recent_voters: None })
}
#[test]
fn a_poll_that_is_started_is_checked() {
let a = |n: &[&str]| n.iter().map(|s| s.to_string()).collect::<Vec<_>>();
let ok = build(" Lunch? ", &a(&["Pizza", " Soup ", ""]), false).unwrap();
assert_eq!(ok.answers.len(), 2, "an empty answer is not one");
assert!(ok.hide_results_until_close);
assert!(!ok.public_voters);
assert!(build("", &a(&["a", "b"]), true).is_err());
assert!(build("Q", &a(&["only"]), true).is_err());
assert!(build("Q", &a(&["1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "11"]), true).is_err());
assert!(build(&"x".repeat(256), &a(&["a", "b"]), true).is_err());
assert!(!build("Q", &a(&["a", "b"]), true).unwrap().hide_results_until_close);
}
#[test]
fn a_poll_becomes_a_card_with_counts_and_the_accounts_own_choice() {
let results = tl::types::PollResults { min: false, has_unread_votes: false, can_view_stats: false, results: Some(vec![voters(b"0", 3, false), voters(b"1", 5, true)]), total_voters: Some(8), recent_voters: None, solution: None, solution_entities: None, solution_media: None };
let card = card("telegram:1|Lunch", 42, &poll(false), &results);
assert_eq!((card["title"].as_str(), card["id"].as_str(), card["open"].as_bool()), (Some("Lunch?"), Some("42"), Some(true)));
assert_eq!(card["options"][0]["votes"], 3);
assert_eq!(card["options"][1]["votes"], 5);
assert_eq!((card["hasVoted"].as_bool(), card["votedOptionId"].as_str()), (Some(true), Some(option_id(b"1").as_str())));
let closed = super::card("b", 1, &poll(true), &results);
assert_eq!(closed["open"], false);
}
#[test]
fn a_poll_nobody_voted_in_has_zeros_and_no_choice() {
let results = tl::types::PollResults { min: false, has_unread_votes: false, can_view_stats: false, results: None, total_voters: None, recent_voters: None, solution: None, solution_entities: None, solution_media: None };
let card = card("b", 1, &poll(false), &results);
assert_eq!(card["options"][0]["votes"], 0);
assert_eq!(card["hasVoted"], false);
}
}
src/backend/telegram/profile.rs
+111
View File
@@ -0,0 +1,111 @@
//! The card for a person: their name, handle, picture, what they wrote about themselves and when they were last seen.
use super::media;
use super::messages;
use super::peers;
use super::send::{explain, peer_for};
use crate::profile;
use crate::state::AppState;
use grammers_client::peer::{Peer, User};
use grammers_client::tl;
use grammers_session::Session;
use serde_json::{json, Value};
/// What they were last seen doing, in the card's words, from Telegram's coarse status.
pub fn presence(status: &tl::enums::UserStatus) -> (Option<&'static str>, Option<i64>) {
use tl::enums::UserStatus as S;
match status {
S::Online(_) => (Some("online"), None),
S::Offline(o) => (Some("offline"), Some(i64::from(o.was_online))),
S::Recently(_) => (Some("offline"), None),
S::LastWeek(_) | S::LastMonth(_) => (Some("offline"), None),
S::Empty => (None, None),
}
}
/// How Telegram words its coarse "last seen" when it will not give a time.
fn coarse(status: &tl::enums::UserStatus) -> Option<&'static str> {
use tl::enums::UserStatus as S;
match status {
S::Recently(_) => Some("recently"),
S::LastWeek(_) => Some("within a week"),
S::LastMonth(_) => Some("within a month"),
_ => None,
}
}
/// The card for the person a conversation is with, or a member of a group the window asked about.
pub async fn build(state: &AppState, account_id: &str, buffer_id: &str, who: &str, user_id: &str) -> Value {
let mut card = profile::pending("telegram", account_id, who);
card["pending"] = json!(false);
let Some(buffer) = state.runtime.get_buffer(buffer_id) else { return card };
// A direct message is with one person; in a group the member asked about is named by number.
let target = if buffer.kind == "dm" { buffer.remote_id.clone() } else { Some(user_id.to_string()) };
let Some(target) = target.and_then(|t| peers::peer_id_of(&t)) else { return card };
let Ok((handle, _)) = peer_for(state, &buffer).await else { return card };
let Some(peer) = handle.session.peer_ref(target).await.ok().flatten() else { return card };
card["id"] = json!(target.bare_id_unchecked().to_string());
let full = handle.client.invoke(&tl::functions::users::GetFullUser { id: peer.into() }).await;
let tl::enums::users::UserFull::Full(full) = match full {
Ok(f) => f,
Err(e) => {
tracing::debug!("telegram profile: {}", explain(&e));
return card;
}
};
let tl::enums::UserFull::Full(info) = &full.full_user;
if let Some(user) = full.users.iter().find(|u| u.id() == target.bare_id_unchecked()).cloned() {
let user = User::from_raw(&handle.client, user);
let as_peer = Peer::User(user.clone());
card["name"] = json!(messages::name_of(&as_peer));
if let Some(username) = user.username() {
card["handle"] = json!(format!("@{username}"));
}
profile::set(&mut card, "avatarUrl", media::avatar_url(handle.own.user_id, &as_peer).map(|u| json!(u)));
let (status, seen) = presence(user.status());
profile::set(&mut card, "status", status.map(|s| json!(s)));
profile::set(&mut card, "lastActiveTs", seen.map(|t| json!(t)));
if let Some(words) = coarse(user.status()) {
profile::note(&mut card, "Last seen", words);
}
if user.is_bot() {
profile::note(&mut card, "Account", "A bot");
}
if user.verified() {
profile::note(&mut card, "Account", "Verified by Telegram");
}
if let Some(phone) = user.phone() {
profile::note(&mut card, "Phone", format!("+{phone}"));
}
}
if let Some(about) = &info.about {
profile::note(&mut card, "About", about.clone());
}
if let Some(tl::enums::Birthday::Birthday(b)) = &info.birthday {
profile::note(&mut card, "Birthday", match b.year {
Some(year) => format!("{}.{}.{year}", b.day, b.month),
None => format!("{}.{}", b.day, b.month),
});
}
if info.common_chats_count > 0 {
profile::note(&mut card, "Groups in common", info.common_chats_count.to_string());
}
card
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn last_seen_is_a_time_when_telegram_gives_one_and_a_word_when_it_does_not() {
let offline = tl::enums::UserStatus::Offline(tl::types::UserStatusOffline { was_online: 1790000000 });
assert_eq!(presence(&offline), (Some("offline"), Some(1790000000)));
assert_eq!(presence(&tl::enums::UserStatus::Online(tl::types::UserStatusOnline { expires: 1 })), (Some("online"), None));
let recently = tl::enums::UserStatus::Recently(tl::types::UserStatusRecently { by_me: false });
assert_eq!(presence(&recently), (Some("offline"), None));
assert_eq!(coarse(&recently), Some("recently"));
assert_eq!(coarse(&offline), None);
assert_eq!(presence(&tl::enums::UserStatus::Empty), (None, None));
}
}
src/backend/telegram/reactions.rs
+138
View File
@@ -0,0 +1,138 @@
//! Reactions: the emoji people put on a message, and the account's own.
//!
//! Telegram sends a message with its reactions as a list of (reaction, count, whether the account chose it), and an
//! update when they change. A reaction is an ordinary emoji, a custom emoji (drawn from Telegram's sticker sets,
//! which moho does not draw yet, so it is counted under a star) or a paid star.
use super::send::{explain, peer_for};
use crate::model::{Buffer, Reaction};
use crate::state::AppState;
use grammers_client::message::InputReactions;
use grammers_client::tl;
/// What stands for a reaction moho cannot draw: custom emoji and paid stars.
const STAND_IN: &str = "⭐";
/// A message's reactions as moho's list.
pub fn from_tl(reactions: &tl::enums::MessageReactions) -> Vec<Reaction> {
let tl::enums::MessageReactions::Reactions(r) = reactions;
let mut out: Vec<Reaction> = Vec::new();
for count in &r.results {
let tl::enums::ReactionCount::Count(c) = count;
let emoji = match &c.reaction {
tl::enums::Reaction::Emoji(e) => e.emoticon.clone(),
tl::enums::Reaction::CustomEmoji(_) | tl::enums::Reaction::Paid => STAND_IN.to_string(),
tl::enums::Reaction::Empty => continue,
};
// Two kinds that share the stand-in are one pill.
if let Some(existing) = out.iter_mut().find(|x| x.emoji == emoji) {
existing.count += i64::from(c.count);
existing.me |= c.chosen_order.is_some();
} else {
out.push(Reaction { emoji, count: i64::from(c.count), me: c.chosen_order.is_some(), animated: false });
}
}
out
}
/// Records what a message's reactions are now, and says so.
pub fn apply(state: &AppState, buffer_id: &str, msg_id: &str, reactions: &[Reaction]) {
match state.store.set_reactions(buffer_id, msg_id, reactions) {
Ok(true) => state.events.emit("reactionsChanged", serde_json::json!({ "bufferId": buffer_id, "id": msg_id, "reactions": reactions })),
Ok(false) => {}
Err(e) => tracing::warn!("telegram: recording reactions: {e}"),
}
}
/// The reactions the account itself has put on a message, as Telegram's own type.
fn chosen(reactions: &[Reaction]) -> Vec<tl::enums::Reaction> {
reactions.iter().filter(|r| r.me && r.emoji != STAND_IN).map(|r| tl::enums::Reaction::Emoji(tl::types::ReactionEmoji { emoticon: r.emoji.clone() })).collect()
}
/// Puts an emoji on a message or takes the account's off. Telegram lets an ordinary account have one reaction on a
/// message, so adding replaces the account's earlier one.
pub async fn toggle(state: &AppState, buffer: &Buffer, msg_id: &str, emoji: &str, add: bool) -> Result<(), String> {
let id: i32 = msg_id.parse().map_err(|_| "that message cannot take reactions".to_string())?;
let (handle, peer) = peer_for(state, buffer).await?;
let stored = state.store.get_message(&buffer.id, msg_id).ok().flatten().map(|m| m.reactions).unwrap_or_default();
let stored_again = stored.clone();
let mut mine = chosen(&stored);
let this = tl::enums::Reaction::Emoji(tl::types::ReactionEmoji { emoticon: emoji.to_string() });
if add {
mine = vec![this];
} else {
mine.retain(|r| *r != this);
}
let input: InputReactions = if mine.is_empty() { InputReactions::remove() } else { mine.into() };
handle.client.send_reactions(peer, id, input).await.map_err(|e| explain(&e))?;
// Telegram answers with the message's new state as an update, which is read below; this is the same fact said
// at once so the pill does not wait on it.
apply(state, &buffer.id, msg_id, &after_change(&stored_again, emoji, add));
Ok(())
}
/// The reactions a message has once the account has put an emoji on it (replacing its earlier one) or taken it off.
pub fn after_change(stored: &[Reaction], emoji: &str, add: bool) -> Vec<Reaction> {
let mut next: Vec<Reaction> = stored.to_vec();
for r in next.iter_mut() {
let own_ordinary = r.me && r.emoji != STAND_IN;
if own_ordinary && (add || r.emoji == emoji) {
r.me = false;
r.count -= 1;
}
}
next.retain(|r| r.count > 0);
if add {
match next.iter_mut().find(|r| r.emoji == emoji) {
Some(r) => {
r.count += 1;
r.me = true;
}
None => next.push(Reaction { emoji: emoji.to_string(), count: 1, me: true, animated: false }),
}
}
next
}
/// A reactions update from Telegram.
pub fn update(state: &AppState, account_id: &str, remote: &str, msg_id: i32, reactions: &tl::enums::MessageReactions) {
let Some(buffer) = super::topics::holding(state, account_id, remote, msg_id) else { return };
apply(state, &buffer.id, &msg_id.to_string(), &from_tl(reactions));
}
#[cfg(test)]
mod tests {
use super::*;
fn count(reaction: tl::enums::Reaction, n: i32, chosen: Option<i32>) -> tl::enums::ReactionCount {
tl::enums::ReactionCount::Count(tl::types::ReactionCount { chosen_order: chosen, reaction, count: n })
}
fn emoji(e: &str) -> tl::enums::Reaction {
tl::enums::Reaction::Emoji(tl::types::ReactionEmoji { emoticon: e.into() })
}
#[test]
fn reactions_become_pills_with_counts_and_whether_the_account_chose_them() {
let results = vec![count(emoji("👍"), 5, Some(0)), count(emoji("🔥"), 2, None), count(tl::enums::Reaction::CustomEmoji(tl::types::ReactionCustomEmoji { document_id: 9 }), 3, None), count(tl::enums::Reaction::Paid, 4, None)];
let reactions = tl::enums::MessageReactions::Reactions(tl::types::MessageReactions { min: false, can_see_list: false, reactions_as_tags: false, results, recent_reactions: None, top_reactors: None });
let got = from_tl(&reactions);
assert_eq!(got.iter().map(|r| (r.emoji.as_str(), r.count, r.me)).collect::<Vec<_>>(), vec![("👍", 5, true), ("🔥", 2, false), ("⭐", 7, false)], "two kinds drawn the same are one pill");
}
#[test]
fn choosing_an_emoji_replaces_the_accounts_earlier_one_and_taking_it_off_leaves_the_rest() {
let r = |e: &str, n: i64, me: bool| Reaction { emoji: e.into(), count: n, me, animated: false };
let stored = vec![r("👍", 5, true), r("🔥", 2, false)];
assert_eq!(after_change(&stored, "🔥", true), vec![r("👍", 4, false), r("🔥", 3, true)]);
assert_eq!(after_change(&stored, "👍", false), vec![r("👍", 4, false), r("🔥", 2, false)]);
assert_eq!(after_change(&[r("👍", 1, true)], "👍", false), Vec::<Reaction>::new(), "a pill nobody has any more goes");
assert_eq!(after_change(&[], "😂", true), vec![r("😂", 1, true)]);
}
#[test]
fn only_the_accounts_own_ordinary_reactions_are_carried_into_a_change() {
let stored = vec![Reaction { emoji: "👍".into(), count: 5, me: true, animated: false }, Reaction { emoji: "🔥".into(), count: 2, me: false, animated: false }, Reaction { emoji: STAND_IN.into(), count: 1, me: true, animated: false }];
assert_eq!(chosen(&stored), vec![emoji("👍")]);
}
}
src/backend/telegram/search.rs
+56
View File
@@ -0,0 +1,56 @@
//! Searching on Telegram's side: in one conversation or in every one the account has.
//!
//! The window searches its own copy first and shows that at once; this adds what the account's history has that this
//! client never read. A hit is answered as a message, and nothing is recorded: pressing one reads the messages around
//! it (see `history::context`) the way a jump to anything old does.
use super::media;
use super::messages;
use super::send::{explain, peer_for};
use crate::model::Buffer;
use crate::state::AppState;
use grammers_client::message::Message;
use serde_json::{json, Value};
/// One hit as the window reads a message.
fn hit(state: &AppState, account_id: &str, own_name: &str, own_id: i64, message: &Message) -> Option<Value> {
let peer = message.peer()?;
let buffer = super::peers::buffer_for(state, account_id, peer);
let from = if message.outgoing() { own_name.to_string() } else { message.sender().map(messages::name_of).unwrap_or_else(|| super::peers::title_of(peer)) };
let remote = buffer.remote_id.clone().unwrap_or_default();
Some(json!({
"id": message.id().to_string(),
"bufferId": buffer.id,
"from": from,
"body": messages::body_of(message),
"ts": message.date().as_second(),
"kind": "chat",
"isAction": false,
"isHighlight": false,
"edited": false,
"isOwn": message.outgoing(),
"attachments": media::attachments_of(own_id, &remote, message),
}))
}
/// The matches for a query, in this conversation or across the account (`account` scope).
pub async fn run(state: &AppState, buffer: &Buffer, query: &str, account_wide: bool, limit: usize) -> Result<Value, String> {
let (handle, peer) = peer_for(state, buffer).await?;
let mut results: Vec<Value> = Vec::new();
if account_wide {
let mut found = handle.client.search_all_messages().query(query);
while results.len() < limit {
let Some(message) = found.next().await.map_err(|e| explain(&e))? else { break };
results.extend(hit(state, &buffer.account_id, &handle.own.name, handle.own.user_id, &message));
}
} else {
let mut found = handle.client.search_messages(peer).query(query);
while results.len() < limit {
let Some(message) = found.next().await.map_err(|e| explain(&e))? else { break };
results.extend(hit(state, &buffer.account_id, &handle.own.name, handle.own.user_id, &message));
}
}
let names: std::collections::HashMap<String, String> = state.runtime.list_buffers().into_iter().filter(|b| b.account_id == buffer.account_id).map(|b| (b.id, b.name)).collect();
let count = results.len();
Ok(json!({ "results": results, "roomNames": names, "count": count }))
}
src/backend/telegram/send.rs
+382
View File
@@ -0,0 +1,382 @@
//! Saying things in a conversation: send, answer, edit, delete, "typing", "read".
//!
//! Each starts from the buffer: its `remote_id` names the peer, and the session turns that into the reference
//! Telegram wants with it (an id with the authority to use it). A message that was sent is recorded here as it is
//! returned, because Telegram does not send an account its own message back as an update.
use super::markup;
use super::messages;
use super::peers;
use super::Handle;
use crate::model::Buffer;
use crate::state::AppState;
use crate::upload::{Phase, Progress};
use grammers_client::media::{InputMedia, Uploaded};
use grammers_client::message::InputMessage;
use grammers_client::{tl, InvocationError};
use grammers_session::types::PeerRef;
use grammers_session::Session;
/// What Telegram allows in one message, counted the way it counts (UTF-16 units).
pub const MAX_UNITS: usize = 4096;
/// Telegram's refusal, said so a person can act on it.
pub fn explain(error: &InvocationError) -> String {
match error {
InvocationError::Rpc(rpc) => {
let n = rpc.value.unwrap_or(0);
match rpc.name.as_str() {
"FLOOD_WAIT" | "FLOOD_PREMIUM_WAIT" => format!("Telegram says to wait {n} seconds before doing that again"),
"SLOWMODE_WAIT" => format!("Slow mode is on here: wait {n} seconds between messages"),
"CHAT_WRITE_FORBIDDEN" | "CHAT_SEND_PLAIN_FORBIDDEN" => peers::NO_SEND_PERMISSION.to_string(),
"USER_BANNED_IN_CHANNEL" => "You are banned from sending messages in this chat".to_string(),
"USER_IS_BLOCKED" => "This person has blocked you".to_string(),
"USER_PRIVACY_RESTRICTED" => "This person's privacy settings do not let you message them".to_string(),
"PEER_FLOOD" => "Telegram is limiting how many people this account may start a conversation with".to_string(),
"MESSAGE_TOO_LONG" => format!("That message is too long: Telegram takes at most {MAX_UNITS} characters"),
"MESSAGE_EMPTY" => "There is nothing to send".to_string(),
"MESSAGE_NOT_MODIFIED" => "That is already what the message says".to_string(),
"MESSAGE_EDIT_TIME_EXPIRED" => "Telegram no longer lets this message be edited".to_string(),
"MESSAGE_ID_INVALID" | "MESSAGE_NOT_FOUND" => "That message is no longer there".to_string(),
"CHAT_ADMIN_REQUIRED" => "Only an administrator may do that here".to_string(),
"MEDIA_EMPTY" | "FILE_PART_INVALID" => "Telegram would not take that file".to_string(),
"FILE_REFERENCE_EXPIRED" => "Telegram needs the message read again before that file can be downloaded".to_string(),
other => format!("Telegram refused: {}", other.to_lowercase().replace('_', " ")),
}
}
InvocationError::Io(e) => format!("Telegram could not be reached: {e}"),
other => format!("{other}"),
}
}
/// The account's connection and Telegram's reference to the conversation a buffer is.
pub async fn peer_for(state: &AppState, buffer: &Buffer) -> Result<(Handle, PeerRef), String> {
let handle = state.runtime.telegram_handle(&buffer.account_id).ok_or_else(|| "that Telegram account is not connected".to_string())?;
let id = buffer.remote_id.as_deref().and_then(peers::peer_id_of).ok_or_else(|| "this is not a Telegram conversation".to_string())?;
let peer = handle.session.peer_ref(id).await.ok().flatten().ok_or_else(|| "Telegram has not told this account about that conversation yet".to_string())?;
Ok((handle, peer))
}
fn units(text: &str) -> usize {
text.encode_utf16().count()
}
/// Text from the message box, marks taken out and entities in place, if Telegram would take it.
pub fn checked(body: &str) -> Result<(String, Vec<tl::enums::MessageEntity>), String> {
let (text, entities) = markup::parse(body);
if text.trim().is_empty() {
return Err("There is nothing to send".to_string());
}
if units(&text) > MAX_UNITS {
return Err(format!("That message is too long: Telegram takes at most {MAX_UNITS} characters, and this has {}", units(&text)));
}
Ok((text, entities))
}
/// The message box's text as a message.
pub fn input_for(body: &str) -> Result<InputMessage, String> {
let (text, entities) = checked(body)?;
Ok(InputMessage::new().text(text).fmt_entities(entities))
}
/// Sends a message, as an answer to another if `reply_to` names one.
pub async fn send(state: &AppState, buffer: &Buffer, body: &str, reply_to: Option<&str>) -> Result<(), String> {
if let Some(topic) = super::topics::topic_of_buffer(buffer) {
return send_in_topic(state, buffer, body, reply_to, topic).await;
}
let input = input_for(body)?;
let (handle, peer) = peer_for(state, buffer).await?;
let input = input.reply_to(reply_to.and_then(|id| id.parse::<i32>().ok()));
let sent = handle.client.send_message(peer, input).await.map_err(|e| explain(&e))?;
messages::record(state, &buffer.account_id, &handle.own, &sent);
Ok(())
}
/// Sends a message into a forum topic. An answer to a message in a topic has to name the topic as well, which the
/// library's own send cannot say, so the request is made directly and the message it made is read back by its id.
async fn send_in_topic(state: &AppState, buffer: &Buffer, body: &str, reply_to: Option<&str>, topic: i32) -> Result<(), String> {
let (text, entities) = checked(body)?;
let (handle, peer) = peer_for(state, buffer).await?;
let reply = reply_to.and_then(|id| id.parse::<i32>().ok()).unwrap_or(topic);
let random_id: i64 = rand::random();
let updates = handle
.client
.invoke(&tl::functions::messages::SendMessage {
no_webpage: false,
silent: false,
background: false,
clear_draft: false,
noforwards: false,
update_stickersets_order: false,
invert_media: false,
allow_paid_floodskip: false,
peer: peer.into(),
reply_to: Some(
tl::types::InputReplyToMessage { reply_to_msg_id: reply, top_msg_id: Some(topic), reply_to_peer_id: None, quote_text: None, quote_entities: None, quote_offset: None, monoforum_peer_id: None, todo_item_id: None, poll_option: None }
.into(),
),
message: text,
random_id,
reply_markup: None,
entities: Some(entities).filter(|e| !e.is_empty()),
schedule_date: None,
schedule_repeat_period: None,
send_as: None,
quick_reply_shortcut: None,
effect: None,
allow_paid_stars: None,
suggested_post: None,
rich_message: None,
})
.await
.map_err(|e| explain(&e))?;
if let Some(id) = sent_id(&updates, random_id) {
if let Ok(found) = handle.client.get_messages_by_id(peer, &[id]).await {
if let Some(Some(sent)) = found.into_iter().next() {
messages::record(state, &buffer.account_id, &handle.own, &sent);
}
}
}
Ok(())
}
/// The id Telegram gave a message that was sent with `random_id`, from what it answered.
fn sent_id(updates: &tl::enums::Updates, random_id: i64) -> Option<i32> {
use tl::enums::Updates as U;
let list: &[tl::enums::Update] = match updates {
U::Updates(u) => &u.updates,
U::Combined(u) => &u.updates,
U::UpdateShort(u) => std::slice::from_ref(&u.update),
U::UpdateShortSentMessage(u) => return Some(u.id),
_ => return None,
};
list.iter().find_map(|u| match u {
tl::enums::Update::MessageId(m) if m.random_id == random_id => Some(m.id),
_ => None,
})
}
/// Changes what one of the account's own messages says.
pub async fn edit(state: &AppState, buffer: &Buffer, msg_id: &str, body: &str) -> Result<(), String> {
let id: i32 = msg_id.parse().map_err(|_| "that message cannot be edited".to_string())?;
let input = input_for(body)?;
let (handle, peer) = peer_for(state, buffer).await?;
handle.client.edit_message(peer, id, input).await.map_err(|e| explain(&e))?;
let (text, _) = markup::parse(body);
state.runtime.update_message(state, &buffer.id, msg_id, &text, &[], &[]);
Ok(())
}
/// Removes a message: for everyone where Telegram allows it, or only from this account's own view.
pub async fn delete(state: &AppState, buffer: &Buffer, msg_id: &str, for_everyone: bool) -> Result<(), String> {
let id: i32 = msg_id.parse().map_err(|_| "that message cannot be deleted".to_string())?;
let (handle, peer) = peer_for(state, buffer).await?;
if for_everyone || peer.id.kind() == grammers_session::types::PeerKind::Channel {
handle.client.delete_messages(peer, &[id]).await.map_err(|e| explain(&e))?;
} else {
handle.client.invoke(&tl::functions::messages::DeleteMessages { revoke: false, id: vec![id] }).await.map_err(|e| explain(&e))?;
}
state.runtime.delete_message(state, &buffer.id, msg_id);
Ok(())
}
/// Says the account is typing. Telegram lets it lapse after a few seconds, so there is no "stopped".
pub async fn typing(state: &AppState, buffer: &Buffer) -> Result<(), String> {
let (handle, peer) = peer_for(state, buffer).await?;
handle.client.action(peer).oneshot(tl::enums::SendMessageAction::SendMessageTypingAction).await.map_err(|e| explain(&e))
}
/// Says everything in the conversation has been read, so its unread count goes down on the account's other apps.
pub async fn mark_read(state: &AppState, buffer: &Buffer) -> Result<(), String> {
let (handle, peer) = peer_for(state, buffer).await?;
// A topic is read on its own: to the newest message of it that is here.
if let Some(topic) = super::topics::topic_of_buffer(buffer) {
// Or to the newest Telegram said it has, which may be a message this client never kept.
let newest = state.store.newest_message_id(&buffer.id).ok().flatten().and_then(|id| id.parse::<i32>().ok()).unwrap_or(0).max(super::topics::top_of(&buffer.id).unwrap_or(0));
if newest > 0 {
handle.client.invoke(&tl::functions::messages::ReadDiscussion { peer: peer.into(), msg_id: topic, read_max_id: newest }).await.map_err(|e| explain(&e))?;
}
state.runtime.set_buffer_unread(state, &buffer.id, 0, 0);
return Ok(());
}
handle.client.mark_as_read(peer).await.map_err(|e| explain(&e))
}
/// A reader that says how much of it has been read, so an upload can show a ring that fills.
struct Counting<'a, R> {
inner: R,
done: u64,
said: u64,
base: u64,
total: u64,
report: &'a (dyn Fn(u64, u64) + Send + Sync),
}
impl<R: tokio::io::AsyncRead + Unpin> tokio::io::AsyncRead for Counting<'_, R> {
fn poll_read(mut self: std::pin::Pin<&mut Self>, cx: &mut std::task::Context<'_>, buf: &mut tokio::io::ReadBuf<'_>) -> std::task::Poll<std::io::Result<()>> {
let before = buf.filled().len();
let polled = std::pin::Pin::new(&mut self.inner).poll_read(cx, buf);
if let std::task::Poll::Ready(Ok(())) = &polled {
self.done += (buf.filled().len() - before) as u64;
if self.done - self.said >= 128 * 1024 {
self.said = self.done;
(self.report)(self.base + self.done, self.total);
}
}
polled
}
}
/// What a file is, for choosing how Telegram is told about it.
#[derive(Clone, Copy, PartialEq, Debug)]
pub enum Sort {
Photo,
Video,
Other,
}
pub fn sort_of(name: &str) -> Sort {
match name.rsplit_once('.').map(|(_, e)| e.to_ascii_lowercase()).as_deref() {
Some("jpg" | "jpeg" | "png" | "webp" | "bmp") => Sort::Photo,
Some("mp4" | "mov" | "webm" | "mkv" | "m4v") => Sort::Video,
_ => Sort::Other,
}
}
async fn upload(client: &grammers_client::Client, path: &str, base: u64, total: u64, report: &(dyn Fn(u64, u64) + Send + Sync)) -> Result<(Uploaded, String), String> {
let file = tokio::fs::File::open(path).await.map_err(|e| format!("could not read {path}: {e}"))?;
let size = file.metadata().await.map_err(|e| e.to_string())?.len();
let name = std::path::Path::new(path).file_name().map(|n| n.to_string_lossy().to_string()).unwrap_or_else(|| "file".to_string());
let mut reader = Counting { inner: file, done: 0, said: 0, base, total, report };
let up = client.upload_stream(&mut reader, size as usize, name.clone()).await.map_err(|e| format!("Telegram would not take {name}: {e}"))?;
Ok((up, name))
}
/// A picture or a video as media. Anything else is sent as a plain file (see `send_files`).
fn media_for(up: Uploaded, sort: Sort, spoiler: bool) -> Option<tl::enums::InputMedia> {
match sort {
Sort::Photo => Some(tl::types::InputMediaUploadedPhoto { spoiler, file: up.raw, stickers: None, ttl_seconds: None, live_photo: false, video: None }.into()),
Sort::Video => Some(grammers_client::media::InputVideo::new(up).supports_streaming(true).spoiler(spoiler).into()),
Sort::Other => None,
}
}
/// Sends files, with the caption on the first: one message each, or an album where there are several pictures and
/// videos. `progress` names the ring the window draws.
pub async fn send_files(state: &AppState, buffer: &Buffer, caption: &str, files: &[(String, bool)], reply_to: Option<&str>, progress: Option<&Progress>) -> Result<(), String> {
if files.is_empty() {
return send(state, buffer, caption, reply_to).await;
}
let (text, entities) = if caption.trim().is_empty() { (String::new(), Vec::new()) } else { checked(caption)? };
// Telegram's limit on a caption is much smaller than on a message.
if units(&text) > 1024 {
return Err(format!("A caption can be at most 1024 characters, and this has {}", units(&text)));
}
let (handle, peer) = peer_for(state, buffer).await?;
// In a topic, a message with nothing to answer answers the topic's first message, which is how it is sent there.
let reply = reply_to.and_then(|id| id.parse::<i32>().ok()).or_else(|| super::topics::topic_of_buffer(buffer));
let total: u64 = files.iter().filter_map(|(p, _)| std::fs::metadata(p).ok().map(|m| m.len())).sum();
if let Some(p) = progress {
p.at(Phase::Preparing, total as usize, "Telegram");
}
let report = |sent: u64, all: u64| {
if let Some(p) = progress {
p.sent(sent, all, "Telegram");
}
};
let mut uploaded: Vec<(Uploaded, Sort, bool)> = Vec::new();
let mut base = 0u64;
for (path, spoiler) in files {
let (up, name) = upload(&handle.client, path, base, total, &report).await?;
base += std::fs::metadata(path).map(|m| m.len()).unwrap_or(0);
uploaded.push((up, sort_of(&name), *spoiler));
}
if let Some(p) = progress {
p.at(Phase::Waiting, total as usize, "Telegram");
}
let album = uploaded.len() >= 2 && uploaded.len() <= 10 && uploaded.iter().all(|(_, sort, _)| *sort != Sort::Other);
if album {
let medias: Vec<InputMedia> = uploaded
.into_iter()
.enumerate()
.map(|(i, (up, sort, spoiler))| {
let mut m = InputMedia::new().media(media_for(up, sort, spoiler).expect("an album is only pictures and videos"));
if i == 0 {
m = m.caption(text.clone()).fmt_entities(entities.clone()).reply_to(reply);
}
m
})
.collect();
for sent in handle.client.send_album(peer, medias).await.map_err(|e| explain(&e))?.into_iter().flatten() {
messages::record(state, &buffer.account_id, &handle.own, &sent);
}
return Ok(());
}
for (i, (up, sort, spoiler)) in uploaded.into_iter().enumerate() {
let mut input = match media_for(up.clone(), sort, spoiler) {
Some(media) => InputMessage::new().media(media),
None => InputMessage::new().file(up),
}
.reply_to(reply);
if i == 0 {
input = input.text(text.clone()).fmt_entities(entities.clone());
}
let sent = handle.client.send_message(peer, input).await.map_err(|e| explain(&e))?;
messages::record(state, &buffer.account_id, &handle.own, &sent);
}
Ok(())
}
/// Sends a recording as a voice message (an Ogg file of Opus, which is what the recorder writes).
pub async fn send_voice(state: &AppState, buffer: &Buffer, path: &std::path::Path, duration_secs: f64) -> Result<(), String> {
let (handle, peer) = peer_for(state, buffer).await?;
let none = |_: u64, _: u64| {};
let (up, _) = upload(&handle.client, &path.to_string_lossy(), 0, 0, &none).await?;
let voice = grammers_client::media::InputAudio::new(up).voice(true).duration(std::time::Duration::from_secs_f64(duration_secs.max(0.0))).mime_type("audio/ogg");
let sent = handle.client.send_message(peer, InputMessage::new().media(voice)).await.map_err(|e| explain(&e))?;
messages::record(state, &buffer.account_id, &handle.own, &sent);
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use grammers_mtsender::RpcError;
fn rpc(code: i32, name: &str, value: Option<u32>) -> InvocationError {
InvocationError::Rpc(RpcError { code, name: name.to_string(), value, caused_by: None })
}
#[test]
fn telegrams_refusals_are_said_in_words_with_their_numbers() {
assert_eq!(explain(&rpc(420, "FLOOD_WAIT", Some(30))), "Telegram says to wait 30 seconds before doing that again");
assert_eq!(explain(&rpc(420, "SLOWMODE_WAIT", Some(10))), "Slow mode is on here: wait 10 seconds between messages");
assert_eq!(explain(&rpc(403, "CHAT_WRITE_FORBIDDEN", None)), peers::NO_SEND_PERMISSION);
assert!(explain(&rpc(400, "MESSAGE_TOO_LONG", None)).contains("4096"));
assert_eq!(explain(&rpc(400, "SOMETHING_NEW_HAPPENED", None)), "Telegram refused: something new happened");
}
#[test]
fn a_file_is_sent_as_a_picture_a_video_or_a_file_by_its_extension() {
assert_eq!(sort_of("Holiday.JPG"), Sort::Photo);
assert_eq!(sort_of("clip.mp4"), Sort::Video);
assert_eq!(sort_of("report.pdf"), Sort::Other);
assert_eq!(sort_of("noextension"), Sort::Other);
// A GIF is sent as a file so Telegram keeps it moving.
assert_eq!(sort_of("cat.gif"), Sort::Other);
}
#[test]
fn what_cannot_be_sent_is_stopped_before_anything_is_sent() {
assert!(checked(" ").unwrap_err().contains("nothing to send"));
let long = "x".repeat(MAX_UNITS + 1);
assert!(checked(&long).unwrap_err().contains("too long"));
assert!(checked(&"x".repeat(MAX_UNITS)).is_ok());
// An emoji is two units, and Telegram counts units.
assert!(checked(&"😀".repeat(MAX_UNITS / 2 + 1)).unwrap_err().contains("too long"));
// Formatting marks do not count against the limit: they are taken out.
let (text, entities) = checked("**hello**").unwrap();
assert_eq!((text.as_str(), entities.len()), ("hello", 1));
}
}
src/backend/telegram/session.rs
+277
View File
@@ -0,0 +1,277 @@
//! What grammers remembers between runs: the authorization key Telegram issued, the datacenters it found, the people
//! and chats it has met (their access hashes, without which a chat cannot be addressed again) and how far the
//! account's updates have been read.
//!
//! The library offers a SQLite store, which would bring a second SQLite beside the two this daemon already links,
//! and a memory store that cannot be saved. This is its own: the same bookkeeping as the memory store, kept in a file
//! of its own in the daemon's data directory (one per account, private, written whole and renamed into place). The file is
//! a credential: the authorization key in it is the account.
//!
//! It is written when something changed and a few seconds have passed (`save_if_dirty`, called from the connection's
//! loop), and once more when the connection ends or a sign-in finishes.
use anyhow::{Context, Result};
use grammers_session::types::{ChannelState, DcOption, PeerId, PeerInfo, UpdateState, UpdatesState};
use grammers_session::{BoxFuture, Session, SessionData};
use serde::{Deserialize, Serialize};
use std::convert::Infallible;
use std::path::{Path, PathBuf};
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Mutex, MutexGuard};
/// Where an account's session lives.
pub fn path_for(user_id: i64) -> PathBuf {
folder().join(format!("{user_id}.session"))
}
/// A sign-in that has not got as far as having an account yet keeps its session here until it has.
pub fn path_for_login(login_id: &str) -> PathBuf {
let safe: String = login_id.chars().filter(|c| c.is_ascii_alphanumeric() || *c == '-' || *c == '_').collect();
folder().join(format!("login-{safe}.session"))
}
fn folder() -> PathBuf {
crate::default_data_dir().join("telegram")
}
/// The file's form: what the library's own type holds, with the maps as lists (a JSON object cannot be keyed by a
/// peer).
#[derive(Serialize, Deserialize, Default)]
struct Saved {
home_dc: i32,
dc_options: Vec<DcOption>,
peers: Vec<PeerInfo>,
updates: UpdatesState,
}
pub struct FileSession {
path: Mutex<PathBuf>,
data: Mutex<SessionData>,
dirty: AtomicBool,
}
impl FileSession {
/// A new session, with Telegram's datacenters as the library knows them and no key yet.
pub fn fresh(path: PathBuf) -> Self {
Self { path: Mutex::new(path), data: Mutex::new(SessionData::default()), dirty: AtomicBool::new(false) }
}
/// The session a file holds, or a fresh one if there is no file.
pub fn open(path: PathBuf) -> Result<Self> {
let text = match std::fs::read_to_string(&path) {
Ok(text) => text,
Err(e) if e.kind() == std::io::ErrorKind::NotFound => return Ok(Self::fresh(path)),
Err(e) => return Err(e).with_context(|| format!("reading {}", path.display())),
};
let saved: Saved = serde_json::from_str(&text).with_context(|| format!("reading the session in {}", path.display()))?;
let mut data = SessionData::default();
if saved.home_dc != 0 {
data.home_dc = saved.home_dc;
}
for option in saved.dc_options {
data.dc_options.insert(option.id, option);
}
data.peer_infos = saved.peers.into_iter().map(|p| (p.id(), p)).collect();
data.updates_state = saved.updates;
Ok(Self { path: Mutex::new(path), data: Mutex::new(data), dirty: AtomicBool::new(false) })
}
fn data(&self) -> MutexGuard<'_, SessionData> {
// Nothing here can leave the data half-changed, so a lock a panic left poisoned is still good.
self.data.lock().unwrap_or_else(|e| e.into_inner())
}
/// Whether the key Telegram issued for the home datacenter is there: what a signed-in session has.
pub fn has_key(&self) -> bool {
let data = self.data();
data.dc_options.get(&data.home_dc).is_some_and(|o| o.auth_key.is_some())
}
/// Moves the file, when a sign-in learns whose session this is.
pub fn move_to(&self, path: PathBuf) -> Result<()> {
let old = std::mem::replace(&mut *self.path.lock().unwrap_or_else(|e| e.into_inner()), path);
self.save()?;
if old != *self.path.lock().unwrap_or_else(|e| e.into_inner()) {
let _ = std::fs::remove_file(old);
}
Ok(())
}
/// Writes the session, whole and private, and renames it over the last.
pub fn save(&self) -> Result<()> {
let path = self.path.lock().unwrap_or_else(|e| e.into_inner()).clone();
let saved = {
let data = self.data();
Saved { home_dc: data.home_dc, dc_options: data.dc_options.values().cloned().collect(), peers: data.peer_infos.values().cloned().collect(), updates: data.updates_state.clone() }
};
write_private(&path, serde_json::to_string(&saved)?.as_bytes())?;
self.dirty.store(false, Ordering::Relaxed);
Ok(())
}
/// Writes it if anything changed since the last time.
pub fn save_if_dirty(&self) -> Result<()> {
if self.dirty.load(Ordering::Relaxed) {
self.save()?;
}
Ok(())
}
/// Takes the file away: an account that was removed, or a sign-in that was given up.
pub fn delete(&self) {
let path = self.path.lock().unwrap_or_else(|e| e.into_inner()).clone();
let _ = std::fs::remove_file(path);
self.dirty.store(false, Ordering::Relaxed);
}
fn touch(&self) {
self.dirty.store(true, Ordering::Relaxed);
}
}
fn write_private(path: &Path, bytes: &[u8]) -> Result<()> {
use std::io::Write;
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent).with_context(|| format!("making {}", parent.display()))?;
}
let tmp = path.with_extension("session.tmp");
{
// Private from the moment it exists, as the accounts file is.
let mut file = crate::secure::create_private_file(&tmp)?;
file.write_all(bytes)?;
}
std::fs::rename(&tmp, path).with_context(|| format!("writing {}", path.display()))
}
impl Session for FileSession {
type Error = Infallible;
fn home_dc_id(&self) -> Result<i32, Infallible> {
Ok(self.data().home_dc)
}
fn set_home_dc_id(&self, dc_id: i32) -> BoxFuture<'_, Result<(), Infallible>> {
Box::pin(async move {
self.data().home_dc = dc_id;
self.touch();
Ok(())
})
}
fn dc_option(&self, dc_id: i32) -> Result<Option<DcOption>, Infallible> {
Ok(self.data().dc_options.get(&dc_id).cloned())
}
fn set_dc_option(&self, dc_option: &DcOption) -> BoxFuture<'_, Result<(), Infallible>> {
let dc_option = dc_option.clone();
Box::pin(async move {
self.data().dc_options.insert(dc_option.id, dc_option);
self.touch();
Ok(())
})
}
fn peer(&self, peer: PeerId) -> BoxFuture<'_, Result<Option<PeerInfo>, Infallible>> {
Box::pin(async move { Ok(self.data().peer_infos.get(&peer).cloned()) })
}
fn cache_peer(&self, peer: PeerInfo) -> BoxFuture<'_, Result<(), Infallible>> {
Box::pin(async move {
let changed = {
let mut data = self.data();
let known = data.peer_infos.entry(peer.id()).or_insert_with(|| peer.clone());
known.extend_info(&peer)
};
if changed {
self.touch();
}
Ok(())
})
}
fn updates_state(&self) -> BoxFuture<'_, Result<UpdatesState, Infallible>> {
Box::pin(async move { Ok(self.data().updates_state.clone()) })
}
fn set_update_state(&self, update: UpdateState) -> BoxFuture<'_, Result<(), Infallible>> {
Box::pin(async move {
{
let mut data = self.data();
match update {
UpdateState::All(state) => data.updates_state = state,
UpdateState::Primary { pts, date, seq } => {
data.updates_state.pts = pts;
data.updates_state.date = date;
data.updates_state.seq = seq;
}
UpdateState::Secondary { qts } => data.updates_state.qts = qts,
UpdateState::Channel { id, pts } => {
data.updates_state.channels.retain(|c| c.id != id);
data.updates_state.channels.push(ChannelState { id, pts });
}
}
}
self.touch();
Ok(())
})
}
}
#[cfg(test)]
mod tests {
use super::*;
fn dir(name: &str) -> PathBuf {
let dir = std::env::temp_dir().join(format!("nobilis-tg-session-{name}-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
dir
}
#[tokio::test]
async fn a_session_is_written_private_and_comes_back_whole() {
let dir = dir("roundtrip");
let path = dir.join("1.session");
let session = FileSession::fresh(path.clone());
assert!(!session.has_key());
let mut option = session.dc_option(2).unwrap().unwrap();
option.auth_key = Some([9u8; 256]);
session.set_dc_option(&option).await.unwrap();
session.set_home_dc_id(2).await.unwrap();
session.set_update_state(UpdateState::Primary { pts: 10, date: 20, seq: 30 }).await.unwrap();
session.set_update_state(UpdateState::Channel { id: 5, pts: 7 }).await.unwrap();
assert!(session.has_key());
session.save_if_dirty().unwrap();
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
assert_eq!(std::fs::metadata(&path).unwrap().permissions().mode() & 0o777, 0o600, "the key in it is the account");
}
let back = FileSession::open(path.clone()).unwrap();
assert!(back.has_key());
assert_eq!(back.dc_option(2).unwrap().unwrap().auth_key, Some([9u8; 256]));
let state = back.updates_state().await.unwrap();
assert_eq!((state.pts, state.date, state.seq), (10, 20, 30));
assert_eq!(state.channels.iter().map(|c| (c.id, c.pts)).collect::<Vec<_>>(), vec![(5, 7)]);
std::fs::remove_dir_all(dir).ok();
}
#[tokio::test]
async fn nothing_is_written_until_something_changed_and_a_missing_file_is_a_fresh_session() {
let dir = dir("clean");
let path = dir.join("2.session");
let session = FileSession::open(path.clone()).unwrap();
session.save_if_dirty().unwrap();
assert!(!path.exists(), "an unchanged session writes nothing");
session.set_home_dc_id(4).await.unwrap();
session.save_if_dirty().unwrap();
assert!(path.exists());
let moved = dir.join("3.session");
session.move_to(moved.clone()).unwrap();
assert!(moved.exists() && !path.exists(), "a sign-in's file moves to the account's name");
session.delete();
assert!(!moved.exists());
std::fs::remove_dir_all(dir).ok();
}
}
src/backend/telegram/stickers.rs
+260
View File
@@ -0,0 +1,260 @@
//! Stickers: the packs an account has installed, the ones it uses most, and sending one.
//!
//! A sticker is a document that lives on Telegram, so sending one is sending that file, the way a GIF is. The account
//! keeps three lists the picker draws: the stickers it sent lately, the ones it marked as favourites, and its
//! installed packs. A pack is asked for on its own, which is a request each, so they are read a few at a time and
//! kept for a while. Each sticker is drawn from the still picture Telegram sends with it (an animated or video
//! sticker has one too), named in the cache and fetched when the picker draws it, from the document remembered
//! here: a file is fetched with its access key and a reference that goes stale, so what was listed is what can be
//! fetched.
use super::gifs::wrapper;
use super::media::{download_thumb, download_to, MEDIA};
use super::messages;
use super::send::{explain, peer_for};
use super::Handle;
use crate::model::Buffer;
use crate::state::AppState;
use anyhow::{Context, Result};
use futures::StreamExt;
use grammers_client::message::InputMessage;
use grammers_client::tl;
use serde_json::{json, Value};
use std::collections::HashMap;
use std::path::{Path, PathBuf};
use std::sync::Mutex;
use std::time::{Duration, Instant};
/// How many of the most recent stickers are listed.
const RECENT: usize = 40;
/// How many installed packs are read. An account can have hundreds; the picker is for the ones in daily use, which
/// Telegram lists first.
const PACKS: usize = 40;
/// How many packs are asked for at once.
const AT_ONCE: usize = 4;
/// How long a pack's stickers are kept before it is asked for again.
const KEEP: Duration = Duration::from_secs(15 * 60);
/// The documents that were listed, by (account, id): what the picker names is what can be fetched or sent.
type Known = Mutex<HashMap<(i64, i64), tl::types::Document>>;
fn known() -> &'static Known {
static K: std::sync::OnceLock<Known> = std::sync::OnceLock::new();
K.get_or_init(Default::default)
}
/// A pack's stickers, when they were read.
type Packs = Mutex<HashMap<(i64, i64), (Instant, Vec<tl::types::Document>)>>;
fn packs() -> &'static Packs {
static P: std::sync::OnceLock<Packs> = std::sync::OnceLock::new();
P.get_or_init(Default::default)
}
/// A sticker's cached picture: `tstk_<account>_<id>.webp`.
fn name(account: i64, id: i64) -> String {
format!("tstk_{account}_{id}.webp")
}
/// What a `tstk_` name says: the account and the document.
fn parse_name(file: &str) -> Option<(i64, i64)> {
let stem = file.strip_prefix("tstk_")?.strip_suffix(".webp")?;
let (account, id) = stem.split_once('_')?;
Some((account.parse().ok()?, id.parse().ok()?))
}
/// The emoji a sticker stands for, which is also what it is called.
fn emoji_of(doc: &tl::types::Document) -> String {
doc.attributes
.iter()
.find_map(|a| match a {
tl::enums::DocumentAttribute::Sticker(s) => Some(s.alt.clone()),
_ => None,
})
.unwrap_or_default()
}
/// A listed document as the picker's cell, its picture named in the cache and fetched when drawn.
fn entry(account: i64, pack: &str, doc: &tl::types::Document) -> Value {
let emoji = emoji_of(doc);
let url = format!("file://{}", crate::media_cache::dir(MEDIA).join(name(account, doc.id)).display());
json!({ "id": doc.id.to_string(), "name": if emoji.is_empty() { "Sticker".to_string() } else { emoji.clone() }, "pack": pack, "body": emoji, "url": url })
}
fn documents(list: Vec<tl::enums::Document>) -> Vec<tl::types::Document> {
list.into_iter()
.filter_map(|d| match d {
tl::enums::Document::Document(d) => Some(d),
_ => None,
})
.collect()
}
/// Adds a heading's stickers to the list and remembers the documents.
fn push(out: &mut Vec<Value>, account: i64, pack: &str, docs: Vec<tl::types::Document>) {
let mut held = known().lock().unwrap_or_else(|e| e.into_inner());
for doc in docs {
// A sticker can be in the recent list and in its pack; the picker shows it under each heading, since the
// heading is how it is found, but it is remembered and fetched once.
out.push(entry(account, pack, &doc));
held.insert((account, doc.id), doc);
}
}
/// One installed pack's stickers, from what was kept or from Telegram.
async fn pack_documents(handle: Handle, account: i64, id: i64, access_hash: i64) -> Result<Vec<tl::types::Document>> {
let key = (account, id);
if let Some((when, docs)) = packs().lock().unwrap_or_else(|e| e.into_inner()).get(&key) {
if when.elapsed() < KEEP {
return Ok(docs.clone());
}
}
let answer = handle
.client
.invoke(&tl::functions::messages::GetStickerSet { stickerset: tl::enums::InputStickerSet::Id(tl::types::InputStickerSetId { id, access_hash }), hash: 0 })
.await
.map_err(|e| anyhow::anyhow!("{}", explain(&e)))?;
let docs = match answer {
tl::enums::messages::StickerSet::Set(s) => documents(s.documents),
tl::enums::messages::StickerSet::NotModified => Vec::new(),
};
packs().lock().unwrap_or_else(|e| e.into_inner()).insert(key, (Instant::now(), docs.clone()));
Ok(docs)
}
/// What the picker shows: the recent stickers, the favourites, then the installed packs in the account's order.
pub async fn list(state: &AppState, buffer: &Buffer) -> Result<Value, String> {
let (handle, _) = peer_for(state, buffer).await?;
let account = handle.own.user_id;
let mut out = Vec::new();
// The two short lists are a nicety: a failure reading them leaves the packs to be listed.
match handle.client.invoke(&tl::functions::messages::GetRecentStickers { attached: false, hash: 0 }).await {
Ok(tl::enums::messages::RecentStickers::Stickers(r)) => {
let docs: Vec<_> = documents(r.stickers).into_iter().take(RECENT).collect();
push(&mut out, account, "Recent", docs);
}
Ok(_) => {}
Err(e) => tracing::debug!("telegram: recent stickers: {}", explain(&e)),
}
match handle.client.invoke(&tl::functions::messages::GetFavedStickers { hash: 0 }).await {
Ok(tl::enums::messages::FavedStickers::Stickers(f)) => push(&mut out, account, "Favourites", documents(f.stickers)),
Ok(_) => {}
Err(e) => tracing::debug!("telegram: favourite stickers: {}", explain(&e)),
}
let sets = match handle.client.invoke(&tl::functions::messages::GetAllStickers { hash: 0 }).await.map_err(|e| explain(&e))? {
tl::enums::messages::AllStickers::Stickers(s) => s.sets,
tl::enums::messages::AllStickers::NotModified => Vec::new(),
};
let sets: Vec<tl::types::StickerSet> = sets
.into_iter()
.map(|s| {
let tl::enums::StickerSet::Set(s) = s;
s
})
// Masks are for putting on photos and custom emoji are typed into text: neither is sent as a sticker.
.filter(|s| !s.masks && !s.emojis && !s.archived)
.take(PACKS)
.collect();
let fetched: Vec<_> = futures::stream::iter(sets.into_iter().map(|set| {
let handle = handle.clone();
async move { (set.title, pack_documents(handle, account, set.id, set.access_hash).await) }
}))
.buffered(AT_ONCE)
.collect()
.await;
for (title, result) in fetched {
match result {
Ok(docs) => push(&mut out, account, &title, docs),
Err(e) => tracing::debug!("telegram: sticker pack {title}: {e}"),
}
}
Ok(Value::Array(out))
}
/// Sends a sticker that was listed, to a conversation.
pub async fn send(state: &AppState, buffer: &Buffer, id: &str) -> Result<(), String> {
let id: i64 = id.parse().map_err(|_| "that is not a sticker".to_string())?;
let (handle, peer) = peer_for(state, buffer).await?;
let doc = known().lock().unwrap_or_else(|e| e.into_inner()).get(&(handle.own.user_id, id)).cloned().ok_or_else(|| "That sticker is no longer listed: open the picker again".to_string())?;
let media = wrapper(&doc).to_raw_input_media();
// In a topic of a group the sticker answers the topic's first message, which is how it is sent there.
let reply = super::topics::topic_of_buffer(buffer);
let sent = handle.client.send_message(peer, InputMessage::new().media(tl::enums::InputMedia::Document(media)).reply_to(reply)).await.map_err(|e| explain(&e))?;
messages::record(state, &buffer.account_id, &handle.own, &sent);
Ok(())
}
/// Fetches a listed sticker's picture again, for the window asking for it.
pub async fn restore(state: &AppState, file: &str, path: &Path) -> Result<Option<PathBuf>> {
let Some((account, id)) = parse_name(file) else { return Ok(None) };
let handle = state.runtime.telegram_handle(&format!("telegram:{account}")).context("that Telegram account is not connected")?;
let doc = known().lock().unwrap_or_else(|e| e.into_inner()).get(&(account, id)).cloned().context("that sticker is no longer listed")?;
let wrapped = wrapper(&doc);
// A still sticker is its own picture; everything else is drawn from the preview that comes with it.
if wrapped.thumbs().is_empty() && doc.mime_type == "image/webp" {
download_to(&handle.client, &wrapped, path, false).await?;
} else {
download_thumb(&handle.client, &wrapped, path).await?;
}
Ok(Some(path.to_path_buf()))
}
#[cfg(test)]
mod tests {
use super::*;
fn sticker(id: i64, alt: &str) -> tl::types::Document {
tl::types::Document {
id,
access_hash: 9,
file_reference: vec![1],
date: 0,
mime_type: "image/webp".into(),
size: 10,
thumbs: None,
video_thumbs: None,
dc_id: 2,
attributes: vec![tl::enums::DocumentAttribute::Sticker(tl::types::DocumentAttributeSticker {
mask: false,
alt: alt.into(),
stickerset: tl::enums::InputStickerSet::Empty,
mask_coords: None,
})],
}
}
#[test]
fn a_stickers_picture_is_named_and_read_back() {
let file = name(1234567890, 5_000_000_000_001);
assert_eq!(file, "tstk_1234567890_5000000000001.webp");
assert_eq!(parse_name(&file), Some((1234567890, 5_000_000_000_001)));
assert_eq!(parse_name("tgif_1_2_full.mp4"), None);
assert_eq!(parse_name("tstk_1_x.webp"), None);
}
#[test]
fn a_sticker_is_called_by_its_emoji() {
let cell = entry(5, "Cats", &sticker(77, "😺"));
assert_eq!(cell["id"], "77");
assert_eq!(cell["name"], "😺");
assert_eq!(cell["body"], "😺");
assert_eq!(cell["pack"], "Cats");
assert!(cell["url"].as_str().unwrap().ends_with("tstk_5_77.webp"));
// One without an emoji is still a cell that can be told from a missing one.
assert_eq!(entry(5, "Cats", &sticker(78, ""))["name"], "Sticker");
}
#[test]
fn a_sticker_listed_twice_is_remembered_once_and_shown_under_each_heading() {
let mut out = Vec::new();
push(&mut out, 424242, "Recent", vec![sticker(1, "a")]);
push(&mut out, 424242, "Cats", vec![sticker(1, "a"), sticker(2, "b")]);
assert_eq!(out.len(), 3);
assert!(known().lock().unwrap().contains_key(&(424242, 2)));
}
}
src/backend/telegram/threads.rs
+200
View File
@@ -0,0 +1,200 @@
//! Replies in a thread, and the comments under a channel's post.
//!
//! A reply chain in a group is a thread the window already knows how to draw from the messages it has (the replies
//! name the message at the top, see `messages::record`). What it cannot have is what was said before this client read
//! it, and a channel's comments, which are messages in the discussion group linked to the channel: those are asked for
//! here when a thread is opened, and a comment is sent to that group as an answer to the post's copy in it.
use super::peers;
use super::messages;
use super::send::{explain, input_for, peer_for};
use crate::model::Buffer;
use crate::state::AppState;
use grammers_client::tl;
use grammers_session::types::{PeerAuth, PeerId, PeerRef};
use serde_json::{json, Value};
use std::collections::HashMap;
/// How many replies a thread is read to.
const LIMIT: i32 = 100;
fn user_names(users: &[tl::enums::User]) -> HashMap<i64, String> {
users
.iter()
.filter_map(|u| match u {
tl::enums::User::User(u) => {
let name = [u.first_name.clone().unwrap_or_default(), u.last_name.clone().unwrap_or_default()].iter().filter(|s| !s.is_empty()).cloned().collect::<Vec<_>>().join(" ");
Some((u.id, if name.is_empty() { u.username.clone().unwrap_or_else(|| "Deleted account".to_string()) } else { name }))
}
_ => None,
})
.collect()
}
/// One of Telegram's own messages as the window reads a message.
pub fn message_json(message: &tl::enums::Message, names: &HashMap<i64, String>, chat_name: &str, buffer_id: &str, own_name: &str) -> Option<Value> {
let tl::enums::Message::Message(m) = message else { return None };
let from = if m.out {
own_name.to_string()
} else {
match &m.from_id {
Some(tl::enums::Peer::User(u)) => names.get(&u.user_id).cloned().unwrap_or_else(|| "Somebody".to_string()),
_ => chat_name.to_string(),
}
};
let mut body = m.message.clone();
if let Some(label) = m.media.as_ref().and_then(super::messages::media_label) {
if !body.is_empty() {
body.push('\n');
}
body.push_str(&label);
}
Some(json!({
"id": m.id.to_string(),
"bufferId": buffer_id,
"from": from,
"body": body,
"ts": m.date,
"kind": "chat",
"isAction": false,
"isHighlight": false,
"edited": m.edit_date.is_some(),
"isOwn": m.out,
}))
}
/// A thread's replies, oldest first.
pub async fn fetch(state: &AppState, buffer: &Buffer, root_id: &str) -> Result<Vec<Value>, String> {
let root: i32 = root_id.parse().map_err(|_| "that message has no thread".to_string())?;
let (handle, peer) = peer_for(state, buffer).await?;
let answer = handle
.client
.invoke(&tl::functions::messages::GetReplies { peer: peer.into(), msg_id: root, offset_id: 0, offset_date: 0, add_offset: 0, limit: LIMIT, max_id: 0, min_id: 0, hash: 0 })
.await
.map_err(|e| explain(&e))?;
let (messages, users, chats) = match answer {
tl::enums::messages::Messages::Messages(m) => (m.messages, m.users, m.chats),
tl::enums::messages::Messages::Slice(m) => (m.messages, m.users, m.chats),
tl::enums::messages::Messages::ChannelMessages(m) => (m.messages, m.users, m.chats),
tl::enums::messages::Messages::NotModified(_) => (Vec::new(), Vec::new(), Vec::new()),
};
let names = user_names(&users);
let mut out: Vec<Value> = messages
.iter()
.filter_map(|m| {
// A comment is in the discussion group, not the channel: filed under that group's buffer when it is one.
let discussion = match m {
tl::enums::Message::Message(raw) => match &raw.peer_id {
tl::enums::Peer::Channel(c) if buffer.remote_id.as_deref() != Some(&peers::remote_id(PeerId::channel_unchecked(c.channel_id))) => {
state.runtime.list_buffers().into_iter().find(|b| b.account_id == buffer.account_id && b.remote_id.as_deref() == Some(&peers::remote_id(PeerId::channel_unchecked(c.channel_id)))).map(|b| b.id)
}
_ => None,
},
_ => None,
};
let chat = chats.first().map(|c| match c {
tl::enums::Chat::Channel(c) => c.title.clone(),
tl::enums::Chat::Chat(c) => c.title.clone(),
_ => buffer.name.clone(),
});
message_json(m, &names, chat.as_deref().unwrap_or(&buffer.name), discussion.as_deref().unwrap_or(&buffer.id), &handle.own.name)
})
.collect();
out.sort_by_key(|m| m["ts"].as_i64().unwrap_or(0));
Ok(out)
}
/// Says something in a thread: under a channel's post it goes to the discussion group, as an answer to the post's copy
/// there; anywhere else it is an answer to the message the thread grew from.
pub async fn reply(state: &AppState, buffer: &Buffer, root_id: &str, body: &str) -> Result<(), String> {
let post: i32 = root_id.parse().map_err(|_| "that message has no thread".to_string())?;
let input = input_for(body)?;
let (handle, peer) = peer_for(state, buffer).await?;
let found = match handle.client.invoke(&tl::functions::messages::GetDiscussionMessage { peer: peer.into(), msg_id: post }).await {
Ok(tl::enums::messages::DiscussionMessage::Message(found)) => Some(found),
// A message nobody has answered yet has no thread to be found in: it is answered like any other.
Err(_) => None,
};
let target = found.as_ref().and_then(|found| {
let Some(tl::enums::Message::Message(copy)) = found.messages.first() else { return None };
let tl::enums::Peer::Channel(group) = &copy.peer_id else { return None };
let hash = found.chats.iter().find_map(|c| match c {
tl::enums::Chat::Channel(c) if c.id == group.channel_id => c.access_hash,
_ => None,
})?;
Some((PeerRef { id: PeerId::channel_unchecked(group.channel_id), auth: PeerAuth::from_hash(hash) }, copy.id))
});
let (to, reply_to) = target.unwrap_or((peer, post));
let sent = handle.client.send_message(to, input.reply_to(Some(reply_to))).await.map_err(|e| explain(&e))?;
messages::record(state, &buffer.account_id, &handle.own, &sent);
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
fn raw(id: i32, out: bool, from: i64, text: &str) -> tl::enums::Message {
tl::enums::Message::Message(tl::types::Message {
out,
mentioned: false,
media_unread: false,
silent: false,
post: false,
from_scheduled: false,
legacy: false,
edit_hide: false,
pinned: false,
noforwards: false,
invert_media: false,
offline: false,
video_processing_pending: false,
paid_suggested_post_stars: false,
paid_suggested_post_ton: false,
id,
from_id: Some(tl::enums::Peer::User(tl::types::PeerUser { user_id: from })),
from_boosts_applied: None,
from_rank: None,
peer_id: tl::enums::Peer::User(tl::types::PeerUser { user_id: from }),
saved_peer_id: None,
fwd_from: None,
via_bot_id: None,
via_business_bot_id: None,
reply_to: None,
date: 1790000000 + id,
message: text.into(),
media: None,
reply_markup: None,
entities: None,
views: None,
forwards: None,
replies: None,
edit_date: None,
post_author: None,
grouped_id: None,
reactions: None,
restriction_reason: None,
ttl_period: None,
quick_reply_shortcut_id: None,
effect: None,
factcheck: None,
report_delivery_until_date: None,
paid_message_stars: None,
suggested_post: None,
schedule_repeat_period: None,
summary_from_language: None,
guestchat_via_from: None,
rich_message: None,
})
}
#[test]
fn a_reply_is_read_as_a_message_with_its_authors_name() {
let names: HashMap<i64, String> = [(7, "Ann".to_string())].into_iter().collect();
let got = message_json(&raw(5, false, 7, "hello"), &names, "Chat", "telegram:1|Chat", "Me").unwrap();
assert_eq!((got["id"].as_str(), got["from"].as_str(), got["body"].as_str(), got["isOwn"].as_bool()), (Some("5"), Some("Ann"), Some("hello"), Some(false)));
let mine = message_json(&raw(6, true, 1, "mine"), &names, "Chat", "telegram:1|Chat", "Me").unwrap();
assert_eq!((mine["from"].as_str(), mine["isOwn"].as_bool()), (Some("Me"), Some(true)));
assert!(message_json(&tl::enums::Message::Empty(tl::types::MessageEmpty { id: 1, peer_id: None }), &names, "c", "b", "m").is_none());
}
}
src/backend/telegram/topics.rs
+277
View File
@@ -0,0 +1,277 @@
//! Forum topics: a group with Topics switched on is several conversations, each its own place to read and write.
//!
//! Telegram keeps a topic as a message (its first, the "topic root") that every message in the topic answers; the
//! General topic is number 1. Each topic is a conversation of its own here, filed under its group's entry on the rail,
//! named `Group/#Topic`, and its `remote_id` is the group's with the topic after a colon (`-100123:45`). A message is
//! routed to its topic from what it answers; the topics come from the group when the account connects, and a topic
//! made while moho is running is found when its first message arrives.
use super::peers;
use super::Handle;
use crate::model::Buffer;
use crate::state::AppState;
use anyhow::{anyhow, Result};
use grammers_client::peer::Peer;
use grammers_client::tl;
/// The topic every message with no other belongs to.
pub const GENERAL: i32 = 1;
/// Whether a conversation is a group with Topics switched on.
pub fn is_forum(peer: &Peer) -> bool {
matches!(peer, Peer::Group(g) if matches!(&g.raw, tl::enums::Chat::Channel(c) if c.forum))
}
/// A topic's `remote_id`: its group's, then the topic.
pub fn remote_of(group_remote: &str, topic: i32) -> String {
format!("{group_remote}:{topic}")
}
/// The topic a conversation is, when it is one.
pub fn topic_of_buffer(buffer: &Buffer) -> Option<i32> {
buffer.remote_id.as_deref()?.split_once(':')?.1.parse().ok()
}
fn header_topic(header: &tl::enums::MessageReplyHeader) -> Option<i32> {
let tl::enums::MessageReplyHeader::Header(h) = header else { return None };
if !h.forum_topic {
return None;
}
// A message in a topic answers the topic's first message, or - when it answers another - names the topic as well.
h.reply_to_top_id.or(h.reply_to_msg_id)
}
/// The topic a message was said in.
pub fn topic_of(raw: &tl::enums::Message) -> i32 {
match raw {
tl::enums::Message::Message(m) => m.reply_to.as_ref().and_then(header_topic).unwrap_or(GENERAL),
// The message that makes a topic is the topic's own root.
tl::enums::Message::Service(s) if matches!(s.action, tl::enums::MessageAction::TopicCreate(_)) => s.id,
tl::enums::Message::Service(s) => s.reply_to.as_ref().and_then(header_topic).unwrap_or(GENERAL),
tl::enums::Message::Empty(_) => GENERAL,
}
}
fn display_name(group: &str, title: &str) -> String {
format!("{group}/#{}", title.trim())
}
/// The conversation of one topic, made if it is not there yet. `title` is its name where known; without it a topic
/// other than General is called "topic N" until the group is asked.
pub fn buffer_for_topic(state: &AppState, account_id: &str, peer: &Peer, topic: i32, title: Option<&str>) -> Buffer {
let group_remote = peers::remote_id(peer.id());
let remote = remote_of(&group_remote, topic);
let mine: Vec<Buffer> = state.runtime.list_buffers().into_iter().filter(|b| b.account_id == account_id).collect();
let group_title = peers::title_of(peer);
if let Some(found) = mine.iter().find(|b| b.remote_id.as_deref() == Some(remote.as_str())) {
// A name learned since: the conversation takes it.
if let Some(title) = title {
let wanted = display_name(&group_title, title);
if found.name != wanted && !mine.iter().any(|b| b.name == wanted) {
state.runtime.rename_buffer(state, account_id, &found.name, &wanted, "channel");
if let Some(renamed) = state.runtime.get_buffer(&crate::model::buffer_id(account_id, &wanted)) {
return renamed;
}
}
}
return found.clone();
}
let title_here = match (title, topic) {
(Some(t), _) => t.to_string(),
(None, GENERAL) => "General".to_string(),
(None, n) => format!("topic {n}"),
};
let taken: Vec<(String, Option<String>)> = mine.iter().map(|b| (b.name.clone(), b.remote_id.clone())).collect();
let name = peers::unique_name(&taken, &remote, &display_name(&group_title, &title_here), None);
let buffer = state.runtime.ensure_buffer(state, account_id, &name, "channel");
state.runtime.set_buffer_remote_id(state, &buffer.id, &remote);
peers::file_in_rail(state, account_id, &buffer, peer, 1000);
let now = chrono::Utc::now().timestamp();
state.runtime.set_buffer_read_only(state, &buffer.id, peers::read_only_for(peer, now));
// A topic that arrived with no name: the group says what its topics are called.
if title.is_none() && topic != GENERAL {
if let Some(handle) = state.runtime.telegram_handle(account_id) {
let (state, account_id, peer) = (state.clone(), account_id.to_string(), peer.clone());
tokio::spawn(async move {
if let Err(e) = sync(&state, &account_id, &handle, &peer).await {
tracing::debug!("telegram: reading a group's topics: {e:#}");
}
});
}
}
state.runtime.get_buffer(&buffer.id).unwrap_or(buffer)
}
/// Where a message goes, for a message in any conversation: its topic's, in a group with topics.
pub fn buffer_of_message(state: &AppState, account_id: &str, peer: &Peer, raw: &tl::enums::Message) -> Buffer {
if is_forum(peer) {
// A topic made, named in the message that makes it.
let title = match raw {
tl::enums::Message::Service(s) => match &s.action {
tl::enums::MessageAction::TopicCreate(a) if !a.title_missing => Some(a.title.clone()),
_ => None,
},
_ => None,
};
buffer_for_topic(state, account_id, peer, topic_of(raw), title.as_deref())
} else {
peers::buffer_for(state, account_id, peer)
}
}
/// The conversation that holds a message, given only its group's `remote_id`: the group's own, or - in a group with
/// topics - the topic it was said in.
pub fn holding(state: &AppState, account_id: &str, remote: &str, msg_id: i32) -> Option<Buffer> {
let mine: Vec<Buffer> = state.runtime.list_buffers().into_iter().filter(|b| b.account_id == account_id).collect();
if let Some(exact) = mine.iter().find(|b| b.remote_id.as_deref() == Some(remote)) {
return Some(exact.clone());
}
let prefix = format!("{remote}:");
let topics: Vec<&Buffer> = mine.iter().filter(|b| b.remote_id.as_deref().is_some_and(|r| r.starts_with(&prefix))).collect();
topics.iter().find(|b| state.store.get_message(&b.id, &msg_id.to_string()).ok().flatten().is_some()).or(topics.first()).map(|b| (*b).clone())
}
/// The newest message of each topic as Telegram last said, by the topic's conversation: what reading it reads up to,
/// since this client may not have kept the messages that are unread.
fn tops() -> &'static std::sync::Mutex<std::collections::HashMap<String, i32>> {
static T: std::sync::OnceLock<std::sync::Mutex<std::collections::HashMap<String, i32>>> = std::sync::OnceLock::new();
T.get_or_init(Default::default)
}
/// The newest message Telegram has told of in a topic's conversation, if it has.
pub fn top_of(buffer_id: &str) -> Option<i32> {
tops().lock().unwrap_or_else(|e| e.into_inner()).get(buffer_id).copied()
}
/// How many messages in a topic are unread, and how many of those mention this account.
fn counts(topic: &tl::types::ForumTopic) -> (u32, u32) {
(topic.unread_count.max(0) as u32, topic.unread_mentions_count.max(0) as u32)
}
/// Whether a read position reaches everything the topic is known to hold.
pub fn read_all(read_max: i32, newest_here: i32, top: Option<i32>) -> bool {
read_max >= newest_here.max(top.unwrap_or(0))
}
/// Reads the group's topics and makes a conversation of each, named as the group names it. The one conversation the
/// group had before it had topics is removed: its messages are in the topics now.
pub async fn sync(state: &AppState, account_id: &str, handle: &Handle, peer: &Peer) -> Result<usize> {
use grammers_session::Session;
let group_remote = peers::remote_id(peer.id());
let peer_ref = handle.session.peer_ref(peer.id()).await.ok().flatten().ok_or_else(|| anyhow!("Telegram has not told this account about that group yet"))?;
let answer = handle
.client
.invoke(&tl::functions::messages::GetForumTopics { peer: peer_ref.into(), q: None, offset_date: 0, offset_id: 0, offset_topic: 0, limit: 100 })
.await
.map_err(|e| anyhow!("{}", super::send::explain(&e)))?;
let tl::enums::messages::ForumTopics::Topics(found) = answer;
// Gone first, so the topics are not named around it.
for old in state.runtime.list_buffers() {
if old.account_id == account_id && old.remote_id.as_deref() == Some(group_remote.as_str()) {
state.runtime.remove_buffer(state, &old.id);
}
}
let mut count = 0;
for topic in &found.topics {
let tl::enums::ForumTopic::Topic(topic) = topic else { continue };
let buffer = buffer_for_topic(state, account_id, peer, topic.id, Some(&topic.title));
let now = chrono::Utc::now().timestamp();
let reason = if topic.closed { Some("This topic is closed.") } else { peers::read_only_for(peer, now) };
state.runtime.set_buffer_read_only(state, &buffer.id, reason);
// The number Telegram shows beside the topic: a topic never opened here has no messages to count.
let (unread, mentions) = counts(topic);
state.runtime.set_buffer_unread(state, &buffer.id, unread, mentions);
tops().lock().unwrap_or_else(|e| e.into_inner()).insert(buffer.id.clone(), topic.top_message);
count += 1;
}
Ok(count)
}
#[cfg(test)]
mod tests {
use super::*;
fn header(forum_topic: bool, msg: i32, top: Option<i32>) -> tl::enums::MessageReplyHeader {
tl::enums::MessageReplyHeader::Header(tl::types::MessageReplyHeader {
reply_to_scheduled: false,
forum_topic,
quote: false,
reply_to_ephemeral: false,
reply_to_msg_id: Some(msg),
reply_to_peer_id: None,
reply_from: None,
reply_media: None,
reply_to_top_id: top,
quote_text: None,
quote_entities: None,
quote_offset: None,
todo_item_id: None,
poll_option: None,
})
}
#[test]
fn a_message_belongs_to_the_topic_it_answers_or_names() {
assert_eq!(header_topic(&header(true, 45, None)), Some(45), "answering the topic's first message is being in it");
assert_eq!(header_topic(&header(true, 99, Some(45))), Some(45), "answering a message names the topic too");
assert_eq!(header_topic(&header(false, 99, None)), None, "an ordinary reply says nothing of topics");
}
#[test]
fn a_topics_conversation_is_named_by_its_group_and_found_by_its_remote_id() {
assert_eq!(remote_of("-1001234", 45), "-1001234:45");
assert_eq!(display_name("Chasm", " Support "), "Chasm/#Support");
assert_eq!(peers::peer_id_of("-1001234:45"), peers::peer_id_of("-1001234"));
}
#[test]
fn a_topics_unread_numbers_are_what_telegram_counts() {
let mut topic = tl::types::ForumTopic {
my: false,
closed: false,
pinned: false,
short: false,
hidden: false,
title_missing: false,
id: 45,
date: 0,
peer: tl::enums::Peer::Channel(tl::types::PeerChannel { channel_id: 1 }),
title: "Support".into(),
icon_color: 0,
icon_emoji_id: None,
top_message: 90,
read_inbox_max_id: 80,
read_outbox_max_id: 0,
unread_count: 7,
unread_mentions_count: 2,
unread_reactions_count: 0,
unread_poll_votes_count: 0,
from_id: tl::enums::Peer::Channel(tl::types::PeerChannel { channel_id: 1 }),
notify_settings: tl::enums::PeerNotifySettings::Settings(tl::types::PeerNotifySettings {
show_previews: None,
silent: None,
mute_until: None,
ios_sound: None,
android_sound: None,
other_sound: None,
stories_muted: None,
stories_hide_sender: None,
stories_ios_sound: None,
stories_android_sound: None,
stories_other_sound: None,
}),
draft: None,
};
assert_eq!(counts(&topic), (7, 2));
topic.unread_count = -1;
assert_eq!(counts(&topic).0, 0, "a count below nothing is nothing");
}
#[test]
fn a_topic_is_read_when_the_position_reaches_what_is_known_of_it() {
assert!(read_all(90, 85, Some(90)));
assert!(!read_all(80, 85, Some(90)), "the position is short of the newest Telegram has told of");
assert!(!read_all(80, 85, None), "and of the newest kept here");
assert!(read_all(0, 0, None), "a topic holding nothing is read");
}
}
src/backend/telegram/updates.rs
+100
View File
@@ -0,0 +1,100 @@
//! What Telegram sends that is not a message: somebody typing, a conversation read on another of the account's apps.
//!
//! The library turns messages, edits and deletions into updates of their own; everything else arrives as the raw
//! update Telegram sent, which is read here.
use super::peers;
use super::Handle;
use crate::state::AppState;
use grammers_client::tl;
use grammers_session::types::PeerId;
/// How long somebody's "typing" lasts without another: Telegram sends one every few seconds while they type.
const TYPING_FOR_MS: u64 = 6000;
/// The buffer for a conversation named by Telegram's id for it, if there is one.
fn buffer_of(state: &AppState, account_id: &str, remote: &str) -> Option<crate::model::Buffer> {
state.runtime.list_buffers().into_iter().find(|b| b.account_id == account_id && b.remote_id.as_deref() == Some(remote))
}
fn is_typing(action: &tl::enums::SendMessageAction) -> bool {
use tl::enums::SendMessageAction as A;
!matches!(action, A::SendMessageCancelAction | A::SpeakingInGroupCallAction)
}
fn who(handle: &Handle, user: i64) -> Option<String> {
if user == handle.own.user_id {
return None;
}
handle.names.lock().unwrap_or_else(|e| e.into_inner()).get(&user).cloned()
}
fn say_typing(state: &AppState, account_id: &str, buffer: &crate::model::Buffer, name: String) {
state.events.emit("typing", serde_json::json!({ "accountId": account_id, "bufferId": buffer.id, "nicks": [name], "expiresInMs": TYPING_FOR_MS }));
}
/// Marks a conversation read here when Telegram says it was read up to its newest message elsewhere.
fn read_up_to(state: &AppState, account_id: &str, remote: &str, max_id: i32) {
let Some(buffer) = buffer_of(state, account_id, remote) else { return };
let newest = state.store.newest_message_id(&buffer.id).ok().flatten().and_then(|id| id.parse::<i32>().ok()).unwrap_or(0);
if max_id >= newest {
state.events.emit("bufferRead", serde_json::json!({ "accountId": account_id, "bufferId": buffer.id }));
}
}
/// A topic was read on another of the account's apps: up to the message it names.
fn topic_read(state: &AppState, account_id: &str, group: &str, topic: i32, max_id: i32) {
let Some(buffer) = buffer_of(state, account_id, &super::topics::remote_of(group, topic)) else { return };
let newest = state.store.newest_message_id(&buffer.id).ok().flatten().and_then(|id| id.parse::<i32>().ok()).unwrap_or(0);
if super::topics::read_all(max_id, newest, super::topics::top_of(&buffer.id)) {
state.runtime.set_buffer_unread(state, &buffer.id, 0, 0);
state.events.emit("bufferRead", serde_json::json!({ "accountId": account_id, "bufferId": buffer.id }));
}
}
/// The pins of a conversation changed: read them again, which says so to every window.
fn pins_changed(state: &AppState, account_id: &str, _handle: &Handle, remote: &str) {
let Some(buffer) = buffer_of(state, account_id, remote) else { return };
let state = state.clone();
tokio::spawn(async move {
let _ = super::pins::list(&state, &buffer).await;
});
}
/// Handles one raw update.
pub fn raw(state: &AppState, account_id: &str, handle: &Handle, update: &tl::enums::Update) {
use tl::enums::Update as U;
match update {
U::UserTyping(u) if is_typing(&u.action) => {
if let (Some(buffer), Some(name)) = (buffer_of(state, account_id, &u.user_id.to_string()), who(handle, u.user_id)) {
say_typing(state, account_id, &buffer, name);
}
}
U::ChatUserTyping(u) if is_typing(&u.action) => {
let user = PeerId::from(u.from_id.clone()).bare_id();
if let (Some(buffer), Some(name)) = (buffer_of(state, account_id, &peers::remote_id(PeerId::chat_unchecked(u.chat_id))), user.and_then(|id| who(handle, id))) {
say_typing(state, account_id, &buffer, name);
}
}
U::ChannelUserTyping(u) if is_typing(&u.action) => {
let user = PeerId::from(u.from_id.clone()).bare_id();
// In a group with topics somebody types in one of them.
let group = peers::remote_id(PeerId::channel_unchecked(u.channel_id));
let found = buffer_of(state, account_id, &group).or_else(|| buffer_of(state, account_id, &super::topics::remote_of(&group, u.top_msg_id.unwrap_or(super::topics::GENERAL))));
if let (Some(buffer), Some(name)) = (found, user.and_then(|id| who(handle, id))) {
say_typing(state, account_id, &buffer, name);
}
}
U::MessageReactions(u) => super::reactions::update(state, account_id, &peers::remote_id(PeerId::from(u.peer.clone())), u.msg_id, &u.reactions),
U::MessagePoll(u) => super::polls::update(state, u),
U::NotifySettings(u) => super::notify::changed(state, account_id, u),
U::PinnedMessages(u) => pins_changed(state, account_id, handle, &peers::remote_id(PeerId::from(u.peer.clone()))),
U::PinnedChannelMessages(u) => pins_changed(state, account_id, handle, &peers::remote_id(PeerId::channel_unchecked(u.channel_id))),
U::ReadHistoryInbox(u) => read_up_to(state, account_id, &peers::remote_id(PeerId::from(u.peer.clone())), u.max_id),
// The account's folders were made, changed, deleted or put in another order, on this client or another.
U::DialogFilter(_) | U::DialogFilters | U::DialogFilterOrder(_) => super::folders::changed(state, account_id, handle),
U::ReadChannelDiscussionInbox(u) => topic_read(state, account_id, &peers::remote_id(PeerId::channel_unchecked(u.channel_id)), u.top_msg_id, u.read_max_id),
U::ReadChannelInbox(u) => read_up_to(state, account_id, &peers::remote_id(PeerId::channel_unchecked(u.channel_id)), u.max_id),
_ => {}
}
}
src/backend/twitch/chat.rs
+1
View File
@@ -345,6 +345,7 @@ pub fn reply_of(line: &Line, text: &str) -> (Option<ReplyPreview>, String) {
thread: false,
forwarded: false,
thread_root: Some(root.to_string()),
chat: None,
}),
body,
)
src/backend/twitch/connect.rs
+1 -1
View File
@@ -813,7 +813,7 @@ where
let reply = match (&parent, reply_to) {
(Some(p), _) => {
let root = p.reply_to.as_ref().and_then(|r| r.thread_root.clone()).unwrap_or_else(|| p.id.clone());
Some(crate::model::ReplyPreview { id: p.id.clone(), from: p.from.clone(), body: p.body.clone(), thread: false, forwarded: false, thread_root: Some(root) })
Some(crate::model::ReplyPreview { id: p.id.clone(), from: p.from.clone(), body: p.body.clone(), thread: false, forwarded: false, thread_root: Some(root), chat: None })
}
_ => None,
};
src/main.rs
+3
View File
@@ -343,6 +343,9 @@ async fn run() -> Result<()> {
for cfg in state.accounts.all_twitch() {
backend::twitch::spawn(state.clone(), cfg);
}
for cfg in state.accounts.all_telegram() {
backend::telegram::connect::spawn(state.clone(), cfg);
}
// What was going on last time, before anything is served: a window that
// connects immediately should see the same list it was looking at.
src/media_cache.rs
+9
View File
@@ -38,6 +38,8 @@ pub const CACHES: &[Cache] = &[
Cache { dir: "matrix-media", group: "media", max_bytes: 250 * MB },
Cache { dir: "sneedchat-attachments", group: "media", max_bytes: 250 * MB },
Cache { dir: "discord-thumbnails", group: "media", max_bytes: 100 * MB },
Cache { dir: "telegram-media", group: "media", max_bytes: 500 * MB },
Cache { dir: "telegram-avatars", group: "avatars", max_bytes: 64 * MB },
Cache { dir: "sneedchat-avatars", group: "avatars", max_bytes: 100 * MB },
Cache { dir: "discord-icons", group: "avatars", max_bytes: 16 * MB },
Cache { dir: "kick-emotes", group: "emotes", max_bytes: 64 * MB },
@@ -209,6 +211,13 @@ pub async fn restore(state: &AppState, path: &str, buffer_id: Option<&str>, mess
}
"discord-icons" => crate::backend::discord::restore_guild_icon(&path).await?,
"kick-emotes" => crate::backend::kick::emotecache::restore_cached(&path).await?,
"telegram-media" | "telegram-avatars" => match crate::backend::telegram::media::restore_cached(state, &path).await {
Ok(path) => path,
Err(e) => {
tracing::warn!("telegram: fetching {} again: {e:#}", path.display());
return Err(e);
}
},
_ => bail!("{} is fetched again by what uses it, not by name", cache.dir),
};
Ok(format!("file://{}", restored.display()))
src/model.rs
+27
View File
@@ -379,6 +379,10 @@ pub struct Buffer {
/// thing - so it is named for what it is rather than after the tag.
#[serde(rename = "serviceRoom", default, skip_serializing_if = "std::ops::Not::not")]
pub service_room: bool,
/// A small glyph drawn in front of the name in the list, for what kind of conversation it is where the service
/// has kinds a person looks for: Telegram's broadcast channels ("campaign", a horn) and bots ("smart_toy").
#[serde(rename = "nameIcon", default, skip_serializing_if = "Option::is_none")]
pub name_icon: Option<String>,
/// What the conversation says it is for: an IRC channel's topic, a Matrix
/// room's, a Discord channel's. Absent when there is none.
#[serde(default, skip_serializing_if = "Option::is_none")]
@@ -391,6 +395,18 @@ pub struct Buffer {
/// posts, each of which is a thread. Opened as a pane of its own.
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
pub forum: bool,
/// How many messages the service itself counts as unread here, where it keeps a count a client could not work
/// out from the messages it has: a Telegram topic of a group that was never opened here. A starting point for
/// the window's own count, which then follows live messages; absent when there is none.
#[serde(rename = "unreadSeed", default, skip_serializing_if = "is_zero")]
pub unread_seed: u32,
/// How many of those mention this account.
#[serde(rename = "mentionsSeed", default, skip_serializing_if = "is_zero")]
pub mentions_seed: u32,
}
fn is_zero(n: &u32) -> bool {
*n == 0
}
/// The rail entry a buffer belongs to when its protocol has no grouping of
@@ -482,6 +498,12 @@ pub struct ReplyPreview {
/// above it, the second for a thread to open.
#[serde(rename = "threadRoot", default, skip_serializing_if = "Option::is_none")]
pub thread_root: Option<String>,
/// The conversation the message answered lives in, when it is not this one.
///
/// Telegram can answer a post in another channel; the quote then names where it came from, and a client can
/// take the reader there. A buffer id, present only when that conversation is one the account has open.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub chat: Option<String>,
}
/// Why a message the service took down is gone, kept with the message.
@@ -910,6 +932,7 @@ pub fn service_of(account_id: &str) -> &'static str {
Some("sneedchat") => "sneedchat",
Some("steam") => "steam",
Some("twitch") => "twitch",
Some("telegram") => "telegram",
Some("jabber") => "jabber",
Some("slack") => "slack",
_ => "irc",
@@ -927,6 +950,7 @@ mod service_tests {
assert_eq!(service_of("kick:someone"), "kick");
assert_eq!(service_of("sneedchat:Ancient Pioneer"), "sneedchat");
assert_eq!(service_of("steam:someone"), "steam");
assert_eq!(service_of("telegram:1234567890"), "telegram", "it was read as IRC, the service with no prefix");
// IRC is the one without a prefix, and a host with a port in it must
// not be read as one.
assert_eq!(service_of("[email protected]"), "irc");
@@ -1032,9 +1056,12 @@ mod tests {
favourite: false,
low_priority: false,
service_room: false,
name_icon: None,
topic: None,
read_only: None,
forum: false,
unread_seed: 0,
mentions_seed: 0,
}
}
src/rpc/methods.rs
+390 -2
View File
@@ -112,6 +112,7 @@ fn adding_service(method: &str) -> Option<&'static str> {
"addKickAccount" => Some("kick"),
"addSteamAccount" | "addSteamAccountQr" => Some("steam"),
"addTwitchAccount" | "startTwitchLogin" => Some("twitch"),
"addTelegramAccount" => Some("telegram"),
_ => None,
}
}
@@ -426,6 +427,7 @@ pub async fn dispatch(
{ "id": "kick", "name": "Kick", "available": true },
{ "id": "steam", "name": "Steam", "available": true },
{ "id": "twitch", "name": "Twitch", "available": true },
{ "id": "telegram", "name": "Telegram", "available": true },
])),
None,
),
@@ -471,7 +473,13 @@ pub async fn dispatch(
// connect: a few hundred channels would mean a few
// hundred requests, most for conversations nobody
// is about to read.
backend::discord::catch_up_channel(state, &cfg.token, &cfg.user_id, cfg.display_name.as_deref(), buffer_id, &channel_id).await;
// A conversation never read here is read for the first time now; one that has been is asked
// only about what came after.
if state.store.has_messages(buffer_id).unwrap_or(true) {
backend::discord::catch_up_channel(state, &cfg.token, &cfg.user_id, cfg.display_name.as_deref(), buffer_id, &channel_id).await;
} else {
backend::discord::backfill_channel_history(state, &cfg.token, &cfg.user_id, cfg.display_name.as_deref(), buffer_id, &channel_id).await;
}
}
}
// Steam keeps every conversation: opening one asks for its
@@ -490,6 +498,14 @@ pub async fn dispatch(
tracing::debug!("steam history: {e}");
}
}
// Telegram keeps every message too: opening a conversation asks for its recent end, scrolling to
// the top for the part before. Best-effort, like Steam's.
if buffer.account_id.starts_with("telegram:") {
let fetched = if before > 0 { backend::telegram::history::older(state, &buffer, before).await } else { backend::telegram::history::open(state, &buffer).await };
if let Err(e) = fetched {
tracing::debug!("telegram history: {e:#}");
}
}
// Kick pages back through its own history endpoint,
// continuing from where the last page stopped rather than
// by timestamp - the endpoint is cursor-based, and asking
@@ -746,6 +762,10 @@ pub async fn dispatch(
// name later.
state.highlights.forget(id);
state.ignores.forget(id);
// And the Discord session kept for a restart to resume: a removed account must not leave its
// servers and friends in a file.
backend::discord::session_store::forget(id);
backend::discord::send_guard::forget_account(id);
// And everything else this process is holding about it: the
// seventy-odd maps keyed by the account or by one of its
// buffers. In memory only, so it mattered exactly when
@@ -781,8 +801,15 @@ pub async fn dispatch(
// hours and a refresh token good for thirty days. Best effort:
// the account is removed either way.
let twitch_tokens = state.accounts.get_twitch(id).filter(|a| a.is_signed_in()).map(|a| (a.access_token, a.refresh_token));
// A Telegram session is ended at Telegram too, and its file taken away.
let telegram_account = state.accounts.get_telegram(id);
match state.accounts.remove(id) {
Ok(true) => {
if let Some(config) = telegram_account {
backend::telegram::folders::forget(&config.account_id());
let state = state.clone();
tokio::spawn(async move { backend::telegram::connect::sign_out(&state, &config).await });
}
if let Some((access, refresh)) = twitch_tokens {
tokio::spawn(async move {
if let Ok(http) = backend::twitch::client() {
@@ -1012,6 +1039,12 @@ pub async fn dispatch(
// Twitch is the same shape: a channel is a streamer, naming one is
// the whole of joining it, and the connection remembers it once
// Twitch has said it is there.
if id.starts_with("telegram:") {
return match backend::telegram::membership::join(state, id, name).await {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(e)),
};
}
if id.starts_with("twitch:") {
let Some(channel) = backend::twitch::connect::normalise_login(name) else {
return (None, Some(format!("\"{name}\" is not a Twitch channel")));
@@ -1080,6 +1113,15 @@ pub async fn dispatch(
}
(Some(ok_node()), None)
}
Some(buffer) if buffer.account_id.starts_with("telegram:") => {
// It lapses by itself, so only "typing" is said.
if typing {
if let Err(e) = backend::telegram::send::typing(state, &buffer).await {
tracing::debug!("sendTyping: {e}");
}
}
(Some(ok_node()), None)
}
// IRC says it with a client-only tag on an empty TAGMSG,
// and says nothing at all where the server has no
// message-tags to carry one - see send_typing.
@@ -1144,6 +1186,27 @@ pub async fn dispatch(
// Muting a room for the account rather than for this window. Matrix
// keeps it server-side, so a room quietened here is quiet on a phone
// too - which is what somebody means by muting a room.
// The article a Telegram link preview carries, as a document for a window of its own.
"telegramInstantView" => {
let Some(buffer) = p_str_opt(params, "bufferId").and_then(|b| state.runtime.get_buffer(b)).filter(|b| b.account_id.starts_with("telegram:")) else {
return (None, Some("telegramInstantView requires a Telegram \"bufferId\"".to_string()));
};
match backend::telegram::instant::article(state, &buffer, p_str(params, "messageId", "")).await {
Ok(article) => (Some(serde_json::json!({ "title": article.title, "url": article.url, "html": article.html })), None),
Err(e) => (None, Some(e)),
}
}
"setTelegramChatMuted" => {
let Some(buffer) = p_str_opt(params, "bufferId").and_then(|b| state.runtime.get_buffer(b)).filter(|b| b.account_id.starts_with("telegram:")) else {
return (None, Some("setTelegramChatMuted requires a Telegram \"bufferId\"".to_string()));
};
match backend::telegram::notify::set_muted(state, &buffer, p_bool(params, "muted", true)).await {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(e)),
}
}
"setMatrixRoomMuted" => {
let Some(buffer_id) = p_str_opt(params, "bufferId") else {
return (None, Some("setMatrixRoomMuted requires \"bufferId\"".to_string()));
@@ -1232,6 +1295,7 @@ pub async fn dispatch(
"kick" => backend::kick::profile(&state, &account_id, &buffer_id, &who).await,
"sneedchat" => backend::sneedchat::profile(&state, &account_id, &buffer_id, &who),
"steam" => backend::steam::profile::build(&state, &account_id, &buffer_id, &who, &user_id).await,
"telegram" => backend::telegram::profile::build(&state, &account_id, &buffer_id, &who, &user_id).await,
// A service with nothing to add still answers, so the
// card stops waiting and shows the name.
other => {
@@ -1257,6 +1321,12 @@ pub async fn dispatch(
Ok(messages) => messages,
Err(e) => return (None, Some(format!("{e:#}"))),
}
} else if account_id.starts_with("telegram:") {
let Some(buffer) = state.runtime.get_buffer(buffer_id) else { return (None, Some("no such conversation".to_string())) };
return match backend::telegram::threads::fetch(state, &buffer, root_id).await {
Ok(messages) => (Some(serde_json::json!({ "messages": messages })), None),
Err(e) => (None, Some(e)),
};
} else {
state.store.thread_messages(buffer_id, root_id).unwrap_or_default()
};
@@ -1307,6 +1377,13 @@ pub async fn dispatch(
}
(Some(ok_node()), None)
}
// Telegram keeps a read position for each conversation too, and tells the account's other apps.
Some(buffer) if buffer.account_id.starts_with("telegram:") => {
if let Err(e) = backend::telegram::send::mark_read(state, &buffer).await {
tracing::debug!("telegram read position: {e}");
}
(Some(ok_node()), None)
}
// Steam keeps a read position for each conversation, and tells
// every other client of the account when it moves.
Some(buffer) if buffer.account_id.starts_with("steam:") => {
@@ -1356,7 +1433,8 @@ pub async fn dispatch(
Err(e) => (None, Some(format!("{e:#}"))),
}
} else {
match backend::discord::accept_member_verification(state, account_id, guild_id).await {
let answers: Vec<Value> = params.get("responses").and_then(|v| v.as_array()).cloned().unwrap_or_default();
match backend::discord::accept_member_verification(state, account_id, guild_id, &answers).await {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(format!("{e:#}"))),
}
@@ -1461,6 +1539,8 @@ pub async fn dispatch(
&finished.waveform,
)
.await
} else if buffer.account_id.starts_with("telegram:") {
backend::telegram::send::send_voice(state, &buffer, &finished.path, finished.duration_secs).await.map_err(|e| anyhow::anyhow!(e))
} else if let Some(cfg) = state.accounts.get_matrix(&buffer.account_id) {
backend::matrix::send_voice_message(
state,
@@ -1820,6 +1900,15 @@ pub async fn dispatch(
false
}
}
Some(buffer) if buffer.account_id.starts_with("telegram:") => {
let (state, buffer) = (state.clone(), buffer);
tokio::spawn(async move {
if let Err(e) = backend::telegram::members::refresh(&state, &buffer).await {
tracing::debug!("telegram members: {e}");
}
});
true
}
_ => backend::discord::request_member_list(state, buffer_id),
};
(Some(serde_json::json!({ "requested": asked })), None)
@@ -2197,6 +2286,29 @@ pub async fn dispatch(
Err(e) => tracing::debug!("discord: listing commands: {e:#}"),
}
}
// A Telegram bot's own menu, in a conversation with that bot: what its profile lists, as Telegram's apps
// offer it beside the message box and while a slash is typed.
if service == "telegram" {
// In a conversation with a bot only the bot's own commands are offered: this client's built-ins are
// things said to a chat, and a bot is not asked to /shrug.
if buffer.name_icon.as_deref() == Some("smart_toy") {
out.clear();
}
match backend::telegram::bots::commands(state, &buffer).await {
Ok(list) => {
for c in list.into_iter().filter(|c| query.is_empty() || crate::commands::score(query, &c.name).is_some()) {
out.push(serde_json::json!({
"name": c.name,
"usage": "",
"description": c.description,
"source": c.source.clone().unwrap_or_else(|| buffer.name.clone()),
"kind": "bot",
}));
}
}
Err(e) => tracing::debug!("telegram: listing a bot's commands: {e}"),
}
}
(Some(serde_json::json!({ "commands": out })), None)
}
@@ -2300,6 +2412,8 @@ pub async fn dispatch(
backend::discord::load_context(state, &buffer.account_id, buffer_id, message_id).await
} else if buffer.account_id.starts_with("matrix:") {
backend::matrix::load_context(state, &buffer.account_id, buffer_id, message_id).await
} else if buffer.account_id.starts_with("telegram:") {
backend::telegram::history::context(state, &buffer, message_id).await
} else {
Err(anyhow::anyhow!("this service cannot be asked for one message"))
};
@@ -2340,6 +2454,8 @@ pub async fn dispatch(
backend::discord::load_newer(state, &buffer.account_id, buffer_id, message_id).await
} else if buffer.account_id.starts_with("matrix:") {
backend::matrix::load_newer(state, &buffer.account_id, buffer_id, message_id).await
} else if buffer.account_id.starts_with("telegram:") {
backend::telegram::history::newer(state, &buffer, message_id).await
} else {
Err(anyhow::anyhow!("this service cannot be read forward"))
};
@@ -2365,6 +2481,8 @@ pub async fn dispatch(
backend::discord::list_pinned(state, &buffer.account_id, buffer_id).await
} else if buffer.account_id.starts_with("matrix:") {
backend::matrix::list_pinned(state, &buffer.account_id, buffer_id).await
} else if buffer.account_id.starts_with("telegram:") {
backend::telegram::pins::list(state, &buffer).await.map_err(|e| anyhow::anyhow!(e))
} else {
Err(anyhow::anyhow!("this service has no pinned messages"))
};
@@ -2414,6 +2532,8 @@ pub async fn dispatch(
backend::discord::set_pinned(state, &buffer.account_id, buffer_id, message_id, pinned).await
} else if buffer.account_id.starts_with("matrix:") {
backend::matrix::set_pinned(state, &buffer.account_id, buffer_id, message_id, pinned).await
} else if buffer.account_id.starts_with("telegram:") {
backend::telegram::pins::set(state, &buffer, message_id, pinned).await.map_err(|e| anyhow::anyhow!(e))
} else {
Err(anyhow::anyhow!("this service has no pinned messages"))
};
@@ -2423,6 +2543,25 @@ pub async fn dispatch(
}
}
// The same question asked of Telegram's own index, for what this client never read.
"searchTelegramMessages" => {
let (buffer_id, query) = match (p_str_opt(params, "bufferId"), p_str_opt(params, "query")) {
(Some(b), Some(q)) => (b, q),
_ => return (None, Some("searchTelegramMessages requires \"bufferId\" and \"query\"".to_string())),
};
if query.trim().is_empty() {
return (Some(serde_json::json!({ "results": [], "roomNames": {}, "count": 0 })), None);
}
let Some(buffer) = state.runtime.get_buffer(buffer_id).filter(|b| b.account_id.starts_with("telegram:")) else {
return (None, Some("that conversation is not on Telegram".to_string()));
};
let account_wide = p_str_opt(params, "scope") == Some("account");
match backend::telegram::search::run(state, &buffer, query, account_wide, p_i64(params, "limit", 25) as usize).await {
Ok(answer) => (Some(answer), None),
Err(e) => (None, Some(e)),
}
}
// The same question asked of the homeserver rather than of this
// window's own copy.
//
@@ -3222,6 +3361,20 @@ pub async fn dispatch(
let _ = sender.send(backend::twitch::Command::Part(buffer.name.clone()));
}
let _ = state.accounts.mutate_twitch(&buffer.account_id, |a| a.channels.retain(|c| *c != buffer.name));
} else if buffer.account_id.starts_with("telegram:") && backend::telegram::topics::topic_of_buffer(&buffer).is_some() {
// Closing one topic of a group only stops showing it: the group is left from its rail entry.
} else if buffer.account_id.starts_with("telegram:") {
if let Err(e) = backend::telegram::membership::leave(state, &buffer).await {
return (None, Some(e));
}
// A group's own entry on the rail goes with it.
if let Some(remote) = buffer.remote_id.as_deref().and_then(|r| r.split(':').next()) {
let rail = backend::telegram::peers::rail_id(&buffer.account_id, remote);
if buffer.group_id.as_deref() == Some(rail.as_str()) {
state.runtime.remove_buffer_group(state, &rail);
return (Some(ok_node()), None);
}
}
} else if buffer.account_id.starts_with("matrix:") {
if let Some(room_id) = state.runtime.get_matrix_room(buffer_id) {
if let Err(e) = backend::matrix::leave_room(state, &buffer.account_id, &room_id).await {
@@ -3304,6 +3457,14 @@ pub async fn dispatch(
backend::discord::leave_guild(state, account_id, guild_id).await
} else if let Some(room_id) = rest.strip_prefix("space:") {
backend::matrix::leave_room(state, account_id, room_id).await
} else if rest.starts_with("tggroup:") {
// A Telegram group is its rail entry and the one conversation under it: leaving is leaving that.
let under: Vec<crate::model::Buffer> = state.runtime.list_buffers().into_iter().filter(|b| b.group_id.as_deref() == Some(group_id)).collect();
// One leave covers the group, however many topics it has.
match under.first() {
Some(buffer) => backend::telegram::membership::leave(state, buffer).await.map_err(anyhow::Error::msg),
None => Ok(()),
}
} else {
return (None, Some("that is not something you can leave".to_string()));
};
@@ -3414,6 +3575,30 @@ pub async fn dispatch(
}
}
},
Some(buffer) if buffer.account_id.starts_with("telegram:") && params.get("thread").and_then(|v| v.as_bool()).unwrap_or(false) && attachment_path.is_none() && reply_to_id.is_some() => {
match backend::telegram::threads::reply(state, &buffer, reply_to_id.unwrap_or_default(), body).await {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(e)),
}
}
Some(buffer) if buffer.account_id.starts_with("telegram:") => {
let paths: Vec<String> = match params.get("attachmentPaths").and_then(|v| v.as_array()) {
Some(list) => list.iter().filter_map(|v| v.as_str().map(str::to_string)).collect(),
None => attachment_path.map(|p| vec![p.to_string()]).unwrap_or_default(),
};
let spoilers: Vec<&str> = params.get("spoilerPaths").and_then(|v| v.as_array()).map(|list| list.iter().filter_map(|v| v.as_str()).collect()).unwrap_or_default();
let files: Vec<(String, bool)> = paths.iter().map(|p| (p.clone(), spoilers.contains(&p.as_str()))).collect();
let sent = backend::telegram::send::send_files(state, &buffer, body, &files, reply_to_id, progress).await;
if let Some(p) = progress {
if !files.is_empty() {
p.done(sent.as_ref().err().map(String::as_str));
}
}
match sent {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(e)),
}
}
Some(buffer) if buffer.account_id.starts_with("steam:") => {
// Steam has no replies or edits for a direct message in
// this client yet: what arrives that it cannot use is
@@ -3610,6 +3795,10 @@ pub async fn dispatch(
Err(e) => (None, Some(format!("{e:#}"))),
},
},
Some(buffer) if buffer.account_id.starts_with("telegram:") => match backend::telegram::send::edit(state, &buffer, msg_id, body).await {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(e)),
},
Some(buffer) if buffer.account_id.starts_with("sneedchat:") => match backend::sneedchat::edit_message(state, &buffer.account_id, &buffer.name, msg_id, body) {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(format!("{e:#}"))),
@@ -3650,6 +3839,10 @@ pub async fn dispatch(
Err(e) => (None, Some(format!("{e:#}"))),
},
},
Some(buffer) if buffer.account_id.starts_with("telegram:") => match backend::telegram::reactions::toggle(state, &buffer, msg_id, emoji, add).await {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(e)),
},
Some(buffer) if buffer.account_id.starts_with("steam:") && backend::steam::chatgroups::is_room(&buffer) => match backend::steam::chatgroups::react(state, &buffer, msg_id, emoji, add).await {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(e)),
@@ -3676,6 +3869,14 @@ pub async fn dispatch(
Err(e) => (None, Some(format!("{e:#}"))),
},
},
Some(buffer) if buffer.account_id.starts_with("telegram:") => {
// For everyone unless the window says it is only to be removed from this account's view.
let everyone = params.get("forEveryone").and_then(|v| v.as_bool()).unwrap_or(true);
match backend::telegram::send::delete(state, &buffer, msg_id, everyone).await {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(e)),
}
}
Some(buffer) if buffer.account_id.starts_with("kick:") => {
match (kick_credential(state, &buffer.account_id), state.runtime.kick_channel(buffer_id)) {
(Err(e), _) => (None, Some(e)),
@@ -4419,6 +4620,52 @@ pub async fn dispatch(
}
}
// Signing in to Telegram takes place over time - Telegram sends a code, and may then ask for the account's
// cloud password - so this answers with an id and the rest is events: see backend/telegram/login.rs.
"addTelegramAccount" => {
let (Some(api_id), Some(api_hash), Some(phone)) = (params.get("apiId").and_then(|v| v.as_i64().or_else(|| v.as_str().and_then(|s| s.trim().parse().ok()))), p_str_opt(params, "apiHash"), p_str_opt(params, "phone")) else {
return (None, Some("addTelegramAccount requires \"apiId\", \"apiHash\" and \"phone\"".to_string()));
};
let Ok(api_id) = i32::try_from(api_id) else {
return (None, Some("that api_id is not a number Telegram would have given".to_string()));
};
let login_id = format!("telegram-{}", crate::model::next_message_id());
backend::telegram::login::start(state.clone(), login_id.clone(), api_id, api_hash.trim().to_string(), phone.to_string());
(Some(serde_json::json!({ "loginId": login_id })), None)
}
// The same sign-in without a number: a QR code to scan with Telegram on a phone that is signed in already.
"addTelegramAccountQr" => {
let (Some(api_id), Some(api_hash)) = (params.get("apiId").and_then(|v| v.as_i64().or_else(|| v.as_str().and_then(|s| s.trim().parse().ok()))), p_str_opt(params, "apiHash")) else {
return (None, Some("addTelegramAccountQr requires \"apiId\" and \"apiHash\"".to_string()));
};
let Ok(api_id) = i32::try_from(api_id) else {
return (None, Some("that api_id is not a number Telegram would have given".to_string()));
};
let login_id = format!("telegram-{}", crate::model::next_message_id());
backend::telegram::login::start_qr(state.clone(), login_id.clone(), api_id, api_hash.trim().to_string());
(Some(serde_json::json!({ "loginId": login_id })), None)
}
// The code Telegram sent, and the cloud password it may ask for next, for a sign-in that is waiting.
"submitTelegramCode" | "submitTelegramPassword" => {
let (Some(login_id), Some(value)) = (p_str_opt(params, "loginId"), p_str_opt(params, "value")) else {
return (None, Some(format!("{method} requires \"loginId\" and \"value\"")));
};
let given = if method == "submitTelegramCode" { backend::telegram::login::submit_code(login_id, value) } else { backend::telegram::login::submit_password(login_id, value) };
match given {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(format!("{e:#}"))),
}
}
"cancelTelegramLogin" => {
if let Some(login_id) = p_str_opt(params, "loginId") {
backend::telegram::login::cancel(login_id);
}
(Some(ok_node()), None)
}
// The form was closed with a sign-in still waiting.
"cancelSteamLogin" => {
if let Some(login_id) = p_str_opt(params, "loginId") {
@@ -4951,6 +5198,12 @@ pub async fn dispatch(
// running tally is what most people mean by a poll, and a hidden
// one is the deliberate choice.
let disclosed = params.get("disclosed").and_then(Value::as_bool).unwrap_or(true);
if buffer.account_id.starts_with("telegram:") {
return match backend::telegram::polls::start(state, &buffer, question, &answers, disclosed).await {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(e)),
};
}
match backend::matrix::start_poll(state, &buffer.account_id, buffer_id, question, &answers, disclosed).await {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(format!("{e:#}"))),
@@ -5001,6 +5254,15 @@ pub async fn dispatch(
};
}
}
if channel_account(state, buffer_id).starts_with("telegram:") {
let (Some(choice), Some(buffer)) = (p_str_opt(params, "answerId"), state.runtime.get_buffer(buffer_id)) else {
return (None, Some("votePoll requires \"answerId\"".to_string()));
};
return match backend::telegram::polls::vote(state, &buffer, choice).await {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(e)),
};
}
if channel_account(state, buffer_id).starts_with("twitch:") {
// Twitch's choices are named by ids of their own, which the window hands back as text.
let Some(choice) = p_str_opt(params, "answerId") else {
@@ -5533,6 +5795,132 @@ pub async fn dispatch(
// bufferId (not roomId) throughout, matching every other message-
// scoped Matrix method - the buffer->room_id lookup happens inside
// moderation.rs itself, same as sendMessage/editMessage/etc.
// The GIFs a Telegram account can send: its saved ones, or what Telegram's GIF search finds; and sending one.
"listTelegramGifs" | "sendTelegramGif" => {
let Some(buffer) = p_str_opt(params, "bufferId").and_then(|b| state.runtime.get_buffer(b)).filter(|b| b.account_id.starts_with("telegram:")) else {
return (None, Some(format!("{method} requires a Telegram \"bufferId\"")));
};
if method == "listTelegramGifs" {
return match backend::telegram::gifs::list(state, &buffer, p_str(params, "query", "")).await {
Ok(list) => (Some(list), None),
Err(e) => (None, Some(e)),
};
}
match backend::telegram::gifs::send(state, &buffer, p_str(params, "id", "")).await {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(e)),
}
}
// A t.me or tg:// link followed: the conversation it points into (joined if need be) and the message to go to.
"openTelegramLink" => {
let Some(url) = p_str_opt(params, "url") else {
return (None, Some("openTelegramLink requires \"url\"".to_string()));
};
match backend::telegram::links::open(state, p_str_opt(params, "accountId"), url).await {
Ok(opened) => (Some(backend::telegram::links::answer(&opened)), None),
Err(e) => (None, Some(e)),
}
}
// The stickers a Telegram account can send, and sending one.
"listTelegramStickers" | "sendTelegramSticker" => {
let Some(buffer) = p_str_opt(params, "bufferId").and_then(|b| state.runtime.get_buffer(b)).filter(|b| b.account_id.starts_with("telegram:")) else {
return (None, Some(format!("{method} requires a Telegram \"bufferId\"")));
};
if method == "listTelegramStickers" {
return match backend::telegram::stickers::list(state, &buffer).await {
Ok(list) => (Some(list), None),
Err(e) => (None, Some(e)),
};
}
match backend::telegram::stickers::send(state, &buffer, p_str(params, "id", "")).await {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(e)),
}
}
// An account's chat folders, which the window shows as categories on the account's page, and the changes it
// makes to them: see backend/telegram/folders.rs.
"telegramFolders" => match p_str_opt(params, "accountId") {
Some(account_id) if account_id.starts_with("telegram:") => (Some(backend::telegram::folders::current(account_id)), None),
_ => (None, Some("telegramFolders requires a Telegram \"accountId\"".to_string())),
},
"createTelegramFolder" | "fileTelegramChat" => {
let Some(buffer) = p_str_opt(params, "bufferId").and_then(|b| state.runtime.get_buffer(b)).filter(|b| b.account_id.starts_with("telegram:")) else {
return (None, Some(format!("{method} requires a Telegram \"bufferId\"")));
};
let done = if method == "createTelegramFolder" {
backend::telegram::folders::create(state, &buffer, p_str(params, "name", "")).await
} else {
let to = params.get("folderId").and_then(|v| v.as_i64()).and_then(|n| i32::try_from(n).ok());
backend::telegram::folders::file(state, &buffer, to).await
};
match done {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(e)),
}
}
"renameTelegramFolder" | "deleteTelegramFolder" | "reorderTelegramFolders" => {
let Some(account_id) = p_str_opt(params, "accountId").filter(|a| a.starts_with("telegram:")) else {
return (None, Some(format!("{method} requires a Telegram \"accountId\"")));
};
let folder = params.get("folderId").and_then(|v| v.as_i64()).and_then(|n| i32::try_from(n).ok());
let done = match (method, folder) {
("renameTelegramFolder", Some(id)) => backend::telegram::folders::rename(state, account_id, id, p_str(params, "name", "")).await,
("deleteTelegramFolder", Some(id)) => backend::telegram::folders::delete(state, account_id, id).await,
("reorderTelegramFolders", _) => {
let order: Vec<i32> = params.get("order").and_then(|v| v.as_array()).map(|a| a.iter().filter_map(|n| n.as_i64().and_then(|n| i32::try_from(n).ok())).collect()).unwrap_or_default();
backend::telegram::folders::reorder(state, account_id, &order).await
}
_ => Err(format!("{method} requires a \"folderId\"")),
};
match done {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(e)),
}
}
// Lets a Discord connection that is being held for having logged in too often through now.
"connectDiscordAnyway" => {
let Some(account_id) = p_str_opt(params, "accountId").filter(|a| a.starts_with("discord:")) else {
return (None, Some("connectDiscordAnyway requires a Discord \"accountId\"".to_string()));
};
if backend::discord::identify_budget::connect_anyway(account_id) {
(Some(ok_node()), None)
} else {
(None, Some("That account is not being held".to_string()))
}
}
// Pressing a button under a bot's message; answers what the bot said back, for the window to show.
"pressTelegramButton" => {
let Some(buffer) = p_str_opt(params, "bufferId").and_then(|b| state.runtime.get_buffer(b)).filter(|b| b.account_id.starts_with("telegram:")) else {
return (None, Some("pressTelegramButton requires a Telegram \"bufferId\"".to_string()));
};
match backend::telegram::keyboards::press(state, &buffer, p_str(params, "messageId", ""), p_str(params, "customId", "")).await {
Ok(answer) => (Some(answer), None),
Err(e) => (None, Some(e)),
}
}
// What this Telegram account may do to others in a conversation, and doing it.
"telegramPermissions" | "moderateTelegramMember" => {
let Some(buffer) = p_str_opt(params, "bufferId").and_then(|b| state.runtime.get_buffer(b)).filter(|b| b.account_id.starts_with("telegram:")) else {
return (None, Some(format!("{method} requires a Telegram \"bufferId\"")));
};
if method == "telegramPermissions" {
return match backend::telegram::admin::permissions(state, &buffer).await {
Ok(permissions) => (Some(permissions), None),
Err(e) => (None, Some(e)),
};
}
match backend::telegram::admin::moderate(state, &buffer, p_str(params, "userId", ""), p_str(params, "action", "")).await {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(e)),
}
}
"getMatrixRoomPermissions" => {
let (account_id, buffer_id) = match (p_str_opt(params, "accountId"), p_str_opt(params, "bufferId")) {
(Some(a), Some(b)) => (a, b),
src/runtime.rs
+94 -2
View File
@@ -543,6 +543,8 @@ pub struct Runtime {
/// Each connected Steam account's connection, by account id: how a message
/// or a status change reaches Steam from outside the connection's own task.
steam_links: Mutex<HashMap<String, crate::backend::steam::link::Link>>,
/// A connected Telegram account's client, by account id (see backend/telegram).
telegram_handles: Mutex<HashMap<String, crate::backend::telegram::Handle>>,
/// Who has asked to be a Steam account's friend, and who it has asked:
/// what the window is shown on the account's page, kept so a window opened
/// after the request arrived can still be told.
@@ -1058,6 +1060,10 @@ fn cap_cache<K: std::hash::Hash + Eq, V>(map: &mut HashMap<K, V>, limit: usize)
/// "Channels" and "Direct Messages" (whispers), the second drawn only while there is a whisper to put under it.
/// They are the service's own headings, so they cannot be renamed, removed or filled by hand.
fn fixed_category(account_id: &str, kind: &str) -> Option<String> {
if account_id.starts_with("telegram:") {
// Groups are the usual channel; a broadcast channel moves to its own heading when its buffer is made.
return Some(if kind == "dm" { "Direct Messages".to_string() } else { "Groups".to_string() });
}
account_id.starts_with("twitch:").then(|| if kind == "dm" { "Direct Messages".to_string() } else { "Channels".to_string() })
}
@@ -1106,6 +1112,7 @@ impl Runtime {
twitch_senders: Mutex::new(HashMap::new()),
twitch_rights: Mutex::new(HashMap::new()),
steam_links: Mutex::new(HashMap::new()),
telegram_handles: Mutex::new(HashMap::new()),
steam_requests: Mutex::new(HashMap::new()),
kick_channels: Mutex::new(HashMap::new()),
kick_streams: Mutex::new(HashMap::new()),
@@ -1256,6 +1263,11 @@ impl Runtime {
let conn = conn_states.get(&id).map(|s| s.as_str()).unwrap_or("disconnected");
crate::accounts::twitch_account_to_json(cfg, conn)
}));
out.extend(state.accounts.all_telegram().iter().map(|cfg| {
let id = cfg.account_id();
let conn = conn_states.get(&id).map(|s| s.as_str()).unwrap_or("disconnected");
crate::accounts::telegram_account_to_json(cfg, conn)
}));
out
}
@@ -1532,6 +1544,24 @@ impl Runtime {
}
}
/// Says a conversation was active at `ts`, if that is later than what is known: for a service that tells when its
/// last message was without sending it.
pub fn raise_buffer_activity(&self, state: &AppState, buffer_id: &str, ts: i64) {
let updated = {
let mut buffers = self.buffers.lock().unwrap();
match buffers.get_mut(buffer_id) {
Some(b) if b.last_activity_ts < ts => {
b.last_activity_ts = ts;
Some(b.clone())
}
_ => None,
}
};
if let Some(b) = updated {
state.events.emit("bufferListChange", serde_json::to_value(&b).unwrap());
}
}
pub fn is_server_buffer(&self, buffer_id: &str) -> bool {
self.buffers
.lock()
@@ -1658,7 +1688,9 @@ impl Runtime {
if let Some(existing) = buffers.get_mut(&id) {
// A Twitch account's list is always headed the same two ways (see `fixed_category`), including
// for what was open before it was.
if existing.category.is_none() {
// Only in the account's own list: a conversation filed under a rail entry of its own (a Telegram group)
// is the whole of that entry and has no heading to sit under.
if existing.category.is_none() && existing.group_id.as_deref().is_none_or(|g| g == model::account_group_id(account_id)) {
existing.category = fixed_category(account_id, kind);
}
return existing.clone();
@@ -1693,9 +1725,12 @@ impl Runtime {
favourite: false,
low_priority: false,
service_room: false,
name_icon: None,
topic: None,
read_only: None,
forum: false,
unread_seed: 0,
mentions_seed: 0,
};
buffers.insert(id, buffer.clone());
state.events.emit("bufferListChange", serde_json::to_value(&buffer).unwrap());
@@ -1759,6 +1794,23 @@ impl Runtime {
/// One way only. A homeserver does not un-designate its notices room, and
/// a client that cleared the mark because one sync happened not to carry
/// the tag would offer a Leave that the server refuses.
/// Draws a glyph in front of a conversation's name in the list (see `Buffer::name_icon`), said once it changes.
pub fn set_buffer_name_icon(&self, state: &AppState, buffer_id: &str, icon: Option<&str>) {
let updated = {
let mut buffers = self.buffers.lock().unwrap();
match buffers.get_mut(buffer_id) {
Some(b) if b.name_icon.as_deref() != icon => {
b.name_icon = icon.map(String::from);
Some(b.clone())
}
_ => None,
}
};
if let Some(b) = updated {
state.events.emit("bufferListChange", serde_json::to_value(&b).unwrap());
}
}
pub fn set_buffer_service_room(&self, state: &AppState, buffer_id: &str) {
let updated = {
let mut buffers = self.buffers.lock().unwrap();
@@ -3247,6 +3299,25 @@ impl Runtime {
}
}
/// What the service counts as unread in a conversation, and how many of those mention this account: see
/// `Buffer::unread_seed`. Said to the windows when it changes, which set their own count to it.
pub fn set_buffer_unread(&self, state: &AppState, buffer_id: &str, unread: u32, mentions: u32) {
let updated = {
let mut buffers = self.buffers.lock().unwrap();
match buffers.get_mut(buffer_id) {
Some(b) if (b.unread_seed, b.mentions_seed) != (unread, mentions) => {
b.unread_seed = unread;
b.mentions_seed = mentions;
Some(b.clone())
}
_ => None,
}
};
if let Some(b) = updated {
state.events.emit("bufferListChange", serde_json::to_value(&b).unwrap());
}
}
/// A Matrix room's topic, kept against the room so a buffer made after it
/// was read still gets it.
pub fn set_matrix_room_topic(&self, state: &AppState, account_id: &str, room_id: &str, topic: &str) {
@@ -4345,6 +4416,18 @@ impl Runtime {
}
/// Registers a signed-in Steam connection.
pub fn set_telegram_handle(&self, account_id: &str, handle: crate::backend::telegram::Handle) {
self.telegram_handles.lock().unwrap().insert(account_id.to_string(), handle);
}
pub fn clear_telegram_handle(&self, account_id: &str) {
self.telegram_handles.lock().unwrap().remove(account_id);
}
pub fn telegram_handle(&self, account_id: &str) -> Option<crate::backend::telegram::Handle> {
self.telegram_handles.lock().unwrap().get(account_id).cloned()
}
pub fn set_steam_link(&self, account_id: &str, link: crate::backend::steam::link::Link) {
self.steam_links.lock().unwrap().insert(account_id.to_string(), link);
}
@@ -4830,6 +4913,15 @@ impl Runtime {
}
}
/// Says who and what a reply answered, once it has been looked up (a reply to something not read here yet).
pub fn fill_reply_preview(&self, state: &AppState, buffer_id: &str, msg_id: &str, from: &str, body: &str) {
match state.store.set_reply_text(buffer_id, msg_id, from, body) {
Ok(true) => state.events.emit("messageReplyRenamed", json!({ "bufferId": buffer_id, "id": msg_id, "from": from, "body": body })),
Ok(false) => {}
Err(e) => tracing::warn!("failed to fill in a reply: {e}"),
}
}
/// Discord's MESSAGE_DELETE - removes the row and tells clients to
/// drop it from view, same as Discord's own clients (no tombstone).
/// A message the service took down, kept and marked as taken down. Quiet
@@ -5619,7 +5711,7 @@ impl Runtime {
discord_channels, discord_gateway_sessions, ringing_calls,
discord_guild_id, discord_buffer_emojis, emoji_catalogue,
kick_watching, matrix_emoticons, discord_friends,
sneedchat_senders, kick_senders, twitch_senders, twitch_rights, steam_links, steam_requests, kick_channels,
sneedchat_senders, kick_senders, twitch_senders, twitch_rights, steam_links, telegram_handles, steam_requests, kick_channels,
kick_streams, kick_stream_fetched, live_cards,
irc_channel_lists, irc_channel_directory, irc_caps, irc_metadata,
discord_last_interaction, matrix_room_creators, matrix_room_versions,
src/store.rs
+140 -12
View File
@@ -193,6 +193,8 @@ impl Store {
// A message the service took down and this client kept - see
// model::Moderated. JSON, and null for a message nobody removed.
"ALTER TABLE messages ADD COLUMN moderated TEXT",
// The conversation a reply's target is in, when it is another - see model::ReplyPreview::chat.
"ALTER TABLE messages ADD COLUMN reply_chat TEXT",
] {
let _ = conn.execute(stmt, []);
}
@@ -449,8 +451,8 @@ impl Store {
// OR IGNORE against the (buffer_id, msg_id) index: recording the
// same message twice is a backend replaying history it already
// has, and the right answer is to keep the copy already stored.
"INSERT OR IGNORE INTO messages (msg_id, buffer_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, buffer_kind, sender_color, badges, reply_is_thread, reply_forwarded, reply_thread_root)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, COALESCE(?12, '[]'), ?13, ?14, COALESCE(?15, '[]'), ?16, COALESCE(?17, '[]'), ?18, ?19, ?20, COALESCE(?21, '[]'), ?22, ?23, ?24)",
"INSERT OR IGNORE INTO messages (msg_id, buffer_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, buffer_kind, sender_color, badges, reply_is_thread, reply_forwarded, reply_thread_root, reply_chat)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, COALESCE(?12, '[]'), ?13, ?14, COALESCE(?15, '[]'), ?16, COALESCE(?17, '[]'), ?18, ?19, ?20, COALESCE(?21, '[]'), ?22, ?23, ?24, ?25)",
params![
msg_id,
buffer_id,
@@ -476,6 +478,7 @@ impl Store {
reply_to.is_some_and(|r| r.thread) as i32,
reply_to.is_some_and(|r| r.forwarded) as i32,
reply_to.and_then(|r| r.thread_root.as_deref()),
reply_to.and_then(|r| r.chat.as_deref()),
],
)?;
// Whether this was actually new. The insert has always ignored a
@@ -519,6 +522,27 @@ impl Store {
Ok(changed > 0)
}
/// Puts a sender's badges on a message already stored without them, without marking it changed.
pub fn set_badges(&self, buffer_id: &str, msg_id: &str, badges: &[crate::backend::kick::api::Badge]) -> Result<bool> {
let conn = self.conn.lock().unwrap();
let json = serde_json::to_string(badges)?;
let changed = conn.execute(
"UPDATE messages SET badges = ?1 WHERE buffer_id = ?2 AND msg_id = ?3 AND (badges IS NULL OR badges = '[]')",
params![json, buffer_id, msg_id],
)?;
Ok(changed > 0)
}
/// Fills in who and what a reply answered, once that has been looked up.
pub fn set_reply_text(&self, buffer_id: &str, msg_id: &str, from: &str, body: &str) -> Result<bool> {
let conn = self.conn.lock().unwrap();
let changed = conn.execute(
"UPDATE messages SET reply_to_from = ?1, reply_to_body = ?2 WHERE buffer_id = ?3 AND msg_id = ?4",
params![from, body, buffer_id, msg_id],
)?;
Ok(changed > 0)
}
/// Puts the cards on a message without marking it edited: a link being
/// described after the fact is not something its sender did.
pub fn set_message_embeds(&self, buffer_id: &str, msg_id: &str, embeds: &[Embed]) -> Result<bool> {
@@ -753,6 +777,44 @@ impl Store {
Ok(id)
}
/// The most recent message this buffer holds, and whether it is the account's own.
pub fn newest_msg(&self, buffer_id: &str) -> Result<Option<(String, bool)>> {
let conn = self.conn.lock().unwrap();
let row = conn
.query_row(
"SELECT msg_id, is_own FROM messages WHERE buffer_id = ?1 AND msg_id IS NOT NULL ORDER BY ts DESC LIMIT 1",
params![buffer_id],
|row| Ok((row.get::<_, String>(0)?, row.get::<_, i64>(1)? != 0)),
)
.optional()?;
Ok(row)
}
/// The ids of the account's own messages in these buffers from `since` (Unix seconds) on, newest first, at most
/// `limit`.
pub fn own_message_ids_since(&self, buffer_ids: &[String], since: i64, limit: usize) -> Result<Vec<String>> {
let mut found = Vec::new();
if buffer_ids.is_empty() {
return Ok(found);
}
let conn = self.conn.lock().unwrap();
for chunk in buffer_ids.chunks(200) {
let marks = vec!["?"; chunk.len()].join(",");
let sql = format!("SELECT msg_id FROM messages WHERE is_own = 1 AND ts >= ? AND msg_id IS NOT NULL AND buffer_id IN ({marks}) ORDER BY ts DESC LIMIT {limit}");
let mut args: Vec<&dyn rusqlite::ToSql> = vec![&since];
args.extend(chunk.iter().map(|b| b as &dyn rusqlite::ToSql));
let mut stmt = conn.prepare(&sql)?;
let rows = stmt.query_map(args.as_slice(), |row| row.get::<_, String>(0))?;
for id in rows {
found.push(id?);
}
if found.len() >= limit {
break;
}
}
Ok(found)
}
/// The most recent message this buffer holds, for asking a service what
/// has happened since.
pub fn newest_msg_id(&self, buffer_id: &str) -> Result<Option<String>> {
@@ -780,7 +842,8 @@ impl Store {
body: row.get::<_, Option<String>>("reply_to_body").ok().flatten().unwrap_or_default(),
thread: row.get::<_, Option<i64>>("reply_is_thread").ok().flatten().unwrap_or(0) != 0,
forwarded: row.get::<_, Option<i64>>("reply_forwarded").ok().flatten().unwrap_or(0) != 0,
thread_root: row.get::<_, Option<String>>("reply_thread_root").ok().flatten() });
thread_root: row.get::<_, Option<String>>("reply_thread_root").ok().flatten(),
chat: row.get::<_, Option<String>>("reply_chat").ok().flatten() });
let json_column = |name: &str| -> String {
row.get::<_, Option<String>>(name).ok().flatten().unwrap_or_else(|| "[]".to_string())
};
@@ -823,7 +886,7 @@ impl Store {
let conn = self.conn.lock().unwrap();
let limit = if limit > 0 { limit } else { 200 };
let mut stmt = conn.prepare(
"SELECT msg_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, edited, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, sender_color, badges, reply_is_thread, reply_forwarded, components, reply_thread_root, moderated
"SELECT msg_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, edited, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, sender_color, badges, reply_is_thread, reply_forwarded, components, reply_thread_root, reply_chat, moderated
FROM messages
WHERE buffer_id = ?1 AND (?2 <= 0 OR ts < ?2)
ORDER BY ts DESC LIMIT ?3",
@@ -854,6 +917,13 @@ impl Store {
Ok(())
}
/// Takes the buttons off a message, for an edit that removed them.
pub fn clear_components(&self, buffer_id: &str, msg_id: &str) -> Result<()> {
let conn = self.conn.lock().unwrap();
conn.execute("UPDATE messages SET components = '[]' WHERE buffer_id = ?1 AND msg_id = ?2", params![buffer_id, msg_id])?;
Ok(())
}
/// The names that have spoken here lately, most recent first.
///
/// Wanted because a roster is not always there to ask: a service may not
@@ -899,7 +969,7 @@ impl Store {
let conn = self.conn.lock().unwrap();
let limit = if limit > 0 { limit } else { 200 };
let mut stmt = conn.prepare(
"SELECT msg_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, edited, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, sender_color, badges, reply_is_thread, reply_forwarded, components, reply_thread_root, moderated
"SELECT msg_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, edited, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, sender_color, badges, reply_is_thread, reply_forwarded, components, reply_thread_root, reply_chat, moderated
FROM messages
WHERE buffer_id = ?1 AND ts > ?2
ORDER BY ts ASC LIMIT ?3",
@@ -922,7 +992,7 @@ impl Store {
let conn = self.conn.lock().unwrap();
let limit = if limit > 0 { limit } else { 500 };
let mut stmt = conn.prepare(
"SELECT msg_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, edited, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, sender_color, badges, reply_is_thread, reply_forwarded, components, reply_thread_root, moderated
"SELECT msg_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, edited, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, sender_color, badges, reply_is_thread, reply_forwarded, components, reply_thread_root, reply_chat, moderated
FROM messages
WHERE buffer_id = ?1 AND ts > ?2 AND ts <= ?3
ORDER BY ts ASC LIMIT ?4",
@@ -963,7 +1033,7 @@ impl Store {
pub fn messages_around(&self, buffer_id: &str, ts: i64, span: i64) -> Result<Vec<Message>> {
let conn = self.conn.lock().unwrap();
let span = if span > 0 { span } else { 50 };
const COLUMNS: &str = "msg_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, edited, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, sender_color, badges, reply_is_thread, reply_forwarded, components, reply_thread_root, moderated";
const COLUMNS: &str = "msg_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, edited, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, sender_color, badges, reply_is_thread, reply_forwarded, components, reply_thread_root, reply_chat, moderated";
let mut older = conn.prepare(&format!(
"SELECT {COLUMNS} FROM messages WHERE buffer_id = ?1 AND ts <= ?2 ORDER BY ts DESC LIMIT ?3"
@@ -1009,7 +1079,7 @@ impl Store {
format!(" OR (buffer_kind IS NULL AND buffer_id IN ({places}))")
};
let sql = format!(
"SELECT msg_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, edited, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, sender_color, badges, reply_is_thread, reply_forwarded, components, reply_thread_root, moderated, buffer_id
"SELECT msg_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, edited, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, sender_color, badges, reply_is_thread, reply_forwarded, components, reply_thread_root, reply_chat, moderated, buffer_id
FROM messages
WHERE is_highlight = 1 AND is_own = 0 AND (buffer_kind = 'channel'{legacy})
ORDER BY ts DESC LIMIT ?"
@@ -1270,7 +1340,7 @@ impl Store {
let escaped = query.replace('\\', "\\\\").replace('%', "\\%").replace('_', "\\_");
let pattern = format!("%{escaped}%");
let mut stmt = conn.prepare(
"SELECT msg_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, edited, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, sender_color, badges, reply_is_thread, reply_forwarded, components, reply_thread_root, moderated
"SELECT msg_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, edited, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, sender_color, badges, reply_is_thread, reply_forwarded, components, reply_thread_root, reply_chat, moderated
FROM messages
WHERE buffer_id = ?1 AND body LIKE ?2 ESCAPE '\\'
ORDER BY ts DESC LIMIT ?3",
@@ -1291,7 +1361,13 @@ fn drop_missing_local_copies(attachments: &mut [Attachment]) {
let gone = |p: &Option<String>| -> bool {
let Some(p) = p.as_deref() else { return false };
let Some(rest) = p.strip_prefix("file://") else { return false };
!std::path::Path::new(rest).exists()
// Telegram's are named before they are fetched, and fetched when the window asks (see backend/telegram/media.rs):
// a file that is not there yet is the usual state of one, not a sign it was swept.
let path = std::path::Path::new(rest);
if path.parent().and_then(|d| d.file_name()).and_then(|n| n.to_str()).is_some_and(|n| n == crate::backend::telegram::media::MEDIA || n == crate::backend::telegram::media::AVATARS) {
return false;
}
!path.exists()
};
for att in attachments.iter_mut() {
if gone(&att.thumbnail_path) {
@@ -1335,7 +1411,7 @@ impl Store {
pub fn thread_messages(&self, buffer_id: &str, root_id: &str) -> Result<Vec<Message>> {
let conn = self.conn.lock().unwrap();
let mut stmt = conn.prepare(
"SELECT msg_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, edited, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, sender_color, badges, reply_is_thread, reply_forwarded, components, reply_thread_root, moderated
"SELECT msg_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, edited, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, sender_color, badges, reply_is_thread, reply_forwarded, components, reply_thread_root, reply_chat, moderated
FROM messages
WHERE buffer_id = ?1 AND ((reply_to_id = ?2 AND reply_is_thread = 1) OR reply_thread_root = ?2 OR msg_id = ?2)
ORDER BY ts ASC, rowid ASC",
@@ -1347,7 +1423,7 @@ impl Store {
pub fn get_message(&self, buffer_id: &str, msg_id: &str) -> Result<Option<Message>> {
let conn = self.conn.lock().unwrap();
let mut stmt = conn.prepare(
"SELECT msg_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, edited, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, sender_color, badges, reply_is_thread, reply_forwarded, components, reply_thread_root, moderated
"SELECT msg_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, edited, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, sender_color, badges, reply_is_thread, reply_forwarded, components, reply_thread_root, reply_chat, moderated
FROM messages
WHERE buffer_id = ?1 AND msg_id = ?2",
)?;
@@ -1815,6 +1891,42 @@ mod tests {
.expect("appending");
}
/// Like `append_at`, but the account's own.
fn append_own_at(s: &Store, buffer: &str, id: &str, ts: i64) {
s.append_message(buffer, id, "me", "hi", ts, false, false, "chat", None, &[], true, None, &[], &[], None, None, "channel", None, &[])
.expect("appending");
}
#[test]
fn finds_the_newest_message_and_whether_it_is_the_accounts_own() {
let (s, _dir) = store();
assert_eq!(s.newest_msg("b").unwrap(), None);
append_at(&s, "b", "m1", 100);
assert_eq!(s.newest_msg("b").unwrap(), Some(("m1".to_string(), false)));
append_own_at(&s, "b", "m2", 200);
assert_eq!(s.newest_msg("b").unwrap(), Some(("m2".to_string(), true)));
append_at(&s, "b", "m3", 300);
assert_eq!(s.newest_msg("b").unwrap(), Some(("m3".to_string(), false)));
}
#[test]
fn finds_what_the_account_has_said_in_a_set_of_rooms_since_a_time() {
let (s, _dir) = store();
let rooms = |names: &[&str]| names.iter().map(|n| n.to_string()).collect::<Vec<_>>();
assert!(s.own_message_ids_since(&[], 0, 5).unwrap().is_empty(), "no rooms is no message");
append_at(&s, "a", "m1", 500);
assert!(s.own_message_ids_since(&rooms(&["a"]), 0, 5).unwrap().is_empty(), "somebody else's message is not the account's");
append_own_at(&s, "b", "m2", 100);
append_own_at(&s, "b", "m3", 200);
assert_eq!(s.own_message_ids_since(&rooms(&["a", "b"]), 0, 5).unwrap(), vec!["m3", "m2"], "newest first");
assert!(s.own_message_ids_since(&rooms(&["a"]), 0, 5).unwrap().is_empty(), "in another room");
assert!(s.own_message_ids_since(&rooms(&["b"]), 201, 5).unwrap().is_empty(), "before the account joined");
assert_eq!(s.own_message_ids_since(&rooms(&["b"]), 0, 1).unwrap(), vec!["m3"], "no more than asked for");
// More rooms than one query holds.
let many: Vec<String> = (0..450).map(|i| format!("r{i}")).chain(std::iter::once("b".to_string())).collect();
assert_eq!(s.own_message_ids_since(&many, 0, 5).unwrap().len(), 2);
}
#[test]
fn finds_the_newest_message_to_mark_a_room_read_up_to() {
let (s, _dir) = store();
@@ -2000,6 +2112,22 @@ mod tests {
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn a_telegram_file_not_fetched_yet_is_still_advertised() {
// Its name is given when the message is recorded and the file is fetched when the window asks, so
// "not there" is how it starts and not a sign it was swept.
let mut attachments = vec![Attachment {
path: Some("file:///home/someone/.cache/nobilis/telegram-media/tg_1_n1_2_full.jpg".to_string()),
thumbnail_path: Some("file:///home/someone/.cache/nobilis/telegram-media/tg_1_n1_2_thumb.jpg".to_string()),
..Default::default()
}];
drop_missing_local_copies(&mut attachments);
assert!(attachments[0].path.is_some() && attachments[0].thumbnail_path.is_some());
let mut swept = vec![Attachment { path: Some("file:///home/someone/.cache/nobilis/matrix-media/gone.jpg".to_string()), ..Default::default() }];
drop_missing_local_copies(&mut swept);
assert!(swept[0].path.is_none(), "another service's swept file is still dropped");
}
#[test]
fn a_remote_url_is_left_alone() {
// Only local copies are checked; a https link is not this code's