Salastil/nobilis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

Compare commits

base: Salastil/nobilis:v1.1.0
Branches Tags
Salastil/nobilis:master Salastil/nobilis:protocol/telegram Salastil/nobilis:development Salastil/nobilis:protocol/twitch
Salastil/nobilis:v1.2.1 Salastil/nobilis:v1.2.0 Salastil/nobilis:v1.1.0 Salastil/nobilis:v1.0.0 Salastil/nobilis:v1.0.0-rc.5 Salastil/nobilis:v1.0.0-rc.4 Salastil/nobilis:v1.0.0-rc.3 Salastil/nobilis:v1.0.0-rc.2 Salastil/nobilis:v1.0.0-rc.1
...
compare: Salastil/nobilis:protocol/telegram
Branches Tags
Salastil/nobilis:protocol/telegram Salastil/nobilis:master Salastil/nobilis:development Salastil/nobilis:protocol/twitch
Salastil/nobilis:v1.2.1 Salastil/nobilis:v1.2.0 Salastil/nobilis:v1.1.0 Salastil/nobilis:v1.0.0 Salastil/nobilis:v1.0.0-rc.5 Salastil/nobilis:v1.0.0-rc.4 Salastil/nobilis:v1.0.0-rc.3 Salastil/nobilis:v1.0.0-rc.2 Salastil/nobilis:v1.0.0-rc.1
58 Commits
v1.1.0 ... protocol/telegram
Author SHA1 Message Date
SalastilandClaude Sonnet 5.5 63093e1f1a Discord: the server's rules dialog shows the rules and any questions, and sends the answers 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-16 20:28:43 -04:00
SalastilandClaude Sonnet 5.5 65635276f3 Telegram: a forwarded message says where it came from, and a post in an open channel is a click away 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-16 20:07:35 -04:00
SalastilandClaude Sonnet 5.5 5094a9a2ec Telegram: an answer to a post in another channel is drawn as Telegram draws it and goes to the original 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-16 19:45:55 -04:00
SalastilandClaude Sonnet 5.5 7a4b0130a0 Telegram: a horn before broadcast channels and a robot before bots in the list 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-16 19:36:54 -04:00
SalastilandClaude Sonnet 5.5 ee0e0f921b Telegram: bots have a heading of their own 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-16 19:32:07 -04:00
SalastilandClaude Sonnet 5.5 a6f35525d0 Telegram: an article's cover is found among the page's own picture, and a missing picture keeps its caption 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-16 15:27:28 -04:00
SalastilandClaude Sonnet 5.5 b1d1a53c2f Telegram: Instant View articles drawn as a document of their own; link cards say so 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-16 15:22:23 -04:00
SalastilandClaude Sonnet 5.5 15d9058604 Telegram: link previews (what Instant View pages carry) as cards with their picture 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-16 15:09:43 -04:00
SalastilandClaude Sonnet 5.5 b4ddb310b6 Telegram: a file not fetched yet keeps its place in a message (it was being dropped as swept) 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-16 10:03:45 -04:00
SalastilandClaude Sonnet 5.5 d368413241 Telegram: each download has its own half-written file; a restore waits for a preview already on its way 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-16 09:55:50 -04:00
SalastilandClaude Sonnet 5.5 5037705cc4 Telegram: previews are fetched as messages are recorded 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-16 09:47:29 -04:00
SalastilandClaude Sonnet 5.5 f725dcc2da Telegram: pictures no longer wait behind whole-file downloads; failures to fetch are logged 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-16 09:38:01 -04:00
SalastilandClaude Sonnet 5.5 4643cb082e Telegram: a message kept without its media gets it when read again 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-16 09:30:15 -04:00
SalastilandClaude Sonnet 5.5 11ce3e974b Telegram: messages from a person in a private chat are recorded (short updates name no one) 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-16 09:23:12 -04:00
SalastilandClaude Sonnet 5.5 0f5003c061 Telegram: start polls, forum messages are not replies to their topic 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-16 00:23:49 -04:00
SalastilandClaude Sonnet 5.5 ca47ffeae7 Telegram: join and leave, admin actions on members, mute synced with the account 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-16 00:17:18 -04:00
SalastilandClaude Sonnet 5.5 6a8c35fe14 Telegram: polls, thread replies and channel comments 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-16 00:08:09 -04:00
SalastilandClaude Sonnet 5.5 584e3b8744 Telegram: reactions, pins, members, profile cards, server search, jumping to old messages 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-15 23:59:03 -04:00
Salastil 84258f4381 Telegram: pictures, video, files, voice and stickers - fetched when looked at - and sending files, albums and voice messages 2026-10-15 23:53:15 -04:00
SalastilandClaude Sonnet 5.5 3fc8cd1a1d Telegram: send, reply, edit, delete, typing both ways, read positions, formatting as entities 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-15 23:46:34 -04:00
SalastilandClaude Sonnet 5.5 22f838417f Telegram: a conversation the account cannot write in shows why in place of the message box 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-15 23:36:21 -04:00
SalastilandClaude Sonnet 5.5 d027d2e39f Telegram: conversations as buffers, history on opening and scrolling back, live messages, edits and deletions 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-15 23:27:29 -04:00
SalastilandClaude Sonnet 5.5 537ee520cd Telegram: accounts, sessions in files of their own, sign-in by phone code and cloud password, a connection that is kept and reconnected 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-15 23:17:27 -04:00
SalastilandClaude Sonnet 5.5 e4a16278ab 1.2.1 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-15 17:14:12 -04:00
SalastilandClaude Sonnet 5.5 f3de870d49 Steam: sign in as a desktop client with a stable machine id and a logon id of this install's own, and wait longer before reconnecting 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-15 01:01:33 -04:00
SalastilandClaude Sonnet 5.5 e9b4470128 1.2.0 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-14 21:54:08 -04:00
SalastilandClaude Sonnet 5.5 6d246264dd Twitch: report watching once a minute while the stage plays, so channel points build 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-14 21:04:28 -04:00
SalastilandClaude Sonnet 5.5 7d726839d3 Twitch emotes: one section per owner with their pictures, no duplicate cells, a changed emote listed with its channel 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-14 20:44:14 -04:00
SalastilandClaude Sonnet 5.5 25a0c07770 Twitch emote picker: usable emotes first; the channel's picture or Twitch's mark on the rail 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-14 19:58:35 -04:00
SalastilandClaude Sonnet 5.5 d348ccfb59 Twitch: drop BetterTTV, FrankerFaceZ and 7TV; the picker shows the channel's whole emote set with the unusable ones greyed out 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-14 19:52:53 -04:00
SalastilandClaude Sonnet 5.5 1bb6e3bfd4 Twitch: the picker reads the account's emotes again when stale, and your own line shows each emote as a picture 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-14 19:42:44 -04:00
SalastilandClaude Sonnet 5.5 81f3aa0f98 Twitch emote rewards: offer the channel's unlockable emotes the account lacks, change only emotes it has; warm the integrity token when rewards open 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-14 19:25:19 -04:00
SalastilandClaude Sonnet 5.5 84c15e8bf8 Twitch: integrity tokens are made as the account for the account's own calls; emote choices as pictures 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-14 18:58:26 -04:00
SalastilandClaude Sonnet 5.5 c4f52e5216 Twitch: Highlight My Message and the emote unlocks as rewards; the account's emotes are read again so unlocks appear and lapse 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-14 18:47:32 -04:00
SalastilandClaude Sonnet 5.5 f4533ff9a7 Twitch: list a channel's rewards and redeem them, as the twitch.tv session 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-14 18:23:00 -04:00
SalastilandClaude Sonnet 5.5 34d15e9315 Twitch: no bonus claiming (out of scope); the points plaque shows the balance only 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-14 18:08:31 -04:00
SalastilandClaude Sonnet 5.5 765f6d07ea Twitch: the live points test can bet on a real prediction with nothing at stake 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-14 18:03:59 -04:00
SalastilandClaude Sonnet 5.5 f1aec36f97 Twitch: a message highlighted with channel points is not a mention and notifies nobody 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-14 17:17:54 -04:00
SalastilandClaude Sonnet 5.5 71d18c8cb4 Twitch: vote in polls, bet on predictions, read the points balance and claim the bonus, as the twitch.tv session 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-14 17:00:03 -04:00
SalastilandClaude Sonnet 5.5 bd031001ab Twitch: a refusal that is about the account itself does not tell the person to sign in again 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-14 16:46:44 -04:00
SalastilandClaude Sonnet 5.5 28481f39e3 IRC: a channel's modes are explained in words when it is joined and when one changes 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-14 16:20:17 -04:00
SalastilandClaude Sonnet 5.5 26a2339ee6 Twitch: Channels and Direct Messages as fixed headings; room mode changes, removals and acts as cards 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-14 15:03:49 -04:00
SalastilandClaude Sonnet 5.5 a286f45ee8 Twitch: a whisper conversation is named @login so it cannot be the same buffer as that person's channel; a timeout or ban is not said twice 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-14 14:48:38 -04:00
SalastilandClaude Sonnet 5.5 6089672200 Twitch: moderation, whispers, shoutouts and blocking are asked for in the one sign-in 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-14 14:02:36 -04:00
SalastilandClaude Sonnet 5.5 c9b21e960b Twitch: a picked emote is sent as its word; its picker cell is named by the token the box draws 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-14 12:05:00 -04:00
SalastilandClaude Sonnet 5.5 278f0dd323 Twitch: a sign-in can bring the twitch.tv session with it 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-14 10:56:28 -04:00
SalastilandClaude Sonnet 5.5 f0cac1df03 Twitch: a signed-in account opens the channels it follows the first time it connects 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-14 10:23:07 -04:00
SalastilandClaude Sonnet 5.5 3c26bc9bfe Twitch: raids, polls and predictions as they happen, live 7TV and BetterTTV emote changes, block and shoutout 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-14 09:56:47 -04:00
SalastilandClaude Sonnet 5.5 4c30be62de Twitch: emotes and badges as pictures, third-party emote sets, cheers, moderation by menu and slash command, whispers, rights per channel 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-14 09:15:00 -04:00
SalastilandClaude Sonnet 5.5 6635af75cf Twitch: what is on air, and the address of its video 
A channel's state (live, title, category, viewers, avatar) is read from Twitch's GraphQL in one batch for
every open channel each minute and said as the same stream event Kick's use, so the header chip, the Watch
button and the player are the code Kick already has. The playlist address is built from the player's own
token query, renewed before its twenty minutes are up; what the token says (ads, restricted renditions,
subscribers only, blocked here) is carried. An integrity token, if Twitch ever demands one, is asked for from
the window. A Twitch website session can be attached to an account for video as the person, and followed
channels can be opened by Helix.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-14 08:53:24 -04:00
SalastilandClaude Sonnet 5.5 cd8a44cbf9 Twitch: Helix sending with IRC as the fallback, recent messages on request 
A message goes by Helix when the account agreed to user:write:chat (it says whether Twitch sent it and why
not, and gives its id, which the shown copy then carries) and falls back to IRC when Helix cannot take it.
A reply names its parent to Twitch even when that message has left the scrollback. A channel's recent
past can be read from the third-party recent-messages service, only when asked, never making the roster.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-14 00:43:21 -04:00
SalastilandClaude Sonnet 5.5 7bd80939a8 Twitch: chat is read and sent over Twitch's IRC, channels opened and remembered 
irc.rs reads a line (tags, prefix, command, parameters); chat.rs turns lines into messages (colour, badges,
emote tokens by code point, replies with their thread), cards (subs, gifts told once per batch, raids,
announcements), removed messages (CLEARMSG, CLEARCHAT marks a person's messages), room modes in the header,
a roster counted without a cap; connect.rs is the connection: greeting, paced joins, a channel Twitch never
answers for is said not to exist, say with the 500-character and rate limits, local echo, reconnect.
Rounds 2 and 3 of #55, without history, Helix sending or blocking yet.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-14 00:32:27 -04:00
SalastilandClaude Sonnet 5.5 33dfca4183 Twitch: an account, signed in by the device code flow, kept good 
Accounts (signed in, or an anonymous reader), their tokens in accounts.toml, the device flow against
id.twitch.tv with moho's public client (no secret), renewal with the single-use refresh token written
before anything else, validation, revocation on removal, and a connection that keeps the sign-in good and
stops in AuthFailed when Twitch will not take it. Round 1 of #55.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-14 00:13:32 -04:00
SalastilandClaude Sonnet 5.5 4c17be082b Kick: the points balance is pushed as it changes, and badges_v2 arrive as pictures 
The watch session also follows private-channelpoints-<user id>, where Kick says
ChannelPointsUpdatedEvent with the channel and the new balance, and tells the window.
badges_v2 are read - the ones the person chose to wear, ranked highest first as Kick's own
apps rank them - and carried beside the role badges as pictures.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-13 23:11:25 -04:00
SalastilandClaude Sonnet 5.5 43f5e94c18 Kick: fix the order of the roster test's assertions 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-13 21:54:55 -04:00
SalastilandClaude Sonnet 5.5 41e93b8f49 Kick: the count of chatters keeps rising past the list's length 
The roster is the most recent 500 speakers (it was 200) and is sent whole each time somebody new joins it;
the number of different people who have spoken is counted apart, never cut off, and sent beside it.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-13 21:51:34 -04:00
SalastilandClaude Sonnet 5.5 bb318cf7de Kick: replies carry their thread, removed messages are kept and marked, the silent chatrooms.<id> subscription is gone 
A reply names the message its chain began with (thread_parent_id, or the quoted message's own place in a
chain when it comes out of history, whose metadata is a string of JSON and was never read for its quote).
A MessageDeletedEvent now marks the message instead of deleting it, with aiModerated and violatedRules.
Six minutes on a 60,000-viewer channel delivered nothing on chatrooms.<room>; it is no longer asked for,
and predictions-channel-<id> is now cancelled with the channel like the rest.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-13 21:17:29 -04:00
SalastilandClaude Sonnet 5.5 9f0ca86fdb Kick: the stream address comes from the playback-url endpoint, with its expiry, renewed before it runs out 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
 2026-10-13 19:57:02 -04:00
69 changed files with 15013 additions and 148 deletions
Expand all files Collapse all files
Cargo.lock
Generated
+661 -20
View File
@@ -841,6 +841,16 @@ dependencies = [
"futures",
]
[[package]]
name = "core-foundation"
version = "0.9.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "91e195e091a93c46f7102ec7818a2aa394e1e1771c3ab4825963fa03e45afb8f"
dependencies = [
"core-foundation-sys",
"libc",
]
[[package]]
name = "core-foundation"
version = "0.10.1"
@@ -944,13 +954,28 @@ dependencies = [
[[package]]
name = "crc32fast"
version = "1.5.0"
version = "1.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9481c1c90cbf2ac953f07c8d4a58aa3945c425b7185c9154d67a65e4230da511"
checksum = "01a7799fd6b852db0e61728dde9a204c423b44d689dbd432522543614b490e78"
dependencies = [
"cfg-if",
]
[[package]]
name = "critical-section"
version = "1.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "790eea4361631c5e7d22598ecd5723ff611904e3344ce8720784c93e3d83d40b"
[[package]]
name = "crossbeam-channel"
version = "0.5.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "98b0cc327b5bc766e7fda9c9260cc0fa81b43a8e240440422dff70788e3f9ef1"
dependencies = [
"crossbeam-utils",
]
[[package]]
name = "crossbeam-deque"
version = "0.8.7"
@@ -997,6 +1022,19 @@ dependencies = [
"zeroize",
]
[[package]]
name = "crypto-bigint"
version = "0.7.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1a52aa3fcda4e6302a9f48734f234d35d4721b96f8fe07d073f07ce9df4f0271"
dependencies = [
"cpubits",
"ctutils",
"num-traits",
"rand_core 0.10.1",
"serdect",
]
[[package]]
name = "crypto-common"
version = "0.1.7"
@@ -1019,6 +1057,16 @@ dependencies = [
"rand_core 0.10.1",
]
[[package]]
name = "crypto-primes"
version = "0.7.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3633a51a39c69ebbaa4feaa694bd83d241e4093901c84a0963b19d9bb3f0cf8f"
dependencies = [
"crypto-bigint 0.7.5",
"rand_core 0.10.1",
]
[[package]]
name = "ctr"
version = "0.9.2"
@@ -1274,6 +1322,37 @@ version = "3.3.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a9244323129647178bf41ac861a2cdb9d9c81b9b09d3d0d1de9cd302b33b8a1d"
[[package]]
name = "defmt"
version = "1.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e4a0795ac740cb54d7b3775c9c17345f6aaecf884c01733b39d6017738e0e3e1"
dependencies = [
"bitflags 1.3.2",
"defmt-macros",
]
[[package]]
name = "defmt-macros"
version = "1.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "840bb056bd99c495c9188198a258ab787c79f306d864db3e6e8451442d6d8366"
dependencies = [
"defmt-parser",
"proc-macro2",
"quote",
"syn 3.0.3",
]
[[package]]
name = "defmt-parser"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "71958f69fb2de856c5dd51ac9377e986ca1aa5919f931bf5395a807277f889a0"
dependencies = [
"thiserror 2.0.21",
]
[[package]]
name = "der"
version = "0.7.10"
@@ -1565,7 +1644,7 @@ dependencies = [
"ed25519 3.0.0",
"rand_core 0.10.1",
"serde",
"sha2 0.11.0",
"sha2 0.11.1",
"signature 3.0.0",
"subtle",
"zeroize",
@@ -1596,7 +1675,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b5e6043086bf7973472e0c7dff2142ea0b680d30e18d9cc40f267efbf222bd47"
dependencies = [
"base16ct 0.2.0",
"crypto-bigint",
"crypto-bigint 0.5.5",
"digest 0.10.7",
"ff",
"generic-array",
@@ -1630,7 +1709,7 @@ version = "3.1.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1bf1fa3f06bbff1ea5b1a9c7b14aa992a39657db60a2759457328d7e058f49ee"
dependencies = [
"num-bigint",
"num-bigint 0.4.8",
"num-traits",
"proc-macro2",
"quote",
@@ -1813,12 +1892,13 @@ checksum = "26b73573e6edcd2af0cdf47bd6cb58f0b3839491263c314eaad1ccf24430e1de"
[[package]]
name = "flate2"
version = "1.1.9"
version = "1.1.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "843fba2746e448b37e26a819579957415c8cef339bf08564fe8b7ddbd959573c"
checksum = "6e634e2e0ebac1ee034020da1ca582e17ffe4e0f5e985823721e168928136dcb"
dependencies = [
"crc32fast",
"miniz_oxide",
"miniz_oxide 0.9.1",
"zlib-rs",
]
[[package]]
@@ -2101,6 +2181,22 @@ dependencies = [
"weezl",
]
[[package]]
name = "glass_pumpkin"
version = "2.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6ff9ef01d1ba41822349a2a9970e706dc6035637957cdddecf32482a8a90e86e"
dependencies = [
"crypto-bigint 0.7.5",
"crypto-primes",
"getrandom 0.4.3",
"num-bigint 0.5.1",
"num-integer",
"num-traits",
"rand_core 0.10.1",
"thiserror 2.0.21",
]
[[package]]
name = "glob-match"
version = "0.2.1"
@@ -2119,6 +2215,121 @@ dependencies = [
"wasm-bindgen",
]
[[package]]
name = "grammers-client"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "08072ddb6eeaca38870a0401d7a25ade7a541a5a3b81ada6877b981fe44606f5"
dependencies = [
"futures-util",
"grammers-crypto",
"grammers-mtsender",
"grammers-session",
"grammers-tl-types",
"jiff",
"md5",
"mime_guess",
"pin-project-lite",
"tokio",
"tracing",
]
[[package]]
name = "grammers-crypto"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c0154344dab25d8365408df205e4440b432cae02ef9dae2df478434be77c7075"
dependencies = [
"aes 0.9.3",
"ctr 0.10.1",
"getrandom 0.4.3",
"glass_pumpkin",
"hmac 0.13.0",
"num-bigint 0.5.1",
"num-traits",
"pbkdf2 0.13.0",
"sha1 0.11.0",
"sha2 0.11.1",
]
[[package]]
name = "grammers-mtproto"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ecead7953adb1c9ad6a2bdcea9f9931414be54246bdb4c722e990f7ade1193d3"
dependencies = [
"bytes",
"crc32fast",
"flate2",
"getrandom 0.4.3",
"grammers-crypto",
"grammers-tl-types",
"num-bigint 0.5.1",
"sha1 0.11.0",
"tracing",
]
[[package]]
name = "grammers-mtsender"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b009d7d0f571cabac15e1ac7f588fefb764f23aba13bf8b9063a0a32a036075d"
dependencies = [
"bytes",
"grammers-crypto",
"grammers-mtproto",
"grammers-session",
"grammers-tl-types",
"hickory-resolver",
"locate-locale",
"os_info",
"tokio",
"tokio-socks",
"tracing",
"url",
]
[[package]]
name = "grammers-session"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3f34260cb63ecf5901e735a5f088ba6cb837d954649c223d5a53e7e980468ae0"
dependencies = [
"grammers-tl-types",
"serde",
"serde_with",
"tokio",
"tracing",
]
[[package]]
name = "grammers-tl-gen"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bf49f6769cffae7ea376669f3073c70eeb16ccc38f00ff019394069f3d401a4c"
dependencies = [
"grammers-tl-parser",
]
[[package]]
name = "grammers-tl-parser"
version = "1.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "646fc5eeb27461ffcc94fcf9fda3ffd9b402a20250447ae4a65c39c1a905e381"
dependencies = [
"crc32fast",
]
[[package]]
name = "grammers-tl-types"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "069e64c424bde5764391bc72f07085a12177d7053c8a0da355425527fd7c221f"
dependencies = [
"grammers-tl-gen",
"grammers-tl-parser",
]
[[package]]
name = "group"
version = "0.13.0"
@@ -2210,7 +2421,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "543f93241d32b3f00569201bfce9d7a93c92c6421b23c77864ac929dc947b9fc"
dependencies = [
"hax-lib-macros",
"num-bigint",
"num-bigint 0.4.8",
"num-traits",
]
@@ -2264,6 +2475,76 @@ version = "0.3.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7ebdb29d2ea9ed0083cd8cece49bbd968021bd99b0849edb4a9a7ee0fdf6a4e0"
[[package]]
name = "hickory-net"
version = "0.26.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c480823ed7c2c5d0f09c41020cb6b7c28029ce60ec42dc942158dcf22f8e0a4d"
dependencies = [
"async-trait",
"cfg-if",
"data-encoding",
"futures-channel",
"futures-io",
"futures-util",
"hickory-proto",
"idna",
"ipnet",
"jni",
"rand 0.10.2",
"thiserror 2.0.21",
"tinyvec",
"tokio",
"tracing",
"url",
]
[[package]]
name = "hickory-proto"
version = "0.26.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "12b92608f679a6fa515dd1d15c1ff89443026e391200a2c840c7afcba482893d"
dependencies = [
"data-encoding",
"idna",
"ipnet",
"jni",
"once_cell",
"prefix-trie",
"rand 0.10.2",
"ring",
"thiserror 2.0.21",
"tinyvec",
"tracing",
"url",
]
[[package]]
name = "hickory-resolver"
version = "0.26.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3f3da5255c95d5a716857d54b5b8f4e8d67c3484d3beaaaae2ce25063b3ba981"
dependencies = [
"cfg-if",
"futures-util",
"hickory-net",
"hickory-proto",
"ipconfig",
"ipnet",
"jni",
"moka",
"ndk-context",
"once_cell",
"parking_lot 0.12.5",
"rand 0.10.2",
"resolv-conf",
"smallvec",
"system-configuration",
"thiserror 2.0.21",
"tokio",
"tracing",
]
[[package]]
name = "hkdf"
version = "0.12.4"
@@ -2318,7 +2599,7 @@ dependencies = [
"hkdf 0.13.0",
"hybrid-array",
"rand_core 0.10.1",
"sha2 0.11.0",
"sha2 0.11.1",
"subtle",
"x25519-dalek 3.0.0",
"zeroize",
@@ -2802,11 +3083,27 @@ dependencies = [
"rustversion",
]
[[package]]
name = "ipconfig"
version = "0.3.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4d40460c0ce33d6ce4b0630ad68ff63d6661961c48b6dba35e5a4d81cfb48222"
dependencies = [
"socket2",
"widestring",
"windows-registry",
"windows-result",
"windows-sys 0.61.2",
]
[[package]]
name = "ipnet"
version = "2.12.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6a756c3fac73139e83f14c2d742155dd2b78d3ee56597b419a0579b7bdd6dd78"
dependencies = [
"serde",
]
[[package]]
name = "iprange"
@@ -2876,6 +3173,60 @@ version = "1.0.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682"
[[package]]
name = "jiff"
version = "0.2.38"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b2b005715dcbeb0089a3c0dab99f2ff1cc3b2525323552703d648585d342a383"
dependencies = [
"defmt",
"jiff-core",
"jiff-static",
"jiff-tzdb-platform",
"log",
"portable-atomic",
"portable-atomic-util",
"serde_core",
"windows-link",
]
[[package]]
name = "jiff-core"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5e52fe76043ccecc9005d2305ebaadf7d7fc0cc89ca6baa10a94d6bc68c7128c"
dependencies = [
"defmt",
"log",
]
[[package]]
name = "jiff-static"
version = "0.2.38"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2cc9817253cf7c7ee4684451bd327e88d6f3658014e54a29198625590650695c"
dependencies = [
"jiff-core",
"proc-macro2",
"quote",
"syn 2.0.119",
]
[[package]]
name = "jiff-tzdb"
version = "0.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fa8377070c6bae868759445e5a77f66d84f0b72f3a054bfb00e6d038b8282da7"
[[package]]
name = "jiff-tzdb-platform"
version = "0.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "875a5a69ac2bab1a891711cf5eccbec1ce0341ea805560dcd90b7a2e925132e8"
dependencies = [
"jiff-tzdb",
]
[[package]]
name = "jni"
version = "0.22.4"
@@ -3332,6 +3683,15 @@ version = "0.8.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "92daf443525c4cce67b150400bc2316076100ce0b3686209eb8cf3c31612e6f0"
[[package]]
name = "locate-locale"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2835eaaed39a92511442aff277d4dca3d7674ca058df3bc45170661c2ccb4619"
dependencies = [
"winapi",
]
[[package]]
name = "lock_api"
version = "0.4.14"
@@ -3485,7 +3845,7 @@ dependencies = [
"ruma",
"serde",
"serde_json",
"sha2 0.11.0",
"sha2 0.11.1",
"subtle",
"thiserror 2.0.21",
"time",
@@ -3557,11 +3917,17 @@ dependencies = [
"serde",
"serde_bytes",
"serde_json",
"sha2 0.11.0",
"sha2 0.11.1",
"thiserror 2.0.21",
"zeroize",
]
[[package]]
name = "md5"
version = "0.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7ebb8d8732c6a6df3d8f032a82911cfc747e00efb95cc46e8d0acd5b5b88570c"
[[package]]
name = "memchr"
version = "2.8.3"
@@ -3621,6 +3987,16 @@ dependencies = [
"simd-adler32",
]
[[package]]
name = "miniz_oxide"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b63fbc4a50860e98e7b2aa7804ded1db5cbc3aff9193adaff57a6931bf7c4b4c"
dependencies = [
"adler2",
"simd-adler32",
]
[[package]]
name = "mio"
version = "1.2.2"
@@ -3633,6 +4009,23 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "moka"
version = "0.12.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4293f18e7567a1caf3c584855554377025c65e0aa445344d04171f5ad63d19b9"
dependencies = [
"crossbeam-channel",
"crossbeam-epoch",
"crossbeam-utils",
"equivalent",
"parking_lot 0.12.5",
"portable-atomic",
"smallvec",
"tagptr",
"uuid",
]
[[package]]
name = "moxcms"
version = "0.8.1"
@@ -3684,9 +4077,21 @@ version = "1.0.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "650eef8c711430f1a879fdd01d4745a7deea475becfb90269c06775983bbf086"
[[package]]
name = "nix"
version = "0.31.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cf20d2fde8ff38632c426f1165ed7436270b44f199fc55284c38276f9db47c3d"
dependencies = [
"bitflags 2.13.2",
"cfg-if",
"cfg_aliases",
"libc",
]
[[package]]
name = "nobilis"
version = "1.1.0"
version = "1.2.1"
dependencies = [
"aes-gcm",
"anyhow",
@@ -3703,6 +4108,9 @@ dependencies = [
"flate2",
"fs2",
"futures",
"grammers-client",
"grammers-mtsender",
"grammers-session",
"hex",
"hmac 0.13.0",
"http-body-util",
@@ -3818,6 +4226,17 @@ dependencies = [
"num-traits",
]
[[package]]
name = "num-bigint"
version = "0.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "93e7820bc0a80a0238e650327316f929ba18d5be054b647490a3a6a339f3e7c0"
dependencies = [
"num-integer",
"num-traits",
"rand_core 0.10.1",
]
[[package]]
name = "num-bigint-dig"
version = "0.8.6"
@@ -3947,6 +4366,17 @@ dependencies = [
"objc2-foundation",
]
[[package]]
name = "objc2-cloud-kit"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "73ad74d880bb43877038da939b7427bba67e9dd42004a18b809ba7d87cee241c"
dependencies = [
"bitflags 2.13.2",
"objc2",
"objc2-foundation",
]
[[package]]
name = "objc2-core-audio"
version = "0.3.2"
@@ -3970,6 +4400,16 @@ dependencies = [
"objc2",
]
[[package]]
name = "objc2-core-data"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0b402a653efbb5e82ce4df10683b6b28027616a2715e90009947d50b8dd298fa"
dependencies = [
"objc2",
"objc2-foundation",
]
[[package]]
name = "objc2-core-foundation"
version = "0.3.2"
@@ -3983,6 +4423,51 @@ dependencies = [
"objc2",
]
[[package]]
name = "objc2-core-graphics"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e022c9d066895efa1345f8e33e584b9f958da2fd4cd116792e15e07e4720a807"
dependencies = [
"bitflags 2.13.2",
"dispatch2",
"objc2",
"objc2-core-foundation",
"objc2-io-surface",
]
[[package]]
name = "objc2-core-image"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e5d563b38d2b97209f8e861173de434bd0214cf020e3423a52624cd1d989f006"
dependencies = [
"objc2",
"objc2-foundation",
]
[[package]]
name = "objc2-core-location"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ca347214e24bc973fc025fd0d36ebb179ff30536ed1f80252706db19ee452009"
dependencies = [
"objc2",
"objc2-foundation",
]
[[package]]
name = "objc2-core-text"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0cde0dfb48d25d2b4862161a4d5fcc0e3c24367869ad306b0c9ec0073bfed92d"
dependencies = [
"bitflags 2.13.2",
"objc2",
"objc2-core-foundation",
"objc2-core-graphics",
]
[[package]]
name = "objc2-encode"
version = "4.1.0"
@@ -4012,11 +4497,69 @@ dependencies = [
"objc2-core-foundation",
]
[[package]]
name = "objc2-io-surface"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "180788110936d59bab6bd83b6060ffdfffb3b922ba1396b312ae795e1de9d81d"
dependencies = [
"bitflags 2.13.2",
"objc2",
"objc2-core-foundation",
]
[[package]]
name = "objc2-quartz-core"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "96c1358452b371bf9f104e21ec536d37a650eb10f7ee379fff67d2e08d537f1f"
dependencies = [
"bitflags 2.13.2",
"objc2",
"objc2-core-foundation",
"objc2-foundation",
]
[[package]]
name = "objc2-ui-kit"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d87d638e33c06f577498cbcc50491496a3ed4246998a7fbba7ccb98b1e7eab22"
dependencies = [
"bitflags 2.13.2",
"block2",
"objc2",
"objc2-cloud-kit",
"objc2-core-data",
"objc2-core-foundation",
"objc2-core-graphics",
"objc2-core-image",
"objc2-core-location",
"objc2-core-text",
"objc2-foundation",
"objc2-quartz-core",
"objc2-user-notifications",
]
[[package]]
name = "objc2-user-notifications"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9df9128cbbfef73cda168416ccf7f837b62737d748333bfe9ab71c245d76613e"
dependencies = [
"objc2",
"objc2-foundation",
]
[[package]]
name = "once_cell"
version = "1.21.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50"
dependencies = [
"critical-section",
"portable-atomic",
]
[[package]]
name = "oneshot-fused-workaround"
@@ -4143,6 +4686,21 @@ dependencies = [
"num-traits",
]
[[package]]
name = "os_info"
version = "3.15.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9cf20a545b305cf1da722b236b5155c9bb35f1d5ceb28c048bd96ca842f41b5b"
dependencies = [
"android_system_properties",
"log",
"nix",
"objc2",
"objc2-foundation",
"objc2-ui-kit",
"windows-sys 0.61.2",
]
[[package]]
name = "os_str_bytes"
version = "6.6.1"
@@ -4284,6 +4842,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "112d82ceb8c5bf524d9af484d4e4970c9fd5a0cc15ba14ad93dccd28873b0629"
dependencies = [
"digest 0.11.3",
"hmac 0.13.0",
]
[[package]]
@@ -4452,7 +5011,7 @@ dependencies = [
"crc32fast",
"fdeflate",
"flate2",
"miniz_oxide",
"miniz_oxide 0.8.9",
]
[[package]]
@@ -4499,6 +5058,21 @@ dependencies = [
"universal-hash 0.6.1",
]
[[package]]
name = "portable-atomic"
version = "1.15.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "05c8b63e8d9609db387f0324918f81d68fe27748f084ef092fb35954d0539a85"
[[package]]
name = "portable-atomic-util"
version = "0.2.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "10ab3eb7f3becc3a1cbc4f2c6f20267996cfc1a6467a873763411b136a122715"
dependencies = [
"portable-atomic",
]
[[package]]
name = "postage"
version = "0.5.0"
@@ -4544,6 +5118,17 @@ version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "925383efa346730478fb4838dbe9137d2a47675ad789c546d150a6e1dd4ab31c"
[[package]]
name = "prefix-trie"
version = "0.8.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4cf6e3177f0684016a5c209b00882e15f8bdd3f3bb48f0491df10cd102d0c6e7"
dependencies = [
"either",
"ipnet",
"num-traits",
]
[[package]]
name = "primeorder"
version = "0.13.6"
@@ -5044,6 +5629,12 @@ dependencies = [
"rand_core 0.10.1",
]
[[package]]
name = "resolv-conf"
version = "0.7.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e061d1b48cb8d38042de4ae0a7a6401009d6143dc80d2e2d6f31f0bdd6470c7"
[[package]]
name = "retry-error"
version = "0.14.0"
@@ -5512,7 +6103,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b7f4bc775c73d9a02cde8bf7b2ec4c9d12743edf609006c7facc23998404cd1d"
dependencies = [
"bitflags 2.13.2",
"core-foundation",
"core-foundation 0.10.1",
"core-foundation-sys",
"libc",
"security-framework-sys",
@@ -5738,9 +6329,9 @@ dependencies = [
[[package]]
name = "sha2"
version = "0.11.0"
version = "0.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "446ba717509524cb3f22f17ecc096f10f4822d76ab5c0b9822c5f9c284e825f4"
checksum = "47d7069beb7d6ac7b9acd1039986e73443f24234f41074da099d6f994ac9ad19"
dependencies = [
"cfg-if",
"cpufeatures 0.3.0",
@@ -6270,6 +6861,33 @@ dependencies = [
"windows",
]
[[package]]
name = "system-configuration"
version = "0.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a13f3d0daba03132c0aa9767f98351b3488edc2c100cda2d2ec2b04f3d8d3c8b"
dependencies = [
"bitflags 2.13.2",
"core-foundation 0.9.4",
"system-configuration-sys",
]
[[package]]
name = "system-configuration-sys"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8e1d1b10ced5ca923a1fcb8d03e96b8d3268065d724548c0211415ff6ac6bac4"
dependencies = [
"core-foundation-sys",
"libc",
]
[[package]]
name = "tagptr"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7b2093cf4c8eb1e67749a6762251bc9cd836b6fc171623bd0a9d324d37af2417"
[[package]]
name = "tap"
version = "1.0.1"
@@ -6428,9 +7046,9 @@ dependencies = [
[[package]]
name = "tokio"
version = "1.53.1"
version = "1.53.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "202caea871b69668250d242070849eb495be178ed697a3e98aebce5bc81a0bed"
checksum = "ce3335fa71841cda333a58d7615b03901380ecf09d59b3296d21f8bbac0dde4e"
dependencies = [
"bytes",
"libc",
@@ -8009,7 +8627,7 @@ dependencies = [
"serde",
"serde_bytes",
"serde_json",
"sha2 0.11.0",
"sha2 0.11.1",
"subtle",
"thiserror 2.0.21",
"x25519-dalek 3.0.0",
@@ -8219,6 +8837,12 @@ dependencies = [
"safe_arch",
]
[[package]]
name = "widestring"
version = "1.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "72069c3113ab32ab29e5584db3c6ec55d416895e60715417b5b883a357c3e471"
[[package]]
name = "wildmatch"
version = "2.6.1"
@@ -8339,6 +8963,17 @@ dependencies = [
"windows-link",
]
[[package]]
name = "windows-registry"
version = "0.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "02752bf7fbdcce7f2a27a742f798510f3e5ad88dbe84871e5168e2120c3d5720"
dependencies = [
"windows-link",
"windows-result",
"windows-strings",
]
[[package]]
name = "windows-result"
version = "0.4.1"
@@ -8784,6 +9419,12 @@ dependencies = [
"syn 2.0.119",
]
[[package]]
name = "zlib-rs"
version = "0.6.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b268e58e7c693d7c271f93ffc4ba3b380412554231c85bf61ca7af91042a4112"
[[package]]
name = "zmij"
version = "1.0.23"
Cargo.toml
+7 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "nobilis"
version = "1.1.0"
version = "1.2.1"
edition = "2021"
description = "Chat daemon: IRC, Discord, Sneedchat and Matrix behind one unified JSON model over a Unix socket"
license = "GPL-3.0-or-later"
@@ -59,6 +59,12 @@ qrcode = { version = "0.14", features = ["image"] }
image = { version = "0.25", default-features = false, features = ["png", "gif"] }
chrono = { version = "0.4", default-features = false, features = ["std"] }
# Telegram backend (MTProto, pure Rust). `grammers-session` without its default `sqlite-storage` (libsql would sit
# beside rusqlite and matrix-sdk-sqlite): the session is serialized into the account instead - see backend/telegram.
grammers-client = { version = "=0.11.0", default-features = false, features = ["proxy", "fs"] }
grammers-mtsender = { version = "=0.11.0", default-features = false, features = ["proxy"] }
grammers-session = { version = "=0.11.0", default-features = false, features = ["serde"] }
# Sneedchat backend (SneedChat over embedded Tor)
arti-client = { version = "0.47", default-features = false, features = ["tokio", "rustls", "onion-service-client", "compression"] }
tor-rtcompat = { version = "0.47", features = ["tokio", "rustls"] }
src/accounts.rs
+505 -7
View File
@@ -390,9 +390,8 @@ pub struct SteamAccountConfig {
/// sign-in so Steam Guard does not ask for a code every time.
#[serde(default)]
pub guard_data: Option<String>,
/// Which kind of session this account signed in as. Always `chat`, the
/// mobile app's, which lives beside the Steam launcher; kept so files
/// written when the desktop kind could still be chosen read the same.
/// Which kind of session this account signed in as. Unused now: moho signs in as a desktop client of its own
/// (see backend/steam/auth.rs); kept so files written when it was chosen read the same.
#[serde(default)]
pub session: String,
/// Whether this account's connections go through Tor or the SOCKS5 proxy
@@ -411,6 +410,133 @@ impl SteamAccountConfig {
}
}
/// A Twitch account: a signed-in person, or an anonymous reader.
///
/// What is kept is the pair of tokens the device flow handed back (see
/// backend/twitch/auth.rs), never a password - Twitch's sign-in happens on
/// twitch.tv, in the person's own browser, and moho only ever sees the result.
/// The access token lasts four hours and is renewed with the refresh token,
/// which Twitch lets go of after thirty days unused.
///
/// An anonymous reader has neither and reads every channel, as a signed-out
/// Kick account does; sending needs the sign-in.
#[derive(Serialize, Deserialize, Clone, Debug, Default)]
pub struct TwitchAccountConfig {
/// The login, lower-case: Twitch does not tell "Someone" and "someone"
/// apart, and neither can the id built from it. Empty when anonymous.
#[serde(default)]
pub login: String,
/// Twitch's own number for the account, which is what its API is keyed by
/// and which survives the login being renamed.
#[serde(default)]
pub user_id: String,
/// What to call this account here. Local: Twitch is never told about it.
#[serde(default)]
pub display_name: Option<String>,
#[serde(default)]
pub access_token: String,
#[serde(default)]
pub refresh_token: String,
/// When the access token stops working, in Unix seconds.
#[serde(default)]
pub expires_at: i64,
/// What the person agreed to when they signed in.
#[serde(default)]
pub scopes: Vec<String>,
/// The channels this account has open, by login, so they come back.
#[serde(default)]
pub channels: Vec<String>,
/// The `auth-token` of a twitch.tv session this person signed in to for
/// video, if they did. A credential of the same standing as the tokens
/// above and kept the same way; it is what makes their Turbo or their
/// subscriptions count when a stream is played (see backend/twitch/playback.rs).
#[serde(default)]
pub web_token: String,
/// Whether the channels this account follows have been opened already. Once, at first connection: a list
/// read again every time would put back every channel the person had closed.
#[serde(default)]
pub followed_synced: bool,
/// An account with no sign-in, which reads and does not send.
#[serde(default)]
pub anonymous: bool,
/// Whether this account's connections go through Tor or the SOCKS5 proxy
/// set in the network settings. Off by default.
#[serde(default)]
pub use_tor: bool,
/// Strict routing: everything the window loads for this account's
/// conversations goes through the route too.
#[serde(default)]
pub strict_route: bool,
}
/// The id of the one anonymous Twitch account. A `~` cannot be in a Twitch
/// login, so no signed-in account can ever have it.
pub const TWITCH_ANONYMOUS_ID: &str = "twitch:~anonymous";
impl TwitchAccountConfig {
pub fn account_id(&self) -> String {
if self.anonymous {
TWITCH_ANONYMOUS_ID.to_string()
} else {
format!("twitch:{}", self.login)
}
}
/// Whether there is a sign-in behind this account: what "signed in" means
/// for one that never keeps a password.
pub fn is_signed_in(&self) -> bool {
!self.anonymous && !self.access_token.is_empty()
}
}
/// A Telegram account: a phone number's, signed in over MTProto (see backend/telegram).
///
/// Telegram gives the owner of a phone number one `api_id` and `api_hash`, and a client sends them with every
/// connection. Moho carries none of its own: the ones the person typed into the sign-in form are kept here, with
/// the account, and are as private as the session is. The session itself (the authorization key Telegram issued,
/// and what a client caches to catch up on updates) is large and changes often, so it is a file of its own in the
/// daemon's data directory (see backend/telegram/session.rs), not a field of this one.
#[derive(Serialize, Deserialize, Clone, Debug, Default)]
pub struct TelegramAccountConfig {
#[serde(default)]
pub api_id: i32,
#[serde(default)]
pub api_hash: String,
/// The number the account was signed in with, in international form (`+15551234567`).
#[serde(default)]
pub phone: String,
/// Telegram's number for the account, which survives the number or the name being changed. The account's id here
/// is built from it.
#[serde(default)]
pub user_id: i64,
/// The `@name`, if the account has one. Telegram's, not ours.
#[serde(default)]
pub username: Option<String>,
/// The account's own name in Telegram (first and last), for the sidebar.
#[serde(default)]
pub name: String,
/// What to call this account here. Local: Telegram is never told about it.
#[serde(default)]
pub display_name: Option<String>,
/// Whether this account's connections go through Tor or the SOCKS5 proxy set in the network settings.
#[serde(default)]
pub use_tor: bool,
/// Strict routing: everything the window loads for this account's conversations goes through the route too.
#[serde(default)]
pub strict_route: bool,
}
impl TelegramAccountConfig {
pub fn account_id(&self) -> String {
format!("telegram:{}", self.user_id)
}
/// Whether there is a sign-in behind this account: one that has been through is what has an id.
pub fn is_signed_in(&self) -> bool {
self.user_id != 0 && self.api_id != 0 && !self.api_hash.is_empty()
}
}
#[derive(Serialize, Deserialize, Clone, Debug, Default)]
struct AccountsFile {
#[serde(default, rename = "account")]
@@ -429,6 +555,10 @@ struct AccountsFile {
kick: Vec<KickAccountConfig>,
#[serde(default, rename = "steam_account")]
steam: Vec<SteamAccountConfig>,
#[serde(default, rename = "twitch_account")]
twitch: Vec<TwitchAccountConfig>,
#[serde(default, rename = "telegram_account")]
telegram: Vec<TelegramAccountConfig>,
}
/// Account/credential persistence at ~/.config/nobilis/accounts.toml,
@@ -446,6 +576,10 @@ pub struct AccountStore {
/// predate Steam did not each need a sixth argument. Anything that
/// changes this map must let go of its lock before it calls `persist`.
steam: Mutex<HashMap<String, SteamAccountConfig>>,
/// Read by `persist` the same way, for the same reason.
twitch: Mutex<HashMap<String, TwitchAccountConfig>>,
/// And this one.
telegram: Mutex<HashMap<String, TelegramAccountConfig>>,
}
impl AccountStore {
@@ -456,7 +590,7 @@ impl AccountStore {
}
pub fn open(path: PathBuf) -> Result<Self> {
let (irc, discord, sneedchat, matrix, kick, steam) = if path.exists() {
let (irc, discord, sneedchat, matrix, kick, steam, twitch, telegram) = if path.exists() {
let text = std::fs::read_to_string(&path)
.with_context(|| format!("reading {}", path.display()))?;
let file: AccountsFile = toml::from_str(&text)
@@ -467,11 +601,23 @@ impl AccountStore {
let matrix = file.matrix.into_iter().map(|a| (a.account_id(), a)).collect();
let kick = file.kick.into_iter().map(|a| (a.account_id(), a)).collect();
let steam = file.steam.into_iter().map(|a| (a.account_id(), a)).collect();
(irc, discord, sneedchat, matrix, kick, steam)
let twitch = file.twitch.into_iter().map(|a| (a.account_id(), a)).collect();
let telegram = file.telegram.into_iter().map(|a| (a.account_id(), a)).collect();
(irc, discord, sneedchat, matrix, kick, steam, twitch, telegram)
} else {
(HashMap::new(), HashMap::new(), HashMap::new(), HashMap::new(), HashMap::new(), HashMap::new())
(HashMap::new(), HashMap::new(), HashMap::new(), HashMap::new(), HashMap::new(), HashMap::new(), HashMap::new(), HashMap::new())
};
Ok(Self { path, irc: Mutex::new(irc), discord: Mutex::new(discord), sneedchat: Mutex::new(sneedchat), matrix: Mutex::new(matrix), kick: Mutex::new(kick), steam: Mutex::new(steam) })
Ok(Self {
path,
irc: Mutex::new(irc),
discord: Mutex::new(discord),
sneedchat: Mutex::new(sneedchat),
matrix: Mutex::new(matrix),
kick: Mutex::new(kick),
steam: Mutex::new(steam),
twitch: Mutex::new(twitch),
telegram: Mutex::new(telegram),
})
}
/// Callers already hold whichever map they just mutated - the others
@@ -492,6 +638,8 @@ impl AccountStore {
matrix: matrix.values().cloned().collect(),
kick: kick.values().cloned().collect(),
steam: self.steam.lock().unwrap().values().cloned().collect(),
twitch: self.twitch.lock().unwrap().values().cloned().collect(),
telegram: self.telegram.lock().unwrap().values().cloned().collect(),
};
let text = toml::to_string_pretty(&file)?;
// Atomic write-temp-then-rename, same spirit as libpurple's periodic
@@ -680,6 +828,97 @@ impl AccountStore {
Ok(true)
}
pub fn get_twitch(&self, account_id: &str) -> Option<TwitchAccountConfig> {
self.twitch.lock().unwrap().get(account_id).cloned()
}
pub fn all_twitch(&self) -> Vec<TwitchAccountConfig> {
self.twitch.lock().unwrap().values().cloned().collect()
}
/// Upserts: signing in again with the same login is how a token Twitch has
/// stopped accepting gets replaced. What the person set up on the account
/// - its route, its name here, the channels it had open - stays.
pub fn add_twitch(&self, mut config: TwitchAccountConfig) -> Result<TwitchAccountConfig> {
config.login = config.login.trim().to_lowercase();
let id = config.account_id();
{
let mut twitch = self.twitch.lock().unwrap();
if let Some(old) = twitch.get(&id) {
config.strict_route |= old.strict_route;
config.use_tor |= old.use_tor;
if config.display_name.is_none() {
config.display_name = old.display_name.clone();
}
if config.channels.is_empty() {
config.channels = old.channels.clone();
}
if config.web_token.is_empty() {
config.web_token = old.web_token.clone();
}
}
twitch.insert(id, config.clone());
}
self.persist(&self.irc.lock().unwrap(), &self.discord.lock().unwrap(), &self.sneedchat.lock().unwrap(), &self.matrix.lock().unwrap(), &self.kick.lock().unwrap())?;
Ok(config)
}
/// Changes one Twitch account, and writes the file. `false` when there is
/// no such account. The lock is let go before the write.
pub fn mutate_twitch(&self, account_id: &str, change: impl FnOnce(&mut TwitchAccountConfig)) -> Result<bool> {
{
let mut twitch = self.twitch.lock().unwrap();
match twitch.get_mut(account_id) {
None => return Ok(false),
Some(account) => change(account),
}
}
self.persist(&self.irc.lock().unwrap(), &self.discord.lock().unwrap(), &self.sneedchat.lock().unwrap(), &self.matrix.lock().unwrap(), &self.kick.lock().unwrap())?;
Ok(true)
}
pub fn get_telegram(&self, account_id: &str) -> Option<TelegramAccountConfig> {
self.telegram.lock().unwrap().get(account_id).cloned()
}
pub fn all_telegram(&self) -> Vec<TelegramAccountConfig> {
self.telegram.lock().unwrap().values().cloned().collect()
}
/// Upserts: signing in again as the same account is how a session Telegram has ended gets replaced. What the
/// person set up on the account - its route, its name here - stays.
pub fn add_telegram(&self, mut config: TelegramAccountConfig) -> Result<TelegramAccountConfig> {
let id = config.account_id();
{
let mut telegram = self.telegram.lock().unwrap();
if let Some(old) = telegram.get(&id) {
config.strict_route |= old.strict_route;
config.use_tor |= old.use_tor;
if config.display_name.is_none() {
config.display_name = old.display_name.clone();
}
}
telegram.insert(id, config.clone());
}
self.persist(&self.irc.lock().unwrap(), &self.discord.lock().unwrap(), &self.sneedchat.lock().unwrap(), &self.matrix.lock().unwrap(), &self.kick.lock().unwrap())?;
Ok(config)
}
/// Changes one Telegram account, and writes the file. `false` when there is no such account.
///
/// The lock is let go before the write: `persist` takes it again.
pub fn mutate_telegram(&self, account_id: &str, change: impl FnOnce(&mut TelegramAccountConfig)) -> Result<bool> {
{
let mut telegram = self.telegram.lock().unwrap();
match telegram.get_mut(account_id) {
None => return Ok(false),
Some(account) => change(account),
}
}
self.persist(&self.irc.lock().unwrap(), &self.discord.lock().unwrap(), &self.sneedchat.lock().unwrap(), &self.matrix.lock().unwrap(), &self.kick.lock().unwrap())?;
Ok(true)
}
/// Upserts, like the others: signing in to Kick again with the same
/// username is how a rejected token gets replaced.
pub fn add_kick(&self, mut config: KickAccountConfig) -> Result<KickAccountConfig> {
@@ -818,6 +1057,14 @@ impl AccountStore {
a.use_tor = routed;
a.strict_route = strict;
true
} else if let Some(a) = self.twitch.lock().unwrap().get_mut(account_id) {
a.use_tor = routed;
a.strict_route = strict;
true
} else if let Some(a) = self.telegram.lock().unwrap().get_mut(account_id) {
a.use_tor = routed;
a.strict_route = strict;
true
} else {
false
};
@@ -845,6 +1092,12 @@ impl AccountStore {
if let Some(a) = self.get_steam(account_id) {
return pick(a.use_tor, a.strict_route);
}
if let Some(a) = self.get_twitch(account_id) {
return pick(a.use_tor, a.strict_route);
}
if let Some(a) = self.get_telegram(account_id) {
return pick(a.use_tor, a.strict_route);
}
self.get_kick(account_id).and_then(|a| pick(a.use_tor, a.strict_route))
}
@@ -925,6 +1178,20 @@ impl AccountStore {
return Ok(true);
}
}
{
let removed = self.twitch.lock().unwrap().remove(account_id).is_some();
if removed {
self.persist(&self.irc.lock().unwrap(), &self.discord.lock().unwrap(), &self.sneedchat.lock().unwrap(), &self.matrix.lock().unwrap(), &self.kick.lock().unwrap())?;
return Ok(true);
}
}
{
let removed = self.telegram.lock().unwrap().remove(account_id).is_some();
if removed {
self.persist(&self.irc.lock().unwrap(), &self.discord.lock().unwrap(), &self.sneedchat.lock().unwrap(), &self.matrix.lock().unwrap(), &self.kick.lock().unwrap())?;
return Ok(true);
}
}
let mut kick = self.kick.lock().unwrap();
if kick.remove(account_id).is_some() {
self.persist(&self.irc.lock().unwrap(), &self.discord.lock().unwrap(), &self.sneedchat.lock().unwrap(), &self.matrix.lock().unwrap(), &kick)?;
@@ -1086,6 +1353,12 @@ impl AccountStore {
if self.mutate_steam(account_id, |a| a.display_name = value.clone())? {
return Ok(true);
}
if self.mutate_twitch(account_id, |a| a.display_name = value.clone())? {
return Ok(true);
}
if self.mutate_telegram(account_id, |a| a.display_name = value.clone())? {
return Ok(true);
}
let mut kick = self.kick.lock().unwrap();
match kick.get_mut(account_id) {
None => Ok(false),
@@ -1119,6 +1392,12 @@ impl AccountStore {
if let Some(a) = self.steam.lock().unwrap().get(account_id) {
return named(&a.display_name);
}
if let Some(a) = self.twitch.lock().unwrap().get(account_id) {
return named(&a.display_name);
}
if let Some(a) = self.telegram.lock().unwrap().get(account_id) {
return named(&a.display_name);
}
self.kick.lock().unwrap().get(account_id).and_then(|a| named(&a.display_name))
}
@@ -1238,6 +1517,8 @@ pub fn irc_account_to_json(a: &IrcAccountConfig, state: &str) -> Account {
rtc_focus_url: None,
sliding_sync: false,
dehydration: false,
web_session: false,
moderation: false,
}
}
@@ -1277,6 +1558,8 @@ pub fn discord_account_to_json(a: &DiscordAccountConfig, state: &str) -> Account
rtc_focus_url: None,
sliding_sync: false,
dehydration: false,
web_session: false,
moderation: false,
}
}
@@ -1315,6 +1598,8 @@ pub fn sneedchat_account_to_json(a: &SneedChatAccountConfig, state: &str) -> Acc
rtc_focus_url: None,
sliding_sync: false,
dehydration: false,
web_session: false,
moderation: false,
}
}
@@ -1362,6 +1647,8 @@ pub fn kick_account_to_json(a: &KickAccountConfig, state: &str) -> Account {
rtc_focus_url: None,
sliding_sync: false,
dehydration: false,
web_session: false,
moderation: false,
}
}
@@ -1408,6 +1695,95 @@ pub fn steam_account_to_json(a: &SteamAccountConfig, state: &str, persona_name:
rtc_focus_url: None,
sliding_sync: false,
dehydration: false,
web_session: false,
moderation: false,
}
}
/// Twitch has none of IRC's concepts either. `has_password` says there is a
/// token, which is what "signed in" means for an account that never keeps a
/// password; an anonymous reader has none and says so.
pub fn twitch_account_to_json(a: &TwitchAccountConfig, state: &str) -> Account {
Account {
id: a.account_id(),
service: "twitch".to_string(),
status: "online".to_string(),
status_text: String::new(),
auto_idle: false,
display_name: a
.display_name
.clone()
.filter(|s| !s.is_empty())
.unwrap_or_else(|| if a.anonymous { "Anonymous".to_string() } else { a.login.clone() }),
state: state.to_string(),
autojoin: String::new(),
has_nickserv_password: false,
sasl_enabled: false,
sasl_username: String::new(),
allow_plaintext_sasl: false,
reconnect_for_sasl: false,
sasl_mechanism: String::new(),
has_sasl_certificate: false,
current_nick: String::new(),
highlight_keywords: Vec::new(),
ssl: true,
has_password: a.is_signed_in(),
avatar_url: None,
sneedchat_rooms: Vec::new(),
tor_mode: None,
tor_proxy: None,
use_tor: a.use_tor,
route_level: route_level(a.use_tor, a.strict_route),
has_key_backup: false,
rtc_focus_url: None,
sliding_sync: false,
dehydration: false,
web_session: !a.web_token.is_empty(),
moderation: crate::backend::twitch::auth::has_moderation(&a.scopes),
}
}
/// Telegram has none of IRC's concepts either. `has_password` says there is a sign-in behind the account, which is
/// what "signed in" means for one that never keeps a password.
pub fn telegram_account_to_json(a: &TelegramAccountConfig, state: &str) -> Account {
Account {
id: a.account_id(),
service: "telegram".to_string(),
status: "online".to_string(),
status_text: String::new(),
auto_idle: false,
display_name: a
.display_name
.clone()
.filter(|s| !s.is_empty())
.or_else(|| Some(a.name.clone()).filter(|s| !s.is_empty()))
.or_else(|| a.username.clone().filter(|s| !s.is_empty()))
.unwrap_or_else(|| a.phone.clone()),
state: state.to_string(),
autojoin: String::new(),
has_nickserv_password: false,
sasl_enabled: false,
sasl_username: String::new(),
allow_plaintext_sasl: false,
reconnect_for_sasl: false,
sasl_mechanism: String::new(),
has_sasl_certificate: false,
current_nick: String::new(),
highlight_keywords: Vec::new(),
ssl: true,
has_password: a.is_signed_in(),
avatar_url: None,
sneedchat_rooms: Vec::new(),
tor_mode: None,
tor_proxy: None,
use_tor: a.use_tor,
route_level: route_level(a.use_tor, a.strict_route),
has_key_backup: false,
rtc_focus_url: None,
sliding_sync: false,
dehydration: false,
web_session: false,
moderation: false,
}
}
@@ -1450,6 +1826,8 @@ pub fn matrix_account_to_json(a: &MatrixAccountConfig, state: &str, has_key_back
rtc_focus_url: a.rtc_focus_url.clone(),
sliding_sync: a.prefer_sliding_sync,
dehydration: a.dehydration_enabled,
web_session: false,
moderation: false,
}
}
@@ -1599,3 +1977,123 @@ mod steam_account_tests {
let _ = std::fs::remove_file(path);
}
}
#[cfg(test)]
mod twitch_account_tests {
use super::*;
fn store(name: &str) -> AccountStore {
let path = std::env::temp_dir().join(format!("nobilis-twitch-accounts-{name}-{}.toml", std::process::id()));
let _ = std::fs::remove_file(&path);
AccountStore::open(path).unwrap()
}
fn account(login: &str, access: &str) -> TwitchAccountConfig {
TwitchAccountConfig { login: login.into(), user_id: "42".into(), access_token: access.into(), refresh_token: "ref".into(), expires_at: 1000, scopes: vec!["chat:read".into()], ..Default::default() }
}
#[test]
fn a_twitch_account_survives_a_restart_with_its_tokens() {
let s = store("restart");
s.add_twitch(account("Someone", "acc")).unwrap();
let text = std::fs::read_to_string(s.path_for_tests()).unwrap();
assert!(text.contains("[[twitch_account]]"), "{text}");
let reopened = AccountStore::open(s.path_for_tests().to_path_buf()).unwrap();
let got = reopened.get_twitch("twitch:someone").expect("found under its lower-case id");
assert_eq!((got.login.as_str(), got.user_id.as_str(), got.access_token.as_str(), got.scopes.len()), ("someone", "42", "acc", 1));
assert!(got.is_signed_in());
}
#[test]
fn an_anonymous_reader_has_an_id_no_login_can_have() {
let s = store("anon");
let saved = s.add_twitch(TwitchAccountConfig { anonymous: true, ..Default::default() }).unwrap();
assert_eq!(saved.account_id(), TWITCH_ANONYMOUS_ID);
assert!(!saved.is_signed_in());
assert!(!TWITCH_ANONYMOUS_ID.trim_start_matches("twitch:").chars().all(|c| c.is_ascii_alphanumeric() || c == '_'));
let json = twitch_account_to_json(&saved, "connected");
assert_eq!((json.service.as_str(), json.display_name.as_str(), json.has_password), ("twitch", "Anonymous", false));
}
#[test]
fn signing_in_again_keeps_the_route_the_name_and_the_open_channels() {
let s = store("again");
s.add_twitch(account("someone", "old")).unwrap();
s.mutate_twitch("twitch:someone", |a| {
a.channels = vec!["xqc".into()];
a.display_name = Some("Mine".into());
})
.unwrap();
s.set_route_level("twitch:someone", "strict").unwrap();
let again = s.add_twitch(account("SOMEONE", "new")).unwrap();
assert_eq!(s.all_twitch().len(), 1, "the same login is the same account");
assert_eq!((again.access_token.as_str(), again.channels.clone(), again.display_name.clone()), ("new", vec!["xqc".to_string()], Some("Mine".to_string())));
assert!(again.strict_route && again.use_tor, "a sign-in does not take a route away");
}
#[test]
fn removing_a_twitch_account_removes_it_from_the_file() {
let s = store("remove");
s.add_twitch(account("someone", "acc")).unwrap();
assert!(s.remove("twitch:someone").unwrap());
assert!(s.get_twitch("twitch:someone").is_none());
let reopened = AccountStore::open(s.path_for_tests().to_path_buf()).unwrap();
assert!(reopened.all_twitch().is_empty());
}
}
#[cfg(test)]
mod telegram_account_tests {
use super::*;
fn store(name: &str) -> (AccountStore, PathBuf) {
let path = std::env::temp_dir().join(format!("nobilis-telegram-accounts-{name}-{}.toml", std::process::id()));
let _ = std::fs::remove_file(&path);
(AccountStore::open(path.clone()).unwrap(), path)
}
fn account(user_id: i64) -> TelegramAccountConfig {
TelegramAccountConfig { api_id: 12345, api_hash: "abcdef".into(), phone: "+15551234567".into(), user_id, name: "Some One".into(), username: Some("someone".into()), ..Default::default() }
}
#[test]
fn a_telegram_account_survives_a_restart_without_its_session_in_the_file() {
let (s, path) = store("restart");
s.add_telegram(account(777)).unwrap();
let text = std::fs::read_to_string(&path).unwrap();
assert!(text.contains("[[telegram_account]]"), "{text}");
assert!(!text.contains("session"), "the session is a file of its own, not a field here: {text}");
let reopened = AccountStore::open(path.clone()).unwrap();
let got = reopened.get_telegram("telegram:777").expect("found under the id built from Telegram's number");
assert_eq!((got.api_id, got.api_hash.as_str(), got.phone.as_str(), got.username.as_deref()), (12345, "abcdef", "+15551234567", Some("someone")));
assert!(got.is_signed_in());
std::fs::remove_file(path).ok();
}
#[test]
fn signing_in_again_keeps_what_the_person_set_up_and_removing_takes_only_that_account() {
let (s, path) = store("again");
s.add_telegram(account(1)).unwrap();
s.add_telegram(account(2)).unwrap();
s.set_route_level("telegram:1", "strict").unwrap();
s.set_display_name("telegram:1", "Me on Telegram").unwrap();
s.add_telegram(TelegramAccountConfig { name: "Renamed".into(), ..account(1) }).unwrap();
let got = s.get_telegram("telegram:1").unwrap();
assert_eq!((got.name.as_str(), got.display_name.as_deref(), got.strict_route), ("Renamed", Some("Me on Telegram"), true));
assert_eq!(s.route_level_of("telegram:1").as_deref(), Some("strict"));
assert_eq!(s.all_telegram().len(), 2);
assert!(s.remove("telegram:1").unwrap());
assert!(!s.remove("telegram:1").unwrap());
assert!(s.get_telegram("telegram:2").is_some());
assert!(AccountStore::open(path.clone()).unwrap().get_telegram("telegram:1").is_none(), "the file forgot too");
std::fs::remove_file(path).ok();
}
#[test]
fn the_listed_account_is_named_for_the_person_and_says_it_is_signed_in() {
let listed = telegram_account_to_json(&account(5), "connected");
assert_eq!((listed.service.as_str(), listed.id.as_str(), listed.display_name.as_str(), listed.has_password), ("telegram", "telegram:5", "Some One", true));
let unnamed = telegram_account_to_json(&TelegramAccountConfig { name: String::new(), username: None, ..account(5) }, "connected");
assert_eq!(unnamed.display_name, "+15551234567");
}
}
src/backend/discord/guilds.rs
+49 -11
View File
@@ -107,6 +107,27 @@ pub async fn member_verification(state: &AppState, account_id: &str, guild_id: &
serde_json::from_str(&text).context("parsing this server's rules")
}
/// The form's fields with the answers filled in: the rules are agreed to, and every other field takes what the person
/// gave for it (their text, or the number of the choice), by its place in the form. A field nobody answered is sent
/// without a response, which Discord accepts only where the server did not require one.
pub fn responses_for(form: &Value, answers: &[Value]) -> Vec<Value> {
form["form_fields"]
.as_array()
.into_iter()
.flatten()
.enumerate()
.map(|(i, f)| {
let mut field = f.clone();
if f["field_type"] == "TERMS" {
field["response"] = Value::Bool(true);
} else if let Some(answer) = answers.get(i).filter(|a| !a.is_null()) {
field["response"] = answer.clone();
}
field
})
.collect()
}
/// Agrees to them, which is what lifts the gate.
///
/// The form is fetched again rather than taken from the caller: what is
@@ -114,20 +135,11 @@ pub async fn member_verification(state: &AppState, account_id: &str, guild_id: &
/// captured when the dialog opened may not still be it. Every field is
/// returned as sent with its response set - Discord rejects a form that has
/// been reshaped, and it is the server's form, not ours to edit.
pub async fn accept_member_verification(state: &AppState, account_id: &str, guild_id: &str) -> Result<()> {
pub async fn accept_member_verification(state: &AppState, account_id: &str, guild_id: &str, answers: &[Value]) -> Result<()> {
let cfg = state.accounts.get_discord(account_id).context("account not connected")?;
let form = member_verification(state, account_id, guild_id).await?;
let version = form["version"].clone();
let fields: Vec<Value> = form["form_fields"]
.as_array()
.into_iter()
.flatten()
.map(|f| {
let mut field = f.clone();
field["response"] = Value::Bool(true);
field
})
.collect();
let fields = responses_for(&form, answers);
let resp = send_write(
http_client_for(&cfg.token)
@@ -1217,3 +1229,29 @@ mod system_dm_tests {
assert!(!is_system_dm(&json!({ "recipients": [{ "id": "1", "system": true }, { "id": "2" }] })));
}
}
#[cfg(test)]
mod verification_tests {
use super::*;
#[test]
fn the_rules_are_agreed_to_and_other_fields_take_the_persons_answers() {
let form = json!({
"version": "2026-01-01",
"form_fields": [
{ "field_type": "TERMS", "label": "Read and agree to the server rules", "values": ["Be kind"], "required": true },
{ "field_type": "TEXT_INPUT", "label": "Why are you joining?", "required": true },
{ "field_type": "MULTIPLE_CHOICE", "label": "Pick one", "values": ["a", "b"], "required": false },
]
});
let sent = responses_for(&form, &[Value::Null, json!("to talk"), json!(1)]);
assert_eq!(sent[0]["response"], true);
assert_eq!(sent[1]["response"], "to talk");
assert_eq!(sent[2]["response"], 1);
assert_eq!(sent[0]["values"], json!(["Be kind"]), "the server's own form is not reshaped");
// Nothing answered: no response key at all, not a made-up one.
let bare = responses_for(&form, &[]);
assert!(bare[1].get("response").is_none() && bare[2].get("response").is_none());
assert_eq!(bare[0]["response"], true);
}
}
src/backend/discord/media.rs
+1
View File
@@ -629,6 +629,7 @@ mod tests {
buffer_id: "b".into(),
html: None,
components: Vec::new(),
moderated: None,
sender_color: None,
badges: Vec::new(),
from: "x".into(),
src/backend/discord/messages.rs
+3 -1
View File
@@ -909,6 +909,8 @@ pub(super) fn extract_reply(d: &Value) -> Option<ReplyPreview> {
body: String::new(),
thread: false,
forwarded: true,
thread_root: None,
chat: None,
});
}
let referenced = &d["referenced_message"];
@@ -920,7 +922,7 @@ pub(super) fn extract_reply(d: &Value) -> Option<ReplyPreview> {
let author = &referenced["author"];
let from = author["global_name"].as_str().filter(|s| !s.is_empty()).or_else(|| author["username"].as_str()).unwrap_or("unknown").to_string();
let body = extract_body(referenced).unwrap_or_default();
Some(ReplyPreview { id: reply_id.to_string(), from, body, thread: false, forwarded: false })
Some(ReplyPreview { id: reply_id.to_string(), from, body, thread: false, forwarded: false, thread_root: None, chat: None })
}
/// Shared by both the initial backfill and extend_history - Discord
src/backend/irc/incoming.rs
+27 -6
View File
@@ -7,6 +7,18 @@
use super::*;
/// The line written in a channel when the server says what modes it has: the letters, and what they mean, so
/// somebody new to IRC need not know them ("#archlinux is +Cnrt -- No CTCP messages to the channel, ...").
pub fn channel_modes_line(channel: &str, modes: &str, params: &[String]) -> String {
if modes.trim_matches('+').is_empty() {
return format!("{channel} has no modes set");
}
match super::modes::describe(modes, params) {
Some(why) => format!("{channel} is {modes} -- {why}"),
None => format!("{channel} is {modes}"),
}
}
pub fn is_channel(target: &str) -> bool {
target.starts_with(['#', '&', '+', '!'])
}
@@ -290,7 +302,10 @@ pub(super) async fn handle_message(
let adding = matches!(m, Mode::Plus(_, None));
let letter = mode_letter(mode);
state.runtime.set_irc_channel_mode(state, account_id, &channel, letter, adding);
let body = format!("{from} sets {}{letter} on {channel}", if adding { "+" } else { "-" });
let mut body = format!("{from} sets {}{letter} on {channel}", if adding { "+" } else { "-" });
if let Some(why) = super::modes::describe_change(letter, adding) {
body = format!("{body} -- {why}");
}
state.runtime.record_message(state, account_id, &channel, "channel", "*", &body, false, "system", None, None, false, None, Vec::new(), Vec::new(), None);
continue;
}
@@ -847,11 +862,7 @@ pub(super) async fn handle_message(
if let Some(channel) = args.get(1) {
let modes = args.get(2).map(String::as_str).unwrap_or("");
state.runtime.set_irc_channel_modes(state, account_id, channel, modes);
let body = if modes.trim_matches('+').is_empty() {
format!("{channel} has no modes set")
} else {
format!("{channel} is {modes}")
};
let body = channel_modes_line(channel, modes, &args.iter().skip(3).cloned().collect::<Vec<_>>());
state.runtime.record_message(state, account_id, channel, "channel", "*", &body, false, "system", None, None, false, None, Vec::new(), Vec::new(), None);
}
}
@@ -1135,6 +1146,16 @@ pub(super) fn buffer_kind_hint(target: &str) -> &'static str {
#[cfg(test)]
mod tests {
#[test]
fn a_channels_modes_are_written_with_what_they_mean() {
assert_eq!(
channel_modes_line("#archlinux", "+Cnrt", &[]),
"#archlinux is +Cnrt -- No CTCP messages to the channel, no messages from outside the channel, registered users only, topic locked to operators."
);
assert_eq!(channel_modes_line("#quiet", "+", &[]), "#quiet has no modes set");
assert_eq!(channel_modes_line("#odd", "+Z", &[]), "#odd is +Z -- +Z is this network's own.");
}
use super::*;
fn person(nick: &str, host: &str) -> Prefix {
src/backend/irc/mod.rs
+1
View File
@@ -35,6 +35,7 @@ pub mod typing;
pub mod dcc;
pub mod nickserv;
pub mod metadata;
pub mod modes;
pub mod sts;
pub mod sasl;
pub mod latesasl;
src/backend/irc/modes.rs
+137
View File
@@ -0,0 +1,137 @@
//! What a channel's modes mean, in words.
//!
//! A server answers `/mode #chan` with letters - `+Cnrt` - which a new IRC user cannot be expected to
//! read. The channel line moho writes on joining explains them: "#archlinux is +Cnrt -- No CTCP messages
//! to the channel, no messages from outside the channel, registered users only, topic locked to operators."
//!
//! The letters that mean the same everywhere (RFC 2811 and what every server inherited) are plain; a few
//! that nearly every server agrees on are included; a letter a network invented for itself is named as
//! one rather than guessed at, since a wrong explanation is worse than none.
/// One mode letter's meaning, given its argument if it takes one. `None` for a letter this does not know.
fn meaning(letter: char, param: Option<&str>) -> Option<String> {
let with = |what: &str| param.map(|p| format!("{what} {p}"));
Some(match letter {
'n' => "no messages from outside the channel".into(),
't' => "topic locked to operators".into(),
'm' => "moderated: only voiced people and operators can talk".into(),
'i' => "invite only".into(),
's' => "secret: left out of the channel list".into(),
'p' => "private".into(),
'r' => "registered users only".into(),
'c' => "no colours or formatting".into(),
'C' => "no CTCP messages to the channel".into(),
'S' => "only people connected over TLS".into(),
'g' => "anybody can invite".into(),
'P' => "permanent: stays when it is empty".into(),
'L' => "a larger ban list".into(),
'k' => "needs a key to join".into(),
'l' => match param {
Some(p) => format!("limited to {p} people"),
None => "limited in size".into(),
},
'f' => with("people who cannot join are sent to")?,
'j' => with("joining is throttled to")?,
_ => return None,
})
}
/// Whether a mode letter takes an argument when it is set, for the ones this explains.
fn takes_param(letter: char) -> bool {
matches!(letter, 'k' | 'l' | 'f' | 'j')
}
/// Explains a channel's modes: the string the server sent (`+Cnrt`, `+lk`) and the arguments after it.
/// `None` when there is nothing to say (no modes set).
pub fn describe(modes: &str, params: &[String]) -> Option<String> {
let mut args = params.iter();
let mut said: Vec<String> = Vec::new();
let mut unknown: Vec<char> = Vec::new();
let mut adding = true;
for letter in modes.chars() {
match letter {
'+' => adding = true,
'-' => adding = false,
c if !adding || !c.is_ascii_alphabetic() => {
let _ = c;
}
c => {
let param = takes_param(c).then(|| args.next().map(String::as_str)).flatten();
// A key is a secret: said that there is one, never what it is.
let param = if c == 'k' { None } else { param };
match meaning(c, param) {
Some(text) => said.push(text),
None => unknown.push(c),
}
}
}
}
if !unknown.is_empty() {
let list = unknown.iter().map(|c| format!("+{c}")).collect::<Vec<_>>().join(", ");
said.push(format!("{list} {} this network's own", if unknown.len() == 1 { "is" } else { "are" }));
}
if said.is_empty() {
return None;
}
let mut text = said.join(", ");
// A sentence: a capital to begin, a full stop to end.
if let Some(first) = text.chars().next() {
text.replace_range(..first.len_utf8(), &first.to_uppercase().to_string());
}
Some(format!("{text}."))
}
/// What one mode being set or cleared means, for the line that says somebody changed it. `None` for a letter
/// this does not know, where the line says no more than it did.
pub fn describe_change(letter: char, adding: bool) -> Option<String> {
let text = meaning(letter, None)?;
if adding {
let mut c = text.chars();
return c.next().map(|f| format!("{}{}.", f.to_uppercase(), c.as_str()));
}
Some(format!("No longer in effect: {text}."))
}
#[cfg(test)]
mod tests {
use super::*;
fn d(modes: &str, params: &[&str]) -> Option<String> {
describe(modes, &params.iter().map(|p| p.to_string()).collect::<Vec<_>>())
}
#[test]
fn the_letters_of_a_busy_channel_are_said_in_the_order_the_server_sent_them() {
assert_eq!(
d("+Cnrt", &[]).as_deref(),
Some("No CTCP messages to the channel, no messages from outside the channel, registered users only, topic locked to operators.")
);
assert_eq!(d("+nt", &[]).as_deref(), Some("No messages from outside the channel, topic locked to operators."));
}
#[test]
fn a_limit_says_its_number_and_a_key_is_never_told() {
assert_eq!(d("+lntk", &["50", "hunter2"]).as_deref(), Some("Limited to 50 people, no messages from outside the channel, topic locked to operators, needs a key to join."));
assert!(!d("+k", &["hunter2"]).unwrap().contains("hunter2"));
assert_eq!(d("+f", &["#overflow"]).as_deref(), Some("People who cannot join are sent to #overflow."));
}
#[test]
fn a_letter_nobody_agrees_on_is_named_and_not_guessed_at() {
assert_eq!(d("+nQ", &[]).as_deref(), Some("No messages from outside the channel, +Q is this network's own."));
assert_eq!(d("+QZ", &[]).as_deref(), Some("+Q, +Z are this network's own."));
}
#[test]
fn a_change_says_what_it_did() {
assert_eq!(describe_change('m', true).as_deref(), Some("Moderated: only voiced people and operators can talk."));
assert_eq!(describe_change('t', false).as_deref(), Some("No longer in effect: topic locked to operators."));
assert_eq!(describe_change('Q', true), None);
}
#[test]
fn no_modes_say_nothing() {
assert_eq!(d("+", &[]), None);
assert_eq!(d("", &[]), None);
}
}
src/backend/kick/api.rs
+115 -1
View File
@@ -83,6 +83,53 @@ pub struct Channel {
pub playback_url: Option<String>,
}
/// Where a stream's video is, and until when that address is good.
#[derive(Clone, Debug, PartialEq)]
pub struct Playback {
/// A signed Amazon IVS playlist address: the signature is in the query, so it is fetched with no
/// credential of ours.
pub url: String,
/// When the signature stops being accepted, in unix seconds, where the address says.
pub expires: Option<i64>,
/// Whether the signature opts out of ad insertion, which Kick's own web page is given.
pub ads_opt_out: Option<bool>,
}
impl Playback {
pub fn of(url: &str) -> Playback {
let claims = token_claims(url);
Playback { url: url.to_string(), expires: claims.as_ref().and_then(|c| c["exp"].as_i64()), ads_opt_out: claims.as_ref().and_then(|c| c["aws:ads-opt-out"].as_bool()) }
}
}
/// What a signed playlist address says about itself: the claims of the token in its query.
fn token_claims(url: &str) -> Option<serde_json::Value> {
use base64::Engine;
let token = url.split_once('?')?.1.split('&').find_map(|pair| pair.strip_prefix("token="))?;
let claims = token.split('.').nth(1)?;
let bytes = base64::engine::general_purpose::URL_SAFE_NO_PAD.decode(claims.trim_end_matches('=')).ok()?;
serde_json::from_slice(&bytes).ok()
}
/// A live channel's playlist address, asked for on its own.
///
/// The channel payload carries one too, but that is eight kilobytes about everything else and is kept
/// by Kick for a while, so the address in it can be most of the way to its expiry. This answers with
/// the address alone, signed fresh, for a tenth of the bytes; it is the call Kick's apps make. The
/// signature lasts about ten minutes, which is what `Playback::expires` is for.
pub async fn playback(http: &reqwest::Client, slug: &str) -> Result<Playback> {
let res = paced(http.get(format!("{API_ROOT}/api/v2/channels/{slug}/playback-url")).header("Accept", "application/json"), Priority::Background)
.await
.context("asking Kick where the stream is")?;
let status = res.status();
if !status.is_success() {
bail!("Kick answered {status} for the stream's address");
}
let body: serde_json::Value = res.json().await.context("unreadable answer")?;
let url = body["data"].as_str().filter(|u| u.starts_with("https://")).context("Kick sent no address for the stream")?;
Ok(Playback::of(url))
}
/// A stream in progress, as the channel endpoint describes it.
#[derive(Clone, Debug, Default)]
pub struct Live {
@@ -855,8 +902,13 @@ pub struct HistoryMessage {
#[serde(default)]
pub created_at: Option<String>,
pub sender: HistorySender,
/// A string of JSON here, not the object the live socket sends.
#[serde(default)]
pub metadata: Option<serde_json::Value>,
/// Where a reply's chain began. This endpoint has never sent it; the
/// other history one does, and so does the socket.
#[serde(default)]
pub thread_parent_id: Option<String>,
}
#[derive(Deserialize, Debug)]
@@ -884,9 +936,12 @@ pub struct ChatIdentity {
pub color: Option<String>,
#[serde(default)]
pub badges: Vec<Badge>,
/// The pictures: level, event and founder badges, and a channel's own.
#[serde(default)]
pub badges_v2: Vec<BadgeV2>,
}
#[derive(Deserialize, Debug, Clone, Serialize)]
#[derive(Deserialize, Debug, Clone, Serialize, Default)]
pub struct Badge {
/// `moderator`, `subscriber`, `verified`, `og`, `founder`, `vip`...
#[serde(rename = "type")]
@@ -897,6 +952,38 @@ pub struct Badge {
/// Months subscribed, for the badges that count.
#[serde(default)]
pub count: Option<u32>,
/// A picture to draw it with, for the badges Kick sends as one: the level,
/// a channel's or an event's own, the founder's. The role badges above
/// have none and are drawn as words.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub image: Option<String>,
/// Where Kick ranks it, highest first. Only used to put the badges in
/// Kick's order, which the role and the picture badges share.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub sort_order: Option<i32>,
}
/// A badge in Kick's newer list, which says what it looks like where the old
/// list names a role. `selected` is the person's own choice to wear it: a
/// level badge comes on every message, and only some people show theirs.
#[derive(Deserialize, Debug, Clone, Default)]
pub struct BadgeV2 {
#[serde(default)]
pub name: String,
#[serde(default)]
pub image_url: Option<String>,
#[serde(default)]
pub selected: Option<bool>,
#[serde(default)]
pub sort_order: Option<i32>,
#[serde(default)]
pub metadata: Option<BadgeV2Metadata>,
}
#[derive(Deserialize, Debug, Clone, Default)]
pub struct BadgeV2Metadata {
#[serde(default)]
pub level: Option<u32>,
}
#[derive(Deserialize)]
@@ -2179,3 +2266,30 @@ pub async fn redeem(http: &reqwest::Client, token: &str, slug: &str, reward_id:
None => bail!("Kick answered {status}"),
}
}
#[cfg(test)]
mod playback_tests {
use super::*;
use base64::Engine;
fn address(claims: serde_json::Value) -> String {
let part = base64::engine::general_purpose::URL_SAFE_NO_PAD.encode(claims.to_string());
format!("https://example.live-video.net/api/video/v1/x.m3u8?token=eyJhbGciOiJFUzM4NCJ9.{part}.sig")
}
#[test]
fn an_address_says_when_it_expires_and_whether_it_opts_out_of_ads() {
let p = Playback::of(&address(serde_json::json!({ "exp": 1791932879, "aws:ads-opt-out": true })));
assert_eq!((p.expires, p.ads_opt_out), (Some(1791932879), Some(true)));
let p = Playback::of(&address(serde_json::json!({ "exp": 5 })));
assert_eq!((p.expires, p.ads_opt_out), (Some(5), None), "silence is not a refusal");
}
#[test]
fn an_address_that_is_not_signed_the_way_it_expects_has_no_expiry_and_is_still_an_address() {
for url in ["https://example.net/x.m3u8", "https://example.net/x.m3u8?other=1", "https://example.net/x.m3u8?token=notajwt", "https://example.net/x.m3u8?token=a.!!!.c"] {
let p = Playback::of(url);
assert_eq!((p.url.as_str(), p.expires), (url, None));
}
}
}
src/backend/kick/cards.rs
+2 -2
View File
@@ -207,7 +207,7 @@ pub async fn refresh_prediction(state: &AppState, buffer_id: &str) {
}
/// Stores a card, writes it down, and tells the client - or says it is gone.
pub(super) fn publish_card(state: &AppState, buffer_id: &str, kind: &str, card: Option<serde_json::Value>) {
pub(crate) fn publish_card(state: &AppState, buffer_id: &str, kind: &str, card: Option<serde_json::Value>) {
// When this was true, stamped here so every card carries one.
//
// The clock is the whole point: `remaining` is a number of seconds that
@@ -263,7 +263,7 @@ pub fn card_is_current(card: &serde_json::Value) -> bool {
}
/// The wall clock, in seconds, as everything here writes it down.
pub(super) fn now_secs() -> i64 {
pub(crate) fn now_secs() -> i64 {
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
src/backend/kick/chat.rs
+260 -39
View File
@@ -79,6 +79,8 @@ pub(super) struct Watched {
/// wants the panel for: to mention them, whisper them, or see who the
/// moderators are.
pub(super) speakers: HashMap<String, Vec<Speaker>>,
/// Every name that has spoken in each channel, for counting.
pub(super) chatters: HashMap<String, std::collections::HashSet<String>>,
}
/// Somebody who has said something, and how they looked saying it.
@@ -91,12 +93,15 @@ pub(super) struct Speaker {
pub(super) badge: Option<String>,
}
/// How many to remember per channel.
/// How many to list per channel.
///
/// A busy stream produces hundreds of distinct names an hour, and a list
/// nobody can scan is no better than an empty one. This is the last two
/// hundred to have spoken, which is the window somebody is actually reading.
pub(super) const SPEAKERS_REMEMBERED: usize = 200;
/// A busy stream produces thousands of distinct names an hour, and the list is
/// sent whole every time a new one joins it and drawn row by row - a few
/// hundred is what that stays cheap for. It is the most recent to have
/// spoken, which is the window somebody is actually reading. How many people
/// have spoken is counted apart from it (`Watched::chatters`) and is not cut
/// off.
pub(super) const SPEAKERS_REMEMBERED: usize = 500;
#[derive(Deserialize)]
pub(super) struct ChatMessage {
@@ -114,6 +119,10 @@ pub(super) struct ChatMessage {
/// the cap. Kick's replies are quotable however old the original is.
#[serde(default)]
metadata: Option<ReplyMetadata>,
/// The message at the top of the chain this one is part of, on a reply.
/// Empty or absent on anything that is not one.
#[serde(default)]
thread_parent_id: Option<String>,
}
#[derive(Deserialize)]
@@ -129,6 +138,12 @@ pub(super) struct OriginalMessage {
id: String,
#[serde(default)]
content: Option<String>,
/// Where the original itself sits: empty when it began the chain, and the
/// chain's first message when it was a reply. It is how a reply read from
/// history, which does not carry the thread on the reply itself, finds its
/// thread.
#[serde(default)]
thread_parent_id: Option<String>,
}
#[derive(Deserialize)]
@@ -173,7 +188,7 @@ pub(super) fn handle_event(
let Ok(msg) = serde_json::from_value::<ChatMessage>(payload) else { return Ok(()) };
let Some(slug) = watched.chatroom(msg.chatroom_id).cloned() else { return Ok(()) };
let from = msg.sender.username.clone().unwrap_or_else(|| "someone".to_string());
let reply_to = reply_preview(msg.metadata.as_ref());
let reply_to = reply_preview(msg.metadata.as_ref(), msg.thread_parent_id.as_deref());
let style = style_of(msg.sender.identity.as_ref());
// Kick has no viewer list to ask for, so the panel is built from
// who is talking - which is who somebody wants to reach anyway.
@@ -188,8 +203,10 @@ pub(super) fn handle_event(
) {
let buffer_id = crate::model::buffer_id(account_id, &slug);
let roster = serde_json::json!(watched.roster(&slug));
let total = watched.chatters(&slug);
state.runtime.set_presence(&buffer_id, roster.clone());
state.events.emit("presenceChange", serde_json::json!({ "bufferId": buffer_id, "members": roster }));
state.runtime.set_presence_total(&buffer_id, total);
state.events.emit("presenceChange", serde_json::json!({ "bufferId": buffer_id, "members": roster, "total": total }));
}
state.runtime.record_message_at(
state,
@@ -257,7 +274,10 @@ pub(super) fn handle_event(
.or_else(|| payload.get("id"))
.and_then(|v| v.as_str());
if let Some(id) = id {
state.runtime.delete_message(state, &crate::model::buffer_id(account_id, &slug), id);
// Kept and marked, not removed: what it said is worth reading
// beside the fact that it is gone, and how a removed message
// is drawn is the window's to decide.
state.runtime.moderate_message(state, &crate::model::buffer_id(account_id, &slug), id, &moderation_of(&payload));
}
}
@@ -437,7 +457,7 @@ pub async fn backfill(
&msg.content,
false,
"chat",
reply_preview_from_value(msg.metadata.as_ref()),
reply_preview_from_value(msg.metadata.as_ref(), msg.thread_parent_id.as_deref()),
Some(msg.id.clone()),
false,
None,
@@ -456,9 +476,33 @@ pub async fn backfill(
///
/// History carries it in the same shape, so this is the one parser rather than
/// a second one that could drift.
pub(super) fn reply_preview_from_value(metadata: Option<&serde_json::Value>) -> Option<crate::model::ReplyPreview> {
let parsed: ReplyMetadata = serde_json::from_value(metadata?.clone()).ok()?;
reply_preview(Some(&parsed))
///
/// An object on the live socket and a string of JSON in history, which is
/// the same object written down: a reply read back from history lost its quote
/// because only the first was understood.
pub(super) fn reply_preview_from_value(metadata: Option<&serde_json::Value>, thread_parent: Option<&str>) -> Option<crate::model::ReplyPreview> {
let metadata = metadata?;
let parsed: ReplyMetadata = match metadata {
serde_json::Value::String(text) => serde_json::from_str(text).ok()?,
other => serde_json::from_value(other.clone()).ok()?,
};
reply_preview(Some(&parsed), thread_parent)
}
/// Who took a message down and for what, as the event says it.
///
/// `aiModerated` and `violatedRules` are on the payload of the event for a
/// message Kick's automatic moderation removed; a person's removal has
/// neither, which reads as not automatic with nothing to say about rules.
pub(super) fn moderation_of(payload: &serde_json::Value) -> crate::model::Moderated {
crate::model::Moderated {
ai: payload.get("aiModerated").and_then(|v| v.as_bool()).unwrap_or(false),
rules: payload
.get("violatedRules")
.and_then(|v| v.as_array())
.map(|rules| rules.iter().filter_map(|r| r.as_str()).map(str::to_string).collect())
.unwrap_or_default(),
}
}
/// The one badge worth marking somebody with in a list.
@@ -483,32 +527,70 @@ pub(super) fn strongest_badge(badges: &[api::Badge]) -> Option<String> {
/// the ones a reader is actually scanning for.
pub(super) fn style_of(identity: Option<&api::ChatIdentity>) -> crate::model::SenderStyle {
let Some(identity) = identity else { return crate::model::SenderStyle::default() };
crate::model::SenderStyle {
color: identity.color.clone().filter(|c| !c.is_empty()),
badges: identity
.badges
// The role badges are drawn as words, and Kick's picture badges beside them
// as pictures - the ones the person chose to wear, in Kick's order. The
// old list's own "level" is dropped as it always was: the picture one says
// the same with a number a reader can see.
let mut badges: Vec<api::Badge> = identity.badges.iter().filter(|b| !matches!(b.kind.as_str(), "level" | "")).cloned().collect();
let mut pictures: Vec<api::Badge> = identity
.badges_v2
.iter()
.filter(|b| !matches!(b.kind.as_str(), "level" | ""))
.cloned()
.collect(),
}
.filter(|b| b.selected.unwrap_or(true) && !b.name.is_empty())
.filter_map(|b| {
let image = b.image_url.clone().filter(|u| u.starts_with("https://"))?;
let level = b.metadata.as_ref().and_then(|m| m.level);
Some(api::Badge {
// Named by what Kick calls it, apart from the role names.
kind: b.name.clone(),
text: match level {
Some(level) if b.name == "level" => format!("Level {level}"),
_ => b.name.clone(),
},
count: None,
image: Some(image),
sort_order: b.sort_order,
})
})
.collect();
// The highest-ranked of them, if there are more than a name can carry.
pictures.sort_by_key(|b| std::cmp::Reverse(b.sort_order.unwrap_or(0)));
pictures.truncate(MAX_BADGE_PICTURES);
badges.extend(pictures);
// Highest first, as Kick's own apps put them, and stable so two of a rank
// stay in the order they came.
badges.sort_by_key(|b| std::cmp::Reverse(b.sort_order.unwrap_or(0)));
crate::model::SenderStyle { color: identity.color.clone().filter(|c| !c.is_empty()), badges }
}
/// How many picture badges one name carries. Some people wear a dozen, and a
/// name with that many beside it is the loudest thing on the line.
const MAX_BADGE_PICTURES: usize = 4;
/// What a reply was replying to, out of the metadata Kick attaches to it.
///
/// Both halves are required rather than filled in with blanks: a preview with
/// no author and no text is drawn as an empty quote above the message, which
/// reads as something failing to load rather than as a reply. Better to show
/// the message plainly than to show a hole above it.
pub(super) fn reply_preview(metadata: Option<&ReplyMetadata>) -> Option<crate::model::ReplyPreview> {
///
/// `thread_parent` is what the message itself says its chain began with. When
/// it says nothing - a reply out of history - the original's own place in its
/// chain gives the same answer, and an original that began one is the root.
pub(super) fn reply_preview(metadata: Option<&ReplyMetadata>, thread_parent: Option<&str>) -> Option<crate::model::ReplyPreview> {
let metadata = metadata?;
let original = metadata.original_message.as_ref()?;
let from = metadata.original_sender.as_ref()?.username.clone()?;
let filled = |s: Option<&str>| s.filter(|s| !s.is_empty()).map(str::to_string);
let thread_root = filled(thread_parent).or_else(|| filled(original.thread_parent_id.as_deref())).unwrap_or_else(|| original.id.clone());
Some(crate::model::ReplyPreview {
id: original.id.clone(),
from,
body: original.content.clone().unwrap_or_default(),
thread: false, forwarded: false })
thread: false,
forwarded: false,
thread_root: Some(thread_root),
chat: None,
})
}
/// One line for something somebody did in a channel, or None if this event is
@@ -908,14 +990,19 @@ mod tests {
w.heard("chan", Speaker { nick: format!("nick{i}"), user_id: None, badge: None });
}
assert_eq!(w.roster("chan").len(), SPEAKERS_REMEMBERED);
// The count is of everybody, not of those listed.
assert_eq!(w.chatters("chan"), SPEAKERS_REMEMBERED + 50);
// The most recent survive; a list nobody can scan is no better than
// an empty one, and the newest names are the ones being read.
assert_eq!(w.roster("chan")[0]["nick"], format!("nick{}", SPEAKERS_REMEMBERED + 49));
// And somebody who spoke long ago and speaks again is not counted twice.
w.heard("chan", Speaker { nick: "nick0".into(), user_id: None, badge: None });
assert_eq!(w.chatters("chan"), SPEAKERS_REMEMBERED + 50);
}
#[test]
fn marks_somebody_by_the_badge_worth_finding_them_by() {
let badge = |kind: &str| api::Badge { kind: kind.into(), text: kind.into(), count: None };
let badge = |kind: &str| api::Badge { kind: kind.into(), text: kind.into(), ..Default::default() };
// Ranked by what a reader is scanning for, not by what is rarest: a
// moderator who is also a subscriber shows as a moderator.
assert_eq!(strongest_badge(&[badge("subscriber"), badge("moderator")]).as_deref(), Some("moderator"));
@@ -946,7 +1033,7 @@ mod tests {
"original_message": { "id": "ddeccb11-29bc-4cf1-ab2a-9b409293de76", "content": "L0L0WERFLGEWRUGHBER" }
}))
.unwrap();
let preview = reply_preview(Some(&metadata)).unwrap();
let preview = reply_preview(Some(&metadata), None).unwrap();
assert_eq!(preview.from, "Greg");
assert_eq!(preview.id, "ddeccb11-29bc-4cf1-ab2a-9b409293de76");
assert_eq!(preview.body, "L0L0WERFLGEWRUGHBER");
@@ -978,7 +1065,7 @@ mod tests {
let msg: ChatMessage = serde_json::from_str(raw).expect("a real reply frame must parse");
assert_eq!(msg.chatroom_id, 2393554);
assert_eq!(msg.sender.username.as_deref(), Some("L3galizeWee"));
let preview = reply_preview(msg.metadata.as_ref()).expect("and carry what it answered");
let preview = reply_preview(msg.metadata.as_ref(), None).expect("and carry what it answered");
assert_eq!(preview.from, "Greg");
assert_eq!(preview.body, "L0L0WERFLGEWRUGHBER");
}
@@ -990,12 +1077,12 @@ mod tests {
let raw = r#"{ "id": "x", "chatroom_id": 1, "content": "hi", "type": "message",
"sender": { "id": 2, "username": "someone" }, "metadata": { "message_ref": "1788305786864" } }"#;
let msg: ChatMessage = serde_json::from_str(raw).unwrap();
assert!(reply_preview(msg.metadata.as_ref()).is_none());
assert!(reply_preview(msg.metadata.as_ref(), None).is_none());
}
#[test]
fn an_ordinary_message_is_not_a_reply() {
assert!(reply_preview(None).is_none());
assert!(reply_preview(None, None).is_none());
}
#[test]
@@ -1004,10 +1091,10 @@ mod tests {
// which is worse than showing the message plainly.
let no_sender: ReplyMetadata =
serde_json::from_value(serde_json::json!({ "original_message": { "id": "x", "content": "y" } })).unwrap();
assert!(reply_preview(Some(&no_sender)).is_none());
assert!(reply_preview(Some(&no_sender), None).is_none());
let no_message: ReplyMetadata =
serde_json::from_value(serde_json::json!({ "original_sender": { "username": "Greg" } })).unwrap();
assert!(reply_preview(Some(&no_message)).is_none());
assert!(reply_preview(Some(&no_message), None).is_none());
}
#[test]
@@ -1256,8 +1343,12 @@ mod tests {
assert_eq!(testkit::lines(&state, "odablock"), vec!["went live", "ended the stream"]);
}
fn backlog(state: &AppState) -> Vec<crate::model::Message> {
state.store.get_backlog(&crate::model::buffer_id("kick:tester", "odablock"), 0, 50).expect("backlog")
}
#[test]
fn a_message_taken_down_leaves_the_log() {
fn a_message_taken_down_stays_in_the_log_marked() {
let state = testkit::simulated_daemon("deleted");
let mut watched = testkit::watching_odablock();
spoken(&state, &mut watched, "keep", "stays");
@@ -1266,7 +1357,94 @@ mod tests {
// Kick nests the id under `message` on one event and sends it flat on another.
feed(&state, &mut watched, "App\\Events\\MessageDeletedEvent", serde_json::json!({ "message": { "id": "nested" } }));
feed(&state, &mut watched, "App\\Events\\MessageDeletedEvent", serde_json::json!({ "id": "flat" }));
assert_eq!(testkit::lines(&state, "odablock"), vec!["stays"]);
let log = backlog(&state);
// Nothing left, and the words of the ones taken down are still there to read.
assert_eq!(log.iter().map(|m| m.body.as_str()).collect::<Vec<_>>(), vec!["stays", "goes", "goes too"]);
assert_eq!(log.iter().map(|m| m.moderated.is_some()).collect::<Vec<_>>(), vec![false, true, true]);
}
#[test]
fn what_the_automatic_moderator_said_is_kept_with_the_message() {
let state = testkit::simulated_daemon("moderated");
let mut watched = testkit::watching_odablock();
spoken(&state, &mut watched, "bad", "something vile");
// As measured on a live channel: the event has its own id, and the message's is nested.
feed(
&state,
&mut watched,
"App\\Events\\MessageDeletedEvent",
serde_json::json!({ "id": "event-id", "message": { "id": "bad" }, "aiModerated": true, "violatedRules": ["hate"] }),
);
let log = backlog(&state);
assert_eq!(log.len(), 1);
assert_eq!(log[0].moderated, Some(crate::model::Moderated { ai: true, rules: vec!["hate".into()] }));
}
#[test]
fn a_person_removing_a_message_is_not_the_automatic_moderator() {
let why = moderation_of(&serde_json::json!({ "message": { "id": "x" } }));
assert_eq!(why, crate::model::Moderated { ai: false, rules: vec![] });
}
#[test]
fn a_reply_names_the_thread_it_belongs_to() {
// The top-level field when there is one: this is a reply to a reply, so what it quotes and
// the thread it is in are different messages.
let metadata: ReplyMetadata = serde_json::from_value(serde_json::json!({
"original_sender": { "id": 1, "username": "Greg" },
"original_message": { "id": "middle", "content": "hello" }
}))
.unwrap();
let preview = reply_preview(Some(&metadata), Some("top")).unwrap();
assert_eq!((preview.id.as_str(), preview.thread_root.as_deref()), ("middle", Some("top")));
// Without it, a reply to a message that began nothing starts the thread itself.
let preview = reply_preview(Some(&metadata), Some("")).unwrap();
assert_eq!(preview.thread_root.as_deref(), Some("middle"));
}
#[test]
fn a_reply_out_of_history_finds_its_thread_in_what_it_quoted() {
// History writes the metadata down as a string and leaves the thread off the reply; the
// quoted message knows its own.
let written = serde_json::Value::String(
serde_json::json!({
"original_sender": { "id": 1, "username": "Greg" },
"original_message": { "id": "middle", "content": "hello", "thread_parent_id": "top" }
})
.to_string(),
);
let preview = reply_preview_from_value(Some(&written), None).expect("a string of JSON is the same metadata");
assert_eq!((preview.id.as_str(), preview.from.as_str(), preview.thread_root.as_deref()), ("middle", "Greg", Some("top")));
}
#[test]
fn replies_are_stored_with_their_thread_and_read_back_whole() {
let state = testkit::simulated_daemon("threads");
let mut watched = testkit::watching_odablock();
feed(&state, &mut watched, "App\\Events\\ChatMessageEvent", serde_json::json!({
"id": "top", "chatroom_id": 2393554, "content": "the question", "created_at": "2026-10-14T00:59:00+00:00",
"sender": { "id": 9, "username": "bob", "identity": null }
}));
for (id, to, root) in [("r1", "top", "top"), ("r2", "r1", "top"), ("elsewhere", "x", "x")] {
feed(
&state,
&mut watched,
"App\\Events\\ChatMessageEvent",
serde_json::json!({
"id": id, "chatroom_id": 2393554, "content": id, "type": "reply",
"created_at": "2026-10-14T01:00:00+00:00",
"sender": { "id": 7, "username": "someone" },
"thread_parent_id": root,
"metadata": { "original_sender": { "id": 1, "username": "Greg" }, "original_message": { "id": to, "content": "quoted" } }
}),
);
}
let buffer = crate::model::buffer_id("kick:tester", "odablock");
let thread = state.store.thread_messages(&buffer, "top").unwrap();
assert_eq!(thread.iter().map(|m| m.id.as_str()).collect::<Vec<_>>(), vec!["top", "r1", "r2"]);
let r2 = thread.iter().find(|m| m.id == "r2").unwrap().reply_to.clone().unwrap();
// It still quotes what it answered, and says which thread that is part of.
assert_eq!((r2.id.as_str(), r2.thread, r2.thread_root.as_deref()), ("r1", false, Some("top")));
}
#[test]
@@ -1337,17 +1515,60 @@ mod tests {
#[test]
fn a_sender_is_drawn_in_their_colour_with_the_badges_that_say_something() {
let badge = |kind: &str| api::Badge { kind: kind.into(), text: kind.into(), count: None };
let badge = |kind: &str| api::Badge { kind: kind.into(), text: kind.into(), ..Default::default() };
assert_eq!(style_of(None).color, None);
let style = style_of(Some(&api::ChatIdentity {
color: Some("#75FD46".into()),
badges: vec![badge("moderator"), badge("level"), badge(""), badge("subscriber")],
..Default::default()
}));
assert_eq!(style.color.as_deref(), Some("#75FD46"));
// An engagement level and a nameless badge say nothing about the person.
assert_eq!(style.badges.iter().map(|b| b.kind.as_str()).collect::<Vec<_>>(), vec!["moderator", "subscriber"]);
// A colour that is empty is no colour.
assert_eq!(style_of(Some(&api::ChatIdentity { color: Some(String::new()), badges: vec![] })).color, None);
assert_eq!(style_of(Some(&api::ChatIdentity { color: Some(String::new()), ..Default::default() })).color, None);
}
/// As measured on a live channel: the picture badges come in their own list,
/// each with the person's choice to wear it and where Kick ranks it.
#[test]
fn the_pictures_a_person_chose_to_wear_come_after_their_role_in_kicks_order() {
let identity: api::ChatIdentity = serde_json::from_value(serde_json::json!({
"color": "#E9113C",
"badges": [
{ "type": "moderator", "text": "Moderator", "sort_order": 12 },
{ "type": "sub_gifter", "text": "Sub Gifter", "count": 1, "sort_order": 8 }
],
"badges_v2": [
{ "name": "level", "badge_type": "global", "image_url": "https://ext.cdn.kick.com/chat/badges/48_a.png", "metadata": { "level": 48 }, "selected": true, "sort_order": 1 },
{ "name": "GOAT", "badge_type": "global", "image_url": "https://ext.cdn.kick.com/chat/badges/GOAT_b.png", "selected": true, "sort_order": 4 },
{ "name": "GoldenK", "badge_type": "global", "image_url": "https://ext.cdn.kick.com/chat/badges/GoldenK_c.png", "selected": false, "sort_order": 4 },
{ "name": "kick_founder22", "badge_type": "global", "image_url": "https://ext.cdn.kick.com/chat/badges/f.png", "selected": true, "sort_order": 2 }
]
}))
.unwrap();
let style = style_of(Some(&identity));
// Not the one the person chose not to wear, and highest rank first.
assert_eq!(style.badges.iter().map(|b| b.kind.as_str()).collect::<Vec<_>>(), vec!["moderator", "sub_gifter", "GOAT", "kick_founder22", "level"]);
// Words for the roles, pictures for the rest, and the level says what it is.
assert!(style.badges[0].image.is_none());
assert_eq!(style.badges[2].image.as_deref(), Some("https://ext.cdn.kick.com/chat/badges/GOAT_b.png"));
assert_eq!(style.badges[4].text, "Level 48");
}
#[test]
fn a_badge_picture_is_only_ever_a_web_address_and_a_crowd_of_them_is_cut() {
let wear = |name: &str, url: &str| api::BadgeV2 { name: name.into(), image_url: Some(url.into()), selected: Some(true), ..Default::default() };
let identity = api::ChatIdentity {
badges_v2: vec![
wear("local", "file:///etc/passwd"),
wear("a", "https://x/a.png"), wear("b", "https://x/b.png"), wear("c", "https://x/c.png"),
wear("d", "https://x/d.png"), wear("e", "https://x/e.png"),
],
..Default::default()
};
let kinds: Vec<_> = style_of(Some(&identity)).badges.into_iter().map(|b| b.kind).collect();
assert_eq!(kinds, vec!["a", "b", "c", "d"]);
}
#[test]
@@ -1356,14 +1577,14 @@ mod tests {
"original_sender": { "id": 1, "username": "Greg" },
"original_message": { "id": "abc", "content": "hello" }
});
let preview = reply_preview_from_value(Some(&whole)).unwrap();
let preview = reply_preview_from_value(Some(&whole), None).unwrap();
assert_eq!((preview.id.as_str(), preview.from.as_str(), preview.body.as_str()), ("abc", "Greg", "hello"));
// Half of one is a hole above the message, and is left out.
assert!(reply_preview_from_value(Some(&serde_json::json!({ "original_message": { "id": "abc", "content": "hello" } }))).is_none());
assert!(reply_preview_from_value(Some(&serde_json::json!({ "original_sender": { "id": 1, "username": "Greg" } }))).is_none());
assert!(reply_preview_from_value(Some(&serde_json::json!({ "original_sender": { "id": 1 }, "original_message": { "id": "abc" } }))).is_none());
assert!(reply_preview_from_value(None).is_none());
assert!(reply_preview_from_value(Some(&serde_json::json!("not metadata"))).is_none());
assert!(reply_preview_from_value(Some(&serde_json::json!({ "original_message": { "id": "abc", "content": "hello" } })), None).is_none());
assert!(reply_preview_from_value(Some(&serde_json::json!({ "original_sender": { "id": 1, "username": "Greg" } })), None).is_none());
assert!(reply_preview_from_value(Some(&serde_json::json!({ "original_sender": { "id": 1 }, "original_message": { "id": "abc" } })), None).is_none());
assert!(reply_preview_from_value(None, None).is_none());
assert!(reply_preview_from_value(Some(&serde_json::json!("not metadata")), None).is_none());
}
#[test]
src/backend/kick/live.rs
+39 -3
View File
@@ -11,7 +11,7 @@ use super::*;
///
/// Kick's own channel endpoint carries all of it and this backend was already
/// calling it - the numbers were parsed away and dropped.
pub(super) fn announce_stream(state: &AppState, buffer_id: &str, channel: &api::Channel, following: Option<bool>) {
pub(super) fn announce_stream(state: &AppState, buffer_id: &str, channel: &api::Channel, following: Option<bool>, playback: Option<&api::Playback>) {
let stream = serde_json::json!({
"bufferId": buffer_id,
"live": channel.live.is_some(),
@@ -29,7 +29,10 @@ pub(super) fn announce_stream(state: &AppState, buffer_id: &str, channel: &api::
// whoever wants to watch, because every frontend that shows video
// needs it and the refresh already running is what replaces the
// signed token before it expires.
"playbackUrl": channel.playback_url,
"playbackUrl": playback.map(|p| p.url.clone()),
// When that address stops being accepted, so whoever plays it can ask for a new one in time
// rather than finding out from a segment that is refused.
"playbackExpires": playback.and_then(|p| p.expires),
// Absent rather than false for an account that is not signed in:
// "not following" and "cannot say" are different, and a button that
// offers to follow when it cannot is a button that fails when pressed.
@@ -39,6 +42,38 @@ pub(super) fn announce_stream(state: &AppState, buffer_id: &str, channel: &api::
state.events.emit("kickStream", stream);
}
/// How long before an address stops being accepted a new one is asked for.
const RENEW_BEFORE: i64 = 180;
/// The address of a live channel's video: the one the stream already has while it is good for a few
/// minutes yet, otherwise a new one from the call that gives only that, and failing that the one the
/// channel payload carried.
pub(super) async fn playback_for(state: &AppState, buffer_id: &str, http: &reqwest::Client, channel: &api::Channel) -> Option<api::Playback> {
channel.live.as_ref()?;
let now = std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).map(|d| d.as_secs() as i64).unwrap_or_default();
if let Some(have) = state.runtime.kick_stream(buffer_id) {
let url = have["playbackUrl"].as_str();
let expires = have["playbackExpires"].as_i64();
if let (Some(url), Some(expires)) = (url, expires) {
if expires - now > RENEW_BEFORE {
return Some(api::Playback::of(url));
}
}
}
match api::playback(http, &channel.slug).await {
Ok(p) => {
if p.ads_opt_out == Some(false) {
tracing::warn!("kick: the address for {} no longer opts out of ads", channel.slug);
}
Some(p)
}
Err(e) => {
tracing::debug!("kick: the address for {}: {e:#}", channel.slug);
channel.playback_url.as_deref().map(api::Playback::of)
}
}
}
/// Whether this account follows the channel, where it can say.
pub(super) async fn following_now(
state: &AppState,
@@ -169,7 +204,8 @@ pub async fn refresh_stream_now(state: &AppState, buffer_id: &str) {
match api::channel(&http, &channel.slug).await {
Ok(fresh) => {
let following = following_now(state, &http, &account_id, &channel.slug, known).await;
announce_stream(state, buffer_id, &fresh, following);
let playback = playback_for(state, buffer_id, &http, &fresh).await;
announce_stream(state, buffer_id, &fresh, following, playback.as_ref());
}
Err(e) => tracing::debug!("kick: refreshing {}: {e:#}", channel.slug),
}
src/backend/kick/mod.rs
+10 -1
View File
@@ -68,6 +68,9 @@ impl Watched {
/// nearly identical. Their badges changing is - somebody subscribes, or
/// is given moderator, and the list should follow.
fn heard(&mut self, slug: &str, speaker: Speaker) -> bool {
// Everybody who has spoken, kept for the count alone: the list below is
// cut off, and what it is cut off at is not how many people have talked.
self.chatters.entry(slug.to_string()).or_default().insert(speaker.nick.clone());
let list = self.speakers.entry(slug.to_string()).or_default();
if let Some(existing) = list.iter_mut().find(|s| s.nick == speaker.nick) {
let changed = existing.badge != speaker.badge;
@@ -79,6 +82,11 @@ impl Watched {
true
}
/// How many different people have spoken in this channel since it was opened.
fn chatters(&self, slug: &str) -> usize {
self.chatters.get(slug).map_or(0, |set| set.len())
}
fn roster(&self, slug: &str) -> Vec<serde_json::Value> {
self.speakers
.get(slug)
@@ -111,17 +119,18 @@ impl Watched {
/// events for a conversation that has been closed.
fn remove(&mut self, slug: &str) -> Vec<String> {
self.speakers.remove(slug);
self.chatters.remove(slug);
let mut names = Vec::new();
if let Some(room) = self.by_chatroom.iter().find(|(_, s)| *s == slug).map(|(i, _)| *i) {
self.by_chatroom.remove(&room);
names.push(format!("chatrooms.{room}.v2"));
names.push(format!("chatrooms.{room}"));
names.push(format!("chatroom_{room}"));
}
if let Some(channel) = self.by_channel.iter().find(|(_, s)| *s == slug).map(|(i, _)| *i) {
self.by_channel.remove(&channel);
names.push(format!("channel.{channel}"));
names.push(format!("channel_{channel}"));
names.push(format!("predictions-channel-{channel}"));
}
names
}
src/backend/kick/socket.rs
+12 -10
View File
@@ -551,7 +551,7 @@ async fn prepare(
// Follow state is not asked for here: this runs for every channel at
// connect, and that would be one extra request per channel before the
// first message arrives. It is filled in when the channel is opened.
announce_stream(state, &buffer.id, &channel, None);
announce_stream(state, &buffer.id, &channel, None, channel.playback_url.as_deref().filter(|_| channel.live.is_some()).map(api::Playback::of).as_ref());
// The streamer's own picture on the channel. Parsed since the first
// version of this backend and used by nothing, so every Kick channel drew
@@ -659,20 +659,24 @@ async fn prepare(
/// Asks the socket for everything that happens in a channel's chat.
///
/// Five subscriptions, not one, because Kick scatters them and only the first
/// Several subscriptions, not one, because Kick scatters them and only the first
/// carries what people say. Watching that one alone is why a chat can look
/// quiet while the stream is visibly reacting to something. Measured against a
/// live subathon rather than guessed - each name here is one an event was
/// actually observed arriving on:
///
/// - `chatrooms.<room>.v2` what people say, and plain subscriptions
/// - `chatrooms.<room>` rewards, and the info lines Kick's own UI draws
/// - `chatroom_<room>` gifted subscriptions, on an older name entirely
/// - `chatroom_<room>` rewards and gifted subscriptions, on an older name entirely
/// - `channel.<channel>` going live, and the channel's own subscriptions
/// - `channel_<channel>` KICKs sent to the streamer, and goal and leaderboard updates
///
/// There used to be a sixth, `chatrooms.<room>` without the `.v2`. Kick's own
/// apps do not ask for it, and six minutes on a channel with 60,000 watching
/// delivered nothing on it - chat came on `.v2` and rewards on `chatroom_` -
/// so it was a subscription to silence, paid for on every channel opened.
///
/// The overlap is real and deliberate on Kick's part: one subscription arrives
/// three times over three of these. Which of the three is believed is settled
/// more than once over several of these. Which of the three is believed is settled
/// in `describe_action`, not here - subscribing to fewer channels would lose
/// whole categories of event to avoid a duplicate that is cheaper to filter.
///
@@ -681,7 +685,6 @@ async fn prepare(
pub(super) async fn subscribe(socket: &mut Socket, chatroom_id: u64, channel_id: u64) -> Result<()> {
for channel in [
format!("chatrooms.{chatroom_id}.v2"),
format!("chatrooms.{chatroom_id}"),
format!("chatroom_{chatroom_id}"),
format!("channel.{channel_id}"),
// A third name for the channel, with an underscore, which is where
@@ -827,7 +830,7 @@ mod tests {
let none = serde_json::json!({});
// All four names this connection subscribes to resolve to the same
// streamer, across both of Kick's number spaces.
for name in ["chatrooms.2393554.v2", "chatrooms.2393554", "chatroom_2393554", "channel.2401072"] {
for name in ["chatrooms.2393554.v2", "chatroom_2393554", "channel.2401072"] {
assert_eq!(channel_of(&w, &Some(name.into()), &none).as_deref(), Some("odablock"), "for {name}");
}
}
@@ -887,18 +890,17 @@ mod tests {
names.sort();
assert_eq!(
names,
vec!["channel.2401072", "channel_2401072", "chatroom_2393554", "chatrooms.2393554", "chatrooms.2393554.v2"]
vec!["channel.2401072", "channel_2401072", "chatroom_2393554", "chatrooms.2393554.v2", "predictions-channel-2401072"]
);
// And it is gone from both maps, so a straggling event has nowhere to
// land rather than reopening the buffer that was just closed.
assert_eq!(channel_of(&w, &Some("chatrooms.2393554".into()), &serde_json::json!({})), None);
assert_eq!(channel_of(&w, &Some("chatrooms.2393554.v2".into()), &serde_json::json!({})), None);
assert!(w.remove("odablock").is_empty());
}
#[test]
fn reads_the_id_out_of_a_subscription_name() {
assert_eq!(id_in_subscription("chatrooms.123.v2"), Some(Subscribed::Chatroom(123)));
assert_eq!(id_in_subscription("chatrooms.123"), Some(Subscribed::Chatroom(123)));
assert_eq!(id_in_subscription("chatroom_123"), Some(Subscribed::Chatroom(123)));
assert_eq!(id_in_subscription("channel.123"), Some(Subscribed::Channel(123)));
assert_eq!(id_in_subscription("channel_123"), Some(Subscribed::Channel(123)));
src/backend/kick/watch.rs
+69 -1
View File
@@ -79,13 +79,39 @@ pub async fn negotiate(http: &reqwest::Client, token: &str, user_id: &str) -> Re
/// A token for one livestream's channel - what actually says who is watching.
pub async fn channel_token(http: &reqwest::Client, token: &str, livestream_id: u64) -> Result<String> {
let body = serde_json::json!({ "channel": channel_name(livestream_id) });
subscription_token(http, token, &channel_name(livestream_id)).await
}
/// A token to subscribe to one of Kick's private channels, by name.
pub async fn subscription_token(http: &reqwest::Client, token: &str, channel: &str) -> Result<String> {
let body = serde_json::json!({ "channel": channel });
let resp = post(http, token, &format!("{WEB_ROOT}/api/v1/realtime/auth/channel"), body)
.await
.context("asking Kick to let us watch")?;
resp["data"]["token"].as_str().map(str::to_string).context("Kick sent no subscription token")
}
/// Where Kick says what this account's points now are, in every channel at once.
///
/// Keyed by the *account*, not the streamer - Kick's own app subscribes to it
/// the same way, once - and each event names the channel it is about.
pub fn points_channel_name(user_id: &str) -> String {
format!("private-channelpoints-{user_id}")
}
/// What a points event says: which channel, and what the balance there now is.
///
/// Kick's app does exactly this with it - finds the channel by `channel_id`
/// and replaces the balance it holds with `balance`. A balance is a whole
/// number of points, though the number is read leniently: it has come as a
/// float elsewhere on this API.
pub fn points_update(data: &serde_json::Value) -> Option<(u64, i64)> {
let number = |v: &serde_json::Value| v.as_i64().or_else(|| v.as_f64().map(|f| f as i64)).or_else(|| v.as_str().and_then(|s| s.parse().ok()));
let channel = data.get("channel_id").and_then(|v| v.as_u64().or_else(|| v.as_str().and_then(|s| s.parse().ok())))?;
let balance = number(data.get("balance")?)?;
Some((channel, balance))
}
/// The channel watch time is counted on.
pub fn channel_name(livestream_id: u64) -> String {
format!("private-livestream.{livestream_id}")
@@ -161,6 +187,17 @@ pub mod frames {
/// two viewers, and merging them would credit one for the other's time.
pub type Watching = std::collections::BTreeMap<String, u64>;
/// Tells the window what this account's points in a channel now are.
fn announce_points(state: &crate::state::AppState, account_id: &str, data: &serde_json::Value) {
let Some((channel_id, balance)) = points_update(data) else {
tracing::debug!("kick[{account_id}]: a points event with no channel or balance: {data}");
return;
};
// A channel this window has not opened has nowhere to show it.
let Some(buffer_id) = state.runtime.kick_buffer_of_channel(account_id, channel_id) else { return };
state.events.emit("kickPoints", serde_json::json!({ "bufferId": buffer_id, "points": balance }));
}
/// Keeps one connection open for as long as anything is being watched.
///
/// Restarted rather than repaired when the socket dies: a subscription is
@@ -211,6 +248,18 @@ async fn session(state: &crate::state::AppState, account_id: &str, user_id: &str
// changes without tearing the connection down.
let mut subscribed: std::collections::BTreeMap<String, u64> = Default::default();
// The balance, pushed as it changes: the earn every five minutes, a bet, a
// reward bought somewhere else. The window asks for it too, once a minute,
// which is the fallback for a channel that is not live and so is not here.
let points = points_channel_name(user_id);
match subscription_token(&http, token, &points).await {
Ok(jwt) => {
socket.send(Message::Text(frames::subscribe(id(), &points, &jwt).to_string())).await?;
tracing::info!("kick[{account_id}]: following {points}");
}
Err(e) => tracing::debug!("kick[{account_id}]: cannot follow points: {e:#}"),
}
loop {
let wanted = state.runtime.kick_watching(account_id);
if wanted.is_empty() || begun.elapsed() > std::time::Duration::from_secs(30 * 60) {
@@ -256,6 +305,12 @@ async fn session(state: &crate::state::AppState, account_id: &str, user_id: &str
Message::Text(text) => {
for line in text.lines().filter(|l| !l.trim().is_empty()) {
let Ok(value) = serde_json::from_str::<serde_json::Value>(line) else { continue };
if let super::realtime::Said::Published { channel, event, data } = super::realtime::read(line).into_iter().next().unwrap_or(super::realtime::Said::Other) {
if channel == points && event.ends_with("ChannelPointsUpdatedEvent") {
announce_points(state, account_id, &data);
}
continue;
}
if frames::is_ping(&value) {
socket.send(Message::Text(frames::pong().to_string())).await?;
} else if let Some(why) = frames::error_in(&value) {
@@ -296,6 +351,19 @@ mod tests {
assert_eq!(frames::pong(), json!({}));
}
/// As Kick's own app reads it: the channel, and the balance that replaces the one held.
#[test]
fn a_points_event_names_the_channel_and_the_new_balance() {
assert_eq!(points_channel_name("1234"), "private-channelpoints-1234");
assert_eq!(points_update(&json!({ "channel_id": 5312671, "balance": 1450 })), Some((5312671, 1450)));
// However the number is written.
assert_eq!(points_update(&json!({ "channel_id": "668", "balance": 12.0 })), Some((668, 12)));
assert_eq!(points_update(&json!({ "channel_id": 668, "balance": "90" })), Some((668, 90)));
// Without both it says nothing.
assert_eq!(points_update(&json!({ "channel_id": 668 })), None);
assert_eq!(points_update(&json!({ "balance": 5 })), None);
}
#[test]
fn a_subscription_carries_the_token_that_names_the_watcher() {
let f = frames::subscribe(2, "private-livestream.1", "jwt");
src/backend/matrix/timeline.rs
+2 -2
View File
@@ -24,7 +24,7 @@ pub(super) fn relation_preview(state: &AppState, buffer_id: &str, content: &Valu
let in_thread = thread.is_some();
let target = thread.or_else(|| protocol::reply_target(content))?;
match state.store.get_message(buffer_id, target) {
Ok(Some(m)) => Some(crate::model::ReplyPreview { id: target.to_string(), from: m.from, body: m.body, thread: in_thread, forwarded: false }),
Ok(Some(m)) => Some(crate::model::ReplyPreview { id: target.to_string(), from: m.from, body: m.body, thread: in_thread, forwarded: false, thread_root: None, chat: None }),
_ => Some(crate::model::ReplyPreview { id: target.to_string(), thread: in_thread, ..Default::default() }),
}
}
@@ -348,7 +348,7 @@ pub(super) async fn handle_timeline_event(
badges: vec![crate::backend::kick::api::Badge {
kind: "unverified".to_string(),
text: "Sent from a device you have not verified".to_string(),
count: None,
..Default::default()
}],
}),
_ => None,
src/backend/mod.rs
+2
View File
@@ -16,3 +16,5 @@ pub mod kick;
pub mod matrix;
pub mod sneedchat;
pub mod steam;
pub mod telegram;
pub mod twitch;
src/backend/steam/auth.rs
+94 -14
View File
@@ -27,19 +27,75 @@ use base64::Engine;
use rsa::{BigUint, Pkcs1v15Encrypt, RsaPublicKey};
use std::time::Duration;
/// What moho signs in as: the Steam mobile app's chat session.
/// What moho signs in as: a desktop Steam client, the way every other third-party client does.
///
/// This is part of the token Steam issues and decides how Steam treats the
/// connection that uses it. The desktop client's session is one per account,
/// so signing in as it would sign the Steam launcher out (and the reverse).
/// The mobile chat session is chat and friends only, and lives beside the
/// launcher. It is the only kind moho offers.
/// This is part of the token Steam issues and decides how Steam treats the connection that uses it. It was the
/// mobile app's chat session (platform 3, app type 2, website "Mobile") until Steam raised an account alert for an
/// unexpected device: a "phone" that reports a Linux desktop's operating system, has no machine id and logs on again
/// every time its connection is closed does not look like any real device. SteamKit2, ArchiSteamFarm, node-steam-user,
/// node-steam-session and steam-vent all sign in as the desktop client (platform 1, website "Client"), and so does this.
///
/// `EAuthTokenPlatformType` 3 is the mobile app.
const PLATFORM_TYPE: i32 = 3;
/// `EAuthTokenAppType` 2 is the mobile chat app.
const APP_TYPE: i32 = 2;
const WEBSITE_ID: &str = "Mobile";
/// `EAuthTokenPlatformType` 1 is the desktop client. It has no app type.
const PLATFORM_TYPE: i32 = 1;
const WEBSITE_ID: &str = "Client";
/// `EGamingDeviceType` 1 is an ordinary PC, as node-steam-session sends for the desktop client.
const GAMING_DEVICE_TYPE: u32 = 1;
/// This install's own random number, made once and kept next to the daemon's other files, from which its machine
/// id and logon id are drawn. One per install and not per account: a machine is one machine however many accounts
/// sign in on it, and two installs (a test window, a second computer) are two.
pub fn install_id() -> [u8; 16] {
// A test never writes to the real data directory.
#[cfg(test)]
return [7u8; 16];
#[cfg(not(test))]
{
static ID: std::sync::OnceLock<[u8; 16]> = std::sync::OnceLock::new();
*ID.get_or_init(|| {
let path = crate::default_data_dir().join("steam-install-id");
if let Some(read) = std::fs::read_to_string(&path).ok().and_then(|t| hex::decode(t.trim()).ok()).and_then(|b| <[u8; 16]>::try_from(b).ok()) {
return read;
}
let made: [u8; 16] = rand::random();
let _ = std::fs::create_dir_all(path.parent().unwrap_or(std::path::Path::new(".")));
let _ = std::fs::write(&path, hex::encode(made));
made
})
}
}
/// The machine id Steam's client sends: a binary key-value object of three SHA-1 hashes (named BB3, FF2 and 3B3 in the
/// client), here of strings made from the install's number, so it is the same each time and different for each install.
pub fn machine_id_from(install: &[u8; 16]) -> Vec<u8> {
use sha1::{Digest, Sha1};
let mut out = vec![0u8];
out.extend_from_slice(b"MessageObject\0");
for name in ["BB3", "FF2", "3B3"] {
let hash = hex::encode(Sha1::digest(format!("moho {name} {}", hex::encode(install)).as_bytes()));
out.push(1);
out.extend_from_slice(name.as_bytes());
out.push(0);
out.extend_from_slice(hash.as_bytes());
out.push(0);
}
out.extend_from_slice(&[8, 8]);
out
}
pub fn machine_id() -> Vec<u8> {
machine_id_from(&install_id())
}
/// What tells this install's sessions from another's on the same account: Steam ends a session when a later one with
/// the same logon id arrives, and lets those with different ones live side by side. The desktop client sends its own
/// private address here; this sends a number of its own, never zero (which is "none").
pub fn logon_id_from(install: &[u8; 16]) -> u32 {
(u32::from_le_bytes([install[0], install[1], install[2], install[3]]) | 1) ^ 0xBAAD_F00D
}
pub fn logon_id() -> u32 {
logon_id_from(&install_id())
}
/// `EOSType` as the connection reports it: Linux 6.x is -184 and Windows 10
/// is 16, sent as the unsigned number the field is.
@@ -155,7 +211,9 @@ fn device_details() -> DeviceDetails {
device_friendly_name: Some(device_name()),
platform_type: Some(PLATFORM_TYPE),
os_type: Some(os_type() as i32),
app_type: Some(APP_TYPE),
gaming_device_type: Some(GAMING_DEVICE_TYPE),
// Sent with a password sign-in, as the desktop client's is.
machine_id: Some(machine_id()),
..DeviceDetails::default()
}
}
@@ -433,8 +491,11 @@ mod tests {
assert_eq!(begin_req.encryption_timestamp, Some(1790000005000));
assert_eq!(begin_req.persistence, Some(1));
assert_eq!(begin_req.guard_data.as_deref(), Some("old-guard"));
assert_eq!(begin_req.platform_type, Some(3), "it signs in as the mobile app, never as the desktop client");
assert_eq!(begin_req.device_details.unwrap().app_type, Some(2));
assert_eq!(begin_req.platform_type, Some(1), "it signs in as a desktop client, like every other third-party client");
assert_eq!(begin_req.website_id.as_deref(), Some("Client"));
let details = begin_req.device_details.unwrap();
assert_eq!((details.platform_type, details.app_type, details.gaming_device_type), (Some(1), None, Some(1)));
assert_eq!(details.machine_id.as_deref(), Some(machine_id().as_slice()), "and with its install's machine id");
assert!(!begin_req.encrypted_password.unwrap().contains("hunter2"));
let code_req = UpdateAuthWithGuardCodeRequest::decode(seen[2].body.as_slice()).unwrap();
@@ -519,4 +580,23 @@ mod tests {
assert_eq!(steamid_of_token("not a token"), None);
assert_eq!(steamid_of_token("a.b.c"), None);
}
#[test]
fn the_machine_id_is_a_key_value_object_of_three_hashes_that_is_the_same_for_one_install_and_not_for_another() {
let one = machine_id_from(&[1u8; 16]);
assert_eq!(one, machine_id_from(&[1u8; 16]));
assert_ne!(one, machine_id_from(&[2u8; 16]));
// 0x00 "MessageObject" 0x00, then three strings named BB3, FF2 and 3B3 of forty hex digits, then 0x08 0x08.
assert!(one.starts_with(b"\0MessageObject\0\x01BB3\0"));
assert!(one.ends_with(&[0, 8, 8]));
let text = String::from_utf8_lossy(&one).to_string();
for name in ["BB3", "FF2", "3B3"] {
assert!(text.contains(name));
}
assert_eq!(one.len(), 1 + 14 + 3 * (1 + 3 + 1 + 40 + 1) + 2);
assert_ne!(logon_id_from(&[1u8; 16]), 0);
assert_ne!(logon_id_from(&[1u8; 16]), logon_id_from(&[2u8; 16]));
// Never the same as the zero a client with no id of its own sends.
assert_ne!(logon_id_from(&[0xFF; 16]), 0);
}
}
src/backend/steam/connect.rs
+4 -3
View File
@@ -39,8 +39,8 @@ pub const PROTOCOL_VERSION: u32 = 65581;
/// How long Steam has to answer a sign-in.
const LOGON_TIMEOUT: Duration = Duration::from_secs(30);
const RECONNECT_INITIAL_DELAY: Duration = Duration::from_secs(3);
const RECONNECT_MAX_DELAY: Duration = Duration::from_secs(60);
const RECONNECT_INITIAL_DELAY: Duration = Duration::from_secs(10);
const RECONNECT_MAX_DELAY: Duration = Duration::from_secs(300);
/// How long to leave an account alone after Steam says it is being asked too
/// often. Seconds would be asking again; this is the length of time that the
@@ -124,10 +124,11 @@ pub fn logon_message(config: &SteamAccountConfig, language: &str) -> ClientLogon
should_remember_password: Some(true),
// Where the machine is on its own network is nothing Steam needs to
// be told; the field exists and asks to be filled.
obfuscated_private_ip: Some(IpAddress { ip: Some(ip_address::Ip::V4(0)) }),
obfuscated_private_ip: Some(IpAddress { ip: Some(ip_address::Ip::V4(auth::logon_id())) }),
chat_mode: Some(2),
account_name: Some(config.username.clone()),
machine_name: Some(auth::device_name()),
machine_id: Some(auth::machine_id()),
supports_rate_limit_response: Some(true),
// The refresh token, which is what this field takes whatever its name.
access_token: Some(config.refresh_token.clone()),
src/backend/steam/login.rs
+1 -1
View File
@@ -305,7 +305,7 @@ pub async fn flow(
display_name: None,
refresh_token: tokens.refresh_token,
guard_data: tokens.guard_data,
// Always the mobile chat session; kept so older files still read.
// The kind is no longer chosen; kept so older files still read.
session: "chat".to_string(),
use_tor: crate::net::route::router().wanted(&key),
strict_route: false,
src/backend/telegram/admin.rs
+46
View File
@@ -0,0 +1,46 @@
//! What an administrator may do to a member: remove, ban, silence.
//!
//! The menu beside the member list asks what this account may do here, and offers only that. Telegram has the last
//! word: an administrator who lacks the particular right is refused ("Only an administrator may do that here").
use super::send::{explain, peer_for};
use crate::model::Buffer;
use crate::state::AppState;
use grammers_session::types::{PeerId, PeerKind};
use grammers_session::Session;
use serde_json::{json, Value};
use std::time::Duration;
/// How long a mute lasts, as everywhere else in moho.
const MUTE: Duration = Duration::from_secs(10 * 60);
/// What this account may do to others in a conversation, in the shape the member menu reads.
pub async fn permissions(state: &AppState, buffer: &Buffer) -> Result<Value, String> {
let none = json!({ "canRedactOthers": false, "canKick": false, "canBan": false, "canMute": false });
if buffer.kind == "dm" {
return Ok(none);
}
let (handle, peer) = peer_for(state, buffer).await?;
let me = handle.client.get_me().await.map_err(|e| explain(&e))?;
let Ok(Some(me)) = me.to_ref().await else { return Ok(none) };
let Ok(found) = handle.client.get_permissions(peer, me).await else { return Ok(none) };
let may = found.is_creator() || found.is_admin();
// A small group can only have a member removed; bans and silences are for supergroups and channels.
let supergroup = peer.id.kind() == PeerKind::Channel;
Ok(json!({ "canRedactOthers": may, "canKick": may, "canBan": may && supergroup, "canMute": may && supergroup }))
}
/// Removes, bans or silences one member.
pub async fn moderate(state: &AppState, buffer: &Buffer, user_id: &str, action: &str) -> Result<(), String> {
let id: i64 = user_id.parse().map_err(|_| "that member cannot be told apart from the others".to_string())?;
let (handle, peer) = peer_for(state, buffer).await?;
let user = handle.session.peer_ref(PeerId::user_unchecked(id)).await.ok().flatten().ok_or_else(|| "Telegram has not told this account about that member yet".to_string())?;
let supergroup = peer.id.kind() == PeerKind::Channel;
match action {
"kick" => handle.client.kick_participant(peer, user).await.map_err(|e| explain(&e)),
"ban" if supergroup => handle.client.set_banned_rights(peer, user).view_messages(false).await.map_err(|e| explain(&e)),
"mute" if supergroup => handle.client.set_banned_rights(peer, user).send_messages(false).duration(MUTE).await.map_err(|e| explain(&e)),
"ban" | "mute" => Err("Telegram only lets a supergroup or a channel ban or silence somebody".to_string()),
_ => Err("Telegram has no such action".to_string()),
}
}
src/backend/telegram/connect.rs
+288
View File
@@ -0,0 +1,288 @@
//! An account's connection: open the session it saved, stay connected, and reconnect when Telegram or the network
//! lets go - without ever signing in afresh to get over a hiccup (see mod.rs on why).
//!
//! Modelled on Steam's loop (`steam/connect.rs run_with_retry`): a wait that grows while a connection falls over at
//! once and starts again after one that held, a long wait when Telegram says to slow down, and a stop - not a retry
//! - when Telegram has ended the session, because the same key can never work again.
use super::link::{self, Link};
use super::messages::{self, Own};
use super::peers;
use super::Handle;
use super::session::{self, FileSession};
use crate::accounts::TelegramAccountConfig;
use crate::runtime::ConnState;
use crate::state::AppState;
use anyhow::{Context, Result};
use grammers_client::InvocationError;
use grammers_mtsender::UpdatesConfiguration;
use std::sync::Arc;
use std::time::Duration;
const RECONNECT_INITIAL_DELAY: Duration = Duration::from_secs(10);
const RECONNECT_MAX_DELAY: Duration = Duration::from_secs(300);
/// How often a changed session is written.
const SAVE_EVERY: Duration = Duration::from_secs(5);
/// Ends the account's session at Telegram, so removing it here does not leave a session in the account's list of
/// devices, and takes the saved session away. Best effort: the saved one is gone either way.
pub async fn sign_out(state: &AppState, config: &TelegramAccountConfig) {
let path = session::path_for(config.user_id);
if let Ok(session) = FileSession::open(path.clone()) {
let session = Arc::new(session);
if session.has_key() {
// The account was removed first, so its own key is the only route it has left: not Tor unless it was.
match link::open(state, &config.account_id(), config.api_id, session.clone()).await {
Ok(link) => {
if let Err(e) = link.client.sign_out().await {
tracing::info!("telegram[{}]: signing out at Telegram: {e}", config.account_id());
}
link.close().await;
}
Err(e) => tracing::info!("telegram[{}]: could not reach Telegram to sign out: {e:#}", config.account_id()),
}
}
session.delete();
} else {
let _ = std::fs::remove_file(path);
}
}
/// How a connection ended.
#[derive(Debug, PartialEq)]
pub enum Ending {
/// Telegram ended the session (or it is gone): sign in again.
SignInAgain(String),
/// Telegram asked for a pause, or the network is down: try again after this.
Later(Duration, String),
/// The connection ended; try again after the usual wait.
Closed,
}
/// What an error from Telegram means for the connection.
pub fn classify(error: &InvocationError) -> Ending {
match error {
InvocationError::Rpc(rpc) => {
let name = rpc.name.as_str();
if rpc.code == 401 || matches!(name, "AUTH_KEY_UNREGISTERED" | "SESSION_REVOKED" | "SESSION_EXPIRED" | "USER_DEACTIVATED" | "USER_DEACTIVATED_BAN" | "AUTH_KEY_DUPLICATED") {
Ending::SignInAgain(super::login::explain(name))
} else if name == "FLOOD_WAIT" || name == "FLOOD_PREMIUM_WAIT" {
// The number of seconds Telegram asked for, and a little more.
Ending::Later(Duration::from_secs(u64::from(rpc.value.unwrap_or(60)) + 5), format!("Telegram asked for a pause of {} seconds", rpc.value.unwrap_or(60)))
} else {
Ending::Closed
}
}
_ => Ending::Closed,
}
}
/// Starts (or restarts) an account's connection.
pub fn spawn(state: AppState, config: TelegramAccountConfig) {
let account_id = config.account_id();
crate::net::route::router().mark(&account_id, config.use_tor);
state.runtime.reset_connection(&account_id);
let handle = tokio::spawn({
let account_id = account_id.clone();
let state = state.clone();
async move {
run_with_retry(&state, &account_id).await;
state.runtime.remove_task_handle(&account_id);
}
});
state.runtime.insert_task_handle(&account_id, handle.abort_handle());
}
async fn run_with_retry(state: &AppState, account_id: &str) {
let mut delay = RECONNECT_INITIAL_DELAY;
state.runtime.set_conn_state(state, account_id, ConnState::Connecting, None);
loop {
// The account as it is now: removed while this slept, it is not reconnected.
let Some(config) = state.accounts.get_telegram(account_id) else { return };
let (detail, wait) = match run(state, &config, account_id).await {
Ok(Ending::SignInAgain(why)) => {
tracing::info!("telegram[{account_id}]: stopped: {why}");
state.runtime.set_conn_state(state, account_id, ConnState::AuthFailed, Some(&format!("{why}. Sign in to Telegram again from Accounts.")));
return;
}
Ok(Ending::Later(wait, why)) => (why, Some(wait)),
Ok(Ending::Closed) => ("connection ended".to_string(), None),
Err(e) => {
tracing::warn!("telegram[{account_id}]: {e:#}");
(format!("{e:#}"), None)
}
};
state.runtime.set_conn_state(state, account_id, ConnState::Connecting, None);
let wait = wait.unwrap_or_else(|| state.runtime.next_retry_delay(account_id, &mut delay, RECONNECT_INITIAL_DELAY, RECONNECT_MAX_DELAY));
tracing::info!("telegram[{account_id}]: not connected ({detail}); trying again in {}s", wait.as_secs());
state.runtime.report_retry(state, account_id, &detail, wait);
tokio::time::sleep(wait).await;
}
}
/// One go at being connected.
async fn run(state: &AppState, config: &TelegramAccountConfig, account_id: &str) -> Result<Ending> {
let session = Arc::new(FileSession::open(session::path_for(config.user_id))?);
if !session.has_key() {
return Ok(Ending::SignInAgain("this account has no saved session".to_string()));
}
let mut link = link::open(state, account_id, config.api_id, session).await?;
let ending = stay(state, config, account_id, &mut link).await;
state.runtime.clear_telegram_handle(account_id);
link.close().await;
ending
}
async fn stay(state: &AppState, config: &TelegramAccountConfig, account_id: &str, link: &mut Link) -> Result<Ending> {
match link.client.is_authorized().await {
Ok(true) => {}
Ok(false) => return Ok(Ending::SignInAgain("Telegram no longer accepts this session".to_string())),
Err(e) => return Ok(classify(&e)).and_then(|ending| if ending == Ending::Closed { Err(anyhow::Error::from(e)).context("asking Telegram whether the session is good") } else { Ok(ending) }),
}
// The name and number the account has now, which are Telegram's to change.
let mut own = Own { name: if config.name.is_empty() { config.phone.clone() } else { config.name.clone() }, user_id: config.user_id };
if let Ok(me) = link.client.get_me().await {
let (name, username) = (me.full_name(), me.username().map(str::to_string));
if !name.is_empty() {
own.name = name.clone();
}
let _ = state.accounts.mutate_telegram(account_id, |a| {
a.name = name;
a.username = username;
});
}
state.runtime.set_own_identity(account_id, &own.name);
let handle = Handle { client: link.client.clone(), session: link.session.clone(), own: own.clone(), names: Default::default() };
state.runtime.set_telegram_handle(account_id, handle.clone());
state.runtime.set_conn_state(state, account_id, ConnState::Connected, None);
tracing::info!("telegram[{account_id}]: connected as {} ({})", config.user_id, config.username.as_deref().unwrap_or("no username"));
// The conversations, in the background: the connection is up whether or not this succeeds.
tokio::spawn({
let (state, account_id, handle) = (state.clone(), account_id.to_string(), handle.clone());
async move {
if let Err(e) = super::dialogs::sync(&state, &account_id, &handle).await {
tracing::warn!("telegram[{account_id}]: reading the conversations: {e:#}");
}
}
});
let updates = link.updates.take().context("the updates were already taken")?;
let mut stream = link.client.stream_updates(updates, UpdatesConfiguration { catch_up: true }).await.map_err(|e| anyhow::anyhow!("starting to read updates: {e}"))?;
let mut save = tokio::time::interval(SAVE_EVERY);
loop {
tokio::select! {
_ = save.tick() => {
if let Err(e) = link.session.save_if_dirty() {
tracing::warn!("telegram[{account_id}]: saving the session: {e:#}");
}
}
update = stream.next() => match update {
Ok(update) => handle_update(state, account_id, &handle, update).await,
Err(e) => {
let _ = stream.sync_update_state().await;
return match classify(&e) {
Ending::Closed => Err(anyhow::Error::from(e)).context("reading updates"),
other => Ok(other),
};
}
},
}
}
}
/// What Telegram sent unasked: a message, an edit, something deleted.
async fn handle_update(state: &AppState, account_id: &str, handle: &Handle, update: grammers_client::update::Update) {
use grammers_client::update::Update;
match update {
Update::NewMessage(message) => {
let recorded = match message.peer() {
Some(_) => messages::record_named(state, account_id, &handle.own, &message, Some(&handle.names)),
// A short update from a person names no one: asked of Telegram, who is the other side.
None => {
let (state, account_id, handle) = (state.clone(), account_id.to_string(), handle.clone());
tokio::spawn(async move { record_unnamed(&state, &account_id, &handle, &message).await });
return;
}
};
tracing::debug!("telegram[{account_id}]: new message in {:?}: recorded {recorded}", message.peer_id());
}
Update::Raw(raw) => super::updates::raw(state, account_id, handle, &raw.raw),
Update::MessageEdited(message) => {
let peer = match message.peer() {
Some(peer) => Some(peer.clone()),
None => resolve(handle, &message).await,
};
if let Some(peer) = peer {
let buffer = peers::buffer_for(state, account_id, &peer);
let body = messages::body_of(&message);
let attachments = buffer.remote_id.as_deref().map(|remote| super::media::attachments_of(handle.own.user_id, remote, &message)).unwrap_or_default();
state.runtime.update_message(state, &buffer.id, &message.id().to_string(), &body, &[], &attachments);
}
}
Update::MessageDeleted(deleted) => {
for id in deleted.messages() {
forget_message(state, account_id, deleted.channel_id(), *id);
}
}
_ => {}
}
}
/// The conversation a message from a short update belongs to, from what the session knows of the person.
async fn resolve(handle: &Handle, message: &grammers_client::message::Message) -> Option<grammers_client::peer::Peer> {
let peer_ref = message.peer_ref().await.ok().flatten()?;
handle.client.resolve_peer(peer_ref).await.ok()
}
async fn record_unnamed(state: &AppState, account_id: &str, handle: &Handle, message: &grammers_client::message::Message) {
match resolve(handle, message).await {
Some(peer) => {
messages::record_in(state, account_id, &handle.own, message, &peer, None, Some(&handle.names));
}
None => tracing::warn!("telegram[{account_id}]: a message in {:?} from somebody this account has no record of was not shown", message.peer_id()),
}
}
/// Removes a message Telegram says is gone. A channel's deletion names the channel; any other names only the number,
/// which is unique across the account's people and small groups, so the conversations are looked through.
fn forget_message(state: &AppState, account_id: &str, channel: Option<i64>, id: i32) {
let msg_id = id.to_string();
let candidates: Vec<crate::model::Buffer> = state
.runtime
.list_buffers()
.into_iter()
.filter(|b| b.account_id == account_id)
.filter(|b| match (channel, b.remote_id.as_deref()) {
(Some(c), Some(remote)) => remote == peers::remote_id(grammers_session::types::PeerId::channel_unchecked(c)),
(Some(_), None) => false,
(None, Some(remote)) => !remote.starts_with("-100"),
(None, None) => false,
})
.collect();
for buffer in candidates {
if state.store.get_message(&buffer.id, &msg_id).ok().flatten().is_some() {
state.runtime.delete_message(state, &buffer.id, &msg_id);
return;
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use grammers_mtsender::RpcError;
fn rpc(code: i32, name: &str, value: Option<u32>) -> InvocationError {
InvocationError::Rpc(RpcError { code, name: name.to_string(), value, caused_by: None })
}
#[test]
fn a_session_telegram_ended_stops_the_connection_and_a_flood_wait_waits_as_long_as_asked() {
assert!(matches!(classify(&rpc(401, "AUTH_KEY_UNREGISTERED", None)), Ending::SignInAgain(_)));
assert!(matches!(classify(&rpc(401, "SESSION_REVOKED", None)), Ending::SignInAgain(_)));
assert!(matches!(classify(&rpc(420, "FLOOD_WAIT", Some(120))), Ending::Later(d, _) if d == Duration::from_secs(125)));
assert_eq!(classify(&rpc(500, "INTERDC_CALL_ERROR", None)), Ending::Closed);
assert_eq!(classify(&InvocationError::Io(std::io::Error::other("reset"))), Ending::Closed);
}
}
src/backend/telegram/dialogs.rs
+53
View File
@@ -0,0 +1,53 @@
//! The conversations an account has: asked for when the connection comes up, and each one's last message with it.
//!
//! Telegram's dialog list answers with every conversation and, in the same response, its last message, so one
//! request gives the list its names, kinds and recent activity without opening any of them. History proper is read
//! when a conversation is opened (see `history`).
use super::media;
use super::messages;
use super::Handle;
use super::peers;
use crate::state::AppState;
use anyhow::Result;
use grammers_client::tl;
/// How many conversations are read on connect. Telegram pages them a hundred at a time.
const LIMIT: usize = 200;
/// Makes a buffer of each of the account's conversations, newest activity first, and records the last message of each.
pub async fn sync(state: &AppState, account_id: &str, handle: &Handle) -> Result<usize> {
let mut dialogs = handle.client.iter_dialogs();
let mut count = 0;
while count < LIMIT {
let Some(dialog) = dialogs.next().await.map_err(|e| anyhow::anyhow!("{e}"))? else { break };
let buffer = peers::buffer_for(state, account_id, dialog.peer());
// Filed again each time: a bot read before bots had a heading of their own was filed with the people.
state.runtime.set_buffer_category(state, &buffer.id, Some(peers::category_of(dialog.peer())), buffer.position);
state.runtime.set_buffer_name_icon(state, &buffer.id, peers::name_icon_of(dialog.peer()));
if let grammers_client::peer::Peer::User(user) = dialog.peer() {
handle.names.lock().unwrap_or_else(|e| e.into_inner()).insert(user.id().bare_id_unchecked(), peers::title_of(dialog.peer()));
}
if let Some(last) = &dialog.last_message {
messages::record_named(state, account_id, &handle.own, last, Some(&handle.names));
}
// Silenced on the account, perhaps on the phone.
if let tl::enums::Dialog::Dialog(d) = &dialog.raw {
state.runtime.set_silenced(state, &buffer.id, super::notify::silent(&d.notify_settings, chrono::Utc::now().timestamp()));
}
// Nothing unread there, as Telegram counts it: the window need not call it unread either.
if matches!(&dialog.raw, tl::enums::Dialog::Dialog(d) if d.unread_count == 0) {
state.events.emit("bufferRead", serde_json::json!({ "accountId": account_id, "bufferId": buffer.id }));
}
if let Some(avatar) = media::avatar_url(handle.own.user_id, dialog.peer()) {
state.runtime.set_buffer_avatar(state, &buffer.id, &avatar);
}
// Whether the account may write there, read off the dialog's own copy of the chat (which carries its rights):
// where it may not, the window shows why in place of the message box.
let now = chrono::Utc::now().timestamp();
state.runtime.set_buffer_read_only(state, &buffer.id, peers::read_only_for(dialog.peer(), now));
count += 1;
}
tracing::info!("telegram[{account_id}]: {count} conversations");
Ok(count)
}
src/backend/telegram/history.rs
+85
View File
@@ -0,0 +1,85 @@
//! A conversation's history, read when it is opened and again when it is scrolled back.
//!
//! Telegram keeps every message, so opening a conversation asks for its recent end, and scrolling to the top asks for
//! the part before. What is stored is returned by `getBacklog` either way, so a failure to reach Telegram leaves what
//! there is rather than an error.
use super::peers;
use crate::model::Buffer;
use crate::state::AppState;
use anyhow::{anyhow, Result};
use grammers_session::Session;
/// How many messages a page asks for.
const PAGE: usize = 50;
async fn read(state: &AppState, buffer: &Buffer, before: Option<i64>) -> Result<usize> {
let handle = state.runtime.telegram_handle(&buffer.account_id).ok_or_else(|| anyhow!("this Telegram account is not connected"))?;
let peer = buffer.remote_id.as_deref().and_then(peers::peer_id_of).ok_or_else(|| anyhow!("this is not a Telegram conversation"))?;
let peer = handle.session.peer_ref(peer).await.ok().flatten().ok_or_else(|| anyhow!("Telegram has not told this account about that conversation yet"))?;
let mut iter = handle.client.iter_messages(peer).limit(PAGE);
if let Some(ts) = before.filter(|t| *t > 0) {
iter = iter.offset_date(ts.clamp(0, i64::from(i32::MAX)) as i32);
}
let mut got = Vec::new();
while let Some(message) = iter.next().await.map_err(|e| anyhow!("{e}"))? {
got.push(message);
}
// Oldest first, so a reply finds the message it answers already stored.
got.reverse();
let count = got.len();
for message in &got {
super::messages::record(state, &buffer.account_id, &handle.own, message);
}
Ok(count)
}
/// Opening a conversation: its recent end.
pub async fn open(state: &AppState, buffer: &Buffer) -> Result<usize> {
read(state, buffer, None).await
}
/// Scrolling back: what came before `before` (a Unix time).
pub async fn older(state: &AppState, buffer: &Buffer, before: i64) -> Result<usize> {
read(state, buffer, Some(before)).await
}
/// Reads the messages around one, so a jump to an old message (a pin, a reply, a search result) arrives with what was
/// said before and after it. Returns the message's time.
pub async fn context(state: &AppState, buffer: &Buffer, msg_id: &str) -> Result<i64> {
let id: i32 = msg_id.parse().map_err(|_| anyhow!("that is not a Telegram message"))?;
let handle = state.runtime.telegram_handle(&buffer.account_id).ok_or_else(|| anyhow!("this Telegram account is not connected"))?;
let peer = buffer.remote_id.as_deref().and_then(peers::peer_id_of).ok_or_else(|| anyhow!("this is not a Telegram conversation"))?;
let peer = handle.session.peer_ref(peer).await.ok().flatten().ok_or_else(|| anyhow!("Telegram has not told this account about that conversation yet"))?;
let mut around = Vec::new();
// Itself and what came before it, newest first...
let mut older = handle.client.iter_messages(peer).offset_id(id.saturating_add(1)).limit(26);
while let Some(message) = older.next().await.map_err(|e| anyhow!("{e}"))? {
around.push(message);
}
// ...and what came after, oldest first.
let mut newer = handle.client.iter_messages(peer).reverse(true).offset_id(id).limit(25);
while let Some(message) = newer.next().await.map_err(|e| anyhow!("{e}"))? {
around.push(message);
}
around.sort_by_key(|m| m.id());
for message in &around {
super::messages::record(state, &buffer.account_id, &handle.own, message);
}
around.iter().find(|m| m.id() == id).map(|m| m.date().as_second()).ok_or_else(|| anyhow!("that message is no longer there"))
}
/// Reads on from a message towards the present, for a reader who arrived in the middle of a conversation.
pub async fn newer(state: &AppState, buffer: &Buffer, msg_id: &str) -> Result<usize> {
let id: i32 = msg_id.parse().map_err(|_| anyhow!("that is not a Telegram message"))?;
let handle = state.runtime.telegram_handle(&buffer.account_id).ok_or_else(|| anyhow!("this Telegram account is not connected"))?;
let peer = buffer.remote_id.as_deref().and_then(peers::peer_id_of).ok_or_else(|| anyhow!("this is not a Telegram conversation"))?;
let peer = handle.session.peer_ref(peer).await.ok().flatten().ok_or_else(|| anyhow!("Telegram has not told this account about that conversation yet"))?;
let mut iter = handle.client.iter_messages(peer).reverse(true).offset_id(id).limit(PAGE);
let mut count = 0;
while let Some(message) = iter.next().await.map_err(|e| anyhow!("{e}"))? {
super::messages::record(state, &buffer.account_id, &handle.own, &message);
count += 1;
}
Ok(count)
}
src/backend/telegram/instant.rs
+447
View File
@@ -0,0 +1,447 @@
//! Instant View: the article a link's preview carries, drawn as a page of its own.
//!
//! A message that links to a page Telegram has made an Instant View of carries the page's blocks (headings,
//! paragraphs with their styling, pictures, lists, quotes, tables) as a tree. The window opens it in a window of its
//! own, so this turns the tree into one self-contained HTML document: everything in it is escaped here, pictures are
//! fetched into the cache and named by their local file, and links are limited to the kinds a page may reasonably have.
//! What it cannot draw (embedded players, maps, other channels' posts) is left out, with the page's own link on the
//! card to open it in a browser instead.
use super::media::{download_thumb, download_to, media_name, pick, Variant, MEDIA};
use super::send::{explain, peer_for};
use crate::model::Buffer;
use crate::state::AppState;
use grammers_client::media::{Document, Media, Photo};
use grammers_client::tl;
use std::collections::HashMap;
use std::path::PathBuf;
use tl::enums::{PageBlock as B, RichText as R};
/// A finished article: its title, for the window, and the document.
pub struct Article {
pub title: String,
pub url: String,
pub html: String,
}
/// What a page needs fetched: a picture, or a video's poster.
#[derive(Clone, Copy, PartialEq, Eq, Hash, Debug)]
enum Want {
Photo(i64),
Poster(i64),
}
struct Cx {
account: i64,
remote: String,
message: i32,
photos: HashMap<i64, tl::enums::Photo>,
documents: HashMap<i64, tl::enums::Document>,
wanted: Vec<Want>,
}
fn escape(text: &str) -> String {
let mut out = String::with_capacity(text.len());
for c in text.chars() {
match c {
'&' => out.push_str("&amp;"),
'<' => out.push_str("&lt;"),
'>' => out.push_str("&gt;"),
'"' => out.push_str("&quot;"),
'\'' => out.push_str("&#39;"),
c => out.push(c),
}
}
out
}
/// An address a page's link may point to: web pages, mail, phones and Telegram's own, or an anchor in the page.
fn safe_url(url: &str) -> String {
let lower = url.trim().to_ascii_lowercase();
let ok = ["http://", "https://", "mailto:", "tel:", "tg://", "#"].iter().any(|p| lower.starts_with(p));
if ok { escape(url.trim()) } else { "#".to_string() }
}
fn percent(path: &str) -> String {
let mut out = String::new();
for b in path.bytes() {
if b.is_ascii_alphanumeric() || b"-_.~".contains(&b) {
out.push(b as char);
} else {
out.push_str(&format!("%{b:02X}"));
}
}
out
}
impl Cx {
fn path(&self, want: Want) -> PathBuf {
let (kind, id) = match want {
Want::Photo(id) => ("iv", id),
Want::Poster(id) => ("ivp", id),
};
let name = media_name(self.account, &self.remote, self.message, Variant::Full, "jpg").replace("_full.jpg", &format!("_{kind}{id}.jpg"));
crate::media_cache::dir(MEDIA).join(name)
}
/// The address of a picture the page uses, once it is fetched; `None` when the page does not carry it.
fn picture(&mut self, want: Want) -> Option<String> {
let known = match want {
Want::Photo(id) => self.photos.contains_key(&id),
Want::Poster(id) => self.documents.contains_key(&id),
};
if !known {
tracing::info!("telegram: an article uses {want:?}, which its page does not carry ({} pictures, {} files)", self.photos.len(), self.documents.len());
return None;
}
if !self.wanted.contains(&want) {
self.wanted.push(want);
}
Some(format!("moho-media://file/?p={}", percent(&self.path(want).display().to_string())))
}
}
fn rich(t: &R, cx: &mut Cx) -> String {
let wrap = |tag: &str, inner: &R, cx: &mut Cx| format!("<{tag}>{}</{tag}>", rich(inner, cx));
match t {
R::TextEmpty => String::new(),
R::TextPlain(p) => escape(&p.text).replace('\n', "<br>"),
R::TextBold(b) => wrap("b", &b.text, cx),
R::TextItalic(i) => wrap("i", &i.text, cx),
R::TextUnderline(u) => wrap("u", &u.text, cx),
R::TextStrike(s) => wrap("s", &s.text, cx),
R::TextFixed(f) => wrap("code", &f.text, cx),
R::TextSubscript(s) => wrap("sub", &s.text, cx),
R::TextSuperscript(s) => wrap("sup", &s.text, cx),
R::TextMarked(m) => wrap("mark", &m.text, cx),
R::TextUrl(u) => format!("<a href=\"{}\">{}</a>", safe_url(&u.url), rich(&u.text, cx)),
R::TextEmail(e) => format!("<a href=\"mailto:{}\">{}</a>", safe_url(&e.email).trim_start_matches("mailto:"), rich(&e.text, cx)),
R::TextPhone(p) => format!("<a href=\"tel:{}\">{}</a>", escape(&p.phone), rich(&p.text, cx)),
R::TextConcat(c) => c.texts.iter().map(|t| rich(t, cx)).collect(),
R::TextAnchor(a) => format!("<span id=\"{}\">{}</span>", escape(&a.name), rich(&a.text, cx)),
R::TextSpoiler(s) => rich(&s.text, cx),
R::TextMention(m) => rich(&m.text, cx),
R::TextHashtag(m) => rich(&m.text, cx),
R::TextBotCommand(m) => rich(&m.text, cx),
R::TextCashtag(m) => rich(&m.text, cx),
R::TextBankCard(m) => rich(&m.text, cx),
R::TextAutoUrl(m) => rich(&m.text, cx),
R::TextAutoEmail(m) => rich(&m.text, cx),
R::TextAutoPhone(m) => rich(&m.text, cx),
_ => String::new(),
}
}
fn plain(t: &R) -> String {
let mut cx = Cx { account: 0, remote: String::new(), message: 0, photos: HashMap::new(), documents: HashMap::new(), wanted: Vec::new() };
let html = rich(t, &mut cx);
// Only for a title: the tags taken out again.
let mut out = String::new();
let mut inside = false;
for c in html.chars() {
match c {
'<' => inside = true,
'>' => inside = false,
c if !inside => out.push(c),
_ => {}
}
}
out.replace("&amp;", "&").replace("&lt;", "<").replace("&gt;", ">").replace("&quot;", "\"").replace("&#39;", "'").trim().to_string()
}
fn is_empty(t: &R) -> bool {
matches!(t, R::TextEmpty) || plain(t).is_empty()
}
fn caption(c: &tl::enums::PageCaption, cx: &mut Cx) -> String {
let tl::enums::PageCaption::Caption(c) = c;
let (text, credit) = (rich(&c.text, cx), rich(&c.credit, cx));
if text.is_empty() && credit.is_empty() {
return String::new();
}
format!("<figcaption>{text}{}</figcaption>", if credit.is_empty() { String::new() } else { format!("<cite>{credit}</cite>") })
}
fn blocks(list: &[B], cx: &mut Cx) -> String {
list.iter().map(|b| block(b, cx)).collect()
}
fn heading(tag: &str, t: &R, cx: &mut Cx) -> String {
format!("<{tag}>{}</{tag}>", rich(t, cx))
}
fn block(b: &B, cx: &mut Cx) -> String {
match b {
B::Title(t) => heading("h1", &t.text, cx),
B::Subtitle(t) => format!("<p class=\"subtitle\">{}</p>", rich(&t.text, cx)),
B::Kicker(t) => format!("<p class=\"kicker\">{}</p>", rich(&t.text, cx)),
B::AuthorDate(a) => {
let who = rich(&a.author, cx);
let when = chrono::DateTime::from_timestamp(i64::from(a.published_date), 0).filter(|_| a.published_date > 0).map(|d| d.format("%B %-d, %Y").to_string());
let parts: Vec<String> = [Some(who).filter(|w| !w.is_empty()), when].into_iter().flatten().collect();
if parts.is_empty() { String::new() } else { format!("<p class=\"byline\">{}</p>", parts.join(" · ")) }
}
B::Header(h) => heading("h2", &h.text, cx),
B::Heading1(h) => heading("h2", &h.text, cx),
B::Heading2(h) => heading("h3", &h.text, cx),
B::Heading3(h) => heading("h4", &h.text, cx),
B::Heading4(h) => heading("h5", &h.text, cx),
B::Heading5(h) => heading("h6", &h.text, cx),
B::Heading6(h) => heading("h6", &h.text, cx),
B::Subheader(t) => heading("h3", &t.text, cx),
B::Paragraph(p) => {
let inner = rich(&p.text, cx);
if inner.is_empty() { String::new() } else { format!("<p>{inner}</p>") }
}
B::Preformatted(p) => format!("<pre><code>{}</code></pre>", rich(&p.text, cx)),
B::Footer(f) => format!("<p class=\"footer\">{}</p>", rich(&f.text, cx)),
B::Divider => "<hr>".to_string(),
B::Anchor(a) => format!("<a id=\"{}\"></a>", escape(&a.name)),
B::List(l) => {
let items: String = l
.items
.iter()
.map(|i| match i {
tl::enums::PageListItem::Text(t) => format!("<li>{}</li>", rich(&t.text, cx)),
tl::enums::PageListItem::Blocks(b) => format!("<li>{}</li>", blocks(&b.blocks, cx)),
})
.collect();
format!("<ul>{items}</ul>")
}
B::OrderedList(l) => {
let items: String = l
.items
.iter()
.map(|i| match i {
tl::enums::PageListOrderedItem::Text(t) => format!("<li>{}</li>", rich(&t.text, cx)),
tl::enums::PageListOrderedItem::Blocks(b) => format!("<li>{}</li>", blocks(&b.blocks, cx)),
})
.collect();
let start = l.start.map(|s| format!(" start=\"{s}\"")).unwrap_or_default();
format!("<ol{start}{}>{items}</ol>", if l.reversed { " reversed" } else { "" })
}
B::Blockquote(q) => {
let who = rich(&q.caption, cx);
format!("<blockquote><p>{}</p>{}</blockquote>", rich(&q.text, cx), if who.is_empty() { String::new() } else { format!("<cite>{who}</cite>") })
}
B::BlockquoteBlocks(q) => format!("<blockquote>{}</blockquote>", blocks(&q.blocks, cx)),
B::Pullquote(q) => {
let who = rich(&q.caption, cx);
format!("<blockquote class=\"pull\"><p>{}</p>{}</blockquote>", rich(&q.text, cx), if who.is_empty() { String::new() } else { format!("<cite>{who}</cite>") })
}
B::Photo(p) => {
// A picture the page does not carry still has its words.
let img = match cx.picture(Want::Photo(p.photo_id)) {
Some(src) => {
let img = format!("<img src=\"{src}\" alt=\"\">");
match &p.url {
Some(url) => format!("<a href=\"{}\">{img}</a>", safe_url(url)),
None => img,
}
}
None => String::new(),
};
format!("<figure>{img}{}</figure>", caption(&p.caption, cx))
}
B::Video(v) => {
let poster = cx.picture(Want::Poster(v.video_id)).map(|src| format!("<img src=\"{src}\" alt=\"\">")).unwrap_or_default();
format!("<figure class=\"video\">{poster}<span class=\"note\">Video: open the link above to watch it</span>{}</figure>", caption(&v.caption, cx))
}
B::Cover(c) => format!("<div class=\"cover\">{}</div>", block(&c.cover, cx)),
B::Collage(c) => format!("<figure class=\"collage\"><div>{}</div>{}</figure>", blocks(&c.items, cx), caption(&c.caption, cx)),
B::Slideshow(c) => format!("<figure class=\"collage\"><div>{}</div>{}</figure>", blocks(&c.items, cx), caption(&c.caption, cx)),
B::Embed(e) => {
let poster = e.poster_photo_id.and_then(|id| cx.picture(Want::Photo(id))).map(|src| format!("<img src=\"{src}\" alt=\"\">")).unwrap_or_default();
let link = e.url.as_deref().map(|u| format!("<a class=\"note\" href=\"{}\">Open the embedded content</a>", safe_url(u))).unwrap_or_default();
format!("<figure>{poster}{link}{}</figure>", caption(&e.caption, cx))
}
B::Table(t) => {
let rows: String = t
.rows
.iter()
.map(|row| {
let tl::enums::PageTableRow::Row(row) = row;
let cells: String = row
.cells
.iter()
.map(|cell| {
let tl::enums::PageTableCell::Cell(c) = cell;
let tag = if c.header { "th" } else { "td" };
let span = format!("{}{}", c.colspan.map(|n| format!(" colspan=\"{n}\"")).unwrap_or_default(), c.rowspan.map(|n| format!(" rowspan=\"{n}\"")).unwrap_or_default());
let align = if c.align_center { " class=\"c\"" } else if c.align_right { " class=\"r\"" } else { "" };
format!("<{tag}{span}{align}>{}</{tag}>", c.text.as_ref().map(|t| rich(t, cx)).unwrap_or_default())
})
.collect();
format!("<tr>{cells}</tr>")
})
.collect();
let title = rich(&t.title, cx);
format!("<div class=\"table\"><table>{}{rows}</table></div>", if title.is_empty() { String::new() } else { format!("<caption>{title}</caption>") })
}
B::Details(d) => format!("<details{}><summary>{}</summary>{}</details>", if d.open { " open" } else { "" }, rich(&d.title, cx), blocks(&d.blocks, cx)),
B::Math(m) => format!("<pre><code>{}</code></pre>", escape(&m.source)),
_ => String::new(),
}
}
const STYLE: &str = "
:root{color-scheme:light dark;--bg:#fff;--fg:#1c2733;--muted:#6b7a89;--line:#d8dee4;--link:#2b7bd3;--soft:#f2f5f8}
@media (prefers-color-scheme:dark){:root{--bg:#0e1621;--fg:#e4e9ee;--muted:#8a99a8;--line:#26323f;--link:#6ab3f3;--soft:#17212b}}
html,body{margin:0;background:var(--bg);color:var(--fg)}
body{font:17px/1.65 system-ui,-apple-system,'Segoe UI','Noto Sans',sans-serif}
main{max-width:740px;margin:0 auto;padding:0 20px 80px}
a{color:var(--link);text-decoration:none}a:hover{text-decoration:underline}
h1{font-size:2rem;line-height:1.2;margin:.8em 0 .4em}h2{font-size:1.5rem;margin:1.4em 0 .4em}h3{font-size:1.25rem;margin:1.3em 0 .3em}h4,h5,h6{margin:1.2em 0 .3em}
p{margin:.9em 0}.kicker{color:var(--muted);text-transform:uppercase;letter-spacing:.06em;font-size:.8rem;margin:1.6em 0 0}
.subtitle{color:var(--muted);font-size:1.15rem}.byline,.footer{color:var(--muted);font-size:.9rem}
figure{margin:1.4em -20px}figure img,.cover img{display:block;width:100%;height:auto}figcaption{padding:.5em 20px 0;color:var(--muted);font-size:.9rem}figcaption cite{display:block;font-style:normal;opacity:.8}
.cover{margin:0 -20px}.cover figure{margin:0}.cover figcaption{padding:.7em 20px;text-transform:uppercase;letter-spacing:.04em;font-size:.8rem}.cover figcaption cite{margin:.6em -20px -.7em;padding:.6em 20px;background:var(--soft);color:var(--fg);font-size:1rem;font-weight:600;text-transform:none;letter-spacing:0;opacity:1}.collage div{display:grid;grid-template-columns:repeat(auto-fit,minmax(220px,1fr));gap:4px}.collage figure{margin:0}
.note{display:block;padding:.6em 20px;color:var(--muted);font-size:.9rem}
blockquote{margin:1.2em 0;padding:.1em 1em;border-left:3px solid var(--link);color:var(--fg)}blockquote.pull{font-size:1.25rem;border-left:0;text-align:center;font-style:italic}cite{display:block;color:var(--muted);font-style:normal;font-size:.9rem}
pre{background:var(--soft);padding:12px 14px;overflow:auto;border-radius:6px;font-size:.9rem}code{font-family:ui-monospace,Menlo,Consolas,monospace}mark{background:#ffe08a55;color:inherit}
hr{border:0;border-top:1px solid var(--line);margin:2em 0}ul,ol{padding-left:1.4em}li{margin:.3em 0}
.table{overflow:auto;margin:1.2em 0}table{border-collapse:collapse;width:100%}td,th{border:1px solid var(--line);padding:6px 10px;vertical-align:top}th{background:var(--soft)}.c{text-align:center}.r{text-align:right}caption{caption-side:top;text-align:left;font-weight:600;padding-bottom:.4em}
details{margin:1em 0;border:1px solid var(--line);border-radius:6px;padding:.4em .9em}summary{cursor:pointer;font-weight:600}
";
/// The article a message's link preview carries, with its pictures fetched.
pub async fn article(state: &AppState, buffer: &Buffer, msg_id: &str) -> Result<Article, String> {
let id: i32 = msg_id.parse().map_err(|_| "that message has no article".to_string())?;
let (handle, peer) = peer_for(state, buffer).await?;
let found = handle.client.get_messages_by_id(peer, &[id]).await.map_err(|e| explain(&e))?;
let Some(Some(message)) = found.into_iter().next() else { return Err("that message is no longer there".to_string()) };
let Some(Media::WebPage(media)) = message.media() else { return Err("that message links to no page".to_string()) };
let tl::enums::WebPage::Page(page) = &media.raw.webpage else { return Err("Telegram has no preview of that page".to_string()) };
let url = page.url.clone();
let title = page.title.clone().unwrap_or_default();
let (preview_photo, preview_document) = (page.photo.clone(), page.document.clone());
let mut cached = page.cached_page.clone();
// A message may carry only the start of the page: the whole of it is asked for.
if matches!(&cached, Some(tl::enums::Page::Page(p)) if p.part) {
if let Ok(tl::enums::messages::WebPage::Page(full)) = handle.client.invoke(&tl::functions::messages::GetWebPage { url: url.clone(), hash: 0 }).await {
if let tl::enums::WebPage::Page(whole) = full.webpage {
cached = whole.cached_page.or(cached);
}
}
}
let Some(tl::enums::Page::Page(doc)) = cached else { return Err("Telegram has no Instant View of that page".to_string()) };
let remote = buffer.remote_id.clone().unwrap_or_default();
let mut cx = Cx {
account: handle.own.user_id,
remote,
message: id,
// The page's own picture (the one its preview shows) is often the cover and is not listed again with the rest.
photos: doc.photos.iter().chain(preview_photo.iter()).filter_map(|p| match p { tl::enums::Photo::Photo(x) => Some((x.id, p.clone())), _ => None }).collect(),
documents: doc.documents.iter().chain(preview_document.iter()).filter_map(|d| match d { tl::enums::Document::Document(x) => Some((x.id, d.clone())), _ => None }).collect(),
wanted: Vec::new(),
};
let body = blocks(&doc.blocks, &mut cx);
// The page's own title, from its first heading when the preview gave none.
let title = if title.is_empty() { doc.blocks.iter().find_map(|b| if let B::Title(t) = b { Some(plain(&t.text)).filter(|t| !t.is_empty()) } else { None }).unwrap_or_default() } else { title };
let has_title = doc.blocks.iter().any(|b| matches!(b, B::Title(t) if !is_empty(&t.text)));
let heading = if has_title || title.is_empty() { String::new() } else { format!("<h1>{}</h1>", escape(&title)) };
// The pictures, fetched together; one that will not come leaves a gap where it was.
let mut jobs = tokio::task::JoinSet::new();
for want in cx.wanted.clone() {
let path = cx.path(want);
if path.exists() {
continue;
}
let client = handle.client.clone();
let (photo, document) = match want {
Want::Photo(id) => (cx.photos.get(&id).cloned(), None),
Want::Poster(id) => (None, cx.documents.get(&id).cloned()),
};
jobs.spawn(async move {
let done = if let Some(photo) = photo {
let photo = Photo::from_raw(photo);
match pick(&photo.thumbs(), 1280) {
Some(size) => download_to(&client, size, &path, false).await,
None => download_to(&client, &photo, &path, false).await,
}
} else if let Some(document) = document {
let document = Document::from_raw_media(tl::types::MessageMediaDocument {
nopremium: false,
spoiler: false,
video: true,
round: false,
voice: false,
document: Some(document),
alt_documents: None,
video_cover: None,
video_timestamp: None,
ttl_seconds: None,
});
download_thumb(&client, &document, &path).await
} else {
Ok(())
};
if let Err(e) = done {
tracing::debug!("telegram: a picture of an article: {e:#}");
}
});
}
while jobs.join_next().await.is_some() {}
let rtl = if doc.rtl { " dir=\"rtl\"" } else { "" };
let html = format!(
"<!doctype html><html lang=\"en\"{rtl}><head><meta charset=\"utf-8\"><meta name=\"viewport\" content=\"width=device-width,initial-scale=1\">\
<meta http-equiv=\"Content-Security-Policy\" content=\"default-src 'none'; img-src moho-media: data:; style-src 'unsafe-inline'\">\
<title>{}</title><style>{STYLE}</style></head><body><main>{heading}{body}<p class=\"footer\"><a href=\"{}\">Open the original page</a></p></main></body></html>",
escape(&title),
safe_url(&url)
);
Ok(Article { title, url, html })
}
#[cfg(test)]
mod tests {
use super::*;
fn cx() -> Cx {
Cx { account: 1, remote: "-100123".into(), message: 7, photos: HashMap::new(), documents: HashMap::new(), wanted: Vec::new() }
}
fn plain_text(t: &str) -> R {
R::TextPlain(tl::types::TextPlain { text: t.into() })
}
#[test]
fn text_is_escaped_and_styled() {
let mut c = cx();
let t = R::TextConcat(tl::types::TextConcat {
texts: vec![plain_text("a <b> & "), R::TextBold(Box::new(tl::types::TextBold { text: plain_text("bold") }))],
});
assert_eq!(rich(&t, &mut c), "a &lt;b&gt; &amp; <b>bold</b>");
}
#[test]
fn a_link_may_only_go_where_a_page_would() {
assert_eq!(safe_url("https://example.com/a?b=1&c=2"), "https://example.com/a?b=1&amp;c=2");
assert_eq!(safe_url("javascript:alert(1)"), "#");
assert_eq!(safe_url("data:text/html,x"), "#");
assert_eq!(safe_url("#section"), "#section");
let mut c = cx();
let link = R::TextUrl(Box::new(tl::types::TextUrl { text: plain_text("x"), url: "javascript:alert(1)".into(), webpage_id: 0 }));
assert_eq!(rich(&link, &mut c), "<a href=\"#\">x</a>");
}
#[test]
fn a_picture_the_page_lacks_keeps_its_words_and_one_it_has_is_named_by_its_file() {
let mut c = cx();
let photo = B::Photo(tl::types::PageBlockPhoto {
spoiler: false,
photo_id: 9,
caption: tl::enums::PageCaption::Caption(tl::types::PageCaption { text: plain_text("a caption"), credit: R::TextEmpty }),
url: None,
webpage_id: None,
});
let without = block(&photo, &mut c);
assert!(!without.contains("<img") && without.contains("a caption"), "{without}");
c.photos.insert(9, tl::enums::Photo::Empty(tl::types::PhotoEmpty { id: 9 }));
let html = block(&photo, &mut c);
assert!(html.contains("moho-media://file/?p=") && html.contains("_iv9.jpg") && html.contains("a caption"), "{html}");
assert_eq!(c.wanted, vec![Want::Photo(9)]);
}
}
src/backend/telegram/link.rs
+49
View File
@@ -0,0 +1,49 @@
//! One connection to Telegram: the library's sender pool, its client and the stream of what Telegram sends.
//!
//! Everything that talks to Telegram - a sign-in, a connected account - opens one of these and closes it when it
//! is done. The pool connects to each datacenter on demand, and a closed one drops its connections gracefully.
use super::session::FileSession;
use crate::state::AppState;
use anyhow::{Context, Result};
use grammers_client::Client;
use grammers_mtsender::{SenderPool, SenderPoolFatHandle};
use grammers_session::updates::UpdatesLike;
use tokio::sync::mpsc::Receiver;
use std::sync::Arc;
pub struct Link {
pub client: Client,
pub session: Arc<FileSession>,
/// What Telegram sends unasked, to be handed to `Client::stream_updates` once (taken by whoever reads it).
pub updates: Option<Receiver<UpdatesLike>>,
handle: SenderPoolFatHandle,
runner: tokio::task::JoinHandle<()>,
}
/// Opens a connection under `key` (the account's id, or a sign-in's pending key), by the account's route if it has
/// one: Tor or the proxy, as a SOCKS5 address the library connects through.
pub async fn open(state: &AppState, key: &str, api_id: i32, session: Arc<FileSession>) -> Result<Link> {
let router = crate::net::route::router();
let proxy = if router.routed(key) {
let (host, port) = router.ready(|msg| state.runtime.report_progress(state, key, msg)).await.context("starting Tor or reaching the proxy")?;
Some(format!("socks5://{host}:{port}"))
} else {
None
};
let SenderPool { runner, handle, updates } = SenderPool::with_configuration(session.clone(), api_id, super::connection_params(proxy.as_deref()));
let client = Client::new(handle.clone());
let runner = tokio::spawn(runner.run());
Ok(Link { client, session, updates: Some(updates), handle, runner })
}
impl Link {
/// Closes the connections, and writes the session if anything changed.
pub async fn close(self) {
self.handle.quit();
let _ = self.runner.await;
if let Err(e) = self.session.save_if_dirty() {
tracing::warn!("telegram: saving the session: {e:#}");
}
}
}
src/backend/telegram/login.rs
+292
View File
@@ -0,0 +1,292 @@
//! Adding an account: the sign-in a person takes part in.
//!
//! The person types an `api_id`, an `api_hash` and a phone number. Telegram sends a code to the account's other
//! sessions (or by SMS), the person types it, and if the account has a cloud password that is asked for next.
//! Neither the code nor the password is kept: what the account keeps is the session Telegram issued.
//!
//! Like Steam's, it is a conversation that can take minutes, so it runs as its own task and talks to the window
//! through events:
//!
//! - `telegramLoginStatus` - what it is doing, for a line of text
//! - `telegramLoginNeeds` - what Telegram is asking for: `step` is `code` or `password` (with the password's hint),
//! and `error` says why the last answer was refused
//! - `telegramLoginResult` - done, with the account, or the reason it failed
//!
//! and the window answers with `submitTelegramCode` and `submitTelegramPassword`.
use super::link::{self, Link};
use super::session::{self, FileSession};
use crate::accounts::TelegramAccountConfig;
use crate::state::AppState;
use anyhow::{anyhow, bail, Context, Result};
use grammers_client::SignInError;
use serde_json::json;
use std::collections::HashMap;
use std::sync::{Arc, Mutex};
use std::time::Duration;
use tokio::sync::mpsc;
/// How long a sign-in may wait for a person. Telegram's own code is good for a few minutes, and a form left open
/// overnight should not leave a task.
const PATIENCE: Duration = Duration::from_secs(5 * 60);
/// What the window can hand to a sign-in that is waiting.
enum Answer {
Code(String),
Password(String),
}
/// The sign-ins waiting for an answer, by id, and the way to give each one.
fn waiting() -> &'static Mutex<HashMap<String, mpsc::UnboundedSender<Answer>>> {
static WAITING: std::sync::OnceLock<Mutex<HashMap<String, mpsc::UnboundedSender<Answer>>>> = std::sync::OnceLock::new();
WAITING.get_or_init(Default::default)
}
fn give(login_id: &str, answer: Answer) -> Result<()> {
let sender = waiting().lock().unwrap().get(login_id).cloned().ok_or_else(|| anyhow!("that sign-in is not waiting for anything any more"))?;
sender.send(answer).map_err(|_| anyhow!("that sign-in has ended"))
}
/// Gives a sign-in the code Telegram sent.
pub fn submit_code(login_id: &str, code: &str) -> Result<()> {
// Telegram's own client shows the code as "12345"; a pasted one may carry spaces or a dash.
let code: String = code.chars().filter(|c| c.is_ascii_digit()).collect();
if code.is_empty() {
bail!("type the code first");
}
give(login_id, Answer::Code(code))
}
/// Gives a sign-in the account's cloud password.
pub fn submit_password(login_id: &str, password: &str) -> Result<()> {
if password.is_empty() {
bail!("type the password first");
}
give(login_id, Answer::Password(password.to_string()))
}
/// Stops a sign-in that is waiting, because the form was closed.
pub fn cancel(login_id: &str) {
waiting().lock().unwrap().remove(login_id);
}
/// A phone number as Telegram wants it: digits with a `+` in front.
pub fn normalize_phone(typed: &str) -> Option<String> {
let digits: String = typed.chars().filter(|c| c.is_ascii_digit()).collect();
// The shortest numbers in use are about seven digits and the longest fifteen.
(7..=15).contains(&digits.len()).then(|| format!("+{digits}"))
}
/// Starts a sign-in. Answers at once; the rest is events.
pub fn start(state: AppState, login_id: String, api_id: i32, api_hash: String, phone: String) {
let (tx, rx) = mpsc::unbounded_channel();
waiting().lock().unwrap().insert(login_id.clone(), tx);
tokio::spawn(async move {
let outcome = flow(&state, &login_id, api_id, &api_hash, &phone, rx).await;
waiting().lock().unwrap().remove(&login_id);
match outcome {
Ok(Some(saved)) => {
let account = crate::accounts::telegram_account_to_json(&saved, "connecting");
super::connect::spawn(state.clone(), saved);
state.events.emit("telegramLoginResult", json!({ "loginId": login_id, "success": true, "account": account }));
}
// The form was closed; nobody is waiting for an answer.
Ok(None) => {}
Err(e) => {
let error = format!("{e:#}");
tracing::info!("telegram: sign-in {login_id}: {error}");
state.events.emit("telegramLoginResult", json!({ "loginId": login_id, "success": false, "error": error }));
}
}
});
}
fn status(state: &AppState, login_id: &str, text: &str) {
state.events.emit("telegramLoginStatus", json!({ "loginId": login_id, "text": text }));
}
fn needs(state: &AppState, login_id: &str, step: &str, hint: Option<&str>, error: Option<&str>) {
state.events.emit("telegramLoginNeeds", json!({ "loginId": login_id, "step": step, "hint": hint, "error": error }));
}
/// What was typed, or `None` when the form was closed or nobody answered in time.
async fn answer(rx: &mut mpsc::UnboundedReceiver<Answer>) -> Option<Answer> {
tokio::time::timeout(PATIENCE, rx.recv()).await.ok().flatten()
}
async fn flow(state: &AppState, login_id: &str, api_id: i32, api_hash: &str, phone: &str, mut rx: mpsc::UnboundedReceiver<Answer>) -> Result<Option<TelegramAccountConfig>> {
if api_id <= 0 || api_hash.trim().is_empty() {
bail!("an api_id and an api_hash are needed: get them at my.telegram.org/apps");
}
let phone = normalize_phone(phone).ok_or_else(|| anyhow!("that does not look like a phone number; type it with its country code"))?;
status(state, login_id, "Connecting to Telegram");
let session = Arc::new(FileSession::fresh(session::path_for_login(login_id)));
let link = link::open(state, &crate::net::route::pending_key("telegram"), api_id, session.clone()).await?;
let outcome = signed_in(state, login_id, &link, api_hash, &phone, &mut rx).await;
let user = match outcome {
Ok(Some(user)) => user,
other => {
link.close().await;
session.delete();
return other.map(|_| None);
}
};
let config = TelegramAccountConfig {
api_id,
api_hash: api_hash.trim().to_string(),
phone,
user_id: user.id().bare_id_unchecked(),
username: user.username().map(str::to_string),
name: user.full_name(),
use_tor: crate::net::route::router().wanted(&crate::net::route::pending_key("telegram")),
..Default::default()
};
// The session is this account's now: under its own name, and written before the connection that made it closes.
session.move_to(session::path_for(config.user_id)).context("saving the session")?;
link.close().await;
let saved = state.accounts.add_telegram(config)?;
Ok(Some(saved))
}
/// The conversation with Telegram, up to being signed in.
async fn signed_in(state: &AppState, login_id: &str, link: &Link, api_hash: &str, phone: &str, rx: &mut mpsc::UnboundedReceiver<Answer>) -> Result<Option<grammers_client::peer::User>> {
let client = &link.client;
if client.is_authorized().await.context("asking Telegram whether this session is signed in")? {
bail!("this session is already signed in");
}
status(state, login_id, "Asking Telegram to send a code");
let token = client.request_login_code(phone, api_hash).await.map_err(|e| anyhow!("{}", explain(&e.to_string())))?;
needs(state, login_id, "code", None, None);
loop {
let Some(given) = answer(rx).await else { return Ok(None) };
match given {
Answer::Code(code) => match client.sign_in(&token, &code).await {
Ok(user) => return Ok(Some(user)),
Err(SignInError::PasswordRequired(password)) => {
let hint = password.hint().map(str::to_string);
needs(state, login_id, "password", hint.as_deref(), None);
return ask_password(state, login_id, client, password, rx).await;
}
Err(SignInError::InvalidCode) => needs(state, login_id, "code", None, Some("That code is not right. Check it and try again")),
Err(SignInError::SignUpRequired) => bail!("this number has no Telegram account. Make one with Telegram's own app first"),
Err(e) => bail!("{}", explain(&e.to_string())),
},
Answer::Password(_) => needs(state, login_id, "code", None, Some("Telegram has not asked for a password")),
}
}
}
async fn ask_password(state: &AppState, login_id: &str, client: &grammers_client::Client, mut token: grammers_client::client::PasswordToken, rx: &mut mpsc::UnboundedReceiver<Answer>) -> Result<Option<grammers_client::peer::User>> {
loop {
let Some(given) = answer(rx).await else { return Ok(None) };
let Answer::Password(password) = given else { continue };
match client.check_password(token, password.as_bytes()).await {
Ok(user) => return Ok(Some(user)),
Err(SignInError::InvalidPassword(again)) => {
needs(state, login_id, "password", again.hint(), Some("That password is not right"));
token = again;
}
Err(e) => bail!("{}", explain(&e.to_string())),
}
}
}
/// Telegram's own wording for what went wrong, said so a person can act on it.
pub fn explain(raw: &str) -> String {
let upper = raw.to_uppercase();
for (code, words) in [
("API_ID_INVALID", "Telegram does not accept that api_id and api_hash pair. Check them at my.telegram.org/apps"),
("PHONE_NUMBER_INVALID", "Telegram says that is not a valid phone number"),
("PHONE_NUMBER_BANNED", "Telegram has banned that phone number"),
("PHONE_CODE_EXPIRED", "That code has expired. Start again to get a new one"),
("PHONE_CODE_INVALID", "That code is not right"),
("PASSWORD_HASH_INVALID", "That password is not right"),
("FLOOD_WAIT", "Telegram says too many attempts were made. Wait before trying again"),
("SESSION_PASSWORD_NEEDED", "This account has a cloud password"),
("AUTH_KEY_UNREGISTERED", "Telegram has ended this session"),
("USER_DEACTIVATED", "Telegram says this account has been deleted"),
] {
if upper.contains(code) {
return words.to_string();
}
}
raw.to_string()
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_phone_number_is_made_the_way_telegram_wants_it() {
assert_eq!(normalize_phone("+1 (555) 123-4567").as_deref(), Some("+15551234567"));
assert_eq!(normalize_phone("44 20 7946 0958").as_deref(), Some("+442079460958"));
assert_eq!(normalize_phone("12345"), None);
assert_eq!(normalize_phone("not a number"), None);
}
#[test]
fn answers_for_a_sign_in_that_is_not_waiting_are_refused_in_words() {
assert!(submit_code("telegram-none", "12345").unwrap_err().to_string().contains("not waiting"));
assert!(submit_code("telegram-none", "abc").unwrap_err().to_string().contains("type the code"));
let (tx, mut rx) = mpsc::unbounded_channel();
waiting().lock().unwrap().insert("telegram-t".into(), tx);
submit_code("telegram-t", "1 2-3 4 5").unwrap();
submit_password("telegram-t", "hunter2").unwrap();
assert!(matches!(rx.try_recv(), Ok(Answer::Code(c)) if c == "12345"), "spaces and dashes in a pasted code are dropped");
assert!(matches!(rx.try_recv(), Ok(Answer::Password(p)) if p == "hunter2"));
cancel("telegram-t");
assert!(submit_code("telegram-t", "12345").is_err());
}
#[test]
fn telegrams_refusals_are_said_so_a_person_can_act_on_them() {
assert!(explain("rpc error 400: API_ID_INVALID").contains("my.telegram.org"));
assert!(explain("PHONE_CODE_EXPIRED").contains("expired"));
assert_eq!(explain("something unforeseen"), "something unforeseen");
}
/// Against Telegram's own test servers, which have no real accounts: a number `99966XYYYY` signs in with the code
/// `XXXXX` (its digit X, five times). Needs the person's `api_id` and `api_hash` (the test servers take a real pair):
///
/// `TELEGRAM_LIVE_API_ID=... TELEGRAM_LIVE_API_HASH=... TELEGRAM_LIVE_PHONE=+99966XYYYY cargo test --release live_ -- --ignored --nocapture`
#[tokio::test]
#[ignore]
async fn live_a_test_number_signs_in_and_out_on_telegrams_test_servers() {
use grammers_session::types::DcOption;
use grammers_session::Session;
let _ = rustls::crypto::ring::default_provider().install_default();
let api_id: i32 = std::env::var("TELEGRAM_LIVE_API_ID").unwrap().parse().unwrap();
let api_hash = std::env::var("TELEGRAM_LIVE_API_HASH").unwrap();
let phone = normalize_phone(&std::env::var("TELEGRAM_LIVE_PHONE").unwrap()).unwrap();
assert!(phone.starts_with("+99966"), "only a test number, which has no real account behind it");
let code: String = std::iter::repeat_n(phone.chars().nth(6).unwrap(), 5).collect();
let state = crate::backend::kick::testkit::simulated_daemon("telegram-live");
let path = std::env::temp_dir().join(format!("nobilis-telegram-live-{}.session", std::process::id()));
let session = Arc::new(FileSession::fresh(path.clone()));
// Telegram's test datacenter 2 stands where the real one does.
let mut dc: DcOption = session.dc_option(2).unwrap().unwrap();
dc.ipv4 = "149.154.167.40:443".parse().unwrap();
dc.ipv6 = "[::ffff:149.154.167.40]:443".parse().unwrap();
session.set_dc_option(&dc).await.unwrap();
session.set_home_dc_id(2).await.unwrap();
let link = link::open(&state, "telegram:live-test", api_id, session.clone()).await.unwrap();
assert!(!link.client.is_authorized().await.unwrap());
let token = link.client.request_login_code(&phone, &api_hash).await.unwrap();
let user = match link.client.sign_in(&token, &code).await {
Ok(user) => user,
Err(e) => panic!("{}", explain(&e.to_string())),
};
println!("signed in as {} ({})", user.full_name(), user.id().bare_id_unchecked());
assert!(link.client.is_authorized().await.unwrap());
assert!(session.has_key());
session.save().unwrap();
let reopened = FileSession::open(path.clone()).unwrap();
assert!(reopened.has_key(), "the saved session still has its key");
link.client.sign_out().await.unwrap();
link.close().await;
std::fs::remove_file(path).ok();
}
}
src/backend/telegram/markup.rs
+284
View File
@@ -0,0 +1,284 @@
//! What the message box holds, as Telegram reads it: plain text and a list of entities.
//!
//! The window writes formatting as Markdown, the way Discord reads it (`**bold**`, `*italic*`, `__underline__`,
//! `~~strike~~`, `||spoiler||`, `` `code` ``, fenced code, `> quote`, `[text](url)`), and Telegram wants the text with
//! the marks taken out and a list saying where each kind starts and how long it runs, counted in UTF-16 code units.
//! A mark with no partner is the character it looks like.
use grammers_client::tl;
#[derive(Debug, PartialEq, Clone, Copy)]
enum Kind {
Bold,
Italic,
Underline,
Strike,
Spoiler,
Code,
Pre,
Quote,
}
#[derive(Debug, PartialEq)]
struct Span {
kind: Kind,
offset: usize,
length: usize,
/// For a link, where it goes.
url: Option<String>,
/// For a fenced block, its language.
language: Option<String>,
}
#[derive(Default)]
struct Out {
text: String,
units: usize,
spans: Vec<Span>,
}
impl Out {
fn push(&mut self, c: char) {
self.text.push(c);
self.units += c.len_utf16();
}
fn push_str(&mut self, s: &str) {
for c in s.chars() {
self.push(c);
}
}
}
/// The text with its marks taken out, and the entities that say what they marked.
pub fn parse(source: &str) -> (String, Vec<tl::enums::MessageEntity>) {
let mut out = Out::default();
let lines: Vec<&str> = source.split('\n').collect();
let mut i = 0;
let mut first = true;
while i < lines.len() {
if !first {
out.push('\n');
}
first = false;
let line = lines[i];
if let Some(fence) = line.strip_prefix("```") {
// A fenced block runs to the next fence, or to the end.
if let Some(end) = lines[i + 1..].iter().position(|l| l.trim_end() == "```") {
let language = fence.trim().to_string();
let body = lines[i + 1..i + 1 + end].join("\n");
let offset = out.units;
out.push_str(&body);
out.spans.push(Span { kind: Kind::Pre, offset, length: out.units - offset, url: None, language: (!language.is_empty()).then_some(language) });
i += end + 2;
continue;
}
}
if line.starts_with("> ") || line == ">" {
// Consecutive quoted lines are one quote.
let start = out.units;
let mut quoted = vec![line.strip_prefix("> ").unwrap_or("")];
while i + 1 < lines.len() && (lines[i + 1].starts_with("> ") || lines[i + 1] == ">") {
i += 1;
quoted.push(lines[i].strip_prefix("> ").unwrap_or(""));
}
for (n, q) in quoted.iter().enumerate() {
if n > 0 {
out.push('\n');
}
inline(&q.chars().collect::<Vec<_>>(), &mut out);
}
out.spans.push(Span { kind: Kind::Quote, offset: start, length: out.units - start, url: None, language: None });
i += 1;
continue;
}
inline(&line.chars().collect::<Vec<_>>(), &mut out);
i += 1;
}
let entities = out.spans.iter().filter(|s| s.length > 0).map(entity).collect();
(out.text, entities)
}
fn entity(s: &Span) -> tl::enums::MessageEntity {
use tl::enums::MessageEntity as E;
use tl::types as t;
let (offset, length) = (s.offset as i32, s.length as i32);
if let Some(url) = &s.url {
return E::TextUrl(t::MessageEntityTextUrl { offset, length, url: url.clone() });
}
match s.kind {
Kind::Bold => E::Bold(t::MessageEntityBold { offset, length }),
Kind::Italic => E::Italic(t::MessageEntityItalic { offset, length }),
Kind::Underline => E::Underline(t::MessageEntityUnderline { offset, length }),
Kind::Strike => E::Strike(t::MessageEntityStrike { offset, length }),
Kind::Spoiler => E::Spoiler(t::MessageEntitySpoiler { offset, length }),
Kind::Code => E::Code(t::MessageEntityCode { offset, length }),
Kind::Pre => E::Pre(t::MessageEntityPre { offset, length, language: s.language.clone().unwrap_or_default() }),
Kind::Quote => E::Blockquote(t::MessageEntityBlockquote { collapsed: false, offset, length }),
}
}
fn find(chars: &[char], from: usize, marker: &[char]) -> Option<usize> {
(from..chars.len().saturating_sub(marker.len() - 1)).find(|&j| chars[j..].starts_with(marker))
}
/// A single `*` or `_` only opens at the start of a word and closes at the end of one, so `snake_case_name` and `2*3*4`
/// stay as written.
fn word_edge(chars: &[char], at: usize, opening: bool) -> bool {
let alnum = |c: char| c.is_alphanumeric();
if opening {
(at == 0 || !alnum(chars[at - 1])) && chars.get(at + 1).is_some_and(|c| !c.is_whitespace())
} else {
at > 0 && !chars[at - 1].is_whitespace() && chars.get(at + 1).is_none_or(|c| !alnum(*c))
}
}
fn inline(chars: &[char], out: &mut Out) {
let mut i = 0;
while i < chars.len() {
let c = chars[i];
if c == '\\' && i + 1 < chars.len() && "\\*_~|`[]()>".contains(chars[i + 1]) {
out.push(chars[i + 1]);
i += 2;
continue;
}
// Code takes its contents as they are.
if c == '`' {
if let Some(end) = find(chars, i + 1, &['`']).filter(|&e| e > i + 1) {
let offset = out.units;
out.push_str(&chars[i + 1..end].iter().collect::<String>());
out.spans.push(Span { kind: Kind::Code, offset, length: out.units - offset, url: None, language: None });
i = end + 1;
continue;
}
}
// [text](url)
if c == '[' {
if let Some(close) = find(chars, i + 1, &[']']) {
if chars.get(close + 1) == Some(&'(') {
if let Some(end) = find(chars, close + 2, &[')']) {
let url: String = chars[close + 2..end].iter().collect();
if ["http://", "https://", "tg://", "mailto:"].iter().any(|p| url.starts_with(p)) && close > i + 1 {
let offset = out.units;
inline(&chars[i + 1..close], out);
out.spans.push(Span { kind: Kind::Bold, offset, length: out.units - offset, url: Some(url), language: None });
i = end + 1;
continue;
}
}
}
}
}
let two = |m: &str| m.chars().collect::<Vec<_>>();
let pairs: [(&str, Kind); 4] = [("**", Kind::Bold), ("__", Kind::Underline), ("~~", Kind::Strike), ("||", Kind::Spoiler)];
let mut done = false;
for (marker, kind) in pairs {
let m = two(marker);
if chars[i..].starts_with(&m) {
if let Some(end) = find(chars, i + 2, &m).filter(|&e| e > i + 2) {
let offset = out.units;
inline(&chars[i + 2..end], out);
out.spans.push(Span { kind, offset, length: out.units - offset, url: None, language: None });
i = end + 2;
done = true;
break;
}
}
}
if done {
continue;
}
if (c == '*' || c == '_') && word_edge(chars, i, true) {
// The closing one: the next of the same mark that ends a word.
let mut j = i + 1;
let mut found = None;
while let Some(e) = find(chars, j, &[c]) {
if e > i + 1 && word_edge(chars, e, false) {
found = Some(e);
break;
}
j = e + 1;
}
if let Some(end) = found {
let offset = out.units;
inline(&chars[i + 1..end], out);
out.spans.push(Span { kind: Kind::Italic, offset, length: out.units - offset, url: None, language: None });
i = end + 1;
continue;
}
}
out.push(c);
i += 1;
}
}
#[cfg(test)]
mod tests {
use super::*;
use tl::enums::MessageEntity as E;
fn kinds(entities: &[E]) -> Vec<(&'static str, i32, i32)> {
entities
.iter()
.map(|e| match e {
E::Bold(x) => ("bold", x.offset, x.length),
E::Italic(x) => ("italic", x.offset, x.length),
E::Underline(x) => ("underline", x.offset, x.length),
E::Strike(x) => ("strike", x.offset, x.length),
E::Spoiler(x) => ("spoiler", x.offset, x.length),
E::Code(x) => ("code", x.offset, x.length),
E::Pre(x) => ("pre", x.offset, x.length),
E::Blockquote(x) => ("quote", x.offset, x.length),
E::TextUrl(x) => ("link", x.offset, x.length),
_ => ("other", 0, 0),
})
.collect()
}
#[test]
fn marks_are_taken_out_and_the_entities_say_where_they_were() {
let (text, entities) = parse("a **bold** and *italic* and ~~gone~~ and ||secret|| and __under__");
assert_eq!(text, "a bold and italic and gone and secret and under");
let got = kinds(&entities);
assert!(got.contains(&("bold", 2, 4)) && got.contains(&("italic", 11, 6)) && got.contains(&("strike", 22, 4)) && got.contains(&("spoiler", 31, 6)) && got.contains(&("underline", 42, 5)), "{got:?}");
}
#[test]
fn marks_nest_and_a_mark_with_no_partner_is_just_the_character() {
let (text, entities) = parse("**a *b* c**");
assert_eq!(text, "a b c");
let got = kinds(&entities);
assert!(got.contains(&("bold", 0, 5)) && got.contains(&("italic", 2, 1)), "{got:?}");
let (text, entities) = parse("2 * 3 and snake_case_name and **unclosed");
assert_eq!(text, "2 * 3 and snake_case_name and **unclosed");
assert!(entities.is_empty());
}
#[test]
fn code_is_taken_as_written_and_a_fence_is_a_block_with_its_language() {
let (text, entities) = parse("see `**not bold**` ok");
assert_eq!(text, "see **not bold** ok");
assert_eq!(kinds(&entities), vec![("code", 4, 12)]);
let (text, entities) = parse("```rust\nlet x = 1;\n```\nafter");
assert_eq!(text, "let x = 1;\nafter");
assert!(matches!(&entities[0], E::Pre(p) if p.language == "rust" && p.offset == 0 && p.length == 10));
}
#[test]
fn a_link_and_a_quote_and_an_escape() {
let (text, entities) = parse("[here](https://example.com/a?b=1) \\*not\\*");
assert_eq!(text, "here *not*");
assert!(matches!(&entities[0], E::TextUrl(u) if u.url == "https://example.com/a?b=1" && u.offset == 0 && u.length == 4));
let (text, entities) = parse("> one\n> two\nthree");
assert_eq!(text, "one\ntwo\nthree");
assert_eq!(kinds(&entities), vec![("quote", 0, 7)]);
// A link to somewhere that is not a web address is left as it was typed.
assert_eq!(parse("[x](javascript:alert(1))").0, "[x](javascript:alert(1))");
}
#[test]
fn offsets_count_utf16_units_so_an_emoji_before_a_mark_moves_it_by_two() {
let (text, entities) = parse("😀 **hi**");
assert_eq!(text, "😀 hi");
assert_eq!(kinds(&entities), vec![("bold", 3, 2)]);
}
}
src/backend/telegram/media.rs
+482
View File
@@ -0,0 +1,482 @@
//! Pictures, video, files, voice and stickers: what a message carries, and fetching it.
//!
//! Nothing is fetched until it is looked at. A message is recorded with attachments that name files in the daemon's
//! caches (`telegram-media`, `telegram-avatars`) - files that need not exist yet - and when the window asks for one
//! that is not there, the "restore a cached picture" route it already has for every other service asks the daemon,
//! which finds the file's message again (a file reference expires, so the one remembered would not do), downloads it
//! and answers. So opening a channel of fifty posts fetches the pictures on screen, not fifty videos.
//!
//! A cached file is named for what it is: `tg_<account>_<conversation>_<message>_<thumb|full>.<ext>` for a message's
//! media, `tg_<account>_<person>_<photo>.jpg` for a profile picture.
use super::peers;
use super::Handle;
use crate::model::{Attachment, Embed};
use crate::state::AppState;
use anyhow::{anyhow, bail, Context, Result};
use grammers_client::media::{Document, Downloadable, Media, PhotoSize};
use grammers_client::message::Message;
use grammers_client::tl;
use grammers_session::types::PeerId;
use grammers_session::Session;
use std::path::{Path, PathBuf};
pub const MEDIA: &str = "telegram-media";
pub const AVATARS: &str = "telegram-avatars";
/// How many downloads run at once: enough for a screenful of pictures, few enough not to look like a scraper.
const AT_ONCE: usize = 4;
/// How many whole files (videos, documents) are fetched at once. They take minutes, and pictures must not wait behind
/// them: those have places of their own.
const BIG_AT_ONCE: usize = 2;
fn permits(big: bool) -> &'static tokio::sync::Semaphore {
static P: std::sync::OnceLock<tokio::sync::Semaphore> = std::sync::OnceLock::new();
static BIG: std::sync::OnceLock<tokio::sync::Semaphore> = std::sync::OnceLock::new();
if big {
BIG.get_or_init(|| tokio::sync::Semaphore::new(BIG_AT_ONCE))
} else {
P.get_or_init(|| tokio::sync::Semaphore::new(AT_ONCE))
}
}
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
pub enum Variant {
/// A small picture to draw in the list.
Thumb,
/// The picture at the size the viewer shows, or the file itself.
Full,
}
impl Variant {
fn word(self) -> &'static str {
match self {
Variant::Thumb => "thumb",
Variant::Full => "full",
}
}
}
/// A conversation's id as a piece of a file name: a group's `-` would be taken for a separator.
fn slug(remote: &str) -> String {
remote.replace('-', "n")
}
fn unslug(text: &str) -> String {
match text.strip_prefix('n') {
Some(rest) => format!("-{rest}"),
None => text.to_string(),
}
}
pub fn media_name(account: i64, remote: &str, message: i32, variant: Variant, ext: &str) -> String {
format!("tg_{account}_{}_{message}_{}.{ext}", slug(remote), variant.word())
}
pub fn avatar_name(account: i64, remote: &str, photo: i64) -> String {
format!("tg_{account}_{}_{photo}.jpg", slug(remote))
}
/// What a cached media file's name says: the account, the conversation, the message and which version of it.
pub fn parse_media_name(name: &str) -> Option<(i64, String, i32, Variant)> {
let stem = name.rsplit_once('.')?.0;
let mut parts = stem.strip_prefix("tg_")?.split('_');
let account = parts.next()?.parse().ok()?;
let remote = unslug(parts.next()?);
let message = parts.next()?.parse().ok()?;
let variant = match parts.next()? {
"thumb" => Variant::Thumb,
"full" => Variant::Full,
_ => return None,
};
parts.next().is_none().then_some((account, remote, message, variant))
}
/// What a cached avatar's name says: the account, the person or chat, and which picture of theirs.
pub fn parse_avatar_name(name: &str) -> Option<(i64, String, i64)> {
let stem = name.strip_suffix(".jpg")?;
let mut parts = stem.strip_prefix("tg_")?.split('_');
let account = parts.next()?.parse().ok()?;
let remote = unslug(parts.next()?);
let photo = parts.next()?.parse().ok()?;
parts.next().is_none().then_some((account, remote, photo))
}
fn local(dir: &str, name: &str) -> String {
format!("file://{}", crate::media_cache::dir(dir).join(name).display())
}
/// The file's extension, from its name or else its type.
pub fn extension_of(name: Option<&str>, mime: Option<&str>) -> String {
if let Some(ext) = name.and_then(|n| n.rsplit_once('.')).map(|(_, e)| e.to_ascii_lowercase()).filter(|e| !e.is_empty() && e.len() <= 8 && e.chars().all(|c| c.is_ascii_alphanumeric())) {
return ext;
}
match mime.unwrap_or("") {
"image/jpeg" => "jpg",
"image/png" => "png",
"image/webp" => "webp",
"image/gif" => "gif",
"video/mp4" => "mp4",
"video/webm" => "webm",
"video/quicktime" => "mov",
"audio/ogg" | "audio/opus" => "ogg",
"audio/mpeg" => "mp3",
"audio/mp4" | "audio/m4a" => "m4a",
"application/pdf" => "pdf",
"application/zip" => "zip",
"application/x-tgsticker" => "tgs",
"text/plain" => "txt",
_ => "bin",
}
.to_string()
}
/// The broad shape of a document, which decides how the window draws it.
fn kind_of_mime(mime: &str) -> &'static str {
if mime.starts_with("image/") && mime != "image/gif" {
"image"
} else if mime.starts_with("video/") || mime == "image/gif" {
"video"
} else if mime.starts_with("audio/") {
"audio"
} else {
"file"
}
}
fn dimensions(size: &PhotoSize) -> Option<(i32, i32)> {
match size {
PhotoSize::Size(s) => Some((s.width, s.height)),
PhotoSize::Cached(s) => Some((s.width, s.height)),
PhotoSize::Progressive(s) => Some((s.width, s.height)),
_ => None,
}
}
/// The version of a photo to draw in the list (about 320 wide) and the one to show in the viewer (about 1280).
pub(super) fn pick(sizes: &[PhotoSize], target: i32) -> Option<&PhotoSize> {
let sized: Vec<(&PhotoSize, i32)> = sizes.iter().filter_map(|s| dimensions(s).map(|(w, h)| (s, w.max(h)))).collect();
choose(&sized, target).copied()
}
/// The smallest that reaches the target, else the largest there is.
fn choose<T>(items: &[(T, i32)], target: i32) -> Option<&T> {
let mut by_size: Vec<&(T, i32)> = items.iter().collect();
by_size.sort_by_key(|(_, longest)| *longest);
by_size.iter().find(|(_, longest)| *longest >= target).or(by_size.last()).map(|(item, _)| item)
}
/// The width a link card's picture is fetched at: it is drawn larger than a list thumbnail.
const CARD: i32 = 800;
/// The picture of the web page a message links to, if the page has one.
fn page_photo(media: &Media) -> Option<grammers_client::media::Photo> {
let Media::WebPage(page) = media else { return None };
let tl::enums::WebPage::Page(page) = &page.raw.webpage else { return None };
page.photo.clone().map(grammers_client::media::Photo::from_raw)
}
/// The link preview a message carries (what Telegram calls a web page, with or without an Instant View): a card with
/// the site, title, description and picture. Its picture is named in the cache and fetched like any preview.
pub fn embeds_of(account: i64, remote: &str, message: &Message) -> Vec<Embed> {
let Some(media) = message.media() else { return Vec::new() };
let Media::WebPage(page) = &media else { return Vec::new() };
let tl::enums::WebPage::Page(page) = &page.raw.webpage else { return Vec::new() };
let has_picture = page_photo(&media).is_some();
if page.title.is_none() && page.description.is_none() && !has_picture {
return Vec::new();
}
let non_empty = |t: &Option<String>| t.clone().filter(|t| !t.trim().is_empty());
vec![Embed {
title: non_empty(&page.title),
description: non_empty(&page.description),
url: Some(page.url.clone()),
provider: non_empty(&page.site_name),
author: non_empty(&page.author),
image_url: has_picture.then(|| local(MEDIA, &media_name(account, remote, message.id(), Variant::Thumb, "jpg"))),
// A page Telegram has made an article of opens as one: the card says so, as Telegram's own does.
kind: page.cached_page.is_some().then(|| "instant".to_string()),
cta: page.cached_page.is_some().then(|| crate::model::EmbedLink { label: "Instant View".to_string(), url: None }),
..Default::default()
}]
}
/// What the message carries, as attachments naming files in the cache. Nothing is downloaded here.
pub fn attachments_of(account: i64, remote: &str, message: &Message) -> Vec<Attachment> {
let id = message.id();
let Some(media) = message.media() else { return Vec::new() };
let thumb = |ext: &str| Some(local(MEDIA, &media_name(account, remote, id, Variant::Thumb, ext)));
let full = |ext: &str| Some(local(MEDIA, &media_name(account, remote, id, Variant::Full, ext)));
match media {
Media::Photo(photo) => {
let sizes = photo.thumbs();
let view = pick(&sizes, 1280).and_then(dimensions);
vec![Attachment {
kind: "image".into(),
mimetype: Some("image/jpeg".into()),
filename: Some(if photo.is_spoiler() { "SPOILER_photo.jpg".into() } else { "photo.jpg".into() }),
size: photo.size().map(|s| s as u64),
width: view.map(|(w, _)| w as u32),
height: view.map(|(_, h)| h as u32),
path: full("jpg"),
thumbnail_path: thumb("jpg"),
..Default::default()
}]
}
Media::Sticker(sticker) => {
let document = sticker.document.clone();
let mime = document.mime_type().unwrap_or("image/webp").to_string();
let still = mime == "image/webp";
let (w, h) = document.resolution().unwrap_or((512, 512));
vec![Attachment {
kind: "image".into(),
mimetype: Some(if still { mime.clone() } else { "image/webp".into() }),
filename: Some(format!("sticker{}", if sticker.emoji().is_empty() { String::new() } else { format!(" {}", sticker.emoji()) })),
size: document.size().map(|s| s as u64),
width: Some(w as u32),
height: Some(h as u32),
// An animated or video sticker is drawn as the still picture Telegram sends with it.
path: if still { full(&extension_of(document.name(), Some(&mime))) } else { thumb("jpg") },
thumbnail_path: thumb("jpg"),
..Default::default()
}]
}
Media::Document(document) => {
let mime = document.mime_type().unwrap_or("application/octet-stream").to_string();
let name = document.name().map(str::to_string);
let ext = extension_of(name.as_deref(), Some(&mime));
let voice = is_voice(&document);
let mut kind = kind_of_mime(&mime);
if document.is_animated() && kind == "image" {
kind = "video";
}
let (w, h) = document.resolution().map(|(w, h)| (Some(w as u32), Some(h as u32))).unwrap_or((None, None));
vec![Attachment {
kind: kind.into(),
mimetype: Some(mime),
filename: Some(match (&name, document.is_spoiler()) {
(Some(n), true) => format!("SPOILER_{n}"),
(Some(n), false) => n.clone(),
(None, _) => format!("file.{ext}"),
}),
size: document.size().map(|s| s as u64),
width: w,
height: h,
duration_secs: document.duration(),
voice,
path: full(&ext),
thumbnail_path: (!document.thumbs().is_empty() && matches!(kind, "video" | "file" | "image")).then(|| local(MEDIA, &media_name(account, remote, id, Variant::Thumb, "jpg"))),
..Default::default()
}]
}
_ => Vec::new(),
}
}
/// Whether a recording was spoken into the app rather than attached as a file.
fn is_voice(document: &Document) -> bool {
match document.raw.document.as_ref() {
Some(tl::enums::Document::Document(d)) => d.attributes.iter().any(|a| matches!(a, tl::enums::DocumentAttribute::Audio(audio) if audio.voice)),
_ => false,
}
}
/// Downloads to a file, whole and renamed into place so a half-written one is never taken for a picture.
pub(super) async fn download_to<D: Downloadable>(client: &grammers_client::Client, what: &D, path: &Path, big: bool) -> Result<()> {
let _permit = permits(big).acquire().await?;
if let Some(parent) = path.parent() {
tokio::fs::create_dir_all(parent).await?;
}
// Its own name for the half-written file: two fetches of one picture (a preview being fetched as the message was
// recorded, and the window asking for it) must not write into the same one.
static SEQUENCE: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(0);
let part = path.with_extension(format!("{}.part", SEQUENCE.fetch_add(1, std::sync::atomic::Ordering::Relaxed)));
if let Err(e) = client.download_media(what, &part).await {
let _ = tokio::fs::remove_file(&part).await;
return Err(anyhow!("{}", super::send::explain(&e)));
}
tokio::fs::rename(&part, path).await.with_context(|| format!("writing {}", path.display()))?;
Ok(())
}
/// Fetches a message's media again and writes the version the name asks for.
async fn fetch(handle: &Handle, remote: &str, message: i32, variant: Variant, path: &Path) -> Result<()> {
let id = peers::peer_id_of(remote).ok_or_else(|| anyhow!("not a Telegram conversation"))?;
let peer = handle.session.peer_ref(id).await.ok().flatten().ok_or_else(|| anyhow!("Telegram has not told this account about that conversation yet"))?;
let found = handle.client.get_messages_by_id(peer, &[message]).await.map_err(|e| anyhow!("{}", super::send::explain(&e)))?;
let Some(Some(message)) = found.into_iter().next() else { bail!("that message is no longer there") };
match message.media().ok_or_else(|| anyhow!("that message has no media"))? {
Media::Photo(photo) => {
let sizes = photo.thumbs();
let wanted = match variant {
Variant::Thumb => pick(&sizes, 320),
Variant::Full => pick(&sizes, 1280),
};
match wanted {
Some(size) => download_to(&handle.client, size, path, false).await,
None => download_to(&handle.client, &photo, path, false).await,
}
}
Media::Sticker(sticker) => match variant {
Variant::Full => download_to(&handle.client, &sticker.document, path, false).await,
Variant::Thumb => download_thumb(&handle.client, &sticker.document, path).await,
},
Media::Document(document) => match variant {
Variant::Full => download_to(&handle.client, &document, path, true).await,
Variant::Thumb => download_thumb(&handle.client, &document, path).await,
},
media @ Media::WebPage(_) => {
let photo = page_photo(&media).ok_or_else(|| anyhow!("that link has no picture"))?;
match pick(&photo.thumbs(), CARD) {
Some(size) => download_to(&handle.client, size, path, false).await,
None => download_to(&handle.client, &photo, path, false).await,
}
}
_ => bail!("that kind of media cannot be fetched"),
}
}
pub(super) async fn download_thumb(client: &grammers_client::Client, document: &Document, path: &Path) -> Result<()> {
let sizes = document.thumbs();
let size = pick(&sizes, 320).or(sizes.first()).ok_or_else(|| anyhow!("Telegram sent no preview of that"))?;
// A tiny inline preview carries its own bytes; the rest are downloaded.
if let Some(bytes) = Downloadable::to_data(size) {
tokio::fs::write(path, bytes).await?;
return Ok(());
}
download_to(client, size, path, false).await
}
/// The previews being fetched now, so a message read twice at once fetches its preview once.
fn fetching() -> &'static std::sync::Mutex<std::collections::HashSet<PathBuf>> {
static F: std::sync::OnceLock<std::sync::Mutex<std::collections::HashSet<PathBuf>>> = std::sync::OnceLock::new();
F.get_or_init(Default::default)
}
/// Fetches the small preview of a message's picture or video in the background, as the message is recorded.
///
/// Previews are a few kilobytes and are what the list draws, so they are not left to be asked for one by one when the
/// window finds them missing (which asks Telegram for the message again each time). The message is in hand here, so
/// nothing more is asked. Whole files are still fetched only when looked at.
pub fn prefetch_preview(state: &AppState, account_id: &str, account: i64, remote: &str, message: &Message) {
let Some(media) = message.media() else { return };
let path = crate::media_cache::dir(MEDIA).join(media_name(account, remote, message.id(), Variant::Thumb, "jpg"));
if path.exists() || !matches!(media, Media::Photo(_) | Media::Document(_) | Media::Sticker(_) | Media::WebPage(_)) {
return;
}
if !fetching().lock().unwrap_or_else(|e| e.into_inner()).insert(path.clone()) {
return;
}
let Some(handle) = state.runtime.telegram_handle(account_id) else {
fetching().lock().unwrap_or_else(|e| e.into_inner()).remove(&path);
return;
};
tokio::spawn(async move {
let done = match &media {
Media::Photo(photo) => match pick(&photo.thumbs(), 320) {
Some(size) => download_to(&handle.client, size, &path, false).await,
None => download_to(&handle.client, photo, &path, false).await,
},
Media::Document(document) => download_thumb(&handle.client, document, &path).await,
Media::Sticker(sticker) => download_thumb(&handle.client, &sticker.document, &path).await,
Media::WebPage(_) => match page_photo(&media) {
Some(photo) => match pick(&photo.thumbs(), CARD) {
Some(size) => download_to(&handle.client, size, &path, false).await,
None => download_to(&handle.client, &photo, &path, false).await,
},
None => Ok(()),
},
_ => Ok(()),
};
if let Err(e) = done {
tracing::debug!("telegram: preview {}: {e:#}", path.display());
}
fetching().lock().unwrap_or_else(|e| e.into_inner()).remove(&path);
});
}
/// Fetches a cached Telegram file again, after it expired, was cleared, or was named before it was ever fetched.
pub async fn restore_cached(state: &AppState, path: &Path) -> Result<PathBuf> {
let name = path.file_name().and_then(|n| n.to_str()).context("not a cached file")?;
let parent = path.parent().and_then(|p| p.file_name()).and_then(|n| n.to_str()).unwrap_or("");
if parent == AVATARS {
let (account, peer, photo) = parse_avatar_name(name).context("not a cached Telegram picture")?;
let handle = state.runtime.telegram_handle(&format!("telegram:{account}")).context("that Telegram account is not connected")?;
fetch_avatar(&handle, &peer, photo, path).await?;
return Ok(path.to_path_buf());
}
let (account, remote, message, variant) = parse_media_name(name).context("not a cached Telegram file")?;
// A preview already on its way is waited for, not fetched a second time.
for _ in 0..120 {
if path.exists() {
return Ok(path.to_path_buf());
}
if !fetching().lock().unwrap_or_else(|e| e.into_inner()).contains(path) {
break;
}
tokio::time::sleep(std::time::Duration::from_millis(250)).await;
}
let handle = state.runtime.telegram_handle(&format!("telegram:{account}")).context("that Telegram account is not connected")?;
fetch(&handle, &remote, message, variant, path).await?;
Ok(path.to_path_buf())
}
/// The file an avatar is cached as, for a person or chat with a profile photo; `None` for one without.
pub fn avatar_url(account: i64, peer: &grammers_client::peer::Peer) -> Option<String> {
use grammers_client::peer::Peer;
let photo_id = match peer {
Peer::User(u) => u.photo().map(|p| p.photo_id),
Peer::Group(g) => g.photo().map(|p| p.photo_id),
Peer::Channel(c) => c.photo().map(|p| p.photo_id),
Peer::Community(_) => None,
}?;
Some(local(AVATARS, &avatar_name(account, &peers::remote_id(peer.id()), photo_id)))
}
async fn fetch_avatar(handle: &Handle, remote: &str, photo_id: i64, path: &Path) -> Result<()> {
let id: PeerId = peers::peer_id_of(remote).ok_or_else(|| anyhow!("not a Telegram conversation"))?;
let peer = handle.session.peer_ref(id).await.ok().flatten().ok_or_else(|| anyhow!("Telegram has not told this account about that picture's owner yet"))?;
let what = grammers_client::media::ChatPhoto { raw: tl::types::InputPeerPhotoFileLocation { big: false, peer: peer.into(), photo_id }.into() };
download_to(&handle.client, &what, path, false).await
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_cached_files_name_says_what_it_is_and_is_read_back() {
let name = media_name(5173409493, "-1001179270258", 21910, Variant::Full, "mp4");
assert_eq!(name, "tg_5173409493_n1001179270258_21910_full.mp4");
assert_eq!(parse_media_name(&name), Some((5173409493, "-1001179270258".to_string(), 21910, Variant::Full)));
let person = media_name(1, "6576171495", 7, Variant::Thumb, "jpg");
assert_eq!(parse_media_name(&person), Some((1, "6576171495".to_string(), 7, Variant::Thumb)));
assert_eq!(parse_media_name("not_ours.jpg"), None);
assert_eq!(parse_media_name("tg_1_n5_2_other.jpg"), None);
let avatar = avatar_name(1, "-100123", 99);
assert_eq!(parse_avatar_name(&avatar), Some((1, "-100123".to_string(), 99)));
assert_eq!(parse_avatar_name("tg_1_n5_2_full.jpg"), None, "a media file is not an avatar");
}
#[test]
fn a_files_extension_comes_from_its_name_else_its_type() {
assert_eq!(extension_of(Some("Report.PDF"), None), "pdf");
assert_eq!(extension_of(None, Some("video/mp4")), "mp4");
assert_eq!(extension_of(Some("noextension"), Some("audio/ogg")), "ogg");
assert_eq!(extension_of(Some("weird.name with spaces"), None), "bin");
assert_eq!(kind_of_mime("image/webp"), "image");
assert_eq!(kind_of_mime("image/gif"), "video");
assert_eq!(kind_of_mime("application/pdf"), "file");
}
#[test]
fn the_size_to_draw_is_the_smallest_that_reaches_the_target_else_the_largest() {
let sizes = vec![("s", 90), ("y", 1280), ("m", 320), ("x", 800)];
assert_eq!(choose(&sizes, 320), Some(&"m"));
assert_eq!(choose(&sizes, 1280), Some(&"y"));
assert_eq!(choose(&sizes, 5000), Some(&"y"));
assert_eq!(choose(&sizes, 100), Some(&"m"));
assert_eq!(choose::<&str>(&[], 320), None);
}
}
src/backend/telegram/members.rs
+59
View File
@@ -0,0 +1,59 @@
//! Who is in a group: the list beside the conversation.
//!
//! Telegram will list the members of a group or a supergroup (a channel's subscribers are for its administrators
//! only), two hundred at a time. The first page is read when somebody opens the conversation, with the group's own
//! count, and the owner and administrators carry a mark in front of their name.
use super::media;
use super::messages;
use super::send::{explain, peer_for};
use crate::model::Buffer;
use crate::state::AppState;
use grammers_client::peer::Role;
use serde_json::{json, Value};
/// How many members are read: Telegram's own page.
const LIMIT: usize = 200;
/// What stands in front of an owner's or an administrator's name.
const OWNER: &str = "👑";
const ADMIN: &str = "@";
/// Reads a group's members and says so, as the list and as how many there are in all.
pub async fn refresh(state: &AppState, buffer: &Buffer) -> Result<usize, String> {
let (handle, peer) = peer_for(state, buffer).await?;
// A person's conversation, a channel somebody only follows, or the account's own: nobody to list.
if buffer.kind == "dm" || buffer.category.as_deref() == Some(super::peers::CHANNELS) {
return Ok(0);
}
let mut iter = handle.client.iter_participants(peer);
let total = iter.total().await.unwrap_or(0);
let mut members: Vec<Value> = Vec::new();
while members.len() < LIMIT {
let Some(found) = iter.next().await.map_err(|e| explain(&e))? else { break };
let (prefix, banned) = match &found.role {
Role::Creator(_) => (OWNER, false),
Role::Admin(_) => (ADMIN, false),
Role::Banned(_) | Role::Left(_) => ("", true),
_ => ("", false),
};
if banned {
continue;
}
let peer = grammers_client::peer::Peer::User(found.user.clone());
handle.names.lock().unwrap_or_else(|e| e.into_inner()).insert(found.user.id().bare_id_unchecked(), messages::name_of(&peer));
members.push(json!({
"nick": messages::name_of(&peer),
"userId": found.user.id().bare_id_unchecked().to_string(),
"prefix": prefix,
"bot": found.user.is_bot(),
"avatarUrl": media::avatar_url(handle.own.user_id, &peer),
}));
}
let count = members.len();
let list = Value::Array(members);
state.runtime.set_presence(&buffer.id, list.clone());
state.runtime.set_presence_total(&buffer.id, total.max(count));
state.events.emit("presenceChange", json!({ "bufferId": buffer.id, "members": list, "total": total.max(count) }));
Ok(count)
}
src/backend/telegram/membership.rs
+134
View File
@@ -0,0 +1,134 @@
//! Joining and leaving: a public chat by its @name or t.me link, a private one by its invite link, and leaving what
//! the account is in.
//!
//! A chat that was joined arrives as a dialog, so the list is read again afterwards and the window opens the new
//! conversation as it does for any service.
use super::dialogs;
use super::peers;
use super::send::{explain, peer_for};
use crate::model::Buffer;
use crate::state::AppState;
use grammers_client::peer::Peer;
use grammers_client::tl;
use grammers_session::types::PeerKind;
/// What a person typed to name somewhere to join.
#[derive(Debug, PartialEq)]
pub enum Target {
/// A public name: a person, a group or a channel.
Username(String),
/// The secret part of a private invite link.
Invite(String),
}
/// Reads `@name`, `name`, `t.me/name`, `https://t.me/+hash`, `t.me/joinchat/hash` and `tg://join?invite=hash`.
pub fn parse_target(input: &str) -> Option<Target> {
let text = input.trim();
if let Some(rest) = text.strip_prefix("tg://join?invite=") {
return clean_invite(rest).map(Target::Invite);
}
let rest = text.strip_prefix("https://").or_else(|| text.strip_prefix("http://")).unwrap_or(text);
let path = ["t.me/", "telegram.me/", "telegram.dog/"].iter().find_map(|host| rest.strip_prefix(host));
let Some(path) = path else {
let name = text.strip_prefix('@').unwrap_or(text);
return valid_username(name).then(|| Target::Username(name.to_string()));
};
let path = path.split(['?', '#']).next().unwrap_or("");
let mut parts = path.split('/').filter(|p| !p.is_empty());
let first = parts.next()?;
if first == "joinchat" {
return clean_invite(parts.next()?).map(Target::Invite);
}
if let Some(hash) = first.strip_prefix('+') {
return clean_invite(hash).map(Target::Invite);
}
valid_username(first).then(|| Target::Username(first.to_string()))
}
fn clean_invite(hash: &str) -> Option<String> {
let hash = hash.trim();
(!hash.is_empty() && hash.chars().all(|c| c.is_ascii_alphanumeric() || c == '_' || c == '-')).then(|| hash.to_string())
}
/// Telegram's rule: 5 to 32 letters, digits and underscores, starting with a letter.
fn valid_username(name: &str) -> bool {
(5..=32).contains(&name.len()) && name.chars().next().is_some_and(|c| c.is_ascii_alphabetic()) && name.chars().all(|c| c.is_ascii_alphanumeric() || c == '_')
}
/// Joins what was named, and makes its conversation.
pub async fn join(state: &AppState, account_id: &str, input: &str) -> Result<(), String> {
let target = parse_target(input).ok_or_else(|| format!("\"{}\" is not a Telegram @name or t.me link", input.trim()))?;
let handle = state.runtime.telegram_handle(account_id).ok_or_else(|| "that Telegram account is not connected".to_string())?;
match target {
Target::Invite(hash) => {
handle.client.invoke(&tl::functions::messages::ImportChatInvite { hash }).await.map_err(|e| match &e {
grammers_client::InvocationError::Rpc(r) if r.name == "INVITE_HASH_EXPIRED" || r.name == "INVITE_HASH_INVALID" => "That invite link is no longer valid".to_string(),
grammers_client::InvocationError::Rpc(r) if r.name == "INVITE_REQUEST_SENT" => "Your request to join was sent: an administrator has to accept it".to_string(),
_ => explain(&e),
})?;
}
Target::Username(name) => {
let peer = handle.client.resolve_username(&name).await.map_err(|e| explain(&e))?.ok_or_else(|| format!("Nobody on Telegram is called @{name}"))?;
match &peer {
// A person: the conversation with them is all there is to open.
Peer::User(_) => {
let buffer = peers::buffer_for(state, account_id, &peer);
if let Some(avatar) = super::media::avatar_url(handle.own.user_id, &peer) {
state.runtime.set_buffer_avatar(state, &buffer.id, &avatar);
}
return Ok(());
}
_ => {
let peer_ref = peer.to_ref().await.ok().flatten().ok_or_else(|| "Telegram did not say how to reach that chat".to_string())?;
handle.client.join_chat(peer_ref).await.map_err(|e| match &e {
grammers_client::InvocationError::Rpc(r) if r.name == "INVITE_REQUEST_SENT" => "Your request to join was sent: an administrator has to accept it".to_string(),
_ => explain(&e),
})?;
}
}
}
}
dialogs::sync(state, account_id, &handle).await.map_err(|e| e.to_string())?;
Ok(())
}
/// Leaves a conversation: a channel or supergroup is left, a small group is left, a private chat is cleared.
pub async fn leave(state: &AppState, buffer: &Buffer) -> Result<(), String> {
let (handle, peer) = peer_for(state, buffer).await?;
match peer.id.kind() {
PeerKind::Channel => {
handle.client.invoke(&tl::functions::channels::LeaveChannel { channel: peer.into() }).await.map_err(|e| explain(&e))?;
}
PeerKind::Chat => {
handle.client.invoke(&tl::functions::messages::DeleteChatUser { revoke_history: false, chat_id: peer.into(), user_id: tl::enums::InputUser::UserSelf }).await.map_err(|e| explain(&e))?;
}
_ => {
// History is deleted from this account's side only; the other person keeps theirs.
handle.client.invoke(&tl::functions::messages::DeleteHistory { just_clear: false, revoke: false, peer: peer.into(), max_id: 0, min_date: None, max_date: None }).await.map_err(|e| explain(&e))?;
}
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn names_and_links_are_read() {
let user = |n: &str| Some(Target::Username(n.to_string()));
let invite = |h: &str| Some(Target::Invite(h.to_string()));
assert_eq!(parse_target("@abcd"), None, "too short for Telegram, so not a name");
assert_eq!(parse_target("@disclosetv"), user("disclosetv"));
assert_eq!(parse_target("disclosetv"), user("disclosetv"));
assert_eq!(parse_target("https://t.me/disclosetv"), user("disclosetv"));
assert_eq!(parse_target("t.me/disclosetv/1234?single"), user("disclosetv"));
assert_eq!(parse_target("https://t.me/+AbC_d-123"), invite("AbC_d-123"));
assert_eq!(parse_target("https://t.me/joinchat/AbC_d-123"), invite("AbC_d-123"));
assert_eq!(parse_target("tg://join?invite=AbC_d-123"), invite("AbC_d-123"));
assert_eq!(parse_target("two words"), None);
assert_eq!(parse_target("1startsdigit"), None);
assert_eq!(parse_target(""), None);
}
}
src/backend/telegram/messages.rs
+460
View File
@@ -0,0 +1,460 @@
//! A Telegram message as moho's: who said it, what it says, what it answers, what came with it.
//!
//! Telegram marks up a message with entities - a kind, an offset and a length, counted in UTF-16 code units - which
//! become the message's `html` (bold, italic, underline, strike, code, a quote, a link, a spoiler) beside its plain
//! `body`. Media is not drawn until Round 4; a message that carries some says what it is in words, after its caption.
use super::media;
use super::peers;
use crate::model::ReplyPreview;
use crate::state::AppState;
use grammers_client::message::Message;
use grammers_client::peer::Peer;
use grammers_client::tl;
/// What the connection knows about the account itself: its name here, which a message it sent is attributed to.
#[derive(Clone)]
pub struct Own {
pub name: String,
pub user_id: i64,
}
fn escape(text: &str) -> String {
text.replace('&', "&amp;").replace('<', "&lt;").replace('>', "&gt;").replace('"', "&quot;")
}
/// A span to mark: where it starts and ends in UTF-16 units, and the tags that open and close it.
struct Span {
from: usize,
to: usize,
open: String,
close: &'static str,
}
fn span_of(entity: &tl::enums::MessageEntity, text16: &[u16]) -> Option<Span> {
use tl::enums::MessageEntity as E;
let (offset, length, open, close): (i32, i32, String, &'static str) = match entity {
E::Bold(e) => (e.offset, e.length, "<b>".into(), "</b>"),
E::Italic(e) => (e.offset, e.length, "<i>".into(), "</i>"),
E::Underline(e) => (e.offset, e.length, "<u>".into(), "</u>"),
E::Strike(e) => (e.offset, e.length, "<s>".into(), "</s>"),
E::Code(e) => (e.offset, e.length, "<code>".into(), "</code>"),
E::Pre(e) => (e.offset, e.length, "<pre>".into(), "</pre>"),
E::Blockquote(e) => (e.offset, e.length, "<blockquote>".into(), "</blockquote>"),
E::Spoiler(e) => (e.offset, e.length, "<span class=\"spoiler\">".into(), "</span>"),
E::TextUrl(e) => (e.offset, e.length, format!("<a href=\"{}\">", escape(&e.url)), "</a>"),
E::Url(e) => {
// The link is its own text.
let (from, to) = (e.offset.max(0) as usize, ((e.offset + e.length).max(0) as usize).min(text16.len()));
let target = String::from_utf16_lossy(text16.get(from..to)?);
let target = if target.contains("://") { target } else { format!("https://{target}") };
(e.offset, e.length, format!("<a href=\"{}\">", escape(&target)), "</a>")
}
_ => return None,
};
let from = offset.max(0) as usize;
let to = ((offset + length).max(0) as usize).min(text16.len());
(from < to).then_some(Span { from, to, open, close })
}
/// The message's text with its entities as html, or `None` when it has none worth drawing.
pub fn entities_html(text: &str, entities: &[tl::enums::MessageEntity]) -> Option<String> {
let text16: Vec<u16> = text.encode_utf16().collect();
let mut spans: Vec<Span> = entities.iter().filter_map(|e| span_of(e, &text16)).collect();
if spans.is_empty() {
return None;
}
// Outer spans open first: by where they start, the longer before the shorter.
spans.sort_by(|a, b| a.from.cmp(&b.from).then(b.to.cmp(&a.to)));
let mut out = String::new();
let mut open: Vec<&Span> = Vec::new();
let mut next = 0;
let mut position = 0;
let mut run = String::new();
let flush = |run: &mut String, out: &mut String| {
if !run.is_empty() {
out.push_str(&escape(run).replace('\n', "<br>"));
run.clear();
}
};
let mut units = text16.iter().copied().peekable();
while position <= text16.len() {
// What ends here closes first, innermost first.
while let Some(last) = open.last() {
if last.to <= position {
flush(&mut run, &mut out);
out.push_str(last.close);
open.pop();
} else {
break;
}
}
while next < spans.len() && spans[next].from <= position {
flush(&mut run, &mut out);
out.push_str(&spans[next].open);
open.push(&spans[next]);
next += 1;
}
let Some(unit) = units.next() else { break };
// A character outside the basic plane is two units: take both together.
let mut pair = vec![unit];
if (0xD800..0xDC00).contains(&unit) {
if let Some(low) = units.next() {
pair.push(low);
position += 1;
}
}
run.push_str(&String::from_utf16_lossy(&pair));
position += 1;
}
flush(&mut run, &mut out);
while let Some(last) = open.pop() {
out.push_str(last.close);
}
Some(out)
}
/// What a message with media says it carries, in words.
pub fn media_label(media: &tl::enums::MessageMedia) -> Option<String> {
use tl::enums::MessageMedia as M;
Some(match media {
M::Empty | M::Unsupported => return None,
// A link's card is the link already in the text.
M::WebPage(_) => return None,
M::Photo(_) => "[Photo]".to_string(),
M::Geo(_) | M::GeoLive(_) => "[Location]".to_string(),
M::Venue(v) => format!("[Place: {}]", v.title),
M::Contact(c) => format!("[Contact: {} {}]", c.first_name, c.last_name).trim_end().replace(" ]", "]"),
M::Poll(p) => format!("[Poll: {}]", poll_question(&p.poll)),
M::Dice(d) => format!("[{} {}]", d.emoticon, d.value),
M::Document(d) => match &d.document {
Some(tl::enums::Document::Document(doc)) => document_label(doc),
_ => "[File]".to_string(),
},
_ => "[Attachment]".to_string(),
})
}
fn poll_question(poll: &tl::enums::Poll) -> String {
match poll {
tl::enums::Poll::Poll(p) => match &p.question {
tl::enums::TextWithEntities::Entities(t) => t.text.clone(),
},
}
}
fn document_label(doc: &tl::types::Document) -> String {
use tl::enums::DocumentAttribute as A;
let name = doc.attributes.iter().find_map(|a| if let A::Filename(f) = a { Some(f.file_name.clone()) } else { None });
for attribute in &doc.attributes {
match attribute {
A::Sticker(s) => return format!("[Sticker {}]", s.alt).replace("[Sticker ]", "[Sticker]"),
A::Audio(a) if a.voice => return "[Voice message]".to_string(),
A::Audio(a) => return format!("[Audio: {}]", a.title.clone().or(name.clone()).unwrap_or_else(|| "track".to_string())),
A::Video(v) if v.round_message => return "[Video message]".to_string(),
A::Video(_) => return format!("[Video{}]", name.map(|n| format!(": {n}")).unwrap_or_default()),
A::Animated => return "[GIF]".to_string(),
_ => {}
}
}
format!("[File{}]", name.map(|n| format!(": {n}")).unwrap_or_default())
}
/// What a service message says (somebody joined, the title changed), or `None` for those with nothing to say.
pub fn service_text(who: &str, action: &tl::enums::MessageAction) -> Option<String> {
use tl::enums::MessageAction as A;
Some(match action {
A::ChatCreate(a) => format!("{who} created the group \"{}\"", a.title),
A::ChatEditTitle(a) => format!("{who} changed the title to \"{}\"", a.title),
A::ChatEditPhoto(_) => format!("{who} changed the group photo"),
A::ChatDeletePhoto => format!("{who} removed the group photo"),
A::ChatAddUser(a) => format!("{who} added {} {}", a.users.len(), if a.users.len() == 1 { "person" } else { "people" }),
A::ChatDeleteUser(_) => format!("{who} removed somebody from the group"),
A::ChatJoinedByLink(_) => format!("{who} joined by invite link"),
A::ChatJoinedByRequest => format!("{who} joined after being approved"),
A::ChannelCreate(a) => format!("The channel \"{}\" was created", a.title),
A::PinMessage => format!("{who} pinned a message"),
A::HistoryClear => "The history was cleared".to_string(),
A::ScreenshotTaken => format!("{who} took a screenshot"),
A::ContactSignUp => format!("{who} joined Telegram"),
A::SetMessagesTtl(a) => if a.period == 0 { "Messages no longer disappear".to_string() } else { format!("Messages now disappear after {} seconds", a.period) },
A::PhoneCall(a) => format!("{who} {} call", if a.video { "had a video" } else { "had a voice" }),
A::TopicCreate(a) => format!("{who} created the topic \"{}\"", a.title),
A::Empty => return None,
_ => format!("{who} did something Telegram describes in a way moho does not show yet"),
})
}
/// A person's name, for a line that says who did something.
pub fn name_of(peer: &Peer) -> String {
peers::title_of(peer)
}
/// Everything about a message that is words: the caption or text and what came with it. Where it was forwarded from is
/// said above it (see `forward_preview`), not in it.
pub fn body_of(message: &Message) -> String {
let mut parts: Vec<String> = Vec::new();
let text = message.text();
if !text.is_empty() {
parts.push(text.to_string());
}
if let tl::enums::Message::Message(m) = &message.raw {
if let Some(label) = m.media.as_ref().and_then(media_label) {
parts.push(label);
}
}
parts.join("\n")
}
/// Records a message in its conversation, as it arrived live or as history. `false` when it was already there.
pub fn record(state: &AppState, account_id: &str, own: &Own, message: &Message) -> bool {
record_named(state, account_id, own, message, None)
}
/// The same, remembering who the sender is for `names` when it is given.
pub fn record_named(state: &AppState, account_id: &str, own: &Own, message: &Message, names: Option<&std::sync::Mutex<std::collections::HashMap<i64, String>>>) -> bool {
let Some(peer) = message.peer() else { return false };
record_in(state, account_id, own, message, peer, message.sender(), names)
}
/// The same for a message whose conversation the caller found. A message a person sent in a private chat arrives as
/// a short update that names no one, so the library cannot say who it is from or where it is: the caller asks
/// Telegram, and the other person in a private chat is the sender of everything not sent by the account.
pub fn record_in(state: &AppState, account_id: &str, own: &Own, message: &Message, peer: &Peer, sender: Option<&Peer>, names: Option<&std::sync::Mutex<std::collections::HashMap<i64, String>>>) -> bool {
let buffer = peers::buffer_for(state, account_id, peer);
let outgoing = message.outgoing();
let sender = sender.or_else(|| (!outgoing && matches!(peer, Peer::User(_))).then_some(peer));
if let (Some(names), Some(Peer::User(user))) = (names, sender) {
names.lock().unwrap_or_else(|e| e.into_inner()).insert(user.id().bare_id_unchecked(), name_of(&Peer::User(user.clone())));
}
let from = if outgoing {
own.name.clone()
} else {
sender.map(name_of).unwrap_or_else(|| peers::title_of(peer))
};
let sender_id = if outgoing { Some(own.user_id.to_string()) } else { message.sender_id().map(peers::remote_id) };
let ts = message.date().as_second();
let id = message.id().to_string();
if let Some(action) = message.action() {
let Some(text) = service_text(&from, action) else { return false };
return state.runtime.record_message_at(state, account_id, &buffer.name, &buffer.kind, "Telegram", &text, false, "system", None, Some(id), false, None, Vec::new(), Vec::new(), None, Some(ts), None, None);
}
let topic_only = matches!(&message.raw, tl::enums::Message::Message(m) if m.reply_to.as_ref().is_some_and(is_topic_root));
let answered = if topic_only { None } else { reply_preview(state, account_id, &buffer, message) };
let forward = message.forward_header().map(|h| forward_preview(state, account_id, &h));
// One slot above a message: an answer wins, and says it was forwarded in its text.
let (reply, forward_line) = match (answered, forward) {
(Some(answer), Some(forwarded)) => (Some(answer), Some(format!("↪ Forwarded from {}", forwarded.from))),
(Some(answer), None) => (Some(answer), None),
(None, forwarded) => (forwarded, None),
};
let ask_for_reply = reply.as_ref().filter(|r| r.body.is_empty() && !r.forwarded).map(|r| (r.id.clone(), r.chat.clone()));
let has_forward_line = forward_line.is_some();
let body = match forward_line {
Some(line) => format!("{line}\n{}", body_of(message)),
None => body_of(message),
};
let html = message.fmt_entities().and_then(|e| entities_html(message.text(), e)).filter(|h| !h.is_empty());
// The first line of a forward is words of our own, so the html (made from the text alone) would lose it: a
// forwarded message is shown as plain text.
let html = if has_forward_line { None } else { html };
let attachments = buffer.remote_id.as_deref().map(|remote| media::attachments_of(own.user_id, remote, message)).unwrap_or_default();
let avatar = if outgoing { None } else { sender.and_then(|p| media::avatar_url(own.user_id, p)) };
if let Some(remote) = buffer.remote_id.as_deref() {
media::prefetch_preview(state, account_id, own.user_id, remote, message);
}
let again = attachments.clone();
let reply_target = ask_for_reply.map(|(reply_id, chat)| {
// Where the answered message is: the other conversation if it is one, else this.
let remote = chat.and_then(|c| state.runtime.get_buffer(&c)).and_then(|b| b.remote_id).or_else(|| buffer.remote_id.clone());
(remote, reply_id)
});
let cards = buffer.remote_id.as_deref().map(|remote| media::embeds_of(own.user_id, remote, message)).unwrap_or_default();
let cards_again = cards.clone();
let recorded = state.runtime.record_message_at(state, account_id, &buffer.name, &buffer.kind, &from, &body, false, "chat", reply, Some(id.clone()), message.mentioned(), avatar, cards, attachments, sender_id, Some(ts), html, None);
if !recorded && !cards_again.is_empty() {
if let Ok(Some(stored)) = state.store.get_message(&buffer.id, &id) {
if stored.embeds != cards_again && state.store.set_message_embeds(&buffer.id, &id, &cards_again).unwrap_or(false) {
state.events.emit("messageUpdated", serde_json::json!({ "bufferId": buffer.id, "id": id, "body": stored.body, "edited": false, "embeds": cards_again, "attachments": stored.attachments }));
}
}
}
// Read again, a message already kept without its picture or file gets it: earlier builds kept the text alone.
if !recorded && !again.is_empty() {
if let Ok(Some(stored)) = state.store.get_message(&buffer.id, &id) {
if stored.attachments.is_empty() && state.store.update_message_attachments(&buffer.id, &id, &again).unwrap_or(false) {
state.events.emit("messageUpdated", serde_json::json!({ "bufferId": buffer.id, "id": id, "body": stored.body, "edited": false, "embeds": stored.embeds, "attachments": again }));
}
}
}
// What a reply answered, when it was not in hand: asked of Telegram, and the row is filled in when it comes.
if let Some((Some(remote), reply_id)) = reply_target {
fill_reply_later(state, account_id, &buffer.id, &id, &remote, &reply_id);
}
// A poll that is still open is the conversation's card.
if let tl::enums::Message::Message(raw) = &message.raw {
if let Some(tl::enums::MessageMedia::Poll(p)) = &raw.media {
let (tl::enums::Poll::Poll(poll), tl::enums::PollResults::Results(results)) = (&p.poll, &p.results);
super::polls::announce(state, &buffer.id, message.id(), poll, results);
}
}
// What people have reacted with, which a message arrives carrying.
if let tl::enums::Message::Message(m) = &message.raw {
// How many have answered it, or commented under it if it is a channel's post.
if let Some(tl::enums::MessageReplies::Replies(r)) = &m.replies {
if r.replies > 0 {
state.events.emit("threadCount", serde_json::json!({ "bufferId": buffer.id, "id": id, "count": r.replies }));
}
}
if let Some(reactions) = &m.reactions {
let list = super::reactions::from_tl(reactions);
if !list.is_empty() {
super::reactions::apply(state, &buffer.id, &id, &list);
}
}
}
recorded
}
/// Where a forwarded message came from, as the line above it: the name of the source and, when it was a post in a
/// channel this account has open, where to find it.
fn forward_preview(state: &AppState, account_id: &str, header: &tl::enums::MessageFwdHeader) -> ReplyPreview {
let tl::enums::MessageFwdHeader::Header(h) = header;
let source = h.from_id.clone().map(|p| peers::remote_id(grammers_session::types::PeerId::from(p)));
let origin = source.as_ref().and_then(|remote| state.runtime.list_buffers().into_iter().find(|b| b.account_id == account_id && b.remote_id.as_deref() == Some(remote.as_str())));
let from = origin.as_ref().map(|b| b.name.clone()).or_else(|| h.from_name.clone()).or_else(|| h.post_author.clone()).unwrap_or_else(|| "somebody".to_string());
// Only a post in a channel can be gone to: a person's forward has no message of theirs to point at.
let (chat, id) = match (&origin, h.channel_post) {
(Some(b), Some(post)) if b.kind == "channel" => (Some(b.id.clone()), post.to_string()),
_ => (None, String::new()),
};
ReplyPreview { id, from, body: String::new(), thread: false, forwarded: true, thread_root: None, chat }
}
/// What a message answers, as the quote above it: the words quoted (or the message answered), who said them, and - when
/// it was said in another conversation - where, so the reader can go there.
fn reply_preview(state: &AppState, account_id: &str, buffer: &crate::model::Buffer, message: &Message) -> Option<ReplyPreview> {
let reply_id = message.reply_to_message_id()?;
let header = match &message.raw {
tl::enums::Message::Message(m) => match &m.reply_to {
Some(tl::enums::MessageReplyHeader::Header(h)) => Some(h.clone()),
_ => None,
},
_ => None,
};
let elsewhere = header
.as_ref()
.and_then(|h| h.reply_to_peer_id.clone())
.map(|p| peers::remote_id(grammers_session::types::PeerId::from(p)))
.filter(|remote| buffer.remote_id.as_deref() != Some(remote.as_str()));
let other = elsewhere.as_ref().and_then(|remote| state.runtime.list_buffers().into_iter().find(|b| b.account_id == account_id && b.remote_id.as_deref() == Some(remote.as_str())));
let source = other.as_ref().map(|b| b.id.clone()).unwrap_or_else(|| buffer.id.clone());
// A message in a conversation this account has not got open is not one to look for here.
let stored = if elsewhere.is_some() && other.is_none() { None } else { state.store.get_message(&source, &reply_id.to_string()).ok().flatten() };
let quote = header.as_ref().and_then(|h| h.quote_text.clone()).filter(|t| !t.trim().is_empty());
let named = header.as_ref().and_then(|h| h.reply_from.as_ref()).and_then(|f| {
let tl::enums::MessageFwdHeader::Header(f) = f;
f.from_name.clone().or_else(|| f.post_author.clone())
});
let shown_media = header.as_ref().and_then(|h| h.reply_media.as_ref()).and_then(media_label);
let from = other.as_ref().map(|b| b.name.clone()).or(named).or_else(|| stored.as_ref().map(|m| m.from.clone())).unwrap_or_default();
let body = quote.or_else(|| stored.map(|m| m.body)).or(shown_media).unwrap_or_default();
Some(ReplyPreview { id: reply_id.to_string(), from, body, thread: false, forwarded: false, thread_root: None, chat: other.map(|b| b.id) })
}
/// Asks Telegram what a reply answered and fills the quote in.
fn fill_reply_later(state: &AppState, account_id: &str, buffer_id: &str, msg_id: &str, remote: &str, reply_id: &str) {
let (Some(handle), Some(id), Ok(reply)) = (state.runtime.telegram_handle(account_id), peers::peer_id_of(remote), reply_id.parse::<i32>()) else { return };
let (state, buffer_id, msg_id) = (state.clone(), buffer_id.to_string(), msg_id.to_string());
tokio::spawn(async move {
use grammers_session::Session;
let Some(peer) = handle.session.peer_ref(id).await.ok().flatten() else { return };
let Ok(found) = handle.client.get_messages_by_id(peer, &[reply]).await else { return };
let Some(Some(original)) = found.into_iter().next() else { return };
let body = body_of(&original);
let from = original.sender().map(name_of).unwrap_or_else(|| original.peer().map(peers::title_of).unwrap_or_default());
if !body.is_empty() {
state.runtime.fill_reply_preview(&state, &buffer_id, &msg_id, &from, &body);
}
});
}
/// In a group with topics every message "replies" to its topic's first message; that is where it is, not who it
/// answers. A real reply names a message and the topic it is in as well.
fn is_topic_root(header: &tl::enums::MessageReplyHeader) -> bool {
matches!(header, tl::enums::MessageReplyHeader::Header(h) if h.forum_topic && h.reply_to_top_id.is_none())
}
#[cfg(test)]
mod tests {
use super::*;
use tl::types as t;
fn bold(offset: i32, length: i32) -> tl::enums::MessageEntity {
tl::enums::MessageEntity::Bold(t::MessageEntityBold { offset, length })
}
fn italic(offset: i32, length: i32) -> tl::enums::MessageEntity {
tl::enums::MessageEntity::Italic(t::MessageEntityItalic { offset, length })
}
fn header(forum_topic: bool, msg: i32, top: Option<i32>) -> tl::enums::MessageReplyHeader {
tl::enums::MessageReplyHeader::Header(t::MessageReplyHeader {
reply_to_scheduled: false,
forum_topic,
quote: false,
reply_to_ephemeral: false,
reply_to_msg_id: Some(msg),
reply_to_peer_id: None,
reply_from: None,
reply_media: None,
reply_to_top_id: top,
quote_text: None,
quote_entities: None,
quote_offset: None,
todo_item_id: None,
poll_option: None,
})
}
#[test]
fn a_message_in_a_topic_is_not_a_reply_to_the_topic() {
assert!(is_topic_root(&header(true, 10, None)));
assert!(!is_topic_root(&header(true, 55, Some(10))), "answering a message inside the topic");
assert!(!is_topic_root(&header(false, 10, None)), "an ordinary reply");
}
#[test]
fn entities_become_html_and_nest() {
assert_eq!(entities_html("hello world", &[bold(0, 5)]).unwrap(), "<b>hello</b> world");
assert_eq!(entities_html("a bc d", &[bold(2, 2), italic(2, 1)]).unwrap(), "a <b><i>b</i>c</b> d");
assert_eq!(entities_html("plain", &[]), None);
// The text is escaped, and a line break is one.
assert_eq!(entities_html("a < b\nc", &[bold(0, 1)]).unwrap(), "<b>a</b> &lt; b<br>c");
}
#[test]
fn offsets_are_in_utf16_units_so_an_emoji_before_the_text_moves_it_by_two() {
// "😀 hi": the emoji is two units, the space one, so "hi" starts at 3.
assert_eq!(entities_html("😀 hi", &[bold(3, 2)]).unwrap(), "😀 <b>hi</b>");
// A span that runs past the end is cut at it rather than panicking.
assert_eq!(entities_html("ab", &[bold(1, 50)]).unwrap(), "a<b>b</b>");
}
#[test]
fn a_link_is_its_own_text_or_its_url_and_a_bare_domain_gets_a_scheme() {
let url = tl::enums::MessageEntity::Url(t::MessageEntityUrl { offset: 4, length: 11 });
assert_eq!(entities_html("see example.com now", &[url]).unwrap(), "see <a href=\"https://example.com\">example.com</a> now");
let text_url = tl::enums::MessageEntity::TextUrl(t::MessageEntityTextUrl { offset: 0, length: 4, url: "https://a.example/?q=1&r=2".into() });
assert_eq!(entities_html("here", &[text_url]).unwrap(), "<a href=\"https://a.example/?q=1&amp;r=2\">here</a>");
}
#[test]
fn service_messages_say_who_did_what() {
let title = tl::enums::MessageAction::ChatEditTitle(t::MessageActionChatEditTitle { title: "Lunch".into() });
assert_eq!(service_text("Ann", &title).as_deref(), Some("Ann changed the title to \"Lunch\""));
assert_eq!(service_text("Ann", &tl::enums::MessageAction::PinMessage).as_deref(), Some("Ann pinned a message"));
assert_eq!(service_text("Ann", &tl::enums::MessageAction::Empty), None);
}
}
src/backend/telegram/mod.rs
+115
View File
@@ -0,0 +1,115 @@
//! Telegram, over MTProto, in pure Rust (the `grammers` crates).
//!
//! # How this folder is arranged
//!
//! Built in rounds, as Twitch was (see Gitea issue #26 for the checklist and what has been verified):
//!
//! - this file - the identity moho gives Telegram, and what every other file shares
//! - `session` - what the library remembers between runs, in a private file of its own per account
//! - `link` - one connection to Telegram: the sender pool, the client and the stream of updates
//! - `login` - adding an account: phone, code, cloud password
//! - `connect` - an account's connection, kept up and reconnected
//! - `dialogs` - the account's conversations, read when it connects
//! - `history` - a conversation's history, read when it is opened and scrolled back
//! - `markup` - what the message box holds, as plain text and entities
//! - `updates` - what Telegram sends that is not a message: typing, reads on other apps
//! - `send` - saying things: send, answer, edit, delete, typing, read
//! - `media` - pictures, video, files and voice: what a message carries, fetched when it is looked at
//! - `reactions` - the emoji on a message, and the account's own
//! - `pins` - a conversation's pinned messages
//! - `members` - who is in a group
//! - `profile` - the card for a person
//! - `search` - searching on Telegram's side
//! - `polls` - a poll as the conversation's card, voted in from it
//! - `threads` - replies in a thread, and a channel post's comments
//! - `membership` - joining by @name or link, leaving
//! - `admin` - what an administrator may do to a member
//! - `notify` - muting a chat on the account
//! - `instant` - an Instant View article as a page of its own
//! - `peers` - a conversation as a buffer: its name, its kind, its heading in the list
//! - `messages` - a message as moho's, with its entities as html
//!
//! # The api_id
//!
//! Telegram gives one `api_id` and `api_hash` to a phone number's owner, and a client sends them with every
//! connection. Moho does not embed one: each account carries its own, typed into the sign-in form, and they are
//! kept with the session like a password. Nothing here logs them.
//!
//! # What moho tells Telegram about itself
//!
//! Telegram lists every session of an account (Settings, Devices) with the app and device each one reported, and
//! treats a stream of unfamiliar sessions as a risk. So the identity is set once, is the same on every connection,
//! and says what this is: the device is "moho" on this operating system, the version is moho's own. A session is
//! saved and reused; nothing here signs in afresh to get over a hiccup.
pub mod admin;
pub mod connect;
pub mod dialogs;
pub mod history;
pub mod instant;
pub mod link;
pub mod login;
pub mod markup;
pub mod media;
pub mod members;
pub mod membership;
pub mod messages;
pub mod notify;
pub mod peers;
pub mod pins;
pub mod polls;
pub mod profile;
pub mod reactions;
pub mod search;
pub mod send;
pub mod threads;
pub mod updates;
pub mod session;
use grammers_mtsender::ConnectionParams;
use std::sync::Arc;
/// What the rest of the daemon needs to act on a connected account: its client, its session (to turn a conversation
/// into something Telegram will accept, an id with the authority to use it) and its own name.
#[derive(Clone)]
pub struct Handle {
pub client: grammers_client::Client,
pub session: Arc<session::FileSession>,
pub own: messages::Own,
/// What people are called, by Telegram's number for them, so a typing notice can say who. Filled as people are
/// met in the conversation list and in messages.
pub names: Arc<std::sync::Mutex<std::collections::HashMap<i64, String>>>,
}
/// The name Telegram shows for this app's sessions: `moho` and the kind of machine it runs on.
pub fn device_model() -> String {
format!("moho ({} {})", std::env::consts::OS, std::env::consts::ARCH)
}
/// How every connection of an account introduces itself.
///
/// `proxy` is the SOCKS5 address of the account's route (Tor or the proxy in the network settings), if it has one.
pub fn connection_params(proxy: Option<&str>) -> ConnectionParams {
ConnectionParams {
device_model: device_model(),
// Moho's own version, which is the daemon's.
app_version: env!("CARGO_PKG_VERSION").to_string(),
proxy_url: proxy.map(str::to_string),
..ConnectionParams::default()
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn the_identity_says_what_this_is_and_is_the_same_each_time() {
let a = connection_params(None);
let b = connection_params(None);
assert!(a.device_model.starts_with("moho ("), "{}", a.device_model);
assert_eq!((a.device_model.as_str(), a.app_version.as_str()), (b.device_model.as_str(), b.app_version.as_str()));
assert_eq!(a.app_version, env!("CARGO_PKG_VERSION"));
assert!(!a.system_version.is_empty() && !a.lang_code.is_empty());
}
}
src/backend/telegram/notify.rs
+77
View File
@@ -0,0 +1,77 @@
//! Muting a conversation on the account itself.
//!
//! Telegram keeps "mute until" per chat with the account, so a chat silenced on the phone is silent here and the
//! other way round. It is read off each dialog and from the update Telegram sends when it changes, and written when
//! the window asks.
use super::peers;
use super::send::{explain, peer_for};
use crate::model::Buffer;
use crate::state::AppState;
use grammers_client::tl;
use grammers_session::types::PeerId;
/// "Forever", as Telegram's own apps write it.
const FOREVER: i32 = i32::MAX;
/// Whether notification settings say the chat is silent at `now`.
pub fn silent(settings: &tl::enums::PeerNotifySettings, now: i64) -> bool {
let tl::enums::PeerNotifySettings::Settings(s) = settings;
s.mute_until.is_some_and(|until| i64::from(until) > now)
}
/// A chat's settings changed (on another of the account's apps, or here): the buffer follows.
pub fn changed(state: &AppState, account_id: &str, update: &tl::types::UpdateNotifySettings) {
let tl::enums::NotifyPeer::Peer(p) = &update.peer else { return };
let remote = peers::remote_id(PeerId::from(p.peer.clone()));
let Some(buffer) = state.runtime.list_buffers().into_iter().find(|b| b.account_id == account_id && b.remote_id.as_deref() == Some(remote.as_str())) else { return };
state.runtime.set_silenced(state, &buffer.id, silent(&update.notify_settings, chrono::Utc::now().timestamp()));
}
/// Mutes a chat, or takes the mute off.
pub async fn set_muted(state: &AppState, buffer: &Buffer, muted: bool) -> Result<(), String> {
let (handle, peer) = peer_for(state, buffer).await?;
let settings = tl::types::InputPeerNotifySettings {
show_previews: None,
silent: None,
mute_until: Some(if muted { FOREVER } else { 0 }),
sound: None,
stories_muted: None,
stories_hide_sender: None,
stories_sound: None,
};
let peer = tl::enums::InputNotifyPeer::Peer(tl::types::InputNotifyPeer { peer: peer.into() });
handle.client.invoke(&tl::functions::account::UpdateNotifySettings { peer, settings: settings.into() }).await.map_err(|e| explain(&e))?;
state.runtime.set_silenced(state, &buffer.id, muted);
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
fn settings(mute_until: Option<i32>) -> tl::enums::PeerNotifySettings {
tl::enums::PeerNotifySettings::Settings(tl::types::PeerNotifySettings {
show_previews: None,
silent: None,
mute_until,
ios_sound: None,
android_sound: None,
other_sound: None,
stories_muted: None,
stories_hide_sender: None,
stories_ios_sound: None,
stories_android_sound: None,
stories_other_sound: None,
})
}
#[test]
fn a_chat_is_silent_until_its_mute_runs_out() {
assert!(silent(&settings(Some(FOREVER)), 1_790_000_000));
assert!(silent(&settings(Some(1_790_000_100)), 1_790_000_000));
assert!(!silent(&settings(Some(1_789_999_000)), 1_790_000_000));
assert!(!silent(&settings(Some(0)), 1_790_000_000));
assert!(!silent(&settings(None), 1_790_000_000));
}
}
src/backend/telegram/peers.rs
+267
View File
@@ -0,0 +1,267 @@
//! A Telegram conversation as a buffer: its name, its kind, its heading in the list.
//!
//! People, groups, supergroups, channels and bots are all "peers". A person (or a bot) is a direct message; a group or
//! a channel is a channel. The list is headed Direct Messages, Groups and Channels, as Twitch's is headed Channels and
//! Direct Messages (see `runtime.rs fixed_category`).
//!
//! The buffer is named for the conversation, and the buffer's name is what its history is filed under, so it must not
//! change when somebody renames themselves: a buffer is found by the peer's own id (Telegram's "bot API dialog id",
//! kept as the buffer's `remote_id`), and keeps the name it was first given. Two people with the same name are told
//! apart by their `@username`, or failing that by the end of their number.
use crate::model::Buffer;
use crate::state::AppState;
use grammers_client::peer::Peer;
use grammers_client::tl;
use grammers_session::types::PeerId;
pub const DIRECT_MESSAGES: &str = "Direct Messages";
pub const BOTS: &str = "Bots";
pub const GROUPS: &str = "Groups";
pub const CHANNELS: &str = "Channels";
/// The id Telegram's own tools (and the bot API) call a peer by: a person's number, minus a group's, and a
/// supergroup's or channel's with `-100` in front.
pub fn remote_id(peer: PeerId) -> String {
peer.bot_api_dialog_id().map(|id| id.to_string()).unwrap_or_default()
}
/// The peer a buffer's `remote_id` names.
pub fn peer_id_of(remote_id: &str) -> Option<PeerId> {
PeerId::from_bot_api_dialog_id(remote_id.parse().ok()?)
}
pub fn kind_of(peer: &Peer) -> &'static str {
match peer {
Peer::User(_) => "dm",
_ => "channel",
}
}
pub fn category_of(peer: &Peer) -> &'static str {
match peer {
// A bot is a conversation with a program, not a person: its own heading (it is still a direct chat).
Peer::User(user) if user.is_bot() => BOTS,
Peer::User(_) => DIRECT_MESSAGES,
Peer::Channel(_) => CHANNELS,
_ => GROUPS,
}
}
/// The glyph in front of a conversation's name: a horn for a broadcast channel, a robot for a bot.
pub fn name_icon_of(peer: &Peer) -> Option<&'static str> {
match peer {
Peer::Channel(channel) if channel.raw.broadcast => Some("campaign"),
Peer::User(user) if user.is_bot() => Some("smart_toy"),
_ => None,
}
}
/// What the conversation is called: the person's name, the group's title, or "Saved Messages" for the account's own.
pub fn title_of(peer: &Peer) -> String {
match peer {
Peer::User(user) if user.is_self() => "Saved Messages".to_string(),
Peer::User(user) => {
let name = user.full_name();
if !name.trim().is_empty() {
name
} else if let Some(username) = user.username() {
username.to_string()
} else {
"Deleted account".to_string()
}
}
other => other.name().filter(|n| !n.trim().is_empty()).unwrap_or("Unnamed chat").to_string(),
}
}
/// A name for a buffer that no other conversation of the account has.
pub fn unique_name(taken: &[(String, Option<String>)], remote_id: &str, title: &str, username: Option<&str>) -> String {
let used = |name: &str| taken.iter().any(|(n, r)| n == name && r.as_deref() != Some(remote_id));
if !used(title) {
return title.to_string();
}
if let Some(username) = username.filter(|u| !u.is_empty()) {
let with = format!("{title} (@{username})");
if !used(&with) {
return with;
}
}
let digits: String = remote_id.chars().filter(|c| c.is_ascii_digit()).collect();
let tail = &digits[digits.len().saturating_sub(4)..];
format!("{title} ·{tail}")
}
/// The conversation's buffer, made if it is not there yet.
pub fn buffer_for(state: &AppState, account_id: &str, peer: &Peer) -> Buffer {
let remote = remote_id(peer.id());
let mine: Vec<Buffer> = state.runtime.list_buffers().into_iter().filter(|b| b.account_id == account_id).collect();
if let Some(found) = mine.iter().find(|b| b.remote_id.as_deref() == Some(remote.as_str())) {
return found.clone();
}
let taken: Vec<(String, Option<String>)> = mine.iter().map(|b| (b.name.clone(), b.remote_id.clone())).collect();
let name = unique_name(&taken, &remote, &title_of(peer), peer.username());
let buffer = state.runtime.ensure_buffer(state, account_id, &name, kind_of(peer));
state.runtime.set_buffer_remote_id(state, &buffer.id, &remote);
state.runtime.set_buffer_category(state, &buffer.id, Some(category_of(peer)), 0);
state.runtime.set_buffer_name_icon(state, &buffer.id, name_icon_of(peer));
state.runtime.get_buffer(&buffer.id).unwrap_or(buffer)
}
/// What the window says in place of the message box where the account may not write: the words every service uses.
pub const NO_SEND_PERMISSION: &str = "You do not have permission to send messages in this channel.";
/// What decides whether an account may write in a group or a channel, read off what Telegram says about it.
#[derive(Default, Clone, Debug, PartialEq)]
pub struct Rights {
/// A broadcast channel, where only those allowed to post may write.
pub broadcast: bool,
pub creator: bool,
/// An administrator (of any kind) of this chat.
pub admin: bool,
/// An administrator who may post in a broadcast channel.
pub may_post: bool,
/// The account has left, or was removed.
pub gone: bool,
/// The chat was upgraded to a supergroup and no longer takes messages.
pub deactivated: bool,
/// This account in particular may not send.
pub banned_here: bool,
/// Nobody but administrators may send.
pub members_banned: bool,
}
/// Why the account cannot write in a chat, or `None` when it can.
pub fn read_only_reason(r: &Rights) -> Option<&'static str> {
if r.gone {
return Some("You are not a member of this chat.");
}
if r.deactivated {
return Some("This group was upgraded and no longer takes messages.");
}
let boss = r.creator || r.admin;
if r.broadcast {
return (!(r.creator || r.may_post)).then_some(NO_SEND_PERMISSION);
}
if r.banned_here || (r.members_banned && !boss) {
return Some(NO_SEND_PERMISSION);
}
None
}
fn sending_banned(rights: &Option<tl::enums::ChatBannedRights>, now: i64) -> bool {
match rights {
Some(tl::enums::ChatBannedRights::Rights(b)) => b.send_messages && (b.until_date == 0 || i64::from(b.until_date) > now),
None => false,
}
}
fn is_admin(rights: &Option<tl::enums::ChatAdminRights>) -> bool {
rights.is_some()
}
fn may_post(rights: &Option<tl::enums::ChatAdminRights>) -> bool {
matches!(rights, Some(tl::enums::ChatAdminRights::Rights(a)) if a.post_messages)
}
fn rights_of_channel(c: &tl::types::Channel, now: i64) -> Rights {
Rights {
broadcast: c.broadcast,
creator: c.creator,
admin: is_admin(&c.admin_rights),
may_post: may_post(&c.admin_rights),
gone: c.left,
deactivated: false,
banned_here: sending_banned(&c.banned_rights, now),
members_banned: sending_banned(&c.default_banned_rights, now),
}
}
/// What the account may do in a conversation. Only meaningful for a peer read from the dialog list: one that arrived
/// with a message may be a "min" copy without the account's rights in it.
pub fn rights_of(peer: &Peer, now: i64) -> Option<Rights> {
match peer {
Peer::Channel(channel) => Some(rights_of_channel(&channel.raw, now)),
Peer::Group(group) => Some(match &group.raw {
tl::enums::Chat::Chat(c) => Rights {
creator: c.creator,
admin: is_admin(&c.admin_rights),
gone: c.left,
deactivated: c.deactivated,
members_banned: sending_banned(&c.default_banned_rights, now),
..Default::default()
},
tl::enums::Chat::Channel(c) => rights_of_channel(c, now),
// A chat that is gone, or that this account cannot see.
_ => Rights { gone: true, ..Default::default() },
}),
Peer::User(user) if user.id().bare_id_unchecked() == 777000 => None,
_ => None,
}
}
/// The reason the window gives for a message box it does not show, for a conversation read from the dialog list.
pub fn read_only_for(peer: &Peer, now: i64) -> Option<&'static str> {
if let Peer::User(user) = peer {
// Telegram's own service account sends its login codes and takes no reply.
return (user.id().bare_id_unchecked() == 777000).then_some("Telegram does not take replies here.");
}
rights_of(peer, now).and_then(|r| read_only_reason(&r))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn two_people_with_one_name_are_told_apart_and_a_rename_never_makes_a_second_buffer() {
let taken = vec![("Anna".to_string(), Some("111".to_string()))];
assert_eq!(unique_name(&taken, "222", "Anna", None), "Anna ·222");
assert_eq!(unique_name(&taken, "222", "Anna", Some("anna_k")), "Anna (@anna_k)");
// The same person asking again is not a clash with themselves.
assert_eq!(unique_name(&taken, "111", "Anna", None), "Anna");
assert_eq!(unique_name(&taken, "333", "Boris", None), "Boris");
assert_eq!(unique_name(&taken, "123456789", "Anna", None), "Anna ·6789");
}
#[test]
fn a_broadcast_channel_takes_messages_only_from_those_allowed_to_post() {
let feed = Rights { broadcast: true, ..Default::default() };
assert_eq!(read_only_reason(&feed), Some(NO_SEND_PERMISSION));
assert_eq!(read_only_reason(&Rights { admin: true, ..feed.clone() }), Some(NO_SEND_PERMISSION), "an administrator who may not post is a reader here");
assert_eq!(read_only_reason(&Rights { admin: true, may_post: true, ..feed.clone() }), None);
assert_eq!(read_only_reason(&Rights { creator: true, ..feed }), None);
}
#[test]
fn a_group_takes_messages_unless_this_account_or_every_member_is_banned_from_sending() {
assert_eq!(read_only_reason(&Rights::default()), None);
assert_eq!(read_only_reason(&Rights { banned_here: true, ..Default::default() }), Some(NO_SEND_PERMISSION));
assert_eq!(read_only_reason(&Rights { members_banned: true, ..Default::default() }), Some(NO_SEND_PERMISSION));
assert_eq!(read_only_reason(&Rights { members_banned: true, admin: true, ..Default::default() }), None, "administrators may still send where members may not");
assert!(read_only_reason(&Rights { gone: true, ..Default::default() }).unwrap().contains("not a member"));
assert!(read_only_reason(&Rights { deactivated: true, ..Default::default() }).unwrap().contains("upgraded"));
}
#[test]
fn a_ban_on_sending_that_has_run_out_is_not_one() {
use tl::types::ChatBannedRights as B;
let banned = |until| Some(tl::enums::ChatBannedRights::Rights(B { send_messages: true, until_date: until, view_messages: false, send_media: false, send_stickers: false, send_gifs: false, send_games: false, send_inline: false, embed_links: false, send_polls: false, change_info: false, invite_users: false, pin_messages: false, manage_topics: false, send_photos: false, send_videos: false, send_roundvideos: false, send_audios: false, send_voices: false, send_docs: false, send_plain: false, edit_rank: false, manage_linked_peers: false, send_reactions: false }));
assert!(sending_banned(&banned(0), 1000), "no end is for ever");
assert!(sending_banned(&banned(2000), 1000));
assert!(!sending_banned(&banned(500), 1000));
assert!(!sending_banned(&None, 1000));
}
#[test]
fn an_id_goes_to_the_buffer_and_back() {
let user = PeerId::user_unchecked(5173409493);
assert_eq!(remote_id(user), "5173409493");
assert_eq!(peer_id_of("5173409493"), Some(user));
let channel = PeerId::channel_unchecked(1234567890);
assert_eq!(remote_id(channel), "-1001234567890");
assert_eq!(peer_id_of("-1001234567890"), Some(channel));
assert_eq!(peer_id_of("not a number"), None);
}
}
src/backend/telegram/pins.rs
+49
View File
@@ -0,0 +1,49 @@
//! A conversation's pinned messages: listed, pinned and unpinned.
//!
//! Telegram keeps the list on its side, so it is asked for when somebody opens the pins and again when an update says
//! the pins changed. The answer has the shape the other services' pins have, so the window's banner and list work
//! the same.
use super::messages;
use super::send::{explain, peer_for};
use crate::model::Buffer;
use crate::state::AppState;
use grammers_client::tl;
use serde_json::{json, Value};
/// How many pins are read: a conversation with more has more than anybody is going to scroll.
const LIMIT: usize = 50;
pub async fn list(state: &AppState, buffer: &Buffer) -> Result<Value, String> {
let (handle, peer) = peer_for(state, buffer).await?;
let mut found = handle.client.search_messages(peer).filter(tl::enums::MessagesFilter::InputMessagesFilterPinned);
let mut pinned: Vec<Value> = Vec::new();
while pinned.len() < LIMIT {
let Some(message) = found.next().await.map_err(|e| explain(&e))? else { break };
let from = if message.outgoing() { handle.own.name.clone() } else { message.sender().map(messages::name_of).unwrap_or_else(|| buffer.name.clone()) };
let body = messages::body_of(&message);
pinned.push(json!({
"id": message.id().to_string(),
"bufferId": buffer.id,
"from": from,
"body": if body.trim().is_empty() { "(no text)".to_string() } else { body },
"ts": message.date().as_second(),
"kind": "chat",
}));
}
let ids: Vec<String> = pinned.iter().filter_map(|m| m["id"].as_str().map(str::to_string)).collect();
// Said as well as answered, so the header's count is right in every window watching this conversation.
state.events.emit("pinnedMessages", json!({ "bufferId": buffer.id, "pinned": ids }));
Ok(json!({ "bufferId": buffer.id, "pinned": pinned }))
}
pub async fn set(state: &AppState, buffer: &Buffer, msg_id: &str, pinned: bool) -> Result<(), String> {
let id: i32 = msg_id.parse().map_err(|_| "that message cannot be pinned".to_string())?;
let (handle, peer) = peer_for(state, buffer).await?;
if pinned {
handle.client.pin_message(peer, id).await.map_err(|e| explain(&e))?;
} else {
handle.client.unpin_message(peer, id).await.map_err(|e| explain(&e))?;
}
Ok(())
}
src/backend/telegram/polls.rs
+280
View File
@@ -0,0 +1,280 @@
//! Polls: a question with answers, shown as the card over the conversation and voted in from it.
//!
//! A poll is part of a message. When one that is still open is read (live or from history) it is announced as the
//! conversation's poll card; Telegram sends an update whenever the tally moves, and the card is drawn again. Only the
//! newest open poll has the card (a conversation has one slot for it); a closed poll stays in the log as its label.
use super::send::{explain, peer_for};
use crate::model::Buffer;
use crate::state::AppState;
use grammers_client::tl;
use serde_json::{json, Value};
use std::collections::HashMap;
use std::sync::Mutex;
/// Which message each poll is, by Telegram's id for the poll, for an update that names only the poll.
fn registry() -> &'static Mutex<HashMap<i64, (String, i32)>> {
static R: std::sync::OnceLock<Mutex<HashMap<i64, (String, i32)>>> = std::sync::OnceLock::new();
R.get_or_init(Default::default)
}
fn text_of(t: &tl::enums::TextWithEntities) -> String {
let tl::enums::TextWithEntities::Entities(t) = t;
t.text.clone()
}
/// An answer's id as text: Telegram names an answer by a few bytes.
pub fn option_id(bytes: &[u8]) -> String {
hex::encode(bytes)
}
/// The card for a poll, from its definition and its tally.
pub fn card(buffer_id: &str, msg_id: i32, poll: &tl::types::Poll, results: &tl::types::PollResults) -> Value {
let tally: HashMap<Vec<u8>, (i32, bool)> = results
.results
.iter()
.flatten()
.map(|r| {
let tl::enums::PollAnswerVoters::Voters(v) = r;
(v.option.clone(), (v.voters.unwrap_or(0), v.chosen))
})
.collect();
let mut voted: Option<String> = None;
let options: Vec<Value> = poll
.answers
.iter()
.map(|a| {
let tl::enums::PollAnswer::Answer(a) = a else { return json!({}) };
let (votes, chosen) = tally.get(&a.option).copied().unwrap_or((0, false));
if chosen {
voted = Some(option_id(&a.option));
}
json!({ "id": option_id(&a.option), "label": text_of(&a.text), "votes": votes })
})
.collect();
json!({
"bufferId": buffer_id,
"kind": "poll",
"id": msg_id.to_string(),
"title": text_of(&poll.question),
"options": options,
// A poll has no clock the card could count: it is open until it is closed.
"open": !poll.closed,
"duration": 0,
"remaining": 0,
"resultDisplayDuration": 0,
"hasVoted": voted.is_some(),
"votedOptionId": voted,
"noEnd": true,
})
}
/// Shows a poll as the conversation's card, and remembers which message it is.
pub fn announce(state: &AppState, buffer_id: &str, msg_id: i32, poll: &tl::types::Poll, results: &tl::types::PollResults) {
registry().lock().unwrap_or_else(|e| e.into_inner()).insert(poll.id, (buffer_id.to_string(), msg_id));
if poll.closed {
return;
}
let card = card(buffer_id, msg_id, poll, results);
state.runtime.set_live_card(buffer_id, "poll", card.clone());
let ts = chrono::Utc::now().timestamp();
if let Err(e) = state.store.record_live_card(buffer_id, "poll", &msg_id.to_string(), card["title"].as_str().unwrap_or(""), &card.to_string(), ts) {
tracing::debug!("telegram: keeping the poll: {e:#}");
}
state.events.emit("pollCard", json!({ "bufferId": buffer_id, "kind": "poll", "poll": card }));
}
/// The tally of a poll moved.
pub fn update(state: &AppState, update: &tl::types::UpdateMessagePoll) {
let Some((buffer_id, msg_id)) = registry().lock().unwrap_or_else(|e| e.into_inner()).get(&update.poll_id).cloned() else { return };
let tl::enums::PollResults::Results(results) = &update.results;
// A poll that closed is drawn as it ended; one whose definition came with the update is redrawn from it.
match &update.poll {
Some(tl::enums::Poll::Poll(poll)) => announce_any(state, &buffer_id, msg_id, poll, results),
None => {
if let Some(mut live) = state.runtime.live_card(&buffer_id, "poll").filter(|c| c["id"].as_str() == Some(&msg_id.to_string())) {
let mut chosen_id: Option<String> = None;
if let Some(options) = live["options"].as_array_mut() {
for option in options.iter_mut() {
let id = option["id"].as_str().unwrap_or("").to_string();
let found = results.results.iter().flatten().find_map(|r| {
let tl::enums::PollAnswerVoters::Voters(v) = r;
(option_id(&v.option) == id).then_some((v.voters.unwrap_or(0), v.chosen))
});
if let Some((votes, chosen)) = found {
option["votes"] = json!(votes);
if chosen {
chosen_id = Some(id);
}
}
}
}
if let Some(id) = chosen_id {
live["votedOptionId"] = json!(id);
live["hasVoted"] = json!(true);
}
state.runtime.set_live_card(&buffer_id, "poll", live.clone());
state.events.emit("pollCard", json!({ "bufferId": buffer_id, "kind": "poll", "poll": live }));
}
}
}
}
/// Redraws a poll that may have closed: the card says so rather than vanishing.
fn announce_any(state: &AppState, buffer_id: &str, msg_id: i32, poll: &tl::types::Poll, results: &tl::types::PollResults) {
let card = card(buffer_id, msg_id, poll, results);
state.runtime.set_live_card(buffer_id, "poll", card.clone());
state.events.emit("pollCard", json!({ "bufferId": buffer_id, "kind": "poll", "poll": card }));
}
/// Votes for an answer in the poll the conversation's card shows.
pub async fn vote(state: &AppState, buffer: &Buffer, answer: &str) -> Result<(), String> {
let card = state.runtime.live_card(&buffer.id, "poll").ok_or_else(|| "there is no poll open here".to_string())?;
let msg_id: i32 = card["id"].as_str().and_then(|i| i.parse().ok()).ok_or_else(|| "that poll cannot be voted in".to_string())?;
let option = hex::decode(answer).map_err(|_| "that is not one of the poll's answers".to_string())?;
let (handle, peer) = peer_for(state, buffer).await?;
handle.client.invoke(&tl::functions::messages::SendVote { peer: peer.into(), msg_id, options: vec![option] }).await.map_err(|e| explain(&e))?;
// Said at once; Telegram's own update with the new tally follows.
let mut live = card;
live["hasVoted"] = json!(true);
live["votedOptionId"] = json!(answer);
if let Some(options) = live["options"].as_array_mut() {
for o in options.iter_mut() {
if o["id"].as_str() == Some(answer) {
o["votes"] = json!(o["votes"].as_i64().unwrap_or(0) + 1);
}
}
}
state.runtime.set_live_card(&buffer.id, "poll", live.clone());
state.events.emit("pollCard", json!({ "bufferId": buffer.id, "kind": "poll", "poll": live }));
Ok(())
}
/// Telegram's limits on a poll: a question of 255 characters, answers of 100, two to ten of them.
const QUESTION_MAX: usize = 255;
const ANSWER_MAX: usize = 100;
const ANSWERS: std::ops::RangeInclusive<usize> = 2..=10;
/// What a poll somebody is starting is, checked and as Telegram's own type. `disclosed` is whether the running
/// count shows before it closes.
pub fn build(question: &str, answers: &[String], disclosed: bool) -> Result<tl::types::Poll, String> {
let question = question.trim();
let answers: Vec<&str> = answers.iter().map(|a| a.trim()).filter(|a| !a.is_empty()).collect();
if question.is_empty() {
return Err("A poll needs a question".to_string());
}
if question.chars().count() > QUESTION_MAX {
return Err(format!("The question can be at most {QUESTION_MAX} characters"));
}
if !ANSWERS.contains(&answers.len()) {
return Err("Telegram polls have between 2 and 10 answers".to_string());
}
if answers.iter().any(|a| a.chars().count() > ANSWER_MAX) {
return Err(format!("An answer can be at most {ANSWER_MAX} characters"));
}
let plain = |t: &str| tl::enums::TextWithEntities::Entities(tl::types::TextWithEntities { text: t.to_string(), entities: Vec::new() });
Ok(tl::types::Poll {
id: 0,
closed: false,
// Anonymous, as Telegram's own apps start one.
public_voters: false,
multiple_choice: false,
quiz: false,
open_answers: false,
revoting_disabled: false,
shuffle_answers: false,
hide_results_until_close: !disclosed,
creator: false,
subscribers_only: false,
question: plain(question),
answers: answers.iter().enumerate().map(|(i, a)| tl::enums::PollAnswer::Answer(tl::types::PollAnswer { text: plain(a), option: vec![i as u8], media: None, added_by: None, date: None })).collect(),
close_period: None,
close_date: None,
hash: 0,
countries_iso2: None,
})
}
/// Starts a poll in a conversation.
pub async fn start(state: &AppState, buffer: &Buffer, question: &str, answers: &[String], disclosed: bool) -> Result<(), String> {
let poll = build(question, answers, disclosed)?;
let (handle, peer) = peer_for(state, buffer).await?;
let media = tl::types::InputMediaPoll { poll: tl::enums::Poll::Poll(poll), correct_answers: None, attached_media: None, solution: None, solution_entities: None, solution_media: None };
let input = grammers_client::message::InputMessage::new().media(tl::enums::InputMedia::Poll(Box::new(media)));
let sent = handle.client.send_message(peer, input).await.map_err(|e| explain(&e))?;
super::messages::record(state, &buffer.account_id, &handle.own, &sent);
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
fn text(s: &str) -> tl::enums::TextWithEntities {
tl::enums::TextWithEntities::Entities(tl::types::TextWithEntities { text: s.into(), entities: vec![] })
}
fn answer(label: &str, option: &[u8]) -> tl::enums::PollAnswer {
tl::enums::PollAnswer::Answer(tl::types::PollAnswer { text: text(label), option: option.to_vec(), media: None, added_by: None, date: None })
}
fn poll(closed: bool) -> tl::types::Poll {
tl::types::Poll {
id: 77,
closed,
public_voters: false,
multiple_choice: false,
quiz: false,
open_answers: false,
revoting_disabled: false,
shuffle_answers: false,
hide_results_until_close: false,
creator: false,
subscribers_only: false,
question: text("Lunch?"),
answers: vec![answer("Pizza", b"0"), answer("Soup", b"1")],
close_period: None,
close_date: None,
hash: 0,
countries_iso2: None,
}
}
fn voters(option: &[u8], n: i32, chosen: bool) -> tl::enums::PollAnswerVoters {
tl::enums::PollAnswerVoters::Voters(tl::types::PollAnswerVoters { chosen, correct: false, option: option.to_vec(), voters: Some(n), recent_voters: None })
}
#[test]
fn a_poll_that_is_started_is_checked() {
let a = |n: &[&str]| n.iter().map(|s| s.to_string()).collect::<Vec<_>>();
let ok = build(" Lunch? ", &a(&["Pizza", " Soup ", ""]), false).unwrap();
assert_eq!(ok.answers.len(), 2, "an empty answer is not one");
assert!(ok.hide_results_until_close);
assert!(!ok.public_voters);
assert!(build("", &a(&["a", "b"]), true).is_err());
assert!(build("Q", &a(&["only"]), true).is_err());
assert!(build("Q", &a(&["1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "11"]), true).is_err());
assert!(build(&"x".repeat(256), &a(&["a", "b"]), true).is_err());
assert!(!build("Q", &a(&["a", "b"]), true).unwrap().hide_results_until_close);
}
#[test]
fn a_poll_becomes_a_card_with_counts_and_the_accounts_own_choice() {
let results = tl::types::PollResults { min: false, has_unread_votes: false, can_view_stats: false, results: Some(vec![voters(b"0", 3, false), voters(b"1", 5, true)]), total_voters: Some(8), recent_voters: None, solution: None, solution_entities: None, solution_media: None };
let card = card("telegram:1|Lunch", 42, &poll(false), &results);
assert_eq!((card["title"].as_str(), card["id"].as_str(), card["open"].as_bool()), (Some("Lunch?"), Some("42"), Some(true)));
assert_eq!(card["options"][0]["votes"], 3);
assert_eq!(card["options"][1]["votes"], 5);
assert_eq!((card["hasVoted"].as_bool(), card["votedOptionId"].as_str()), (Some(true), Some(option_id(b"1").as_str())));
let closed = super::card("b", 1, &poll(true), &results);
assert_eq!(closed["open"], false);
}
#[test]
fn a_poll_nobody_voted_in_has_zeros_and_no_choice() {
let results = tl::types::PollResults { min: false, has_unread_votes: false, can_view_stats: false, results: None, total_voters: None, recent_voters: None, solution: None, solution_entities: None, solution_media: None };
let card = card("b", 1, &poll(false), &results);
assert_eq!(card["options"][0]["votes"], 0);
assert_eq!(card["hasVoted"], false);
}
}
src/backend/telegram/profile.rs
+111
View File
@@ -0,0 +1,111 @@
//! The card for a person: their name, handle, picture, what they wrote about themselves and when they were last seen.
use super::media;
use super::messages;
use super::peers;
use super::send::{explain, peer_for};
use crate::profile;
use crate::state::AppState;
use grammers_client::peer::{Peer, User};
use grammers_client::tl;
use grammers_session::Session;
use serde_json::{json, Value};
/// What they were last seen doing, in the card's words, from Telegram's coarse status.
pub fn presence(status: &tl::enums::UserStatus) -> (Option<&'static str>, Option<i64>) {
use tl::enums::UserStatus as S;
match status {
S::Online(_) => (Some("online"), None),
S::Offline(o) => (Some("offline"), Some(i64::from(o.was_online))),
S::Recently(_) => (Some("offline"), None),
S::LastWeek(_) | S::LastMonth(_) => (Some("offline"), None),
S::Empty => (None, None),
}
}
/// How Telegram words its coarse "last seen" when it will not give a time.
fn coarse(status: &tl::enums::UserStatus) -> Option<&'static str> {
use tl::enums::UserStatus as S;
match status {
S::Recently(_) => Some("recently"),
S::LastWeek(_) => Some("within a week"),
S::LastMonth(_) => Some("within a month"),
_ => None,
}
}
/// The card for the person a conversation is with, or a member of a group the window asked about.
pub async fn build(state: &AppState, account_id: &str, buffer_id: &str, who: &str, user_id: &str) -> Value {
let mut card = profile::pending("telegram", account_id, who);
card["pending"] = json!(false);
let Some(buffer) = state.runtime.get_buffer(buffer_id) else { return card };
// A direct message is with one person; in a group the member asked about is named by number.
let target = if buffer.kind == "dm" { buffer.remote_id.clone() } else { Some(user_id.to_string()) };
let Some(target) = target.and_then(|t| peers::peer_id_of(&t)) else { return card };
let Ok((handle, _)) = peer_for(state, &buffer).await else { return card };
let Some(peer) = handle.session.peer_ref(target).await.ok().flatten() else { return card };
card["id"] = json!(target.bare_id_unchecked().to_string());
let full = handle.client.invoke(&tl::functions::users::GetFullUser { id: peer.into() }).await;
let tl::enums::users::UserFull::Full(full) = match full {
Ok(f) => f,
Err(e) => {
tracing::debug!("telegram profile: {}", explain(&e));
return card;
}
};
let tl::enums::UserFull::Full(info) = &full.full_user;
if let Some(user) = full.users.iter().find(|u| u.id() == target.bare_id_unchecked()).cloned() {
let user = User::from_raw(&handle.client, user);
let as_peer = Peer::User(user.clone());
card["name"] = json!(messages::name_of(&as_peer));
if let Some(username) = user.username() {
card["handle"] = json!(format!("@{username}"));
}
profile::set(&mut card, "avatarUrl", media::avatar_url(handle.own.user_id, &as_peer).map(|u| json!(u)));
let (status, seen) = presence(user.status());
profile::set(&mut card, "status", status.map(|s| json!(s)));
profile::set(&mut card, "lastActiveTs", seen.map(|t| json!(t)));
if let Some(words) = coarse(user.status()) {
profile::note(&mut card, "Last seen", words);
}
if user.is_bot() {
profile::note(&mut card, "Account", "A bot");
}
if user.verified() {
profile::note(&mut card, "Account", "Verified by Telegram");
}
if let Some(phone) = user.phone() {
profile::note(&mut card, "Phone", format!("+{phone}"));
}
}
if let Some(about) = &info.about {
profile::note(&mut card, "About", about.clone());
}
if let Some(tl::enums::Birthday::Birthday(b)) = &info.birthday {
profile::note(&mut card, "Birthday", match b.year {
Some(year) => format!("{}.{}.{year}", b.day, b.month),
None => format!("{}.{}", b.day, b.month),
});
}
if info.common_chats_count > 0 {
profile::note(&mut card, "Groups in common", info.common_chats_count.to_string());
}
card
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn last_seen_is_a_time_when_telegram_gives_one_and_a_word_when_it_does_not() {
let offline = tl::enums::UserStatus::Offline(tl::types::UserStatusOffline { was_online: 1790000000 });
assert_eq!(presence(&offline), (Some("offline"), Some(1790000000)));
assert_eq!(presence(&tl::enums::UserStatus::Online(tl::types::UserStatusOnline { expires: 1 })), (Some("online"), None));
let recently = tl::enums::UserStatus::Recently(tl::types::UserStatusRecently { by_me: false });
assert_eq!(presence(&recently), (Some("offline"), None));
assert_eq!(coarse(&recently), Some("recently"));
assert_eq!(coarse(&offline), None);
assert_eq!(presence(&tl::enums::UserStatus::Empty), (None, None));
}
}
src/backend/telegram/reactions.rs
+138
View File
@@ -0,0 +1,138 @@
//! Reactions: the emoji people put on a message, and the account's own.
//!
//! Telegram sends a message with its reactions as a list of (reaction, count, whether the account chose it), and an
//! update when they change. A reaction is an ordinary emoji, a custom emoji (drawn from Telegram's sticker sets,
//! which moho does not draw yet, so it is counted under a star) or a paid star.
use super::send::{explain, peer_for};
use crate::model::{Buffer, Reaction};
use crate::state::AppState;
use grammers_client::message::InputReactions;
use grammers_client::tl;
/// What stands for a reaction moho cannot draw: custom emoji and paid stars.
const STAND_IN: &str = "⭐";
/// A message's reactions as moho's list.
pub fn from_tl(reactions: &tl::enums::MessageReactions) -> Vec<Reaction> {
let tl::enums::MessageReactions::Reactions(r) = reactions;
let mut out: Vec<Reaction> = Vec::new();
for count in &r.results {
let tl::enums::ReactionCount::Count(c) = count;
let emoji = match &c.reaction {
tl::enums::Reaction::Emoji(e) => e.emoticon.clone(),
tl::enums::Reaction::CustomEmoji(_) | tl::enums::Reaction::Paid => STAND_IN.to_string(),
tl::enums::Reaction::Empty => continue,
};
// Two kinds that share the stand-in are one pill.
if let Some(existing) = out.iter_mut().find(|x| x.emoji == emoji) {
existing.count += i64::from(c.count);
existing.me |= c.chosen_order.is_some();
} else {
out.push(Reaction { emoji, count: i64::from(c.count), me: c.chosen_order.is_some(), animated: false });
}
}
out
}
/// Records what a message's reactions are now, and says so.
pub fn apply(state: &AppState, buffer_id: &str, msg_id: &str, reactions: &[Reaction]) {
match state.store.set_reactions(buffer_id, msg_id, reactions) {
Ok(true) => state.events.emit("reactionsChanged", serde_json::json!({ "bufferId": buffer_id, "id": msg_id, "reactions": reactions })),
Ok(false) => {}
Err(e) => tracing::warn!("telegram: recording reactions: {e}"),
}
}
/// The reactions the account itself has put on a message, as Telegram's own type.
fn chosen(reactions: &[Reaction]) -> Vec<tl::enums::Reaction> {
reactions.iter().filter(|r| r.me && r.emoji != STAND_IN).map(|r| tl::enums::Reaction::Emoji(tl::types::ReactionEmoji { emoticon: r.emoji.clone() })).collect()
}
/// Puts an emoji on a message or takes the account's off. Telegram lets an ordinary account have one reaction on a
/// message, so adding replaces the account's earlier one.
pub async fn toggle(state: &AppState, buffer: &Buffer, msg_id: &str, emoji: &str, add: bool) -> Result<(), String> {
let id: i32 = msg_id.parse().map_err(|_| "that message cannot take reactions".to_string())?;
let (handle, peer) = peer_for(state, buffer).await?;
let stored = state.store.get_message(&buffer.id, msg_id).ok().flatten().map(|m| m.reactions).unwrap_or_default();
let stored_again = stored.clone();
let mut mine = chosen(&stored);
let this = tl::enums::Reaction::Emoji(tl::types::ReactionEmoji { emoticon: emoji.to_string() });
if add {
mine = vec![this];
} else {
mine.retain(|r| *r != this);
}
let input: InputReactions = if mine.is_empty() { InputReactions::remove() } else { mine.into() };
handle.client.send_reactions(peer, id, input).await.map_err(|e| explain(&e))?;
// Telegram answers with the message's new state as an update, which is read below; this is the same fact said
// at once so the pill does not wait on it.
apply(state, &buffer.id, msg_id, &after_change(&stored_again, emoji, add));
Ok(())
}
/// The reactions a message has once the account has put an emoji on it (replacing its earlier one) or taken it off.
pub fn after_change(stored: &[Reaction], emoji: &str, add: bool) -> Vec<Reaction> {
let mut next: Vec<Reaction> = stored.to_vec();
for r in next.iter_mut() {
let own_ordinary = r.me && r.emoji != STAND_IN;
if own_ordinary && (add || r.emoji == emoji) {
r.me = false;
r.count -= 1;
}
}
next.retain(|r| r.count > 0);
if add {
match next.iter_mut().find(|r| r.emoji == emoji) {
Some(r) => {
r.count += 1;
r.me = true;
}
None => next.push(Reaction { emoji: emoji.to_string(), count: 1, me: true, animated: false }),
}
}
next
}
/// A reactions update from Telegram.
pub fn update(state: &AppState, account_id: &str, remote: &str, msg_id: i32, reactions: &tl::enums::MessageReactions) {
let Some(buffer) = state.runtime.list_buffers().into_iter().find(|b| b.account_id == account_id && b.remote_id.as_deref() == Some(remote)) else { return };
apply(state, &buffer.id, &msg_id.to_string(), &from_tl(reactions));
}
#[cfg(test)]
mod tests {
use super::*;
fn count(reaction: tl::enums::Reaction, n: i32, chosen: Option<i32>) -> tl::enums::ReactionCount {
tl::enums::ReactionCount::Count(tl::types::ReactionCount { chosen_order: chosen, reaction, count: n })
}
fn emoji(e: &str) -> tl::enums::Reaction {
tl::enums::Reaction::Emoji(tl::types::ReactionEmoji { emoticon: e.into() })
}
#[test]
fn reactions_become_pills_with_counts_and_whether_the_account_chose_them() {
let results = vec![count(emoji("👍"), 5, Some(0)), count(emoji("🔥"), 2, None), count(tl::enums::Reaction::CustomEmoji(tl::types::ReactionCustomEmoji { document_id: 9 }), 3, None), count(tl::enums::Reaction::Paid, 4, None)];
let reactions = tl::enums::MessageReactions::Reactions(tl::types::MessageReactions { min: false, can_see_list: false, reactions_as_tags: false, results, recent_reactions: None, top_reactors: None });
let got = from_tl(&reactions);
assert_eq!(got.iter().map(|r| (r.emoji.as_str(), r.count, r.me)).collect::<Vec<_>>(), vec![("👍", 5, true), ("🔥", 2, false), ("⭐", 7, false)], "two kinds drawn the same are one pill");
}
#[test]
fn choosing_an_emoji_replaces_the_accounts_earlier_one_and_taking_it_off_leaves_the_rest() {
let r = |e: &str, n: i64, me: bool| Reaction { emoji: e.into(), count: n, me, animated: false };
let stored = vec![r("👍", 5, true), r("🔥", 2, false)];
assert_eq!(after_change(&stored, "🔥", true), vec![r("👍", 4, false), r("🔥", 3, true)]);
assert_eq!(after_change(&stored, "👍", false), vec![r("👍", 4, false), r("🔥", 2, false)]);
assert_eq!(after_change(&[r("👍", 1, true)], "👍", false), Vec::<Reaction>::new(), "a pill nobody has any more goes");
assert_eq!(after_change(&[], "😂", true), vec![r("😂", 1, true)]);
}
#[test]
fn only_the_accounts_own_ordinary_reactions_are_carried_into_a_change() {
let stored = vec![Reaction { emoji: "👍".into(), count: 5, me: true, animated: false }, Reaction { emoji: "🔥".into(), count: 2, me: false, animated: false }, Reaction { emoji: STAND_IN.into(), count: 1, me: true, animated: false }];
assert_eq!(chosen(&stored), vec![emoji("👍")]);
}
}
src/backend/telegram/search.rs
+56
View File
@@ -0,0 +1,56 @@
//! Searching on Telegram's side: in one conversation or in every one the account has.
//!
//! The window searches its own copy first and shows that at once; this adds what the account's history has that this
//! client never read. A hit is answered as a message, and nothing is recorded: pressing one reads the messages around
//! it (see `history::context`) the way a jump to anything old does.
use super::media;
use super::messages;
use super::send::{explain, peer_for};
use crate::model::Buffer;
use crate::state::AppState;
use grammers_client::message::Message;
use serde_json::{json, Value};
/// One hit as the window reads a message.
fn hit(state: &AppState, account_id: &str, own_name: &str, own_id: i64, message: &Message) -> Option<Value> {
let peer = message.peer()?;
let buffer = super::peers::buffer_for(state, account_id, peer);
let from = if message.outgoing() { own_name.to_string() } else { message.sender().map(messages::name_of).unwrap_or_else(|| super::peers::title_of(peer)) };
let remote = buffer.remote_id.clone().unwrap_or_default();
Some(json!({
"id": message.id().to_string(),
"bufferId": buffer.id,
"from": from,
"body": messages::body_of(message),
"ts": message.date().as_second(),
"kind": "chat",
"isAction": false,
"isHighlight": false,
"edited": false,
"isOwn": message.outgoing(),
"attachments": media::attachments_of(own_id, &remote, message),
}))
}
/// The matches for a query, in this conversation or across the account (`account` scope).
pub async fn run(state: &AppState, buffer: &Buffer, query: &str, account_wide: bool, limit: usize) -> Result<Value, String> {
let (handle, peer) = peer_for(state, buffer).await?;
let mut results: Vec<Value> = Vec::new();
if account_wide {
let mut found = handle.client.search_all_messages().query(query);
while results.len() < limit {
let Some(message) = found.next().await.map_err(|e| explain(&e))? else { break };
results.extend(hit(state, &buffer.account_id, &handle.own.name, handle.own.user_id, &message));
}
} else {
let mut found = handle.client.search_messages(peer).query(query);
while results.len() < limit {
let Some(message) = found.next().await.map_err(|e| explain(&e))? else { break };
results.extend(hit(state, &buffer.account_id, &handle.own.name, handle.own.user_id, &message));
}
}
let names: std::collections::HashMap<String, String> = state.runtime.list_buffers().into_iter().filter(|b| b.account_id == buffer.account_id).map(|b| (b.id, b.name)).collect();
let count = results.len();
Ok(json!({ "results": results, "roomNames": names, "count": count }))
}
src/backend/telegram/send.rs
+304
View File
@@ -0,0 +1,304 @@
//! Saying things in a conversation: send, answer, edit, delete, "typing", "read".
//!
//! Each starts from the buffer: its `remote_id` names the peer, and the session turns that into the reference
//! Telegram wants with it (an id with the authority to use it). A message that was sent is recorded here as it is
//! returned, because Telegram does not send an account its own message back as an update.
use super::markup;
use super::messages;
use super::peers;
use super::Handle;
use crate::model::Buffer;
use crate::state::AppState;
use crate::upload::{Phase, Progress};
use grammers_client::media::{InputMedia, Uploaded};
use grammers_client::message::InputMessage;
use grammers_client::{tl, InvocationError};
use grammers_session::types::PeerRef;
use grammers_session::Session;
/// What Telegram allows in one message, counted the way it counts (UTF-16 units).
pub const MAX_UNITS: usize = 4096;
/// Telegram's refusal, said so a person can act on it.
pub fn explain(error: &InvocationError) -> String {
match error {
InvocationError::Rpc(rpc) => {
let n = rpc.value.unwrap_or(0);
match rpc.name.as_str() {
"FLOOD_WAIT" | "FLOOD_PREMIUM_WAIT" => format!("Telegram says to wait {n} seconds before doing that again"),
"SLOWMODE_WAIT" => format!("Slow mode is on here: wait {n} seconds between messages"),
"CHAT_WRITE_FORBIDDEN" | "CHAT_SEND_PLAIN_FORBIDDEN" => peers::NO_SEND_PERMISSION.to_string(),
"USER_BANNED_IN_CHANNEL" => "You are banned from sending messages in this chat".to_string(),
"USER_IS_BLOCKED" => "This person has blocked you".to_string(),
"USER_PRIVACY_RESTRICTED" => "This person's privacy settings do not let you message them".to_string(),
"PEER_FLOOD" => "Telegram is limiting how many people this account may start a conversation with".to_string(),
"MESSAGE_TOO_LONG" => format!("That message is too long: Telegram takes at most {MAX_UNITS} characters"),
"MESSAGE_EMPTY" => "There is nothing to send".to_string(),
"MESSAGE_NOT_MODIFIED" => "That is already what the message says".to_string(),
"MESSAGE_EDIT_TIME_EXPIRED" => "Telegram no longer lets this message be edited".to_string(),
"MESSAGE_ID_INVALID" | "MESSAGE_NOT_FOUND" => "That message is no longer there".to_string(),
"CHAT_ADMIN_REQUIRED" => "Only an administrator may do that here".to_string(),
"MEDIA_EMPTY" | "FILE_PART_INVALID" => "Telegram would not take that file".to_string(),
"FILE_REFERENCE_EXPIRED" => "Telegram needs the message read again before that file can be downloaded".to_string(),
other => format!("Telegram refused: {}", other.to_lowercase().replace('_', " ")),
}
}
InvocationError::Io(e) => format!("Telegram could not be reached: {e}"),
other => format!("{other}"),
}
}
/// The account's connection and Telegram's reference to the conversation a buffer is.
pub async fn peer_for(state: &AppState, buffer: &Buffer) -> Result<(Handle, PeerRef), String> {
let handle = state.runtime.telegram_handle(&buffer.account_id).ok_or_else(|| "that Telegram account is not connected".to_string())?;
let id = buffer.remote_id.as_deref().and_then(peers::peer_id_of).ok_or_else(|| "this is not a Telegram conversation".to_string())?;
let peer = handle.session.peer_ref(id).await.ok().flatten().ok_or_else(|| "Telegram has not told this account about that conversation yet".to_string())?;
Ok((handle, peer))
}
fn units(text: &str) -> usize {
text.encode_utf16().count()
}
/// Text from the message box, marks taken out and entities in place, if Telegram would take it.
pub fn checked(body: &str) -> Result<(String, Vec<tl::enums::MessageEntity>), String> {
let (text, entities) = markup::parse(body);
if text.trim().is_empty() {
return Err("There is nothing to send".to_string());
}
if units(&text) > MAX_UNITS {
return Err(format!("That message is too long: Telegram takes at most {MAX_UNITS} characters, and this has {}", units(&text)));
}
Ok((text, entities))
}
/// The message box's text as a message.
pub fn input_for(body: &str) -> Result<InputMessage, String> {
let (text, entities) = checked(body)?;
Ok(InputMessage::new().text(text).fmt_entities(entities))
}
/// Sends a message, as an answer to another if `reply_to` names one.
pub async fn send(state: &AppState, buffer: &Buffer, body: &str, reply_to: Option<&str>) -> Result<(), String> {
let input = input_for(body)?;
let (handle, peer) = peer_for(state, buffer).await?;
let input = input.reply_to(reply_to.and_then(|id| id.parse::<i32>().ok()));
let sent = handle.client.send_message(peer, input).await.map_err(|e| explain(&e))?;
messages::record(state, &buffer.account_id, &handle.own, &sent);
Ok(())
}
/// Changes what one of the account's own messages says.
pub async fn edit(state: &AppState, buffer: &Buffer, msg_id: &str, body: &str) -> Result<(), String> {
let id: i32 = msg_id.parse().map_err(|_| "that message cannot be edited".to_string())?;
let input = input_for(body)?;
let (handle, peer) = peer_for(state, buffer).await?;
handle.client.edit_message(peer, id, input).await.map_err(|e| explain(&e))?;
let (text, _) = markup::parse(body);
state.runtime.update_message(state, &buffer.id, msg_id, &text, &[], &[]);
Ok(())
}
/// Removes a message: for everyone where Telegram allows it, or only from this account's own view.
pub async fn delete(state: &AppState, buffer: &Buffer, msg_id: &str, for_everyone: bool) -> Result<(), String> {
let id: i32 = msg_id.parse().map_err(|_| "that message cannot be deleted".to_string())?;
let (handle, peer) = peer_for(state, buffer).await?;
if for_everyone || peer.id.kind() == grammers_session::types::PeerKind::Channel {
handle.client.delete_messages(peer, &[id]).await.map_err(|e| explain(&e))?;
} else {
handle.client.invoke(&tl::functions::messages::DeleteMessages { revoke: false, id: vec![id] }).await.map_err(|e| explain(&e))?;
}
state.runtime.delete_message(state, &buffer.id, msg_id);
Ok(())
}
/// Says the account is typing. Telegram lets it lapse after a few seconds, so there is no "stopped".
pub async fn typing(state: &AppState, buffer: &Buffer) -> Result<(), String> {
let (handle, peer) = peer_for(state, buffer).await?;
handle.client.action(peer).oneshot(tl::enums::SendMessageAction::SendMessageTypingAction).await.map_err(|e| explain(&e))
}
/// Says everything in the conversation has been read, so its unread count goes down on the account's other apps.
pub async fn mark_read(state: &AppState, buffer: &Buffer) -> Result<(), String> {
let (handle, peer) = peer_for(state, buffer).await?;
handle.client.mark_as_read(peer).await.map_err(|e| explain(&e))
}
/// A reader that says how much of it has been read, so an upload can show a ring that fills.
struct Counting<'a, R> {
inner: R,
done: u64,
said: u64,
base: u64,
total: u64,
report: &'a (dyn Fn(u64, u64) + Send + Sync),
}
impl<R: tokio::io::AsyncRead + Unpin> tokio::io::AsyncRead for Counting<'_, R> {
fn poll_read(mut self: std::pin::Pin<&mut Self>, cx: &mut std::task::Context<'_>, buf: &mut tokio::io::ReadBuf<'_>) -> std::task::Poll<std::io::Result<()>> {
let before = buf.filled().len();
let polled = std::pin::Pin::new(&mut self.inner).poll_read(cx, buf);
if let std::task::Poll::Ready(Ok(())) = &polled {
self.done += (buf.filled().len() - before) as u64;
if self.done - self.said >= 128 * 1024 {
self.said = self.done;
(self.report)(self.base + self.done, self.total);
}
}
polled
}
}
/// What a file is, for choosing how Telegram is told about it.
#[derive(Clone, Copy, PartialEq, Debug)]
pub enum Sort {
Photo,
Video,
Other,
}
pub fn sort_of(name: &str) -> Sort {
match name.rsplit_once('.').map(|(_, e)| e.to_ascii_lowercase()).as_deref() {
Some("jpg" | "jpeg" | "png" | "webp" | "bmp") => Sort::Photo,
Some("mp4" | "mov" | "webm" | "mkv" | "m4v") => Sort::Video,
_ => Sort::Other,
}
}
async fn upload(client: &grammers_client::Client, path: &str, base: u64, total: u64, report: &(dyn Fn(u64, u64) + Send + Sync)) -> Result<(Uploaded, String), String> {
let file = tokio::fs::File::open(path).await.map_err(|e| format!("could not read {path}: {e}"))?;
let size = file.metadata().await.map_err(|e| e.to_string())?.len();
let name = std::path::Path::new(path).file_name().map(|n| n.to_string_lossy().to_string()).unwrap_or_else(|| "file".to_string());
let mut reader = Counting { inner: file, done: 0, said: 0, base, total, report };
let up = client.upload_stream(&mut reader, size as usize, name.clone()).await.map_err(|e| format!("Telegram would not take {name}: {e}"))?;
Ok((up, name))
}
/// A picture or a video as media. Anything else is sent as a plain file (see `send_files`).
fn media_for(up: Uploaded, sort: Sort, spoiler: bool) -> Option<tl::enums::InputMedia> {
match sort {
Sort::Photo => Some(tl::types::InputMediaUploadedPhoto { spoiler, file: up.raw, stickers: None, ttl_seconds: None, live_photo: false, video: None }.into()),
Sort::Video => Some(grammers_client::media::InputVideo::new(up).supports_streaming(true).spoiler(spoiler).into()),
Sort::Other => None,
}
}
/// Sends files, with the caption on the first: one message each, or an album where there are several pictures and
/// videos. `progress` names the ring the window draws.
pub async fn send_files(state: &AppState, buffer: &Buffer, caption: &str, files: &[(String, bool)], reply_to: Option<&str>, progress: Option<&Progress>) -> Result<(), String> {
if files.is_empty() {
return send(state, buffer, caption, reply_to).await;
}
let (text, entities) = if caption.trim().is_empty() { (String::new(), Vec::new()) } else { checked(caption)? };
// Telegram's limit on a caption is much smaller than on a message.
if units(&text) > 1024 {
return Err(format!("A caption can be at most 1024 characters, and this has {}", units(&text)));
}
let (handle, peer) = peer_for(state, buffer).await?;
let reply = reply_to.and_then(|id| id.parse::<i32>().ok());
let total: u64 = files.iter().filter_map(|(p, _)| std::fs::metadata(p).ok().map(|m| m.len())).sum();
if let Some(p) = progress {
p.at(Phase::Preparing, total as usize, "Telegram");
}
let report = |sent: u64, all: u64| {
if let Some(p) = progress {
p.sent(sent, all, "Telegram");
}
};
let mut uploaded: Vec<(Uploaded, Sort, bool)> = Vec::new();
let mut base = 0u64;
for (path, spoiler) in files {
let (up, name) = upload(&handle.client, path, base, total, &report).await?;
base += std::fs::metadata(path).map(|m| m.len()).unwrap_or(0);
uploaded.push((up, sort_of(&name), *spoiler));
}
if let Some(p) = progress {
p.at(Phase::Waiting, total as usize, "Telegram");
}
let album = uploaded.len() >= 2 && uploaded.len() <= 10 && uploaded.iter().all(|(_, sort, _)| *sort != Sort::Other);
if album {
let medias: Vec<InputMedia> = uploaded
.into_iter()
.enumerate()
.map(|(i, (up, sort, spoiler))| {
let mut m = InputMedia::new().media(media_for(up, sort, spoiler).expect("an album is only pictures and videos"));
if i == 0 {
m = m.caption(text.clone()).fmt_entities(entities.clone()).reply_to(reply);
}
m
})
.collect();
for sent in handle.client.send_album(peer, medias).await.map_err(|e| explain(&e))?.into_iter().flatten() {
messages::record(state, &buffer.account_id, &handle.own, &sent);
}
return Ok(());
}
for (i, (up, sort, spoiler)) in uploaded.into_iter().enumerate() {
let mut input = match media_for(up.clone(), sort, spoiler) {
Some(media) => InputMessage::new().media(media),
None => InputMessage::new().file(up),
}
.reply_to(reply);
if i == 0 {
input = input.text(text.clone()).fmt_entities(entities.clone());
}
let sent = handle.client.send_message(peer, input).await.map_err(|e| explain(&e))?;
messages::record(state, &buffer.account_id, &handle.own, &sent);
}
Ok(())
}
/// Sends a recording as a voice message (an Ogg file of Opus, which is what the recorder writes).
pub async fn send_voice(state: &AppState, buffer: &Buffer, path: &std::path::Path, duration_secs: f64) -> Result<(), String> {
let (handle, peer) = peer_for(state, buffer).await?;
let none = |_: u64, _: u64| {};
let (up, _) = upload(&handle.client, &path.to_string_lossy(), 0, 0, &none).await?;
let voice = grammers_client::media::InputAudio::new(up).voice(true).duration(std::time::Duration::from_secs_f64(duration_secs.max(0.0))).mime_type("audio/ogg");
let sent = handle.client.send_message(peer, InputMessage::new().media(voice)).await.map_err(|e| explain(&e))?;
messages::record(state, &buffer.account_id, &handle.own, &sent);
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use grammers_mtsender::RpcError;
fn rpc(code: i32, name: &str, value: Option<u32>) -> InvocationError {
InvocationError::Rpc(RpcError { code, name: name.to_string(), value, caused_by: None })
}
#[test]
fn telegrams_refusals_are_said_in_words_with_their_numbers() {
assert_eq!(explain(&rpc(420, "FLOOD_WAIT", Some(30))), "Telegram says to wait 30 seconds before doing that again");
assert_eq!(explain(&rpc(420, "SLOWMODE_WAIT", Some(10))), "Slow mode is on here: wait 10 seconds between messages");
assert_eq!(explain(&rpc(403, "CHAT_WRITE_FORBIDDEN", None)), peers::NO_SEND_PERMISSION);
assert!(explain(&rpc(400, "MESSAGE_TOO_LONG", None)).contains("4096"));
assert_eq!(explain(&rpc(400, "SOMETHING_NEW_HAPPENED", None)), "Telegram refused: something new happened");
}
#[test]
fn a_file_is_sent_as_a_picture_a_video_or_a_file_by_its_extension() {
assert_eq!(sort_of("Holiday.JPG"), Sort::Photo);
assert_eq!(sort_of("clip.mp4"), Sort::Video);
assert_eq!(sort_of("report.pdf"), Sort::Other);
assert_eq!(sort_of("noextension"), Sort::Other);
// A GIF is sent as a file so Telegram keeps it moving.
assert_eq!(sort_of("cat.gif"), Sort::Other);
}
#[test]
fn what_cannot_be_sent_is_stopped_before_anything_is_sent() {
assert!(checked(" ").unwrap_err().contains("nothing to send"));
let long = "x".repeat(MAX_UNITS + 1);
assert!(checked(&long).unwrap_err().contains("too long"));
assert!(checked(&"x".repeat(MAX_UNITS)).is_ok());
// An emoji is two units, and Telegram counts units.
assert!(checked(&"😀".repeat(MAX_UNITS / 2 + 1)).unwrap_err().contains("too long"));
// Formatting marks do not count against the limit: they are taken out.
let (text, entities) = checked("**hello**").unwrap();
assert_eq!((text.as_str(), entities.len()), ("hello", 1));
}
}
src/backend/telegram/session.rs
+277
View File
@@ -0,0 +1,277 @@
//! What grammers remembers between runs: the authorization key Telegram issued, the datacenters it found, the people
//! and chats it has met (their access hashes, without which a chat cannot be addressed again) and how far the
//! account's updates have been read.
//!
//! The library offers a SQLite store, which would bring a second SQLite beside the two this daemon already links,
//! and a memory store that cannot be saved. This is its own: the same bookkeeping as the memory store, kept in a file
//! of its own in the daemon's data directory (one per account, private, written whole and renamed into place). The file is
//! a credential: the authorization key in it is the account.
//!
//! It is written when something changed and a few seconds have passed (`save_if_dirty`, called from the connection's
//! loop), and once more when the connection ends or a sign-in finishes.
use anyhow::{Context, Result};
use grammers_session::types::{ChannelState, DcOption, PeerId, PeerInfo, UpdateState, UpdatesState};
use grammers_session::{BoxFuture, Session, SessionData};
use serde::{Deserialize, Serialize};
use std::convert::Infallible;
use std::path::{Path, PathBuf};
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Mutex, MutexGuard};
/// Where an account's session lives.
pub fn path_for(user_id: i64) -> PathBuf {
folder().join(format!("{user_id}.session"))
}
/// A sign-in that has not got as far as having an account yet keeps its session here until it has.
pub fn path_for_login(login_id: &str) -> PathBuf {
let safe: String = login_id.chars().filter(|c| c.is_ascii_alphanumeric() || *c == '-' || *c == '_').collect();
folder().join(format!("login-{safe}.session"))
}
fn folder() -> PathBuf {
crate::default_data_dir().join("telegram")
}
/// The file's form: what the library's own type holds, with the maps as lists (a JSON object cannot be keyed by a
/// peer).
#[derive(Serialize, Deserialize, Default)]
struct Saved {
home_dc: i32,
dc_options: Vec<DcOption>,
peers: Vec<PeerInfo>,
updates: UpdatesState,
}
pub struct FileSession {
path: Mutex<PathBuf>,
data: Mutex<SessionData>,
dirty: AtomicBool,
}
impl FileSession {
/// A new session, with Telegram's datacenters as the library knows them and no key yet.
pub fn fresh(path: PathBuf) -> Self {
Self { path: Mutex::new(path), data: Mutex::new(SessionData::default()), dirty: AtomicBool::new(false) }
}
/// The session a file holds, or a fresh one if there is no file.
pub fn open(path: PathBuf) -> Result<Self> {
let text = match std::fs::read_to_string(&path) {
Ok(text) => text,
Err(e) if e.kind() == std::io::ErrorKind::NotFound => return Ok(Self::fresh(path)),
Err(e) => return Err(e).with_context(|| format!("reading {}", path.display())),
};
let saved: Saved = serde_json::from_str(&text).with_context(|| format!("reading the session in {}", path.display()))?;
let mut data = SessionData::default();
if saved.home_dc != 0 {
data.home_dc = saved.home_dc;
}
for option in saved.dc_options {
data.dc_options.insert(option.id, option);
}
data.peer_infos = saved.peers.into_iter().map(|p| (p.id(), p)).collect();
data.updates_state = saved.updates;
Ok(Self { path: Mutex::new(path), data: Mutex::new(data), dirty: AtomicBool::new(false) })
}
fn data(&self) -> MutexGuard<'_, SessionData> {
// Nothing here can leave the data half-changed, so a lock a panic left poisoned is still good.
self.data.lock().unwrap_or_else(|e| e.into_inner())
}
/// Whether the key Telegram issued for the home datacenter is there: what a signed-in session has.
pub fn has_key(&self) -> bool {
let data = self.data();
data.dc_options.get(&data.home_dc).is_some_and(|o| o.auth_key.is_some())
}
/// Moves the file, when a sign-in learns whose session this is.
pub fn move_to(&self, path: PathBuf) -> Result<()> {
let old = std::mem::replace(&mut *self.path.lock().unwrap_or_else(|e| e.into_inner()), path);
self.save()?;
if old != *self.path.lock().unwrap_or_else(|e| e.into_inner()) {
let _ = std::fs::remove_file(old);
}
Ok(())
}
/// Writes the session, whole and private, and renames it over the last.
pub fn save(&self) -> Result<()> {
let path = self.path.lock().unwrap_or_else(|e| e.into_inner()).clone();
let saved = {
let data = self.data();
Saved { home_dc: data.home_dc, dc_options: data.dc_options.values().cloned().collect(), peers: data.peer_infos.values().cloned().collect(), updates: data.updates_state.clone() }
};
write_private(&path, serde_json::to_string(&saved)?.as_bytes())?;
self.dirty.store(false, Ordering::Relaxed);
Ok(())
}
/// Writes it if anything changed since the last time.
pub fn save_if_dirty(&self) -> Result<()> {
if self.dirty.load(Ordering::Relaxed) {
self.save()?;
}
Ok(())
}
/// Takes the file away: an account that was removed, or a sign-in that was given up.
pub fn delete(&self) {
let path = self.path.lock().unwrap_or_else(|e| e.into_inner()).clone();
let _ = std::fs::remove_file(path);
self.dirty.store(false, Ordering::Relaxed);
}
fn touch(&self) {
self.dirty.store(true, Ordering::Relaxed);
}
}
fn write_private(path: &Path, bytes: &[u8]) -> Result<()> {
use std::io::Write;
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent).with_context(|| format!("making {}", parent.display()))?;
}
let tmp = path.with_extension("session.tmp");
{
// Private from the moment it exists, as the accounts file is.
let mut file = crate::secure::create_private_file(&tmp)?;
file.write_all(bytes)?;
}
std::fs::rename(&tmp, path).with_context(|| format!("writing {}", path.display()))
}
impl Session for FileSession {
type Error = Infallible;
fn home_dc_id(&self) -> Result<i32, Infallible> {
Ok(self.data().home_dc)
}
fn set_home_dc_id(&self, dc_id: i32) -> BoxFuture<'_, Result<(), Infallible>> {
Box::pin(async move {
self.data().home_dc = dc_id;
self.touch();
Ok(())
})
}
fn dc_option(&self, dc_id: i32) -> Result<Option<DcOption>, Infallible> {
Ok(self.data().dc_options.get(&dc_id).cloned())
}
fn set_dc_option(&self, dc_option: &DcOption) -> BoxFuture<'_, Result<(), Infallible>> {
let dc_option = dc_option.clone();
Box::pin(async move {
self.data().dc_options.insert(dc_option.id, dc_option);
self.touch();
Ok(())
})
}
fn peer(&self, peer: PeerId) -> BoxFuture<'_, Result<Option<PeerInfo>, Infallible>> {
Box::pin(async move { Ok(self.data().peer_infos.get(&peer).cloned()) })
}
fn cache_peer(&self, peer: PeerInfo) -> BoxFuture<'_, Result<(), Infallible>> {
Box::pin(async move {
let changed = {
let mut data = self.data();
let known = data.peer_infos.entry(peer.id()).or_insert_with(|| peer.clone());
known.extend_info(&peer)
};
if changed {
self.touch();
}
Ok(())
})
}
fn updates_state(&self) -> BoxFuture<'_, Result<UpdatesState, Infallible>> {
Box::pin(async move { Ok(self.data().updates_state.clone()) })
}
fn set_update_state(&self, update: UpdateState) -> BoxFuture<'_, Result<(), Infallible>> {
Box::pin(async move {
{
let mut data = self.data();
match update {
UpdateState::All(state) => data.updates_state = state,
UpdateState::Primary { pts, date, seq } => {
data.updates_state.pts = pts;
data.updates_state.date = date;
data.updates_state.seq = seq;
}
UpdateState::Secondary { qts } => data.updates_state.qts = qts,
UpdateState::Channel { id, pts } => {
data.updates_state.channels.retain(|c| c.id != id);
data.updates_state.channels.push(ChannelState { id, pts });
}
}
}
self.touch();
Ok(())
})
}
}
#[cfg(test)]
mod tests {
use super::*;
fn dir(name: &str) -> PathBuf {
let dir = std::env::temp_dir().join(format!("nobilis-tg-session-{name}-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
dir
}
#[tokio::test]
async fn a_session_is_written_private_and_comes_back_whole() {
let dir = dir("roundtrip");
let path = dir.join("1.session");
let session = FileSession::fresh(path.clone());
assert!(!session.has_key());
let mut option = session.dc_option(2).unwrap().unwrap();
option.auth_key = Some([9u8; 256]);
session.set_dc_option(&option).await.unwrap();
session.set_home_dc_id(2).await.unwrap();
session.set_update_state(UpdateState::Primary { pts: 10, date: 20, seq: 30 }).await.unwrap();
session.set_update_state(UpdateState::Channel { id: 5, pts: 7 }).await.unwrap();
assert!(session.has_key());
session.save_if_dirty().unwrap();
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
assert_eq!(std::fs::metadata(&path).unwrap().permissions().mode() & 0o777, 0o600, "the key in it is the account");
}
let back = FileSession::open(path.clone()).unwrap();
assert!(back.has_key());
assert_eq!(back.dc_option(2).unwrap().unwrap().auth_key, Some([9u8; 256]));
let state = back.updates_state().await.unwrap();
assert_eq!((state.pts, state.date, state.seq), (10, 20, 30));
assert_eq!(state.channels.iter().map(|c| (c.id, c.pts)).collect::<Vec<_>>(), vec![(5, 7)]);
std::fs::remove_dir_all(dir).ok();
}
#[tokio::test]
async fn nothing_is_written_until_something_changed_and_a_missing_file_is_a_fresh_session() {
let dir = dir("clean");
let path = dir.join("2.session");
let session = FileSession::open(path.clone()).unwrap();
session.save_if_dirty().unwrap();
assert!(!path.exists(), "an unchanged session writes nothing");
session.set_home_dc_id(4).await.unwrap();
session.save_if_dirty().unwrap();
assert!(path.exists());
let moved = dir.join("3.session");
session.move_to(moved.clone()).unwrap();
assert!(moved.exists() && !path.exists(), "a sign-in's file moves to the account's name");
session.delete();
assert!(!moved.exists());
std::fs::remove_dir_all(dir).ok();
}
}
src/backend/telegram/threads.rs
+200
View File
@@ -0,0 +1,200 @@
//! Replies in a thread, and the comments under a channel's post.
//!
//! A reply chain in a group is a thread the window already knows how to draw from the messages it has (the replies
//! name the message at the top, see `messages::record`). What it cannot have is what was said before this client read
//! it, and a channel's comments, which are messages in the discussion group linked to the channel: those are asked for
//! here when a thread is opened, and a comment is sent to that group as an answer to the post's copy in it.
use super::peers;
use super::messages;
use super::send::{explain, input_for, peer_for};
use crate::model::Buffer;
use crate::state::AppState;
use grammers_client::tl;
use grammers_session::types::{PeerAuth, PeerId, PeerRef};
use serde_json::{json, Value};
use std::collections::HashMap;
/// How many replies a thread is read to.
const LIMIT: i32 = 100;
fn user_names(users: &[tl::enums::User]) -> HashMap<i64, String> {
users
.iter()
.filter_map(|u| match u {
tl::enums::User::User(u) => {
let name = [u.first_name.clone().unwrap_or_default(), u.last_name.clone().unwrap_or_default()].iter().filter(|s| !s.is_empty()).cloned().collect::<Vec<_>>().join(" ");
Some((u.id, if name.is_empty() { u.username.clone().unwrap_or_else(|| "Deleted account".to_string()) } else { name }))
}
_ => None,
})
.collect()
}
/// One of Telegram's own messages as the window reads a message.
pub fn message_json(message: &tl::enums::Message, names: &HashMap<i64, String>, chat_name: &str, buffer_id: &str, own_name: &str) -> Option<Value> {
let tl::enums::Message::Message(m) = message else { return None };
let from = if m.out {
own_name.to_string()
} else {
match &m.from_id {
Some(tl::enums::Peer::User(u)) => names.get(&u.user_id).cloned().unwrap_or_else(|| "Somebody".to_string()),
_ => chat_name.to_string(),
}
};
let mut body = m.message.clone();
if let Some(label) = m.media.as_ref().and_then(super::messages::media_label) {
if !body.is_empty() {
body.push('\n');
}
body.push_str(&label);
}
Some(json!({
"id": m.id.to_string(),
"bufferId": buffer_id,
"from": from,
"body": body,
"ts": m.date,
"kind": "chat",
"isAction": false,
"isHighlight": false,
"edited": m.edit_date.is_some(),
"isOwn": m.out,
}))
}
/// A thread's replies, oldest first.
pub async fn fetch(state: &AppState, buffer: &Buffer, root_id: &str) -> Result<Vec<Value>, String> {
let root: i32 = root_id.parse().map_err(|_| "that message has no thread".to_string())?;
let (handle, peer) = peer_for(state, buffer).await?;
let answer = handle
.client
.invoke(&tl::functions::messages::GetReplies { peer: peer.into(), msg_id: root, offset_id: 0, offset_date: 0, add_offset: 0, limit: LIMIT, max_id: 0, min_id: 0, hash: 0 })
.await
.map_err(|e| explain(&e))?;
let (messages, users, chats) = match answer {
tl::enums::messages::Messages::Messages(m) => (m.messages, m.users, m.chats),
tl::enums::messages::Messages::Slice(m) => (m.messages, m.users, m.chats),
tl::enums::messages::Messages::ChannelMessages(m) => (m.messages, m.users, m.chats),
tl::enums::messages::Messages::NotModified(_) => (Vec::new(), Vec::new(), Vec::new()),
};
let names = user_names(&users);
let mut out: Vec<Value> = messages
.iter()
.filter_map(|m| {
// A comment is in the discussion group, not the channel: filed under that group's buffer when it is one.
let discussion = match m {
tl::enums::Message::Message(raw) => match &raw.peer_id {
tl::enums::Peer::Channel(c) if buffer.remote_id.as_deref() != Some(&peers::remote_id(PeerId::channel_unchecked(c.channel_id))) => {
state.runtime.list_buffers().into_iter().find(|b| b.account_id == buffer.account_id && b.remote_id.as_deref() == Some(&peers::remote_id(PeerId::channel_unchecked(c.channel_id)))).map(|b| b.id)
}
_ => None,
},
_ => None,
};
let chat = chats.first().map(|c| match c {
tl::enums::Chat::Channel(c) => c.title.clone(),
tl::enums::Chat::Chat(c) => c.title.clone(),
_ => buffer.name.clone(),
});
message_json(m, &names, chat.as_deref().unwrap_or(&buffer.name), discussion.as_deref().unwrap_or(&buffer.id), &handle.own.name)
})
.collect();
out.sort_by_key(|m| m["ts"].as_i64().unwrap_or(0));
Ok(out)
}
/// Says something in a thread: under a channel's post it goes to the discussion group, as an answer to the post's copy
/// there; anywhere else it is an answer to the message the thread grew from.
pub async fn reply(state: &AppState, buffer: &Buffer, root_id: &str, body: &str) -> Result<(), String> {
let post: i32 = root_id.parse().map_err(|_| "that message has no thread".to_string())?;
let input = input_for(body)?;
let (handle, peer) = peer_for(state, buffer).await?;
let found = match handle.client.invoke(&tl::functions::messages::GetDiscussionMessage { peer: peer.into(), msg_id: post }).await {
Ok(tl::enums::messages::DiscussionMessage::Message(found)) => Some(found),
// A message nobody has answered yet has no thread to be found in: it is answered like any other.
Err(_) => None,
};
let target = found.as_ref().and_then(|found| {
let Some(tl::enums::Message::Message(copy)) = found.messages.first() else { return None };
let tl::enums::Peer::Channel(group) = &copy.peer_id else { return None };
let hash = found.chats.iter().find_map(|c| match c {
tl::enums::Chat::Channel(c) if c.id == group.channel_id => c.access_hash,
_ => None,
})?;
Some((PeerRef { id: PeerId::channel_unchecked(group.channel_id), auth: PeerAuth::from_hash(hash) }, copy.id))
});
let (to, reply_to) = target.unwrap_or((peer, post));
let sent = handle.client.send_message(to, input.reply_to(Some(reply_to))).await.map_err(|e| explain(&e))?;
messages::record(state, &buffer.account_id, &handle.own, &sent);
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
fn raw(id: i32, out: bool, from: i64, text: &str) -> tl::enums::Message {
tl::enums::Message::Message(tl::types::Message {
out,
mentioned: false,
media_unread: false,
silent: false,
post: false,
from_scheduled: false,
legacy: false,
edit_hide: false,
pinned: false,
noforwards: false,
invert_media: false,
offline: false,
video_processing_pending: false,
paid_suggested_post_stars: false,
paid_suggested_post_ton: false,
id,
from_id: Some(tl::enums::Peer::User(tl::types::PeerUser { user_id: from })),
from_boosts_applied: None,
from_rank: None,
peer_id: tl::enums::Peer::User(tl::types::PeerUser { user_id: from }),
saved_peer_id: None,
fwd_from: None,
via_bot_id: None,
via_business_bot_id: None,
reply_to: None,
date: 1790000000 + id,
message: text.into(),
media: None,
reply_markup: None,
entities: None,
views: None,
forwards: None,
replies: None,
edit_date: None,
post_author: None,
grouped_id: None,
reactions: None,
restriction_reason: None,
ttl_period: None,
quick_reply_shortcut_id: None,
effect: None,
factcheck: None,
report_delivery_until_date: None,
paid_message_stars: None,
suggested_post: None,
schedule_repeat_period: None,
summary_from_language: None,
guestchat_via_from: None,
rich_message: None,
})
}
#[test]
fn a_reply_is_read_as_a_message_with_its_authors_name() {
let names: HashMap<i64, String> = [(7, "Ann".to_string())].into_iter().collect();
let got = message_json(&raw(5, false, 7, "hello"), &names, "Chat", "telegram:1|Chat", "Me").unwrap();
assert_eq!((got["id"].as_str(), got["from"].as_str(), got["body"].as_str(), got["isOwn"].as_bool()), (Some("5"), Some("Ann"), Some("hello"), Some(false)));
let mine = message_json(&raw(6, true, 1, "mine"), &names, "Chat", "telegram:1|Chat", "Me").unwrap();
assert_eq!((mine["from"].as_str(), mine["isOwn"].as_bool()), (Some("Me"), Some(true)));
assert!(message_json(&tl::enums::Message::Empty(tl::types::MessageEmpty { id: 1, peer_id: None }), &names, "c", "b", "m").is_none());
}
}
src/backend/telegram/updates.rs
+84
View File
@@ -0,0 +1,84 @@
//! What Telegram sends that is not a message: somebody typing, a conversation read on another of the account's apps.
//!
//! The library turns messages, edits and deletions into updates of their own; everything else arrives as the raw
//! update Telegram sent, which is read here.
use super::peers;
use super::Handle;
use crate::state::AppState;
use grammers_client::tl;
use grammers_session::types::PeerId;
/// How long somebody's "typing" lasts without another: Telegram sends one every few seconds while they type.
const TYPING_FOR_MS: u64 = 6000;
/// The buffer for a conversation named by Telegram's id for it, if there is one.
fn buffer_of(state: &AppState, account_id: &str, remote: &str) -> Option<crate::model::Buffer> {
state.runtime.list_buffers().into_iter().find(|b| b.account_id == account_id && b.remote_id.as_deref() == Some(remote))
}
fn is_typing(action: &tl::enums::SendMessageAction) -> bool {
use tl::enums::SendMessageAction as A;
!matches!(action, A::SendMessageCancelAction | A::SpeakingInGroupCallAction)
}
fn who(handle: &Handle, user: i64) -> Option<String> {
if user == handle.own.user_id {
return None;
}
handle.names.lock().unwrap_or_else(|e| e.into_inner()).get(&user).cloned()
}
fn say_typing(state: &AppState, account_id: &str, buffer: &crate::model::Buffer, name: String) {
state.events.emit("typing", serde_json::json!({ "accountId": account_id, "bufferId": buffer.id, "nicks": [name], "expiresInMs": TYPING_FOR_MS }));
}
/// Marks a conversation read here when Telegram says it was read up to its newest message elsewhere.
fn read_up_to(state: &AppState, account_id: &str, remote: &str, max_id: i32) {
let Some(buffer) = buffer_of(state, account_id, remote) else { return };
let newest = state.store.newest_message_id(&buffer.id).ok().flatten().and_then(|id| id.parse::<i32>().ok()).unwrap_or(0);
if max_id >= newest {
state.events.emit("bufferRead", serde_json::json!({ "accountId": account_id, "bufferId": buffer.id }));
}
}
/// The pins of a conversation changed: read them again, which says so to every window.
fn pins_changed(state: &AppState, account_id: &str, _handle: &Handle, remote: &str) {
let Some(buffer) = buffer_of(state, account_id, remote) else { return };
let state = state.clone();
tokio::spawn(async move {
let _ = super::pins::list(&state, &buffer).await;
});
}
/// Handles one raw update.
pub fn raw(state: &AppState, account_id: &str, handle: &Handle, update: &tl::enums::Update) {
use tl::enums::Update as U;
match update {
U::UserTyping(u) if is_typing(&u.action) => {
if let (Some(buffer), Some(name)) = (buffer_of(state, account_id, &u.user_id.to_string()), who(handle, u.user_id)) {
say_typing(state, account_id, &buffer, name);
}
}
U::ChatUserTyping(u) if is_typing(&u.action) => {
let user = PeerId::from(u.from_id.clone()).bare_id();
if let (Some(buffer), Some(name)) = (buffer_of(state, account_id, &peers::remote_id(PeerId::chat_unchecked(u.chat_id))), user.and_then(|id| who(handle, id))) {
say_typing(state, account_id, &buffer, name);
}
}
U::ChannelUserTyping(u) if is_typing(&u.action) => {
let user = PeerId::from(u.from_id.clone()).bare_id();
if let (Some(buffer), Some(name)) = (buffer_of(state, account_id, &peers::remote_id(PeerId::channel_unchecked(u.channel_id))), user.and_then(|id| who(handle, id))) {
say_typing(state, account_id, &buffer, name);
}
}
U::MessageReactions(u) => super::reactions::update(state, account_id, &peers::remote_id(PeerId::from(u.peer.clone())), u.msg_id, &u.reactions),
U::MessagePoll(u) => super::polls::update(state, u),
U::NotifySettings(u) => super::notify::changed(state, account_id, u),
U::PinnedMessages(u) => pins_changed(state, account_id, handle, &peers::remote_id(PeerId::from(u.peer.clone()))),
U::PinnedChannelMessages(u) => pins_changed(state, account_id, handle, &peers::remote_id(PeerId::channel_unchecked(u.channel_id))),
U::ReadHistoryInbox(u) => read_up_to(state, account_id, &peers::remote_id(PeerId::from(u.peer.clone())), u.max_id),
U::ReadChannelInbox(u) => read_up_to(state, account_id, &peers::remote_id(PeerId::channel_unchecked(u.channel_id)), u.max_id),
_ => {}
}
}
src/backend/twitch/auth.rs
+436
View File
@@ -0,0 +1,436 @@
//! Signing in to Twitch: the device code flow, and the tokens it leaves.
//!
//! The person never types a password into moho. Moho asks Twitch for a short
//! code, shows it, and the person approves it on twitch.tv in their own
//! browser, signed in however they like; moho polls until Twitch says yes and
//! then holds two tokens:
//!
//! - an **access token**, which every call carries and which lasts four hours
//! - a **refresh token**, single use, which buys the next pair and which
//! Twitch lets go of after thirty days without one
//!
//! moho is a *public* client (it is a program on somebody's own computer, and
//! cannot keep a secret), so there is no client secret anywhere in this: the
//! device flow and the refresh both go without one, and the Client ID below
//! is not a secret either.
//!
//! The functions take the server's address, so the tests can answer them.
//! Everything here is checked against a real Twitch account in the plan's
//! round 1; what the answers look like is from the docs
//! (dev.twitch.tv/docs/authentication) and from watching them.
use anyhow::{bail, Context, Result};
use serde::Deserialize;
/// The application registered at dev.twitch.tv for moho: a *public* client.
pub const CLIENT_ID: &str = "ybpi52doogxn2gzsa7415i18bq0vlb";
/// Where Twitch's sign-in lives.
pub const ID_ROOT: &str = "https://id.twitch.tv";
/// What moho asks to be allowed, and what each is for.
///
/// No more than the features need, and said beside each in the account form.
/// Moderation and whispers are asked for when somebody turns them on, not
/// here: a scope nobody can see a use for is a reason to refuse.
pub const SCOPES: &[(&str, &str)] = &[
("chat:read", "read chat"),
("chat:edit", "send messages"),
("user:write:chat", "hear why Twitch turned a message down"),
("user:read:follows", "list the channels you follow"),
("user:read:emotes", "show the emotes you can use"),
];
/// What moderation, whispers, shoutouts, blocking and a name colour need, and what each is for.
/// Asked for in the one sign-in with the rest, so a person signs in once and has every feature: none of
/// it does anything in a channel where the account has no standing (Twitch checks that on every call), and
/// what moho offers follows what the account may do there.
pub const MODERATION_SCOPES: &[(&str, &str)] = &[
("moderator:manage:banned_users", "time people out, ban them and let them back"),
("moderator:manage:chat_messages", "delete messages"),
("moderator:manage:chat_settings", "change the room's modes (slow, followers-only and the rest)"),
("moderator:manage:announcements", "make announcements"),
("user:manage:chat_color", "change your name colour"),
("user:manage:whispers", "send whispers"),
("whispers:read", "read whispers sent to you"),
("moderator:manage:shoutouts", "give shoutouts"),
("user:manage:blocked_users", "block and unblock people"),
];
/// The scopes as Twitch wants them in a request: one string, space between.
pub fn scope_string() -> String {
SCOPES.iter().chain(MODERATION_SCOPES.iter()).map(|(s, _)| *s).collect::<Vec<_>>().join(" ")
}
/// Whether a set of scopes is the one that turned moderation on.
pub fn has_moderation(scopes: &[String]) -> bool {
scopes.iter().any(|s| s == "moderator:manage:banned_users")
}
/// The code to show, and where to enter it.
#[derive(Debug, Clone, Deserialize, PartialEq)]
pub struct DeviceCode {
pub device_code: String,
pub user_code: String,
/// Twitch's own page, with the code already in it.
pub verification_uri: String,
/// How long the code can be used, in seconds.
pub expires_in: u64,
/// How often to ask whether it has been approved, in seconds.
pub interval: u64,
}
/// A pair of tokens, as Twitch hands them out.
#[derive(Debug, Clone, Deserialize, PartialEq)]
pub struct Tokens {
pub access_token: String,
pub refresh_token: String,
/// Seconds from now until the access token stops working.
pub expires_in: i64,
#[serde(default, deserialize_with = "scopes_either_way")]
pub scope: Vec<String>,
}
/// Twitch says the scopes as a list; some of its endpoints have said them as
/// one string. Both are read.
fn scopes_either_way<'de, D: serde::Deserializer<'de>>(d: D) -> std::result::Result<Vec<String>, D::Error> {
#[derive(Deserialize)]
#[serde(untagged)]
enum Either {
List(Vec<String>),
Text(String),
}
Ok(match Option::<Either>::deserialize(d)? {
Some(Either::List(list)) => list,
Some(Either::Text(text)) => text.split_whitespace().map(str::to_string).collect(),
None => Vec::new(),
})
}
/// What asking whether the code was approved can answer.
#[derive(Debug, Clone, PartialEq)]
pub enum Poll {
/// Not yet. Ask again after the interval.
Pending,
/// Twitch wants asking less often.
SlowDown,
/// The person said no.
Denied,
/// The code ran out before anyone approved it.
Expired,
Approved(Tokens),
}
/// Who a token belongs to and what it may do.
#[derive(Debug, Clone, Deserialize, PartialEq)]
pub struct Validation {
pub client_id: String,
pub login: String,
pub user_id: String,
#[serde(default)]
pub scopes: Vec<String>,
/// Seconds the token has left.
pub expires_in: i64,
}
/// Why a token could not be renewed.
#[derive(Debug)]
pub enum Renewal {
/// Twitch will not accept it any more: revoked, used twice, or thirty
/// days old. The only answer is to sign in again.
Refused(String),
/// Anything else - the network, a server having a bad moment. The token
/// may well be fine; try again later.
Failed(anyhow::Error),
}
fn form(pairs: &[(&str, &str)]) -> String {
let mut out = url::form_urlencoded::Serializer::new(String::new());
for (k, v) in pairs {
out.append_pair(k, v);
}
out.finish()
}
async fn post_form(http: &reqwest::Client, url: &str, pairs: &[(&str, &str)]) -> Result<(u16, serde_json::Value)> {
let res = http
.post(url)
.header("Content-Type", "application/x-www-form-urlencoded")
.header("Accept", "application/json")
.body(form(pairs))
.send()
.await
.with_context(|| format!("reaching Twitch at {url}"))?;
let status = res.status().as_u16();
// An error answer is JSON too, and says why; one that is not is read as nothing.
let body = res.json::<serde_json::Value>().await.unwrap_or(serde_json::Value::Null);
Ok((status, body))
}
fn message_of(body: &serde_json::Value) -> String {
body.get("message").and_then(|m| m.as_str()).unwrap_or("").to_string()
}
/// Asks Twitch for a code for somebody to approve.
pub async fn start(http: &reqwest::Client, root: &str) -> Result<DeviceCode> {
let (status, body) = post_form(http, &format!("{root}/oauth2/device"), &[("client_id", CLIENT_ID), ("scopes", &scope_string())]).await?;
if status != 200 {
bail!("Twitch would not start a sign-in ({status}): {}", message_of(&body));
}
serde_json::from_value(body).context("reading Twitch's sign-in code")
}
/// Asks whether the code has been approved.
pub async fn poll(http: &reqwest::Client, root: &str, device_code: &str) -> Result<Poll> {
let (status, body) = post_form(
http,
&format!("{root}/oauth2/token"),
&[
("client_id", CLIENT_ID),
("scopes", &scope_string()),
("device_code", device_code),
("grant_type", "urn:ietf:params:oauth:grant-type:device_code"),
],
)
.await?;
if status == 200 {
return Ok(Poll::Approved(serde_json::from_value(body).context("reading the tokens Twitch gave")?));
}
// The refusals are the ordinary course of a sign-in, said in a message
// rather than a status, and spelled as the standard does.
let message = message_of(&body).to_lowercase();
Ok(match message.as_str() {
"authorization_pending" => Poll::Pending,
m if m.contains("slow_down") => Poll::SlowDown,
m if m.contains("access_denied") || m.contains("denied") => Poll::Denied,
m if m.contains("expired") || m.contains("invalid device code") || m.contains("invalid_device_code") => Poll::Expired,
_ => bail!("Twitch refused the sign-in ({status}): {}", message_of(&body)),
})
}
/// Buys a new pair of tokens with the refresh token. The old pair is spent
/// whether or not this works out, so what comes back must be kept.
pub async fn refresh(http: &reqwest::Client, root: &str, refresh_token: &str) -> std::result::Result<Tokens, Renewal> {
let (status, body) = match post_form(
http,
&format!("{root}/oauth2/token"),
&[("client_id", CLIENT_ID), ("grant_type", "refresh_token"), ("refresh_token", refresh_token)],
)
.await
{
Ok(answer) => answer,
Err(e) => return Err(Renewal::Failed(e)),
};
match status {
200 => serde_json::from_value(body).map_err(|e| Renewal::Failed(anyhow::Error::new(e).context("reading renewed tokens"))),
// Twitch says 400 "Invalid refresh token" for one that is used, revoked or old, and 401 for the same family.
400 | 401 => Err(Renewal::Refused(message_of(&body))),
other => Err(Renewal::Failed(anyhow::anyhow!("Twitch answered {other} to a renewal: {}", message_of(&body)))),
}
}
/// Who a token belongs to, or `None` when Twitch no longer accepts it.
///
/// Twitch asks apps to do this when they start and from time to time, and it
/// is how the login, the id and the scopes are read: the sign-in does not say.
pub async fn validate(http: &reqwest::Client, root: &str, access_token: &str) -> Result<Option<Validation>> {
let res = http
.get(format!("{root}/oauth2/validate"))
.header("Authorization", format!("OAuth {access_token}"))
.send()
.await
.context("reaching Twitch to check a sign-in")?;
match res.status().as_u16() {
200 => Ok(Some(res.json().await.context("reading who the token belongs to")?)),
401 => Ok(None),
other => bail!("Twitch answered {other} when checking a sign-in"),
}
}
/// Tells Twitch to forget a token, so signing out here ends it there too
/// rather than leaving it good for the rest of its four hours.
pub async fn revoke(http: &reqwest::Client, root: &str, token: &str) -> Result<()> {
let (status, body) = post_form(http, &format!("{root}/oauth2/revoke"), &[("client_id", CLIENT_ID), ("token", token)]).await?;
// 400 is a token that was already gone, which is what was wanted.
if status != 200 && status != 400 {
bail!("Twitch answered {status} when ending a sign-in: {}", message_of(&body));
}
Ok(())
}
/// When a token handed out `expires_in` seconds from `now` should be renewed:
/// a little before it stops, so a call made at the edge does not meet a dead one.
pub fn renew_at(now: i64, expires_in: i64) -> i64 {
now + (expires_in - RENEW_BEFORE).max(0)
}
/// How long before its end a token is renewed, in seconds.
pub const RENEW_BEFORE: i64 = 300;
#[cfg(test)]
mod tests {
use super::*;
use std::sync::{Arc, Mutex};
use tokio::io::{AsyncReadExt, AsyncWriteExt};
/// A Twitch that answers each request with the next canned answer, and
/// remembers what it was asked: the request line and the body.
async fn twitch(answers: Vec<(u16, &'static str)>) -> (String, Arc<Mutex<Vec<(String, String)>>>) {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let root = format!("http://{}", listener.local_addr().unwrap());
let seen: Arc<Mutex<Vec<(String, String)>>> = Default::default();
let log = seen.clone();
let answers = Arc::new(Mutex::new(answers.into_iter().collect::<std::collections::VecDeque<_>>()));
tokio::spawn(async move {
loop {
let Ok((mut sock, _)) = listener.accept().await else { return };
let (log, answers) = (log.clone(), answers.clone());
tokio::spawn(async move {
let mut buf = vec![0u8; 8192];
let mut got = 0;
// Until the headers and as much body as it says there is.
loop {
let n = sock.read(&mut buf[got..]).await.unwrap_or(0);
if n == 0 {
break;
}
got += n;
let text = String::from_utf8_lossy(&buf[..got]).to_string();
if let Some(end) = text.find("\r\n\r\n") {
let length = text[..end]
.lines()
.find_map(|l| l.to_lowercase().strip_prefix("content-length:").map(|v| v.trim().parse::<usize>().unwrap_or(0)))
.unwrap_or(0);
if got >= end + 4 + length {
break;
}
}
}
let text = String::from_utf8_lossy(&buf[..got]).to_string();
let (head, body) = text.split_once("\r\n\r\n").unwrap_or((&text, ""));
log.lock().unwrap().push((head.lines().next().unwrap_or("").to_string(), body.to_string()));
let (status, answer) = answers.lock().unwrap().pop_front().unwrap_or((500, "{}"));
let reply = format!("HTTP/1.1 {status} X\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{answer}", answer.len());
let _ = sock.write_all(reply.as_bytes()).await;
});
}
});
(root, seen)
}
fn http() -> reqwest::Client {
reqwest::Client::new()
}
const CODE: &str = r#"{"device_code":"dev123","expires_in":1800,"interval":5,"user_code":"ABCDEFGH","verification_uri":"https://www.twitch.tv/activate?device-code=ABCDEFGH"}"#;
const TOKENS: &str = r#"{"access_token":"acc","expires_in":14344,"refresh_token":"ref","scope":["chat:read","chat:edit"],"token_type":"bearer"}"#;
#[tokio::test]
async fn a_sign_in_asks_for_the_scopes_moho_needs_with_its_own_client_id() {
let (root, seen) = twitch(vec![(200, CODE)]).await;
let code = start(&http(), &root).await.unwrap();
assert_eq!((code.user_code.as_str(), code.interval, code.expires_in), ("ABCDEFGH", 5, 1800));
assert!(code.verification_uri.ends_with("device-code=ABCDEFGH"));
let seen = seen.lock().unwrap();
assert_eq!(seen[0].0, "POST /oauth2/device HTTP/1.1");
assert!(seen[0].1.contains(&format!("client_id={CLIENT_ID}")), "{}", seen[0].1);
// Space is a plus in a form, which is how Twitch reads a list of scopes.
assert!(seen[0].1.contains("scopes=chat%3Aread+chat%3Aedit"), "{}", seen[0].1);
// And no secret anywhere: moho is a public client.
assert!(!seen[0].1.contains("secret"));
// One sign-in has every feature: moderation, whispers, shoutouts and blocking are asked for with the rest.
for scope in ["moderator%3Amanage%3Abanned_users", "whispers%3Aread", "user%3Amanage%3Awhispers", "moderator%3Amanage%3Ashoutouts", "user%3Amanage%3Ablocked_users"] {
assert!(seen[0].1.contains(scope), "{scope} in {}", seen[0].1);
}
assert!(scope_string().contains("moderator:manage:banned_users") && scope_string().contains("whispers:read"));
assert!(has_moderation(&["chat:read".to_string(), "moderator:manage:banned_users".to_string()]) && !has_moderation(&["chat:read".to_string()]));
}
#[tokio::test]
async fn the_poll_tells_waiting_from_a_no_from_an_old_code() {
let (root, _) = twitch(vec![
(400, r#"{"status":400,"message":"authorization_pending"}"#),
(400, r#"{"status":400,"message":"slow_down"}"#),
(400, r#"{"status":400,"message":"access_denied"}"#),
(400, r#"{"status":400,"message":"invalid device code"}"#),
(200, TOKENS),
])
.await;
let client = http();
let ask = || poll(&client, &root, "dev123");
assert_eq!(ask().await.unwrap(), Poll::Pending);
assert_eq!(ask().await.unwrap(), Poll::SlowDown);
assert_eq!(ask().await.unwrap(), Poll::Denied);
assert_eq!(ask().await.unwrap(), Poll::Expired);
match ask().await.unwrap() {
Poll::Approved(t) => assert_eq!((t.access_token.as_str(), t.refresh_token.as_str(), t.scope.len()), ("acc", "ref", 2)),
other => panic!("{other:?}"),
}
}
#[tokio::test]
async fn the_poll_names_the_grant_and_the_code() {
let (root, seen) = twitch(vec![(400, r#"{"message":"authorization_pending"}"#)]).await;
let _ = poll(&http(), &root, "dev123").await;
let seen = seen.lock().unwrap();
assert!(seen[0].1.contains("device_code=dev123"), "{}", seen[0].1);
assert!(seen[0].1.contains("grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Adevice_code"), "{}", seen[0].1);
}
#[tokio::test]
async fn an_answer_nobody_expected_is_an_error_with_twitchs_words() {
let (root, _) = twitch(vec![(500, r#"{"message":"the sky fell"}"#)]).await;
let e = poll(&http(), &root, "d").await.unwrap_err();
assert!(format!("{e}").contains("the sky fell"), "{e}");
}
#[tokio::test]
async fn a_refresh_returns_the_new_pair_and_a_used_one_is_refused() {
let (root, seen) = twitch(vec![(200, TOKENS), (400, r#"{"status":400,"message":"Invalid refresh token"}"#), (503, "{}")]).await;
let t = refresh(&http(), &root, "old").await.unwrap();
assert_eq!(t.access_token, "acc");
assert!(seen.lock().unwrap()[0].1.contains("grant_type=refresh_token"));
assert!(seen.lock().unwrap()[0].1.contains("refresh_token=old"));
match refresh(&http(), &root, "old").await {
Err(Renewal::Refused(why)) => assert_eq!(why, "Invalid refresh token"),
other => panic!("{other:?}"),
}
// A server having a bad moment is not a verdict on the token.
assert!(matches!(refresh(&http(), &root, "old").await, Err(Renewal::Failed(_))));
}
#[tokio::test]
async fn validating_reads_who_a_token_belongs_to_and_a_dead_one_is_none() {
let (root, seen) = twitch(vec![
(200, r#"{"client_id":"c","login":"someone","scopes":["chat:read"],"user_id":"42","expires_in":3600}"#),
(401, r#"{"status":401,"message":"invalid access token"}"#),
])
.await;
let v = validate(&http(), &root, "tok").await.unwrap().expect("accepted");
assert_eq!((v.login.as_str(), v.user_id.as_str(), v.scopes.len()), ("someone", "42", 1));
assert!(seen.lock().unwrap()[0].0.starts_with("GET /oauth2/validate"));
assert!(validate(&http(), &root, "tok").await.unwrap().is_none());
}
#[tokio::test]
async fn revoking_a_token_that_is_already_gone_is_fine() {
let (root, _) = twitch(vec![(200, "{}"), (400, r#"{"message":"Invalid token"}"#), (500, "{}")]).await;
revoke(&http(), &root, "t").await.unwrap();
revoke(&http(), &root, "t").await.unwrap();
assert!(revoke(&http(), &root, "t").await.is_err());
}
#[test]
fn scopes_are_read_as_a_list_or_as_text() {
let list: Tokens = serde_json::from_str(r#"{"access_token":"a","refresh_token":"r","expires_in":1,"scope":["a","b"]}"#).unwrap();
let text: Tokens = serde_json::from_str(r#"{"access_token":"a","refresh_token":"r","expires_in":1,"scope":"a b"}"#).unwrap();
let none: Tokens = serde_json::from_str(r#"{"access_token":"a","refresh_token":"r","expires_in":1}"#).unwrap();
assert_eq!((list.scope, text.scope, none.scope), (vec!["a".to_string(), "b".into()], vec!["a".to_string(), "b".into()], vec![]));
}
#[test]
fn a_token_is_renewed_before_it_runs_out() {
assert_eq!(renew_at(1000, 14400), 1000 + 14400 - RENEW_BEFORE);
assert_eq!(renew_at(1000, 60), 1000, "a token with less time than the margin is renewed at once");
}
}
src/backend/twitch/badges.rs
+118
View File
@@ -0,0 +1,118 @@
//! The pictures for the badges beside a name: Twitch's global ones and each
//! channel's own (the subscriber badge is a different picture in every
//! channel, and changes with the months).
//!
//! A message's `badges` tag names each as `set/version`; what it looks like is
//! in a table Twitch publishes, asked for once for the global set and once per
//! channel when it is joined. Through Twitch's GraphQL (see `gql.rs`), because
//! the official API will answer neither without an account.
use super::gql;
use serde_json::{json, Value};
use std::collections::HashMap;
use std::sync::{Arc, Mutex};
/// One badge as Twitch describes it.
#[derive(Clone, Debug, PartialEq)]
pub struct Picture {
pub title: String,
pub image: String,
}
#[derive(Default)]
struct Held {
global: Option<Arc<HashMap<String, Picture>>>,
channels: HashMap<String, Arc<HashMap<String, Picture>>>,
}
fn held() -> &'static Mutex<Held> {
static HELD: std::sync::OnceLock<Mutex<Held>> = std::sync::OnceLock::new();
HELD.get_or_init(Default::default)
}
const GLOBAL: &str = "query{ badges { setID version title imageURL(size:DOUBLE) } }";
const CHANNEL: &str = "query($login:String!){ user(login:$login){ broadcastBadges { setID version title imageURL(size:DOUBLE) } } }";
/// Reads a list of badges, whichever query it answered.
pub fn read(list: &Value) -> HashMap<String, Picture> {
let mut out = HashMap::new();
for b in list.as_array().into_iter().flatten() {
let (Some(set), Some(version), Some(image)) = (b["setID"].as_str(), b["version"].as_str(), b["imageURL"].as_str()) else { continue };
if !image.starts_with("https://") {
continue;
}
out.insert(format!("{set}/{version}"), Picture { title: b["title"].as_str().unwrap_or(set).to_string(), image: image.to_string() });
}
out
}
/// The picture for a badge in a channel: the channel's own, else the global one.
pub fn find(channel: &str, set_and_version: &str) -> Option<Picture> {
let held = held().lock().unwrap();
held.channels.get(channel).and_then(|c| c.get(set_and_version)).or_else(|| held.global.as_ref().and_then(|g| g.get(set_and_version))).cloned()
}
pub fn have_global() -> bool {
held().lock().unwrap().global.is_some()
}
pub fn forget(channel: &str) {
held().lock().unwrap().channels.remove(channel);
}
pub async fn load_global(http: &reqwest::Client, root: &str) {
if have_global() {
return;
}
let Ok(answer) = gql::call(http, root, &gql::query(GLOBAL, json!({})), &gql::Who::default()).await else { return };
let table = read(&answer["data"]["badges"]);
if !table.is_empty() {
held().lock().unwrap().global = Some(Arc::new(table));
}
}
pub async fn load_channel(http: &reqwest::Client, root: &str, login: &str) {
let Ok(answer) = gql::call(http, root, &gql::query(CHANNEL, json!({ "login": login })), &gql::Who::default()).await else { return };
let table = read(&answer["data"]["user"]["broadcastBadges"]);
held().lock().unwrap().channels.insert(login.to_string(), Arc::new(table));
}
#[cfg(test)]
pub fn keep_for_test(channel: Option<&str>, table: HashMap<String, Picture>) {
let mut held = held().lock().unwrap();
match channel {
Some(c) => {
held.channels.insert(c.to_string(), Arc::new(table));
}
None => held.global = Some(Arc::new(table)),
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_badge_list_is_read_by_set_and_version_and_an_address_that_is_not_https_is_refused() {
let table = read(&json!([
{ "setID": "subscriber", "version": "36", "title": "3-Year Subscriber", "imageURL": "https://static-cdn.jtvnw.net/badges/v1/a/2" },
{ "setID": "moderator", "version": "1", "title": "Moderator", "imageURL": "https://static-cdn.jtvnw.net/badges/v1/b/2" },
{ "setID": "bad", "version": "1", "title": "X", "imageURL": "file:///etc/passwd" },
{ "setID": "no-picture", "version": "1" }
]));
assert_eq!(table.len(), 2);
assert_eq!(table["subscriber/36"].title, "3-Year Subscriber");
}
#[test]
fn a_channels_own_picture_comes_before_the_global_one() {
let pic = |t: &str| Picture { title: t.into(), image: format!("https://x/{t}") };
keep_for_test(None, [("subscriber/0".to_string(), pic("global")), ("vip/1".to_string(), pic("vip"))].into_iter().collect());
keep_for_test(Some("badgechan"), [("subscriber/0".to_string(), pic("channel"))].into_iter().collect());
assert_eq!(find("badgechan", "subscriber/0").unwrap().title, "channel");
assert_eq!(find("badgechan", "vip/1").unwrap().title, "vip");
assert_eq!(find("other", "subscriber/0").unwrap().title, "global");
assert!(find("badgechan", "no/such").is_none());
forget("badgechan");
}
}
src/backend/twitch/chat.rs
+1204
View File
File diff suppressed because it is too large Load Diff
src/backend/twitch/connect.rs
+1418
View File
File diff suppressed because it is too large Load Diff
src/backend/twitch/emotes.rs
+207
View File
@@ -0,0 +1,207 @@
//! The emotes of Twitch's own that a chat is written in: whether one is animated, when an account's own were
//! last read (they change under it: a reward unlocks one, a 24-hour unlock lapses), and how a message the
//! account sends is turned between the picture the box holds and the word Twitch reads.
//!
//! BetterTTV, FrankerFaceZ and 7TV are not here: they are browser extensions that draw over Twitch's chat, and
//! Twitch itself shows those words as words, so moho does not pretend to.
use std::collections::{HashMap, HashSet};
use std::sync::Mutex;
/// Who owns emotes an account has: found out once from Twitch and kept, since a channel's name and picture change
/// rarely and a subscription list can name dozens.
#[derive(Clone)]
pub struct Owner {
pub login: String,
pub name: String,
pub picture: String,
}
fn known_owners() -> &'static Mutex<HashMap<String, Owner>> {
static KNOWN: std::sync::OnceLock<Mutex<HashMap<String, Owner>>> = std::sync::OnceLock::new();
KNOWN.get_or_init(Default::default)
}
/// The channels these numbers belong to, by number; one whose owner Twitch would not name is not in the answer.
pub async fn owners(http: &reqwest::Client, who: &super::helix::Caller<'_>, ids: &[String]) -> HashMap<String, Owner> {
let missing: Vec<String> = {
let known = known_owners().lock().unwrap();
ids.iter().filter(|i| !known.contains_key(*i)).cloned().collect()
};
if !missing.is_empty() {
let found = super::helix::users_by_id(http, who, &missing).await;
let mut known = known_owners().lock().unwrap();
for (id, login, name, picture) in found {
known.insert(id, Owner { login, name, picture });
}
}
let known = known_owners().lock().unwrap();
ids.iter().filter_map(|i| known.get(i).map(|o| (i.clone(), o.clone()))).collect()
}
/// When an account's own emotes in a channel were last fetched, and the channel's number (for fetching them again
/// from anywhere): Twitch lets an emote lapse (a 24-hour unlock, a limited-time one) and gives no word of it, so the
/// list is read again when it is asked for and has gone stale, and the picker shows only what is there.
fn refreshed() -> &'static Mutex<HashMap<String, (std::time::Instant, String)>> {
static REFRESHED: std::sync::OnceLock<Mutex<HashMap<String, (std::time::Instant, String)>>> = std::sync::OnceLock::new();
REFRESHED.get_or_init(Default::default)
}
/// Says the account's emotes in a channel were just fetched.
pub fn note_user_emotes(account_id: &str, channel: &str, room_id: &str) {
refreshed().lock().unwrap().insert(format!("{account_id}|{channel}"), (std::time::Instant::now(), room_id.to_string()));
}
/// The channel's number, and whether the account's emotes there are older than `max_age` (or never fetched).
pub fn user_emotes_state(account_id: &str, channel: &str, max_age: std::time::Duration) -> (Option<String>, bool) {
match refreshed().lock().unwrap().get(&format!("{account_id}|{channel}")) {
Some((at, room)) => (Some(room.clone()), at.elapsed() > max_age),
None => (None, true),
}
}
/// The words an emote token stands for: `[twitch:emote:25:Kappa]` is sent as `Kappa`.
///
/// The window puts a picture in the message box where an emote was picked, and what that picture is made of is
/// this token (the same one an incoming message carries, so one piece of code draws both). Twitch reads an emote
/// as its word, so that is what leaves.
pub fn tokens_to_names(text: &str) -> String {
let mut out = String::with_capacity(text.len());
let mut rest = text;
while let Some(start) = rest.find("[twitch:") {
out.push_str(&rest[..start]);
let after = &rest[start..];
let parsed = after.find(']').and_then(|end| {
let mut parts = after[1..end].splitn(4, ':');
let (_, kind, _id, name) = (parts.next()?, parts.next()?, parts.next()?, parts.next()?);
matches!(kind, "emote" | "aemote").then_some((name, end))
});
match parsed {
Some((name, end)) => {
out.push_str(name);
rest = &after[end + 1..];
}
None => {
out.push_str("[twitch:");
rest = &after["[twitch:".len()..];
}
}
}
out.push_str(rest);
out
}
/// What the sender's own message is shown as: each word that is one of the sender's emotes becomes its token, so
/// the line the sender sees has the same pictures the others see (Twitch does not send a message back to the one
/// who sent it, so nothing else would draw them). `mine` is the sender's emotes in this channel by word, as the
/// picker holds them (the entry's id is the token); a token already in the text, from the picker, is kept.
pub fn tokens_for_own(text: &str, mine: &HashMap<String, String>) -> String {
let mut out = String::with_capacity(text.len());
let mut word = String::new();
let flush = |word: &mut String, out: &mut String| {
if word.is_empty() {
return;
}
match mine.get(word.as_str()) {
Some(token) if !word.starts_with("[twitch:") => out.push_str(token),
_ => out.push_str(word),
}
word.clear();
};
for c in text.chars() {
if c.is_whitespace() {
flush(&mut word, &mut out);
out.push(c);
} else {
word.push(c);
}
}
flush(&mut word, &mut out);
out
}
// ---------------------------------------------------------------------------
// Twitch's own, animated or not
// ---------------------------------------------------------------------------
#[derive(Default)]
struct Animated {
known: HashMap<String, bool>,
asking: HashSet<String>,
}
fn animated_cache() -> &'static Mutex<Animated> {
static CACHE: std::sync::OnceLock<Mutex<Animated>> = std::sync::OnceLock::new();
CACHE.get_or_init(Default::default)
}
/// Whether one of Twitch's own emotes is animated, if that is known yet.
///
/// Twitch's picture address has an animated form for those that are and answers
/// 404 for those that are not, and the message carries no word on which, so the
/// first time an emote is seen it is asked about, in the background, and drawn
/// still until the answer is in; every message after that draws it moving.
pub fn animated(id: &str) -> Option<bool> {
let mut cache = animated_cache().lock().unwrap();
if let Some(known) = cache.known.get(id) {
return Some(*known);
}
if cache.asking.insert(id.to_string()) {
// Only where there is something to ask with: a unit test calling this has no runtime.
if let Ok(runtime) = tokio::runtime::Handle::try_current() {
let id = id.to_string();
runtime.spawn(async move {
let yes = match super::client() {
Ok(http) => http.head(animated_url(&id)).send().await.map(|r| r.status().is_success()).unwrap_or(false),
Err(_) => false,
};
let mut cache = animated_cache().lock().unwrap();
cache.asking.remove(&id);
// Bounded: a long session sees a great many emotes.
if cache.known.len() > 20_000 {
cache.known.clear();
}
cache.known.insert(id, yes);
});
} else {
cache.asking.remove(id);
}
}
None
}
/// Twitch's address for the moving form of one of its own emotes.
pub fn animated_url(id: &str) -> String {
format!("https://static-cdn.jtvnw.net/emoticons/v2/{id}/animated/dark/2.0")
}
/// For a test: says how an emote is, as if it had been asked.
#[cfg(test)]
pub fn say_animated(id: &str, yes: bool) {
animated_cache().lock().unwrap().known.insert(id.to_string(), yes);
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn the_senders_own_line_shows_each_emote_as_a_picture_whether_it_was_picked_or_typed() {
let mine: HashMap<String, String> = [("Kappa".to_string(), "[twitch:emote:25:Kappa]".to_string())].into_iter().collect();
assert_eq!(tokens_for_own("hi Kappa [twitch:emote:9:Pog] plain", &mine), "hi [twitch:emote:25:Kappa] [twitch:emote:9:Pog] plain");
}
#[test]
fn a_picked_emote_leaves_as_its_word_and_nothing_else_is_touched() {
assert_eq!(tokens_to_names("hi [twitch:emote:25:Kappa] and [twitch:emote:9:Pog]!"), "hi Kappa and Pog!");
assert_eq!(tokens_to_names("[twitch:aemote:emotesv2_ab12:Wow]"), "Wow");
assert_eq!(tokens_to_names("[twitch:cheer:cheer:100] stays, [twitch:bad"), "[twitch:cheer:cheer:100] stays, [twitch:bad");
assert_eq!(tokens_to_names("plain [text] here"), "plain [text] here");
}
#[test]
fn twitchs_own_animated_form_is_a_different_address_and_an_unasked_emote_is_unknown() {
assert_eq!(animated_url("301445381"), "https://static-cdn.jtvnw.net/emoticons/v2/301445381/animated/dark/2.0");
assert_eq!(animated("never-asked-for"), None);
}
}
src/backend/twitch/gql.rs
+82
View File
@@ -0,0 +1,82 @@
//! Twitch's own GraphQL, the one twitch.tv's pages are built on.
//!
//! It is private - Twitch does not offer it to anybody else, and it has been
//! changed before under the clients that use it (the plan, Gitea issue #55,
//! says why moho uses it anyway and what happens if it breaks). Three things
//! are asked of it, none of which Twitch's official API will answer without an
//! account: whether a channel is live and what it is called, the signed token
//! a stream's playlist needs, and the badges a channel's chat draws.
//!
//! It is spoken as twitch.tv speaks it: the site's own Client ID, and queries
//! that are either plain GraphQL or, for the player's token, the *persisted*
//! query the player sends by hash. Answers are checked with care - a response
//! that is not what was expected is an error to say, never a crash.
use anyhow::{bail, Context, Result};
use serde_json::{json, Value};
pub const ROOT: &str = "https://gql.twitch.tv/gql";
/// The Client ID twitch.tv's own pages send. Public (it is in every page),
/// and the only one the GraphQL endpoint accepts: it answers 400 "The
/// Client-ID header is invalid" to moho's own.
pub const SITE_CLIENT_ID: &str = "kimne78kx3ncx6brgo4mv6wki5h1ko";
/// A device id for this run, the one the site makes up for a browser: random
/// and sent with every request, so the token Twitch signs is for one device.
pub fn device_id() -> &'static str {
static ID: std::sync::OnceLock<String> = std::sync::OnceLock::new();
ID.get_or_init(|| {
let a = std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).map(|d| d.as_nanos()).unwrap_or(1);
let b = std::process::id() as u128;
format!("{:032x}", a.wrapping_mul(6364136223846793005).wrapping_add(b))
})
}
/// Extra things a call may carry: who is asking, and proof that it is a
/// person's browser asking.
#[derive(Default, Clone)]
pub struct Who<'a> {
/// The `auth-token` of a twitch.tv session, for a call that should be
/// answered for that person (their Turbo, their subscriptions).
pub web_token: Option<&'a str>,
/// An integrity token Twitch demanded, and the device id it was minted for.
pub integrity: Option<(&'a str, &'a str)>,
}
/// Sends one query or persisted query and answers with its JSON, or says what
/// Twitch said when it refused or reported errors.
pub async fn call(http: &reqwest::Client, root: &str, body: &Value, who: &Who<'_>) -> Result<Value> {
let mut req = http
.post(root)
.header("Client-ID", SITE_CLIENT_ID)
// The site sends its JSON as text/plain, and so does this.
.header("Content-Type", "text/plain;charset=UTF-8")
.header("X-Device-Id", who.integrity.map(|(device, _)| device).unwrap_or_else(|| device_id()))
.body(body.to_string());
if let Some(token) = who.web_token.filter(|t| !t.is_empty()) {
req = req.header("Authorization", format!("OAuth {token}"));
}
if let Some((_, token)) = who.integrity {
req = req.header("Client-Integrity", token);
}
let res = req.send().await.context("reaching Twitch")?;
let status = res.status();
let text = res.text().await.unwrap_or_default();
let value: Value = serde_json::from_str(&text).unwrap_or(Value::Null);
if !status.is_success() {
let said = value.get("message").and_then(|m| m.as_str()).unwrap_or("");
bail!("Twitch answered {status}{}", if said.is_empty() { String::new() } else { format!(": {said}") });
}
Ok(value)
}
/// The first error message Twitch put in an answer, if it put any.
pub fn error_of(answer: &Value) -> Option<String> {
answer.get("errors")?.as_array()?.first()?.get("message")?.as_str().map(str::to_string)
}
/// A plain GraphQL query with variables.
pub fn query(text: &str, variables: Value) -> Value {
json!({ "query": text, "variables": variables })
}
src/backend/twitch/helix.rs
+538
View File
@@ -0,0 +1,538 @@
//! Twitch's official API, Helix: only what moho uses of it.
//!
//! Sending a chat message is the first use. Over IRC a message that Twitch
//! drops (a banned word, slow mode, a link it will not have) is dropped
//! silently, or explained by a NOTICE that nothing ties to the message; Helix
//! answers each message with whether it was sent, and if not, why. It also
//! answers with the message's id, which IRC never does for one's own.
//!
//! Every call carries the account's access token and moho's own Client ID.
//! The functions take the server's address, so the tests can answer them.
use anyhow::Context;
use serde::Deserialize;
pub const API_ROOT: &str = "https://api.twitch.tv/helix";
/// What Twitch said about a message sent.
#[derive(Debug, PartialEq)]
pub struct Sent {
/// Twitch's id for the message: what a reply to it, or its removal, names.
pub message_id: String,
/// Why it was not sent, when it was not: Twitch's words.
pub dropped: Option<String>,
}
/// Why a message could not be sent by Helix at all.
#[derive(Debug)]
pub enum SendError {
/// Twitch understood and said no, in words that should reach the person.
Refused(String),
/// It never got a hearing - the network, a server error, a token or a
/// scope Twitch will not take for this call. Sending can fall back to IRC.
Unavailable(anyhow::Error),
}
#[derive(Deserialize)]
struct SendAnswer {
#[serde(default)]
data: Vec<SendResult>,
}
#[derive(Deserialize)]
struct SendResult {
#[serde(default)]
message_id: String,
#[serde(default)]
is_sent: bool,
#[serde(default)]
drop_reason: Option<DropReason>,
}
#[derive(Deserialize)]
struct DropReason {
#[serde(default)]
code: String,
#[serde(default)]
message: String,
}
/// Where Helix is and who is asking it.
pub struct Caller<'a> {
pub root: &'a str,
pub client_id: &'a str,
pub token: &'a str,
}
/// A message to send: to which channel, as whom, saying what.
pub struct Outgoing<'a> {
pub broadcaster_id: &'a str,
pub sender_id: &'a str,
pub message: &'a str,
/// The id of the message this answers.
pub reply_parent: Option<&'a str>,
}
/// Sends a message as the account, optionally as a reply to one.
pub async fn send_chat_message(http: &reqwest::Client, who: &Caller<'_>, out: &Outgoing<'_>) -> std::result::Result<Sent, SendError> {
let mut body = serde_json::json!({ "broadcaster_id": out.broadcaster_id, "sender_id": out.sender_id, "message": out.message });
if let Some(parent) = out.reply_parent {
body["reply_parent_message_id"] = serde_json::json!(parent);
}
let res = http
.post(format!("{}/chat/messages", who.root))
.header("Authorization", format!("Bearer {}", who.token))
.header("Client-Id", who.client_id)
.json(&body)
.send()
.await
.context("reaching Twitch")
.map_err(SendError::Unavailable)?;
let status = res.status().as_u16();
let text = res.text().await.unwrap_or_default();
let said = || serde_json::from_str::<serde_json::Value>(&text).ok().and_then(|v| v.get("message").and_then(|m| m.as_str()).map(str::to_string)).unwrap_or_default();
match status {
200 => {
let answer: SendAnswer = serde_json::from_str(&text).context("reading Twitch's answer to a message").map_err(SendError::Unavailable)?;
let result = answer.data.into_iter().next().ok_or_else(|| SendError::Unavailable(anyhow::anyhow!("Twitch answered a message with nothing")))?;
let dropped = (!result.is_sent).then(|| match result.drop_reason {
Some(r) if !r.message.is_empty() => r.message,
Some(r) if !r.code.is_empty() => format!("Twitch did not send it ({})", r.code),
_ => "Twitch did not send it".to_string(),
});
Ok(Sent { message_id: result.message_id, dropped })
}
// Twitch understood and said no: banned, not allowed to talk, a bad id.
400 | 403 | 422 => Err(SendError::Refused(match said() {
m if m.is_empty() => format!("Twitch refused the message ({status})"),
m => m,
})),
// The token or what it may do is the problem, which is not a verdict on the message.
401 => Err(SendError::Unavailable(anyhow::anyhow!("Twitch does not accept this sign-in for sending: {}", said()))),
429 => Err(SendError::Refused("Twitch says you are sending too fast. Wait a moment.".to_string())),
other => Err(SendError::Unavailable(anyhow::anyhow!("Twitch answered {other} to a message: {}", said()))),
}
}
/// Why a call to Helix did not work.
#[derive(Debug)]
pub enum HelixError {
/// Twitch understood and said no, in words that should reach the person.
Refused(String),
/// The token or what it was allowed to do will not do for this.
Auth(String),
Failed(anyhow::Error),
}
impl std::fmt::Display for HelixError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
HelixError::Refused(why) => write!(f, "{why}"),
// A reason about the account itself (no verified phone number) is not cured by signing in again.
HelixError::Auth(why) if why.to_lowercase().contains("verified") || why.to_lowercase().contains("phone") => write!(f, "Twitch does not allow this for this account ({why}). Verify the account on twitch.tv to enable it."),
HelixError::Auth(why) => write!(f, "Twitch does not allow this for this account{}. Sign in again from Accounts to refresh what it may do.", if why.is_empty() { String::new() } else { format!(" ({why})") }),
HelixError::Failed(e) => write!(f, "{e:#}"),
}
}
}
/// One call to Helix with the account's token: the answer's JSON, or why not.
async fn request(http: &reqwest::Client, who: &Caller<'_>, method: reqwest::Method, path: &str, query: &[(&str, &str)], body: Option<serde_json::Value>) -> Result<serde_json::Value, HelixError> {
let mut url = format!("{}{path}", who.root);
if !query.is_empty() {
let mut q = url::form_urlencoded::Serializer::new(String::new());
for (k, v) in query {
q.append_pair(k, v);
}
url.push('?');
url.push_str(&q.finish());
}
let mut req = http.request(method, &url).header("Authorization", format!("Bearer {}", who.token)).header("Client-Id", who.client_id);
if let Some(body) = body {
req = req.json(&body);
}
let res = req.send().await.context("reaching Twitch").map_err(HelixError::Failed)?;
let status = res.status().as_u16();
let text = res.text().await.unwrap_or_default();
let value: serde_json::Value = serde_json::from_str(&text).unwrap_or(serde_json::Value::Null);
let said = value.get("message").and_then(|m| m.as_str()).unwrap_or("").to_string();
match status {
200..=204 => Ok(value),
401 => Err(HelixError::Auth(said)),
// 403 is "you may not do that here", and is a verdict on the action when it carries Twitch's reason.
400 | 403 | 404 | 409 | 422 | 425 => Err(HelixError::Refused(if said.is_empty() { format!("Twitch refused ({status})") } else { said })),
429 => Err(HelixError::Refused("Twitch says you are doing that too fast. Wait a moment.".to_string())),
other => Err(HelixError::Failed(anyhow::anyhow!("Twitch answered {other}{}", if said.is_empty() { String::new() } else { format!(": {said}") }))),
}
}
/// Twitch's number for a login, or `None` where it has no such user.
pub async fn user_id_of(http: &reqwest::Client, who: &Caller<'_>, login: &str) -> Result<Option<String>, HelixError> {
let answer = request(http, who, reqwest::Method::GET, "/users", &[("login", login)], None).await?;
Ok(answer.pointer("/data/0/id").and_then(|i| i.as_str()).map(str::to_string))
}
/// Which channel and who is acting in it: what every moderation call carries.
pub struct Room<'a> {
pub broadcaster_id: &'a str,
pub moderator_id: &'a str,
}
/// Times somebody out for `seconds`, or bans them when there is no length.
pub async fn ban(http: &reqwest::Client, who: &Caller<'_>, room: &Room<'_>, user_id: &str, seconds: Option<u64>, reason: &str) -> Result<(), HelixError> {
let mut data = serde_json::json!({ "user_id": user_id });
if let Some(seconds) = seconds {
data["duration"] = serde_json::json!(seconds);
}
if !reason.is_empty() {
data["reason"] = serde_json::json!(reason);
}
request(http, who, reqwest::Method::POST, "/moderation/bans", &[("broadcaster_id", room.broadcaster_id), ("moderator_id", room.moderator_id)], Some(serde_json::json!({ "data": data }))).await.map(|_| ())
}
/// Lets somebody back: ends a ban, or a timeout early.
pub async fn unban(http: &reqwest::Client, who: &Caller<'_>, room: &Room<'_>, user_id: &str) -> Result<(), HelixError> {
request(http, who, reqwest::Method::DELETE, "/moderation/bans", &[("broadcaster_id", room.broadcaster_id), ("moderator_id", room.moderator_id), ("user_id", user_id)], None).await.map(|_| ())
}
/// Removes one message, or - with no id - everything in the chat.
pub async fn delete_chat(http: &reqwest::Client, who: &Caller<'_>, room: &Room<'_>, message_id: Option<&str>) -> Result<(), HelixError> {
let mut query = vec![("broadcaster_id", room.broadcaster_id), ("moderator_id", room.moderator_id)];
if let Some(id) = message_id {
query.push(("message_id", id));
}
request(http, who, reqwest::Method::DELETE, "/moderation/chat", &query, None).await.map(|_| ())
}
/// Changes the chat's modes: only the fields given are changed.
pub async fn chat_settings(http: &reqwest::Client, who: &Caller<'_>, room: &Room<'_>, patch: serde_json::Value) -> Result<(), HelixError> {
request(http, who, reqwest::Method::PATCH, "/chat/settings", &[("broadcaster_id", room.broadcaster_id), ("moderator_id", room.moderator_id)], Some(patch)).await.map(|_| ())
}
/// Makes an announcement in the channel, in one of Twitch's colours.
pub async fn announce(http: &reqwest::Client, who: &Caller<'_>, room: &Room<'_>, message: &str, color: &str) -> Result<(), HelixError> {
request(http, who, reqwest::Method::POST, "/chat/announcements", &[("broadcaster_id", room.broadcaster_id), ("moderator_id", room.moderator_id)], Some(serde_json::json!({ "message": message, "color": color }))).await.map(|_| ())
}
/// Changes the account's own name colour: one of Twitch's names, or a `#rrggbb` where the account may.
pub async fn set_color(http: &reqwest::Client, who: &Caller<'_>, user_id: &str, color: &str) -> Result<(), HelixError> {
request(http, who, reqwest::Method::PUT, "/chat/color", &[("user_id", user_id), ("color", color)], None).await.map(|_| ())
}
/// Blocks somebody (for the account, on Twitch's side, everywhere).
pub async fn block(http: &reqwest::Client, who: &Caller<'_>, user_id: &str) -> Result<(), HelixError> {
request(http, who, reqwest::Method::PUT, "/users/blocks", &[("target_user_id", user_id)], None).await.map(|_| ())
}
/// Lets somebody through again.
pub async fn unblock(http: &reqwest::Client, who: &Caller<'_>, user_id: &str) -> Result<(), HelixError> {
request(http, who, reqwest::Method::DELETE, "/users/blocks", &[("target_user_id", user_id)], None).await.map(|_| ())
}
/// Points the channel's viewers at another channel, in chat.
pub async fn shoutout(http: &reqwest::Client, who: &Caller<'_>, room: &Room<'_>, to_id: &str) -> Result<(), HelixError> {
request(http, who, reqwest::Method::POST, "/chat/shoutouts", &[("from_broadcaster_id", room.broadcaster_id), ("to_broadcaster_id", to_id), ("moderator_id", room.moderator_id)], None).await.map(|_| ())
}
/// Sends a whisper.
pub async fn whisper(http: &reqwest::Client, who: &Caller<'_>, from_id: &str, to_id: &str, message: &str) -> Result<(), HelixError> {
request(http, who, reqwest::Method::POST, "/whispers", &[("from_user_id", from_id), ("to_user_id", to_id)], Some(serde_json::json!({ "message": message }))).await.map(|_| ())
}
/// One emote an account can use in a channel.
#[derive(Debug, PartialEq)]
pub struct UserEmote {
/// The broadcaster who owns it: a channel's number, or empty / `twitch` for Twitch's own.
pub owner: String,
pub id: String,
pub name: String,
pub url: String,
pub animated: bool,
}
/// The emotes an account can use in a channel: Twitch's global ones, the
/// channel's, and any the account has from elsewhere (a subscription to
/// another streamer's that carries over, a follower emote). Needs the
/// `user:read:emotes` scope.
pub async fn user_emotes(http: &reqwest::Client, who: &Caller<'_>, user_id: &str, broadcaster_id: &str, cap: usize) -> anyhow::Result<Vec<UserEmote>> {
let mut out: Vec<UserEmote> = Vec::new();
let mut after = String::new();
loop {
let mut url = format!("{}/chat/emotes/user?user_id={user_id}&broadcaster_id={broadcaster_id}", who.root);
if !after.is_empty() {
url.push_str(&format!("&after={after}"));
}
let res = http
.get(&url)
.header("Authorization", format!("Bearer {}", who.token))
.header("Client-Id", who.client_id)
.send()
.await
.context("reaching Twitch for the emotes you can use")?;
let status = res.status();
let body: serde_json::Value = res.json().await.unwrap_or(serde_json::Value::Null);
if !status.is_success() {
anyhow::bail!("Twitch would not list your emotes ({status})");
}
let template = body.get("template").and_then(|t| t.as_str()).unwrap_or("https://static-cdn.jtvnw.net/emoticons/v2/{{id}}/{{format}}/{{theme_mode}}/{{scale}}");
for e in body.get("data").and_then(|d| d.as_array()).into_iter().flatten() {
let (Some(id), Some(name)) = (e.get("id").and_then(|i| i.as_str()), e.get("name").and_then(|n| n.as_str())) else { continue };
let animated = e.get("format").and_then(|f| f.as_array()).is_some_and(|f| f.iter().any(|x| x == "animated"));
// The still form at twice the size: a cell in the picker is small, and the window animates by token.
let url = template.replace("{{id}}", id).replace("{{format}}", "static").replace("{{theme_mode}}", "dark").replace("{{scale}}", "2.0");
let owner = e.get("owner_id").and_then(|o| o.as_str()).unwrap_or("").to_string();
out.push(UserEmote { owner, id: id.to_string(), name: name.to_string(), url, animated });
}
after = body.pointer("/pagination/cursor").and_then(|c| c.as_str()).unwrap_or("").to_string();
if after.is_empty() || out.len() >= cap {
break;
}
}
out.truncate(cap);
Ok(out)
}
/// Every emote a channel has of its own (subscriber tiers, Bits tiers, follower emotes), whether or not an account
/// can use them: what the picker greys out is what is in here and not in `user_emotes`.
pub async fn channel_emotes(http: &reqwest::Client, who: &Caller<'_>, broadcaster_id: &str) -> anyhow::Result<Vec<UserEmote>> {
let res = http
.get(format!("{}/chat/emotes?broadcaster_id={broadcaster_id}", who.root))
.header("Authorization", format!("Bearer {}", who.token))
.header("Client-Id", who.client_id)
.send()
.await
.context("reaching Twitch for the channel's emotes")?;
let status = res.status();
let body: serde_json::Value = res.json().await.unwrap_or(serde_json::Value::Null);
if !status.is_success() {
anyhow::bail!("Twitch would not list the channel's emotes ({status})");
}
let template = body.get("template").and_then(|t| t.as_str()).unwrap_or("https://static-cdn.jtvnw.net/emoticons/v2/{{id}}/{{format}}/{{theme_mode}}/{{scale}}");
Ok(body
.get("data")
.and_then(|d| d.as_array())
.into_iter()
.flatten()
.filter_map(|e| {
let (id, name) = (e.get("id")?.as_str()?, e.get("name")?.as_str()?);
let animated = e.get("format").and_then(|f| f.as_array()).is_some_and(|f| f.iter().any(|x| x == "animated"));
let url = template.replace("{{id}}", id).replace("{{format}}", "static").replace("{{theme_mode}}", "dark").replace("{{scale}}", "2.0");
let owner = e.get("owner_id").and_then(|o| o.as_str()).unwrap_or("").to_string();
Some(UserEmote { owner, id: id.to_string(), name: name.to_string(), url, animated })
})
.collect())
}
/// Who some accounts are: (id, login, name shown, picture), as many as are found. A hundred to a request.
pub async fn users_by_id(http: &reqwest::Client, who: &Caller<'_>, ids: &[String]) -> Vec<(String, String, String, String)> {
let mut out = Vec::new();
for chunk in ids.chunks(100) {
let query: Vec<(&str, &str)> = chunk.iter().map(|i| ("id", i.as_str())).collect();
let Ok(answer) = request(http, who, reqwest::Method::GET, "/users", &query, None).await else { continue };
for u in answer.get("data").and_then(|d| d.as_array()).into_iter().flatten() {
let text = |k: &str| u.get(k).and_then(|v| v.as_str()).unwrap_or("").to_string();
out.push((text("id"), text("login"), text("display_name"), text("profile_image_url")));
}
}
out
}
/// The channels an account follows, by login: only the ones live now when
/// `live_only` (one request), otherwise all of them up to `cap`, a hundred a page.
///
/// Needs the `user:read:follows` scope. A 401 means the token will not do
/// for it and is said as such, since the likely cause is an account that
/// signed in before the scope was asked for.
pub async fn followed(http: &reqwest::Client, who: &Caller<'_>, user_id: &str, live_only: bool, cap: usize) -> anyhow::Result<Vec<String>> {
let mut logins: Vec<String> = Vec::new();
let mut after = String::new();
loop {
let mut url = format!("{}/{}?user_id={user_id}&first=100", who.root, if live_only { "streams/followed" } else { "channels/followed" });
if !after.is_empty() {
url.push_str(&format!("&after={after}"));
}
let res = http
.get(&url)
.header("Authorization", format!("Bearer {}", who.token))
.header("Client-Id", who.client_id)
.send()
.await
.context("reaching Twitch for the channels you follow")?;
let status = res.status();
let body: serde_json::Value = res.json().await.unwrap_or(serde_json::Value::Null);
if !status.is_success() {
let said = body.get("message").and_then(|m| m.as_str()).unwrap_or("");
anyhow::bail!("Twitch would not list the channels you follow ({status}){}. Signing in to Twitch again from Accounts may fix it.", if said.is_empty() { String::new() } else { format!(": {said}") });
}
for row in body.get("data").and_then(|d| d.as_array()).into_iter().flatten() {
if let Some(login) = row.get("broadcaster_login").or_else(|| row.get("user_login")).and_then(|l| l.as_str()) {
logins.push(login.to_lowercase());
}
}
after = body.pointer("/pagination/cursor").and_then(|c| c.as_str()).unwrap_or("").to_string();
// The live list is one page of a hundred, which is every channel a person follows that is on air at once.
if live_only || after.is_empty() || logins.len() >= cap {
break;
}
}
logins.truncate(cap);
Ok(logins)
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::{Arc, Mutex};
use tokio::io::{AsyncReadExt, AsyncWriteExt};
async fn twitch(answers: Vec<(u16, &'static str)>) -> (String, Arc<Mutex<Vec<String>>>) {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let root = format!("http://{}", listener.local_addr().unwrap());
let seen: Arc<Mutex<Vec<String>>> = Default::default();
let log = seen.clone();
let answers = Arc::new(Mutex::new(answers.into_iter().collect::<std::collections::VecDeque<_>>()));
tokio::spawn(async move {
loop {
let Ok((mut sock, _)) = listener.accept().await else { return };
let (log, answers) = (log.clone(), answers.clone());
tokio::spawn(async move {
let mut buf = vec![0u8; 8192];
let mut got = 0;
loop {
let n = sock.read(&mut buf[got..]).await.unwrap_or(0);
if n == 0 {
break;
}
got += n;
let text = String::from_utf8_lossy(&buf[..got]).to_string();
if let Some(end) = text.find("\r\n\r\n") {
let length = text[..end].lines().find_map(|l| l.to_lowercase().strip_prefix("content-length:").map(|v| v.trim().parse::<usize>().unwrap_or(0))).unwrap_or(0);
if got >= end + 4 + length {
break;
}
}
}
log.lock().unwrap().push(String::from_utf8_lossy(&buf[..got]).to_string());
let (status, body) = answers.lock().unwrap().pop_front().unwrap_or((500, "{}"));
let reply = format!("HTTP/1.1 {status} X\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}", body.len());
let _ = sock.write_all(reply.as_bytes()).await;
});
}
});
(root, seen)
}
async fn send(root: &str, reply: Option<&str>) -> std::result::Result<Sent, SendError> {
let who = Caller { root, client_id: "cid", token: "tok" };
let out = Outgoing { broadcaster_id: "100", sender_id: "42", message: "hello", reply_parent: reply };
send_chat_message(&reqwest::Client::new(), &who, &out).await
}
#[tokio::test]
async fn a_sent_message_comes_back_with_its_id_and_carries_the_account_and_client() {
let (root, seen) = twitch(vec![(200, r#"{"data":[{"message_id":"abc","is_sent":true}]}"#)]).await;
let sent = send(&root, Some("parent-1")).await.unwrap();
assert_eq!(sent, Sent { message_id: "abc".into(), dropped: None });
let request = seen.lock().unwrap()[0].clone();
assert!(request.starts_with("POST /chat/messages"), "{request}");
assert!(request.to_lowercase().contains("authorization: bearer tok"));
assert!(request.to_lowercase().contains("client-id: cid"));
assert!(request.contains(r#""broadcaster_id":"100""#) && request.contains(r#""sender_id":"42""#) && request.contains(r#""reply_parent_message_id":"parent-1""#), "{request}");
}
#[tokio::test]
async fn a_message_twitch_drops_says_why() {
let (root, _) = twitch(vec![
(200, r#"{"data":[{"message_id":"","is_sent":false,"drop_reason":{"code":"msg_duplicate","message":"Your message was not sent because it is identical to the previous one you sent, less than 30 seconds ago."}}]}"#),
(200, r#"{"data":[{"message_id":"","is_sent":false,"drop_reason":{"code":"x","message":""}}]}"#),
])
.await;
assert!(send(&root, None).await.unwrap().dropped.unwrap().contains("identical to the previous one"));
assert_eq!(send(&root, None).await.unwrap().dropped.as_deref(), Some("Twitch did not send it (x)"));
}
#[tokio::test]
async fn a_no_is_a_refusal_and_a_bad_token_is_a_reason_to_try_another_way() {
let (root, _) = twitch(vec![
(403, r#"{"error":"Forbidden","status":403,"message":"The sender is not permitted to send chat messages in this channel."}"#),
(401, r#"{"message":"Missing scope: user:write:chat"}"#),
(429, "{}"),
(500, "{}"),
])
.await;
match send(&root, None).await {
Err(SendError::Refused(why)) => assert!(why.contains("not permitted"), "{why}"),
other => panic!("{other:?}"),
}
assert!(matches!(send(&root, None).await, Err(SendError::Unavailable(_))));
assert!(matches!(send(&root, None).await, Err(SendError::Refused(_))));
assert!(matches!(send(&root, None).await, Err(SendError::Unavailable(_))));
assert!(matches!(send("http://127.0.0.1:9", None).await, Err(SendError::Unavailable(_))));
}
#[tokio::test]
async fn the_channels_somebody_follows_are_listed_page_by_page_up_to_a_cap_and_the_live_ones_in_one_request() {
let (root, seen) = twitch(vec![
(200, r#"{"data":[{"broadcaster_login":"Alpha"},{"broadcaster_login":"beta"}],"pagination":{"cursor":"next-page"}}"#),
(200, r#"{"data":[{"broadcaster_login":"gamma"}],"pagination":{}}"#),
(200, r#"{"data":[{"user_login":"Live1"},{"user_login":"live2"}],"pagination":{"cursor":"ignored"}}"#),
(200, r#"{"data":[{"broadcaster_login":"a"},{"broadcaster_login":"b"},{"broadcaster_login":"c"}],"pagination":{"cursor":"more"}}"#),
(401, r#"{"message":"Missing scope: user:read:follows"}"#),
])
.await;
let who = Caller { root: &root, client_id: "cid", token: "tok" };
let client = reqwest::Client::new();
assert_eq!(followed(&client, &who, "42", false, 200).await.unwrap(), vec!["alpha", "beta", "gamma"]);
let asked = seen.lock().unwrap().clone();
assert!(asked[0].starts_with("GET /channels/followed?user_id=42&first=100 "), "{}", asked[0]);
assert!(asked[1].contains("after=next-page"), "{}", asked[1]);
assert_eq!(followed(&client, &who, "42", true, 200).await.unwrap(), vec!["live1", "live2"]);
assert!(seen.lock().unwrap()[2].starts_with("GET /streams/followed?"), "the live list is its own call, once");
assert_eq!(followed(&client, &who, "42", false, 2).await.unwrap(), vec!["a", "b"], "cut at the cap, and no further page asked for");
let e = followed(&client, &who, "42", false, 200).await.unwrap_err();
assert!(e.to_string().contains("Missing scope") && e.to_string().contains("Signing in"), "{e}");
}
#[tokio::test]
async fn the_emotes_somebody_may_use_in_a_channel_come_with_their_pictures_and_whether_they_move() {
let (root, seen) = twitch(vec![
(200, r#"{"data":[{"id":"25","name":"Kappa","emote_type":"globals","format":["static"]},{"id":"emotesv2_abc","name":"Dance","emote_type":"subscriptions","format":["static","animated"]}],"template":"https://static-cdn.jtvnw.net/emoticons/v2/{{id}}/{{format}}/{{theme_mode}}/{{scale}}","pagination":{"cursor":"p2"}}"#),
(200, r#"{"data":[{"id":"9","name":"Last","format":["static"]}],"pagination":{}}"#),
(401, "{}"),
])
.await;
let who = Caller { root: &root, client_id: "cid", token: "tok" };
let client = reqwest::Client::new();
let got = user_emotes(&client, &who, "42", "100", 1000).await.unwrap();
assert_eq!(got.iter().map(|e| (e.name.as_str(), e.animated)).collect::<Vec<_>>(), vec![("Kappa", false), ("Dance", true), ("Last", false)]);
assert_eq!(got[0].url, "https://static-cdn.jtvnw.net/emoticons/v2/25/static/dark/2.0");
assert!(seen.lock().unwrap()[0].starts_with("GET /chat/emotes/user?user_id=42&broadcaster_id=100 "));
assert!(seen.lock().unwrap()[1].contains("after=p2"));
assert!(user_emotes(&client, &who, "42", "100", 1000).await.is_err());
}
#[tokio::test]
async fn the_owners_of_emotes_are_named_with_their_pictures() {
let (root, seen) = twitch(vec![(200, r#"{"data":[{"id":"7","login":"skitzy_va","display_name":"Skitzy_VA","profile_image_url":"https://img/s.png"}]}"#)]).await;
let who = Caller { root: &root, client_id: "cid", token: "tok" };
let got = users_by_id(&reqwest::Client::new(), &who, &["7".to_string(), "8".to_string()]).await;
assert_eq!(got, vec![("7".to_string(), "skitzy_va".to_string(), "Skitzy_VA".to_string(), "https://img/s.png".to_string())]);
assert!(seen.lock().unwrap()[0].starts_with("GET /users?id=7&id=8 "), "{}", seen.lock().unwrap()[0]);
}
#[tokio::test]
async fn a_channels_own_emotes_are_listed_whether_or_not_they_can_be_used() {
let (root, seen) = twitch(vec![
(200, r#"{"data":[{"id":"emotesv2_a","name":"binxSub","emote_type":"subscriptions","format":["static","animated"]},{"id":"7","name":"binxFollow","emote_type":"follower","format":["static"]}],"template":"https://static-cdn.jtvnw.net/emoticons/v2/{{id}}/{{format}}/{{theme_mode}}/{{scale}}"}"#),
(401, "{}"),
])
.await;
let who = Caller { root: &root, client_id: "cid", token: "tok" };
let client = reqwest::Client::new();
let got = channel_emotes(&client, &who, "100").await.unwrap();
assert_eq!(got.iter().map(|e| (e.name.as_str(), e.animated)).collect::<Vec<_>>(), vec![("binxSub", true), ("binxFollow", false)]);
assert_eq!(got[1].url, "https://static-cdn.jtvnw.net/emoticons/v2/7/static/dark/2.0");
assert!(seen.lock().unwrap()[0].starts_with("GET /chat/emotes?broadcaster_id=100 "));
assert!(channel_emotes(&client, &who, "100").await.is_err());
}
}
src/backend/twitch/history.rs
+129
View File
@@ -0,0 +1,129 @@
//! A channel's recent past.
//!
//! Twitch has no chat history: a client that joins a channel sees it from that
//! moment. The usual answer, Chatterino's among others, is a third-party
//! service that keeps the last few hundred lines of every channel it has been
//! asked about and hands them back as the raw IRC lines Twitch sent
//! (`recent-messages.robotty.de`). They go through the same reading as live
//! ones, recorded under Twitch's own message ids so that one already heard is
//! not shown twice, and timed by Twitch's own timestamps so they sort before
//! the present.
//!
//! It is somebody else's server being told which channel is being opened, so
//! it is off until the person turns it on (Settings, Twitch), and goes through
//! the account's route like every other request.
use super::chat::{self, Rooms};
use super::irc;
use crate::state::AppState;
use anyhow::{bail, Context, Result};
/// The service, and how many lines to ask for.
pub const ROOT: &str = "https://recent-messages.robotty.de/api/v2/recent-messages";
const LIMIT: usize = 400;
/// What the service answers.
#[derive(serde::Deserialize)]
struct Answer {
#[serde(default)]
messages: Vec<String>,
#[serde(default)]
error: Option<String>,
}
/// Reads a channel's recent lines out of the service's answer into the
/// scrollback. Only what was said - messages, and subscriptions and raids
/// announced - and not what the room did then (a ban, a mode, a notice),
/// which is past and would be wrong to apply now.
pub async fn load(state: &AppState, account_id: &str, channel: &str, root: &str) -> Result<usize> {
let http = super::client()?;
let res = http
.get(format!("{root}/{channel}?limit={LIMIT}&hide_moderated_messages=true&clearchat_to_notice=false"))
.send()
.await
.context("reaching the recent-messages service")?;
let status = res.status();
let answer: Answer = res.json().await.unwrap_or(Answer { messages: Vec::new(), error: None });
if !status.is_success() {
bail!("the recent-messages service answered {status}{}", answer.error.map(|e| format!(": {e}")).unwrap_or_default());
}
let cfg = state.accounts.get_twitch(account_id);
let mut rooms = Rooms::new(cfg.as_ref().map(|c| c.login.as_str()).unwrap_or(""), "");
let mut read = 0;
for raw in &answer.messages {
let Some(line) = irc::parse(raw) else { continue };
if line.channel().as_deref() != Some(channel) {
continue;
}
if matches!(line.command.as_str(), "PRIVMSG" | "USERNOTICE") {
chat::handle(state, account_id, &mut rooms, &line, true);
read += 1;
}
}
Ok(read)
}
#[cfg(test)]
mod tests {
use super::*;
use crate::backend::kick::testkit;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
async fn service(body: String, status: u16) -> (String, std::sync::Arc<std::sync::Mutex<Vec<String>>>) {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let root = format!("http://{}", listener.local_addr().unwrap());
let seen: std::sync::Arc<std::sync::Mutex<Vec<String>>> = Default::default();
let log = seen.clone();
tokio::spawn(async move {
loop {
let Ok((mut sock, _)) = listener.accept().await else { return };
let (log, body) = (log.clone(), body.clone());
tokio::spawn(async move {
let mut buf = vec![0u8; 4096];
let n = sock.read(&mut buf).await.unwrap_or(0);
log.lock().unwrap().push(String::from_utf8_lossy(&buf[..n]).lines().next().unwrap_or("").to_string());
let reply = format!("HTTP/1.1 {status} X\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}", body.len());
let _ = sock.write_all(reply.as_bytes()).await;
});
}
});
(root, seen)
}
#[tokio::test]
async fn the_past_is_read_as_the_present_was_and_what_is_already_here_is_not_doubled() {
let state = testkit::simulated_daemon("twitch-history");
state.accounts.add_twitch(crate::accounts::TwitchAccountConfig { anonymous: true, ..Default::default() }).unwrap();
let lines = [
"@display-name=Old;id=h1;room-id=1;tmi-sent-ts=1791949000000;user-id=5;historical=1 :[email protected] PRIVMSG #xqc :said a while ago",
"@display-name=Fan;id=h2;room-id=1;tmi-sent-ts=1791949001000;user-id=6;historical=1 :[email protected] PRIVMSG #xqc :still here",
"@display-name=Fan;login=fan;msg-id=sub;room-id=1;tmi-sent-ts=1791949002000;user-id=6 :tmi.twitch.tv USERNOTICE #xqc",
"@room-id=1;target-user-id=5 :tmi.twitch.tv CLEARCHAT #xqc :old",
"@emote-only=1;room-id=1 :tmi.twitch.tv ROOMSTATE #xqc",
"@display-name=Elsewhere;id=h9;room-id=2;tmi-sent-ts=1791949003000;user-id=7 :[email protected] PRIVMSG #other :another channel",
];
let body = serde_json::json!({ "messages": lines, "error": null }).to_string();
let (root, seen) = service(body, 200).await;
// One of them was already heard live.
state.runtime.record_message_at(&state, "twitch:~anonymous", "xqc", "channel", "Fan", "still here", false, "chat", None, Some("h2".into()), false, None, vec![], vec![], None, Some(1791949001), None, None);
let read = load(&state, "twitch:~anonymous", "xqc", &root).await.unwrap();
assert_eq!(read, 3, "two messages and a subscription; not the ban, the mode or another channel");
assert!(seen.lock().unwrap()[0].contains("/xqc?limit="), "{seen:?}");
let log = state.store.get_backlog("twitch:~anonymous|xqc", 0, 20).unwrap();
let bodies: Vec<&str> = log.iter().filter(|m| m.kind == "chat").map(|m| m.body.as_str()).collect();
assert_eq!(bodies, vec!["said a while ago", "still here"], "in the order they were said, with the one heard live not repeated");
assert!(log.iter().all(|m| m.moderated.is_none()), "an old ban is not applied now");
assert!(state.runtime.get_presence("twitch:~anonymous|xqc").is_none(), "the past does not make the roster");
assert_eq!(state.runtime.get_buffer("twitch:~anonymous|xqc").unwrap().channel_modes, None, "nor the modes");
}
#[tokio::test]
async fn a_service_that_will_not_answer_is_an_error_and_nothing_else() {
let state = testkit::simulated_daemon("twitch-history-down");
let (root, _) = service(r#"{"messages":[],"error":"Channel not joined yet"}"#.to_string(), 404).await;
let e = load(&state, "twitch:~anonymous", "xqc", &root).await.unwrap_err();
assert!(format!("{e}").contains("404") && format!("{e}").contains("Channel not joined yet"), "{e}");
assert!(load(&state, "twitch:~anonymous", "xqc", "http://127.0.0.1:9").await.is_err());
}
}
src/backend/twitch/irc.rs
+202
View File
@@ -0,0 +1,202 @@
//! Twitch's IRC, as a line: tags, a prefix, a command and its parameters.
//!
//! Twitch speaks IRC with IRCv3 message tags - `@key=value;key=value` ahead of
//! the line - and nearly everything worth knowing about a message is in them:
//! who it was, what colour, what badges, which parts are emotes, what it was
//! a reply to. This reads one line into that shape and does nothing else; what
//! the lines mean is `chat.rs`'s. Written against lines captured from live
//! channels (the tests keep some), and tolerant of what it does not know: a tag
//! it has no use for is carried along, and a line it cannot read is nothing
//! rather than an error, because one odd line must not end a connection.
use std::collections::HashMap;
/// One line from Twitch.
#[derive(Debug, Clone, PartialEq, Default)]
pub struct Line {
pub tags: HashMap<String, String>,
/// Who sent it: `nick!user@host`, or a server name.
pub prefix: Option<String>,
pub command: String,
/// Everything after the command; the trailing parameter (the one that may
/// hold spaces, after `:`) is last.
pub params: Vec<String>,
}
impl Line {
/// A tag's value, or an empty string where it is absent or empty - which
/// Twitch writes the same way (`emotes=` and no `emotes` tag are one thing).
pub fn tag(&self, key: &str) -> &str {
self.tags.get(key).map(String::as_str).unwrap_or("")
}
/// The nick the line is from, out of `nick!user@host`.
pub fn nick(&self) -> Option<&str> {
let prefix = self.prefix.as_deref()?;
Some(prefix.split('!').next().unwrap_or(prefix)).filter(|n| !n.is_empty())
}
/// The channel it is about, without the `#`, lower-cased as Twitch's are.
pub fn channel(&self) -> Option<String> {
self.params.first().and_then(|p| p.strip_prefix('#')).map(|c| c.to_lowercase())
}
/// The last parameter: what was said, for a message.
pub fn text(&self) -> &str {
if self.params.len() > 1 { self.params.last().map(String::as_str).unwrap_or("") } else { "" }
}
}
/// Reads a tag's value as the IRCv3 spec escapes it.
///
/// `\:` is `;`, `\s` a space, `\\` a backslash, `\r` and `\n` the controls; a
/// backslash before anything else is dropped and the character kept, and a
/// lone one at the end is dropped. A reply quotes the message it answers in a
/// tag, so this is what turns `hello\sthere` back into "hello there".
pub fn unescape(value: &str) -> String {
let mut out = String::with_capacity(value.len());
let mut chars = value.chars();
while let Some(c) = chars.next() {
if c != '\\' {
out.push(c);
continue;
}
match chars.next() {
Some(':') => out.push(';'),
Some('s') => out.push(' '),
Some('r') => out.push('\r'),
Some('n') => out.push('\n'),
Some('\\') => out.push('\\'),
Some(other) => out.push(other),
None => {}
}
}
out
}
/// Reads one line, without its line ending. `None` for a line with no command.
pub fn parse(raw: &str) -> Option<Line> {
let mut rest = raw.trim_end_matches(['\r', '\n']);
let mut line = Line::default();
if let Some(after) = rest.strip_prefix('@') {
let (tags, tail) = after.split_once(' ')?;
for tag in tags.split(';').filter(|t| !t.is_empty()) {
let (key, value) = tag.split_once('=').unwrap_or((tag, ""));
line.tags.insert(key.to_string(), unescape(value));
}
rest = tail.trim_start();
}
if let Some(after) = rest.strip_prefix(':') {
let (prefix, tail) = after.split_once(' ').unwrap_or((after, ""));
line.prefix = Some(prefix.to_string());
rest = tail.trim_start();
}
let (command, mut tail) = rest.split_once(' ').unwrap_or((rest, ""));
if command.is_empty() {
return None;
}
line.command = command.to_string();
while !tail.is_empty() {
if let Some(trailing) = tail.strip_prefix(':') {
line.params.push(trailing.to_string());
break;
}
let (param, next) = tail.split_once(' ').unwrap_or((tail, ""));
line.params.push(param.to_string());
tail = next.trim_start();
}
Some(line)
}
/// What a client sends to get in: the capabilities that put the tags on the
/// lines, then the password and the nick. In that order - Twitch wants the
/// capability request first.
pub fn greeting(login: &str, token: Option<&str>) -> Vec<String> {
// Any password does for an anonymous reader, and a `justinfan` nick marks it.
let (nick, pass) = match token {
Some(token) => (login.to_string(), format!("oauth:{token}")),
None => (format!("justinfan{}", 10_000 + (std::process::id() % 80_000)), "SCHMOOPIIE".to_string()),
};
vec!["CAP REQ :twitch.tv/tags twitch.tv/commands".to_string(), format!("PASS {pass}"), format!("NICK {nick}")]
}
#[cfg(test)]
mod tests {
use super::*;
/// A line captured from xqc's chat, tags and all.
const MESSAGE: &str = "@badge-info=subscriber/37;badges=subscriber/36,premium/1;client-nonce=0c55da2c71da4ff5af5bb3a8ee0ef46d;color=#FF69B4;display-name=Zakktheking13;emotes=301445381:0-7;first-msg=0;flags=;id=6a349c82-69f2-4ed6-ad45-30b2d3e57102;mod=0;returning-chatter=0;room-id=71092938;subscriber=1;tmi-sent-ts=1791949381404;turbo=0;user-id=494821572;user-type= :[email protected] PRIVMSG #xqc :xqcPeepo eyebrow busta";
#[test]
fn a_message_is_read_with_its_tags_prefix_channel_and_text() {
let line = parse(MESSAGE).unwrap();
assert_eq!(line.command, "PRIVMSG");
assert_eq!(line.nick(), Some("zakktheking13"));
assert_eq!(line.channel().as_deref(), Some("xqc"));
// The two spaces inside the text are the sender's, and are kept.
assert_eq!(line.text(), "xqcPeepo eyebrow busta");
assert_eq!(line.tag("display-name"), "Zakktheking13");
assert_eq!(line.tag("emotes"), "301445381:0-7");
assert_eq!(line.tag("color"), "#FF69B4");
assert_eq!(line.tag("tmi-sent-ts"), "1791949381404");
// An empty tag and an absent one are the same thing.
assert_eq!(line.tag("flags"), "");
assert_eq!(line.tag("no-such-tag"), "");
}
#[test]
fn tag_values_are_unescaped() {
assert_eq!(unescape(r"what\sdoes\sno\seyebrows\smean\sbru"), "what does no eyebrows mean bru");
assert_eq!(unescape(r"a\:b\\c\nd\re"), "a;b\\c\nd\re");
assert_eq!(unescape(r"trailing\"), "trailing");
assert_eq!(unescape(r"odd\qescape"), "oddqescape");
}
#[test]
fn a_reply_names_what_it_answers_in_its_tags() {
let line = parse("@reply-parent-display-name=deathsashes112;reply-parent-msg-body=what\\sdoes\\sno\\seyebrows\\smean\\sbru;reply-parent-msg-id=bc4a0663-4e5d-49c4-8d5b-0c78ec73d901;reply-parent-user-login=deathsashes112;reply-thread-parent-msg-id=bc4a0663-4e5d-49c4-8d5b-0c78ec73d901;id=b4d3 :a_t_m_0_s!a_t_m_0_s@a_t_m_0_s.tmi.twitch.tv PRIVMSG #xqc :@deathsashes112 it means nothing").unwrap();
assert_eq!(line.tag("reply-parent-msg-body"), "what does no eyebrows mean bru");
assert_eq!(line.tag("reply-thread-parent-msg-id"), "bc4a0663-4e5d-49c4-8d5b-0c78ec73d901");
}
#[test]
fn the_lines_with_no_tags_are_read_too() {
let line = parse(":tmi.twitch.tv 001 justinfan26983 :Welcome, GLHF!").unwrap();
assert_eq!((line.command.as_str(), line.prefix.as_deref(), line.params.clone()), ("001", Some("tmi.twitch.tv"), vec!["justinfan26983".to_string(), "Welcome, GLHF!".to_string()]));
let ping = parse("PING :tmi.twitch.tv\r\n").unwrap();
assert_eq!((ping.command.as_str(), ping.text()), ("PING", ""));
assert_eq!(ping.params, vec!["tmi.twitch.tv".to_string()]);
let cap = parse(":tmi.twitch.tv CAP * ACK :twitch.tv/tags twitch.tv/commands").unwrap();
assert_eq!((cap.command.as_str(), cap.params.len()), ("CAP", 3));
}
#[test]
fn a_roomstate_is_just_tags() {
let line = parse("@emote-only=0;followers-only=10;r9k=0;room-id=71092938;slow=0;subs-only=0 :tmi.twitch.tv ROOMSTATE #xqc").unwrap();
assert_eq!((line.command.as_str(), line.channel().as_deref(), line.tag("followers-only"), line.tag("room-id")), ("ROOMSTATE", Some("xqc"), "10", "71092938"));
}
#[test]
fn an_action_keeps_its_markers_for_the_caller_to_read() {
let line = parse(":[email protected] PRIVMSG #xqc :\u{1}ACTION waves\u{1}").unwrap();
assert_eq!(line.text(), "\u{1}ACTION waves\u{1}");
}
#[test]
fn what_is_not_a_line_is_nothing() {
assert!(parse("").is_none());
assert!(parse("@tags-and-nothing-else").is_none());
assert!(parse("\r\n").is_none());
}
#[test]
fn a_greeting_asks_for_the_tags_first_and_is_anonymous_without_a_token() {
let signed = greeting("someone", Some("abc"));
assert_eq!(signed, vec!["CAP REQ :twitch.tv/tags twitch.tv/commands", "PASS oauth:abc", "NICK someone"]);
let anon = greeting("", None);
assert!(anon[2].starts_with("NICK justinfan"), "{anon:?}");
assert!(anon[1].starts_with("PASS "));
}
}
src/backend/twitch/login.rs
+277
View File
@@ -0,0 +1,277 @@
//! Adding an account: the sign-in a person takes part in.
//!
//! Twitch's sign-in is a conversation held in two places. Moho asks for a
//! code and shows it; the person types it (or follows the link, which has it
//! in) on twitch.tv, signed in however they sign in, and says yes; moho,
//! meanwhile, asks every few seconds whether that has happened yet. It can
//! take minutes, so it runs as its own task and talks to the window through
//! events rather than answering one request:
//!
//! - `twitchLoginCode` - the code to show, where to enter it, and how long it
//! is good for
//! - `twitchLoginStatus` - what it is doing, for a line of text
//! - `twitchLoginResult` - done, with the account, or the reason it failed
//!
//! The window closing the form ends the wait with `cancelTwitchLogin`.
use super::auth::{self, Poll};
use crate::accounts::TwitchAccountConfig;
use crate::state::AppState;
use anyhow::{bail, Result};
use serde_json::json;
use std::collections::HashMap;
use std::sync::{Arc, Mutex};
use std::time::Duration;
use tokio::sync::Notify;
/// The sign-ins waiting on a person, by id, and the way to end each one.
fn waiting() -> &'static Mutex<HashMap<String, Arc<Notify>>> {
static WAITING: std::sync::OnceLock<Mutex<HashMap<String, Arc<Notify>>>> = std::sync::OnceLock::new();
WAITING.get_or_init(Default::default)
}
/// The twitch.tv session some sign-ins bring with them, by sign-in: the window that signs a person in to twitch.tv
/// and then has them approve the code is one action, and the session it saw on the way is for the account the
/// code becomes. Kept only for as long as the sign-in lasts.
fn web_tokens() -> &'static Mutex<HashMap<String, String>> {
static TOKENS: std::sync::OnceLock<Mutex<HashMap<String, String>>> = std::sync::OnceLock::new();
TOKENS.get_or_init(Default::default)
}
/// Hands a sign-in the person's twitch.tv session, to be saved with the account it makes.
pub fn set_web_token(login_id: &str, token: &str) -> bool {
if token.trim().is_empty() || !waiting().lock().unwrap().contains_key(login_id) {
return false;
}
web_tokens().lock().unwrap().insert(login_id.to_string(), token.trim().to_string());
true
}
/// Ends a sign-in that is waiting, because the form was closed.
pub fn cancel(login_id: &str) {
web_tokens().lock().unwrap().remove(login_id);
if let Some(stop) = waiting().lock().unwrap().remove(login_id) {
stop.notify_one();
}
}
/// Starts a sign-in. Answers at once; the rest is events.
pub fn start(state: AppState, login_id: String) {
start_at(state, login_id, auth::ID_ROOT.to_string(), true)
}
/// The same against a given server, and with or without starting the
/// connection once it is done - which is what lets a test do it without one.
pub fn start_at(state: AppState, login_id: String, root: String, connect: bool) {
let stop = Arc::new(Notify::new());
waiting().lock().unwrap().insert(login_id.clone(), stop.clone());
tokio::spawn(async move {
let outcome = flow(&state, &login_id, &root, stop).await;
waiting().lock().unwrap().remove(&login_id);
web_tokens().lock().unwrap().remove(&login_id);
match outcome {
Ok(Some(saved)) => {
if connect {
super::spawn(state.clone(), saved.clone());
}
let account = crate::accounts::twitch_account_to_json(&saved, if connect { "connecting" } else { "disconnected" });
state.events.emit("twitchLoginResult", json!({ "loginId": login_id, "success": true, "account": account }));
}
// The form was closed: nobody is waiting to be told.
Ok(None) => {}
Err(e) => {
let error = format!("{e:#}");
tracing::warn!("twitch sign-in {login_id}: {error}");
state.events.emit("twitchLoginResult", json!({ "loginId": login_id, "success": false, "error": error }));
}
}
});
}
/// The sign-in itself. `None` when it was cancelled.
pub async fn flow(state: &AppState, login_id: &str, root: &str, stop: Arc<Notify>) -> Result<Option<TwitchAccountConfig>> {
let status = |detail: &str| state.events.emit("twitchLoginStatus", json!({ "loginId": login_id, "detail": detail }));
let http = super::client()?;
status("Asking Twitch for a code");
let code = auth::start(&http, root).await?;
state.events.emit(
"twitchLoginCode",
json!({
"loginId": login_id,
"userCode": code.user_code,
"verificationUri": code.verification_uri,
"expiresIn": code.expires_in,
"scopes": auth::SCOPES.iter().chain(auth::MODERATION_SCOPES.iter()).map(|(s, why)| json!({ "scope": s, "why": why })).collect::<Vec<_>>(),
}),
);
status("Waiting for you to approve it on twitch.tv");
let deadline = tokio::time::Instant::now() + Duration::from_secs(code.expires_in);
let mut interval = Duration::from_secs(code.interval.max(1));
let tokens = loop {
tokio::select! {
_ = stop.notified() => return Ok(None),
_ = tokio::time::sleep(interval) => {}
}
if tokio::time::Instant::now() >= deadline {
bail!("The code ran out before it was approved. Start again for a new one.");
}
match auth::poll(&http, root, &code.device_code).await {
Ok(Poll::Pending) => {}
Ok(Poll::SlowDown) => interval += Duration::from_secs(5),
Ok(Poll::Denied) => bail!("The sign-in was refused on twitch.tv."),
Ok(Poll::Expired) => bail!("The code ran out before it was approved. Start again for a new one."),
Ok(Poll::Approved(tokens)) => break tokens,
// A network blip while waiting is not the end of a sign-in the
// person may be halfway through approving; the deadline is.
Err(e) => tracing::debug!("twitch sign-in {login_id}: {e:#}"),
}
};
status("Checking who signed in");
let Some(who) = auth::validate(&http, root, &tokens.access_token).await? else {
bail!("Twitch gave a token and then would not accept it. Try again.");
};
let now = chrono::Utc::now().timestamp();
let existing = state.accounts.get_twitch(&format!("twitch:{}", who.login.to_lowercase()));
let saved = state.accounts.add_twitch(TwitchAccountConfig {
login: who.login,
user_id: who.user_id,
display_name: existing.as_ref().and_then(|e| e.display_name.clone()),
access_token: tokens.access_token,
refresh_token: tokens.refresh_token,
expires_at: now + tokens.expires_in,
scopes: if who.scopes.is_empty() { tokens.scope } else { who.scopes },
// Signing in again keeps the channels that were open: a renewed
// sign-in is not a reason to forget where the person was.
followed_synced: existing.as_ref().map(|e| e.followed_synced).unwrap_or(false),
channels: existing.map(|e| e.channels).unwrap_or_default(),
web_token: String::new(),
anonymous: false,
use_tor: crate::net::route::router().wanted(&crate::net::route::pending_key("twitch")),
// Never strict at sign-in; the account store keeps an existing
// account's strict routing across a sign-in.
strict_route: false,
})?;
// The twitch.tv session the window signed in to on the way, if it did: video plays as the person from the start.
if let Some(token) = web_tokens().lock().unwrap().remove(login_id) {
state.accounts.mutate_twitch(&saved.account_id(), |a| a.web_token = token.clone())?;
return Ok(state.accounts.get_twitch(&saved.account_id()));
}
Ok(Some(saved))
}
#[cfg(test)]
mod tests {
use super::*;
use crate::backend::kick::testkit;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
/// A Twitch whose answers depend on what is asked and when: the code at
/// once, "pending" a couple of times, then the tokens, then who they are.
async fn twitch(approve_after: usize) -> String {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let root = format!("http://{}", listener.local_addr().unwrap());
let polls = Arc::new(Mutex::new(0usize));
tokio::spawn(async move {
loop {
let Ok((mut sock, _)) = listener.accept().await else { return };
let polls = polls.clone();
tokio::spawn(async move {
let mut buf = vec![0u8; 8192];
let n = sock.read(&mut buf).await.unwrap_or(0);
let head = String::from_utf8_lossy(&buf[..n]).to_string();
let line = head.lines().next().unwrap_or("").to_string();
let (status, body) = if line.starts_with("POST /oauth2/device") {
(200, r#"{"device_code":"dev","expires_in":60,"interval":1,"user_code":"WXYZ1234","verification_uri":"https://www.twitch.tv/activate?device-code=WXYZ1234"}"#.to_string())
} else if line.starts_with("POST /oauth2/token") {
let mut p = polls.lock().unwrap();
*p += 1;
if *p > approve_after {
(200, r#"{"access_token":"acc","refresh_token":"ref","expires_in":14000,"scope":["chat:read"],"token_type":"bearer"}"#.to_string())
} else {
(400, r#"{"status":400,"message":"authorization_pending"}"#.to_string())
}
} else if line.starts_with("GET /oauth2/validate") {
(200, r#"{"client_id":"c","login":"Someone","scopes":["chat:read"],"user_id":"42","expires_in":14000}"#.to_string())
} else {
(404, "{}".to_string())
};
let reply = format!("HTTP/1.1 {status} X\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}", body.len());
let _ = sock.write_all(reply.as_bytes()).await;
});
}
});
root
}
async fn next_event(rx: &mut tokio::sync::broadcast::Receiver<crate::events::Event>, name: &str) -> serde_json::Value {
loop {
let event = tokio::time::timeout(Duration::from_secs(15), rx.recv()).await.expect("an event in time").expect("the bus is open");
if event.name == name {
return event.data;
}
}
}
#[tokio::test]
async fn a_person_is_shown_a_code_approves_it_and_has_an_account() {
let state = testkit::simulated_daemon("twitch-login");
let mut events = state.events.subscribe();
let root = twitch(1).await;
start_at(state.clone(), "login-1".into(), root, false);
let code = next_event(&mut events, "twitchLoginCode").await;
assert_eq!(code["userCode"], "WXYZ1234");
assert!(code["verificationUri"].as_str().unwrap().contains("device-code=WXYZ1234"));
assert!(code["scopes"].as_array().unwrap().iter().any(|s| s["scope"] == "chat:send" || s["scope"] == "chat:edit"));
let result = next_event(&mut events, "twitchLoginResult").await;
assert_eq!(result["success"], true, "{result}");
assert_eq!(result["account"]["id"], "twitch:someone", "the login is read from Twitch and lower-cased");
assert_eq!(result["account"]["service"], "twitch");
let saved = state.accounts.get_twitch("twitch:someone").expect("saved");
assert_eq!((saved.user_id.as_str(), saved.access_token.as_str(), saved.refresh_token.as_str()), ("42", "acc", "ref"));
assert!(saved.expires_at > chrono::Utc::now().timestamp() + 13000);
}
#[tokio::test]
async fn closing_the_form_ends_the_wait_without_saving_anything() {
let state = testkit::simulated_daemon("twitch-login-cancel");
let mut events = state.events.subscribe();
let root = twitch(1000).await;
start_at(state.clone(), "login-2".into(), root, false);
next_event(&mut events, "twitchLoginCode").await;
cancel("login-2");
tokio::time::sleep(Duration::from_millis(1500)).await;
assert!(state.accounts.all_twitch().is_empty());
assert!(waiting().lock().unwrap().get("login-2").is_none());
}
#[tokio::test]
async fn an_unreachable_twitch_is_said_by_the_form() {
let state = testkit::simulated_daemon("twitch-login-down");
let mut events = state.events.subscribe();
start_at(state.clone(), "login-3".into(), "http://127.0.0.1:9".into(), false);
let result = next_event(&mut events, "twitchLoginResult").await;
assert_eq!(result["success"], false);
assert!(result["error"].as_str().unwrap().to_lowercase().contains("twitch"), "{result}");
}
#[tokio::test]
async fn a_twitch_session_handed_over_while_waiting_is_saved_with_the_account_the_code_becomes() {
let state = testkit::simulated_daemon("twitch-login-web");
let mut events = state.events.subscribe();
let root = twitch(1).await;
start_at(state.clone(), "login-4".into(), root, false);
next_event(&mut events, "twitchLoginCode").await;
assert!(set_web_token("login-4", " web-session "));
assert!(!set_web_token("nobody", "x"), "a sign-in that is not waiting takes nothing");
let result = next_event(&mut events, "twitchLoginResult").await;
assert_eq!(result["success"], true, "{result}");
assert_eq!(result["account"]["webSession"], true);
assert_eq!(state.accounts.get_twitch("twitch:someone").unwrap().web_token, "web-session");
assert!(web_tokens().lock().unwrap().get("login-4").is_none(), "nothing is left lying about");
}
}
src/backend/twitch/mod.rs
+64
View File
@@ -0,0 +1,64 @@
//! Twitch - chat and live video, over Twitch's own IRC service and the same
//! playlists its website plays.
//!
//! Twitch is the plan in Gitea issue #55, read before anything here. In short:
//! chat is IRC over a websocket (anonymously readable, signed in to send),
//! the official REST API is Helix, and the video comes from the playlist
//! service twitch.tv's own player uses, so the stream can be drawn in the
//! window's player the way Kick's is.
//!
//! # How this folder is arranged
//!
//! - `auth` - Twitch's sign-in (the device code flow) and what comes of it:
//! tokens, renewing them, finding out who they belong to
//! - `login` - adding an account: the sign-in a person takes part in
//! - `moderation` - removing messages, timeouts and bans, the room's modes, whispers: slash commands and menus alike
//! - `irc` - one line of Twitch's chat: tags, prefix, command, parameters
//! - `emotes` - whether one of Twitch's own emotes is animated, and how an account's own are turned between picture and word
//! - `badges` - the pictures for the badges beside a name
//! - `watch` - telling Twitch a channel is being watched, so its points build while the video plays
//! - `chat` - what the lines mean: messages, cards, removals, a channel's modes
//! - `gql` - Twitch's own GraphQL, which answers what its official API will not without an account
//! - `stream` - what is on air in a channel, said as the stream event the window draws
//! - `playback` - the address of a channel's video, and the integrity token Twitch sometimes wants for it
//! - `points` - channel points, the bonus, bets and votes, as the person's twitch.tv session
//! - `realtime` - what Twitch says as it happens beyond the chat: raids, polls and predictions
//! - `helix` - Twitch's official API, as far as moho uses it
//! - `history` - a channel's recent past, from a third-party service, if asked
//! - `connect` - an account's connection: keeping its token good, and the chat
//! connection with its channels
//!
//! As in the other backends, `spawn` is re-exported so callers say
//! `backend::twitch::spawn` without knowing which file that is.
pub mod auth;
pub mod badges;
pub mod chat;
pub mod connect;
pub mod emotes;
pub mod gql;
pub mod helix;
pub mod history;
pub mod irc;
pub mod login;
pub mod moderation;
pub mod playback;
pub mod points;
pub mod realtime;
pub mod stream;
pub mod watch;
pub use connect::{spawn, Command};
/// What routes this service's requests when any Twitch account goes through
/// Tor or the proxy. One key for the whole service, as Kick's is: the
/// requests that are made before an account exists (the sign-in) have no
/// account to name.
pub const SHARED_KEY: &str = "twitch:any-routed";
/// An HTTP client for Twitch's servers, through the route.
pub fn client() -> anyhow::Result<reqwest::Client> {
let router = crate::net::route::router();
let routed = router.routed(SHARED_KEY) || router.routed(&crate::net::route::pending_key("twitch"));
Ok(router.client_if("twitch", routed, |builder| builder.user_agent(concat!("moho/", env!("CARGO_PKG_VERSION"))).timeout(std::time::Duration::from_secs(20))))
}
src/backend/twitch/moderation.rs
+433
View File
@@ -0,0 +1,433 @@
//! What a person can do to a channel's chat, and to themselves, through
//! Twitch's official API: remove a message, time somebody out or ban them (and
//! let them back), change the room's modes, announce, change a name colour,
//! whisper.
//!
//! Twitch no longer reads these as chat text (`/ban` typed into IRC does
//! nothing), so they are API calls, and this reads the slash commands people
//! are used to typing and makes the calls. The window's own menus - a message's
//! Delete, a name's Time out - make the same ones.
//!
//! The rights come with signing in (see `auth::MODERATION_SCOPES`), so one sign-in has every feature;
//! what an account may actually do in a channel is Twitch's to say on each call, and moho offers
//! the menus only where the account is a moderator or the broadcaster. An account signed in before
//! they were part of it is told to sign in again.
use super::helix::{self, Caller, Room};
use crate::state::AppState;
/// One thing to do.
#[derive(Debug, Clone, PartialEq)]
pub enum Action {
/// A person, by login, for this many seconds.
Timeout { user: String, seconds: u64, reason: String },
Ban { user: String, reason: String },
Unban { user: String },
/// One message by id, or all of them.
Delete { message_id: Option<String> },
Slow(Option<u32>),
/// Followers-only, with the minutes a person must have followed for.
Followers(Option<u32>),
Subscribers(bool),
EmoteOnly(bool),
Unique(bool),
Announce { text: String, color: String },
Color(String),
Whisper { to: String, text: String },
/// Twitch's own block list, for the account.
Block { user: String },
Unblock { user: String },
/// A shoutout to another channel, in this one.
Shoutout { user: String },
}
impl Action {
/// Whether it is something a moderator does to the room, which needs the
/// right to, as against what a person does as themselves.
pub fn needs_moderator(&self) -> bool {
!matches!(self, Action::Color(_) | Action::Whisper { .. } | Action::Block { .. } | Action::Unblock { .. })
}
}
/// Twitch's longest timeout is fourteen days.
const LONGEST_TIMEOUT: u64 = 14 * 24 * 3600;
fn login_of(word: &str) -> Option<String> {
let login = word.trim().trim_start_matches('@').to_lowercase();
(!login.is_empty() && login.len() <= 25 && login.chars().all(|c| c.is_ascii_alphanumeric() || c == '_')).then_some(login)
}
/// A length of time as people type it: `600`, `10m`, `2h`, `1d`, `2w`, in seconds.
pub fn duration_seconds(word: &str) -> Option<u64> {
let word = word.trim().to_lowercase();
let digits: String = word.chars().take_while(|c| c.is_ascii_digit()).collect();
let unit = &word[digits.len()..];
let n: u64 = digits.parse().ok()?;
let per = match unit {
"" | "s" => 1,
"m" => 60,
"h" => 3600,
"d" => 86_400,
"w" => 604_800,
_ => return None,
};
n.checked_mul(per)
}
/// Reads a slash command. `None` for text that is not one (and for `/me`,
/// which Twitch reads itself); an error for one that is wrong or unknown, in
/// words, so a mistyped command is never sent as a message.
pub fn parse(text: &str) -> Option<Result<Action, String>> {
let text = text.trim();
let body = text.strip_prefix('/')?;
// Twitch's own: sent as typed.
if body == "me" || body.starts_with("me ") || body.starts_with('/') || body.is_empty() {
return None;
}
let (name, rest) = body.split_once(char::is_whitespace).map(|(n, r)| (n, r.trim())).unwrap_or((body, ""));
let mut words = rest.split_whitespace();
let user = |w: Option<&str>| w.and_then(login_of).ok_or_else(|| format!("/{name} needs a user: /{name} <name>"));
Some(match name.to_lowercase().as_str() {
"timeout" => (|| {
let who = user(words.next())?;
let mut seconds = 600;
let mut reason: Vec<&str> = Vec::new();
if let Some(first) = words.next() {
match duration_seconds(first) {
Some(s) => seconds = s,
None => reason.push(first),
}
}
reason.extend(words);
if seconds == 0 || seconds > LONGEST_TIMEOUT {
return Err("A timeout is from 1 second to 14 days".to_string());
}
Ok(Action::Timeout { user: who, seconds, reason: reason.join(" ") })
})(),
"ban" => user(words.next()).map(|who| Action::Ban { user: who, reason: words.collect::<Vec<_>>().join(" ") }),
"unban" | "untimeout" => user(words.next()).map(|who| Action::Unban { user: who }),
"delete" => match words.next() {
Some(id) => Ok(Action::Delete { message_id: Some(id.to_string()) }),
None => Err("/delete needs the id of a message; use a message's own menu to delete one".to_string()),
},
"clear" => Ok(Action::Delete { message_id: None }),
"slow" => match words.next().map(|w| w.parse::<u32>()) {
None => Ok(Action::Slow(Some(30))),
Some(Ok(s)) if (3..=120).contains(&s) => Ok(Action::Slow(Some(s))),
_ => Err("Slow mode is 3 to 120 seconds: /slow <seconds>".to_string()),
},
"slowoff" => Ok(Action::Slow(None)),
"followers" => match words.next() {
None => Ok(Action::Followers(Some(0))),
Some(w) => match duration_seconds(&if w.chars().all(|c| c.is_ascii_digit()) { format!("{w}m") } else { w.to_string() }) {
Some(s) if s / 60 <= 129_600 => Ok(Action::Followers(Some((s / 60) as u32))),
_ => Err("Followers-only takes a length up to 3 months: /followers 30m, /followers 2h, /followers 1d".to_string()),
},
},
"followersoff" => Ok(Action::Followers(None)),
"subscribers" | "subscriberson" | "subs" => Ok(Action::Subscribers(true)),
"subscribersoff" | "subsoff" => Ok(Action::Subscribers(false)),
"emoteonly" | "emoteonlyon" => Ok(Action::EmoteOnly(true)),
"emoteonlyoff" => Ok(Action::EmoteOnly(false)),
"uniquechat" | "r9kbeta" => Ok(Action::Unique(true)),
"uniquechatoff" | "r9kbetaoff" => Ok(Action::Unique(false)),
"announce" | "announceblue" | "announcegreen" | "announceorange" | "announcepurple" => {
let color = name.to_lowercase().trim_start_matches("announce").to_string();
let color = if color.is_empty() { "primary".to_string() } else { color };
if rest.is_empty() { Err(format!("/{name} needs something to announce")) } else { Ok(Action::Announce { text: rest.to_string(), color }) }
}
"color" => {
if rest.is_empty() { Err("/color needs a colour: a name like coral or blue_violet, or #rrggbb with Twitch Turbo or Prime".to_string()) } else { Ok(Action::Color(rest.split_whitespace().next().unwrap_or("").to_string())) }
}
"block" => user(words.next()).map(|who| Action::Block { user: who }),
"unblock" => user(words.next()).map(|who| Action::Unblock { user: who }),
"shoutout" | "so" => user(words.next()).map(|who| Action::Shoutout { user: who }),
"w" | "whisper" => match (words.next().and_then(login_of), words.collect::<Vec<_>>().join(" ")) {
(Some(to), text) if !text.is_empty() => Ok(Action::Whisper { to, text }),
_ => Err("/w needs a name and a message: /w <name> <message>".to_string()),
},
other => Err(format!("Twitch has no /{other} command here. Type // to send a message that starts with a slash.")),
})
}
/// What is needed to act: who and where.
pub struct Ctx<'a> {
pub token: &'a str,
pub my_id: &'a str,
/// Twitch's number for the channel, from its room state.
pub room_id: &'a str,
/// Whether this account is a moderator (or the broadcaster) there.
pub moderator: bool,
pub api_root: &'a str,
pub client_id: &'a str,
}
fn said(r: Result<(), helix::HelixError>, done: String) -> Result<String, String> {
r.map(|_| done).map_err(|e| e.to_string())
}
/// Does it, and says what was done in words.
pub async fn run(http: &reqwest::Client, cx: &Ctx<'_>, action: &Action, target_id: Option<&str>) -> Result<String, String> {
if action.needs_moderator() && !cx.moderator {
return Err("You are not a moderator in this channel".to_string());
}
let who = Caller { root: cx.api_root, client_id: cx.client_id, token: cx.token };
let room = Room { broadcaster_id: cx.room_id, moderator_id: cx.my_id };
let resolve = |login: &str| {
let (who, login) = (&who, login.to_string());
let known = target_id.map(str::to_string);
async move {
if let Some(id) = known {
return Ok(id);
}
match helix::user_id_of(http, who, &login).await {
Ok(Some(id)) => Ok(id),
Ok(None) => Err(format!("Twitch has no user called {login}")),
Err(e) => Err(e.to_string()),
}
}
};
match action {
Action::Timeout { user, seconds, reason } => {
let id = resolve(user).await?;
said(helix::ban(http, &who, &room, &id, Some(*seconds), reason).await, format!("{user} was timed out for {}", super::chat::span(*seconds)))
}
Action::Ban { user, reason } => {
let id = resolve(user).await?;
said(helix::ban(http, &who, &room, &id, None, reason).await, format!("{user} was banned"))
}
Action::Unban { user } => {
let id = resolve(user).await?;
said(helix::unban(http, &who, &room, &id).await, format!("{user} is allowed back"))
}
Action::Delete { message_id } => said(
helix::delete_chat(http, &who, &room, message_id.as_deref()).await,
if message_id.is_some() { "The message was deleted".to_string() } else { "Chat was cleared".to_string() },
),
Action::Slow(Some(s)) => said(helix::chat_settings(http, &who, &room, serde_json::json!({ "slow_mode": true, "slow_mode_wait_time": s })).await, format!("Slow mode is on: {s} seconds")),
Action::Slow(None) => said(helix::chat_settings(http, &who, &room, serde_json::json!({ "slow_mode": false })).await, "Slow mode is off".to_string()),
Action::Followers(Some(minutes)) => said(
helix::chat_settings(http, &who, &room, serde_json::json!({ "follower_mode": true, "follower_mode_duration": minutes })).await,
if *minutes == 0 { "Followers-only mode is on".to_string() } else { format!("Followers-only mode is on: followers of {}", super::chat::span(*minutes as u64 * 60)) },
),
Action::Followers(None) => said(helix::chat_settings(http, &who, &room, serde_json::json!({ "follower_mode": false })).await, "Followers-only mode is off".to_string()),
Action::Subscribers(on) => said(helix::chat_settings(http, &who, &room, serde_json::json!({ "subscriber_mode": on })).await, format!("Subscribers-only mode is {}", if *on { "on" } else { "off" })),
Action::EmoteOnly(on) => said(helix::chat_settings(http, &who, &room, serde_json::json!({ "emote_mode": on })).await, format!("Emote-only mode is {}", if *on { "on" } else { "off" })),
Action::Unique(on) => said(helix::chat_settings(http, &who, &room, serde_json::json!({ "unique_chat_mode": on })).await, format!("Unique-chat mode is {}", if *on { "on" } else { "off" })),
Action::Announce { text, color } => said(helix::announce(http, &who, &room, text, color).await, "Announced".to_string()),
Action::Color(color) => said(helix::set_color(http, &who, cx.my_id, color).await, format!("Your name is {color}")),
Action::Block { user } => {
let id = resolve(user).await?;
said(helix::block(http, &who, &id).await, format!("{user} is blocked"))
}
Action::Unblock { user } => {
let id = resolve(user).await?;
said(helix::unblock(http, &who, &id).await, format!("{user} is unblocked"))
}
Action::Shoutout { user } => {
let id = resolve(user).await?;
said(helix::shoutout(http, &who, &room, &id).await, format!("Shouted out {user}"))
}
Action::Whisper { to, text } => {
let id = resolve(to).await?;
said(helix::whisper(http, &who, cx.my_id, &id, &super::emotes::tokens_to_names(text)).await, format!("Whisper sent to {to}"))
}
}
}
/// Said in the channel, as a line of its own, so the one who acted - and any
/// other window of theirs - sees what was done.
pub fn say_done(state: &AppState, account_id: &str, channel: &str, action: &Action, what: &str) {
// What Twitch tells the whole room comes back as its own card (modes, removals, timeouts, bans, clearing
// the chat, announcements); what only this account did is said here, as one.
let card = |icon: &str, subject: &str, what: &str| super::chat::moderation_card(state, account_id, channel, icon, subject, what, None);
match action {
Action::Unban { user } => return card("lock_open", user, "is allowed back"),
Action::Block { user } => return card("block", user, "was blocked"),
Action::Unblock { user } => return card("check_circle", user, "was unblocked"),
Action::Shoutout { user } => return card("campaign", user, "was given a shoutout"),
Action::Timeout { .. } | Action::Ban { .. } | Action::Delete { .. } | Action::Slow(_) | Action::Followers(_) | Action::Subscribers(_) | Action::EmoteOnly(_) | Action::Unique(_) | Action::Announce { .. } | Action::Whisper { .. } => return,
Action::Color(_) => {}
}
state.runtime.record_message(state, account_id, channel, "channel", channel, what, false, "system", None, None, false, None, Vec::new(), Vec::new(), None);
}
#[cfg(test)]
mod tests {
use super::*;
fn ok(text: &str) -> Action {
parse(text).expect("a command").expect("a good one")
}
#[test]
fn what_is_not_a_command_is_not_read_as_one_and_me_is_twitchs_own() {
assert!(parse("hello").is_none());
assert!(parse("/me waves").is_none());
assert!(parse("//not a command").is_none(), "a doubled slash is how a message starting with one is sent");
assert!(parse("/").is_none());
}
#[test]
fn a_timeout_has_a_length_and_a_reason_either_of_which_may_be_left_out() {
assert_eq!(ok("/timeout @Bob"), Action::Timeout { user: "bob".into(), seconds: 600, reason: String::new() });
assert_eq!(ok("/timeout bob 30"), Action::Timeout { user: "bob".into(), seconds: 30, reason: String::new() });
assert_eq!(ok("/timeout bob 2h being rude"), Action::Timeout { user: "bob".into(), seconds: 7200, reason: "being rude".into() });
assert_eq!(ok("/timeout bob spamming links"), Action::Timeout { user: "bob".into(), seconds: 600, reason: "spamming links".into() });
assert!(parse("/timeout").unwrap().unwrap_err().contains("needs a user"));
assert!(parse("/timeout bob 99w").unwrap().unwrap_err().contains("14 days"));
}
#[test]
fn bans_unbans_and_deletes() {
assert_eq!(ok("/ban bob spam"), Action::Ban { user: "bob".into(), reason: "spam".into() });
assert_eq!(ok("/unban bob"), Action::Unban { user: "bob".into() });
assert_eq!(ok("/untimeout bob"), Action::Unban { user: "bob".into() });
assert_eq!(ok("/delete 6a349c82"), Action::Delete { message_id: Some("6a349c82".into()) });
assert_eq!(ok("/clear"), Action::Delete { message_id: None });
assert!(parse("/delete").unwrap().is_err());
}
#[test]
fn the_rooms_modes() {
assert_eq!(ok("/slow"), Action::Slow(Some(30)));
assert_eq!(ok("/slow 10"), Action::Slow(Some(10)));
assert!(parse("/slow 1").unwrap().is_err());
assert_eq!(ok("/slowoff"), Action::Slow(None));
assert_eq!(ok("/followers"), Action::Followers(Some(0)));
assert_eq!(ok("/followers 30"), Action::Followers(Some(30)));
assert_eq!(ok("/followers 2h"), Action::Followers(Some(120)));
assert_eq!(ok("/followers 1d"), Action::Followers(Some(1440)));
assert!(parse("/followers soon").unwrap().is_err());
assert_eq!(ok("/followersoff"), Action::Followers(None));
assert_eq!((ok("/subscribers"), ok("/subscribersoff")), (Action::Subscribers(true), Action::Subscribers(false)));
assert_eq!((ok("/emoteonly"), ok("/emoteonlyoff")), (Action::EmoteOnly(true), Action::EmoteOnly(false)));
assert_eq!((ok("/uniquechat"), ok("/uniquechatoff")), (Action::Unique(true), Action::Unique(false)));
}
#[test]
fn announcements_colours_and_whispers() {
assert_eq!(ok("/announce Giveaway at noon"), Action::Announce { text: "Giveaway at noon".into(), color: "primary".into() });
assert_eq!(ok("/announcegreen Hello"), Action::Announce { text: "Hello".into(), color: "green".into() });
assert!(parse("/announce").unwrap().is_err());
assert_eq!(ok("/color coral"), Action::Color("coral".into()));
assert_eq!((ok("/block @Bob"), ok("/unblock bob")), (Action::Block { user: "bob".into() }, Action::Unblock { user: "bob".into() }));
assert_eq!(ok("/so Bob"), Action::Shoutout { user: "bob".into() });
assert!(!Action::Block { user: "x".into() }.needs_moderator() && Action::Shoutout { user: "x".into() }.needs_moderator());
assert_eq!(ok("/w @Bob hello there"), Action::Whisper { to: "bob".into(), text: "hello there".into() });
assert!(parse("/w bob").unwrap().is_err());
assert!(!Action::Color("x".into()).needs_moderator() && Action::Slow(None).needs_moderator());
}
#[test]
fn an_unknown_command_is_said_and_never_sent_as_a_message() {
let e = parse("/frobnicate now").unwrap().unwrap_err();
assert!(e.contains("no /frobnicate command") && e.contains("//"), "{e}");
}
#[test]
fn lengths_are_read_as_people_write_them() {
for (typed, seconds) in [("90", 90), ("30s", 30), ("10m", 600), ("2H", 7200), ("1d", 86_400), ("1w", 604_800)] {
assert_eq!(duration_seconds(typed), Some(seconds), "{typed}");
}
for typed in ["", "m", "1y", "ten"] {
assert_eq!(duration_seconds(typed), None, "{typed}");
}
}
// The calls themselves, against a Twitch that is a server in a test.
use std::sync::{Arc, Mutex};
use tokio::io::{AsyncReadExt, AsyncWriteExt};
async fn helix_server(answers: Vec<(u16, &'static str)>) -> (String, Arc<Mutex<Vec<String>>>) {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let root = format!("http://{}", listener.local_addr().unwrap());
let seen: Arc<Mutex<Vec<String>>> = Default::default();
let log = seen.clone();
let answers = Arc::new(Mutex::new(answers.into_iter().collect::<std::collections::VecDeque<_>>()));
tokio::spawn(async move {
loop {
let Ok((mut sock, _)) = listener.accept().await else { return };
let (log, answers) = (log.clone(), answers.clone());
tokio::spawn(async move {
let mut buf = vec![0u8; 8192];
let mut got = 0;
loop {
let n = sock.read(&mut buf[got..]).await.unwrap_or(0);
if n == 0 {
break;
}
got += n;
let text = String::from_utf8_lossy(&buf[..got]).to_string();
if let Some(end) = text.find("\r\n\r\n") {
let length = text[..end].lines().find_map(|l| l.to_lowercase().strip_prefix("content-length:").map(|v| v.trim().parse::<usize>().unwrap_or(0))).unwrap_or(0);
if got >= end + 4 + length {
break;
}
}
}
log.lock().unwrap().push(String::from_utf8_lossy(&buf[..got]).to_string());
let (status, body) = answers.lock().unwrap().pop_front().unwrap_or((500, "{}"));
let reply = format!("HTTP/1.1 {status} X\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}", body.len());
let _ = sock.write_all(reply.as_bytes()).await;
});
}
});
(root, seen)
}
fn cx<'a>(root: &'a str, moderator: bool) -> Ctx<'a> {
Ctx { token: "tok", my_id: "42", room_id: "100", moderator, api_root: root, client_id: "cid" }
}
#[tokio::test]
async fn a_timeout_looks_the_person_up_and_asks_twitch_for_it_in_the_rooms_name() {
let (root, seen) = helix_server(vec![(200, r#"{"data":[{"id":"7","login":"bob"}]}"#), (200, r#"{"data":[{}]}"#)]).await;
let done = run(&reqwest::Client::new(), &cx(&root, true), &Action::Timeout { user: "bob".into(), seconds: 600, reason: "spam".into() }, None).await.unwrap();
assert_eq!(done, "bob was timed out for 10 minutes");
let asked = seen.lock().unwrap().clone();
assert!(asked[0].starts_with("GET /users?login=bob "), "{}", asked[0]);
assert!(asked[1].starts_with("POST /moderation/bans?broadcaster_id=100&moderator_id=42 "), "{}", asked[1]);
assert!(asked[1].contains(r#""user_id":"7""#) && asked[1].contains(r#""duration":600"#) && asked[1].contains(r#""reason":"spam""#), "{}", asked[1]);
assert!(asked[1].to_lowercase().contains("authorization: bearer tok") && asked[1].to_lowercase().contains("client-id: cid"));
}
#[tokio::test]
async fn a_person_whose_id_is_known_is_not_looked_up_and_one_who_does_not_exist_is_said() {
let (root, seen) = helix_server(vec![(200, "{}"), (200, r#"{"data":[]}"#)]).await;
let client = reqwest::Client::new();
run(&client, &cx(&root, true), &Action::Ban { user: "bob".into(), reason: String::new() }, Some("7")).await.unwrap();
assert_eq!(seen.lock().unwrap().len(), 1, "the id was given, so only the ban was asked for");
let e = run(&client, &cx(&root, true), &Action::Unban { user: "ghost".into() }, None).await.unwrap_err();
assert!(e.contains("no user called ghost"), "{e}");
}
#[tokio::test]
async fn the_rooms_modes_are_patched_and_deleting_names_the_message() {
let (root, seen) = helix_server(vec![(200, "{}"), (200, "{}"), (204, "{}")]).await;
let client = reqwest::Client::new();
run(&client, &cx(&root, true), &Action::Slow(Some(30)), None).await.unwrap();
run(&client, &cx(&root, true), &Action::Followers(Some(60)), None).await.unwrap();
run(&client, &cx(&root, true), &Action::Delete { message_id: Some("m1".into()) }, None).await.unwrap();
let asked = seen.lock().unwrap().clone();
assert!(asked[0].starts_with("PATCH /chat/settings?broadcaster_id=100&moderator_id=42 ") && asked[0].contains(r#""slow_mode_wait_time":30"#), "{}", asked[0]);
assert!(asked[1].contains(r#""follower_mode":true"#) && asked[1].contains(r#""follower_mode_duration":60"#), "{}", asked[1]);
assert!(asked[2].starts_with("DELETE /moderation/chat?broadcaster_id=100&moderator_id=42&message_id=m1 "), "{}", asked[2]);
}
#[tokio::test]
async fn not_being_a_moderator_stops_it_before_twitch_is_asked_and_twitchs_no_is_said() {
let (root, seen) = helix_server(vec![(400, r#"{"message":"The user specified in user_id is already banned."}"#), (401, r#"{"message":"Missing scope: moderator:manage:banned_users"}"#)]).await;
let client = reqwest::Client::new();
let e = run(&client, &cx(&root, false), &Action::Slow(None), None).await.unwrap_err();
assert!(e.contains("not a moderator") && seen.lock().unwrap().is_empty());
let e = run(&client, &cx(&root, true), &Action::Ban { user: "bob".into(), reason: String::new() }, Some("7")).await.unwrap_err();
assert!(e.contains("already banned"), "{e}");
let e = run(&client, &cx(&root, true), &Action::Ban { user: "bob".into(), reason: String::new() }, Some("7")).await.unwrap_err();
assert!(e.contains("Sign in again"), "{e}");
// Whispers and colours are what a person does as themselves, with no moderator needed.
let (root, _) = helix_server(vec![(204, "{}"), (204, "{}")]).await;
run(&client, &cx(&root, false), &Action::Color("coral".into()), None).await.unwrap();
run(&client, &cx(&root, false), &Action::Whisper { to: "bob".into(), text: "hi".into() }, Some("7")).await.unwrap();
}
}
src/backend/twitch/playback.rs
+471
View File
@@ -0,0 +1,471 @@
//! The address of a live channel's video.
//!
//! twitch.tv's own player asks Twitch for a signed token (a persisted GraphQL
//! query, see `gql.rs`) and then for the channel's playlist from the playlist
//! service, passing the token along. Both are reproduced here, and the second is
//! only *built* - the player in the window fetches it, as it fetches every
//! playlist and segment after it. What this returns is that address, when it
//! stops being good (the token lasts twenty minutes), and what the token says
//! about the account: whether ads are for it, which renditions it may not have.
//!
//! Anonymous works: no account, no sign-in, no browser. An account's own
//! `auth-token` (the cookie of a twitch.tv session, see Settings) is sent when
//! there is one, and is what makes a person's Turbo or subscription count.
//!
//! Twitch can demand an *integrity token*, which is proof that a real browser
//! is asking, and sometimes has, for a week at a time. It is not demanded
//! today. When it is, the daemon asks the window to make one (in a hidden page
//! on twitch.tv, which is the only place it can be made), keeps it until it is
//! nearly out, and asks again.
use super::gql::{self, Who};
use crate::state::AppState;
use anyhow::anyhow;
use serde_json::{json, Value};
use std::collections::HashMap;
use std::sync::Mutex;
/// Where the playlist service is.
pub const USHER_ROOT: &str = "https://usher.ttvnw.net";
/// The player's own query for the token, by the hash it is stored under at Twitch.
const TOKEN_HASH: &str = "ed230aa1e33e07eebb8928504583da78a5173989fadfb1ac94be06a04f3cdbe9";
/// The address of a stream and what is known about it.
#[derive(Debug, Clone, PartialEq)]
pub struct Playback {
/// The playlist, signed. Replaced before `expires`.
pub url: String,
/// When the signature stops being accepted, in Unix seconds.
pub expires: i64,
/// Whether Twitch says ads are not for this account.
pub hide_ads: bool,
/// Renditions this account may not have, by Twitch's names for them.
pub restricted: Vec<String>,
}
/// Why there is no address.
#[derive(Debug)]
pub enum PlaybackError {
/// The channel is not broadcasting.
Offline,
/// Twitch will not give this account the stream: subscribers only, and so on.
Forbidden(String),
/// Not available where this is being watched from.
Blocked(String),
/// Twitch wanted proof that a browser is asking, and none could be had.
NeedsIntegrity,
Failed(anyhow::Error),
}
impl std::fmt::Display for PlaybackError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
PlaybackError::Offline => write!(f, "this channel is not streaming"),
PlaybackError::Forbidden(why) if why.is_empty() => write!(f, "Twitch will not play this stream for this account (it may be subscribers only)"),
PlaybackError::Forbidden(why) => write!(f, "Twitch will not play this stream for this account: {why}"),
PlaybackError::Blocked(why) if why.is_empty() => write!(f, "this stream is not available where you are"),
PlaybackError::Blocked(why) => write!(f, "this stream is not available where you are ({why})"),
PlaybackError::NeedsIntegrity => write!(f, "Twitch is asking for proof that this is a browser, and moho could not give it"),
PlaybackError::Failed(e) => write!(f, "{e:#}"),
}
}
}
fn token_query(login: &str) -> Value {
json!({
"operationName": "PlaybackAccessToken",
"variables": { "isLive": true, "login": login, "isVod": false, "vodID": "", "playerType": "site", "platform": "web" },
"extensions": { "persistedQuery": { "version": 1, "sha256Hash": TOKEN_HASH } },
})
}
/// The playlist's address, from the signed token. Codecs are limited to H.264:
/// the window's player has no use for the AV1 and HEVC renditions Twitch offers
/// big channels, and a playlist that lists only what plays is one nothing has to
/// filter.
pub fn usher_url(root: &str, login: &str, signature: &str, token: &str) -> String {
let pass = std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).map(|d| d.subsec_nanos() % 999_999).unwrap_or(1);
let mut query = url::form_urlencoded::Serializer::new(String::new());
for (k, v) in [
("allow_source", "true"),
("allow_audio_only", "true"),
("fast_bread", "true"),
("p", &pass.to_string()),
("platform", "web"),
("player_backend", "mediaplayer"),
("playlist_include_framerate", "true"),
("reassignments_supported", "true"),
("sig", signature),
("supported_codecs", "h264"),
("token", token),
("transcode_mode", "cbr_v1"),
] {
query.append_pair(k, v);
}
format!("{root}/api/v2/channel/hls/{}.m3u8?{}", login.to_lowercase(), query.finish())
}
/// Reads Twitch's answer to the token query.
pub fn read_token(answer: &Value, login: &str, usher_root: &str) -> Result<Playback, PlaybackError> {
let token = answer.pointer("/data/streamPlaybackAccessToken").filter(|t| !t.is_null()).ok_or(PlaybackError::Offline)?;
let value = token.get("value").and_then(|v| v.as_str()).ok_or_else(|| PlaybackError::Failed(anyhow!("Twitch's token had no value")))?;
let signature = token.get("signature").and_then(|v| v.as_str()).ok_or_else(|| PlaybackError::Failed(anyhow!("Twitch's token had no signature")))?;
let about: Value = serde_json::from_str(value).map_err(|e| PlaybackError::Failed(anyhow!("Twitch's token was not readable: {e}")))?;
if about.pointer("/authorization/forbidden").and_then(|f| f.as_bool()).unwrap_or(false) {
return Err(PlaybackError::Forbidden(about.pointer("/authorization/reason").and_then(|r| r.as_str()).unwrap_or("").to_string()));
}
if let Some(why) = about.get("geoblock_reason").and_then(|g| g.as_str()).filter(|g| !g.is_empty()) {
return Err(PlaybackError::Blocked(why.to_string()));
}
Ok(Playback {
url: usher_url(usher_root, login, signature, value),
expires: about.get("expires").and_then(|e| e.as_i64()).unwrap_or_else(|| chrono::Utc::now().timestamp() + 20 * 60),
hide_ads: about.get("hide_ads").and_then(|h| h.as_bool()).unwrap_or(false),
restricted: about
.pointer("/chansub/restricted_bitrates")
.and_then(|r| r.as_array())
.map(|a| a.iter().filter_map(|n| n.as_str().map(str::to_string)).collect())
.unwrap_or_default(),
})
}
// ---------------------------------------------------------------------------
// The integrity token
// ---------------------------------------------------------------------------
#[derive(Clone)]
struct Integrity {
device_id: String,
token: String,
/// Unix seconds.
expires_at: i64,
}
/// Integrity tokens by the session they were made for ("" for anonymous): Twitch wants the one made as the
/// account for a call made as the account, and refuses one made anonymously for it.
fn cache() -> &'static Mutex<HashMap<String, Integrity>> {
static CACHE: std::sync::OnceLock<Mutex<HashMap<String, Integrity>>> = std::sync::OnceLock::new();
CACHE.get_or_init(Default::default)
}
type Waiting = HashMap<String, tokio::sync::oneshot::Sender<Result<(String, i64), String>>>;
fn waiting() -> &'static Mutex<Waiting> {
static WAITING: std::sync::OnceLock<Mutex<Waiting>> = std::sync::OnceLock::new();
WAITING.get_or_init(Default::default)
}
/// How long before its end an integrity token is let go of.
const INTEGRITY_MARGIN: i64 = 600;
fn good_integrity(session: Option<&str>) -> Option<Integrity> {
let held = cache().lock().unwrap().get(session.unwrap_or("")).cloned()?;
(held.expires_at - chrono::Utc::now().timestamp() > INTEGRITY_MARGIN).then_some(held)
}
/// The window's answer to a request for an integrity token: the token and when
/// it ends in Unix milliseconds, or why one could not be made.
pub fn provide_integrity(request_id: &str, outcome: Result<(String, i64), String>) -> bool {
match waiting().lock().unwrap().remove(request_id) {
Some(reply) => reply.send(outcome).is_ok(),
None => false,
}
}
/// Asks the window to make an integrity token and waits for it.
async fn obtain_integrity(state: &AppState, session: Option<&str>) -> Option<Integrity> {
if let Some(held) = good_integrity(session) {
return Some(held);
}
let request_id = format!("integrity-{}", crate::model::next_message_id());
let (reply, answer) = tokio::sync::oneshot::channel();
waiting().lock().unwrap().insert(request_id.clone(), reply);
let device_id = gql::device_id().to_string();
// The window that makes it asks Twitch as the account (the session goes to main alone: it is not forwarded on).
state.events.emit("twitchIntegrityNeeded", json!({ "requestId": request_id, "deviceId": device_id, "authToken": session.unwrap_or("") }));
let outcome = tokio::time::timeout(std::time::Duration::from_secs(60), answer).await;
waiting().lock().unwrap().remove(&request_id);
match outcome {
Ok(Ok(Ok((token, expires_ms)))) => {
let made = Integrity { device_id, token, expires_at: expires_ms / 1000 };
cache().lock().unwrap().insert(session.unwrap_or("").to_string(), made.clone());
Some(made)
}
Ok(Ok(Err(why))) => {
tracing::debug!("twitch: no integrity token: {why}");
None
}
_ => None,
}
}
/// Makes the integrity token for a session ahead of the call that will want it (it takes a few seconds and lasts
/// hours), so that call does not wait for it. Nothing is said if it cannot be made: the call tries again then.
pub async fn warm_up(state: &AppState, session: &str) {
let _ = obtain_integrity(state, Some(session)).await;
}
/// One call to the GraphQL as the person's twitch.tv session, with an integrity token if Twitch asks for one.
/// Answers the JSON, whose own `errors` are the caller's to read; a refusal for want of integrity is a `NeedsIntegrity`.
pub async fn call_as_session(state: &AppState, http: &reqwest::Client, gql_root: &str, body: &Value, web_token: &str) -> Result<Value, PlaybackError> {
let mut integrity = good_integrity(Some(web_token));
for attempt in 0..2 {
let who = Who { web_token: Some(web_token), integrity: integrity.as_ref().map(|i| (i.device_id.as_str(), i.token.as_str())) };
let answer = gql::call(http, gql_root, body, &who).await.map_err(PlaybackError::Failed)?;
if gql::error_of(&answer).map(|s| s.to_lowercase().contains("integrity")).unwrap_or(false) {
if attempt == 0 {
integrity = obtain_integrity(state, Some(web_token)).await;
if integrity.is_some() {
continue;
}
}
return Err(PlaybackError::NeedsIntegrity);
}
return Ok(answer);
}
Err(PlaybackError::NeedsIntegrity)
}
/// The address of a channel's video, or why there is none.
pub async fn playback(state: &AppState, http: &reqwest::Client, gql_root: &str, usher_root: &str, login: &str, web_token: Option<&str>) -> Result<Playback, PlaybackError> {
let mut integrity = good_integrity(web_token.filter(|t| !t.is_empty()));
for attempt in 0..2 {
let who = Who { web_token, integrity: integrity.as_ref().map(|i| (i.device_id.as_str(), i.token.as_str())) };
let answer = gql::call(http, gql_root, &token_query(login), &who).await.map_err(PlaybackError::Failed)?;
if let Some(said) = gql::error_of(&answer) {
if said.to_lowercase().contains("integrity") {
if attempt == 0 {
integrity = obtain_integrity(state, web_token.filter(|t| !t.is_empty())).await;
if integrity.is_some() {
continue;
}
}
return Err(PlaybackError::NeedsIntegrity);
}
return Err(PlaybackError::Failed(anyhow!("Twitch would not give the stream a token: {said}")));
}
return read_token(&answer, login, usher_root);
}
Err(PlaybackError::NeedsIntegrity)
}
#[cfg(test)]
mod tests {
use super::*;
use crate::backend::kick::testkit;
use std::sync::Arc;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
/// The integrity cache and the requests waiting on the window are the process's, so the tests that touch
/// them take turns.
fn turn() -> &'static tokio::sync::Mutex<()> {
static TURN: std::sync::OnceLock<tokio::sync::Mutex<()>> = std::sync::OnceLock::new();
TURN.get_or_init(Default::default)
}
/// What the token's JSON says, as captured from a live answer, with the fields varied.
fn token(extra: Value) -> Value {
let mut about = json!({
"adblock": false, "authorization": { "forbidden": false, "reason": "" }, "channel": "tylorturner", "channel_id": 106959846,
"chansub": { "restricted_bitrates": [], "view_until": 1924905600 }, "device_id": "abc", "expires": 1791950724,
"hide_ads": false, "https_required": true, "platform": "web", "player_type": "site", "server_ads": true, "show_ads": true
});
for (k, v) in extra.as_object().cloned().unwrap_or_default() {
about[k] = v;
}
json!({ "data": { "streamPlaybackAccessToken": { "value": about.to_string(), "signature": "deadbeef" } } })
}
#[test]
fn a_token_becomes_a_playlist_address_that_names_the_channel_and_the_signature() {
let p = read_token(&token(json!({})), "TylorTurner", USHER_ROOT).unwrap();
assert!(p.url.starts_with("https://usher.ttvnw.net/api/v2/channel/hls/tylorturner.m3u8?"), "{}", p.url);
assert!(p.url.contains("sig=deadbeef") && p.url.contains("allow_source=true") && p.url.contains("supported_codecs=h264"), "{}", p.url);
assert!(p.url.contains("token=%7B%22adblock%22"), "the token is the JSON, escaped: {}", p.url);
assert_eq!((p.expires, p.hide_ads), (1791950724, false));
}
#[test]
fn what_the_token_says_about_the_account_is_kept() {
let p = read_token(&token(json!({ "hide_ads": true, "chansub": { "restricted_bitrates": ["chunked", "720p60"] } })), "x", USHER_ROOT).unwrap();
assert!(p.hide_ads);
assert_eq!(p.restricted, vec!["chunked", "720p60"]);
}
#[test]
fn a_channel_that_is_not_streaming_has_no_token_and_a_stream_that_may_not_be_had_says_why() {
assert!(matches!(read_token(&json!({ "data": { "streamPlaybackAccessToken": null } }), "x", USHER_ROOT), Err(PlaybackError::Offline)));
match read_token(&token(json!({ "authorization": { "forbidden": true, "reason": "This stream is for subscribers only" } })), "x", USHER_ROOT) {
Err(PlaybackError::Forbidden(why)) => assert!(why.contains("subscribers only")),
other => panic!("{other:?}"),
}
match read_token(&token(json!({ "geoblock_reason": "blocked in your country" })), "x", USHER_ROOT) {
Err(PlaybackError::Blocked(why)) => assert_eq!(why, "blocked in your country"),
other => panic!("{other:?}"),
}
assert!(matches!(read_token(&json!({ "data": { "streamPlaybackAccessToken": { "value": "not json", "signature": "s" } } }), "x", USHER_ROOT), Err(PlaybackError::Failed(_))));
let said = PlaybackError::Forbidden(String::new()).to_string();
assert!(said.contains("subscribers only"), "{said}");
}
/// A GraphQL that answers each call with the next canned answer, and remembers the headers it was sent.
async fn graphql(answers: Vec<(u16, String)>) -> (String, Arc<Mutex<Vec<String>>>) {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let root = format!("http://{}", listener.local_addr().unwrap());
let seen: Arc<Mutex<Vec<String>>> = Default::default();
let log = seen.clone();
let answers = Arc::new(Mutex::new(answers.into_iter().collect::<std::collections::VecDeque<_>>()));
tokio::spawn(async move {
loop {
let Ok((mut sock, _)) = listener.accept().await else { return };
let (log, answers) = (log.clone(), answers.clone());
tokio::spawn(async move {
let mut buf = vec![0u8; 16384];
let mut got = 0;
loop {
let n = sock.read(&mut buf[got..]).await.unwrap_or(0);
if n == 0 {
break;
}
got += n;
let text = String::from_utf8_lossy(&buf[..got]).to_string();
if let Some(end) = text.find("\r\n\r\n") {
let length = text[..end].lines().find_map(|l| l.to_lowercase().strip_prefix("content-length:").map(|v| v.trim().parse::<usize>().unwrap_or(0))).unwrap_or(0);
if got >= end + 4 + length {
break;
}
}
}
log.lock().unwrap().push(String::from_utf8_lossy(&buf[..got]).to_string());
let (status, body) = answers.lock().unwrap().pop_front().unwrap_or((500, "{}".to_string()));
let reply = format!("HTTP/1.1 {status} X\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}", body.len());
let _ = sock.write_all(reply.as_bytes()).await;
});
}
});
(root, seen)
}
#[tokio::test]
async fn the_call_is_the_players_own_and_carries_the_accounts_token_only_when_there_is_one() {
let _turn = turn().lock().await;
cache().lock().unwrap().clear();
let state = testkit::simulated_daemon("twitch-playback");
let (root, seen) = graphql(vec![(200, token(json!({})).to_string()), (200, token(json!({ "hide_ads": true })).to_string())]).await;
let http = reqwest::Client::new();
let anonymous = playback(&state, &http, &root, USHER_ROOT, "xqc", None).await.unwrap();
assert!(!anonymous.hide_ads);
let signed = playback(&state, &http, &root, USHER_ROOT, "xqc", Some("web-session")).await.unwrap();
assert!(signed.hide_ads);
let asked = seen.lock().unwrap().clone();
let (first, second) = (asked[0].to_lowercase(), asked[1].to_lowercase());
assert!(first.contains("client-id: kimne78kx3ncx6brgo4mv6wki5h1ko") && first.contains("x-device-id: "), "{first}");
assert!(first.contains(r#""operationname":"playbackaccesstoken""#) && first.contains(&TOKEN_HASH.to_lowercase()), "{first}");
assert!(first.contains(r#""playertype":"site""#) && first.contains(r#""login":"xqc""#));
assert!(!first.contains("authorization:"), "an anonymous request carries no account");
assert!(second.contains("authorization: oauth web-session"), "{second}");
}
#[tokio::test]
async fn an_error_that_is_not_about_integrity_is_said_and_a_refusal_is_said_with_its_status() {
let _turn = turn().lock().await;
cache().lock().unwrap().clear();
let state = testkit::simulated_daemon("twitch-playback-errors");
let (root, _) = graphql(vec![
(200, json!({ "errors": [{ "message": "service timeout" }] }).to_string()),
(400, json!({ "message": "The \"Client-ID\" header is invalid." }).to_string()),
])
.await;
let http = reqwest::Client::new();
let e = playback(&state, &http, &root, USHER_ROOT, "xqc", None).await.unwrap_err();
assert!(e.to_string().contains("service timeout"), "{e}");
let e = playback(&state, &http, &root, USHER_ROOT, "xqc", None).await.unwrap_err();
assert!(e.to_string().contains("400") && e.to_string().contains("Client-ID"), "{e}");
}
#[tokio::test]
async fn when_twitch_wants_an_integrity_token_the_window_is_asked_and_the_call_is_made_again_with_it() {
let _turn = turn().lock().await;
cache().lock().unwrap().clear();
let state = testkit::simulated_daemon("twitch-playback-integrity");
cache().lock().unwrap().clear();
let (root, seen) = graphql(vec![(200, json!({ "errors": [{ "message": "failed integrity check" }] }).to_string()), (200, token(json!({})).to_string())]).await;
let mut events = state.events.subscribe();
let answerer = tokio::spawn(async move {
loop {
let event = events.recv().await.expect("the bus is open");
if event.name == "twitchIntegrityNeeded" {
let id = event.data["requestId"].as_str().unwrap().to_string();
assert_eq!(event.data["deviceId"], gql::device_id());
let ends = (chrono::Utc::now().timestamp() + 16 * 3600) * 1000;
assert!(provide_integrity(&id, Ok(("v4.local.made".to_string(), ends))));
return;
}
}
});
let got = playback(&state, &reqwest::Client::new(), &root, USHER_ROOT, "xqc", None).await.expect("works the second time");
answerer.await.unwrap();
assert!(got.url.contains("xqc.m3u8"));
let second = seen.lock().unwrap()[1].to_lowercase();
assert!(second.contains("client-integrity: v4.local.made"), "{second}");
// Kept: the next call carries it without asking again.
assert!(good_integrity(None).is_some());
cache().lock().unwrap().clear();
}
#[tokio::test]
async fn an_integrity_token_that_cannot_be_made_is_a_reason_not_a_hang() {
let _turn = turn().lock().await;
cache().lock().unwrap().clear();
let state = testkit::simulated_daemon("twitch-playback-integrity-fails");
cache().lock().unwrap().clear();
let (root, _) = graphql(vec![(200, json!({ "errors": [{ "message": "failed integrity check" }] }).to_string())]).await;
let mut events = state.events.subscribe();
tokio::spawn(async move {
loop {
let event = events.recv().await.expect("the bus is open");
if event.name == "twitchIntegrityNeeded" {
provide_integrity(event.data["requestId"].as_str().unwrap(), Err("the page would not load".to_string()));
return;
}
}
});
let e = playback(&state, &reqwest::Client::new(), &root, USHER_ROOT, "xqc", None).await.unwrap_err();
assert!(matches!(e, PlaybackError::NeedsIntegrity), "{e}");
assert!(!provide_integrity("nobody-asked", Err("late".into())), "a late answer to nothing is refused");
}
#[tokio::test]
async fn a_call_made_as_the_account_asks_for_an_integrity_token_made_as_the_account_and_keeps_it_apart_from_the_anonymous_one() {
let _turn = turn().lock().await;
cache().lock().unwrap().clear();
let state = testkit::simulated_daemon("twitch-session-integrity");
let (root, seen) = graphql(vec![(200, json!({ "errors": [{ "message": "failed integrity check" }] }).to_string()), (200, json!({ "data": { "x": 1 } }).to_string()), (200, json!({ "data": { "x": 2 } }).to_string())]).await;
let mut events = state.events.subscribe();
let answerer = tokio::spawn(async move {
loop {
let event = events.recv().await.expect("the bus is open");
if event.name == "twitchIntegrityNeeded" {
// Said to the window that makes it, so it can make it as the account.
assert_eq!(event.data["authToken"], "web-session");
let id = event.data["requestId"].as_str().unwrap().to_string();
let ends = (chrono::Utc::now().timestamp() + 16 * 3600) * 1000;
assert!(provide_integrity(&id, Ok(("v4.local.as-account".to_string(), ends))));
return;
}
}
});
let http = reqwest::Client::new();
call_as_session(&state, &http, &root, &json!({ "query": "mutation { x }" }), "web-session").await.unwrap();
answerer.await.unwrap();
assert!(seen.lock().unwrap()[1].contains("client-integrity: v4.local.as-account"));
// The token made as the account is the account's; an anonymous call has none held.
assert!(good_integrity(Some("web-session")).is_some());
assert!(good_integrity(None).is_none());
// And the next call as the account uses it without asking again.
call_as_session(&state, &http, &root, &json!({ "query": "mutation { x }" }), "web-session").await.unwrap();
assert!(seen.lock().unwrap()[2].contains("client-integrity: v4.local.as-account"));
cache().lock().unwrap().clear();
}
}
src/backend/twitch/points.rs
+746
View File
@@ -0,0 +1,746 @@
//! What a signed-in twitch.tv session can do in a channel that the official API cannot: see the channel
//! points balance, put points on a prediction, vote in a poll. (The bonus chest is deliberately not claimed from
//! here: it only appears to a viewer that reports watching, which moho does not do.)
//!
//! All of it is Twitch's private GraphQL (see gql.rs), and all of it as the person's own twitch.tv session - the
//! `auth-token` that signing in leaves with the account (see `playback`), since the token moho uses for chat
//! and the official API is not accepted there. Without that session these say so, in words, and do nothing.
//!
//! Nothing here runs by itself. A balance is read when the window shows the channel; a bet is placed and a vote
//! cast only when the person presses the button for it. The player's "watch time" reports,
//! which a points bot sends to earn faster, are not sent (see the plan, Gitea issue #55).
use super::gql;
use super::playback::{self, PlaybackError};
use crate::state::AppState;
use serde_json::{json, Value};
/// The balance in a channel.
#[derive(Debug, PartialEq, Clone)]
pub struct Points {
pub balance: Option<u64>,
/// What the channel calls its points, if it renamed them.
pub name: Option<String>,
}
const CONTEXT: &str = "query($login:String!){ user(login:$login){ channel{ self{ communityPoints{ balance } } communityPointsSettings{ name } } } }";
pub fn read_points(answer: &Value) -> Option<Points> {
let channel = answer.pointer("/data/user/channel").filter(|c| !c.is_null())?;
let points = channel.pointer("/self/communityPoints").filter(|p| !p.is_null());
Some(Points {
balance: points.and_then(|p| p.get("balance")).and_then(|b| b.as_u64()),
name: channel.pointer("/communityPointsSettings/name").and_then(|n| n.as_str()).filter(|n| !n.is_empty()).map(str::to_string),
})
}
/// What spending on a reward does, which decides what it takes to redeem it.
#[derive(Debug, PartialEq, Clone, Copy)]
pub enum Kind {
/// One the streamer made: by id, with a word if it asks for one.
Custom,
/// Twitch's "Highlight My Message": the message is typed.
Highlight,
/// Twitch's "Unlock a Random Sub Emote".
RandomEmote,
/// "Choose an Emote to Unlock": one of the channel's subscriber emotes the account does not have.
ChosenEmote,
/// "Modify a Single Emote": one of the changed versions Twitch offers of a subscriber emote.
ModifiedEmote,
/// One that cannot be had from here (it is sent from twitch.tv's own chat box).
Other,
}
/// One of the things a reward asks the person to pick from.
#[derive(Debug, PartialEq, Clone)]
pub struct Choice {
pub id: String,
pub label: String,
pub image: Option<String>,
/// For a changed version of an emote: the emote it is a change of (its name), and that emote's picture, so the
/// window can show the emotes first and then what can be done to the one picked.
pub group: Option<String>,
pub group_image: Option<String>,
}
/// One thing a channel trades its points for.
#[derive(Debug, PartialEq, Clone)]
pub struct Reward {
pub id: String,
pub title: String,
pub cost: u64,
pub prompt: Option<String>,
/// Whether it can be had right now: switched on, in stock and not paused.
pub enabled: bool,
pub paused: bool,
/// It asks the person to type something.
pub needs_input: bool,
pub color: Option<String>,
pub image: Option<String>,
pub kind: Kind,
/// What it asks the person to pick from, for the emote unlocks.
pub choices: Vec<Choice>,
/// Why it cannot be had from here, where that is known beforehand.
pub unavailable: Option<String>,
}
const REWARDS: &str = "query($login:String!){ user(login:$login){ channel{ id self{ communityPoints{ balance } } communityPointsSettings{ name customRewards{ id title cost prompt isEnabled isPaused isInStock isUserInputRequired backgroundColor image{ url } defaultImage{ url } } automaticRewards{ id type cost defaultCost isEnabled defaultBackgroundColor backgroundColor defaultImage{ url } } emoteVariants{ id isUnlockable emote{ id token } modifications{ id } } } } } }";
fn emote_image(id: &str) -> String {
format!("https://static-cdn.jtvnw.net/emoticons/v2/{id}/default/dark/1.0")
}
/// What a modification's two letters do to an emote, in words.
fn modification_name(id: &str) -> &'static str {
match id.rsplit('_').next().unwrap_or("") {
"BW" => "black and white",
"HF" => "flipped",
"SG" => "with sunglasses",
"SQ" => "squished",
"TK" => "thick",
_ => "changed",
}
}
/// A channel's rewards, its number, and this account's balance, from one answer. `owned` is the ids of the
/// emotes the account can already use there: an unlock does not offer those again, and a change is offered
/// only for those.
pub fn read_rewards(answer: &Value, owned: &std::collections::HashSet<String>) -> Option<(String, Vec<Reward>, Points)> {
let channel = answer.pointer("/data/user/channel").filter(|c| !c.is_null())?;
let flag = |r: &Value, k: &str, default: bool| r.get(k).and_then(|b| b.as_bool()).unwrap_or(default);
let mut rewards: Vec<Reward> = channel
.pointer("/communityPointsSettings/customRewards")
.and_then(|r| r.as_array())
.map(|list| {
list.iter()
.filter_map(|r| {
Some(Reward {
id: r.get("id")?.as_str()?.to_string(),
title: r.get("title")?.as_str()?.to_string(),
cost: r.get("cost")?.as_u64()?,
prompt: r.get("prompt").and_then(|p| p.as_str()).filter(|p| !p.is_empty()).map(str::to_string),
enabled: flag(r, "isEnabled", false) && flag(r, "isInStock", true) && !flag(r, "isPaused", false),
paused: flag(r, "isPaused", false),
needs_input: flag(r, "isUserInputRequired", false),
color: r.get("backgroundColor").and_then(|c| c.as_str()).map(str::to_string),
image: r.pointer("/image/url").or_else(|| r.pointer("/defaultImage/url")).and_then(|u| u.as_str()).map(str::to_string),
kind: Kind::Custom,
choices: Vec::new(),
unavailable: None,
})
})
.collect()
})
.unwrap_or_default();
// The emotes Twitch lets be unlocked or changed are the channel's `emoteVariants`, and only the ones it marks
// unlockable are in the pool (asking for another is refused as an invalid emote). The unlocks offer those the
// account does not have; a change is offered for those it does.
struct Variant {
id: String,
token: String,
unlockable: bool,
modifications: Vec<String>,
}
let variants: Vec<Variant> = channel
.pointer("/communityPointsSettings/emoteVariants")
.and_then(|v| v.as_array())
.into_iter()
.flatten()
.filter_map(|v| {
Some(Variant {
id: v.get("id")?.as_str()?.to_string(),
token: v.pointer("/emote/token").and_then(|t| t.as_str())?.to_string(),
unlockable: flag(v, "isUnlockable", false),
modifications: v.get("modifications").and_then(|m| m.as_array()).into_iter().flatten().filter_map(|m| m.get("id").and_then(|i| i.as_str()).map(str::to_string)).collect(),
})
})
.collect();
let unowned: Vec<Choice> = variants.iter().filter(|v| v.unlockable && !owned.contains(&v.id)).map(|v| Choice { id: v.id.clone(), label: v.token.clone(), image: Some(emote_image(&v.id)), group: None, group_image: None }).collect();
let changes: Vec<Choice> = variants
.iter()
.filter(|v| owned.contains(&v.id))
.flat_map(|v| {
v.modifications.iter().map(move |m| Choice { label: format!("{}, {}", v.token, modification_name(m)), image: Some(emote_image(m)), id: m.clone(), group: Some(v.token.clone()), group_image: Some(emote_image(&v.id)) })
})
.collect();
let variants = changes;
for r in channel.pointer("/communityPointsSettings/automaticRewards").and_then(|r| r.as_array()).into_iter().flatten() {
let (Some(id), Some(kind)) = (r.get("id").and_then(|i| i.as_str()), r.get("type").and_then(|t| t.as_str())) else { continue };
let cost = r.get("cost").and_then(|c| c.as_u64()).or_else(|| r.get("defaultCost").and_then(|c| c.as_u64())).unwrap_or(0);
let (title, prompt, kind, needs_input, choices) = match kind {
"SEND_HIGHLIGHTED_MESSAGE" => ("Highlight My Message", "Write something special and give it some glitz in chat", Kind::Highlight, true, Vec::new()),
"RANDOM_SUB_EMOTE_UNLOCK" => ("Unlock a Random Sub Emote", "Unlocks a random subscriber emote you do not have", Kind::RandomEmote, false, Vec::new()),
"CHOSEN_SUB_EMOTE_UNLOCK" => ("Choose an Emote to Unlock", "Pick a subscriber emote to unlock for 24 hours", Kind::ChosenEmote, false, unowned.clone()),
"CHOSEN_MODIFIED_SUB_EMOTE_UNLOCK" => ("Modify a Single Emote", "Change one of this channel's emotes you have", Kind::ModifiedEmote, false, variants.clone()),
"SINGLE_MESSAGE_BYPASS_SUB_MODE" => ("Send a Message in Sub-Only Mode", "Send one message past subscribers-only mode", Kind::Other, false, Vec::new()),
// Paid for in Bits and not in points: not for here.
_ => continue,
};
let mut unavailable = None;
if kind == Kind::Other {
unavailable = Some("Send this one from twitch.tv's chat box".to_string());
} else if (matches!(kind, Kind::ChosenEmote | Kind::ModifiedEmote) && choices.is_empty()) || (kind == Kind::RandomEmote && unowned.is_empty()) {
unavailable = Some(if kind == Kind::ModifiedEmote { "You have none of this channel's emotes to change yet".to_string() } else { "There is nothing left here to unlock".to_string() });
}
rewards.push(Reward {
id: id.to_string(),
title: title.to_string(),
cost,
prompt: Some(prompt.to_string()),
enabled: flag(r, "isEnabled", false),
paused: false,
needs_input,
color: r.get("backgroundColor").and_then(|c| c.as_str()).or_else(|| r.get("defaultBackgroundColor").and_then(|c| c.as_str())).map(str::to_string),
image: r.pointer("/defaultImage/url").and_then(|u| u.as_str()).map(str::to_string),
kind,
choices,
unavailable,
});
}
Some((channel.get("id")?.as_str()?.to_string(), rewards, read_points(answer)?))
}
fn explain(e: PlaybackError) -> String {
match e {
PlaybackError::NeedsIntegrity => "Twitch wants proof this is a browser and moho could not make it. Try again in a moment".to_string(),
other => other.to_string(),
}
}
const NEEDS_SESSION: &str = "Sign in to Twitch from Accounts again: channel points are only available to the twitch.tv session a sign-in brings";
/// A fresh id for one vote or bet, so a request that is repeated is the same one and not a second. In the form
/// of a UUID, which is what Twitch's own page sends.
fn fresh_id() -> String {
use rand::RngCore;
let mut b = [0u8; 16];
rand::thread_rng().fill_bytes(&mut b);
b[6] = (b[6] & 0x0f) | 0x40;
b[8] = (b[8] & 0x3f) | 0x80;
let h: String = b.iter().map(|x| format!("{x:02x}")).collect();
format!("{}-{}-{}-{}-{}", &h[0..8], &h[8..12], &h[12..16], &h[16..20], &h[20..32])
}
/// A mutation's own refusal, in words. Twitch answers with a code in the payload, not an HTTP error.
fn refusal(answer: &Value, field: &str) -> Option<String> {
if let Some(said) = gql::error_of(answer) {
return Some(match said.as_str() {
"unauthenticated" => "Twitch no longer accepts the twitch.tv sign-in. Sign in to Twitch again from Accounts".to_string(),
other => other.to_string(),
});
}
let code = answer.pointer(&format!("/data/{field}/error/code")).and_then(|c| c.as_str())?;
Some(match code {
"NOT_ENOUGH_POINTS" | "INSUFFICIENT_FUNDS" => "Not enough channel points".to_string(),
"ALREADY_VOTED" | "ALREADY_PREDICTED" => "You have already answered that".to_string(),
"CANNOT_PREDICT_ON_BOTH_OUTCOMES" | "OTHER_OUTCOME_ALREADY_PREDICTED" => "You already put points on the other outcome".to_string(),
"EVENT_NOT_ACTIVE" | "POLL_NOT_ACTIVE" | "PREDICTION_LOCKED" => "That is closed".to_string(),
"GLOBAL_COOLDOWN" | "COOLDOWN" => "That reward is cooling down".to_string(),
"MAX_PER_STREAM" | "MAX_PER_USER_PER_STREAM" | "MAX_REDEMPTIONS" => "That reward has been had as many times as this stream allows".to_string(),
"REWARD_DISABLED" | "REWARD_NOT_ENABLED" | "REWARD_PAUSED" | "REWARD_OUT_OF_STOCK" => "That reward is not available right now".to_string(),
"CHANNEL_NOT_LIVE" | "STREAM_NOT_LIVE" => "That reward can be had while the channel is live".to_string(),
"NOT_FOUND" | "POLL_NOT_FOUND" | "EVENT_NOT_FOUND" | "CLAIM_NOT_FOUND" => "That is no longer there".to_string(),
"INVALID_EMOTE" | "EMOTE_NOT_UNLOCKABLE" | "EMOTE_ALREADY_UNLOCKED" | "EMOTE_NOT_FOUND" => "Twitch will not unlock that emote (it may already be yours, or not in this channel's pool)".to_string(),
"MAX_POINTS_EXCEEDED" => "That is more than the channel allows on one prediction".to_string(),
other => format!("Twitch refused: {}", other.to_lowercase().replace('_', " ")),
})
}
fn token(web_token: Option<&str>) -> Result<&str, String> {
web_token.filter(|t| !t.is_empty()).ok_or_else(|| NEEDS_SESSION.to_string())
}
/// The balance and any bonus waiting, for a channel by login.
pub async fn context(state: &AppState, http: &reqwest::Client, root: &str, login: &str, web_token: Option<&str>) -> Result<Points, String> {
let token = token(web_token)?;
let answer = playback::call_as_session(state, http, root, &gql::query(CONTEXT, json!({ "login": login })), token).await.map_err(explain)?;
if let Some(said) = gql::error_of(&answer) {
return Err(said);
}
read_points(&answer).ok_or_else(|| format!("Twitch has no channel points for {login}"))
}
/// Puts points on an outcome of a prediction.
pub async fn bet(state: &AppState, http: &reqwest::Client, root: &str, event_id: &str, outcome_id: &str, points: u64, web_token: Option<&str>) -> Result<(), String> {
let token = token(web_token)?;
const BET: &str = "mutation($i:MakePredictionInput!){ makePrediction(input:$i){ error{ code } } }";
let body = gql::query(BET, json!({ "i": { "eventID": event_id, "outcomeID": outcome_id, "points": points, "transactionID": fresh_id() } }));
let answer = playback::call_as_session(state, http, root, &body, token).await.map_err(explain)?;
refusal(&answer, "makePrediction").map_or(Ok(()), Err)
}
/// Votes in a poll with the free vote everybody has.
pub async fn vote(state: &AppState, http: &reqwest::Client, root: &str, poll_id: &str, choice_id: &str, user_id: &str, web_token: Option<&str>) -> Result<(), String> {
let token = token(web_token)?;
const VOTE: &str = "mutation($i:VoteInPollInput!){ voteInPoll(input:$i){ error{ code } } }";
let body = gql::query(VOTE, json!({ "i": { "pollID": poll_id, "choiceID": choice_id, "userID": user_id, "voteID": fresh_id() } }));
let answer = playback::call_as_session(state, http, root, &body, token).await.map_err(explain)?;
refusal(&answer, "voteInPoll").map_or(Ok(()), Err)
}
/// A channel's rewards and this account's balance: the panel over the chat lists them.
pub async fn rewards(state: &AppState, http: &reqwest::Client, root: &str, login: &str, owned: &std::collections::HashSet<String>, web_token: Option<&str>) -> Result<(String, Vec<Reward>, Points), String> {
let token = token(web_token)?;
let answer = playback::call_as_session(state, http, root, &gql::query(REWARDS, json!({ "login": login })), token).await.map_err(explain)?;
if let Some(said) = gql::error_of(&answer) {
return Err(said);
}
read_rewards(&answer, owned).ok_or_else(|| format!("Twitch has no channel points for {login}"))
}
/// Spends points on a reward. `input` is what a reward that asks for something was given.
pub async fn redeem(state: &AppState, http: &reqwest::Client, root: &str, channel_id: &str, reward: &Reward, input: Option<&str>, web_token: Option<&str>) -> Result<(), String> {
let token = token(web_token)?;
const REDEEM: &str = "mutation($i:RedeemCommunityPointsCustomRewardInput!){ redeemCommunityPointsCustomReward(input:$i){ error{ code } } }";
let mut i = json!({ "channelID": channel_id, "cost": reward.cost, "prompt": reward.prompt, "rewardID": reward.id, "title": reward.title, "transactionID": fresh_id() });
if let Some(text) = input.map(str::trim).filter(|t| !t.is_empty()) {
i["textInput"] = json!(text);
}
let answer = playback::call_as_session(state, http, root, &gql::query(REDEEM, json!({ "i": i })), token).await.map_err(explain)?;
refusal(&answer, "redeemCommunityPointsCustomReward").map_or(Ok(()), Err)
}
/// What the person gave a reward: the words typed for it, the emote picked for it.
#[derive(Default, Clone, Copy)]
pub struct Given<'a> {
pub input: Option<&'a str>,
pub choice: Option<&'a str>,
}
/// Spends points on one of Twitch's own rewards (a highlighted message, an emote unlock).
pub async fn redeem_automatic(state: &AppState, http: &reqwest::Client, root: &str, channel_id: &str, reward: &Reward, given: Given<'_>, web_token: Option<&str>) -> Result<(), String> {
let (input, choice) = (given.input, given.choice);
let token = token(web_token)?;
let base = json!({ "channelID": channel_id, "cost": reward.cost, "transactionID": fresh_id() });
let (name, input_type, mut i) = match reward.kind {
Kind::Highlight => {
let text = input.map(str::trim).filter(|t| !t.is_empty()).ok_or_else(|| "That reward asks you to type something".to_string())?;
let mut i = base;
i["message"] = json!(text);
("sendHighlightedChatMessage", "SendHighlightedChatMessageInput", i)
}
Kind::RandomEmote => ("unlockRandomSubscriberEmote", "UnlockRandomSubscriberEmoteInput", base),
Kind::ChosenEmote | Kind::ModifiedEmote => {
let emote = choice.filter(|c| reward.choices.iter().any(|o| o.id == *c)).ok_or_else(|| "Pick an emote first".to_string())?;
let mut i = base;
i["emoteID"] = json!(emote);
if reward.kind == Kind::ChosenEmote {
("unlockChosenSubscriberEmote", "UnlockChosenSubscriberEmoteInput", i)
} else {
("unlockChosenModifiedSubscriberEmote", "UnlockChosenModifiedSubscriberEmoteInput", i)
}
}
_ => return Err(reward.unavailable.clone().unwrap_or_else(|| "That reward cannot be had from here".to_string())),
};
let mutation = format!("mutation($i:{input_type}!){{ {name}(input:$i){{ error{{ code }} }} }}");
i["cost"] = json!(reward.cost);
let answer = playback::call_as_session(state, http, root, &gql::query(&mutation, json!({ "i": i })), token).await.map_err(explain)?;
refusal(&answer, name).map_or(Ok(()), Err)
}
// ---------------------------------------------------------------------------
// From a conversation: what the window's buttons call
// ---------------------------------------------------------------------------
/// The account behind a Twitch channel's conversation, and the channel's login.
pub(super) fn session_of(state: &AppState, buffer_id: &str) -> Result<(crate::accounts::TwitchAccountConfig, String), String> {
let buffer = state.runtime.get_buffer(buffer_id).filter(|b| b.account_id.starts_with("twitch:")).ok_or_else(|| "that is not a Twitch channel".to_string())?;
let cfg = state.accounts.get_twitch(&buffer.account_id).ok_or_else(|| "no such Twitch account".to_string())?;
Ok((cfg, buffer.name))
}
/// The id Twitch knows a card by, from the one the window knows it by (`poll:<id>`).
fn card_id(state: &AppState, buffer_id: &str, kind: &str) -> Result<(String, serde_json::Value), String> {
let card = state.runtime.live_card(buffer_id, kind).ok_or_else(|| format!("there is no {kind} open here"))?;
let id = card["id"].as_str().and_then(|i| i.strip_prefix(&format!("{kind}:"))).ok_or_else(|| format!("that {kind} has no id to answer"))?.to_string();
Ok((id, card))
}
/// Votes in the poll on screen, and shows the vote on its card at once (the tally follows from Twitch).
pub async fn vote_in_buffer(state: &AppState, root: &str, buffer_id: &str, choice_id: &str) -> Result<(), String> {
let (cfg, _) = session_of(state, buffer_id)?;
let (poll_id, mut card) = card_id(state, buffer_id, "poll")?;
let http = super::client().map_err(|e| format!("{e:#}"))?;
vote(state, &http, root, &poll_id, choice_id, &cfg.user_id, Some(&cfg.web_token)).await?;
card["hasVoted"] = json!(true);
card["votedOptionId"] = json!(choice_id);
crate::backend::kick::cards::publish_card(state, buffer_id, "poll", Some(card));
Ok(())
}
/// Puts points on the prediction on screen, and shows the bet on its card at once.
pub async fn bet_in_buffer(state: &AppState, root: &str, buffer_id: &str, outcome_id: &str, amount: u64) -> Result<(), String> {
let (cfg, channel) = session_of(state, buffer_id)?;
let (event_id, mut card) = card_id(state, buffer_id, "prediction")?;
if amount < super::realtime::MIN_BET as u64 {
return Err(format!("Bets are {} points or more", super::realtime::MIN_BET));
}
if let Some(balance) = card["balance"].as_u64() {
if amount > balance {
return Err("Not enough channel points".to_string());
}
}
let http = super::client().map_err(|e| format!("{e:#}"))?;
bet(state, &http, root, &event_id, outcome_id, amount, Some(&cfg.web_token)).await?;
let stake = card["stake"].as_f64().unwrap_or(0.0) + amount as f64;
card["hasVoted"] = json!(true);
card["votedOptionId"] = json!(outcome_id);
card["stake"] = json!(stake);
crate::backend::kick::cards::publish_card(state, buffer_id, "prediction", Some(card));
// What is left, from Twitch rather than this one minus the bet: a refund or a cap would make that wrong.
super::realtime::refresh_balance(state, root, &cfg.account_id(), &channel).await;
Ok(())
}
/// The balance and any bonus waiting in a conversation's channel; also said to the window as `twitchPoints`.
pub async fn points_in_buffer(state: &AppState, root: &str, buffer_id: &str) -> Result<Points, String> {
let (cfg, channel) = session_of(state, buffer_id)?;
let http = super::client().map_err(|e| format!("{e:#}"))?;
let points = context(state, &http, root, &channel, Some(&cfg.web_token)).await?;
state.events.emit("twitchPoints", json!({ "bufferId": buffer_id, "points": points.balance, "name": points.name }));
Ok(points)
}
/// The ids of the emotes this account can already use in a conversation's channel (the picker's entries are named by
/// a token that carries the id: `[twitch:emote:<id>:<name>]`).
fn owned_emotes(state: &AppState, account_id: &str, buffer_id: &str) -> std::collections::HashSet<String> {
state
.runtime
.emoji_catalogue(account_id)
.into_iter()
.filter(|s| s.buffers.iter().any(|b| b == buffer_id))
.flat_map(|s| s.emoji)
.filter(|e| !e.locked)
.filter_map(|e| e.id.strip_prefix("[twitch:").and_then(|t| t.split(':').nth(1)).map(str::to_string))
.collect()
}
/// The rewards of a conversation's channel, and the balance, for the panel; the balance is also said to the window.
pub async fn rewards_in_buffer(state: &AppState, root: &str, buffer_id: &str) -> Result<(Vec<Reward>, Points), String> {
let (cfg, channel) = session_of(state, buffer_id)?;
let http = super::client().map_err(|e| format!("{e:#}"))?;
let owned = owned_emotes(state, &cfg.account_id(), buffer_id);
let (_, rewards, points) = rewards(state, &http, root, &channel, &owned, Some(&cfg.web_token)).await?;
state.events.emit("twitchPoints", json!({ "bufferId": buffer_id, "points": points.balance, "name": points.name }));
// Twitch's proof that this is a browser takes a few seconds to make the first time (and then lasts for hours):
// made now, while the panel is being read, so a press of Redeem is not what waits for it.
if rewards.iter().any(|r| r.enabled && r.kind != Kind::Other) {
let (state, token) = (state.clone(), cfg.web_token.clone());
tokio::spawn(async move {
playback::warm_up(&state, &token).await;
});
}
Ok((rewards, points))
}
/// Redeems a reward of a conversation's channel by its id, after checking it is there, is on offer, and is affordable,
/// and answers the balance after. An emote unlock also reads the account's emotes again, so the new one is in the picker.
pub async fn redeem_in_buffer(state: &AppState, root: &str, buffer_id: &str, reward_id: &str, input: Option<&str>, choice: Option<&str>) -> Result<Points, String> {
let (cfg, channel) = session_of(state, buffer_id)?;
let http = super::client().map_err(|e| format!("{e:#}"))?;
let owned = owned_emotes(state, &cfg.account_id(), buffer_id);
let (channel_id, rewards, points) = rewards(state, &http, root, &channel, &owned, Some(&cfg.web_token)).await?;
let reward = rewards.into_iter().find(|r| r.id == reward_id).ok_or_else(|| "That reward is no longer there".to_string())?;
if let Some(why) = &reward.unavailable {
return Err(why.clone());
}
if !reward.enabled {
return Err(if reward.paused { "That reward is paused right now".to_string() } else { "That reward is not available right now".to_string() });
}
if reward.needs_input && input.map(str::trim).unwrap_or("").is_empty() {
return Err("That reward asks you to type something".to_string());
}
if let Some(balance) = points.balance {
if balance < reward.cost {
return Err(format!("You have {balance} of the {} points that costs", reward.cost));
}
}
match reward.kind {
Kind::Custom => redeem(state, &http, root, &channel_id, &reward, input, Some(&cfg.web_token)).await?,
_ => redeem_automatic(state, &http, root, &channel_id, &reward, Given { input, choice }, Some(&cfg.web_token)).await?,
}
if matches!(reward.kind, Kind::RandomEmote | Kind::ChosenEmote | Kind::ModifiedEmote) {
super::connect::refresh_user_emotes(state, &cfg.account_id(), &channel, &channel_id).await;
}
points_in_buffer(state, root, buffer_id).await
}
#[cfg(test)]
pub(super) mod tests {
use super::*;
#[test]
fn the_balance_and_the_bonus_are_read_and_a_signed_out_answer_has_neither() {
let signed_in = json!({ "data": { "user": { "channel": { "self": { "communityPoints": { "balance": 4200 } }, "communityPointsSettings": { "name": "Sloths" } } } } });
assert_eq!(read_points(&signed_in), Some(Points { balance: Some(4200), name: Some("Sloths".into()) }));
let anonymous = json!({ "data": { "user": { "channel": { "self": { "communityPoints": null }, "communityPointsSettings": { "name": null } } } } });
assert_eq!(read_points(&anonymous), Some(Points { balance: None, name: None }));
assert_eq!(read_points(&json!({ "data": { "user": null } })), None);
}
#[test]
fn a_refusal_is_said_in_words_whether_it_came_as_an_error_or_in_the_payload() {
let payload = json!({ "data": { "makePrediction": { "error": { "code": "NOT_ENOUGH_POINTS" } } } });
assert_eq!(refusal(&payload, "makePrediction").as_deref(), Some("Not enough channel points"));
let auth = json!({ "errors": [{ "message": "unauthenticated" }], "data": { "makePrediction": null } });
assert!(refusal(&auth, "makePrediction").unwrap().contains("Sign in to Twitch again"));
let fine = json!({ "data": { "makePrediction": { "error": null } } });
assert_eq!(refusal(&fine, "makePrediction"), None);
let odd = json!({ "data": { "voteInPoll": { "error": { "code": "SOMETHING_NEW" } } } });
assert_eq!(refusal(&odd, "voteInPoll").as_deref(), Some("Twitch refused: something new"));
}
#[test]
fn an_id_for_a_vote_is_a_uuid_and_is_new_each_time() {
let a = fresh_id();
assert_eq!(a.len(), 36);
assert_eq!(a.matches('-').count(), 4);
assert_eq!(&a[14..15], "4");
assert_ne!(a, fresh_id());
}
use crate::accounts::TwitchAccountConfig;
use crate::backend::kick::testkit;
use std::sync::{Arc, Mutex};
use tokio::io::{AsyncReadExt, AsyncWriteExt};
/// A GraphQL that answers by what was asked, and keeps what it was sent (headers and body).
pub(in crate::backend::twitch) async fn fake_gql(answer: impl Fn(&str) -> String + Send + Sync + 'static) -> (String, Arc<Mutex<Vec<String>>>) {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let root = format!("http://{}", listener.local_addr().unwrap());
let seen: Arc<Mutex<Vec<String>>> = Default::default();
let (log, answer) = (seen.clone(), Arc::new(answer));
tokio::spawn(async move {
loop {
let Ok((mut sock, _)) = listener.accept().await else { return };
let (log, answer) = (log.clone(), answer.clone());
tokio::spawn(async move {
let mut buf = vec![0u8; 16384];
let mut got = 0;
loop {
let n = sock.read(&mut buf[got..]).await.unwrap_or(0);
if n == 0 {
break;
}
got += n;
let text = String::from_utf8_lossy(&buf[..got]).to_string();
if let Some(end) = text.find("\r\n\r\n") {
let length = text[..end].lines().find_map(|l| l.to_lowercase().strip_prefix("content-length:").map(|v| v.trim().parse::<usize>().unwrap_or(0))).unwrap_or(0);
if got >= end + 4 + length {
break;
}
}
}
let request = String::from_utf8_lossy(&buf[..got]).to_string();
let body = answer(&request);
log.lock().unwrap().push(request);
let reply = format!("HTTP/1.1 200 X\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}", body.len());
let _ = sock.write_all(reply.as_bytes()).await;
});
}
});
(root, seen)
}
pub(in crate::backend::twitch) fn channel(name: &str, web_token: &str) -> (AppState, String) {
let state = testkit::simulated_daemon(name);
state.accounts.add_twitch(TwitchAccountConfig { login: "me".into(), user_id: "42".into(), access_token: "t".into(), refresh_token: "r".into(), expires_at: chrono::Utc::now().timestamp() + 9000, web_token: web_token.into(), ..Default::default() }).unwrap();
state.runtime.ensure_buffer(&state, "twitch:me", "xqc", "channel");
(state, "twitch:me|xqc".to_string())
}
#[tokio::test]
async fn a_vote_is_sent_as_the_twitch_session_and_shows_on_the_card() {
let (state, buffer) = channel("twitch-points-vote", "websession");
state.runtime.set_live_card(&buffer, "poll", json!({ "id": "poll:q1", "kind": "poll", "options": [] }));
let (root, seen) = fake_gql(|_| r#"{"data":{"voteInPoll":{"error":null}}}"#.to_string()).await;
vote_in_buffer(&state, &root, &buffer, "choice-x").await.unwrap();
let request = seen.lock().unwrap()[0].clone();
assert!(request.to_lowercase().contains("authorization: oauth websession"), "{request}");
assert!(request.contains("\"pollID\":\"q1\"") && request.contains("\"choiceID\":\"choice-x\"") && request.contains("\"userID\":\"42\""), "{request}");
assert!(request.contains("voteID"), "{request}");
let card = state.runtime.live_card(&buffer, "poll").unwrap();
assert_eq!((card["hasVoted"].as_bool(), card["votedOptionId"].as_str()), (Some(true), Some("choice-x")));
}
#[tokio::test]
async fn a_refusal_is_said_and_the_card_is_left_alone() {
let (state, buffer) = channel("twitch-points-refused", "websession");
state.runtime.set_live_card(&buffer, "poll", json!({ "id": "poll:q1", "kind": "poll", "options": [] }));
let (root, _) = fake_gql(|_| r#"{"data":{"voteInPoll":{"error":{"code":"ALREADY_VOTED"}}}}"#.to_string()).await;
assert_eq!(vote_in_buffer(&state, &root, &buffer, "x").await.unwrap_err(), "You have already answered that");
assert!(state.runtime.live_card(&buffer, "poll").unwrap()["hasVoted"].is_null());
// And no session, no call.
let (state, buffer) = channel("twitch-points-nosession", "");
state.runtime.set_live_card(&buffer, "poll", json!({ "id": "poll:q1", "kind": "poll" }));
let e = vote_in_buffer(&state, &root, &buffer, "x").await.unwrap_err();
assert!(e.contains("Sign in to Twitch from Accounts again"), "{e}");
}
#[tokio::test]
async fn a_bet_checks_its_size_places_it_and_asks_what_is_left() {
let (state, buffer) = channel("twitch-points-bet", "websession");
state.runtime.set_live_card(&buffer, "prediction", json!({ "id": "prediction:e1", "kind": "prediction", "balance": 500, "options": [] }));
let (root, seen) = fake_gql(|req| {
if req.contains("makePrediction") {
r#"{"data":{"makePrediction":{"error":null}}}"#.to_string()
} else {
r#"{"data":{"user":{"channel":{"self":{"communityPoints":{"balance":400}},"communityPointsSettings":{"name":null}}}}}"#.to_string()
}
})
.await;
assert!(bet_in_buffer(&state, &root, &buffer, "o1", 5).await.unwrap_err().contains("10 points or more"));
assert_eq!(bet_in_buffer(&state, &root, &buffer, "o1", 900).await.unwrap_err(), "Not enough channel points");
assert!(seen.lock().unwrap().is_empty(), "a bet that cannot be placed is not sent");
bet_in_buffer(&state, &root, &buffer, "o1", 100).await.unwrap();
let requests = seen.lock().unwrap().clone();
assert!(requests[0].contains("\"eventID\":\"e1\"") && requests[0].contains("\"outcomeID\":\"o1\"") && requests[0].contains("\"points\":100") && requests[0].contains("transactionID"), "{}", requests[0]);
let card = state.runtime.live_card(&buffer, "prediction").unwrap();
assert_eq!((card["votedOptionId"].as_str(), card["stake"].as_f64(), card["balance"].as_u64()), (Some("o1"), Some(100.0), Some(400)));
}
/// Against the real Twitch, with nothing at stake: reads a balance, and sends a vote and a bet that
/// name things that do not exist, which Twitch refuses in words. Shows the session is accepted and the
/// requests are shaped as it wants. `TWITCH_LIVE_WEB_TOKEN=... TWITCH_LIVE_USER_ID=... TWITCH_LIVE_CHANNEL=...
/// cargo test --release live_ -- --ignored --nocapture`.
#[tokio::test]
#[ignore]
async fn live_the_session_reads_a_balance_and_twitch_refuses_what_does_not_exist() {
let _ = rustls::crypto::ring::default_provider().install_default();
let (token, user, channel) = (std::env::var("TWITCH_LIVE_WEB_TOKEN").unwrap(), std::env::var("TWITCH_LIVE_USER_ID").unwrap(), std::env::var("TWITCH_LIVE_CHANNEL").unwrap());
let state = testkit::simulated_daemon("twitch-points-live");
let http = crate::backend::twitch::client().unwrap();
let root = gql::ROOT;
// A real prediction and outcome (from `TWITCH_LIVE_EVENT` and `TWITCH_LIVE_OUTCOME`), with more points than
// the account has: Twitch refuses it for want of points, which proves the bet is understood and spends nothing.
if let (Ok(event), Ok(outcome)) = (std::env::var("TWITCH_LIVE_EVENT"), std::env::var("TWITCH_LIVE_OUTCOME")) {
let amount: u64 = std::env::var("TWITCH_LIVE_BET").ok().and_then(|a| a.parse().ok()).unwrap_or(10);
println!("a bet of {amount} on a real prediction: {:?}", bet(&state, &http, root, &event, &outcome, amount, Some(&token)).await);
}
println!("balance: {:?}", context(&state, &http, root, &channel, Some(&token)).await);
// The channel's rewards, and a redeem of one that does not exist (nothing is spent).
match rewards(&state, &http, root, &channel, &Default::default(), Some(&token)).await {
Ok((channel_id, list, points)) => {
println!("rewards: {} offered, balance {:?}; first: {:?}", list.len(), points.balance, list.first().map(|r| (&r.title, r.cost, r.enabled)));
for r in list.iter().filter(|r| r.kind != Kind::Custom) {
println!(" own reward: {} | {} | enabled {} | {} choices | {:?}", r.title, r.cost, r.enabled, r.choices.len(), r.unavailable);
}
let fake = Reward { id: "00000000-0000-0000-0000-000000000000".into(), title: "nothing".into(), cost: 10, prompt: None, enabled: true, paused: false, needs_input: false, color: None, image: None, kind: Kind::Custom, choices: Vec::new(), unavailable: None };
println!("redeem a reward that is not there: {:?}", redeem(&state, &http, root, &channel_id, &fake, None, Some(&token)).await);
}
Err(e) => println!("rewards: {e}"),
}
println!("vote in a poll that is not there: {:?}", vote(&state, &http, root, "00000000-0000-0000-0000-000000000000", "00000000-0000-0000-0000-000000000001", &user, Some(&token)).await);
println!("bet on a prediction that is not there: {:?}", bet(&state, &http, root, "00000000-0000-0000-0000-000000000000", "00000000-0000-0000-0000-000000000001", 10, Some(&token)).await);
}
const REWARDS_ANSWER: &str = r##"{"data":{"user":{"channel":{"id":"71","self":{"communityPoints":{"balance":600}},"communityPointsSettings":{"name":"Sloths","customRewards":[
{"id":"r1","title":"Drink water","cost":500,"prompt":null,"isEnabled":true,"isPaused":false,"isInStock":true,"isUserInputRequired":false,"backgroundColor":"#6BB7FA","image":null,"defaultImage":{"url":"https://img/default.png"}},
{"id":"r2","title":"Say a name","cost":100,"prompt":"Who?","isEnabled":true,"isPaused":false,"isInStock":true,"isUserInputRequired":true,"backgroundColor":"#E37C10","image":{"url":"https://img/own.png"},"defaultImage":null},
{"id":"r3","title":"Dance","cost":50,"prompt":null,"isEnabled":true,"isPaused":true,"isInStock":true,"isUserInputRequired":false,"backgroundColor":null,"image":null,"defaultImage":null},
{"id":"r4","title":"Costly","cost":9000,"prompt":null,"isEnabled":true,"isPaused":false,"isInStock":true,"isUserInputRequired":false,"backgroundColor":null,"image":null,"defaultImage":null}],"automaticRewards":[
{"id":"71:SEND_HIGHLIGHTED_MESSAGE","type":"SEND_HIGHLIGHTED_MESSAGE","cost":null,"defaultCost":100,"isEnabled":true,"defaultBackgroundColor":"#FF6905","backgroundColor":null,"defaultImage":{"url":"https://img/hl.png"}},
{"id":"71:RANDOM_SUB_EMOTE_UNLOCK","type":"RANDOM_SUB_EMOTE_UNLOCK","cost":null,"defaultCost":320,"isEnabled":true,"defaultBackgroundColor":"#8205B4","backgroundColor":null,"defaultImage":null},
{"id":"71:CHOSEN_SUB_EMOTE_UNLOCK","type":"CHOSEN_SUB_EMOTE_UNLOCK","cost":null,"defaultCost":480,"isEnabled":true,"defaultBackgroundColor":"#00C8AF","backgroundColor":null,"defaultImage":null},
{"id":"71:CHOSEN_MODIFIED_SUB_EMOTE_UNLOCK","type":"CHOSEN_MODIFIED_SUB_EMOTE_UNLOCK","cost":900,"defaultCost":900,"isEnabled":true,"defaultBackgroundColor":"#00FA05","backgroundColor":null,"defaultImage":null},
{"id":"71:SINGLE_MESSAGE_BYPASS_SUB_MODE","type":"SINGLE_MESSAGE_BYPASS_SUB_MODE","cost":350,"defaultCost":350,"isEnabled":true,"defaultBackgroundColor":"#9146FF","backgroundColor":null,"defaultImage":null},
{"id":"71:SEND_GIGANTIFIED_EMOTE","type":"SEND_GIGANTIFIED_EMOTE","cost":null,"defaultCost":0,"isEnabled":true,"defaultBackgroundColor":"#69B6FF","backgroundColor":null,"defaultImage":null}],
"emoteVariants":[{"id":"e1","isUnlockable":true,"emote":{"id":"e1","token":"binxbWut"},"modifications":[]},{"id":"e2","isUnlockable":true,"emote":{"id":"e2","token":"binxbYAP"},"modifications":[]},{"id":"e3","isUnlockable":true,"emote":{"id":"e3","token":"binxbDrama"},"modifications":[{"id":"e3_BW"},{"id":"e3_SG"}]},{"id":"e4","isUnlockable":false,"emote":{"id":"e4","token":"binxbNope"},"modifications":[{"id":"e4_BW"}]}]}}}}}"##;
#[test]
fn a_channels_rewards_are_read_with_what_makes_each_available() {
let (channel, rewards, points) = read_rewards(&serde_json::from_str(REWARDS_ANSWER).unwrap(), &Default::default()).unwrap();
assert_eq!((channel.as_str(), points.balance, points.name.as_deref()), ("71", Some(600), Some("Sloths")));
let custom: Vec<(&str, u64, bool, bool, bool)> = rewards.iter().filter(|r| r.kind == Kind::Custom).map(|r| (r.id.as_str(), r.cost, r.enabled, r.paused, r.needs_input)).collect();
assert_eq!(custom, vec![("r1", 500, true, false, false), ("r2", 100, true, false, true), ("r3", 50, false, true, false), ("r4", 9000, true, false, false)]);
assert_eq!(rewards[0].image.as_deref(), Some("https://img/default.png"));
assert_eq!(rewards[1].image.as_deref(), Some("https://img/own.png"));
assert_eq!(rewards[1].prompt.as_deref(), Some("Who?"));
}
#[test]
fn twitchs_own_rewards_are_listed_with_what_each_takes_and_the_bits_ones_are_left_out() {
let owned: std::collections::HashSet<String> = ["e1", "e3"].iter().map(|s| s.to_string()).collect();
let (_, rewards, _) = read_rewards(&serde_json::from_str(REWARDS_ANSWER).unwrap(), &owned).unwrap();
let auto: Vec<&Reward> = rewards.iter().filter(|r| r.kind != Kind::Custom).collect();
let kinds: Vec<(Kind, u64, bool)> = auto.iter().map(|r| (r.kind, r.cost, r.needs_input)).collect();
// Highlight (typed), random, chosen, modified, and the one that is sent from twitch.tv; the Bits one is not here.
assert_eq!(kinds, vec![(Kind::Highlight, 100, true), (Kind::RandomEmote, 320, false), (Kind::ChosenEmote, 480, false), (Kind::ModifiedEmote, 900, false), (Kind::Other, 350, false)]);
// What the account already has is not offered again, nor is an emote Twitch does not put in the pool; a change is
// offered only for an emote the account has.
let chosen: Vec<&str> = auto[2].choices.iter().map(|c| c.label.as_str()).collect();
assert_eq!(chosen, vec!["binxbYAP"]);
let modified: Vec<&str> = auto[3].choices.iter().map(|c| c.label.as_str()).collect();
assert_eq!(modified, vec!["binxbDrama, black and white", "binxbDrama, with sunglasses"]);
assert_eq!(auto[4].unavailable.as_deref(), Some("Send this one from twitch.tv's chat box"));
// With every emote owned there is nothing to unlock, which is said.
let all: std::collections::HashSet<String> = ["e1", "e2", "e3"].iter().map(|s| s.to_string()).collect();
let (_, none_left, _) = read_rewards(&serde_json::from_str(REWARDS_ANSWER).unwrap(), &all).unwrap();
let chosen_none = none_left.iter().find(|r| r.kind == Kind::ChosenEmote).unwrap();
assert_eq!(chosen_none.unavailable.as_deref(), Some("There is nothing left here to unlock"));
}
#[tokio::test]
async fn the_automatic_rewards_are_redeemed_through_their_own_calls_with_what_each_needs() {
let (state, buffer) = channel("twitch-points-auto", "websession");
let (root, seen) = fake_gql(|req| {
if req.contains("\"mutation(") || req.contains("mutation($i") {
r#"{"data":{"x":{"error":null}}}"#.replace("\"x\"", if req.contains("sendHighlightedChatMessage") { "\"sendHighlightedChatMessage\"" } else if req.contains("unlockRandomSubscriberEmote") { "\"unlockRandomSubscriberEmote\"" } else if req.contains("unlockChosenModifiedSubscriberEmote") { "\"unlockChosenModifiedSubscriberEmote\"" } else { "\"unlockChosenSubscriberEmote\"" })
} else {
REWARDS_ANSWER.to_string()
}
})
.await;
redeem_in_buffer(&state, &root, &buffer, "71:SEND_HIGHLIGHTED_MESSAGE", Some("look at me"), None).await.unwrap();
redeem_in_buffer(&state, &root, &buffer, "71:RANDOM_SUB_EMOTE_UNLOCK", None, None).await.unwrap();
redeem_in_buffer(&state, &root, &buffer, "71:CHOSEN_SUB_EMOTE_UNLOCK", None, Some("e2")).await.unwrap();
redeem_in_buffer(&state, &root, &buffer, "71:CHOSEN_MODIFIED_SUB_EMOTE_UNLOCK", None, Some("e3_SG")).await.unwrap_err();
let requests = seen.lock().unwrap().clone();
let sent = |name: &str| requests.iter().find(|r| r.contains(name) && r.contains("transactionID")).cloned().unwrap_or_default();
let highlight = sent("sendHighlightedChatMessage");
for part in ["\"channelID\":\"71\"", "\"cost\":100", "\"message\":\"look at me\"", "SendHighlightedChatMessageInput"] {
assert!(highlight.contains(part), "{part} in {highlight}");
}
assert!(sent("unlockRandomSubscriberEmote").contains("\"cost\":320"));
let chosen = sent("unlockChosenSubscriberEmote");
assert!(chosen.contains("\"emoteID\":\"e2\"") && chosen.contains("\"cost\":480"), "{chosen}");
// An emote that was not offered, and a message that was not typed, are stopped before anything is sent.
let before = requests.iter().filter(|r| r.contains("transactionID")).count();
assert_eq!(redeem_in_buffer(&state, &root, &buffer, "71:CHOSEN_SUB_EMOTE_UNLOCK", None, Some("not-offered")).await.unwrap_err(), "Pick an emote first");
assert_eq!(redeem_in_buffer(&state, &root, &buffer, "71:SEND_HIGHLIGHTED_MESSAGE", Some(" "), None).await.unwrap_err(), "That reward asks you to type something");
assert_eq!(redeem_in_buffer(&state, &root, &buffer, "71:SINGLE_MESSAGE_BYPASS_SUB_MODE", None, None).await.unwrap_err(), "Send this one from twitch.tv's chat box");
assert_eq!(seen.lock().unwrap().iter().filter(|r| r.contains("transactionID")).count(), before);
}
#[tokio::test]
async fn a_reward_is_redeemed_with_what_twitch_needs_and_a_refusal_is_stopped_before_it_is_sent() {
let (state, buffer) = channel("twitch-points-redeem", "websession");
let asked = Arc::new(Mutex::new(0));
let counter = asked.clone();
let (root, seen) = fake_gql(move |req| {
if req.contains("redeemCommunityPointsCustomReward") {
return r#"{"data":{"redeemCommunityPointsCustomReward":{"error":null}}}"#.to_string();
}
let mut n = counter.lock().unwrap();
*n += 1;
if *n <= 4 { REWARDS_ANSWER.to_string() } else { REWARDS_ANSWER.replace("\"balance\":600", "\"balance\":500") }
})
.await;
// Paused, too dear, asks for a word and was not given one, and not there: each said, none sent.
assert_eq!(redeem_in_buffer(&state, &root, &buffer, "r3", None, None).await.unwrap_err(), "That reward is paused right now");
assert_eq!(redeem_in_buffer(&state, &root, &buffer, "r4", None, None).await.unwrap_err(), "You have 600 of the 9000 points that costs");
assert_eq!(redeem_in_buffer(&state, &root, &buffer, "r2", Some(" "), None).await.unwrap_err(), "That reward asks you to type something");
assert_eq!(redeem_in_buffer(&state, &root, &buffer, "nope", None, None).await.unwrap_err(), "That reward is no longer there");
assert!(!seen.lock().unwrap().iter().any(|r| r.contains("redeemCommunityPointsCustomReward")), "nothing was spent");
// One that is on offer, with what it asks for.
let after = redeem_in_buffer(&state, &root, &buffer, "r2", Some("Greg"), None).await.unwrap();
assert_eq!(after.balance, Some(500));
let request = seen.lock().unwrap().iter().find(|r| r.contains("redeemCommunityPointsCustomReward")).cloned().unwrap();
for part in ["\"channelID\":\"71\"", "\"rewardID\":\"r2\"", "\"cost\":100", "\"title\":\"Say a name\"", "\"prompt\":\"Who?\"", "\"textInput\":\"Greg\"", "transactionID"] {
assert!(request.contains(part), "{part} in {request}");
}
assert!(request.to_lowercase().contains("authorization: oauth websession"));
}
}
src/backend/twitch/realtime.rs
+695
View File
@@ -0,0 +1,695 @@
//! What Twitch says as it happens, beyond the chat: a channel that raids another,
//! the poll it runs, the prediction it opens.
//!
//! twitch.tv's own pages hear these over a websocket of their own (Hermes), by
//! subscribing to a topic per channel and kind of thing. It can be listened to
//! with no account at all - the subscriptions are accepted anonymously, which
//! is how a logged-out visitor sees a poll - so this works for every channel
//! every account has open. Like the GraphQL it is private and not Twitch's to
//! promise (see gql.rs and the plan, Gitea issue #55), and a socket that will
//! not open costs nothing but these cards.
//!
//! One socket per account (so it goes by that account's route), subscribed to
//! three topics per open channel: `raid.<id>`, `polls.<id>` and
//! `predictions-channel-v1.<id>`. Each message is a JSON string inside the
//! envelope; what it says is read by the plain functions below and the cards
//! are the ones Kick's polls and predictions already use.
//!
//! A poll or a prediction that began before the channel was opened is asked for
//! once, from the GraphQL, since the socket only says what changes.
use super::gql;
use crate::state::AppState;
use anyhow::{Context, Result};
use futures::{SinkExt, StreamExt};
use serde_json::{json, Value};
use std::collections::HashMap;
use std::sync::Mutex;
use std::time::Duration;
use tokio::sync::mpsc;
use tokio_tungstenite::tungstenite::Message as WsMessage;
pub const HERMES_URL: &str = "wss://hermes.twitch.tv/v1?clientId=kimne78kx3ncx6brgo4mv6wki5h1ko";
/// How long without a word (it sends a keepalive every 15 seconds) before the connection is taken for dead.
const SILENCE: Duration = Duration::from_secs(60);
/// What the connection is asked to do.
pub enum Watch {
Add { channel: String, room_id: String },
Remove { channel: String },
}
fn registry() -> &'static Mutex<HashMap<String, mpsc::UnboundedSender<Watch>>> {
static REGISTRY: std::sync::OnceLock<Mutex<HashMap<String, mpsc::UnboundedSender<Watch>>>> = std::sync::OnceLock::new();
REGISTRY.get_or_init(Default::default)
}
/// Makes the way to tell an account's socket what to listen to. Called when
/// the account starts; the receiver is `run`'s.
pub fn register(account_id: &str) -> mpsc::UnboundedReceiver<Watch> {
let (tx, rx) = mpsc::unbounded_channel();
registry().lock().unwrap().insert(account_id.to_string(), tx);
rx
}
/// Listens for a channel, now and after any reconnection.
pub fn watch(account_id: &str, channel: &str, room_id: &str) {
if let Some(tx) = registry().lock().unwrap().get(account_id) {
let _ = tx.send(Watch::Add { channel: channel.to_string(), room_id: room_id.to_string() });
}
}
pub fn unwatch(account_id: &str, channel: &str) {
if let Some(tx) = registry().lock().unwrap().get(account_id) {
let _ = tx.send(Watch::Remove { channel: channel.to_string() });
}
}
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
enum Kind {
Raid,
Polls,
Predictions,
}
impl Kind {
const ALL: [Kind; 3] = [Kind::Raid, Kind::Polls, Kind::Predictions];
fn topic(self, room_id: &str) -> String {
match self {
Kind::Raid => format!("raid.{room_id}"),
Kind::Polls => format!("polls.{room_id}"),
Kind::Predictions => format!("predictions-channel-v1.{room_id}"),
}
}
}
// ---------------------------------------------------------------------------
// Reading what was said
// ---------------------------------------------------------------------------
/// What a message means.
#[derive(Debug, PartialEq)]
pub enum Event {
/// The channel is sending its viewers to another: who, and how many.
Raid { id: String, login: String, display: String, viewers: Option<u64> },
RaidCalledOff { id: String },
/// A poll or prediction card, and whether it is the last word on it.
Card { kind: &'static str, card: Value },
/// The poll or prediction is over and filed away: take the card down.
CardGone { kind: &'static str },
}
fn text<'a>(v: &'a Value, keys: &[&str]) -> Option<&'a str> {
keys.iter().find_map(|k| v.get(*k).and_then(|x| x.as_str())).filter(|s| !s.is_empty())
}
fn number(v: &Value, keys: &[&str]) -> Option<f64> {
keys.iter().find_map(|k| v.get(*k).and_then(|x| x.as_f64()))
}
/// Reads one pubsub message of the given kind. `now` is unix seconds, for the clock of a prediction (which carries
/// when it began and not how long is left).
pub fn read(kind: &str, payload: &Value, now: i64) -> Option<Event> {
let what = payload.get("type")?.as_str()?;
match kind {
"raid" => {
let raid = payload.get("raid")?;
let id = text(raid, &["id"])?.to_string();
match what {
"raid_update_v2" => {
let login = text(raid, &["target_login"])?.to_string();
Some(Event::Raid {
display: text(raid, &["target_display_name"]).unwrap_or(&login).to_string(),
viewers: number(raid, &["viewer_count"]).map(|v| v as u64).filter(|v| *v > 0),
login,
id,
})
}
"raid_cancel_v2" => Some(Event::RaidCalledOff { id }),
_ => None,
}
}
"polls" => {
let poll = payload.pointer("/data/poll")?;
match what {
"POLL_ARCHIVE" => Some(Event::CardGone { kind: "poll" }),
_ => poll_card(poll, now).map(|card| Event::Card { kind: "poll", card }),
}
}
"predictions" => {
let event = payload.pointer("/data/event")?;
let card = prediction_card(event, now)?;
// Called off: nothing was decided, and there is nothing to show of it.
if card["state"].as_str() == Some("canceled") {
return Some(Event::CardGone { kind: "prediction" });
}
Some(Event::Card { kind: "prediction", card })
}
_ => None,
}
}
fn parse_time(v: &Value, keys: &[&str]) -> Option<i64> {
text(v, keys).and_then(|t| chrono::DateTime::parse_from_rfc3339(t).ok()).map(|d| d.timestamp())
}
/// A poll in the shape the card reads, from either of the shapes Twitch has it in: the socket's
/// (`poll_id`, `choice_id`, `duration_seconds`) and the GraphQL's (`id`, `durationSeconds`).
pub fn poll_card(poll: &Value, _now: i64) -> Option<Value> {
let id = text(poll, &["poll_id", "id"])?;
let status = text(poll, &["status"]).unwrap_or("ACTIVE").to_uppercase();
let open = status == "ACTIVE";
let duration = number(poll, &["duration_seconds", "durationSeconds"]).unwrap_or(0.0).max(0.0) as u32;
let remaining = if open { (number(poll, &["remaining_duration_milliseconds", "remainingDurationMilliseconds"]).unwrap_or(duration as f64 * 1000.0) / 1000.0).max(0.0).ceil() as u32 } else { 0 };
let choices = poll.get("choices")?.as_array()?;
let votes_of = |c: &Value| c.pointer("/votes/total").and_then(|t| t.as_f64()).unwrap_or(0.0);
let top = choices.iter().map(votes_of).fold(0.0_f64, f64::max);
let options: Vec<Value> = choices
.iter()
.filter_map(|c| {
let id = text(c, &["choice_id", "id"])?;
Some(json!({
"id": id,
"label": text(c, &["title"]).unwrap_or(""),
"votes": votes_of(c),
"backers": number(c, &["total_voters", "totalVoters"]),
// The one with the most, once it is decided.
"winner": !open && top > 0.0 && votes_of(c) == top,
}))
})
.collect();
Some(json!({
"kind": "poll",
"id": format!("poll:{id}"),
"title": text(poll, &["title"]).unwrap_or(""),
"options": options,
"duration": duration,
"remaining": remaining,
// The result stays up a while after it closes, as Twitch's own does.
"resultDisplayDuration": 20,
"hasVoted": false,
"votedOptionId": null,
"state": status.to_lowercase(),
"open": open,
// Twitch's polls are ended on twitch.tv by the streamer or a moderator, not from a viewer's card.
"noEnd": true,
}))
}
/// A prediction in the shape the card reads, from the socket's event or the GraphQL's.
pub fn prediction_card(event: &Value, now: i64) -> Option<Value> {
let id = text(event, &["id"])?;
let status = text(event, &["status"]).unwrap_or("ACTIVE").to_uppercase();
let open = status == "ACTIVE";
let window = number(event, &["prediction_window_seconds", "predictionWindowSeconds"]).unwrap_or(0.0).max(0.0) as u32;
let started = parse_time(event, &["created_at", "createdAt"]);
let remaining = match (open, started) {
(true, Some(started)) => (window as i64 - (now - started)).max(0) as u32,
(true, None) => window,
_ => 0,
};
let outcomes = event.get("outcomes")?.as_array()?;
let points_of = |o: &Value| number(o, &["total_points", "totalPoints"]).unwrap_or(0.0);
let pot: f64 = outcomes.iter().map(points_of).sum();
let winner = text(event, &["winning_outcome_id"]).map(str::to_string).or_else(|| event.pointer("/winningOutcome/id").and_then(|w| w.as_str()).map(str::to_string));
let options: Vec<Value> = outcomes
.iter()
.filter_map(|o| {
let oid = text(o, &["id"])?;
let points = points_of(o);
Some(json!({
"id": oid,
"label": text(o, &["title"]).unwrap_or(""),
"votes": points,
"backers": number(o, &["total_users", "totalUsers"]),
// What a point put on it pays back, written the way Kick's page writes it.
"odds": (points > 0.0 && pot > 0.0).then(|| format!("1:{:.1}", pot / points)),
"winner": winner.as_deref() == Some(oid),
}))
})
.collect();
let state = match status.as_str() {
"ACTIVE" => "open",
"LOCKED" => "locked",
"RESOLVE_PENDING" | "RESOLVED" => "resolved",
"CANCEL_PENDING" | "CANCELED" => "canceled",
other => return Some(json!({ "kind": "prediction", "id": format!("prediction:{id}"), "state": other.to_lowercase(), "options": options, "title": text(event, &["title"]).unwrap_or(""), "duration": window, "remaining": 0, "resultDisplayDuration": 0, "hasVoted": false, "votedOptionId": null, "total": pot, "minBet": MIN_BET, "noEnd": true })),
};
Some(json!({
"kind": "prediction",
"id": format!("prediction:{id}"),
"title": text(event, &["title"]).unwrap_or(""),
"options": options,
"duration": window,
"remaining": remaining,
"resultDisplayDuration": 0,
"hasVoted": false,
"votedOptionId": null,
"total": pot,
"state": state,
"minBet": MIN_BET,
"noEnd": true,
}))
}
/// The smallest bet Twitch takes.
pub const MIN_BET: u32 = 10;
// ---------------------------------------------------------------------------
// Saying it
// ---------------------------------------------------------------------------
/// Whether this account can answer a poll or a prediction: it has the twitch.tv session that Twitch takes votes
/// and bets from.
fn answerable(state: &AppState, account_id: &str) -> bool {
state.accounts.get_twitch(account_id).is_some_and(|a| !a.web_token.is_empty())
}
/// Publishes a card, carrying across what the socket cannot know: this account's own vote and bet and what it has to spend.
pub fn publish(state: &AppState, account_id: &str, buffer_id: &str, kind: &str, mut card: Value) {
card["readOnly"] = json!(!answerable(state, account_id));
if let Some(known) = state.runtime.live_card(buffer_id, kind) {
if known["id"] == card["id"] {
for key in ["hasVoted", "votedOptionId", "stake", "balance"] {
if !known[key].is_null() {
card[key] = known[key].clone();
}
}
}
}
crate::backend::kick::cards::publish_card(state, buffer_id, kind, Some(card));
}
fn gone(state: &AppState, buffer_id: &str, kind: &str) {
crate::backend::kick::cards::publish_card(state, buffer_id, kind, None);
}
/// A raid, said in the channel with a way to follow it.
fn raid_said(state: &AppState, account_id: &str, channel: &str, event: &Event) {
match event {
Event::Raid { login, display, viewers, .. } => {
let embed = crate::model::Embed { kind: Some("raid-out".into()), author: Some(display.clone()), title: viewers.map(|v| v.to_string()), description: Some(login.clone()), ..Default::default() };
state.runtime.record_message(state, account_id, channel, "channel", channel, &format!("{channel} is raiding {display}"), false, "raid", None, None, false, None, vec![embed], Vec::new(), None);
}
Event::RaidCalledOff { .. } => {
state.runtime.record_message(state, account_id, channel, "channel", channel, "The raid was called off", false, "system", None, None, false, None, Vec::new(), Vec::new(), None);
}
_ => {}
}
}
/// What has to be known to act on a message: which channel a subscription was for, and what kind it was.
type Subscriptions = HashMap<String, (String, Kind)>;
/// Acts on one pubsub message.
fn act(state: &AppState, account_id: &str, channel: &str, kind: Kind, payload: &Value, raids: &mut Vec<String>) {
let now = crate::backend::kick::cards::now_secs();
let name = match kind {
Kind::Raid => "raid",
Kind::Polls => "polls",
Kind::Predictions => "predictions",
};
let buffer_id = crate::model::buffer_id(account_id, channel);
match read(name, payload, now) {
Some(event @ Event::Raid { .. }) => {
if let Event::Raid { id, .. } = &event {
// The same raid is described again as its clock runs: said once.
if raids.contains(id) {
return;
}
raids.push(id.clone());
if raids.len() > 32 {
raids.remove(0);
}
}
raid_said(state, account_id, channel, &event);
}
Some(event @ Event::RaidCalledOff { .. }) => raid_said(state, account_id, channel, &event),
Some(Event::Card { kind, card }) => {
let fresh = kind == "prediction" && state.runtime.live_card(&buffer_id, kind).is_none_or(|known| known["id"] != card["id"]);
publish(state, account_id, &buffer_id, kind, card);
// A new prediction asks what this account has to bet with, once.
if fresh && answerable(state, account_id) {
let (state, account_id, channel) = (state.clone(), account_id.to_string(), channel.to_string());
tokio::spawn(async move { refresh_balance(&state, gql::ROOT, &account_id, &channel).await });
}
}
Some(Event::CardGone { kind }) => gone(state, &buffer_id, kind),
None => {}
}
}
/// Pulls out the pubsub payload a message carries: the topic's subscription id and the JSON inside.
pub fn notification(frame: &Value) -> Option<(String, Value)> {
let n = frame.get("notification")?;
let sub = n.pointer("/subscription/id")?.as_str()?.to_string();
let payload: Value = serde_json::from_str(n.get("pubsub")?.as_str()?).ok()?;
Some((sub, payload))
}
/// Reads what this account has to bet with in a channel and puts it on the prediction card that is up, if any.
pub async fn refresh_balance(state: &AppState, root: &str, account_id: &str, channel: &str) {
let Some(cfg) = state.accounts.get_twitch(account_id) else { return };
let Ok(http) = super::client() else { return };
let Ok(points) = super::points::context(state, &http, root, channel, Some(&cfg.web_token)).await else { return };
let (buffer_id, Some(balance)) = (crate::model::buffer_id(account_id, channel), points.balance) else { return };
if let Some(mut card) = state.runtime.live_card(&buffer_id, "prediction") {
card["balance"] = json!(balance);
// Directly: `publish` would carry the old balance back over the new one.
crate::backend::kick::cards::publish_card(state, &buffer_id, "prediction", Some(card));
}
state.events.emit("twitchPoints", json!({ "bufferId": buffer_id, "points": balance, "name": points.name }));
}
/// A poll or prediction the channel already has going, asked for from the GraphQL when it is opened.
pub async fn current(state: &AppState, account_id: &str, channel: &str, gql_root: &str) {
let Ok(http) = super::client() else { return };
const ASK: &str = "query($login:String!){ user(login:$login){ viewablePoll{ id title status durationSeconds remainingDurationMilliseconds choices{ id title votes{ total } totalVoters } } channel{ activePredictionEvents{ id title status createdAt predictionWindowSeconds winningOutcome{ id } outcomes{ id title totalPoints totalUsers } } } } }";
let answer = match gql::call(&http, gql_root, &gql::query(ASK, json!({ "login": channel })), &gql::Who::default()).await {
Ok(a) => a,
Err(e) => {
tracing::debug!("twitch[{account_id}]: {channel}'s poll and prediction: {e:#}");
return;
}
};
let now = crate::backend::kick::cards::now_secs();
let buffer_id = crate::model::buffer_id(account_id, channel);
if let Some(card) = answer.pointer("/data/user/viewablePoll").filter(|p| !p.is_null()).and_then(|p| poll_card(p, now)) {
publish(state, account_id, &buffer_id, "poll", card);
}
if let Some(card) = answer.pointer("/data/user/channel/activePredictionEvents/0").filter(|p| !p.is_null()).and_then(|p| prediction_card(p, now)) {
publish(state, account_id, &buffer_id, "prediction", card);
if answerable(state, account_id) {
refresh_balance(state, gql_root, account_id, channel).await;
}
}
}
// ---------------------------------------------------------------------------
// The connection
// ---------------------------------------------------------------------------
/// Listens for as long as the account runs, reconnecting with a growing wait.
pub async fn run(state: AppState, account_id: String, mut rx: mpsc::UnboundedReceiver<Watch>, url: String, gql_root: String) {
let mut watched: HashMap<String, String> = HashMap::new();
let mut wait = 3u64;
loop {
match session(&state, &account_id, &mut rx, &mut watched, &url, &gql_root).await {
Ok(()) => return,
Err(e) => {
tracing::debug!("twitch[{account_id}]: realtime ended: {e:#}");
// Anything asked for while it was down is kept in `watched` by the drain below.
while let Ok(w) = rx.try_recv() {
apply(&mut watched, w);
}
}
}
tokio::time::sleep(Duration::from_secs(wait)).await;
wait = (wait * 2).min(120);
}
}
fn apply(watched: &mut HashMap<String, String>, w: Watch) {
match w {
Watch::Add { channel, room_id } => {
watched.insert(channel, room_id);
}
Watch::Remove { channel } => {
watched.remove(&channel);
}
}
}
fn subscribe_frame(id: &str, topic: &str) -> String {
json!({
"type": "subscribe",
"id": id,
"subscribe": { "id": id, "type": "pubsub", "pubsub": { "topic": topic } },
"timestamp": chrono::Utc::now().to_rfc3339(),
})
.to_string()
}
async fn session(state: &AppState, account_id: &str, rx: &mut mpsc::UnboundedReceiver<Watch>, watched: &mut HashMap<String, String>, url: &str, gql_root: &str) -> Result<()> {
let (mut socket, _) = crate::net::route::websocket(super::SHARED_KEY, url).await.context("connecting to Twitch's realtime service")?;
let mut subs: Subscriptions = HashMap::new();
let mut next = 0u64;
let mut raids: Vec<String> = Vec::new();
// Nothing to listen to is nothing to hold open: the socket waits for the first channel.
let mut listening: Vec<String> = Vec::new();
let mut last_heard = tokio::time::Instant::now();
let mut check = tokio::time::interval(Duration::from_secs(15));
check.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Delay);
macro_rules! listen {
($channel:expr, $room:expr) => {{
let (channel, room): (String, String) = ($channel, $room);
if !room.is_empty() && !listening.contains(&channel) {
listening.push(channel.clone());
for kind in Kind::ALL {
let id = format!("s{next}");
next += 1;
socket.send(WsMessage::Text(subscribe_frame(&id, &kind.topic(&room)))).await?;
subs.insert(id, (channel.clone(), kind));
}
let (state, account_id, gql_root) = (state.clone(), account_id.to_string(), gql_root.to_string());
tokio::spawn(async move { current(&state, &account_id, &channel, &gql_root).await });
}
}};
}
for (channel, room) in watched.clone() {
listen!(channel, room);
}
loop {
tokio::select! {
frame = socket.next() => {
let Some(frame) = frame else { anyhow::bail!("the connection closed") };
match frame.context("reading from Twitch's realtime service")? {
WsMessage::Text(text) => {
last_heard = tokio::time::Instant::now();
let Ok(value) = serde_json::from_str::<Value>(&text) else { continue };
if value.get("type").and_then(|t| t.as_str()) != Some("notification") {
continue;
}
let Some((sub, payload)) = notification(&value) else { continue };
let Some((channel, kind)) = subs.get(&sub).cloned() else { continue };
// A channel closed since is no longer listened for.
if !watched.contains_key(&channel) {
continue;
}
act(state, account_id, &channel, kind, &payload, &mut raids);
}
WsMessage::Ping(data) => socket.send(WsMessage::Pong(data)).await?,
WsMessage::Close(_) => anyhow::bail!("Twitch's realtime service closed the connection"),
_ => {}
}
}
command = rx.recv() => {
let Some(command) = command else { return Ok(()) };
match command {
Watch::Add { channel, room_id } => {
watched.insert(channel.clone(), room_id.clone());
listen!(channel, room_id);
}
Watch::Remove { channel } => {
watched.remove(&channel);
listening.retain(|c| *c != channel);
}
}
}
_ = check.tick() => {
if !listening.is_empty() && last_heard.elapsed() > SILENCE {
anyhow::bail!("nothing heard from Twitch's realtime service for {} seconds", SILENCE.as_secs());
}
if listening.is_empty() {
last_heard = tokio::time::Instant::now();
}
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn prediction(status: &str) -> Value {
json!({ "type": "event-updated", "data": { "event": {
"id": "p1", "title": "Map 2", "status": status, "created_at": "2026-10-14T13:00:00Z", "prediction_window_seconds": 600,
"winning_outcome_id": if status == "RESOLVED" { json!("b") } else { Value::Null },
"outcomes": [{ "id": "a", "title": "Falcons", "total_points": 750, "total_users": 7 }, { "id": "b", "title": "NIP", "total_points": 250, "total_users": 3 }]
} } })
}
#[test]
fn a_prediction_counts_down_from_when_it_began_and_pays_by_its_pot() {
let began = chrono::DateTime::parse_from_rfc3339("2026-10-14T13:00:00Z").unwrap().timestamp();
let Some(Event::Card { kind, card }) = read("predictions", &prediction("ACTIVE"), began + 100) else { panic!("a card") };
assert_eq!(kind, "prediction");
assert_eq!(card["id"], "prediction:p1");
assert_eq!(card["remaining"], 500);
assert_eq!(card["state"], "open");
assert_eq!(card["total"], 1000.0);
assert_eq!(card["options"][0]["odds"], "1:1.3");
assert_eq!(card["options"][1]["odds"], "1:4.0");
assert_eq!(card["options"][1]["backers"], 3.0);
}
#[test]
fn a_locked_or_decided_prediction_has_no_time_left_and_names_its_winner() {
let began = chrono::DateTime::parse_from_rfc3339("2026-10-14T13:00:00Z").unwrap().timestamp();
let Some(Event::Card { card, .. }) = read("predictions", &prediction("LOCKED"), began + 100) else { panic!() };
assert_eq!((card["remaining"].as_u64(), card["state"].as_str()), (Some(0), Some("locked")));
let Some(Event::Card { card, .. }) = read("predictions", &prediction("RESOLVED"), began + 700) else { panic!() };
assert_eq!(card["state"], "resolved");
assert_eq!((card["options"][0]["winner"].as_bool(), card["options"][1]["winner"].as_bool()), (Some(false), Some(true)));
}
#[test]
fn a_called_off_prediction_is_taken_down() {
assert_eq!(read("predictions", &prediction("CANCELED"), 0), Some(Event::CardGone { kind: "prediction" }));
}
#[test]
fn a_poll_is_read_in_the_sockets_shape_and_in_the_graphqls() {
let socket = json!({ "type": "POLL_UPDATE", "data": { "poll": { "poll_id": "q1", "title": "Next?", "status": "ACTIVE", "duration_seconds": 60, "remaining_duration_milliseconds": 41_500,
"choices": [{ "choice_id": "x", "title": "Chess", "votes": { "total": 3 }, "total_voters": 3 }, { "choice_id": "y", "title": "Go", "votes": { "total": 1 }, "total_voters": 1 }] } } });
let Some(Event::Card { kind, card }) = read("polls", &socket, 0) else { panic!() };
assert_eq!((kind, card["id"].as_str(), card["remaining"].as_u64(), card["open"].as_bool()), ("poll", Some("poll:q1"), Some(42), Some(true)));
assert_eq!(card["options"][0], json!({ "id": "x", "label": "Chess", "votes": 3.0, "backers": 3.0, "winner": false }));
let graphql = json!({ "id": "q1", "title": "Next?", "status": "ACTIVE", "durationSeconds": 60, "remainingDurationMilliseconds": 10_000, "choices": [{ "id": "x", "title": "Chess", "votes": { "total": 3 }, "totalVoters": 3 }] });
assert_eq!(poll_card(&graphql, 0).unwrap()["remaining"], 10);
}
#[test]
fn a_finished_poll_shows_how_it_went_and_an_archived_one_goes() {
let done = json!({ "type": "POLL_COMPLETE", "data": { "poll": { "poll_id": "q1", "title": "Next?", "status": "COMPLETED", "duration_seconds": 60,
"choices": [{ "choice_id": "x", "title": "Chess", "votes": { "total": 3 } }, { "choice_id": "y", "title": "Go", "votes": { "total": 1 } }] } } });
let Some(Event::Card { card, .. }) = read("polls", &done, 0) else { panic!() };
assert_eq!((card["open"].as_bool(), card["remaining"].as_u64()), (Some(false), Some(0)));
assert_eq!((card["options"][0]["winner"].as_bool(), card["options"][1]["winner"].as_bool()), (Some(true), Some(false)));
let archived = json!({ "type": "POLL_ARCHIVE", "data": { "poll": { "poll_id": "q1" } } });
assert_eq!(read("polls", &archived, 0), Some(Event::CardGone { kind: "poll" }));
}
#[test]
fn a_raid_names_where_it_is_going_and_can_be_called_off() {
let raid = json!({ "type": "raid_update_v2", "raid": { "id": "r1", "target_login": "friend", "target_display_name": "Friend", "viewer_count": 120 } });
assert_eq!(read("raid", &raid, 0), Some(Event::Raid { id: "r1".into(), login: "friend".into(), display: "Friend".into(), viewers: Some(120) }));
let off = json!({ "type": "raid_cancel_v2", "raid": { "id": "r1" } });
assert_eq!(read("raid", &off, 0), Some(Event::RaidCalledOff { id: "r1".into() }));
assert_eq!(read("raid", &json!({ "type": "raid_go_v2", "raid": { "id": "r1" } }), 0), None);
}
#[test]
fn the_envelope_gives_up_the_payload_inside_it() {
let frame = json!({ "type": "notification", "notification": { "subscription": { "id": "s3" }, "type": "pubsub", "pubsub": prediction("ACTIVE").to_string() } });
let (sub, payload) = notification(&frame).unwrap();
assert_eq!(sub, "s3");
assert_eq!(payload["data"]["event"]["id"], "p1");
assert!(notification(&json!({ "type": "keepalive" })).is_none());
}
use crate::backend::kick::testkit;
use std::sync::Arc;
use tokio::net::TcpListener;
use tokio_tungstenite::tungstenite::Message as Ws;
/// A Hermes that, once a client has subscribed to its three topics, announces a prediction, a raid (twice) and a poll.
async fn fake_hermes(subscribed: Arc<Mutex<Vec<String>>>) -> String {
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
let url = format!("ws://{}", listener.local_addr().unwrap());
tokio::spawn(async move {
let (tcp, _) = listener.accept().await.unwrap();
let mut ws = tokio_tungstenite::accept_async(tcp).await.unwrap();
let _ = ws.send(Ws::Text(json!({ "type": "welcome", "welcome": { "keepaliveSec": 15 } }).to_string())).await;
let mut ids: Vec<(String, String)> = Vec::new();
while let Some(Ok(Ws::Text(text))) = ws.next().await {
let v: Value = serde_json::from_str(&text).unwrap();
if v["type"] != "subscribe" {
continue;
}
let (id, topic) = (v["subscribe"]["id"].as_str().unwrap().to_string(), v["subscribe"]["pubsub"]["topic"].as_str().unwrap().to_string());
subscribed.lock().unwrap().push(topic.clone());
let _ = ws.send(Ws::Text(json!({ "type": "subscribeResponse", "subscribeResponse": { "subscription": { "id": id }, "result": "ok" } }).to_string())).await;
ids.push((id, topic));
if ids.len() == 3 {
let say = |topic: &str, payload: Value| {
let id = ids.iter().find(|(_, t)| t.starts_with(topic)).unwrap().0.clone();
Ws::Text(json!({ "type": "notification", "notification": { "subscription": { "id": id }, "type": "pubsub", "pubsub": payload.to_string() } }).to_string())
};
let began = chrono::Utc::now().to_rfc3339();
let _ = ws.send(say("predictions-channel-v1", json!({ "type": "event-created", "data": { "event": { "id": "p1", "title": "Map 2", "status": "ACTIVE", "created_at": began, "prediction_window_seconds": 600,
"outcomes": [{ "id": "a", "title": "Falcons", "total_points": 0, "total_users": 0 }, { "id": "b", "title": "NIP", "total_points": 0, "total_users": 0 }] } } }))).await;
let raid = json!({ "type": "raid_update_v2", "raid": { "id": "r1", "target_login": "friend", "target_display_name": "Friend", "viewer_count": 120 } });
let _ = ws.send(say("raid", raid.clone())).await;
// Described again as its clock runs: said once.
let _ = ws.send(say("raid", raid)).await;
let _ = ws.send(say("polls", json!({ "type": "POLL_CREATE", "data": { "poll": { "poll_id": "q1", "title": "Next?", "status": "ACTIVE", "duration_seconds": 60, "choices": [{ "choice_id": "x", "title": "Chess", "votes": { "total": 0 } }] } } }))).await;
}
}
});
url
}
#[tokio::test]
async fn a_channels_prediction_poll_and_raid_are_heard_from_the_realtime_service_and_shown() {
let state = testkit::simulated_daemon("twitch-realtime");
let account = "twitch:~anonymous";
let buffer = crate::model::buffer_id(account, "xqc");
state.runtime.ensure_buffer(&state, account, "xqc", "channel");
let subscribed = Arc::new(Mutex::new(Vec::new()));
let url = fake_hermes(subscribed.clone()).await;
let rx = register("twitch:realtime-test");
// Told about the channel before the socket is up, as a join is.
watch("twitch:realtime-test", "xqc", "100");
let task = tokio::spawn(run(state.clone(), account.to_string(), rx, url, "http://127.0.0.1:9".to_string()));
for _ in 0..100 {
if state.runtime.live_card(&buffer, "prediction").is_some() && state.runtime.live_card(&buffer, "poll").is_some() && !state.store.get_backlog(&buffer, 0, 10).unwrap().is_empty() {
break;
}
tokio::time::sleep(Duration::from_millis(30)).await;
}
let mut topics = subscribed.lock().unwrap().clone();
topics.sort();
assert_eq!(topics, vec!["polls.100", "predictions-channel-v1.100", "raid.100"]);
let prediction = state.runtime.live_card(&buffer, "prediction").expect("the prediction is on screen");
assert_eq!((prediction["title"].as_str(), prediction["state"].as_str()), (Some("Map 2"), Some("open")));
assert!(prediction["remaining"].as_u64().unwrap() > 590);
assert_eq!(state.runtime.live_card(&buffer, "poll").unwrap()["options"][0]["label"], "Chess");
let log = state.store.get_backlog(&buffer, 0, 10).unwrap();
let raids: Vec<_> = log.iter().filter(|m| m.body.contains("raiding")).collect();
assert_eq!(raids.len(), 1, "a raid is said once: {log:?}");
assert_eq!(raids[0].embeds[0].kind.as_deref(), Some("raid-out"));
assert_eq!(raids[0].embeds[0].description.as_deref(), Some("friend"));
task.abort();
}
/// Against the real services: `TWITCH_LIVE_CHANNEL=<login> cargo test --release live_ -- --ignored --nocapture`.
/// Opens the channel the way a join does and prints what Twitch says about its poll and prediction.
#[tokio::test]
#[ignore]
async fn live_a_real_channels_prediction_is_read_from_the_graphql_and_the_socket() {
let _ = rustls::crypto::ring::default_provider().install_default();
let channel = std::env::var("TWITCH_LIVE_CHANNEL").expect("TWITCH_LIVE_CHANNEL=<login>");
let state = testkit::simulated_daemon("twitch-realtime-live");
let account = "twitch:~anonymous";
let buffer = crate::model::buffer_id(account, &channel);
state.runtime.ensure_buffer(&state, account, &channel, "channel");
let http = crate::backend::twitch::client().unwrap();
let id = crate::backend::twitch::stream::channels(&http, gql::ROOT, std::slice::from_ref(&channel)).await.unwrap()[0].clone().unwrap().id;
let rx = register("twitch:live");
watch("twitch:live", &channel, &id);
let task = tokio::spawn(run(state.clone(), account.to_string(), rx, HERMES_URL.to_string(), gql::ROOT.to_string()));
tokio::time::sleep(Duration::from_secs(20)).await;
println!("prediction: {:?}", state.runtime.live_card(&buffer, "prediction"));
println!("poll: {:?}", state.runtime.live_card(&buffer, "poll"));
task.abort();
}
}
src/backend/twitch/stream.rs
+258
View File
@@ -0,0 +1,258 @@
//! What is on air in a channel: whether it is live, what it is called, how many
//! are watching, and the address of its video.
//!
//! The answer comes from Twitch's GraphQL (see `gql.rs`) because it can be
//! asked anonymously and in one request for many channels at once, where the
//! official API wants a signed-in account and one call per kind of thing. It is
//! said to the window as the same `kickStream` event Kick's streams use, with
//! the same fields, so the header's live chip, the Watch button and the player
//! are the code Kick already has - the event is named for the service that had
//! it first and carries any service's stream.
use super::gql;
use super::playback::{self, Playback, PlaybackError};
use crate::state::AppState;
use anyhow::{Context, Result};
use serde_json::{json, Value};
/// A channel as Twitch describes it.
#[derive(Debug, Clone, PartialEq)]
pub struct Channel {
/// Twitch's number for it, which the official API and a shared chat are keyed by.
pub id: String,
pub login: String,
pub display_name: String,
/// The streamer's own picture.
pub avatar: Option<String>,
pub followers: Option<u64>,
pub live: Option<Live>,
}
/// A broadcast in progress.
#[derive(Debug, Clone, PartialEq)]
pub struct Live {
/// The broadcast's own id: a different one is a different stream, even in the same channel.
pub id: u64,
pub title: String,
pub category: Option<String>,
pub viewers: Option<u64>,
pub started_ts: Option<i64>,
}
const USERS: &str = "query($logins:[String!]!){ users(logins:$logins){ id login displayName profileImageURL(width:300) followers{ totalCount } broadcastSettings{ title game{ displayName } } stream{ id title viewersCount createdAt type game{ displayName } } } }";
/// How many channels one question asks about. The poll covers every channel an
/// account has open, and a list longer than this is asked in pieces.
const BATCH: usize = 30;
fn read_channel(v: &Value) -> Option<Channel> {
let login = v.get("login")?.as_str()?.to_lowercase();
let live = v.get("stream").filter(|s| !s.is_null()).map(|s| Live {
id: s.get("id").and_then(|i| i.as_str()).and_then(|i| i.parse().ok()).unwrap_or(0),
title: s.get("title").and_then(|t| t.as_str()).filter(|t| !t.is_empty()).or_else(|| v.pointer("/broadcastSettings/title").and_then(|t| t.as_str())).unwrap_or("").to_string(),
category: s.pointer("/game/displayName").and_then(|g| g.as_str()).map(str::to_string),
viewers: s.get("viewersCount").and_then(|n| n.as_u64()),
started_ts: s.get("createdAt").and_then(|d| d.as_str()).and_then(|d| chrono::DateTime::parse_from_rfc3339(d).ok()).map(|d| d.timestamp()),
});
Some(Channel {
id: v.get("id")?.as_str()?.to_string(),
display_name: v.get("displayName").and_then(|d| d.as_str()).filter(|d| !d.is_empty()).unwrap_or(&login).to_string(),
avatar: v.get("profileImageURL").and_then(|a| a.as_str()).map(str::to_string),
followers: v.pointer("/followers/totalCount").and_then(|n| n.as_u64()),
login,
live,
})
}
/// Asks about several channels at once. A login Twitch has no channel for comes
/// back as `None` in its place, so the answers line up with the question.
pub async fn channels(http: &reqwest::Client, root: &str, logins: &[String]) -> Result<Vec<Option<Channel>>> {
let mut out = Vec::with_capacity(logins.len());
for chunk in logins.chunks(BATCH) {
let answer = gql::call(http, root, &gql::query(USERS, json!({ "logins": chunk })), &gql::Who::default()).await?;
if let Some(why) = gql::error_of(&answer) {
anyhow::bail!("Twitch would not say what is on air: {why}");
}
let users = answer.pointer("/data/users").and_then(|u| u.as_array()).context("Twitch's answer about channels had no list of them")?;
for (i, login) in chunk.iter().enumerate() {
out.push(users.get(i).filter(|u| !u.is_null()).and_then(read_channel).filter(|c| c.login == login.to_lowercase()));
}
}
Ok(out)
}
/// What a stream event says, built from what is known. Public for the tests.
pub fn stream_event(buffer_id: &str, channel: &Channel, playback: Option<&Playback>, why_not: Option<&str>) -> Value {
json!({
"bufferId": buffer_id,
"service": "twitch",
"live": channel.live.is_some(),
"title": channel.live.as_ref().map(|l| l.title.clone()),
"category": channel.live.as_ref().and_then(|l| l.category.clone()),
"livestreamId": channel.live.as_ref().map(|l| l.id).filter(|id| *id != 0),
"viewers": channel.live.as_ref().and_then(|l| l.viewers),
"startedTs": channel.live.as_ref().and_then(|l| l.started_ts),
"followers": channel.followers,
"playbackUrl": playback.map(|p| p.url.clone()),
"playbackExpires": playback.map(|p| p.expires),
// Whether this account's own token says ads are not for it, which is
// the one thing a client does not choose: a subscriber's or Turbo's.
"adFree": playback.map(|p| p.hide_ads),
// Why there is no picture, in Twitch's words or ours, for a stream that
// is live: sub-only, blocked in this country, and so on.
"playbackError": why_not,
})
}
/// How long before an address stops being accepted a new one is made: the
/// token is good for twenty minutes, and the player can be several minutes
/// into a playlist that carries it.
pub const RENEW_BEFORE: i64 = 300;
/// The address for a live channel: the one already held while it is good for
/// a while yet, else a new one.
async fn playback_for(state: &AppState, account_id: &str, buffer_id: &str, http: &reqwest::Client, login: &str) -> (Option<Playback>, Option<String>) {
let now = chrono::Utc::now().timestamp();
if let Some(have) = state.runtime.kick_stream(buffer_id) {
if let (Some(url), Some(expires)) = (have["playbackUrl"].as_str(), have["playbackExpires"].as_i64()) {
if expires - now > RENEW_BEFORE {
return (Some(Playback { url: url.to_string(), expires, hide_ads: have["adFree"].as_bool().unwrap_or(false), restricted: Vec::new() }), None);
}
}
}
let web = state.accounts.get_twitch(account_id).map(|a| a.web_token).filter(|t| !t.is_empty());
match playback::playback(state, http, gql::ROOT, playback::USHER_ROOT, login, web.as_deref()).await {
Ok(p) => (Some(p), None),
Err(PlaybackError::Offline) => (None, None),
Err(e) => {
tracing::debug!("twitch: the address for {login}: {e}");
(None, Some(e.to_string()))
}
}
}
/// Reads a batch of channels' state into the stream events, saying going live
/// and going off the air in the channel when that is what changed.
///
/// Each live channel is made sure to have a good address too. That costs a
/// call only for a channel whose address is not yet held or is running out:
/// one that is good for a while yet is kept as it is.
pub async fn refresh(state: &AppState, account_id: &str, logins: &[String]) {
let Ok(http) = super::client() else { return };
let rows = match channels(&http, gql::ROOT, logins).await {
Ok(rows) => rows,
Err(e) => {
tracing::debug!("twitch[{account_id}]: asking what is on air: {e:#}");
return;
}
};
for (login, row) in logins.iter().zip(rows) {
let buffer_id = crate::model::buffer_id(account_id, login);
let Some(channel) = row else { continue };
let before = state.runtime.kick_stream(&buffer_id);
let was_live = before.as_ref().map(|s| s["live"].as_bool().unwrap_or(false));
let (playback, why_not) = if channel.live.is_some() { playback_for(state, account_id, &buffer_id, &http, login).await } else { (None, None) };
let next = stream_event(&buffer_id, &channel, playback.as_ref(), why_not.as_deref());
if before.as_ref() != Some(&next) {
state.runtime.set_kick_stream(&buffer_id, next.clone());
state.events.emit("kickStream", next);
}
if let Some(avatar) = channel.avatar.as_deref() {
state.runtime.set_buffer_avatar(state, &buffer_id, avatar);
}
// Said in the channel when it changes, never on first sight of a
// channel (which would announce every channel as it is joined).
if let (Some(was), now) = (was_live, channel.live.is_some()) {
if was != now {
say_live_change(state, account_id, &channel, now);
}
}
}
}
fn say_live_change(state: &AppState, account_id: &str, channel: &Channel, live: bool) {
let what = if live { "went live" } else { "ended the stream" };
state.runtime.record_message(
state,
account_id,
&channel.login,
"channel",
&channel.login,
what,
false,
"stream",
None,
None,
false,
None,
vec![crate::model::Embed {
kind: Some("stream".into()),
author: Some(channel.display_name.clone()),
title: Some(what.to_string()),
icon: Some(if live { "live" } else { "offline" }.to_string()),
..Default::default()
}],
Vec::new(),
None,
);
}
/// One channel, now: used when it has just been joined and when the window
/// asks on its own minute for the one on screen.
pub async fn refresh_one(state: &AppState, buffer_id: &str) {
let Some(buffer) = state.runtime.get_buffer(buffer_id) else { return };
if !buffer.account_id.starts_with("twitch:") {
return;
}
refresh(state, &buffer.account_id, std::slice::from_ref(&buffer.name)).await
}
#[cfg(test)]
mod tests {
use super::*;
fn answer() -> Value {
json!({ "data": { "users": [
{ "id": "71092938", "login": "xqc", "displayName": "xQc", "profileImageURL": "https://static-cdn.jtvnw.net/x.jpeg", "followers": { "totalCount": 12641353 },
"broadcastSettings": { "title": "old title", "game": { "displayName": "Just Chatting" } },
"stream": { "id": "321500000001", "title": "LOCK IN", "viewersCount": 25551, "createdAt": "2026-10-14T01:02:03Z", "type": "live", "game": { "displayName": "GTA V" } } },
{ "id": "43001749", "login": "kotbarsik_twt", "displayName": "kotbarsik_twt", "profileImageURL": null, "followers": { "totalCount": 0 }, "broadcastSettings": { "title": "Afk cloaking", "game": { "displayName": "EVE Online" } }, "stream": null },
null
] } })
}
#[test]
fn a_live_channel_is_read_with_its_broadcast_and_an_offline_one_without() {
let users = answer()["data"]["users"].as_array().unwrap().clone();
let live = read_channel(&users[0]).unwrap();
assert_eq!((live.login.as_str(), live.display_name.as_str(), live.followers), ("xqc", "xQc", Some(12641353)));
let on = live.live.unwrap();
assert_eq!((on.id, on.title.as_str(), on.category.as_deref(), on.viewers), (321500000001, "LOCK IN", Some("GTA V"), Some(25551)));
assert_eq!(on.started_ts, Some(1791939723));
let off = read_channel(&users[1]).unwrap();
assert!(off.live.is_none());
assert_eq!(off.avatar, None);
assert!(read_channel(&users[2]).is_none(), "a login Twitch has no channel for comes back as null");
}
#[test]
fn the_event_is_the_shape_the_window_already_draws_for_kick() {
let users = answer()["data"]["users"].as_array().unwrap().clone();
let channel = read_channel(&users[0]).unwrap();
let playback = Playback { url: "https://usher.ttvnw.net/x".into(), expires: 1800000000, hide_ads: false, restricted: vec![] };
let e = stream_event("twitch:~anonymous|xqc", &channel, Some(&playback), None);
assert_eq!(e["bufferId"], "twitch:~anonymous|xqc");
assert_eq!((e["live"].clone(), e["title"].clone(), e["category"].clone(), e["viewers"].clone()), (json!(true), json!("LOCK IN"), json!("GTA V"), json!(25551)));
assert_eq!((e["playbackUrl"].clone(), e["playbackExpires"].clone(), e["adFree"].clone()), (json!("https://usher.ttvnw.net/x"), json!(1800000000), json!(false)));
assert_eq!(e["livestreamId"], json!(321500000001u64));
// An offline channel has no broadcast and no address, and no `following` for a button to offer.
let off = stream_event("b", &read_channel(&users[1]).unwrap(), None, None);
assert_eq!((off["live"].clone(), off["playbackUrl"].clone(), off["livestreamId"].clone()), (json!(false), Value::Null, Value::Null));
assert!(e.get("following").is_none());
// Live but with nothing to play, and why.
let blocked = stream_event("b", &channel, None, Some("subscribers only"));
assert_eq!((blocked["live"].clone(), blocked["playbackError"].clone()), (json!(true), json!("subscribers only")));
}
}
src/backend/twitch/watch.rs
+160
View File
@@ -0,0 +1,160 @@
//! Telling Twitch a channel is being watched.
//!
//! Twitch gives a viewer channel points for time spent watching, and counts that time from what the website's
//! player reports: once a minute, a small "minute-watched" event naming the channel, the broadcast and the viewer,
//! sent to Twitch's telemetry address. Moho plays the video itself, so without this Twitch never hears that anyone
//! is watching and no points build up. This sends the same report, and only while the window is really playing
//! the channel's video: the window asks once a minute while the picture is playing and visible, and stops when it
//! is paused, hidden, closed or minimised.
//!
//! What it does not do: it claims nothing (the bonus chest is out of scope), it reports no faster than once a
//! minute for a channel however many windows ask, it reports nothing for a channel that is not live or an
//! account with no twitch.tv session, and it can be turned off in Settings -> Twitch.
use super::stream;
use crate::state::AppState;
use base64::Engine;
use std::collections::HashMap;
use std::sync::Mutex;
use std::time::{Duration, Instant};
/// Twitch's own page, which says where its player reports to.
pub const PAGE_ROOT: &str = "https://www.twitch.tv";
/// Just under the minute the window asks on, so two windows on one channel cost one report.
const MIN_GAP: Duration = Duration::from_secs(50);
fn last_reports() -> &'static Mutex<HashMap<String, Instant>> {
static LAST: std::sync::OnceLock<Mutex<HashMap<String, Instant>>> = std::sync::OnceLock::new();
LAST.get_or_init(Default::default)
}
fn known_address() -> &'static Mutex<Option<String>> {
static ADDRESS: std::sync::OnceLock<Mutex<Option<String>>> = std::sync::OnceLock::new();
ADDRESS.get_or_init(Default::default)
}
/// The address of the website's settings script, which a channel's page names.
pub fn settings_script(page: &str) -> Option<String> {
let found = regex::Regex::new(r#"https?://[A-Za-z0-9.:/_-]*?/config/settings\.[A-Za-z0-9]+\.js"#).ok()?.find(page)?;
Some(found.as_str().to_string())
}
/// Where the player reports to, from the settings script.
pub fn spade_address(script: &str) -> Option<String> {
let found = regex::Regex::new(r#""spade_url":"([^"]+)""#).ok()?.captures(script)?;
Some(found[1].replace("\\/", "/"))
}
/// The report, as the player writes it: a list with one event, as JSON, in base64.
pub fn minute_watched(channel_id: &str, broadcast_id: u64, user_id: &str) -> String {
let events = serde_json::json!([{
"event": "minute-watched",
"properties": { "channel_id": channel_id, "broadcast_id": broadcast_id.to_string(), "player": "site", "user_id": user_id }
}]);
base64::engine::general_purpose::STANDARD.encode(events.to_string())
}
async fn address(http: &reqwest::Client, page_root: &str, channel: &str, fresh: bool) -> Result<String, String> {
if !fresh {
if let Some(known) = known_address().lock().unwrap().clone() {
return Ok(known);
}
}
let ask = |url: String| async move { http.get(url).send().await.map_err(|e| format!("Twitch's page would not load: {e}"))?.text().await.map_err(|e| format!("Twitch's page would not load: {e}")) };
let page = ask(format!("{page_root}/{channel}")).await?;
let script_url = settings_script(&page).ok_or("Twitch's page does not say where its player reports")?;
let script = ask(script_url).await?;
let found = spade_address(&script).ok_or("Twitch's settings do not say where its player reports")?;
*known_address().lock().unwrap() = Some(found.clone());
Ok(found)
}
/// Reports that the account is watching the channel of this conversation, if it is live and the account has a
/// twitch.tv session and this channel was not reported in the last minute. True when a report was sent.
pub async fn report(state: &AppState, gql_root: &str, page_root: &str, buffer_id: &str) -> Result<bool, String> {
let (cfg, channel) = super::points::session_of(state, buffer_id)?;
if cfg.web_token.is_empty() || cfg.user_id.is_empty() {
return Ok(false);
}
{
let mut last = last_reports().lock().unwrap();
let key = format!("{}|{channel}", cfg.user_id);
if last.get(&key).is_some_and(|at| at.elapsed() < MIN_GAP) {
return Ok(false);
}
last.insert(key, Instant::now());
}
let http = super::client().map_err(|e| format!("{e:#}"))?;
let found = stream::channels(&http, gql_root, std::slice::from_ref(&channel)).await.map_err(|e| format!("{e:#}"))?;
let Some((id, live)) = found.into_iter().next().flatten().and_then(|c| c.live.map(|l| (c.id, l))) else { return Ok(false) };
let body = format!("data={}", minute_watched(&id, live.id, &cfg.user_id));
for fresh in [false, true] {
let to = address(&http, page_root, &channel, fresh).await?;
let sent = http.post(&to).header("Content-Type", "application/x-www-form-urlencoded").body(body.clone()).send().await;
match sent {
Ok(res) if res.status().is_success() => return Ok(true),
// The address moved with a new script: look again once.
Ok(res) if fresh => return Err(format!("Twitch did not take the report ({})", res.status())),
Err(e) if fresh => return Err(format!("Twitch's reports are not answering: {e}")),
_ => {}
}
}
Ok(false)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn the_report_is_the_players_own_shape() {
let encoded = minute_watched("71", 1234, "9");
let decoded: serde_json::Value = serde_json::from_slice(&base64::engine::general_purpose::STANDARD.decode(encoded).unwrap()).unwrap();
assert_eq!(decoded[0]["event"], "minute-watched");
assert_eq!(decoded[0]["properties"]["channel_id"], "71");
assert_eq!(decoded[0]["properties"]["broadcast_id"], "1234");
assert_eq!(decoded[0]["properties"]["user_id"], "9");
assert_eq!(decoded[0]["properties"]["player"], "site");
}
#[tokio::test]
async fn a_live_channel_is_reported_once_a_minute() {
use super::super::points::tests::{channel, fake_gql};
*known_address().lock().unwrap() = None;
last_reports().lock().unwrap().clear();
let (state, buffer) = channel("twitch-watch", "websession");
let (root, seen) = fake_gql(|req| {
let line = req.lines().next().unwrap_or("").to_string();
if line.starts_with("POST /gql") || req.contains("users(logins") {
r#"{"data":{"users":[{"id":"71","login":"xqc","displayName":"xQc","stream":{"id":"555","title":"t","viewersCount":5,"createdAt":"2026-10-14T10:00:00Z","type":"live"}}]}}"#.to_string()
} else if line.starts_with("GET /xqc ") {
// The page names the script on the same fake server.
let host = req.lines().find_map(|l| l.strip_prefix("host: ").or_else(|| l.strip_prefix("Host: "))).unwrap_or("").trim().to_string();
format!(r#"<script src="http://{host}/config/settings.abc.js"></script>"#)
} else if line.starts_with("GET /config/") {
let host = req.lines().find_map(|l| l.strip_prefix("host: ").or_else(|| l.strip_prefix("Host: "))).unwrap_or("").trim().to_string();
format!(r#"x={{"spade_url":"http://{host}/track"}}"#)
} else {
String::new()
}
})
.await;
assert!(report(&state, &format!("{root}/gql"), &root, &buffer).await.unwrap());
let requests = seen.lock().unwrap().clone();
let posted = requests.iter().find(|r| r.starts_with("POST /track")).expect("a report was sent");
assert!(posted.contains("data="), "{posted}");
// Asked again at once: nothing more goes out.
assert!(!report(&state, &format!("{root}/gql"), &root, &buffer).await.unwrap());
assert_eq!(seen.lock().unwrap().iter().filter(|r| r.starts_with("POST /track")).count(), 1);
}
#[test]
fn where_to_report_is_read_from_the_page_and_its_script() {
let page = r#"<script src="https://assets.twitch.tv/config/settings.abc123.js"></script>"#;
assert_eq!(settings_script(page).as_deref(), Some("https://assets.twitch.tv/config/settings.abc123.js"));
assert_eq!(spade_address(r#"x={"spade_url":"https:\/\/spade.example\/track","a":1}"#).as_deref(), Some("https://spade.example/track"));
assert_eq!(settings_script("<html></html>"), None);
assert_eq!(spade_address("nothing"), None);
}
}
src/commands.rs
+5 -4
View File
@@ -25,11 +25,12 @@ pub struct Builtin {
}
const IRC: &[&str] = &["irc"];
const IRC_AND_MATRIX: &[&str] = &["irc", "matrix"];
/// `/me` is the one command Twitch's chat itself understands, so it is sent as typed.
const ME: &[&str] = &["irc", "matrix", "twitch"];
const MATRIX: &[&str] = &["matrix"];
/// Everywhere, including the services with no list of their own - moho keeps
/// one for those.
const EVERY: &[&str] = &["irc", "matrix", "discord", "kick", "sneedchat"];
const EVERY: &[&str] = &["irc", "matrix", "discord", "kick", "sneedchat", "twitch"];
/// The ones that are only text, and so work wherever this daemon does the
/// sending itself.
///
@@ -37,7 +38,7 @@ const EVERY: &[&str] = &["irc", "matrix", "discord", "kick", "sneedchat"];
/// anybody else sees it, and inventing four more that only this client knows
/// about would mean the same typing doing different things depending on which
/// client you were at.
const TEXT_ONLY: &[&str] = &["irc", "matrix", "discord", "kick"];
const TEXT_ONLY: &[&str] = &["irc", "matrix", "discord", "kick", "twitch"];
/// Spoilers exist on the two services that have a syntax for them.
const SPOILERS: &[&str] = &["discord", "matrix"];
const SNEEDCHAT: &[&str] = &["sneedchat"];
@@ -45,7 +46,7 @@ const SNEEDCHAT: &[&str] = &["sneedchat"];
/// Ordered roughly by how often each is wanted, since a fuzzy match that ties
/// keeps this order.
pub const BUILTINS: &[Builtin] = &[
Builtin { name: "me", usage: "<action>", description: "Say something as an action", services: IRC_AND_MATRIX },
Builtin { name: "me", usage: "<action>", description: "Say something as an action", services: ME },
Builtin { name: "join", usage: "<#channel>", description: "Join a channel", services: IRC },
Builtin { name: "part", usage: "", description: "Leave this channel", services: IRC },
Builtin { name: "msg", usage: "<nick> <message>", description: "Send somebody a private message", services: IRC },
src/main.rs
+6
View File
@@ -340,6 +340,12 @@ async fn run() -> Result<()> {
for cfg in state.accounts.all_steam() {
backend::steam::spawn(state.clone(), cfg);
}
for cfg in state.accounts.all_twitch() {
backend::twitch::spawn(state.clone(), cfg);
}
for cfg in state.accounts.all_telegram() {
backend::telegram::connect::spawn(state.clone(), cfg);
}
// What was going on last time, before anything is served: a window that
// connects immediately should see the same list it was looking at.
src/media_cache.rs
+9
View File
@@ -38,6 +38,8 @@ pub const CACHES: &[Cache] = &[
Cache { dir: "matrix-media", group: "media", max_bytes: 250 * MB },
Cache { dir: "sneedchat-attachments", group: "media", max_bytes: 250 * MB },
Cache { dir: "discord-thumbnails", group: "media", max_bytes: 100 * MB },
Cache { dir: "telegram-media", group: "media", max_bytes: 500 * MB },
Cache { dir: "telegram-avatars", group: "avatars", max_bytes: 64 * MB },
Cache { dir: "sneedchat-avatars", group: "avatars", max_bytes: 100 * MB },
Cache { dir: "discord-icons", group: "avatars", max_bytes: 16 * MB },
Cache { dir: "kick-emotes", group: "emotes", max_bytes: 64 * MB },
@@ -209,6 +211,13 @@ pub async fn restore(state: &AppState, path: &str, buffer_id: Option<&str>, mess
}
"discord-icons" => crate::backend::discord::restore_guild_icon(&path).await?,
"kick-emotes" => crate::backend::kick::emotecache::restore_cached(&path).await?,
"telegram-media" | "telegram-avatars" => match crate::backend::telegram::media::restore_cached(state, &path).await {
Ok(path) => path,
Err(e) => {
tracing::warn!("telegram: fetching {} again: {e:#}", path.display());
return Err(e);
}
},
_ => bail!("{} is fetched again by what uses it, not by name", cache.dir),
};
Ok(format!("file://{}", restored.display()))
src/model.rs
+51
View File
@@ -188,6 +188,15 @@ pub struct Account {
/// Matrix only: whether this account keeps a dehydrated device.
#[serde(rename = "dehydration", default)]
pub dehydration: bool,
/// Twitch only: whether this account holds a twitch.tv website session for
/// playing video as the person (their Turbo, their subscriptions). Said as
/// a yes or no and never the session itself.
#[serde(rename = "webSession", default, skip_serializing_if = "std::ops::Not::not")]
pub web_session: bool,
/// Twitch only: whether this account was signed in with the rights to
/// moderate, whisper and change its colour.
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
pub moderation: bool,
}
/// One room in a Sneedchat account's currently-enabled list, as exposed on
@@ -370,6 +379,10 @@ pub struct Buffer {
/// thing - so it is named for what it is rather than after the tag.
#[serde(rename = "serviceRoom", default, skip_serializing_if = "std::ops::Not::not")]
pub service_room: bool,
/// A small glyph drawn in front of the name in the list, for what kind of conversation it is where the service
/// has kinds a person looks for: Telegram's broadcast channels ("campaign", a horn) and bots ("smart_toy").
#[serde(rename = "nameIcon", default, skip_serializing_if = "Option::is_none")]
pub name_icon: Option<String>,
/// What the conversation says it is for: an IRC channel's topic, a Matrix
/// room's, a Discord channel's. Absent when there is none.
#[serde(default, skip_serializing_if = "Option::is_none")]
@@ -463,6 +476,38 @@ pub struct ReplyPreview {
/// something else entirely.
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
pub forwarded: bool,
/// The message a whole chain of replies grew from, where the service
/// groups them into one conversation.
///
/// Kick does: every reply carries the id of the message at the top of its
/// chain, however deep it is, while `id` stays the message it answered. A
/// reply therefore points two ways - back at what it quoted, and up at the
/// thread it belongs to - and a client wants both: the first for the quote
/// above it, the second for a thread to open.
#[serde(rename = "threadRoot", default, skip_serializing_if = "Option::is_none")]
pub thread_root: Option<String>,
/// The conversation the message answered lives in, when it is not this one.
///
/// Telegram can answer a post in another channel; the quote then names where it came from, and a client can
/// take the reader there. A buffer id, present only when that conversation is one the account has open.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub chat: Option<String>,
}
/// Why a message the service took down is gone, kept with the message.
///
/// Kick tells everyone in a channel when a message is removed, and says by
/// whom only as far as "a machine did it, and for breaking these rules". The
/// text stays here rather than leaving the log, so that what it said is still
/// readable - how it is drawn is up to the client.
#[derive(Serialize, serde::Deserialize, Clone, Debug, Default, PartialEq)]
pub struct Moderated {
/// Whether Kick's automatic moderation did it, rather than a person.
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
pub ai: bool,
/// The rules it was said to break: `hate`, `harassment`, and so on.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub rules: Vec<String>,
}
/// How a service says a sender should look.
@@ -791,6 +836,10 @@ pub struct Message {
/// Buttons and menus on the message, where the service has them.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub components: Vec<Component>,
/// Set once the service has taken this message down: it is still here, and
/// this says it should be drawn as removed.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub moderated: Option<Moderated>,
}
pub fn buffer_id(account_id: &str, name: &str) -> String {
@@ -870,6 +919,7 @@ pub fn service_of(account_id: &str) -> &'static str {
Some("kick") => "kick",
Some("sneedchat") => "sneedchat",
Some("steam") => "steam",
Some("twitch") => "twitch",
Some("jabber") => "jabber",
Some("slack") => "slack",
_ => "irc",
@@ -992,6 +1042,7 @@ mod tests {
favourite: false,
low_priority: false,
service_room: false,
name_icon: None,
topic: None,
read_only: None,
forum: false,
src/rpc/methods.rs
+614 -4
View File
@@ -82,6 +82,10 @@ fn respawn_account(state: &AppState, id: &str) {
if let Some(cfg) = state.accounts.get_steam(id) {
backend::steam::spawn(state.clone(), cfg);
}
} else if id.starts_with("twitch:") {
if let Some(cfg) = state.accounts.get_twitch(id) {
backend::twitch::spawn(state.clone(), cfg);
}
} else if let Some(cfg) = state.accounts.get_irc(id) {
backend::irc::spawn(state.clone(), cfg);
}
@@ -107,6 +111,8 @@ fn adding_service(method: &str) -> Option<&'static str> {
| "matrixRegistrationFlows" | "matrixLoginFlows" | "addMatrixAccountSso" | "addMatrixAccountQr" => Some("matrix"),
"addKickAccount" => Some("kick"),
"addSteamAccount" | "addSteamAccountQr" => Some("steam"),
"addTwitchAccount" | "startTwitchLogin" => Some("twitch"),
"addTelegramAccount" => Some("telegram"),
_ => None,
}
}
@@ -178,6 +184,15 @@ pub async fn dispatch(
if account_id.starts_with("steam:") {
backend::steam::emoticons::refresh(state, &account_id).await;
}
// Twitch's: what the account can use in a channel changes under it (a reward unlocks one, a 24-hour
// unlock lapses), so the picker reads it again when what it holds is a couple of minutes old.
if account_id.starts_with("twitch:") {
if let Some(buffer) = state.runtime.get_buffer(buffer_id) {
if let (Some(room), true) = backend::twitch::emotes::user_emotes_state(&buffer.account_id, &buffer.name, std::time::Duration::from_secs(120)) {
backend::twitch::connect::refresh_user_emotes(state, &buffer.account_id, &buffer.name, &room).await;
}
}
}
let mut sources = state.runtime.emoji_catalogue(&account_id);
// Steam's stickers are sent, not typed: they have their own tab, asked for
// by `listSteamStickers`, and are not a section of emoji.
@@ -209,9 +224,19 @@ pub async fn dispatch(
// Whether each source reaches this conversation. Worked out here
// rather than stored, because it is a fact about the pair.
// A Twitch channel's section is marked by the channel's own picture, which is known to the
// conversation and not to the source.
let channel_picture = state.runtime.get_buffer(buffer_id).filter(|_| account_id.starts_with("twitch:")).and_then(|b| b.avatar_url.map(|a| (format!("twitch:{}", b.name), a)));
let answer: Vec<serde_json::Value> = sources
.into_iter()
.map(|s| {
.map(|mut s| {
if s.icon_url.is_none() {
if let Some((id, picture)) = &channel_picture {
if s.id == *id {
s.icon_url = Some(picture.clone());
}
}
}
let here = s.buffers.is_empty() || s.buffers.iter().any(|b| b == buffer_id);
let usable = here || s.sendable_anywhere;
let mut v = serde_json::to_value(&s).unwrap_or_default();
@@ -265,6 +290,25 @@ pub async fn dispatch(
None,
),
},
// Twitch's: what the account can use there, and the other services'.
Some(buffer_id) if buffer_id.starts_with("twitch:") => {
let Some(buffer) = state.runtime.get_buffer(buffer_id) else { return (Some(serde_json::json!([])), None) };
// What this account can use there changes under it (a 24-hour unlock lapses, a reward unlocks one):
// read again when the picker asks and what it was told is a couple of minutes old.
if let (Some(room), true) = backend::twitch::emotes::user_emotes_state(&buffer.account_id, &buffer.name, std::time::Duration::from_secs(120)) {
backend::twitch::connect::refresh_user_emotes(state, &buffer.account_id, &buffer.name, &room).await;
}
let mut sources = state.runtime.emoji_catalogue(&buffer.account_id);
sources.retain(|s| s.buffers.iter().any(|b| b == buffer_id));
let mut seen = std::collections::HashSet::new();
let emoji: Vec<serde_json::Value> = sources
.into_iter()
.flat_map(|s| s.emoji)
.filter(|e| !e.locked && seen.insert(e.name.clone()))
.map(|e| serde_json::json!({ "id": e.id, "name": e.name, "animated": e.animated, "url": e.url, "locked": false }))
.collect();
(Some(serde_json::json!(emoji)), None)
}
// Steam's emoticons are the account's, the same in every conversation.
Some(buffer_id) if buffer_id.starts_with("steam:") => {
let emoji: Vec<serde_json::Value> = state
@@ -382,6 +426,8 @@ pub async fn dispatch(
{ "id": "sneedchat", "name": "Sneedchat", "available": true },
{ "id": "kick", "name": "Kick", "available": true },
{ "id": "steam", "name": "Steam", "available": true },
{ "id": "twitch", "name": "Twitch", "available": true },
{ "id": "telegram", "name": "Telegram", "available": true },
])),
None,
),
@@ -446,6 +492,14 @@ pub async fn dispatch(
tracing::debug!("steam history: {e}");
}
}
// Telegram keeps every message too: opening a conversation asks for its recent end, scrolling to
// the top for the part before. Best-effort, like Steam's.
if buffer.account_id.starts_with("telegram:") {
let fetched = if before > 0 { backend::telegram::history::older(state, &buffer, before).await } else { backend::telegram::history::open(state, &buffer).await };
if let Err(e) = fetched {
tracing::debug!("telegram history: {e:#}");
}
}
// Kick pages back through its own history endpoint,
// continuing from where the last page stopped rather than
// by timestamp - the endpoint is cursor-based, and asking
@@ -539,7 +593,11 @@ pub async fn dispatch(
// until the next incremental change (see Runtime::
// set_presence's doc comment).
if let Some(members) = state.runtime.get_presence(buffer_id) {
state.events.emit("presenceChange", serde_json::json!({ "bufferId": buffer_id, "members": members }));
let mut change = serde_json::json!({ "bufferId": buffer_id, "members": members });
if let Some(total) = state.runtime.get_presence_total(buffer_id) {
change["total"] = serde_json::json!(total);
}
state.events.emit("presenceChange", change);
}
// Read markers are replayed for the same reason: they arrive
// on sync and are not repeated once they stop moving, so a
@@ -569,6 +627,11 @@ pub async fn dispatch(
}
}
// Whether this account may moderate a Twitch channel, which Twitch
// says once on joining and a window opened later missed.
if let Some(moderator) = state.runtime.twitch_rights(buffer_id) {
state.events.emit("twitchRights", serde_json::json!({ "bufferId": buffer_id, "moderator": moderator }));
}
// And what a Kick channel is broadcasting, which changes while
// nobody is looking: a viewer count from an hour ago is worse
// than none, so opening the channel asks again.
@@ -723,8 +786,29 @@ pub async fn dispatch(
if id.starts_with("discord:") {
backend::discord::golive::forget(id);
}
// A Twitch sign-in is ended at Twitch as well, so removing the
// account here does not leave a token good for four more
// hours and a refresh token good for thirty days. Best effort:
// the account is removed either way.
let twitch_tokens = state.accounts.get_twitch(id).filter(|a| a.is_signed_in()).map(|a| (a.access_token, a.refresh_token));
// A Telegram session is ended at Telegram too, and its file taken away.
let telegram_account = state.accounts.get_telegram(id);
match state.accounts.remove(id) {
Ok(true) => (Some(ok_node()), None),
Ok(true) => {
if let Some(config) = telegram_account {
let state = state.clone();
tokio::spawn(async move { backend::telegram::connect::sign_out(&state, &config).await });
}
if let Some((access, refresh)) = twitch_tokens {
tokio::spawn(async move {
if let Ok(http) = backend::twitch::client() {
let _ = backend::twitch::auth::revoke(&http, backend::twitch::auth::ID_ROOT, &access).await;
let _ = backend::twitch::auth::revoke(&http, backend::twitch::auth::ID_ROOT, &refresh).await;
}
});
}
(Some(ok_node()), None)
}
Ok(false) => (None, Some("no such account".to_string())),
Err(e) => (None, Some(format!("{e:#}"))),
}
@@ -776,6 +860,14 @@ pub async fn dispatch(
}
None => (None, Some("no such account".to_string())),
}
} else if id.starts_with("twitch:") {
match state.accounts.get_twitch(id) {
Some(cfg) => {
backend::twitch::spawn(state.clone(), cfg);
(Some(ok_node()), None)
}
None => (None, Some("no such account".to_string())),
}
} else {
match state.accounts.get_irc(id) {
Some(cfg) => {
@@ -933,6 +1025,33 @@ pub async fn dispatch(
Err(_) => (None, Some("Kick took too long to answer for that channel".to_string())),
};
}
// Twitch is the same shape: a channel is a streamer, naming one is
// the whole of joining it, and the connection remembers it once
// Twitch has said it is there.
if id.starts_with("telegram:") {
return match backend::telegram::membership::join(state, id, name).await {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(e)),
};
}
if id.starts_with("twitch:") {
let Some(channel) = backend::twitch::connect::normalise_login(name) else {
return (None, Some(format!("\"{name}\" is not a Twitch channel")));
};
let Some(sender) = state.runtime.twitch_sender(id) else {
return (None, Some("that Twitch account is not connected".to_string()));
};
let (reply, answer) = tokio::sync::oneshot::channel();
if sender.send(backend::twitch::Command::Join(channel, Some(reply))).is_err() {
return (None, Some("that Twitch account is not connected".to_string()));
}
return match tokio::time::timeout(std::time::Duration::from_secs(60), answer).await {
Ok(Ok(Ok(()))) => (Some(ok_node()), None),
Ok(Ok(Err(e))) => (None, Some(e)),
Ok(Err(_)) => (None, Some("the Twitch connection dropped before the channel opened".to_string())),
Err(_) => (None, Some("Twitch took too long to answer for that channel".to_string())),
};
}
match state.runtime.irc_sender(id) {
None => (None, Some("join failed (account not connected?)".to_string())),
Some(sender) => match sender.send_join(name) {
@@ -983,6 +1102,15 @@ pub async fn dispatch(
}
(Some(ok_node()), None)
}
Some(buffer) if buffer.account_id.starts_with("telegram:") => {
// It lapses by itself, so only "typing" is said.
if typing {
if let Err(e) = backend::telegram::send::typing(state, &buffer).await {
tracing::debug!("sendTyping: {e}");
}
}
(Some(ok_node()), None)
}
// IRC says it with a client-only tag on an empty TAGMSG,
// and says nothing at all where the server has no
// message-tags to carry one - see send_typing.
@@ -1047,6 +1175,27 @@ pub async fn dispatch(
// Muting a room for the account rather than for this window. Matrix
// keeps it server-side, so a room quietened here is quiet on a phone
// too - which is what somebody means by muting a room.
// The article a Telegram link preview carries, as a document for a window of its own.
"telegramInstantView" => {
let Some(buffer) = p_str_opt(params, "bufferId").and_then(|b| state.runtime.get_buffer(b)).filter(|b| b.account_id.starts_with("telegram:")) else {
return (None, Some("telegramInstantView requires a Telegram \"bufferId\"".to_string()));
};
match backend::telegram::instant::article(state, &buffer, p_str(params, "messageId", "")).await {
Ok(article) => (Some(serde_json::json!({ "title": article.title, "url": article.url, "html": article.html })), None),
Err(e) => (None, Some(e)),
}
}
"setTelegramChatMuted" => {
let Some(buffer) = p_str_opt(params, "bufferId").and_then(|b| state.runtime.get_buffer(b)).filter(|b| b.account_id.starts_with("telegram:")) else {
return (None, Some("setTelegramChatMuted requires a Telegram \"bufferId\"".to_string()));
};
match backend::telegram::notify::set_muted(state, &buffer, p_bool(params, "muted", true)).await {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(e)),
}
}
"setMatrixRoomMuted" => {
let Some(buffer_id) = p_str_opt(params, "bufferId") else {
return (None, Some("setMatrixRoomMuted requires \"bufferId\"".to_string()));
@@ -1135,6 +1284,7 @@ pub async fn dispatch(
"kick" => backend::kick::profile(&state, &account_id, &buffer_id, &who).await,
"sneedchat" => backend::sneedchat::profile(&state, &account_id, &buffer_id, &who),
"steam" => backend::steam::profile::build(&state, &account_id, &buffer_id, &who, &user_id).await,
"telegram" => backend::telegram::profile::build(&state, &account_id, &buffer_id, &who, &user_id).await,
// A service with nothing to add still answers, so the
// card stops waiting and shows the name.
other => {
@@ -1160,6 +1310,12 @@ pub async fn dispatch(
Ok(messages) => messages,
Err(e) => return (None, Some(format!("{e:#}"))),
}
} else if account_id.starts_with("telegram:") {
let Some(buffer) = state.runtime.get_buffer(buffer_id) else { return (None, Some("no such conversation".to_string())) };
return match backend::telegram::threads::fetch(state, &buffer, root_id).await {
Ok(messages) => (Some(serde_json::json!({ "messages": messages })), None),
Err(e) => (None, Some(e)),
};
} else {
state.store.thread_messages(buffer_id, root_id).unwrap_or_default()
};
@@ -1210,6 +1366,13 @@ pub async fn dispatch(
}
(Some(ok_node()), None)
}
// Telegram keeps a read position for each conversation too, and tells the account's other apps.
Some(buffer) if buffer.account_id.starts_with("telegram:") => {
if let Err(e) = backend::telegram::send::mark_read(state, &buffer).await {
tracing::debug!("telegram read position: {e}");
}
(Some(ok_node()), None)
}
// Steam keeps a read position for each conversation, and tells
// every other client of the account when it moves.
Some(buffer) if buffer.account_id.starts_with("steam:") => {
@@ -1259,7 +1422,8 @@ pub async fn dispatch(
Err(e) => (None, Some(format!("{e:#}"))),
}
} else {
match backend::discord::accept_member_verification(state, account_id, guild_id).await {
let answers: Vec<Value> = params.get("responses").and_then(|v| v.as_array()).cloned().unwrap_or_default();
match backend::discord::accept_member_verification(state, account_id, guild_id, &answers).await {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(format!("{e:#}"))),
}
@@ -1364,6 +1528,8 @@ pub async fn dispatch(
&finished.waveform,
)
.await
} else if buffer.account_id.starts_with("telegram:") {
backend::telegram::send::send_voice(state, &buffer, &finished.path, finished.duration_secs).await.map_err(|e| anyhow::anyhow!(e))
} else if let Some(cfg) = state.accounts.get_matrix(&buffer.account_id) {
backend::matrix::send_voice_message(
state,
@@ -1723,6 +1889,15 @@ pub async fn dispatch(
false
}
}
Some(buffer) if buffer.account_id.starts_with("telegram:") => {
let (state, buffer) = (state.clone(), buffer);
tokio::spawn(async move {
if let Err(e) = backend::telegram::members::refresh(&state, &buffer).await {
tracing::debug!("telegram members: {e}");
}
});
true
}
_ => backend::discord::request_member_list(state, buffer_id),
};
(Some(serde_json::json!({ "requested": asked })), None)
@@ -2203,6 +2378,8 @@ pub async fn dispatch(
backend::discord::load_context(state, &buffer.account_id, buffer_id, message_id).await
} else if buffer.account_id.starts_with("matrix:") {
backend::matrix::load_context(state, &buffer.account_id, buffer_id, message_id).await
} else if buffer.account_id.starts_with("telegram:") {
backend::telegram::history::context(state, &buffer, message_id).await
} else {
Err(anyhow::anyhow!("this service cannot be asked for one message"))
};
@@ -2243,6 +2420,8 @@ pub async fn dispatch(
backend::discord::load_newer(state, &buffer.account_id, buffer_id, message_id).await
} else if buffer.account_id.starts_with("matrix:") {
backend::matrix::load_newer(state, &buffer.account_id, buffer_id, message_id).await
} else if buffer.account_id.starts_with("telegram:") {
backend::telegram::history::newer(state, &buffer, message_id).await
} else {
Err(anyhow::anyhow!("this service cannot be read forward"))
};
@@ -2268,6 +2447,8 @@ pub async fn dispatch(
backend::discord::list_pinned(state, &buffer.account_id, buffer_id).await
} else if buffer.account_id.starts_with("matrix:") {
backend::matrix::list_pinned(state, &buffer.account_id, buffer_id).await
} else if buffer.account_id.starts_with("telegram:") {
backend::telegram::pins::list(state, &buffer).await.map_err(|e| anyhow::anyhow!(e))
} else {
Err(anyhow::anyhow!("this service has no pinned messages"))
};
@@ -2317,6 +2498,8 @@ pub async fn dispatch(
backend::discord::set_pinned(state, &buffer.account_id, buffer_id, message_id, pinned).await
} else if buffer.account_id.starts_with("matrix:") {
backend::matrix::set_pinned(state, &buffer.account_id, buffer_id, message_id, pinned).await
} else if buffer.account_id.starts_with("telegram:") {
backend::telegram::pins::set(state, &buffer, message_id, pinned).await.map_err(|e| anyhow::anyhow!(e))
} else {
Err(anyhow::anyhow!("this service has no pinned messages"))
};
@@ -2326,6 +2509,25 @@ pub async fn dispatch(
}
}
// The same question asked of Telegram's own index, for what this client never read.
"searchTelegramMessages" => {
let (buffer_id, query) = match (p_str_opt(params, "bufferId"), p_str_opt(params, "query")) {
(Some(b), Some(q)) => (b, q),
_ => return (None, Some("searchTelegramMessages requires \"bufferId\" and \"query\"".to_string())),
};
if query.trim().is_empty() {
return (Some(serde_json::json!({ "results": [], "roomNames": {}, "count": 0 })), None);
}
let Some(buffer) = state.runtime.get_buffer(buffer_id).filter(|b| b.account_id.starts_with("telegram:")) else {
return (None, Some("that conversation is not on Telegram".to_string()));
};
let account_wide = p_str_opt(params, "scope") == Some("account");
match backend::telegram::search::run(state, &buffer, query, account_wide, p_i64(params, "limit", 25) as usize).await {
Ok(answer) => (Some(answer), None),
Err(e) => (None, Some(e)),
}
}
// The same question asked of the homeserver rather than of this
// window's own copy.
//
@@ -3118,6 +3320,17 @@ pub async fn dispatch(
let left: Vec<String> = cfg.channels.into_iter().filter(|c| *c != buffer.name).collect();
let _ = state.accounts.set_kick_channels(&buffer.account_id, left);
}
} else if buffer.account_id.starts_with("twitch:") {
// As Kick's: nothing to leave, so it parts the
// connection's channel and forgets the handle.
if let Some(sender) = state.runtime.twitch_sender(&buffer.account_id) {
let _ = sender.send(backend::twitch::Command::Part(buffer.name.clone()));
}
let _ = state.accounts.mutate_twitch(&buffer.account_id, |a| a.channels.retain(|c| *c != buffer.name));
} else if buffer.account_id.starts_with("telegram:") {
if let Err(e) = backend::telegram::membership::leave(state, &buffer).await {
return (None, Some(e));
}
} else if buffer.account_id.starts_with("matrix:") {
if let Some(room_id) = state.runtime.get_matrix_room(buffer_id) {
if let Err(e) = backend::matrix::leave_room(state, &buffer.account_id, &room_id).await {
@@ -3310,6 +3523,30 @@ pub async fn dispatch(
}
}
},
Some(buffer) if buffer.account_id.starts_with("telegram:") && params.get("thread").and_then(|v| v.as_bool()).unwrap_or(false) && attachment_path.is_none() && reply_to_id.is_some() => {
match backend::telegram::threads::reply(state, &buffer, reply_to_id.unwrap_or_default(), body).await {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(e)),
}
}
Some(buffer) if buffer.account_id.starts_with("telegram:") => {
let paths: Vec<String> = match params.get("attachmentPaths").and_then(|v| v.as_array()) {
Some(list) => list.iter().filter_map(|v| v.as_str().map(str::to_string)).collect(),
None => attachment_path.map(|p| vec![p.to_string()]).unwrap_or_default(),
};
let spoilers: Vec<&str> = params.get("spoilerPaths").and_then(|v| v.as_array()).map(|list| list.iter().filter_map(|v| v.as_str()).collect()).unwrap_or_default();
let files: Vec<(String, bool)> = paths.iter().map(|p| (p.clone(), spoilers.contains(&p.as_str()))).collect();
let sent = backend::telegram::send::send_files(state, &buffer, body, &files, reply_to_id, progress).await;
if let Some(p) = progress {
if !files.is_empty() {
p.done(sent.as_ref().err().map(String::as_str));
}
}
match sent {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(e)),
}
}
Some(buffer) if buffer.account_id.starts_with("steam:") => {
// Steam has no replies or edits for a direct message in
// this client yet: what arrives that it cannot use is
@@ -3380,6 +3617,50 @@ pub async fn dispatch(
Err(e) => (None, Some(format!("{e:#}"))),
}
}
Some(buffer) if buffer.account_id.starts_with("twitch:") => {
// Twitch's chat is one connection with the account's
// channels on it: the message goes down it, and the
// answer says whether Twitch can have it. No attachments:
// Twitch chat has none, so one is ignored as Kick's is.
let Some(sender) = state.runtime.twitch_sender(&buffer.account_id) else {
return (None, Some("that Twitch account is not connected".to_string()));
};
// A whisper is a conversation of its own and is sent by the official API.
if buffer.kind == "dm" {
let (respond, answer) = tokio::sync::oneshot::channel();
let command = backend::twitch::Command::Act {
channel: String::new(),
action: backend::twitch::moderation::Action::Whisper { to: buffer.name.trim_start_matches('@').to_string(), text: body.to_string() },
target_id: None,
respond,
};
if sender.send(command).is_err() {
return (None, Some("that Twitch account is not connected".to_string()));
}
return match tokio::time::timeout(std::time::Duration::from_secs(20), answer).await {
Ok(Ok(Ok(_))) => (Some(ok_node()), None),
Ok(Ok(Err(e))) => (None, Some(e)),
Ok(Err(_)) => (None, Some("the Twitch connection dropped before the whisper was sent".to_string())),
Err(_) => (None, Some("Twitch took too long to take that whisper".to_string())),
};
}
let (respond, answer) = tokio::sync::oneshot::channel();
let command = backend::twitch::Command::Say {
channel: buffer.name.clone(),
text: body.to_string(),
reply_to: reply_to_id.map(str::to_string),
respond,
};
if sender.send(command).is_err() {
return (None, Some("that Twitch account is not connected".to_string()));
}
match tokio::time::timeout(std::time::Duration::from_secs(20), answer).await {
Ok(Ok(Ok(()))) => (Some(ok_node()), None),
Ok(Ok(Err(e))) => (None, Some(e)),
Ok(Err(_)) => (None, Some("the Twitch connection dropped before the message was sent".to_string())),
Err(_) => (None, Some("Twitch took too long to take that message".to_string())),
}
}
Some(buffer) if buffer.account_id.starts_with("sneedchat:") => {
// Sneedchat answers somebody by name rather than by
// message id, so a reply needs whoever wrote the message
@@ -3462,6 +3743,10 @@ pub async fn dispatch(
Err(e) => (None, Some(format!("{e:#}"))),
},
},
Some(buffer) if buffer.account_id.starts_with("telegram:") => match backend::telegram::send::edit(state, &buffer, msg_id, body).await {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(e)),
},
Some(buffer) if buffer.account_id.starts_with("sneedchat:") => match backend::sneedchat::edit_message(state, &buffer.account_id, &buffer.name, msg_id, body) {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(format!("{e:#}"))),
@@ -3502,6 +3787,10 @@ pub async fn dispatch(
Err(e) => (None, Some(format!("{e:#}"))),
},
},
Some(buffer) if buffer.account_id.starts_with("telegram:") => match backend::telegram::reactions::toggle(state, &buffer, msg_id, emoji, add).await {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(e)),
},
Some(buffer) if buffer.account_id.starts_with("steam:") && backend::steam::chatgroups::is_room(&buffer) => match backend::steam::chatgroups::react(state, &buffer, msg_id, emoji, add).await {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(e)),
@@ -3528,6 +3817,14 @@ pub async fn dispatch(
Err(e) => (None, Some(format!("{e:#}"))),
},
},
Some(buffer) if buffer.account_id.starts_with("telegram:") => {
// For everyone unless the window says it is only to be removed from this account's view.
let everyone = params.get("forEveryone").and_then(|v| v.as_bool()).unwrap_or(true);
match backend::telegram::send::delete(state, &buffer, msg_id, everyone).await {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(e)),
}
}
Some(buffer) if buffer.account_id.starts_with("kick:") => {
match (kick_credential(state, &buffer.account_id), state.runtime.kick_channel(buffer_id)) {
(Err(e), _) => (None, Some(e)),
@@ -4095,6 +4392,171 @@ pub async fn dispatch(
(Some(serde_json::json!({ "loginId": login_id })), None)
}
// Adding a Twitch account that only reads: no sign-in, nothing to
// wait for. Twitch's chat is public, so an anonymous reader sees every
// channel it is pointed at, and one that signs in later gets to send.
"addTwitchAccount" => {
let config = crate::accounts::TwitchAccountConfig {
anonymous: true,
use_tor: crate::net::route::router().wanted(&crate::net::route::pending_key("twitch")),
..Default::default()
};
match state.accounts.add_twitch(config) {
Ok(saved) => {
backend::twitch::spawn(state.clone(), saved.clone());
(Some(serde_json::json!({ "accountId": saved.account_id() })), None)
}
Err(e) => (None, Some(format!("{e:#}"))),
}
}
// Signing in to Twitch takes place over time - the person approves a
// code on twitch.tv - so this answers with an id and the rest is
// events: see backend/twitch/login.rs.
"startTwitchLogin" => {
let login_id = format!("twitch-{}", crate::model::next_message_id());
backend::twitch::login::start(state.clone(), login_id.clone());
(Some(serde_json::json!({ "loginId": login_id })), None)
}
// The form was closed with a sign-in still waiting.
"cancelTwitchLogin" => {
if let Some(login_id) = p_str_opt(params, "loginId") {
backend::twitch::login::cancel(login_id);
}
(Some(ok_node()), None)
}
// The window's answer to a request for an integrity token, which
// Twitch sometimes demands before it will give a stream's address and
// which only a real page on twitch.tv can make. See playback.rs.
"provideTwitchIntegrity" => {
let Some(request_id) = p_str_opt(params, "requestId") else {
return (None, Some("provideTwitchIntegrity requires \"requestId\"".to_string()));
};
let outcome = match (p_str_opt(params, "token"), params.get("expiration").and_then(|v| v.as_i64())) {
(Some(token), Some(expiration)) if !token.is_empty() => Ok((token.to_string(), expiration)),
_ => Err(p_str_opt(params, "error").unwrap_or("no token").to_string()),
};
(Some(serde_json::json!({ "accepted": backend::twitch::playback::provide_integrity(request_id, outcome) })), None)
}
// The `auth-token` of a twitch.tv session, for playing a stream as
// the person who signed in (their Turbo, their subscriptions). Sent
// here by the window's sign-in page and by nothing else; an empty one
// forgets it.
// The twitch.tv session the sign-in window saw while the person signed in there, handed to the sign-in
// that is waiting for them to approve its code, so one action gives the account both.
"setTwitchLoginWebToken" => {
let (Some(login_id), Some(token)) = (p_str_opt(params, "loginId"), p_str_opt(params, "token")) else {
return (None, Some("setTwitchLoginWebToken requires \"loginId\" and \"token\"".to_string()));
};
if backend::twitch::login::set_web_token(login_id, token) {
(Some(ok_node()), None)
} else {
(None, Some("no Twitch sign-in is waiting".to_string()))
}
}
"setTwitchWebToken" => {
let Some(account_id) = p_str_opt(params, "accountId") else {
return (None, Some("setTwitchWebToken requires \"accountId\"".to_string()));
};
let token = p_str_opt(params, "token").unwrap_or("").trim().to_string();
match state.accounts.mutate_twitch(account_id, |a| a.web_token = token.clone()) {
Ok(true) => (Some(ok_node()), None),
Ok(false) => (None, Some("no such Twitch account".to_string())),
Err(e) => (None, Some(format!("{e:#}"))),
}
}
// Opens the channels a Twitch account follows: the ones on air now, or
// all of them up to two hundred. Joined one at a time by the
// connection, which paces them to what Twitch allows; this answers
// with how many were asked for.
"syncTwitchFollows" => {
let Some(account_id) = p_str_opt(params, "accountId") else {
return (None, Some("syncTwitchFollows requires \"accountId\"".to_string()));
};
let Some(cfg) = state.accounts.get_twitch(account_id).filter(|a| a.is_signed_in()) else {
return (None, Some("sign in to Twitch from Accounts to see the channels you follow".to_string()));
};
let live_only = params.get("liveOnly").and_then(|v| v.as_bool()).unwrap_or(true);
let Some(sender) = state.runtime.twitch_sender(account_id) else {
return (None, Some("that Twitch account is not connected".to_string()));
};
let http = match backend::twitch::client() {
Ok(c) => c,
Err(e) => return (None, Some(format!("{e:#}"))),
};
let who = backend::twitch::helix::Caller { root: backend::twitch::helix::API_ROOT, client_id: backend::twitch::auth::CLIENT_ID, token: &cfg.access_token };
match backend::twitch::helix::followed(&http, &who, &cfg.user_id, live_only, 200).await {
Ok(logins) => {
let count = logins.len();
for login in logins {
let _ = sender.send(backend::twitch::Command::Join(login, None));
}
(Some(serde_json::json!({ "count": count })), None)
}
Err(e) => (None, Some(format!("{e:#}"))),
}
}
// Something a moderator does to a Twitch channel from a menu: remove a
// message, time somebody out, ban them, let them back. The slash
// commands do the same through the composer (backend/twitch/moderation.rs).
"twitchModerate" => {
use backend::twitch::moderation::Action;
let (Some(buffer_id), Some(kind)) = (p_str_opt(params, "bufferId"), p_str_opt(params, "action")) else {
return (None, Some("twitchModerate requires \"bufferId\" and \"action\"".to_string()));
};
let Some(buffer) = state.runtime.get_buffer(buffer_id).filter(|b| b.account_id.starts_with("twitch:")) else {
return (None, Some("that is not a Twitch channel".to_string()));
};
let login = p_str_opt(params, "login").unwrap_or("").trim().trim_start_matches('@').to_lowercase();
let reason = p_str_opt(params, "reason").unwrap_or("").to_string();
let action = match kind {
"delete" => Action::Delete { message_id: p_str_opt(params, "messageId").map(str::to_string) },
"timeout" => Action::Timeout { user: login, seconds: params.get("seconds").and_then(|v| v.as_u64()).unwrap_or(600), reason },
"ban" => Action::Ban { user: login, reason },
"unban" => Action::Unban { user: login },
"block" => Action::Block { user: login },
"unblock" => Action::Unblock { user: login },
"shoutout" => Action::Shoutout { user: login },
other => return (None, Some(format!("no Twitch moderation action called {other}"))),
};
let Some(sender) = state.runtime.twitch_sender(&buffer.account_id) else {
return (None, Some("that Twitch account is not connected".to_string()));
};
let (respond, answer) = tokio::sync::oneshot::channel();
let command = backend::twitch::Command::Act { channel: buffer.name.clone(), action, target_id: p_str_opt(params, "userId").map(str::to_string), respond };
if sender.send(command).is_err() {
return (None, Some("that Twitch account is not connected".to_string()));
}
match tokio::time::timeout(std::time::Duration::from_secs(30), answer).await {
Ok(Ok(Ok(done))) => (Some(serde_json::json!({ "done": done })), None),
Ok(Ok(Err(e))) => (None, Some(e)),
Ok(Err(_)) => (None, Some("the Twitch connection dropped".to_string())),
Err(_) => (None, Some("Twitch took too long to answer".to_string())),
}
}
// A Twitch channel's recent past, from the third-party service the
// window was told it may ask (Settings, Twitch). Twitch keeps no
// history of its own; see backend/twitch/history.rs.
"twitchRecentMessages" => {
let Some(buffer_id) = p_str_opt(params, "bufferId") else {
return (None, Some("twitchRecentMessages requires \"bufferId\"".to_string()));
};
let Some(buffer) = state.runtime.get_buffer(buffer_id).filter(|b| b.account_id.starts_with("twitch:")) else {
return (None, Some("that is not a Twitch channel".to_string()));
};
match backend::twitch::history::load(state, &buffer.account_id, &buffer.name, backend::twitch::history::ROOT).await {
Ok(read) => (Some(serde_json::json!({ "read": read })), None),
Err(e) => (None, Some(format!("{e:#}"))),
}
}
// The code Steam asked for, for a sign-in that is waiting on one.
"submitSteamGuardCode" => {
let (Some(login_id), Some(code)) = (p_str_opt(params, "loginId"), p_str_opt(params, "code")) else {
@@ -4106,6 +4568,39 @@ pub async fn dispatch(
}
}
// Signing in to Telegram takes place over time - Telegram sends a code, and may then ask for the account's
// cloud password - so this answers with an id and the rest is events: see backend/telegram/login.rs.
"addTelegramAccount" => {
let (Some(api_id), Some(api_hash), Some(phone)) = (params.get("apiId").and_then(|v| v.as_i64().or_else(|| v.as_str().and_then(|s| s.trim().parse().ok()))), p_str_opt(params, "apiHash"), p_str_opt(params, "phone")) else {
return (None, Some("addTelegramAccount requires \"apiId\", \"apiHash\" and \"phone\"".to_string()));
};
let Ok(api_id) = i32::try_from(api_id) else {
return (None, Some("that api_id is not a number Telegram would have given".to_string()));
};
let login_id = format!("telegram-{}", crate::model::next_message_id());
backend::telegram::login::start(state.clone(), login_id.clone(), api_id, api_hash.trim().to_string(), phone.to_string());
(Some(serde_json::json!({ "loginId": login_id })), None)
}
// The code Telegram sent, and the cloud password it may ask for next, for a sign-in that is waiting.
"submitTelegramCode" | "submitTelegramPassword" => {
let (Some(login_id), Some(value)) = (p_str_opt(params, "loginId"), p_str_opt(params, "value")) else {
return (None, Some(format!("{method} requires \"loginId\" and \"value\"")));
};
let given = if method == "submitTelegramCode" { backend::telegram::login::submit_code(login_id, value) } else { backend::telegram::login::submit_password(login_id, value) };
match given {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(format!("{e:#}"))),
}
}
"cancelTelegramLogin" => {
if let Some(login_id) = p_str_opt(params, "loginId") {
backend::telegram::login::cancel(login_id);
}
(Some(ok_node()), None)
}
// The form was closed with a sign-in still waiting.
"cancelSteamLogin" => {
if let Some(login_id) = p_str_opt(params, "loginId") {
@@ -4594,6 +5089,12 @@ pub async fn dispatch(
let Some(buffer_id) = p_str_opt(params, "bufferId") else {
return (None, Some("refreshKickStream requires \"bufferId\"".to_string()));
};
// Twitch's channels answer the same question, in the same event.
if buffer_id.starts_with("twitch:") {
let (state, buffer_id) = (state.clone(), buffer_id.to_string());
tokio::spawn(async move { backend::twitch::stream::refresh_one(&state, &buffer_id).await });
return (Some(ok_node()), None);
}
if state.runtime.kick_channel(buffer_id).is_none() {
return (None, Some("that is not a Kick channel".to_string()));
}
@@ -4632,6 +5133,12 @@ pub async fn dispatch(
// running tally is what most people mean by a poll, and a hidden
// one is the deliberate choice.
let disclosed = params.get("disclosed").and_then(Value::as_bool).unwrap_or(true);
if buffer.account_id.starts_with("telegram:") {
return match backend::telegram::polls::start(state, &buffer, question, &answers, disclosed).await {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(e)),
};
}
match backend::matrix::start_poll(state, &buffer.account_id, buffer_id, question, &answers, disclosed).await {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(format!("{e:#}"))),
@@ -4682,6 +5189,25 @@ pub async fn dispatch(
};
}
}
if channel_account(state, buffer_id).starts_with("telegram:") {
let (Some(choice), Some(buffer)) = (p_str_opt(params, "answerId"), state.runtime.get_buffer(buffer_id)) else {
return (None, Some("votePoll requires \"answerId\"".to_string()));
};
return match backend::telegram::polls::vote(state, &buffer, choice).await {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(e)),
};
}
if channel_account(state, buffer_id).starts_with("twitch:") {
// Twitch's choices are named by ids of their own, which the window hands back as text.
let Some(choice) = p_str_opt(params, "answerId") else {
return (None, Some("votePoll requires \"answerId\"".to_string()));
};
return match backend::twitch::points::vote_in_buffer(state, backend::twitch::gql::ROOT, buffer_id, choice).await {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(e)),
};
}
let Some(channel) = state.runtime.kick_channel(buffer_id) else {
return (None, Some("that is not a Kick channel".to_string()));
};
@@ -4709,6 +5235,12 @@ pub async fn dispatch(
return (None, Some("betPrediction requires \"optionId\"".to_string()));
};
let amount = params.get("amount").and_then(|v| v.as_i64()).unwrap_or(0);
if channel_account(state, buffer_id).starts_with("twitch:") {
return match backend::twitch::points::bet_in_buffer(state, backend::twitch::gql::ROOT, buffer_id, option_id, amount.max(0) as u64).await {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(e)),
};
}
let Some(channel) = state.runtime.kick_channel(buffer_id) else {
return (None, Some("that is not a Kick channel".to_string()));
};
@@ -4785,6 +5317,58 @@ pub async fn dispatch(
(Some(serde_json::json!({ "watching": live_id.is_some() })), None)
}
// A Twitch channel's points balance, as the person's twitch.tv session; only when asked for (see
// backend/twitch/points.rs).
"twitchPoints" => {
let Some(buffer_id) = p_str_opt(params, "bufferId") else {
return (None, Some("twitchPoints requires \"bufferId\"".to_string()));
};
match backend::twitch::points::points_in_buffer(state, backend::twitch::gql::ROOT, buffer_id).await {
Ok(p) => (Some(serde_json::json!({ "points": p.balance, "name": p.name })), None),
Err(e) => (None, Some(e)),
}
}
// The window is playing a Twitch channel's video: told once a minute, and passed on to Twitch as the
// player on twitch.tv does it (see backend/twitch/watch.rs).
"reportTwitchWatching" => {
let Some(buffer_id) = p_str_opt(params, "bufferId") else {
return (None, Some("reportTwitchWatching requires \"bufferId\"".to_string()));
};
match backend::twitch::watch::report(state, backend::twitch::gql::ROOT, backend::twitch::watch::PAGE_ROOT, buffer_id).await {
Ok(sent) => (Some(serde_json::json!({ "sent": sent })), None),
Err(e) => (None, Some(e)),
}
}
// What a Twitch channel trades its points for, and spending them on one: the panel over the chat lists
// them and a button redeems (see backend/twitch/points.rs).
"listTwitchRewards" => {
let Some(buffer_id) = p_str_opt(params, "bufferId") else {
return (None, Some("listTwitchRewards requires \"bufferId\"".to_string()));
};
match backend::twitch::points::rewards_in_buffer(state, backend::twitch::gql::ROOT, buffer_id).await {
Ok((rewards, points)) => {
let list: Vec<serde_json::Value> = rewards
.into_iter()
.map(|r| serde_json::json!({ "id": r.id, "title": r.title, "cost": r.cost, "description": r.prompt, "enabled": r.enabled, "paused": r.paused, "needsInput": r.needs_input, "color": r.color, "image": r.image, "unavailable": r.unavailable, "choices": r.choices.iter().map(|c| serde_json::json!({ "id": c.id, "label": c.label, "image": c.image, "group": c.group, "groupImage": c.group_image })).collect::<Vec<_>>() }))
.collect();
(Some(serde_json::json!({ "rewards": list, "points": points.balance, "name": points.name })), None)
}
Err(e) => (None, Some(e)),
}
}
"redeemTwitchReward" => {
let (Some(buffer_id), Some(reward_id)) = (p_str_opt(params, "bufferId"), p_str_opt(params, "rewardId")) else {
return (None, Some("redeemTwitchReward requires \"bufferId\" and \"rewardId\"".to_string()));
};
match backend::twitch::points::redeem_in_buffer(state, backend::twitch::gql::ROOT, buffer_id, reward_id, p_str_opt(params, "input"), p_str_opt(params, "choice")).await {
Ok(p) => (Some(serde_json::json!({ "points": p.balance })), None),
Err(e) => (None, Some(e)),
}
}
"kickPoints" => {
let Some(buffer_id) = p_str_opt(params, "bufferId") else {
return (None, Some("kickPoints requires \"bufferId\"".to_string()));
@@ -5146,6 +5730,23 @@ pub async fn dispatch(
// bufferId (not roomId) throughout, matching every other message-
// scoped Matrix method - the buffer->room_id lookup happens inside
// moderation.rs itself, same as sendMessage/editMessage/etc.
// What this Telegram account may do to others in a conversation, and doing it.
"telegramPermissions" | "moderateTelegramMember" => {
let Some(buffer) = p_str_opt(params, "bufferId").and_then(|b| state.runtime.get_buffer(b)).filter(|b| b.account_id.starts_with("telegram:")) else {
return (None, Some(format!("{method} requires a Telegram \"bufferId\"")));
};
if method == "telegramPermissions" {
return match backend::telegram::admin::permissions(state, &buffer).await {
Ok(permissions) => (Some(permissions), None),
Err(e) => (None, Some(e)),
};
}
match backend::telegram::admin::moderate(state, &buffer, p_str(params, "userId", ""), p_str(params, "action", "")).await {
Ok(()) => (Some(ok_node()), None),
Err(e) => (None, Some(e)),
}
}
"getMatrixRoomPermissions" => {
let (account_id, buffer_id) = match (p_str_opt(params, "accountId"), p_str_opt(params, "bufferId")) {
(Some(a), Some(b)) => (a, b),
@@ -5275,6 +5876,15 @@ pub async fn dispatch(
backend::matrix::open_dm(state, account_id, user_id, nick).await
}
}
// A Twitch whisper is a conversation with a login: it exists as soon as it is named.
"twitch" => {
let who = if nick.is_empty() { user_id } else { nick }.trim().trim_start_matches('@').to_lowercase();
if who.is_empty() || !who.chars().all(|c| c.is_ascii_alphanumeric() || c == '_') {
Err(anyhow::anyhow!("no login to whisper to"))
} else {
Ok(state.runtime.ensure_buffer(state, account_id, &backend::twitch::chat::whisper_buffer(&who), "dm").id)
}
}
other => Err(anyhow::anyhow!("{other} has no direct messages")),
};
match opened {
src/runtime.rs
+169 -4
View File
@@ -449,6 +449,13 @@ pub struct Runtime {
/// snapshot here lets subscribe() replay it immediately - see
/// rpc/methods.rs's subscribe handler.
presence: Mutex<HashMap<String, serde_json::Value>>,
/// How many people a roster stands for, where that is more than it lists.
///
/// Kick's roster is the most recent speakers and is cut off to something
/// a window can draw, while the count of everybody who has spoken keeps
/// going. Beside the snapshot above so that a window opened later is told
/// it as well.
presence_total: Mutex<HashMap<String, usize>>,
/// Generic (protocol-agnostic) "what's my own display name on this
/// account" - record_message's highlight check needs this for every
/// backend, not just IRC (which already tracked it per-connection via
@@ -530,9 +537,14 @@ pub struct Runtime {
/// One per connected Kick account, since one websocket carries every
/// channel that account watches - see backend::kick's module doc.
kick_senders: Mutex<HashMap<String, tokio::sync::mpsc::UnboundedSender<crate::backend::kick::Command>>>,
twitch_senders: Mutex<HashMap<String, tokio::sync::mpsc::UnboundedSender<crate::backend::twitch::Command>>>,
/// Whether this account may moderate each Twitch channel it has open, as Twitch last said.
twitch_rights: Mutex<HashMap<String, bool>>,
/// Each connected Steam account's connection, by account id: how a message
/// or a status change reaches Steam from outside the connection's own task.
steam_links: Mutex<HashMap<String, crate::backend::steam::link::Link>>,
/// A connected Telegram account's client, by account id (see backend/telegram).
telegram_handles: Mutex<HashMap<String, crate::backend::telegram::Handle>>,
/// Who has asked to be a Steam account's friend, and who it has asked:
/// what the window is shown on the account's page, kept so a window opened
/// after the request arrived can still be told.
@@ -1044,6 +1056,17 @@ fn cap_cache<K: std::hash::Hash + Eq, V>(map: &mut HashMap<K, V>, limit: usize)
});
}
/// The heading a conversation is always filed under, for a service whose list has fixed ones: Twitch's is
/// "Channels" and "Direct Messages" (whispers), the second drawn only while there is a whisper to put under it.
/// They are the service's own headings, so they cannot be renamed, removed or filled by hand.
fn fixed_category(account_id: &str, kind: &str) -> Option<String> {
if account_id.starts_with("telegram:") {
// Groups are the usual channel; a broadcast channel moves to its own heading when its buffer is made.
return Some(if kind == "dm" { "Direct Messages".to_string() } else { "Groups".to_string() });
}
account_id.starts_with("twitch:").then(|| if kind == "dm" { "Direct Messages".to_string() } else { "Channels".to_string() })
}
impl Runtime {
pub fn new() -> Self {
Self {
@@ -1065,6 +1088,7 @@ impl Runtime {
discord_member_windows: Mutex::new(HashMap::new()),
buffers: Mutex::new(HashMap::new()),
presence: Mutex::new(HashMap::new()),
presence_total: Mutex::new(HashMap::new()),
own_identity: Mutex::new(HashMap::new()),
discord_channels: Mutex::new(HashMap::new()),
discord_gateway_sessions: Mutex::new(HashMap::new()),
@@ -1085,7 +1109,10 @@ impl Runtime {
discord_friends: Mutex::new(HashMap::new()),
sneedchat_senders: Mutex::new(HashMap::new()),
kick_senders: Mutex::new(HashMap::new()),
twitch_senders: Mutex::new(HashMap::new()),
twitch_rights: Mutex::new(HashMap::new()),
steam_links: Mutex::new(HashMap::new()),
telegram_handles: Mutex::new(HashMap::new()),
steam_requests: Mutex::new(HashMap::new()),
kick_channels: Mutex::new(HashMap::new()),
kick_streams: Mutex::new(HashMap::new()),
@@ -1231,6 +1258,16 @@ impl Runtime {
let conn = conn_states.get(&id).map(|s| s.as_str()).unwrap_or("disconnected");
crate::accounts::steam_account_to_json(cfg, conn, self.own_identity(&id).as_deref())
}));
out.extend(state.accounts.all_twitch().iter().map(|cfg| {
let id = cfg.account_id();
let conn = conn_states.get(&id).map(|s| s.as_str()).unwrap_or("disconnected");
crate::accounts::twitch_account_to_json(cfg, conn)
}));
out.extend(state.accounts.all_telegram().iter().map(|cfg| {
let id = cfg.account_id();
let conn = conn_states.get(&id).map(|s| s.as_str()).unwrap_or("disconnected");
crate::accounts::telegram_account_to_json(cfg, conn)
}));
out
}
@@ -1630,7 +1667,12 @@ impl Runtime {
let id = model::buffer_id(account_id, name);
self.confirm_restored(account_id, &id, false);
let mut buffers = self.buffers.lock().unwrap();
if let Some(existing) = buffers.get(&id) {
if let Some(existing) = buffers.get_mut(&id) {
// A Twitch account's list is always headed the same two ways (see `fixed_category`), including
// for what was open before it was.
if existing.category.is_none() {
existing.category = fixed_category(account_id, kind);
}
return existing.clone();
}
let last_activity_ts = state.store.last_activity(&id).unwrap_or(0);
@@ -1646,7 +1688,7 @@ impl Runtime {
// Filled in by whichever backend knows: a Discord guild files its
// channels under the server's own headings, and everything else
// has none.
category: None,
category: fixed_category(account_id, kind),
position: 0,
syncing: false,
link: None,
@@ -1663,6 +1705,7 @@ impl Runtime {
favourite: false,
low_priority: false,
service_room: false,
name_icon: None,
topic: None,
read_only: None,
forum: false,
@@ -1729,6 +1772,23 @@ impl Runtime {
/// One way only. A homeserver does not un-designate its notices room, and
/// a client that cleared the mark because one sync happened not to carry
/// the tag would offer a Leave that the server refuses.
/// Draws a glyph in front of a conversation's name in the list (see `Buffer::name_icon`), said once it changes.
pub fn set_buffer_name_icon(&self, state: &AppState, buffer_id: &str, icon: Option<&str>) {
let updated = {
let mut buffers = self.buffers.lock().unwrap();
match buffers.get_mut(buffer_id) {
Some(b) if b.name_icon.as_deref() != icon => {
b.name_icon = icon.map(String::from);
Some(b.clone())
}
_ => None,
}
};
if let Some(b) = updated {
state.events.emit("bufferListChange", serde_json::to_value(&b).unwrap());
}
}
pub fn set_buffer_service_room(&self, state: &AppState, buffer_id: &str) {
let updated = {
let mut buffers = self.buffers.lock().unwrap();
@@ -2020,6 +2080,14 @@ impl Runtime {
self.presence.lock().unwrap().insert(buffer_id.to_string(), members);
}
pub fn set_presence_total(&self, buffer_id: &str, total: usize) {
self.presence_total.lock().unwrap().insert(buffer_id.to_string(), total);
}
pub fn get_presence_total(&self, buffer_id: &str) -> Option<usize> {
self.presence_total.lock().unwrap().get(buffer_id).copied()
}
pub fn get_presence(&self, buffer_id: &str) -> Option<serde_json::Value> {
self.presence.lock().unwrap().get(buffer_id).cloned()
}
@@ -2038,6 +2106,14 @@ impl Runtime {
.insert(source.id.clone(), source);
}
/// Forgets a source and the ones named under it (`id:...`), for a set that is rebuilt whole.
pub fn clear_emoji_sources(&self, account_id: &str, id: &str) {
if let Some(sources) = self.emoji_catalogue.lock().unwrap().get_mut(account_id) {
let under = format!("{id}:");
sources.retain(|have, _| have != id && !have.starts_with(&under));
}
}
/// Everything this account can reach, newest knowledge of each source.
pub fn emoji_catalogue(&self, account_id: &str) -> Vec<crate::model::EmojiSource> {
self.emoji_catalogue
@@ -3797,6 +3873,18 @@ impl Runtime {
self.kick_streams.lock().unwrap().get(buffer_id).cloned()
}
/// The conversation an account has open for a Kick channel, by the channel's
/// own number - which is how the points event names it.
pub fn kick_buffer_of_channel(&self, account_id: &str, channel_id: u64) -> Option<String> {
let prefix = format!("{account_id}|");
self.kick_channels
.lock()
.unwrap()
.iter()
.find(|(buffer_id, channel)| buffer_id.starts_with(&prefix) && channel.channel_id == channel_id)
.map(|(buffer_id, _)| buffer_id.clone())
}
pub fn kick_channel(&self, buffer_id: &str) -> Option<KickChannel> {
self.kick_channels.lock().unwrap().get(buffer_id).cloned()
}
@@ -3875,6 +3963,22 @@ impl Runtime {
self.kick_senders.lock().unwrap().insert(account_id.to_string(), sender);
}
pub fn set_twitch_sender(&self, account_id: &str, sender: tokio::sync::mpsc::UnboundedSender<crate::backend::twitch::Command>) {
self.twitch_senders.lock().unwrap().insert(account_id.to_string(), sender);
}
pub fn set_twitch_rights(&self, buffer_id: &str, moderator: bool) {
self.twitch_rights.lock().unwrap().insert(buffer_id.to_string(), moderator);
}
pub fn twitch_rights(&self, buffer_id: &str) -> Option<bool> {
self.twitch_rights.lock().unwrap().get(buffer_id).copied()
}
pub fn twitch_sender(&self, account_id: &str) -> Option<tokio::sync::mpsc::UnboundedSender<crate::backend::twitch::Command>> {
self.twitch_senders.lock().unwrap().get(account_id).cloned()
}
pub fn kick_sender(&self, account_id: &str) -> Option<tokio::sync::mpsc::UnboundedSender<crate::backend::kick::Command>> {
self.kick_senders.lock().unwrap().get(account_id).cloned()
}
@@ -4271,6 +4375,18 @@ impl Runtime {
}
/// Registers a signed-in Steam connection.
pub fn set_telegram_handle(&self, account_id: &str, handle: crate::backend::telegram::Handle) {
self.telegram_handles.lock().unwrap().insert(account_id.to_string(), handle);
}
pub fn clear_telegram_handle(&self, account_id: &str) {
self.telegram_handles.lock().unwrap().remove(account_id);
}
pub fn telegram_handle(&self, account_id: &str) -> Option<crate::backend::telegram::Handle> {
self.telegram_handles.lock().unwrap().get(account_id).cloned()
}
pub fn set_steam_link(&self, account_id: &str, link: crate::backend::steam::link::Link) {
self.steam_links.lock().unwrap().insert(account_id.to_string(), link);
}
@@ -4608,6 +4724,7 @@ impl Runtime {
// Filled in by the backend that has them, immediately after this
// returns - only Discord has any, and it is the one that knows.
components: Vec::new(),
moderated: None,
id: msg_id,
buffer_id: buffer.id.clone(),
from: from.to_string(),
@@ -4755,8 +4872,56 @@ impl Runtime {
}
}
/// Says who and what a reply answered, once it has been looked up (a reply to something not read here yet).
pub fn fill_reply_preview(&self, state: &AppState, buffer_id: &str, msg_id: &str, from: &str, body: &str) {
match state.store.set_reply_text(buffer_id, msg_id, from, body) {
Ok(true) => state.events.emit("messageReplyRenamed", json!({ "bufferId": buffer_id, "id": msg_id, "from": from, "body": body })),
Ok(false) => {}
Err(e) => tracing::warn!("failed to fill in a reply: {e}"),
}
}
/// Discord's MESSAGE_DELETE - removes the row and tells clients to
/// drop it from view, same as Discord's own clients (no tombstone).
/// A message the service took down, kept and marked as taken down. Quiet
/// about one that is not stored: a removal of something older than this
/// client's scrollback is not news here.
pub fn moderate_message(&self, state: &AppState, buffer_id: &str, msg_id: &str, why: &model::Moderated) {
match state.store.mark_moderated(buffer_id, msg_id, why) {
Ok(true) => state.events.emit("messageModerated", json!({ "bufferId": buffer_id, "id": msg_id, "moderated": why })),
Ok(false) => {}
Err(e) => tracing::warn!("failed to mark a message as removed: {e}"),
}
}
/// Everything a person said in a conversation, kept and marked as removed:
/// what a timeout or a ban does to their messages.
pub fn moderate_sender(&self, state: &AppState, buffer_id: &str, sender_id: &str, why: &model::Moderated) {
match state.store.mark_moderated_by_sender(buffer_id, sender_id, why) {
Ok(ids) => {
for id in ids {
state.events.emit("messageModerated", json!({ "bufferId": buffer_id, "id": id, "moderated": why }));
}
}
Err(e) => tracing::warn!("failed to mark a person's messages as removed: {e}"),
}
}
/// Says how a channel's chat is restricted, in the words the header shows,
/// where the service has its own modes (Twitch's room state).
pub fn set_channel_modes(&self, state: &AppState, buffer_id: &str, modes: Option<String>) {
let updated = {
let mut buffers = self.buffers.lock().unwrap();
let Some(buffer) = buffers.get_mut(buffer_id) else { return };
if buffer.channel_modes == modes {
return;
}
buffer.channel_modes = modes;
buffer.clone()
};
state.events.emit("bufferListChange", serde_json::to_value(&updated).unwrap());
}
pub fn delete_message(&self, state: &AppState, buffer_id: &str, msg_id: &str) {
match state.store.delete_message(buffer_id, msg_id) {
Ok(true) => state.events.emit("messageDeleted", json!({ "bufferId": buffer_id, "id": msg_id })),
@@ -5501,11 +5666,11 @@ impl Runtime {
restored_buffers, restored_groups, layout_confirmed,
connect_generation, matrix_invites, discord_resync,
discord_own_roles, discord_guild_roles, discord_guild_owners,
discord_member_windows, presence, own_identity,
discord_member_windows, presence, presence_total, own_identity,
discord_channels, discord_gateway_sessions, ringing_calls,
discord_guild_id, discord_buffer_emojis, emoji_catalogue,
kick_watching, matrix_emoticons, discord_friends,
sneedchat_senders, kick_senders, steam_links, steam_requests, kick_channels,
sneedchat_senders, kick_senders, twitch_senders, twitch_rights, steam_links, telegram_handles, steam_requests, kick_channels,
kick_streams, kick_stream_fetched, live_cards,
irc_channel_lists, irc_channel_directory, irc_caps, irc_metadata,
discord_last_interaction, matrix_room_creators, matrix_room_versions,
src/store.rs
+83 -13
View File
@@ -187,6 +187,14 @@ impl Store {
// Whether that reference is a message brought here rather than
// one answered - see model::ReplyPreview::forwarded.
"ALTER TABLE messages ADD COLUMN reply_forwarded INTEGER NOT NULL DEFAULT 0",
// The message a chain of replies grew from - see
// model::ReplyPreview::thread_root.
"ALTER TABLE messages ADD COLUMN reply_thread_root TEXT",
// A message the service took down and this client kept - see
// model::Moderated. JSON, and null for a message nobody removed.
"ALTER TABLE messages ADD COLUMN moderated TEXT",
// The conversation a reply's target is in, when it is another - see model::ReplyPreview::chat.
"ALTER TABLE messages ADD COLUMN reply_chat TEXT",
] {
let _ = conn.execute(stmt, []);
}
@@ -443,8 +451,8 @@ impl Store {
// OR IGNORE against the (buffer_id, msg_id) index: recording the
// same message twice is a backend replaying history it already
// has, and the right answer is to keep the copy already stored.
"INSERT OR IGNORE INTO messages (msg_id, buffer_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, buffer_kind, sender_color, badges, reply_is_thread, reply_forwarded)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, COALESCE(?12, '[]'), ?13, ?14, COALESCE(?15, '[]'), ?16, COALESCE(?17, '[]'), ?18, ?19, ?20, COALESCE(?21, '[]'), ?22, ?23)",
"INSERT OR IGNORE INTO messages (msg_id, buffer_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, buffer_kind, sender_color, badges, reply_is_thread, reply_forwarded, reply_thread_root, reply_chat)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, COALESCE(?12, '[]'), ?13, ?14, COALESCE(?15, '[]'), ?16, COALESCE(?17, '[]'), ?18, ?19, ?20, COALESCE(?21, '[]'), ?22, ?23, ?24, ?25)",
params![
msg_id,
buffer_id,
@@ -469,6 +477,8 @@ impl Store {
serde_json::to_string(badges).ok(),
reply_to.is_some_and(|r| r.thread) as i32,
reply_to.is_some_and(|r| r.forwarded) as i32,
reply_to.and_then(|r| r.thread_root.as_deref()),
reply_to.and_then(|r| r.chat.as_deref()),
],
)?;
// Whether this was actually new. The insert has always ignored a
@@ -512,6 +522,16 @@ impl Store {
Ok(changed > 0)
}
/// Fills in who and what a reply answered, once that has been looked up.
pub fn set_reply_text(&self, buffer_id: &str, msg_id: &str, from: &str, body: &str) -> Result<bool> {
let conn = self.conn.lock().unwrap();
let changed = conn.execute(
"UPDATE messages SET reply_to_from = ?1, reply_to_body = ?2 WHERE buffer_id = ?3 AND msg_id = ?4",
params![from, body, buffer_id, msg_id],
)?;
Ok(changed > 0)
}
/// Puts the cards on a message without marking it edited: a link being
/// described after the fact is not something its sender did.
pub fn set_message_embeds(&self, buffer_id: &str, msg_id: &str, embeds: &[Embed]) -> Result<bool> {
@@ -579,6 +599,30 @@ impl Store {
Ok(bodies)
}
/// Keeps a message the service has removed, marked as removed. Answers
/// whether it was here to mark.
pub fn mark_moderated(&self, buffer_id: &str, msg_id: &str, why: &crate::model::Moderated) -> Result<bool> {
let conn = self.conn.lock().unwrap();
let json = serde_json::to_string(why)?;
let rows = conn.execute("UPDATE messages SET moderated = ?3 WHERE buffer_id = ?1 AND msg_id = ?2", params![buffer_id, msg_id, json])?;
Ok(rows > 0)
}
/// Marks everything one person said in a conversation as removed, and answers
/// which messages that was. Twitch does this to a person's messages when it
/// times them out or bans them: no message is named, the person is.
pub fn mark_moderated_by_sender(&self, buffer_id: &str, sender_id: &str, why: &crate::model::Moderated) -> Result<Vec<String>> {
let conn = self.conn.lock().unwrap();
let json = serde_json::to_string(why)?;
let mut stmt = conn.prepare("SELECT msg_id FROM messages WHERE buffer_id = ?1 AND sender_id = ?2 AND kind = 'chat' AND moderated IS NULL")?;
let ids: Vec<String> = stmt.query_map(params![buffer_id, sender_id], |row| row.get::<_, String>(0))?.filter_map(|r| r.ok()).collect();
drop(stmt);
for id in &ids {
conn.execute("UPDATE messages SET moderated = ?3 WHERE buffer_id = ?1 AND msg_id = ?2", params![buffer_id, id, json])?;
}
Ok(ids)
}
pub fn delete_message(&self, buffer_id: &str, msg_id: &str) -> Result<bool> {
let conn = self.conn.lock().unwrap();
let rows = conn.execute("DELETE FROM messages WHERE buffer_id = ?1 AND msg_id = ?2", params![buffer_id, msg_id])?;
@@ -748,7 +792,9 @@ impl Store {
from: row.get::<_, Option<String>>("reply_to_from").ok().flatten().unwrap_or_default(),
body: row.get::<_, Option<String>>("reply_to_body").ok().flatten().unwrap_or_default(),
thread: row.get::<_, Option<i64>>("reply_is_thread").ok().flatten().unwrap_or(0) != 0,
forwarded: row.get::<_, Option<i64>>("reply_forwarded").ok().flatten().unwrap_or(0) != 0 });
forwarded: row.get::<_, Option<i64>>("reply_forwarded").ok().flatten().unwrap_or(0) != 0,
thread_root: row.get::<_, Option<String>>("reply_thread_root").ok().flatten(),
chat: row.get::<_, Option<String>>("reply_chat").ok().flatten() });
let json_column = |name: &str| -> String {
row.get::<_, Option<String>>(name).ok().flatten().unwrap_or_else(|| "[]".to_string())
};
@@ -758,6 +804,7 @@ impl Store {
drop_missing_local_copies(&mut attachments);
let badges = serde_json::from_str(&json_column("badges")).unwrap_or_default();
let components = serde_json::from_str(&json_column("components")).unwrap_or_default();
let moderated = row.get::<_, Option<String>>("moderated").ok().flatten().and_then(|json| serde_json::from_str(&json).ok());
Ok(Message {
id: row.get::<_, Option<String>>("msg_id")?.unwrap_or_default(),
buffer_id: buffer_id.to_string(),
@@ -782,6 +829,7 @@ impl Store {
badges,
html: row.get("html")?,
components,
moderated,
})
}
@@ -789,7 +837,7 @@ impl Store {
let conn = self.conn.lock().unwrap();
let limit = if limit > 0 { limit } else { 200 };
let mut stmt = conn.prepare(
"SELECT msg_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, edited, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, sender_color, badges, reply_is_thread, reply_forwarded, components
"SELECT msg_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, edited, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, sender_color, badges, reply_is_thread, reply_forwarded, components, reply_thread_root, reply_chat, moderated
FROM messages
WHERE buffer_id = ?1 AND (?2 <= 0 OR ts < ?2)
ORDER BY ts DESC LIMIT ?3",
@@ -865,7 +913,7 @@ impl Store {
let conn = self.conn.lock().unwrap();
let limit = if limit > 0 { limit } else { 200 };
let mut stmt = conn.prepare(
"SELECT msg_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, edited, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, sender_color, badges, reply_is_thread, reply_forwarded, components
"SELECT msg_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, edited, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, sender_color, badges, reply_is_thread, reply_forwarded, components, reply_thread_root, reply_chat, moderated
FROM messages
WHERE buffer_id = ?1 AND ts > ?2
ORDER BY ts ASC LIMIT ?3",
@@ -888,7 +936,7 @@ impl Store {
let conn = self.conn.lock().unwrap();
let limit = if limit > 0 { limit } else { 500 };
let mut stmt = conn.prepare(
"SELECT msg_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, edited, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, sender_color, badges, reply_is_thread, reply_forwarded, components
"SELECT msg_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, edited, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, sender_color, badges, reply_is_thread, reply_forwarded, components, reply_thread_root, reply_chat, moderated
FROM messages
WHERE buffer_id = ?1 AND ts > ?2 AND ts <= ?3
ORDER BY ts ASC LIMIT ?4",
@@ -929,7 +977,7 @@ impl Store {
pub fn messages_around(&self, buffer_id: &str, ts: i64, span: i64) -> Result<Vec<Message>> {
let conn = self.conn.lock().unwrap();
let span = if span > 0 { span } else { 50 };
const COLUMNS: &str = "msg_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, edited, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, sender_color, badges, reply_is_thread, reply_forwarded, components";
const COLUMNS: &str = "msg_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, edited, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, sender_color, badges, reply_is_thread, reply_forwarded, components, reply_thread_root, reply_chat, moderated";
let mut older = conn.prepare(&format!(
"SELECT {COLUMNS} FROM messages WHERE buffer_id = ?1 AND ts <= ?2 ORDER BY ts DESC LIMIT ?3"
@@ -975,7 +1023,7 @@ impl Store {
format!(" OR (buffer_kind IS NULL AND buffer_id IN ({places}))")
};
let sql = format!(
"SELECT msg_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, edited, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, sender_color, badges, reply_is_thread, reply_forwarded, components, buffer_id
"SELECT msg_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, edited, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, sender_color, badges, reply_is_thread, reply_forwarded, components, reply_thread_root, reply_chat, moderated, buffer_id
FROM messages
WHERE is_highlight = 1 AND is_own = 0 AND (buffer_kind = 'channel'{legacy})
ORDER BY ts DESC LIMIT ?"
@@ -1236,7 +1284,7 @@ impl Store {
let escaped = query.replace('\\', "\\\\").replace('%', "\\%").replace('_', "\\_");
let pattern = format!("%{escaped}%");
let mut stmt = conn.prepare(
"SELECT msg_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, edited, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, sender_color, badges, reply_is_thread, reply_forwarded, components
"SELECT msg_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, edited, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, sender_color, badges, reply_is_thread, reply_forwarded, components, reply_thread_root, reply_chat, moderated
FROM messages
WHERE buffer_id = ?1 AND body LIKE ?2 ESCAPE '\\'
ORDER BY ts DESC LIMIT ?3",
@@ -1257,7 +1305,13 @@ fn drop_missing_local_copies(attachments: &mut [Attachment]) {
let gone = |p: &Option<String>| -> bool {
let Some(p) = p.as_deref() else { return false };
let Some(rest) = p.strip_prefix("file://") else { return false };
!std::path::Path::new(rest).exists()
// Telegram's are named before they are fetched, and fetched when the window asks (see backend/telegram/media.rs):
// a file that is not there yet is the usual state of one, not a sign it was swept.
let path = std::path::Path::new(rest);
if path.parent().and_then(|d| d.file_name()).and_then(|n| n.to_str()).is_some_and(|n| n == crate::backend::telegram::media::MEDIA || n == crate::backend::telegram::media::AVATARS) {
return false;
}
!path.exists()
};
for att in attachments.iter_mut() {
if gone(&att.thumbnail_path) {
@@ -1301,9 +1355,9 @@ impl Store {
pub fn thread_messages(&self, buffer_id: &str, root_id: &str) -> Result<Vec<Message>> {
let conn = self.conn.lock().unwrap();
let mut stmt = conn.prepare(
"SELECT msg_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, edited, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, sender_color, badges, reply_is_thread, reply_forwarded, components
"SELECT msg_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, edited, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, sender_color, badges, reply_is_thread, reply_forwarded, components, reply_thread_root, reply_chat, moderated
FROM messages
WHERE buffer_id = ?1 AND ((reply_to_id = ?2 AND reply_is_thread = 1) OR msg_id = ?2)
WHERE buffer_id = ?1 AND ((reply_to_id = ?2 AND reply_is_thread = 1) OR reply_thread_root = ?2 OR msg_id = ?2)
ORDER BY ts ASC, rowid ASC",
)?;
let rows = stmt.query_map(params![buffer_id, root_id], |row| Self::row_to_message(buffer_id, row))?;
@@ -1313,7 +1367,7 @@ impl Store {
pub fn get_message(&self, buffer_id: &str, msg_id: &str) -> Result<Option<Message>> {
let conn = self.conn.lock().unwrap();
let mut stmt = conn.prepare(
"SELECT msg_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, edited, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, sender_color, badges, reply_is_thread, reply_forwarded, components
"SELECT msg_id, from_nick, body, ts, is_action, is_highlight, kind, reply_to_id, reply_to_from, reply_to_body, edited, reactions, is_own, avatar_url, embeds, sender_id, attachments, html, sender_color, badges, reply_is_thread, reply_forwarded, components, reply_thread_root, reply_chat, moderated
FROM messages
WHERE buffer_id = ?1 AND msg_id = ?2",
)?;
@@ -1966,6 +2020,22 @@ mod tests {
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn a_telegram_file_not_fetched_yet_is_still_advertised() {
// Its name is given when the message is recorded and the file is fetched when the window asks, so
// "not there" is how it starts and not a sign it was swept.
let mut attachments = vec![Attachment {
path: Some("file:///home/someone/.cache/nobilis/telegram-media/tg_1_n1_2_full.jpg".to_string()),
thumbnail_path: Some("file:///home/someone/.cache/nobilis/telegram-media/tg_1_n1_2_thumb.jpg".to_string()),
..Default::default()
}];
drop_missing_local_copies(&mut attachments);
assert!(attachments[0].path.is_some() && attachments[0].thumbnail_path.is_some());
let mut swept = vec![Attachment { path: Some("file:///home/someone/.cache/nobilis/matrix-media/gone.jpg".to_string()), ..Default::default() }];
drop_missing_local_copies(&mut swept);
assert!(swept[0].path.is_none(), "another service's swept file is still dropped");
}
#[test]
fn a_remote_url_is_left_alone() {
// Only local copies are checked; a https link is not this code's