Let the homeserver say what a link is

No /preview_url and no m.room.preview_urls: a posted link showed as a
bare URL while Element showed a title, a description and a picture
(#220).

The homeserver does the fetching, which is the part worth having - the
preview costs the reader no request to a stranger's server, the opposite
trade from this client's own content sniffing.

Never in an encrypted room, and no setting here turns that back on.
Asking the homeserver to unfurl a link handed to it in plaintext gives
it a URL that room went to some trouble to keep from it; a room is
encrypted because its contents are nobody else's business, and a picture
is not worth undoing that for. Element defaults the same way and this
does not offer the override.

Both switches are honoured beside that - the account's
org.matrix.preview_urls and the room's own m.room.preview_urls - and a
room that has turned previews off wins over an account that has said
nothing, because that decision was made for everybody in the room. Both
events say "disable", so silence means on.

One link per message. Six cards under one line is an index of a message
rather than a glance at it. The card is fetched after the line is
stored and in the background: the words are what somebody is waiting
for.

Verified against matrix.poast.org, which answers with a title and an
mxc picture - so the image takes the same token-bearing fetch and local
cache as every other piece of Matrix media.

Co-Authored-By: Claude Opus 5 <[email protected]>

src/backend/discord/messages.rs

@@ -709,6 +709,10 @@ pub(super) fn extract_embeds(d: &Value) -> Vec<Embed> {
                 color: embed["color"].as_i64(),
                 timestamp: embed["timestamp"].as_str().map(|s| s.to_string()),
                 url: embed["url"].as_str().map(|s| s.to_string()),
+                // Discord's own thumbnails are already reached through the
+                // message's media sniffing, which pairs them with the embed
+                // in the frontend. Nothing to fetch here.
+                image_url: None,
             })
         })
         .collect()

src/backend/matrix/mod.rs

@@ -50,6 +50,7 @@ pub mod history;
 pub mod media;
 pub mod notifications;
 pub mod probes;
+pub mod previews;
 pub mod profile;
 pub mod receipts;
 pub mod recentemoji;

src/backend/matrix/previews.rs

@@ -0,0 +1,182 @@
+//! What a link in a message turns out to be.
+//!
+//! The homeserver does the fetching, which is the part worth having. Element
+//! works this way and so does this: the preview costs the reader no request
+//! to a stranger's server, which is the opposite trade from this client's own
+//! content sniffing - there, the reader's own machine fetches the page.
+//!
+//! Never in an encrypted room. Asking the homeserver to unfurl a link handed
+//! to it in plaintext gives it a URL that room went to some trouble to keep
+//! from it, and no setting here turns that back on: a room is encrypted
+//! because its contents are nobody else's business, and a preview is not
+//! worth undoing that for. Element defaults the same way; this does not
+//! offer the override.
+//!
+//! Two switches are honoured beside that, both spelled the Matrix way: the
+//! account's `org.matrix.preview_urls`, and the room's own
+//! `m.room.preview_urls` state - a room that has turned previews off must not
+//! have them, whatever the account says.
+
+use super::*;
+
+/// The room state event that turns previews off for everybody in a room.
+pub const ROOM_SETTING: &str = "m.room.preview_urls";
+
+/// The account's own switch. Still under its `org.matrix.` prefix, which is
+/// where every client that reads it looks.
+pub const ACCOUNT_SETTING: &str = "org.matrix.preview_urls";
+
+/// Whether a `disable` flag says to stop.
+///
+/// The events say "disable" rather than "enable", so an absent or unreadable
+/// setting means previews are on - which is what every client does with one.
+pub fn disabled_by(content: &Value) -> bool {
+    content["disable"].as_bool().unwrap_or(false)
+}
+
+/// Whether a link in this room should be unfurled at all.
+pub fn allowed(state: &AppState, account_id: &str, buffer_id: &str, room_id: &str) -> bool {
+    if state.runtime.is_matrix_room_encrypted(buffer_id) {
+        return false;
+    }
+    state.runtime.matrix_previews_allowed(account_id, room_id)
+}
+
+/// Reads the account's own switch at connect.
+///
+/// Asked for directly rather than waited for, like every other piece of
+/// account data here: a resumed session is told only what changed since its
+/// cursor, and a preference set a year ago has not changed.
+pub(super) async fn fetch_account_setting(state: &AppState, account_id: &str, homeserver_url: &str, access_token: &str, user_id: &str) {
+    let content = ssss::read_account_data(homeserver_url, access_token, user_id, ACCOUNT_SETTING)
+        .await
+        .unwrap_or_else(|| serde_json::json!({}));
+    state.runtime.set_matrix_previews_off(account_id, "", disabled_by(&content));
+}
+
+/// The first link in a message body, if it has one.
+///
+/// One rather than all of them. A message with six links would mean six
+/// requests and six cards under one line, and the card is a glance at what
+/// somebody posted rather than an index of it - Element shows one too.
+pub fn first_link(body: &str) -> Option<&str> {
+    body.split_whitespace()
+        .find(|word| word.starts_with("https://") || word.starts_with("http://"))
+        // Trailing punctuation belongs to the sentence, not to the URL:
+        // "look at https://example.org." is a link to example.org.
+        .map(|word| word.trim_end_matches(|c| matches!(c, '.' | ',' | ')' | ']' | '!' | '?' | ';' | ':' | '"' | '\'')))
+        .filter(|url| url.len() > "https://".len())
+}
+
+/// Asks the homeserver what a link is.
+///
+/// Quiet on every failure. A server with previews switched off, a page that
+/// does not answer, a link to something that is not a page - none of those
+/// are worth telling a reader about, because the link itself is still there
+/// and still works. A missing card says "no preview"; an error message where
+/// a card should be says "this client is broken".
+pub async fn fetch(state: &AppState, account_id: &str, url: &str, ts_ms: i64) -> Option<crate::model::Embed> {
+    let account = state.accounts.get_matrix(account_id)?;
+    let asked = format!(
+        "{}/_matrix/client/v1/media/preview_url?url={}&ts={ts_ms}",
+        account.homeserver_url.trim_end_matches('/'),
+        url::form_urlencoded::byte_serialize(url.as_bytes()).collect::<String>()
+    );
+    let answer = http::get_json(&asked, &account.access_token).await.ok()?;
+
+    let text = |key: &str| answer[key].as_str().filter(|v| !v.is_empty()).map(str::to_string);
+    let title = text("og:title");
+    let description = text("og:description");
+    // The picture comes back as an mxc URI - the homeserver fetched it and
+    // kept a copy, so it needs the same token-bearing download everything
+    // else here does and cannot be handed to a frontend as it stands.
+    let image = match answer["og:image"].as_str().filter(|v| v.starts_with("mxc://")) {
+        Some(mxc) => {
+            let ext = extension_for_mimetype(answer["og:image:type"].as_str().unwrap_or(""));
+            cached_media_path(&account.homeserver_url, &account.access_token, mxc, ext).await
+        }
+        None => None,
+    };
+
+    // A card with nothing on it is not a card. Servers answer `{}` for a page
+    // they could not read, and drawing an empty box under the message would
+    // be worse than drawing nothing.
+    if title.is_none() && description.is_none() && image.is_none() {
+        return None;
+    }
+    Some(crate::model::Embed {
+        title,
+        description,
+        color: None,
+        timestamp: None,
+        url: Some(url.to_string()),
+        image_url: image,
+    })
+}
+
+/// Unfurls a message's link and puts the card on it, if there is one.
+///
+/// After the message is stored rather than before: the line is what somebody
+/// is waiting for, and a homeserver fetching somebody else's slow page is not
+/// something to hold a conversation up for. Spawned for the same reason.
+pub(super) fn unfurl_later(
+    state: &AppState,
+    account_id: &str,
+    buffer_id: &str,
+    room_id: &str,
+    event_id: &str,
+    body: &str,
+    ts_ms: i64,
+) {
+    if !allowed(state, account_id, buffer_id, room_id) {
+        return;
+    }
+    let Some(link) = first_link(body) else { return };
+    let (state, account_id, buffer_id, event_id, body, link) = (
+        state.clone(),
+        account_id.to_string(),
+        buffer_id.to_string(),
+        event_id.to_string(),
+        body.to_string(),
+        link.to_string(),
+    );
+    tokio::spawn(async move {
+        let Some(embed) = fetch(&state, &account_id, &link, ts_ms).await else { return };
+        // The body again, unchanged: update_message takes the whole message
+        // and this only means to add the card to it.
+        state.runtime.update_message(&state, &buffer_id, &event_id, &body, &[embed], &[]);
+    });
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    /// One link, and the sentence's punctuation left out of it.
+    #[test]
+    fn the_first_link_is_the_one_that_gets_a_card() {
+        assert_eq!(first_link("look at https://example.org/page"), Some("https://example.org/page"));
+        // The full stop ends the sentence, not the address.
+        assert_eq!(first_link("see https://example.org."), Some("https://example.org"));
+        assert_eq!(first_link("(https://example.org/a)"), Some("(https://example.org/a"));
+        // First of several: a card is a glance at what somebody posted, not
+        // an index of it.
+        assert_eq!(first_link("https://one.example https://two.example"), Some("https://one.example"));
+
+        assert_eq!(first_link("nothing here"), None);
+        assert_eq!(first_link("mailto:[email protected]"), None);
+        // A bare scheme is not a link.
+        assert_eq!(first_link("https://"), None);
+    }
+
+    /// The events say "disable", so silence means previews are on - which is
+    /// what every client does with one.
+    #[test]
+    fn nothing_said_leaves_previews_on() {
+        assert!(!disabled_by(&serde_json::json!({})));
+        assert!(!disabled_by(&serde_json::json!({ "disable": false })));
+        assert!(disabled_by(&serde_json::json!({ "disable": true })));
+        // Something that is not a boolean is not a refusal.
+        assert!(!disabled_by(&serde_json::json!({ "disable": "yes" })));
+    }
+}

src/backend/matrix/probes.rs

@@ -796,6 +796,24 @@ mod live {
         assert_eq!(source["room_id"].as_str(), Some(room_id.as_str()), "{source}");
         println!("event source: type {}", source["type"]);
 
+        // And what the homeserver makes of a link. Its own endpoint under
+        // /media, and a server is free to have it switched off entirely -
+        // which is a result rather than a failure, so it is printed either
+        // way.
+        let ts = std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).unwrap().as_millis() as i64;
+        let preview = http::get_json(
+            &format!(
+                "{base}/_matrix/client/v1/media/preview_url?url={}&ts={ts}",
+                url::form_urlencoded::byte_serialize(b"https://matrix.org/").collect::<String>()
+            ),
+            &token,
+        )
+        .await;
+        match preview {
+            Ok(answer) => println!("preview_url: title {:?}, image {:?}", answer["og:title"], answer["og:image"]),
+            Err(e) => println!("preview_url: unavailable on this server ({e})"),
+        }
+
         let _ = http::post_json(
             &format!("{base}/_matrix/client/v3/rooms/{escaped_room}/leave"),
             Some(&token),

src/backend/matrix/roomstate.rs

@@ -22,7 +22,7 @@
 //! cached: the joined-member roster, power levels (sort key), and presence
 //! (online/offline split) - called any time one of those three changes.
 
-use super::{cached_media_path, moderation, protocol, widgets};
+use super::{cached_media_path, moderation, previews, protocol, widgets};
 use crate::model;
 use crate::state::AppState;
 use serde_json::Value;
@@ -121,6 +121,12 @@ pub async fn process_state_events(state: &AppState, account_id: &str, room_id: &
                     );
                 }
             }
+            // Whether this room lets a link in it be unfurled. A room's own
+            // decision, made for everybody in it, so it outranks an account
+            // that has said nothing - see Runtime::matrix_previews_allowed.
+            previews::ROOM_SETTING => {
+                state.runtime.set_matrix_previews_off(account_id, room_id, previews::disabled_by(&event["content"]));
+            }
             "m.room.avatar" => {
                 if let Some(mxc) = event["content"]["url"].as_str() {
                     if let Some(path) = cached_media_path(homeserver_url, access_token, mxc, "").await {

src/backend/matrix/sync.rs

@@ -153,6 +153,7 @@ pub(super) async fn run_sync(state: &AppState, config: &MatrixAccountConfig, acc
     // this late would leave a conversation filed as an ordinary room for the
     // rest of the session.
     directs::fetch(state, account_id, &config.homeserver_url, &access_token, &config.user_id).await;
+    previews::fetch_account_setting(state, account_id, &config.homeserver_url, &access_token, &config.user_id).await;
 
     if next_batch.is_some() {
         if let Err(e) = bootstrap_joined_rooms(state, account_id, &config.user_id, &config.homeserver_url, &access_token).await {
@@ -512,6 +513,13 @@ pub(super) async fn process_sync_response(state: &AppState, account_id: &str, ow
             if event["type"].as_str() == Some("m.direct") {
                 directs::apply(state, account_id, &event["content"]);
             }
+            // Whether this account wants links unfurled at all. A switch
+            // rather than a preference here, because the unfurling is done by
+            // the homeserver and somebody who turned it off in Element meant
+            // to stop telling it which links they read.
+            if event["type"].as_str() == Some(previews::ACCOUNT_SETTING) {
+                state.runtime.set_matrix_previews_off(account_id, "", previews::disabled_by(&event["content"]));
+            }
             // The account's own sticker pack, which travels with it between
             // clients - see backend/matrix/stickers.rs for the two places a
             // pack lives.

src/backend/matrix/timeline.rs

@@ -353,6 +353,20 @@ pub(super) async fn handle_timeline_event(
             _ => None,
         },
     );
+
+    // What a link in it turns out to be, asked of the homeserver rather than
+    // of the page. After the message is stored, and in the background: the
+    // line is what somebody is waiting for, and a server fetching somebody
+    // else's slow page is not worth holding a conversation up for.
+    previews::unfurl_later(
+        state,
+        account_id,
+        buffer_id,
+        room_id,
+        event_id,
+        &body,
+        event["origin_server_ts"].as_i64().unwrap_or(0),
+    );
 }
 
 /// Applies an incoming `m.room.redaction`: either a real message delete

src/model.rs

@@ -428,6 +428,16 @@ pub struct Embed {
     pub timestamp: Option<String>,
     #[serde(skip_serializing_if = "Option::is_none")]
     pub url: Option<String>,
+    /// The picture on the card, as a local path.
+    ///
+    /// A link preview is mostly its picture, and the one a homeserver hands
+    /// back is an mxc URI needing a token no frontend holds - so it is
+    /// fetched and cached like every other piece of Matrix media, and what
+    /// travels is the file. Discord embeds carry none of these today; the
+    /// field is on the shared shape because a card with a picture is what an
+    /// unfurled link is, whoever unfurled it.
+    #[serde(rename = "imageUrl", skip_serializing_if = "Option::is_none")]
+    pub image_url: Option<String>,
 }
 
 /// Something on a message that can be pressed.

src/runtime.rs

@@ -706,6 +706,13 @@ pub struct Runtime {
     /// "is this room a DM"; the account data is keyed by person, since one
     /// person can have several rooms with you.
     matrix_directs: Mutex<HashMap<(String, String), String>>,
+    /// (account, room) for every place that has turned link previews off; an
+    /// empty room id is the account's own switch.
+    ///
+    /// A set of the refusals rather than a map of every room's answer,
+    /// because the ordinary state is "nobody has said anything" and a map
+    /// would have to hold that answer for every room in order to mean it.
+    matrix_previews_off: Mutex<HashSet<(String, String)>>,
     /// Matrix-specific: (account id, room id) -> {user id -> display name}
     /// for every member currently *joined* to that room (see roomstate.rs's
     /// m.room.member handling - a leave/ban removes the entry entirely,
@@ -1035,6 +1042,7 @@ impl Runtime {
             matrix_pinned: Mutex::new(HashMap::new()),
             matrix_widgets: Mutex::new(HashMap::new()),
             matrix_directs: Mutex::new(HashMap::new()),
+            matrix_previews_off: Mutex::new(HashSet::new()),
             matrix_verification_peers: Mutex::new(HashMap::new()),
             matrix_ignored: Mutex::new(HashMap::new()),
             matrix_room_members: Mutex::new(HashMap::new()),
@@ -1399,6 +1407,30 @@ impl Runtime {
         state.events.emit("bufferListChange", serde_json::to_value(&updated).unwrap());
     }
 
+    /// Whether a room lets a link in it be unfurled.
+    ///
+    /// Two switches, both of which say "disable" rather than "enable" - so an
+    /// account or a room that has never said anything allows them, which is
+    /// what every other client does with the same events. A room that has
+    /// turned them off wins over an account that has not, because that
+    /// decision was made for everybody in the room.
+    pub fn matrix_previews_allowed(&self, account_id: &str, room_id: &str) -> bool {
+        let held = self.matrix_previews_off.lock().unwrap();
+        !held.contains(&(account_id.to_string(), String::new()))
+            && !held.contains(&(account_id.to_string(), room_id.to_string()))
+    }
+
+    /// Records one of those two switches. An empty room id is the account's.
+    pub fn set_matrix_previews_off(&self, account_id: &str, room_id: &str, off: bool) {
+        let key = (account_id.to_string(), room_id.to_string());
+        let mut held = self.matrix_previews_off.lock().unwrap();
+        if off {
+            held.insert(key);
+        } else {
+            held.remove(&key);
+        }
+    }
+
     /// Marks a conversation as the service talking rather than a person.
     ///
     /// One way only. A homeserver does not un-designate its notices room, and
@@ -4661,6 +4693,20 @@ impl Runtime {
                 $( self.$field.lock().unwrap().retain(|key, _| !belongs_to(account_id, &key.0)); )*
             };
         }
+        // A set has no value half, so `retain` hands it one argument rather
+        // than two - which is the whole reason these are separate macros, and
+        // the reason three sets quietly escaped the sweep until the test
+        // below learned to look for them.
+        macro_rules! sweep_set {
+            ($($field:ident),* $(,)?) => {
+                $( self.$field.lock().unwrap().retain(|key| !belongs_to(account_id, key)); )*
+            };
+        }
+        macro_rules! sweep_set_pairs {
+            ($($field:ident),* $(,)?) => {
+                $( self.$field.lock().unwrap().retain(|key| !belongs_to(account_id, &key.0)); )*
+            };
+        }
 
         sweep!(
             conn_states, irc_handles, buffers,
@@ -4695,6 +4741,11 @@ impl Runtime {
             matrix_own_reactions, irc_splits, matrix_widgets,
             matrix_directs,
         );
+        sweep_set!(
+            discord_history_inflight, matrix_encrypted_rooms, matrix_backup_enabled,
+            emoji_unrestricted,
+        );
+        sweep_set_pairs!(matrix_previews_off);
     }
 }
 
@@ -4717,14 +4768,22 @@ mod forget_tests {
             .lines()
             .filter_map(|line| {
                 let line = line.trim_start();
-                let name = line.split(": Mutex<HashMap<").next()?;
-                (line.contains(": Mutex<HashMap<") && !name.contains(' ')).then_some(name)
+                // Sets as well as maps. Three of them escaped this test for
+                // as long as it only looked for HashMap - which is the
+                // failure it exists to catch, happening to itself.
+                let name = line.split(": Mutex<HashMap<").next()?.split(": Mutex<HashSet<").next()?;
+                ((line.contains(": Mutex<HashMap<") || line.contains(": Mutex<HashSet<")) && !name.contains(' '))
+                    .then_some(name)
             })
             .collect();
         assert!(fields.len() > 70, "found only {} maps - the parse is wrong", fields.len());
 
+        // To the end of the function rather than a fixed number of bytes: a
+        // cap is a thing to outgrow silently, and outgrowing this one would
+        // make the test start reporting maps that are in fact swept.
         let sweep = src.find("pub fn forget_account").expect("the sweep");
-        let sweep_body = &src[sweep..sweep + 4000];
+        let end = src[sweep..].find("\n    }\n}").expect("its end") + sweep;
+        let sweep_body = &src[sweep..end];
         let missing: Vec<&&str> = fields.iter().filter(|f| !sweep_body.contains(**f)).collect();
         assert!(missing.is_empty(), "these maps would survive removing an account: {missing:?}");
     }