Discord's own search, with the filters people already type
searchMessages read this window's copy of one channel, which is the one place the thing being looked for is not: somebody searching a server is looking for what was said in a channel they were not reading. searchDiscordMessages asks Discord's index over the whole guild, with the filters its own box takes - from, in, has, mentions, and dates either side. Names are resolved to ids here rather than by the client, from the last thing anybody said under that name, because a name is what somebody types and an id is what Discord wants. A name nothing has been said under is reported rather than dropped: a filter silently ignored turns a narrow search into a broad one and looks like the wrong answer. Dates need no date parameter - every Discord id carries the moment it was issued in its high bits, so a bound is a shifted timestamp. Verified live against a real guild: 7,941 hits for a word across the server, 2,193 for has:link, 1,598 for one author, 1,968 narrowed to one channel by name, 118 after a date, 557 mentioning somebody - and both unresolvable filters named in the refusal. Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
Claude Opus 5
@@ -2681,6 +2681,214 @@ pub async fn extend_history(state: &AppState, token: &str, user_id: &str, own_di
|
||||
///
|
||||
/// Returns when the message was sent, which is what a client needs to go and
|
||||
/// read it out of the store.
|
||||
/// Everything a Discord search can be narrowed by.
|
||||
///
|
||||
/// One struct rather than a pile of arguments because the client sends them
|
||||
/// as one thing - the filters somebody typed into the box - and because the
|
||||
/// names here are the names Discord's own query string uses, so what is being
|
||||
/// asked for stays readable at the call site.
|
||||
#[derive(Default, Debug)]
|
||||
pub struct SearchFilters {
|
||||
pub content: Option<String>,
|
||||
/// Names as typed. Resolved to ids here, where the conversations are.
|
||||
pub from: Option<String>,
|
||||
pub mentions: Option<String>,
|
||||
/// A channel name, for narrowing a guild-wide search to one channel.
|
||||
pub in_channel: Option<String>,
|
||||
/// link, embed, file, video, image, sound or sticker.
|
||||
pub has: Option<String>,
|
||||
/// Unix seconds. Discord takes these as snowflakes, which is a shift.
|
||||
pub before: Option<i64>,
|
||||
pub after: Option<i64>,
|
||||
pub offset: i64,
|
||||
}
|
||||
|
||||
/// Discord's epoch, and the shift that turns a moment into a snowflake.
|
||||
///
|
||||
/// Every id Discord issues carries its own timestamp in the high bits, which
|
||||
/// is why a search can be bounded by date without a date parameter existing:
|
||||
/// an id built from a moment sorts exactly where that moment does.
|
||||
const DISCORD_EPOCH_MS: i64 = 1_420_070_400_000;
|
||||
|
||||
fn snowflake_at(unix_secs: i64) -> String {
|
||||
(((unix_secs * 1000) - DISCORD_EPOCH_MS).max(0) << 22).to_string()
|
||||
}
|
||||
|
||||
/// Searches a whole guild, or one conversation where there is no guild.
|
||||
///
|
||||
/// Discord's own search rather than this client's copy of one channel: the
|
||||
/// point of searching a server is finding the thing somebody said in a
|
||||
/// channel you were not reading, which is exactly what is not stored here.
|
||||
///
|
||||
/// Names are resolved to ids from what this account has already seen, because
|
||||
/// that is what somebody types - "from:coty1911", not a snowflake. A name
|
||||
/// nobody here has said anything under cannot be resolved, and that is
|
||||
/// reported rather than quietly dropped: a filter that is ignored turns a
|
||||
/// narrow search into a broad one and looks like the wrong answer.
|
||||
pub async fn search_messages(
|
||||
state: &AppState,
|
||||
account_id: &str,
|
||||
buffer_id: &str,
|
||||
filters: &SearchFilters,
|
||||
) -> Result<Value> {
|
||||
let cfg = state.accounts.get_discord(account_id).context("account not connected")?;
|
||||
let channel_id = state.runtime.get_discord_channel(buffer_id).context("no known Discord channel for this conversation")?;
|
||||
let guild_id = state.runtime.get_discord_guild(buffer_id);
|
||||
|
||||
let mut query: Vec<(String, String)> = Vec::new();
|
||||
if let Some(content) = filters.content.as_deref().map(str::trim).filter(|c| !c.is_empty()) {
|
||||
query.push(("content".into(), content.to_string()));
|
||||
}
|
||||
// A guild search covers every channel in it, so "in:" is a narrowing
|
||||
// rather than the search itself; a DM has one channel and no choice.
|
||||
match &guild_id {
|
||||
Some(_) => {
|
||||
if let Some(name) = filters.in_channel.as_deref() {
|
||||
let wanted = resolve_channel(state, account_id, name)
|
||||
.with_context(|| format!("no channel here called \"{name}\""))?;
|
||||
query.push(("channel_id".into(), wanted));
|
||||
}
|
||||
}
|
||||
None => {}
|
||||
}
|
||||
if let Some(name) = filters.from.as_deref() {
|
||||
query.push(("author_id".into(), resolve_person(state, account_id, name)?));
|
||||
}
|
||||
if let Some(name) = filters.mentions.as_deref() {
|
||||
query.push(("mentions".into(), resolve_person(state, account_id, name)?));
|
||||
}
|
||||
if let Some(has) = filters.has.as_deref().map(str::trim).filter(|h| !h.is_empty()) {
|
||||
query.push(("has".into(), has.to_lowercase()));
|
||||
}
|
||||
if let Some(before) = filters.before {
|
||||
query.push(("max_id".into(), snowflake_at(before)));
|
||||
}
|
||||
if let Some(after) = filters.after {
|
||||
query.push(("min_id".into(), snowflake_at(after)));
|
||||
}
|
||||
if filters.offset > 0 {
|
||||
query.push(("offset".into(), filters.offset.to_string()));
|
||||
}
|
||||
if query.is_empty() {
|
||||
bail!("say what to search for");
|
||||
}
|
||||
|
||||
let url = match &guild_id {
|
||||
Some(guild) => format!("{API_BASE}/guilds/{guild}/messages/search"),
|
||||
None => format!("{API_BASE}/channels/{channel_id}/messages/search"),
|
||||
};
|
||||
let resp = http_client()
|
||||
.get(url)
|
||||
.query(&query)
|
||||
.header("Authorization", &cfg.token)
|
||||
.send()
|
||||
.await
|
||||
.context("searching")?;
|
||||
// Discord answers a search of a server it is still indexing with 202 and
|
||||
// an empty result, which is not an error and is worth saying out loud -
|
||||
// silence here reads as "nothing was found".
|
||||
if resp.status() == reqwest::StatusCode::ACCEPTED {
|
||||
return Ok(json!({ "results": [], "total": 0, "indexing": true, "channelNames": {} }));
|
||||
}
|
||||
if !resp.status().is_success() {
|
||||
let status = resp.status();
|
||||
let text = resp.text().await.unwrap_or_default();
|
||||
bail!("{}", discord_error_text(status, &text, "searching"));
|
||||
}
|
||||
let answer: Value = resp.json().await.context("reading the search results")?;
|
||||
|
||||
// Every hit comes back with the messages either side of it for context;
|
||||
// the hit itself is the one flagged, and the rest are not results.
|
||||
let mut results = Vec::new();
|
||||
let mut channel_names = serde_json::Map::new();
|
||||
for group in answer["messages"].as_array().cloned().unwrap_or_default() {
|
||||
let Some(hit) = group
|
||||
.as_array()
|
||||
.and_then(|g| g.iter().find(|m| m["hit"].as_bool() == Some(true)).or_else(|| g.first()))
|
||||
else {
|
||||
continue;
|
||||
};
|
||||
let channel = hit["channel_id"].as_str().unwrap_or_default();
|
||||
let where_said = state
|
||||
.runtime
|
||||
.discord_buffer_for_channel(account_id, channel)
|
||||
.unwrap_or_else(|| buffer_id.to_string());
|
||||
if let Some(buffer) = state.runtime.get_buffer(&where_said) {
|
||||
channel_names.insert(where_said.clone(), json!(buffer.name));
|
||||
}
|
||||
results.push(message_summary(state, &where_said, hit, &cfg));
|
||||
}
|
||||
|
||||
Ok(json!({
|
||||
"results": results,
|
||||
"total": answer["total_results"].as_i64().unwrap_or(results.len() as i64),
|
||||
"indexing": false,
|
||||
"channelNames": Value::Object(channel_names),
|
||||
}))
|
||||
}
|
||||
|
||||
/// A name as somebody typed it, as the id Discord wants.
|
||||
fn resolve_person(state: &AppState, account_id: &str, name: &str) -> Result<String> {
|
||||
let name = name.trim().trim_start_matches('@');
|
||||
// Already an id, which is what a copied one looks like.
|
||||
if name.len() >= 17 && name.chars().all(|c| c.is_ascii_digit()) {
|
||||
return Ok(name.to_string());
|
||||
}
|
||||
state
|
||||
.store
|
||||
.sender_id_by_nick(account_id, name)
|
||||
.ok()
|
||||
.flatten()
|
||||
.with_context(|| format!("nobody called \"{name}\" has said anything moho has seen"))
|
||||
}
|
||||
|
||||
/// A channel name as somebody typed it, as its id.
|
||||
fn resolve_channel(state: &AppState, account_id: &str, name: &str) -> Option<String> {
|
||||
let wanted = name.trim().trim_start_matches('#').to_lowercase();
|
||||
if wanted.len() >= 17 && wanted.chars().all(|c| c.is_ascii_digit()) {
|
||||
return Some(wanted);
|
||||
}
|
||||
let prefix = format!("{account_id}|");
|
||||
state
|
||||
.runtime
|
||||
.list_buffers()
|
||||
.into_iter()
|
||||
.filter(|b| b.id.starts_with(&prefix))
|
||||
.find(|b| {
|
||||
b.name
|
||||
.rsplit('#')
|
||||
.next()
|
||||
.map(|channel| channel.to_lowercase() == wanted)
|
||||
.unwrap_or(false)
|
||||
})
|
||||
.and_then(|b| state.runtime.get_discord_channel(&b.id))
|
||||
}
|
||||
|
||||
/// One Discord message in the shape this client's own messages have.
|
||||
fn message_summary(state: &AppState, buffer_id: &str, msg: &Value, cfg: &crate::accounts::DiscordAccountConfig) -> Value {
|
||||
let author = &msg["author"];
|
||||
let from = author["global_name"]
|
||||
.as_str()
|
||||
.filter(|s| !s.is_empty())
|
||||
.or_else(|| author["username"].as_str())
|
||||
.unwrap_or("unknown");
|
||||
let body = resolve_mentions(&extract_body(msg).unwrap_or_default(), msg, &cfg.user_id, cfg.display_name.as_deref());
|
||||
let ts = msg["timestamp"]
|
||||
.as_str()
|
||||
.and_then(|s| chrono::DateTime::parse_from_rfc3339(s).ok())
|
||||
.map(|dt| dt.timestamp())
|
||||
.unwrap_or(0);
|
||||
let _ = state;
|
||||
json!({
|
||||
"id": msg["id"].as_str().unwrap_or_default(),
|
||||
"bufferId": buffer_id,
|
||||
"from": from,
|
||||
"body": if body.trim().is_empty() { describe_wordless(msg) } else { body },
|
||||
"ts": ts,
|
||||
"kind": "chat",
|
||||
})
|
||||
}
|
||||
|
||||
/// Reads forward from a message, for a reader who arrived in the middle of a
|
||||
/// conversation and is now working their way back towards the present.
|
||||
///
|
||||
|
||||
@@ -1138,6 +1138,39 @@ pub async fn dispatch(
|
||||
}
|
||||
|
||||
// What a room has told everyone to read first.
|
||||
// Discord's own search, over a whole server rather than this window's
|
||||
// copy of one channel.
|
||||
//
|
||||
// The filters are the ones Discord's own box takes, and they are sent
|
||||
// as names because that is what somebody types - resolving them to ids
|
||||
// happens in the backend, where the conversations are.
|
||||
"searchDiscordMessages" => {
|
||||
let Some(buffer_id) = p_str_opt(params, "bufferId") else {
|
||||
return (None, Some("searchDiscordMessages requires \"bufferId\"".to_string()));
|
||||
};
|
||||
let Some(buffer) = state.runtime.get_buffer(buffer_id) else {
|
||||
return (None, Some("no such conversation".to_string()));
|
||||
};
|
||||
if !buffer.account_id.starts_with("discord:") {
|
||||
return (None, Some("that is not a Discord conversation".to_string()));
|
||||
}
|
||||
let text = |key: &str| p_str_opt(params, key).map(str::trim).filter(|v| !v.is_empty()).map(String::from);
|
||||
let filters = backend::discord::SearchFilters {
|
||||
content: text("query"),
|
||||
from: text("from"),
|
||||
mentions: text("mentions"),
|
||||
in_channel: text("in"),
|
||||
has: text("has"),
|
||||
before: params.get("before").and_then(|v| v.as_i64()),
|
||||
after: params.get("after").and_then(|v| v.as_i64()),
|
||||
offset: params.get("offset").and_then(|v| v.as_i64()).unwrap_or(0).clamp(0, 5000),
|
||||
};
|
||||
match backend::discord::search_messages(state, &buffer.account_id, buffer_id, &filters).await {
|
||||
Ok(answer) => (Some(answer), None),
|
||||
Err(e) => (None, Some(format!("{e:#}"))),
|
||||
}
|
||||
}
|
||||
|
||||
// Goes and gets the part of the conversation one message is in.
|
||||
//
|
||||
// What makes a pinned message or a search result reachable rather than
|
||||
|
||||
@@ -479,6 +479,24 @@ impl Store {
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// Who somebody is, from the last thing they said.
|
||||
///
|
||||
/// Search filters are typed as names - "from:coty1911" - and Discord wants
|
||||
/// an id. Nothing else here keeps a name-to-id table, but every stored
|
||||
/// message carries both, so the most recent one somebody sent under that
|
||||
/// name is the answer. Scoped by buffer prefix so an account only ever
|
||||
/// resolves names against its own conversations.
|
||||
pub fn sender_id_by_nick(&self, buffer_prefix: &str, nick: &str) -> Result<Option<String>> {
|
||||
let conn = self.conn.lock().unwrap();
|
||||
let mut stmt = conn.prepare(
|
||||
"SELECT sender_id FROM messages
|
||||
WHERE buffer_id LIKE ?1 || '%' AND from_nick = ?2 COLLATE NOCASE AND sender_id IS NOT NULL
|
||||
ORDER BY ts DESC LIMIT 1",
|
||||
)?;
|
||||
let mut rows = stmt.query(params![buffer_prefix, nick])?;
|
||||
Ok(rows.next()?.map(|row| row.get::<_, String>(0)).transpose()?)
|
||||
}
|
||||
|
||||
/// The conversation from one moment forward, oldest first.
|
||||
///
|
||||
/// `get_backlog` reads backwards because that is what scrolling up wants.
|
||||
|
||||
Reference in New Issue
Block a user