feat(discovery): detect llama.cpp servers and label local providers (#4729)

* feat(discovery): detect llama.cpp servers and label local providers

Scan port 8080 (llama-server) and 11435 (APFEL) during discovery, fingerprint
llama.cpp via its native /props endpoint, and label well-known local serving
ports (8080 llama.cpp, 8000 vLLM, 1234 LM Studio, 11434 Ollama) consistently
in both the Python provider helper and the JS endpoint UI. Adds a llama.cpp
hint to the /setup slash command.

* fix(discovery): don't infer the serving tool from the port alone

Per review: vLLM, SGLang, llama.cpp and plain OpenAI-compatible servers all
share 8000/8080, so labeling by port mislabels real setups (a vLLM box on 8080
shown as llama.cpp). Drop the port->tool assertions from _provider_label and
providerLabel; the authoritative signal is the /props fingerprint done during
discovery, which is unchanged. Loopback now reads a neutral 'local endpoint' /
'Local'. Tests updated to assert the neutral labels.
This commit is contained in:
Joel Alejandro Escareño Fernández
2026-06-23 23:39:56 +02:00
committed by GitHub
parent 72c0bde8a9
commit e0ccf250a4
9 changed files with 330 additions and 15 deletions
+13 -4
View File
@@ -93,10 +93,19 @@ class TestProviderLabel:
def test_known_labels(self, url, expected):
assert _provider_label(url) == expected
def test_local_non_ollama_endpoint(self):
# A loopback host that isn't on the native Ollama /api path is just a
# generic local endpoint (e.g. an OpenAI-compatible local server).
assert _provider_label("http://localhost:8080/v1") == "local endpoint"
@pytest.mark.parametrize("url", [
"http://localhost:8080/v1",
"http://127.0.0.1:8080/v1",
"http://localhost:8000/v1",
"http://localhost:1234/v1",
"http://localhost:9999/v1",
])
def test_local_non_ollama_endpoint(self, url):
# The serving tool is NOT inferred from the port: vLLM, SGLang, llama.cpp
# and plain OpenAI-compatible servers all share 8000/8080, so a port-only
# label would mislabel real setups. The tool is identified by /props
# fingerprinting during discovery; this helper stays neutral.
assert _provider_label(url) == "local endpoint"
def test_unknown_host_returns_host(self):
assert _provider_label("https://api.unknown-llm.example/v1") == "api.unknown-llm.example"