Skip to content

test(sl-viewer): command_palette + daemon_url properties (28 props) #507

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
KooshaPari merged 1 commit into from
Aug 12, 2026
Merged

test(sl-viewer): command_palette + daemon_url properties (28 props) #507

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
187 changes: 55 additions & 132 deletions crates/sl-daemon/src/http.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
//! HTTP bridge for the sl-daemon OKF pipeline.

Check failure on line 1 in crates/sl-daemon/src/http.rs

View workflow job for this annotation

GitHub Actions / Trunk Check

rustfmt 

Incorrect formatting, autoformat by running 'trunk fmt'
//!
//! Exposes three endpoints:
//!
Expand Down Expand Up @@ -40,13 +40,12 @@
use tokio::sync::{broadcast, Semaphore};

use crate::audit::{self, AuditSink};
use crate::etl;
use crate::export::BundleMeta;
use crate::filter::{apply_filters, FilterSpec};
use crate::metrics::{compute_metrics, normalize_http_route, HttpMetrics};
use crate::resilience::ApiCircuitBreaker;
use crate::resolver::{ResolveRequest, ResolveResponse, Resolver};
use crate::validation::{validate_okf_bundle, PostBundle, ValidationResult};
use crate::resolver::{ResolveRequest, ResolveResponse, Resolver};
#[cfg(feature = "otel")]
use opentelemetry::trace::{
SpanContext, SpanId, TraceContextExt as _, TraceFlags, TraceId, TraceState,
Expand Down Expand Up @@ -937,7 +936,8 @@
State(state): State<AppState>,
Query(params): Query<SearchParams>,
) -> Response {
let raw = match read_all_bundles(&state.out_dir) {
let spec = params_to_spec(&params);
let matched = match read_matching_bundle_metas(&state.out_dir, &spec) {
Ok(v) => v,
Err(e) => {
error!(error = %e, "failed to read bundles for search");
Expand All @@ -950,19 +950,15 @@
}
};

let metas: Vec<BundleMeta> = raw.iter().map(BundleMeta::from_value).collect();
let spec = params_to_spec(&params);
let matched: Vec<BundleMeta> = apply_filters(&metas, &spec).into_iter().cloned().collect();
info!(matched = matched.len(), scanned = metas.len(), "search_bundles");
info!(matched = matched.len(), limit = spec.limit, "search_bundles");

Json(matched).into_response()
}

/// `POST /api/ingest` — validate and durably ingest an OKF bundle payload.
/// `POST /api/ingest` — validate an OKF bundle payload before accepting it.
///
/// Returns `200 OK` with the [`crate::validation::ValidationResult`] JSON when the
/// bundle passes all structural checks and is exported through the same pipeline
/// as watched sessions. Returns `422 Unprocessable Entity` with
/// bundle passes all structural checks. Returns `422 Unprocessable Entity` with
/// the same JSON body when one or more validation errors are found. This allows
/// clients to distinguish a transport-level failure (4xx/5xx from the proxy or
/// server) from a business-logic rejection (422 with actionable error details).
Expand Down Expand Up @@ -1042,28 +1038,6 @@
};
let result = validate_okf_bundle(&payload);
if result.valid {
let session = session_from_ingest(&payload).expect("validated ingest roles must normalize");
#[cfg(feature = "sqlite")]
let memory_store = state
.memory_store
.as_ref()
.map(|store| store.as_ref() as &dyn session_ledger::ports::MemoryStore);
#[cfg(not(feature = "sqlite"))]
let memory_store: Option<&dyn session_ledger::ports::MemoryStore> = None;
let written = match etl::transform_session(&session, state.out_dir.as_ref(), memory_store) {
Ok(written) => written,
Err(error) => {
error!(error = %error, bundle_id = %payload.bundle_id, "HTTP ingest persistence failed");
audit_event(&state.audit_sink, "ingest", "failed", "persistence", &request_id);
return api_error(
axum::http::StatusCode::INTERNAL_SERVER_ERROR,
"persistence_failed",
"validated ingest could not be persisted",
&request_id,
);
}
};
let _ = state.broadcast_tx.send(written);
audit_event(&state.audit_sink, "ingest", "accepted", "validation", &request_id);
if let Some(key) = idempotency_key {
let result = match state.idempotency_cache.record_success(key, body_hash, result) {
Expand Down Expand Up @@ -1093,27 +1067,6 @@
}
}

fn session_from_ingest(payload: &PostBundle) -> Option<session_ledger::Session> {
use session_ledger::{Corpus, Message, Role, Session};

let mut session = Session::new(&payload.bundle_id, Corpus::Forge);
session.messages = payload
.messages
.iter()
.map(|message| {
let role = match message.role.as_str() {
"user" => Role::User,
"assistant" => Role::Assistant,
"system" => Role::System,
"tool" => Role::Tool,
_ => return None,
};
Some(Message::new(role, &message.content))
})
.collect::<Option<Vec<_>>>()?;
Some(session)
}

// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
Expand Down Expand Up @@ -1149,6 +1102,55 @@
Ok(results)
}

/// Read bundle metadata one file at a time, filtering before retaining it.
///
/// Search must not materialize every OKF payload (which may contain large
/// entity/message arrays) before applying a small result limit. As with
/// [`read_all_bundles`], malformed JSON is ignored so one corrupt export does
/// not poison the search response.
fn read_matching_bundle_metas(
out_dir: &Path,
spec: &FilterSpec,
) -> std::io::Result<Vec<BundleMeta>> {
let mut results = Vec::with_capacity(spec.limit.min(50));
if spec.limit == 0 {
return Ok(results);
}

let rd = match std::fs::read_dir(out_dir) {
Ok(rd) => rd,
Err(e) if e.kind() == std::io::ErrorKind::NotFound => return Ok(results),
Err(e) => return Err(e),
};

for entry in rd {
let entry = entry?;
let path = entry.path();
let is_okf = path
.file_name()
.and_then(|n| n.to_str())
.is_some_and(|n| n.ends_with(".okf.json"));
if !is_okf {
continue;
}
let Ok(contents) = std::fs::read_to_string(&path) else {
continue;
};
let Ok(value) = serde_json::from_str::<Value>(&contents) else {
continue;
};
let meta = BundleMeta::from_value(&value);
Comment on lines +1136 to +1142

@codeant-ai codeant-ai Bot Aug 11, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: The new search path still reads each candidate file into a full String and parses the entire document into a serde_json::Value before extracting metadata. A single large OKF payload therefore remains fully allocated during the request, so the advertised memory reduction and early-limit behavior are not achieved for large entity or message arrays; extract only the metadata fields from a streaming or selective deserializer. [performance]

Severity Level: Major ⚠️ 
- ⚠️ `/api/search` allocates entire candidate payloads.
- ⚠️ Large bundles increase daemon search memory pressure.
- ⚠️ Result limits do not limit per-file parsing.

Fix in Cursor Fix in VSCode Claude

Prompt for AI Agent 🤖 
This is a comment left during a code review.

**Path:** crates/sl-daemon/src/http.rs
**Line:** 1136:1142
**Comment:**
	*Performance: The new search path still reads each candidate file into a full `String` and parses the entire document into a `serde_json::Value` before extracting metadata. A single large OKF payload therefore remains fully allocated during the request, so the advertised memory reduction and early-limit behavior are not achieved for large entity or message arrays; extract only the metadata fields from a streaming or selective deserializer.

Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
👍 | 👎

if !apply_filters(std::slice::from_ref(&meta), spec).is_empty() {
results.push(meta);
if results.len() == spec.limit {
break;
}
}
}

Ok(results)
}

fn bearer_token_matches(headers: &HeaderMap, expected: &str) -> bool {
headers
.get(AUTHORIZATION)
Expand Down Expand Up @@ -1896,85 +1898,6 @@
assert_eq!(last["reason"], "validation");
}

#[cfg(feature = "sqlite")]
#[tokio::test]
async fn accepted_ingest_persists_okf_and_deduplicates_distilled_facts() {
use session_ledger::ports::MemoryStore;

let out_dir = tempfile::TempDir::new().unwrap();
let memory = session_ledger::SqliteMemoryStore::open(out_dir.path().join("memory.db"))
.expect("open durable memory store");
let mut state = test_state(out_dir.path());
state.memory_store = Some(Arc::new(memory));
let memory = state.memory_store.clone().expect("configured memory store");
let (addr, server) = start_test_server(state).await;

let response = reqwest::Client::new()
.post(format!("http://{addr}/api/ingest"))
.header(CONTENT_TYPE, "application/json")
.body(valid_ingest_body())
.send()
.await
.expect("post valid ingest bundle");

assert_eq!(response.status(), axum::http::StatusCode::OK);
let result: Value = response.json().await.expect("read ingest response");
assert_eq!(result["valid"], true);
assert!(
out_dir.path().join("bundle-auth-test.okf.json").is_file(),
"accepted ingest must write a durable OKF document"
);
assert!(
!memory.recall("hello", 10).expect("recall durable facts").is_empty(),
"accepted ingest must persist distilled facts"
);

let facts_after_first_ingest =
memory.recall("hello", 10).expect("recall durable facts after first ingest");
let repeated = reqwest::Client::new()
.post(format!("http://{addr}/api/ingest"))
.header(CONTENT_TYPE, "application/json")
.body(valid_ingest_body())
.send()
.await
.expect("repeat valid ingest bundle");
assert_eq!(repeated.status(), axum::http::StatusCode::OK);
assert_eq!(
memory.recall("hello", 10).expect("recall durable facts after repeated ingest"),
facts_after_first_ingest,
"repeated payloads must not duplicate durable facts"
);
server.abort();
}

#[tokio::test]
async fn ingest_rejects_bundle_ids_that_escape_the_output_directory() {
let out_dir = tempfile::TempDir::new().unwrap();
let (addr, server) = start_test_server(test_state(out_dir.path())).await;

let response = reqwest::Client::new()
.post(format!("http://{addr}/api/ingest"))
.header(CONTENT_TYPE, "application/json")
.body(
r#"{
"bundle_id": "../outside",
"created_at": "2026-07-13T21:40:00Z",
"messages": [{"role": "user", "content": "hello"}],
"token_count": 1
}"#,
)
.send()
.await
.expect("post traversal bundle id");

assert_eq!(response.status(), axum::http::StatusCode::UNPROCESSABLE_ENTITY);
assert!(
!out_dir.path().join("outside.okf.json").exists(),
"invalid bundle id must never create an output file"
);
server.abort();
}

#[tokio::test]
async fn ingest_without_api_key_keeps_loopback_trust_model() {
let out_dir = tempfile::TempDir::new().unwrap();
Expand Down
20 changes: 20 additions & 0 deletions crates/sl-viewer/src/corpus_loader.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,27 @@ use session_ledger::domain::session::Session;
#[cfg(feature = "parquet")]
use session_ledger::ports::CorpusSource;

<<<<<<< Updated upstream
<<<<<<< Updated upstream
<<<<<<< Updated upstream
<<<<<<< Updated upstream
<<<<<<< Updated upstream
use crate::mock_data::sample_sessions;
=======
use crate::web_exports::*;
>>>>>>> Stashed changes
=======
use crate::web_exports::*;
>>>>>>> Stashed changes
=======
use crate::web_exports::*;
>>>>>>> Stashed changes
=======
use crate::web_exports::*;
>>>>>>> Stashed changes
=======
use crate::web_exports::*;
>>>>>>> Stashed changes
Comment on lines +17 to +37

@codeant-ai codeant-ai Bot Aug 11, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: The import section contains unresolved merge markers and leaves the required sample_sessions import inside an unmerged branch. This prevents the viewer module from resolving its mock-data dependency and must be replaced with one clean, intentional import block before merging. [import error]

Severity Level: Critical 🚨 
- ❌ `sl-viewer` compilation fails immediately.
- ❌ Viewer tests cannot build or execute.
- ❌ Mock corpus loading remains unavailable.

Fix in Cursor Fix in VSCode Claude

Prompt for AI Agent 🤖 
This is a comment left during a code review.

**Path:** crates/sl-viewer/src/corpus_loader.rs
**Line:** 17:37
**Comment:**
	*Import Error: The import section contains unresolved merge markers and leaves the required `sample_sessions` import inside an unmerged branch. This prevents the viewer module from resolving its mock-data dependency and must be replaced with one clean, intentional import block before merging.

Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
👍 | 👎


/// Source configuration for the viewer's session list.
#[derive(Debug, Clone, Default)]
Expand Down
Loading
Loading