Testing in Spacedrive Core ensures reliability across single-device operations and multi-device networking scenarios. This guide covers the available frameworks, patterns, and best practices.
Testing Infrastructure
Spacedrive Core provides two primary testing approaches:
- Standard Tests - For unit and single-core integration testing
- Subprocess Framework - For multi-device networking and distributed scenarios
Test Organization
Tests live in two locations:
core/tests/ - Integration tests that verify complete workflows
core/src/testing/ - Test framework utilities and helpers
Standard Testing
For single-device tests, use Tokio’s async test framework:
#[tokio::test]
async fn test_library_creation() {
let setup = IntegrationTestSetup::new("library_test").await.unwrap();
let core = setup.create_core().await.unwrap();
let library = core.libraries
.create_library("Test Library", None)
.await
.unwrap();
assert!(!library.id.is_empty());
}
Integration Test Setup
The IntegrationTestSetup utility provides isolated test environments:
// Basic setup
let setup = IntegrationTestSetup::new("test_name").await?;
// Custom configuration
let setup = IntegrationTestSetup::with_config("test_name", |builder| {
builder
.log_level("debug")
.networking_enabled(true)
.volume_monitoring_enabled(false)
}).await?;
Key features:
- Isolated temporary directories per test
- Structured logging to
test_data/{test_name}/library/logs/
- Automatic cleanup on drop
- Configurable app settings
Multi-Device Testing
Spacedrive provides two approaches for testing multi-device scenarios:
When to Use Subprocess Framework
Use CargoTestRunner subprocess framework when:
- Testing real networking with actual network discovery, NAT traversal, and connections
- Testing device pairing workflows that require independent network stacks
- Scenarios need true process isolation (separate memory spaces, different ports)
- You want to test network reconnection, timeout, and failure handling
- Testing cross-platform network behavior
Examples: Device pairing, network discovery, connection management
// Uses real networking, separate processes
let mut runner = CargoTestRunner::new()
.add_subprocess("alice", "alice_pairing_scenario")
.add_subprocess("bob", "bob_pairing_scenario");
When to Use Custom Transport/Harness
Use custom harness with mock transport when:
- Testing sync logic without network overhead
- Fast iteration on data synchronization algorithms
- Testing deterministic scenarios without network timing issues
- Verifying database state and conflict resolution
- Need precise control over sync event ordering
Examples: Real-time sync, backfill, content identity linking, conflict resolution
// Uses mock transport, single process, fast and deterministic
let harness = TwoDeviceHarnessBuilder::new("sync_test")
.collect_events(true)
.build()
.await?;
Comparison
| Aspect | Subprocess Framework | Custom Harness |
|---|
| Speed | Slower (real networking) | Fast (in-memory) |
| Networking | Real (discovery, NAT) | Mock transport |
| Isolation | True process isolation | Shared process |
| Debugging | Harder (multiple processes) | Easier (single process) |
| Determinism | Network timing varies | Fully deterministic |
| Use Case | Network features | Sync/data logic |
Subprocess Testing Framework
The subprocess framework spawns separate cargo test processes for each device role:
let mut runner = CargoTestRunner::new()
.with_timeout(Duration::from_secs(90))
.add_subprocess("alice", "alice_scenario")
.add_subprocess("bob", "bob_scenario");
runner.run_until_success(|outputs| {
outputs.values().all(|output| output.contains("SUCCESS"))
}).await?;
Writing Multi-Device Tests
Create separate test functions for each device role:
#[tokio::test]
async fn test_device_pairing() {
let mut runner = CargoTestRunner::new()
.add_subprocess("alice", "alice_pairing")
.add_subprocess("bob", "bob_pairing");
runner.run_until_success(|outputs| {
outputs.values().all(|o| o.contains("PAIRING_SUCCESS"))
}).await.unwrap();
}
#[tokio::test]
#[ignore]
async fn alice_pairing() {
if env::var("TEST_ROLE").unwrap_or_default() != "alice" {
return;
}
let data_dir = PathBuf::from(env::var("TEST_DATA_DIR").unwrap());
let core = create_test_core(data_dir).await.unwrap();
// Alice initiates pairing
let (code, _) = core.start_pairing_as_initiator().await.unwrap();
fs::write("/tmp/pairing_code.txt", &code).unwrap();
// Wait for connection
wait_for_connection(&core).await;
println!("PAIRING_SUCCESS");
}
Device scenario functions must be marked with #[ignore] to prevent direct execution. They only run when called by the subprocess framework.
Process Coordination
Processes coordinate through:
- Environment variables:
TEST_ROLE and TEST_DATA_DIR
- Temporary files: Share data like pairing codes
- Output patterns: Success markers for the runner to detect
Common Test Patterns
Filesystem Watcher Testing
When testing filesystem watcher functionality, several critical setup steps are required:
Enable Watcher in Test Config
The default TestConfigBuilder disables the filesystem watcher (for performance in sync tests). Tests that verify watcher events must explicitly enable it:
let mut config = TestConfigBuilder::new(test_root.clone())
.build()?;
// CRITICAL: Enable watcher for change detection tests
config.services.fs_watcher_enabled = true;
config.save()?;
let core = Core::new(config.data_dir.clone()).await?;
Use Home Directory Paths on macOS
macOS temp directories (/var/folders/...) don’t reliably deliver filesystem events. Use home directory paths instead:
// ❌ Don't use TempDir for watcher tests
let temp_dir = TempDir::new()?;
// ✅ Use home directory
let home = std::env::var("HOME").unwrap_or_else(|_| "/tmp".to_string());
let test_root = PathBuf::from(home).join(".spacedrive_test_my_test");
// Clean up before
let _ = tokio::fs::remove_dir_all(&test_root).await;
tokio::fs::create_dir_all(&test_root).await?;
// ... run test ...
// Clean up after
tokio::fs::remove_dir_all(&test_root).await?;
Ephemeral Watching Requirements
Ephemeral paths must be indexed before watching:
// 1. Index the directory (ephemeral mode)
let config = IndexerJobConfig::ephemeral_browse(
SdPath::local(dest_dir.clone()),
IndexScope::Current
);
let job = IndexerJob::new(config);
library.jobs().dispatch(job).await?.wait().await?;
// 2. Mark indexing complete (indexer job does this automatically)
context.ephemeral_cache().mark_indexing_complete(&dest_dir);
// 3. Register for watching (indexer job does this automatically)
watcher.watch_ephemeral(dest_dir.clone()).await?;
// Now filesystem events will be detected
The IndexerJob automatically calls watch_ephemeral() after successful indexing, so manual registration is only needed when bypassing the indexer.
Persistent Location Watching
For persistent locations, the watcher auto-loads locations at startup. New locations created during tests must be manually registered:
// After creating and indexing a location
let location_meta = LocationMeta {
id: location_uuid,
library_id: library.id(),
root_path: location_path.clone(),
rule_toggles: RuleToggles::default(),
};
watcher.watch_location(location_meta).await?;
The IndexingHarness handles this automatically.
Event Collection Best Practices
Start collecting events after initialization to avoid library statistics noise:
// Complete all setup first
let harness = IndexingHarnessBuilder::new("test").build().await?;
let location = harness.add_and_index_location(...).await?;
// Wait for setup to settle
tokio::time::sleep(Duration::from_millis(500)).await;
// Start collecting BEFORE the operation you're testing
let mut collector = EventCollector::new(&harness.core.events);
let handle = tokio::spawn(async move {
collector.collect_events(Duration::from_secs(5)).await;
collector
});
// Perform operation
perform_copy_operation().await?;
// Collect and verify
let collector = handle.await.unwrap();
let stats = collector.analyze().await;
assert!(stats.resource_changed.get("file").copied().unwrap_or(0) >= 2);
The EventCollector automatically filters out:
- Library statistics updates (
LibraryStatisticsUpdated)
- Library resource events (non-file/entry events)
Expected Event Types
Different handlers emit different event types:
- Ephemeral handler: Individual
ResourceChanged events per file (CREATE + MODIFY)
- Persistent handler: Batched
ResourceChangedBatch events
// Ephemeral assertion
let file_events = stats.resource_changed.get("file").copied().unwrap_or(0);
assert!(file_events >= 2, "Expected file ResourceChanged events");
// Persistent assertion
let batch_count = stats.resource_changed_batch.get("file").copied().unwrap_or(0);
assert!(batch_count >= 2, "Expected file ResourceChangedBatch events");
Event Monitoring
Waiting for Specific Events
Wait for specific Core events with timeouts:
let mut events = core.events.subscribe();
let event = wait_for_event(
&mut events,
|e| matches!(e, Event::JobCompleted { .. }),
Duration::from_secs(30)
).await?;
Collecting All Events for Analysis
For tests that need to verify event emission patterns (e.g., ResourceChanged events during operations), use the shared EventCollector helper:
use helpers::EventCollector;
// Create collector with full event capture for debugging
let mut collector = EventCollector::with_capture(&harness.core.events);
// Spawn collection task
let collection_handle = tokio::spawn(async move {
collector.collect_events(Duration::from_secs(10)).await;
collector
});
// Perform operations that emit events
perform_copy_operation().await?;
location.reindex().await?;
// Retrieve collector and analyze
let collector = collection_handle.await.unwrap();
// Print statistics summary
let stats = collector.analyze().await;
stats.print();
// Print full event details for debugging (when using with_capture)
collector.print_events().await;
// Write events to JSON file for later inspection
collector.write_to_file(&snapshot_dir.join("events.json")).await?;
// Filter specific events
let file_events = collector.get_resource_batch_events("file").await;
let indexing_events = collector.get_events_by_type("IndexingCompleted").await;
The EventCollector tracks:
- ResourceChanged/ResourceChangedBatch events by resource type
- Indexing start/completion events
- Job lifecycle events (started/completed)
- Entry events (created/modified/deleted/moved)
Statistics Output:
Event Statistics:
==================
ResourceChangedBatch events:
file → 45 resources
Indexing events:
Started: 1
Completed: 1
Entry events:
Created: 3
Modified: 0
Job events:
Started:
indexer → 1
Completed:
indexer → 1
Detailed Event Output (with with_capture()):
=== Collected Events (8) ===
[1] IndexingStarted
Location: 550e8400-e29b-41d4-a716-446655440000
[2] JobStarted
Job: indexer (job_123)
[3] ResourceChangedBatch
Type: file
Resources: 45 items
Paths: 1 affected
[4] IndexingCompleted
Location: 550e8400-e29b-41d4-a716-446655440000
Files: 42, Dirs: 3
[5] JobCompleted
Job: indexer (job_123)
Output: Success
Use Cases:
- Verifying watcher events during file operations
- Testing normalized cache updates
- Debugging event emission patterns
- Creating test fixtures with real event data
- Inspecting actual resource payloads in events
Database Verification
Query the database directly to verify state:
use sd_core::entities;
let entries = entities::entry::Entity::find()
.filter(entities::entry::Column::Name.contains("test"))
.all(db.conn())
.await?;
assert_eq!(entries.len(), expected_count);
Job Testing
Test job execution and resumption:
// Start a job
let job_id = core.jobs.dispatch(IndexingJob::new(...)).await?;
// Monitor progress
wait_for_event(&mut events, |e| matches!(
e,
Event::JobProgress { id, .. } if *id == job_id
), timeout).await?;
// Verify completion
let job = core.jobs.get_job(job_id).await?;
assert_eq!(job.status, JobStatus::Completed);
Mock Transport for Sync Testing
Test synchronization without real networking:
let transport = Arc::new(Mutex::new(Vec::new()));
let mut core_a = create_test_core().await?;
let mut core_b = create_test_core().await?;
// Connect cores with mock transport
connect_with_mock_transport(&mut core_a, &mut core_b, transport).await?;
// Verify sync
perform_operation_on_a(&core_a).await?;
wait_for_sync(&core_b).await?;
Test Helpers
Common Utilities
The framework provides comprehensive test helpers in core/tests/helpers/:
Event Collection:
EventCollector - Collect and analyze all events from the event bus
EventStats - Statistics about collected events with formatted output
Indexing Tests:
IndexingHarnessBuilder - Create isolated test environments with indexing support
TestLocation - Builder for test locations with files
LocationHandle - Handle to indexed locations with verification methods
Sync Tests:
TwoDeviceHarnessBuilder - Pre-configured two-device sync test environments
MockTransport - Mock network transport for deterministic sync testing
wait_for_sync() - Sophisticated sync completion detection
TestConfigBuilder - Custom test configurations
Database & Jobs:
wait_for_event() - Wait for specific events with timeout
wait_for_indexing() - Wait for indexing job completion
register_device() - Register a device in a library
See core/tests/helpers/README.md for detailed documentation on all available helpers including usage examples and migration guides.
Test Volumes
For volume-related tests, use the test volume utilities:
use helpers::test_volumes;
let volume = test_volumes::create_test_volume().await?;
// Test volume operations
test_volumes::cleanup_test_volume(volume).await?;
Running Tests
All Tests
Specific Test
cargo test test_device_pairing --nocapture
Debug Subprocess Tests
# Run individual scenario
TEST_ROLE=alice TEST_DATA_DIR=/tmp/test cargo test alice_scenario -- --ignored --nocapture
With Logging
RUST_LOG=debug cargo test test_name --nocapture
Best Practices
Test Structure
- Use descriptive names:
test_cross_device_file_transfer over test_transfer
- One concern per test: Focus on a single feature or workflow
- Clean up resources: Use RAII patterns or explicit cleanup
Subprocess Tests
- Always use
#[ignore] on scenario functions
- Check TEST_ROLE early: Return immediately if role doesn’t match
- Use clear success patterns: Print distinct markers for the runner
- Set appropriate timeouts: Balance between test speed and reliability
Debugging
When tests fail, check the logs in test_data/{test_name}/library/logs/ for detailed information about what went wrong.
Common debugging approaches:
- Run with
--nocapture to see all output
- Check job logs in
test_data/{test_name}/library/job_logs/
- Run scenarios individually with manual environment variables
- Use
RUST_LOG=trace for maximum verbosity
- Run tests in parallel: Use
cargo test default parallelism
- Minimize sleeps: Use event waiting instead of fixed delays
- Share setup code: Extract common initialization into helpers
Writing New Tests
Single-Device Test Checklist
Multi-Device Test Checklist
TypeScript Integration Testing
Spacedrive provides a bridge infrastructure for running TypeScript tests against a real Rust daemon. This enables true end-to-end testing across the Rust backend and TypeScript frontend, verifying that cache updates, WebSocket events, and React hooks work correctly with real data.
Architecture
The TypeScript bridge test pattern works as follows:
- Rust test creates a daemon with indexed locations using
IndexingHarnessBuilder
- Connection info (TCP socket address, library ID, paths) written to JSON config file
- Rust spawns
bun test with specific TypeScript test file
- TypeScript test reads config, connects to daemon via
SpacedriveClient.fromTcpSocket()
- TypeScript test performs file operations and validates cache updates via React hooks
- Rust validates test exit code and cleans up
This pattern tests the entire stack: Rust daemon → RPC transport → TypeScript client → React hooks → cache updates.
Writing Bridge Tests
Rust Side
Create a test in core/tests/ that spawns the daemon and TypeScript test:
#[tokio::test]
async fn test_typescript_cache_updates() -> anyhow::Result<()> {
// Create daemon with RPC server enabled
let harness = IndexingHarnessBuilder::new("typescript_bridge_test")
.enable_daemon() // Start RPC server for TypeScript client
.build()
.await?;
// Create test location with files
let test_location = harness.create_test_location("test_files").await?;
test_location.create_dir("folder_a").await?;
test_location.write_file("folder_a/file1.txt", "Content").await?;
// Index the location
let location = test_location
.index("Test Location", IndexMode::Shallow)
.await?;
// Get daemon socket address
let socket_addr = harness
.daemon_socket_addr()
.expect("Daemon should be enabled")
.to_string();
// Prepare bridge config for TypeScript
let bridge_config = TestBridgeConfig {
socket_addr,
library_id: harness.library.id().to_string(),
location_db_id: location.db_id,
location_path: test_location.path().to_path_buf(),
test_data_path: harness.temp_path().to_path_buf(),
};
// Write config to temp file
let config_path = harness.temp_path().join("bridge_config.json");
tokio::fs::write(&config_path, serde_json::to_string_pretty(&bridge_config)?).await?;
// Spawn TypeScript test
let ts_test_file = "packages/ts-client/tests/integration/mytest.test.ts";
let workspace_root = std::env::current_dir()?.parent().unwrap().to_path_buf();
let output = tokio::process::Command::new("bun")
.arg("test")
.arg(workspace_root.join(ts_test_file))
.env("BRIDGE_CONFIG_PATH", config_path.to_str().unwrap())
.current_dir(&workspace_root)
.output()
.await?;
// Verify TypeScript test passed
if !output.status.success() {
anyhow::bail!("TypeScript test failed: {:?}", output.status.code());
}
harness.shutdown().await?;
Ok(())
}
Use .enable_daemon() on IndexingHarnessBuilder to start the RPC server. The daemon listens on a random TCP port returned by .daemon_socket_addr().
TypeScript Side
Create a test in packages/ts-client/tests/integration/:
import { describe, test, expect, beforeAll } from "bun:test";
import { readFile } from "fs/promises";
import { SpacedriveClient } from "../../src/client";
import { renderHook, waitFor } from "@testing-library/react";
import { SpacedriveProvider } from "../../src/hooks/useClient";
import { useNormalizedQuery } from "../../src/hooks/useNormalizedQuery";
interface BridgeConfig {
socket_addr: string;
library_id: string;
location_db_id: number;
location_path: string;
test_data_path: string;
}
let bridgeConfig: BridgeConfig;
let client: SpacedriveClient;
beforeAll(async () => {
// Read bridge config from Rust test
const configPath = process.env.BRIDGE_CONFIG_PATH;
const configJson = await readFile(configPath, "utf-8");
bridgeConfig = JSON.parse(configJson);
// Connect to daemon via TCP socket
client = SpacedriveClient.fromTcpSocket(bridgeConfig.socket_addr);
client.setCurrentLibrary(bridgeConfig.library_id);
});
describe("Cache Update Tests", () => {
test("should update cache when files move", async () => {
const wrapper = ({ children }) =>
React.createElement(SpacedriveProvider, { client }, children);
// Query directory listing with useNormalizedQuery
const { result } = renderHook(
() => useNormalizedQuery({
wireMethod: "query:files.directory_listing",
input: { path: { Physical: { path: folderPath } } },
resourceType: "file",
pathScope: { Physical: { path: folderPath } },
debug: true, // Enable debug logging
}),
{ wrapper }
);
// Wait for initial data
await waitFor(() => {
expect(result.current.data).toBeDefined();
});
// Perform file operation
await rename(oldPath, newPath);
// Wait for watcher to detect change (500ms buffer + processing)
await new Promise(resolve => setTimeout(resolve, 2000));
// Verify cache updated
expect(result.current.data.files).toContainEqual(
expect.objectContaining({ name: "newfile" })
);
});
});
TCP Transport
TypeScript tests connect to the daemon via TCP socket using TcpSocketTransport. This transport is designed for Bun/Node.js environments and enables testing outside the browser.
// Automatic with factory method
const client = SpacedriveClient.fromTcpSocket("127.0.0.1:6969");
// Manual construction
import { TcpSocketTransport } from "@sd/ts-client/transports";
const transport = new TcpSocketTransport("127.0.0.1:6969");
const client = new SpacedriveClient(transport);
The TCP transport:
- Uses JSON-RPC 2.0 over TCP
- Supports WebSocket-style subscriptions for events
- Automatically reconnects on connection loss
- Works in both Bun and Node.js runtimes
Testing Cache Updates
The primary use case for bridge tests is verifying that useNormalizedQuery cache updates work correctly when the daemon emits ResourceChanged or ResourceChangedBatch events.
Key patterns:
- Enable debug logging with
debug: true in useNormalizedQuery options
- Wait for watcher delays (500ms buffer + processing time, typically 2-8 seconds)
- Collect events by wrapping the subscription manager to log all received events
- Verify cache state using React Testing Library’s
waitFor and assertions
// Enable debug logging
const { result } = renderHook(
() => useNormalizedQuery({
wireMethod: "query:files.directory_listing",
input: { /* ... */ },
resourceType: "file",
pathScope: { /* ... */ },
debug: true, // Logs event processing
}),
{ wrapper }
);
// Collect all events for debugging
const allEvents: any[] = [];
const originalCreateSubscription = (client as any).subscriptionManager.createSubscription;
(client as any).subscriptionManager.createSubscription = function(filter: any, callback: any) {
const wrappedCallback = (event: any) => {
allEvents.push({ timestamp: new Date().toISOString(), event });
console.log(`Event received:`, JSON.stringify(event, null, 2));
callback(event);
};
return originalCreateSubscription.call(this, filter, wrappedCallback);
};
Running Bridge Tests
# Run all TypeScript bridge tests
cargo test --package sd-core --test typescript_bridge_test -- --nocapture
# Run specific bridge test
cargo test test_typescript_use_normalized_query_with_file_moves -- --nocapture
# Run only the TypeScript side (requires manual daemon setup)
cd packages/ts-client
BRIDGE_CONFIG_PATH=/path/to/config.json bun test tests/integration/mytest.test.ts
Use --nocapture to see TypeScript test output. The Rust test prints all stdout/stderr from the TypeScript test process.
Common Scenarios
File moves between folders:
- Tests that files removed from one directory appear in another
- Verifies UUID preservation (move detection vs delete+create)
Folder renames:
- Tests that nested files update their paths correctly
- Verifies parent path updates propagate to descendants
Bulk operations:
- Tests 20+ file moves with mixed Physical/Content paths
- Verifies cache updates don’t miss files during batched events
Content-addressed files:
- Uses
IndexMode::Content to enable content identification
- Tests that files with
alternate_paths update correctly
- Verifies metadata-only updates don’t add duplicate cache entries
Debugging Bridge Tests
Check Rust logs:
RUST_LOG=debug cargo test typescript_bridge -- --nocapture
Check TypeScript output:
The Rust test prints all TypeScript stdout/stderr. Look for:
[TS] prefixed log messages
- Event payloads with
🔔 emoji
- Final event summary at test end
Verify daemon is running:
# In Rust test output, look for:
Socket address: 127.0.0.1:XXXXX
Library ID: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Check bridge config:
# The config file is written to test_data directory
cat /tmp/test_data/typescript_bridge_test/bridge_config.json
Common issues:
- TypeScript test times out: Increase watcher wait time (filesystem events can be slow)
- Cache not updating: Enable
debug: true to see if events are received
- Connection refused: Verify daemon started with
.enable_daemon()
- Wrong library: Check that
client.setCurrentLibrary() uses correct ID from config
Examples
For complete examples, refer to:
Single Device Tests:
tests/copy_action_test.rs - Event collection during file operations (persistent + ephemeral)
tests/job_resumption_integration_test.rs - Job interruption handling
Subprocess Framework (Real Networking):
tests/device_pairing_test.rs - Device pairing with real network discovery
Custom Harness (Mock Transport):
tests/sync_realtime_test.rs - Real-time sync testing with deterministic transport
tests/sync_integration_test.rs - Complex sync scenarios with mock networking
tests/file_transfer_test.rs - Cross-device file operations
TypeScript Bridge Tests:
tests/typescript_bridge_test.rs - Rust harness that spawns TypeScript tests
packages/ts-client/tests/integration/useNormalizedQuery.test.ts - File move cache updates
packages/ts-client/tests/integration/useNormalizedQuery.folder-rename.test.ts - Folder rename propagation
packages/ts-client/tests/integration/useNormalizedQuery.bulk-moves.test.ts - Bulk operations with content-addressed files