Skip to main content
Testing in Spacedrive Core ensures reliability across single-device operations and multi-device networking scenarios. This guide covers the available frameworks, patterns, and best practices.

Testing Infrastructure

Spacedrive Core provides two primary testing approaches:
  1. Standard Tests - For unit and single-core integration testing
  2. Subprocess Framework - For multi-device networking and distributed scenarios

Test Organization

Tests live in two locations:
  • core/tests/ - Integration tests that verify complete workflows
  • core/src/testing/ - Test framework utilities and helpers

Standard Testing

For single-device tests, use Tokio’s async test framework:
#[tokio::test]
async fn test_library_creation() {
    let setup = IntegrationTestSetup::new("library_test").await.unwrap();
    let core = setup.create_core().await.unwrap();
    
    let library = core.libraries
        .create_library("Test Library", None)
        .await
        .unwrap();
    
    assert!(!library.id.is_empty());
}

Integration Test Setup

The IntegrationTestSetup utility provides isolated test environments:
// Basic setup
let setup = IntegrationTestSetup::new("test_name").await?;

// Custom configuration
let setup = IntegrationTestSetup::with_config("test_name", |builder| {
    builder
        .log_level("debug")
        .networking_enabled(true)
        .volume_monitoring_enabled(false)
}).await?;
Key features:
  • Isolated temporary directories per test
  • Structured logging to test_data/{test_name}/library/logs/
  • Automatic cleanup on drop
  • Configurable app settings

Multi-Device Testing

Spacedrive provides two approaches for testing multi-device scenarios:

When to Use Subprocess Framework

Use CargoTestRunner subprocess framework when:
  • Testing real networking with actual network discovery, NAT traversal, and connections
  • Testing device pairing workflows that require independent network stacks
  • Scenarios need true process isolation (separate memory spaces, different ports)
  • You want to test network reconnection, timeout, and failure handling
  • Testing cross-platform network behavior
Examples: Device pairing, network discovery, connection management
// Uses real networking, separate processes
let mut runner = CargoTestRunner::new()
    .add_subprocess("alice", "alice_pairing_scenario")
    .add_subprocess("bob", "bob_pairing_scenario");

When to Use Custom Transport/Harness

Use custom harness with mock transport when:
  • Testing sync logic without network overhead
  • Fast iteration on data synchronization algorithms
  • Testing deterministic scenarios without network timing issues
  • Verifying database state and conflict resolution
  • Need precise control over sync event ordering
Examples: Real-time sync, backfill, content identity linking, conflict resolution
// Uses mock transport, single process, fast and deterministic
let harness = TwoDeviceHarnessBuilder::new("sync_test")
    .collect_events(true)
    .build()
    .await?;

Comparison

AspectSubprocess FrameworkCustom Harness
SpeedSlower (real networking)Fast (in-memory)
NetworkingReal (discovery, NAT)Mock transport
IsolationTrue process isolationShared process
DebuggingHarder (multiple processes)Easier (single process)
DeterminismNetwork timing variesFully deterministic
Use CaseNetwork featuresSync/data logic

Subprocess Testing Framework

The subprocess framework spawns separate cargo test processes for each device role:
let mut runner = CargoTestRunner::new()
    .with_timeout(Duration::from_secs(90))
    .add_subprocess("alice", "alice_scenario")
    .add_subprocess("bob", "bob_scenario");

runner.run_until_success(|outputs| {
    outputs.values().all(|output| output.contains("SUCCESS"))
}).await?;

Writing Multi-Device Tests

Create separate test functions for each device role:
#[tokio::test]
async fn test_device_pairing() {
    let mut runner = CargoTestRunner::new()
        .add_subprocess("alice", "alice_pairing")
        .add_subprocess("bob", "bob_pairing");
        
    runner.run_until_success(|outputs| {
        outputs.values().all(|o| o.contains("PAIRING_SUCCESS"))
    }).await.unwrap();
}

#[tokio::test]
#[ignore]
async fn alice_pairing() {
    if env::var("TEST_ROLE").unwrap_or_default() != "alice" {
        return;
    }
    
    let data_dir = PathBuf::from(env::var("TEST_DATA_DIR").unwrap());
    let core = create_test_core(data_dir).await.unwrap();
    
    // Alice initiates pairing
    let (code, _) = core.start_pairing_as_initiator().await.unwrap();
    fs::write("/tmp/pairing_code.txt", &code).unwrap();
    
    // Wait for connection
    wait_for_connection(&core).await;
    println!("PAIRING_SUCCESS");
}
Device scenario functions must be marked with #[ignore] to prevent direct execution. They only run when called by the subprocess framework.

Process Coordination

Processes coordinate through:
  • Environment variables: TEST_ROLE and TEST_DATA_DIR
  • Temporary files: Share data like pairing codes
  • Output patterns: Success markers for the runner to detect

Common Test Patterns

Filesystem Watcher Testing

When testing filesystem watcher functionality, several critical setup steps are required:

Enable Watcher in Test Config

The default TestConfigBuilder disables the filesystem watcher (for performance in sync tests). Tests that verify watcher events must explicitly enable it:
let mut config = TestConfigBuilder::new(test_root.clone())
    .build()?;

// CRITICAL: Enable watcher for change detection tests
config.services.fs_watcher_enabled = true;
config.save()?;

let core = Core::new(config.data_dir.clone()).await?;

Use Home Directory Paths on macOS

macOS temp directories (/var/folders/...) don’t reliably deliver filesystem events. Use home directory paths instead:
// ❌ Don't use TempDir for watcher tests
let temp_dir = TempDir::new()?;

// ✅ Use home directory
let home = std::env::var("HOME").unwrap_or_else(|_| "/tmp".to_string());
let test_root = PathBuf::from(home).join(".spacedrive_test_my_test");

// Clean up before
let _ = tokio::fs::remove_dir_all(&test_root).await;
tokio::fs::create_dir_all(&test_root).await?;

// ... run test ...

// Clean up after
tokio::fs::remove_dir_all(&test_root).await?;

Ephemeral Watching Requirements

Ephemeral paths must be indexed before watching:
// 1. Index the directory (ephemeral mode)
let config = IndexerJobConfig::ephemeral_browse(
    SdPath::local(dest_dir.clone()),
    IndexScope::Current
);
let job = IndexerJob::new(config);
library.jobs().dispatch(job).await?.wait().await?;

// 2. Mark indexing complete (indexer job does this automatically)
context.ephemeral_cache().mark_indexing_complete(&dest_dir);

// 3. Register for watching (indexer job does this automatically)
watcher.watch_ephemeral(dest_dir.clone()).await?;

// Now filesystem events will be detected
The IndexerJob automatically calls watch_ephemeral() after successful indexing, so manual registration is only needed when bypassing the indexer.

Persistent Location Watching

For persistent locations, the watcher auto-loads locations at startup. New locations created during tests must be manually registered:
// After creating and indexing a location
let location_meta = LocationMeta {
    id: location_uuid,
    library_id: library.id(),
    root_path: location_path.clone(),
    rule_toggles: RuleToggles::default(),
};

watcher.watch_location(location_meta).await?;
The IndexingHarness handles this automatically.

Event Collection Best Practices

Start collecting events after initialization to avoid library statistics noise:
// Complete all setup first
let harness = IndexingHarnessBuilder::new("test").build().await?;
let location = harness.add_and_index_location(...).await?;

// Wait for setup to settle
tokio::time::sleep(Duration::from_millis(500)).await;

// Start collecting BEFORE the operation you're testing
let mut collector = EventCollector::new(&harness.core.events);
let handle = tokio::spawn(async move {
    collector.collect_events(Duration::from_secs(5)).await;
    collector
});

// Perform operation
perform_copy_operation().await?;

// Collect and verify
let collector = handle.await.unwrap();
let stats = collector.analyze().await;
assert!(stats.resource_changed.get("file").copied().unwrap_or(0) >= 2);
The EventCollector automatically filters out:
  • Library statistics updates (LibraryStatisticsUpdated)
  • Library resource events (non-file/entry events)

Expected Event Types

Different handlers emit different event types:
  • Ephemeral handler: Individual ResourceChanged events per file (CREATE + MODIFY)
  • Persistent handler: Batched ResourceChangedBatch events
// Ephemeral assertion
let file_events = stats.resource_changed.get("file").copied().unwrap_or(0);
assert!(file_events >= 2, "Expected file ResourceChanged events");

// Persistent assertion
let batch_count = stats.resource_changed_batch.get("file").copied().unwrap_or(0);
assert!(batch_count >= 2, "Expected file ResourceChangedBatch events");

Event Monitoring

Waiting for Specific Events

Wait for specific Core events with timeouts:
let mut events = core.events.subscribe();

let event = wait_for_event(
    &mut events,
    |e| matches!(e, Event::JobCompleted { .. }),
    Duration::from_secs(30)
).await?;

Collecting All Events for Analysis

For tests that need to verify event emission patterns (e.g., ResourceChanged events during operations), use the shared EventCollector helper:
use helpers::EventCollector;

// Create collector with full event capture for debugging
let mut collector = EventCollector::with_capture(&harness.core.events);

// Spawn collection task
let collection_handle = tokio::spawn(async move {
    collector.collect_events(Duration::from_secs(10)).await;
    collector
});

// Perform operations that emit events
perform_copy_operation().await?;
location.reindex().await?;

// Retrieve collector and analyze
let collector = collection_handle.await.unwrap();

// Print statistics summary
let stats = collector.analyze().await;
stats.print();

// Print full event details for debugging (when using with_capture)
collector.print_events().await;

// Write events to JSON file for later inspection
collector.write_to_file(&snapshot_dir.join("events.json")).await?;

// Filter specific events
let file_events = collector.get_resource_batch_events("file").await;
let indexing_events = collector.get_events_by_type("IndexingCompleted").await;
The EventCollector tracks:
  • ResourceChanged/ResourceChangedBatch events by resource type
  • Indexing start/completion events
  • Job lifecycle events (started/completed)
  • Entry events (created/modified/deleted/moved)
Statistics Output:
Event Statistics:
==================

ResourceChangedBatch events:
  file → 45 resources

Indexing events:
  Started: 1
  Completed: 1

Entry events:
  Created: 3
  Modified: 0

Job events:
  Started:
    indexer → 1
  Completed:
    indexer → 1
Detailed Event Output (with with_capture()):
=== Collected Events (8) ===

[1] IndexingStarted
  Location: 550e8400-e29b-41d4-a716-446655440000

[2] JobStarted
  Job: indexer (job_123)

[3] ResourceChangedBatch
  Type: file
  Resources: 45 items
  Paths: 1 affected

[4] IndexingCompleted
  Location: 550e8400-e29b-41d4-a716-446655440000
  Files: 42, Dirs: 3

[5] JobCompleted
  Job: indexer (job_123)
  Output: Success
Use Cases:
  • Verifying watcher events during file operations
  • Testing normalized cache updates
  • Debugging event emission patterns
  • Creating test fixtures with real event data
  • Inspecting actual resource payloads in events

Database Verification

Query the database directly to verify state:
use sd_core::entities;

let entries = entities::entry::Entity::find()
    .filter(entities::entry::Column::Name.contains("test"))
    .all(db.conn())
    .await?;

assert_eq!(entries.len(), expected_count);

Job Testing

Test job execution and resumption:
// Start a job
let job_id = core.jobs.dispatch(IndexingJob::new(...)).await?;

// Monitor progress
wait_for_event(&mut events, |e| matches!(
    e, 
    Event::JobProgress { id, .. } if *id == job_id
), timeout).await?;

// Verify completion
let job = core.jobs.get_job(job_id).await?;
assert_eq!(job.status, JobStatus::Completed);

Mock Transport for Sync Testing

Test synchronization without real networking:
let transport = Arc::new(Mutex::new(Vec::new()));

let mut core_a = create_test_core().await?;
let mut core_b = create_test_core().await?;

// Connect cores with mock transport
connect_with_mock_transport(&mut core_a, &mut core_b, transport).await?;

// Verify sync
perform_operation_on_a(&core_a).await?;
wait_for_sync(&core_b).await?;

Test Helpers

Common Utilities

The framework provides comprehensive test helpers in core/tests/helpers/: Event Collection:
  • EventCollector - Collect and analyze all events from the event bus
  • EventStats - Statistics about collected events with formatted output
Indexing Tests:
  • IndexingHarnessBuilder - Create isolated test environments with indexing support
  • TestLocation - Builder for test locations with files
  • LocationHandle - Handle to indexed locations with verification methods
Sync Tests:
  • TwoDeviceHarnessBuilder - Pre-configured two-device sync test environments
  • MockTransport - Mock network transport for deterministic sync testing
  • wait_for_sync() - Sophisticated sync completion detection
  • TestConfigBuilder - Custom test configurations
Database & Jobs:
  • wait_for_event() - Wait for specific events with timeout
  • wait_for_indexing() - Wait for indexing job completion
  • register_device() - Register a device in a library
See core/tests/helpers/README.md for detailed documentation on all available helpers including usage examples and migration guides.

Test Volumes

For volume-related tests, use the test volume utilities:
use helpers::test_volumes;

let volume = test_volumes::create_test_volume().await?;
// Test volume operations
test_volumes::cleanup_test_volume(volume).await?;

Running Tests

All Tests

cargo test --workspace

Specific Test

cargo test test_device_pairing --nocapture

Debug Subprocess Tests

# Run individual scenario
TEST_ROLE=alice TEST_DATA_DIR=/tmp/test cargo test alice_scenario -- --ignored --nocapture

With Logging

RUST_LOG=debug cargo test test_name --nocapture

Best Practices

Test Structure

  1. Use descriptive names: test_cross_device_file_transfer over test_transfer
  2. One concern per test: Focus on a single feature or workflow
  3. Clean up resources: Use RAII patterns or explicit cleanup

Subprocess Tests

  1. Always use #[ignore] on scenario functions
  2. Check TEST_ROLE early: Return immediately if role doesn’t match
  3. Use clear success patterns: Print distinct markers for the runner
  4. Set appropriate timeouts: Balance between test speed and reliability

Debugging

When tests fail, check the logs in test_data/{test_name}/library/logs/ for detailed information about what went wrong.
Common debugging approaches:
  • Run with --nocapture to see all output
  • Check job logs in test_data/{test_name}/library/job_logs/
  • Run scenarios individually with manual environment variables
  • Use RUST_LOG=trace for maximum verbosity

Performance

  1. Run tests in parallel: Use cargo test default parallelism
  2. Minimize sleeps: Use event waiting instead of fixed delays
  3. Share setup code: Extract common initialization into helpers

Writing New Tests

Single-Device Test Checklist

  • Create test with #[tokio::test]
  • Use IntegrationTestSetup for isolation
  • Wait for events instead of sleeping
  • Verify both positive and negative cases
  • Clean up temporary files

Multi-Device Test Checklist

  • Create orchestrator function with CargoTestRunner
  • Create scenario functions with #[ignore]
  • Add TEST_ROLE guards to scenarios
  • Define clear success patterns
  • Handle process coordination properly
  • Set reasonable timeouts

TypeScript Integration Testing

Spacedrive provides a bridge infrastructure for running TypeScript tests against a real Rust daemon. This enables true end-to-end testing across the Rust backend and TypeScript frontend, verifying that cache updates, WebSocket events, and React hooks work correctly with real data.

Architecture

The TypeScript bridge test pattern works as follows:
  1. Rust test creates a daemon with indexed locations using IndexingHarnessBuilder
  2. Connection info (TCP socket address, library ID, paths) written to JSON config file
  3. Rust spawns bun test with specific TypeScript test file
  4. TypeScript test reads config, connects to daemon via SpacedriveClient.fromTcpSocket()
  5. TypeScript test performs file operations and validates cache updates via React hooks
  6. Rust validates test exit code and cleans up
This pattern tests the entire stack: Rust daemon → RPC transport → TypeScript client → React hooks → cache updates.

Writing Bridge Tests

Rust Side

Create a test in core/tests/ that spawns the daemon and TypeScript test:
#[tokio::test]
async fn test_typescript_cache_updates() -> anyhow::Result<()> {
    // Create daemon with RPC server enabled
    let harness = IndexingHarnessBuilder::new("typescript_bridge_test")
        .enable_daemon() // Start RPC server for TypeScript client
        .build()
        .await?;

    // Create test location with files
    let test_location = harness.create_test_location("test_files").await?;
    test_location.create_dir("folder_a").await?;
    test_location.write_file("folder_a/file1.txt", "Content").await?;

    // Index the location
    let location = test_location
        .index("Test Location", IndexMode::Shallow)
        .await?;

    // Get daemon socket address
    let socket_addr = harness
        .daemon_socket_addr()
        .expect("Daemon should be enabled")
        .to_string();

    // Prepare bridge config for TypeScript
    let bridge_config = TestBridgeConfig {
        socket_addr,
        library_id: harness.library.id().to_string(),
        location_db_id: location.db_id,
        location_path: test_location.path().to_path_buf(),
        test_data_path: harness.temp_path().to_path_buf(),
    };

    // Write config to temp file
    let config_path = harness.temp_path().join("bridge_config.json");
    tokio::fs::write(&config_path, serde_json::to_string_pretty(&bridge_config)?).await?;

    // Spawn TypeScript test
    let ts_test_file = "packages/ts-client/tests/integration/mytest.test.ts";
    let workspace_root = std::env::current_dir()?.parent().unwrap().to_path_buf();
    let output = tokio::process::Command::new("bun")
        .arg("test")
        .arg(workspace_root.join(ts_test_file))
        .env("BRIDGE_CONFIG_PATH", config_path.to_str().unwrap())
        .current_dir(&workspace_root)
        .output()
        .await?;

    // Verify TypeScript test passed
    if !output.status.success() {
        anyhow::bail!("TypeScript test failed: {:?}", output.status.code());
    }

    harness.shutdown().await?;
    Ok(())
}
Use .enable_daemon() on IndexingHarnessBuilder to start the RPC server. The daemon listens on a random TCP port returned by .daemon_socket_addr().

TypeScript Side

Create a test in packages/ts-client/tests/integration/:
import { describe, test, expect, beforeAll } from "bun:test";
import { readFile } from "fs/promises";
import { SpacedriveClient } from "../../src/client";
import { renderHook, waitFor } from "@testing-library/react";
import { SpacedriveProvider } from "../../src/hooks/useClient";
import { useNormalizedQuery } from "../../src/hooks/useNormalizedQuery";

interface BridgeConfig {
    socket_addr: string;
    library_id: string;
    location_db_id: number;
    location_path: string;
    test_data_path: string;
}

let bridgeConfig: BridgeConfig;
let client: SpacedriveClient;

beforeAll(async () => {
    // Read bridge config from Rust test
    const configPath = process.env.BRIDGE_CONFIG_PATH;
    const configJson = await readFile(configPath, "utf-8");
    bridgeConfig = JSON.parse(configJson);

    // Connect to daemon via TCP socket
    client = SpacedriveClient.fromTcpSocket(bridgeConfig.socket_addr);
    client.setCurrentLibrary(bridgeConfig.library_id);
});

describe("Cache Update Tests", () => {
    test("should update cache when files move", async () => {
        const wrapper = ({ children }) =>
            React.createElement(SpacedriveProvider, { client }, children);

        // Query directory listing with useNormalizedQuery
        const { result } = renderHook(
            () => useNormalizedQuery({
                wireMethod: "query:files.directory_listing",
                input: { path: { Physical: { path: folderPath } } },
                resourceType: "file",
                pathScope: { Physical: { path: folderPath } },
                debug: true, // Enable debug logging
            }),
            { wrapper }
        );

        // Wait for initial data
        await waitFor(() => {
            expect(result.current.data).toBeDefined();
        });

        // Perform file operation
        await rename(oldPath, newPath);

        // Wait for watcher to detect change (500ms buffer + processing)
        await new Promise(resolve => setTimeout(resolve, 2000));

        // Verify cache updated
        expect(result.current.data.files).toContainEqual(
            expect.objectContaining({ name: "newfile" })
        );
    });
});

TCP Transport

TypeScript tests connect to the daemon via TCP socket using TcpSocketTransport. This transport is designed for Bun/Node.js environments and enables testing outside the browser.
// Automatic with factory method
const client = SpacedriveClient.fromTcpSocket("127.0.0.1:6969");

// Manual construction
import { TcpSocketTransport } from "@sd/ts-client/transports";
const transport = new TcpSocketTransport("127.0.0.1:6969");
const client = new SpacedriveClient(transport);
The TCP transport:
  • Uses JSON-RPC 2.0 over TCP
  • Supports WebSocket-style subscriptions for events
  • Automatically reconnects on connection loss
  • Works in both Bun and Node.js runtimes

Testing Cache Updates

The primary use case for bridge tests is verifying that useNormalizedQuery cache updates work correctly when the daemon emits ResourceChanged or ResourceChangedBatch events. Key patterns:
  1. Enable debug logging with debug: true in useNormalizedQuery options
  2. Wait for watcher delays (500ms buffer + processing time, typically 2-8 seconds)
  3. Collect events by wrapping the subscription manager to log all received events
  4. Verify cache state using React Testing Library’s waitFor and assertions
// Enable debug logging
const { result } = renderHook(
    () => useNormalizedQuery({
        wireMethod: "query:files.directory_listing",
        input: { /* ... */ },
        resourceType: "file",
        pathScope: { /* ... */ },
        debug: true, // Logs event processing
    }),
    { wrapper }
);

// Collect all events for debugging
const allEvents: any[] = [];
const originalCreateSubscription = (client as any).subscriptionManager.createSubscription;
(client as any).subscriptionManager.createSubscription = function(filter: any, callback: any) {
    const wrappedCallback = (event: any) => {
        allEvents.push({ timestamp: new Date().toISOString(), event });
        console.log(`Event received:`, JSON.stringify(event, null, 2));
        callback(event);
    };
    return originalCreateSubscription.call(this, filter, wrappedCallback);
};

Running Bridge Tests

# Run all TypeScript bridge tests
cargo test --package sd-core --test typescript_bridge_test -- --nocapture

# Run specific bridge test
cargo test test_typescript_use_normalized_query_with_file_moves -- --nocapture

# Run only the TypeScript side (requires manual daemon setup)
cd packages/ts-client
BRIDGE_CONFIG_PATH=/path/to/config.json bun test tests/integration/mytest.test.ts
Use --nocapture to see TypeScript test output. The Rust test prints all stdout/stderr from the TypeScript test process.

Common Scenarios

File moves between folders:
  • Tests that files removed from one directory appear in another
  • Verifies UUID preservation (move detection vs delete+create)
Folder renames:
  • Tests that nested files update their paths correctly
  • Verifies parent path updates propagate to descendants
Bulk operations:
  • Tests 20+ file moves with mixed Physical/Content paths
  • Verifies cache updates don’t miss files during batched events
Content-addressed files:
  • Uses IndexMode::Content to enable content identification
  • Tests that files with alternate_paths update correctly
  • Verifies metadata-only updates don’t add duplicate cache entries

Debugging Bridge Tests

Check Rust logs:
RUST_LOG=debug cargo test typescript_bridge -- --nocapture
Check TypeScript output: The Rust test prints all TypeScript stdout/stderr. Look for:
  • [TS] prefixed log messages
  • Event payloads with 🔔 emoji
  • Final event summary at test end
Verify daemon is running:
# In Rust test output, look for:
Socket address: 127.0.0.1:XXXXX
Library ID: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Check bridge config:
# The config file is written to test_data directory
cat /tmp/test_data/typescript_bridge_test/bridge_config.json
Common issues:
  • TypeScript test times out: Increase watcher wait time (filesystem events can be slow)
  • Cache not updating: Enable debug: true to see if events are received
  • Connection refused: Verify daemon started with .enable_daemon()
  • Wrong library: Check that client.setCurrentLibrary() uses correct ID from config

Examples

For complete examples, refer to: Single Device Tests:
  • tests/copy_action_test.rs - Event collection during file operations (persistent + ephemeral)
  • tests/job_resumption_integration_test.rs - Job interruption handling
Subprocess Framework (Real Networking):
  • tests/device_pairing_test.rs - Device pairing with real network discovery
Custom Harness (Mock Transport):
  • tests/sync_realtime_test.rs - Real-time sync testing with deterministic transport
  • tests/sync_integration_test.rs - Complex sync scenarios with mock networking
  • tests/file_transfer_test.rs - Cross-device file operations
TypeScript Bridge Tests:
  • tests/typescript_bridge_test.rs - Rust harness that spawns TypeScript tests
  • packages/ts-client/tests/integration/useNormalizedQuery.test.ts - File move cache updates
  • packages/ts-client/tests/integration/useNormalizedQuery.folder-rename.test.ts - Folder rename propagation
  • packages/ts-client/tests/integration/useNormalizedQuery.bulk-moves.test.ts - Bulk operations with content-addressed files