Skip to content

feat(io): add streaming_file input for real-time file tailing #3876

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
aronchick wants to merge 1 commit into
base: main
Choose a base branch
from
Open

feat(io): add streaming_file input for real-time file tailing #3876

aronchick wants to merge 1 commit into from

Conversation

@aronchick
Copy link

@aronchick aronchick commented Dec 28, 2025

Summary

  • Add new streaming_file input component for real-time file tailing (like tail -F)
  • Handles log rotation via inode-based detection on Unix, size-based fallback on Windows
  • Handles file truncation with automatic position reset
  • Exposes position metadata for custom persistence via cache

Features

  • Inode-based file identity tracking for rotation detection
  • Automatic truncation detection and position reset
  • Optional fsnotify integration for efficient file watching
  • Configurable polling intervals and buffer sizes
  • Position metadata (path, inode, offset) exposed on each message

Platform Support

  • Unix: Full inode-based rotation detection via syscall.Stat_t
  • Windows: Size-based fallback for rotation detection

Test plan

  • Unit tests pass locally (7 tests, 2.046s)
  • Build succeeds
  • CI validation

🤖 Generated with Claude Code
@aronchick @claude
feat(io): add streaming_file input for real-time file tailing
9dac6a8 
Add a new input component that reads from files continuously with
automatic handling of log rotation and truncation, similar to 'tail -F'.

Features:
- Inode-based file identity tracking for rotation detection
- Automatic truncation detection and position reset
- Optional fsnotify integration for efficient file watching
- Configurable polling intervals and buffer sizes
- Position metadata exposed for custom persistence via cache

Platform support:
- Unix: Full inode-based rotation detection
- Windows: Size-based fallback for rotation detection

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@aronchick aronchick marked this pull request as ready for review December 29, 2025 00:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant

@aronchick