gh-ops

A reusable, deterministic GitHub operations and intelligence platform — now frozen for maintenance.

"A personal GitHub operations layer running on GitHub Actions."

Implementation status: COMPLETE / FROZEN 🔒

All planned implementation phases and the post-Phase-8 hardening/intelligence line are complete. The phase labels below are historical milestones only. Phase 8 is the final numbered milestone; the complete frozen system is represented by the post-Phase-8 batches and the 1,495-test baseline.

Total: 1495/1495 tests passing.

Figures are cumulative historical checkpoints. The complete system is the final 1,495-test frozen baseline; the security, reporting, repository/event, Telegram, Laya/LLM, and deterministic-intelligence work all landed after the original Phase 8 milestone.

Implementation: Complete · Architecture: Complete · Intelligence: Complete · Telegram: Hardened · OSS Radar: Complete

The deterministic core, intelligence layer, reporting pipeline, Telegram contracts, OSS Radar, optional Laya/LLM interpretation, GitHub Actions boundaries, and documentation are complete. The repository is now in maintenance mode: bug fixes and evidence-driven hardening are welcome; no further feature expansion is planned for this frozen baseline.

Final verification: 1495/1495 tests passing · compileall clean · touched-path Ruff clean · Telegram contracts green · secret audit clean · master == origin/master.

See ARCHITECTURE.md for the full system design.

# Install dependencies
pip install -r requirements.txt

# Configure secrets locally (the file is gitignored)
cp .env.example .env
# then edit .env and fill in the values

# Run locally
python scripts/run_local.py daily

# Or run a job directly — the same entry point the workflows use
python -m src.jobs status        # read-only: report current state + config
python -m src.jobs daily

# Check Telegram delivery configuration (prints no secrets)
python scripts/check_telegram.py

The dispatcher is the same code path in CI and locally:

python -m src.jobs <job_name>          # daily | monitoring | weekly-report
                                       # oss-hunt | security | status
GHOPS_JOB=weekly-report python -m src.jobs

python -m src.jobs                     # prints usage and exits 1
python -m src.jobs not-a-job           # prints available jobs, exits 1

Jobs read GITHUB_TOKEN for collection and TELEGRAM_BOT_TOKEN for delivery. A job never exits 0 without having executed one.

Monitoring alerts remain event-driven. When telegram.run_summary is enabled, successful daily and weekly runs may send briefs built from the shared reporting ledger (only when meaningful events exist; a quiet run sends nothing). Briefs render in a fixed attention-first order: 🚨 Attention (action-required only) → 🛡️ Security → CI → 🧭 OSS → 🧑‍💻 Developer → 🚀 Releases & Repositories → ⚠️ Data quality, closed by an Overall: line, followed by deterministic intelligence blocks (🎯 Focus, 📦 Repository Signals, 📈 Trends) built from the bounded evidence pack. Data-quality warnings are ledger-gated: each distinct state is briefed once, then silent until it changes. Successful CI runs (and gh-ops's own workflow runs) are silenced at conversion; failures of gh-ops's own enumerated workflows render as GH-OPS · SYSTEM alerts. Completion summaries (run/oss/security) are no longer sent by jobs; their formatters remain exported for API compatibility. Repository counts come only from config/repositories.yml; accessible account repositories are never added automatically. OSS opportunities are deduplicated across runs via the oss_opportunities resource key in Phase 3 state (at-least-once: identities are marked only after successful delivery); a previously delivered opportunity whose comments, state, stars, or updated timestamp change is re-announced exactly once under the distinct 🧭 GH-OPS · OSS RADAR title, while unchanged opportunities stay silent. Security findings are likewise deduplicated across runs via the security_notified resource key (findings live under security; only actionable severities are batch-delivered, marked only after successful delivery; recovery events clear the mark and may send a RESOLVED report).

Secrets live in environment variables, never in source or config. For local development you can keep them in a gitignored .env file:

cp .env.example .env

.env is gitignored (.gitignore: .env, .env.*) — confirm with git check-ignore -v .env. Only .env.example, which holds no values, is tracked.

Loading rules (src/utils/env.py, standard library only — no new dependency):

• The process environment always wins. .env never overrides a variable that is already set, so a CI secret cannot be shadowed by a stale local file.
• Values are never logged or printed. load_env_file() returns variable names.
• A missing .env is a no-op, so production runs are unaffected.
• Comment lines, blank lines, an export prefix, quoted values, and # inline comments are supported.

python scripts/check_telegram.py reports readiness using booleans, counts, and the token length only — it never prints the token, so its output is safe to paste into a terminal or a CI log. It exits non-zero when delivery is not ready.

Daily and weekly briefs gain one advisory 🧠 headline line — an addition, not a replacement: event collection, ordering, deduplication, priority, lifecycle, delivery, and every contract test stay exactly as they were, and the brief is byte-identical to the deterministic rendering whenever interpretation is unavailable.

built brief (deterministic)
  → intelligence context: evidence pack → temporal state → correlations
      → deterministic focus (attention | security | operations | … | quiet)
      → non-meaningful/quiet context stops here: no model is consulted
  → Laya System-1 triage proposes a label from a fixed allow-list
      (local, lazy, one ~2.2 GB checkpoint, cached, runs only at brief time)
  → reconcile: deterministic attention/security always win; Laya "quiet"
      may only silence a non-urgent focus
  → "quiet" skips the LLM; otherwise the cloud LLM writes one ≤60-word
      sentence (provider chain groq → gemini, OpenAI-compatible, env credentials)
  → grounding: every URL, repository, and item number in the sentence must
      exist in the evidence pack, else the claim is dropped
  → strict validation (length, printable text, no JSON, no internal strings)
  → 🧠 line inserted after Coverage, display-only

Rules:

• Fully functional with neither layer: no keys, dead provider, timeout, or a missing Laya install all collapse to None and the brief renders unchanged.
• A context that is not meaningful (quiet) never reaches either layer, so a calm day costs zero model calls.
• Laya's numeric score is an uncalibrated model score, never a probability.
• Laya is advisory: its label is validated against a fixed allow-list before use; it never mutates state, executes anything, or alters event output.
• Credentials come from the environment only (.env is gitignored; never committed, never logged). No Ollama, no local LLM — cloud providers only.
• GitHub Actions runs stay deterministic: workflows configure no LLM keys and never install the Laya extra, so interpretation is a no-op there by design.

gh-ops resolves GitHub tokens from environment variables:

• Token is never logged, printed, or serialized
• AuthConfig.__repr__ shows only the source variable name, not the token
• If no token is found, resolve_auth() raises GhOpsError
• Whitespace-only tokens are treated as empty

src/github/client.py is the single HTTP exit point for all GitHub API communication.

gh-ops V1 is read-only with respect to GitHub. The client enforces this:

• client.execute("POST", ...) raises WriteDeniedError
• client.execute("PUT", ...) raises WriteDeniedError
• client.execute("PATCH", ...) raises WriteDeniedError
• client.execute("DELETE", ...) raises WriteDeniedError

Only GET requests are permitted. This is a hard boundary — no collectors, handlers, or utilities can bypass it.

client.get_paginated() handles pagination automatically:

• Follows Link header rel="next" references
• Configurable per_page (max 100) and max_pages (safety limit, default 10)
• Returns PaginatedResult with all items collected across pages
• Supports ETag conditional requests (304 Not Modified)

• Pass etag parameter to get_paginated() for conditional requests
• On 304 Not Modified, returns PaginatedResult with not_modified=True
• ETag values are passed through for the state layer to persist (Phase 3)

• Reads X-RateLimit-Remaining, X-RateLimit-Limit, X-RateLimit-Reset headers
• Updates RateLimitTracker on every response
• On 429 (rate limited): parses Retry-After header, retries with exponential backoff
• On 5xx (server errors): retries with exponential backoff + jitter
• Exponential backoff: min(1.0 * 2^attempt + random(0, 0.5), 30.0) seconds
• Maximum 3 retries (configurable via max_retries)
• on_rate_limit callback for monitoring integration

All errors are typed GitHubClientError instances with status code, endpoint, and rate-limit metadata.

Collectors are thin wrappers that call client.get() or client.get_paginated() and return normalized models.

• No requests import — collectors use only GitHubClient and models
• No credential access — collectors never see the token
• No HTTP logic — pagination, retries, rate limits are in client.py
• Deterministic parsing — Model.from_api() handles all field extraction
• Frozen dataclasses — models are immutable after creation

All API responses are parsed into frozen dataclasses in src/github/models.py:

All models provide to_dict() for serialization and from_api() for parsing.

gh-ops V1 performs zero write operations against the GitHub API.

• No issue creation, commenting, or labeling
• No pull request creation, merging, or reviewing
• No release creation or asset uploading
• No workflow triggering
• No repository starring, watching, or forking

The WriteDeniedError enforcement in client.py makes this a hard guarantee, not a convention.

gh-ops implements a deterministic state engine that detects changes between collection runs.

diff_resources() produces NEW/CHANGED/REMOVED/UNCHANGED events with field-level change tracking:

from src.core.events import diff_resources

events = diff_resources(
    resource_type="issues",
    previous_items={"#1": {"state": "open", "title": "bug"}},
    current_items={"#1": {"state": "closed", "title": "bug"}},
)
# events[0].event_type == EventType.CHANGED
# events[0].changes == (FieldChange(field="state", before="open", after="closed"),)

A failed collector MUST NOT cause valid previous state to disappear:

• SUCCESS — overwrite that collector's items
• NOT_MODIFIED — preserve existing items (304 response)
• FAILURE — preserve previous items, record the failure

• Schema versioning with STATE_SCHEMA_VERSION = 1
• Atomic writes: temp file → flush → fsync → atomic replace
• Corruption handling: MISSING / VALID / CORRUPT / UNSUPPORTED_VERSION
• ETag persistence for conditional requests

from src.core.state import load_current_state, apply_snapshot
from src.core.models import Snapshot, CollectorResult, CollectorStatus

# Load previous state
previous = load_current_state()

# Apply a snapshot (partial failure safe)
snapshot = Snapshot(results={
    "repos": CollectorResult(collector="repos", status=CollectorStatus.SUCCESS, items={...}),
    "releases": CollectorResult(collector="releases", status=CollectorStatus.FAILURE, error="timeout"),
})
new_state = apply_snapshot(previous, snapshot)

src/developer/ turns data already collected by Phase 2/3 into factual, descriptive activity statistics. It performs no network calls of its own, introduces no LLM, and makes no judgement about the developer.

CurrentState (resources) + Phase 3 events
        ↓
activity.py     — normalize into ActivityRecord
        ↓
statistics.py   — descriptive counts and breakdowns
        ↓
reports.py      — structured DeveloperReport
        ↓
Phase 7 Telegram delivery (escaped, split, per-chat isolation)

A record is emitted only when the corresponding resource field exists and parses to a UTC datetime. Nothing is fabricated.

• Closed PR ≠ merged PR. pr_merged requires the source data to establish a merge (merged true / merged_at present). A closed PR without merge evidence is pr_closed.
• Current state ≠ historical event. Timestamps come from the resource fields (created_at, closed_at, merged_at), never from collection time.
• Missing ≠ zero. A failed collector produces a DataQuality entry with is_complete=False and its error message; the report reports completeness rather than reporting zero activity.
• First observation. When an issue is already closed, or a PR is already merged, at first observation, both the create (or open) and the close/merge are emitted using the resource's own timestamps.

Pure aggregation over ActivityRecord lists — no I/O:

• ActivityCounts — per-type counts plus total
• compute_by_repository() — RepoActivity per repository
• compute_by_period() — fixed-width PeriodActivity buckets
• compute_active_days(), compute_active_repositories(), compute_date_range()
• summarize_data_quality() — collector completeness percentage

ReportPeriod presets: last(days), this_week() (Monday 00:00 UTC), and this_month() (1st 00:00 UTC).

generate_report() deduplicates records, bounds them to the period (since and until both inclusive), then computes counts, the per-repository breakdown, by_period buckets, and data-quality context.

The aggregation is a pure function of its input:

• Frozen dataclasses; no randomness in aggregation
• Records sorted by timestamp; by_repository sorted by total descending then repository name; by_period chronological; active_repositories sorted
• Dedup key is (activity_type, item_id, timestamp)

Two values are generation-time by construction and differ between runs: generated_at, and the trailing by_period bucket's period_end (clamped to the reference time). Pass an explicit reference_time to compute_by_period() to make bucket boundaries reproducible.

• Coverage is bounded by what Phase 2/3 collected; there is no backfill
• Workflow runs are not attributed to a developer, so per-user reports still include collected workflow runs
• Releases and workflow runs carry no repository name in the normalized record, so they group under (unknown) in per-repository breakdowns
• A PR already closed-without-merge at first observation yields only pr_opened, because that close transition was never observed
• A report describes one period from one snapshot; no history or trends

src/notifications/telegram.py delivers already-computed results to Telegram. It is a thin outbound delivery layer: it does not query GitHub, contains no intelligence or scoring, does not mutate state, performs no GitHub writes, accepts no inbound control, and calls no LLM/agent/MCP/UEA runtime.

structured results (MonitorResult / Opportunity / DeveloperReport)
        ↓
formatters         — escaped MarkdownV2, split to fit Telegram's limit
        ↓
TelegramTransport  — finite timeout, bounded retries, 429 / Retry-After
        ↓
Telegram Bot API   — https://api.telegram.org (fixed module constant)

• Read from TELEGRAM_BOT_TOKEN only — never from config, CLI arguments, or state
• Config that sets bot_token_ref to any other variable is rejected at load time
• The token is never logged, never placed in errors or error context, and never persisted
  • The token is embedded in the request URL, so raw URLs are excluded from logs and errors
  • Raw exception text is excluded too, since third-party exceptions can embed the URL
• logging.py additionally redacts Telegram token patterns as defence-in-depth

telegram:
  enabled: false
  bot_token_ref: "TELEGRAM_BOT_TOKEN"
  chat_ids:
    - chat_id: 123456789
      topics: [alerts, oss_opportunities]
    - chat_id: 987654321      # no topics = receives everything
  message:
    max_length: 4096
    parse_mode: "MarkdownV2"
    retry_attempts: 3
    retry_delay_seconds: 5
    timeout_seconds: 10

• Attempts are bounded at retry_attempts + 1; every request has a finite timeout
• Delivery is a safe no-op when disabled, when no chat matches the topic, or when no token is set — it never raises for a delivery problem
• Failures are isolated per chat: one failing chat does not block the others, and a failing chat receives no further messages in that run

• All text — including the message title — goes through escape_markdown_v2, so a hostile repository name or issue title cannot open or close a MarkdownV2 entity
• Blocks are escaped independently and then packed, so packing never splits an escape sequence; oversized blocks are split on their escaped form with a dangling backslash carried onto the next chunk
• Every message is at most max_length characters, even when escaping doubles the text
• Output is byte-identical for identical input

• Outbound only: no commands, no polling, no webhooks
• URLs are sent as escaped plain text rather than inline links
• Digest scheduling (daily/weekly jobs) belongs to Phase 8; this phase provides formatting and delivery only

Actions is the scheduler; the application stays in Python.

trigger (cron / workflow_dispatch)
        ↓
python -m src.jobs          ← job name read from GHOPS_JOB
        ↓
core/dispatcher.py
        ↓
src/jobs/jobs.py            ← collect → state/events → monitors/intelligence/developer
        ↓
Telegram                    ← outbound only

The only substantive step in any workflow is python -m src.jobs. No expression is interpolated into a shell command, and no business logic lives in YAML.

Schedules are staggered so no two state-writing jobs share a minute (monitoring uses :30, others :00/:30 on non-overlapping slots). IST = UTC+5:30 (fixed offset; no tzdata dependency).

All six also support manual dispatch. Every job has a 25-minute timeout.

Every workflow declares explicit read-only scopes. Nothing has write.

Note the key is security-events (hyphen); security_events is not a valid permission name.

Both are provided through the env: block only — never as CLI arguments, never in a cache key, never in run: text. Add TELEGRAM_BOT_TOKEN under Settings → Secrets and variables → Actions. GITHUB_TOKEN is automatic.

Actions cache carries data/state/ between runs. It is a rolling snapshot, not a database:

• Restore uses the shared prefix gh-ops-state-, which resolves to the most recently created snapshot regardless of which workflow wrote it.
• Save uses a fresh immutable key (gh-ops-state-<job>-${{ github.run_id }}-${{ github.run_attempt }}), because cache entries cannot be overwritten.
• A ${{ github.run_id }}-only key would be unusable as the restore path, since the next run can never hit it exactly.
• All state-writing workflows share one concurrency: gh-ops-state group with cancel-in-progress: false, serializing the read-modify-write cycle against the single shared state file.

A job invocation never exits 0 without having executed a job.

• The state cache is evicted after 7 idle days of no runs, and lives in a shared 10 GB LRU pool that grows by one entry per successful run.
• GitHub runs at most one running plus one pending run per concurrency group; a third queued run replaces the pending one.
• GITHUB_TOKEN may be refused by the Dependabot alerts endpoint. When that happens the collector is recorded as a partial failure and previous state is preserved — the system never reports "no alerts" for a check it could not run.

All tests use mocked HTTP responses — no live GitHub contact:

# Run full test suite
python -m pytest tests/ -v

# Run specific module tests
python -m pytest tests/github/ -v
python -m pytest tests/core/ -v

# Validate the workflow YAML, permissions, pins, and state strategy
python -m pytest tests/ci/ -v

Workflow validation is static: it parses .github/workflows/*.yml and asserts the security and state invariants documented below. It never triggers a workflow or contacts GitHub.

The post-Phase-8 hardening suites add Telegram transport-boundary contracts, unified reporting, security-intelligence delivery, cross-run deduplication, repository/event lifecycle hardening, and the final response-UX batch (attention-first briefs, gh-ops system-workflow classification, data-quality ledger gating). The deterministic intelligence batch adds the evidence-pack, temporal, correlation, attention, grounding, and narrative suites plus job-wiring tests.

The portfolio is actively contributing upstream alongside GH-OPS maintenance. The latest open PRs are:

• LlamaIndex #23260 — preserves originally retrieved node scores during overlapping Prev/Next expansion, with regression coverage.
• Pydantic AI #8823 — closed by the upstream issue-assignment guard before merge; Macroscope approved the implementation, but issue #8778 was not assigned.

These are tracked in the profile-level OSS ledger; GH-OPS itself remains FROZEN / maintenance mode.

gh-ops/
├── src/
│   ├── core/           # Config, state, events, errors, rate limiting, monitor model
│   ├── github/         # API client, auth, collectors, models
│   │   ├── auth.py     # Token resolution (env vars only)
│   │   ├── client.py   # Single HTTP exit point (GET-only)
│   │   ├── models.py   # Frozen dataclasses for API responses
│   │   └── collectors/ # Thin data-fetching wrappers
│   ├── intelligence/   # OSS hunter, radars, evidence/temporal/attention layer
│   ├── developer/      # Personal activity, statistics, reports
│   ├── monitors/       # Repository, CI, release, security, endpoint monitors
│   │   ├── __init__.py # Monitor registry and evaluator
│   │   ├── repository.py # Repository metadata monitoring
│   │   ├── ci.py       # CI/workflow failure/recovery detection
│   │   ├── release.py  # Release event monitoring
│   │   ├── security.py # Security alert change evaluation
│   │   └── endpoint.py # HTTP endpoint health checks (SSRF-protected)
│   ├── notifications/  # Outbound Telegram delivery (Phase 7)
│   ├── reporting/      # Ledger, priority, briefs, narratives
│   ├── jobs/           # Thin job orchestration for Actions (Phase 8)
│   │   ├── __main__.py # python -m src.jobs <job_name>
│   │   ├── pipeline.py # Shared plumbing: config, collection, state, diff
│   │   └── jobs.py     # The six jobs + registration
│   └── utils/          # Logging, text, time helpers
├── .github/workflows/  # Six Actions workflows (thin: they call python -m src.jobs)
├── config/             # YAML configuration files (monitoring.yml, oss_hunter.yml, telegram.yml)
├── .env.example        # Tracked template; real .env is gitignored
├── data/               # Runtime state (gitignored, carried by Actions cache)
├── tests/              # Test suite (1495 tests)
└── scripts/            # Local development scripts

• Python 3.10+
• requests — GitHub API HTTP client (client.py only)
• pyyaml — Configuration file parsing

MIT