Korgalore Changelog
===================

v0.6.1 (2026-06-23)
-------------------

Bug Fixes
~~~~~~~~~

* Fix thread tracking not delivering new messages after a few days

  - Thread tracking used a ``msgid:<msgid>`` query combined with
    ``--threads``, which interacted badly with how ``lei update`` limits
    queries to ~2 days before the last retrieved message
  - Once a thread kept receiving messages past that window, the date filter
    caused the query to return nothing and delivery silently stopped

* Fall back to ``--depth=1`` when a shallow fetch fails on stale repos

  - When a feed's epoch repository received no commits within the
    ``--shallow-since=1.week.ago`` window, git aborted the fetch with
    "error processing shallow info: 4" (exit 128) and skipped the feed
  - ``update_feed()`` now mirrors ``clone_epoch()`` and retries with
    ``--depth=1`` so the fetch self-heals instead of surfacing a cryptic
    error

* Fix noop retry rewind causing infinite re-delivery of messages

  - The noop (rm/purge) retry path failed to forward the ``was_failing``
    flag, so retried entries were never removed from the failed list
  - ``mark_successful_delivery`` also rewound the "last" commit pointer for
    retried commits, causing all subsequent commits to be re-delivered on
    every pull cycle (hundreds of duplicate messages per sync)

* Fix crash on a bad-object commit during failed delivery retry

  - When a commit in the failed delivery list was no longer available
    locally (missing packfile), ``is_noop_commit()`` mistook it for a noop
    and the downstream ``save_delivery_info()`` crashed
  - ``is_noop_commit()`` now verifies the commit object exists first and
    raises ``GitError`` for genuinely missing commits, which is caught and
    recorded as a failed delivery instead of crashing the retry loop

v0.6.0 (2026-03-05)
-------------------

New Features
~~~~~~~~~~~~

* ``track-subsystem --list`` (``-L``) flag to display all tracked subsystems

  - Shows the subsystem name, configuration file path, and delivery details
    (target and labels) for each tracked subsystem
  - Subsystem name is read from the ``[subsystem]`` section in each conf.d
    configuration file
  - Falls back to deriving the display name from the filename for legacy
    configurations without a ``[subsystem]`` section

* ``[subsystem]`` metadata section in generated conf.d configuration files

  - Stores the human-readable subsystem name (e.g., ``name = 'SELINUX
    SECURITY MODULE'``)
  - Used by ``--list`` for display and by ``--forget`` for log messages

* ``kgl subscribe`` command group for managing mailing list subscriptions

  - ``subscribe add URL`` validates and subscribes to a public-inbox feed
    (lore.kernel.org or any public-inbox server) or a local lei search
  - ``subscribe list`` shows all active and paused subscriptions
  - ``subscribe pause/resume`` toggles delivery without losing feed data
  - ``subscribe resume --skip`` discards messages received while paused
  - ``subscribe stop`` removes the subscription; ``--delete`` also removes
    feed data
  - The ``add`` subcommand is the default, so ``kgl subscribe URL`` works
    as a shorthand for ``kgl subscribe add URL``
  - Subscriptions are stored as individual ``conf.d/sub-{feed_key}.toml``
    files, keeping them separate from the main configuration
  - Duplicate detection checks both conf.d files and the main configuration

Enhancements
~~~~~~~~~~~~

* ``--target`` / ``-t`` now defaults to the first configured target

  - Applies to ``yank``, ``track add``, ``subscribe add``, and
    ``track-subsystem``
  - Previously, commands with multiple configured targets required an
    explicit ``-t`` flag or they would abort
  - Matches the existing GNOME applet behaviour, which already
    pre-selects the first target

Bug Fixes
~~~~~~~~~

* Fix non-lore feed URLs creating invalid directory paths

  - ``normalize_feed_key()`` returned raw URLs (e.g.,
    ``https://someothersite.com/foo``) as-is, resulting in a ``https:``
    directory under the data path
  - URLs are now sanitised by stripping the scheme, replacing special
    characters with hyphens, and falling back to a hash-based name for
    very long URLs

* Fix ``track-subsystem`` crash when a lei query matches no messages

  - ``get_first_commit()`` now detects empty repositories and returns an
    empty string instead of raising a ``GitError``

* Cache ``is_empty_repo()`` check to avoid redundant git subprocesses

  - Result is cached per epoch and cleared on ``feed_unlock()``

* Skip public-inbox commits that carry no message file

  - Public-inbox v2 repositories can contain commits without an ``m``
    (message) blob: ``rm`` commits record message removals (they store
    the deleted message in a ``d`` file instead), and ``purged`` commits
    record content scrubbing via ``replace_oids()``
  - Previously, attempting to extract a message from these commits raised
    ``StateError``, which was caught as a delivery failure and retried
    indefinitely
  - Both types are now detected by checking for the absence of the
    ``m`` object in the commit tree and treated as successful no-ops

* Handle remote errors gracefully during feed updates

  - When a remote server returns a transient error (e.g. 503 Service
    Unavailable), ``kgl pull`` now logs a one-line warning instead of
    dumping a full Python traceback
  - A failure on one feed no longer aborts updates to the remaining feeds

* Fix new commit discovery failing on shallow clones

  - The ``--ancestry-path`` flag in ``git rev-list`` requires an unbroken
    parent chain between endpoints, which cannot be established in shallow
    clones where parent links are severed at graft boundaries
  - Since public-inbox v2 epoch repositories have linear history,
    ``--ancestry-path`` provides no filtering benefit and is now removed

* Fix crash in legacy migration when git directory has no epoch repos

  - If a previous clone was interrupted, the ``git/`` directory could exist
    without any epoch subdirectories, causing ``_perform_legacy_migration()``
    to raise an unhandled ``PublicInboxError``
  - The migration now returns early when no epochs are found, allowing the
    normal feed initialisation path to proceed

* Initialise delivery state after first feed clone

  - When a new feed was cloned on the first ``kgl pull``, delivery state
    was not set up until the next run, wasting one full pull cycle
  - ``update_all_feeds()`` now returns both updated and initialized feed
    lists, and ``perform_pull()`` immediately creates delivery state for
    newly cloned feeds

* Fix crash in ``kgl pull`` after ``kgl track add``

  - ``map_tracked_threads()`` stored a 3-tuple ``(feed, target, labels)``
    but ``retry_all_failed_deliveries()`` and ``perform_pull()`` expected a
    4-tuple ``(feed, target, labels, subfolder)``
  - Tracked thread deliveries now include ``subfolder=None`` to match the
    format used by regular deliveries

* Include stderr and exit code in git error messages

  - ``run_git_command()`` only returned stdout, but git writes errors to
    stderr, resulting in empty error messages (e.g., "Git clone failed:")
  - The function now returns stderr alongside stdout, and all error
    messages include the exit code

* Fix shallow clone failure for dormant lists

  - Cloning with ``--shallow-since=1.week.ago`` fails for lists with no
    recent messages, because git cannot establish a shallow boundary when
    no commits fall in the time window
  - The clone now falls back to ``--depth=1`` when the time-based shallow
    clone fails

* Improve GUI startup error reporting and debug logging

  - Missing GTK dependency error is now shown before stdout/stderr are
    redirected to ``/dev/null``, so users see it without ``-v DEBUG``
  - Detect pipx environments and show ``pipx install "korgalore[gui]"``
    instead of the pip command
  - Running with ``-v DEBUG`` keeps stdout/stderr connected and preserves
    the log level for easier troubleshooting

* GUI: automatically detect and reload configuration files changed on disk

  - Before each sync cycle, the GUI now checks mtimes of the main config
    file and all ``conf.d/*.toml`` files
  - If any file has been modified (e.g., by ``kgl track-subsystem`` or
    manual editing), the configuration is validated and reloaded before
    syncing
  - Invalid configuration changes are logged and skipped, keeping the
    previous working configuration
  - Prevents crashes caused by stale in-memory configuration after
    external config changes

v0.5 (2026-01-28)
-----------------

New Features
~~~~~~~~~~~~

* JMAP and IMAP message deduplication by Message-ID

  - Before importing a message, targets now check if a message with the same
    Message-ID already exists in the target folder
  - Prevents duplicate messages when the same email arrives via multiple
    mailing lists or deliveries
  - Uses JMAP ``Email/query`` with header filter for JMAP targets
  - Uses IMAP ``SEARCH HEADER Message-ID`` for IMAP targets
  - Gmail already deduplicates automatically via its API

* New ``RawMessage`` class for unified message handling

  - Shared message wrapper used by all targets (JMAP, IMAP, Gmail, Maildir, Pipe)
  - Lazy parsing with cached properties (``message_id``, ``parsed``)
  - ``as_bytes()`` method for consistent message transformation before delivery
  - Centralised CRLF line ending normalisation

* X-Korgalore-Trace header injection

  - Messages now include provenance information when delivered to targets
  - Header format: ``X-Korgalore-Trace: from feed=<feed> for delivery=<delivery>;
    v<version>; <RFC 2822 date>``
  - Helps identify message origin when the same email arrives via multiple paths
  - Header is inserted at the end of the header section before delivery

* Subfolder support for IMAP and Maildir targets

  - New ``subfolder`` parameter at the delivery level allows routing messages to
    a target folder/directory without creating separate targets for each folder
  - For IMAP: combines base folder with subfolder (e.g., ``INBOX`` + ``Lists/LKML``
    becomes ``INBOX/Lists/LKML``)
  - For Maildir: creates maildir subdirectories under the base path
  - Subfolder maildirs are cached for efficient reuse within a session
  - JMAP, Gmail, and Pipe targets ignore subfolder (use ``labels`` for JMAP/Gmail)
  - Maildir subfolders support strftime format codes (e.g., ``%Y/%m``) for
    date-based directory organisation; templates are validated at startup and
    refreshed before each sync (ensuring correct folder for long-running GUI)

  Example configuration::

    [deliveries.lkml]
    feed = 'lkml'
    target = 'imap-server'
    subfolder = 'Lists/LKML'

    # Maildir with date-based archiving
    [deliveries.archive]
    feed = 'lkml'
    target = 'local-maildir'
    subfolder = 'Archive/%Y/%m'  # e.g., Archive/2026/01

* New ``main.user_agent_plus`` configuration option to append a unique identifier
  to the User-Agent string sent to remote servers. This helps server operators
  identify traffic from specific korgalore installations and may be used to
  prioritize requests.

* New ``main.catchall_lists`` configuration option to customize which mailing
  lists are excluded from ``track-subsystem`` queries. Default excludes
  ``linux-kernel@vger.kernel.org`` and ``patches@lists.linux.dev``.

Improvements
~~~~~~~~~~~~

* ``track-subsystem`` no longer requires ``-m/--maintainers``. The command now
  automatically looks for ``./MAINTAINERS`` in the current directory, and falls
  back to fetching from kernel.org if not found. The fetched file is cached for
  24 hours.

* ``track-subsystem`` now excludes catch-all mailing lists (linux-kernel,
  patches@lists.linux.dev) from queries by default to avoid flooding results
  with irrelevant messages. Override via ``main.catchall_lists`` in config.

* Centralized User-Agent handling across all remote connections (git, lei, HTTP).
  All requests now consistently identify as korgalore with the current version.

* Git availability is now checked at startup with a friendly error message if
  git is not installed or fails to run.

* GUI now redirects stdout/stderr to /dev/null and logs only critical messages
  to the systemd journal. This prevents I/O errors when the terminal is closed
  after launching with ``kgl gui &``.

Bug Fixes
~~~~~~~~~

* Maildir target now creates parent directories if they don't exist. Python's
  ``mailbox.Maildir`` only creates the maildir structure (cur/new/tmp), not
  parent directories, causing failures when using paths like ``~/Mail/rfl``
  where ``~/Mail`` exists but ``rfl`` does not.

v0.4 (2026-01-20)
-----------------

New Features
~~~~~~~~~~~~

* IMAP OAuth2 Authentication for Microsoft 365

  - Support for modern OAuth2 authentication (XOAUTH2) with Microsoft 365
  - Uses PKCE authorization flow for secure desktop application authentication
  - Built-in Azure AD application ID for zero-configuration setup
  - Custom client_id support for organizations that block third-party apps
  - Automatic token refresh with secure local storage
  - Full GUI support for re-authentication when tokens expire

  Example configuration::

    [targets.office365]
    type = 'imap'
    auth_type = 'oauth2'
    server = 'outlook.office365.com'
    username = 'user@company.com'

Improvements
~~~~~~~~~~~~

* Close network connections after sync runs to avoid keeping idle connections
  open between periodic syncs (IMAP, HTTP sessions)

* JMAP target now uses shared requests session for consistent User-Agent
  header and proper connection cleanup

* Add GUI optional dependencies to pyproject.toml (``pip install korgalore[gui]``)

* Support AyatanaAppIndicator3 as fallback for AppIndicator3 (needed on
  Debian and derivatives)

* GUI now monitors network availability via Gio.NetworkMonitor and skips
  sync attempts when network is unavailable (airplane mode, resume from
  suspend, etc.). Sync is automatically scheduled when network is restored.

* GUI error messages now display actual error text instead of unhelpful
  "see logs" messages, since logs are not collected by default.

* Add hashed requirements file for GUI dependencies (requirements-gui.txt)

v0.3 (2026-01-15)
-----------------

New Features
~~~~~~~~~~~~

* GNOME Taskbar Application (``kgl gui``)

  - System tray status indicator with automatic background syncing
  - Configurable sync interval via ``[gui] sync_interval`` config option
  - Real-time status updates showing current feed/delivery being processed
  - "Sync Now" menu item for manual sync triggering
  - "Yank..." dialog to fetch messages by message-id or URL without terminal
  - "Edit Config..." menu item to edit configuration in preferred editor
  - "Edit Bozofilter..." menu item to manage blocked senders
  - Automatic TOML validation after config edits with live reload on success
  - Gmail re-authentication support: detects expired tokens and shows
    "Authenticate..." menu item to re-auth via browser without terminal
  - Deduplicates message count by Message-ID for accurate reporting
  - Desktop entry file (korgalore.desktop) for application menu integration

* Thread Tracking (``kgl track``)

  - Track specific email threads for ongoing updates without full list subscription
  - Subcommands: add, list, stop, pause, resume
  - Uses lei to create persistent searches that monitor threads
  - Tracked threads automatically updated during regular ``pull`` operations
  - Threads auto-expire after 30 days of inactivity (can be resumed)

* Subsystem Tracking (``kgl track-subsystem``)

  - Track Linux kernel subsystems by parsing MAINTAINERS file
  - Automatically creates lei queries for mailing list traffic and patches
  - Supports substring matching for subsystem names (case-insensitive)
  - ``--threads`` option to include entire threads when any message matches
  - ``--since`` option to control query date range (default: 7 days)
  - ``--forget`` flag to remove tracking and clean up lei queries
  - Configuration stored in conf.d/ for easy management

* Modular Configuration (conf.d)

  - Additional config files automatically loaded from ``~/.config/korgalore/conf.d/*.toml``
  - Files merged alphabetically into main configuration
  - Useful for separating auto-generated configs from manual configuration

* Pipe Target

  - New target type for delivering messages to external commands
  - Labels passed as command-line arguments to the pipe command
  - Useful for custom processing pipelines

* TOML Validation

  - ``edit-config`` command now validates syntax after editor closes
  - Reports errors with line numbers for easy debugging

* Bozofilter (``kgl bozofilter``)

  - Block messages from unwanted senders
  - Blocked messages are silently skipped during delivery
  - Simple text file format (one address per line, supports comments)
  - CLI options: ``--add``, ``--edit``, ``--list``, ``--reason``
  - GUI integration: "Edit Bozofilter..." menu item

Improvements
~~~~~~~~~~~~

* Gmail token handling: gracefully handle expired/revoked tokens instead of
  crashing, with clear error messages guiding re-authentication

* Suppress googleapiclient discovery cache warning by disabling file_cache

* Defer feed initialization messages until after progressbar completes for
  cleaner output

* Add comprehensive test suite with 300+ tests

* Optimize delivery lookup algorithms for better performance

* Labels can now be specified as comma-separated strings (e.g., ``-l INBOX,UNREAD``)
  in addition to repeated options (``-l INBOX -l UNREAD``)

Bug Fixes
~~~~~~~~~

* Fix crash when initializing a new feed for the first time

* Fix feed_lock failing when feed directory does not exist

* Drop dependency on python-dateutil (use stdlib datetime instead)

* Fix git commands failing with ``safe.bareRepository=explicit`` by using
  ``--git-dir`` instead of ``-C``

Documentation
~~~~~~~~~~~~~

* Add documentation for thread tracking feature

* Add comprehensive GUI documentation covering installation, usage,
  configuration, and systemd service setup

* Add documentation for subsystem tracking and conf.d modular configuration

* Add manpage (``man/kgl.1``) for distribution packages

* Add shell completion scripts for bash, zsh, and fish (``completions/``)

v0.2
----

Initial public release with support for:

* Gmail, JMAP, IMAP, and Maildir targets
* Lore.kernel.org and lei feed sources
* Basic pull and yank commands
