Filesystem Repository

Tags: repo, storage

© R.A.Sol

This spec defines filesystem storage for HPPR packets.

It follows packet rules in 010-PACKETS.md and address rules in 020-URC.md.

Repository Layout

repo/
├── hash/<Type>/<hh>/<tail>.H3
├── ref/B/<hh>/<tail>/...
├── ref/P/<hh>/<tail>/...
├── index/<group>/<api...>/||/<key...>/|/...
├── detach/<hash>
└── .tmp/

• hash/: packet storage by type B, P, S
• ref/: back-reference trees
• index/: coordinate index
• detach/: detached hashes
• .tmp/: atomic write staging

Filesystem requirements:

• MUST be case-sensitive
• MUST support UTF-8 names except / and NUL
• MUST support | in path names
• implementation MUST refuse startup if unsupported

Hash Storage (hash/)

Path form:

hash/<T>/<hh>/<tail>.H3

Where:

• <T> is B, P, or S
• <hh> is first two B64A digest chars
• <tail> is remaining digest chars

Storage format:

• Blob (B): raw content bytes only
• Plex (P): thin format
• Seal (S): thin format

Blob file size defines Data-Length during reconstruction.

Index Storage (index/)

Coordinate syntax uses //<group>/<api>//<key>. Filesystem storage does not store the second // as an empty path component. It maps the API/Key boundary to ||, which is forbidden in API and Key segments because | is forbidden.

Plex path:

index/<group>/<api...>/||/<key...>/|/plex/<tai>/<hash>

Seal path:

index/<group>/<api...>/||/<key...>/|/seal/<verifier>/<tai>/<hash>

Examples:

Rules:

• marker files are zero-byte
• filename/path encodes identity and version
• <api...> is the API header value split on /
• <key...> is the Key header value split on /
• <tai> uses seconds:subseconds form
• || is the filesystem API/Key boundary only
• | is the filesystem version-selector boundary

LIST Mapping

LIST over the API tree reads children before ||.

When an exact API has a Key tree, LIST of that API prefix includes // as the API/Key boundary marker. When an exact Key has versions, LIST of that Key prefix includes |/ as the version boundary marker.

The filesystem names || and | are not returned directly by LIST except as these coordinate markers.

Back-Reference Storage (ref/)

Blob to Plex references:

ref/B/<hh>/<tail>/<plex-hash>

Plex to Seal references:

ref/P/<hh>/<tail>/<seal-hash>/<verifier>

These support reverse traversal from inner to outer packet.

Store Flow

1. Write Blob raw bytes to hash/B/....
2. For Plex:
   • write thin Plex to hash/P/...
   • add index/.../|/plex/<tai>/<hash>
   • add ref/B/.../<plex-hash>
3. For Seal:
   • write thin Seal to hash/S/...
   • ensure embedded Plex and Blob exist
   • add index/.../|/seal/<verifier>/<tai>/<seal-hash>
   • add embedded Plex index entry
   • add ref/B/.../<plex-hash> and ref/P/.../<seal-hash>/<verifier>

Duplicates SHOULD succeed as idempotent writes. Use .tmp/ plus atomic rename for final placement.

Detach Cleanup

When detaching Plex or Seal:

1. remove index entries
2. remove matching ref/B/.../<plex-hash>
3. if that ref/B/... directory becomes empty, record blob hash in detach/
4. never detach the empty-blob constant hash

Reconstruction

• Blob: synthesize markline and Data-Length from file size + raw bytes.
• Plex/Seal: resolve embedded hashes from hash/ and rebuild full packet.

Hash verification requires full reconstructed packet bytes.

Tip Tracking

Tip links are required materialized state for the filesystem repository storage profile. They live under index/.../|/.

Required tip links:

• top tip: |/tip
• plex tip: |/plex/tip
• seal type tip: |/seal/tip
• per-signer seal tip: |/seal/<verifier>/tip

Tip update rule:

• compare (tai, hash) lexicographically
• newer value replaces existing link target

Tip Recovery

If a tip link is missing while version directories exist, implementation MUST:

1. scan affected |/plex/ and |/seal/ once
2. recompute latest entries
3. recreate missing tips

If scan finds no packets, empty |/ structure SHOULD be cleaned up.

After detach, recompute tip links for the affected coordinate.

Portability

• Atomic rename is required.
• If filesystem lacks symlinks, emulate tip links with files.
• Emulated tip links MUST be hidden from 🖧LIST exactly as symlinks are.