Worktrees and shared indexes

When you work in several linked Git worktrees of one repository, most of their code is identical. greppy indexes this shared code once, in an immutable Base, and gives each worktree a private Delta with exactly the paths that differ.

greppy index .                  # the first linked worktree builds the Base, every further one indexes its Delta alone
greppy index status --json      # phase, progress and readiness, without blocking
greppy index --agent-worktree   # build or check the built-in agent's Base ahead of time

An example with serde

The example uses a copy of serde 1.0.229 as the main checkout, plus one linked worktree per task:

git worktree add ../serde-feature -b feature
cd ../serde-feature && greppy index .

The first linked worktree builds the Base. greppy index says which Base it bound and whether it built it (created) or reused it (reused).

greppy: index started for ~/src/serde-feature (phase=preparing_base, pid=43356); progress: `greppy index status --json`
greppy: index — writing_graph: 41/219 files; phase ETA about 60s; pid 43356
greppy: index — writing_graph: 134/219 files; phase ETA about 20s; pid 43356
…
greppy: index — base_graph_ready: steps total unknown; phase ETA unavailable until total is known; pid 43356
greppy index: linked worktree uses shared Base dc1b99a0c8ed at 7fc3b4c30c94f73a96ebd1553f2b090d928fc3a8 (created); only the Git/dirty Delta will be indexed
greppy: index — finalizing_graph: 1/1 steps; phase ETA complete; pid 43356
indexed Delta generation 1: 0 changed/deleted paths, 0 private nodes (project: serde)

Every further worktree on the same version binds the finished Base and indexes nothing but its Delta.

greppy: index started for ~/src/serde-fix (phase=preparing_base, pid=55352); progress: `greppy index status --json`
greppy index: linked worktree uses shared Base dc1b99a0c8ed at 7fc3b4c30c94f73a96ebd1553f2b090d928fc3a8 (reused); only the Git/dirty Delta will be indexed
indexed Delta generation 1: 0 changed/deleted paths, 0 private nodes (project: serde)

When a worktree changes a file, that file alone lands in the Delta. Here Buf in serde_core/src/format.rs gets two new methods, len and remaining:

greppy: index started for ~/src/serde-feature (phase=preparing_base, pid=52216); progress: `greppy index status --json`
greppy index: linked worktree uses shared Base dc1b99a0c8ed at 7fc3b4c30c94f73a96ebd1553f2b090d928fc3a8 (reused); only the Git/dirty Delta will be indexed
indexed Delta generation 2: 1 changed/deleted paths, 8 private nodes (project: serde)

Queries see the combined view of the Base and their own Delta. The worktree that contains the new method finds it, and no other worktree does:

$ cd serde-feature && greppy search-symbol remaining
serde_core/src/format.rs:20  Buf::remaining  method
$ cd ../serde-fix && greppy search-symbol remaining
status: no_matches
scope: indexed definitions in the repository
message: no definition named `remaining`
…

On an Apple M5 Mac on October 7, 2026, the three runs measured as follows:

Base and Delta

The Base holds the unchanged repository data exactly once. It is identified by its content: its identity includes the Git tree, the store schema and indexer versions, and the contracts of the summary and embedding models. An incompatible identity selects a new Base instead of migrating an old one in place.

Every worktree and every agent run gets a writable private Delta with exactly the paths that differ from the Base: changed, deleted, renamed or new files, committed or not.

• The binding between a worktree and its Base is stored per worktree and stays pinned when the main checkout moves on. In the example, the main checkout now sits on a newer commit, while serde-feature keeps working on Base dc1b99a0c8ed.
• Further worktrees use a verified Base without a new checkout and without a full pass through the repository.
• greppy publishes a Base atomically. Readers hold leases that protect a Base from removal while it is in use.
• Corrupt or incomplete Bases go into quarantine and are rebuilt under an exclusive lease. If preparing a Base fails, greppy reports it; a private store does not silently replace it.

doctor, diagnostics and index status --json report the selected mode (overlay for a worktree on a shared Base, single for a private store), the Base and Delta identities, completeness, cache hits, the number of changed paths and any reason for a fallback.

status: ok
root: ~/src/serde-feature
…
store_mode: overlay
base_identity: dc1b99a0c8ed570e166b806f59b11821a79ff46b56616d78cdb5e05331cefed8
embedding_complete: true
…
generation: 2
stats: files=339 nodes=4575 edges=9430
…
dirty_overlay: total=1 staged=0 unstaged=1 untracked=0 deleted=0 renamed=0 ignored=0

Inference is shared too

Embeddings and summaries are cached user-wide, keyed by the exact model, the prompt, the task profile and the input. A linked worktree or an agent uses the vectors that are already computed; changed Delta content is the one thing that costs inference. The index output and the live status count local reuse, global cache hits and genuine inference misses separately.

indexed 219 files (120 unsupported, 0 unreadable, 0 oversize, 0 file-limit, 0 time-budget); 4571 nodes extracted; generation 1 (project: serde)
embedded 2443 code spans (0 local-store reused, 2443 global-cache hits, 0 inference misses, 4573 considered, 2150 non-definition skipped, 0 missing-file, 0 invalid-span, 0 oversize, 0 failed, 0 stale pruned)

Waiting for another builder

A Base is always built by a single process. If another linked worktree is building it right now, greppy index waits (the status then shows the phase waiting_for_base_builder) and afterwards uses the published Base instead of building a second copy.

In the example, greppy index . starts in serde-w3 and builds a new Base; ten seconds later the same command starts in serde-w4 at the same commit.

$ cd serde-w3 && greppy index .
…
greppy index: linked worktree uses shared Base bcfab63b0bf9 at d8bb8f787170a832eaec9eef4aa26bcd9512cecf (created); only the Git/dirty Delta will be indexed
$ cd serde-w4 && greppy index .
…
greppy index: linked worktree uses shared Base bcfab63b0bf9 at d8bb8f787170a832eaec9eef4aa26bcd9512cecf (reused); only the Git/dirty Delta will be indexed

serde-w4 waited and took over the Base that serde-w3 built.

greppy index status --json      # never blocks, not even while another process is building

While a build is running, index status --json exits with code 75 and reports the current state. A status whose progress has not moved for too long carries "progress_stale": true. The threshold is 120 seconds, 300 seconds while preparing a Base (preparing_base) and 600 seconds while loading the model (loading_model).

{
  …
  "background_state": "refreshing",
  …
  "status": "indexing",
  …
  "writer_active": true
}

Where the data lives

By default, the store lives in greppy's cache directory outside your repository. For another volume, you set GREPPY_STORE_DIR to an absolute directory before indexing and use the same value for status, queries and later edits:

GREPPY_STORE_DIR=/absolute/volume/greppy-store greppy index .
GREPPY_STORE_DIR=/absolute/volume/greppy-store greppy index status --json

The store then lives under GREPPY_STORE_DIR/workspaces/. The variable selects a different store; it does not move or delete an existing one. TMPDIR controls temporary files, not the location of the index, and XDG_CACHE_HOME does not select it either.

Models and inference caches follow GREPPY_STORE_DIR unless GREPPY_SHARED_INFERENCE_ROOT is set. If you use several isolated stores, point them all at one shared inference root on the volume you want, and model and cache stay shared.

Confidentiality

Base stores and Deltas contain paths, spans, graph relations, summaries and embeddings from the source code. Protect them like the repository itself. Agent sandboxes read a published Base but do not write it; writable Delta state stays isolated per run.

Clean up

greppy cache status --json        # stores, Bases, models, sizes, locks
greppy cache clear --root . --yes # this worktree's store and the repository's shared agent Bases
greppy cache gc --dry-run         # run the TTL/LRU garbage collector without deleting anything

cache clear --root also selects shared Bases that other linked worktrees of the same repository use. Without --yes, it removes nothing. Locked stores are kept and reported. See Local data, privacy and uninstall.

Scope

The graph namespace is per repository: one root, one Base plus private Deltas. Multi-package workspaces inside this root (Cargo, npm, Go) already form one shared relation space. You query several repositories with --root per repository; a federated query space across several roots is a separate, later project.