Remote/@git tracking branches

This is a plan to implement more Git-like remote tracking branch UX.

Objective

jj imports all remote branches to local branches by default. As described in #1136, this doesn’t interact nicely with Git if we have multiple Git remotes with a number of branches. The git.auto-local-bookmark config can mitigate this problem, but we’ll get locally-deleted branches instead.

The goal of this plan is to implement

proper support for tracking/non-tracking remote branches
logically consistent data model for importing/exporting Git refs

Current data model (as of jj 0.8.0)

Under the current model, all remote branches are “tracking” branches, and remote changes are merged into the local counterparts.

branches
  [name]:
    local_target?
    remote_targets[remote]: target
tags
  [name]: target
git_refs
  ["refs/heads/{name}"]: target             # last-known local branches
  ["refs/remotes/{remote}/{name}"]: target  # last-known remote branches
                                            # (copied to remote_targets)
  ["refs/tags/{name}"]: target              # last-known tags
git_head: target?

Remote branches are stored in both branches[name].remote_targets and git_refs["refs/remotes"]. These two are mostly kept in sync, but there are two scenarios where remote-tracking branches and git refs can diverge:
1. jj branch forget
2. jj op revert/restore in colocated workspace
Pseudo @git tracking branches are stored in git_refs["refs/heads"]. We need special case to resolve @git branches, and their behavior is slightly different from the other remote-tracking branches.

Proposed data model

We’ll add a per-remote-branch state to distinguish non-tracking branches from tracking ones.

state = new        # not merged in the local branch or tag
      | tracking   # merged in the local branch or tag
# `ignored` state could be added if we want to manage it by view, not by
# config file. target of ignored remote branch would be absent.

We’ll add a per-remote view-like object to record the last known remote branches. It will replace branches[name].remote_targets in the current model. @git branches will be stored in remotes["git"].

branches
  [name]: target
tags
  [name]: target
remotes
  ["git"]:
    branches
      [name]: target, state                 # refs/heads/{name}
    tags
      [name]: target, state = tracking      # refs/tags/{name}
    head: target?, state = TBD              # refs/HEAD
  [remote]:
    branches
      [name]: target, state                 # refs/remotes/{remote}/{name}
    tags: (empty)
    head: (empty)
git_refs                                    # last imported/exported refs
  ["refs/heads/{name}"]: target
  ["refs/remotes/{remote}/{name}"]: target
  ["refs/tags/{name}"]: target

With the proposed data model, we can

naturally support remote branches which have no local counterparts
deduplicate branches[name].remote_targets and git_refs["refs/remotes"]

Import/export data flow

       export flow                              import flow
       -----------                              -----------
                        +----------------+                   --.
   +------------------->|backing Git repo|---+                 :
   |                    +----------------+   |                 : unchanged
   |[update]                                 |[copy]           : on "op restore"
   |                      +----------+       |                 :
   |      +-------------->| git_refs |<------+                 :
   |      |               +----------+       |               --'
   +--[compare]                            [diff]--+
          |   .--       +---------------+    |     |         --.
          |   :    +--->|remotes["git"] |    |     |           :
          +---:    |    |               |<---+     |           :
              :    |    |remotes[remote]|          |           : restored
              '--  |    +---------------+          |[merge]    : on "op restore"
                   |                               |           : by default
             [copy]|    +---------------+          |           :
                   +----| (local)       |<---------+           :
                        | branches/tags |                      :
                        +---------------+                    --'

jj git import applies diff between git_refs and remotes[]. git_refs is always copied from the backing Git repo.
jj git export copies jj’s remotes view back to the Git repo. If a ref in the Git repo has been updated since the last import, the ref isn’t exported.
jj op restore never rolls back git_refs.

Tracking state

The git.auto-local-bookmark config knob is applied when importing new remote branch. jj branch sub commands will be added to change the tracking state.

fn default_state_for_newly_imported_branch(config, remote) {
    if remote == "git" {
        State::Tracked
    } else if config["git.auto-local-bookmark"] {
        State::Tracked
    } else {
        State::New
    }
}

A branch target to be merged is calculated based on the state.

fn target_in_merge_context(known_target, state) {
    match state {
        State::New => RefTarget::absent(),
        State::Tracked => known_target,
    }
}

Mapping to the current data model

New remotes["git"].branches corresponds to git_refs["refs/heads"], but forgotten branches are removed from remotes["git"].branches.
New remotes["git"].tags corresponds to git_refs["refs/tags"].
New remotes["git"].head corresponds to git_head.
New remotes[remote].branches corresponds to branches[].remote_targets[remote].
state = new|tracking doesn’t exist in the current model. It’s determined by git.auto-local-bookmark config.

Common command behaviors

In the following sections, a merge is expressed as adds - removes. In particular, a merge of local and remote targets is [local, remote] - [known_remote].

fetch/import

jj git fetch
1. Fetches remote changes to the backing Git repo.
2. Import changes only for remotes[remote].branches[glob] (see below)
  - TODO: how about fetched .tags?
jj git import
1. Copies git_refs from the backing Git repo.
2. Calculates diff from the known remotes to the new git_refs.
  - git_refs["refs/heads"] - remotes["git"].branches
  - git_refs["refs/tags"] - remotes["git"].tags
  - TBD: "HEAD" - remotes["git"].head (unused)
  - git_refs["refs/remotes/{remote}"] - remotes[remote]
3. Merges diff in local branches and tags if state is tracking.
  - If the known target is absent, the default state should be calculated. This also applies to previously-forgotten branches.
4. Updates remotes reflecting the import.
5. Abandons commits that are no longer referenced.

push/export

jj git push
1. Calculates diff from the known remotes[remote] to the local changes.
  - branches - remotes[remote].branches
    - If state is new (i.e. untracked), the known remote branch target is considered absent.
    - If state is new, and if the local branch target is absent, the diff [absent, remote] - absent is noop. So it’s not allowed to push deleted branch to untracked remote.
    - TODO: Copy Git’s --force-with-lease behavior?
  - ~~tags~~ (not implemented, but should be the same as branches)
2. Pushes diff to the remote Git repo (as well as remote tracking branches in the backing Git repo.)
3. Updates remotes[remote] and git_refs reflecting the push.
jj git export
1. Copies local branches/tags back to remotes["git"].
  - Conceptually, remotes["git"].branches[name].state can be set to untracked. Untracked local branches won’t be exported to Git.
  - If remotes["git"].branches[name] is absent, the default state = tracking applies. This also applies to forgotten branches.
  - ~~tags~~ (not implemented, but should be the same as branches)
2. Calculates diff from the known git_refs to the new remotes[remote].
3. Applies diff to the backing Git repo.
4. Updates git_refs reflecting the export.
If a ref failed to export at the step 3, the preceding steps should also be rolled back for that ref.

init/clone

jj init
- Import, track, and merge per git.auto_local_branch config.
- If !git.auto_local_branch, no tracking state will be set.
jj git clone
- Import, track, and merge per git.auto_local_branch config.
- The default branch will be tracked regardless of git.auto_local_branch config. This isn’t technically needed, but will help users coming from Git.

branch

jj branch set {name}
1. Sets local branches[name] entry.
jj branch delete {name}
1. Removes local branches[name] entry.
jj branch forget {name}
1. Removes local branches[name] entry if exists.
2. Removes remotes[remote].branches[name] entries if exist. TODO: maybe better to not remove non-tracking remote branches?
jj branch track {name}@{remote} (new command)
1. Merges [local, remote] - [absent] in local branch.
  - Same as “fetching/importing existing branch from untracked remote”.
2. Sets remotes[remote].branches[name].state = tracking.
jj branch untrack {name}@{remote} (new command)
1. Sets remotes[remote].branches[name].state = new.
jj branch list
- TODO: hide non-tracking branches by default? …

Note: desired behavior of jj branch forget is to

discard both local and remote branches (without actually removing branches at remotes)
not abandon commits which belongs to those branches (even if the branch is removed at a remote)

Command behavior examples

fetch/import

Fetching/importing new branch
1. Decides new state = new|tracking based on git.auto_local_branch
2. If new state is tracking, merges [absent, new_remote] - [absent] (i.e. creates local branch with new_remote target)
3. Sets remotes[remote].branches[name].state
Fetching/importing existing branch from tracking remote
1. Merges [local, new_remote] - [known_remote]
Fetching/importing existing branch from untracked remote
1. Decides new state = new|tracking based on git.auto_local_branch
2. If new state is tracking, merges [local, new_remote] - [absent]
3. Sets remotes[remote].branches[name].state
Fetching/importing remotely-deleted branch from tracking remote
1. Merges [local, absent] - [known_remote]
2. Removes remotes[remote].branches[name] (target becomes absent) (i.e. the remote branch is no longer tracked)
3. Abandons commits in the deleted branch
Fetching/importing remotely-deleted branch from untracked remote
1. Decides new state = new|tracking based on git.auto_local_branch
2. Noop anyway since [local, absent] - [absent] -> local
Fetching previously-forgotten branch from remote
1. Decides new state = new|tracking based on git.auto_local_branch
2. If new state is tracking, merges [absent, new_remote] - [absent] -> new_remote
3. Sets remotes[remote].branches[name].state
Fetching forgotten and remotely-deleted branch
- Same as “remotely-deleted branch from untracked remote” since forgotten remote branch should be state = new
- Therefore, no local commits should be abandoned

push

Pushing new branch, remote doesn’t exist
1. Pushes [local, absent] - [absent] -> local
2. Sets remotes[remote].branches[name].target = local, .state = tracking
Pushing new branch, untracked remote exists
1. Pushes [local, remote] - [absent]
  - Fails if local moved backwards or sideways
2. Sets remotes[remote].branches[name].target = local, .state = tracking
Pushing existing branch to tracking remote
1. Pushes [local, remote] - [remote] -> local
  - Fails if local moved backwards or sideways, and if remote is out of sync
2. Sets remotes[remote].branches[name].target = local
Pushing existing branch to untracked remote
- Same as “new branch”
Pushing deleted branch to tracking remote
1. Pushes [absent, remote] - [remote] -> absent
  - TODO: Fails if remote is out of sync?
2. Removes remotes[remote].branches[name] (target becomes absent)
Pushing deleted branch to untracked remote
- Noop since [absent, remote] - [absent] -> remote
- Perhaps, UI will report error
Pushing forgotten branch to untracked remote
- Same as “deleted branch to untracked remote”
Pushing previously-forgotten branch to remote
- Same as “new branch, untracked remote exists”
- The target of forgotten remote branch is absent

export

Exporting new local branch, git branch doesn’t exist
1. Sets remotes["git"].branches[name].target = local, .state = tracking
2. Exports [local, absent] - [absent] -> local
Exporting new local branch, git branch is out of sync
1. Exports [local, git] - [absent] -> fail
Exporting existing local branch, git branch is synced
1. Sets remotes["git"].branches[name].target = local
2. Exports [local, git] - [git] -> local
Exporting deleted local branch, git branch is synced
1. Removes remotes["git"].branches[name]
2. Exports [absent, git] - [git] -> absent
Exporting forgotten branches, git branches are synced
1. Exports [absent, git] - [git] -> absent for forgotten local/remote branches

undo fetch

Exporting undone fetch, git branches are synced
1. Exports [old, git] - [git] -> old for undone local/remote branches
Redoing undone fetch without exporting
- Same as plain fetch since the known git_refs isn’t diffed against the refs in the backing Git repo.

`@git` remote

jj branch untrack {name}@git
- Maybe rejected (to avoid confusion)?
- Allowing this would mean different local branches of the same name coexist in jj and git.
jj git fetch --remote git
- Rejected. The implementation is different.
- Conceptually, it’s git::import_refs() only for local branches.
jj git push --remote git
- Rejected. The implementation is different.
- Conceptually, it’s jj branch track and git::export_refs() only for local branches.

Remaining issues

https://github.com/jj-vcs/jj/issues/1278 pushing to tracked remote
- Option could be added to push to all tracking remotes?
Track remote branch locally with different name
- Local branch name could be stored per remote branch
- Consider UI complexity
“private” state (suggested by @ilyagr)
- “private” branches can be pushed to their own remote, but not to the upstream repo
- This might be a state attached to a local branch (similar to Mercurial’s “secret” phase)

Remote/@git tracking branches

Objective

Current data model (as of jj 0.8.0)

Proposed data model

Import/export data flow

Tracking state

Mapping to the current data model

Common command behaviors

fetch/import

push/export

init/clone

branch

Command behavior examples

fetch/import

push

export

undo fetch

@git remote

Remaining issues

References

`@git` remote