Automatic GitHub Mirroring for Self-Hosted Repositories

While there are many benefits to self-hosting git repositories, there are a number of reasons to mirror them to GitHub: Redundancy, discoverability, integration with other users, etc. It’s actually very easy to set up automatic GitHub repository creation and mirroring using the GitHub CLI and simple git hooks. The process is explained below.

Generating a GitHub Personal Access Token

The first step is to go to “Developer Settings” in settings, then generate a fine-grained personal access token with read/write access to “Administration”. This token will be used to create and update repositories.

Creating a Post-Update Hook

Once the token is generated and the GitHub CLI (gh) is installed, create a post-update hook like the one shown below which creates a repository on GitHub if it doesn’t exist, updates its visibility and description based on files in the local repository, then pushes updates to the repository.

#!/bin/sh

set -eu

# Exit with an error if GIT_DIR isn't set
: ${GIT_DIR:?Expected GIT_DIR to be defined.}

# Load GitHub PAT token for gh
GH_TOKEN="$(cat ~/gh_token)"
export GH_TOKEN

# Get user & repository information
# for git push; gh defaults to the authenticated user
user="$(gh api user --jq .login)"
name="$(basename "${GIT_DIR%.git}")"
path="$user/$name"

# Set to public if git-daemon-export-ok exists
visibility=private
test -f "$GIT_DIR"/git-daemon-export-ok &&
	visibility=public

# Create/update repository on GitHub
echo gh repo create "$path" --"$visibility" ||
    true
echo gh repo edit "$path" \
    --visibility "$visibility" \
    -d "$(cat "$GIT_DIR"/description)" \
    --accept-visibility-change-consequences

# Push updates
git push --mirror git@github.com:"$path"

To use the hook, either put it in hooks/post-update under $GIT_DIR (the root of bare repositories) or place it in a global hooks directory and configure a global hooks directory using git’s core.hooksPath option. The latter method is often preferable because it’s automatically used by new repositories and can be updated in a central location. Once the hook is set up, whenever updates are pushed to the repository they’ll automatically be mirrored to GitHub along with their metadata.

Update Consolidation and the gitsrv Project

While not necessary, to prevent excessive updates I created conque, a simple script which stores only unique arguments in a queue for later execution. In addition, I maintain gitsrv (GitHub)—a set of utilities for serving git repositories, including github-mirror for mirroring to GitHub repositories.

In the following example, cron(8) is used to call gitsrv on the consolidated queue every minute, and the hook—located in ~/hooks, the globally configured hooks directory— adds directories to the queue instead of directly updating them.

$ cat ~/hooks/post-update
#!/bin/sh

conque ~/gitsrv-q add "$(git rev-parse --absolute-git-dir)"
$ crontab -l
* * * * *  conque ~/gitsrv-q run gitsrv