Cluster-wide GitHub/Docker Download Acceleration Solution: Unified Proxy Configuration in Practice

Wapuu
1

Background

VMs within the cluster and production servers experience slow download speeds when fetching code, release files, and Docker images from GitHub. Previous workarounds included hardcoding GitHub IP addresses in /etc/hosts (prone to expiration) or relying on various third-party proxies (e.g., ghproxy.net), which are unstable and fragmented.

On feicode.com, the source URLs for 484 mirrored repositories used five different third-party proxies—resulting in chaotic management.

Solution

Leverage our own proxy services—gh.wpcy.net (GitHub file proxy) and docker.wpcy.net (Docker Hub image proxy)—and configure them transparently via native git and container engine mechanisms. This eliminates the need to modify application code or repository URLs.

1. Accelerating GitHub Code/File Downloads

Git natively supports URL rewriting via the insteadOf configuration. Once configured, all git clone/fetch/pull operations automatically route through the proxy:

git config --global url."https://gh.wpcy.net/https://github.com/".insteadOf "https://github.com/"

For the Forgejo instance hosting 484 mirrored repositories—each previously using one of five distinct third-party proxies—we unify all traffic by configuring multiple insteadOf rules inside containers. This requires no database modifications or service downtime:

[url "https://gh.wpcy.net/https://github.com/"]
    insteadOf = https://github.com/
    insteadOf = https://ghproxy.net/https://github.com/
    insteadOf = https://hk.gh-proxy.com/https://github.com/
    insteadOf = https://gh.llkk.cc/https://github.com/
    insteadOf = https://git.yylx.win/https://github.com/
    insteadOf = https://fast.feibisi.com/https://github.com/

Benefit: Switching proxies in the future only requires updating this single gitconfig, without touching configurations for any of the 484 repositories.

Additionally, we deployed the gh-download command for manual downloads of GitHub release assets—automatically accelerated via the proxy.

2. Accelerating Docker Image Pulls

  • VM clusters (using Podman): Configure registry priority via registries.conf.d.
  • Production servers (using Docker): Configure registry-mirrors in daemon.json.

Both prioritize local caching first, then fall back to our own proxy.

3. Cleaning /etc/hosts

We systematically removed hardcoded GitHub IP entries across the entire cluster (~100 lines per host), along with redundant domain mappings. These hardcoded IPs frequently expire and become outdated; with the proxy in place, they are entirely unnecessary.

Deployment Scope

  • git insteadOf: 9 VM cluster nodes + 7 production servers + feicode.com (including multi-proxy rewriting)
  • gh-download command: 9 VM cluster nodes
  • Docker/Podman image proxy: VM cluster + production servers + feicode.com
  • /etc/hosts cleanup: VM cluster + production servers

Notes

  • git insteadOf applies only to HTTPS URLs—not SSH.
  • The proxy uses Cloudflare, which imposes rate limits on large files.
  • If the proxy becomes unavailable, temporarily unset the configuration to restore direct connections.
  • Non-GitHub sources (e.g., codeberg.org) remain unaffected.