Creating libs and sharing them among applications is one strategy to shorten the time needed to develop a new project and make your team's work easier. Each library is treated as a repository by Golang, and you can store them directly on GitHub. It means that we can write our own custom code, push it, and share it with our team. However, how do we accomplish this with a private repository? In today's post, I will introduce to you several ways to get and import private repositories to your project.
Using go get to import private Github repos
In Golang, we can use the go get
command to download a library/module to your local computer before importing them to your code.
Get downloads the packages named by the import paths, along with their dependencies. It then installs the named packages, like 'go install'
For example:
go get -v github.com/capotej/groupcache-db-experiment/..
This command uses GOPATH and GOMODCACHE locations to essentially clone the repository on your computer (to see these locations you can use the command go env [var ...] ).
Go internally pulls the specified versions of your dependencies via Git. Therefore, the git configuration must have the proper credentials to access any private repositories wherever Go is executing. GitHub typically uses an SSH key for communication, but go get clones through HTTPS, thus you must use your login information or a personal token.
The Golang FAQ contains some information on this. If you attempt to use go get
in a private repository, you will see the same link on your console, for instance:
C:\Users\nguye>go get github.com/nguyenanhtuan11041998/testPrivateRepo # cd C:\Users\nguye\go\src\github.com\nguyenanhtuan11041998; git clone -- https://github.com/nguyenanhtuan11041998/testPrivateRepo C:\Users\nguye\go\src\github.com\nguyenanhtuan11041998\testPrivateRepo Cloning into 'C:\Users\nguye\go\src\github.com\nguyenanhtuan11041998\testPrivateRepo'... fatal: Cannot prompt because user interactivity has been disabled. fatal: could not read Username for 'https://github.com': terminal prompts disabled package github.com/nguyenanhtuan11041998/testPrivateRepo: exit status 128
If you use an automated process or script to collect the project dependencies, such as your pipeline to build the application, using HTTPS will cause an issue because your console or terminal must be able to ask you for your credentials in order to insert them. This is completely unsafe and counterproductive. There are 2 ways to solve this problem:
- Using a personal access token
- Use ssh instead of HTTPS for the repository
Using SSH instead of HTTPS
To complete this process, simply enter the following commands into your terminal:
git config --global url."ssh://git@github.com".insteadOf "https://github.com"
To check if the configuration was added, check your .gitconfig
file:
git config --list --show-origin
Because the Golang server cannot access your private repository, you must inform your local environment that it is private and request that it only be accessed directly from your repository by establishing the environment variable GOPRIVATE:
go env -w GOPRIVATE="github.com/<my_user>/<my_privaterepo>"
If you are using the public domain version of Github and not the enterprise edition, I strongly advise setting the GOPRIVATE with your repository rather than just with your user/org. This will prevent certain unexpected behavior.
Generate a GitHub token
First of all, you need to generate a token from GitHub. You can follow the GitHub official guide to create a personal access token.
Next step, you have to export your GitHub token as an environment variable:
export GITHUB_TOKEN=123
The final step is to set git config to use the basic auth URL:
git config --global url."https://$GITHUB_TOKEN:x-oauth-basic@github.com/".insteadOf "https://github.com/"
This is not a recommended way because you can see we send the token along with the URL, there is maybe some risk to leak our information and the repositories will no longer be secured.
Summary
You can reuse code by using private repositories to build libraries and use them in your projects. However, keep in mind that Golang will always attempt to clone your repository or module to a Module Mirror, establishing a checksum in the process. For this reason, when you go retrieve your private repository or module, you must explicitly specify the Go environment to ignore this behavior.
References
https://go.dev/doc/go-get-install-deprecation
https://pkg.go.dev/cmd/go/internal/get
https://go.dev/ref/mod#private-modules
What's the proper way to "go get" a private repository?