Go Modules

Go modules are Go's built-in mechanism for managing dependencies, versions, and package distribution since Go 1.11 (and enabled by default since Go 1.13). With modules, you no longer need the legacy GOPATH workspace style for dependency management. Instead, each project has its own module, and dependencies are listed in the go.mod file. Below is an in-depth explanation of how Go modules work, including several illustrative examples.

1. Key Concepts

Module
- A module is a collection of Go packages stored in a file tree.
- A module is defined by a go.mod file at its root, which specifies the module path (an import path) and its dependencies.
go.mod File
- Declares the module path (e.g., module example.com/myproject).
- Specifies the minimum required version for each dependency.
- May contain directives like require, replace, and exclude to manage dependencies.
go.sum File
- Accompanies go.mod.
- Contains cryptographic checksums for each module version to ensure integrity.
- Automatically updated by the Go tool to verify no tampering occurs.
Semantic Import Versioning
- Modules can have versioned import paths for major versions v2+. For example, module example.com/mylib/v2.
- Ensures code using v2+ major versions can coexist alongside earlier versions.
Fetching Dependencies
- The Go tool automatically fetches dependencies when needed.
- Modules are cached locally in the module cache (often in GOPATH/pkg/mod).
Transitive Dependencies
- The go.mod file only lists direct dependencies.
- Transitive dependencies (dependencies of dependencies) are also tracked in the go.sum file.

2. Creating a New Module

When you start a new Go project, you initialize a module by creating a go.mod file.

Example: Initializing a Module

# Create a new directory
mkdir myproject
cd myproject

# Initialize the module
go mod init example.com/myproject

Explanation:

go mod init example.com/myproject creates a go.mod file in the current directory with module example.com/myproject.
This path is how other modules import your code (e.g., import "example.com/myproject/foo").

Sample go.mod (generated):

module example.com/myproject

go 1.19

3. Adding and Using Dependencies

When your code imports a package from another module, Go automatically updates go.mod and go.sum with the required module version.

Example: Adding a Dependency

Suppose you have a file main.go:

package main

import (
    "fmt"

    "github.com/google/uuid" // external dependency
)

func main() {
    id := uuid.New()
    fmt.Println("Generated UUID:", id)
}

Steps:

Initialize (if not already done):
go mod init example.com/myproject
Build or run:
go build # or go run main.go

Outcome:

The Go tool automatically fetches github.com/google/uuid from its default repository.
go.mod is updated with a require line for github.com/google/uuid at the appropriate version.
go.sum is updated with checksums for that version.

Updated go.mod (truncated example):

module example.com/myproject

go 1.19

require github.com/google/uuid v1.3.0

4. Version Upgrades and Downgrades

You can explicitly manage dependency versions using the go get command.

Example: Changing Dependency Versions

Assume your go.mod looks like:

module example.com/myproject

go 1.19

require github.com/google/uuid v1.3.0

To upgrade github.com/google/uuid to version v1.4.0, run:

go get github.com/google/uuid@v1.4.0

This updates:

The go.mod to require github.com/google/uuid v1.4.0.
The go.sum with new checksums.

If you want to downgrade, you can specify an older version similarly:

go get github.com/google/uuid@v1.2.0

5. Replace and Exclude Directives

5.1 `replace` Directive

The replace directive in go.mod overrides the source location or version of a dependency. Useful for:

Using a local module copy (e.g., for testing changes) instead of the remote repo.
Redirecting a module to a forked version.

Example:

module example.com/myproject

go 1.19

require github.com/google/uuid v1.3.0

replace github.com/google/uuid => ../localuuid

Explanation:

Any import of github.com/google/uuid will be replaced with the local path ../localuuid.
This does not affect the module path seen by the code; it remains github.com/google/uuid but code is physically sourced from the local directory.

5.2 `exclude` Directive

The exclude directive prevents a specific module version from being used.

module example.com/myproject

go 1.19

require github.com/google/uuid v1.4.0

exclude github.com/google/uuid v1.3.0

Explanation:

Go will not use version v1.3.0 of github.com/google/uuid, even if transitive dependencies request it.
Typically used to avoid problematic versions (e.g., containing bugs or vulnerabilities).

6. Semantic Import Versioning for Major Releases

Go modules handle major versions (v2+) by encoding them into the module path. For example, if your project releases a breaking change and needs a v2 version, you must update your import paths to example.com/myproject/v2.

Example: Releasing `v2` of a module

Change the module path in go.mod:
module example.com/myproject/v2 go 1.19
Update import paths in your code:
// old import // import "example.com/myproject" // new import import "example.com/myproject/v2"
Tag your repository with v2.x.x (e.g., v2.0.0).

Consumers who need v2 must update their imports to "example.com/myproject/v2".

7. Using Private Modules

If you host private Go modules (e.g., on GitHub with private repos), you may need to set up:

GOPRIVATE environment variable: Tells the Go tool not to attempt to fetch certain modules from public proxies.
Credentials: Provide authentication for private repositories (Git over SSH, HTTPS with tokens, etc.).

Example:

export GOPRIVATE="github.com/myorg/*"
go get github.com/myorg/privatemodule

Explanation:

GOPRIVATE="github.com/myorg/*" instructs Go to skip the public module proxy for any repositories under github.com/myorg/ and fetch them directly from the source.

8. Running Commands with Go Modules

8.1 `go build`/`go run`

go build compiles all packages and fetches dependencies if needed.
go run <file.go> builds and runs a program from source.

8.2 `go mod tidy`

go mod tidy cleans up unused dependencies from go.mod and ensures go.sum is accurate.
Removes modules not imported by any package in your module.

8.3 `go list -m all`

Lists all modules in use (including transitive dependencies) for the current build.

8.4 `go test ./...`

Runs all tests in the current module (and subpackages).
Ensures that all necessary dependencies are fetched.

9. Multi-Module Repositories

Sometimes a single repo can contain multiple modules. Each subdirectory with its own go.mod is treated as an independent module.

Pros:

Fine-grained versioning for different components within the same repository.

Cons:

Additional overhead of managing multiple go.mod files.

Example Directory Structure:

myrepo/
  go.mod            # Root module
  pkg1/
    go.mod          # Submodule
    ...
  pkg2/
    go.mod          # Another submodule
    ...

Usage:

Each go.mod can have a distinct module path like example.com/myrepo/pkg1 and example.com/myrepo/pkg2.

10. Example Workflow Summary

Initialize a new module in your project directory:
go mod init example.com/myproject
Write code that imports packages from standard library or external modules.
Build or run your code:
go build go run main.go
The Go tool fetches dependencies and updates go.mod and go.sum.
Check / Update versions as needed:
go get github.com/some/dependency@v1.2.3
Clean up unused dependencies:
go mod tidy
Tag your releases in VCS and update import paths accordingly for major version changes.

Best Practices

Use Meaningful Module Paths:
- Typically use a domain name followed by the repo path (github.com/username/repo).
Keep go.mod and go.sum in Version Control:
- Ensures reproducible builds.
Don’t Commit vendor unless necessary:
- By default, the Go module proxy/cache is sufficient.
- vendor can be used for offline builds or if corporate policy requires bundling dependencies.
Regularly Run go mod tidy:
- Keeps go.mod clean. Removes unnecessary dependencies.
Handle Major Version Changes Properly:
- If you need backward-incompatible changes, rename your module to include v2, v3, etc.
Use Replace Carefully:
- Use replace for local development or custom forks. Avoid shipping code with permanent replace directives unless absolutely required.

Conclusion

Go modules provide a robust, self-contained approach to managing dependencies and versioning in Go. By creating a go.mod file in your project’s root, you declare your module name and track dependencies automatically. The Go tool handles fetching, verifying, and updating dependencies. With semantic import versioning, private module support, and simple commands (go mod tidy, go get, etc.), Go modules simplify the build process and enable reproducible, maintainable Go projects.

Last modified: 01 January 2025

Go Modules

1. Key Concepts

2. Creating a New Module

Example: Initializing a Module

3. Adding and Using Dependencies

Example: Adding a Dependency

4. Version Upgrades and Downgrades

Example: Changing Dependency Versions

5. Replace and Exclude Directives

5.1 replace Directive

5.2 exclude Directive

6. Semantic Import Versioning for Major Releases

Example: Releasing v2 of a module

7. Using Private Modules

8. Running Commands with Go Modules

8.1 go build/go run

8.2 go mod tidy

8.3 go list -m all

8.4 go test ./...

9. Multi-Module Repositories

Example Directory Structure:

10. Example Workflow Summary

Best Practices

Conclusion

5.1 `replace` Directive

5.2 `exclude` Directive

Example: Releasing `v2` of a module

8.1 `go build`/`go run`

8.2 `go mod tidy`

8.3 `go list -m all`

8.4 `go test ./...`