Golang

Last Updated: 2024-01-14

Go is about composition and concurrency.

Golang Project Layout

There's no official layout, but many golang projects follow this layout:

my-project
 |- /cmd                # binaries / executables
 |- /pkg                # code that can be imported in other projects
 |- /internal           # code that cannot be imported in other projects
 |                      # `internal` folder is enforced by Golang; it can be at any level.
 |- /vendor             # managed by the `go mod vendor` command
 |- /third_party        # forked code, NOT managed by `go mod vendor`
 |- /api                # API definitions; implementation should be in `/pkg`
 |- /build              # scripts for building the project
 |- /scripts or /hack   # scripts for various tasks
 |- go.mod              # modules
 |- go.sum              # checksums

Note

The /cmd, /internal, and /pkg directories group packages. /cmd groups command packages, /internal groups internal packages, and /pkg groups public packages.
/internal is official, /pkg is not. /internal is the only directory named in Go’s documentation and has special compiler treatment.
if you are coming from the Java world: there's no /src in a go project; there used to be a /src under GOPATH, but it is no longer needed in the module world.

Packages and Modules

Go Package: a collection of source files in the same directory that are compiled together.
- a package is a folder of go code, most likely living in the /pkg folder.
- each go file starts with package foo.
- naming convention: the package name is the same as the last element of the import path.
- within a directory, you cannot have two package names.
- functions, types, variables, and constants defined in one source file are visible to all other source files within the same package.
- special:
  - main package, used for executables;
  - main function, an entry point of the executable. No arguments, no return, no explicit call of main().
Go Module: a collection of Go packages stored in a file tree with a go.mod file at its root.
- these packages are versioned together, and the contents of each version are immutable.
- a Go repository typically contains only one module, located at the root of the repository.
- naming convention: module path should match the URL for the repository.
- the go.mod file defines:
  - module path (e.g. module k8s.io/kubernetes), serves as an import path prefix.
  - go version (e.g. go 1.19).
  - dependencies (require()).

Example go.mod: https://github.com/kubernetes/kubernetes/blob/master/go.mod

Export and Import

Functions begin with an upper-case letter are exported.

Import path:

import path = module path + subdirectory within the module

If you want to use code from another file:

In the same package: no need to be imported.
In the same module but a different package: no change in go.mod, just import in .go files.
From a different module: require() in go.mod (and run go mod tidy), and import in .go files.

For example, kubernetes/apimachinery project has module k8s.io/apimachinery in its go.mod, and it has a package runtime in the /pkg/runtime folder. The file pkg/runtime/interfaces.go defines a type Encoder.

Then in kubernetes/kubernetes project, it imports the package using the module path (k8s.io/apimachinery) plus the subdirectories (/pkg/runtime):

In go.mod:

require (
    k8s.io/apimachinery v0.0.0
)

In .go code:

import (
    "k8s.io/apimachinery/pkg/runtime"
)

// To use types defined in `runtime`:
runtime.Encoder

Services

A module mirror is a special kind of module proxy that caches metadata and source code in its own storage system, allowing the mirror to continue to serve source code that is no longer available from the original locations. This can speed up downloads and protect you from disappearing dependencies. The Go team maintains a module mirror, served at proxy.golang.org, which the go command will use by default

golang.org is being merged to go.dev.
proxy.golang.org: a module mirror implementing the GOPROXY protocol. User can download go modules from it. Maintained by Google.
- A module proxy is an HTTP server that can respond to GET requests.
sum.golang.org - an auditable checksum database which will be used by the go command to authenticate modules.
https://index.golang.org/index - an index which serves a feed of new module versions that become available by proxy.golang.org.
https://pkg.go.dev/: Discover pakcages and modules

Release Cycle

Roughly twice a year. "Each major Go release is supported until there are two newer major releases." (i.e. only the latest 2 versions are supported)

Go 1.20 (February 2023)
Go 1.19 (August 2022)

Note that 1.20 is considered as a major release, instead of a minor release in semver.

Standard Library

Packages in the standard library do not have a module path prefix. E.g. import "os".

https://pkg.go.dev/std

Runtime and GC

The Go standard toolchain provides a runtime library that ships with every application, and this runtime library includes a garbage collector.

Go's runtime is analogous to libc, the C library. Go's runtime does not include a virtual machine, such as is provided by the Java runtime. Go programs are compiled ahead of time to native machine code.

Stack allocation: non-pointer Go values stored in local variables will likely not be managed by the Go GC at all, the space is stored on the goroutine stack.
Dynamic memory allocation: Go values whose memory cannot be allocated this way, because the Go compiler cannot determine its lifetime, are said to escape to the heap.

When will Go 2 be released?

Never, based on the GopherCon 2022. https://www.youtube.com/watch?v=v24wrd3RwGo

Go 2 means all the Go 1 programs will stop working. Instead Go team prioritizes compatibility.

Install multiple versions

$ go install golang.org/dl/go1.16@latest
$ go1.16 download

Then you can use the go1.16 binary (the binary is in GOPATH/bin):

$ go1.16 version
go version go1.16 linux/amd64

# Different version should have the same GOPATH, but different GOROOT
$ go1.16 env GOROOT

Defer

A defer statement defers the execution of a function until the surrounding function returns.

The deferred call's arguments are evaluated immediately, but the function call is not executed until the surrounding function returns, avoiding worries about variables changing values as the function executes.

Deferred function calls are pushed onto a stack. When a function returns, its deferred calls are executed in last-in-first-out order.

Features to avoid

Do NOT use import .

Go specific features

The name rune is Go terminology for a single Unicode code point.
functions and methods can return multiple values.
func Min(a ...int) int {}, a is a slice of ints.

init()

takes no arguments, returns no values.
the init() function is optional.
called implicitly by Go.
all init() functions are always executed prior to the main() function.
a source file can contain zero or multiple init() functions, they are executed in the order of declaration.
each init() is executed only once, even if the package is imported multiple times.

Use context.Context for cancellation, error to return error

// Fetch returns the requested item.
func Fetch(context.Context, name string) (Item, error) {
  // ...
}

Closures

In Go, function literals are closures: the implementation makes sure the variables referred to by the function survive as long as they are active.

Run commands in Go

cmd := exec.Command()

fmt.Printf("Executing %v", cmd.Args)

if out, err := cmd.CombinedOutput(); err != nil {
  fmt.Printf("failed to execute: %s", out)
}

cmd.Output(): runs the command and returns its standard output.
cmd.CombinedOutput(): runs the command and returns its combined standard output AND standard error.

Implements

Golang does not provide an implements keyword or mechanism like Java.

To say a struct implements an interface in go:

type FooInterface interface {
// ...
}

type barType struct{}

var _ FooInterface = barType{}

This provides a static (compile time) check that barType satisfies the FooInterface interface.

The _ is the name of the variable, it tells the compiler to effectively discard the RHS value, but to type-check it and evaluate it if it has any side effects; _ doesn't take any process space.

os.SetEnv

os.SetEnv only changes the environment definition inside your process.

Environment variables are a per process thing.

embed

list all files; embed is just another FS.

if err := fs.WalkDir(AbsFS, "/manifests/testdeps", func(path string, d fs.DirEntry, err error) error {
  if err != nil {
    fmt.Println(err)
  }
  fmt.Println(path)
  return nil
}); err != nil {
  t.Errorf("failed")
}

SSH Client

SSH Client: vendor/golang.org/x/crypto/ssh/client.go

laddr means local address.
raddr means remote address of the socket.

CLI

Commuly used: Cobra. https://github.com/spf13/cobra