Generics in Go and Effective Abstraction

In the quest for better performance and easier concurrency, many backend server engineering teams are turning to Go to power their APIs and build tools. According to the Stack Overflow 2024 Developer Survey, Go is the 12th most popular language used extensively by professional developers — behind the heavy-hitters like JavaScript and Python, slightly ahead of Rust, and well ahead of languages like Ruby, Scala, or Elixir.

There is good reason for this enthusiasm. It uses an intentionally minimal type system without a huge learning curve, and boasts a well-designed channel-based concurrency system using flexible green threads. Performance is often better than alternatives like Python or JavaScript with a lower memory and resource footprint than Java or .NET.

Despite the strong momentum, however, Go isn’t perfect. Many have criticized the limitations of its type system when attempting to build flexible libraries without using runtime features like reflection or assertions. Many others have criticized the addition of a major feature intended to improve that situation — generic type parameters. After using them in production to build internal libraries for microservices, however, I find them to be a great addition to the language that gives me more confidence in building usable abstractions for large teams.

Abstraction & Polymorphism

Go has long had what I consider to be a weakness in the area of abstractions. Abstraction is a way to make complex operations more simple by generalizing — focusing only on relevant details and hiding irrelevant complexity away. In software engineering, it often takes the form of an “interface” that defines the important attributes or methods, letting implementations deal with the details of how they make it happen. Abstractions are often intended to take advantage of “polymorphism”, which means that they can be reused across many different types. Before v1.18 was released in 2022, Go’s polymorphism relied entirely on runtime behavior, with no built-in way to apply static analysis to prevent bugs.

Stepping back for a moment — if a language didn’t have polymorphism at all, simple utility functions like Sum() or Map() or Contains() would have to be specific to a particular type. You would need SumInt(), SumUInt(), SumFloat32(), SumFloat64(), and more! Thankfully, Go has had interfaces from the beginning which can be “downcast” from their abstract type down to their underlying concrete type using runtime type guards/assertions. These guards will return an error if the underlying type isn’t what is requested, allowing for simple trial-and-error polymorphism. Go also has a detailed runtime type reflection system in the reflect package, which allows for inspecting types and dynamically invoking methods at runtime.

This runtime polymorphism carries with it an unfortunate loss of type safety, however, as well as a some performance overhead. It’s easy to over-use the any interface (also known as interface{}) that matches all types, and passing an unexpected type could lead to errors if your code needs access to the underlying concrete type for any reason. The usage of type assertions and reflection comes with a runtime cost that can be a big problem in loops and other performance-sensitive contexts. Before generics, it was nearly impossible to concisely implement some algorithms or data structures with any kind of compile-time confidence, resulting in verbose or error-prone APIs in the standard library and elsewhere. In addition, you can only work with types the library author anticipated ahead of time, and adding new types requires adding new code to the library.

Generic Types at Last!

In March of 2022, Go finally released generics in version v1.18! Generics are type parameters that a function can accept to enable polymorphism. They allow for much more sophisticated abstraction without runtime costs, since they are evaluated at compile-time. Using generics together with interfaces, you can strike a good balance between paying attention to the relevant details and ignoring the irrelevant ones. Complexity can be pushed out to the writer of the abstraction, while giving them ways to guarantee the functionality they need from generic types.

This new feature was implemented in a backwards-compatible way, adding new syntax to function and type declarations to allow for generic type parameters. There are some limitations, however. Perhaps the biggest limitation is that type parameters cannot be used in methods. There is a proposal to implement this functionality, but it is not a part of the language yet. Inference is unfortunately not very strong yet, so you will need to annotate types in many places other languages would be able to infer. Go also doesn’t yet support higher-kinded types — types within parameters which themselves take other types as parameters. Haskell enthusiasts are surely disappointed that they cannot yet fully represent a Monad or Functor.

Instead of dealing with type assertions and runtime checking behavior as above, the Go compiler now automatically generates different versions of a function for each type they are used with.

If the generic function is later used with a “type D” in the example above, Go would automatically generate a new version of the generic function behind the scenes that would work for type D.

Interfacing

Interfaces with type generic parameters are very useful for library authors who want to guarantee some functionality in the generic types they are working with. For example, consider an internal library author who wants to provide an abstraction for consistent event ingestion. The library author will likely want to make their Consumer struct flexible and able to be used for multiple event sources and payloads by many different applications across different teams within the organization. To support this, they might define a few interfaces first:

type AWSEvent interface {
	events.SQSEvent | events.KinesisEvent | events.DynamoDBEvent | events.KafkaEvent
}

type Decoder[Event AWSEvent, Item constraints.Ordered] interface {
	Decode(ctx context.Context, event Event) ([]Item, error)
}

type Handler[Item constraints.Ordered] interface {
	Handle(ctx context.Context, item Item) error
}

The AWSEvent interface is a union of all the different AWS events that the library author’s Consumer can support. The Decoder and Handler interfaces allow the users of the internal library (likely working on different applications) to supply their own decoders and handlers, decoupling the library from the data it will work with. The implementation of the Consumer might start out very simple, like this:

type Consumer[Event AWSEvent, Item constraints.Ordered] struct {
	decoder Decoder[Event, Item]
	handler Handler[Item]
}

func (c Consumer[Event, Item]) Consume(ctx context.Context, event Event) error {
	items, err := c.decoder.Decode(ctx, event)
	if err != nil {
		return err
	}
	if len(items) == 0 {
		return nil
	}

	sort.Slice(items, func(i, j int) bool {
		return items[i] < items[j]
	})

	for _, item := range items {
		if err := c.handler.Handle(ctx, item); err != nil {
			return err
		}
	}

	return nil
}

As usual, the library author can use Decode() and Handle() on the decoder and handler without knowing anything about the actual underlying types because of the interfaces they defined. In addition, since the Ordered constraint (from the experimental section of the standard library) was used with the generic Item parameter, the library author can use the < operator to sort items with confidence, knowing that the underlying type for Item will support it. If the Item type does not support it, the Go compiler will throw an error during compilation, ensuring that an error around the use of the < operator will never make it into Production. Similarly, if an Event type is used that doesn’t match one of the types in the AWSEvent interface a compile error will prevent the project from building.

Users of the library can write a decoder for each AWS Event type they want to work with, and one handler that works for any decoded Item. They don’t have to think about details like sorting the items before handling them, because the library author hid that complexity away within the abstraction.

Users of the library can write a decoder for each AWS Event type they want to work with, and one handler that works for any decoded Item. They don’t have to think about details like sorting the items before handling them, because the library author hid that complexity away within the abstraction.

Functional Programming

Generic types also enable the type of function composition essential for effective functional programming. This pattern takes advantage of pure, first-class functions to build up complex functionality from smaller pieces and push mutable state out to the edges of a software system to make the results more predictable.

Previously, runtime reflection was required to make any kind of real functional programming possible, and the verbosity and performance hit of this made it an unpopular paradigm in the Go community. With generics, Go now has a much better way to make functions polymorphic so that they can be reused easily with good performance, relying on static analysis at build time rather than error-prone runtime checks.

There are some simple ways that this can be useful in Go. Options can be used to represent values that can be empty, or be populated with an actual value:

type Option[T any] struct {
	isPresent bool
	value     T
}

func Some[T any](value T) Option[T] {
	return Option[T]{
		isPresent: true,
		value:     value,
	}
}

func None[T any]() Option[T] {
	return Option[T]{
		isPresent: false,
	}
}

We can implement a Get() method, and a top-level Map() function to work with our optional value:

func (o Option[T]) Get() (T, bool) {
	return o.value, o.isPresent
}

func Map[A, B any](o Option[A], mapper func(value A) B) Option[B] {
	if o.isPresent {
		return Some(mapper(o.value))
	}

	return None[B]()
}

This can help reduce the dependency on error-prone pointers, and also to eliminate the ambiguity around emptiness with types like string and int. Go’s string type automatically initializes as "" for its empty state, and int automatically initializes as 0. There are times when you actually want to differentiate between an empty string or a 0 and an absent value, however. An Option type allows you to distinguish between Some("") and None, and it also allows you to pass around an optional value with no implicit nil state — meaning there’s no chance of a nil pointer dereference (the equivalent of a null-pointer exception in other languages).

With tools like Map and Get, you can build up more complicated operations from smaller pieces of functionality while keeping each piece easily testable in isolation and keeping the code tight and concise. For example, if you had a simple sensor and you needed to parse a string response to understand the status, you might write a struct like this:

type SensorStatus struct {
	Message string
	Code    int
}

func NewSensorStatus(status string) SensorStatus {
	if status == "OK" {
		return SensorStatus{status, 2}
	}

	return SensorStatus{status, 1}
}

func (s SensorStatus) IsOK() bool {
	return s.Code == 2
}

func (s SensorStatus) Report() Option[string] {
	if !s.IsOK() {
		return Some(fmt.Sprintf("Sensor error: %s\n", s.Message))
	}

	return None[string]()
}

To work with it in a clean, functional way as part of a main entry point you could take advantage of Map and Get like this (admittedly rather contrived) example:

func getSensorStatus() Option[string] {
	/// TODO: get sensor status from sensor
	return Some("OK")
}

func sendSensorReport(report string) {
	/// TODO: send sensor error reports somewhere
}

func sendMissingSensorReport() {
	/// TODO: send a report that the sensor is unreachable
}

func main() {
	status, ok := Map(getSensorStatus(), NewSensorStatus).Get()
	if !ok {
		sendMissingSensorReport()
		return
	}

	if report, ok := status.Report().Get(); ok {
		sendSensorReport(report)
		return
	}

	// Sensor is OK
}

With the generics at work, Go can automatically infer the type of status, and can allow the inner scope of the if statement using Get() access to it without any risk of an error and without having to remember to explicitly check for nil as you would with a pointer.

Other common functional programming structs such as Result, Either, and Future are also possible, making some “monadic” functional tools actually usable in Go without too much overhead.

Better Organization

Generics in Go are still evolving, and we’ve already seen several changes like the performance improvements in v1.19, improvements to inference in v1.21, and preview support for generic type aliases in v1.23 (currently planned for full release in v1.24). More enhancements are planned in the future to allow for more complex generic programming and to overcome the limitations mentioned earlier. Even in their current form, however, they’re a powerful tool for flexible libraries with good type safety and low performance overhead.

There are some in the community who criticize generics, lamenting the syntax and the complexity that they bring to the language. There was even some discussion around a fork of the language when generics were announced. As Go’s blog points out, however, “In three years of Go surveys, lack of generics has always been listed as one of the top three problems to fix in the language.” It was a feature that was in high demand, and I think it provides strong tools for using Go with more confidence in large teams, encoding more build-time constraints into the types to ensure predictability and supporting better organization in our codebases.

At Nearform, we have a lot of experience building internal tools and shared frameworks for large teams. Our skilled engineers can craft usable and maintainable abstractions in many language ecosystems, and we can use that experience to help your team build with Go more reliably. Contact us today to find out how we can help your team build well organized, easily maintainable applications with Go with strong performance and solid stability!