Using Optics #

Locating values to update #

Go optics is intended to update immutable data structures. This is achieved by composing optics together to focus the intended targets of an update. This however means that we must fully describe the update using optics rooted at the top level data structure.

Consider this data model.

type BlogPost struct {
	author string
	title string
	content string
	ratings []Rating
}

type Rating struct {
	author string
	stars int
}

For this exercise we want to increase all the ratings of author “Max Mustermann” by 1 star. For a mutable data model we might implement it like this.

blogPost := BlogPost{...}

for _,rating := range blogPost.ratings {
	if rating.author == "Max Mustermann" {
		rating.stars += 1
	}
}

In order to make an immutable update using optics we need some way to represent these concepts.

There are 6 things that need to be mapped

1. Accessing the blogpost.ratings field
2. looping over the []Rating slice
3. Accessing the rating.author field
4. Checking if a value is equal to “Max Mustermann”
5. Accessing the rating.stars field
6. Incrementing an integer value by 1.

A FieldLens can be used to access the field of a struct so we can define our 3 field lenses to access blogPost.ratings, rating.author and rating.rating

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

	blogPostRatings := FieldLens(func(source *BlogPost) *[]Rating {
		return &source.Ratings
	})

	ratingAuthor := FieldLens(func(source *Rating) *string {
		return &source.Author
	})

	ratingStars := FieldLens(func(source *Rating) *int {
		return &source.Stars
	})

Note: normally these field lenses would be generated by the makelens tool

Looping over a slice is handled by the built in TraverseSlice[Rating]() optic.

1

	traverseRatings := TraverseSlice[Rating]()

Checking if a value is equal is handled by the builtin Eq() predicate

1

	eqMustermann := Eq("Max Mustermann")

Incrementing a value by 1 is handled by the builtin Add() operation

1

	incOne := Add(1)

We now need to combine these optics together to the equivalent statements in the original mutable example.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

	// for _,rating := range blogPost.ratings
	traverseBlogPostRatings := Compose(blogPostRatings, traverseRatings)

	// rating.author == "Max Mustermann"
	ratingAuthorEqMaxMustermann := Compose(ratingAuthor, eqMustermann)

	// if rating.author == "Max Mustermann"
	ifRatingAuthorMustermann := Filtered(traverseBlogPostRatings, ratingAuthorEqMaxMustermann)

	// rating.stars = ....
	matchingRatings := Compose(
		ifRatingAuthorMustermann,
		ratingStars,
	)

We can now use these optics with the MustModify action to apply the incOne operation to each matching blog post rating.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

	result := MustModify(
		matchingRatings,
		incOne,
		BlogPost{
			Ratings: []Rating{
				Rating{
					Author: "Max Mustermann",
					Stars:  0,
				},
				Rating{
					Author: "Erika Mustermann",
					Stars:  0,
				},
			},
		},
	)
	fmt.Println(result)

In production code when using makelens this would most likely be written in this way.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

	result := MustModify(
		Compose(
			Filtered(
				O.BlogPost().Ratings().Traverse(),
				O.Rating().Author().Eq("Max Mustermann"),
			),
			O.Rating().Stars(),
		),
		Add(1),
		BlogPost{
			Ratings: []Rating{
				Rating{
					Author: "Max Mustermann",
					Stars:  0,
				},
				Rating{
					Author: "Erika Mustermann",
					Stars:  0,
				},
			},
		},
	)
	fmt.Println(result)

In order to identify the nested elements within an immutable data structure to update go optics must provide a powerful query language. As a side effect this query language can also be used to extract data from the data structure.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

	result := MustGet(
		SliceOf(
			Filtered(
				O.BlogPost().Ratings().Traverse(),
				O.Rating().Author().Eq("Max Mustermann"),
			),
			10, //Initial slice length
		),
		BlogPost{
			Ratings: []Rating{
				Rating{
					Author: "Max Mustermann",
					Stars:  0,
				},
				Rating{
					Author: "Erika Mustermann",
					Stars:  0,
				},
			},
		},
	)
	fmt.Println(result)

Will return a slice of all the ratings with author == “Max Mustermann”

Filtering Combinators #

The following combinators are provided to locate elements to update.

• Taking focuses on the first N elements.
• Dropping skips the first N elements.
• TakingWhile focuses on the elements until the predicate fails.
• DroppingWhile skips elements until the predicate fails.
• TrimmingWhile skips leading and trailing elements until the predicate fails.
• Filtered focuses on elements that match a predicate.
• Element focuses on an element as a given 0 based offset.
• At focuses on a specific key in a map.
• Coalesce focuses on the elements of the first of a list of optics that has any elements.
• First focuses on the first element
• FirstOrDefault focuses on the first element or a default value.
• Last focuses on the last element
• MaxOf focuses on the element with the maximum value.
• MinOf focuses on the element with the minimum value.
• Index focuses on the elements with the given index.
• Indices focuses on the elements whose index matched a predicate.

Many of these combinators have index aware version suffixed with an I that are able to apply their filtering taking the index value into account.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

	result := MustModify(
		Compose(
			FilteredI(
				O.BlogPost().Ratings().Traverse(),
				OpOnIx[Rating](Even[int]()),
			),
			O.Rating().Stars(),
		),
		Add(1),
		BlogPost{
			Ratings: []Rating{
				Rating{
					Author: "Max Mustermann",
					Stars:  0,
				},
				Rating{
					Author: "Erika Mustermann",
					Stars:  0,
				},
			},
		},
	)
	fmt.Println(result)

Here FilteredI is used instead of Filtered meaning the index will be taken into account during filtering. OpOnIx is used to make the built in Even predicate operate on the index rather than the focus. The net effect is that the Add(1) will now be applied to only Ratings with an even slice index.

Predicate Combinators #

So far we have only focused on simple predicates matching a single value.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

	result := MustModify(
		Compose(
			Filtered(
				O.BlogPost().Ratings().Traverse(),
				O.Rating().Author().Eq("Max Mustermann"),
			),
			O.Rating().Stars(),
		),
		Add(1),
		BlogPost{
			Ratings: []Rating{
				Rating{
					Author: "Max Mustermann",
					Stars:  0,
				},
				Rating{
					Author: "Erika Mustermann",
					Stars:  0,
				},
			},
		},
	)
	fmt.Println(result)

Here the filter predicate only compares the Author field to a fixed value. Lets extend this to only match ratings for “Max Mustermann” that are also less than 5.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

	result := MustModify(
		Compose(
			Filtered(
				O.BlogPost().Ratings().Traverse(),
				AndOp(
					O.Rating().Author().Eq("Max Mustermann"),
					O.Rating().Stars().Lt(5),
				),
			),
			O.Rating().Stars(),
		),
		Add(1),
		BlogPost{
			Ratings: []Rating{
				Rating{
					Author: "Max Mustermann",
					Stars:  0,
				},
				Rating{
					Author: "Erika Mustermann",
					Stars:  0,
				},
			},
		},
	)
	fmt.Println(result)

AndOp is a combinator that is able to take 2 predicate optics with the same source and perform a logical “and” on their results. Any 2 optics can be used as parameters to AndOp as long as they have the same source type and focus a boolean value.

The following logical combinators are provided.

• AndOp performs a logical and over 2 optics.
• OrOp performs a logical or over 2 optics.
• NotOp performs a logical not over an optic.

Reordering Combinators #

All combinators we have encountered so far have preserved the order of the focused elements. Go optics provides 2 built in combinators for re-ordering.

1. Ordered
2. Reversed

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

	data := []int{3, 2, 5, 4, 1}

	ordered := Ordered(
		TraverseSlice[int](),
		OrderBy[int](Identity[int]()),
	)

	result := MustGet(
		SliceOf(
			ordered,
			len(data),
		),
		data,
	)

	fmt.Println(result)

This example sorts a slice of ints (TraverseSlice ) by the int value themselves (Identity). The result is.

1

		[]int{1, 2, 3, 4, 5},

When using the MustModify action the results are a little un-intuitive.

1
2
3
4
5

	result := MustModify(
		ordered,
		Mul(2),
		data,
	)

In this case the result is

1

		[]int{6, 4, 10, 8, 2},

Each value has been multiplied by 2 but the results are in the original order. This is due to the Identity rule that defines that the result of using Identity as the operation in should yield the original results.

See The Identity Rule for a more detailed explanation.

Given the identity rule it may seem there is little use for a re-ordering combinator as it must always restore the original order under modification. However when used with additional filtering combinators some useful functionality is revealed.

1
2
3
4
5
6
7
8

	result := MustModify(
		Taking(
			ordered,
			2,
		),
		Mul(10),
		data,
	)

Taking(2) focuses on the first 2 elements of the ordered slice of ints. Which are then multiplied by 10 Mul(10)

The result of this action is

1

		[]int{3, 20, 5, 4, 10},

The 2 lowest values (1 & 2) have been multiplied by 10 but in their original locations. This is a consequence of the Identity rule. The taking optic only delivered the first 2 sorted values to the Mul(10) operation, but had to retain the original values of all other elements. Which the Ordered combinator then put back into the original order.

This hints at some of the power of optics to focus on arbitrary nested elements but retain the original structure.

An interesting feature of Ordered is that it uses a heap sort internally so when used in conjunction with filtering combinators like Taking sorting the entire set of values is avoided.

Collection Operations #

Up to this point all updates have never removed or added any elements to a collection. Whenever we have encountered a collection like a slice we have always performed a TraverseSlice() to focus down into the elements of the collection. In order to add or remove elements in a collection we need to operate on the collection itself.

ratingsSlice := O.BlogPost().Ratings()

Note we have not used a .Traverse() here. The returned optic is a Lens that takes a BlogPost as source and returns an Collection[int,Rating]. The Collection is an immutable wrapper around the underlying slice. This wrapper is provided by the makelens tool. If we were to directly use a FieldLens it would focus the underlying slice. makelens returns a wrapper to ensure we don’t accidentally modify the slice by accident as slices are mutable and Collection is immutable.

We can use the FilteredColcombinator to remove elements from a Collection

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

	result := MustModify(
		O.BlogPost().Ratings(),
		FilteredCol[int](
			O.Rating().Author().Eq("Max Mustermann"),
		),
		BlogPost{
			Ratings: []Rating{
				Rating{
					Author: "Max Mustermann",
					Stars:  0,
				},
				Rating{
					Author: "Erika Mustermann",
					Stars:  0,
				},
			},
		},
	)

	fmt.Println(result)

The result in this case will be a blog post with all ratings with author “Max Mustermann” removed.

The following add/remove optics are provided.

• FilteredCol removes elements from the collection that match a predicate.
• AppendCol adds elements to the end of the collection.
• PrependCol adds elements to the start of the collection.

As these optics have a source and focus of Collection they are difficult to use with a slice or map directly. Go optics provides the helper functions post fixed with Slice or Map to help when dealing directly with these collections.

1
2
3
4
5
6
7

	result := MustModify(
		Identity[[]int](),
		ReversedSlice[int](),
		[]int{1, 2, 3},
	)

	fmt.Println(result)

In this case the result will be.

1

		[]int{3, 2, 1},

Re-ordering & Sorting #

In addition to adding and removing elements from a collection operations are also provided that re-order the elements within a collection. Go optics provides 2 built in operations for re-ordering collections

1. Reversed
2. OrderedCol

Optic Type Parameters and Passing Optics as Parameters #

There are 6 base interfaces types for receiving an optic as a parameter.

• Optic
• OpticRO
• Operation
• OperationI
• Predicate
• PredicateI

Optic is the widest interface and supports all optic methods. OpticRO is slightly narrower and supports read only optic methods. Operation,IxOperation,Predicate and IxPredicate are narrower again supporting only the OpGet method. They are primarily used to reduce the amount of type parameters when passing optics as parameters.

Optic #

Optic has 9 type parameters

Optic[I, S, T, A, B, RET, RW, DIR, ERR]

They have the following meaning:

• I : Index type.
• S : Source type.
• T : Modified source type.
• A : View focus type.
• B: Modified focus type.
• RET : Return type
• RW : Read Write type
• DIR : Direction type.
• ERR: Error type. Together these parameters define the types the optic is able to handle and which actions the optic supports.

The S type parameter defines input to the optic. The A type parameter defines the the output of a view operation on the optic. The I type parameter defines the type of the index for the A type parameter.

The T any B type parameters are used in polymorphic optics to convert the types under modification. For non polymorphic optics T is identical to S and B is identical to A.

Most built in optics are non polymorphic and have this signature. e.g. TraverseSlice returns an optic with the following type signature.

Optic[I, S, S, A, A, ReturnMany, ReadWrite, UniDir]

Polymorphic optics are covered in detail in a later chapter.

The RET, RW, DIR and ERR type parameters define the properties of the Optic.

RET defines how many results the optic focuses. ReturnOne means the optic focuses exactly 1 value. ReturnMany means the optic focuses 0 or more values. A ReturnMany optic cannot be passed to an action that expects exactly one value like View

RW defines whether the optic supports modification actions. A ReadOnly optic cannot be used with modification actions like Modify

DIR defines whether the optic supports reversing its operation. A UniDir optic cannot be used with BiDir actions like ReverseGet .

ERR defines whether the optic is Pure (never returns an error) or may return an error.

This table provides a mapping from these capabilities to the optic type.

ERR does not affect the optic type it only prevents usage of Must actions for impure optics.

When receiving an optic as a function parameter it is important to specify the correct constraints to enable any needed actions to be called on the optic.

To specify these constraints in a functions type parameters the following constraints are provided.

• TReturnOne
• TReadWrite
• TBiDir
• TPure The any type constraint should be used for the ReturnMany , ReadOnly, UniDir and Err cases. This is because a

1. ReturnOne optic is compatible with the ReturnMany semantics as 1 is more than 0.
2. ReadWrite optics always support reading.
3. BiDir optics always support the UniDir direction.
4. Pure optics can be safely passed to actions returning an error.

This may sound complex but a few examples will make this clearer. Here is the definition of the Get action

func Get[I, S, T, A, B any, RET TReturnOne, RW any, DIR any, ERR any](
	o Optic[I, S, T, A, B, RET, RW, DIR, ERR], source S
) (A, error) {
	...
}

He we see the get action is agnostic to the I,S,T,A and B types. It is important to define the full I,S,T,A and B types and not just fallback to the simple Optic[I,S,S,A,A] in order to support getting from polymorphic optics.

For the capabilities we see that RW DIR and ERR are set to any. This is because Get does not perform any modification or reverse operations and it returns an error so impure optics are allowed. It does however constrain RET to be TReturnOne as we expect to return exactly one result.

Trying to call the Get action with a ReturnMany optic will cause a compilation error.

//Compile error: ReturnMany does not satisfy comparable
Get(TraverseSlice[string](), []string{"a", "b"})

These compile errors can guide you to either set the correct constraints on your parameters or call compatible actions.

It is important to note these constraints are only enforced at the action level the “As*” methods of the Optic interface do not enforce these constraints. and will return undefined results if an incompatible method is called.

Lets take a look at a few more common actions.

func Modify[I, S, T, A, B any, RET any, RW TReadWrite, DIR any, ERR any, RETOP TReturnOne, ERROP any](
	o Optic[I, S, T, A, B, RET, RW, DIR],
	op Operation[A, B, RETOP, ERROP],
	source S
) (T, error) {
	...
}

The Modify action applies the op parameter to each focus in the o optic and returns the new result. We can visualise this action in this way.

The optic is able to focus on the A elements with the S The op then converts from an A to a B the optic then builds up a new T from the B’s provided by the op.

Again the RET is unconstrained we can accept ReturnOne and process the single focus or we can accept a ReturnMany and process all the focuses. RW is constraint to TReadWrite as we are performing a modification. DIR is unconstrained as we never call the reverse function. ERR is unconstrained as Modify returns an error.

Operation is a subset of the Optic interface that reduces the amount of type parameters required when dealing with operations ans predicates.

Unfortunately it is not possible to hide the RET or ERR type parameters for Operation RET should always be set to TReturnOne

Here is the AsReverseGet combinator which reverses the direction of an Iso

func AsReverseGet[I any, S, T, A, B any, RET TReturnOne, RW TReadWrite, DIR TBiDir](
	o Optic[I, S, T, A, B, RET, RW, DIR]
) Optic[Void, B, A, T, S, ReturnOne, ReadWrite, BiDir] {

This is the most constrained an optic action possible.

1. RET is constrained to only allow ReturnOne optics.
2. RW is constrained to only allow ReadWrite optics.
3. DIR is constrained to BiDir optics. The only optic that matches all these constraints is an Iso

OpticRO #

As OpticRO only supports read operations the T and B type parameters are omitted.

type OpticRO[I, S, A, RET, RW, DIR, ERR any] interface

This leads to simplified function signatures

func Get[I, S, A, any, RET TReturnOne, RW any, DIR any, ERR any](
	o OpticRO[I, S, T, RET, RW, DIR, ERR], source S
) (A, error) {
	...
}

However this comes at the cost of losing compatibility with compose. The OpticRO interface type should never be returned even if the optic is read only. You should always return the full Optic interface.

Operation & OperationI #

As Operation only support the AsOpGet method the I,T and B type parameters are omitted.

type Operation[S, A any, RET TReturnOne, ERR any] interface {
...
}

This comes at the cost of no longer being compatible with Compose. The Operation interface type should never be returned even if the optic is an operation. You should always return the full Optic interface.

OperationI re-introduces the I index type parameter.

type OperationI[I, S, A any, RET TReturnOne, ERR any] Operation[ValueI[I, S], A, RET, ERR]

Again this comes at the cost of no longer being compatible with Compose. The OperationI interface type should never be returned even if the optic is an operation. You should always return the full Optic interface.

Predicate & PredicateI #

Predicate only supports AsOpGet with a bool return type. The RETtype parameter is also omitted as the empty return is converted to a false return.

type Predicate[S any, ERR any] interface

The PredGet action should be used to perform this empty to false conversion.