I kept hearing about the term wire protocol especially Postgres wire protocol in the recent days (Looking at you cockroachdb, yugabytedb - in a good way) but never really quite understood it. Decided to implement something simple in Go to understand it better. As always, if you find anything wrong or I misunderstood please correct me.

In simple terms,

“wire” - something over network generally (but PG also supports over domain sockets)

I saw this pattern of a function taking optional parameters (variadic arguments) and I never understood why they do that until today.

There is a bit of a background on why I need to use functional pattern.

Let’s say I don’t use functional option. Then, all the arguments need to be passed. Also, difficult to set default values and implement logic for default values.

package main

import (
	"fmt"
	"log"
)

type Config struct {
	Host       string
	Port       uint
	Visibility bool
}

type clientOption func(c *Config) error

func newConfig(host string, opts ...clientOption) Config {
	c := Config{
		Host: host,
	}
	for _, option := range opts {
		err := option(&c)
		if err != nil {
			log.Printf("error %v", err)
		}
	}
	return c
}

func withPort(port uint) clientOption {
	return func(c *Config) error {
		if port > 65000 {
			return fmt.Errorf("port can't be higher than 65000")
		}
		c.Port = port
		if c.Port == 0 {
			c.Port = 5432
		}
		return nil
	}
}

func main() {
	c := newConfig("localhost", withPort(100000))
	fmt.Println("config: ", c)
}

1. double pointers

A pointer (whose value is another variable’s address) which points to another pointer

2. copying a pointer to another pointer copies the address

3. copying a de-referenced pointer to another de-referenced pointer updates the value of the one to another

https://gist.github.com/viggy28/604cda186ff1b872d562b31ca3976732

package main

import "fmt"

type DB struct {
	Name string
}

// double pointer
// a pointer which points to another pointer
func wrapperFunc(config **DB) func() error {
	return func() error {
		cfg := &DB{}
		(*cfg).Name = "hello"
		*config = cfg
		return nil
	}
}

// config = cfg assigns the address which cfg has to config
// hence dd1 in main() doesn't get impacted
func wrapperFunc1(config *DB) func() error {
	return func() error {
		cfg := &DB{}
		fmt.Println("address of config: ", &config)
		fmt.Printf("  wrapperFunc1 A: config is %p, cfg is %p\n", config, cfg)
		cfg.Name = "hello"
		config = cfg
		fmt.Printf("  wrapperFunc1 B: config is %p, cfg is %p\n", config, cfg)
		return nil
	}
}

// *config = *cfg value at address cfg has will be copied to value at the address that config has
// hence dd2 in main() does gets updated
func wrapperFunc2(config *DB) func() error {
	return func() error {
		cfg := &DB{}
		fmt.Printf("  wrapperFunc2 A: config is %p, cfg is %p\n", config, cfg)
		cfg.Name = "hello"
		fmt.Println("value at ", *cfg)
		*config = *cfg
		fmt.Printf("  wrapperFunc2 B: config is %p, cfg is %p\n", config, cfg)
		return nil
	}
}

func wrapperFunc3(config *DB) func() error {
	return func() error {
		config.Name = "hello"
		return nil
	}
}

func wrapperFunc4(config DB) func() error {
	return func() error {
		config.Name = "hello"
		return nil
	}
}

func main() {
	{
		var d *DB
		fn := wrapperFunc(&d)
		fn()
		fmt.Printf("d.Name: %q\n", d.Name)
	}

	{
		dd1 := &DB{}
		fmt.Println("address of dd1: ", &dd1)
		fmt.Printf("main A: dd1 is %p\n", dd1)
		fn1 := wrapperFunc1(dd1)
		fn1()
		fmt.Printf("main B: dd1 is %p\n", dd1)
		fmt.Printf("dd1.Name: %q\n", dd1.Name)
	}

	{
		dd2 := &DB{}
		fmt.Printf("main C: dd2 is %p\n", dd2)
		fn2 := wrapperFunc2(dd2)
		fn2()
		fmt.Printf("main D: dd2 is %p\n", dd2)
		fmt.Printf("dd2.Name: %q\n", dd2.Name)
	}

	{
		dd3 := &DB{}
		fn3 := wrapperFunc3(dd3)
		fn3()
		fmt.Printf("dd3.Name: %q\n", dd3.Name)
	}

	{
		dd4 := DB{}
		fn4 := wrapperFunc4(dd4)
		fn4()
		fmt.Printf("dd4.Name: %q\n", dd4.Name)
	}
}

context is a standard package. Context is an interface in context package.

type Context interface {
	Deadline() (deadline time.Time, ok bool)
	Done() <-chan struct{}
	Err() error
	Value(key interface{}) interface{}
}

There are 4 methods in Context interface. Most of the times, we don’t define our own types which satisfy the interface (which off course we can do), but we mostly use the factory functions provided by the context package itself.

They are Background(), TODO(), WithCancel(), WithTimeout() etc. The only difference is WithDeadline sets the termination time point, whileWithTimeout sets the maximum running time.

1. go run command go run command is a quick way to run go program. However, what it does behind the screen is it compiles the program and runs an executable and removes it once the program is stopped.

2. byte is a built-in alias of uint8. rune is a built-in alias of int32.

3. The semicolon insertion rule In Go, I thought there is no need for semicolon at the end of the statement. It is true (as a developer I don’t need to) however lexer (which I don’t know what it is; sounds similar to compiler, interpretor) does that behind the screen. Refer https://golang.org/doc/effective_go#semicolons

closure is a function inside another function where you can reference variable defined in the outer function.

I have seen them useful in dealing with packages where a function expects an type func() as an argument where the func() doesn’t take any argument.

example from package github.com/cenkalti/backoff

signature of backoff.Retry is as below

backoff.Retry(o backoff.Operation, b backoff.Backoff)

where backoff.Operation is of type func() error

type Operation func() error

func main() {
    func retryWrapper(imp string) func() error{
        return func() {
            funcThatCouldFail(imp)
            retun nil
        }
    }
    err := backoff.Retry(retryWrapper("hello world"), backoff.NewExponentialBackOff())
}

we can’t directly call funcThatCouldFail(imp) in backoff.Retry(). So, we create a wrapper which takes an argument and that argument is being used inside of the func that being returned.

defer is one of the unique keywords in Go.

What does defer do? As the name suggests, it defers something. It defers function calls. A deferred function gets invoked right after the surrounding function returns.

func outer() {
    defer inner()
    log.Println("outer got called")
}

func inner() {
    log.Println("inner got called")
}

outer got called prints first then inner got called.

What the main use of defer statement? It helps mainly with actions that need to happen but not immediately. For eg. closing a file that’s opened or closing a database connection.

I came across this piece of code from The Go Programming Language book.

loop:
	for {
		select {
		case size, ok := <-fileSizes:
			if !ok {
				break loop // fileSizes was closed
			}
			nfiles++
			nbytes += size
		case <-tick:
			printDiskUsage(nfiles, nbytes)
		}
	}

The complete program is available in Github

Named break statement allows to break perhaps multiple control statements. In this example, break loop breaks the select and for statement.

I was going through The Go Programming Language and in chapter 5 there was a section Caveat: Capturing Iteration Variables.

The example is like this:

var rmdirs []func()

for _, dir := range tempDirs() {                    // 1
    os.MkdirAll(dir, 0755)                          // 2
    rmdirs = append(rmdirs, func() {
        os.RemoveAll(dir) // NOTE: incorrect!       // 3
    })
}

1. dir is the same variable. i.e. the address of the variable is same
2. dir’s value evaluated at this point is different at each iteration
3. IMPORTANT dir is not evaluated here. Its just being used. It will be evaluated only when the func() is called

The following explanation from Ref[1]:

program1

package main

import (
	"log"
)

func main() {
	done := make(chan struct{})
	go func() {
		log.Println("done")
	}()
	<-done //wait for background goroutine to finish
}

program2 run nc -l 8000 on another terminal

package main

import (
	"log"
	"net"
)

func main() {
	conn, err := net.Dial("tcp", "localhost:8000")
	if err != nil {
		log.Fatal(err)
	}
	conn.Close()
	done := make(chan struct{})
	go func() {
		log.Println("done")
	}()
	<-done //wait for background goroutine to finish
}

Both of those examples have a receive statement from a channel that no one ever sends to. Such a statement must block forever, meaning the goroutine that contains it can never run. If there are no goroutines that can run, Go terminates the program with a deadlock message.

1. Initializing embedded field: The field name of an anonymous field during a struct instantiation is the name of the struct (Time)

type Event struct {
    ID int
    time.Time 
    }

event := Event{
    ID:   1234,
    Time: time.Now(),
}

2. If an embedded field type implements an interface, the struct containing the embedded field will also implement this interface

Concurrency

channels are like typed pipes, Where you can send something from one side and receive from other side. using channel direction operator <-. Depends on the direction of the variables around the operator, its either sending or receiving. x := <-c // receive from c c <- sum // send sum to c fmt.Println(“channel receive happening”, <-c) channels are reference type. similar to maps. So when we pass channel as an argument to a function or copy we are passing the reference.

In Go, copying generally does a copy by value. So modifying the destination, doesn’t modify the source. However, that holds good only if you are copying value types not reference types (i.e. pointer, slice, map etc.)

package main

import (
	"fmt"
)

func main() {

	type Cat struct {
		age     int
		name    string
		Friends []string //reference type
	}
	wilson := Cat{7, "Wilson", []string{"Tom", "Tabata", "Willie"}}
	// shallow copy
	nikita := wilson
	// modifies both nikita and wilson
	nikita.Friends[0] = "cruise"
	fmt.Println("nikita:", wilson)
	fmt.Println("wilson:", wilson)
	// Here I am modifying the slice too, but whats stored in nikita.Friends is the address of new slice which append returns not wilson.Friends
	nikita.Friends = append(wilson.Friends, "jerry")
	fmt.Println("nikita:", nikita)
	fmt.Println("wilson:", wilson)
	// If the above doesn't makes sense this should. Created a new slice data
	data := []string{"popeye"}
	// shallow copy data to nikita Friends
	nikita.Friends = data
	// changing nikita Friends should change data too
	nikita.Friends[0] = "pluto"
	fmt.Println("nikita:", nikita)
	fmt.Println("wilson:", wilson)
	fmt.Println("data:", data)
}

This code explains one of the gotchas that I ran into. nikita.Friends = append(wilson.Friends, "jerry") I was expecting wilson.Friends will also have jerry but since i did shallow copy nikita := wilson. However, that’s not the case. I realized nikita.Friends = append(wilson.Friends, "jerry") actually creates a new slice and stores that’s reference. The data should clarify that further.

Disclaimer: Whatever I write here is my learnings. So, don’t take these words as granted or reference. Also, please lmk if there are any errors.

I have often come across this term JWT. I knew the definition - Json Web Token, but honestly that’s all I knew for a veryyyy longgg time. Recently, I came across a problem where I have been told that JWT might be a good solution for it. That piqued my interest.

Libpq-Postgres-Go

After I started working on Postgres, I have heard this term libpq enough times, but never had a good grasp of it. After digging around this topic for a couple of days, here is my understanding.

From the Postgres doc https://www.postgresql.org/docs/9.5/libpq.html,

libpq is the C application programmer’s interface to PostgreSQL. libpq is a set of library functions that allow client programs to pass queries to the PostgreSQL backend server and to receive the results of these queries

Understanding context in Golang through Postgres

I was trying to learn Go package context especially with respect to Postgres. On a very high level context provides context to the operation. Yeah, I agree, the previous statement doesn’t really add much value, but hold on I don’t really know how to explain it, rather let’s go over some code. Sometimes its easier to understand by seeing it in action :)

The complete program is here