Seven attempts, seven different wrong answers — lessons from setting up a local LLM.

It’s tax season, which means I’ve been staring at a notes file full of stock sales — 23 transactions across the year that I needed to total up. The kind of data I’d rather not paste into a chat window I don’t control.

I’d been meaning to set up a local LLM anyway, and this seemed like the perfect low-stakes test. M3 Max, 64GB of unified memory, plenty of headroom for a real model. I installed Ollama, pulled Qwen 2.5 Coder, pasted the list, asked the question.

Postgres doesn’t support active-active replication natively. As of this writing, we have to rely on 3rd party tools. I decided to go with [BDR] (https://www.2ndquadrant.com/en/resources/postgres-bdr-2ndquadrant/).

I didn’t want to spin up multiple VMs. So, the obvious choice is docker. Make sure you have docker on mac & docker compose.

Step1: Running 2 Postgres instances using docker container

Thanks to [jgiannuzzi] (https://github.com/jgiannuzzi), who created a docker [image] (https://hub.docker.com/r/jgiannuzzi/postgres-bdr) with Postgres and BDR.

This is a guest post by Suvam Prasad (https://www.linkedin.com/in/suvamprasad/). I am very glad to have his post.

This book is very special to me because it helped me to discover new perspectives of web designing mythology. The book is written by Steve Krug(a famous user experience professional). The book contained hardly 216 pages that give you enough knowledge and idea to understand web design usability, user accessibility, and how to be a great user experience designer. Most of the concepts mentioned are very beginner centric. The book revolves around web usability with actionable guidelines, conduct usability testing, understanding user interface design, which should be UCD(User-Centered Design). The book starts with the user interface myths. Whenever the user visits any site, they try to search the specific keyword or word; they are looking for their needs. The users are not going to read the long passage or articles on the site. It is unnecessary to include long paragraphs on the website if it is an e-commerce website or a selling-product-based website. The user always in a hurry to satisfy their itch, and the product should act as a painkiller.

TLDR; A short book on Product Development and especially explains how B2C (especially social media) companies make their users hooked on their products. Though Product owners, designers, and founders can use this book in improving their products, every user can read this to understand why some of the popular apps do things (like infinite scroll, weekly digest emails, notifications) which we use day-to-day. Hooked also overlaps with Nir Eyal’s other book Undistractable

Disclaimer: This book was published in 1989. So, we have to time travel to understand a few things, but overall the concepts are still relevant. Also, read this during COVID-19 (middle of a global pandemic).

This review is a little different than other books. Going to share most of the notes as points.

Peter Lynch is America’s money manager. He was Vice-Chairman of the Fidelity Management & Research Company. He was the portfolio manager of Fidelity Magellan Fund, which was the best performing fund in the world from May 1977 to May 1990.

Part 1 is about Master Internal Triggers which talks about that distractions are mainly caused by discomfort. Most people don’t want to acknolwedge the uncomfortable truth that distraction is always an unhealthy escape from reality. Being indistractable means striving to do what you say you will do.

We are compelled to reach for things we supposedly need but really don’t. (eg. checking emails)

We tend to blame things like television, junk food, social media etc - but these are all promiate causes of our distraction, but not the root cause.

Here is a list of books which I really enjoyed reading. They are mostly around the topics such as Self Development, Management, Finance, Technology.

• Give and Take by Adam Grant
• Atomic Habits by James Clear
• A random walkdown wall street
• Designing Data Intensive Application
• surely you’re joking mr. feynman
• How To Win Friends and Influence People
• Lean Startup
• The Go programming languate
• Make your bed
• Hard thing about hard things
• One Up On Wall Street
• Learn to Earn
• Indistractable
• Thinking fast and slow (I couldn’t complete this book. Its too much content)
• Hooked

Chapter 1 is Good Returns which explains the Dangers and Rewards of Giving more than you get with multiple examples. One of the things which striked me

You can’t just ignore someone because you don’t think they’re important enough.

I can’t agree more on that. I have seen first class that founders making this mistake. Its so easy to just ignore, but think about the long term or big picture. First chapter, goes over some of the examples of givers approaches. Especially David Hornik some of his quotes and thoughts are note worthy

Admiral William H. McRaven served as Navy SEAL for 37 years. As a Four-Start Admiral, his last assignment was as Commander of all U.S. Special Operations Forces.

He attended the University of Texas, Austin. This book is based on his commencement speech at his alma mater. He served as the Chancellor for the University of Texas system between 2015 and 2018.

Summary

Chapter 1: Start Your Day With a Task Completed

If you want to change the world… start off by making your bed.

Summary

Chapter 1: Tutorial

Introduction to various major compoents of Golang

Topics such as data types, functions, packages, concurrency in Go was illustrated using simple programs.

Chapter 2: Program Structure

For every type T, there is a corresponding conversion operation T(x) that converts value of x to type T (if applicable)

I was trying to play with Phil’s pgtam and the first step is that to install Postgres version 16. Sounds fairly innocous. When I ran configure, I was getting the below error:

checking for icu-uc icu-i18n... no
configure: error: ICU library not found
If you have ICU already installed, see config.log for details on the
failure.  It is possible the compiler isn't looking in the proper directory.
Use --without-icu to disable ICU support.

Obviously, one solution is to disable ICU support as recommended by the error message. However, that’s an anti-feature as per the Postgres documentation.

Libpq-Postgres-Go

After I started working on Postgres, I have heard this term libpq enough times, but never had a good grasp of it. After digging around this topic for a couple of days, here is my understanding.

From the Postgres doc https://www.postgresql.org/docs/9.5/libpq.html,

libpq is the C application programmer’s interface to PostgreSQL. libpq is a set of library functions that allow client programs to pass queries to the PostgreSQL backend server and to receive the results of these queries

closure is a function inside another function where you can reference variable defined in the outer function.

I have seen them useful in dealing with packages where a function expects an type func() as an argument where the func() doesn’t take any argument.

example from package github.com/cenkalti/backoff

signature of backoff.Retry is as below

backoff.Retry(o backoff.Operation, b backoff.Backoff)

where backoff.Operation is of type func() error

type Operation func() error

func main() {
    func retryWrapper(imp string) func() error{
        return func() {
            funcThatCouldFail(imp)
            retun nil
        }
    }
    err := backoff.Retry(retryWrapper("hello world"), backoff.NewExponentialBackOff())
}

we can’t directly call funcThatCouldFail(imp) in backoff.Retry(). So, we create a wrapper which takes an argument and that argument is being used inside of the func that being returned.

I was writing an article for Infoq on the topic less spoken costs of managed databases and one question that the editor asked is how much margin RDS makes compared to running a Postgres instance on EC2? That intrigued me and honestly, I never did the math so far. I have been using AWS-managed databases since 2016 and I thought what’s a better time to do an analysis of Cloud cost than today ?!

In Google Cloud Next 2019, they introduced a new product called [Cloud Run] (https://cloud.google.com/run/). I’ve been using it from the EAP (Early Access Program) days. [As the name suggests, basically it runs your docker image. You might be familiar with other serverless products such as [Cloud Function] (https://cloud.google.com/functions/) or [lambda] (https://aws.amazon.com/lambda/) where you provide your source code instead of a docker image. In my opinion, Cloud Run is more flexible than functions. Let me explain why.

This is part of my Today I Learnt series where I share whatever I am learning something new. After having issues with port forwarding in Xfinity I decided to look for alternative solutions. I have used Cloudflare tunnel (used to be called Argo Tunnel) in the past to expose websites running on my laptop to the Internet. So, I decided to try it out for Ghost blogging site which I am setting up for my dad.

This aricle is inspired by [Give and Take] (https://www.adamgrant.net/give-and-take) by [Adam Grant] (https://www.adamgrant.net/). I was wondering how can I help my friends and network. Searching a job is something stressful and we all have been through. Once my wife and I searched for a year and finally we moved to San Francisco.

So far its been a challenge for my network to know whether I can refer to a particular company. To solve this, I am publishing a list of companies where I can refer. If you don’t know, now you know (Yeah, we have been watching a lot of Trevor Noah shows)

Mark my word. We had so many competitions in the past. Eg. vim vs emacs, tabs vs spaces etc. The new comparison which is added in this decade is functions vs containers

Both the functions and containers can be serverless. So, please avoid the notion of serverless means functions. In fact, predominantly all the functions (like lambda, cloud functions) are running behind the screen as containers. (there are few products like Cloudflare workers are running without containers)

In this article we will see how to convert a simple python application to a containerized (Docker) one

Go to the profile of vignesh ravichandran vignesh ravichandran Sep 23, 2017 TL;DR -> Containerize a python app. Push the image and execute from Google Cloud.

On a leisurely Saturday afternoon, I thought its nice to play with containers. I was fascinated with the idea of containerizing an app and running it across different environments. Its easier (for me) to understand something when I do, instead of just reading.

The analytics databases optimized for queries. The real leverage was always where data gets written.

Your application writes to Postgres. So does almost everyone else’s. Postgres is the most widely used database in the world, topping Stack Overflow’s developer survey two years running. It didn’t win as a data warehouse. It won as the place applications write.

For years, the data infrastructure companies competed on what happens after — faster queries, better compression, smarter optimizers. The traditional model relied on storage gravity: move your data into our walls, pay us to store and query it. Once data lived inside a warehouse, the switching cost kept it there.

The last line of defense for those OOPS moments — so even DBAs can sleep peacefully.

Have you ever accidentally run a DELETE without a WHERE clause? Or typed DROP TABLE thinking you were in staging, only to realize a half-second too late that you were in prod?

If yes — keep reading, this is for you. If no — keep reading anyway. Nobody who uses Postgres long enough stays immune.

Citus: Distributed PostgreSQL for Data-Intensive Applications paper can be downloaded here.

Recently, our team got a request to provide a solution to shard Postgres. One of the solutions that we discussed was Citus. I have heard about the product and seen their blogs related to Postgres in the past but never used it. I thought it would be fun to read about its internal workings.

If you find something wrong on the notes, please send a pull request. Before digging their white paper, let’s take a step back and ask what is sharding and why do we need sharding?

In Google Cloud Next 2019, they introduced a new product called [Cloud Run] (https://cloud.google.com/run/). I’ve been using it from the EAP (Early Access Program) days. [As the name suggests, basically it runs your docker image. You might be familiar with other serverless products such as [Cloud Function] (https://cloud.google.com/functions/) or [lambda] (https://aws.amazon.com/lambda/) where you provide your source code instead of a docker image. In my opinion, Cloud Run is more flexible than functions. Let me explain why.

Step 1: Setting up HAProxy

I hope you gone through [Part1] (https://viggy28.dev/article/postgres-active-active-replication/) of this series. Perhaps, one thing you might have noticed is that I’ve to connect to the specific master database. In our case, since both the databases are running on docker, only the localhost port is different. (In a production environment, most likely you going to run the databases on a different host). The main reason for active-active replication is high availability. If one of the nodes goes down, you still want your application to behave normally. You don’t want to hard-code your DSN on the application or keep checking the health of the database every time before you make a connection. Fair enough. [HAProxy] (<www.haproxy.org>), an open source project solves this particular change.

Postgres doesn’t support active-active replication natively. As of this writing, we have to rely on 3rd party tools. I decided to go with [BDR] (https://www.2ndquadrant.com/en/resources/postgres-bdr-2ndquadrant/).

I didn’t want to spin up multiple VMs. So, the obvious choice is docker. Make sure you have docker on mac & docker compose.

Step1: Running 2 Postgres instances using docker container

Thanks to [jgiannuzzi] (https://github.com/jgiannuzzi), who created a docker [image] (https://hub.docker.com/r/jgiannuzzi/postgres-bdr) with Postgres and BDR.

In this article we will see how to convert a simple python application to a containerized (Docker) one

Go to the profile of vignesh ravichandran vignesh ravichandran Sep 23, 2017 TL;DR -> Containerize a python app. Push the image and execute from Google Cloud.

On a leisurely Saturday afternoon, I thought its nice to play with containers. I was fascinated with the idea of containerizing an app and running it across different environments. Its easier (for me) to understand something when I do, instead of just reading.

The last line of defense for those OOPS moments — so even DBAs can sleep peacefully.

Have you ever accidentally run a DELETE without a WHERE clause? Or typed DROP TABLE thinking you were in staging, only to realize a half-second too late that you were in prod?

If yes — keep reading, this is for you. If no — keep reading anyway. Nobody who uses Postgres long enough stays immune.

I am writing this around the time where we are at the beginning of a global pandemic. Everyone is trying to figure out how to keep their business up and running during this challenging period. Most of the employers have mandated work from remote policy. One of the main challenges is how to connect to your applications remotely. Traditionally the answer is VPN. There are so many challenges wrt scalability, availability, and performance of these traditional VPN software. Don’t worry, there are alternatives.

When I first heard GPG it was little daunting. I haven’t used it exactly, even though I was familar with the concept. So, I decided to dig deeper and understand how to use it.

GnuPG (GPG) is an implementation of PGP

The simple idea is that you generate a key pair: secret or private key and a public key. You can provide your public key to anyone. They can then encrypt either a secret or file using your public key. You are the only one who can decrypt the secret or file (assuming you aren’t sharing your secret key with others, which is bad).

We all at some point in time thought about having a personal website. It may be for writing about your travel trips, posting pictures of food that you cooked or ranting about the tech which you don’t like. I will leave the reason to you

Step1: Buying a domain

[This] (https://viggy28.dev) websites domain name is viggy28.dev. You want a human friendly name, so that your friends, family and rest of the internet can reach your website.

In Google Cloud Next 2019, they introduced a new product called [Cloud Run] (https://cloud.google.com/run/). I’ve been using it from the EAP (Early Access Program) days. [As the name suggests, basically it runs your docker image. You might be familiar with other serverless products such as [Cloud Function] (https://cloud.google.com/functions/) or [lambda] (https://aws.amazon.com/lambda/) where you provide your source code instead of a docker image. In my opinion, Cloud Run is more flexible than functions. Let me explain why.

I kept hearing about the term wire protocol especially Postgres wire protocol in the recent days (Looking at you cockroachdb, yugabytedb - in a good way) but never really quite understood it. Decided to implement something simple in Go to understand it better. As always, if you find anything wrong or I misunderstood please correct me.

In simple terms,

“wire” - something over network generally (but PG also supports over domain sockets)

I saw this pattern of a function taking optional parameters (variadic arguments) and I never understood why they do that until today.

There is a bit of a background on why I need to use functional pattern.

Let’s say I don’t use functional option. Then, all the arguments need to be passed. Also, difficult to set default values and implement logic for default values.

package main

import (
	"fmt"
	"log"
)

type Config struct {
	Host       string
	Port       uint
	Visibility bool
}

type clientOption func(c *Config) error

func newConfig(host string, opts ...clientOption) Config {
	c := Config{
		Host: host,
	}
	for _, option := range opts {
		err := option(&c)
		if err != nil {
			log.Printf("error %v", err)
		}
	}
	return c
}

func withPort(port uint) clientOption {
	return func(c *Config) error {
		if port > 65000 {
			return fmt.Errorf("port can't be higher than 65000")
		}
		c.Port = port
		if c.Port == 0 {
			c.Port = 5432
		}
		return nil
	}
}

func main() {
	c := newConfig("localhost", withPort(100000))
	fmt.Println("config: ", c)
}

1. double pointers

A pointer (whose value is another variable’s address) which points to another pointer

2. copying a pointer to another pointer copies the address

3. copying a de-referenced pointer to another de-referenced pointer updates the value of the one to another

https://gist.github.com/viggy28/604cda186ff1b872d562b31ca3976732

package main

import "fmt"

type DB struct {
	Name string
}

// double pointer
// a pointer which points to another pointer
func wrapperFunc(config **DB) func() error {
	return func() error {
		cfg := &DB{}
		(*cfg).Name = "hello"
		*config = cfg
		return nil
	}
}

// config = cfg assigns the address which cfg has to config
// hence dd1 in main() doesn't get impacted
func wrapperFunc1(config *DB) func() error {
	return func() error {
		cfg := &DB{}
		fmt.Println("address of config: ", &config)
		fmt.Printf("  wrapperFunc1 A: config is %p, cfg is %p\n", config, cfg)
		cfg.Name = "hello"
		config = cfg
		fmt.Printf("  wrapperFunc1 B: config is %p, cfg is %p\n", config, cfg)
		return nil
	}
}

// *config = *cfg value at address cfg has will be copied to value at the address that config has
// hence dd2 in main() does gets updated
func wrapperFunc2(config *DB) func() error {
	return func() error {
		cfg := &DB{}
		fmt.Printf("  wrapperFunc2 A: config is %p, cfg is %p\n", config, cfg)
		cfg.Name = "hello"
		fmt.Println("value at ", *cfg)
		*config = *cfg
		fmt.Printf("  wrapperFunc2 B: config is %p, cfg is %p\n", config, cfg)
		return nil
	}
}

func wrapperFunc3(config *DB) func() error {
	return func() error {
		config.Name = "hello"
		return nil
	}
}

func wrapperFunc4(config DB) func() error {
	return func() error {
		config.Name = "hello"
		return nil
	}
}

func main() {
	{
		var d *DB
		fn := wrapperFunc(&d)
		fn()
		fmt.Printf("d.Name: %q\n", d.Name)
	}

	{
		dd1 := &DB{}
		fmt.Println("address of dd1: ", &dd1)
		fmt.Printf("main A: dd1 is %p\n", dd1)
		fn1 := wrapperFunc1(dd1)
		fn1()
		fmt.Printf("main B: dd1 is %p\n", dd1)
		fmt.Printf("dd1.Name: %q\n", dd1.Name)
	}

	{
		dd2 := &DB{}
		fmt.Printf("main C: dd2 is %p\n", dd2)
		fn2 := wrapperFunc2(dd2)
		fn2()
		fmt.Printf("main D: dd2 is %p\n", dd2)
		fmt.Printf("dd2.Name: %q\n", dd2.Name)
	}

	{
		dd3 := &DB{}
		fn3 := wrapperFunc3(dd3)
		fn3()
		fmt.Printf("dd3.Name: %q\n", dd3.Name)
	}

	{
		dd4 := DB{}
		fn4 := wrapperFunc4(dd4)
		fn4()
		fmt.Printf("dd4.Name: %q\n", dd4.Name)
	}
}

context is a standard package. Context is an interface in context package.

type Context interface {
	Deadline() (deadline time.Time, ok bool)
	Done() <-chan struct{}
	Err() error
	Value(key interface{}) interface{}
}

There are 4 methods in Context interface. Most of the times, we don’t define our own types which satisfy the interface (which off course we can do), but we mostly use the factory functions provided by the context package itself.

They are Background(), TODO(), WithCancel(), WithTimeout() etc. The only difference is WithDeadline sets the termination time point, whileWithTimeout sets the maximum running time.

1. go run command go run command is a quick way to run go program. However, what it does behind the screen is it compiles the program and runs an executable and removes it once the program is stopped.

2. byte is a built-in alias of uint8. rune is a built-in alias of int32.

3. The semicolon insertion rule In Go, I thought there is no need for semicolon at the end of the statement. It is true (as a developer I don’t need to) however lexer (which I don’t know what it is; sounds similar to compiler, interpretor) does that behind the screen. Refer https://golang.org/doc/effective_go#semicolons

closure is a function inside another function where you can reference variable defined in the outer function.

I have seen them useful in dealing with packages where a function expects an type func() as an argument where the func() doesn’t take any argument.

example from package github.com/cenkalti/backoff

signature of backoff.Retry is as below

backoff.Retry(o backoff.Operation, b backoff.Backoff)

where backoff.Operation is of type func() error

type Operation func() error

func main() {
    func retryWrapper(imp string) func() error{
        return func() {
            funcThatCouldFail(imp)
            retun nil
        }
    }
    err := backoff.Retry(retryWrapper("hello world"), backoff.NewExponentialBackOff())
}

we can’t directly call funcThatCouldFail(imp) in backoff.Retry(). So, we create a wrapper which takes an argument and that argument is being used inside of the func that being returned.

defer is one of the unique keywords in Go.

What does defer do? As the name suggests, it defers something. It defers function calls. A deferred function gets invoked right after the surrounding function returns.

func outer() {
    defer inner()
    log.Println("outer got called")
}

func inner() {
    log.Println("inner got called")
}

outer got called prints first then inner got called.

What the main use of defer statement? It helps mainly with actions that need to happen but not immediately. For eg. closing a file that’s opened or closing a database connection.

I came across this piece of code from The Go Programming Language book.

loop:
	for {
		select {
		case size, ok := <-fileSizes:
			if !ok {
				break loop // fileSizes was closed
			}
			nfiles++
			nbytes += size
		case <-tick:
			printDiskUsage(nfiles, nbytes)
		}
	}

The complete program is available in Github

Named break statement allows to break perhaps multiple control statements. In this example, break loop breaks the select and for statement.

I was going through The Go Programming Language and in chapter 5 there was a section Caveat: Capturing Iteration Variables.

The example is like this:

var rmdirs []func()

for _, dir := range tempDirs() {                    // 1
    os.MkdirAll(dir, 0755)                          // 2
    rmdirs = append(rmdirs, func() {
        os.RemoveAll(dir) // NOTE: incorrect!       // 3
    })
}

1. dir is the same variable. i.e. the address of the variable is same
2. dir’s value evaluated at this point is different at each iteration
3. IMPORTANT dir is not evaluated here. Its just being used. It will be evaluated only when the func() is called

The following explanation from Ref[1]:

program1

package main

import (
	"log"
)

func main() {
	done := make(chan struct{})
	go func() {
		log.Println("done")
	}()
	<-done //wait for background goroutine to finish
}

program2 run nc -l 8000 on another terminal

package main

import (
	"log"
	"net"
)

func main() {
	conn, err := net.Dial("tcp", "localhost:8000")
	if err != nil {
		log.Fatal(err)
	}
	conn.Close()
	done := make(chan struct{})
	go func() {
		log.Println("done")
	}()
	<-done //wait for background goroutine to finish
}

Both of those examples have a receive statement from a channel that no one ever sends to. Such a statement must block forever, meaning the goroutine that contains it can never run. If there are no goroutines that can run, Go terminates the program with a deadlock message.

1. Initializing embedded field: The field name of an anonymous field during a struct instantiation is the name of the struct (Time)

type Event struct {
    ID int
    time.Time 
    }

event := Event{
    ID:   1234,
    Time: time.Now(),
}

2. If an embedded field type implements an interface, the struct containing the embedded field will also implement this interface

Concurrency

channels are like typed pipes, Where you can send something from one side and receive from other side. using channel direction operator <-. Depends on the direction of the variables around the operator, its either sending or receiving. x := <-c // receive from c c <- sum // send sum to c fmt.Println(“channel receive happening”, <-c) channels are reference type. similar to maps. So when we pass channel as an argument to a function or copy we are passing the reference.

In Go, copying generally does a copy by value. So modifying the destination, doesn’t modify the source. However, that holds good only if you are copying value types not reference types (i.e. pointer, slice, map etc.)

package main

import (
	"fmt"
)

func main() {

	type Cat struct {
		age     int
		name    string
		Friends []string //reference type
	}
	wilson := Cat{7, "Wilson", []string{"Tom", "Tabata", "Willie"}}
	// shallow copy
	nikita := wilson
	// modifies both nikita and wilson
	nikita.Friends[0] = "cruise"
	fmt.Println("nikita:", wilson)
	fmt.Println("wilson:", wilson)
	// Here I am modifying the slice too, but whats stored in nikita.Friends is the address of new slice which append returns not wilson.Friends
	nikita.Friends = append(wilson.Friends, "jerry")
	fmt.Println("nikita:", nikita)
	fmt.Println("wilson:", wilson)
	// If the above doesn't makes sense this should. Created a new slice data
	data := []string{"popeye"}
	// shallow copy data to nikita Friends
	nikita.Friends = data
	// changing nikita Friends should change data too
	nikita.Friends[0] = "pluto"
	fmt.Println("nikita:", nikita)
	fmt.Println("wilson:", wilson)
	fmt.Println("data:", data)
}

This code explains one of the gotchas that I ran into. nikita.Friends = append(wilson.Friends, "jerry") I was expecting wilson.Friends will also have jerry but since i did shallow copy nikita := wilson. However, that’s not the case. I realized nikita.Friends = append(wilson.Friends, "jerry") actually creates a new slice and stores that’s reference. The data should clarify that further.

Disclaimer: Whatever I write here is my learnings. So, don’t take these words as granted or reference. Also, please lmk if there are any errors.

I have often come across this term JWT. I knew the definition - Json Web Token, but honestly that’s all I knew for a veryyyy longgg time. Recently, I came across a problem where I have been told that JWT might be a good solution for it. That piqued my interest.

Libpq-Postgres-Go

After I started working on Postgres, I have heard this term libpq enough times, but never had a good grasp of it. After digging around this topic for a couple of days, here is my understanding.

From the Postgres doc https://www.postgresql.org/docs/9.5/libpq.html,

libpq is the C application programmer’s interface to PostgreSQL. libpq is a set of library functions that allow client programs to pass queries to the PostgreSQL backend server and to receive the results of these queries

Understanding context in Golang through Postgres

I was trying to learn Go package context especially with respect to Postgres. On a very high level context provides context to the operation. Yeah, I agree, the previous statement doesn’t really add much value, but hold on I don’t really know how to explain it, rather let’s go over some code. Sometimes its easier to understand by seeing it in action :)

The complete program is here

Step 1: Setting up HAProxy

I hope you gone through [Part1] (https://viggy28.dev/article/postgres-active-active-replication/) of this series. Perhaps, one thing you might have noticed is that I’ve to connect to the specific master database. In our case, since both the databases are running on docker, only the localhost port is different. (In a production environment, most likely you going to run the databases on a different host). The main reason for active-active replication is high availability. If one of the nodes goes down, you still want your application to behave normally. You don’t want to hard-code your DSN on the application or keep checking the health of the database every time before you make a connection. Fair enough. [HAProxy] (<www.haproxy.org>), an open source project solves this particular change.

I am writing this around the time where we are at the beginning of a global pandemic. Everyone is trying to figure out how to keep their business up and running during this challenging period. Most of the employers have mandated work from remote policy. One of the main challenges is how to connect to your applications remotely. Traditionally the answer is VPN. There are so many challenges wrt scalability, availability, and performance of these traditional VPN software. Don’t worry, there are alternatives.

When I first heard GPG it was little daunting. I haven’t used it exactly, even though I was familar with the concept. So, I decided to dig deeper and understand how to use it.

GnuPG (GPG) is an implementation of PGP

The simple idea is that you generate a key pair: secret or private key and a public key. You can provide your public key to anyone. They can then encrypt either a secret or file using your public key. You are the only one who can decrypt the secret or file (assuming you aren’t sharing your secret key with others, which is bad).

We all at some point in time thought about having a personal website. It may be for writing about your travel trips, posting pictures of food that you cooked or ranting about the tech which you don’t like. I will leave the reason to you

Step1: Buying a domain

[This] (https://viggy28.dev) websites domain name is viggy28.dev. You want a human friendly name, so that your friends, family and rest of the internet can reach your website.

I am writing this around the time where we are at the beginning of a global pandemic. Everyone is trying to figure out how to keep their business up and running during this challenging period. Most of the employers have mandated work from remote policy. One of the main challenges is how to connect to your applications remotely. Traditionally the answer is VPN. There are so many challenges wrt scalability, availability, and performance of these traditional VPN software. Don’t worry, there are alternatives.

When I first heard GPG it was little daunting. I haven’t used it exactly, even though I was familar with the concept. So, I decided to dig deeper and understand how to use it.

GnuPG (GPG) is an implementation of PGP

The simple idea is that you generate a key pair: secret or private key and a public key. You can provide your public key to anyone. They can then encrypt either a secret or file using your public key. You are the only one who can decrypt the secret or file (assuming you aren’t sharing your secret key with others, which is bad).

We all at some point in time thought about having a personal website. It may be for writing about your travel trips, posting pictures of food that you cooked or ranting about the tech which you don’t like. I will leave the reason to you

Step1: Buying a domain

[This] (https://viggy28.dev) websites domain name is viggy28.dev. You want a human friendly name, so that your friends, family and rest of the internet can reach your website.

Disclaimer: Whatever I write here is my learnings. So, don’t take these words as granted or reference. Also, please lmk if there are any errors.

I have often come across this term JWT. I knew the definition - Json Web Token, but honestly that’s all I knew for a veryyyy longgg time. Recently, I came across a problem where I have been told that JWT might be a good solution for it. That piqued my interest.

From the wiki,

In computing, data storage, and data transmission, character encoding is used to represent a repertoire of characters by some kind of encoding system that assigns a number to each character for digital representation.

Okay, they mean converting character to some number for storing and transmitting data.

There are two popular character sets

1. ASCII
   1. American Standard Code for Information Interchange
   2. Pretty much all the characters and symbols in modern keyboard comes under ASCII
   3. Total of 128 characters
   4. Decimal value between 65 and 90 represents captial alphabets and 97 to 122 represents small alphabets
2. UTF-8
   1. ASCII doesn’t cover characters from other languages.
   2. UTF-8 is capable of encoding all 1,112,064 valid character code points in Unicode using one to four one-byte (8-bit) code units.
   3. It has backward compatible with ASCII

I run into character set in postgres. There is client_encoding and server_encoding setting.

There are two types of variable declaration.

1. recursively expanded variables - foo is assigned with bar which inturns assigned to ugh whose value is Huh?

foo = $(bar)
bar = $(ugh)
ugh = Huh?

all:;echo $(foo)

$ make
$ Huh?

2. simply expanded variables - foo is assigned with bar however bar is not defined yet. so it outputs to nothing.

foo := $(bar)
bar := $(ugh)
ugh := Huh?

all:;echo $(foo)

$ make
$

NOTE: To echo something it needs to be in a target. Here the target name is all.

This is a crosspost from my TIL

What the heck is .PHONY in makefile? I have seen this for so long but I always shrugged off. Whatever that is. Well, finally I decided to learn about it.

A bit of introduction to Makefile syntax.

target: prerequisites
<TAB> recipe

The first target in a Makefile will be executed by default when we call make.

blah: blah.o
    cc blah.o -o blah # Runs third

blah.o: blah.c
    cc -c blah.c -o blah.o # Runs second

blah.c:
    echo "int main() { return 0; }" > blah.c # Runs first

In the above example, blah, blah.o and blah.c are targets and they are all files.

Seven attempts, seven different wrong answers — lessons from setting up a local LLM.

It’s tax season, which means I’ve been staring at a notes file full of stock sales — 23 transactions across the year that I needed to total up. The kind of data I’d rather not paste into a chat window I don’t control.

I’d been meaning to set up a local LLM anyway, and this seemed like the perfect low-stakes test. M3 Max, 64GB of unified memory, plenty of headroom for a real model. I installed Ollama, pulled Qwen 2.5 Coder, pasted the list, asked the question.

Seven attempts, seven different wrong answers — lessons from setting up a local LLM.

It’s tax season, which means I’ve been staring at a notes file full of stock sales — 23 transactions across the year that I needed to total up. The kind of data I’d rather not paste into a chat window I don’t control.

I’d been meaning to set up a local LLM anyway, and this seemed like the perfect low-stakes test. M3 Max, 64GB of unified memory, plenty of headroom for a real model. I installed Ollama, pulled Qwen 2.5 Coder, pasted the list, asked the question.

There are two types of variable declaration.

1. recursively expanded variables - foo is assigned with bar which inturns assigned to ugh whose value is Huh?

foo = $(bar)
bar = $(ugh)
ugh = Huh?

all:;echo $(foo)

$ make
$ Huh?

2. simply expanded variables - foo is assigned with bar however bar is not defined yet. so it outputs to nothing.

foo := $(bar)
bar := $(ugh)
ugh := Huh?

all:;echo $(foo)

$ make
$

NOTE: To echo something it needs to be in a target. Here the target name is all.

This is a crosspost from my TIL

What the heck is .PHONY in makefile? I have seen this for so long but I always shrugged off. Whatever that is. Well, finally I decided to learn about it.

A bit of introduction to Makefile syntax.

target: prerequisites
<TAB> recipe

The first target in a Makefile will be executed by default when we call make.

blah: blah.o
    cc blah.o -o blah # Runs third

blah.o: blah.c
    cc -c blah.c -o blah.o # Runs second

blah.c:
    echo "int main() { return 0; }" > blah.c # Runs first

In the above example, blah, blah.o and blah.c are targets and they are all files.

TL;DR:

Connect with people whom you admire. Things can take longer so stay in touch. At the right moment, don’t hesitate to ask.

I recently became an Engineering Manager after being an Individual Contributor for almost 8 years. If you are interested to know more about the reasoning behind the change. As like many other jobs in Tech, there is no formal training for an Engineering Manager. Reminds me of my dad’s saying “How do you teach one to swim? Just throw on the water !”. I felt it would be great to have a mentor with whom I can chat/discuss regularly around management, leadership, politics, team building etc etc…

Last week, I got promoted to Engineering Manager (EM) of the Database team at Cloudflare. I joined the team almost 3 years ago. A senior team member and myself were hired to form a then dedicated database team. I have been enjoying it and the team is amazing to work with. Great people. Opportunities came to either advance on the Individual Contributor (IC) or the leadership side. I picked the latter. My thoughts on why did I transition to leadership.

TL;DR:

Connect with people whom you admire. Things can take longer so stay in touch. At the right moment, don’t hesitate to ask.

I recently became an Engineering Manager after being an Individual Contributor for almost 8 years. If you are interested to know more about the reasoning behind the change. As like many other jobs in Tech, there is no formal training for an Engineering Manager. Reminds me of my dad’s saying “How do you teach one to swim? Just throw on the water !”. I felt it would be great to have a mentor with whom I can chat/discuss regularly around management, leadership, politics, team building etc etc…

Capturing thoughts from https://twitter.com/viggy28/status/1530800893842444289

1. When you are doing major DML changes, other than locks one more thing to keep in mind is replication lag. Especially if you use your replicas in hot standby mode.

2. When you need to delete most of the records in a massive table, its better to create a new table and just copy the records that you need to preserve. When you need to delete all the records in a massive table, just truncate it instead of deleting them.

This aricle is inspired by [Give and Take] (https://www.adamgrant.net/give-and-take) by [Adam Grant] (https://www.adamgrant.net/). I was wondering how can I help my friends and network. Searching a job is something stressful and we all have been through. Once my wife and I searched for a year and finally we moved to San Francisco.

So far its been a challenge for my network to know whether I can refer to a particular company. To solve this, I am publishing a list of companies where I can refer. If you don’t know, now you know (Yeah, we have been watching a lot of Trevor Noah shows)

In v6, /48 is the minimum prefix that can be advertised by the routers in the Internet.

I believe in v4, its /24.

Until yesterday I wasn’t aware of that. I might have thought you could even advertise one IP (ie. /128 or /32) but that’s not possible.

Ref: https://blog.apnic.net/2020/06/01/why-is-a-48-the-recommended-minimum-prefix-size-for-routing

There are tools like https://tehnoblog.org/ip-tools/ip-address-in-cidr-range/ which does the job. It doesn’t support IPv6. One of my colleagues today taught me how to do it by hand quickly.

Say we have an IP 2b06:4600:1101:0:abcd:efa:dbd:ea60:f5a6 is in the CIDR 2b06:4600:1101::/64

First step is to check what’s the mask bits. Here it is 64. So you take the address, take the first 64 bits and see if they are the same as the CIDR.

This article is part of my Today I Learnt. I have been trying to share whatever I learnt newly on the day. Some of them are basic, but still I want to write it down for two reasons.

1. To help me understand the concepts better
2. Potential help other people who running in to same issue.

Alright, It’s all started when I bought a new domain (synergy.net) for my Dad. Who doesn’t like yet another domain for their future project? He wanted a blog, so I thought it would be nice to host a ghost [1] blog on my Raspberry Pi which has been sitting idle on my desk. As like every good netizen, I did a Google search and found this medium article [2] which exactly solves what I want. Software installation went fine and I reached the step of forwarding traffic to my Raspberry Pi.

Seven attempts, seven different wrong answers — lessons from setting up a local LLM.

It’s tax season, which means I’ve been staring at a notes file full of stock sales — 23 transactions across the year that I needed to total up. The kind of data I’d rather not paste into a chat window I don’t control.

I’d been meaning to set up a local LLM anyway, and this seemed like the perfect low-stakes test. M3 Max, 64GB of unified memory, plenty of headroom for a real model. I installed Ollama, pulled Qwen 2.5 Coder, pasted the list, asked the question.

Last week, I got promoted to Engineering Manager (EM) of the Database team at Cloudflare. I joined the team almost 3 years ago. A senior team member and myself were hired to form a then dedicated database team. I have been enjoying it and the team is amazing to work with. Great people. Opportunities came to either advance on the Individual Contributor (IC) or the leadership side. I picked the latter. My thoughts on why did I transition to leadership.

This is something new for this site. In fact, First of its kind. Mostly, I write about Tech, Finance, books etc. Recently, I thought about writing/journaling more of my personal life and experiences. Also, interactions with the professional people whom I met.

If you have known Sri or myself personally, we moved to San Francisco from Boston (Sri) and Los Angeles (myself) in 2019. We don’t have that many friends or family here in the Bay area. Slowly but steadily we are building our circle though :) First place lived is in Burlingame. During 2020, we thought of buying home. Yes, during the pandemic year. Also, we decided to move to East Bay/Tri-Valley. That’s all we knew at the mid 2020. The first challenge is finding a good realtor. Started with a Redfin agent but didn’t find the chemistry. Then, we tapped our network and got some recommendation. Worked with them but it became very obvious even at the first meet that the dynamic wasn’t there. Kinda, we hit a roadblock. That’s when we got reference of Nivi from the most (un)usual place - Blind app.

1. double pointers

A pointer (whose value is another variable’s address) which points to another pointer

2. copying a pointer to another pointer copies the address

3. copying a de-referenced pointer to another de-referenced pointer updates the value of the one to another

https://gist.github.com/viggy28/604cda186ff1b872d562b31ca3976732

package main

import "fmt"

type DB struct {
	Name string
}

// double pointer
// a pointer which points to another pointer
func wrapperFunc(config **DB) func() error {
	return func() error {
		cfg := &DB{}
		(*cfg).Name = "hello"
		*config = cfg
		return nil
	}
}

// config = cfg assigns the address which cfg has to config
// hence dd1 in main() doesn't get impacted
func wrapperFunc1(config *DB) func() error {
	return func() error {
		cfg := &DB{}
		fmt.Println("address of config: ", &config)
		fmt.Printf("  wrapperFunc1 A: config is %p, cfg is %p\n", config, cfg)
		cfg.Name = "hello"
		config = cfg
		fmt.Printf("  wrapperFunc1 B: config is %p, cfg is %p\n", config, cfg)
		return nil
	}
}

// *config = *cfg value at address cfg has will be copied to value at the address that config has
// hence dd2 in main() does gets updated
func wrapperFunc2(config *DB) func() error {
	return func() error {
		cfg := &DB{}
		fmt.Printf("  wrapperFunc2 A: config is %p, cfg is %p\n", config, cfg)
		cfg.Name = "hello"
		fmt.Println("value at ", *cfg)
		*config = *cfg
		fmt.Printf("  wrapperFunc2 B: config is %p, cfg is %p\n", config, cfg)
		return nil
	}
}

func wrapperFunc3(config *DB) func() error {
	return func() error {
		config.Name = "hello"
		return nil
	}
}

func wrapperFunc4(config DB) func() error {
	return func() error {
		config.Name = "hello"
		return nil
	}
}

func main() {
	{
		var d *DB
		fn := wrapperFunc(&d)
		fn()
		fmt.Printf("d.Name: %q\n", d.Name)
	}

	{
		dd1 := &DB{}
		fmt.Println("address of dd1: ", &dd1)
		fmt.Printf("main A: dd1 is %p\n", dd1)
		fn1 := wrapperFunc1(dd1)
		fn1()
		fmt.Printf("main B: dd1 is %p\n", dd1)
		fmt.Printf("dd1.Name: %q\n", dd1.Name)
	}

	{
		dd2 := &DB{}
		fmt.Printf("main C: dd2 is %p\n", dd2)
		fn2 := wrapperFunc2(dd2)
		fn2()
		fmt.Printf("main D: dd2 is %p\n", dd2)
		fmt.Printf("dd2.Name: %q\n", dd2.Name)
	}

	{
		dd3 := &DB{}
		fn3 := wrapperFunc3(dd3)
		fn3()
		fmt.Printf("dd3.Name: %q\n", dd3.Name)
	}

	{
		dd4 := DB{}
		fn4 := wrapperFunc4(dd4)
		fn4()
		fmt.Printf("dd4.Name: %q\n", dd4.Name)
	}
}

The last line of defense for those OOPS moments — so even DBAs can sleep peacefully.

Have you ever accidentally run a DELETE without a WHERE clause? Or typed DROP TABLE thinking you were in staging, only to realize a half-second too late that you were in prod?

If yes — keep reading, this is for you. If no — keep reading anyway. Nobody who uses Postgres long enough stays immune.

The analytics databases optimized for queries. The real leverage was always where data gets written.

Your application writes to Postgres. So does almost everyone else’s. Postgres is the most widely used database in the world, topping Stack Overflow’s developer survey two years running. It didn’t win as a data warehouse. It won as the place applications write.

For years, the data infrastructure companies competed on what happens after — faster queries, better compression, smarter optimizers. The traditional model relied on storage gravity: move your data into our walls, pay us to store and query it. Once data lived inside a warehouse, the switching cost kept it there.

I was writing an article for Infoq on the topic less spoken costs of managed databases and one question that the editor asked is how much margin RDS makes compared to running a Postgres instance on EC2? That intrigued me and honestly, I never did the math so far. I have been using AWS-managed databases since 2016 and I thought what’s a better time to do an analysis of Cloud cost than today ?!

I was trying to play with Phil’s pgtam and the first step is that to install Postgres version 16. Sounds fairly innocous. When I ran configure, I was getting the below error:

checking for icu-uc icu-i18n... no
configure: error: ICU library not found
If you have ICU already installed, see config.log for details on the
failure.  It is possible the compiler isn't looking in the proper directory.
Use --without-icu to disable ICU support.

Obviously, one solution is to disable ICU support as recommended by the error message. However, that’s an anti-feature as per the Postgres documentation.

Abstract

On a high level, the talk addressed following questions:

• What is Edge and why are the benefits of edge?
• What are the challenges of implementing Postgres at the edge?
• How we implemented it at Cloudflare?
• Where Postgres at the edge is heading?

Slides from the presentation

Link from Developerweek Cloudx

Link from Developerweek Cloudx virtual

Abstract

We talked about the importance of having a good toolset, of practicing incidents and of internally advocating database best practices to a large engineering organization.

Recordings of the presentation

Abstract

With the advent of cloud/managed offerings, running Postgres on their own hardware is becoming rare. It’s great that a lot of the Ops work is taken care of by the provider, however, understanding a layer or two beneath these abstractions will be useful for anyone in strengthening their knowledge.

For eg. When you can’t connect to an instance how do you find where the problem is?

a. Are you able to reach the server i.e. Is ICMP even working?

Abstract

This talk explores how Cloudflare uses HAProxy for health checks, load balancing and reading traffic among nodes set up with Postgres streaming replication in hot standby mode. Cloudflare operates multiple Postgres Clusters across four data centers, and all of these clusters are made up of six nodes.

During a primary failure, Cloudflare’s high availability system promotes a replica to become a primary, and HAProxy makes sure there is no write traffic between two primaries to avoid a split-brain scenario.

I kept hearing about the term wire protocol especially Postgres wire protocol in the recent days (Looking at you cockroachdb, yugabytedb - in a good way) but never really quite understood it. Decided to implement something simple in Go to understand it better. As always, if you find anything wrong or I misunderstood please correct me.

In simple terms,

“wire” - something over network generally (but PG also supports over domain sockets)

I use pg_rewind for rewind a Postgres cluster which is ahead of the primary. Yesterday, after the rewind Postgres didn’t start. It failed with the below error. This is using 9.6 version pg_rewind client.

Sep  2 20:06:14.371503 productiondb[1]: [1-1] time=2022-09-02 20:06:14.338 GMT,pid=78197,user=admin,db=postgres,client=[local],appname=[unknown],vid=,xid=0 FATAL:  the database system is starting up
Sep  2 20:06:14.371611 productiondb[1]: [1-1] time=2022-09-02 20:06:14.370 GMT,pid=78184,user=,db=,client=,appname=,vid=,xid=0 LOG:  entering standby mode
Sep  2 20:06:14.371991 productiondb[1]: [1-1] time=2022-09-02 20:06:14.371 GMT,pid=78184,user=,db=,client=,appname=,vid=,xid=0 PANIC:  could not open file "pg_replslot/ae0750b1/state": No such file or directory
Sep  2 20:06:14.371991 productiondb[1]: [1-1] time=2022-09-02 20:06:14.371 GMT,pid=78182,user=,db=,client=,appname=,vid=,xid=0 LOG:  startup process (PID 78184) was terminated by signal 6: Aborted

Of course, the /state file didn’t exist inside the replication slot sub-directory ae0750b1.

Citus: Distributed PostgreSQL for Data-Intensive Applications paper can be downloaded here.

Recently, our team got a request to provide a solution to shard Postgres. One of the solutions that we discussed was Citus. I have heard about the product and seen their blogs related to Postgres in the past but never used it. I thought it would be fun to read about its internal workings.

If you find something wrong on the notes, please send a pull request. Before digging their white paper, let’s take a step back and ask what is sharding and why do we need sharding?

Capturing thoughts from https://twitter.com/viggy28/status/1530800893842444289

1. When you are doing major DML changes, other than locks one more thing to keep in mind is replication lag. Especially if you use your replicas in hot standby mode.

2. When you need to delete most of the records in a massive table, its better to create a new table and just copy the records that you need to preserve. When you need to delete all the records in a massive table, just truncate it instead of deleting them.

Before I forget let me write this down here. $$ log_stament - none - all

log_min_duration_statment - millisecond value

When you set log_statement=none and log_min_duration_statement=1 then any statement which takes longer than 1 millisecond will be logged.

When you set log_statement=all and log_min_duration_statement=1 then all statements are logged; however it only shows duration on statements longer than 1 millisecond.

From the wiki,

In computing, data storage, and data transmission, character encoding is used to represent a repertoire of characters by some kind of encoding system that assigns a number to each character for digital representation.

Okay, they mean converting character to some number for storing and transmitting data.

There are two popular character sets

1. ASCII
   1. American Standard Code for Information Interchange
   2. Pretty much all the characters and symbols in modern keyboard comes under ASCII
   3. Total of 128 characters
   4. Decimal value between 65 and 90 represents captial alphabets and 97 to 122 represents small alphabets
2. UTF-8
   1. ASCII doesn’t cover characters from other languages.
   2. UTF-8 is capable of encoding all 1,112,064 valid character code points in Unicode using one to four one-byte (8-bit) code units.
   3. It has backward compatible with ASCII

I run into character set in postgres. There is client_encoding and server_encoding setting.

I have know about the postgres wire protocol, but first I ran into streaming replication protocol.

I was looking at this code in stolon

	replConnParams["replication"] = "1"
	db, err := sql.Open("postgres", replConnParams.ConnString())
	if err != nil {
		return nil, err
	}
	defer db.Close()

	rows, err := query(ctx, db, "IDENTIFY_SYSTEM")
	if err != nil {
		return nil, err
	}

I was wondering what is this IDENTIFY_SYSTEM. A G-search pointed me to this

visi@viggy28-MacBook-Pro ~ % psql "dbname=postgres user=postgres replication=database password=replaceme" -c "IDENTIFY_SYSTEM;"
      systemid       | timeline |  xlogpos  |  dbname
---------------------+----------+-----------+----------
 6852440194718025990 |        1 | 0/1542DB8 | postgres

visi@visis-MacBook-Pro ~ % psql "dbname=postgres user=postgres replication=database password=replaceme" -c "TIMELINE_HISTORY 1;"
ERROR:  could not open file "pg_xlog/00000001.history": No such file or directory
(1 row)

I learnt you can directly postgres using other than SQL by sending messages to the backend (server) through the wire protocol !!

Libpq-Postgres-Go

After I started working on Postgres, I have heard this term libpq enough times, but never had a good grasp of it. After digging around this topic for a couple of days, here is my understanding.

From the Postgres doc https://www.postgresql.org/docs/9.5/libpq.html,

libpq is the C application programmer’s interface to PostgreSQL. libpq is a set of library functions that allow client programs to pass queries to the PostgreSQL backend server and to receive the results of these queries

Understanding context in Golang through Postgres

I was trying to learn Go package context especially with respect to Postgres. On a very high level context provides context to the operation. Yeah, I agree, the previous statement doesn’t really add much value, but hold on I don’t really know how to explain it, rather let’s go over some code. Sometimes its easier to understand by seeing it in action :)

The complete program is here

Step 1: Setting up HAProxy

I hope you gone through [Part1] (https://viggy28.dev/article/postgres-active-active-replication/) of this series. Perhaps, one thing you might have noticed is that I’ve to connect to the specific master database. In our case, since both the databases are running on docker, only the localhost port is different. (In a production environment, most likely you going to run the databases on a different host). The main reason for active-active replication is high availability. If one of the nodes goes down, you still want your application to behave normally. You don’t want to hard-code your DSN on the application or keep checking the health of the database every time before you make a connection. Fair enough. [HAProxy] (<www.haproxy.org>), an open source project solves this particular change.

Postgres doesn’t support active-active replication natively. As of this writing, we have to rely on 3rd party tools. I decided to go with [BDR] (https://www.2ndquadrant.com/en/resources/postgres-bdr-2ndquadrant/).

I didn’t want to spin up multiple VMs. So, the obvious choice is docker. Make sure you have docker on mac & docker compose.

Step1: Running 2 Postgres instances using docker container

Thanks to [jgiannuzzi] (https://github.com/jgiannuzzi), who created a docker [image] (https://hub.docker.com/r/jgiannuzzi/postgres-bdr) with Postgres and BDR.

In this article we will see how to convert a simple python application to a containerized (Docker) one

Go to the profile of vignesh ravichandran vignesh ravichandran Sep 23, 2017 TL;DR -> Containerize a python app. Push the image and execute from Google Cloud.

On a leisurely Saturday afternoon, I thought its nice to play with containers. I was fascinated with the idea of containerizing an app and running it across different environments. Its easier (for me) to understand something when I do, instead of just reading.

This is part of my Today I Learnt series where I share whatever I am learning something new. After having issues with port forwarding in Xfinity I decided to look for alternative solutions. I have used Cloudflare tunnel (used to be called Argo Tunnel) in the past to expose websites running on my laptop to the Internet. So, I decided to try it out for Ghost blogging site which I am setting up for my dad.

Mark my word. We had so many competitions in the past. Eg. vim vs emacs, tabs vs spaces etc. The new comparison which is added in this decade is functions vs containers

Both the functions and containers can be serverless. So, please avoid the notion of serverless means functions. In fact, predominantly all the functions (like lambda, cloud functions) are running behind the screen as containers. (there are few products like Cloudflare workers are running without containers)

namespaces is a Linux Kernel feature. There are different types of namespaces

• Partitions Kernel resources such that one set of Processes see a different resources while others different
• Apartment complex analogy - 7 different namespaces
• PID namespace - Watching TV show
• Net namespace - each namespace gets an unique IP address s list of port. - apartment address analogy
• Uts namespace - host names for Ip address - Telling taxi driver apartment name instead of address
• User namespace - Files are associated with VID Euser Identification
• Mailbox associated to apartment unit # not with the name of the person - Mit namespace o to isolate mount points such that processes in different namespaces cannot view each others files.
• Similar to chroot

Commands:

I have seen this term here and there. I remember fiddling with this in pain of setting up port forwarding in xfinity/.

DHCP - Dynamic Host Configuration Protocol (DHCP) is a network management protocol used on Internet Protocol (IP) networks for automatically assigning IP addresses and other communication parameters to devices connected to the network using a client–server architecture (from wiki )

client-server architecture? In a home network, router is the server and client is my PCs, laptops, Raspberry Pis etc.

I have used many signals with kill like SIGHUP, SIGKILL etc. Today I came across USR2.

Did a Google search and found the manual page from GNU. According to it,

These signals are used for various other purposes. In general, they will not affect your program unless it explicitly uses them for something.

Another one from the BSD mailing list

USR2 is a "user defined signal" (from "man signal")

It doesn't "mean" anything by definition.  Each application is free to define its meaning as it sees fit.  It's there specifically so that applications can use signals for special purposes without reusing the defined signals.