#620·pq

Context cancel hangs if there is no network connectivity

Author: banksCreated May 31, 2017Updated Aug 3, 2026
Labelsbug

Hi,

Not sure if this is a bug or if it is effectively the same as #584. But it appears to be a significant Gotcha if I'm understanding correctly.

Setup

go
package main

import (
	"context"
	"database/sql"
	"fmt"
	"time"

	_ "github.com/lib/pq"
)

func main() {
	db, err := sql.Open("postgres", "postgresql://127.0.0.1:5432/postgres?sslmode=disable&connect_timeout=3")
	if err != nil {
		panic(err)
	}

	for {
		ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
		rows, err := db.QueryContext(ctx, "SELECT 1")
		if err != nil {
			fmt.Println("ERROR:", err)
		}
		rows.Next()
		var i int
		err = rows.Scan(&i)
		rows.Close()
		cancel()
		if err != nil {
			panic(err)
		}
		fmt.Println("Got", i)
		time.Sleep(1 * time.Second)
	}
}

Behaviour

  • Code above works fine with a local postgres instance up
  • If you stop the PG instance the connect will be reset and it will fail loudly as expected
  • Do any of the following to simulate failure:
  • simulate process busy/hung by finding the PID of the connected postgres process and then kill -stop <pid>
  • simulate packet loss by adding echo "block drop quick on lo0 proto tcp from any to any port = 5432" | sudo pfctl -ef - on OS X or equivalent iptables rule to drop packets to the local database
  • Expected behavior: that the client's timeout after 5 seconds with an error and print to screen before retrying
  • ** Actual Behaviour**: the client remains hung indefinitiely in either case until the "failure" is resolved

Cause

  • It seems to be due to the fact that the code handling Context cancelation (https://github.com/lib/pq/blob/master/conn_go18.go#L67) attempts to send a cancel op to the database and waits forever for a response
  • The cancel method even calls dial internally but this is not even respecting the connect_timeout option, possibly because it's returning a pooled connection? Even my trivial test example doesn't timeout though and there should only be a single connection made.

Ramifications

In my case the motivation for wanting to use Contexts in the first place is that I need tight bounds on execution time and can't afford to have clients hang indefinitely on DB whatever the error condition is in the real-world (both packet loss and hung server processes happen).

I could implement this externally presumably with my own watcher on the context in another Goroutine that tears down the whole connection, but it seems like this mechanism should work for my case without re-inventing it outside.

Is there an alternative option?