Graceful Retries In Go

Failures are Inevitable

Network calls might time out, APIs might be temporarily unavailable, or databases might become unreachable. To handle such transient errors gracefully, implementing a retry mechanism is crucial. This article will guide you through the basics of why you need a retry mechanism, starting with simple retries and gradually introducing backoff strategies using a Go example.

Simple Retries

A basic retry mechanism involves reattempting an operation a fixed number of times immediately after a failure. Here's how you can implement this in Go:

func immediatelyRetry(f func() error, retriesLeft int) error {
    err := f()
    if err == nil {
        return nil
    }

    if retriesLeft == 0 {
        return err
    }

    return immediatelyRetry(f, retriesLeft-1)
}

In this function, f is the operation that might fail. It is retried immediately until it either succeeds or the retry limit is reached.

Adding Backoff Strategies

While immediate retries can be effective, they might overwhelm the system or the external service you're interacting with. To address this, backoff strategies introduce delays between retries, with the delay increasing over time.

Exponential Backoff

Exponential backoff is a popular strategy where the delay between retries grows exponentially. This reduces the load on the system and gives it more time to recover. Here's an example implementation:

func retryWithBackoff(f func() error, retriesLeft int, delay time.Duration, backoff float64) error {
    err := f()
    if err == nil {
        return nil
    }

    if retriesLeft == 0 {
        return err
    }

    time.Sleep(delay)
    return retryWithBackoff(f, retriesLeft-1, time.Duration(float64(delay)*backoff), backoff)
}

In this function, the delay is multiplied by the backoff factor after each retry, resulting in exponentially increasing delays.

Defining a Custom Retry Policy

To provide flexibility, you can define a custom retry policy that combines immediate retries and retries with backoff. Here’s how you can achieve this in Go:

package xretry

import (
    "time"
)

// RetryPolicy is the retry policy
type RetryPolicy struct {
    immediateRetries   int
    retriesWithBackoff int
    delay              time.Duration
    backoffFactor      float64
}

// RetryPolicyOption is the option for the retry policy
type RetryPolicyOption func(*RetryPolicy)

// WithImmediateRetries sets the immediate retries
func WithImmediateRetries(retries int) RetryPolicyOption {
    return func(p *RetryPolicy) {
        p.immediateRetries = retries
    }
}

// WithRetriesWithBackoff sets the retries with backoff
func WithRetriesWithBackoff(retries int, delay time.Duration, backoffFactor float64) RetryPolicyOption {
    return func(p *RetryPolicy) {
        p.retriesWithBackoff = retries
        p.delay = delay
        p.backoffFactor = backoffFactor
    }
}

// NewRetryPolicy creates a new RetryPolicy
func NewRetryPolicy(opts ...RetryPolicyOption) RetryPolicy {
    p := RetryPolicy{
        immediateRetries:   0,
        retriesWithBackoff: 0,
        delay:              0,
        backoffFactor:      0,
    }

    for _, opt := range opts {
        opt(&p)
    }

    return p
}

// Retrier is the interface that wraps the Retry method
type Retrier struct {
    p RetryPolicy
}

// NewRetrier creates a new Retrier
func NewRetrier(p RetryPolicy) *Retrier {
    return &Retrier{
        p: p,
    }
}

// Retry will retry the given function
func (r *Retrier) Retry(f func() error) error {
    err := immediatelyRetry(f, r.p.immediateRetries)
    if err != nil {
        err = retryWithBackoff(f, r.p.retriesWithBackoff, r.p.delay, r.p.backoffFactor)
        if err != nil {
            return err
        }
    }

    return nil
}

Usage Example

Here's an example of how to use the custom retry policy and retrier in your application:

func main() {
    // Define a retry policy
    policy := xretry.NewRetryPolicy(
        xretry.WithImmediateRetries(3),
        xretry.WithRetriesWithBackoff(3, 1*time.Second, 2.0),
    )

    // Create a new retrier with the policy
    retrier := xretry.NewRetrier(policy)

    // Define a function that may fail
    f := func() error {
        // Your code here
    }

    // Use the retrier to retry the function if it fails
    err := retrier.Retry(f)
    if err != nil {
        fmt.Println("Operation failed after retries:", err)
    }
}

In this example, the function f will be retried immediately 3 times if it fails. If it still fails after these retries, it will be retried 3 more times with a delay that doubles after each retry, starting from 1 second.

Conclusion

Implementing a retry mechanism with customizable policies and backoff strategies can significantly enhance the resilience of your Go applications. By defining flexible retry policies, you can handle transient errors more gracefully, providing a better experience for your users and reducing the need for manual intervention.

Hardik Sachan