29. Concurrency patterns: fan-in/fan-out, pipelines, rate limiting, errgroup

29.1 Fan-out / fan-in

Distribute work across multiple goroutines (fan-out), then merge their results back into one stream (fan-in):

func fanIn(channels ...<-chan int) <-chan int {
	out := make(chan int)
	var wg sync.WaitGroup
	wg.Add(len(channels))
	for _, c := range channels {
		go func(c <-chan int) {
			defer wg.Done()
			for v := range c {
				out <- v
			}
		}(c)
	}
	go func() {
		wg.Wait()
		close(out)
	}()
	return out
}

29.2 Rate limiting with time.Ticker or golang.org/x/time/rate

limiter := rate.NewLimiter(rate.Limit(10), 1) // 10 events/sec, burst 1
for _, req := range requests {
	if err := limiter.Wait(ctx); err != nil {
		break // ctx cancelled
	}
	send(req)
}

29.3 errgroup: WaitGroup that also propagates the first error

import "golang.org/x/sync/errgroup"

g, ctx := errgroup.WithContext(ctx)
for _, url := range urls {
	url := url
	g.Go(func() error {
		return fetch(ctx, url)
	})
}
if err := g.Wait(); err != nil {
	return err // the first error any goroutine returned
}

errgroup.WithContext also cancels the shared ctx the moment any goroutine returns a non-nil error — so a failing fetch stops the others from doing pointless work, without you wiring that cancellation by hand.

29.4 Real-world example

A service that fans out to five downstream APIs and needs "all succeed or report which failed, and don't wait for slow ones once one has failed" is errgroup in five lines — this exact pattern replaces pages of manual WaitGroup + mutex + error-slice bookkeeping.

Applied in the series: the HTTPS guide's resilience lab implements rate limiting, retries with jitter and a budget, and a circuit breaker.