gRPC Load Balancing

Giriş

Açıklaması şöyle.

There are two types of load balancing options available in gRPC — proxy and client-side.

1. Proxy load balancing

2. Client-side load balancing

Proxy load balancing

Bir gRPC servisi istekleri karşılar, diğer gRPC servislerine dağıtır. Açıklaması şöyle.

In Proxy load balancing, the client issues RPCs to a Load Balancer (LB) proxy. The LB distributes the RPC call to one of the available backend servers that implement the actual logic for serving the call. The LB keeps track of load on each backend and implements algorithms for distributing load fairly. The clients themselves do not know about the backend servers. Clients can be untrusted. This architecture is typically used for user-facing services where clients from open internet can connect to the servers

Client-side load balancing

gRPC istemcisi gRPC servislerini bilir ve isteği kendi uygun gördüğü birine gönderir. Açıklaması şöyle.

In Client-side load balancing, the client is aware of many backend servers and chooses one to use for each RPC. If the client wishes it can implement the load balancing algorithms based on load report from the server. For simple deployment, clients can round-robin requests between available servers.

Client-side load balancing ve Kubernetes

Açıklaması şöyle. Yani gRPC  bir Connection Pool kullansaydı ve bağlantıları arada bir kapatıp tekrar açsaydı sorun olmayacaktı. Ancak gRPC bağlantıyı hep açık tutmak üzere tasarlanmış. Çözüm Serve-side Connection Management kullanmak

gRPC is built on HTTP/2, and HTTP/2 is designed to maintain a long-living TCP connection where all requests can be active on the same connection at any point in time. It reduces the overhead of connection management. ... once the connection is established, there’s no more balancing to be done. All requests get pinned to original destinations pods, as shown below until a new DNS discovery happens (with headless service). And it won’t happen until at least one of the existing connections breaks.

Serve-side Connection Management

Örnek

Şöyle yaparız

grpc.KeepaliveParams(keepalive.ServerParameters{
  MaxConnectionAge:      time.Second * 30,  // THIS one does the trick
  MaxConnectionAgeGrace: time.Second * 10,
})

Açıklaması şöyle

MAX_CONNECTION_AGE to 30 seconds. This time period is long enough to have low-latency communication without an expensive and frequent connection establishment process. Also, it allows services to react relatively quickly to the existence of new pods so the traffic distribution will be balanced.

MAX_CONNECTION_AGE_GRACE to 10 seconds. Defines the maximum time for the connection to be kept alive for outstanding RPCs to complete.