Back Pressure (BackPressure) Nedir

Giriş

Back Pressure karışık bir konu ama öğrenilmesi gerekiyor. Açıklaması şöyle

A 2024 study of around 300 backend incidents found that roughly 35 percent traced back to missing or misconfigured backpressure. Not missing databases or network failures but the absence of a mechanism to say no when the system is overwhelmed. Systems design interviews ask you to sketch load balancers and caches but rarely test whether you know how to stop accepting work when you can't keep up.

Tanımı ise şöyle

Backpressure is what happens when a slow consumer tells a fast producer to slow down. Without it you get unbounded queues that eat memory until the process dies.

Örnekler

Açıklaması şöyle

Backpressure goes further. Instead of optimistically accepting work and hoping to catch up, you block at the boundary. TCP does this with sliding windows. Application-level backpressure means refusing to pull messages when your workers are saturated, returning 503s when your connection pool is full. You're deliberately creating a buffer where requests wait instead of executing and failing.

1. Aynı Bellekte Çalışıyorsak

Aynı bellekte çalışıyorsak kuyruğu daha büyük yapmak çözüm değil. Açıklaması şöyle

A bigger queue doesn't fix the problem. It just delays the failure. Your workers are still slow. The queue still grows. Now you're OOMing with a million items queued instead of a hundred thousand. And when you restart, all that queued work is lost anyway so you didn't even help the users you were trying to protect.

Back Pressure için kullanılan çözümlerle ilgili notlarım şöyle. 

1.1 Blocking Back Pressure Nedir - Yazan Tarafın Kuyrukta Bloke Olması

Bu yöntemde Producer bekler. Açıklaması şöyle

A bounded queue has a maximum size. When it's full, new items either block the producer or get rejected immediately. This is backpressure in its most basic form.

Kuyruk doluysa hata kodu dönmek lazım. Açıklaması şöyle

... when the queue fills up your API returns 503 Service Unavailable instead of accepting the request.

Dolayısıyla kuyruk büüyklüğünü iyi belirlemek lazım

Ancak asenkron çalışmada Producer'ı bekletme şansımız yok. Dolayısıyla "Non-blocking back-pressure" yöntemi kullanılıyor.

1.2 Load Shedding and Adaptive Concurrency

Load shedding düşük önceliğe sahip işlerin kuyruktan çıkarılması demek. Açıklaması şöyle

Load shedding is smarter. It drops low priority work first.

Örnek

Açıklaması şöyle

Imagine you have two types of requests. Premium users and free users. Under normal load you process both. Under heavy load you start dropping free user requests while still serving premium users. This requires priority queues or separate queues per tier. When the system is overloaded you shed the low priority queue entirely.

Adaptive Concurrency

Açıklaması şöyle

Another pattern is adaptive concurrency. Instead of a fixed number of worker threads, you scale the worker pool based on current latency. If requests start taking longer than normal, that's a sign the system is struggling. You can reduce the concurrency limit to prevent further slowdown. Libraries like Tokio in Rust and Hystrix in Java implement this. The idea is measure your p95 latency, and if it crosses a threshold, stop accepting new work until latency drops.

Retry

Açıklaması şöyle

Some systems combine this with retries. If you reject a request with a 503, the client can retry with exponential backoff. The first retry might succeed if the spike was brief. This turns a temporary overload into a slight delay rather than total failure. But you need idempotency for this to work safely. If retrying a request causes duplicate charges or double writes, load shedding makes things worse not better.

1.3. Non-Blocking Back Pressure Nedir - Okuyan Taraf Veriyi Çeker
ReactiveX ise Reactive Manifesto ile ilgili. Reactive Manifesto'nun bir amacı şunu sunmak.

"a standard for asynchronous stream processing with non-blocking back pressure"

Açıklaması şöyle.

If we have a producer which is emitting events to a consumer faster than it can process them, then eventually the consumer will be overwhelmed with events, running out of system resources. Backpressure means that our consumer should be able to tell the producer how much data to send in order to prevent this, and this is what is laid out in the specification.

Producer'ın hızına yetişemeyen Consumer, bir şekilde Producer'a yavaşlaması gerektiğini bildirir. Yani Consumer hazır olduğunda Producer'a kaç tane nesne istediğini söyler. Yani push yöntemi yerine pull yöntemi kullanılır. Producer da elindekileri gönderir.

1.3.1 Back Pressure vs Flow Control
Okuyan tarafın veriyi çekmesi yönteminde haberleşme protokollerindeki "flow control" kavramı ile aynı şeymiş gibi düşünmek başlamayı kolaylaştırıyor.

2. Farklı Bellekte Çalışıyorsak

Back Pressure'ın kötü ele alındığı bir örnek burada. Bu örnekteki normal akış şöyle

Read : 

- The eventloop threads read requests coming off of netty channels and enqueue them into a bounded inbound queue in the Cassandra node.

Process : 

- A thread pool dequeues requests from the inbound queue, processes them asynchronously and enqueues the response into an outbound queue. There exist multiple outbound queues, one for each eventloop thread to avoid races.

Send:

- The same eventloop threads that are responsible for enqueuing incoming requests into the inbound queue, are also responsible for dequeuing responses off from the outbound queue and shipping responses back to the client.

- Yani bir eventloop başına bir Netty Channel ve bir de Outbound kuyruk var. Outbound üst sınıra sahip değil

- Inbound queue tek bir tane ve üst sınıra sahip, yani tüm eventloop'lar için ortak

Sıkıntı şöyle

Eğer Process'in threadler yavaşlarsa eventloop üst sınırı belli (bounded) Inbound kuyruğa yazmak için çok fazla bloke oluyor ve Outbound kuyruğu boşaltmıyor. Açıklaması şöyle

Let us take a scenario where there is a spike in operations from the client. The eventloop threads are now enqueuing requests at a much higher rate than the rate at which the requests are being processed by the native transport thread pool. Eventually, the inbound queue reaches its limit and says it cannot store any more requests in the queue.

Consequently, the eventloop threads get into a blocked state as they try to enqueue more requests into an already full inbound queue. They wait until they can successfully enqueue the request in hand, into the queue.

As noted earlier, these blocked eventloop threads are also supposed to dequeue responses from the outbound queue. Given they are in blocked state, the outbound queue (which is unbounded) grows endlessly, with all the responses, eventually resulting in C* going out of memory. This is a vicious cycle because, since the eventloop threads are blocked, there is no one to ship responses back to the client; eventually client side timeout triggers, and clients may send more requests due to retries. This is an unfortunate situation to be in,...

Çözüm

Inbound kuyruk sınırsız hale getiriliyor ancak bir eşik değer konuluyor. Bu eşik değeri aşınca, sunucu konfigürasyon ayarına göre şu iki seçenekten birini uyguluyor

1. Okuma işlemini bırakır. Burada istemciyi yavaşlatma işi TCP protokolüne bıraklıyor. Açıklaması şöyle

Consequently, the kernel socket inbound buffer becomes full since no bytes are being read off of it by netty eventloop.

Once the Kernel Socket Inbound Buffer is full on the server side, things start getting piled up in the Kernel Socket Outbound Buffer on the client side, and once this buffer gets full, client will start experiencing backpressure.

2. İstemciye bir mesaj gönderir. Açıklaması şöyle

If the inbound queue is full (i.e. the thresholds are met), eventloop threads do not enqueue the request into the Inbound Queue. Instead, the eventloop thread creates an OverloadedException response message and enqueues it into the flusher queue, which will then be shipped back to the client.