AWS Lambda Cold Start

Giriş

Cold Start probleminin sebebi Scale To Zero

Scale To Zero

Açıklaması şöyle. Yani istek yoksa, ortam söndürülüyor, ve her şey en baştan başlıyor

What is “Scale to Zero”?

Simply put, “Scale to Zero” is a regular container deployment that automatically scales to zero instances when there is no incoming traffic. The moment a request hits the service, an instance is started to handle it. This is also commonly referred to as “cold start,” similar to e.g., AWS Lambdas.

Nearly all major cloud providers support “Scale to Zero” in some form or the other.

Cold Start Problemi

Açıklaması şöyle

When Lambda receives a request to execute a task, it starts by downloading the code from S3 buckets and creating an execution environment based on the predefined memory and its corresponding compute resources. If there is any initialization code, Lambda runs it outside the environment and then runs the handler code. The time required for downloading the code and preparing the execution environment is counted as the cold start duration. After executing the code, Lambda freezes the environment so that the same function can run quickly if invoked again. If you run the function concurrently, each invocation gets a cold start. There will also be a code start if the code is updated. The typical time for cold starts falls between 100 ms and 1 second. In light of the foregoing, Lambda falls short in the Lambda vs Fargate race regarding cold starts. However, Provisioned Concurrency is a solution to reduce cold starts.

The runtime choice will also have an impact on Lambda cold starts. For instance, Java runtime involves multiple resources to run the JVM environment, which delays the start. On the other hand, C# or Node.js runtime environments offer lower latencies.

Çözümler şöyle

1. Lambda Provisioned Concurrency

2. SnapStart

1. Lambda Provisioned Concurrency

Açıklaması şöyle. Yani sürekli çalışan lambda demek

In other words, this facilitates the creation of pre-warmed Lambdas waiting to serve incoming requests. As this is pre-provisioned, the configured number of provisioned environments will be up and running all the time even if there are no requests to cater to. Therefore, this contradicts the very essence of serverless environments. Also, since environments are provisioned upfront, this feature is not free and comes with a considerable price.

2. SnapStart

Açıklaması şöyle. Yani sürekli çalışmaya hazır lambda demek. “AWS console”—> “Configuration”—> “General Configuration”—> “Edit.” menüsünden etkinleştirmek gerekir.

With SnapStart, Lambda initializes your function when you publish a function version. Lambda takes a Firecracker micro VM snapshot of the memory and disk state of the initialized execution environment, encrypts the snapshot, and caches it for low-latency access. When you invoke the function version for the first time, and as the invocations scale up, Lambda resumes new execution environments from the cached snapshot instead of initializing them from scratch, improving startup latency. The best part is that, unlike provisioned concurrency, there is no additional cost for SnapStart. SnapStart is currently only available for Java 11 (Corretto) runtime.

Açıklaması şöyle

Lambda SnapStart for Java is a new feature that enables you to resume new execution environments from cached snapshots without initializing them from scratch. It helps improve startup latency. However, this feature is only available for Java 11-managed runtime environments. There are other limitations as well. It does not support provisioned concurrency, Amazon X-Ray, Amazon EFS, or arm64 architecture. Moreover, you cannot use ephemeral storage of more than 512 MB.

Hard Real-time Nedir?

Giriş

Hard Real-time kelimesinin tercümesi konusunda bir uzlaşma yok. Bazen critical real-time da deniliyor.

1. Sıkı Gerçek Zaman

2. Katı Gerçek-Zaman

3. Sert Gerçek-Zaman

gibi tercümeler var

 

Açıklaması şöyle.

Hard real-time: Missing a deadline is a total system failure. Delays or spikes are not accepted. Hence, the goal is to ensure that all deadlines are met,

Açıklaması şöyle. 

Hard real-time is a deterministic network with zero spikes and zero latency. That’s a requirement for embedded systems using programming languages like C, C++, or Rust to implement safety-critical software like flight control systems or collaborative robots (cobots).... the right technology for safety-critical latency requirements.

Cache Stratejileri - Cache Access Patterns Write-Around

Giriş

Veri tabanı güncellenir ancak cache güncellenmez. Cache verisi bayatlayınca veri tabanından son durumu okur.

Açıklaması şöyle

This strategy populates the underlying store but not the cache itself. In other words, the write bypasses the cache and writes to the underlying store only.

Açıklaması şöyle

Write request goes around the cache straight to DB and acknowledge is sent back, data is not sent to cache. Data is written to the cache when there is the first cache miss.

Açıklaması şöyle

In this design, cache entry only expires when exceeds the pre-set TTL. There is no cache invalidation nor cache update in the write path. The advantage is that the implementation is very simple, but at the cost of even more cache staleness — as long as the TTL window.

Dezavantajı

Açıklaması şöyle

Written data won't immediately be read back from cache

Ne Zaman Kullanılır

Açıklaması şöyle

This is often used when write volumes are large but read volumes are significantly lower.

Write-Around vs Cache-Aside

Açıklaması şöyle

The difference is that with Cache-Aside, the focus is on the reads and lazy loading — only populating the data into the cache when it is first read from the datastore. Whereas with Write-Around caching, the focus is on write performance instead. This technique is often used to avoid cache pollution when data is being written often but is infrequently read.

Cache Stratejileri - Cache Access Patterns Write-Behind veya Write Back

Giriş

Şeklen şöyle

Aslında Write-Through ile aynıdır. Tek fark veri tabanı güncellemesi senkron değil asenkron yapılır

Açıklaması şöyle

Write-behind approach is very much similar to write-through, just that the database write calls are asynchronous in fashion.

Açıklaması şöyle. 

In a write-behind cache, a write request only updates the cache. Then another background process asynchronously updates the DB with the new entries in the cache. The asynchronous DB update can be implemented as periodic batch update, and the workload can be scheduled to run during mid-night, i.e. when the DB load is low.

Açıklaması şöyle

first write into database and then into cache.

Avantajı

Açıklaması şöyle

In write-heavy environments where slight data loss is tolerable

Cache Stratejileri - Cache Access Patterns Write-Through - Implemented by Cache Provider

Açıklaması şöyle. Yani önce cache güncellenir, sonra veri tabanı güncellenir,

1. The application writes the data directly to the cache.

2. The cache updates the data in the main database. When the write is complete, both the cache and the database have the same value and the cache always remains consistent.

MapWriter kullanılır. Açıklaması şöyle

whenever any “write” request comes, it will come through the cache to the DB. Write is considered successful only if data is written successfully in the cache and in DB.

Eğer Spring ile bunu taklit etmek istersek şöyle yaparız

// With Spring you can mimic it
// Both DB + cache updated synchronously.
@CachePut(value = "users", key = "#user.id")
public User saveUser(User user) {
    return userRepository.save(user);
}   

Cache Stratejileri - Cache Access Patterns Read-Through

Giriş

MapLoader kullanılır. Açıklaması şöyle. Uygulama sadece Cache'e erişir. Cache gerekiyorsa, veri tabanından sorgulama yapar.

1. The App never interacts with DB directly but always via Cache.

2. On a cache miss, the cache will read from DB and enrich the cache storage.

3. On a cache hit, data is served from the cache.

You can see, the DB is reached very infrequently and the response is fast since the caches are mostly in-memory (Redis/ Memcached). 

Avantajı

Açıklaması şöyle

Keeps cache consistently populated by handling misses automatically

Read-Through ve Request Collapsing Kavramı

Bir nesne için aynı anda çok fazla istek gelirse, Cache veri tabanına çok fazla sayıda istek gönderir. Bu isteklerin birleştirilmesine Request Collapsing deniliyor.

Cache-aside Okuma vs Read-Through

Şeklen şöyle

Yazılım Mimarisi - Hexagonal Architecture İle Repository

Giriş

Domain sadece bir arayüzü yani portu biliyor.

Örnek

Elimizde şöyle bir arayüz olsun

public interface StudentRepository {

    Student save(Student student);
    Optional<Student> retrieveStudentWithEmail(ContactInfo contactInfo);
    Publisher<Student> saveReactive(Student student);

}

Bu arayüz için domain tarafında test yazalım

public class StudentRepositoryTest {

  StudentRepository studentRepository;

  @Test
  public void shouldCreateStudent() {
    Student expected = ...;
    Student actual = studentRepository.save(expected);
    ...
  }

  @Test
  public void shouldUpdateExistingStudent() {
    Student expected = randomExistingStudent();
    Student actual = studentRepository.save(expected);
    ...
  }
}

Şimdi bellekte repository gerçekleştirimi yani adapter için test yazalım

public class StudentRepositoryInMemoryIT extends StudentRepositoryTest {

  @BeforeEach
  public void setup() {
    super.studentRepository = new StudentRepositoryInMemory();
  }
}

Şimdi de gerçek veri tabanı gerçekleştirimi yani adapter için test yazalım. Burada SpringBoot kullanılıyor

@Testcontainers
@ContextConfiguration(classes = {PersistenceConfig.class})
@DataJpaTest
@AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE)
public class StudentRepositoryJpaIT extends StudentRepositoryTest {

  @Autowired
  public StudentRepository studentRepository;

  @Container
  public static PostgreSQLContainer container = new PostgreSQLContainer("postgres:latest")
    .withDatabaseName("students_db")
    .withUsername("sa")
    .withPassword("sa");


  @DynamicPropertySource
  public static void overrideProperties(DynamicPropertyRegistry registry){
    registry.add("spring.datasource.url", container::getJdbcUrl);
    registry.add("spring.datasource.username", container::getUsername);
    registry.add("spring.datasource.password", container::getPassword);
    registry.add("spring.datasource.driver-class-name", container::getDriverClassName);
  }

  @BeforeEach
  public void setup() {
    super.studentRepository = studentRepository;
  }
}