Materialized Views

Giriş

Materialized View iki açından incelenebilir.

1. Stream'ler açısından - Kafka kullanıyorsanız Apache Kafka ksqlDB tercih edilebilir.

2. İlişkisel Veri tabanı açısından - Tercih Etmeyin

1. Stream'ler Açısından Materialized View Nedir?

Açıklaması şöyle. Yani bir sorunun cevabı hazır olarak Materialized View içinde vardır

Meaning, a table can contain a pre-computed answer to a query in a ready-to-be-served form. That is known as a materialized view.

Sorunun cevabını hesaplamak için Stream'in tabloya dönüşmesi gerekir. Açıklaması şöyle

An interesting observation is that there is a close relationship between streams and tables. We call this the so-called stream-table duality. Essentially, this duality means that a stream can be viewed as a table, and a table can be viewed as a stream.

Stream Tabloya Nasıl Dönüşür

Açıklaması şöyle. Yani Stateful Aggregation yapılır

We can turn a stream into a table by grouping the keys of individual events in the stream and then performing aggregate operations like COUNT() or SUM() on the event values. The table keeps only the latest value for events that share the same key.

Tabloyu Stream'e  Dönüştürmek

Açıklaması şöyle

We can turn a table into a stream by capturing the changes made to the table — inserts, updates, and deletes — into a “changelog stream.” This process is often called change data capture or CDC for short.

Materialized View Ne İşe Yarar

Açıklaması şöyle

The materialized view subscribes to an incoming stream of events to refresh itself in an event-driven manner. For example, an order-summary view updates itself by listening to events coming from the orders topic.

Materialized views enable us to query the current state of a system. For example, it answers questions like:

- What is the average temperature of room 213 within the last hour?

- What is the status of order X?

- How many packages have been delivered during the last week?

Materialized View Verisi Nerede Saklanır?

Açıklaması şöyle.

There are two ways:

1.Using an internal state store: The application stores the current state in the local VM/container/process.

2. Using an external state store: The application uses external storage like a database, search index, or a file system to store the state globally.

Internal state stores come in handy when the event processing application needs to perform low-latency aggregations on the stream. Examples include SUM and COUNT operations. 

Stateful event processing frameworks such as Kafka Streams and ksqlDB advocate for fault-tolerant and scalable internal state stores based on RocksDB.

2. İlişkisel Veri Tabanında Normalization Açısında Materialized Views Nedir - Tercih Etmeyin

Açıklaması şöyle. Denormalization yerine Materialized Views da düşünülebilir.

Most relational databases have something called Materialized Views. It basically has the database precompute a query and keep that around for quick response. The database is then responsible for keeping the View consistent with the data, including any complications from atomicity transactions. Basically like database-managed caching.

Read the documentation for your database, because this can get complicated.

Using Materialized Views is no more a denormalization antipattern than using indices. But making the application layer ensure data consistency when the database could be doing it: that is an antipattern.

Söz dizimi olarak şeklen şöyledir ve sonuçları bi yerde saklar.

SELECT MATERIALIZED VIEW timeline AS
SELECT user_id, display_name, avatart_url ...
FROM foo
JOIN bar ON ...

2.1 Materialized views vs. traditional views

Açıklaması şöyle

Even though their syntax is almost identical, materialized views are different from traditional database ‘views.’

A view or a non-materialized view is just an alias for a query; when you read from the view, the database translates it into the underlying query. It gives you a convenient wrapper around a complicated query to shield you from the underlying complexity. Other than that, you will not gain any performance benefits.

...

Each time you query the view, the database does all the joins and aggregations on the fly. There’s no cached result.

2.2 Keeping the view synchronized with source tables

İki yöntem var

1. Periodic full rebuilds

2. Incremental refreshes

2.3 Veri tabanı üzerindeki etkisi

Açıklaması şöyle

Maintaining a materialized view imposes a massive tax on the database. Especially, periodic rebuilds of views can severely impact the performance because every change since the last refresh needs to be computed from scratch. Also, it is often challenging to choose a refresh schedule that has a balance between stale data and the cost of the rebuild.

While incremental refreshes are available in a handful of databases, they often come with a long list of restrictions.