Shuffle Algoritmaları

Giriş

Shuffle yani karıştırma işlemi de random sayı üretici kullanır. 

1. Fisher-Yates Algoritması

Fisher-Yates Algoritması yazısına taşıdım.

2. Daha Kötü Bir Shuffle Algoritması
Fisher-Yates algoritmasını bilmeyen insanlar safça şuna benzer bir kod yazıyorlar.

Random r = new Random();
for (int i = 0; i < a.size(); i++) {
    int index = r.nextInt(a.size());
    int temp = a.get(i);
    a.set(i, a.get(index));
    a.set(index, temp);
}

Bu kod sıradaki nesneyi, rastgele bir nesne ile yer değiştiriyor. Ancak zaten yer değiştirmiş bir nesnenin tekrar seçilme olasılığı yüksek. Bu yüzden Fisher-Yates algoritmasına göre daha yavaş çalışır. Fisher-Yates ise sondan başa doğru ilerler ve yer değiştiren bir nesneye tekrar dokunmaz.

3. Java İle Shuffle

Şöyle yaparız

// assume `cards` is a `List`, 
Collections.shuffle(cards); 
// cards is now shuffled

4. Belli Bir Aralıkta Karıştırılmış ve Tekrar Etmeyen Sayı Üretme

Belli bir aralıkta ve tekrar etmeyen sayı üretmek için önce sayılar bir listeye doldurulur daha sonra Collections.shuffle() çağırılarak rastgelelik sağlanır.

Örnek - Java

Şöyle yaparız

final List<Integer> l = new ArrayList<Integer>();
for (int j = 1; j < 7; j++ ) {
  l.add( j );
}
Collections.shuffle( l );

GraphQL Query

Giriş

Açıklaması şöyle

In REST API, we fetch data from specific endpoints. Each endpoint has a particular structure. In other words, the client needs to adhere to the API structure. Basically, the request URL determines the query parameters in a REST API.

However, GraphQL has a considerably different approach. Instead of multiple endpoints, a GraphQL server typically exposes only one endpoint. The structure of the response is not fixed. Instead, it is quite flexible. Basically, the client specifies what data is required and the server responds with the required data.

Root veya Hiyerarşik Sorgular

Nesneler başka nesneleri de içerebilir. Elimizde şöyle bir schema olsun

type Author {
  name: String!
  country: Int!
  books: [Book!]!
}

type Book {
  title: String!
  publishYear: Int!
  author: Author!
}

Sadece kök nesneyi istiyorsak sorgu olarak şöyle yaparız. Burada kitapların sadece title alanını istiyoruz

{
  allBooks {
    title
  }
}

Cevap olarak şunu alırız

{
  "allBooks": [
    { "title": "Eye of the World" },
    { "title": "The Way of Kings" },
    { "title": "The Mistborn" }
  ]
}

Eğer kök nesnenin diğer alanlarını da istiyorsak sorgu olarak şöyle yaparız. Burada kitapların title  ve publishYear alanlarını istiyoruz

{
  allBooks {
    title
    publishYear
  }
}

Hiyerarşik sorgu için şöyle yaparız. Burada yazarların name alanını ve yazdıkları kitapların title alanını istiyoruz. 

{
  allAuthors {
    name
    books {
      title
    }
  }
}

Bazı Örnekler

Örnek - parametresiz

Şöyle yaparız

{
  "query":"{findAllBooks { id title } }"
}

Örnek - parametreli

name + email çekmek için şöyle yaparız.

{
  "query": "query($id: String!){ retrieveUser (id: $id) {name email} }",
  "parameters": {
    "id": 1
  }
}

Sadece email çekmek için şöyle yaparız.

{
  "query": "query($id: String!){ retrieveUser (id: $id) {email} }",
  "parameters": {
    "id": 1
  }
}

Örnek - parametreli

Şöyle yaparız

query order($id: ID!){
  order(id: $id) {
    quantity
    price
    orderDate {
      day,
      month,
      year
    }
    isConfirmed
  }
}

Örnek - parametreli

Şöyle yaparız

void getPostById(WebClient client, String id) {
  client.post("/graphql")
        .sendJson(Map.of(
"query","query post($id:String!){ postById(postId:$id){ id title content author{ name }
  comments{ content createdAt} createdAt}}",
"variables", Map.of("id", id)
        ))
        .onSuccess(
          data -> log.info("data of postByID: {}", data.bodyAsString())
        )
        .onFailure(e -> log.error("error: {}", e));
}  

Örnek - sabit parametre

Şöyle yaparız

query {
    recentPosts(count: 10, offset: 0) {
        id
        title
        category
        author {
            id
            name
            thumbnail
        }
    }
}

Örnek - last kelimesi

Şöyle yaparız

  
{
  "data": {
    "User": {
      "name": "Fernando Doglio",
      "posts": [{
        "title": "Post #1",
        "post_date": "2021-06-28"
      }
      ...
      ],
      "followers": [
        { "name": "Follower #1" },
        { "name": "Follower #2" }
      ]
    }
  }
}

Çıktı olarak şunu alırız

{
  "data": {
    "User": {
      "name": "Fernando Doglio",
      "posts": [{
        "title": "Post #1",
        "post_date": "2021-06-28"
      }
      ...
      ],
      "followers": [
        { "name": "Follower #1" },
        { "name": "Follower #2" }
      ]
    }
  }
}

Örnek - DataLoader

Şöyle yaparız. Bir tane viewer nesnesi döner.

query CompanyData {
  viewer { # DataLoader called usersById
    company(id: "1") { # DataLoader called companiesById
      name

      transports { # DataLoader called transportsByCompanyId
        number

        carrier { # Dataloader called companiesById
          name
        }
      }
    }
  }
}

Açıklaması şöyle

Assuming, for example, that the user has access to 5 transports. Without DataLoaders, we would be looking at 8 database queries:

1 call to fetch the user (viewer)

1 call to fetch company with ID 1

1 call to fetch the 5 transports

5 calls to fetch the carriers for each of the transports

In this example, the benefit comes from the companiesById DataLoader, as it groups the 5 calls for getting the carriers of the transports into a single database query, therefore, saving us from making 4 database queries.

You can imagine scaling this example to something more plausible, such as the user having access to 500 transports, with DataLoaders, the database query count is still 4, without DataLoaders, it would now be 503.

This approach helps us solve the aforementioned N+1 problem, whilst maintaining privacy by sandboxing each request to a separate set of DataLoaders — the cached results are not shared between requests, meaning any permissions logic is never shared.

The DataLoaders also avoid errors in database queries by avoiding duplication of database queries, as well as speeding up the developer experience. More often than not, there already exists a DataLoader for the entity that the code is working with.

Graph Notlarım - Bipartite

Giriş
Şeklen şöyle

Bipartite aynı zamanda 2-colorable olarak ta bilinir. Açıklaması şöyle

Start at an arbitrary node, assigning it any of two colors. For each edge that is iterated, give the target node the color different from the one of its parent. If this is not possible, terminate the search as the input is not bipartite. Otherwise, you have generated a 2-coloring of the graph which indicates the two partitions.

Package By Feature

Giriş

Package By Feature kullanım her zaman Package By Layer'a tercih edilmeli.

Package By Layer Neden Zor

Kod katmanlara ayrılsa bile aralarındaki ilişkiyi anlamak gerçekten zor olabiliyor. Bir örnek şöyle. Burada problem katmanlar arasında kodun takip edilememesi.

“ If you try to organize a car on multiple layers, then you have to define at least : a mechanical layer, an electrical layer and a hydraulic layer. The day you have a problem with, let’s say the steering wheel, you are in big trouble, because in order to fix the problem, you need to go and find in each layer the component used by that feature. What you need to do instead, is to organize the car around features (steering, shift, air conditioning ..).”

domain

Bir başka açıklama şöyle

This approach has two main disadvantages:

- From a visibility point-of-view, to use classes outside their package, you need to mark them as public. FirstController uses FirstService, hence the latter must be public. Because of this, any other class can use it, whereas I want it to be used only for "First"-related classes.

- If you want to split the application, you’ll first need to analyze the dependencies to understand the coupling between packages.

Örnek

Elimizde şöyle "Package by layer" kod olsun

ch.frankel
  ├─ controller
  │  ├─ FirstController
  │  └─ SecondController
  ├─ service
  │  ├─ FirstService
  │  └─ SecondService
  └─ dao
     ├─ FirstDao
     └─ SecondDao

Bu kodu Package by feature ile şu hale getiririz

ch.frankel
  ├─ first
  │  ├─ FirstController
  │  ├─ FirstService
  │  └─ FirstDao
  └─  second
     ├─ SecondController
     ├─ SecondService
     └─ SecondDao

Package by Feature ve DCI Architecture anlaşılabilirlik açısından daha iyi özellikler sunuyor. Açıklaması şöyle.

The first system I worked on had layers separated into Maven modules (model, business-logic, webapp). It worked well at first – the structure was simple and easy to explain. The app was successful and as the codebase grew, it became obvious that this approach did not scale.

Back then I read about package-by-feature instead of package-by-layer approach, it made sense, and so we started moving in that direction. First, we’d simply move all classes related to a given feature into its own package, without changing them. This opened the door for further improvements and simplifications (like leveraging package-private visibility, removing some unnecessary mapping between layers, etc.), but I no longer recall which of those materialized while I was with the company.

One thing was clear – dropping packaging by layer undoubtedly improved the system.

Feature'ları Kendi İçinde Bölümlemek

Package By Feature kullanılsa bile paketleri kendi içinde bölümlemek gerekiyor. 

Ortak kullanılabilecek bazı paketler şöyle

config

domain

infrastructure

interfaces

Her bir feature altında da alt feature konuları olabilir.  Örneğin bir cihazı modellecek olayım. Cihazın

- signal detection

- position finder

- target finder

- effect calculator

- dispenser

- foo determiner

- bar receiver

- countermeasure manager

gibi alt özellikleri olsun. Bu karışık bir cihaz. Her bir alt feature da düzgünce paketlenmezse kod çok karışık olacak. 

Alt feature için benim düşündüğüm bir örnek şöyle

processors : Gelen mesajları işler

timers : Timer kullanılıyorsa buradadır

model : Domain nesnelerini saklar

requests : Domain'e gelen istekler
responses : Domain tarafından verilen cevaplar

Örnek

Açıklaması şöyle

In package-by-feature, the package names correspond to important, high-level aspects of the problem domain. For example, a drug prescription application might have these packages:

- com.app.doctor

- com.app.drug

- com.app.patient

- com.app.presription

- com.app.report

- com.app.security

- com.app.webmaster

- com.app.util

- and so on...

Each package usually contains only the items related to that particular feature, and no other feature. For example, the com.app.doctor package might contain these items:

- DoctorAction.java - an action or controller object

- Doctor.java - a Model Object

- DoctorDAO.java - Data Access Object

- database items (SQL statements)

- user interface items (perhaps a JSP, in the case of a web app)

Package-by-layer olarak düşünseydik elimizde şöyle bir yapı olurdu

- config

- controller

- domain

- repository

- service

Rust Programming Language

Giriş

Açıklaması şöyle

[S]ome common Rust packages depend on C code and will need a C compiler.

Kalıtım

Örnek

Şöyle yaparız

trait Animal {
  fn talk(&self);
}

struct Dog;
struct Cat;
impl Animal for Dog {
  fn talk(&self) {
    println("I am a dog");
  }
}
impl Animal for Cat {
  fn talk(&self) {
    println("I am a cat");
  }
}

Şöyle yaparız. Box<> C++’taki unique_ptr<> gibidir.

fn main() {
  let animals : Vector<Box<dyn Animal>> = vec![Box::new(Dog{}),Box::new(Cat{})];
  for animal in animals.iter() {
    animal.talk();
  }
}

Ownership

Örnek

Elimizde şöyle bir kod olsun. p1 değişkeni p2 değişkenine atanınca nesnenin sahipliği devredilir. p1 değişkenine erişmek istersek derleyici bunu yakalar.

let p1 = Person::new();
p2 = p1;
println!("person {#:?}", p2)
println!("person {#:?}", p1) //compile error

Clone

Deep copy şeklinde çalışır

Örnek

Şöyle yaparız.

let p1 = Person::new();
let p2 = p1.clone();
do_something(p2);
println!("{:?}", p1); //works ok

Borrowing

Nesnenin sahipliği devredilmez.

Örnek - Numeric Primitive Tipler

Nesne sahipliği yoktur. Görmek için şöyle yaparız.

fn sum(left: i32, right: i32) -> i32 {
  left + right
}

fn main() {
  let a = 42;
  let b = 1;
  let s = sum(a, b);

  println!("this sum of {} and {} is {}", a, b, s); // no error!
}

Örnek - Passing By Reference Olmazsa

Hata alırız. Görmek için şöyle yaparız.

fn sum(vector: Vec<i32>) -> i32 {
  let mut sum = 0;

  for item in vector {
    sum = sum + item
  }

  sum
}

fn main() {
  let v = vec![1,2,3];
  let s = sum(v);

  println!("sum of {:?}: {}", v, s); // ERROR: v was MOVED!
}

Düzeltmek için şöyle yaparız.

fn sum(vector: &Vec<i32>) -> i32 { // borrow signature
  let mut sum = 0;

  for item in vector {
    sum = sum + item
  }

  sum
}

fn main() {
  let v = vec![1,2,3];
  let v_ref = &v;  // v_ref borrows v
  let s = sum(v_ref);

  println!("sum of {:?}: {}", v_ref, s); // no error
}

Örnek -  Passing By reference

Şöyle yaparız.

let p1 = Person::new();
do_something(&p1);
println!("p = {}",p1); //continue using p1 since ownership has been was never passed on

fn do_something(p: &Person) {
  //logic
}

Örnek - Passing By Mutable reference

Şöyle yaparız.

let mut p1 = Person::new();
do_something(&mut p1);
println!("mutated info {}",p2)

fn do_something(p: &mut Person, new_name: String) {
   p.name = String::from(new_name);
}

Vector

Örnek

Şöyle yaparız

fn main() {
  let a = vec![1, 2, 3]; // a growable array literal
  let b = a;             // move: `a` can no longer be used

  println!("a: {:?}", b);
}

Yazılım Mimarisi - Kappa Architecture - Stream Processing İçindir

Giriş

Açıklaması şöyle

In 2014 Jay Kreps started a discussion where he pointed out some discrepancies of Lambda architecture that further led the big data world to another alternate architecture that used less code resource and was capable of performing well in certain enterprise scenarios where using multi layered Lambda architecture seemed like extravagance.

Kappa Architecture cannot be taken as a substitute of Lambda architecture on the contrary it should be seen as an alternative to be used in those circumstances where active performance of batch layer is not necessary for meeting the standard quality of service. This architecture finds its applications in real-time processing of distinct events.

Açıklaması şöyle

In 2014, Martin Kleppman, author of the book Designing Data-Intensive Applications gave a seminal talk entitled Turning the database inside out. He outlined a new architectural pattern for processing data in real-time. The key insight is to externalize the write-ahead-log (WAL), which up to this point had been an internal (yet fundamental) component of every operational database. A streaming framework could then be used to process the log in near real-time, thereby providing an always-up-to-date equivalent to a materialized view.

...

In recent years, Kleppman’s inside-out database concept has gained popularity, and even acquired a name: Kappa architecture.

Bu mimari şeklen şöyle Hadoop tabanlı Batch Processing kısmı çıkarılıyor ve her şey Stream Processing ile hallediliyor.

Bu mimaride veri Hadoop gibi bir yerde devamlı saklanmadığı için hata yapılması durumunda geri dönüş daha zor.

Apache Kafka Streams API

Giriş 

Açıklaması şöyle

Kafka Streams is a client library that uses Kafka topics as sources and sinks for the data processing pipeline.

Bir başka açıklama şöyle

Kafka Streams is a simple Java library that enables streaming application development within the Kafka framework.

Açıklaması şöyle

Kafka Streams is a JVM based library for performing streaming transformations on data sourced from Kafka. It is a very good default choice for data processing backed by Kafka due to its simple deployment model, horizontal scalability, resilience to failures and straightforward well documented public API.

The API lets you read data from Kafka then perform a number of transformations on the data including filtering, mapping, aggregation and joining data from multiple topics. The API supports both stateless and stateful transformations. Stateful transformations back up their state into changelog topics in Kafka and cache values local to the processing nodes, typically using RocksDB.

Streams API kullanmak için iki yöntem var. Bunlar şöyle

There are two approaches to writing a Kafka Streams application:

- The high level DSL,

- And the low level Processor API.

Java örneklerini Kafka Streams API yazısına taşıdım

Exactly once semantic support

Açıklaması şöyle

Exactly once semantic support was added to Kafka Streams in the 0.11.0 Kafka release. Enabling exactly once is a simple configuration change setting the streaming configuration parameter processing.guaranteeto exactly_once(the default is at_least_once).

A typical workload for a Kafka Streams application is to read data from one or more partitions, perform some transformations on the data, update a state store (such as a count), then write the result to an output topic. When exactly once semantics is enabled, Kafka Streams atomically updates consumer offsets, local state stores, state store changelog topics and production to output topics all together. If any one of these steps fail, all of the changes are rolled back.

Side effects
The Kafka Streams API lets you perform any action when processing input data, for instance you can write the data directly to a database, or fire off an email. These “side effect” operations are explicitly not covered by the exactly once guarantee. If a stream job fails after processing data, but just prior to writing it back to Kafka, it will be reprocessed the next time the Kafka Streams worker is restarted re-running any side-effect operations such as emailing a customer.

It is best to keep your Kafka Streams transform “pure” with no side effects beyond updating state stores and writing back to Kafka. This way your application will be more resilient (it won’t fail if it can’t contact the email server) and will shorten the length of transactions.

Side effects can then be performed via Kafka Connect Sink connectors or custom consumers.

Enterprise Messaging vs Event Streaming

Enterprise Messaging ve Event Streaming şu açılardan farklıdır

1. Message processing style

2. Message consumption style

3. Access to message history

4. Fine-grained subscription to messages

5. Message delivery guarantees

1. Message Processing Style

Enterprise Messaging uygulamaları her mesajı bireysel olarak el alır. Streaming uygulamaları mesajları bütünsel olarak ele alabilir.

Örnek

Farkları gösteren bir örnek şöyle

For example, if the reading is less than 20, the thermostat turns on the heater. It repeats the same logic the next time a message comes. Even though this is unnecessary, that is how the messaging works.

For example, the thermostat can calculate the average reading over the last minute and decides whether to turn on the heater. The average seems a realistic measure here.

2. Message consumption style

Enterprise Messaging uygulamaları mesajları siler, Streaming uygulamaları mesajları silmez. 

3. Access to message history

Message consumption style maddesinin sonucu olarak mesaj tarihçesi de oluşur veya oluşmaz

4. Fine-grained subscription to messages

Enterprise Messaging uygulamaları mesajlara filtreler koyabilir, Streaming uygulamaları bir partitiondaki tüm mesajları alır ve  kendisi filtreler.

5. Message delivery guarantees

Bazı seviyeler şöyle

- At least once delivery

-Exactly once delivery

- Transactionally coordinated delivery

Açıklaması şöyle

Messaging systems are good at handling exactly once and transactionally coordinated delivery use cases while streaming is good at handling at least once delivery scenarios.

Stream Analytics

Aslında tüm bu farkların söylemeye çalıştığı şey Stream Analytics. Açıklaması şöyle

But, capturing the data streams is only one part of the challenge. Some data processing has to be performed simultaneously with the incoming data to be able to use the results promptly for decision-making. For example, a selection of products in a shopping cart system can be the trigger for a recommendation system to be executed in parallel. This type of requirement creates another building block in the streaming architecture – called Stream Analytics.

Sometimes a single event from the data stream is enough to trigger a predefined business logic. However, it is often necessary to be able to recognize connections between different events in order to run a high-level business process generating real business value. Such a connection can be established between time-shifted similar events by accumulating them over a given period of time. For example, short-term increased demand for a certain product in an online shop system could trigger the start of an additional production line. In other cases, it may be necessary to correlate certain events of different types and merge the data to trigger the corresponding business process. These methods are also known as Windowing and  Joining.

Stream Analytics için iki temel yöntem var. Windowing ve Joining. Şeklen şöyle

Kafka Streams ve CQRS

Kafka Stream'lerini Event Source olarak kullanmak mümkün.