Software Release Dates and Features - Project Management

Giriş

Bu yazıda proje yönetemiyle ilgili bazı notlar var.

Yayım Yöntemi - Software Release Dates and Features

Yazılımın içerdiği özellikler ve yayım tarihi için 4 tane yöntem izlenebilir. Bunlar şöyle.

We have historically seen over and over again that there are two working and two non-working ways of combining the two fundamental constraints on software releases: dates and features.

1. Fixed date, flexible features, aka "release what's ready": you release at a pre-determined date, but you only release what is working. This is a model that is successfully used by Ubuntu, Windows, Linux, and many others.

2. Fixed features, flexible date, aka "release when ready" or "It's done when it's done": you determine the set of features beforehand, and then you simply work until the features are finished. Some Open Source projects work this way.

3. Fixed date and features.

4. Flexible date and features.

1. Fixed date, flexible features ve Fixed features, flexible date

Windows ve Linux 1 numarayı takip ediyorlar.  Yani her 6 ayda bir hazır olan özellikleri yayımlıyorlar. Açıklama şöyle. Linux daha sonra 2 numarayı uyguluyor. Yeni özelliklerdeki hatayı gideriyor ve bir sürüm daha yayınlıyor.

#1 and #2 have been shown to work well in many different projects. For example, both Ubuntu and Windows are released with a fixed 6-month cadence with whatever features are ready in time for the release. If you make the cadence fast enough, even if a feature misses the release, customers don't have to wait a very long time for the next release.

Linux actually uses an interesting staging of the two: as soon as there is a new release, there is a fixed-time "merge window" of two weeks, during which new features are added. When this merge window closes, the set of merged features up to that point is fixed, and a "stabilization period" starts, during which the fixed set of features is stabilized, any bugs fixed, etc. This process takes as long as it takes, there is no deadline. When everything is stable, a new release is made, and the process starts anew. It turns out that this actually leads to a fairly stable release cadence of 6-8 weeks, but the point is that this cadence is not enforced, it emerges naturally.

Note that this does not invalidate my assertion that #3 doesn't work: Linux development does not fix dates and features. They do #1, then make a cutoff point and switch over to #2.

2. Fixed Date And Features

Eğer süre uzunsa sıkıntılı bir süreç. Açıklaması şöyle

#3 is always a big problem, especially with a larger feature list and longer timeframes. It is pretty much impossible to predict the future (many have tried), so your estimates are almost always off. Either you have finished all the features and are sitting around bored twiddling your thumbs, or, more likely, you bump up against the deadline and frantically try to finish all the features in a hellish death march.

It does work if you keep the feature list and timeframe short enough. E.g. this is essentially what a Sprint is in Agile Methodologies: a fixed set of features in a fixed timeframe. However, the timeframes are reasonably short (typically a Sprint is one week or two), and it is ensured that there is rapid and immediate feedback and adjustment. You generally have a Sprint Retrospective after every Sprint, where you gather all the problems and successes of the Sprint and incorporate what you have learned into the next Sprint. And of course there is a Sprint Planning Meeting where the team discusses the next Sprint with the customer and agrees on a set of features to be implemented during that week.

Weekly (or two-weekly) Sprint Retrospectives are still not fast enough feedback, though, so there is also a Daily Standup Meeting with essentially the same goals as the Sprint Retrospective, except being able to react even faster: check whether the previous day's goals were met, and if they weren't, figure out what the problem was and fix it. (Note, I wrote "what" the problem was, not "who"!)

It is also very important that every Sprint ends with the release of a working product, so that the customer can immediately start using the new features, play around with them, get a feel for them, and give feedback for the next Sprint what is good, what isn't, what should be changed, etc.

3. Flexible date and features

Sürekli bir şey eklendiği için hiç bitmiyor. Açıklaması şöyle

#4 almost always leads to never-ending releases with feature creep. Debian 3 and Windows Longhorn were famous examples that interestingly happened around the same time. Neither of the two had a fixed release date, and neither of the two had a fixed set of features. Longhorn took 5 years, Debian 3.1 took 3. In both cases, what happened was that they didn't want to cut features because the long release meant that people would have to wait even longer for the features to appear in the next release. But because of not cutting features the release date slipped even further, so they added even more features because otherwise users would have to wait even longer, but that made the release date slip, and so on and so forth. An even more famous example might be ECMAScript 4.

Eğer Proje Gecikiyorsa

Daha fazla adam yığmak ta çözüm değilse, #3 olan proje #1 veya #2'ye dönüştürülmeye çalışılabilir. Açıklaması şöyle.

So, what can you actually do in your situation? Well, you are currently in situation #3, and that simply does not work. You have to turn your situation #3 either into a #1 or a #2 by either relaxing the release date or dropping features. There simply is nothing else you can do.

The damage was done 6 months ago, and it cannot be magically fixed. You are in the situation where the amount of features cannot be delivered in the amount of time, and one of the two has to give.

Bazı özelliklerden vazgeçilmesi,  sözleşmedeki bazı muğlak ifadeleri kullanarak yapmak mümkün. Açıklaması şöyle.

For several projects, it is not uncommon to pretend #3 to the customer, but make use of the fact feature specs from a contract often leave room for details. An experienced project manager decides about how these details are filled to match the schedule. So from a higher level of abstraction, the project goes for 3, but from a more detailed level of abstraction, it is #2.

Bazı özelliklerden vazgeçilmesi için her şeye karşı çıkan adam rolünü de oynamadan şu pazarlık yöntemi seçileblir.

The solution to this is very easy: for a while, say yes to absolutely everything, but make sure it comes at a cost they need to negotiate with other stakeholders.

For example, Marketing asks you to change a deadline because they want to show the product at some trade show. You go along with it:

"yes, I like the idea, I want to help you. In order to achieve it, we will need to cut x and y, or z from the product. Let me set up a meeting with the x, y and z stakeholders so you guys can decide how you want to proceed."

Or you are told that feature A is absolutely needed for the next release:

"of course, I love feature A, we should add it. It will require that we don't finish feature B or C though, let me bring in Mike and Gary so you can decide with them what should be cut. Or, we could extend the deadline, should we set up a meeting to discuss the cost of pushing it for your feature?"

This goes nowhere very quickly, but by doing this you are actively training people to understand that every request has a cost. You are not blocking anything, you are actively facilitating the process but anyone that wants something will soon realize that they need to negotiate with others.

Collision Detection

Giriş 

Özellikle 2D oyunlarda sıkça karşımıza çıkar. Bu konuyu burada gördüğüm için yazmak istedim

1. Axis-Aligned Bounding Box

Açıklaması şöyle

One of the simpler forms of collision detection is between two rectangles that are axis aligned — meaning no rotation. The algorithm works by ensuring there is no gap between any of the 4 sides of the rectangles. Any gap means a collision does not exist.

Onun sağ üst köşesi benim sol üst köşemin sağındaysa ve //X ekseni

Onun sol üst köşesi benim sağ üst köşemin solundaysa ve //X ekseni

Onun sağ alt köşesi benim sağ alt köşemin altındaysa ve //Y ekseni

Onun sol üst köşesi benim sağ alt köşemin üstündeyse  //Y ekseni

mutlaka üst üste gelme vardır

Örnek

Bu metodun ismi overlaps() veya intersects() olabilirdi. Şöyle yaparız

var rect1 = {x: 5, y: 5, width: 50, height: 50}
var rect2 = {x: 20, y: 10, width: 10, height: 10}

if (rect1.x < rect2.x + rect2.width && 
   rect1.x + rect1.width > rect2.x &&  
   rect1.y < rect2.y + rect2.height && 
   rect1.y + rect1.height > rect2.y) {
    // collision detected!
}

Örnek

Şöyle yaparız

bool valueInRange(int value, int min, int max)
{ return (value >= min) && (value <= max); }

bool rectOverlap(rect A, rect B)
{
  bool xOverlap = valueInRange(A.x, B.x, B.x + B.width) ||
                  valueInRange(B.x, A.x, A.x + A.width);

  bool yOverlap = valueInRange(A.y, B.y, B.y + B.height) ||
                  valueInRange(B.y, A.y, A.y + A.height);

  return xOverlap && yOverlap;
}

2. Circle Collision

Açıklaması şöyle. İki dairenin merkezleri arasındaki mesafeyi bularak bu mesafenin iki dairenin yarıçaplarını toplamından küçük olup olmamasın dayalı

Another simple shape for collision detection is between two circles. This algorithm works by taking the centre points of the two circles and ensuring the distance between the centre points are less than the two radii added together.

Örnek

Şöyle yaparız

var circle1 = {radius: 20, x: 5, y: 5};
var circle2 = {radius: 12, x: 10, y: 5};

var dx = circle1.x - circle2.x;
var dy = circle1.y - circle2.y;
var distance = Math.sqrt(dx * dx + dy * dy);

if (distance < circle1.radius + circle2.radius) {
    // collision detected!
}

3. Circle + Square Collision

Açıklaması şöyle

We have a circle with the position (cx,cy) with a radius r and a square at (rx,ry) with a width/height (rw,rh).

Our code will first test which edge of the rectangle is closest to the circle, then see if there is a collision using the Pythagorean Theorem. 

Dairenin hangi tarafta olduğunu anlamak için kare sınıfının kodunu şöyle yaparız. Burada karenin sol üst köşesi rx , ry noktası. Dairenin merkezi ise cx,cy noktası

Elimizde şöyle iki değişken olsunn

// temporary variables to set edges for testing
float testX = cx;
float testY = cy;

Şöyle yaparız

if (cx < rx)         
  testX = rx;        // left edge
else if (cx > rx+rw) 
  testX = rx+rw;     // right edge

if (cy < ry)         
  testY = ry;        // top edge
else if (cy > ry+rh) 
  testY = ry+rh;     // bottom edge

Daha sonra şöyle yaparız

// get distance from closest edges
float distX = cx-testX;
float distY = cy-testY;
float distance = sqrt( (distX*distX) + (distY*distY) );

// if the distance is less than the radius, collision!
if (distance <= radius) {
  return true;
}
return false;    

AES - Advanced Encryption Standard Counter (CTR) Mod - Kullanmayın

Giriş

Açıklaması şöyle

CTR mode was introduced by Whitfield Diffie and Martin Hellman in 1979

AES CTR modda çalışabilir. Veriyi padlemez, veri ile aynı büyüklükte çıktı verir. Bu yüzden Stream Cipher olarak sınıflandırılır.

TLS ve CTR

Açıklaması şöyle. TLS 3 tane CTR mode cipher'a sahip.

  +------------------------------+-------------+
  | Description                  | Value       |
  +------------------------------+-------------+
  | TLS_AES_128_GCM_SHA256       | {0x13,0x01} |
  |                              |             |
  | TLS_AES_256_GCM_SHA384       | {0x13,0x02} |
  |                              |             |
  | TLS_CHACHA20_POLY1305_SHA256 | {0x13,0x03} |
  |                              |             |
  | TLS_AES_128_CCM_SHA256       | {0x13,0x04} |
  |                              |             |
  | TLS_AES_128_CCM_8_SHA256     | {0x13,0x05} |
  +------------------------------+-------------+

Tercih Etmeyin

Açıklaması şöyle. Bu sıkıntıyı CBC ile aşabilsek te yine de CBC tercih edilmemeli.

Schneier and Ferguson initially recommended CTR mode, but switched to recommending CBC mode due to potential difficulties in ensuring that the counter was never re-used with a key.

Esas sıkıntı şifrelenen verinin değişmesi durumunda CTR'nin tesbit edememesi. Aslında aynı sıkıntı CBC modunda da var. Açıklaması şöyle. 

Its main drawbacks are that:

1. it does not protect message integrity (and in fact allows an attacker to easily flip arbitrary bits in an encrypted message without having to decrypt it), and

2. it may leak plaintext if a counter block is ever reused; in particular, CTR mode encryption requires a unique nonce input that absolutely must not be reused for two different messages encrypted with the same key.

Bu yüzden bu mod yerine GCM daha çok tercih ediliyor. Açıklaması şöyle

You can prevent this by using a message authentication system like HMAC or, even better, an authenticated encryption mode like GCM, which functions like CTR mode but also authenticates the data and thereby makes it impossible to change the ciphertext without being detected.

Nonce/IV

Nonce ve IV farkının açıklaması şöyle

If the IV is a nonce, that means it is a number used once (eg CTR mode). 

Bu modda IV'nin unique olması gerekir. Genellikle IV için bir sayaç tutulur ve bir artırılarak kullanılır. IV bir önceki bloğun çıktısı olmadığı için her blok birbirini beklemek zorunda kalmaz. Böylece CTR paralel çalıştırılabilir. CTR yöntemi ECB mod kullanılarak yapılabilir. Açıklaması şöyle.

You can implement CTR mode if all you have is ECB. All you have to do is use ECB to encrypt the successive counter values and use the resulting ECB-ciphertext as the CTR keystream to xor with the actual plaintext you want to encrypt.

Nonce Reset Saldırısı

Nonce reset saldırısı ile ilgili açıklama şöyle.

In many ciphers, a key is used to produce a series of ones and zeros that are xor'd with the data to produce an encrypted value. This string of ones and zeros is known as the keystream. If you are able to reuse a nonce and the keystream isn't altered based on the content of previous plaintext blocks, then you can look for known packets to appear in the content. When these known packets occur, you will see a predictable pattern occur and will be able to determine that you know that portion of the keystream.


By repeating the process, you eventually can gain more and more knowledge of the keystream and can then read or manipulate anything found on a known portion of the keystream. RC4 is EXTREMELY vulnerable to nonce reuse as it is a pure xor'd keystream. Similarly CTR modes of AES will be extremely susceptible. AES CBC-MAC will be a bit less impacted, however doing constant key resetting could potentially bypass the chaining due to always using the same set of initial blocks, so some information leakage would potentially still be possible, especially for the attacker faking a new client where racing will not be a factor.

NoSQL

NoSQL Kelimesi Nereden Geliyor?

Designing Data Intensive Applications kitabındaki açıklama şöyle. 2009 yılında kullanılmaya başlanmış.

The name “NoSQL” is unfortunate, since it doesn’t actually refer to any particular technology—it was originally intended simply as a catchy Twitter hashtag for a meetup on open source, distributed, nonrelational databases in 2009. Nevertheless, the term struck a nerve and quickly spread through the web startup community and beyond. A number of interesting database systems are now associated with the #NoSQL hashtag, and it has been retroactively reinterpreted as Not Only SQL.

Neden NoSQL

Designing Data Intensive Applications kitabındaki açıklama şöyle. Bence en önemli sebep veri miktarındaki büyük zıplama

There are several driving forces behind the adoption of NoSQL databases, including:

• A need for greater scalability than relational databases can easily achieve, including very large datasets or very high write throughput

• A widespread preference for free and open source software over commercial database products

• Specialized query operations that are not well supported by the relational model

• Frustration with the restrictiveness of relational schemas, and a desire for a more dynamic and expressive data model

Different applications have different requirements, and the best choice of technology for one use case may well be different from the best choice for another use case. It therefore seems likely that in the foreseeable future, relational databases will continue to be used alongside a broad variety of nonrelational datastores—an idea that is sometimes called polyglot persistence.

NoSQL Veri tabanları

MongoDB - Document Database

Couchbase - Document Database 

Apache Cassandra - Column Oriented Database

CAP Teoremi - Distributed Data Store İçindir

Giriş

1998 yılında Eric Brewer adındaki bir profesör tarafından ortaya atılmıştır. Dağıtık veri tabanlarıyla (Distributed Data tore) ilgilidir. Eğer kullandığımız veri tabanı dağıtık değilse CAP geçerli değil. Açıklaması şöyle.

The CAP theorem asserts that in any distributed data store only two out of three guarantees can be provided regarding consistency, availability, and partition tolerance.

BASE İle Olan İlişkisi

Açıklaması şöyle. BASE Properties For Distributed Database Transactions yazısına bakabilirsiniz.

ACID focuses on Consistency and availability.

BASE focuses on Partition tolerance and availability and throws consistency out the window.

1. Tanımlar

Önce tanımlara bakmak gerekir. Kısa bir açıklama şöyle. 

- Availability: All working members in a distributed system return a valid response to any request, without exceptions.

- Consistency: All the clients connected to the system see the same data at the same time, no matter which member they connect to; whenever data is written to a member, it is instantly replicated to all the other members in the system.

- Partition Tolerance: Partition refers to a lost or temporarily delayed connection between the members; partition tolerance refers to continued work of the cluster despite any number of communication breakdowns between the members of the cluster.

Availability mutlaka bir cevap alacağımızı belirtir, ancak aldığımız cevap en son bilgi olmayabilir.

Consistency mutlaka bir cevap alacağımızı belirtir, ayrıca eğer bir cevap aldıysak bunun mutlaka en son son bilgi olduğunu garanti eder

Partition Tolerance  üyeler arasındaki ağda koyma veya gecikme olsa bile her birinin çalışmaya devam edeceğini söyler.

2. CAP Teoremi Neyi Söyle

CAP bize şunu söyler. Aynı anda bu 3 özellikten sadece ikisini sağlayabiliriz. Bir tanesinden feragat etmek gerekir. Şeklen şöyle. Her kenarın kesişimi hangi özellikten feragat ettiğimizi gösterir.

Eğer sadece Consistency ve Availability istersek Partitioning-Intolerance seçiliyor. Bu da dağıtık yapı olmaması anlamına geliyor. Klasik dağıtık yapıda olmayan ACID veri tabanları böyle. Ama zaten konumuz bu değil. Bu yüzden Partition Tolerance özelliğinin mutlaka olacağını varsayıyoruz. Bu yüzden Consistency ve Availability arasında seçim yapmak gerekiyor.

2. Seçilen Özellikler

Şeklen şöyle

Bir başka şekil şöyle. İlk sütunda CA sistemler görülüyor. Yani dağıtık olmayan sistemler. Cp ve AP sistemler ikinci ve üçüncü sütunlarda

2.1 Partition Tolerance + Availability Özelliği Seçilirse

Availability seçilirse, Consistency 'den feragat etmek gerekir çünkü ağdaki kopmalar ve gecikmeler yüzünden veri her yerde aynı anda tek bir değere sahip olamaz.

Bu durumda 

1. BASE yani NoSQL veri tabalarından birisini seçmek gerekiyor. Örneğin Apache Cassandra seçilebilir. Apache Cassandra'nın available olmasının sebebi birden fazla master node olması. Açıklaması şöyle

Cassandra has multiple master nodes. To input data in the other nodes, these master nodes are used. Therefore, the more master nodes a cluster has, the better it will scale. After comparing the both, Cassandra has higher scalability than MongoDB.

Peki AP Özelliği Eventual Consistency Anlamına Gelir mi ?

Hayır gelmez. Çünkü Replication algoritmasının kopmaları dikkate alacak şekilde olması gerekir. Eğer Replication algoritması bunları dikkate almazsa, Eventual Consistency olmaz, Data Loss olur

2.2. Partition Tolerance + Consistency Özelliği Seçilirse

Consistency seçilirse, Availability 'den feragat etmek gerekir yani en son yazılan veriyi okuyamayabiliriz. 

Bu durumda 

1. Dağıtık ACID yani bir anlamda distributed relational (dağıtık ilişkisel) veri tabanlarından birisi. seçilebilir. Açıklaması şöyle

However, MySQL also has another configuration which is a clustered configuration. It prioritizes CP over availability eg. the cluster will shutdown if there are not enough live nodes to serve all the data.

 2. BASE yani NoSQL ancak Consistency özelliğine önem veren veri tabalarından birisi seçilebilir. Örneğin MongoDB seçilebilir. MongoDB'nin consistent olmasının sebebi tek bir master node olması. Açıklaması şöyle.

In MongoDB, there is only one master node. This master node only accepts the input. Apart from this, all the nodes are used as an output. Therefore, if the data has to be written in the slave nodes, it has to pass through the master node.

3. NoSQL ve Consistency İlişkisi

Açıklaması şöyle

The main distinction between database systems is not the language used to query the database, but rather the consistency model of the system. 

4. İlişkisel (Relational) Veri tabanı ve CAP İlişkisi

Açıklaması şöyle. Yani klasik bir ilişkisel veri tabanı olan PostgreSQL, Partition Tolerance özelliğini desteklemiyor.

- PostgreSQL is a CA system.

- Cassandra is an AP system

- Mongo DB is a CA system by default

Different databases have different goals. No matter what data type PostgreSQL offers, it will ultimately be a relational database and will sit within the CA part of the CAP theorem.

Açıklaması şöyle.

Transactions

ACID transactions by nature are typically hard to scale across multiple machines. This is the reason why replication in PostgreSQL or any relational database is done via a Write Ahead Log or WAL log.

This means a transaction is only sent across the wire after it is written to the WAL log, ensuring that there is consistency across different database instances. This is very different from Cassandra's consistency level (BASE) which scales across multiple nodes and uses something called eventual consistency.

Data Models

- Mongo DB is a document store

- Cassandra is a column family store

- PostgreSQL is a relational DB row store

At the end of the day, PostgreSQL is still a relational data model and does not have all the features of a NoSQL database such as an aggregation pipeline.

When you try to model data in PostgreSQL, the best practice is to default to a relational model and only use JSON when it makes sense.

Write Ahead Log

Açıklaması şöyle. Asenkron replication durumunda veri kaybı olabilir.

PostgreSQL Streaming Replication is asynchronous by default, so it is possible to have some transactions committed in the primary node which have not yet been replicated to the standby server. This means there is the possibility of some potential data loss.

This delay in the commit process is supposed to be very small... if the standby server is powerful enough to keep up with the load. If this small data loss risk is not acceptable in the company, you can also use synchronous replication instead of the default.

In synchronous replication, each commit of a write transaction will wait until the confirmation that the commit has been written to the write-ahead log on disk of both the primary and standby server.

This method minimizes the possibility of data loss. For data loss to occur you would need both the primary and the standby to fail at the same time.

The disadvantage of this method is the same for all synchronous methods as with this method the response time for each write transaction increases. This is due to the need to wait until all the confirmations that the transaction was committed. Luckily, read-only transactions will not be affected by this but; only the write transactions.

ACID - Read Committed Seviyesi Nedir ?

Giriş

Şeklen şöyle. İkinci satırda Repeatable Read görülebilir.

SQL Server için şeklen şöyle

PostgreSQL için açıklama şöyle. Yani PostgreSQL'in desteklediği en düşük ve ilk seviye bu

In PostgreSQL, you can request any of the four standard transaction isolation levels, but internally only three distinct isolation levels are implemented, i.e. PostgreSQL's Read Uncommitted mode behaves like Read Committed. This is because it is the only sensible way to map the standard isolation levels to PostgreSQL's multiversion concurrency control architecture.

Designing Data Intensive Applications kitabındaki açıklama şöyle.

The most basic level of transaction isolation is read committed. It makes two guarantees:

1. When reading from the database, you will only see data that has been committed

(no dirty reads).

2. When writing to the database, you will only overwrite data that has been committed

(no dirty writes).

Yani

- Sadece Dirty Read ve Dirty Write problemini engeller. 

- Non Repeatable Read problemini engellemez. Aynı transaction içinde iki okuma farklı sonuç verebilir, çünkü başka transaction veriyi değiştirebilir.

- Lost Update problemini engellemez

- Phantom Read problemini engellemez. Benim dokunmadığım satırlar kilitlenmediği için bunlar değiştirilebilir veya yeni satır eklenebilir

- Phantom satır kapsamında Write skew problemini engellemez.

Non-Repetable Read

Örnek

Şeklen şöyle. Açıklaması şöyle

In the following diagram, the concurrent transactions see the real-time change the other made. Transaction 1 gets chris as the latest committed name even though Transaction 2 began after Transaction 1.

Burada Transaction 1 başlıyor ve 2 numaralı satırı okuyor, ancak 3 numaralı satıra dokunmuyor. Transaction 2 ise 3 numaralı satırı değiştirip commit'liyor. Transaciton 2, 3 numaralı satırı okursa değişmiş veriyi görüyor. Bu problemin ismi "Non-Repeatable Read" ve "Repeatable Read" isolation seviyesi bu problemi engeller.

Select İşlemi - Non Repeatable Read Problemi

Read Committed seviyesinde select cümlesinin başında konulan shared lock, cümle bitiminde transaction bitmemiş olsa bile kaldırılır. Açıklaması şöyle

T1 gets Read Lock, reads, and releases the Read lock asap once the read is done.

T2 can execute and update the row as it is unlocked now.

So When T1 tries to re-read the same row, it will get a different result.

Örnek

Şöyledir

T1 begins

T1 Reads A‘s Balance=Rs. 1000 [Read-1]

T2 begins

T2 Read A’s Balance= Rs. 1000 (Can read as Row= Read Locked)

(T1 reads and releases Read Lock ASAP, and the row is unlocked now)

T2 Update A’s Balance= Rs. 500 (Can Write as Write lock is free for this row)

T2 Commit. (Releases Write Lock, now row is free to Read/Write)

T1 Reads A‘s Balance=Rs. 500 [Read-2] [Different Result] [non-Repeatable]

Select İşlemi - Phantom Reads Problem

Örnek

Şöyledir

T1 begins

T1Queries select * from Tbl where X>100 → 3 rows

T2 begins

T2 Inserts 1 additional row with X=150.

T2 Commits.

T1 Queries select * from Tbl where X>100 → 4rows [Phantom Read].

ACID - Phantom Read Nedir ? (Eksik veya fazla satır)

Giriş

Şeklen şöyle

Açıklaması şöyle

A phantom read occurs when two identical queries are executed during a transaction and the row collection returned by the second query is different from the first

Phantom Read 2 şekilde ortaya çıkar

1. Belli bir koşula bakılarak satır eklenmesi/çıkarılması

1. Belli bir koşula bakmadan satır eklenmesi/çıkarılması

1. Belli bir koşula bakılarak satır eklenmesi/çıkarılması

Açıklaması şöyle. Burada SELECT ile bir koşula veya sayaç değerine bakılıyor. Daha sonra satır ekleniyor/çıkartılıyor

Having said that, a phantom read would always follow the same pattern

1. Run a SELECT query to get data that matches certain conditions.

2. Depending on the result of first the transaction either proceeds or aborts.

3. If the transaction proceeds ahead then you make some INSERT, UPDATE, DELETE call and commits the transaction.

Açıklaması şöyle

This is a very common problem and can happen in these sort of systems

1. Booking system (Two people book the same room)

2. User management system (Two people get the same username)

3. Customer service system (One agent gets assigned multiple calls at the same time)

Örnek

Bir örnek şöyle

The item count is 1 in the database.

1. User 1 fetches the count and gets 1 in return.

2. User 2 fetches the count and gets 1 in return.

3. User 1 sees count > 0, assumes the item is available and places the order, and decrements the count by 1.

4. User 2 sees count > 0, assumes the item is available and places the order, and decrements the count by 1.

5. User 1 order placed.

6. User 2 order placed.

Açıklaması şöyle

So both the user was actually able to book the item even though only 1 item was available. ... In simple words, multiple transactions read the same object to decide if a transaction should go through and then might update that or other objects.

2. Belli bir koşula bakmadan satır eklenmesi/çıkarılması

Açıklaması şöyle

This can happen if T1 queries some range of rows (say N rows), and meanwhile, T2 inserts an extra row matching the same query conditions of T1.

Then if T1 searches again, it will get an additional row(phantom read).

Kısaca aynı transaction içinde aynı sorguyu iki defa çalıştırıp, bir başka transaction'ının satır ekleyip silmesi yüzünden, eksik veya fazla satır sayısının alınması anlamına geliyor. 

Burada dikkat edilmesi gereken şey her iki sorguda da mevcut olan satırların aynı değerlere sahip olmasının garanti edilmesi. Sadece aradaki satırlar eklenip çıkarılabiliyor. Aynı değere sahip olmasının garanti edilmesi Read Committed Seviyesinde Select başlığındaki SCN numarası ile alakalı.

- Eksik veya fazla satırın olması select sum(x) from table; gibi sql cümlelerinde probleme sebep olabilir.

- Eğer primary key kullanarak tek bir satır çekiyorsak phantom read olma olasılığı ortandan kalkar.

Örnek

Bir örnek burada

Çözüm

1. "Select For Update" Kullanılabilir. "Select For Update" dokunduğu her satırı kilitleyeceği için, arkadan gelen transaction beklemek zorunda kalır.

2. Serializable Isolation Level kullanılabilir.

DO-332 Object Oriented Technology and Related Techniques Supplement

Giriş

DO-178B Object Oriented geliştirme yönteminden çok daha önce yayınlandığı için isteseydi de OO konularına değinemezdi. Ancak günümüzde OO çok daha fazla kullanılıyor. DO-178C yazılım geliştirme açısından daha fazla Object Oriented yönelimli. Garbage Collection,Virtualization, Generics (template) kullanımı konularına değiniyor.

Bu belge 2011 yılında yayınlandı ve 150 sayfa civarında. 4 tane ANNEX ve 2 tane APPENDIX'ten oluşuyor. ANNEX ve APPENDIX 'in farkının açıklaması şöyle. Dolayısıyla ANNEX'leri tek başına okumak mümkün.

An appendix contains data that cannot be placed in the main document and has references in the original copy or file. An annex, on the other hand, is usually a standalone document that offers additional information than contained in the main document.

Bazı önemli başlılar şöyle

1.0 INTRODUCTION

  1.6 Characteristics of Object-oriented Technology and Related Techniques

  Bu başlık altında genel geçer OO kodlamayla ilgili şeyler var.

  1.6.1.2.1 Liskov Substitution Principle

  Burada kalıtan her sınıfın Liskov prensibine uyması gerektiği belirtiliyor.

2.0 SYSTEM ASPECTS RELATING TO SOFTWARE DEVELOPMENT

Burada yazan şey sadece şöyle. Yani DO-178C belgesini okuyun diyor.

Section 2 of DO-178C is unchanged

3.0 SOFTWARE LIFECYCLE

Burada yazan şey sadece şöyle. Yani DO-178C belgesini okuyun diyor.

Section 3 of DO-178C is unchanged

4.0 SOFTWARE PLANNING PROCESS

Açıklama yaz

5.0 SOFTWARE DEVELOPMENT PROCESS 

Açıklama yaz

6.0 SOFTWARE VERIFICATION PROCESS

Bu bölümdeki alt başlıkların çoğu DO-178C belgesine atıfta bulunuyor. 

  6.3 Software Reviews and Analysis

  Burada review ve analysis arasındaki farkı anlamak önemli. Açıklaması şöyle.

Reviews are qualitative. They typically compare the artifact being reviewed to a standard using a checklist.

Analyses are quantitative. They could be functional analysis, data flow analysis, timing analysis, or other evaluation of the software artifact.

  Belgedeki açıklama şöyle

6.3 Software Reviews and Analyses

Reviews and analyses are applied to the outputs of the software development processes. One distinction between reviews and analyses is that analyses provide repeatable evidence of correctness and reviews provide a qualitative assessment of correctness. A review may consist of an inspection of an output of a process guided by a checklist or similar aid. An analysis may examine in detail the functionality, performance, traceability, and safety implications of a software component, and its relationship to other components within the system or equipment.

7.0 SOFTWARE CONFIGURATION MANAGEMENT PROCESS

Burada yazan şey sadece şöyle. Yani DO-178C belgesini okuyun diyor.

Section 7 of DO-178C is unchanged

Burada bir süreç olduğunu ispatlamak gerekiyor. Açıklaması şöyle. Code Review'ların bir standarda dayandırılması, kodun static kod analizinden geçmesi ve bir issue tracking yazılımının kullanılması gibi şeyleri göstermek gerekiyor.

Configuration management, including change review and problem tracking. If you're following DO-178C you can't merge in an unreviewed code change or hide a bug report

8.0 SOFTWARE QUALITY ASSURANCE PROCESS

Açıklama yaz

9.0 CERTIFICATION LIAISON PROCESS

Açıklama yaz

10.0 OVERVIEW OF CERTIFICATION PROCESS

Açıklama yaz

11.0 SOFTWARE LIFECYCLE DATA

Açıklama yaz

12.0 ADDITIONAL CONSIDERATONS

Açıklama yaz

ANNEX A PROCESS OBJECTIVES AND OUTPUTS BY SOFTWARE LEVEL IN DO-178C

Açıklama yaz

ANNEX B ACRONYMS AND GLOSSARY TERMS

Açıklama yaz

ANNEX C PROCESS OBJECTIVES AND OUTPUTS BY ASSURANCE LEVEL IN DO-278A

Açıklama yaz

ANNEX D VULNERABILITY ANALYSIS

Burada OOT kullanırken çıkabilecek zaafiyetlerden bahsediliyor

D.1 Key Features and Related Techniques

  Burada kodlada çıkabilece beklenmedik yan etkilerden bahsediliyor.

  D.1.1 Inheritance

  D.1.2 Parametric Polymorphism

  D.1.3 Overloading

  D.1.4 Type Conversion

  D.1.5 Exception Management

  D.1.6 Dynamic Memory Management

  D.1.7 Virtualization

D.2 General Issues

  Burada özellikle analiz yöntemlerinden bahsediliyor.

APPENDIX A COMMITTEE MEMBERSHIP

Burada sadece komite üyelerinin isimleri yazıyor.

APPENDIX B FREQUENTLY ASKED QUESTIONS

Açıklama yaz

Apache Kafka API

Giriş 

Kafka API'si başlıklara ayrılmış. Başlıklar şöyle.

Producer API
Consumer API
Streams API
Connector API

API'lerin açıklaması şöyle

1) Producer API: It provides permission to the application to publish the stream of records.
2) Consumer API: This API is being used to subscribe to the topics.
3) Stream API: This Stream provides the result after converting the input stream into the output stream.
4) Connector API: This links the topics with existing applications.

Producer API

Kafka Producer API yazısına taşıdım

Consumer API
Açıklaması şöyle.

The Consumer API allows an application to subscribe to one or more topics and process the stream of records produced to them.

Java örneklerini Kafka Consumer API yazısına taşıdım

Subscribe ve Self Assign Farkı

Consumer API iki farklı şekilde kullanılabilir.

1. Subscribe Olarak

2. Self Assign'ı seçip partition 'a direkt gidip okuyarak 

Subscribe yönteminde Rebalance sorumluluğu Kafka üzerinde. 

Self Assign yönteminde Rebalance sorumluluğu uygulamanın üzerinde

Consumer Config Parametreleri

Consumer Config Parametreleri yazısına taşıdım.

Streams API

Kafka Streams API yazısına taşıdım