Machine Learning ve Overfitting - Öğrenme Yerine Ezber Yöntemini Kullanma

Overfitting Nedir?

Açıklaması şöyle

Overfitting, modelinizin öğrenme yerine ezber yöntemine geçmesine verilen isimdir. Aşırı öğrenme durumunu tespit etmek için en iyi yöntemlerden birisi; eğitim verisi dışında, ilk kez karşılaşacağı veriler ile modeli test etmektir. Eğitim verileri ile %90'ın üzerinde (hatta %100'e yakın) başarılı sonuç elde edebilirken harici veriler ile bu oran çok çok düşük olabilmektedir.

Overfitting problemini görsel olarak soran bir soru burada

Aşırı Öğrenmenin Önüne Nasıl Geçebiliriz?

Bazı önlemler burada

Önlem - Verileri Artırın

Açıklaması şöyle

If you're training a complex model with small amount of data, your model is very likely to overfit.

Örnek

Çözmek istediğimiz problem şu olsun

Imagine that you're building a model that predicts house price based on the floor area.

Elimizdeki veri şöyle olsun

area  price
30    100001
50    150002
80    200003

Problemin açıklaması şöyle

You train your model, then ask it to predict the price for a house of area=50, and it tells you that the price should be 150002. Is that impressively accurate? Not really. It's just memorizing the training data.
Overfitting is commonly detected through a large difference in performance between the training and test set. If you test on the training set, you're unable to detect overfitting.

Önlem - Aynı Veri Training ve Test İçin Kullanmayın

Bir örnek şöyle

As a teacher, you wouldn’t give your students an exam that’s got the exact same exercises you have provided as homework: you want to find out whether they (a) have actually understood the intuition behind the methods you taught them and (b) make sure they haven’t just memorized the homework exercises.

Machine Learning - Support Vector Machines (SVM) Yöntemi

Giriş

Bu yöntem popüler. Açıklaması şöyle

Support Vector Machines (SVM) are one of the most popular supervised learning methods in Machine Learning(ML). Many researchers have reported superior results compared with older ML techniques.

SVM can be applied on regression problems as well as classification problems, ....

SVM Linear Applications ve Nonlinear Applications için kullanılabilir.

Linear Regression elimizdeki noktalara bakarak bir doğrusal denklem yani çizgi denklemi bulmak demek

SVM Linear Applications

Açıklaması şöyle

A popular classifier for linear applications because SVM’s have yielded excellent generalization performance on many statistical problems with minimal prior knowledge and also when the dimension of the input space(features) is very high.

SVM Nonlinear Applications

Açıklaması şöyle

A popular classifier for linear applications because SVM’s have yielded excellent generalization performance on many statistical problems with minimal prior knowledge and also when the dimension of the input space(features) is very high.

Şeklen şöyle

Maximum margin hyperplane

Açıklaması şöyle

The objective is to find the line passing as far as possible from all points

Kernel Trick

Açıklaması şöyle

SVM uses a Kernel trick to transform to a higher nonlinear dimension where an optimal hyperplane can more easily be defined.

Kernel tipleri şöyle

- Linear Kernel

- Polynomial Kernel     
- RBF - Radial Basis Function Kernel

- Gaussian     kernel

- Hyperbolic     tangent kernel

Neural Networks vs SVMs

SVM Convex veri için daha uygun. Açıklaması şöyle

One important argument is SVM is convex but NN is generally not. Having a convex problem is desirable because we have more tools to solve it more reliable.

If we know our data, we can pick a better model to fit data better. For example, if we have some data like donut shape. Like this

Doğru kernel seçimi de önemli. Açıklaması şöyle

using SVM with right kernel is better than using NN and NN may overfit data in this case.

Neural Networks aslında SVN'den daha eski. Açıklaması şöyle.

Historically, neural networks are older than SVMs and SVMs were initially developed as a method of efficiently training the neural networks. So, when SVMs matured in 1990s, there was a reason why people switched from neural networks to SVMs. Later, as data sets grew larger and more complex, so that feature selection became a (even bigger) problem, while, at the same time, computational power rose, people switched back again.

This development already suggests that both have their strengths and weaknesses and that there is, as Haitao says, no free lunch.

Essentially, both methods do some kind of data transformation to "send" them into a higher dimensional space. What the kernel function does for the SVMs, the hidden layers do for neural networks. The last, output layer in the network also performs a linear separation of the so transformed data. So this is not the core difference.

Her iki yöntem de verinin boyutunu (dimension) artırır. Açıklaması şöyle.

As you can see below, a two-layer neural network, with 5 neurons in the hidden layer, can perfectly separate the two classes. The blue class can be fully enclosed in a pentagon (pale blue) area. Each neuron in the hidden layer determines a linear boundary---a side of the pentagon, producing, say, +1 when its input is a point on the "blue" side of the line and -1 otherwise (it could also produce 0, it doesn't really matter).

I have used different colours to highlight which neuron is responsible for which boundary. The output neuron (black) simply checks (performs a logical AND, which is again a linearly separable function) whether all hidden neurons give the same, "positive" answer. Observe that this last neuron has five inputs. I.e. its input is a 5-dimensional vector. So the hidden layers have transformed 2D data into 5D data.

Burada SVM'nin veri boyutu büyüdükçe boundary çizerken zorlandığı anlatılıyor

Notice, however, that the boundaries drawn by the neural network are somewhat arbitrary. You can shift and rotate them slightly without really affecting the result. How the network draws the boundary is somewhat random; it depends on the initialisation of the weights and on the order you present the training set to it. This is where SVMs differ: They are guaranteed to draw the boundary mid-way between the closest points of the two classes! It can be (has been) shown that this boundary is the optimal one. Finding the boundary is a convex (quadratic) optimisation problem for which fast algorithms exist. Also, the kernel trick has the computational advantage that it's usually much faster to compute a single non-linear function than to pass the vector through many hidden layers.

However, since SVMs never compute the boundary explicitly, but through the weighted sum of the kernel functions over the pairs of the input data, the computational effort scales quadratically with the data set size. For large data sets this quickly becomes impractical.

Also, when the data are high-dimensional (think of images, with millions of pixels) the SVMs might become overwhelmed by the curse of dimensionality: It becomes too easy to draw a good boundary on the training set, but which has poor generalisation properties. Convolutional neural networks, on the other hand, are capable of learning the relevant features from the data.

Yani temel tavsiye şöyle

In summary, my suggestion is to use SVMs for low-dimensional, small data sets and neural networks for high-dimensional large data sets.

Machine Learning - Reinforcement Learning - Learn To Perform The Most Optimal Action Based On Feedback

Giriş
Bazı notlarım şöyle. İlk bölüm için Machine Learning yazına bakabilirsiniz.

Reinforcement Learning Nedir

Açıklaması şöyle

Reinforcement Learning algorithms are complex and advanced where the model learns from its previous predictions and correctness.

Bir başka açıklama şöyle. Yani bildiğimiz bir çoçuğa öğretir gibi öğretmek gerekiyor.

Given the environment, the machine learns to perform the most optimal action based on feedback it gets by performing an action in a simulation or training environment. Some of the key aspects of reinforcement learning include environment, current state, action, future state, reward, etc.

Machine Learning - Unsupervised Learning - Output Values Are Not Present

Giriş

Bazı notlarım şöyle. İlk bölüm için Machine Learning yazına bakabilirsiniz.

Unsupervised Learning Nedir

Açıklaması şöyle

Where in Unsupervised Learning, the data is not labeled i.e. there are no clearly defined target variables (nature of email, amount of rainfall and nature of tumor are the target variables in the previous cases).

Cevaplar bilinmediği için girdi veri çeşitli kümelere ayrılmaya çalışılır. Açıklaması şöyle

These are problems where output values or labels ain't present. Clustering is one common type of unsupervised learning problems. The machine learns the clusters of data given the data set.

Algoritmalar Nelerdir

Çok kullanılan bazı algoritmalar şöyle

Clustering

Neural Networks

Anomaly Detection

Machine Learning - Supervised Learning - Actual Values Related To The Variable That Needs To Be Predicted Are Available

Giriş

Bazı notlarım şöyle. İlk bölüm için Machine Learning yazına bakabilirsiniz.

Supervised Learning Nedir?

Açıklaması şöyle. Bu yöntemde elimizdeki girdiye karşılık, bazı cevaplar (label deniyor) vardır. Bilgisayar bu ikisini kullanarak, daha önce görülmemiş bir veriye göre tahminde bulunur

Supervised Learning takes advantage of already known labels, like whether an email is reported spam or not, how much rainfall has occurred in the last 7 days, whether a lump in body is carcinogenic or not etc.

Açıklaması şöyle

Supervised Learning takes advantage of already known labels, like whether an email is reported spam or not, how much rainfall has occurred in the last 7 days, whether a lump in body is carcinogenic or not etc. Supervised learning is a high level categorization of ML problems which defines all challenges where we have at least some solved/labeled data. This is opposed to unsupervised learning (we don't know the solution) and reinforcement learning (data and labels are generated procedurally).

Algoritmalar Nelerdir

Çok kullanılan bazı algoritmalar şöyle

Naive Bayes Classifier

Support Vector Machine

Decision Tree

Random Forest

Regression

Classification

Regression ve Classification Farkı Nedir?

Açıklaması şöyle

Regression (Predict the numerical value given the data set)

Classification (Predict the class or the label of the dataset)

1.1 Decision tree

Örnek

Açıklaması şöyle

... below is a diagram representing a decision tree algorithm predicting if a passenger survived the Titanic. By looking at the figure, we see that the model is picking up on the "women and children first" policy that was enforced during the tragedy.

Şeklen şöyle

Random forests

Açıklaması şöyle

Decision trees by themselves generally don't perform very well, which is why there are ensemble methods such as random forests. These algorithms use many trees, each of which makes a prediction. They then all "vote" to get a final prediction.

1.2 Support Vector Machines (SVM) Yöntemi

Machine - Learning - Support Vector Machines yazısına taşıdım

1.2 Regression Yöntemi

Açıklaması şöyle

Finally regression is a specific mathematical algorithm which can help us achieve tasks and might be opposed to algorithms such as a Neural Net, Naive Bayes, etc.

1.4 Classification Yöntemi
Açıklaması şöyle

Machine learning classification is a type of supervised learning in which an algorithm maps a set of inputs to discrete output. 

Classification için sadece Machine Learning kullanmak şart değil. Açıklaması şöyle

Machine/statistical learning is one approach to classification, but not the only one. Simple rules created by humans are probably more common in computer programs than ones created by ML.

Benzer bir açıklama şöyle

Actually, classification methodology has been around in classical probability and statistics for the better part of a century, well before "machine learning" was a deal. ... Machine learning algorithms are simply attempts to approximate this historically well-known (at least by statisticians) optimal solution.

Machine Learning

Giriş
Machine Learning gelişimine tarihsel olarak şöyle bakılabilir.

1980'ler Business Rules
2000'ler Finite State Machines
2010'lar Behavior Trees
2020'ler Machine Learning

Machine Learning
Açıklaması şöyle

A subset of AI, ML, can automatically learn from past experiences and improve itself without any manual intervention. ML categorizes new pieces of data by analyzing how the old ones were processed.

Probabilistic Machine Learning

Algoritmalarda Probabilistic yaklaşım kullanmak bir şekilde daha iyi sonuç veriyor. Özellikle yetersiz veri varsa. Açıklaması şöyle. ML benim konum olmadığı için yorum yapamıyor, ama ileride kullanmak üzere not almak istedim

Ghahramani elaborates on these points in many great tutorials and in this non-specialist overview article from Nature (2015) on Probabilistic Machine Learning and Artificial Intelligence.

Ghahramani's article emphasizes that probabilistic methods are crucial whenever you don't have enough data. He explains (section 7) that nonparametric Bayesian models can expand to match datasets of any size with a potentially infinite number of parameters. And he notes that many datasets that may seem enormous (millions of training examples) are in fact large collections of small datasets, where probabilistic methods remain crucial to handle the uncertainties stemming from insufficient data. A similar thesis grounds Part III of the renowned book Deep Learning, where Ian Goodfellow, Yoshua Bengio, and Aaron Courville argue that "Deep Learning Research" must become probabilistic in order to become more data efficient.

Because probabilistic models effectively "know what they don't know", they can help prevent terrible decisions based on unfounded extrapolations from insufficient data. As the questions we ask and the models we build become increasingly complex, the risks of insufficient data rise. And as the decisions we base upon our ML models become increasingly high-stake, the dangers associated with models that are confidently wrong (unable to pull back and say "hey, wait, I've never really seen inputs like this before") increase as well. Since both of those trends seem irreversible--ML growing in both popularity and importance--I expect probabilistic methods to become more and more widespread over time. As long as our datasets remain small relative to the complexity of our questions and to the risks of giving bad answers, we should use probabilistic models that know their own limitations. The best probabilistic models have something analogous to our human capacity to recognize feelings of confusion and disorientation (registering huge or compounding uncertainties). They can effectively warn us when they are entering uncharted territory and thereby prevent us from making potentially catastrophic decisions when they are nearing or exceeding their limits.

Deep Learning

Deep Learning yazısına taşıdım

Machine Learning Algoritmaları

Machine Learning algoritmaları kabaca şu kategorilere ayrılabilir.

                                  Machine Learning Algorithms
                                              |
                                              |
---------------------------------------------------------------------------------
|                                    |                                          |
supervised learning         unsupervised learning           reinforcement learning
|                                            |
|--->Naive Bayes Classifier                  |--->Clustering
|--->Support Vector Machine                  |--->Neural Networks
|--->Decision Tree                           |--->Anomaly Detection
|--->Random Forest
|--->Regression
|--->Classification

Machine Learning Kullanacağımıza Nasıl Karar Veririz?

Bazı kriterler şöyle. Eğer kurallar yani cevaplar bilinmiyorsa, kurallar çok hızlı değişiyorsa, çok fazla veri varsa kullanılabilir.

Here are a few rules that you can use to classify a problem as a machine learning problem or otherwise:

- It is not easy to identify a finite set of rules based on which one can determine output related to numerical problems or classification problems.

- Although the finite set of rules can be identified, however, the fact that rules change very fast makes it difficult to deploy the solution changes in the production

- Whether the solution requires a large volume of data for testing/quality assurance (QA)

- Whether the solution improves with the improvement in a variety of data

Why Data Matters to Machine Learning

Açıklaması şöyle. Verinin kalitesi çok önemli

All machine learning relies on data. Generally speaking, the more data that you can provide your model, the better the model. Your ML model needs to have high-quality data, which must be related to the problem you aim to solve. So in addition to volume, data quality matters as well. Finding relationships within your data and exposing them in your model’s training data can greatly improve its predictability.

Put candidly, high-quality data creates high-quality training features, producing a high-quality model that can more accurately generalize unseen data. As a result, understanding and explaining what your ML model means, and its behavior, is much easier.

Overfitting Problemi

Overfitting Problemi yazısına taşıdım

1. Supervised Learning

Supervised Learning yazısına taşıdım

2. Unsupervised Learning
Unsupervised Learning yazısına taşıdım.

3. Reinforcement Learning

Reinforcement Learning yazısına taşıdım.

Cross Validation Nedir

Açıklaması şöyle

In simplified terms you can think about it like this: suppose you are preparing pupils for an exam. You have three sets exercises from previous exams: A, B, and C. But the exercises in the upcoming exam will be different. Nonetheless you want to test how well students will do on the unseen exam tests, when trained on similar tests from the past.

Here is how you can do it: you give exercises A and B to student one, and after he learns to solve them you test his ability on C. For another student you give exercises A and C and test how well she does on the remaining set B. And for the third student you give B and C and test on A.

This way the scores obtained on the unseen tests, by all students, will be the average score you can reasonably assume those students will get in the upcoming exam. However if instead you show all your exercise sets: A, B, and C, to a student - then how will you able to test how well is he or she prepared? If you give the student the exercise he or she saw in training then the student might answer it perfectly from memory alone.

Same with classification methods. If you show them all the data - what data will you use to check how well "trained" they are? A simple silly method that memorises everything would score 100% on such a testing strategy. But the same method might be completely lost when presented with an unseen data point.

Bir başka açıklama şöyle

Cross-validation is generally used to find parameters or model structure to ensure it works well on new, unseen data. If you train your model using all the data A, B and C without doing the cross-validation, you risk overfitting and ending up with a model that performs well during training, but doesn't generalize to new data.

By performing the training on two of the folds and testing its performance on the third, you can optimize for performance on the new data