Machine Learning Basics

Machine learning is a process to building models from data and using these models to make predictions. Most widely used machine learning methods logistic regression, support vector machines, kernel methods, principal component analysis, random forests, neural nets, and so on.

Logistic Regression

Support Vector Machines

                                   

Neural Nets
 


Kernel Methods
Principal Component Analysis

 Applications of these methods to problems in image understanding, personality testing, sentiment analysis, language processing, and many other domains. There are basically three types of math that we will need: probability, linear algebra, and optimization.First, a comfortable familiarity with the most widely-used machine learning methods. Before starting machine learning we are try to know answer of following question.

• What are they?
• How do they work?
• Why do they work?
• What kinds of data are they good for?
• What are their strengths and weaknesses?
• What is going on under the hood?

                It will also help you understand and interpret the results that you're getting. The second skill that you'll get from this course is the ability to take existing methods and adapt them for a particular application.
                Introduce some basic machine learning terminology like training set, test set, training error, and test error.

• Training set: Which you run your learning algorithm on..

• Test set : which you use to evaluate the performance of the algorithm, but not to make any decisions regarding what learning algorithm or parameters to use.

We're given a picture or image of a handwritten digit and we want to say what digit it is.  This, for example, is a three. Here are some more examples: we have a zero, a one, a two, and so on and so forth. So how might we approach this problem? Here's an idea. A zero has got one loop in it. An eight has got two loops. A one has a single straight line. A four has three straight lines, and so on. What if we had a piece of software that took an image and figured out how many loops it had and how many straight lines, and also the relative positions of these loops and straight lines.
Maybe we could then use this information and write down a bunch of simple rules to decide what digit it is.  Actually, people tried this a long time ago and they ran into a lot of problems.
                So first of all, handwritten digits are super noisy and so it's hard to robustly pull out this information about the loops and lines and so on. Then, there's a lot of variability in the way people write fours, sevens, nines, etc. What this meant was that a huge number of rules were needed               to account for all the different special cases and then, after all this trouble, the systems didn't really work well at all.

So let's look into that. So first off, we will represent images as vectors, okay? Now an MNIST image is 28 pixels by 28. So it's 28 pixels across, 28 pixels high. That means the total number of pixels is 28 and 28 which is 784.  And each pixel is grayscale.

So it's a value in the range zero to 255    where zero means black and 255 means white.
                So we've taken the image and converted it into a 784 dimensional vector. Our data space then, which we're have denote by script X   is 784 dimensional Euclidean space and we'll often write it like this: R to the 784th.

Well, if you look at these two points, X and Z, along the first coordinate, they defer by two and along the second coordinate, they defer by three. So the length of the line, the distance from X to Z is simply the square root of two squared plus three squared which is the square root of 13. That's the Euclidean distance between X and Z in two dimensions, okay?
Now of course we aren't working in two dimensions. We're working in a much higher dimensional space but the basic idea is the same. When you want to compute the distance    between two vectors, X and Z, you simply find out how much they differ on each individual coordinate, you square these values, you add them up and then you take the square root of the whole thing. That's Euclidean distance.

So now we have a representation of the images as vectors in 784 dimensional space and we have a distance function between images. So we're ready to use nearest neighbor. Each time we get a new image, we simply find its nearest neighbor using Euclidean distance in 784 dimensional space and we return the label of this training image.

How to improve Learning model :

• ·         More hidden layers or hidden units are required

• ·         Data needs to have more diversity

• ·         More data may be required

• ·         Algorithm needs longer training

• ·         Add Regularization

• ·         Change the Neural network architecture like activation function etc.

Bias:  The algorithm’s error rate on the training set is  bias.
Variance : How much worse the algorithm does on the dev (or test) set than the training set is variance .

More Details