Machine learning as a field has been around for a long time before deep neural networks took over the scene. Here is a list of the most essential machine learning algorithms you need to know, so you can tackle any problem that comes your way. This isn’t an exhaustive list, but your bases will be mostly covered.
Also wanted to announce that my medium blog will be transitioning from a general ML focus to a deep learning focus. Most of my work nowadays involves creating novel deep learning systems, so I want to spend more time writing about it!
1. Regression Algorithms
Regression algorithms model relationships between variables. Originally a technique from statistics they have become an important tool in every Machine learning engineer’s tool kit.
Common Regression Algorithms
- Least Squares Regression
- Linear Regression
- Logistic Regression
Coursera Course by Johns Hopkins on regression models
2. Clustering Algorithms
Clustering algorithms can divide data points into groups with similar properties. They work by finding inherent structures in data to best organize data into distinct groups. Things in the group are more closely related to each other than things in other groups.
There are two types of clustering algorithms. hard clustering refers to when a data point is in a group or not. soft clustering refers to when a data point can belong to many different groups to different degrees.
Common Clustering algorithms
- Hierarchical Clustering
An amazing introductory video on clustering
3. Dimensionality reduction algorithms
When the number of features is very large compared to the number of data points you have. Dimensional reduction algorithms help you reduce the number of features to only what is necessary for the problem at hand. They can remove redundant or useless features, helping you get better results.
There are two ways that dimensional reduction algorithms work. The first method is through feature selection, where the algorithm picks a subset of the available features. The second way is feature extraction, which reduces the data in a high dimensional space to a lower one.
Common Dimensionality Reduction Algorithms
- Principle component analysis
- Low Variance Filter
- High Correlation Filter
- Random Forests
- Backward Feature Elimination / Forward Feature construction
This is not an exhaustive list, just some that I have used. If you want to read up on this some more as well as see the ROI for some of these algorithms check out the KDnuggets blog post on it.
4. Decision Tree Algorithms
Decision trees create models of decisions made on values from your data. A fork is made in the tree structure until there is a prediction for every data point. Their results are easy to understand unlike other algorithms (Deep Learning) and they are easy to use on many different data types.
Common Decision Tree Algorithms
- Classification and Regression Tree
- C4.5 and C5.0
- Random Forests
- Chi-squared automatic interaction detector
Analytics Vidhya has a great article that goes in-depth on decision trees. Listing out the different algorithms and their advantages and disadvantages
The hype behind machine learning and “AI” is caused by deep learning. They are modern versions of artificial neural networks that exploit cheap computation to train ever larger neural networks. They are powerful universal function approximates that have proven their ability in solving some of the hardest problems. See Alpha Go.
Common Deep Learning Algorithms
- Stacked Auto-encoders
- Convolution Neural networks
- Recurrent neural networks
- Capsule Networks (more information here)
Check out this book snippet. It goes over the major architectures for deep learning.
If you are serious about machine learning you have to understand the tools that are available to you. Having a good understanding of these tools will give you a leg up on any problems you come across.