The Machine Learning Process Explained

Introduction.

• In an earlier post of mine, I mentioned that the process of Machine Learning to create an Artificial Intelligence may be summarized in a simple 4-phase diagram.
• This article will seek to explore and explain the entire process to you, in an effective and efficient way.

The machine learning process simplified

1. The Data Preparation Stage.

• During this stage the data is prepared for the algorithm. Data needs to be carefully handled before sent into a Machine Learning Algorithm.
• In the same way that we require food to be cooked before consumption, an Algorithm requires its fuel (data) to be carefully prepared before being fed into it.
• Simply put, the nature and effort invested into perfecting the fuel (data) for the Algorithm will have a direct impact on the Computational Power/Predictive Power of the model and the quality of predictions. Because the data on which the model is trained determines how it will make predictions for its lifespan.
• During the Preparation stage of the data, one is required to select the feature that one would like to predict called the Target Vector, and set it aside from the features that will be used to predict, called the Features Matrix.
• Hence, the greater the quantity of data, the better the quality of predictions. This is because the Training process spans a longer duration, thereby impacting the level of insight gained during the Learning process.
• A good example of preparing the data would be: Deciding how to handle those rows in which a field has missing values. Do we delete those rows entirely? Do we fill the missing values with the average value in that specific column?

A few of the many data preparation activities

2. The Algorithm Selection Stage.

• The Algorithm refers to the particular methodology that you will select. The Algorithm is that component of the Machine Learning Framework which does the Learning.
• When the Algorithm is exposed to/applied to the dataset, it will analyse each feature and attempt to learn trends and patterns in the dataset.
• It is good to understand that the type of Algorithm that one selects for the task at hand, is dependent on the nature and type of data you are working with.
• There are several categories and types of Algorithms:
  • Some Categories include Regression, Classification, Clustering.
  • Some Types include Linear Regression, Logistic Regression, BIRCH Algorithm.
• We will explore a variety of Algorithms in detail at a later stage in this course.

Selection of a Machine Learning Algorithm requires deep insight and understanding into the data you are working with

3. The Model Compilation Stage.

• During this stage, there are essential training events that occur. You see, most, if not all, Algorithms can be represented as a mathematical expression.
• And a model is successfully compiled and completed when the Algorithm has successfully found the best, useful coefficients to use in the formula
• Think of a simple line graph: f(x) = mx + c
• During the model compilation stage the model will attempt to find the most suitable coefficients for m and c, by applying the algorithm to the data. Remember that the Algorithm does the learning. The model can be thought of as being the memory during the training process, which remembers the teachings and stores it for future use.
• As the model finds suitable coefficients, it gradually builds up, bigger, better, and possessing higher predictive power.
• Model Compilation is completed successfully, when you are able to use the model itself, to make predictions on test values.

A model is the output of the training process of a machine learning algorithm

4. The Prediction/Evaluation Stage.

• Before we conduct a prediction using test values with our model, we should proceed to evaluate the models predictive power.
• The evaluation of a model, gives us, the human being, insight into the accuracy and usefulness of a model. Because a model is only as useful as its worst prediction.
• Evaluation of a model is done using evaluation metrics. These are specific formulae that are applied to a model, to test and see how accurate the predictions are on data that is has never seen before.
• To increase our level of understanding of the 4-phase process, let us analyse a simple example.

Before utilizing it to make predictions, one has to first train the machine learning model

Conclusion.

• This article provided us with the advanced explanation of the Machine Learning Process.
• This process is what enables a machine to exhibit characteristics of an Artificial Intelligence.
• I do hope that you found this article enlightening and interesting.
• Thank you for your time.