What is a training set?

Nowadays, when the concepts of artificial intelligence and machine learning come out of science-fiction novels and become a part of our lives, artificial intelligence still cannot search and learn information on its own under most conditions. In fact, it is quite similar to teaching a child the concepts of certain subjects. However, there are some topics you need to know before creating a training set. In this article, we shall look at what a training set is, what you should pay attention to when creating a training set and what makes a training set efficient.

What is a training set?

If you plan to build a custom machine learning model for text analysis, you need a training set as the first step. You usually need a sheet consisting of two columns to create a training set from the customer feedback you have collected from any corner of the internet. You can use Microsoft Excel or Google Sheets, which are the most popular options. If you wonder how you can provide data from any website, read our article on How to generate an NLP dataset from any internet source?

You should have the content you will analyze and the labels in those two columns. Label selection is an essential point for model building. The labels must contain the concepts that form the basis of the model you want to create. You can get help from this article on choosing as useful labels as possible: "Choosing Labels for Text Classification". Also, if you want to see a well-prepared training set an example, you can take a look at the datasets on Kimola's Github profile.

This is an example of a training set. You can have your own training set by following these steps:

1. Be careful when choosing your labels, and you should not use identically similar labels. Also, you should keep your label count minimal, between 5-8 labels at most. The more labels there are, the more you need data.
2. Each label should contain an equal amount of data; each label must contain at least 500 data.
3. Remember that the data sets must be saved as .xls files.
4. Create your custom machine learning model with your training set. You can read our article on How to Create A Custom Machine Learning Model before you upload your training set and create a custom machine learning model on Kimola Cognitive.
5. After creating your machine learning model with your training set on Kimola Cognitive, you can control the accuracy rate of the ML model. Read our article on How to Optimize Accuracy Rate?

TL;DR

A training set is a .xls file that has labelled text data. Training sets are used to create a custom machine, and learning models. After creating a custom machine learning model, you can analyze your text data by using that model.

The Quantity of Your Training Set

In order to create a training set from the data set you have collected on the subject you want to create a model, you need to consider and have an answer ready for these basic questions around the quantity of data:

• How many records need to be collected to create a great training set?

Remember, larger labelled data means better results. You can think of this as the more resources a person reads on a specific topic, the more knowledge they will have and the ability to analyze the patterns easily.

• How much data will your sample consist of?

While preparing your sample data, you must consider that the values included in your model are equally distributed. For instance, if your training set contains more positive data than negative data while preparing a sentiment classification model, the model you create will be more prone to classify your test data mostly as positive.

• Splitting the data for training and testing and deciding on the model validation method, such as cross-validation.

After you create your machine learning model, you must test it. Your test dataset should be realistically prepared. The data set you want to be classified must be well-shuffled, and the different data types must be equally distributed.

The machine learning system is only as certain as its trainer.

The human mind tends to perceive a sentence based on different social and cultural backgrounds. Our judgment and (let's say) classification of the sentences we hear even daily are based on our biases. Therefore, while creating a machine learning model, artificial intelligence deciding the data it analyzes and labelling is also associated with the labels of the person who trained the AI.

In these terms, you need to train the person(s) labelling to think as bias-free as possible and to help them label the data as open-minded as possible. Thus, some of the inconsistencies can be eliminated.

However, since each data is unique, there may still be some data that even the trainer is uncertain about. In such cases, we should not expect the machine learning model to label data that even a human cannot label. But we have good news for you! A machine learning model never has to have a 100% accuracy rate and this is not possible either. A machine learning model can be considered successful only if it is as successful as a human being, meaning at least an 85% accuracy rate is enough.