A summary of categorical data encoding for ML modelling

By Dr Mabrouka Abuhmida

Categorical data encoding techniques are important because they allow us to convert categorical data, which is data that can be divided into categories, into numerical data that can be used in machine learning algorithms. There are several different techniques for encoding categorical data, including:

Ordinal encoding: Ordinal encoding assigns a numerical value to each category, based on the ordinal relationship between the categories. For example, if we have a categorical column with three categories: “low”, “medium”, and “high”, we could assign the values 1, 2, and 3 to each category, respectively. This can be done using the pandas map() function or the OrdinalEncoder in scikit-learn.

Target encoding: Target encoding assigns a numerical value to each category based on the mean value of the target variable for that category. This is useful when the categories have a clear relationship with the target variable. For example, if we have a categorical column with three categories: “red”, “green”, and “blue”, and the mean value of the target variable for each category is 10, 20, and 30, respectively, we would assign the values 10, 20, and 30 to each category. This can be done using the pandas groupby() and mean() functions, or the TargetEncoder in scikit-learn.

These encoding techniques are important because they allow us to use categorical data in machine learning algorithms, which typically only work with numerical data. By converting the categorical data into numerical data, we can build more accurate models and make better predictions.

The effectiveness of each encoding technique will depend on the specific characteristics of the dataset and the goals of the model. It is important to carefully evaluate the different encoding techniques and choose the one that is most appropriate for the given task.

One-hot encoding is more effective for supervised learning because it creates a separate column for each category, which allows the model to more easily learn the relationships between the categories and the target variable. This is especially useful when the categories are independent of each other, as it allows the model to differentiate between them more easily.

On the other hand, target encoding is more effective for unsupervised learning because it assigns a numerical value to each category based on the mean value of the target variable for that category. This is useful when the categories have a clear relationship with the target variable, as it allows the model to learn this relationship and make better predictions.

There are several ways to measure the impact of a categorical data encoding technique:

Evaluation metrics: One way to measure the impact of an encoding technique is to compare the performance of the model using different encoding techniques. This can be done by training and evaluating the model using different encoding techniques and comparing the evaluation metrics, such as accuracy, precision, or recall.

Correlation analysis: Another way to measure the impact of an encoding technique is to perform a correlation analysis between the encoded columns and the target variable. This can be done using tools such as the pandas corr() function or the Spearman’s rank correlation coefficient. This can help identify which encoded columns have a strong relationship with the target variable and how they contribute to the overall prediction.

There are a few more techniques that are briefly mentioned below:

Binary encoding: Binary encoding assigns a binary code to each category, based on the ordinal relationship between the categories. This is similar to ordinal encoding, but the numerical values are represented in a binary format rather than as integers.

Hashing encoding: Hashing encoding assigns a numerical value to each category using a hash function. This can be useful when there are a large number of categories and one-hot encoding would result in a very large number of columns.

Base-n encoding: Base-n encoding assigns a numerical value to each category using a base-n numbering system. This can be useful when there are a large number of categories and one-hot encoding would result in a very large number of columns.

Weight of evidence encoding: Weight of evidence encoding assigns a numerical value to each category based on the ratio of the probability of the target variable occurring in that category to the probability of the target variable occurring in the entire dataset. This is useful when the categories have a clear relationship with the target variable.

Frequency encoding: Frequency encoding assigns a numerical value to each category based on the frequency of that category in the dataset. This is similar to count encoding, but the values are normalized to a range between 0 and 1.

It is important to carefully evaluate the impact of the encoding technique on the model’s performance and the relationships between the encoded columns and the target variable.

This can help identify the most effective encoding technique for the given task.

The following table summarises a compassion summary encoding techniques

Related posts:

A wise man once told me that the best way to invest for the future is to spend less. This is so true and it’s something that I thought about when I gave up coffee before Christmas. I wondered how…

Muna farin cikin sanar da ƙaddamar da zagaye na 2 na testnets ɗin mu, tare da haifar da sabon zamani na karkatattun tsabar kudi. Domin murnar ƙaddamar da Testnet Round 2 kuma mu gode wa membobin…

Hyperparameter optimization is the process of finding the best set of hyperparameters for a machine learning model. These hyperparameters are settings that can be adjusted to improve the performance…