Categorical to Numeric problem (2024)

12 views (last 30 days)

Show older comments

Stephen Gray on 8 Jan 2024

  • Link

    Direct link to this question

    https://support.mathworks.com/matlabcentral/answers/2067721-categorical-to-numeric-problem

  • Link

    Direct link to this question

    https://support.mathworks.com/matlabcentral/answers/2067721-categorical-to-numeric-problem

Commented: Cris LaPierre on 11 Jan 2024

Accepted Answer: Hassaan

Hi

I have a table that has numeric and categorical items in it. I have converted the catergorical items to numeric using the unique() function which works very well and I can then feed the matrix into an NN for training. The problem is when I feed new data to get results, I don't know how to make sure the converted categirical data in the new table matches ther numbers in the training data. i.e. if a categorical field in the training data is converted to the number 5, how do I make sure if that categorical data is in the new data, that it gets assigned the same number? I'm begining to think it may be a manual thing

SPG

0 Comments

Show -2 older commentsHide -2 older comments

Sign in to comment.

Sign in to answer this question.

Accepted Answer

Hassaan on 8 Jan 2024

  • Link

    Direct link to this answer

    https://support.mathworks.com/matlabcentral/answers/2067721-categorical-to-numeric-problem#answer_1385666

  • Link

    Direct link to this answer

    https://support.mathworks.com/matlabcentral/answers/2067721-categorical-to-numeric-problem#answer_1385666

Open in MATLAB Online

% Example Training Data (Categorical)

training_categorical_data = {'cat', 'dog', 'fish', 'dog', 'cat'};

% Convert Categorical Data to Numeric for Training

[unique_categories, ~, numeric_categories] = unique(training_categorical_data);

category_to_number_map = containers.Map(unique_categories, num2cell(1:length(unique_categories)));

numeric_training_data = cell2mat(values(category_to_number_map, num2cell(training_categorical_data)));

% Training Process with numeric_training_data

% [Your neural network training code goes here]

% Example New Data (Categorical)

new_categorical_data = {'dog', 'cat', 'bird'};

% Convert New Categorical Data to Numeric Using Training Mapping

numeric_new_data = zeros(size(new_categorical_data));

for i = 1:length(new_categorical_data)

if isKey(category_to_number_map, new_categorical_data{i})

numeric_new_data(i) = category_to_number_map(new_categorical_data{i});

else

% Handle unseen categories, e.g., assign a special number or ignore

numeric_new_data(i) = NaN; % Assign NaN for unseen categories

end

end

% Now, numeric_new_data is ready for use with the trained model

% [Your prediction or evaluation code goes here]

  • The training data training_categorical_data is a cell array of categorical strings. This is converted to numeric_training_data using a mapping (category_to_number_map).
  • The new data new_categorical_data is then converted using the same mapping. Unseen categories (like 'bird' in this example) are handled separately; here, I've assigned NaN to them, but you can choose another method as appropriate.
  • You'll need to insert your specific neural network training and prediction code where indicated. The numeric_training_data and numeric_new_data arrays are what you'd use for training and prediction, respectively.

------------------------------------------------------------------------------------------------------------------------------------------------

If you find the solution helpful and it resolves your issue, it would be greatly appreciated if you could accept the answer. Also, leaving an upvote and a comment are also wonderful ways to provide feedback.

Professional Interests

  • Technical Services and Consulting
  • Embedded Systems | Firmware Developement | Simulations
  • Electrical and Electronics Engineering
4 Comments

Show 2 older commentsHide 2 older comments

Stephen Gray on 8 Jan 2024

Direct link to this comment

https://support.mathworks.com/matlabcentral/answers/2067721-categorical-to-numeric-problem#comment_3022166

  • Link

    Direct link to this comment

    https://support.mathworks.com/matlabcentral/answers/2067721-categorical-to-numeric-problem#comment_3022166

I shall give it a go thanks. It looks really good.

Stephen Gray on 9 Jan 2024

Direct link to this comment

https://support.mathworks.com/matlabcentral/answers/2067721-categorical-to-numeric-problem#comment_3023401

  • Link

    Direct link to this comment

    https://support.mathworks.com/matlabcentral/answers/2067721-categorical-to-numeric-problem#comment_3023401

Open in MATLAB Online

WHen I run your code as a test, I get

Error using containers.Map/values

Specified key type does not match the type expected for this container.

Error in untitled1 (line 7)

numeric_training_data = cell2mat(values(category_to_number_map, num2cell(training_categorical_data)));

SPG

Stephen Gray on 10 Jan 2024

Direct link to this comment

https://support.mathworks.com/matlabcentral/answers/2067721-categorical-to-numeric-problem#comment_3024381

  • Link

    Direct link to this comment

    https://support.mathworks.com/matlabcentral/answers/2067721-categorical-to-numeric-problem#comment_3024381

OK, using dictionary instead and it's working so far.

Stephen Gray on 11 Jan 2024

Direct link to this comment

https://support.mathworks.com/matlabcentral/answers/2067721-categorical-to-numeric-problem#comment_3025446

  • Link

    Direct link to this comment

    https://support.mathworks.com/matlabcentral/answers/2067721-categorical-to-numeric-problem#comment_3025446

OK. I've got it to work now using dictionaries. Both this answer and the next one helped me get it working. AS yours includes how to use new data to I'll mark it as the answer. Thanks both for answering.

Sign in to comment.

More Answers (1)

Cris LaPierre on 8 Jan 2024

  • Link

    Direct link to this answer

    https://support.mathworks.com/matlabcentral/answers/2067721-categorical-to-numeric-problem#answer_1385676

  • Link

    Direct link to this answer

    https://support.mathworks.com/matlabcentral/answers/2067721-categorical-to-numeric-problem#answer_1385676

Moved: Cris LaPierre on 8 Jan 2024

Open in MATLAB Online

Could you provide more details about your NN? I would think you should be able to pass categorical data into your network without having to convert it to numeric first.

If not, then I'd look into creating a dictionary, where you pass in the categorical value, and it returns the numberic value.

A = categorical({'medium' 'large' 'small' 'medium' 'large' 'small'});

names = unique(A)

names = 1×3 categorical array

large medium small

values = (1:length(names));

d = dictionary(names,values)

d = dictionary (categorical --> double) with 3 entries: large --> 1 medium --> 2 small --> 3

A(4)

ans = categorical

medium

x = d(A(4))

x = 2

4 Comments

Show 2 older commentsHide 2 older comments

Stephen Gray on 8 Jan 2024

Direct link to this comment

https://support.mathworks.com/matlabcentral/answers/2067721-categorical-to-numeric-problem#comment_3022161

  • Link

    Direct link to this comment

    https://support.mathworks.com/matlabcentral/answers/2067721-categorical-to-numeric-problem#comment_3022161

It's a feedforwardnet type and you need to feed a matrix into it and not a table unfotunately but your idea looks good thanks.

Cris LaPierre on 9 Jan 2024

Direct link to this comment

https://support.mathworks.com/matlabcentral/answers/2067721-categorical-to-numeric-problem#comment_3023206

  • Link

    Direct link to this comment

    https://support.mathworks.com/matlabcentral/answers/2067721-categorical-to-numeric-problem#comment_3023206

They also accept cell arrays. What happens if you use table2cell to convert your input table to a cell array? Does it work then?

Stephen Gray on 9 Jan 2024

Direct link to this comment

https://support.mathworks.com/matlabcentral/answers/2067721-categorical-to-numeric-problem#comment_3023276

  • Link

    Direct link to this comment

    https://support.mathworks.com/matlabcentral/answers/2067721-categorical-to-numeric-problem#comment_3023276

Open in MATLAB Online

Unfortunately not. The code part is

InpsM = table2cell(Inps);

OutsM =table2cell(Outs);

InpsM=InpsM';

OutsM=OutsM';

net=feedforwardnet([96,48,24]);

net.trainFcn = 'trainlm';

net.inputs{1}.processFcns = {'mapstd'};

net=train(net,InpsM,OutsM,'useParallel','yes');

The error I get is

Error using nntraining.setup>setupPerWorker

Inputs X{1,1} is not numeric or logical.

Error in nntraining.setup (line 77)

[net,data,tr,err] = setupPerWorker(net,trainFcn,X,Xi,Ai,T,EW,enableConfigure);

Error in network/train (line 336)

[net,data,tr,err] = nntraining.setup(net,net.trainFcn,X,Xi,Ai,T,EW,enableConfigure,isComposite);

Error in untitled (line 52)

net=train(net,InpsM,OutsM,'useParallel','yes');

SPG

Cris LaPierre on 11 Jan 2024

Direct link to this comment

https://support.mathworks.com/matlabcentral/answers/2067721-categorical-to-numeric-problem#comment_3025881

  • Link

    Direct link to this comment

    https://support.mathworks.com/matlabcentral/answers/2067721-categorical-to-numeric-problem#comment_3025881

Found this, albeit on the trainnetwork page and not train, but it appears to still be applicable.

"To train a network using categorical features, you must first convert the categorical features to numeric."

Sign in to comment.

Sign in to answer this question.

See Also

Categories

AI, Data Science, and StatisticsDeep Learning ToolboxSequence and Numeric Feature Data Workflows

Find more on Sequence and Numeric Feature Data Workflows in Help Center and File Exchange

Tags

  • data
  • categorical
  • unique

Products

  • MATLAB

Release

R2023b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

An Error Occurred

Unable to complete the action because of changes made to the page. Reload the page to see its updated state.


Categorical to Numeric problem (12)

Select a Web Site

Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .

You can also select a web site from the following list

Americas

  • América Latina (Español)
  • Canada (English)
  • United States (English)

Europe

  • Belgium (English)
  • Denmark (English)
  • Deutschland (Deutsch)
  • España (Español)
  • Finland (English)
  • France (Français)
  • Ireland (English)
  • Italia (Italiano)
  • Luxembourg (English)
  • Netherlands (English)
  • Norway (English)
  • Österreich (Deutsch)
  • Portugal (English)
  • Sweden (English)
  • Switzerland
    • Deutsch
    • English
    • Français
  • United Kingdom(English)

Asia Pacific

  • Australia (English)
  • India (English)
  • New Zealand (English)
  • 中国
  • 日本Japanese (日本語)
  • 한국Korean (한국어)

Contact your local office

Categorical to Numeric problem (2024)

FAQs

How do you convert categorical data to numeric? ›

One common method is to assign labels based on the alphabetical order of categories, though the labels could also be assigned randomly or based on the order of appearance in the data. Once these assignments are determined, the categorical values in the dataset are replaced with their corresponding numerical labels.

Which method is used to convert categorical to numeric attributes? ›

Target encoding is a technique used in machine learning and data preprocessing to transform categorical variables into numerical values.

What is the process of transforming categorical variables into numerical values called? ›

Categorical encoding is the process of transforming a categorical column into one (or more) numeric column(s). This is necessary because computers are more at ease working with numbers than with strings.

Which are the ways to replace the categorical data to the numerical ones? ›

Converting Categorical Data to Numerical Data
  1. Method 1: Using the cat.codes Attribute. The easiest way to convert categorical data to numerical data in Pandas is to use the cat. ...
  2. Method 2: Using the replace() Method. ...
  3. Method 3: Using the LabelEncoder Class.
Jun 19, 2023

How do you convert categorical features to numerical features? ›

The method of transforming categorical features to numerical generally includes the following stages:
  1. Permutating the set of input objects in a random order.
  2. Converting the label value from a floating point to an integer. ...
  3. Transforming categorical features to numerical features.

Can a categorical variable be numeric? ›

Although categorical data is qualitative, it can also be calculated in numerical values. However, these possible values don't have quantitative qualities—meaning you can't calculate anything from them. Categorical data may also be classified as binary and nonbinary depending on its nature.

How do you convert character to numeric data type? ›

Using as.

numeric() function in R is your go-to for converting character data into numeric form. This transformation is crucial for subsequent data analysis tasks that require numerical input.

How can I convert categorical data into numeric by Excel? ›

How to Convert Categorical Data to Numeric in Excel
  1. Step 1: Enter the Data. First, enter the data values into Excel: ...
  2. Step 2: Use the IFS Function to Convert Categorical Values to Numeric Values. ...
  3. Step 3: Drag the Formula Down to All Cells.
May 12, 2022

How to deal with categorical data? ›

  1. Step 1: Drop columns with categorical data. You'll get started with the most straightforward approach. ...
  2. Step 2: Label encoding. Before jumping into label encoding, we'll investigate the dataset. ...
  3. Step 3: Investigating cardinality. ...
  4. Step 4: One-hot encoding.

What is the primary objective of transforming categorical variables into numerical? ›

But to make it more simpler to predict we convert that categorical variable to numeric like suppose we have 'yes' and 'no' 2 classes then we will convert 'yes' to 1 and 'no' to 0 . There's no such reason but just to make it easier to predict.

What is the process of converting categorical variables into numerical values called in R? ›

Target Encoding: Target Encoding is a process of converting categorical variables into numerical variables by replacing each category with the mean of the target variable for that category. You can use the targetMean() function from the cattonum package to create target encoding.

Why converting categorical data to numerical data? ›

However, most ML algorithms cannot process categorical data directly, because they are designed to work with numbers. For example, you cannot perform arithmetic operations or calculate distances on categorical data. Therefore, you need to convert categorical data to numerical data before feeding it to your ML model.

How do you convert numeric to categorical? ›

To associate numeric value 62.5 to a categorical value, the discretizer determines which of the three mean values is closest to 62.5 (which is 61.0) and then assigns the cluster value associated with 61.0 (which is 0) as the category value for 62.5.

How to convert categorical data to numerical data using one hot encoding? ›

One-hot encoding is a technique in machine learning that turns categorical data, like colors (red, green, blue), into numerical data for machines to understand. It creates new binary columns for each category, with a 1 marking the presence of that category and 0s elsewhere.

How to convert categorical data into numeric in Excel? ›

How to Convert Categorical Data to Numeric in Excel
  1. Step 1: Enter the Data. First, enter the data values into Excel: ...
  2. Step 2: Use the IFS Function to Convert Categorical Values to Numeric Values. ...
  3. Step 3: Drag the Formula Down to All Cells.
May 12, 2022

Which encoding method is used to convert categorical data into numerical data? ›

There are other methods to convert categorical data to numerical data, such as binary encoding, frequency encoding, target encoding, or embedding. Each method has its own advantages and disadvantages, and you should choose the one that best suits your data and your ML model.

How to convert multiple categorical columns to numerical data? ›

One of the methods to create dummy variables involves following steps: 1) creating dummy variables for each of the columns, 2) concatenate the new columns to the main data frame, 3) drop corresponding categorical columns. Above code is dropping first dummy variable columns to avoid dummy variable trap.

Top Articles
Latest Posts
Article information

Author: Carmelo Roob

Last Updated:

Views: 5915

Rating: 4.4 / 5 (65 voted)

Reviews: 88% of readers found this page helpful

Author information

Name: Carmelo Roob

Birthday: 1995-01-09

Address: Apt. 915 481 Sipes Cliff, New Gonzalobury, CO 80176

Phone: +6773780339780

Job: Sales Executive

Hobby: Gaming, Jogging, Rugby, Video gaming, Handball, Ice skating, Web surfing

Introduction: My name is Carmelo Roob, I am a modern, handsome, delightful, comfortable, attractive, vast, good person who loves writing and wants to share my knowledge and understanding with you.