Evolutionary feature selection for big data classification: a MapReduce approach
type: Item
instance of: scholarly article
title: Evolutionary feature selection for big data classification: a MapReduce approach
zbMATH Open document ID: 1395.62170
DOI: 10.1155/2015/246139
author: Y. Aharonov
published in: Mathematical Problems in Engineering
publication date: 2018-08-27T00:00:00Z
review text: Summary: Nowadays, many disciplines have to deal with big datasets that additionally involve a high number of features. Feature selection methods aim at eliminating noisy, redundant, or irrelevant features that may deteriorate the classification performance. However, traditional methods lack enough scalability to cope with datasets of millions of instances and extract successful results in a delimited time. This paper presents a feature selection algorithm based on evolutionary computation that uses the MapReduce paradigm to obtain subsets of features from big datasets. The algorithm decomposes the original dataset in blocks of instances to learn from them in the map phase; then, the reduce phase merges the obtained partial results into a final vector of feature weights, which allows a flexible application of the feature selection procedure using a threshold to determine the selected subset of features. The feature selection method is evaluated by using three well-known classifiers (SVM, Logistic Regression, and Naive Bayes) implemented within the Spark framework to address big data problems. In the experiments, datasets up to 67 millions of instances and up to 2000 attributes have been managed, showing that this is a suitable framework to perform evolutionary feature selection, improving both the classification accuracy and its runtime when dealing with big data problems.
Mathematics Subject Classification ID: 62H30
Mathematics Subject Classification ID: 62-07
zbMATH DE Number: 6925853
Wikidata QID: Q59117715
describes a project that uses: Hadoop
describes a project that uses: MPI
describes a project that uses: SparkSeq
describes a project that uses: MapReduce
MaRDI profile type: Publication
full work available at URL: https://doi.org/10.1155/2015/246139
OpenAlex ID: W1952835952
cites work: Q3655274
cites work: Data and task parallelism in ILP using mapreduce
cites work: Advances in instance selection for instance-based learning algorithms
cites work: 10.1162/153244303322753616
cites work: Computational Methods of Feature Selection
cites work: Nonlinear Dimensionality Reduction
cites work: Wrappers for feature subset selection
cites work: Q2934043
cites work: Nearest neighbor pattern classification
cites work: Q4403756
description: scientific article; zbMATH DE number 6925853