Machine learning with squared-loss mutual information
type: Item
instance of: scholarly article
title: Machine learning with squared-loss mutual information
zbMATH Open document ID: 1371.68241
author: Masashi Sugiyama
published in: Entropy
publication date: 2014-09-19T00:00:00Z
review text: Summary: Mutual information (MI) is useful for detecting statistical independence between random variables, and it has been successfully applied to solving various machine learning problems. Recently, an alternative to MI called \textit{squared-loss} MI (SMI) was introduced. While ordinary MI is the Kullback-Leibler divergence from the joint distribution to the product of the marginal distributions, SMI is its Pearson divergence variant. Because both the divergences belong to the \(f\)-divergence family, they share similar theoretical properties. However, a notable advantage of SMI is that it can be approximated from data in a computationally more efficient and numerically more stable way than ordinary MI. In this article, we review recent development in SMI approximation based on direct density-ratio estimation and SMI-based machine learning techniques such as independence testing, dimensionality reduction, canonical dependency analysis, independent component analysis, object matching, clustering, and causal inference.
Mathematics Subject Classification ID: 68T05
Mathematics Subject Classification ID: 62B10
Mathematics Subject Classification ID: 62H25
Mathematics Subject Classification ID: 62H30
zbMATH DE Number: 6346066
zbMATH Keywords: squared-loss mutual information
zbMATH Keywords: Pearson divergence
zbMATH Keywords: density-ratio estimation
zbMATH Keywords: independence testing
zbMATH Keywords: dimensionality reduction
zbMATH Keywords: independent component analysis
zbMATH Keywords: object matching
zbMATH Keywords: clustering
zbMATH Keywords: causal inference
zbMATH Keywords: machine learning
describes a project that uses: DIFFRAC
MaRDI profile type: Publication
full work available at URL: https://doi.org/10.3390/e15010080
OpenAlex ID: W2008901952
cites work: On Information and Sufficiency
cites work: Independent coordinates for strange attractors from mutual information
cites work: Estimation of the information by an adaptive partitioning of the observation space
cites work: Divergence Estimation of Continuous Distributions Based on Data-Dependent Partitions
cites work: Edgeworth Approximation of Multivariate Differential Entropy
cites work: Direct importance estimation for covariate shift adaptation
cites work: Estimating Divergence Functionals and the Likelihood Ratio by Convex Risk Minimization
cites work: Robust and efficient estimation by minimising a density power divergence
cites work: Density-ratio matching under the Bregman divergence: a unified framework of density-ratio estimation
cites work: Q5601960
cites work: A least-squares approach to direct importance estimation
cites work: Statistical analysis of kernel-based least-squares density-ratio estimation
cites work: Canonical dependency analysis based on squared-loss mutual information
cites work: Least-Squares Independent Component Analysis
cites work: Weak convergence and empirical processes. With applications to statistics
cites work: Asymptotic Statistics
cites work: Least angle regression. (With discussion)
cites work: Q3093229
cites work: On the influence of the kernel on the consistency of support vector machines
cites work: 10.1162/153244303322753616
cites work: Q4864293
cites work: Q5396651
cites work: Q4217298
cites work: Save: a method for dimension reduction and graphics in regression
cites work: Kernel dimension reduction in regression
cites work: The Geometry of Algorithms with Orthogonality Constraints
cites work: RELATIONS BETWEEN TWO SETS OF VARIATES
cites work: 10.1162/153244303768966085
cites work: Canonical correlation analysis based on information theory
cites work: Nonparametric and semiparametric models.
cites work: Sequential Fixed-Point ICA Based on Mutual Information Minimization
cites work: Blind separation of sources. I: An adaptive algorithm based on neuromimetic architecture
cites work: Q5617409
cites work: The estimation of the gradient of a density function, with applications in pattern recognition
cites work: Q3394879
cites work: Least-squares two-sample test
cites work: $f$-Divergence Estimation and Two-Sample Homogeneity Test Under Semiparametric Density-Ratio Models
cites work: Dimensionality reduction for density ratio estimation in high-dimensional spaces
cites work: Direct density-ratio estimation with dimensionality reduction via least-squares hetero-distributional subspace search
DOI: 10.3390/E15010080
description: scientific article; zbMATH DE number 6346066