Min-Max Bilevel Multi-objective Optimization with Applications in Machine Learning

arXiv2203.01924MaRDI QIDQ6392734

Author name not available (Why is that?)

Publication date: 3 March 2022

Abstract: We consider a generic min-max multi-objective bilevel optimization problem with applications in robust machine learning such as representation learning and hyperparameter optimization. We design MORBiT, a novel single-loop gradient descent-ascent bilevel optimization algorithm, to solve the generic problem and present a novel analysis showing that MORBiT converges to the first-order stationary point at a rate of $widetildemathcalO(n^{1/2}{K}^{-2/5})$ for a class of weakly convex problems with $n$ objectives upon $K$ iterations of the algorithm. Our analysis utilizes novel results to handle the non-smooth min-max multi-objective setup and to obtain a sublinear dependence in the number of objectives $n$. Experimental results on robust representation learning and robust hyperparameter optimization showcase (i) the advantages of considering the min-max multi-objective setup, and (ii) convergence properties of the proposed MORBiT. Our code is at https://github.com/minimario/MORBiT.

Has companion code repository: https://github.com/minimario/morbit

This page was built for publication: Min-Max Bilevel Multi-objective Optimization with Applications in Machine Learning

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6392734)