ICML is a renowned platform for presenting and publishing cutting-edge research on all aspects of machine learning, statistics and data science.
Last year, ICML Conference attracted 4,990 submissions, with 1088 approved papers at a 21.8 percent acceptance rate. For this year’s conference, held online from July 18-24, the number of accepted papers stood at 1184.
Below, we have listed research papers from India accepted at ICML 2021.
RRL: Resnet as representation for Reinforcement Learning
Researchers from the Indian Institute of Technology (IIT), Kharagpur, in collaboration with the University of Washington and Facebook AI — Rutav Shah and Vikash Kumar, proposed a straightforward yet effective approach that can learn complex behaviours directly from proprioceptive inputs. RRL fuses features extracted from pre-trained Resnet into the standard reinforcement learning pipeline and delivers results comparable to learning directly from the state.
In a simulated dexterous manipulated benchmark, where the SOTA methods fail to make significant progress, researchers noted their RRL delivers contact rich behaviours.
On Characterizing GAN Convergence Through Proximal Duality Gap
In this work, IIT Ropar researchers Sahil Sidheekh, Aroof Aimen, and Narayanan C Krishnan extended the notion of duality gap to proximal duality gap that applies to the general context of training Generative Adversarial Networks (GANs), where Nash equilibria may not exist.
The researchers theoretically showed the proximal duality gap could monitor the convergence of GANs to a wider spectrum of equilibria that subsumes Nash equilibria. They also established the relationship between the proximal duality gap and the divergence between the real and generated data distributions for different GAN formulations. The result provided new insights into the nature of GAN convergence and validates the usefulness of the proximal duality gap for monitoring and influencing GAN training.
SiameseXML: Siamese Networks meet Extreme Classifiers with 100M Labels
Researchers from IIT Delhi, in partnership with Microsoft Research and IIT Kanpur, developed the SiameseXML framework based on a novel probabilistic model that naturally motivates a modular approach melding Siamese architecture with high-capacity extreme classifiers, and a training pipeline that effortlessly scales to tasks with 100 million labels. The research team included Kunal Dahiya, Ananye Agarwal, Deepak Saini, Gururaj K, Jian Jiao, Amit Singh, Sumeet Agarwal, Puru Kar, and Manik Varma.
The proposed technique offers 2 to 13 percent more accurate predictions than leading XML methods on public benchmark datasets, as well as in live A/B tests on the Bing search engine. It also offers significant gains in click-through rates, coverage, revenue and other online metrics over SOTA techniques currently in production.
Bayesian Structural Adaptation for Continual Learning
At present, (i) variational Bayes based regularisation by learning priors from previous tasks, and (ii) learning the structure of deep networks to adapt to new tasks have been the recent advances in continual learning with neural networks. The two approaches are orthogonal.
Addressing the shortcoming of both these approaches, researchers from IIT, Kanpur — Abhishek Kumar, Sunabha Chatterjee and Piyush Rai– presented a novel Bayesian approach to continual learning. The proposed model learns the deep structure for each task by learning which weights to be used and supports inter-task transfer by overlapping different sparse subsets of weights learned by various tasks.
Further, experimental results on supervised and unsupervised benchmarks showed the model performed comparably or better than recent advances in a continual learning setting.
Training Data Subset Selection for Regression with Controlled Generalization Error
Researchers from IIT Bombay, in collaboration with UT Dallas, designed an algorithm for selecting a subset of the training data so that the model can be trained quickly, without compromising on accuracy. The paper was co-authored by Durga Sivasubramanian, Rishab Iyer, Ganesh Ramakrishnan, and Abir De.
The researchers focused on data subset selection for L2 regularised regression problems and provided a novel problem formulation that looks to minimize the training loss with respect to both the trainable parameters and the subset of training data, subject to error bounds on the validation set.
Further, the researchers simplified constraints using the dual of the original training problem and showed the objective of this new representation is a monotone and α-submodular function, for a wide variety of modeling choices. It inevitably led them to develop SELCON, an algorithm for data subset selection.
