2022_ICLR.md

ICLR 2022

Causality

• From Intervention to Domain Transportation: A Novel Perspective to Optimize Recommendation；Da Xu, Yuting Ye, Chuanwei Ruan, Evren Korpeoglu, Sushant Kumar, Kannan Achan
  • keywords: Information retrieval, Learning theory, Causal inference, Missing data, Overlapping, Reweighting, Optimal transport
• Invariant Causal Representation Learning for Out-of-Distribution Generalization；Chaochao Lu, Yuhuai Wu, José Miguel Hernández-Lobato, Bernhard Schölkopf
  • keywords: Causal Representation Learning,Out-of-distribution
  • We propose invariant Causal Representation Learning (iCaRL), an approach that enables out-of-distribution (OOD) generalization in the nonlinear setting (i.e., nonlinear representations and nonlinear classifiers).
• **Learning Causal Relationships from Conditional Moment Restrictions by Importance Weighting; *; Masahiro Kato, Masaaki Imaizumi, Kenichiro McAlinn, Shota Yasui, Haruo Kakehi
  • keywords: Causal inference, Conditional moment restrictions
• Optimal Transport for Causal Discovery; Ruibo Tu, Kun Zhang, Hedvig Kjellstrom, Cheng Zhang
  • keywords: causal discovery, optimal transport, functional causal model
• Granger causal inference on DAGs identifies genomic loci regulating transcription; Alexander P Wu, Rohit Singh, Bonnie Berger
  • keywords: Granger causality, causal inference, graph neural networks, gene regulation, single-cell genomics, chromatin accessibility, directed acyclic graphs, single-cell multimodal
• Causal Contextual Bandits with Targeted Interventions；Chandrasekar Subramanian, Balaraman Ravindran
  • keywords: causality, contextual bandits, causal inference, bandits
  • A new, more realistic, formalism of contextual bandits involving causal side-information and targeted interventions, along with a novel algorithm that exploits features of the new setting such as information leakage to learn good policies quickly.
• Learning Temporally Causal Latent Processes from General Temporal Data;Weiran Yao;Yuewen Sun;Alex Ho;Changyin Sun;Kun Zhang
  • *key words:
• On Covariate Shift of Latent Confounders in Imitation and Reinforcement Learning;Guy Tennenholtz;Assaf Hallak;Gal Dalal;Shie Mannor;Gal Chechik;Uri Shalit
  • key words:imitation learning,reinforcement learning,expert data,hidden confounding,causal inference,covariate shift,

Graph mining

• **A Biologically Interpretable Graph Convolutional Network to Link Genetic Risk Pathways and Imaging Phenotypes of Disease **;Sayan Ghosal;Qiang Chen;Giulio Pergola;Aaron L Goldman;William Ulrich;Daniel R Weinberger;Archana Venkataraman

  • key words: Imaging-genetics,Hierarchical Graph Convolution,Gene Ontology,Bayesian Feature Selection,Schizophrenia,

• You are AllSet: A Multiset Function Framework for Hypergraph Neural Networks;Eli Chien;Chao Pan;Jianhao Peng;Olgica Milenkovic

  • key words: Hypergraph neural networks,multiset functions,deep sets,set transformer,

• Why Propagate Alone? Parallel Use of Labels and Features on Graphs;Yangkun Wang;Jiarui Jin;Weinan Zhang;Yang Yongyi;Jiuhai Chen;Quan Gan;Yong Yu;Zheng Zhang;Zengfeng Huang;David Wipf

  • key words:,

• Equivariant and Stable Positional Encoding for More Powerful Graph Neural Networks;Haorui Wang;Haoteng Yin;Muhan Zhang;Pan Li

  • key words:Graph Neural Network,Spectral Graph Theory,System Stability,

• Graph Auto-Encoder via Neighborhood Wasserstein Reconstruction;Mingyue Tang;Pan Li;Carl Yang key words:graph representation learning,unsupervised learning,autoencoder,wasserstein distance,

• Understanding and Improving Graph Injection Attack by Promoting Unnoticeability;Yongqiang Chen;Han Yang;Yonggang Zhang;MA KAILI;Tongliang Liu;Bo Han;James Cheng key words:Graph Neural Networks,Adversarial Attacks,Node Classification,

• GRAND++: Graph Neural Diffusion with A Source Term;Matthew Thorpe;Tan Minh Nguyen;Hedi Xia;Thomas Strohmer;Andrea Bertozzi;Stanley Osher;Bao Wang key words:graph deep learning,low-labeling rates,diffusion on graphs,random walk,

• Query Embedding on Hyper-Relational Knowledge Graphs;Dimitrios Alivanistos;Max Berrendorf;Michael Cochez;Mikhail Galkin key words:Query embedding,Approximate Query Answering,Graph Neural Network,Hyper-relational Graph,Knowledge Graph,

• Space-Time Graph Neural Networks;Samar Hadou;Charilaos I Kanatsoulis;Alejandro Ribeiro key words:ST-GNNs,GNNs,stability,graph-time perturbations,

• PipeGCN: Efficient Full-Graph Training of Graph Convolutional Networks with Pipelined Feature Communication;Cheng Wan;Youjie Li;Cameron R. Wolfe;Anastasios Kyrillidis;Nam Sung Kim;Yingyan Lin key words:Graph Neural Networks,Graph Convolutional Networks,Distributed Training,Asynchronous Training,Full-Graph Training,Large-Graph Training,Stale Features,

• Handling Distribution Shifts on Graphs: An Invariance Perspective；Qitian Wu, Hengrui Zhang, Junchi Yan, David Wipf

  • Representation Learning on Graphs, Out-of-Distribution Generalization, Domain Shift, Graph Structure Learning, Invariant Models

• Expressiveness and Approximation Properties of Graph Neural Networks；Floris Geerts, Juan L Reutter

  • Graph Neural Networks, Colour Refinement, Weisfeiler-Leman, Separation Power, Universality
  • A general methodology for assessing the expressive and approximation power of GNNs is presented.

• Understanding over-squashing and bottlenecks on graphs via curvature；Jake Topping, Francesco Di Giovanni, Benjamin Paul Chamberlain, Xiaowen Dong, Michael M. Bronstein

  • Graph neural networks, Geometric deep learning, Differential geometry, Ricci curvature

• Neural Structured Prediction for Inductive Node Classification；Meng Qu, Huiyu Cai, Jian Tang

  • Graph neural networks
  • This paper studies node classification in the inductive setting, i.e., aiming to learn a model on labeled training graphs and generalize it to infer node labels on unlabeled test graphs.

• Data-Efficient Graph Grammar Learning for Molecular Generation；Minghao Guo, Veronika Thost, Beichen Li, Payel Das, Jie Chen, Wojciech Matusik

  • molecular generation, graph grammar, data efficient generative model

• A New Perspective on "How Graph Neural Networks Go Beyond Weisfeiler-Lehman?"；Asiri Wijesinghe, Qing Wang

  • Graph Neural Networks, Graph Isomorphism, Weisfeiler Lehman

• Topological Graph Neural Networks；Max Horn, Edward De Brouwer, Michael Moor, Yves Moreau, Bastian Rieck, Karsten Borgwardt

  • topology, persistent homology, gnn, graph neural networks, graph classification, node classification, filtrations, topological data analysis, tda
  • We describe a new layer for graph neural networks that incorporates multi-scale (ranging from local to global) topological information.

• Simple GNN Regularisation for 3D Molecular Property Prediction and Beyond；Jonathan Godwin, Michael Schaarschmidt, Alexander L Gaunt, Alvaro Sanchez-Gonzalez, Yulia Rubanova, Petar Veličković, James Kirkpatrick, Peter Battaglia

  • Graph Neural Networks, GNNs, Deep Learning, Molecular Property Prediction
  • A simple regularisation technique for GNNs applied to 3D molecular property prediction & beyond.

• Learning to Extend Molecular Scaffolds with Structural Motifs；Krzysztof Maziarz, Henry Richard Jackson-Flux, Pashmina Cameron, Finton Sirockin, Nadine Schneider, Nikolaus Stiefl, Marwin Segler, Marc Brockschmidt

  • molecules, graph neural networks, scaffold, generative model
  • We propose a new fragment-based generative model of molecules that can be constrained to include an arbitrary subgraph (scaffold).

• DEGREE: Decomposition Based Explanation for Graph Neural Networks；Qizhang Feng, Ninghao Liu, Fan Yang, Ruixiang Tang, Mengnan Du, Xia Hu

  • XAI, GNN
  • We propose a new decomposition based explanation for Graph Neural Networks.

• GNN is a Counter? Revisiting GNN for Question Answering；Kuan Wang, Yuyu Zhang, Diyi Yang, Le Song, Tao Qin

  • GNN, Question Answering, QA, Reasoning, ML
  • Counting is essential for reasoning and our simplistic graph neural counter is efficient and effective for QA tasks.

• NodePiece: Compositional and Parameter-Efficient Representations of Large Knowledge Graphs；Mikhail Galkin, Etienne Denis, Jiapeng Wu, William L. Hamilton

  • knowledge graphs, graph representation learning, tokenization, link prediction, node classification
  • Node hashing in graphs for 10-100x embedding size reduction without significant performance losses on many tasks and inductive out of the box.

• Convergent Graph Solvers；Junyoung Park, Jinhyun Choo, Jinkyoo Park

  • Graph, Graph Neural Network, Fixed point, Implicit model, Implicit function theorem, Convergent
  • We propose the convergent graph solver (CGS), a deep learning method that learns iterative mappings to predict the properties of a graph system at its stationary state (fixed point) with guaranteed convergence.

• How Attentive are Graph Attention Networks?；Shaked Brody, Uri Alon, Eran Yahav

  • graph attention networks, dynamic attention, GAT, GNN
  • We identify that Graph Attention Networks (GAT) compute a very weak form of attention. We show its empirical implications and propose a fix.

• Graph Neural Networks with Learnable Structural and Positional Representations；Vijay Prakash Dwivedi, Anh Tuan Luu, Thomas Laurent, Yoshua Bengio, Xavier Bresson

  • graph neural networks, graph representation learning, transformers, positional encoding
  • We propose a novel GNN architecture (LSPE) which decouples structural and positional representations to make easy for the network to learn the two essential properties.

• **Equivariant Subgraph Aggregation Networks; **Beatrice Bevilacqua, Fabrizio Frasca, Derek Lim, Balasubramaniam Srinivasan, Chen Cai, Gopinath Balamurugan, Michael M. Bronstein, Haggai Maron

  • keywords: Graph Neural Networks, Expressive power, Equivariance, Weisfeiler-Leman

• **Convergent Boosted Smoothing for Modeling GraphData with Tabular Node Features; **Jiuhai Chen, Jonas Mueller, Vassilis N. Ioannidis, Soji Adeshina, Yangkun Wang, Tom Goldstein, David Wipf

  • keywords: Graph Neural Network, Boosting, Node classification, Tabular Data

• **Ab-Initio Potential Energy Surfaces by Pairing GNNs with Neural Wave Functions; **Nicholas Gao, Stephan Günnemann

  • keywords: Graph Neural Networks, Computational Physics, Self-Generative Learning, Machine Learning for Science

• **Relational Multi-Task Learning: Modeling Relations between Data and Tasks; **Kaidi Cao, Jiaxuan You, Jure Leskovec

  • keywords: Graph Neural Networks, Relational Representation Learning, Multi-task Learning, Meta Learning

• **Is Homophily a Necessity for Graph Neural Networks?; **Yao Ma, Xiaorui Liu, Neil Shah, Jiliang Tang

• MoReL: Multi-omics Relational Learning; Arman Hasanzadeh, Ehsan Hajiramezanali, Nick Duffield, Xiaoning Qian

  • keywords: relational learning, data integration, multi-view learning, Bayesian generative model

• Automated Self-Supervised Learning for Graphs; Wei Jin, Xiaorui Liu, Xiangyu Zhao, Yao Ma, Neil Shah, Jiliang Tang

  • keywords: Self-supervised learning, Graph neural networks, AutoML

• Graph Condensation for Graph Neural Networks; Wei Jin, Lingxiao Zhao, Shichang Zhang, Yozen Liu, Jiliang Tang, Neil Shah

  • keywords: data-efficient learning, graph generation, graph neural networks

• Spherical Message Passing for 3D Molecular Graphs; Yi Liu, Limei Wang, Meng Liu, Yuchao Lin, Xuan Zhang, Bora Oztekin, Shuiwang Ji

• Top-N: Equivariant Set and Graph Generation without Exchangeability;Clement Vignac;Pascal Frossard key words:set generation,graph generation,permutation equivariance,generative models,Top-N,

• Inductive Relation Prediction Using Analogy Subgraph Embeddings;Jiarui Jin;Yangkun Wang;Kounianhua Du;Weinan Zhang;Zheng Zhang;David Wipf;Yong Yu;Quan Gan key words:Link Prediction,Relation Modelling,Heterogeneous Graphs,Knowledge Graphs,

• Benchmarking the Spectrum of Agent Capabilities;Danijar Hafner key words:Evaluation,Reinforcement Learning,Environment,Benchmark,Unsupervised Reinforcement Learning,Exploration, **Neural Relational Inference with Node-Specific Information **;Ershad Banijamali key words:Graph Neural Networks,Variational Inference,Trajectory Prediction,

• Filling the G_ap_s: Multivariate Time Series Imputation by Graph Neural Networks;Andrea Cini;Ivan Marisca;Cesare Alippi key words:graph neural networks,missing data,time series analysis,time series imputation,

• Information Gain Propagation: a New Way to Graph Active Learning with Soft Labels;Wentao Zhang;Yexin Wang;Zhenbang You;Meng Cao;Ping Huang;Jiulong Shan;Zhi Yang;Bin CUI

  • key words:Active Learning,Graph,Information Gain,
  • 关注图上的主动学习，利用软标签实现效果的提升。

• LEARNING GUARANTEES FOR GRAPH CONVOLUTIONAL NETWORKS ON THE STOCHASTIC BLOCK MODEL;Wei Lu

  • key words: detect communties
  • 关于图上训练过程中非凸优化过程的理论分析

• Learning 3D Representations of Molecular Chirality with Invariance to Bond Rotations;Keir Adams;Lagnajit Pattanaik;Connor W. Coley

  • key words:geometric deep learning,equivariance,molecules,
  • 蛋白质预测结合内部分子不变性思考

• EXACT: Scalable Graph Neural Networks Training via Extreme Activation Compression;Zirui Liu;Kaixiong Zhou;Fan Yang;Li Li;Rui Chen;Xia Hu

  • key words:graph neural networks,scalable GNN training,quantization,random projection,
  • 大型图神经网络压缩

• Towards Training Billion Parameter Graph Neural Networks for Atomic Simulations;Anuroop Sriram;Abhishek Das;Brandon M Wood;Siddharth Goyal;C. Lawrence Zitnick

  • key words:Graph Neural Networks,Atomic Simulations,Computational Chemistry,
  • 大规模原子图建模

• Using Graph Representation Learning with Schema Encoders to Measure the Severity of Depressive Symptoms;Simin Hong;Anthony Cohn;David Crossland Hogg

  • key words:Graph neural networks sentiment analysis node-embedding algorithm diagnostic prediction task,
  • 医疗GNN

• Do We Need Anisotropic Graph Neural Networks?;Shyam A. Tailor;Felix Opolka;Pietro Lio;Nicholas Donald Lane

  • key words:graph neural networks,efficiency,latency reduction,memory reduction,architecture design,benchmarking,hardware-aware,
  • GNN效率优化

• Automated Self-Supervised Learning for Graphs;Wei Jin, Xiaorui Liu, Xiangyu Zhao, Yao Ma, Neil Shah, Jiliang Tang

  • key words:Self-supervised learning, Graph neural networks, AutoML
  • An automated self-supervised learning algorithm for graph neural networks.

• Cold Brew: Distilling Graph Node Representations with Incomplete or Missing Neighborhoods;Wenqing Zheng, Edward W Huang, Nikhil Rao, Sumeet Katariya, Zhangyang Wang, Karthik Subbian

  • key words:Graph Neural Networks, Cold Start, Knowledge Distillation
  • Improve strict cold start performances for graph minings with a knowledge distillation framework.

• Spatial Graph Attention and Curiosity-driven Policy for Antiviral Drug Discovery；Yulun Wu, Nicholas Choma, Andrew Deru Chen, Mikaela Cashman, Erica Teixeira Prates, Veronica G Melesse Vergara, Manesh B Shah, Austin Clyde, Thomas Brettin, Wibe Albert de Jong, Neeraj Kumar, Martha S Head, Rick L. Stevens, Peter Nugent, Daniel A Jacobson, James B Brown

  • keywords: reinforcement learning, graph neural network, molecule generation, drug discovery, curiosity-driven policy
  • We developed a reinforcement learning framework that advances in exploiting spatial and attributional molecular information as well as exploring novel and synthesizable chemical structures for the purpose of antiviral drug discovery.

• Crystal Diffusion Variational Autoencoder for Periodic Material Generation;Tian Xie, Xiang Fu, Octavian-Eugen Ganea, Regina Barzilay, Tommi S. Jaakkola

  • key words:materials, graph neural networks, periodic, diffusion models, score matching, molecule, 3D, generative

• Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Prediction;Eli Chien, Wei-Cheng Chang, Cho-Jui Hsieh, Hsiang-Fu Yu, Jiong Zhang, Olgica Milenkovic, Inderjit S Dhillon

  • key words:Self-supervised learning, Graph Neural Networks, Extreme multi-label classification
  • We design a self-supervised learning method for extracting node representations from raw data.

• Equivariant Graph Mechanics Networks with Constraints;Wenbing Huang;Jiaqi Han;Yu Rong;Tingyang Xu;Fuchun Sun;Junzhou Huang

  • key words:,

• GraphENS: Neighbor-Aware Ego Network Synthesis for Class-Imbalanced Node Classification;Joonhyung Park;Jaeyun Song;Eunho Yang

  • key words:Deep learning,Node classification,Class imbalance,Data Augmentation,

Debiasing、fairness、long-tail、ODD、domain adaption、鲁棒性、降噪 （包括推荐和非推荐的都算）

• Revisiting flow generative models for Out-of-distribution detection;Dihong Jiang;Sun Sun;Yaoliang Yu

  • key words:flow models,out-of-distribution detection,random projection,distribution comparison,

• VOS: Learning What You Don't Know by Virtual Outlier Synthesis;Xuefeng Du;Zhaoning Wang;Mu Cai;Yixuan Li

  • key words:,

• Optimal Transport for Long-Tailed Recognition with Learnable Cost Matrix;Hanyu Peng;Mingming Sun;Ping Li

  • key words:Long-tailed recognition,imbalanced classification,optimal transport,

• Provable Adaptation across Multiway Domains via Representation Learning;Zhili Feng;Shaobo Han;Simon Shaolei Du

  • key words:Representation learning,tensor,statistical learning theory,

• Visual Representation Learning Does Not Generalize Strongly Within the Same Domain;Lukas Schott;Julius Von Kügelgen;Frederik Träuble;Peter Vincent Gehler;Chris Russell;Matthias Bethge;Bernhard Schölkopf;Francesco Locatello;Wieland Brendel key words:Generalization,Composition,Out of distribution,Disentanglement,

• Adversarially Robust Conformal Prediction;Asaf Gendler;Tsui-Wei Weng;Luca Daniel;Yaniv Romano key words:Conformal Prediction,Adversarial Robustness,Randomized Smoothing,Uncertainty Estimation,Calibration,

• Gradient Matching for Domain Generalization;Yuge Shi;Jeffrey Seely;Philip Torr;Siddharth N;Awni Hannun;Nicolas Usunier;Gabriel Synnaeve key words:Domain generalization,multi-source domain adaptation,

• Hierarchical Variational Memory for Few-shot Learning Across Domains;Yingjun Du;Xiantong Zhen;Ling Shao;Cees G. M. Snoek key words:Meta-learning,Variational hierarchical memory,Variational hierarchical prototype,Cross-domain few-shot learning,

• PriorGrad: Improving Conditional Denoising Diffusion Models with Data-Dependent Adaptive Prior;Sang-gil Lee;Heeseung Kim;Chaehun Shin;Xu Tan;Chang Liu;Qi Meng;Tao Qin;Wei Chen;Sungroh Yoon;Tie-Yan Liu key words:diffusion-based model,generative model,speech synthesis,

• Gradient Step Denoiser for convergent Plug-and-Play;Samuel Hurault;Arthur Leclaire;Nicolas Papadakis key words:Plug-and-Play,Inverse Problem,Image Restoration,Denoising,

• Switch to Generalize: Domain-Switch Learning for Cross-Domain Few-Shot Classification;Zhengdong Hu;Yifan Sun;Yi Yang key words:,

• Is Fairness Only Metric Deep? Evaluating and Addressing Subgroup Gaps in Deep Metric Learning;Natalie Dullerud;Karsten Roth;Kimia Hamidieh;Nicolas Papernot;Marzyeh Ghassemi key words:deep metric learning,fairness,representation learning,

• Learning to Generalize across Domains on Single Test Samples;Zehao Xiao;Xiantong Zhen;Ling Shao;Cees G. M. Snoek key words:domain generalization,single test sample generalization,meta learning,variational inference,

• Certified Robustness for Deep Equilibrium Models via Interval Bound Propagation;Colin Wei;J Zico Kolter key words:deep equilibrium models,certified robustness,interval bound propagation,

• Variational autoencoders in the presence of low-dimensional data: landscape and implicit bias;Frederic Koehler;Viraj Mehta;Chenghui Zhou;Andrej Risteski key words:variational autoencoders,encoder,optima,stability,low-dimensional manifold,

• Attacking deep networks with surrogate-based adversarial black-box methods is easy;Nicholas A. Lord;Romain Mueller;Luca Bertinetto key words:adversarial attacks,black-box attacks,network robustness,network analysis,

• Handling Distribution Shifts on Graphs: An Invariance Perspective；Qitian Wu, Hengrui Zhang, Junchi Yan, David Wipf

  • Representation Learning on Graphs, Out-of-Distribution Generalization, Domain Shift, Graph Structure Learning, Invariant Models

• From Intervention to Domain Transportation: A Novel Perspective to Optimize Recommendation；Da Xu, Yuting Ye, Chuanwei Ruan, Evren Korpeoglu, Sushant Kumar, Kannan Achan

  • Information retrieval, Learning theory, Causal inference, Missing data, Overlapping, Reweighting, Optimal transport

• Cross-Domain Imitation Learning via Optimal Transport；Junguang Jiang, Baixu Chen, Jianmin Wang, Mingsheng Long

  • optimal transportation, imitation learning, cross-domain imitation learning, gromov-Wasserstein
  • We study the use of Gromov-Wasserstein for cross-domain imitation learning

• Decoupled Adaptation for Cross-Domain Object Detection；Arnaud Fickinger, Samuel Cohen, Stuart Russell, Brandon Amos

  • Object Detection, Domain Adaptation, Object Localization, Deep Learning, Transfer Learning
  • To deal with the challenges in cross-domain object detection, we propose D-adapt to decouple the adversarial adaptation and the training of the detector, and also decouple the category adaptation and the bounding box adaptation.

• Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution；Ananya Kumar, Aditi Raghunathan, Robbie Matthew Jones, Tengyu Ma, Percy Liang

  • fine-tuning theory, transfer learning theory, fine-tuning, distribution shift, implicit regularization
  • Fine-tuning does better than linear probing (training a linear classifier on pretrained features) in-distribution, but worse out-of-distribution (OOD)---we analyze why this happens and propose a way to get the benefits of both.

• Asymmetry Learning for Counterfactually-invariant Classification in OOD Tasks；S Chandra Mouli, Bruno Ribeiro

  • out-of-distribution classification, symmetries, counterfactual invariances, geometric deep learning
  • Counterfactual-invariant representations for symmetry transformations

• Extending the WILDS Benchmark for Unsupervised Adaptation；Shiori Sagawa, Pang Wei Koh, Tony Lee, Irena Gao, Sang Michael Xie, Kendrick Shen, Ananya Kumar, Weihua Hu, Michihiro Yasunaga, Henrik Marklund, Sara Beery, Etienne David, Ian Stavness, Wei Guo, Jure Leskovec, Kate Saenko, Tatsunori Hashimoto, Sergey Levine, Chelsea Finn, Percy Liang

  • distribution shifts, adaptation, unlabeled data
  • We introduce U-WILDS, which augments the WILDS distribution shift benchmark with realistic unlabeled data, and benchmark existing methods for unlabeled data on these in-the-wild distribution shifts.

• Non-Transferable Learning: A New Approach for Model Ownership Verification and Applicability Authorization；Lixu Wang, Shichao Xu, Ruiqi Xu, Xiao Wang, Qi Zhu

  • Domain Adaptation, Transfer Learning, Societal Considerations of Representation Learning, Model Watermark
  • We propose a novel Non-Transferable Learning (NTL) method to restrict the model generalization ability to certain domains for model ownership verification and applicability authorization.

• A Fine-Grained Analysis on Distribution Shift；Olivia Wiles, Sven Gowal, Florian Stimberg, Sylvestre-Alvise Rebuffi, Ira Ktena, Krishnamurthy Dj Dvijotham, Ali Taylan Cemgil

  • robustness, distribution shifts
  • We investigate and analyse the robustness of a variety of methods under distribution shifts using our flexible experimental framework.

• Resolving Training Biases via Influence-based Data Relabeling；Shuming Kong, Yanyan Shen, Linpeng Huang

  • Training bias, influence functions, data relabeling
  • We propose an influence-based relabeling framework for solving training bias with a theoretical guarantee.

• Practical Integration via Separable Bijective Networks；Christopher M Bender, Patrick Emmanuel, Michael K. Reiter, Junier Oliva

  • integration, flow, likelihood, classification, regression, out of distribution, regularization
  • We explore a method that enables learning over hypervolumes within the data space.

• Focus on the Common Good: Group Distributional Robustness Follows；Vihari Piratla, Praneeth Netrapalli, Sunita Sarawagi

  • sub-population shift, robust optimization, domain generalization
  • We propose a new and simple algorithm for the sub-population shift problem that enables learning of shared features and performed consistently well over several standard, and real-world, benchmarks of the problem.

• Discrepancy-Based Active Learning for Domain Adaptation；Antoine De mathelin, François Deheeger, Mathilde MOUGEOT, Nicolas Vayatis

  • active learning, domain adaptation, discrepancy, kmedoids, single batch, covariate shift
  • This paper presents an active learning for domain adaptation method based on a localized discrepancy between source and target distributions..

• Domain Adversarial Training: A Game Perspective；David Acuna, Marc T Law, Guojun Zhang, Sanja Fidler

  • Domain Adversarial Training, Domain Adaptation, Neural Networks Optimization, Game Theory
  • A novel perspective on domain-adversarial training that leads to more stable and performant optimizers.

• Source-Free Adaptation to Measurement Shift via Bottom-Up Feature Restoration; Cian Eastwood, Ian Mason, Chris Williams, Bernhard Schölkopf

  • keywords: Transfer learning, dataset shift, unsupervised domain adaptation, source-free domain adaptation

• Conditional Contrastive Learning with Kernel; Yao-Hung Hubert Tsai, Tianqin Li, Martin Q. Ma, Han Zhao, Kun Zhang, Louis-Philippe Morency, Ruslan Salakhutdinov

  • keywords: Contrastive Learning, Conditional Sampling, Kernel methods

• Surrogate Gap Minimization Improves Sharpness-Aware Training; Juntang Zhuang, Boqing Gong, Liangzhe Yuan, Yin Cui, Hartwig Adam, Nicha C Dvornek, sekhar tatikonda, James s Duncan, Ting Liu

  • keywords: generalization, sharpness-aware minimization, surrogate gap, deep learning

• Towards Building A Group-based Unsupervised Representation Disentanglement Framework;Tao Yang, Xuanchi Ren, Yuwang Wang, Wenjun Zeng, Nanning Zheng

  • keywords: Disentangled representation learning, Group theory, VAE

• Stein Latent Optimization for Generative Adversarial Networks; Uiwon Hwang, Heeseung Kim, Dahuin Jung, Hyemi Jang, Hyungyu Lee, Sungroh Yoon

  • keywords: Generative Adversarial Networks, Unsupervised Conditional GANs

• Deep AutoAugment;Yu Zheng, Zhi Zhang, Shen Yan, Mi Zhang

  • keywords: automated machine learning, data augmentation

• AdaAug: Learning Class- and Instance-adaptive Data Augmentation Policies;Tsz-Him Cheung, Dit-Yan Yeung

  • keywords: Data Augmentation, Automated Data Augmentation

• Tracking the risk of a deployed model and detecting harmful distribution shifts;Aleksandr Podkopaev, Aaditya Ramdas

  • keywords: Distribution shift, sequential testing

• Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotations; Jiaheng Wei, Zhaowei Zhu, Hao Cheng, Tongliang Liu, Gang Niu, Yang Liu

  • keywords: Learning with noisy labels, benchmark, real-world label noise, human annotations

• Distributionally Robust Fair Principal Components via Geodesic Descents; Hieu Vu, Toan Tran, Man-Chung Yue, Viet Anh Nguyen

  • keywords: fair principal component analysis, distributionally robust optimization, manifold optimization

• PI3NN: Out-of-distribution-aware Prediction Intervals from Three Neural Networks;Siyan Liu, Pei Zhang, Dan Lu, Guannan Zhang

• Meta Learning Low Rank Covariance Factors for Energy Based Deterministic Uncertainty; Jeffrey Ryan Willette, Hae Beom Lee, Juho Lee, Sung Ju Hwang

  • keywords: calibration, meta-learning

• Graph Neural Network Guided Local Search for the Traveling Salesperson Problem; Benjamin Hudson, Qingbiao Li, Matthew Malencia, Amanda Prorok

  • keywords: Traveling Salesman Problem, Graph Neural Network, Metaheuristic, Guided Local Search, Hybrid

• From Stars to Subgraphs: Uplifting Any GNN with Local Structure Awareness; Lingxiao Zhao, Wei Jin, Leman Akoglu, Neil Shah

  • keywords: Graph Neural Networks, Expressiveness, Message Passing Neural Network, Graph Classification

• Learning to Schedule Learning rate with Graph Neural Networks; Yuanhao Xiong, Li-Cheng Lan, Xiangning Chen, Ruochen Wang, Cho-Jui Hsieh

  • keywords: learning rate scheduling, graph neural networks

• The Rich Get Richer: Disparate Impact of Semi-Supervised Learning; Zhaowei Zhu, Tianyi Luo, Yang Liu

  • keywords: semi-supervised learning, fairness, disparate impact, Matthew effect, consistency regularization

• Learning Distributionally Robust Models at Scale via Composite Optimization; Farzin Haddadpour, Mohammad Mahdi Kamani, Mehrdad Mahdavi, amin karbasi

  • keywords: Composite Optimization, Distributionally Robust Optimization

• MaGNET: Uniform Sampling from Deep Generative Network Manifolds Without Retraining;Ahmed Imtiaz Humayun;Randall Balestriero;Richard Baraniuk

  • key words:Deep Generative Networks,Uniform Sampling,Fairness,Data Augmentation,

• CrossMatch: Cross-Classifier Consistency Regularization for Open-Set Single Domain Generalization;Ronghang Zhu;Sheng Li

  • key words:Single Domain Generalization,Open-Set Recognition,
  • 提出了一个新的问题Open Set Single Domain Generalization

• Mind the Gap: Domain Gap Control for Single Shot Domain Adaptation for Generative Adversarial Networks;Peihao Zhu;Rameen Abdal;John Femiani;Peter Wonka

  • key words:GAN,StyleGAN,Clip,Domain Adaptation,Style Transfer,Single Shot,

• The Role of Pretrained Representations for the OOD Generalization of RL Agents;Frederik Träuble;Andrea Dittadi;Manuel Wuthrich;Felix Widmaier;Peter Vincent Gehler;Ole Winther;Francesco Locatello;Olivier Bachem;Bernhard Schölkopf;Stefan Bauer

  • key words:representations,out-of-distribution,generalization,deep learning,reinforcement learning,
  • OOD结合RL解决实际问题

• An Agnostic Approach to Federated Learning with Class Imbalance;Zebang Shen;Juan Cervino;Hamed Hassani;Alejandro Ribeiro

  • key words:Federated Learning,Class Imbalance,

• Towards Evaluating the Robustness of Neural Networks Learned by Transduction;Jiefeng Chen;Xi Wu;Yang Guo;Yingyu Liang;Somesh Jha

  • key words:adversarial robustness,transductive learning,test-time defense,dynamic defense,attacking model spaces,
  • 模型鲁棒性

• A Unified Wasserstein Distributional Robustness Framework for Adversarial Training;Anh Tuan Bui;Trung Le;Quan Hung Tran;He Zhao;Dinh Phung

  • key words:Adversarial Machine Learning,Distributional Robustness,
  • 推土机距离 对抗的新进展

• Sparsity Winning Twice: Better Robust Generalization from More Efficient Training;Tianlong Chen;Zhenyu Zhang;pengjun wang;Santosh Balachandra;Haoyu Ma;Zehao Wang;Zhangyang Wang

  • key words:,
  • 在对抗性训练中注入适当形式的稀疏性显著缩小鲁棒泛化差距，缓解鲁棒过度拟合

• Task Affinity with Maximum Bipartite Matching in Few-Shot Learning;Cat Phuoc Le, Juncheng Dong, Mohammadreza Soltani, Vahid Tarokhg

  • key words:Task Affinity, Transfer Learning, Few-Shot Learning,
  • Task affinity and its application in few-shot learning

• VOS: Learning What You Don't Know by Virtual Outlier Synthesis;Xuefeng Du, Zhaoning Wang, Mu Cai, Yixuan Li

  • key words: Out-of-distribution

• Out-of-distribution Generalization in the Presence of Nuisance-Induced Spurious Correlations;Aahlad Manas Puli;Lily H Zhang;Eric Karl Oermann;Rajesh Ranganath key words:spurious correlations,out of distribution generalization,ml for health,representation learning,

• Plant 'n' Seek: Can You Find the Winning Ticket?;Jonas Fischer;Rebekka Burkholz key words:lottery tickets,ground truth,planting,LTH,

• Illiterate DALL-E Learns to Compose;Gautam Singh;Fei Deng;Sungjin Ahn key words:Zero-Shot Image Generation,Compositional Representation,Object-Centric Representation,Out-of-Distribution Generalization,Image Transformers,

• MAML is a Noisy Contrastive Learner in Classification;Chia Hsiang Kao;Wei-Chen Chiu;Pin-Yu Chen key words:Meta learning,contrastive learning,few shot learning,

• Fairness Guarantees under Demographic Shift;Stephen Giguere;Blossom Metevier;Bruno Castro da Silva;Yuriy Brun;Philip S. Thomas;Scott Niekum key words:Fairness and Bias in Artificial Intelligence,Machine Learning,

• Universalizing Weak Supervision;Changho Shin;Winfred Li;Harit Vishwakarma;Nicholas Carl Roberts;Frederic Sala key words:Weak supervision, ''

• Hindsight: Posterior-guided training of retrievers for improved open-ended generation;Ashwin Paranjape;Omar Khattab;Christopher Potts;Matei Zaharia;Christopher D Manning key words:retrieval,generation,retrieval-augmented generation,open-ended generation,informative conversations,free-form QA,posterior distribution,ELBo,

• Do deep networks transfer invariances across classes?;Allan Zhou;Fahim Tajwar;Alexander Robey;Tom Knowles;George J. Pappas;Hamed Hassani;Chelsea Finn key words:invariance,augmentation,nuisance transformation,imbalance,long tail,

• Autonomous Learning of Object-Centric Abstractions for High-Level Planning;Steven James;Benjamin Rosman;George Konidaris key words:reinforcement learning,planning,multitask,transfer,objects,

• Rethinking Class-Prior Estimation for Positive-Unlabeled Learning;Yu Yao;Tongliang Liu;Bo Han;Mingming Gong;Gang Niu;Masashi Sugiyama;Dacheng Tao key words:Positive-Unlabeled Learning,Class-Prior Estimation,

• CDTrans: Cross-domain Transformer for Unsupervised Domain Adaptation;Tongkun Xu;Weihua Chen;Pichao WANG;Fan Wang;Hao Li;Rong Jin key words:,

• Rethinking Adversarial Transferability from a Data Distribution Perspective;Yao Zhu;Jiacheng Sun;Zhenguo Li key words:Adversarial Attack,Adversarial Transferability,Black-box Attack,

• Generative Pseudo-Inverse Memory;Kha Pham;Hung Le;Man Ngo;Truyen Tran;Bao Ho;Svetha Venkatesh key words:,

• Discrete Representations Strengthen Vision Transformer Robustness;Chengzhi Mao;Lu Jiang;Mostafa Dehghani;Carl Vondrick;Rahul Sukthankar;Irfan Essa key words:vision transformer,robustness,image recognition,

• Beyond ImageNet Attack: Towards Crafting Adversarial Examples for Black-box Domains;Qilong Zhang;Xiaodan Li;YueFeng Chen;Jingkuan Song;Lianli Gao;Yuan He;Hui Xue' key words:practice black-box attack,cross-domain transferability,

• Label-Efficient Semantic Segmentation with Diffusion Models;Dmitry Baranchuk;Andrey Voynov;Ivan Rubachev;Valentin Khrulkov;Artem Babenko key words:,

• Online Adversarial Attacks;Andjela Mladenovic;Joey Bose;Hugo berard;William L. Hamilton;Simon Lacoste-Julien;Pascal Vincent;Gauthier Gidel key words:Online Algorithms,Adversarial Attacks,

• On the Role of Neural Collapse in Transfer Learning;Tomer Galanti;András György;Marcus Hutter key words:,

交互式推荐、Bandit、强化学习

• Maximum Entropy RL (Provably) Solves Some Robust RL Problems;Benjamin Eysenbach;Sergey Levine\

  • key words:reinforcement learning,robustness,maximum entropy,

• Pessimistic Model-based Offline Reinforcement Learning under Partial Coverage;*Masatoshi Uehara;Wen Sun

  • key words:Reinforcement learning Theory,Offline reinforcement learning,PAC Bounds,

• When Can We Learn General-Sum Markov Games with a Large Number of Players Sample-Efficiently?;Ziang Song;Song Mei;Yu Bai

• key words:reinforcement learning theory,multi-agent RL,Markov games,general-sum games,

• Variational oracle guiding for reinforcement learning;Dongqi Han;Tadashi Kozuno;Xufang Luo;Zhao-Yun Chen;Kenji Doya;Yuqing Yang;Dongsheng Li

  • key words:variational Bayes,oracle guiding,reinforcement learning,decision making,probabilistic modeling,game,Mahjong,

• Bregman Gradient Policy Optimization;Feihu Huang;Shangqian Gao;Heng Huang

  • key words:,

• Actor-Critic Policy Optimization in a Large-Scale Imperfect-Information Game;Haobo Fu;Weiming Liu;Shuang Wu;Yijia Wang;Tao Yang;Kai Li;Junliang Xing;Bin Li;Bo Ma;QIANG FU;Yang Wei

  • key words:Policy Optimization,Nash Equilibrium,Mahjong AI,

• Hindsight Foresight Relabeling for Meta-Reinforcement Learning;Michael Wan;Jian Peng;Tanmay Gangwani

  • key words:Reinforcement Learning,Meta-Learning,

• Convergent and Efficient Deep Q Learning Algorithm;Zhikang T. Wang;Masahito Ueda

  • key words:DQN,reinforcement learning,convergence,

• Learning Value Functions from Undirected State-only Experience;Matthew Chang;Arjun Gupta;Saurabh Gupta

  • key words:Reinforcement Learning,Offline RL,Offline RL without actions,

• COPA: Certifying Robust Policies for Offline Reinforcement Learning against Poisoning Attacks;Fan Wu;Linyi Li;Huan Zhang;Bhavya Kailkhura;Krishnaram Kenthapadi;Ding Zhao;Bo Li

  • key words:certified robustness,poisoning attacks,reinforcement learning, CROP: Certifying Robust Policies for Reinforcement Learning through Functional Smoothing;Fan Wu;Linyi Li;Zijian Huang;Yevgeniy Vorobeychik;Ding Zhao;Bo Li key words:

• Learning Graphon Mean Field Games and Approximate Nash Equilibria;Kai Cui;Heinz Koeppl key words:Mean Field Games,Reinforcement Learning,Multi Agent Systems,

• Learning Neural Contextual Bandits through Perturbed Rewards;Yiling Jia;Weitong ZHANG;Dongruo Zhou;Quanquan Gu;Hongning Wang key words:contextual bandit,neural bandit,

• Policy Gradients Incorporating the Future;David Venuto;Elaine Lau;Doina Precup;Ofir Nachum key words:,

• Pareto Policy Adaptation;Panagiotis Kyriakis;Jyotirmoy Deshmukh;Paul Bogdan key words:multi-objective reinforcement learning,policy gradient,pareto optimality,policy adaptation,

• HyAR: Addressing Discrete-Continuous Action Reinforcement Learning via Hybrid Action Representation；Boyan Li, Hongyao Tang, YAN ZHENG, Jianye HAO, Pengyi Li, Zhen Wang, Zhaopeng Meng, LI Wang

  • Reinforcement Learning,Discrete-continuous hybrid action space

• Know Your Action Set: Learning Action Relations for Reinforcement Learning；Ayush Jain, Norio Kosaka, Kyung-Min Kim, Joseph J Lim

  • reinforcement learning, varying action space, relational reasoning

• A Relational Intervention Approach for Unsupervised Dynamics Generalization in Model-Based Reinforcement Learning；Jiaxian Guo, Mingming Gong, Dacheng Tao

  • Model-Based Reinforcement Learning, Unsupervised Dynamics Generalization

• Topological Experience Replay；Zhang-Wei Hong ~Zhang-Wei_Hong1 , Tao Chen, Yen-Chen Lin, Joni Pajarinen, Pulkit Agrawal

  • Deep reinforcement learning, experience replay
  • We rearrange the update order of experience for training the Q-function by a dependency graph.

• The Boltzmann Policy Distribution: Accounting for Systematic Suboptimality in Human Models；Cassidy Laidlaw, Anca Dragan

  • human model, boltzmann rationality, suboptimality, HRI, human-robot collaboration, generative models, reinforcement learning, deep RL
  • We propose modeling human behavior with a Boltzmann distribution over policies—not trajectories—and show it is more accurate and useful.

• Value Function Spaces: Skill-Centric State Abstractions for Long-Horizon Reasoning；Dhruv Shah, Peng Xu, Yao Lu, Ted Xiao, Alexander T Toshev, Sergey Levine, brian ichter

  • hierarchical reinforcement learning, planning, representation learning, robotics
  • We introduce value function spaces, a learned representation of state through the values of low-level skills, which capture affordances and ignores distractors to enable long-horizon reasoning and zero-shot generalization.

• Generalisation in Lifelong Reinforcement Learning through Logical Composition；Geraud Nangue Tasse, Steven James, Benjamin Rosman

  • Reinforcement Learning, Lifelong learning, Multi task learning, Transfer learning, Logical composition, Deep Reinforcement Learning
  • A framework with theoretical guarantees for an agent to quickly generalize over a task space by autonomously determining whether a new task can be solved zero-shot using existing skills, or whether a task-specific skill should be learned few-shot.

• Provably Filtering Exogenous Distractors using Multistep Inverse Dynamics；Yonathan Efroni, Dipendra Misra, Akshay Krishnamurthy, Alekh Agarwal, John Langford

  • Reinforcement Learning Theory, Invariant Representation, Rich Observation Reinforcement Learning, Exogenous Noise, Inverse Dynamics

• Transform2Act: Learning a Transform-and-Control Policy for Efficient Agent Design；Ye Yuan, Yuda Song, Zhengyi Luo, Wen Sun, Kris M. Kitani

  • Agent Design, Morphology Optimization, Reinforcement Learning

• The Information Geometry of Unsupervised Reinforcement Learning；Benjamin Eysenbach, Ruslan Salakhutdinov, Sergey Levine

  • unsupervised skill learning, reward-free RL, mutual information, DIAYN
  • We show that mutual information skill learning is optimal in one sense but not optimal in another sense.

• Accelerated Policy Learning with Parallel Differentiable Simulation；Jie Xu, Viktor Makoviychuk, Yashraj Narang, Fabio Ramos, Wojciech Matusik, Animesh Garg, Miles Macklin

  • Robot Control, Policy Learning, Differentiable Simulation, Reinforcement Learning
  • We propose an efficient policy learning method leveraging the recent advance of differentiable simulation, and our method outperforms state-of-the-art algorithms in both sample efficiency and wall clock time on multiple challenging control tasks.

• Model-Based Offline Meta-Reinforcement Learning with Regularization；Sen Lin, Jialin Wan, Tengyu Xu, Yingbin Liang, Junshan Zhang

  • offline reinforcement learning, model-based reinforcement learning, behavior policy, Meta-reinforcement learning
  • This paper proposes a novel offline Meta-RL algorithm with regularization, which has provable performance improvement and outperforms the existing baselines empirically.

• Learning State Representations via Retracing in Reinforcement Learning；Changmin Yu, Dong Li, Jianye HAO, Jun Wang, Neil Burgess

  • Representation learning, model-based reinforcement learning
  • We introduce Learning via Retracing, a novel self-supervised framework based on temporal cycle-consistency assumption of the transition dynamics, for improved learning of the representation (and the dynamics model) in RL tasks.

• LIGS: Learnable Intrinsic-Reward Generation Selection for Multi-Agent Learning；David Henry Mguni, Taher Jafferjee, Jianhong Wang, Nicolas Perez-Nieves, Oliver Slumbers, Feifei Tong, Yang Li, Jiangcheng Zhu, Yaodong Yang, Jun Wang

  • multi-agent, reinforcement learning, intrinsic rewards, exploration

• Efficient Active Search for Combinatorial Optimization Problems；André Hottung, Yeong-Dae Kwon, Kevin Tierney

  • heuristic search, combinatorial optimization, learning to optimize, reinforcement learning, traveling salesperson problem, vehicle routing problem, job shop scheduling problem
  • We propose active search approaches for combinatorial optimization problems that search for solutions by adjusting a subset of (model) parameters to a single instance at test time.

• Imitation Learning from Observations under Transition Model Disparity；Tanmay Gangwani, Yuan Zhou, Jian Peng

  • Imitation Learning, Deep Reinforcement Learning
  • Imitation learning from observations when the expert and the learner agents operate in environments with dissimilar transition dynamics models.

• Pareto Policy Pool for Model-based Offline Reinforcement Learning；Yijun Yang, Jing Jiang, Tianyi Zhou, Jie Ma, Yuhui Shi

  • model-based offline RL, Pareto front, multi-objective optimization, policy pool, model return-uncertainty trade-off
  • We propose a model-based offline RL method that builds a diverse set of optimal policies on Pareto front providing different levels of model return-uncertainty trade-off and it significantly outperforms single-policy methods.

• Maximizing Ensemble Diversity in Deep Reinforcement Learning；Hassam Sheikh, Mariano Phielipp, Ladislau Boloni

  • Ensemble Based Reinforcement Learning, Ensemble Diversity
  • Maximizing diversity in neural network improves performance ensemble based reinforcement learning.

• Is High Variance Unavoidable in RL? A Case Study in Continuous Control；Johan Bjorck, Carla P Gomes, Kilian Q Weinberger

  • reinforcement learning, continuous control
  • we study sources of variance in RL and propose methods to decrease it.

• Provably Filtering Exogenous Distractors using Multistep Inverse Dynamics；Yonathan Efroni, Dipendra Misra, Akshay Krishnamurthy, Alekh Agarwal, John Langford

  • Reinforcement Learning Theory, Invariant Representation, Rich Observation Reinforcement Learning, Exogenous Noise, Inverse Dynamics

• Transform2Act: Learning a Transform-and-Control Policy for Efficient Agent Design；Ye Yuan, Yuda Song, Zhengyi Luo, Wen Sun, Kris M. Kitani

  • Agent Design, Morphology Optimization, Reinforcement Learning

• The Information Geometry of Unsupervised Reinforcement Learning；Benjamin Eysenbach, Ruslan Salakhutdinov, Sergey Levine

  • unsupervised skill learning, reward-free RL, mutual information, DIAYN
  • We show that mutual information skill learning is optimal in one sense but not optimal in another sense.

• Learning Altruistic Behaviours in Reinforcement Learning without External Rewards; Tim Franzmeyer, Mateusz Malinowski, Joao F. Henriques

  • keywords: reinforcement learning, altruistic behavior in AI, multi-agent systems

• POETREE: Interpretable Policy Learning with Adaptive Decision Trees; Alizée Pace, Alex Chan, Mihaela van der Schaar

  • keywords: Imitation Learning, Interpretable ML, Clinical Decision Support, Sequential Decision-Making

• Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformers; Ruihan Yang, Minghao Zhang, Nicklas Hansen, Huazhe Xu, Xiaolong Wang

  • keywords: Reinforcement Learning, Robotics, Locomotion Control, Multi-Modal Transformer

• Planning in Stochastic Environments with a Learned Model; Ioannis Antonoglou, Julian Schrittwieser, Sherjil Ozair, Thomas K Hubert, David Silver

  • keywords: model-based reinforcement learning, deep reinforcement learning, tree based search, MCTS

• AdaRL: What, Where, and How to Adapt in Transfer Reinforcement Learning; Biwei Huang, Fan Feng, Chaochao Lu, Sara Magliacane, Kun Zhang

  • keywords: Transfer RL, Graphical models, Efficient adaptation

• R5: Rule Discovery with Reinforced and Recurrent Relational Reasoning; Shengyao Lu, Bang Liu, Keith G Mills, SHANGLING JUI, Di Niu

  • keywords: systematicity, graph reasoning

• Possibility Before Utility: Learning And Using Hierarchical Affordances; Robby Costales, Shariq Iqbal, Fei Sha

  • keywords: RL, HRL, reinforcement learning, hierarchical reinforcement learning, affordances, hierarchical affordances

• COptiDICE: Offline Constrained Reinforcement Learning via Stationary Distribution Correction Estimation; Jongmin Lee, Cosmin Paduraru, Daniel J Mankowitz, Nicolas Heess, Doina Precup, Kee-Eung Kim, Arthur Guez

  • keywords: Offline Reinforcement Learning, Offline Constrained Reinforcement Learning, Stationary Distribution Correction Estimation

• SO(2)-Equivariant Reinforcement Learning; Dian Wang, Robin Walters, Robert Platt

  • keywords: Reinforcement Learning, Equivariance, Robotic Manipulation

• Pessimistic Bootstrapping for Uncertainty-Driven Offline Reinforcement Learning; Chenjia Bai, Lingxiao Wang, Zhuoran Yang, Zhi-Hong Deng, Animesh Garg, Peng Liu, Zhaoran Wang

  • keywords: Pessimistic Bootstrapping, Bootstrapped Q-functions, Uncertainty Estimation, Offline Reinforcement Learning

• Learning transferable motor skills with hierarchical latent mixture policies; Dushyant Rao, Fereshteh Sadeghi, Leonard Hasenclever, Markus Wulfmeier, Martina Zambelli, Giulia Vezzani, Dhruva Tirumala, Yusuf Aytar, Josh Merel, Nicolas Heess, raia hadsell

  • keywords: Robotics, Reinforcement Learning, Hierarchical, Latent Variable Models, Skills, Transfer

• C-Planning: An Automatic Curriculum for Learning Goal-Reaching Tasks； Tianjun Zhang, Benjamin Eysenbach, Ruslan Salakhutdinov, Sergey Levine, Joseph E. Gonzalez

  • keywords: reinforcement learning, planning, variational inference, curriculum learning, waypoints, subgoals

• Information Prioritization through Empowerment in Visual Model-based RL; Homanga Bharadhwaj, Mohammad Babaeizadeh, Dumitru Erhan, Sergey Levine

  • keywords: model-based reinforcement learning, visual distractors, empowerment

• Cross-Trajectory Representation Learning for Zero-Shot Generalization in RL;Bogdan Mazoure;Ahmed M Ahmed;R Devon Hjelm;Andrey Kolobov;Patrick MacAlpine

  • key words:reinforcement learning,representation learning,self-supervised learning,procgen,

• Transfer RL across Observation Feature Spaces via Model-Based Regularization;Yanchao Sun;Ruijie Zheng;Xiyao Wang;Andrew E Cohen;Furong Huang

  • key words:transfer reinforcement learning,representation learning,observation space change,latent dynamics model,

• Task-Induced Representation Learning;Jun Yamada;Karl Pertsch;Anisha Gunjal;Joseph J Lim

  • key words:representation learning,reinforcement learning,transfer learning,visually complex observations,
  • 文章针对复杂环境下强化学习的效率

• Distributional Reinforcement Learning with Monotonic Splines;Yudong Luo;Guiliang Liu;Haonan Duan;Oliver Schulte;Pascal Poupart

  • key words:Distributional RL,

• Understanding and Leveraging Overparameterization in Recursive Value Estimation;Chenjun Xiao;Bo Dai;Jincheng Mei;Oscar A Ramirez;Ramki Gummadi;Chris Harris;Dale Schuurmans

  • key words:Temporal Difference Learning,Residual Minimization,Value Estimation,Overparameterization,
  • 思考RL中过参数化问题

• A First-Occupancy Representation for Reinforcement Learning;Ted Moskovitz;Spencer R Wilson;Maneesh Sahani

  • key words:successor representation,successor features,generalized policy improvement,GPI,

• Policy Smoothing for Provably Robust Reinforcement Learning;Aounon Kumar;Alexander Levine;Soheil Feizi

  • key words:Reinforcement Learning,Provable Adversarial Robustness,Randomized Smoothing,

• Causal Contextual Bandits with Targeted Interventions；Chandrasekar Subramanian, Balaraman Ravindran

  • keywords: causality, contextual bandits, causal inference, bandits
  • A new, more realistic, formalism of contextual bandits involving causal side-information and targeted interventions, along with a novel algorithm that exploits features of the new setting such as information leakage to learn good policies quickly.

• Pessimistic Model-based Offline Reinforcement Learning under Partial Coverage；Masatoshi Uehara, Wen Sun

  • keywords: Reinforcement learning Theory, Offline reinforcement learning, PAC Bounds
  • We study model-based offline Reinforcement Learning with general function approximation without a full coverage assumption on the offline data distribution.

• Communication-Efficient Actor-Critic Methods for Homogeneous Markov Games；Dingyang Chen, Yile Li, Qi Zhang

  • keywords: multi-agent reinforcement learning, multi-agent communication

• Spatial Graph Attention and Curiosity-driven Policy for Antiviral Drug Discovery；Yulun Wu, Nicholas Choma, Andrew Deru Chen, Mikaela Cashman, Erica Teixeira Prates, Veronica G Melesse Vergara, Manesh B Shah, Austin Clyde, Thomas Brettin, Wibe Albert de Jong, Neeraj Kumar, Martha S Head, Rick L. Stevens, Peter Nugent, Daniel A Jacobson, James B Brown

  • keywords: reinforcement learning, graph neural network, molecule generation, drug discovery, curiosity-driven policy
  • We developed a reinforcement learning framework that advances in exploiting spatial and attributional molecular information as well as exploring novel and synthesizable chemical structures for the purpose of antiviral drug discovery.

• A Relational Intervention Approach for Unsupervised Dynamics Generalization in Model-Based Reinforcement Learning；Jiaxian_Guo2 , Mingming Gong, Dacheng Tao

  • keywords: Model-Based Reinforcement Learning, Unsupervised Dynamics Generalization
  • This paper proposes a new model-based RL that could generalize to new environments.

• LIGS: Learnable Intrinsic-Reward Generation Selection for Multi-Agent Learning；David Henry Mguni ~David_Henry_Mguni1 , Taher Jafferjee, Jianhong Wang, Nicolas Perez-Nieves, Oliver Slumbers, Feifei Tong, Yang Li, Jiangcheng Zhu, Yaodong Yang, Jun Wang

  • keywords: multi-agent, reinforcement learning, intrinsic rewards, exploration

• HyperDQN: A Randomized Exploration Method for Deep Reinforcement Learning；Ziniu Li ~Ziniu_Li1 , Yingru Li, Yushun Zhang, Tong Zhang, Zhi-Quan Luo

  • keywords: exploration, reinforcement learning
  • We design a practical randomized exploration method to address the sample efficiency issue in online reinforcement learning.

• POETREE: Interpretable Policy Learning with Adaptive Decision Trees; Alizée Pace, Alex Chan, Mihaela van der Schaar

  • keywords: Imitation Learning, Interpretable ML, Clinical Decision Support, Sequential Decision-Making

• Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformers; Ruihan Yang, Minghao Zhang, Nicklas Hansen, Huazhe Xu, Xiaolong Wang

  • keywords: Reinforcement Learning, Robotics, Locomotion Control, Multi-Modal Transformer

• Planning in Stochastic Environments with a Learned Model; Ioannis Antonoglou, Julian Schrittwieser, Sherjil Ozair, Thomas K Hubert, David Silver

  • keywords: model-based reinforcement learning, deep reinforcement learning, tree based search, MCTS

• AdaRL: What, Where, and How to Adapt in Transfer Reinforcement Learning; Biwei Huang, Fan Feng, Chaochao Lu, Sara Magliacane, Kun Zhang

  • keywords: Transfer RL, Graphical models, Efficient adaptation

• R5: Rule Discovery with Reinforced and Recurrent Relational Reasoning; Shengyao Lu, Bang Liu, Keith G Mills, SHANGLING JUI, Di Niu

  • keywords: systematicity, graph reasoning

• Possibility Before Utility: Learning And Using Hierarchical Affordances; Robby Costales, Shariq Iqbal, Fei Sha

  • keywords: RL, HRL, reinforcement learning, hierarchical reinforcement learning, affordances, hierarchical affordances

• COptiDICE: Offline Constrained Reinforcement Learning via Stationary Distribution Correction Estimation; Jongmin Lee, Cosmin Paduraru, Daniel J Mankowitz, Nicolas Heess, Doina Precup, Kee-Eung Kim, Arthur Guez

  • keywords: Offline Reinforcement Learning, Offline Constrained Reinforcement Learning, Stationary Distribution Correction Estimation

• SO(2)-Equivariant Reinforcement Learning; Dian Wang, Robin Walters, Robert Platt

  • keywords: Reinforcement Learning, Equivariance, Robotic Manipulation

• Pessimistic Bootstrapping for Uncertainty-Driven Offline Reinforcement Learning; Chenjia Bai, Lingxiao Wang, Zhuoran Yang, Zhi-Hong Deng, Animesh Garg, Peng Liu, Zhaoran Wang

  • keywords: Pessimistic Bootstrapping, Bootstrapped Q-functions, Uncertainty Estimation, Offline Reinforcement Learning

• Learning transferable motor skills with hierarchical latent mixture policies; Dushyant Rao, Fereshteh Sadeghi, Leonard Hasenclever, Markus Wulfmeier, Martina Zambelli, Giulia Vezzani, Dhruva Tirumala, Yusuf Aytar, Josh Merel, Nicolas Heess, raia hadsell

  • keywords: Robotics, Reinforcement Learning, Hierarchical, Latent Variable Models, Skills, Transfer

• C-Planning: An Automatic Curriculum for Learning Goal-Reaching Tasks； Tianjun Zhang, Benjamin Eysenbach, Ruslan Salakhutdinov, Sergey Levine, Joseph E. Gonzalez

  • keywords: reinforcement learning, planning, variational inference, curriculum learning, waypoints, subgoals

• Information Prioritization through Empowerment in Visual Model-based RL; Homanga Bharadhwaj, Mohammad Babaeizadeh, Dumitru Erhan, Sergey Levine

  • keywords: model-based reinforcement learning, visual distractors, empowerment

• Cross-Trajectory Representation Learning for Zero-Shot Generalization in RL;Bogdan Mazoure;Ahmed M Ahmed;R Devon Hjelm;Andrey Kolobov;Patrick MacAlpine

  • key words:reinforcement learning,representation learning,self-supervised learning,procgen,

• Transfer RL across Observation Feature Spaces via Model-Based Regularization;Yanchao Sun;Ruijie Zheng;Xiyao Wang;Andrew E Cohen;Furong Huang

  • key words:transfer reinforcement learning,representation learning,observation space change,latent dynamics model,

• Task-Induced Representation Learning;Jun Yamada;Karl Pertsch;Anisha Gunjal;Joseph J Lim

  • key words:representation learning,reinforcement learning,transfer learning,visually complex observations,
  • 文章针对复杂环境下强化学习的效率

• Distributional Reinforcement Learning with Monotonic Splines;Yudong Luo;Guiliang Liu;Haonan Duan;Oliver Schulte;Pascal Poupart

  • key words:Distributional RL,

• Understanding and Leveraging Overparameterization in Recursive Value Estimation;Chenjun Xiao;Bo Dai;Jincheng Mei;Oscar A Ramirez;Ramki Gummadi;Chris Harris;Dale Schuurmans

  • key words:Temporal Difference Learning,Residual Minimization,Value Estimation,Overparameterization,
  • 思考RL中过参数化问题

• A First-Occupancy Representation for Reinforcement Learning;Ted Moskovitz;Spencer R Wilson;Maneesh Sahani

  • key words:successor representation,successor features,generalized policy improvement,GPI,

• Policy Smoothing for Provably Robust Reinforcement Learning;Aounon Kumar;Alexander Levine;Soheil Feizi

  • key words:Reinforcement Learning,Provable Adversarial Robustness,Randomized Smoothing,

• RvS: What is Essential for Offline RL via Supervised Learning?;Scott Emmons;Benjamin Eysenbach;Ilya Kostrikov;Sergey Levine key words:reinforcement learning,deep reinforcement learning,offline reinforcement learning,

• GPT-Critic: Offline Reinforcement Learning for End-to-End Task-Oriented Dialogue Systems;Youngsoo Jang;Jongmin Lee;Kee-Eung Kim key words:task-oriented dialogue,pre-trained language model,offline reinforcement learning

• An Experimental Design Perspective on Model-Based Reinforcement Learning;Viraj Mehta;Biswajit Paria;Jeff Schneider;Stefano Ermon;Willie Neiswanger key words:reinforcement learning,acquisition function,information gain,

• Case-based reasoning for better generalization in textual reinforcement learning;Mattia Atzeni;Shehzaad Zuzar Dhuliawala;Keerthiram Murugesan;Mrinmaya Sachan

  ​ key words:,

• CrowdPlay: Crowdsourcing Human Demonstrations for Offline Learning;Matthias Gerstgrasser;Rakshit Trivedi;David C. Parkes key words:,

• Know Thyself: Transferable Visual Control Policies Through Robot-Awareness;Edward S. Hu;Kun Huang;Oleh Rybkin;Dinesh Jayaraman key words:visual foresight,dynamics models,visuomotor control,video prediction,planning,transfer,

• Offline Reinforcement Learning with Implicit Q-Learning;Ilya Kostrikov;Ashvin Nair;Sergey Levine key words:Deep Reinforcement Learning,Offline Reinforcement Learning,Batch Reinforcement Learning,Continuous Control,

• Online Ad Hoc Teamwork under Partial Observability;Pengjie Gu;Mengchen Zhao;Jianye Hao;Bo An key words:coordination,reinforcement learning,

• Offline Reinforcement Learning with Value-based Episodic Memory;Xiaoteng Ma;Yiqin Yang;Hao Hu;Jun Yang;Chongjie Zhang;Qianchuan Zhao;Bin Liang;Qihan Liu key words:Reinforcement Learning,Offline Learning,Episodic Memory Control,

• Overcoming The Spectral Bias of Neural Value Approximation;Ge Yang;Anurag Ajay;Pulkit Agrawal key words:spectral bias,neural value approximation,Q learning,reinforcement learning,neural tangent kernels,kernel regression,

• Offline Neural Contextual Bandits: Pessimism, Optimization and Generalization;Thanh Nguyen-Tang;Sunil Gupta;A. Tuan Nguyen;Svetha Venkatesh key words:offline policy learning,offline contextual bandits,neural network function approximation,

• Online Target Q-learning with Reverse Experience Replay: Efficiently finding the Optimal Policy for Linear MDPs;Naman Agarwal;Syomantak Chaudhuri;Prateek Jain;Dheeraj Mysore Nagaraj;Praneeth Netrapalli key words:Q Learning,RL with Function Approximation,Experience Replay,Online Target Learning,

• Who Is the Strongest Enemy? Towards Optimal and Efficient Evasion Attacks in Deep RL;Yanchao Sun;Ruijie Zheng;Yongyuan Liang;Furong Huang key words:adversarial RL,robustness of RL,evasion attack,optimal attack,observation perturbation,

• Anti-Concentrated Confidence Bonuses For Scalable Exploration;Jordan T. Ash;Cyril Zhang;Surbhi Goel;Akshay Krishnamurthy;Sham M. Kakade key words:deep reinforcement learning,reinforcement learning,bandits,exploration,

• Continuously Discovering Novel Strategies via Reward-Switching Policy Optimization;Zihan Zhou;Wei Fu;Bingliang Zhang;Yi Wu key words:diverse behavior,deep reinforcement learning,multi-agent reinforcement learning,

• Multi-Critic Actor Learning: Teaching RL Policies to Act with Style;Siddharth Mysore;George Cheng;Yunqi Zhao;Kate Saenko;Meng Wu key words:Reinforcement Learning,Multi-Style Learning,Multi-Task Learning,Actor-Critic,

• Reward Uncertainty for Exploration in Preference-based Reinforcement Learning;Xinran Liang;Katherine Shu;Kimin Lee;Pieter Abbeel key words:,

• Trust Region Policy Optimisation in Multi-Agent Reinforcement Learning;Jakub Grudzien Kuba;Ruiqing Chen;Muning Wen;Ying Wen;Fanglei Sun;Jun Wang;Yaodong Yang key words:Multi-Agent Reinforcement Learning,trust-region method,policy gradient method,

其他方面的推荐

• Bridging Recommendation and Marketing via Recurrent Intensity Modeling；Yifei Ma, Ge Liu, Anoop Deoras
  • keywords: Recommender systems, marketing, push notifications, temporal point processes, sequence models
  • 我们将项目推荐系统的目的重新定义为向项目提供者推荐用户，以达到内容推广和多样性的目的。

搜索

• ARTEMIS: Attention-based Retrieval with Text-Explicit Matching and Implicit Similarity；Ginger Delmas, Rafael S. Rezende, Gabriela Csurka, Diane Larlus
  • keywords: we exploit the specific relation of each query element with the targeted image and derive light-weight attention mechanisms which enable to mediate between the two complementary modalities.

模型可解释性

• FastSHAP: Real-Time Shapley Value Estimation；Neil Jethani, Mukund Sudarshan, Ian Connick Covert, Su-In Lee, Rajesh Ranganath
  • keywords: interpretability, shapley, amortization, explainability, game theory
  • We introduce FastSHAP, a new method for estimating Shapley values in a single forward pass using an explainer model that is learned via stochastic gradient optimization using a weighted least squares-like objective function.
• DISCOVERING AND EXPLAINING THE REPRESENTATION BOTTLENECK OF DNNS；Huiqi Deng, Qihan Ren, Hao Zhang, Quanshi Zhang
  • keywords: representation bottleneck, representation ability, interaction, explanation
• DISSECT: Disentangled Simultaneous Explanations via Concept Traversals；Asma Ghandeharioun, Been Kim, Chun-Liang Li, Brendan Jou, Brian Eoff, Rosalind Picard
  • keywords: Explainability, Interpretability, Counterfactual generation, Generative Adversarial Network, Variational Autoencoder
  • We propose a novel counterfactual explainability method that simultaneously satisfies several desirable qualities where other methods fail by training a generator, a discriminator, and a concept disentangler using the classifier’s signal.
• Fooling Explanations in Text Classifiers；Adam Ivankay, Ivan Girardi, Chiara Marchiori, Pascal Frossard
  • keywords: robustness, explainability, text classification, natural language processing
  • Our work shows that explanation methods in text classifiers are susceptible to imperceptible perturbations that alter the explanation outcomes without changing the predictions of the classifiers.
• Explanations of Black-Box Models based on Directional Feature Interactions; Aria Masoomi, Davin Hill, Zhonghui Xu, Craig P Hersh, Edwin K. Silverman, Peter J. Castaldi, Stratis Ioannidis, Jennifer Dy
• keywords: Explainability, Shapley values, Interpretability, Directional interaction, feature interaction
• Bridging the Gap: Providing Post-Hoc Symbolic Explanations for Sequential Decision-Making Problems with Inscrutable Representations;Sarath Sreedharan;Utkarsh Soni;Mudit Verma;Siddharth Srivastava;Subbarao Kambhampati
  • key words:Explanations,XAI,Post-hoc explanations,
• A Zest of LIME: Towards Architecture-Independent Model Distances;Hengrui Jia;Hongyu Chen;Jonas Guan;Ali Shahin Shamsabadi;Nicolas Papernot
  • key words:model distance,model stealing,machine unlearning,fairwashing,
  • 研究模型之间的相关性问题
• Recursive Disentanglement Network;Yixuan Chen;Yubin Shi;Dongsheng Li;Yujiang Wang;Mingzhi Dong;Yingying Zhao;Robert Dick;Qin Lv;Fan Yang;Li Shang
  • key words:disentanglement,representation learning,compositional,
  • 从信息论角度思考深层模型的特征空间组合
• Deep ReLU Networks Preserve Expected Length;Boris Hanin;Ryan S Jeong;David Rolnick
  • key words:deep learning theory,random ReLU networks,length distortion,initialization,expressivity,
  • 神经网络的ReLU思考
• Modeling Label Space Interactions in Multi-label Classification using Box Embeddings;Dhruvesh Patel;Pavitra Dangati;Jay-Yoon Lee;Michael Boratko;Andrew McCallum
  • key words:Multi-label classification,Box Embeddings,Representation Learning,Embeddings,
  • 将神经网络的编码能力与盒嵌入的归纳偏差和概率语义相结合
• Stochastic Training is Not Necessary for Generalization;Jonas Geiping;Micah Goldblum;Phil Pope;Michael Moeller;Tom Goldstein
  • key words:Optimization,Generalization,Stochasticity,SGD,full-batch,implicit regularization,implicit bias,
• Resonance in Weight Space: Covariate Shift Can Drive Divergence of SGD with Momentum;Kirby Banman;Garnet Liam Peet-Pare;Nidhi Hegde;Alona Fyshe;Martha White
  • key words:optimization,momentum,stochastic gradient descent,non-iid sampling,
  • SGD 训练不稳定思考
• Fooling Explanations in Text Classifiers;Adam Ivankay, Ivan Girardi, Chiara Marchiori, Pascal Frossard
  • key words:robustness, explainability, text classification, natural language processing
  • Our work shows that explanation methods in text classifiers are susceptible to imperceptible perturbations that alter the explanation outcomes without changing the predictions of the classifiers.
• Symbolic Learning to Optimize: Towards Interpretability and Scalability;Wenqing Zheng;Tianlong Chen;Ting-Kuei Hu;Zhangyang Wang
  • key words:Symbolic Regression,Learning To Optimize,Interpretability,

其他

• Understanding the Variance Collapse of SVGD in High Dimensions；Jimmy Ba, Murat A Erdogdu, Marzyeh Ghassemi, Shengyang Sun, Taiji Suzuki, Denny Wu, Tianzong Zhang

  • keywords: Stein Variational Gradient Descent,Approximate Inference, Particle-based Variational Inference

• Discriminative Similarity for Data Clustering；Yingzhen Yang, Ping Li

  • keywords: Discriminative Similarity, Rademacher Complexity, Generalization Bound, Data Clustering

• Generalization of Neural Combinatorial Solvers Through the Lens of Adversarial Robustness；Simon Geisler, Johanna Sommer, Jan Schuchardt, Aleksandar Bojchevski, Stephan Günnemann

  • keywords: Generalization, Neural Combinatorial Optimization, Adversarial Robustness

• Constructing Orthogonal Convolutions in an Explicit Manner；Tan Yu, Jun Li, YUNFENG CAI, Ping Li

  • keywords: orthogonal convolution

• Concurrent Adversarial Learning for Large-Batch Training；Yong Liu, Xiangning Chen, Minhao Cheng, Cho-Jui Hsieh, Yang Youi

  • keywords: Distributed Machine Learnig, Large-Batch Training, Adversarial Learning

• Transformer Embeddings of Irregularly Spaced Events and Their Participants；Hongyuan Mei, Chenghao Yang, Jason Eisner

  • keywords: irregular time series, generative Transformers, neuro-symbolic architectures, logic programming

• Self-Joint Supervised Learnin；Navid Kardan ~Navid_Kardan1 , Mubarak Shah, Mitch Hill

  • keywords: Supervised learning， i.i.d. assumption，
  • 模型明确地学习了条件独立性的样本与样本之间的关系，而不是假设样本是独立的。

• Minimax Optimization with Smooth Algorithmic Adversaries；Tanner Fiez, Chi Jin, Praneeth Netrapalli, Lillian J Ratliff

  • keywords: Minimax optimization, two player zero sum games, generative adversarial networks, adversarial training

• Dual Lottery Ticket Hypothesis；Yue Bai, Huan Wang, ZHIQIANG TAO, Kunpeng Li, Yun Fuf

  • keywords: Dual Lottery Ticket Hypothesis, Sparse Network Training
  • 我们提出了一个双彩票假设(DLTH)和一个训练随机稀疏网络策略来验证DLTH。

• Incremental False Negative Detection for Contrastive Learning；Tsai-Shien Chen, Wei-Chih Hung, Hung-Yu Tseng, Shao-Yi Chien, Ming-Hsuan Yang

  • keywords: Self-supervised learning, Contrastive learning, Representation learning, Clustering-based learning
  • 本文探讨了假阴性样本在自我监督对比学习中的作用，并引入了一个增量检测和显式删除假阴性样本的框架

• On the Convergence of Certified Robust Training with Interval Bound Propagation；Yihan Wang, Zhouxing Shi, Quanquan Gu, Cho-Jui Hsieh

  • keywords: Certified robustness, Adversarial robustness, Convergence
  • We present the first theoretical analysis on the convergence of certified robust training with interval bound propagation.

• Particle Stochastic Dual Coordinate Ascent: Exponential convergent algorithm for mean field neural network optimization；Kazusato Oko, Taiji Suzuki, Atsushi Nitanda, Denny Wu

  • keywords: Neural Network Optimization, Mean field Regime, Overparameterization
  • Proposed a new algorithm for optimizing two-layer neural network in the mean field regime that achieves exponential convergence in regularized empirical risk minimization (w.r.t. outer loop iterations).

• How many degrees of freedom do we need to train deep networks: a loss landscape perspective；Brett W Larsen ~Brett_W_Larsen1 , Stanislav Fort, Nic Becker, Surya Ganguli

  • keywords: loss landscape, high-dimensional geometry, random hyperplanes, optimization

• Monotonic Differentiable Sorting Networks；Felix Petersen, Christian Borgelt, Hilde Kuehne, Oliver Deussen

  • keywords: differentiable sorting, monotonic, sorting, ranking, sorting networks

• Comparing Distributions by Measuring Differences that Affect Decision Making；Shengjia Zhao, Abhishek Sinha, Yutong He, Aidan Perreault, Jiaming Song, Stefano Ermon

• keywords: probability divergence, two sample test, generative model

• 这篇文章提出了一种新的不同数据分布差异性的测量方法

• Bootstrapped Meta-Learning；Sebastian Flennerhag, Yannick Schroecker, Tom Zahavy, Hado van Hasselt, David Silver, Satinder Singh

  • keywords: meta-learning, meta-gradients, meta-reinforcement learning
  • We propose an algorithm for meta-learning with gradients that bootstraps the meta-learner from itself or another update rule.

• Minibatch vs Local SGD with Shuffling: Tight Convergence Bounds and Beyond；Chulhee Yun, Shashank Rajput, Suvrit Sra

  • keywords: Local SGD, Minibatch SGD, Shuffling, Without-replacement, Convex Optimization, Stochastic Optimization, Federated Learning, Large Scale Learning, Distributed Learning
  • We provide tight upper and lower bounds on convergence rates of shuffling-based minibatch SGD and local SGD, and propose an algorithmic modification that improves convergence rates beyond our lower bounds.

• The Hidden Convex Optimization Landscape of Regularized Two-Layer ReLU Networks: an Exact Characterization of Optimal Solutions；Yifei Wang, Jonathan Lacotte, Mert Pilanci

  • keywords: Neural networks, global optimization, convex optimization, convex analysis
  • We prove that finding all globally optimal two-layer ReLU neural networks can be performed by solving a convex optimization program with cone constraints.

• Meta-Learning with Fewer Tasks through Task Interpolation；Huaxiu Yao, Linjun Zhang, Chelsea Finn

  • keywords: meta-learning, task interpolation, meta-regularization
  • A new framework to densify the task distribution via task interpolation.

• Hyperparameter Tuning with Renyi Differential Privacy；Nicolas Papernot, Thomas Steinke

  • keywords: differential privacy, hyperparameter tuning
  • We provide privacy guarantees for hyperparameter search procedures, showing that tuning hyperparameters leaks private information, but that, under certain assumptions, this leakage is modest.

• Robust Unlearnable Examples: Protecting Data Privacy Against Adversarial Learning；Shaopeng Fu, Fengxiang He, Yang Liu, Li Shen, Dacheng Tao

  • keywords: unlearnable examples, adversarial training, privacy
  • This paper proposes an robust error-minimizing noise that can protect data from being learned under adversarial training.

• Demystifying Batch Normalization in ReLU Networks: Equivalent Convex Optimization Models and Implicit Regularization；Tolga Ergen, Arda Sahiner, Batu Ozturkler, John M. Pauly, Morteza Mardani, Mert Pilanci

  • keywords: batch normalization, ReLU networks, deep networks, convex optimization, whitening, implicit regularization, algorithmic bias
  • We introduce an analytic framework based on convex duality to obtain exact and polynomial-time trainable convex representations of weight-decay regularized ReLU networks with BN.

• Prospect Pruning: Finding Trainable Weights at Initialization using Meta-Gradients；Milad Alizadeh, Shyam A. Tailor, Luisa M Zintgraf, Joost van Amersfoort, Sebastian Farquhar, Nicholas Donald Lane, Yarin Gal

  • keywords: pruning, lottery ticket hypothesis, pruning at initialization
  • We use meta-gradients to prune neural networks at initialization based on "trainability" of weights instead of their impact on the loss at a single step.

• On the Convergence of mSGD and AdaGrad for Stochastic Optimization；ruinan Jin, Yu Xing, Xingkang He

  • keywords: stochastic gradient descent, adaptive gradient algorithm, asymptotic convergence
  • A theoretical paper focusing on the investigation for the convergence of mSGD and AdaGrad optimization algorithms.

• Continual Normalization: Rethinking Batch Normalization for Online Continual Learning；Quang Pham, Chenghao Liu, Steven HOI

  • keywords: ontinual Learning, Batch Normalization
  • A negative effect of BN in online continual learning and a simple strategy to alleviate it.

• Label Encoding for Regression Networks Deval Shah, Zi Yu Xue, Tor Aamodt

  • keywords: Regression, Label encoding, Output codes

• Properties from mechanisms: an equivariance perspective on identifiable representation learning Kartik Ahuja, Jason Hartford, Yoshua Bengio

  • keywords: representation learning, equivariance, independent component analysis, ICA, autoencoders

• On the relation between statistical learning and perceptual distances Alexander Hepburn, Valero Laparra, Raul Santos-Rodriguez, Johannes Ballé, Jesus Malo

• VAE Approximation Error: ELBO and Exponential Families Alexander Shekhovtsov, Dmitrij Schlesinger, Boris Flach

• SOSP: Efficiently Capturing Global Correlations by Second-Order Structured Pruning Manuel Nonnenmacher, Thomas Pfeil, Ingo Steinwart, David Reeb

  • keywords: Structured Pruning, Saliency-based Pruning, Network Compression, Hessian Approximation, Neural Architecture Search, Deep Learning, Computer Vision

• TRGP: Trust Region Gradient Projection for Continual Learning Sen Lin, Li Yang, Deliang Fan, Junshan Zhang

  • keywords: trust region, gradient projection, scaled weight projection, continual learning, forward knowledge transfer, task correlation

• Learning Pruning-Friendly Networks via Frank-Wolfe: One-Shot, Any-Sparsity, And No Retraining Lu Miao, Xiaolong Luo, Tianlong Chen, Wuyang Chen, Dong Liu, Zhangyang Wang

  • keywords: Pruning, Frank-Wolfe

• Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh, Albert Webson, Colin Raffel, Stephen Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal Nayak, Debajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong, Harshit Pandey, Rachel Bawden, Thomas Wang, Trishala Neeraj, Jos Rozen, Abheesht Sharma, Andrea Santilli, Thibault Fevry, Jason Alan Fries, Ryan Teehan, Teven Le Scao, Stella Biderman, Leo Gao, Thomas Wolf, Alexander M Rush

• Independent SE(3)-Equivariant Models for End-to-End Rigid Protein Docking Octavian-Eugen Ganea, Xinyuan Huang, Charlotte Bunne, Yatao Bian, Regina Barzilay, Tommi S. Jaakkola, Andreas Krause

  • keywords: protein complexes, protein structure, rigid body docking, SE(3) equivariance, graph neural networks

• Progressive Distillation for Fast Sampling of Diffusion Models Tim Salimans, Jonathan Ho

  • keywords: Diffusion Models, Generative Models, fast sampling

• IntSGD: Adaptive Floatless Compression of Stochastic Gradients Konstantin Mishchenko, Bokun Wang, Dmitry Kovalev, Peter Richtárik

  • keywords: optimization, distributed optimization, compression, theory, parallel training, switchML

• RelaxLoss: Defending Membership Inference Attacks without Losing Utility Dingfan Chen, Ning Yu, Mario Fritz

  • keywords: membership inference attack, defense

• Continual Learning with Recursive Gradient Optimization Hao Liu, Huaping Liu

  • keywords: continual learning, lifelong learning

• Responsible Disclosure of Generative Models Using Scalable Fingerprinting Ning Yu, Vladislav Skripniuk, Dingfan Chen, Larry S. Davis, Mario Fritz

  • keywords: Generative models, fingerprinting, responsible disclosure, deep fake detection and attribution

• Online Hyperparameter Meta-Learning with Hypergradient Distillation Hae Beom Lee, Hayeon Lee, JaeWoong Shin, Eunho Yang, Timothy Hospedales, Sung Ju Hwang

  • keywords: Hyperparameter Optimization, Meta-learning

• DEPTS: Deep Expansion Learning for Periodic Time Series Forecasting Wei Fan, Shun Zheng, Xiaohan Yi, Wei Cao, Yanjie Fu, Jiang Bian, Tie-Yan Liu

• On the Connection between Local Attention and Dynamic Depth-wise Convolution Qi Han, Zejia Fan, Qi Dai, Lei Sun, Ming-Ming Cheng, Jiaying Liu, Jingdong Wang

  • keywords: local attention, depth-wise convolution, dynamic depth-wise convolution, weight sharing, dynamic weight

• Understanding Latent Correlation-Based Multiview Learning and Self-Supervision: An Identifiability Perspective Qi Lyu, Xiao Fu, Weiran Wang, Songtao Lu

• Scaling Laws for Neural Machine Translation Behrooz Ghorbani, Orhan Firat, Markus Freitag, Ankur Bapna, Maxim Krikun, Xavier Garcia, Ciprian Chelba, Colin Cherry

  • keywords: Scaling Laws, Neural Machine Translation, NMT, Model Scaling

• Latent Variable Sequential Set Transformers for Joint Multi-Agent Motion Prediction Roger Girgis, Florian Golemo, Felipe Codevilla, Martin Weiss, Jim Aldon D'Souza, Samira Ebrahimi Kahou, Felix Heide, Christopher Pal

  • keywords: trajectory prediction, motion forecasting, transformers, latent variable models

• Meta Discovery: Learning to Discover Novel Classes given Very Limited Data Haoang Chi, Feng Liu, Wenjing Yang, Long Lan, Tongliang Liu, Bo Han, Gang Niu, Mingyuan Zhou, Masashi Sugiyama

• Tackling the Generative Learning Trilemma with Denoising Diffusion GANs Zhisheng Xiao, Karsten Kreis, Arash Vahdat

• Half-Inverse Gradients for Physical Deep Learning Patrick Schnell, Philipp Holl, Nils Thuerey

  • keywords: physical simulation, partial differential equations, physical loss functions, optimization

• How to Robustify Black-Box ML Models? A Zeroth-Order Optimization Perspective Yimeng Zhang, Yuguang Yao, Jinghan Jia, Jinfeng Yi, Mingyi Hong, Shiyu Chang, Sijia Liu

  • keywords: Zeroth-Order Optimization, Black-Box Defense, Gradient-Free, Adversarial Robustness, Certified Defense

• Learning meta-features for AutoML Herilalaina Rakotoarison, Louisot Milijaona, Andry RASOANAIVO, Michele Sebag, Marc Schoenauer

  • keywords: AutoML, Meta-features, Hyper-parameter Optimization, Optimal Transport

• Learning more skills through optimistic exploration DJ Strouse, Kate Baumli, David Warde-Farley, Volodymyr Mnih, Steven Stenberg Hansen

  • keywords: intrinsic control, skill discovery, unsupervised skill learning, uncertainty estimation, optimistic exploration, variational information maximization

• Learning the Dynamics of Physical Systems from Sparse Observations with Finite Element Networks Marten Lienen, Stephan Günnemann

  • keywords: spatio-temporal, finite, elements, forecasting, continuous, partial, differential, equation, PDE, graph, gnn, time-series

• Scalable Sampling for Nonsymmetric Determinantal Point Processes Insu Han, Mike Gartrell, Jennifer Gillenwater, Elvis Dohmatob, amin karbasi

  • keywords: determinantal point processes, sampling

• On the Optimal Memorization Power of ReLU Neural Networks Gal Vardi, Gilad Yehudai, Ohad Shamir

  • keywords: Expressivness, Memorization, Theory, VC-dimension, Deep learning theory

• Churn Reduction via Distillation Heinrich Jiang, Harikrishna Narasimhan, Dara Bahri, Andrew Cotter, Afshin Rostamizadeh

  • keywords: distillation, churn, constraints

• Path Auxiliary Proposal for MCMC in Discrete Space Haoran Sun, Hanjun Dai, Wei Xia, Arun Ramamurthy

• Learning Prototype-oriented Set Representations for Meta-Learning Dan dan Guo, Long Tian, Minghe Zhang, Mingyuan Zhou, Hongyuan Zha

  • keywords: Summary Networks, Distribution Matching, Optimal Transport, Few-shot Classification, Meta Generative Models

• Safe Neurosymbolic Learning with Differentiable Symbolic Execution Chenxi Yang, Swarat Chaudhuri

  • keywords: Verified Learning, Neurosymbolic Programs, Safe Learning, Symbolic Execution

• CodeTrek: Flexible Modeling of Code using an Extensible Relational Representation Pardis Pashakhanloo, Aaditya Naik, Yuepeng Wang, Hanjun Dai, Petros Maniatis, Mayur Naik

  • keywords: relational database, code representation, knowledge graph reasoning, program understanding

• Peek-a-Boo: What (More) is Disguised in a Randomly Weighted Neural Network, and How to Find It Efficiently Xiaohan Chen, Jason Zhang, Zhangyang Wang

  • keywords: Sparse Neural Network, Lottery Ticket Hypothesis, Efficient Machine Leanring

• Minimax Optimality (Probably) Doesn't Imply Distribution Learning for GANs Sitan Chen, Jerry Li, Yuanzhi Li, Raghu Meka

  • keywords: theory of GANs, distribution learning, pseudorandom generators, cryptography

• Image BERT Pre-training with Online Tokenizer Jinghao Zhou, Chen Wei, Huiyu Wang, Wei Shen, Cihang Xie, Alan Yuille, Tao Kong

  • keywords: online tokenizer, masked image modeling, vision transformer

• Neural Networks as Kernel Learners: The Silent Alignment Effect Alexander Atanasov, Blake Bordelon, Cengiz Pehlevan

  • keywords: Neural Tangent Kernel, Feature Learning, Inductive Bias of Neural Networks

• Declarative nets that are equilibrium models Russell Tsuchida, Suk Yee Yong, Mohammad Ali Armin, Lars Petersson, Cheng Soon Ong

  • keywords: deep equilibrium models, deep declarative networks, implicit layers, kernel methods, generalised linear models

• Towards General Function Approximation in Zero-Sum Markov Games Baihe Huang, Jason D. Lee, Zhaoran Wang, Zhuoran Yang

• MetaMorph: Learning Universal Controllers with Transformers Agrim Gupta, Linxi Fan, Surya Ganguli, Li Fei-Fei

  • keywords: RL, Modular Robots, Transformers

• Distilling GANs with Style-Mixed Triplets for X2I Translation with Limited Data Yaxing Wang, Joost van de weijer, Lu Yu, SHANGLING JUI

  • keywords: Transfer learning, image synthesis, limited data

• Multimeasurement Generative Models Saeed Saremi, Rupesh Kumar Srivastava

  • keywords: energy based models, Langevin MCMC, score matching, denoising autoencoders, empirical Bayes

• Hindsight is 20/20: Leveraging Past Traversals to Aid 3D Perception Yurong You, Katie Z Luo, Xiangyu Chen, Junan Chen, Wei-Lun Chao, Wen Sun, Bharath Hariharan, Mark Campbell, Kilian Q Weinberger

  • keywords: 3D object detection, perception with historical context

• Environment Predictive Coding for Visual Navigation Santhosh Kumar Ramakrishnan, Tushar Nagarajan, Ziad Al-Halah, Kristen Grauman

  • keywords: Self-supervised learning, visual navigation, representation learning

• Equivariant Self-Supervised Learning: Encouraging Equivariance in Representations Rumen Dangovski, Li Jing, Charlotte Loh, Seungwook Han, Akash Srivastava, Brian Cheung, Pulkit Agrawal, Marin Soljacic

  • keywords: self-supervised learning, contrastive learning, photonics science

• Sparse Attention with Learning to Hash Zhiqing Sun, Yiming Yang, Shinjae Yoo

  • keywords: Sparse Attention, Transformer, Learning-to-Hash, Natural Language Processing

• Non-Linear Operator Approximations for Initial Value Problems Gaurav Gupta, Xiongye Xiao, Radu Balan, Paul Bogdan

  • keywords: exponential operators, initial value problem, pade approximation, multiwavelets, partial differential equations

• Doubly Adaptive Scaled Algorithm for Machine Learning Using Second-Order Information Majid Jahani, Sergey Rusakov, Zheng Shi, Peter Richtárik, Michael W. Mahoney, Martin Takac

  • keywords: Convex Optimization, Non-Convex Optimization, Stochastic Optimization, Second-Order Optimization, Deep Learning

• Implicit Bias of MSE Gradient Optimization in Underparameterized Neural Networks Benjamin Bowman, Guido Montufar

  • keywords: underparameterized regime, spectral bias, neural tangent kernel, implicit bias, implicit regularization, gradient flow

• A Loss Curvature Perspective on Training Instabilities of Deep Learning Models Justin Gilmer, Behrooz Ghorbani, Ankush Garg, Sneha Kudugunta, Behnam Neyshabur, David Cardoze, George Edward Dahl, Zachary Nado, Orhan Firat

  • keywords: Optimization, Deep Learning, Training Instability, Curvature, Loss Landscape, Hessian

• VC dimension of partially quantized neural networks in the overparametrized regime Yutong Wang, Clayton Scott

  • keywords: VC dimension, quantized neural networks, classification, minimax theory, overparametrization

• Generalized Natural Gradient Flows in Hidden Convex-Concave Games and GANs Andjela Mladenovic, Iosif Sakos, Gauthier Gidel, Georgios Piliouras

• A fast and accurate splitting method for optimal transport: analysis and implementation Vien V. Mai, Jacob Lindbäck, Mikael Johansson

  • keywords: Optimal transport, Operator splitting, Douglas-Rachford, ADMM, GPUs

• Learning to Downsample for Segmentation of Ultra-High Resolution Images Chen Jin, Ryutaro Tanno, Thomy Mertzanidou, Eleftheria Panagiotaki, Daniel C. Alexander

  • keywords: ultra-high resolution image segmentation, non-uniform dowmsampling, efficient segmentation, large volume image segmentation, medical image segmentation

• StyleNeRF: A Style-based 3D Aware Generator for High-resolution Image Synthesis Jiatao Gu, Lingjie Liu, Peng Wang, Christian Theobalt

  • keywords: Neural Radiance Field, StyleGAN, high resolution image generation

• NAS-Bench-Suite: NAS Evaluation is (Now) Surprisingly Easy Yash Mehta, Colin White, Arber Zela, Arjun Krishnakumar, Guri Zabergja, Shakiba Moradian, Mahmoud Safari, Kaicheng Yu, Frank Hutter

  • keywords: neural architecture search, AutoML

• Generative Models as a Data Source for Multiview Representation Learning;Ali Jahanian;Xavier Puig;Yonglong Tian;Phillip Isola

  • key words:Generative models,GANs,Contrastive Learning,Representation Learning,
  • 对比和生成模型的结合

• LOSSY COMPRESSION WITH DISTRIBUTION SHIFT AS ENTROPY CONSTRAINED OPTIMAL TRANSPORT;Huan Liu;George Zhang;Jun Chen;Ashish J Khisti

  • key words:Image Compression,Image Restoration,Optimal Transport,Deep Learning,
  • 关于分布偏移中熵相关思考

• A Fine-Tuning Approach to Belief State Modeling;Samuel Sokota;Hengyuan Hu;David J Wu;J Zico Kolter;Jakob Nicolaus Foerster;Noam Brown

  • key words:imperfect-information,partial observability,search,decision-time planning,

• Augmented Sliced Wasserstein Distances;Xiongjie Chen;Yongxin Yang;Yunpeng Li

  • key words: Wasserstein Distances
  • 本文主要是通过对推土机距离的计算效率分析，从超曲面上思考其实际应用

• Neural Processes with Stochastic Attention: Paying more attention to the context dataset;Mingyu Kim;Kyeong Ryeol Go;Se-Young Yun

  • key words:neural processes,stochastic attention,variational inference,information theory,

• Unsupervised Learning of Full-Waveform Inversion: Connecting CNN and Partial Differential Equation in a Loop;Peng Jin;Xitong Zhang;Yinpeng Chen;Sharon X Huang;Zicheng Liu;Youzuo Lin

  • key words:Unsupervised Learning,Full-Waveform Inversion,Partial Differential Equation,Physics-Informed Machine Learning,

• Diurnal or Nocturnal? Federated Learning of Multi-branch Networks from Periodically Shifting Distributions;Chen Zhu;Zheng Xu;Mingqing Chen;Jakub Konečný;Andrew Hard;Tom Goldstein

  • key words:Federated Learning,Peroredical Distribution Shift,

• Diverse Client Selection for Federated Learning via Submodular Maximization;Ravikumar Balakrishnan;Tian Li;Tianyi Zhou;Nageen Himayat;Virginia Smith;Jeff Bilmes

  • key words:federated learning,submodularity,diversity,

• R4D: Utilizing Reference Objects for Long-Range Distance Estimation;Yingwei Li;Tiffany Chen;Maya Kabkab;Ruichi Yu;Longlong Jing;Yurong You;Hang Zhao

  • key words:Self-driving,distance estimation,long-range objects,

• Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Framework;Xu Ma;Can Qin;Haoxuan You;Haoxi Ran;Yun Fu

  • key words:point cloud representation,local relation,mlp,

• Learning to Dequantise with Truncated Flows;Shawn Tan;Chin-Wei Huang;Alessandro Sordoni;Aaron Courville

  • key words:variational inference,variational bayes,dequantisation,normalizing flows,

• Auto-Transfer: Learning to Route Transferable Representations;Keerthiram Murugesan;Vijay Sadashivaiah;Ronny Luss;Karthikeyan Shanmugam;Pin-Yu Chen;Amit Dhurandhar

  • key words:Feature routing,Transferable Representations,

• Pix2seq: A Language Modeling Framework for Object Detection;Ting Chen;Saurabh Saxena;Lala Li;David J. Fleet;Geoffrey Hinton

  • key words:language modeling,object detection,

• Evaluating Distributional Distortion in Neural Language Modeling;Benjamin LeBrun;Alessandro Sordoni;Timothy J. O'Donnell

  • key words:,

• Visual hyperacuity with moving sensor and recurrent neural computations;Alexander Rivkind;Or Ram;Eldad Assa;Michael Kreiserman;Ehud Ahissar

  • key words:visual system,convolutional neural networks,recurrent neural networks,active vision,active sensing,ocular drift,

• Robust and Scalable SDE Learning: A Functional Perspective;Scott Alexander Cameron;Tyron Luke Cameron;Arnu Pretorius;Stephen J. Roberts

  • key words:SDE Learning,Parallelization,Importance Sampling,
  • 随机微分方程提供了一类丰富的灵活的生成函数

• Eliminating Sharp Minima from SGD with Truncated Heavy-tailed Noise;Xingyu Wang;Sewoong Oh;Chang-Han Rhee

  • key words:Stochastic Gradient Descent,SGD,Heavy-Tails,Generalization,
  • SAM相应延伸

• Learning Towards The Largest Margins;Xiong Zhou;Xianming Liu;Deming Zhai;Junjun Jiang;Xin Gao;Xiangyang Ji

  • key words:loss function design,margin-based loss,classification,
  • 损失函数的研究

• Variational Predictive Routing with Nested Subjective Timescales;Alexey Zakharov;Qinghai Guo;Zafeirios Fountas key words:Hierarchical temporal abstraction,event discovery,hierarchical generative models,variational inference,

• ClimateGAN: Raising Climate Change Awareness by Generating Images of Floods;Victor Schmidt;Alexandra Luccioni;Mélisande Teng;Tianyu Zhang;Alexia Reynaud;Sunand Raghupathi;Gautier Cosne;Adrien Juraver;Vahe Vardanyan;Alex Hernández-García;Yoshua Bengio key words:GAN,Climate Change,Domain Adaptation,Representation Learning,Computer Vision,Application,

• On the Learning and Learnability of Quasimetrics;Tongzhou Wang;Phillip Isola key words:embedding learning,quasimetric learning,deep learning,

• $\pi$BO: Augmenting Acquisition Functions with User Beliefs for Bayesian Optimization;Carl Hvarfner;Danny Stoll;Artur Souza;Marius Lindauer;Frank Hutter;Luigi Nardi key words:Bayesian Optimization,Hyperparameter Optimization,Meta-Learning,

• Fast Model Editing at Scale;Eric Mitchell;Charles Lin;Antoine Bosselut;Chelsea Finn;Christopher D Manning key words:editing,transfomers,meta-learning,

• The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks;Rahim Entezari;Hanie Sedghi;Olga Saukh;Behnam Neyshabur key words:Permutation,Invariance,Mode Connectivity,Energy Barrier,Loss landscape,Deep Learning,

• Robbing the Fed: Directly Obtaining Private Data in Federated Learning with Modified Models;Liam H Fowl;Jonas Geiping;Wojciech Czaja;Micah Goldblum;Tom Goldstein key words:Privacy,Federated Learning,Gradient Inversion,

• Fast AdvProp;Jieru Mei;Yucheng Han;Yutong Bai;Yixiao Zhang;Yingwei Li;Xianhang Li;Alan Yuille;Cihang Xie key words:Adversarial examples,efficient training,generalization,

• Information Bottleneck: Exact Analysis of (Quantized) Neural Networks;Stephan Sloth Lorenzen;Christian Igel;Mads Nielsen key words:information bottleneck,quantization,neural network,

• Effect of scale on catastrophic forgetting in neural networks;Vinay Venkatesh Ramasesh;Aitor Lewkowycz;Ethan Dyer key words:Catastrophic forgetting,continual learning,scaling,language modeling,image classification,

• Pseudo Numerical Methods for Diffusion Models on Manifolds;Luping Liu;Yi Ren;Zhijie Lin;Zhou Zhao key words:diffusion model,generative model,numerical method,manifold,

• Hybrid Random Features;Krzysztof Marcin Choromanski;Han Lin;Haoxian Chen;Arijit Sehanobish;Yuanzhe Ma;Deepali Jain;Jake Varley;Andy Zeng;Michael S Ryoo;Valerii Likhosherstov;Dmitry Kalashnikov;Vikas Sindhwani;Adrian Weller key words:random features,softmax kernel,attention mechanism,compositional kernels,

• Multi-Mode Deep Matrix and Tensor Factorization;Jicong Fan key words:,

• Learning Weakly-supervised Contrastive Representations;Yao-Hung Hubert Tsai;Tianqin Li;Weixin Liu;Peiyuan Liao;Ruslan Salakhutdinov;Louis-Philippe Morency key words:Self-supervised Learning,Weakly Supervised Learning,Learning with Auxiliary Information,Clustering-based Representation Learning,

• Towards Better Understanding and Better Generalization of Low-shot Classification in Histology Images with Contrastive Learning;Jiawei Yang;Hanbo Chen;Jiangpeng Yan;Xiaoyu Chen;Jianhua Yao key words:Few shot learning,Histology Image,Knowledge Transferring,

• A global convergence theory for deep ReLU implicit networks via over-parameterization;Tianxiang Gao;Hailiang Liu;Jia Liu;Hridesh Rajan;Hongyang Gao key words:Deep learning,Deep implicit learning,deep equilibrium model,gradient descent,stochastic gradient descent,over-parameterization,

• Learning by Directional Gradient Descent;David Silver;Anirudh Goyal;Ivo Danihelka;Matteo Hessel;Hado van Hasselt key words:credit assignment,directional derivative,recurrent networks,

• Wisdom of Committees: An Overlooked Approach To Faster and More Accurate Models;Xiaofang Wang;Dan Kondratyuk;Eric Christiansen;Kris M. Kitani;Yair Movshovitz-Attias;Elad Eban key words:Ensemble,Cascade,Efficiency,

• cosFormer: Rethinking Softmax In Attention;Zhen Qin;Weixuan Sun;Hui Deng;Dongxu Li;Yunshen Wei;Baohong Lv;Junjie Yan;Lingpeng Kong;Yiran Zhong key words:Linear Transformer,softmax attention,

• iFlood: A Stable and Effective Regularizer;Yuexiang Xie;Zhen WANG;Yaliang Li;Ce Zhang;Jingren Zhou;Bolin Ding key words:overfitting,regularizer,

• How unlabeled data improve generalization in self-training? A one-hidden-layer theoretical analysis;Shuai Zhang;Meng Wang;Sijia Liu;Pin-Yu Chen;Jinjun Xiong key words:Self-training,Semi-supervised learning,Convergence analysis,Generalization analysis,

• On Incorporating Inductive Biases into VAEs;Ning Miao;Emile Mathieu;Siddharth N;Yee Whye Teh;Tom Rainforth key words:VAEs,Variational autoencoders,Variational auto-encoders,Representation learning,Inductive biases,

• Learning curves for continual learning in neural networks: Self-knowledge transfer and forgetting;Ryo Karakida, Shotaro Akaho

  • key words:continual learning, neural tangent kernel, statistical mechanics
  • We analyze the generalization performance of continual learning in the NTK regime and identify key properties of knowledge transfer and forgetting.

• iFlood: A Stable and Effective Regularizer;Yuexiang Xie, Zhen WANG, Yaliang Li, Ce Zhang, Jingren Zhou, Bolin Ding

  • key words: overfitting, regularizer
  • We propose a novel regularizer named iFlood, which encourages the trained models to better fit the under-fitted instances while suppressing the confidence on over-fitted ones.

• On Incorporating Inductive Biases into VAEs;Ning Miao;Emile Mathieu;Siddharth N;Yee Whye Teh;Tom Rainforth

  • key words:VAEs,Variational autoencoders,Variational auto-encoders,Representation learning,Inductive biases,

• On feature learning in neural networks with global convergence guarantees;Zhengdao Chen;Eric Vanden-Eijnden;Joan Bruna

  • key words:neural networks,feature learning,gradient descent,global convergence,

• FedChain: Chained Algorithms for Near-optimal Communication Cost in Federated Learning;Charlie Hou;Kiran Koshy Thekumparampil;Giulia Fanti;Sewoong Oh

  • key words:Federated Learning,Optimization,Distributed Optimization,

• MCMC Should Mix: Learning Energy-Based Model with Neural Transport Latent Space MCMC;Erik Nijkamp;Ruiqi Gao;Pavel Sountsov;Srinivas Vasudevan;Bo Pang;Song-Chun Zhu;Ying Nian Wu

  • key words:Generative models,energy-based models,MCMC,

• Target-Side Input Augmentation for Sequence to Sequence Generation;Shufang Xie;Ang Lv;Yingce Xia;Lijun Wu;Tao Qin;Tie-Yan Liu;Rui Yan

  • key words:Sequence Gerneration,Data Augmentation,

• Anytime Dense Prediction with Confidence Adaptivity;Zhuang Liu;Zhiqiu Xu;Hung-Ju Wang;Trevor Darrell;Evan Shelhamer

  • key words:Efficient Inference,Anytime Inference,Semantic Segmentation,Dense Prediction,Computer Vision,

• The Close Relationship Between Contrastive Learning and Meta-Learning;Renkun Ni;Manli Shu;Hossein Souri;Micah Goldblum;Tom Goldstein

  • key words:meta-learning,contrastive learning,self-supervised learning,

• Tuformer: Data-driven Design of Transformers for Improved Generalization or Efficiency;Xiaoyu Liu;Jiahao Su;Furong Huang

  • key words:Attention Modules,Transformers,Data-driven Model Design,Trainable Heads,Expressive Power,Tensor Methods.

• How Does SimSiam Avoid Collapse Without Negative Samples? A Unified Understanding with Self-supervised Contrastive Learning;Chaoning Zhang;Kang Zhang;Chenshuang Zhang;Trung X. Pham;Chang D. Yoo;In So Kweon

  • key words:SimSiam,Negative samples,SSL,Collapse,Covariance,

• Chaos is a Ladder: A New Theoretical Understanding of Contrastive Learning via Augmentation Overlap;Yifei Wang;Qi Zhang;Yisen Wang;Jiansheng Yang;Zhouchen Lin

  • key words:Contrastive Learning,Representation Learning,Self-supervised Learning,

• Federated Learning from Only Unlabeled Data with Class-conditional-sharing Clients;Nan Lu;Zhao Wang;Xiaoxiao Li;Gang Niu;Qi Dou;Masashi Sugiyama

  • key words:unsupervised federated learning,unlabeled data,class prior shift,、

• switch-GLAT: Multilingual Parallel Machine Translation Via Code-Switch Decoder;Zhenqiao Song;Hao Zhou;Lihua Qian;Jingjing Xu;Shanbo Cheng;Mingxuan Wang;Lei Li

  • key words:multilingual non-autoregressive machine translation,contextualized code-switching,back-translation,

• Language-biased image classification: evaluation based on semantic representations;Yoann Lemesle;Masataka Sawayama;Guillermo Valle-Perez;Maxime Adolphe;Hélène Sauzéon;Pierre-Yves Oudeyer

  • key words:interpretation of learned representations,language and visual processing,language-biased image classification,cognitive science,

• A Comparison of Hamming Errors of Representative Variable Selection Methods;Tracy Ke;Longlin Wang

  • key words:Lasso,Hamming error,phase diagram,rare and weak signals,elastic net,SCAD,thresholded Lasso,forward selection,forward backward selection,

• Pareto Set Learning for Neural Multi-Objective Combinatorial Optimization;Xi Lin;Zhiyuan Yang;Qingfu Zhang

  • key words:Multiobjective Combinatorial Optimization,Combinatorial Optimization,Neural Combinatorial Optimization,Multiobjective Optimization,

• Neural Network Approximation based on Hausdorff distance of Tropical Zonotopes;Panagiotis Misiakos;Georgios Smyrnis;George Retsinas;Petros Maragos

  • key words:Tropical Geometry,Zonotopes,Hausdorff Approximation,Neural Network Compression,

• Approximation and Learning with Deep Convolutional Models: a Kernel Perspective;Alberto Bietti

  • key words:kernel methods,deep learning theory,convolution,approximation,generalization,

• DemoDICE: Offline Imitation Learning with Supplementary Imperfect Demonstrations;Geon-Hyeong Kim;Seokin Seo;Jongmin Lee;Wonseok Jeon;HyeongJoo Hwang;Hongseok Yang;Kee-Eung Kim

  • key words:imitation learning,offline imitation learning,imperfect demonstration,non-expert demonstration,

• Generalized Kernel Thinning;Raaz Dwivedi;Lester Mackey

  • key words:coresets,maximum mean discrepancy,Markov chain Monte Carlo,reproducing kernel Hilbert space,thinning,compression,

• Distribution Compression in Near-Linear Time;Abhishek Shetty;Raaz Dwivedi;Lester Mackey

  • key words:Distribution compression,linear time,thinning,i.i.d. sampling,Markov chain Monte Carlo,maximum mean discrepancy,reproducing kernel Hilbert space,

• Feature Kernel Distillation;Bobby He;Mete Ozay

  • key words:Knowledge distillation,Neural Network (NN) Feature learning,ensembling NNs,Deep learning fundamentals,Image classification,

• Hidden Parameter Recurrent State Space Models For Changing Dynamics Scenarios;Vaisakh Shaj;Dieter Büchler;Rohit Sonker;Philipp Becker;Gerhard Neumann

  • key words:State Space Models,Changing Dynamics,Recurrent Neural Networks,Multi Task Learning,

• On Distributed Adaptive Optimization with Gradient Compression;Xiaoyun Li;Belhal Karimi;Ping Li

  • key words:,

• Amortized Implicit Differentiation for Stochastic Bilevel Optimization;Michael Arbel;Julien Mairal

  • key words:bilevel optimization,stochastic optimization,

• Optimization and Adaptive Generalization of Three layer Neural Networks;Khashayar Gatmiry;Stefanie Jegelka;Jonathan Kelner

  • key words:deep learning theory,adaptive kernel,robust deep learning,neural tangent kernel,adaptive generalization,non-convex optimization,

• A Theory of Tournament Representations;Arun Rajkumar;Vishnu Veerathu;Abdul Bakey Mir

  • key words:tournament,skew-symmetric,pairwise ranking,

• Phase Collapse in Neural Networks;Florentin Guth;John Zarka;Stéphane Mallat

  • key words:phase collapse,neural collapse,concentration,classification,imagenet,deep networks,complex networks,sparsity in deep networks,

• Adversarial Unlearning of Backdoors via Implicit Hypergradient;Yi Zeng;Si Chen;Won Park;Zhuoqing Mao;Ming Jin;Ruoxi Jia

  • key words:backdoor defense,backdoor removal,backdoor,minimax,implicit hypergradient,

• Hindsight: Posterior-guided training of retrievers for improved open-ended generation;Ashwin Paranjape;Omar Khattab;Christopher Potts;Matei Zaharia;Christopher D Manning

  • key words:retrieval,generation,retrieval-augmented generation,open-ended generation,informative conversations,free-form QA,posterior distribution,ELBo,

• Can an Image Classifier Suffice For Action Recognition?;Quanfu Fan;Chun-Fu Chen;Rameswar Panda

  • key words:action recognition,image classifier,super image,vision transformer,

• DiffSkill: Skill Abstraction from Differentiable Physics for Deformable Object Manipulations with Tools;Xingyu Lin;Zhiao Huang;Yunzhu Li;Joshua B. Tenenbaum;David Held;Chuang Gan

  • key words:Deformable Object Manipulation,Differentiable Physics,

• EigenGame Unloaded: When playing games is better than optimizing;Ian Gemp;Brian McWilliams;Claire Vernade;Thore Graepel

  • key words:pca,principal components analysis,nash,games,eigendecomposition,svd,singular value decomposition,

• NASI: Label- and Data-agnostic Neural Architecture Search at Initialization;Yao Shu;Shaofeng Cai;Zhongxiang Dai;Beng Chin Ooi;Bryan Kian Hsiang Low

  • key words:Neural Architecture Search,Initialization,Label- and Data-agnostic,Transferability,Neural Tangent Kernel,

• Data Poisoning Won’t Save You From Facial Recognition;Evani Radiya-Dixit;Sanghyun Hong;Nicholas Carlini;Florian Tramer

  • key words:Poisoning attacks,adversarial examples,facial recognition,arms race,defenses,

• Meta-Imitation Learning by Watching Video Demonstrations;Jiayi Li;Tao Lu;Xiaoge Cao;Yinghao Cai;Shuo Wang

  • key words:Meta-imitation Learning,One-shot Learning,Learning by Watching,Generative Adversarial Networks,

• Entroformer: A Transformer-based Entropy Model for Learned Image Compression;Yichen Qian;Xiuyu Sun;Ming Lin;Zhiyu Tan;Rong Jin

  • key words:Image compression,Entropy Model,Global Dependencies,

• Eigencurve: Optimal Learning Rate Schedule for SGD on Quadratic Objectives with Skewed Hessian Spectrums;Rui Pan;Haishan Ye;Tong Zhang

  • key words:optimization,learning rate schedule,optimal convergence rate,

• W-CTC: a Connectionist Temporal Classification Loss with Wild Cards;Xingyu Cai;Jiahong Yuan;Yuchen Bian;Guangxu Xun;Jiaji Huang;Kenneth Church

  • key words:CTC,wild cards,dynamic programing,partial alignment,

• Learning Curves for Gaussian Process Regression with Power-Law Priors and Targets;Hui Jin;Pradeep Kr. Banerjee;Guido Montufar

  • key words:Gaussian process regression,kernel ridge regression,generalization error,power law,neural tangent kernel,

• Learning curves for continual learning in neural networks: Self-knowledge transfer and forgetting;Ryo Karakida;Shotaro Akaho

  • key words:continual learning,neural tangent kernel,statistical mechanics,

• Clean Images are Hard to Reblur: Exploiting the Ill-Posed Inverse Task for Dynamic Scene Deblurring;Seungjun Nah;Sanghyun Son;Jaerin Lee;Kyoung Mu Lee

  • key words:Deblur,Reblur,Loss,Test-time adaptation,Self-supervised,

• P-Adapters: Robustly Extracting Factual Information from Language Models with Diverse Prompts;Benjamin Newman;Prafulla Kumar Choubey;Nazneen Rajani

  • key words:NLP,Prompting,Commonsense,information extraction,factual extraction,Large Language Models,

• Rethinking Supervised Pre-Training for Better Downstream Transferring;Yutong Feng;Jianwen Jiang;Mingqian Tang;Rong Jin;Yue Gao

  • key words:Pre-Training,Contrastive Learning,Representation Learning,Downstream Transferring,

• Optimal ANN-SNN Conversion for High-accuracy and Ultra-low-latency Spiking Neural Networks;Tong Bu;Wei Fang;Jianhao Ding;PENGLIN DAI;Zhaofei Yu;Tiejun Huang

  • key words:Spiking Neural Networks,ANN-SNN Conversion,Ultra-low Latency,Quantization Clip-floor-shift Activation,

• TAPEX: Table Pre-training via Learning a Neural SQL Executor;Qian Liu;Bei Chen;Jiaqi Guo;Morteza Ziyadi;Zeqi Lin;Weizhu Chen;Jian-Guang Lou

  • key words:table pre-training,sythetic pre-training,SQL execution,table-based question answering,table-based fact verification,

• Neural Markov Controlled SDE: Stochastic Optimization for Continuous-Time Data;Sung Woo Park;Kyungjae Lee;Junseok Kwon

  • key words:controlled stochastic differential equation,time-series prediction,

• FP-DETR: Detection Transformer Advanced by Fully Pre-training;Wen Wang;Yang Cao;Jing Zhang;Dacheng Tao

  • key words:Object Detection,Detection Transformer,Pre-training,Visual Prompt,

• Scale Efficiently: Insights from Pretraining and Finetuning Transformers;Yi Tay;Mostafa Dehghani;Jinfeng Rao;William Fedus;Samira Abnar;Hyung Won Chung;Sharan Narang;Dani Yogatama;Ashish Vaswani;Donald Metzler

  • key words:transformers,attention,deep learning,

• RegionViT: Regional-to-Local Attention for Vision Transformers;Chun-Fu Chen;Rameswar Panda;Quanfu Fan

  • key words:vision transformer,image recognition,multi-scale feature,

• Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigm;Yangguang Li;Feng Liang;Lichen Zhao;Yufeng Cui;Wanli Ouyang;Jing Shao;Fengwei Yu;Junjie Yan

  • key words:,

• How to Inject Backdoors with Better Consistency: Logit Anchoring on Clean Data;Zhiyuan Zhang;Lingjuan Lyu;Weiqiang Wang;Lichao Sun;Xu Sun

  • key words:backdoor learning,weight perturbation,consistency,

• IFR-Explore: Learning Inter-object Functional Relationships in 3D Indoor Scenes;QI LI;Kaichun Mo;Yanchao Yang;Hang Zhao;Leonidas Guibas

  • key words:Inter-object Functional Relationship,Learning Interactive Policy for Exploration,Interactive Perception,3D Scene Understanding,

• GDA-AM: ON THE EFFECTIVENESS OF SOLVING MIN-IMAX OPTIMIZATION VIA ANDERSON MIXING;Huan He;Shifan Zhao;Yuanzhe Xi;Joyce Ho;Yousef Saad

  • key words:,

• Learning Continuous Environment Fields via Implicit Functions;Xueting Li;Sifei Liu;Shalini De Mello;Xiaolong Wang;Ming-Hsuan Yang;Jan Kautz

  • key words:Continuous Scene Representation,Implicit Neural Networks,

• OBJECT DYNAMICS DISTILLATION FOR SCENE DECOMPOSITION AND REPRESENTATION;Qu Tang;Xiangyu Zhu;Zhen Lei;Zhaoxiang Zhang

  • key words:,

• Capacity of Group-invariant Linear Readouts from Equivariant Representations: How Many Objects can be Linearly Classified Under All Possible Views?;Matthew Farrell;Blake Bordelon;Shubhendu Trivedi;Cengiz Pehlevan

  • key words:representation learning,perceptron capacity,perceptual manifolds,equivariance,cover's theorem,vc dimension,

• Permutation-Based SGD: Is Random Optimal?;Shashank Rajput;Kangwook Lee;Dimitris Papailiopoulos

  • key words:Convex Optimization,Stochastic Optimization,Large Scale Learning,

• Fast topological clustering with Wasserstein distance;Tananun Songdechakraiwut;Bryan M Krause;Matthew I Banks;Kirill V Nourski;Barry D Van Veen

  • key words:Topological data analysis,cluster analysis,persistent homology,Wasserstein distance,Wasserstein barycenter,brain networks,intracranial electrophysiology,consciousness,

• Omni-Scale CNNs: a simple and effective kernel size configuration for time series classification;Wensi Tang;Guodong Long;Lu Liu;Tianyi Zhou;Michael Blumenstein;Jing Jiang

  • key words:Time series classification,

• Trans-Encoder: Unsupervised sentence-pair modelling through self- and mutual-distillations;Fangyu Liu;Yunlong Jiao;Jordan Massiah;Emine Yilmaz;Serhii Havrylov

  • key words:self-supervised learning,sentence embeddings,sentence representations,knowledge distillation,

• Learning Curves for SGD on Structured Features;Blake Bordelon;Cengiz Pehlevan

  • key words:Stochastic Gradient Descent,Generalization,