Continuously Indexed Domain Adaptation (CIDA)

This is the authors' official PyTorch implementation for CIDA. This repo contains code for experiments in the ICML 2020 paper 'Continuously Indexed Domain Adaptation'.

Outline for This README

• Beyond Domain Adaptation: Brief Introduction for CIDA
• Sample Results
• Method Overview
• IPython Notebooks and Environment
• Quantitative Results
• Theory
• Reference

Beyond Domain Adaptation: Brief Introduction for CIDA

Essentially CIDA asks the question of whether and how to go beyond current (categorical) domain adaptation regime and proposes the first approach to adapt across continuously indexed domains. For example, instead of adapting from domain A to domain B, we would like to simultaneously adapt across infintely many domains in a manifold. This allows us to go beyond domain adaption and perform both domain interpolation and domain extrapolation. See the following toy example.

Naturally categorical domain adaptation works on a finite number of domains while CIDA works on infinitely many domains. See the following comparison.

For a more visual introduction, feel free to take a look at this video.

Sample Results

If we use domains [1, 6] as source domains and the rest as target domains, below are some sample results from previous domain adaptation methods and CIDA, where CIDA successfully learns how the decision boundary evolves with the domain index.

Method Overview

We provide a simple yet effective learning framework with theoretical guarantees (see the Theory section at the end of this README). Below is a quick comparison between previous domain adaptation methods and CIDA (differences marked in red).

• Previous domain adaptation methods use a discriminator is classifify different domains (as categorical values), while CIDA's discriminator directly regresses the domain indices (as continuous values).
• Previous domain adaptation methods' encoders ignore domain IDs, while CIDA takes the domain index as input.

IPython Notebooks and Environment

Below are some IPython Notebooks for the experiments. We strongly recommend starting from the simplest case, i.e., Experiments for Toy Datasets (Quarter Circle) to get familar with the data and settings.

IPython Notebooks

• Experiments for Toy Datasets (Quarter Circle)
• Experiments for Toy Datasets (Half Circle)
• Experiments for Toy Datasets (Sine)
• Experiments for Rotating MNIST

Besides using IPython notebooks, you can also directly run the following command for the Rotating MNIST experiments inside the folder 'rotatingMNIST':

bash run_all_exp.sh

Environment

• Python 3.6
• PyTorch 1.5.1
• Numpy 1.19

Quantitative Results

Rotating MNIST

Intra-Dataset Results on Real-World Medical Datasets

In the intra-dataset setting, we consider both domain extrapolation and domain interpolation (see the figure below).

Cross-Dataset Results on Real-World Medical Datasets

Multi-Dimensional CIDA on Real-World Medical Datasets

Theory

Denoting the domain index as u and the encoding as z, we have (check the paper for full theorems):

• Theorem 1 (Informal). CIDA converges, if and only if, the expectation of domain index 𝔼[𝑢 | z] is identical for any embedding z.
• Theorem 2 (Informal). PCIDA converges, if and only if, the expectation and the variance of domain index, 𝔼[𝑢 | z] and 𝕍[𝑢 | z] are identical for any z.
• Theorem 3 (Informal). The global optimum of the two-player game between 𝐸 and 𝐷 matches the global optimum of the three-play game between 𝐸, 𝐹, and 𝐷.

Also Check Out Relevant Work

Graph-Relational Domain Adaptation
Zihao Xu, Hao He, Guang-He Lee, Yuyang Wang, Hao Wang
Tenth International Conference on Learning Representations (ICLR), 2022
[Paper] [Code] [Talk] [Slides]

Reference

Continuously Indexed Domain Adaptation

@inproceedings{DBLP:conf/icml/WangHK20,
  author    = {Hao Wang and
               Hao He and
               Dina Katabi},
  title     = {Continuously Indexed Domain Adaptation},
  booktitle = {ICML},
  year      = {2020}
}