Universal Perturbations

[dxoigmn]

What does this PR do?

A big missing feature of MART is the ability to generate universal perturbations. This PR tries to change that.

The way universal perturbations should work is that we "train" a perturbation on a dataset and frozen model. Right now we can only generate perturbations in "test" mode, however. This is because it is unclear how to "freeze" a model in MART.

python -m mart experiment=COCO_TorchvisionFasterRCNN_ShapeShifter trainer=gpu

My sense is that Perturber should really just be a LitModular. That will greatly unify everything and make it easier to implement universal perturbations.

This PR depends upon the following PRs:

• Add callback that freezes specified module #141
• Better Composer #154
• Don't lookup non-str keys in SequentialDict #174
• Enable overriding of special args in SequentialDict.forward #175
• Add _train_mode_ and _inference_mode_ special args to SequentialDict #176
• Add _call_ special arg to SequentialDict #184

Type of change

Please check all relevant options.

• Improvement (non-breaking)
• Bug fix (non-breaking)
• New feature (non-breaking)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

Testing

Please describe the tests that you ran to verify your changes. Consider listing any relevant details of your test configuration.

• Test A
• Test B

Before submitting

• The title is self-explanatory and the description concisely explains the PR
• My PR does only one thing, instead of bundling different changes together
• I list all the breaking changes introduced by this pull request
• I have commented my code
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• I have run pre-commit hooks with pre-commit run -a command without errors

Did you have fun?

Make sure you had fun coding 🙃