diff --git a/docs/source/explanations/aa_learners/autoaug.autoaugment_learners.AaLearner.rst b/docs/source/explanations/aa_learners/autoaug.autoaugment_learners.AaLearner.rst new file mode 100644 index 0000000000000000000000000000000000000000..ac34fb251c8002231f2ed806d5df876e97df8472 --- /dev/null +++ b/docs/source/explanations/aa_learners/autoaug.autoaugment_learners.AaLearner.rst @@ -0,0 +1,7 @@ +:mod:`autoaug.autoaugment_learners`.AaLearner +============================================== + +.. currentmodule:: autoaug.autoaugment_learners + +.. autoclass:: AaLearner + :members: \ No newline at end of file diff --git a/docs/source/explanations/aa_learners/autoaug.autoaugment_learners.EvoLearner.rst b/docs/source/explanations/aa_learners/autoaug.autoaugment_learners.EvoLearner.rst new file mode 100644 index 0000000000000000000000000000000000000000..2d1a8e0c966478a793831e74b8c4919fe67a2d1a --- /dev/null +++ b/docs/source/explanations/aa_learners/autoaug.autoaugment_learners.EvoLearner.rst @@ -0,0 +1,7 @@ +:mod:`autoaug.autoaugment_learners`.EvoLearner +============================================== + +.. currentmodule:: autoaug.autoaugment_learners + +.. autoclass:: EvoLearner + :members: \ No newline at end of file diff --git a/docs/source/explanations/aa_learners/autoaug.autoaugment_learners.GruLearner.rst b/docs/source/explanations/aa_learners/autoaug.autoaugment_learners.GruLearner.rst new file mode 100644 index 0000000000000000000000000000000000000000..842afcc47ea001c3c9abb278551a8af57a7c52fa --- /dev/null +++ b/docs/source/explanations/aa_learners/autoaug.autoaugment_learners.GruLearner.rst @@ -0,0 +1,7 @@ +:mod:`autoaug.autoaugment_learners`.GruLearner +============================================== + +.. currentmodule:: autoaug.autoaugment_learners + +.. autoclass:: GruLearner + :members: \ No newline at end of file diff --git a/docs/source/explanations/aa_learners/autoaug.autoaugment_learners.RsLearner.rst b/docs/source/explanations/aa_learners/autoaug.autoaugment_learners.RsLearner.rst new file mode 100644 index 0000000000000000000000000000000000000000..2693d7fdbc0c92ae357481da66aee894d459c7a5 --- /dev/null +++ b/docs/source/explanations/aa_learners/autoaug.autoaugment_learners.RsLearner.rst @@ -0,0 +1,7 @@ +:mod:`autoaug.autoaugment_learners`.RsLearner +============================================== + +.. currentmodule:: autoaug.autoaugment_learners + +.. autoclass:: RsLearner + :members: \ No newline at end of file diff --git a/docs/source/explanations/aa_learners/autoaug.autoaugment_learners.UcbLearner.rst b/docs/source/explanations/aa_learners/autoaug.autoaugment_learners.UcbLearner.rst new file mode 100644 index 0000000000000000000000000000000000000000..6f68b8e04392b3fda9973dea7a16bf1d01df9f99 --- /dev/null +++ b/docs/source/explanations/aa_learners/autoaug.autoaugment_learners.UcbLearner.rst @@ -0,0 +1,7 @@ +:mod:`autoaug.autoaugment_learners`.UcbLearner +============================================== + +.. currentmodule:: autoaug.autoaugment_learners + +.. autoclass:: UcbLearner + :members: \ No newline at end of file diff --git a/docs/source/explanations/autoaugment_learners.rst b/docs/source/explanations/autoaugment_learners.rst new file mode 100644 index 0000000000000000000000000000000000000000..3cd0dbc23c67c6c722a03debf58960e792f44d1d --- /dev/null +++ b/docs/source/explanations/autoaugment_learners.rst @@ -0,0 +1,12 @@ +AutoAugment learners +-------------------- + +.. toctree:: + :maxdepth: 3 + :caption: autoaugment_learners + + aa_learners/autoaug.autoaugment_learners.AaLearner + aa_learners/autoaug.autoaugment_learners.EvoLearner + aa_learners/autoaug.autoaugment_learners.GruLearner + aa_learners/autoaug.autoaugment_learners.RsLearner + aa_learners/autoaug.autoaugment_learners.UcbLearner \ No newline at end of file diff --git a/docs/source/howto/AaLearner.rst b/docs/source/howto/AaLearner.rst new file mode 100644 index 0000000000000000000000000000000000000000..4fd0823430e07fe56e4fccb4229f0c190f432d0a --- /dev/null +++ b/docs/source/howto/AaLearner.rst @@ -0,0 +1,83 @@ +AaLearner object and its children +------------------------------------------------------------------------------------------------ + +This is a page dedicated to demonstrating functionalities of :class:`AaLearner`. + +This is a how-to guide (in the sense describe in https://documentation.divio.com/structure/). + +###################################################################################################### +How to use the ``AaLearner`` class to find an optimal policy for a dataset-child_network pair +###################################################################################################### + +This section can also be read as a ``.py`` file in ``./tutorials/how_use_aalearner.py``. + + +.. code-block:: + + import autoaug.autoaugment_learners as aal + import autoaug.child_networks as cn + + import torchvision.datasets as datasets + import torchvision + + + +Defining the problem setting: + +.. code-block:: + + train_dataset = datasets.MNIST(root='./autoaug/datasets/mnist/train', + train=True, download=True, transform=None) + test_dataset = datasets.MNIST(root='./autoaug/datasets/mnist/test', + train=False, download=True, transform=torchvision.transforms.ToTensor()) + child_network_architecture = cn.lenet + + +.. warning:: + + In earlier versions, we had to write ``child_network_architecture=cn.LeNet`` + and not ``child_network_architecture=cn.LeNet()``. But now we can do both. + Both types of objects can be input into ``AaLearner.learn()``. + + More precisely, the ``child_network_architecture`` parameter has to be either + as ``nn.Module``, a ``function`` which returns a ``nn.Module``, or a ``type`` + which inherits ``nn.Module``. + + A downside (or maybe the upside??) of doing one of the latter two is that + the same randomly initialized weights are used for every policy. + +Using the random search learner to evaluate randomly generated policies: (You +can use any other learner in place of random search learner as well) + +.. code-block:: + + # aa_agent = aal.GruLearner() + # aa_agent = aal.EvoLearner() + # aa_agent = aal.UcbLearner() + # aa_agent = aal.ac_learner() + aa_agent = aal.RsLearner( + sp_num=7, + toy_size=0.01, + batch_size=4, + learning_rate=0.05, + max_epochs=float('inf'), + early_stop_num=35, + ) + aa_agent.learn(train_dataset, + test_dataset, + child_network_architecture=child_network_architecture, + iterations=15000) + +You can set further hyperparameters when defining a AaLearner. + +Also, depending on what learner you are using, there might be unique hyperparameters. +For example, in the GRU learner you can tune the exploration parameter ``alpha``. + +Viewing the results: + +``.history`` is a list containing all the policies tested and the respective +accuracies obtained when trained using them. + +.. code-block:: + + print(aa_agent.history) \ No newline at end of file diff --git a/docs/source/howto/AutoAugment.rst b/docs/source/howto/AutoAugment.rst new file mode 100644 index 0000000000000000000000000000000000000000..d4c1e609928f05e8adc1cbf9ec8b3372cfe47ac9 --- /dev/null +++ b/docs/source/howto/AutoAugment.rst @@ -0,0 +1,78 @@ +AutoAugment object +------------------ + +###################################################################################################### +How to use a ``AutoAugment`` object to apply AutoAugment policies to ``Datasets`` objects +###################################################################################################### + +This is a page dedicated to demonstrating functionalities of :class:`AutoAugment`, which +we use as a helper class to help us apply AutoAugment policies to datasets. + +This is a tutorial (in the sense describe in https://documentation.divio.com/structure/). + +For an example of how the material is used in our library, see the source code of +:meth:`AaLearner._test_autoaugment_policy <autoaug.autoaugment_learners.AaLearner>`. + +Let's say we have a policy within the search space specified by the original +AutoAugment paper: + +.. code-block:: + + my_policy = [ + (("Invert", 0.8, None), ("Contrast", 0.2, 6)), + (("Rotate", 0.7, 2), ("Invert", 0.8, None)), + (("Sharpness", 0.8, 1), ("Sharpness", 0.9, 3)), + (("ShearY", 0.5, 8), ("Invert", 0.7, None)), + (("AutoContrast", 0.5, None), ("Equalize", 0.9, None)) + ] + +And that we also have a dataset that we want to apply this policy to: + +.. code-block:: + + train_dataset = datasets.MNIST(root='./datasets/mnist/train', train=True) + test_dataset = datasets.MNIST(root='./datasets/mnist/test', train=False, + transform=torchvision.transforms.ToTensor()) + +The ``train_dataset`` object will have an attribute ``.transform`` with the +default value ``None``. +The ``.transform`` attribute takes a function which takes an image as an input +and returns a transformed image. + +We need a function which will apply the ``my_policy`` and we use +an ``AutoAugment`` for this job. + +.. code-block:: + :caption: Creating an ``AutoAugment`` object and imbueing it with ``my_policy``. + + aa_transform = AutoAugment() + aa_transform.subpolicies = my_policy + train_transform = transforms.Compose([ + aa_transform, + transforms.ToTensor() + ]) + +We can use ``train_transform`` as an image function: + +.. code-block:: + :caption: This function call will return an augmented image + + augmented_image = train_transform(original_image) + +We usually apply an image function to a ``Dataset`` like this: + +.. code-block:: + + train_dataset = datasets.MNIST(root='./datasets/mnist/train', train=True, transform=my_function) + +However, in our library we often have to apply a image function *after* the ``Dataset`` +object was already created. (For example, a ``Dataset`` object is created and trained on +multiple times using different policies). +In this case, we alter the ``.transform`` attribute: + +.. code-block:: + + train_dataset.transform = train_transform + +Now if we can create a ``DataLoader`` object from ``train_dataset``, it will automatically +apply ``my_policy``. \ No newline at end of file diff --git a/docs/source/howto/howto_main.rst b/docs/source/howto/howto_main.rst new file mode 100644 index 0000000000000000000000000000000000000000..e7385febfc4d3b40fc8ed5fd5311bc59d858ef30 --- /dev/null +++ b/docs/source/howto/howto_main.rst @@ -0,0 +1,10 @@ +How-to Guides for accomplishing common goals +-------------------------------------------- + +.. toctree:: + :maxdepth: 2 + :caption: How-to Guides: + + installation + AaLearner + AutoAugment diff --git a/docs/source/howto/installation.rst b/docs/source/howto/installation.rst new file mode 100644 index 0000000000000000000000000000000000000000..0e74525ea75cfcacc6afca2aeb087f6055bce8b9 --- /dev/null +++ b/docs/source/howto/installation.rst @@ -0,0 +1,8 @@ +Installing ``autoaug`` +---------------------- + +As simple as + +.. code-block:: + + pip install auto-augmentation \ No newline at end of file