.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "build/examples_auto_module/train_image_classifier_basic.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        Click :ref:`here <sphx_glr_download_build_examples_auto_module_train_image_classifier_basic.py>`
        to download the full example code

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_build_examples_auto_module_train_image_classifier_basic.py:

02. Train Image Classification with Auto Estimator
=====================================================

This tutorial goes through the basic steps of using GluonCV auto estimator to train an image classifier
with custom hyper-parameters.

.. GENERATED FROM PYTHON SOURCE LINES 9-11

Train with default configurations
---------------------------------

.. GENERATED FROM PYTHON SOURCE LINES 11-13

.. code-block:: default

    from gluoncv.auto.estimators import ImageClassificationEstimator








.. GENERATED FROM PYTHON SOURCE LINES 14-15

In this tutorial, we use a small sample dataset

.. GENERATED FROM PYTHON SOURCE LINES 15-19

.. code-block:: default

    train, _, test = ImageClassificationEstimator.Dataset.from_folders(
        'https://autogluon.s3.amazonaws.com/datasets/shopee-iet.zip')
    train, val, _ = train.random_split(val_size=0.1, test_size=0)





.. rst-class:: sphx-glr-script-out

 Out:

 .. code-block:: none

    data/
    ├── test/
    └── train/




.. GENERATED FROM PYTHON SOURCE LINES 20-28

Create an estimator with default configurations.
We only change the number of GPUs to reflect the hardware constraint.

Note that you may still launch training if no gpu is available(with {'gpus': []})
but be prepared that this is painfully slow and not even possible to finish
in case the dataset isn't tiny.

We recommend that you use at least one nvidia gpu with more than 6GB free GPU memory.

.. GENERATED FROM PYTHON SOURCE LINES 28-32

.. code-block:: default

    classifier = ImageClassificationEstimator(
        {'gpus': [0], 'train': {'batch_size': 16, 'epochs': 2}})









.. GENERATED FROM PYTHON SOURCE LINES 33-34

run fit on train/validation data

.. GENERATED FROM PYTHON SOURCE LINES 34-36

.. code-block:: default

    classifier.fit(train, val)





.. rst-class:: sphx-glr-script-out

 Out:

 .. code-block:: none

    Downloading /root/.mxnet/models/resnet50_v1-cc729d95.zip from https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/models/resnet50_v1-cc729d95.zip...

      0%|          | 0/57421 [00:00<?, ?KB/s]
      0%|          | 99/57421 [00:00<01:12, 793.84KB/s]
      1%|          | 510/57421 [00:00<00:25, 2253.13KB/s]
      4%|3         | 2189/57421 [00:00<00:07, 7331.41KB/s]
     14%|#3        | 7815/57421 [00:00<00:02, 24063.87KB/s]
     25%|##4       | 14265/57421 [00:00<00:01, 37334.23KB/s]
     40%|###9      | 22933/57421 [00:00<00:00, 53084.51KB/s]
     53%|#####3    | 30480/57421 [00:00<00:00, 60094.47KB/s]
     67%|######7   | 38483/57421 [00:00<00:00, 65610.12KB/s]
     82%|########1 | 46854/57421 [00:00<00:00, 71114.66KB/s]
     95%|#########5| 54600/57421 [00:01<00:00, 73035.76KB/s]
    100%|##########| 57421/57421 [00:01<00:00, 51737.76KB/s]




.. GENERATED FROM PYTHON SOURCE LINES 37-38

Evaluate the final model on test set

.. GENERATED FROM PYTHON SOURCE LINES 38-41

.. code-block:: default

    eval_result = classifier.evaluate(test)
    print("Top1/Top5 on test dataset: {}".format(eval_result))





.. rst-class:: sphx-glr-script-out

 Out:

 .. code-block:: none

    Top1/Top5 on test dataset: (0.8875, 1.0)




.. GENERATED FROM PYTHON SOURCE LINES 42-43

save to/from disk to be used later

.. GENERATED FROM PYTHON SOURCE LINES 43-46

.. code-block:: default

    classifier.save('classifier.pkl')
    classifier2 = ImageClassificationEstimator.load('classifier.pkl')








.. GENERATED FROM PYTHON SOURCE LINES 47-48

run prediction on test images

.. GENERATED FROM PYTHON SOURCE LINES 48-52

.. code-block:: default

    pred = classifier2.predict(test.iloc[0]['image'])
    print('GroundTruth:', test.iloc[0])
    print('prediction:', pred)





.. rst-class:: sphx-glr-script-out

 Out:

 .. code-block:: none

    GroundTruth: image    /root/.gluoncv/datasets/shopee-iet/data/test/B...
    label                                                    0
    Name: 0, dtype: object
    prediction:               class     score  id
    0         BabyPants  0.587603   0
    1         BabyShirt  0.388625   1
    2   womenchiffontop  0.014393   3
    3  womencasualshoes  0.009378   2




.. GENERATED FROM PYTHON SOURCE LINES 53-57

Customize your training configurations
--------------------------------------
You may modify configurations to customize your training,
supported fields:

.. GENERATED FROM PYTHON SOURCE LINES 57-59

.. code-block:: default

    print(ImageClassificationEstimator._default_cfg)





.. rst-class:: sphx-glr-script-out

 Out:

 .. code-block:: none

    ImageClassificationCfg(img_cls=ImageClassification(model=resnet50_v1, use_pretrained=True, use_gn=False, batch_norm=False, use_se=False, last_gamma=False), train=TrainCfg(pretrained_base=True, batch_size=128, epochs=10, lr=0.1, lr_decay=0.1, lr_decay_period=0, lr_decay_epoch=40, 60, lr_mode=step, warmup_lr=0.0, warmup_epochs=0, num_training_samples=1281167, num_workers=4, wd=0.0001, momentum=0.9, teacher=None, hard_weight=0.5, dtype=float32, input_size=224, crop_ratio=0.875, use_rec=False, rec_train=~/.mxnet/datasets/imagenet/rec/train.rec, rec_train_idx=~/.mxnet/datasets/imagenet/rec/train.idx, rec_val=~/.mxnet/datasets/imagenet/rec/val.rec, rec_val_idx=~/.mxnet/datasets/imagenet/rec/val.idx, data_dir=~/.mxnet/datasets/imagenet, mixup=False, no_wd=False, label_smoothing=False, temperature=20, resume_epoch=0, mixup_alpha=0.2, mixup_off_epoch=0, log_interval=50, mode=, start_epoch=0, transfer_lr_mult=0.01, output_lr_mult=0.1, early_stop_patience=-1, early_stop_min_delta=0.001, early_stop_baseline=0.0, early_stop_max_value=1.0), valid=ValidCfg(batch_size=128, num_workers=4), gpus=(0,))




.. GENERATED FROM PYTHON SOURCE LINES 60-61

For example, we could change the learning rate and batch size

.. GENERATED FROM PYTHON SOURCE LINES 61-63

.. code-block:: default

    new_classifier = ImageClassificationEstimator({'gpus': [0], 'train': {'batch_size': 16, 'lr': 0.01}})








.. GENERATED FROM PYTHON SOURCE LINES 64-69

A more natural format for modifying individual hyperparameter
is to edit the yaml file saved automatically in `self._logdir`, here we just show an example
of how to copy/edit and load the modified configuration file back to the estimator in python

You may edit the yaml file directly with a text editor

.. GENERATED FROM PYTHON SOURCE LINES 69-76

.. code-block:: default

    import shutil
    import os
    config_name = 'config.yaml'
    shutil.copyfile(os.path.join(classifier2._logdir, config_name), os.path.join('.', config_name))
    cfg = open(config_name).read()
    print(cfg)





.. rst-class:: sphx-glr-script-out

 Out:

 .. code-block:: none

    # ImageClassificationCfg
    gpus: !!python/tuple
    - 0
    img_cls:
      batch_norm: false
      last_gamma: false
      model: resnet50_v1
      use_gn: false
      use_pretrained: true
      use_se: false
    train:
      batch_size: 16
      crop_ratio: 0.875
      data_dir: ~/.mxnet/datasets/imagenet
      dtype: float32
      early_stop_baseline: 0.0
      early_stop_max_value: 1.0
      early_stop_min_delta: 0.001
      early_stop_patience: -1
      epochs: 10
      hard_weight: 0.5
      input_size: 224
      label_smoothing: false
      log_interval: 50
      lr: 0.01
      lr_decay: 0.1
      lr_decay_epoch: 40, 60
      lr_decay_period: 0
      lr_mode: step
      mixup: false
      mixup_alpha: 0.2
      mixup_off_epoch: 0
      mode: ''
      momentum: 0.9
      no_wd: false
      num_training_samples: 1281167
      num_workers: 4
      output_lr_mult: 0.1
      pretrained_base: true
      rec_train: ~/.mxnet/datasets/imagenet/rec/train.rec
      rec_train_idx: ~/.mxnet/datasets/imagenet/rec/train.idx
      rec_val: ~/.mxnet/datasets/imagenet/rec/val.rec
      rec_val_idx: ~/.mxnet/datasets/imagenet/rec/val.idx
      resume_epoch: 0
      start_epoch: 0
      teacher: null
      temperature: 20
      transfer_lr_mult: 0.01
      use_rec: false
      warmup_epochs: 0
      warmup_lr: 0.0
      wd: 0.0001
    valid:
      batch_size: 128
      num_workers: 4





.. GENERATED FROM PYTHON SOURCE LINES 77-78

Let's modify the network from resnet50_v1 to resnet18_v1b

.. GENERATED FROM PYTHON SOURCE LINES 78-88

.. code-block:: default

    import fileinput
    with fileinput.FileInput(config_name,
                             inplace = True, backup ='.bak') as f:
        for line in f:
            if 'resnet50_v1' in line:
                new_line = line.replace('resnet50_v1', 'resnet18_v1b')
                print(new_line, end='')
            else:
                print(line, end='')








.. GENERATED FROM PYTHON SOURCE LINES 89-90

The new classifier now should reflect the new configs we just edited

.. GENERATED FROM PYTHON SOURCE LINES 90-91

.. code-block:: default

    new_classifier2 = ImageClassificationEstimator(config_name)








.. rst-class:: sphx-glr-timing

   **Total running time of the script:** ( 0 minutes  33.125 seconds)


.. _sphx_glr_download_build_examples_auto_module_train_image_classifier_basic.py:


.. only :: html

 .. container:: sphx-glr-footer
    :class: sphx-glr-footer-example



  .. container:: sphx-glr-download sphx-glr-download-python

     :download:`Download Python source code: train_image_classifier_basic.py <train_image_classifier_basic.py>`



  .. container:: sphx-glr-download sphx-glr-download-jupyter

     :download:`Download Jupyter notebook: train_image_classifier_basic.ipynb <train_image_classifier_basic.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_