Skip to main content

Towards Meta-learning of Deep Architectures for Efficient Domain Adaptation.

Ali, A.R., Budka, M. and Gabrys, B., 2019. Towards Meta-learning of Deep Architectures for Efficient Domain Adaptation. In: 16th Pacific Rim International Conference on Artificial Intelligence, 26-30 August 2019, Fiji.

Full text available as:

[img]
Preview
PDF
_Abbas__Meta_Cognition_of_Deep_Architectures_for_an_Efficient_and_Accurate_Domain_Adaptation.pdf - Accepted Version
Available under License Creative Commons Attribution Non-commercial.

972kB

Official URL: https://www.pricai.org/2019/10-main-page/11-the-16...

Abstract

This paper proposes an efficient domain adaption approach using deep learning along with transfer and meta-level learning. The objective is to identify how many blocks (i.e. groups of consecutive layers) of a pre-trained image classification network need to be fine-tuned based on the characteristics of the new task. In order to investigate it, a number of experiments have been conducted using different pre-trained networks and image datasets. The networks were fine-tuned, starting from the blocks containing the output layers and progressively moving towards the input layer, on various tasks with characteristics different from the original task. The amount of fine-tuning of a pre-trained network (i.e. the number of top layers requiring adaptation) is usually dependent on the complexity, size, and domain similarity of the original and new tasks. Considering these characteristics, a question arises of how many blocks of the network need to be fine-tuned to get maximum possible accuracy? Which of a number of available pre-trained networks require fine-tuning of the minimum number of blocks to achieve this accuracy? The experiments, that involve three network architectures each divided into 10 blocks on average and five datasets, empirically confirm the intuition that there exists a relationship between the similarity of the original and new tasks and the depth of network needed to fine-tune in order to achieve accuracy comparable with that of a model trained from scratch. Further analysis shows that the fine-tuning of the final top blocks of the network, which represent the high-level features, is sufficient in most of the cases. Moreover, we have empirically verified that less similar tasks require fine-tuning of deeper portions of the network, which however is still better than training a network from scratch.

Item Type:Conference or Workshop Item (Paper)
Group:Faculty of Science & Technology
ID Code:32528
Deposited By: Unnamed user with email symplectic@symplectic
Deposited On:15 Jul 2019 10:46
Last Modified:22 Jun 2020 14:33

Downloads

Downloads per month over past year

More statistics for this item...
Repository Staff Only -