Zero-Shot Learning for Semantic Segmentation - Revision history

Tiago at 08:40, 9 October 2020

2020-10-09T08:40:01Z

Tiago at 18:09, 8 October 2020

2020-10-08T18:09:10Z

Tiago: Created page with "{{StudentProjectTemplate |Summary=Zero-Shot Learning for Semantic Segmentation |Keywords=Deep Learning, Computer Vision, Semantic Segmentation |References=https://arxiv.org/pd..."

2020-10-08T18:07:31Z

Created page with "{{StudentProjectTemplate |Summary=Zero-Shot Learning for Semantic Segmentation |Keywords=Deep Learning, Computer Vision, Semantic Segmentation |References=https://arxiv.org/pd..."

New page

{{StudentProjectTemplate
|Summary=Zero-Shot Learning for Semantic Segmentation
|Keywords=Deep Learning, Computer Vision, Semantic Segmentation
|References=https://arxiv.org/pdf/1707.00600.pdf
https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/41473.pdf
https://arxiv.org/pdf/1906.00817.pdf
https://openaccess.thecvf.com/content_ICCVW_2019/papers/MDALC/Kato_Zero-Shot_Semantic_Segmentation_via_Variational_Mapping_ICCVW_2019_paper.pdf
https://github.com/daooshee/Few-Shot-Learning

|Prerequisites=Excellent Programming Skills; Excellent Grasp of Deep Learning and Pytorch
|Supervisor=Tiago Cortinhal, Eren Erdal Aksoy,
|Author=Tiago Cortinhal
|Level=Master
|Status=Internal Draft
}}
Currently we are working of a GAN approach to be able to generate segmentation maps from a sensor modality to another (from/to LiDAR and RGB).

The currently available datasets are not perfectly aligned (SemanticKitti and Cityscape), to try to overcome this problem techniques like Zero-Shot Learning could be applied.

Zero-Shot Learning can be seen as a type of Domain Adaptation, where a given set of classes are used for training and we have another set of unseen classes to which we wish to have segmentation as well.

To be able to perform this, a kind of embedding of the classes need to be done, as to be able to adapt the pre-trained model to unseen classes by combining the embeddings of the learnt classes.

The project consists of using the pre-trained GAN generator model and apply Zero-Shot Learning.

Research Questions:
Can Zero-Shot Learning besides of learning unseen classes also improve the overall IoU (Intersection over Union)?
Should Zero-Shot or N-Shot be applied in this scenario (N-Shot: N examples are showed during training)
Compared with Domain transfer/finetuning techniques which could provide faster/better results?

@@ Line 15: / Line 15: @@
 |Author=Tiago Cortinhal
 |Level=Master
-|Status=Internal Draft
+|Status=Open
 }}
 Currently we are working of a GAN approach to be able to generate segmentation maps from a sensor modality to another (from/to LiDAR and RGB).

@@ Line 3: / Line 3: @@
 |Keywords=Deep Learning, Computer Vision, Semantic Segmentation
 |References=https://arxiv.org/pdf/1707.00600.pdf
 https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/41473.pdf
 https://arxiv.org/pdf/1906.00817.pdf
 https://openaccess.thecvf.com/content_ICCVW_2019/papers/MDALC/Kato_Zero-Shot_Semantic_Segmentation_via_Variational_Mapping_ICCVW_2019_paper.pdf
 https://github.com/daooshee/Few-Shot-Learning
 |Prerequisites=Excellent Programming Skills; Excellent Grasp of Deep Learning and Pytorch
 |Supervisor=Tiago Cortinhal, Eren Erdal Aksoy,
 |Author=Tiago Cortinhal
 |Level=Master