UI Design Dataset Crawling

and Analysis

▪ Presenter: Ruiqi Wang (U6342380)

▪ Supervisor: Dr. Zhenchang Xing （ANU）Dr. Chunyang Chen (Monash University)

▪ COMP 4560 – Advanced Computing Project

▪ Semester 1, 2019

Content

Project motivation and objectives.

1 Background

The contributions of this project.

2 Contributions

The user study we conducted to demonstrate the usefulness of this project.

3 Usefulness evaluation

Conclusions and future work.

4 Conclusion & Future Work

Background

Content

Background

Background

◆ Project goal:

➢ Enhance tagging based search on design sharing websites by predicting additional tags for existing UI designs.

◆ Why is it important?

➢ Inability of search interface to translate design requirements into design components.

➢ Problems with tagging-based search:

• Abbreviations and synonyms among tag vocabulary.

• Missing tags and wrong tags.

Contributions

Content

Contributions

Contributions

◆ What did we do?

1. Introduce a large-scale UI design dataset derived from Dribbble for UI design

analysis.

2. Construct a vocabulary for UI design semantics based on the tags in our

dataset.

3. Develop a deep-learning based method for specifically recommending semantic

tags to the existing design to assist designers with the UI search.

Contributions -- Dataset Overview

Contributions

◆ Dataset

Collected 240,000 designs from Dribbble.com1 including the meta-data

➢ title, designer, description, tags, attachment, comments, number of likes, saves, etc.

Semantic Annotation

Contributions -- UI vocabulary

Contributions

◆To discover the correlation

of tags, we conduct

community detection.

◆We also performed an

iterative open coding of

1,000 most frequent co-

occurring tags.

Contributions -- UI vocabulary

Contributions

◆ Five main semantic UI categories:- PLATFORM

- COLOR

- APP FUNCTIONALITY

- SCREEN FUNTIONALITY

- SCREEN LAYOUT

◆ Tag Normalization

- Use “DomainThesaurus” to generate abbreviations

and synonyms for each tag.

- Manually check the morphological forms.

Contributions -- Tag Prediction

Contributions

◆Dataset preparing

- Positive data: UI designs with a target tag (including its

morphological forms)

- Negative data: UI designs attached with tags which are

in the same category of the target tag

(excluding the target tag).

◆Preprocessing

- Apply AutoAugment to enrich the dataset and increase

diversity.

Contributions -- Tag Prediction

Contributions

◆Binary Convolutional Neural Network

- 25 models

- Average accuracy: 89.1%

- Platform: 94.9%

- Color: 97.6%

- App Function: 86.3%

- Screen Function: 83.5%

- Screen Layout: 86%

◆Deep Learning visualization- To gain the insight into our CNN

classifier for the prediction results.

Contributions -- Tag Prediction Results

Contributions

Usefulness

Evaluation

Content

Usefulness Evaluation

Usefulness Evaluation

◆Randomly select three tags from three categories

respectively as the queries.

- Experiment group: Search our dataset with

normalized tags and complemented with additional

predicted tags.

- Control group 1: All UIs contain all exact

keywords in the query.

- Control group 2: Directly search the Dribbble

website with our query.

◆Recruit 10 participants.

They individually marked each result as related to

the query or not and filled out a questionnaire with

questions about the retrieved results.

Usefulness Evaluation

Usefulness Evaluation

◆ Questionnaire.

Usefulness Evaluation Result

Usefulness Evaluation

◆Experiment group retrieves more related

candidates than the other two control groups.

◆Mann-Whitney U test demonstrates the

the significance of the differences.

Conclusion

Content

Conclusion & Future Work

Conclusion

★ Conclusion

○ Create a large-scale UI design dataset.

○ Constructed vocabulary of UI semantics.

○ Adopt a deep learning method to automatically recommend the existing UIs with additional

tags.

★ Future Work:

○ Improve the current accuracy performance.

○ Extend our work to enhance the search for dynamic animation UI designs.