一句话:提出从 GUI 图像自动识别视觉元素并生成可复用 Web/Android/iOS 代码的模型。
该研究面向界面导向软件开发中把 GUI 转换为代码这一重复且繁琐的任务,目标是通过图像自动生成代码。其挑战包括起点是手绘还是高保真图像、视觉元素检测与识别、数据表示以及代码生成,同时需支持跨多个 GUI 的复用识别与同一应用内 GUI 间的导航。作者采用设计科学研究(DSR)方法,已发现若干问题类别与有助于理解如何从 GUI 提取/表示数据并生成 Web、Android 和 iOS 代码的工件。当前开放问题包括如何识别单个与成组元素、如何表示视觉元素以供代码生成器使用,以及何种代码生成器最有效;这些将作为后续工作。
In the context of user interface-oriented software development, the task of translating a GUI into code requires sufficient knowledge to identify visual elements and how to code it for one or more platforms. In addition, other issues are important, such as reuse, componentization and understanding of the behavior of trivial visual elements. This is a repetitive and tedious task that could be automated. To perform automation this task many challenges depend on the starting point (hand-draw or hi-fidelity images), the detection and recognition of visual elements from images, their data representation and the code generation itself. This work aims to build a model that makes it possible to automate the process of code generation from images so that it is possible to infer which visual elements are reusable across a range of GUIs and in such a way that you can navigate between GUIs in the same application. This study is being conducted through a DSR (Design Science Research) and so far some classes of problems and artifacts have been found that help to understand how to extract and represent data from GUI and generate web, android and ios application code. The open questions about this problem concern how to identify individual elements and in a group, how to represent these visual elements in a way that can be used in code generators and what is the most efficient code generator for this type of problem. The answers to these questions are part of the next steps of this work.
DOI 原文 ·
@inproceedings{paperbot7158,
title = {A model for automatic generating reusable code from multiple GUIs},
author = {Cícero A. G. Araújo},
booktitle = {ASE},
year = {2022},
doi = {10.1145/3551349.3559563}
}