一句话：提出一种约束优化后处理，保证场景图满足一致性约束并提升生成准确率。
场景图生成从图像中抽取关键对象及其关系的图表示，常用于自动驾驶等安全关键场景，领域或安全专家可给出一致性约束。现有机器学习方法只关注关系级精度，难以保证生成场景图满足这些约束。本文在概率场景图之上引入约束优化作为后处理步骤，在已有ML生成结果之后修复约束违反，从而保证不违反一致性约束，并通过修复违规提升整体精度。作者在两个常用ML数据集上增加一致性约束，使用两种ML场景图生成方法作为基线，以多种已知和新颖指标评估有效性。
Scene graph generation takes an image and derives a graph representation of key objects in the image and their relations. This core computer vision task is often used in autonomous driving, where traditional software and machine learning (ML) components are used in tandem. However, in such a safety-critical context, valid scene graphs can be further restricted by consistency constraints captured by domain or safety experts. Existing ML approaches for scene graph generation focus exclusively on relation-level accuracy but provide little to no guarantee that consistency constraints are satisfied in the generated scene graphs. In this paper, we aim to complement existing ML-based approaches by a post-processing step using constraint optimization over probabilistic scene graphs that can (1) guarantee that no consistency constraints are violated and (2) improve the overall accuracy of scene graph generation by fixing constraint violations. We evaluate the effectiveness of our approach using well-known, and novel metrics in the context of two popular ML datasets augmented with consistency constraints and two ML-based scene graph generation approaches as baselines.
@inproceedings{paperbot7267,
title = {Consistent Scene Graph Generation by Constraint Optimization},
author = {Boqi Chen and Kristóf Marussy and Sebastian Pilarski and Oszkár Semeráth and Daniel Varro},
booktitle = {ASE},
year = {2022},
doi = {10.1145/3551349.3560433}
}