Fusing object detection and region appearance for image-text alignment

Luca Del Pero; Philip Lee; James Magahern; Emily Hartley; Kobus Barnard

doi:10.1145/2072298.2071951

Fusing object detection and region appearance for image-text alignment

Luca Del Pero, Philip Lee, James Magahern, Emily Hartley, Kobus Barnard

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

We present a method for automatically aligning words to image regions that integrates specific object classifiers (e.g., "car"detectors) with weak models based on appearance features. Previous strategies have largely focused on the latter, and thus have not exploited progress on object category recognition. Hence, we augment region labeling with object detection, which simplifies the problem by reliably identifying a subset of the labels, and thereby reducing correspondence ambiguity overall. Comprehensive testing on the SAIAPR TC dataset shows that principled integration of object detection improves the region labeling task.

Original language	English (US)
Title of host publication	MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops
Pages	1113-1116
Number of pages	4
DOIs	https://doi.org/10.1145/2072298.2071951
State	Published - 2011
Event	19th ACM International Conference on Multimedia ACM Multimedia 2011, MM'11 - Scottsdale, AZ, United States Duration: Nov 28 2011 → Dec 1 2011

Publication series

Name	MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops

Other

Other	19th ACM International Conference on Multimedia ACM Multimedia 2011, MM'11
Country/Territory	United States
City	Scottsdale, AZ
Period	11/28/11 → 12/1/11

Keywords

Algorithms
Performance

ASJC Scopus subject areas

Computer Graphics and Computer-Aided Design
Human-Computer Interaction

Access to Document

10.1145/2072298.2071951

Cite this

Del Pero, L., Lee, P., Magahern, J., Hartley, E., & Barnard, K. (2011). Fusing object detection and region appearance for image-text alignment. In MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops (pp. 1113-1116). (MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops). https://doi.org/10.1145/2072298.2071951

Fusing object detection and region appearance for image-text alignment. / Del Pero, Luca; Lee, Philip; Magahern, James et al.
MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops. 2011. p. 1113-1116 (MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Del Pero, L, Lee, P, Magahern, J, Hartley, E & Barnard, K 2011, Fusing object detection and region appearance for image-text alignment. in MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops. MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops, pp. 1113-1116, 19th ACM International Conference on Multimedia ACM Multimedia 2011, MM'11, Scottsdale, AZ, United States, 11/28/11. https://doi.org/10.1145/2072298.2071951

@inproceedings{6c7f6020b5b14eaa8d825323ca851af3,

title = "Fusing object detection and region appearance for image-text alignment",

abstract = "We present a method for automatically aligning words to image regions that integrates specific object classifiers (e.g., {"}car{"}detectors) with weak models based on appearance features. Previous strategies have largely focused on the latter, and thus have not exploited progress on object category recognition. Hence, we augment region labeling with object detection, which simplifies the problem by reliably identifying a subset of the labels, and thereby reducing correspondence ambiguity overall. Comprehensive testing on the SAIAPR TC dataset shows that principled integration of object detection improves the region labeling task.",

keywords = "Algorithms, Performance",

author = "{Del Pero}, Luca and Philip Lee and James Magahern and Emily Hartley and Kobus Barnard",

year = "2011",

doi = "10.1145/2072298.2071951",

language = "English (US)",

isbn = "9781450306164",

series = "MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops",

pages = "1113--1116",

booktitle = "MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops",

note = "19th ACM International Conference on Multimedia ACM Multimedia 2011, MM'11 ; Conference date: 28-11-2011 Through 01-12-2011",

}

TY - GEN

T1 - Fusing object detection and region appearance for image-text alignment

AU - Del Pero, Luca

AU - Lee, Philip

AU - Magahern, James

AU - Hartley, Emily

AU - Barnard, Kobus

PY - 2011

Y1 - 2011

N2 - We present a method for automatically aligning words to image regions that integrates specific object classifiers (e.g., "car"detectors) with weak models based on appearance features. Previous strategies have largely focused on the latter, and thus have not exploited progress on object category recognition. Hence, we augment region labeling with object detection, which simplifies the problem by reliably identifying a subset of the labels, and thereby reducing correspondence ambiguity overall. Comprehensive testing on the SAIAPR TC dataset shows that principled integration of object detection improves the region labeling task.

AB - We present a method for automatically aligning words to image regions that integrates specific object classifiers (e.g., "car"detectors) with weak models based on appearance features. Previous strategies have largely focused on the latter, and thus have not exploited progress on object category recognition. Hence, we augment region labeling with object detection, which simplifies the problem by reliably identifying a subset of the labels, and thereby reducing correspondence ambiguity overall. Comprehensive testing on the SAIAPR TC dataset shows that principled integration of object detection improves the region labeling task.

KW - Algorithms

KW - Performance

UR - http://www.scopus.com/inward/record.url?scp=84455206096&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84455206096&partnerID=8YFLogxK

U2 - 10.1145/2072298.2071951

DO - 10.1145/2072298.2071951

M3 - Conference contribution

AN - SCOPUS:84455206096

SN - 9781450306164

T3 - MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops

SP - 1113

EP - 1116

BT - MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops

T2 - 19th ACM International Conference on Multimedia ACM Multimedia 2011, MM'11

Y2 - 28 November 2011 through 1 December 2011

ER -

Fusing object detection and region appearance for image-text alignment

Abstract

Publication series

Other

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this