liu.seSearch for publications in DiVA
Change search
Link to record
Permanent link

Direct link
BETA
Publications (10 of 17) Show all publications
Berg, A. (2019). Learning to Analyze what is Beyond the Visible Spectrum. (Doctoral dissertation). Linköping: Linköping University Electronic Press
Open this publication in new window or tab >>Learning to Analyze what is Beyond the Visible Spectrum
2019 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Thermal cameras have historically been of interest mainly for military applications. Increasing image quality and resolution combined with decreasing camera price and size during recent years have, however, opened up new application areas. They are now widely used for civilian applications, e.g., within industry, to search for missing persons, in automotive safety, as well as for medical applications. Thermal cameras are useful as soon as there exists a measurable temperature difference. Compared to cameras operating in the visual spectrum, they are advantageous due to their ability to see in total darkness, robustness to illumination variations, and less intrusion on privacy.

This thesis addresses the problem of automatic image analysis in thermal infrared images with a focus on machine learning methods. The main purpose of this thesis is to study the variations of processing required due to the thermal infrared data modality. In particular, three different problems are addressed: visual object tracking, anomaly detection, and modality transfer. All these are research areas that have been and currently are subject to extensive research. Furthermore, they are all highly relevant for a number of different real-world applications.

The first addressed problem is visual object tracking, a problem for which no prior information other than the initial location of the object is given. The main contribution concerns benchmarking of short-term single-object (STSO) visual object tracking methods in thermal infrared images. The proposed dataset, LTIR (Linköping Thermal Infrared), was integrated in the VOT-TIR2015 challenge, introducing the first ever organized challenge on STSO tracking in thermal infrared video. Another contribution also related to benchmarking is a novel, recursive, method for semi-automatic annotation of multi-modal video sequences. Based on only a few initial annotations, a video object segmentation (VOS) method proposes segmentations for all remaining frames and difficult parts in need for additional manual annotation are automatically detected. The third contribution to the problem of visual object tracking is a template tracking method based on a non-parametric probability density model of the object's thermal radiation using channel representations.

The second addressed problem is anomaly detection, i.e., detection of rare objects or events. The main contribution is a method for truly unsupervised anomaly detection based on Generative Adversarial Networks (GANs). The method employs joint training of the generator and an observation to latent space encoder, enabling stratification of the latent space and, thus, also separation of normal and anomalous samples. The second contribution is the previously unaddressed problem of obstacle detection in front of moving trains using a train-mounted thermal camera. Adaptive correlation filters are updated continuously and missed detections of background are treated as detections of anomalies, or obstacles. The third contribution to the problem of anomaly detection is a method for characterization and classification of automatically detected district heat leakages for the purpose of false alarm reduction.

Finally, the thesis addresses the problem of modality transfer between thermal infrared and visual spectrum images, a previously unaddressed problem. The contribution is a method based on Convolutional Neural Networks (CNNs), enabling perceptually realistic transformations of thermal infrared to visual images. By careful design of the loss function the method becomes robust to image pair misalignments. The method exploits the lower acuity for color differences than for luminance possessed by the human visual system, separating the loss into a luminance and a chrominance part.

Place, publisher, year, edition, pages
Linköping: Linköping University Electronic Press, 2019. p. 94
Series
Linköping Studies in Science and Technology. Dissertations, ISSN 0345-7524 ; 2024
National Category
Computer Vision and Robotics (Autonomous Systems)
Identifiers
urn:nbn:se:liu:diva-161077 (URN)10.3384/diss.diva-161077 (DOI)9789179299811 (ISBN)
Public defence
2019-12-18, Ada Lovelace, B-huset, Campus Valla, Linköping, 13:15 (English)
Opponent
Supervisors
Funder
Swedish Research Council, D0570301
Available from: 2019-11-13 Created: 2019-10-23 Last updated: 2019-12-12Bibliographically approved
Kristan, M., Matas, J., Leonardis, A., Felsberg, M., Pflugfelder, R., Kamarainen, J.-K., . . . al., e. (2019). The seventh visual object tracking vot2019 challenge results. In: : . Paper presented at IEEE International Conference on Computer Vision Workshops.
Open this publication in new window or tab >>The seventh visual object tracking vot2019 challenge results
Show others...
2019 (English)Conference paper, Published paper (Refereed)
Abstract [en]

The Visual Object Tracking challenge VOT2019 is the seventh annual tracker benchmarking activity organized by the VOT initiative. Results of 81 trackers are presented; many are state-of-the-art trackers published at major computer vision conferences or in journals in the recent years. The evaluation included the standard VOT and other popular methodologies for short-term tracking analysis as well as the standard VOT methodology for long-term tracking analysis. The VOT2019 challenge was composed of five challenges focusing on different tracking domains: (i) VOTST2019 challenge focused on short-term tracking in RGB, (ii) VOT-RT2019 challenge focused on “real-time” shortterm tracking in RGB, (iii) VOT-LT2019 focused on longterm tracking namely coping with target disappearance and reappearance. Two new challenges have been introduced: (iv) VOT-RGBT2019 challenge focused on short-term tracking in RGB and thermal imagery and (v) VOT-RGBD2019 challenge focused on long-term tracking in RGB and depth imagery. The VOT-ST2019, VOT-RT2019 and VOT-LT2019 datasets were refreshed while new datasets were introduced for VOT-RGBT2019 and VOT-RGBD2019. The VOT toolkit has been updated to support both standard shortterm, long-term tracking and tracking with multi-channel imagery. Performance of the tested trackers typically by far exceeds standard baselines. The source code for most of the trackers is publicly available from the VOT page. The dataset, the evaluation kit and the results are publicly available at the challenge website1 .

National Category
Computer Vision and Robotics (Autonomous Systems)
Identifiers
urn:nbn:se:liu:diva-163195 (URN)
Conference
IEEE International Conference on Computer Vision Workshops
Available from: 2020-01-22 Created: 2020-01-22 Last updated: 2020-02-06Bibliographically approved
Berg, A., Ahlberg, J. & Felsberg, M. (2018). Generating Visible Spectrum Images from Thermal Infrared. In: Proceedings 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops CVPRW 2018: . Paper presented at The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 8-22 June 2018, Salt Lake City, UT, USA (pp. 1224-1233). Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>Generating Visible Spectrum Images from Thermal Infrared
2018 (English)In: Proceedings 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops CVPRW 2018, Institute of Electrical and Electronics Engineers (IEEE), 2018, p. 1224-1233Conference paper, Published paper (Refereed)
Abstract [en]

Transformation of thermal infrared (TIR) images into visual, i.e. perceptually realistic color (RGB) images, is a challenging problem. TIR cameras have the ability to see in scenarios where vision is severely impaired, for example in total darkness or fog, and they are commonly used, e.g., for surveillance and automotive applications. However, interpretation of TIR images is difficult, especially for untrained operators. Enhancing the TIR image display by transforming it into a plausible, visual, perceptually realistic RGB image presumably facilitates interpretation. Existing grayscale to RGB, so called, colorization methods cannot be applied to TIR images directly since those methods only estimate the chrominance and not the luminance. In the absence of conventional colorization methods, we propose two fully automatic TIR to visual color image transformation methods, a two-step and an integrated approach, based on Convolutional Neural Networks. The methods require neither pre- nor postprocessing, do not require any user input, and are robust to image pair misalignments. We show that the methods do indeed produce perceptually realistic results on publicly available data, which is assessed both qualitatively and quantitatively.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2018
Series
IEEE Computer Society Conference on Computer Vision and Pattern Recognition workshops, E-ISSN 2160-7516
National Category
Computer Vision and Robotics (Autonomous Systems) Engineering and Technology
Identifiers
urn:nbn:se:liu:diva-149429 (URN)10.1109/CVPRW.2018.00159 (DOI)000457636800152 ()9781538661000 (ISBN)9781538661017 (ISBN)
Conference
The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 8-22 June 2018, Salt Lake City, UT, USA
Funder
Swedish Research Council, 2013-5703Swedish Research Council, 2014-6227
Note

Print on Demand(PoD) ISSN: 2160-7508.

Available from: 2018-06-29 Created: 2018-06-29 Last updated: 2020-02-03Bibliographically approved
Nawaz, T., Berg, A., Ferryman, J., Ahlberg, J. & Felsberg, M. (2017). Effective evaluation of privacy protection techniques in visible and thermal imagery. Journal of Electronic Imaging (JEI), 26(5), Article ID 051408.
Open this publication in new window or tab >>Effective evaluation of privacy protection techniques in visible and thermal imagery
Show others...
2017 (English)In: Journal of Electronic Imaging (JEI), ISSN 1017-9909, E-ISSN 1560-229X, Vol. 26, no 5, article id 051408Article in journal (Refereed) Published
Abstract [en]

Privacy protection may be defined as replacing the original content in an image region with a new (less intrusive) content having modified target appearance information to make it less recognizable by applying a privacy protection technique. Indeed the development of privacy protection techniques needs also to be complemented with an established objective evaluation method to facilitate their assessment and comparison. Generally, existing evaluation methods rely on the use of subjective judgements or assume a specific target type in image data and use target detection and recognition accuracies to assess privacy protection. This work proposes a new annotation-free evaluation method that is neither subjective nor assumes a specific target type. It assesses two key aspects of privacy protection: protection and utility. Protection is quantified as an appearance similarity and utility is measured as a structural similarity between original and privacy-protected image regions. We performed an extensive experimentation using six challenging datasets (having 12 video sequences) including a new dataset (having six sequences) that contains visible and thermal imagery. The new dataset, called TST-Priv, is made available online below for community. We demonstrate effectiveness of the proposed method by evaluating six image-based privacy protection techniques, and also show comparisons of the proposed method over existing methods.

Place, publisher, year, edition, pages
SPIE - International Society for Optical Engineering, 2017
National Category
Computer Vision and Robotics (Autonomous Systems)
Identifiers
urn:nbn:se:liu:diva-140495 (URN)10.1117/1.JEI.26.5.051408 (DOI)000414251400009 ()
Funder
Swedish Research Council, D0570301EU, FP7, Seventh Framework Programme, 312784
Note

Funding agencies:  Swedish Research Council through the project Learning Systems for Remote Thermography [D0570301]; European Community [312784]

Available from: 2017-09-05 Created: 2017-09-05 Last updated: 2018-01-13Bibliographically approved
Berg, A., Ahlberg, J. & Felsberg, M. (2017). Object Tracking in Thermal Infrared Imagery based on Channel Coded Distribution Fields. In: : . Paper presented at Swedish Symposium on Image Analysis. Svenska sällskapet för automatiserad bildanalys (SSBA)
Open this publication in new window or tab >>Object Tracking in Thermal Infrared Imagery based on Channel Coded Distribution Fields
2017 (English)Conference paper, Oral presentation only (Other academic)
Abstract [en]

We address short-term, single-object tracking, a topic that is currently seeing fast progress for visual video, for the case of thermal infrared (TIR) imagery. Tracking methods designed for TIR are often subject to a number of constraints, e.g., warm objects, low spatial resolution, and static camera. As TIR cameras become less noisy and get higher resolution these constraints are less relevant, and for emerging civilian applications, e.g., surveillance and automotive safety, new tracking methods are needed. Due to the special characteristics of TIR imagery, we argue that template-based trackers based on distribution fields should have an advantage over trackers based on spatial structure features. In this paper, we propose a templatebased tracking method (ABCD) designed specifically for TIR and not being restricted by any of the constraints above. The proposed tracker is evaluated on the VOT-TIR2015 and VOT2015 datasets using the VOT evaluation toolkit and a comparison of relative ranking of all common participating trackers in the challenges is provided. Experimental results show that the ABCD tracker performs particularly well on thermal infrared sequences.

Place, publisher, year, edition, pages
Svenska sällskapet för automatiserad bildanalys (SSBA), 2017
National Category
Computer Vision and Robotics (Autonomous Systems)
Identifiers
urn:nbn:se:liu:diva-136743 (URN)
Conference
Swedish Symposium on Image Analysis
Funder
Swedish Research Council, D0570301EU, FP7, Seventh Framework Programme, 312784EU, FP7, Seventh Framework Programme, 607567Swedish Research Council, 2014-6227
Available from: 2017-04-24 Created: 2017-04-24 Last updated: 2019-05-09Bibliographically approved
Berg, A., Felsberg, M., Häger, G. & Ahlberg, J. (2016). An Overview of the Thermal Infrared Visual Object Tracking VOT-TIR2015 Challenge. In: : . Paper presented at Swedish Symposium on Image Analysis.
Open this publication in new window or tab >>An Overview of the Thermal Infrared Visual Object Tracking VOT-TIR2015 Challenge
2016 (English)Conference paper, Oral presentation only (Other academic)
Abstract [en]

The Thermal Infrared Visual Object Tracking (VOT-TIR2015) Challenge was organized in conjunction with ICCV2015. It was the first benchmark on short-term,single-target tracking in thermal infrared (TIR) sequences. The challenge aimed at comparing short-term single-object visual trackers that do not apply pre-learned models of object appearance. It was based on the VOT2013 Challenge, but introduced the following novelties: (i) the utilization of the LTIR (Linköping TIR) dataset, (ii) adaption of the VOT2013 attributes to thermal data, (iii) a similar evaluation to that of VOT2015. This paper provides an overview of the VOT-TIR2015 Challenge as well as the results of the 24 participating trackers.

Series
Svenska sällskapet för automatiserad bildanalys (SSBA)
National Category
Computer Vision and Robotics (Autonomous Systems)
Identifiers
urn:nbn:se:liu:diva-127598 (URN)
Conference
Swedish Symposium on Image Analysis
Available from: 2016-05-03 Created: 2016-05-03 Last updated: 2018-01-10Bibliographically approved
Berg, A. (2016). Detection and Tracking in Thermal Infrared Imagery. (Licentiate dissertation). Linköping: Linköping University Electronic Press
Open this publication in new window or tab >>Detection and Tracking in Thermal Infrared Imagery
2016 (English)Licentiate thesis, comprehensive summary (Other academic)
Abstract [en]

Thermal cameras have historically been of interest mainly for military applications. Increasing image quality and resolution combined with decreasing price and size during recent years have, however, opened up new application areas. They are now widely used for civilian applications, e.g., within industry, to search for missing persons, in automotive safety, as well as for medical applications. Thermal cameras are useful as soon as it is possible to measure a temperature difference. Compared to cameras operating in the visual spectrum, they are advantageous due to their ability to see in total darkness, robustness to illumination variations, and less intrusion on privacy.

This thesis addresses the problem of detection and tracking in thermal infrared imagery. Visual detection and tracking of objects in video are research areas that have been and currently are subject to extensive research. Indications oftheir popularity are recent benchmarks such as the annual Visual Object Tracking (VOT) challenges, the Object Tracking Benchmarks, the series of workshops on Performance Evaluation of Tracking and Surveillance (PETS), and the workshops on Change Detection. Benchmark results indicate that detection and tracking are still challenging problems.

A common belief is that detection and tracking in thermal infrared imagery is identical to detection and tracking in grayscale visual imagery. This thesis argues that the preceding allegation is not true. The characteristics of thermal infrared radiation and imagery pose certain challenges to image analysis algorithms. The thesis describes these characteristics and challenges as well as presents evaluation results confirming the hypothesis.

Detection and tracking are often treated as two separate problems. However, some tracking methods, e.g. template-based tracking methods, base their tracking on repeated specific detections. They learn a model of the object that is adaptively updated. That is, detection and tracking are performed jointly. The thesis includes a template-based tracking method designed specifically for thermal infrared imagery, describes a thermal infrared dataset for evaluation of template-based tracking methods, and provides an overview of the first challenge on short-term,single-object tracking in thermal infrared video. Finally, two applications employing detection and tracking methods are presented.

Place, publisher, year, edition, pages
Linköping: Linköping University Electronic Press, 2016. p. 66
Series
Linköping Studies in Science and Technology. Thesis, ISSN 0280-7971 ; 1744
Keywords
thermal, infrared, detection, tracking
National Category
Computer Vision and Robotics (Autonomous Systems)
Identifiers
urn:nbn:se:liu:diva-126955 (URN)10.3384/lic.diva-126955 (DOI)978-91-7685-789-2 (ISBN)
Presentation
2016-05-10, Visionen, Hus B, Campus Valla, Linköpings universitet, Linköping, 16:16 (English)
Opponent
Supervisors
Funder
Swedish Research Council, D0570301EU, FP7, Seventh Framework Programme, 312784EU, FP7, Seventh Framework Programme, 607567
Available from: 2016-04-11 Created: 2016-04-08 Last updated: 2019-10-29Bibliographically approved
Felsberg, M., Kristan, M., Matas, J., Leonardis, A., Pflugfelder, R., Häger, G., . . . He, Z. (2016). The Thermal Infrared Visual Object Tracking VOT-TIR2016 Challenge Results. In: Hua G., Jégou H. (Ed.), Computer Vision – ECCV 2016 Workshops. ECCV 2016.: . Paper presented at 14th European Conference on Computer Vision (ECCV) (pp. 824-849). SPRINGER INT PUBLISHING AG
Open this publication in new window or tab >>The Thermal Infrared Visual Object Tracking VOT-TIR2016 Challenge Results
Show others...
2016 (English)In: Computer Vision – ECCV 2016 Workshops. ECCV 2016. / [ed] Hua G., Jégou H., SPRINGER INT PUBLISHING AG , 2016, p. 824-849Conference paper, Published paper (Refereed)
Abstract [en]

The Thermal Infrared Visual Object Tracking challenge 2016, VOT-TIR2016, aims at comparing short-term single-object visual trackers that work on thermal infrared (TIR) sequences and do not apply pre-learned models of object appearance. VOT-TIR2016 is the second benchmark on short-term tracking in TIR sequences. Results of 24 trackers are presented. For each participating tracker, a short description is provided in the appendix. The VOT-TIR2016 challenge is similar to the 2015 challenge, the main difference is the introduction of new, more difficult sequences into the dataset. Furthermore, VOT-TIR2016 evaluation adopted the improvements regarding overlap calculation in VOT2016. Compared to VOT-TIR2015, a significant general improvement of results has been observed, which partly compensate for the more difficult sequences. The dataset, the evaluation kit, as well as the results are publicly available at the challenge website.

Place, publisher, year, edition, pages
SPRINGER INT PUBLISHING AG, 2016
Series
Lecture Notes in Computer Science, ISSN 0302-9743, E-ISSN 1611-3349 ; 9914
Keywords
Performance evaluation; Object tracking; Thermal IR; VOT
National Category
Computer Vision and Robotics (Autonomous Systems)
Identifiers
urn:nbn:se:liu:diva-133773 (URN)10.1007/978-3-319-48881-3_55 (DOI)000389501700055 ()978-3-319-48881-3 (ISBN)978-3-319-48880-6 (ISBN)
Conference
14th European Conference on Computer Vision (ECCV)
Available from: 2017-01-11 Created: 2017-01-09 Last updated: 2018-10-15
Berg, A., Ahlberg, J. & Felsberg, M. (2015). A thermal infrared dataset for evaluation of short-term tracking methods. In: : . Paper presented at Swedish Symposium on Image Analysis.
Open this publication in new window or tab >>A thermal infrared dataset for evaluation of short-term tracking methods
2015 (English)Conference paper, Oral presentation only (Other academic)
Abstract [en]

During recent years, thermal cameras have decreased in both size and cost while improving image quality. The area of use for such cameras has expanded with many exciting applications, many of which require tracking of objects. While being subject to extensive research in the visual domain, tracking in thermal imagery has historically been of interest mainly for military purposes. The available thermal infrared datasets for evaluating methods addressing these problems are few and the ones that do are not challenging enough for today’s tracking algorithms. Therefore, we hereby propose a thermal infrared dataset for evaluation of short-term tracking methods. The dataset consists of 20 sequences which have been collected from multiple sources and the data format used is in accordance with the Visual Object Tracking (VOT) Challenge.

Series
Svenska sällskapet för automatiserad bildanalys (SSBA)
National Category
Computer Vision and Robotics (Autonomous Systems)
Identifiers
urn:nbn:se:liu:diva-127541 (URN)
Conference
Swedish Symposium on Image Analysis
Available from: 2016-05-03 Created: 2016-05-03 Last updated: 2018-01-10Bibliographically approved
Berg, A., Ahlberg, J. & Felsberg, M. (2015). A Thermal Object Tracking Benchmark. In: : . Paper presented at 12th IEEE International Conference on Advanced Video- and Signal-based Surveillance, Karlsruhe, Germany, August 25-28 2015. IEEE
Open this publication in new window or tab >>A Thermal Object Tracking Benchmark
2015 (English)Conference paper, Published paper (Refereed)
Abstract [en]

Short-term single-object (STSO) tracking in thermal images is a challenging problem relevant in a growing number of applications. In order to evaluate STSO tracking algorithms on visual imagery, there are de facto standard benchmarks. However, we argue that tracking in thermal imagery is different than in visual imagery, and that a separate benchmark is needed. The available thermal infrared datasets are few and the existing ones are not challenging for modern tracking algorithms. Therefore, we hereby propose a thermal infrared benchmark according to the Visual Object Tracking (VOT) protocol for evaluation of STSO tracking methods. The benchmark includes the new LTIR dataset containing 20 thermal image sequences which have been collected from multiple sources and annotated in the format used in the VOT Challenge. In addition, we show that the ranking of different tracking principles differ between the visual and thermal benchmarks, confirming the need for the new benchmark.

Place, publisher, year, edition, pages
IEEE, 2015
National Category
Computer Vision and Robotics (Autonomous Systems)
Identifiers
urn:nbn:se:liu:diva-121001 (URN)10.1109/AVSS.2015.7301772 (DOI)000380619700052 ()978-1-4673-7632-7 (ISBN)
Conference
12th IEEE International Conference on Advanced Video- and Signal-based Surveillance, Karlsruhe, Germany, August 25-28 2015
Available from: 2015-09-02 Created: 2015-09-02 Last updated: 2019-10-23Bibliographically approved
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0002-6591-9400

Search in DiVA

Show all publications