Nowadays, the necessity of safeguarded environments is stronger than ever. The defence of public areas against terroristic threats requires intelligent security assistance systems that comprise state-of-the-art surveillance technology to localize persons with hazardous materials. The recent progress in the detection of hazardous materials by a new generation of chemical sensors leads to an increasing need of appropriate sensor models. Though, the detection capability of such sensors is quite high, their spatio-temporal resolution is very limited. Hence, a single chemical sensor is not able to localize hazardous material and assign it to a person. This drawback can be compensated by fusing the information of multiple chemical sensors with the location estimates of persons in an observed area. In this work, we are describing a Rao- Blackwellized Particle Filter (RBPF) that fuses person tracks with chemical sensors and thereby localizes persons carrying hazardous material.