a variational level set approach to multiphase multi-object tracking in camera network base on deep features

Background and objectives: object tracking in video streams is one of the issues in machine vision that has many applications. depending on the type of the object, the number of objects and other inputs used in tracking, object tracking is divided into several different categories. multi-object tracking in a camera network is one of the most complex types of object tracking. in this type of tracking, the goal of the algorithm is to extract the persistent trace of several objects moving simultaneously in a wide area that is monitored by a network of cameras. this type of tracking is often done in two steps. in the first step, the traces of each object in each camera is called tracklets are extracted. then, the persistent trace of the objects are obtained by associating the extracted tracklets of all cameras in the monitored wide area. here, we introduce a novel variational approach based on the deep features to associate the tracklets. methods: for this purpose a variational model with multiphase level set representation is introduced. the persistent trace of all objects are obtained by optimizing the proposed variational model. the proposed variational model is optimized by employing the euler-lagrange equation. cnn and deep learning are used to extract the deep features of appearance and motion of objects. here, a resnet50 network that is pre-trained on imagenet and a transformer neural network which is trained with motion parameters of tracklets such as acceleration and orientation change rate are used for extracting deep features. results: the multiphase model using deep features presented in this paper provide 9% better results than the multiphase model without deep features based on tcf and fs metrics and 8% better results based on mt metric. conclusion: the results on the three well-known datasets which are real and a synthesized dataset show that the proposed model takes competitive performance, while using less extra context information of the camera network and objects, compared to the other proposed methods. the evaluations show the quality of the proposed model in solving complex problems using the minimum required initial knowledge.