Task recognition from joint tracking data in an operational manufacturing cell

This paper investigates the feasibility of using inexpensive, general-purpose automated methods for recognition of worker activity in manufacturing processes. a novel aspect of this study is that it is based on live data collected from an operational manufacturing cell without any guided or scripted work. activity in a single-worker cell was recorded using the microsoft kinect, a commodity-priced sensor that records depth data and includes built-in functions for the detection of human skeletal positions, including the positions of all major joints. joint position data for two workers on different shifts was used as input to a collection of learning algorithms with the goal of classifying the activities of each worker at each moment in time. results show that unsupervised and semisupervised algorithms, such as unsupervised hidden markov models, show little loss of accuracy compared to supervised methods trained with ground truth data. this conclusion is important because it implies that automated activity recognition can be accomplished without the use of ground truth labels, which can only be obtained by time-consuming manual review of videos. the results of this study suggest that intelligent manufacturing can now include detailed process-control measures of human workers with systems that are affordable enough to be installed permanently for continuous data collection.