طراحی و آزمایش داشبورد بلادرنگ تحلیل متن شبکه اجتماعی توییتر

یکی‌ از مباحث پژوهشی‌ مهم امروز در حوزه فناوری اطلاعات و فناوری استفاده از دانش نهفته در داده هایی است که امروزه با سرعت بالا، حجم زیاد و با تنوع فراوان در فرمت داده تولید می شوند. داده هایی با چنین ویژگی هایی را کلان‌داده می نامند. استخراج، پردازش و بصری‌سازی نتایج حاصل از کلان‌داده امروزه به یکی‌ از دغدغه های دانشمندان علم داده تبدیل شده است. گفتنی است که امروزه زیر ساخت ها، روش‌ ها و ابزارهای بسیاری برای تحلیل کلان‌داده توسعه یافته‌ اند. هدف این مقاله ارائه راهکاری برای استخراج و بصری سازی داده‌های شبکه اجتماعی توییتر به‌صورت بلادرنگ با حذف پایگاه‌ های داده به‌عنوان نمونه‌ ای از تحلیل کلان‌داده است. در این پژوهش یکی‌ از راه حل‌های بصری سازی‌ بلادرنگ، با استفاده از داده ‌های توییتر به‌عنوان جریان ورودی، از آپاچی استورم به‌عنوان پلتفرم پردازشی و از d3.jsبرای نمایش داده ها ارائه خواهد شد؛ در‌نهایت داشبورد طراحی شده با استفاده از روش طراحی آزمایش‌ها و آزمون ‌های آماری از نظر زمان طی‌شده برای پاسخ (latency). در انواع پیکره ‌بندی‌های مختلف آپاچی استورم مورد ارزیابی قرار گرفته و در‌نهایت بلادرنگ‌بودن با میانگین زمان پاسخ برابر یک دقیقه و سی ثانیه تایید شد.

Design and Test of the Real-time Text mining dashboard for Twitter

One of today's major research trends in the field of information systems is the discovery of implicit knowledge hidden in dataset that is currently being produced at high speed, large volumes and with a wide variety of formats. Data with such features is called big data. Extracting, processing, and visualizing the huge amount of data, today has become one of the concerns of data science scholars.The impact of big data on information analysis can be traced to four different parts. The first part is data extraction and processing, the second part is data analysis, the third part is data storage, and finally the visualization of the data. In the field of big data processing, in various studies, different categories have been presented. For example, in the studies of Hashim et al., big data processing is divided into two categories. These two types are: batch and real time. These two categories of processing, which nowadays are standard in any comprehensive big data solution, also have been introduced in Abawajy studies: batch processing is related to offline processing, and realtime processing is usually used to analyze the streaming data without any need to storage of data on disk. As data flows from various sources, the data is analyzed and processed real time, for immediate insight. As today's world is rapidly changing and survival in today's competitive world requires instant decisionmaking based on flows of data, streaming data analysis is becoming increasingly important.On the other hand, one of the great valuable sources of streaming data is the data generated by social networks rsquo; users such as Twitter. Social networks data sources are very rich sources for analysis as they come from the opinions and opinions of their users.As discussed earlier, and since previous studies such as Flash's studies have focused more on batch analysis (offline data), this study has attempted to investigate a variety of tools and infrastructures related to big streaming data, and finally design a realtime dashboard based on Twitter social network streaming data.The following article addresses two research questions: 1) How to design and implement a realtime dashboard based on social networks data? 2) Which different configurations are best suited for realtime dashboard analysis and visualization?In other words, the purpose of this article is to provide a solution for extracting and visualizing Twitter's social network streaming data by deleting databases, as an examples of big data real time analysis. In this research, we used Twitter streaming data as an input, Apache Storm as a processing platform and D3.js as a visualization tool.Finally, the designed dashboard was evaluated using Design of Experiment method and other statistical tests in various types of Apache Storm configurations and eventually it was proved that the dashboard is real time with an average response time for 1 minute and 30 seconds.