A Multi-Agent Framework for Multi-Criteria Business Intelligence driven Smart Data in a Big Data Environment

Abstract

Business Intelligence (BI) is a very important pro- cess in an entreprise because it provides decision support and enables business strategy managers to have an overview of the activity being treated. The application of new technologies and the era of Big Data pose new challenges for BI, a common problem affecting data quality is the presence of noise and irrelevant information wich can lead decision makers to a wrong decision. In this paper, a Multi-Agent Framework for BI driven Smart Data in a Big Data Environment is presented. For the unstructured and semi-structured data collection and prepro- cessing we use the framework Hadoop; Apache Flume, Apache Sqoop and ODBC/JDBC Connectors for data extraction and intagration, the Hadoop Distributed File System (HDFS) for data storage and MapReduce for preprocessing. For the structured data, an Extraction,Transformation and Loading (ETL) process based agents is used. Agents perform specific task assigned to them for treating the noise in Big Data problems by applying Analytic Hierarchy Process (AHP) that is one method of Multi- Criteria Decision Making (MCDM), providing high quality and relevant information, also known as Smart Data.