Data and Business Intelligence Systems for Competitive Advantage: prospects, challenges, and real-world applications

 

Sistemas de datos e inteligencia empresarial para una ventaja competitiva: perspectivas, desafíos y aplicaciones del mundo real

 

Mohamed Djerdjouri

State University of New York at Plattsburgh

 (New York)

djerdjm@plattsburgh.edu

 

Received: October 12th, 2019

Accepted: December 9th, 2019

 

ABSTRACT

 

This paper is intended as a short introduction to Business Intelligence (BI) and Analytics systems. The main aim of the paper is to raise awareness of organizations in the developing world, about the benefits of these technologies and the crucial role they play in the survival and competitiveness of the firm in the complex and turbulent global market.  For many years, many small and medium-sized businesses (SMBs) have not followed large organizations in the implementation of BI technologies. The main reason stated by SMBs is the complexity and high cost of deploying and managing BI systems. However, according to recent IT industry survey of SMBs executives, they now realize the crucial role BI systems play in the company’s performance, and competitiveness and they are now increasingly investing in and implementing BI technologies.

 

Keywords: SMBs, turbulent global market, managing BI systems, IT industry.

 

Jel Code: M15.

 

 

 

 

 

 

 

 

 

 

 

RESUMEN

 

El objetivo principal del documento es sensibilizar a las organizaciones en desarrollo sobre los beneficios de la tecnología y el papel crucial que desempeña en la supervivencia y competitividad de la empresa ante el complejo y turbulento mercado global. Durante muchos años, las pequeñas y medianas empresas (PYMES) no han seguido a las organizaciones grandes en la implementación del modelo Business Intelligence (BI). La razón principal declarada por las pymes es la complejidad y el alto costo de implementar y administrar sistemas de BI. Sin embargo, según una encuesta reciente de la industria de TI a los ejecutivos de la PYME, ahora se dan cuenta del papel crucial que juegan los sistemas de BI en el rendimiento y la competitividad de la empresa y ahora están invirtiendo cada vez más en su implementación.

 

Palabras clave: PYME, turbulento mercado global, gestión del Business Intelligence, Industria IT.

 

Código Jel: M15.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 


 

 

 

INTRODUCCIÓN

 

Second to its people, a company’s most valuable asset is information.  Information is a critical resource for any organization. In this rapidly changing global market, consumers are now demanding quicker, more efficient service from businesses. To stay competitive, companies must meet or exceed the expectations of consumers. Moreover, the world has witnessed an information explosion. Data is being generated at a very high pace, and more and more of this Data is unstructured, which makes its analysis challenging to say the least.  Nowadays Data is seen as a new class of economic assets, just like currency or gold.

Figure 1

The Information Explosion

 

(zettabyte =  unit of information equal to one sextillion (1021) or, strictly, 270 bytes)

Source: Own elaboration.

 

So to stay competitive and to improve its own performance, a company must make decisions, often promptly, based on timely and accurate information. To this end, many leading innovative companies are adopting and relying on Business Intelligence systems to stay ahead of trends and future events. Also, Business Intelligence (BI) expedites decision making. This, in turn, helps companies to act quickly and correctly on information before competing businesses do. The result of all this is a competitively superior performance for the company, which allows for an appropriate and timely response to customer problems and primary concerns.

 

The ultimate achievement is improved customer experience. BI refers to technologies, applications and approaches practices for the collection, integration, analysis, and presentation of business information (Hedgebeth, 2007). BI helps managers gain insights into their own business as well as into the market in general, and it provides them with valuable facts and information that improves the quality of their decisions. (Chaudhuri, Dayal & Narasayya, 2011)

 

Analytics, on the other hand, is defined as the scientific process of transforming data into insight for making better decisions. A sound BI system provides the decision-maker with valuable information, at the appropriate time and in the right format. The ability to mine and analyze big data gives organizations deeper and richer insights into business patterns and trends, helping drive operational efficiencies and competitive advantage in manufacturing, security, marketing, and IT (Ghasemghaei, 2019). Sun and Wang  (2017) state that big data have become a strategic resource for industry, business, and national security. Moreover, Sun and Wang (2017) affirm that data nowadays have also become a strategic enabler of exploring business insights and the economics of services.

Figure 2 

Data mining

 

Source: Own elaboration.

 

BI systems merge data with different formats and from various sources and gather it into data warehouses or data marts.  Then they use Analytics to process these data to provide historical, current and predictive outlook of business operations and the market in which they operate. The information is usually presented through a dashboard or analytics interface.  BI software makes analysis and report-making much faster and more reliable.

 

In her article, Loshin (2012) reported that BI is used to understand and improve performance and to cut costs and identify new business opportunities, this can include, among many other things:

o          Analyzing customer behaviors, buying patterns, and sales trends

o          Identifying opportunities to reduce costs

o          Measuring, tracking and predicting sales and financial performance

o          Budgeting and financial planning and forecasting

o          Tracking the performance of marketing campaigns

o          Optimizing processes and operational performance

o          Improving delivery and supply chain effectiveness

o          Web and e-commerce analytics

o          Customer relationship management

o          Risk analysis

o          Strategic value driver análisis

 

Jennifer Lonoff Schiff (2013), reports that CIO.com surveyed a sample of BI experts and IT executives about the benefits of investing in BI systems. The consensus among these experts is that BI improves the bottom line of businesses. And the fundamental reasons for that are that BI helps organizations: - Get fast answers to critical business questions; align business activities with corporate strategy; empower employees; reduce time spent on data entry and manipulation; gain insights into customers; benchmark sales channel partners; identify areas for cost-cutting; and boost productivity.

 

BI simplifies information discovery and analysis, making it possible for decision-makers at all levels of an organization to quickly and more easily access, understand, analyze, collaborate, and act on information, anytime and anywhere. BI helps move from just consuming information to developing in-depth contextual knowledge about that information. By tying strategy to metrics, organizations can gain competitive advantage by making better decisions faster, at all levels of the organization. BI is the capability that transforms data into meaningful, actionable information.

 

BI software consolidates data from different sources and assembles it in “data warehouses” or “data marts” that eliminate distinctions in data formats. It then presents the results through a reporting, analytics or dashboard interface. BI software thus serves as a common platform for shared, company-wide insight. BI software makes analysis and report making much faster and more reliable.

 

 

TECHNOLOGY AND TOOLS

 

A typical architecture for supporting BI within a firm is shown in figure 3 below. A BI architecture is a framework for organizing the data, information management, and technology components that are used to build BI systems for reporting and data analytics. The underlying BI architecture plays a vital role in BI projects because it affects the development and implementation of timely decisions. The data over which BI tasks are performed are typically loaded into a repository called the data warehouse that is managed by one or multiple data warehouse servers. The data often comes from different sources, operational databases across departments within the firm, as well as external sources. The data have different formats and structures. Also, both structured and unstructured data may be used. All these data need to be standardized and integrated in preparation for BI tasks. The technologies for preparing the data for BI are known as Extract-Transform-Load (ETL) tools.  Also, a popular engine tool for storing and querying data warehouses is Relational Database Management Systems (RDBMS). Large data warehouses usually deploy parallel RDBMS engines so that SQL queries can be executed over large volumes of data.

 

 

Figure 3

Typical Business Intelligence (BI) architecture

 

Source: Own elaboration.

The technology components, referred to as BI tools in figure 4 above, are used to present information to business users and enable them to analyze the data. This includes the BI tools (or BI software suite) to be used within an organization as well as the supporting IT infrastructure such as hardware, database software, and networking devices. There are various types of BI applications that can be built into an architecture: - reporting, ad hoc query, and data visualization tools, as well as online analytical processing (OLAP) software, dashboards, performance scorecards, data mining engines, and web analytics, to name a few.  

Figure 4

Data Integration Architecture

 

Source: Own elaboration

 

Reporting tools are an essential way to present data and easily convey the results of analysis. BI users are increasingly business users who need quick, easy-to-understand displays of information (Mikalef et al., 2019). And report writers allow users to design and generate custom reports Ad hoc query tool is an end-user tool that accepts an English-like or point-and-click request for data and constructs an ad hoc query to retrieve the desired result. Visualization tools: help users create advanced graphical representations of data via simple user interfaces. This tool help users uncover patterns, outliers, and relevant facts. Online Analytical Processing (OLAP) tools enable users to analyze different dimensions of multidimensional data. The OLAP server understands how data is organized in the database and uses special functions for analyzing the data. Examples of analysis tools are time series and trend analysis.

 

Dashboards typically highlight key performance indicators (KPIs), which help managers focus on the metrics that are most important to them. Dashboards are often browser-based, making them easily accessible by anyone with permission. Performance scorecards attach a numerical weight to performance and map progress toward goals. Think of it as dashboards taken one step further. Scorecards are an effective way to keep tabs on key metrics.

 

Data mining tools allow users to analyze data from many different dimensions or angles, categorize it, and summarize the relationships identified. Technically, data mining is the process of finding correlations or patterns among dozens of fields in large relational databases

Web analytics tools enable users to understand how visitors to a company’s website interact with the pages (Imhoff, Galemmo & Geiger, 2003; Shen, 2013). They perform the measurement, collection, analysis, and reporting of Web data for purposes of understanding and optimizing Web usage. They are also used for business and market research, and to assess and improve the effectiveness of a web site.

 

 

BENEFITS

 

A well-implemented BI strategy can deliver real insight for an organization. BI systems help the organization make better decisions with higher speed and confidence; recognize and maximize the firm’s strengths; shorten marketing efforts; improve customer relationships; align effort with the firm’s strategy and improve revenues and profits (Williams & Williams, 2010).

 

Moreover, BI systems help firms quantify the value of relationships with suppliers and customers, and this gives them more leverage during negotiations. Jennifer Lonoff Schiff (2013) reports that in a survey of executives of “500” companies, they revealed a variety of benefits these firms, the main ones include:  Eliminate guesswork; get faster answers to your business questions; get key business metrics reports when and where you need them; gain insight into customer behavior; identify cross-selling and up-selling opportunities; learn how to streamline operations; improve efficiency; learn what your real manufacturing costs are; manage inventory better and; see where your business has been, where it is now and where it is going.

 

Without business intelligence, a firm runs the risk of making critical decisions based on either insufficient or inaccurate information. Robert Eugene Miller (2013) also reports that executives that a well-implemented BI strategy helps firms in the following ways:

  - Quickly identify and respond to business trends

  - Empowered staff using timely, meaningful information and trend reports

  - Easily create in-depth financial, operations, customer, and vendor reports

  - Efficiently view, manipulate, analyze, and distribute reports using many familiar tools

  - Extract up-to-the-minute high-level summaries, account groupings, or detail transactions

  - Consolidate data from multiple companies, divisions, and databases

  - Minimize manual and repetitive work

 

It is reported in the literature that successful implementation and usage of BI has shown excellent results in all sectors of the economy- healthcare, e-commerce, government, industry, etc. On average, companies have reported an ROI of $10.66 for every dollar spent on business intelligence/analytics. Real-world applications in different sectors of the economy will be presented in section 5 below.

 

CHALLENGES

According to the Garner Analytics firm research, 70% to 80% of corporate BI projects fail. Firms encounter many challenges when developing and implementing a BI strategy. The two main ones are: user resistance for adoption, Poor data quality, and Others challenges

 

User resistance for adoption

Like for any new IT system, user resistance is one significant barrier to BI success. Users resist changing the way they do things unless their current methods are tedious and time-consuming. Also, many firms make the mistake of believing that if they implement the system first, people will use it (build it, and they will come cliché). The way around this pitfall is for the firm to involve all the stakeholders from the beginning o the project and throughout the implementation process. Users should define what they really need from a BI project. When the implementation ends, the majority of the users will already be familiar with the system and know how to use it. They also feel empowered when their suggestions are implemented. Thus to ensure success, the firm must high rates of user adoption.

 

Poor data quality

Without the collection, storage, and access to reliable data, a firm cannot get any valuable and accurate insights into their business and the business environment. Data is the most essential component of any BI system. The main challenge here is for the firm to make sure the data stores and data warehouses are in good working order before they can begin extracting and acting on insights. The risk is that if that is not done correctly, critical and strategic decisions will be made based on unreliable information. The firm must establish and maintain an appropriate level of data quality to feed into the BI system.

 

Others challenges

The other challenges include breaking down departmental knowledge silos; integrating the BI tool with other operational, performance management and transactional system; transforming the workplace from a culture of ‘gut feel’ to one of data-based decision-making; securing executive sponsorship and necessary financial backing,

 

Finally, measuring the performance of BI is a significant challenge and can be problematic. The firm should develop and employ a set of key metrics to help evaluate performance and return on investment. In practice, many firms use metrics such as the time it takes to answer user queries, the depth, and usability of the information obtained from the BI tool and, the number and quality of decisions made as a result of insights generated via the BI tool

 

 

BUSINESS AND GOVERNMENT APPLICATIONS

 

Proper implementation of BI technologies can reap many benefits for the firm. Excellent results have been reported across all sectors of the economy: healthcare, government, and industry. It is estimated that for each dollar spent in BI technologies and Analytics technology, there is, on average a ten dollars return on investment. In this section, a few successful implementations of BI will be presented. The summaries below are “literally” taken from the articles in which the cases were published.

 

New York State Department of Taxation and Finance: Using Business Intelligence to improve tax revenues and citizen equity (IBM Smarter Planet Leadership Series, 2011)

The New York State Department of Taxation and Finance resolved to make its processes more data-driven. The Tax Audits department has a team of 1600 auditors. Research has shown that more than half of U.S. taxpayers willing to take liberties with their taxes when they sense that the government lacks the information to catch them. The core of the deterrent is the incorporation of more data sources–combined with the use of predictive intelligence capabilities–to accurately identify potentially questionable returns.

 

The main flaw with the current process (“pay and then chase”) was that the problems were often detected only after refund checks had been sent and cashed. Also, the process was time-consuming, drained valuable resource and was often fruitless. The department wanted to change the process to catch and rectify such refunds before they were sent out.

 

The system: The New York State Department of Taxation and Finance achieved this goal by developing a BI system called Case Identification and Selection System (CISS). The system is not merely used to search for questionable returns patterns with historical data stored in the department’s warehouse.

 

The analytics are embedded directly into the mainstream return process. The department uses predictive intelligence to determine dynamically when to process a refund request and when to set it aside for further analysis or to reject the refund directly. In a nutshell, the system compares each open case with profiles of past similar cases to recommend which cases should be pursued and through which means, to maximize the overall amount of revenue collected. The results were outstanding. The New York State Taxation and Finance Department reported the following critical results and benefits:

 - $1.2 billion reduction in improper or questionable refunds paid from the State of New York’s coffers, plus another $400 million reduction projected in 2011

-  Dramatic reduction in the costs and inefficiencies associated with “pay and chase” policies

-  $100 million increase in delinquent tax collections through the use of optimization algorithms

-  Over a 350% increase in criminal tax fraud investigations due to greater interdepartmental

   collaboration on cases.

 

Business Intelligence and Analytics in Politics: The Real story behind President   OBAMA Election Victory (Siegel, 2013)  

Barack Obama’s 2012 campaign for a second term employed more than 50 Business Intelligence/Analytics experts. The traditional political campaigns up to now spent large amounts of money focusing on trying to sway swing voters in swing states. The Obama campaign management hired a multi-disciplinary team of statisticians, predictive modelers, data-mining experts, mathematicians, software programmers, and quantitative analysts. It eventually built an entire Business Intelligence/Analytics department five times as large as that of its 2008 campaign.

 

What the Obama BI team realized is that presidential campaigns must focus even more narrowly than that. They applied predictive analytics (BI technology) that pinpoints truly persuadable voters. The BI team moved beyond simple poll analysis. Its real power came from in trying to influence the future rather than to speculate on it. Forecasting calculates an aggregate view for each US state, whereas predictive analytics (BI technology) delivers predictions for each individual voter.

 

During the six months leading up to the election, the Obama team launched a full-scale and all-front campaign, leveraging Web, mobile, TV, call, social media, and analytics to directly micro-target potential voters and donors with tailored messages. Instead of focusing on just “swing” voters (mostly independent voters who have not made up their minds and are persuadable to vote one way or another “swingable.” The Obama BI team realized that a “persuadable voter” (swingable) is a person who will be influenced to vote for the candidate by a call, a door knock, flyer, or TV ad?

 

The benefits: The Obama BI team predicted an entirely new thing. Beyond predicting which a constituent was destined to vote, they also predicted whether each individual voter would be persuaded by campaign contact. The best way to do persuasion is to predict it. Beyond identifying voters who will come out for Obama if contacted, the BI models had to distinguish those voters who would come out for Obama in any case as well as those who were at risk of being turned off by campaign contact and switching over to for vote for the opponent.

 

The necessity was to learn to discriminate, voter by voter, whether contact would persuade. There were only four especially close states in the 2012 election. Only Florida, North Carolina, Ohio, and Virginia were decided by less than 5 percentage points. The smallest number in 30 years (Reagan vs Mondale).

 

The results: More voters were convinced to choose Obama, in comparison to traditional campaign targeting. Most people predicted the election to be very close, but in fact, Obama won a decisive victory. Obama got 51.1 percent of the popular vote to Mitt Romney's 47.2 percent, a four-point margin. Moreover, President Obama won 26 states and the District of Columbia, and he also won 332 electoral votes against 206 for Romney (It takes 270 electorate votes to win the Presidential election). It is widely believed that the use of BI/Analytics by Obama’s Campaign led to the landslide victory by Barack Obama. (Scherer, 2012)

 

Improving Financial Reserve Management in the Insurance Industry  (Microsoft, 2019)

EM Insurance company located in the state of Iowa employs more than 2100 people. With assets of approximately $3 billion, it sells its products through independent insurance agencies throughout the United States. EMC Insurances Companies struggled with pinpointing the right amount of money to hold in reserve against potential case payouts; keeping too much or too little could be disadvantageous to the firm’s performance.

 

After experience a run-up in reserves, EMC took steps to improve its financial reserve management. The company had a great deal of insurance claim data but a limited ability to analyze the information. Unexpected fluctuations of financial reserves prompted EMC to use BI technologies to uncover anomalies, correlations, relationships, and patterns hidden within the firm’s warehouse of claim data. The BI system included predictive modeling for improved claim outcomes.

 

Results/Benefits: The company can identify casualty and worker’s compensation claims that are likely to have a negative outcome. There is also an apparent enhancement of the accuracy and reliability of data. Executive decision making is supported with improved analysis. Expenses are now more effectively controlled. 

 

There are many more success stories in business and government of organizations which decision process and quality improved significantly with the appropriate implementation of BI technologies. The main benefit for these organizations was the improvement of their competitiveness in the Marketplace.

 

The Gartner report (2019) mentioned that the benefits of fact-based decision-making are clear to business managers in a broad range of disciplines, including marketing, sales, supply chain management, manufacturing, engineering, risk management, and finance and HR. Significant changes are imminent to the world of BI and analytics, including the dominance of data discovery techniques, more extensive use of real-time streaming event data, and the eventual acceleration in BI and analytics spending when big data finally matures, said Roy Schulte, vice president, and distinguished analyst at Gartner. As the cost of acquiring, storing and managing data continues to fall, companies are finding it practical to apply BI and analytics in a more extensive range of situations. Nowadays thousands of businesses in all sizes, in all industries, all around the world are implementing and utilizing Strategic Business Intelligence (Stackpole, 2011).

 

The Chief Information Officers focus on BI, and analytics looks set to continue through 2017, according to Gartner (2013). Gartner's user surveys show that "improved decision making" is the key driver of BI purchases. Capabilities that will evolve BI from an information delivery system to a decision platform will increase the value of BI and drive its growth (Gartner Report, 2011 and 2019).

 

 

CONCLUSION

 

According to the 2019 Gartner report, by 2020, the number of data and analytics experts in business units will grow at three times the rate of experts in IT departments, and by 2021, analytics and BI adoption will increase from 35% of employees to over 50%, including new classes of users, particularly front-office workers.

 

BI is essential for the firm’s growth and decision-making. It gives companies a more structured way to look at data while providing in-depth interpretations. It aids decision making via real-time, interactive access to and analysis of vital corporate information. The business and technological advances promised by BI are still being developed, explored, and enhanced.

 

For many years, many small and medium-sized businesses (SMBs) have not followed large organizations in the implementation of BI technologies. The main reason stated by SMBs is the complexity and high cost of deploying and managing BI systems.   However, according to recent IT industry survey of SMBs executives, they now realize the crucial role BI systems play in the company’s performance, and competitiveness and they are now increasingly investing in and implementing BI technologies.

 

In the majority of developing economies, firms face much more significant and numerous challenges because most organizations do not have access to the latest technologies. However, the biggest obstacle to implementing BI systems stems from the lack of reliable and quality data. As mentioned earlier in this paper, data is the lifeblood of BI systems. Today’s data-driven business culture has given organizations new resources and competitive advantages through the integration of data into everyday operations and strategic business decisions.

 

However, the managerial culture should change to adopt more a data-driven decision-making process.  Organizations should realize the importance of collecting, storing, and analyzing internal as well as external data to harness the information obtained from BI systems and Analytics to improve business processes, uncover insights into customer buying patterns, internal cots, revenues, and profitability trends and of other critical business issues.

 

 

REFERENCES

 

Chaudhuri, S., Dayal, U., & Narasayya, V. (2011). An overview of business intelligence technology. Communications of the ACM, 54(8), 88-98.

 

Deloitte Report (2014). The 2014 Global Report. UK: Deloitte.

 

Gartner. (2011). Magic Quadrant for Business Intelligence Platforms. Core Research Note G00210036. Gartner.

 

Gartner. (2019). Gartner market trends report: how to win as wan edge and security converge into secure access service edge. Core Research Note G0035476. Gartner.

 

Ghasemghaei, M. (2019). Does data analytics use improve firm decision making quality? The role of knowledge sharing and data analytics competency. Decision Support Systems120, 14-24.

 

Hedgebeth, D. (2007). Data-driven decision making for the enterprise: an overview of business intelligence applications. Vine37(4), 414-420.

 

IBM (2011). Smarter Planet Leadership Series. New York: IBM. Link:  ibm.com/smarterplanet

 

IDC  (2014). The Digital Universe of Opportunities: Rich Data and the Increasing Value of the Internet of Things. Massachusetts: EMC.

 

Imhoff, C., Galemmo, N., & Geiger, G. (2003). Mastering data warehouse design: relational and dimensional techniques. John Wiley & Sons.

 

Lonoff, J. (2013). 8 Ways Business Intelligence Software Improves the Bottom Line. CIO FEATURE. Link: https://www.cio.com/article/2384577/8-ways-business-intelligence-software-improves-the-bottom-line.html

 

Loshin, D. (2012). Business intelligence: the savvy manager's guide. Massachusetts: Morgan Kaufmann.

 

Mikalef, P., Krogstie, J., Pappas, O., & Pavlou, P. (2019). Exploring the relationship between big data analytics capability and competitive performance: The mediating roles of dynamic and operational capabilities. Information & Management.

 

Microsoft (2019). Customer Stories. Toronto: Microsoft. Link: https://customers.microsoft.com/en-CA/search?sq=EMC&ff=&p=0&so=story_publish_date%20desc  

 

Scherer, M. (2012) Inside the Secret World of the Data Crunchers Who Helped Obama Win. Time, Nov. 7, 2012.

 

Shen, G. (2013) Big Data, Analytics, and Elections. Analytics Magazine, The Fiscal Times, January 21, 2013).

 

Stackpole, B. (2011). A midmarket guide to leveraging data as an asset with business intelligence and analytics.  SearchBusinessAnalytics.com.

 

Sun, Z., & Wang, P. (2017). Big data, analytics and intelligence: an editorial perspective. Journal of New Mathematics and Natural Computation13(2), 75-81.

 

Sun, Z., Sun, L., & Strang, K. (2018). Big data analytics services for enhancing business intelligence. Journal of Computer Information Systems58(2), 162-169.

 

Williams, S., & Williams, N. (2010). The Profit Impact of Business Intelligence. San Francisco: Morgan Kaufmann (Elsevier).