Espacios. Vol. 35 (Nº 7) Año 2014. Pág. 2

Assessing business intelligence systems in a pharmaceutical company: a case study

Avaliando sistemas de inteligência de negócios em uma empresa farmacêutica: um estudo de caso

Catalina RAMIREZ Aristizábal 1, Renato Oliveira de MORAES 2

Recibido: 12/04/14 • Aprobado: 30/05/14


Business intelligence (BI) systems have become fundamental support tools in the decision-making process because the systems transform large amounts of raw data into information and knowledge. This paper examines the various methods of evaluating BI systems based on a review of the previous literature, like financial method, case studies, subjective evaluation and strategic evaluation of BI systems. Using a pharmaceutical company as a case study, the paper then evaluates the BI systems and processes according to findings in the literature review. This study proves empirically the importance of the alignment the BI systems with organizational strategy and the BI evaluation, specifically the subjective assessment that allow to taking in count the point of view of final users.
Keywords: business intelligence, business intelligence assessment and BI systems.

Sistemas de inteligência de negócios (IN) tornaram-se ferramentas de apoio fundamentais no processo de tomada de decisão, pois os sistemas de transformar grandes quantidades de dados brutos em informação e conhecimento. Este artigo examina os vários métodos de avaliação de sistemas de IN com base em uma revisão da literatura anterior, como método financeiro, estudos de caso, avaliação subjetiva e avaliação estratégica dos sistemas de IN. Usando uma empresa farmacêutica como um estudo de caso, o papel, em seguida, avalia os sistemas e processos de IN, de acordo com achados na revisão da literatura. Este estudo comprova empiricamente a importância do alinhamento dos sistemas de BI com a estratégia organizacional ea avaliação BI, especificamente a avaliação subjetiva que permite tomar em conta o ponto de vista dos utilizadores finais.
Palavras-chave: inteligência de negócio, avaliação da inteligência de negócios e sistemas de IN.

1. Introduction

A study by D'Aveni (1994) asserts that the contemporary business environment is characterized by hypercompetition that requires sensible and responsive capabilities. The success of a company is the real-time synchronization of critical, tactical and operational strategic factors with opportunities available in the business environment. According to Porter (1996), for almost two decades, managers have been learning to play with a new set of rules. Companies must be flexible to respond rapidly to competitive and market changes. In these high competitively markets, the decision-making process becomes a critical factor in companies' success.

Information on the internal and external environments of the organization is a crucial factor in the decision-making process of industry executives. However, recent technological advances have generated an information overload for managers, negatively influencing the speed of the decision-making process (CITROEN, 2011). Given this scenario, business intelligence (BI) systems have become a powerful tool for companies because the systems allow large volumes of data from different systems and sources to be processed rapidly into useful information that supports the decision-making process. Accordingly with Santos (2011), the BI systems collects data inside organizations, with the objective of provide useful and relevant information to final users and give support for decision making process. This paper aims to provide an overview of the BI research field and to present a case study of the BI model using a multinational company in the pharmaceutical industry.

2. Literature review

The concepts of business intelligence and competitive intelligence (CI) have been used without distinction in the previous literature, although some authors highlight the differences between the two terms. Cavalcanti (2005) indicates that BI covers a wider spectrum than competitive intelligence. Whereas CI focuses mainly on the microenvironment, BI includes both the microenvironment and the macroenvironment. The following concepts are related to the construct of BI: competitive intelligence, customer intelligence, competitor intelligence, strategic intelligence and technical intelligence. In American literature, the concept of CI is often used and is emphasized within the contexts of environment and external information, whereas in European literature, the term BI is considered a wider concept that covers IC and the other constructs associated with intelligence as mentioned above (LÖNQVIST and PIRTIMÄKI, 2006).

However, other streams of research claim that BI is the research field that promotes information technology (IT) solutions to the processes of competitive intelligence. In this sense, it can be inferred that the following objectives of BI are related to data management: categorization, storage, retrieval, processing, exploitation, mining, analysis, summarization, visualization and learning (CABRAL, 2011). This paper adheres to the latter definition, which understands BI as the IT solution needs of information and knowledge proposed in CI.

2.1. Definition of BI

Table 1 includes several definitions of BI that are found in the previous literature. In this research, the concept of BI is understood as the process that transforms large amounts of opaque data from different sources, into information and knowledge to create competitive advantages for the company and supports the process of strategic decision-making. For some researchers, BI is not only the process of gathering information but also the end product of this process, i.e., the relevant knowledge and information that describes the business environment, the organization itself, and its situation in relation to markets, customers, competitors, and economic issues (LÖNNQVIST and PIRTTIMÄKI et al., 2006).

Another line of research discriminates between the management approach and the technical approach of BI. The managerial approach views BI as a process in which data gathered from inside and outside the enterprise are integrated to generate information relevant to the decision-making process. Using this approach, the role of BI is to create an informational environment in which operational data gathered from internal and external sources can be analyzed to extract strategic business knowledge to support the unstructured decisions of management. In contrast, the technical approach considers BI to be a set of tools to support the process described above. The focus is not on the process itself, but rather on the technologies, algorithms and tools that enable the saving, recovery, manipulation and analysis of data and information. However, in the overall view, there are two important issues. First, the core of BI is the gathering, analysis and distribution of information. Second, the objective of BI is to support the strategic decision-making process (PETRINI and POZZEBON, 2008; GHANZANFARI, JAFARI and ROUHANI, 2011).

Further, Ghanzanfari et al. (2011) include the system approach in the BI definition, i.e., the addition of characteristics with aggregate value to organizational systems. According to these authors, organizations would create a more supportive decision-making environment if they enhanced their systems with value-added features and functionalities.

Table 1. The definitions of BI found in the previous literature



Goshal, S.; Kim, S. K. (1986)

BI is the collection and analysis of information about markets, new technologies, customers, competitors and broad social trends, is becoming an essential competitive tool.

Hannula, M.; Pritimäki, V. (2003)

The BI concept is defined as an organized and systematic process that is used to acquire, analyze and disseminate information significant to the business activities.

Cavalcantti, E. P. (2005)

Intelligence is the process of ethically collecting, analyzing and disseminating precise, pertinent, specific, opportunistic, predictable and actionable information about the business environment, the competition and the organization itself.

Lönnqvist, A.; Pirttimäki, V. (2006)

The purpose of BI is to assist in controlling the vast amounts and flow of business information within the organization by identifying and processing the information into condensed and useful managerial knowledge and intelligence.

Pirttimäki, V.; Lönnqvist, A.; Karjaluoto, A. (2006)

The term BI is dualistic, referring to the following: (1) relevant information and knowledge describing the business environment, the organization itself and its own situation in relation to its markets, customers, competitors and economic issues and (2) the process that produces the intelligence described.

Source: Author

2.2. BI systems

BI refers to an important class of systems for data analysis and reporting that provides managers at various levels of an organization with timely, relevant and easy-to-use information, enabling them to make better decisions. Because BI systems often require specialized IT infrastructure to operate effectively, including tools for query, analysis and reporting and the underlying databases, BI systems often are implemented as improvements of ERP systems (ELBASHIR, COLLIER and DAVERN, 2008).

Hočevar and Jacklič (2010) indicate that BI systems include informational tools that help users to obtain required information in an efficient and easy way. These researchers note that a BI system is usually not a single application; rather, the system consists of different components that are closely related to each other, enabling users to select and analyze data, make aggregations and display the results in a form that is easy to use and understand.

From an operational standpoint, a BI system is composed of the following three main systems:

  • Front end: the platform that allows the viewing of transaction results performed by the BI system. The platform consists of two elements, the dashboard or control panel that contains the graphs, charts and indicators generated by BI, and the reports.
  • Data warehouse: the database in which the data of the company system are collected and stored.
  • BI system: a core of BI, it transforms the data obtained from the data warehouse into information as tables, graphics, indicators and reports.

2.3.1. Assessing BI

Elbashir, Collier and Davern (2008) note that measuring the bottom-line contribution of information technology has long been seen as a major challenge for researchers and professionals. Like most IT investments, money invested in BI is also a controversial issue that has generated multiple studies seeking to determine the actual return of such investments for companies.

It can be argued that investments in BI are even more difficult to measure than other IT investments because the effects of BI are not primarily financial and may be intangible. The effects may produce a direct financial impact; however there may be a substantial time lag between intelligence production and financial gains. Therefore, in practice, the measurement of BI investments is very complex. The typical method of measuring the monetary value of the investment is to calculate the return on investment (ROI); however, the problem with calculating the ROI for BI is that the outcome of BI intelligence is processed information, whose value is difficult to assess (HANNULA and PIRTTIMÄKI, 2003; LÖNNQVIST and PIRTTIMÄKI, 2006).

Different approaches are used for measuring BI, such as assessing BI in a general way or assessing BI systems specifically. In the following section, these approaches of measurement are presented.

2.3.2 General assessment of BI

In the literature review, Lönnqvist and Pirttimäki (2006) indicate that IT measures can be classified according to the following objectives:

  • Measures for determining the value of BI. The objective of this type of measure is that executives can justify BI investments, demonstrate the actual effects of BI, and increase the credibility of BI as a managerial tool. However, the financial measure is a complex issue. Models include the one proposed by Davinson (2001), known as the CI measurement model, in which the value of IC is assessed for individual CI projects and CI output is measured by assessing effects such as objective fulfillment and decision maker satisfaction. According to Lönnqvist and Pirttimäki  (2006), the four measures of CI effectiveness are as follows: time savings, cost savings, cost avoidance and revenue enhancement. These researchers note that the effectiveness of BI can also be measured by evaluating the contribution of BI to a specific decision or action and then looking at the benefit or detriment that this specific decision brings to the company.
  • Measures for managing the BI process. Based on the previous literature, the measures for managing the BI process have not been examined as often as the measures of the BI effects. Several measures of BI value are useful from the point of view of managing the BI process. The main difference with the measurement of the value of BI is that in this case, the BI professional is the primary user and the goal is to efficiently produce intelligence based on the specific needs of users. Key questions relate to the most effective allocation of staff to BI, the effective allocation of available resources, the quality of BI products and user satisfaction.

An example of this assessment method can be found in the case study conducted by Pirttimäki, Lönnqvist and Karjaluoto (2006) in the Finnish telecommunications company, Elisa Corporation.

2.3. Assessing BI systems

Hočevar and Jacklič (2010) propose the following four methods for measuring BI systems: (1) the financial method, which includes the return on investment, the net present value, the internal rate of return and the cost-benefit analysis and total cost of ownership method; (2) case studies, which measure the effectiveness of investments in information technology but do not deal specifically with the evaluation of investments in BI; (3) subjective evaluation, an alternative approach based on the observations of users that can fairly accurately depict the outcomes of business intelligence. One positive aspect of this approach is that the results demonstrate the effectiveness of the business intelligence solutions to users; and (4) strategic analysis, which determines whether the BI systems increase company competitiveness and support the management.   

Another stream of research approaches BI systems assessment from a quantitative perspective. For example, Elbashir, Collier and Davern (2008) conducted a research survey to measure the relation between the BI process and organizational performance, in this research were identified 22 items about the benefits achieved by organizations using BI systems, exploratory factor analysis was applied to examine the underlying dimensions, this process resulted in four factors: (1) business supplier/partner relations benefits, such as reduction in transaction costs, enhanced coordination with a business suppliers and partners and better inventory management; (2) internal process benefits, refers to the benefits arise from the improvement in the efficiency of internal processes, (3) customers intelligence benefits, arise from better understanding of customers buying acts, prediction of customers future needs and introducing new products and services accordingly and (4) organizational benefits. These dimensions demonstrate the current move in the deployment of BI systems at operational level creating benefits along their value chain activities.

Ghazanfari, Jafari and Rouhani (2011) also conducted a survey to determine and extract six critical factors for assessing intelligence systems business: (1) analytical and intelligent decision support, (2) providing related experimentation and integration with environmental information, (3) optimization and recommended model for transform data in information, (4) reasoning, (5) enhanced decision making tools and (6) stakeholder satisfaction. This tool can diagnose the intelligence level of expertise systems. According with the authors enterprises can use this approach to evaluate, select, and buy software and systems that provide better decision support for the organizational environment, enabling them to achieve competitive advantage.

3. Methodology

This research is developed through a longitudinal case study. Like most longitudinal research, the study is based on only one case (Voss, Tsikriktsis and Frohlich; 2003) enabling a more in-depth study (MIGUEL, 2007).

3.1. Case description

The case study examines a company in the pharmaceutical sector with a regional center in Brazil. In the past, the company produced lines of vitamin supplements, essences and vitamins for industrialized food and over-the-counter medicines. However, in 2005, the board of directors performed a process of strategic reformulation that focused the company's research activities on the development of medicines for diseases such as cancer, diabetes and AIDS. As part of the operationalization of the new business strategy, the company sold all other product lines and focused on the development of new complex molecules in its clinical research. Notably, this strategic reformulation allowed the company to diminish its customer scope and to focus its marketing investments on physicians in selected specialties.

3.2. Information sources

This case study uses two primary sources of information. One source includes semi-structured interviews guided by instruments designed to collect information for assess the process and the results of company BI systems. The two employees interviewed from the BI team are responsible for the development and maintenance of BI architecture. For the following functional areas, one employee was interviewed: finance, marketing and IT. It guarantees an unbiased approach because it takes into account BI users perceptions. The secondary source is the presentation of company BI systems and the direct observation of BI systems including the front end and the data warehouse.

4. Discussion    

This case study examines in depth the company's BI process and systems as well as its methodologies for evaluating and measuring the BI.  An analysis is conducted with regard to the performance of the BI systems and process. In the next section, the findings on these topics are discussed.

4.1. BI systems' alignment with organizational strategy

For several years, BW was the provider of BI systems for entire company. In the marketing and sales sectors of the Latin American region, the selection of BI systems was made at the discretion of each country. For this reason, BI systems were composed of several legacy systems that were incompatible with each other. This fact is highlighted in an interview with one member of the BI staff:

The BI area is new in this structure. Previously, each country individually chose their BI tool, making the process difficult for management. Moreover, using different tools was complex and imposed high economic costs.

However, due the company's strategic reformulation in 2005 and the subsequent change of opportunity for customer segmentation, the company acquired the customer relationship management (CRM) system of Oracle, creating the need for a new BI system for sales and marketing to extract information from the new CRM system. Under this scenario, the company's BI system is composed of two different systems: the BI for sales and marketing and the BI for other business areas such as finance, the supply chain, human resources and the factory. In the table 2, the components of the two BI systems are illustrated.

Table 2. Components of the company BI systems



Sales and Marketing

Other areas

Front end

Business object

Business object

Data warehouse



BI system

Power center


Source: Author

The fact that the company has two BI systems distorts the main objective of BI systems, i.e., to unify and compile company data and information. A sales and marketing BI system manager recognizes this distortion in the following response: 

Generally BI is the area where all information is consolidated, and then makes sense to have only one area; however, in this company the technologies used by each area are different. One explanation for that is that the transactional systems that support the other areas like SAP are globalized and they have a separate group. However, sales and marketing BI are closer businesses and the local business has their own particularities; for this area, the regional approach is used.

Moreover, the sales and marketing BI team highlight that their systems are more complete, intuitive and interactive. Figure 1 presents the dashboard of Business Objects.

Figure 1. Dashboard of Business Objects

Source: Company´s sales and marketing BI system

As part of the process of strategic reformulation and the new marketing segmentation that the company began in 2005, all members of the sales force were provided with tablets and smart phones to enable them to work more efficiently when outside the office. BI was incorporated into the mobile project through the application of RoamBi which allowed the users to assess the company BI systems remotely. Despite the application's limitations in the query process, it is useful for the sales force. Figure 2 depicts the RoamBi dashboard.

Figure 2. RoamBi Dashboard

From a strategic point of view, it may be concluded that the company's BI systems are evolving to follow and contribute to the new strategic orientation of the company, supporting the management's strategic decision-making process and guaranteeing the competitiveness of the sales force.

4.2. Assessment of BI systems

As previously discussed in the literature review, it is difficult to measure the value of BI systems because their final product is information and the value of information cannot be calculated in an objective way. In the studied company, the BI area classifies the measurement in the following two main stages:

Measurement before implementation: Before the implementation process, the BI area creates a business case with the objective of assessing the cost/benefit and the return of the investment in the new application. However, the employees indicate that is very difficult achieve the real numbers of the project, and after implementation, the results obtained are not compared with the projected data in the business case. Another way to use measurement more informally is to compare the total cost of the project with the cost of the employees that can be released after the automation of the process.

Measurement after implementation: The main criterion is the usability indicator, which measures the use of each BI resource and allows the system to remain parsimonious by avoiding redundant or little used tools.

The BI staff recognizes the intrinsic difficulty of measuring the BI systems and process:

In the projects of BI, the people try to make a business case; however, it is difficult to quantify in financial terms how much some information costs. Consistent and true information serves for a faster and certain decision-making process. And is difficult to measure how much you win or lose for making more or less right decision.

In short, although the company has some methodologies for evaluating BI systems and process, this assessing is informal or not applied with the necessary rigor. The company can improve this weakness by implementing some of the methodologies of measurement presented in the literature revision, section 2.3. In the context of evaluating information as an intangible product, the company can apply the methodology of the case study presented at the same point. Noteworthy, none of the measurements performed by area evaluates BI user satisfaction. According to the literature review, subjective evaluation is a recommended method for approaching user satisfaction. In the next point, the paper presents a subjective evaluation performed with the system users.

4.3. Subjective assessment

The subjective evaluation of BI systems seeks to understand the utility of BI products in an accurate and objective way by taking into account the point of view of final users. In the interviews, BI users evaluated issues such as the availability and accuracy of information, the ease of use, time saved, improvements in the decision-making process and system flexibility.

  • Availability and accuracy of information: A finance area employee indicates that she does not use the regional BI system because it is less accurate. She had difficulties with the exchange rates of the different countries because the use of this system requires manual adjustments. Another problem is the availability of the reports, as not all countries send the reports on the scheduled dates.
  • Ease of use: One IT employee expressed difficulty in using the BI system, explaining that the system has a large amount of information but that the majority of the information is never used, making difficult to find useful information.
  • Time saved: The employees who were interviewed agreed that the BI systems allow users to save time, i.e., by eliminating manual reports and indicator calculations.
  • Improvements in the decision-making process: The users affirm that BI systems help them to make better decisions. As one employee said, "One thing is the perception that people have and other is the real data. Sometimes it seems that a problem exists in a given process; however, the graphic can show that error occurs indeed, but in another process. It helps to know which action of improvement must be implemented, allows us to quantify the results of the made decisions, and helps us to be more objective and less interpretative".
  • Flexibility: The interviewed employees agreed that the BI systems are not flexible and do not allow modifications. Moreover, in the implementation phase, the users were not consulted about the reports required, and for this reason, there exists a large quantity of useless reports that make the system more complex and slow.

The interviewed users believe that company BI systems are useful. However, this subjective evaluation indicated that despite the company's efforts to update and unify the BI systems, there are still difficulties in its precision, objectiveness, its ease of use and its flexibility.  In the company that was studied, the absence of subjective assessment of the BI systems is a weakness of the company´s BI process, because this kind of evaluation is focused in the final users, who can give an important point of view of the system quality.

5. Conclusion

In this section, the paper summarizes the main findings in the conducted case study as follows:

  • Although efforts were made by the company to improve its decision support systems, the company's current system is divided into two parts. This fact limits the informational power of both tools because they are not working with the integrated information company, which is one of the most prominent characteristics of BI systems. Is highlighted that according with the empirical evidence the BI systems are aligned with the company strategy.
  • The organization, specifically, the area of ​​BI presents substantial opportunities for improvement related to the evaluation and measurement of BI systems, because it has placed low importance on this topic and assessment processes are informal. The measurement of user satisfaction is a notably weak area. The improved evaluation of BI systems and processes would allow the company to tap the full potential of the systems in which it has invested.
  • Subjective evaluation has demonstrated the importance of BI systems for users with regard to improving decision-making processes and saving time. However, there is opportunity for improvement in areas such as accuracy, timeliness, ease of use and flexibility. For example, one suggestion would be the implementation of a self-service BI system that allowed users to customize the dashboard BI based on their own preferences.

Overall, this case study showed in practice the substantial challenges of implementing a BI system, particularly in a multinational company, as observed in the example of the exchange rates of different countries.

Notably, this study also highlights the importance of measuring processes and BI systems because if companies do not measure, they cannot control and manage. Finally, this study demonstrates the importance of BI systems in supporting the strategic decision-making process because objective and correct decisions can only be made based on timely and reliable information.

Opportunities for future research include the development of additional case studies in the pharmaceutical industry, which would create a road map of best practices in BI as well as a consistent and comprehensive method for evaluating BI systems and processes.


CABRAL NETTO, O. V. (2011); Uma visão holística da inteligência competitiva para a construção de uma teoria. 164p. Dissertação (Mestrado em Engenharia de Produção) – Escola Politécnica da Universidade de São Paulo, São Paulo.

CAVALCANTI, E. P. (2005); "The relationship between Business Intelligence and Business Success", Journal or Competitive Intelligence and Management, 3, 1.

CITROEN, C. L. (2011); "The role of information in strategic decision-making", International Journal of Information Management, 31, 493-501.

D'AVENI, R. A. (1994); Hypercompetition: managing de the dynamics of strategic maneuvering. New York: Free Press.

DAVISON, L. (2001); "Measuring Competitive Intelligence Effectiveness: Insights from Advertising Industry", Competitive Intelligence Review, 19, 4, 27-30.

ELBASHIR, M. Z., Collier, P. A., Davern, M. J. (2008); "Measuring the effects of the business intelligence systems: The relationship between business process and organizational performance", International Journal of Accounting Information Systems, 9, 135-153.

GHAZANFARI, M., Jafari, M., Rouhani, S. (2011); "A tool to evaluate the business intelligence of enterprise systems", Scientia Iranica, 18, 6, 1579-1590.

GOSHAL, S., Kim, S. K. (1986); "Building Effective Intelligence Systems for Competitive Advantage", Sloan Management Review, 28, 1.

HANNULA, M., Pirttimäki, V. (2003); "Business Intelligence empirical study on the top 50 Finish Companies", Journal of American Academy of Business, 2, 2, 593- 599.

HOČEVAR, B., Jaklič, J. (2010); "Assessing benefits of business intelligence systems- A case study", Management, 15, 1, 87-119.

LÖNNQVIST, A., Pirttimäki, V. (2006); "The measurement of business intelligence", Information Systems Management, 23, 1.

MIGUEL, P. A. (2007); "Estudo de caso na engenharia de produção: estruturação e recomendações para sua condução", Produção, 17, 1, 216-229.

PETRINI, M., Pozzebon, M. (2008); "What role is business intelligence playing in developing countries? a picture of Brazilian companies", Data Mining Applications for Empowering Knowledge Societies, Hakikur Rahman, ED., 237-257, IGI Global.

PORTER, M. E. (1996); "What is strategy?", Harvard Business Review, 61-78.

PIRTTIMÄKI, V., Lönnqvist, A., Karjalouto, A. (2006); "Measurement of Business Intelligence in a Finish Telecommunications Company", The electronic journal of knowledge management, 4, 1, 83-90.

SANTOS dos, S. R. (2011); "Proposta de reconfiguração estratégica de módulo financeiro de business intelligence no contexto de governança de tecnologia da informação (GTI): estudo realizado em uma indústria do setor plástico", Espacios, 32, 1, 41-42.

VOSS, C., Tsikriktsis, N., Frohlich, M. (2002); "Case research in operations management", International Journal of Operations & Production Management, 22, 2, 195 – 219.

1 Production Engineering Department University of São Paulo. Email:
2 Production Engineering Department University of São Paulo. Email:

Vol. 35 (Nº7) Año 2014

[En caso de encontrar algún error en este website favor enviar email a webmaster]