Survival of the most informed
Business intelligence for changing Japanese marketsWhile corporate Japan still lags behind the US in computerization, the amount of data being collected and stored is growing exponentially. And with domestic markets becoming more competitive, companies are frantically searching for ways to derive market intelligence from the accumulated knowledge hidden away in their databases. Business Intelligence applications are designed to perform this near-miracle.
by R.A. Lemos
With the boom years behind them, companies in Japan are quickly learning Darwin's famous evolutionary rule. The doors are opening for foreign companies to compete in several market segments (including the personal computer and telecommunications areas), because these companies have learned how to supply goods more cheaply and efficiently than the traditional Japanese distribution channels. In response, Japanese firms are belatedly struggling to cut costs by such techniques as "downsizing" or "reengineering."
Seemingly minor differences in the way products or services are sold can result in major profits or losses for a company. Even companies that have been doing business in Japan for years are worrying about how the changes in the market will affect their business. As David Deputy, Japan sales manager of IRI Software (a business information applications provider) notes, "In many sectors, the Japanese relationship-oriented market is rapidly giving way to markets based on efficiency and inexpensive goods and services." The stress on cost competition over traditional relationships have made information about emerging markets a prime commodity.
To help management make informed decisions about operating in these markets, many companies in Japan have begun to bring in information systems and increase the number of desktop computer systems. A sad fact is that many companies still think a computer on every desk as a fantasy. However, as the 3 million computers sold in Japan shows, the domestic industry is trying to dig themselves out of the undercomputerization pit. The next logical step should be to rethink how to best use the data created by their new systems. The data contained in the company+s distributed databases, usually sales transaction receipts and operational statistics, can provide a detailed view of a company+s operation. To effectively access and analyze corporate operations from this data, business intelligence applications have become a necessity.
Follow the BI ways
The problems with current information management are many and varied, but mainly center on a lack of efficient collection and communication of information on corporate operations. In the past, finding the information hidden in this ocean of data required learning a query language, accessing the data, and then copying (often even rekeying) the data into your spreadsheet application. While most foreign companies have passed beyond this limited form of business intelligence, many companies in Japan operating their information systems in this agonizingly slow way. Many large corporations can afford to assign a team of market analysts to obtain and work with the data, but these specialists tend to be a limited resource that managers can rarely utilize. In addition, most relational database systems (and older nonrelational systems) are optimized for data entry and storage, not analysis, resulting in long waits as the system manipulates the records of data, which in turn results in less time for effective analysis and less information for your company. In short, the information department of companies in Japan are not operating efficiently, leaving them ill-equipped to follow market changes.Enter business intelligence, a buzzword that has migrated from amongst the covens of corporate data analysts to the towers of executive management. In its simplest form, business intelligence (BI) can be described as any analysis that converts corporate data (examples of which range from sales data to employee work hours and from computer usage data to production costs) into information that can be used to build an analytical model of a company+s business. Under this definition, spreadsheets can be considered rudimentary business intelligence tools, since they provide a method of analyzing this data. However, on a higher level, BI applications must support the kind of analysis that corporate managers need to perform - from structured searches for operational problems to whimsical -What if I do this?+ modeling. IRI Software+s David Deputy explains that -what characterizes BI is the ability for an end user to find a problem then explore it fully from various angles.+
The desire to do this sort of analysis results in three requirements of BI systems: 1) the queries and analysis must be resolved quickly, 2) the interface must be easy to understand and intuitive to use, and 3) knowledge and expert systems relevant to the analysis should help the user in their analysis. These requirements are necessary if managers are going to be able to understand how their businesses interact with the Japanese marketplace. Howard Dresner of the Gartner Group has forecast that BI will become so important that -instead of a small number of analysts spending 100 percent of their time analyzing data, all managers and professionals will spend 10 percent of their time using business intelligence systems.+
The main reason behind the sudden appearance of business intelligence is the fact that, the essential ingredients, which had not historically been packaged as an application, are now available for a variety of platforms (DOS, Windows, and UNIX) and integrated into applications aimed at various kinds of analysis (marketing and finance are the most common). The current concept of business intelligence sprouted from the minds of the MIT professors and Wharton Business School graduates who founded Management Decision Systems (MDS), a consultancy researching better ways for companies to analyze their performance. In the 1972, the company pioneered the use of multidimensional databases as the basis of decision-support systems and analysis techniques that the company built into their Express server and BI application development system. The multidimensional database is the fundamental technology that makes online analytical processing (OLAP) efficient and fast enough to empower the average manager to simplify mountains of data into relevant information. However, the Express development system was similar to many proprietary database systems (including the ones still sold today), which package the database server with the proprietary software needed to organize and analyze the data on the server. Each application is tailored to the client corporation+s needs.
Today, BI has evolved into an interface and development toolset and, in some cases, into full fledged applications usable by certain kinds of managers. By 1998, the Gartner Group has predicted BI applications and OLAP tools will graduate to a level where BI applications will work with many kinds of OLAP platforms and multidimensional databases. If this happens, BI will become a necessary technology for companies who want to remain efficient.
To BI or not to BI
Business intelligence is not a technology that can help every company. IRI Software, a leader in the BI applications market, admits most of their customers are Fortune 1000 companies whose data management requrements are enormous. However, with packaged applications now on the market, business intelligence is becoming available to masses of midsized companies who need to better use their information. If your company has a problem analyzing a market from the constant stream of data being received - data which has many attributes or dimensions - then business intelligence applications will make a difference. As the more advanced applications are localized for the Japanese market, their potential for simplifying complex analyses will be a huge benefit for the companies that use them.An example may help illuminate the possibilities. Since marketing and finance are best supported by the current crop of applications, the example will be a mythical US firm that has been selling their cellular telephones on the Japanese market and, after the first year, has had only mild success. Using the 12-dimensional sales data from the 20,000 telephones purchased with a credit card (five of the dimensions are model #, district of Japan, month sold, occupation of owner, age of owner, as shown in the figure). Restricting the analysis to the Kanto (Tokyo) region, the sales manager calls up a graph of the number sold of each model by time. The manager notices that March was a big month and looks at the sales and % change from previous and following months for each model. Noticing that the sales of the 100X model increased by about 75% in March over those sold in April and February, the manager attempts to see whose buying the telephones and pivots the data on the 100X. Calling up the same graph for only March sales of the 100X, the manager finds out that sales to students between 21 and 25 years of age more than doubled for that month. This information suggests to the manager that students who are about to graduate are buying phones to use for work. With this information, the company can now plan a promotion to that particular group of customers.
However, sales is not the only field that can derive value from its transactional data. Shipping receipts can be analyzed to streamline ordering, the data from production forms can be graphed to make production more responsive to demand, and employee time cards could be used to understand employee work habits and problems. While all this could have been done with spreadsheets before, the difference is that data can be analyzed faster (often 30 to 100 times faster than the same complex query to a relational database) using OLAP tools and multidimensional databases, the interfaces makes the systems much easier to use, and the expert knowledge programmed into the applications make analyses easy to perform.
Japan's intelligent future
Business people do not need to be told that future business will be based on information. For any foreign company in Japan, though, information is the key to the unfamiliar workings of the domestic marketplace. However, Japan has always had an information deficit, making getting at relevant information a difficult task at best. Business intelligence will be a key technology in the restructuring of the Japanese marketplace - a key to finding problems in the way a company operates and obtaining information to solve those problems intelligently.As business intelligence applications come into wider use in Japan, companies will create a more efficient, knowledge-based market. As the current trend in moving manufacturing overseas shows, long-avoided necessities will gradually become accepted. While many keiretsu will evolve to survive these changes, relationships will be sacrified for efficiency. After an informed jump to more effcient operation, BI will find a large following (as it now does in the US) in sales. Company eill try to better identify their customers and tailor promotional campaigns and services to the individual. In the end, the result of using information to target sales, may lead Japan into a new kind of relationship - not between companies, but between the company and consumer.
No mystery behind MDBs
Two years ago, relational database systems (RDBS) were being touted as the be all and end all of the database world. However, even the original developer of the relational database concept, Dr. E. F. Codd, admits that they were never intended to provide the powerful functions for data synthesis, analysis, and consolidation for which multidimensional databases (MDB) were created. Multidimensional databases has, as the name suggests, a multidimensional data model, rather than the two-dimensional table used by relational databases. The need for this kind of model and the complex functions came from the questions that business managers have always asked. These questions are rarely -How much is the 100S phone?+ (a 1-dimensional question: phone model), but -What was the percent change over a year ago in 21-year-old students in Tokyo that used bought our 100X phone in March?+ (a 5-dimensional question: owner+s age, owner+s occupation, region, phone model, and month). Questions of this type are group-oriented and answers found by set intersections, a function that multidimensional databases can do efficiently, while the relational database model must approximate by complex and time-consuming join calculations.Business intelligence (BI) applications are a manager+s way of being able to delve down into the workings of the product, market, or section of the company that is his/her responsibility. To achieve this goal a certain structure is needed, one that optimizes the speed with which queries are resolved, yet is tailored to the way that business analysts think.
This results in a four-layered structure. The bottom layer consists of the physical databases, usually relational or (heaven forbid!) flat-file. These databases have been structured for online transaction processing (record entry, searching, and sorting) and are accessed by the SQL query language. The data could also come from a POS information service; for example AC Nielson+s ScanTrack which covers about 50% of consumer packaged goods sales in Japan, or IRI+s InfoScan which cover about 80% of CPGs in the US market and will be starting in Japan within the next year.
The next layer needs to organize the physical data for faster analytical query access, the sort of queries frequently asked in online analytical processing. It was for that purpose that multidimensional databases were developed, though they may be replaced instead by a OLAP definition and calculation layer, one that holds the indices to precalculated summary data and formulas to quickly calculate othe data. The OLAP layer usually manifests as an OLAP server and database which answers queries from BI applications and formulates SQL queries to the relational level for data that it does not have stored.
At present, both the OLAP/multidimensional database and remaining two layers are usually provided by the same source. The laer that sits on top of the database is an interface between the user and the OLAP/multidimenional database, and provides all the functions that the typical business user might need. The fourth layer is the main application part that a user sees when they use a BI application. This application is usually created in-house or by the multidimensional database provider. For example, IRI Software provides Express EIS, which includes the multidimensional database, the interface, and an application toolkit, with which the company can build an analysis application to the specifications of the client. Other companies add functionality to spreadsheets to create their interface.
However, companies are slowly building up a select of prepackaged applications for the business user. For marketing and finance, IRI Software+s has two prebuilt applications that have the bases covered. An important feature of these applications is that people knowledgeable about the data being analyzed created them. Thus, marketing and finance manager do not have to learn computerese, but use the vocabulary of their trade. In a way, the application is somewhat of an expert in navigating through the transaction records where your informationis hidden.
IRI Software has been a leader in bringing business-intelligence applications to the desktop. The company's Japanese subsidiary has close relations with Mitsui & Co., Ltd., and has created localized versions of their Express EIS and Dataserver Analyzer products. Computing Japan spoke with David Deputy, IRI Software's Japan sales manager, about business intelligence and its future in Japan.
Computing Japan: What is IRI's current role in Japan?
David Deputy: Currently, we are a BI (business intelligence) software company. But our worldwide operations are actually two-thirds focused on collecting and reselling POS (point-of-sale) data from supermarkets as well as providing logistics solutions known as ECR for the consumer packaged goods industry.
Express (IRI Software's OLAP package) was originally designed as a marketing decision-support system. But many of the issues in marketing are the same ones in finance and operations, so the technology grew to become a corporate-wide decision-support environment. Starting this year, we will be offering ECR and POS data services in Japan. If we realize our vision for these services here in Japan, we will be able to dramatically reduce the cost of goods to the consumer. Some of this should ó in theory ó trickle down directly to the consumer.
CJ: Could you fill us in on the history of "business intelligence?"
Deputy: The history of BI, as it relates to IRI, consists of two key events. The first was the acquisition of Management Decision Systems by IRI in 1985. It started as a company founded by students and professors at MIT looking for a better way to analyze marketing data. They ended up inventing the multidimensional database.
The second key event that helped along BI was the endorsement of the OLAP underpinnings by Dr. E. F. Codd, a well-known industry visionary who supported BI market growth by showing that relational databases, OLTP, and SQL will not meet a user's analysis needs ó multidimensional databases and OLAP are needed. Dr. Codd did the industry a great service by validating the OLAP model for analysis.
CJ: Does the Japanese market need your products?
Deputy: They didn't used to. The relationship-based system worked really well here: it used to be a cozy market based on personal networks. If you asked a buchou (general manager) what his information system was, he would have pointed to his kachou (section chief). But the bubble has burst ó the gates opened to the competition ó and, suddenly, they do need the benefits that business intelligence can offer. The greatest reason this market needs our products is because the only thing that is keeping prices high is the lack of information. As companies get the ability and tools to be able to see what parts of the market are not adding value to the process, there will be a radical restructuring of the market.
CJ: What applications does IRI offer?
Deputy: For marketing and finance, there are BI applications that are already developed: for instance, our Dataserver Analyzer is for marketing and FMS is for finance. Dataserver Analyzer, for example, understands market-share calculations and indices, has extensive market-oriented online help, and lets you phrase queries in natural English.
However, if your needs are outside of marketing or finance, then you can build your own BI application using a tool like Express EIS, where Express is the core multidimensional database and EIS is the GUI. The interface could just as easily be Visual BASIC or C as it is a GUI. Late in 1995, we will be releasing a new object-oriented Visual Basic superset interface to Express localized for the Japanese market.
CJ: What kind of problems have you had here?
Deputy: Fortunately, we have our reputation from the US. Everyone knows who we are and takes what we have to say seriously. The problem is a lack of understanding of the benefits. The Japanese market is perhaps eight years behind the US. But it's not just the Japanese companies; some of the multinationals I have talked with haven't yet adopted Windows; they are still using DOS.
How do we convince them of the benefits? We will build up reference sites here, where we showcase our successes. The best way to convince people here is to show them examples of other successful companies that we have helped. We can demonstrate to them how our product and services will have a great impact on the profitability of their operations. We can point to these examples and the benefits they have derived. In this way, we become their partner and gain their trust by showing them the benefits of opening up their data.
ECR in the US could generate $30 billion in savings in a year if everyone did it. And everyone knows Japan's distribution system is more complicated than the US.
CJ: What is the business information future?
Deputy: How will it affect how companies are doing business? Eventually, it will lead to a point where companies are so informed about their customers that they can pinpoint a customer and target his or her particular needs. This will all be stored in a database. They will know you intimately, and the business will become more relationship-oriented. The old-fashioned grocers could do this with a couple hundred customers. In the future, companies can do it with tens of thousands of customers.
