Technology the enabler


Technological requirements for business intelligence and predictive analytics applications involve a generational leap. The change comes as a result of the multi-dimensional nature of the beast. Operational CRM was content to manage single dimensional information such as a sales person’s leads, pipeline and performance at a point in time. Business Analytics add complexity by requiring historical data to be able to compare performance over a period of time. This could go further if the company requires performance comparisons across regions. In addition, the data could become cross-functional as companies look at the financial impact of sales performance. The volume of the data that has to be managed grows exponentially as the complexity of the queries grows. Finally, the very nature of predictive analytics is to conduct a variety of different queries which conflict with the more static design of CRM applications. The need to integrate information from a variety of departments and regions stretches the capability of CRM applications further. Most times, companies have no choice but to switch to data warehouses.

Companies can gain an edge from predictive analytics by completing the cycle beginning with collection of data followed by its analysis and concluding with decisions as quickly as possible. Time delays can occur as data is transferred from the operational sources or external sources to a central point such as a data warehouse. Again, analysis of data contributes to latency. Finally, time is lost when decisions are taken based on the analysis conducted. Technology can help to lower the costs and time delays in the first two stages while decision latency is hard to reduce except when companies take action based on rules. Marketing campaigns thrive on making offers as quickly as possible often responding to events, such as the release of the latest Harry Porter novel, as they unfold.

The timeliness and quality of the data are critical for wider adoption of business intelligence and analytical tools in business. According the recent survey of 6000 marketing and IT executives, data (quality, access, timeliness and usage) were considered to be the most important barrier to the implementation of customer relationship management software.

At one end, companies choose data warehouses which are batch systems and use ETL (extract-transform and load tools) for transferring data from operational sources to the central warehouse. The extraction of data involves conversion of the data structure in the source files to a flat file, the transformation turns the flat files into the data structures of the target database. Finally, FTP (File Transfer Protocol) helps to transfer data from the source to the target database. ETL accounts for 60-80% of the time spent on BI projects. While a data warehouse accumulates multi-dimensional data which is clean, the time delays are far too long for companies to be able to respond to events as they happen.

The time delays in the transfer of data from transactional data sources to the data warehouse take place as a result of the custom coding that has to often take place when data is converted from especially formats that are not widely used such as legacy systems, data stored in applications and proprietary software. As the variety of data incorporated in data warehouses increases, so does the need to take recourse to custom coding. In addition, data warehouses can either load data or be used for analytical processing at any given point of time. The conflict between analytical processing and real time data is aggravated as larger volumes of data have to be transferred, on the one hand, and the demand for increasingly complex queries grows on the other.

The time delays can be shortened by tools that automate the process of conversion of data. These tools employ proprietary scripting languages running within an ETL or DBMS server which use language interpreters, stored in a meta-data repository, to process the incoming data. These engines, however, cannot process unique data structures and need code written by humans to process the data.

Another recent trend is the use of Enterprise Application Integration software which uses web services to interlink heterogeneous platforms and help to transfer transaction data from them in near real time. These systems are not as efficient as ETL tools in transformation of the data so some companies have taken the initiative to integrate these two types of tools.

Companies which are interested in shortening the time delays have to consider other means. One approach is to deploy data warehouse appliances, such as Netezza, which bundle the storage, database server and campaign management server in a single system, which substantially increases query performance without raising the investment costs.


Related Information

 

  
Home | Contact | All Rights Reserved. © Marketing-Analytics.net