Marketing and Business Development

Robert Eve

Subscribe to Robert Eve: eMailAlertsEmail Alerts
Get Robert Eve: homepageHomepage mobileMobile rssRSS facebookFacebook twitterTwitter linkedinLinkedIn

Related Topics: Virtualization Magazine, SOA & WOA Magazine, ERP Journal on Ulitzer

Virtualization: Article

Virtualization - Fast-Forward to Today's Virtual Data Marts

Don't get stuck in the slow lane

Fast Forward to 2008
Today, virtual data marts based on more advanced data federation technology are gaining in popularity. In a recent report, “Data Federation Adoption Increases as Part of Complete Data Integration Strategy,” Stamford, Conn.-based industry analyst firm Gartner highlights increased production deployment of virtual data marts. In its report, Gartner recommends that

“Information architects and data integration designers/developers should acknowledge data federation capabilities as an increasingly important component of a comprehensive data integration strategy. Organizations should adopt a portfolio of data integration tools which support a range of data delivery styles, and federated views are one of those styles.”(1)

What has changed to enable virtual data marts to move to the mainstream?

Moore’s and Mores Laws
For one thing, computing resources are faster and cheaper, as described by Moore’s Law. Think about it. In 1998, the 350 Megahertz Pentium II was the industry’s standard microprocessor. Today, the standard is a 3 Gigahertz Xeon Quad Core with each of its four CPUs running 10 times faster than the single CPU Pentium II. Similar advancements in network bandwidth have added significant available computing resources

In addition, enterprises have witnessed another “Mores” law in the form of more software, more data, and more complexity. Enterprise ERP adoption that accelerated in advance of Y2K, new Web-based applications for e-commerce and internal uses, new types of BI such as operational BI and performance management analytics have all contributed. Reporting on this data explosion, a recent study by University of California, Berkeley (2) estimates a forty- to fiftyfold rise in enterprise data over the last decade. More complexity includes more data types, spanning structured relational data from transactions and analysis systems, semi-structured XML documents from the Web and trading partners, and desktop-generated data such as unstructured e-mail, spreadsheets, PowerPoints, images, audio and video files, and more.

Innovations Provide Additional Breakthroughs
More powerful computing resources alone have not been enough to overcome the data integration challenges resulting from more applications, data and complexity.

New algorithms for high-performance query, based on advanced mathematical and probabilistic models, have been developed and deployed on more powerful hardware and bandwidth to overcome many of the latency and source system performance concerns. In addition, industry adoption of new data integration standards such as ODBC, JDBC, and SOAP helped simplify the task of passing data from diverse sources to diverse consumers. Finally, model-driven design tools, originally the province of complex CASE tools, have become commonplace. These factors have helped developers become more comfortable using abstracted schemas as the loosely coupled intermediary between diverse data source and uses.

Customers Demand New Approach
Interestingly, the successful adoption of ETL and physical data marts has also helped drive the adoption of virtual data marts. As these tools fulfilled demand for complex BI projects, especially those where multidimensional data analysis is required, other types of BI demands have moved up the priority list.

A second driver: companies with advanced data management practices typically store, at most, 10 percent of their data in marts, operational data stores and warehouses. Therefore, replicating all data is only a partial answer.

Third, more data marts mean more data silos, requiring more data integration. Instead of fixing the problem, companies have made things more complex.

Finally, ETL and physical marts take longer to build, cost more to support, and are more difficult to change than virtual marts. As such, business users have become less tolerant of these delays and charge-backs. IT wanted new answers, prompting a second-generation of virtual data marts.


Virtual Data Marts Solutions Today
Enabled by a middleware technology known as distributed query, data federation, or enterprise information integration (EII), today’s virtual data marts provide three key capabilities:


  • Data virtualization serves up data as if it is available from one place, regardless of how it is physically distributed across data silos. Virtualized data is available whenever it is needed, and, therefore, it’s always up to date.
  • Data abstraction simplifies complex data by transforming it from its native structure and syntax into reusable views and Web services that are easy for business solutions developers to understand and business solutions to consume.
  • Data federation securely accesses diverse operational and historical data, combining it into meaningful information for BI or other uses. Query optimization and other techniques such as caching enable high performance.

At build time, virtual data mart solutions provide an easy-to-use data modeler and code generator that abstract data to the form of relational views for reporting and other BI uses or Web data services for SOA initiatives, portals, etc.

At runtime, virtual data marts execute high-performance queries that securely access, federate, transform, and deliver data to consuming business solutions in real-time.

While typically applied to BI use cases, virtual data marts are also applied across a range of initiatives that require data integration, including service-oriented architecture (SOA), customer data integration (CDI)-master data management (MDM), and more. Further, virtual data marts can be deployed on a project or enterprise-wide level.



More Stories By Robert Eve

Robert "Bob" Eve is vice president of marketing at Composite Software. Prior to joining Composite, he held executive-level marketing and business development roles at several other enterprise software companies. At Informatica and Mercury Interactive, he helped penetrate new segments in his role as the vice president of Market Development. Bob ran Marketing and Alliances at Kintana (acquired by Mercury Interactive in 2003) where he defined the IT Governance category. As vice president of Alliances at PeopleSoft, Bob was responsible for more than 300 partners and 100 staff members. Bob has an MS in management from MIT and a BS in business administration with honors from University of California, Berkeley. He is a frequent contributor to publications including SYS-CON's SOA World Magazine and Virtualization Journal.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.