In a previous post titled “BI & Analytics: Top Trends in 2012”, my colleague Peter LePine discussed the macro trends that we anticipate for the BI industry in the coming year. With this follow up post, I want to highlight some of my thoughts on the IBM InfoSphere Information Server suite for data integration, also known as ETL for my fellow old-schoolers. For those unfamiliar, the suite includes integration products such as DataStage, QualityStage, Business Glossary, and Information Analyzer.
Since I have been spending a lot of time in the air traveling to client sites lately, I will employ some airline vernacular to explain the history and direction of the suite as I see it.
Cruising Altitude
“We have now reached our maximum cruising altitude”. For most people, it’s a cue…run to the lavatory, rummage through their overhead bag, pull out their gadgets. For me personally, it means I can finally pry myself from the “sardine” can they call an airline seat, stretch my legs and feel free again… at least until I need to make way for one of those lavatory-seeking passengers.
InfoSphere Information Server has been running at cruising speed for quite a few years now. Kudos go out to IBM and Ascential Software (prior to the acquisition) for continuous investment and focus on the product suite. It is assuring to choose a product that has this level of commitment and dedication from the vendor.
If anyone recalls Torrent Systems, I was fortunate to having worked with their Orchestrate parallel processing engine dating back to the mid-90’s, when it was at a beta validation level. In fact, I worked on implementing the technology on one of the first clients to use it for a large-scale need. About a year after acquiring Torrent in 2001, Ascential integrated the parallel engine into its flagship DataStage product and positioned it to suddenly become a data integration player for organizations of all sizes.
With that leapfrog, DataStage could scale to handle greater and greater volumes of data and compete with the likes of Ab Initio for enterprise grade high-performance processing. Subsequent versions by Ascential, and then by IBM following the acquisition of Ascential in 2005, added more and more creditability to the roadmap. The parallel capabilities continue to extend into more and more components including QualityStage, allowing greater scalability at more stages of processing. Other investments included greater administration and job management capabilities, tighter integration with IBM and other vendor products, solid version control, ability to execute on more operating systems, and end-to-end leveraging of metadata.
I believe this level of progression allows it to be nicely positioned to support the trends and demands of the industry to come.
Ready for Take-off
“Ladies and gentlemen, we are waiting for clearance. We will take off momentarily”. Let’s face it; they don’t know how close we really are to taking off. “Momentarily” could mean anywhere from 1 moment to 120 moments! Yes- I encountered the latter recently, almost wishing for the return of horse-drawn carriages for cross-country travel. I am pretty sure they didn’t charge baggage fees.
Information Server 8.7, the latest version from IBM, brings a number of advancements that align nicely with industry trends. As highlighted in the Top Trends post, Big Data and In-Memory Analytics will be the focus of more and more organizations. Last year saw a significant rise in client interest for this technology, bringing the promise of delivering integration and insight capabilities in a way never previously imagined. Teradata’s acquisition of Aster Data, IBM’s acquisition of Neteeza, the rise of Greenplum, newer appliances such as SAP Hana, Oracle Exadata & Exalytics and XtremeData dbX prove that vendors see the trend as well and are gearing up for a big uptick in client demand.
The days of the same ole’ forms of business intelligence and analytics delivery are over. The Information Server suite can now support big data and Hadoop and has optimized integration with Neteeza, SAS and database products. Despite the desire of vendors to gain the lion’s share of a client’s wallet, I expect that clients will want a best-of-breed class of applications and products. The result will be a heterogeneous environment with a blend of vendors and products. I also anticipate that the Information Server will recognize this and continue to build integration flexibility into their products, including extending it to integrate with more vendor appliances.
Another nice capability with the 8.7 version is more comprehensive information governance. I believe that data quality and governance will be hot topics in the coming year. Organizations are realizing that all of these powerful analytical and BI tools provide exciting opportunities but aren’t much good without trusted data. Information Server is nicely positioned to instill quality in the integration process, integrate with master data management initiatives and allow the same consistent integration policies to be extended to consumers such as real-time services.
Bumpy Air
“Folks, please take your seats, we are about to experience some ‘bumpy air’”. Interpreted…turbulence may be nothing or it could be really bad. Get ready to buckle up, grab your glass in one hand and the dreaded unmarked “bag” in the other. I admit it’s good they use the less unnerving words ‘bumpy air’ rather than turbulence.
IBM has already announced formally that, as of April 30, 2012, a number of older versions of data integration products that were previously packaged under the WebSphere family will no longer be supported. In general, support will end for versions 7.5.0 and 7.5.1 of various products including DataStage, QualityStage, ProfileStage, and MetaStage. For those organizations still on or near those product versions, it’s decision time. This post will hopefully provide some considerations for you to make the decision easier. Last year we encountered many clients struggling with this decision to upgrade. Some were brave enough and took it on themselves. For other clients, we created an approach to help them migrate from older versions of Information Server to the latest and greatest but more importantly help them learn how to take advantage of new features.
A caution to those organizations that are not actively investing to enhance their internal data integration capabilities- the “turbulence” is yet to come. These trends bringing advanced analytical capabilities into your organization are inevitable and if your integration strategy, regardless of the choice of tool, is not in a position to support the potentially complex needs of these initiatives, plan for considerable challenges ahead. This is likely a good time to assess your data integration environment, and really your overall information platform, and make the necessary strategic and tactical investments now in support of anticipated demand.
With good planning we can all hope for a smooth and pleasant journey ahead, whether it’s technology, travel or otherwise.
