Skip to main content

TIBCO Spotfire Aims for a TERRific Approach to R

1. very great or intense: a terrific noise
2. (informal) very good; excellent: a terrific singer
The British Dictionary

R is quickly becoming the most important letter in the world of analytics. The open source environment for statistical computing is now at the center of major strategies within many software companies. R is here to stay.

As mentioned in my previous post Microsoft and the Revolution . . . Analytics, R has become the new language of choice for the development of statistical and predictive analysis applications. And due to its open source nature and widespread adoption in the academic world, its popularity is extending far beyond the campus to become the foundation of the world of business.

Thus it’s not unfathomable that Microsoft would acquire Revolution Analytics—in order to expand its presence in the analytics market, and that TIBCO Spotfire would continue to increase its presence in this same area by looking for a strong way to support R.

Still, one thing is worth mentioning, and that is the way TIBCO has approached R. Taking the time to work through a very neat and clever approach, and not just jumping on the R bandwagon, TIBCO has developed its comprehensive strategy for R. As a result, TIBCO has redeveloped R with an enterprise approach, manufacturing a new solution for R that it calls TERR.

This post takes a closer look at TIBCO’s new approach to predictive analytics and R.

TIBCO Spotfire: A holistic approach to predictive analytics

The first noteworthy aspect of TIBCO’s style for providing predictive analytics is the targeting of a wider variety of users, besides data scientists and savvy math personnel. So after a couple of years of re-engineering and intensive work, the TIBCO Spotfire team has come up with a three-component approach —TERR, TSSS, and predictive models, as explained later—to providing predictive analytics solutions to its users.

TIBCO can adjust its offering to meet different set of needs for consumption according to the type of user: application authors and business users:

  • Data Scientists and application authors can make full use of the functionality provided by TIBCO Spotfire in the realm of predictive analysis: create models; use the complete set of Spofire’s tools for developing/using enhanced analytics; develop custom tools and scripts in R, SAS, MATLAB, and others; and build complete analytics applications customized for special needs.
  • Business users can use a set of available and targeted advanced analytics applications that can be embedded within dashboards and other applications (see Figure 1).

Figure 1. R scripting for Ad Hoc analysis  (image courtesy of TIBCO)

Hence, for this purpose, TIBCO has come up with the current TIBCO analytics platform, which is composed of these elements:

  • TIBCO Spotfire Statistics Services (TSSS) provides the core platform and integration with existing predictive analytics solutions, and enables custom application development.
  • TIBCO’s predictive modeling tools enable Spotfire to generate predictive analytics models from scratch.
  • TIBCO Enterprise Runtime for R (TERR) is TIBCO’s native environment for running R scripts.  

This wide yet solid approach has enabled TIBCO to develop a number of solutions in order to provide users with more efficient ways to consume analytics for different scenarios and needs. It also allows users of different backgrounds and levels of expertise with predictive analysis to choose the tool that best fits their needs. And with respect to R, the community of users is growing and becoming a dominant force within the predictive analytics landscape of solutions—something that TIBCO is well aware of.

R is by no means a strange letter for TIBCO

For TIBCO, R is everything but new. A year after acquiring Spotfire in 2007, TIBCO acquired a company called Insightful, and got a hold of a commercial version of one of R’s conceptual predecessors, a language called S and later released as S+. S was developed in the mid-70s by John Chambers, Rick Becker, and Allan Wilks from Bell Laboratories and, despite some key differences, S and R share much the same structure and philosophy on coding and open source ideal. With the acquisition, TIBCO gained much of the required expertise in the development of predictive analytics applications—in essence a more natural approach to R.

TIBCO was fully aware of the tremendous potential and benefits that R could bring to the enterprise, but was not blind in considering its limitations. The California-based company knew that if it wanted to offer a fully commercial and enterprise-ready offering in R—and not just a form of integration—it would be necessary to work on keeping R’s benefits while solving some of its limitations. TIBCO needed to preserve not only R’s memory administration and ability to work with large data sets at optimal performance for the business, but also R’s flexibility to develop powerful and relevant predictive models.

Here is where TIBCO Spotfire’s team got creative.

TERR: Same R, new engine

Taking a “simple approach”, TIBCO decided to just re-engineer R’s runtime engine to develop—from the ground up—a runtime engine that is fully compatible with the R language and has significant improvements at its core. TIBCO called this engine the TIBCO Enterprise Runtime for R (TERR).

With the development of TERR, TIBCO offers users the opportunity to keep developing and using the open source code R as it is. It also enables the execution of R within a proprietary runtime engine that is equipped and fine-tuned with full enterprise and commercial support, thus avoiding the recoding of existing R scripts to adjust to the new engine.

The redesign of the runtime engine enables TIBCO to also offer significant performance improvements, and valuable management of computing resources. The offering has also been architected and designed for 64-bit platforms. It has a new proprietary core engine, but the same R syntax, which users already know.

The runtime engine is licensed and fully supported by TIBCO. In fact, the enterprise support from TIBCO’s team has enabled TIBCO to embed the R code within many of its predictive analytics offerings. Additionally, TIBCO has made TERR a central part of its overall predictive analytics initiatives, allowing users to easily develop new R applications that work on TIBCO Spotfire.

So far, TERR is compatible with approximately 2,000 CRAN packages. Users can thus take advantage of the wide R code library via TIBCO Spotfire’s different levels of integration with the vendor’s Predictive Services, as well as the embeddable service for custom R application development, which is available for many TIBCO products and platforms.

TERR: More than meets the eye

Another beneficial aspect of TIBCO’s approach to consider is that the vendor allows TERR users to evaluate and test the engine. A free developer Edition of TERR, which can be found in its Community site, enables users to test the R code under non-production environments before final deployment. It’s an interesting option to let users test the environment, even a console-only version.

Additionally, and perhaps an aspect with great potential, is TIBCO’s ability to embed TERR within tibbr, TIBCO’s social networking application, and TIBCO Business Events, TIBCO’s event processing solution. This enables users to develop and deploy predictive analytics functionality on top of real-time applications and perform analysis on social interactions. These capabilities carry important benefits: they allow for naturally extending both Spotfire’s predictive analysis reach to a greater variety of audiences and data sources and Spotfire’s predictive analysis use cases to real-time analysis scenarios.

So in contrast to the radical strategies of some other vendors, TIBCO is planning to make a smooth takeover of the growing R user community. It will do this by offering less disruptive tools that are designed from the inside—reducing user disturbance during development while improving performance and management capabilities.

Perhaps one important question to raise is whether TIBCO can remain agile enough so that its re-engineering process can keep pace with not only the continuous evolution of R but also the complete incorporation of existing and new CRAN packages into the TERR core.

In this regard, in an interview, Lou Bajuk-Yorgan, TIBCO’s Sr. Director of Product Management, who is directly involved with TERR’s development, mentioned:

“We regularly test TERR with a wide variety of R packages, and extend TERR to greater R coverage over time. We are currently compatible with ~1800 CRAN packages, as well as many bio-conductor packages. The full list of compatible CRAN packages is available at the TERR Community site at”

Of course, TIBCO Spotfire is aware of the importance of maintaining a close relationship with the R community, as do other players such as Microsoft, to ensure that the evolution of TERR remains in line with the R community, and vice versa.

With TERR, TIBCO wants on the one hand to tighten the R licensing, gaining the ability to maneuver and improve its proprietary engine, while on the other hand to provide full support for users but let them keep working on an “open source mode” during the design and programming phases of development.

Certainly TIBCO has realized an outstanding idea. The implications are for practicality, efficiency, and agility—if TIBCO can remain firm without falling to the temptation, or being forced, to follow widely separate ways with the R community forced by commercial needs—converting R to yet another niche programming platform.

Finally, you can take a look at a demo provided by TIBCO, using TERR under the TIBCO Spotfire platform.

Cross-sell revenue optimization demo (Courtesy of TIBCO)

Have a comment? Please leave me a message below, and I’ll respond to you as soon as I can.


Popular posts from this blog

Machine Learning and Cognitive Systems, Part 2: Big Data Analytics

In the first part of this series, I described a bit of what machine learning is and its potential to become a mainstream technology in the industry of enterprise software, and serve as the basis for many other advances in the incorporation of other technologies related to artificial intelligence and cognitive computing. I also mentioned briefly how machine language is becoming increasingly important for many companies in the business intelligence and analytics industry. In this post I will discuss further the importance that machine learning already has and can have in the analytics ecosystem, especially from a Big Data perspective. Machine learning in the context of BI and Big Data analytics Just as in the lab, and other areas, one of the reasons why machine learning became extremely important and useful in enterprise software is its potential to deal not just with huge amounts of data and extract knowledge from it—which can somehow be addressed with disciplines such as data

Next-generation Business Process Management (BPM)—Achieving Process Effectiveness, Pervasiveness, and Control

The range of what we think and do is limited by what we fail to notice. And because we fail to notice that we fail to notice there is little we can do to change until we notice how failing to notice shapes our thoughts and deeds. —R.D. Laing Amid the hype surrounding technology trends such as big data, cloud computing, or the Internet of Things, for a vast number of organizations, a quiet, persistent question remains unanswered: how do we ensure efficiency and control of our business operations? Business process efficiency and proficiency are essential ingredients for ensuring business growth and competitive advantage. Every day, organizations are discovering that their business process management (BPM) applications and practices are insufficient to take them to higher levels of effectiveness and control. Consumers of BPM technology are now pushing the limits of BPM practices, and BPM software providers are urging the technology forward. So what can we expect from the next

Teradata Open its Data Lake Management Strategy with Kylo: Literally

Still distilling good results from the acquisition of former consultancy company Think Big Analytics , Teradata , a powerhouse in the data management market took one step further to expand its data management stack and to make an interesting contribution to the open source community. Fully developed by the team at Think Big Analytics, in March of 2017 the company launched Kylo –a full data lake management solution– but with an interesting twist: as a contribution to the open source community. Offered as an open source project under the Apache 2.0 license Kylo is, according to Teradata, a new enterprise-ready data lake management platform that enables self-service data ingestion and preparation, as well the necessary functionality for managing metadata, governance and security. One appealing aspect of Kylo is it was developed over an eight year period, as the result of number of internal projects with Fortune 1000 customers which has enabled Teradata to incorporate several be