Machine Learning and Cognitive Systems, Part 2: Big Data Analytics


In the first part of this series, I described a bit of what machine learning is and its potential to become a mainstream technology in the industry of enterprise software, and serve as the basis for many other advances in the incorporation of other technologies related to artificial intelligence and cognitive computing. I also mentioned briefly how machine language is becoming increasingly important for many companies in the business intelligence and analytics industry.

In this post I will discuss further the importance that machine learning already has and can have in the analytics ecosystem, especially from a Big Data perspective.

Machine learning in the context of BI and Big Data analytics

Just as in the lab, and other areas, one of the reasons why machine learning became extremely important and useful in enterprise software is its potential to deal not just with huge amounts of data and extract knowledge from it—which can somehow be addressed with disciplines such as data mining or predictive analysis—but also with complex problems in which the algorithms used need to adapt to frequent changing conditions. This is the case for successful applications of machine learning techniques in software applications such as those for spam detection, or those from Amazon to automate employee access control or Cornell for protecting animals.


But the incorporation of machine learning techniques within enterprise software is rapidly expanding to many other areas of business, especially those related to business intelligence or analytics, or in general as part of the decision support framework of an organization. As I mentioned in Part 1, as information collection increases in volume, velocity, and variety (the three Vs of big data) and as business pressures to expedite and decrease the latency of the analysis grow, new and existing business software solutions are incorporating improved ways to analyze these large data sets, taking new approaches to perform effective analysis over large and complex amounts of data sets, but most importantly, furthering the reach of what analytics and BI solutions can do.


As data sources become increasingly complex, so do the means of analyzing them, and the maturity model of the analytics BI platform is forced to accommodate the process and expand to the next level of evolution—and sometimes even revolution—of the decision-making process. So the role of a BI and analytics framework is changing from being solely a decision support companion to a framework that can trigger decision automation. To show this, I have taken the following standard BI maturity model from TEC’s BI Maturity and Software Selection Perspectives report (Figure 1) to show in a simple form some of the pressures that this complexity has on the maturity process. As a consequence, the process is expanded to a double-phase decision-making process, which implies giving the system an increased role in the decision.




Figure 1. Standard BI maturity model is being expanded by complexity of data and processes

The decision phase can happen in two ways: as a supported decision made by users, or by enabling the system to delegate the ability to make a decision to itself, automating the decision-making process based on a previous analysis and letting the system learn and adapt. By delegating the decision to the system, for the process extends the reach of analytics to prediction analysis, early warning messaging, and data discovery.


At this stage we might find more permutations of analytics platforms and frameworks that combine both assisted and automated decisions, ideally increasing the effectiveness of the process and streamlining it (Figure 2).



Figure 2. Standard BI maturity model expands to be able to automate decisions

In this context, due to new requirements coming from different directions, especially from Big Data sources in which systems deal with greater and more complex sets of data, BI and analytics platforms become, most of the time, hubs containing dynamic information that changes in volume, structure, and value over time.


In many cases decisions are still made by humans, but with software assistance to different degrees. In some more advanced cases, decisions are made by the system with no human intervention, triggering the evolution of analytics systems, especially in areas such as decision management, and closing the gap between analytics and operations, which can mean boosting tighter relations between the operations, management, and strategy of an organization.
(post-ads)

Opportunities and challenges

The opportunities for implementing machine learning within the context of Big Data, and especially Big Data analytics, are enormous. From the point of view of decision support, it can enhance the complete decision management cycle by



  1. Enhancing existing business analytics capabilities such as mining and predictive which enable organizations to address more complex problems and enhance precision of the analysis process.
  2. Enhancing the level of support for decisions by providing increased system abilities for performing adaptable data discovery features such as detecting patterns, enabling more advanced search capabilities, reinforcing knowledge discovery by identifying correlations, and many other things, much along the same line of what data mining and predictive analytics can do.
  3. Boosting the incorporation of early detection capabilities within traditional or new BI and analytics systems, a key component of modern organizations that want to anticipate or detect short-term trends that might have great impact on an organization.
  4. Enabling the process of enabling a system to perform autonomous decisions, at least at early stages, to optimize the decision process in cases where the application can decide by itself.


Many organizations that already use machine learning can be considered to be exploiting the first level of this list—improving and enabling the analysis of large volumes of complex data. A smaller number of organizations can be considered to be transitioning to the subsequent levels of Big Data analysis using machine learning.

At this point in time, much of the case for the application of machine learning is based on reinforcing the first point of the list. But aside from its intrinsic relevance, it is, in my view, in the area of early detection and automation of decisions where machine learning has a great deal of potential to help boost BI and analytics to the next level. Of course this will occur most probably alongside other new information technologies in artificial intelligence and other fields.

Many organizations that already have robust analytics infrastructures need to take steps to incorporate them within their existing BI and analytics platforms, for example, building machine learning into their strategies. But organizations that wish to leverage machine learning potential may encounter some challenges:


  1. The complexity of applying machine learning requires a great deal of expertise. This in turn leads to the challenge of gaining the expertise to interpret the right patterns for the right causes.
  2. There may be a shortage of people who can take care of a proper deployment. Intrinsically, the challenge is to find the best people in this discipline.
  3. As an emerging technology, for some organizations it still is a challenge to measure the value of applying these types of advance analytics disciplines, especially if they don’t have sufficiently mature BI and Big Data analytics platforms.
  4. Vendors need to make these technologies increasingly suitable for the business world, easing both deployment and development processes.


Despite these challenges, there is little doubt that over time an increasing number of organizations will continue to implement machine learning techniques, all in order to enhance their analytics potential and consequently mature their analytics offerings.

Some real-life use cases

As we mentioned earlier, there are a number of cases where machine learning is being used to boost an organization’s ability to satisfy analytics needs, especially for analytics applied to Big Data platforms. Following are a couple of examples of what some organizations are doing with machine learning applied to Big Data analytics, which surprisingly are tied to solving not complex scientific projects but more business-oriented ones. These cases were taken from existing machine learning and Big Data analytics vendors, which we will describe in more detail in the next post of this series:

Improving and optimizing energy consumption



  • NV Energy, the electricity utility in northern Nevada, is now using software from Big Data analytics company BuildingIQ for an energy-efficient pilot project using machine learning at their headquarters building in Las Vegas. The 270,000-square-foot building uses BuildingIQ to reduce energy consumption by using large sets of data such as weather forecasts, energy costs and tariffs, and other datasets within proprietary algorithms to continuously improve energy consumption for the building


Optimizing revenue for online advertising



  • Adconion Media Group, an important Media Company with international reach, uses software from machine learning and Big Data analytics provider Skytree for ad arbitrage, improving predictions for finding the best match between buyers and sellers of web advertising.


Finding the right partner


  • eHarmony, the well-known matchmaking site uses advanced analytics provided by Skytree to find the best possible matches for prospective relationship seekers. Skytree machine learning finds the best possible matching scenarios for each customer, using profile data and website behavior along with specific algorithms.


This is just a small sample of real use cases of machine learning in the context of Big Data analytics. There is new but fertile ground for machine learning to take root in and grow.

So what?


Well, in the context of analytics, and specifically Big Data analytics, the application of machine learning has a lot of potential for boosting the use of analytics to higher levels, and extend its use alongside other disciplines, such as artificial intelligence and cognition. But the applications need to be approached within the context of machine learning as enabler and enhancer, and must be integrated within an organizational analytics strategy.

As with other disciplines, the success of the implementation of machine learning and its evolution to higher stages needs to be ensured by an organization’s extensive adaptability to business needs, operations, and processes.

One of the most interesting trends in analytics is its increasing pervasiveness and tighter relation with all levels of an organization. As the adoption of new features increases the power of analytics, it also closes the gap of two traditionally separated worlds within the IT space, the transactional and the non-transactional, enabling analytics to be consumed and used in ways that just a decade ago were unimaginable. The line between business operations and analysis is blurrier than ever, and disappearing. The new IT space will live within these colliding worlds with analytics being performing at each level of an organization, from operations to strategy.

In upcoming posts in this series, we will address the machine learning market landscape and look at some vendors that currently use machine learning to perform Big Data analytics. And we will go a step further, into the space of cognitive systems.

In the meantime, please feel free to drop me a line with your comment. I’ll respond as soon as I can.

Machine Learning and Cognitive Systems, Part 2: Big Data Analytics Machine Learning and Cognitive Systems, Part 2: Big Data Analytics Reviewed by Jorge Garcia on March 04, 2014 Rating: 5

31 comments:

  1. I just got to this amazing site not long ago. I was actually captured with the piece of resources you have got here. Big thumbs up for making such wonderful blog page! uniqson

    ReplyDelete
  2. Just saying thanks will not just be sufficient, for the fantasti c lucidity in your writing. I will instantly grab your rss feed to stay informed of any updates. big data analytics

    ReplyDelete
  3. Thanks for the informative and helpful post, obviously in your blog everything is good.. big data

    ReplyDelete
  4. Remarkable article, it is particularly useful! I quietly began in this, and I'm becoming more acquainted with it better! Delights, keep doing more and extra impressive! KISS English Center

    ReplyDelete
  5. The achievement of a business, all things considered, relies upon a logo structure not on the grounds that it looks great but rather on the grounds that it helps with showcasing and business development.
    logo design service

    ReplyDelete
  6. How Artificial Intelligence Company Helps To Collect Data ? GTS AI An Artificial Intelligence Company is responsible for delivering excellent services to their clients. We deliver services in a timely manner to our esteemed patrons. GTS understands the advantages of artificial intelligence and helps its client use it for various purposes.

    ReplyDelete
  7. Very good written article. It will be supportive to anyone who utilizes it, including me. Keep doing what you are doing – can’r wait to read more posts. click here

    ReplyDelete
  8. One of the most prominent issues the students have to deal with while writing assignments is plagiarism. Hence, they extensively use plagiarism checker free to check if there are any copied content in the paper.
    Another major reason for using plagiarism checkers is that universities do not accept plagiarized content. Plagiarism is a serious offence. Hence, if found, the students are suspended or might even lose the grades. Choose Myassignmenthelp.com.

    Due to these limitations, it is evident that the plagiarism checking & wordcounter tool are not at all effective to check plagiarism. The term plagiarism is actually very broad. It is merely not coping with words. But these tools, unfortunately, detect words but not ideas. Hence, the chances of plagiarized papers remain.

    ReplyDelete
  9. This post is not just informative but impressive also. The post is so convincing that it created an urge to choose Nursing Assignment Help services. You can email us at cs@Myassignmenthelpau.Com or Phone Number: +61-2-8005-8227

    ReplyDelete
  10. Great write-up! All doubts regarding Assignment Help solved after reading the post. Sometimes, it becomes difficult to manage the time for writing homework due to some unavoidable reasons. I have recommendation for website greatassignmenthelp for ordering best writing services to complete your academic task.
    Assignment Helper
    Assignment Help Online
    Online Assignment Help
    Assignment Help Online Services
    Assignment Assistance
    Assignment Help Experts
    Assignment Writing Help

    ReplyDelete
  11. Thanks for your info, Get network assignment help online, Our online availability ensures that you get network assignment Help cleared while sitting at a single place. You get a chance to communicate with our professionals while resting at your comfort zone. Hence call up, live-chat or email your computer science issues to release your stress and worries without any hassles.
    Network Assignment Help
    Network Homework Help

    ReplyDelete
  12. Livewebtutors aims at providing the students with a one stop solution for all the academic related issues. The company has
    Livewebtutors is an online Assignment Help and assignment writing service offering expert assignment help. Hire Australian assignment writing experts.
    visit now:- Assignment Help

    ReplyDelete
  13. Are you constantly trying to look for perfect ArcGIS Assignment help services online? No worries! We, at MyAssignmenhelp.com can assist you with the finest of all solutions on time. Being one of the renowned assignment writing service providers, we are dedicated to offering comprehensive assignment assistance on any subject matter associated with the ArcGIS project.

    ReplyDelete
  14. Livewebtutors aims at providing the students with a one stop solution for all the academic related issues. The company has Livewebtutors is an online Assignment Help and assignment writing service offering expert assignment help. Hire Australian assignment writing experts.
    visit now:- Assignment Help Australia

    ReplyDelete
  15. Thunderbird email convert to outlook software serves to migrate from Thunderbird to Outlook with no hassle, providing fast and accurate result. Convert MSF files

    ReplyDelete
  16. Livewebtutors aims at providing the students with a one stop solution for all the academic related issues. The company has Livewebtutors is an online Assignment Help and assignment writing service offering expert assignment help. Hire Australian assignment writing experts.
    visit now:- Tafe Assignment Help

    ReplyDelete

  17. You did really good work. I really appreciate your new and different post.If you are looking for Quicken Technical Support, we are always available for your support. We provide Quicken Customer Service Phone Number for your help and query. Quicken is a personal finance management tool, with this tool you can manage your money easily. With this tool you can view your invest your money. For example you can view your banking, other investment, retirement and credit card accounts. You can manage all type of investment in one place. In this money management tool if you face any type of problem or you want to ask any type of query you can freely contact with our customer service. Our Quicken Customer Serviceteam solves your problem or query as shortly as possible.

    quicken error cc-506
    Quicken Error CC-800
    Quicken Error OL-221-A
    Quicken Password problems
    Quicken one step update not working
    Quicken Online Backup
    quicken cloud sync errors
    quicken can't open your data file

    ReplyDelete
  18. Credit card debts are normally unsecured as for any normal credit cards an individual is not required to give any collateral or mortgage an asset.

    ReplyDelete
  19. Hii i am donaldgeorge and i am a senior technician in Quickbooks .If you face any technical issues in quickbooks then must dial Quickbooks Support to get an instant solutions.

    ReplyDelete
  20. Online assignment writer provides essay writing NZ help to write your college essay assignment accurately without any duplicity and provide 100% unique essay help in Students assignment help.com. Students assignment help.com provides active customer support with highly professional writers.

    ReplyDelete
  21. Significance Of Ai Datasets. How Gts Can Provide Good Quality Of Ai Datasets For Ml Models? “Artificial Intelligence”: The Maximum Mentioned Topic Of The 12 Months With Globalization And Industrialization We Want To Automate The Strategies So That Performance Can Be Increased Inside The Average Perspective For Which We're Using The New Concept Which Has Emerged Called Artificial Intelligence

    ReplyDelete
  22. Great Article! One of the 10 Free Tarot Apps to give you accurate predictions is Tarot Life! Try this amazing app today!

    ReplyDelete
  23. Thanks for sharing this information. We need such information to create CDR Reports. You can get complete CDR Report Writing Service from us. We also provide CDR Reports in Australia, Summary Statement Writing and Career Episode Writing.

    ReplyDelete
  24. In any case, I didn't let this standard me. I began looking into on all the aeronautical organizations and the activity profiles they are advertising.data science course in pune

    ReplyDelete
  25. Avail assignment experts for academic writing services with APA, MLA citation style at lowest price. Achieve your targets of academics and get higher marks in college, our experts provides you unique academic writing services support in limited time.

    ReplyDelete
  26. Thank you so much for helping me out to find the Data analytics course in Mumbai Organisations and introducing reputed stalwarts in the industry dealing with data analyzing & assorting it in a structured and precise manner. Keep up the good work. Looking forward to view more from you.

    ReplyDelete

  27. Always so interesting to visit your site.What a great info, thank you for sharing. this will help me so much in my learning.

    data science training

    ReplyDelete
  28. It’s really a cool and useful piece of information. I’m glad that you shared this useful information with us. Please keep us up to date like this. Thanks for sharing.
    and Are you looking Latest Web Development Trends tips - https://www.pnjsharptech.com/blog/latest-web-development-trends-level-up-coding-skills/

    ReplyDelete
  29. It’s really a cool and useful piece of information. I’m glad that you shared this useful information with us. Please keep us up to date like this. Thanks for sharing.
    and Are you looking Latest Web Development Trends tips

    ReplyDelete
  30. Thank you for sharing this informative post.MyAssignmenthelp.co.uk is giving assignment help to students.we are already trusted by thousands of students who struggle to write their academic papers and also by those students who simply want law dissertation help to save their time and make life easy.

    ReplyDelete

Powered by Blogger.