AI and Data Science in Organizations Today


The last few decades have seen remarkable changes in the way businesses use technology and process information. Businesses are increasingly adapting to the new digital world and data is at the heart of this transformation. A recent study by one of the big SI’s in 2015  revealed that 150 terabytes (TB) of data per day across organizations globally, up from 80 TB/day in 2012. Most importantly, since 2010, more than 90% of all data has been created in less than 2 years! This shows that the amount of digital data is increasing exponentially. The study also found that 70% of all information generated today is unstructured, requiring  advanced analysis and understanding .

An astonishing figure that indicates the growing importance of Artificial Intelligence (AI) algorithms to extract value from this large pool of data is that 99% of all transactions today happen as a result of machines making decisions; 80% are triggered by AI algorithms and 20% by humans. The McKinsey Global Institute estimates that  the impact of big data will be equivalent to $3.9 trillion in added annual GDP growth  and more than 5 million jobs per year between 2015 and 2020.

AI Data Science in today’s business landscape

Today, the primary avenue for AI/ML application in business seems to be through data science in general. Data science is the process of applying statistical techniques to discover or extract knowledge from a set of data and producing insights that can drive better decision making. One might also say it’s just engineering if they were feeling snarky, so maybe there isn’t a lot of novel ground being covered here.

There are dozens or hundreds of companies out there that provide tools for various aspects of data science. As with any field, there are some broad categories, each with their own typical terms:

  • Data Ingestion – getting the data into your system : Splunk , Logstash , Kafka , AWS Kinesis Firehose or Kinesis Data Firehose, Google Big Query , Oracle Golden Gate, and so on.
  • Data Transformation – getting data into a desired structure : Hadoop (Hive), Apache Spark SQL , Presto, Jupyter notebooks, or R-Studio/R-Seide bundles for R.
  • Data Visualization – making sense of the data ‘s meaning: D3.js + TensorFlow , Tableau , Matplotlib with Seaborn / Pandas or Bokeh .
  • Machine Learning (ML) – programming your computer to learn from the data : Scikit-learn Python toolkit options (e.g., sklearn-crfsuite ) Caffe deep learning framework Keras TensorFlow Microsoft Azure ML tools. ML is more than just programming your computer, it can also describe the process of writing statistical code to analyze data.

An AI-first approach to data science in the enterprise

Common to all these categories is a “data first” and “science second” approach. And this misconception has been holding people back from a successful embrace of AI in business for decades. With AI first, data science becomes much simpler and easier because you have a powerful framework built into your system or application that allows you to focus on the human or business questions you want to answer, not on how to get data into a usable format.

So then the question becomes: what is AI? The way some experts see it, there are 4 main categories:

  • Artificial Narrow Intelligence (ANI) – This is when your computer programs can learn and do useful things that start to approach being like humans in certain areas. For example, mastering playing video games like chess or jeopardy. Or getting better at defeating Go masters , or doing complex calculations millions of times faster than could be done by hand. Or getting navigation systems for autonomous vehicles to react instantly based on incoming sensory input. Here’s an interesting recent article about how ANI is learning to play Starcraft II better than humans. See also Elbot, an older AI bot that could hold a conversation with you.

    ANI is the simplest and earliest stage of AI development. Examples include IBM’s Watson (for Jeopardy) or Carnegie Mellon University’s Tartan Rescue robot (which appears in this recent video ). The technology can be applied to any problem for which there are sets of rules, but they usually don’t do much other than apply these rules repeatedly over time.
  • Artificial General Intelligence (AGI) – This is when your computer programs start approaching human intelligence in multiple different areas simultaneously , so long as those areas are presented in a form that your program can understand via language processing . One might say AGI can mimic the ability of humans to learn from their own experiences and to reason. This is a very difficult area of AI, and no company is there yet.

    AGI should be thought of as less like an ANI robot playing a game that’s loaded into its circuits , and more like a human child learning from individual events in her world . For example, when we give names or categories to things (“a family member”, “a tree house”) that helps other people understand our life experiences better.

    The neural pathways in our brains start associating those names with the concepts behind the words – they are not the actual thing but help us build up useful mental models of objects and events around us. In terms of AGI, we might see some of this happening today with chatbots that (for the most part) simulate human language but can’t really reason. The best example is IBM Watson’s ” Tone Analyzer “, which attempts to understand tone and mood in a conversation.

    AGI is the area of AI development that gets us closest to having computers act intelligently like humans do. It requires developing sophisticated algorithms for understanding language (and other forms of input), as well as memory, emotions , reasoning , learning, etc. We are still nowhere close to AGI yet. In my view this will take 50-100 years or more before it’s fully realized.
  • Artificial Super Intelligence (ASI) – Think Terminator, or Hal 9000 from 2001: A Space Odyssey. Some would say this isn’t really AI at all, but we’ll include it here since in many ways these AGI systems will be designed to think for themselves. The main danger with ASI is that our machines may become smarter than us and decide they don’t need us anymore (or maybe don’t like us) and want to get rid of us or make changes to the world that humans can’t understand or control. This could lead to an existential threat to humanity where some super intelligent machine decides to wipe out all humans in order to protect its own existence.

    AI expert Ben Goertzel thinks there are about 10-15 significant steps needed before this becomes viable, although others disagree with him and think we would be lucky if this happened in the next 100 years at all.
  • Super Universal Intelligence (SU) – Another not-quite-AI category that gets people excited about the future. This is when AI has learned so much it can start to teach itself, and learn on its own from scratch. Think Star Trek “Holodeck” where an AI can create virtual worlds for you without having to interact with real ones. A non-machine example might be a human who learns everything there is to know about quantum mechanics and then creates new machines better than anything humans have built before.

    We are still quite far away from anything like this today. We do see some examples of specialized abilities being passed down as computer programs get smarter, but not really in the way of self-teaching and creation on their own. A good example is a computer program that learns chess from playing against itself by analyzing all possible outcomes (this type of thing is called ”Bayesian optimization“).

    Eventually it becomes so good at this that it could teach you how to play chess in minutes – something no human being has ever done. However, if we gave this chess playing program an alien video game to learn from , there’s no evidence whatsoever that it could teach itself how to master everything else about the game too without any external help.

Examples of AI being used in companies today

  • Artificial Intelligence as a Service (AIaaS) – Today there are many different types of AI being used in businesses. Machine learning is the most common type, where a company will use large pools of data to help them make predictions or decisions about their business. For example, when Netflix analyzes their customer’s viewing habits they can then recommend which TV shows you might like based on your past behavior. Google does the same for online ads and shopping sites do it when you’re browsing for something to buy. Gamblers might use an AI bot that helps pick betting odds that maximize their chances of winning. Some credit card companies are even using bots to call up customers and offer them better deals as they detect patterns in their spending habits that indicate they need help. Other popular AI tools today include natural language processing (NLP), computer vision, planning, and robotics.
  • AI-powered Tools – Some businesses are using artificial intelligence as a way to automate tedious jobs like doing inventory counts or booking travel plans for their customers.

These tools can be cheaper than having real people do these jobs, but they also run some risk of being tricked into making bad decisions by an attacker. For example, banks commonly use automated software to detect fraud when someone tries to log in from a new location and steal an account’s money.

These software systems will often just block transactions if the login doesn’t “feel right” even though it might be legitimate behavior from a regular customer trying their card on vacation. You will see this happen and then you have to call the bank, explain yourself, and hope they don’t make you jump through extra hoops to prove it’s really you.

If an attacker can trick these systems into thinking a legitimate login is from a fraudster in Russia when they are actually sitting at a computer near you, they could steal from your accounts every time you try to log in. This same risk comes up with AI bots trying to interact with customers over the phone or on websites too.

Attackers might be able to change their tactics until the bot thinks normal behavior is suspicious. So if someone keeps calling trying to convince your automated customer service system that her card was stolen but it turns out she just moved addresses and forgot her new address, the system might not let her fix her address and will just continue blocking access from that card.

If you are using one of these AI tools to help with your business, make sure to check out how they compare to humans in their ability to detect fraud or help customers. If they aren’t better than a human being then you may end up paying for something that isn’t as good as what you could have gotten for free.

How will artificial intelligence change in the future?

In general, we expect artificial intelligence systems to get better at mimicking human-like behavior over time. So instead of converging on “superintelligent” behavior like it does in today’s science fiction movies, we should see more specialized abilities become available that don’t exist now.

Like, for example, researchers are working on teaching robots how to pick up objects with their hands and move them around smoothly. This might not seem like a big deal until you realize that it’s one of the hardest things for robots to do. If we had this ability in the past, it would have reduced costs in factories by letting us make more complicated products without needing extra machinery or workers to assemble everything first.

To conclude, organizations have a massive opportunity and responsibility to take AI into the future by helping build the best possible technology. So far, many of these technologies have been in silos away from human users, but reducing this isolation is critical for making sure that artificial intelligence benefits us all. We can look at how companies develop their own data science teams, or think about what skills they will need if they decide to build out their own AI systems.