Haley AI-as-a-Service and Tensorflow

There is a lot of excitement in the Machine Learning world around Deep Learning and Neural Networks, and one of the most popular libraries available for creating Deep Learning models is Tensorflowhttps://www.tensorflow.org/ ).

What follows is an example of using a Tensorflow model within a Haley AI-as-a-Service dialog.

A few quick notes about Haley AI-as-a-Service:

Haley provides a software platform for Artificial Intelligence Agents, which enables automation of business processes — such as chatting with a customer service agent, reacting to Internet-of-Thing data to control devices, or classifying loan applications for risk assessment.  Haley integrates with a number of endpoints, which are the sources and destinations of Haley messages.  These endpoints include email, web applications, mobile applications, Facebook, Twitter, Slack, SMS, IoT Devices, Amazon Alexa, and others.  Communication with an endpoint is over a channel which groups messages together, such as a channel #sales-team for communication among members of a sales team.  AI Agents, called bots, receive messages and send messages on channels, and bots use dialogs  a series of steps to handle events — and workflows — business processes that may be composed of many dialogs — to accomplish tasks.

Haley AI-as-a-Service and ML Models:

Haley AI-as-a-Service supports a variety of machine learning models including those from Apache Spark MLlib and Tensorflow.  We also have support for libraries such as Keras used with Tensorflow and BigDL from Intel ( https://bigdl-project.github.io/ ) used with Spark.  Based on customer demand, we continue to add support for others.

In this example, first, we’ll create a model to classify text.  Then, we’ll use this model in a dialog to classify the text that occurs in a channel.  To see the results, we’ll use a chat web application to communicate on the channel.  This classification of the text into a topic could be used in the dialog to react to what a person is saying, but in our short example we’ll just report back the topic.  So, if someone says something like: “My ford needs an oil change”, we want to classify this to be about “Cars” and respond, “Your message appears to be about Cars.”

Creating the Tensorflow Model

For the tensorflow model, we’ll be using the Keras Deep Learning Library ( https://keras.io/ ) running on the tensorflow backend.

There is a great tutorial here: https://blog.keras.io/using-pre-trained-word-embeddings-in-a-keras-model.html  which covers creating a text classification model which uses word2vec-style word embeddings using the GloVe word embeddings from Stanford: https://nlp.stanford.edu/projects/glove/ .  This tutorial uses the 20news dataset which consists of around 20,000 documents equally divided into 20 categories from USENET postings.

The categories include:

(and 16 others)

The complete code for training the model can be found here:


The critical training part of the code is:

print(‘Training model.’)

# train a 1D convnet with global maxpooling
sequence_input = Input(shape=(MAX_SEQUENCE_LENGTH,), dtype=’int32′)
embedded_sequences = embedding_layer(sequence_input)
x = Conv1D(128, 5, activation=’relu’)(embedded_sequences)
x = MaxPooling1D(5)(x)
x = Conv1D(128, 5, activation=’relu’)(x)
x = MaxPooling1D(5)(x)
x = Conv1D(128, 5, activation=’relu’)(x)
x = MaxPooling1D(35)(x)
x = Flatten()(x)
x = Dense(128, activation=’relu’)(x)
preds = Dense(len(labels_index), activation=’softmax’)(x)

model = Model(sequence_input, preds)

model.fit(x_train, y_train,
validation_data=(x_val, y_val))

We can use Jupyter ( http://jupyter.org/ ) to run this to create the model, and then save the model to a file.

In a production application, the model would be trained within Haley, with new models swapped in on an ongoing basis, but in this example we are uploading the trained model file from a local machine.

Here’s a screenshot from within Jupyter:


Note we’re saving the model with model.save() at the end of the training.  We’ve also turned on logging for tensorboard in the above screenshot.

The tutorial reports an accuracy of around 95%.

Once we have our trained model file we upload it to the Haley Admin Dashboard and deploy it.  Now we’re ready to call it from a dialog.

Creating the Dialog


The above screenshot is the Haley Dialog Designer tool, which is a visual drag-and-drop interface to create dialogs.  We drag-and-drop and configure a handful of steps to create the dialog.  The important ones are:



This step in the dialog gets a text message on the channel and puts it into a fact variable called textFact



This step in the dialog (shown selected, with it’s Configure panel on the right) calls the tensorflow model passing in the parameter textFact which is classified by the model, putting the results into the variable classifyResults.





This step in the dialog sends a message out on the channel, reporting back the classification using the classifyResults fact.



For reporting back the classification, we take the top-most category and its score and send the message:  “That appears to be about: $category with a score of $score”.  For diagnostic purposes we also send the full list of results back in a JSON list.

Once we’ve created the dialog, we then need to connect it to a bot and a channel.

Haley Admin Dashboard, Bot Screen:


Here in the Haley Admin Dashboard, we create a new bot that just contains our new dialog, and set the dialog as the default, so it is the default action for messages that the bot receives.

Haley Admin Dashboard, Channel Screen:


And here in the dashboard we connect the bot up to the channel “docclassify”.  Now, any user which has access to that channel over an endpoint, such as in a web application, can send messages on the channel and access our new classifying bot.


Using the Tensorflow Model in a Chat Interface


Now, by logging into a web application connected to Haley we can see the available channels on the left, select the “docclassify” channel, and send a message like:

Person: “sam hit the ball over the fence for a homerun”

and we get our answer back:

Haley: “That appears to be about: rec.sport.baseball with a score of 0.7076626”

We also send the complete classification and score list for diagnostics:


Based on the score, the “baseball” category is the far winner, with a score of 0.70 compared to the next best score of 0.07 for “motorcycles”, so the model is roughly 70% “sure” that the correct answer is “baseball”.

Using other Tensorflow & ML Models on Haley AI-as-a-Service

In this example, we’ve created a new model, trained it, and uploaded it to Haley.

If you would like to incorporate ML models into Haley AIaaS, there are a few options:

  • You create the model, train it, deploy it on Haley AIaaS as we have done in this example
  • Vital AI creates the model, trains it, and/or deploys it, for you to use
  • Use an “off the shelf” model that Haley already uses or one taken from open sources, potentially training it with your data

Additionally, the training of the models can take place on our infrastructure — this is particularly useful for ongoing training scenarios where a data pipeline re-trains the model to incorporate new data periodically, or an external vendor could be used for this training, such as Databricks or Google, with some additional data coordination to share the training data.  To reduce latency, it’s usually best that the “inference” step (using the model to make a prediction) is as closely integrated as possible, thus it is usually best that this resides within Haley AIaaS, although there can always be exceptional cases.


In this example, we have:

  • Trained a text classification model using Tensorflow and Jupyter
  • Uploaded the model and deployed it using the Haley Admin Dashboard
  • Using the Visual Designer, created a dialog that uses the model to classify incoming text messages
  • Added dialog steps to generate response messages based on the classification, and connected the dialog to a bot, and connected the bot to a channel
  • Used a web application logged in to Haley to send messages on the channel and receive replies

This example can be extended in many ways, including:

  • Connect to other endpoints besides a web application such as classifying Tweets, Facebook Messages, EMails, SMS Messages, and others
  • Use a Tensorflow model to process different types of messages, such as those from IoT devices or images
  • Use a generative Tensorflow model that creates a response to an input rather than classifying the input.  Such models can generate text, audio, images, or actions — such as a proactive step to prevent fraud
  • Add Tensorflow models to workflows to incorporate them into business processes, such as processing insurance claims

If you would like to incorporate Tensorflow or other ML Models into Haley AIaaS, you could create the model, we at Vital AI could create it for you, or an off the shelf model could be used.

To train the model, either you could train it, we could train it on our infrastructure, or a third party vendor could be used — such as Google’s Cloud ML Engine for Tensorflow.

I hope you have enjoyed learning about the Haley AI-as-a-Service platform can utilize Tensorflow Models.  Please contact us to learn more!

Vote for Haley AI-as-a-Service to speak at Botscamp

Voting is open for one more day at Botscamp to select speakers and we’re in the running!

Please check out our short video below pitching our presentation, and please vote for us to see the full presentation about Haley AI-as-a-Service online later this month at Botscamp!

Our presentation will cover the Haley AI-as-a-Service platform providing A.I. automation for business tasks.

To learn more and vote, you can go to:


The main Botscamp website can be found at http://www.botscamp.co/



Big Data Modeling at NoSQLNow! / Semantic Technology Conference, San Jose 2014

We had a wonderful time in San Jose last week at the NoSQLNow! / Semantic Technology Conference.

Many thanks to the organizers Tony Shaw, Eric Franzon, and the rest of the Dataversity team for putting on a great event!

My presentation on Thursday afternoon was “Big Data Modeling”.

The presentation is available below:

Vital AI: Big Data Modeling from Vital.AI

Using the Vital AI software to make predictions

In this series of blog posts, I’ll introduce components of the Vital AI software used to make predictions via machine learning models.


We’ll use the venerable “20 Newsgroup” dataset often used in text classification, which consists of around 20,000 text articles across 20 categories.  The dataset is available here: https://github.com/vital-ai/vital-datasets/tree/master/20news

The primary steps are:

  • Set up a data model
  • Create the data set
  • Define the prediction model
  • Run the machine learning training
  • Evaluate the trained model
  • Use the model to make ongoing predictions

In this example, our predictions will be the categories assigned to the text – such as a category like “baseball” if the text is about baseball.

Next: Introduction to Big Data Models

Introduction to Big Data Models with Vital AI

Part of a series to introduce the Vital AI software used to make predictions.

Go to beginning: Using the Vital AI software to make predictions

At the heart of any data-driven application is a data model – but often the data model is never is fully captured.  It is spread out over many schema files, databases, source code files, and the minds of the developers working on the application.

By capturing the data model in one place:

  • Developers can easily reference it across different software components
  • Developers have a single place to look for data definitions
  • Code can be generated directly from the data model
  • Errors can be detected much more easily by checking data against the model

Typical schema formats are very limited in what can be specified – to truly capture the full model for Big Data applications a much richer format must be used.

At Vital AI, we use the OWL standard to describe Big Data Models.  OWL is a standard used to create data models – also known as “ontologies.”

Some background on the standard is here:

And documentation on the standard is here:

We’ll use a graphical user interface to edit our data model called Protégé, which is an open-source application.

It’s available here: http://protege.stanford.edu/

For our data model, we need a single class (type of data object) to represent the articles in the 20 Newsgroup dataset.

Vital AI provides a core data model defining the most fundamental data types, and a base application data model which defines typical data objects such as “User” and “Document”.

For our Twenty Newsgroup dataset, we’ll extend the “Document” class and create the TwentyNewsArticle class.


The Vital AI “vitalsigns” application generates code from a data model, so our data object definitions can be used in our software.

From the command line we can enter:

vitalsigns generate -o twentynews-1.0.1.owl -p com.twentynews.model -j twentynews-groovy-1.0.2.jar

to create a JAR file which we can then include in our application.

In our IDE, we can use our new data model directly in our code.


In addition to our data objects, we need to define the categories we want to use in our predictions.

Next: Defining categories to use in predictions 

Creating a classification Taxonomy with Vital AI

Part of a series to introduce the Vital AI software used to make predictions.

Go to beginning: Using the Vital AI software to make predictions

To categorize data, we need to define the categories and add them into our data model.

We can use a simple text file and helper application to do this.

First, our categories:



vitaltaxonomy -i twentynews_taxonomy.txt -o twentynews_categories.owl

The vitaltaxonomy command creates an OWL file that contains the list of categories.

These can be merged into our data model using:

vitalsigns mergeindividuals -o twentynews-1.0.2.owl -i ../taxonomy/twentynews_categories.owl

Now our categories are added into our data model.  We can check it by listing them.

vitalsigns listindividuals -o twentynews-1.0.1.owl


And we can see them added into our model with Protege:


Next: Adding features and target to a Vital AI predictive model

Adding features and target to a Vital AI predictive model

Part of a series to introduce the Vital AI software used to make predictions.

Go to beginning: Using the Vital AI software to make predictions

We can edit our data model to define a predictive model.

For our predictive model, we define:

  • A unique identifier for the model (URI)
  • A name for the model
  • The features (inputs) to the model, including their datatype. In this example, the inputs will be textual.
  • The target (output) of the model, including its datatype. In this example, the output is categorical (one of a list of options).
  • The machine learning algorithm to use with the predictive model. In this case, we’ll use complementary bayes.

The predictive model is defined by a combination of individuals and annotations.


Features are specified, such as the “hasBody” property.


The Target property is specified:


We can also specify how results of the prediction will be asserted:


We can check the definition of the model using the “showmodels” option of the vitalpredict command.

vitalpredict showmodels


Now that we have our model defined, we can create our dataset.

Next: Creating a predictive model training set with Vital AI

Creating a predictive model training set with Vital AI

Part of an ongoing series to introduce the Vital AI software used to make predictions.

Go to beginning: Using the Vital AI software to make predictions

To process data with a machine learning algorithm to build a predictive model, a dataset must be created.

The Twenty Newsgroup source data is comprised of around 20,000 individual text files – one per article.

The Vital AI software uses a standardized data format for datasets, with each data object conforming to the data model.

To convert the source data into the Vital AI data format, we use a simple script.

The key lines of the script are:


def doc = new TwentyNewsDocument()

doc.URI = "http://example.org/twentynews/${newsgroup}/${id}";

doc.title = subject

doc.body = body

doc.newsGroup = 'http://vital.ai/twentynews/Category/' + newsgroup;





The resulting data file is in the “Vital Block” format, called “block” format as data objects can be grouped together in “blocks” for processing.

Next: Processing datasets using Machine Learning on Hadoop with Vital AI