Content Classification on Office 365 SharePoint using MS Flow and LUIS

In this post, let us look how the Office 365 SharePoint list content can be classified using Microsoft Flow with the help of LUIS text prediction techniques. Text added to Office 365 SharePoint list are classified and saved back to SharePoint using Microsoft Flow and LUIS.

Consider a scenario of having queries list, and admin wants the queries to be auto classified before routing the queries for solution. Microsoft Flow provides a LUIS connector, which helps in predicting and classifying the text being saved to SharePoint.

Note: In a layman scenario on LUIS portal, categories will be intents and queries will be utterances.

Connecting SharePoint and LUIS using Flow

The solution consists of three steps.

• Query is created on SharePoint: Flow is triggered, whenever a new item is created in the SharePoint list.
• Query category is predicted using LUIS: Flow uses LUIS Get Prediction action for classifying the item created. The action predicts the data using the trained app created on the LUIS portal. The output of this step will be matching intents (predicted scores) with the relevant scores.
• Query is updated with the category predicted: After classification, the top scoring intent from the above step is saved to the respective SharePoint list item’s category field.

Speech Recognition on Office 365 SharePoint with Azure Cognitive Service Speech API

In this post, let us see how speech recognition can be implemented using Microsoft Speech API on SharePoint portals using JavaScript client side libraries/SDKs. This shows how speech to text can be converted on Office 365 SharePoint using Azure Cognitive Services.

In our previous post, we have seen implementing Speech recognition using browser SpeechRecognition objects on SharePoint portals. This post is special for those people who love implementing speech recognition using Microsoft Speech API (Azure Cognitive service speech API).

Note: If you are interested only in the implementation, scroll down to the bottom section. :)

Azure Cognitive Service - Bing Speech API

Microsoft Speech API supports both speech to text and text to speech conversions. In this case, only we are focusing on speech to text conversion. Microsoft Speech API provides two approaches of speech to text conversion. One using the REST API and the other way is using the client libraries. We will be leveraging the client libraries, which provides speech SDK bundles.

Speech Recognition on Office 365 SharePoint

Let us look how the speech recognition can be implemented on Office 365 SharePoint portals. The article contains the introduction to speech recognition service (speech to text conversion), detailed approach for SharePoint, code and snapshots for easier implementation.

Speech Recognition

Speech recognition helps recognizing the real-time audio from the microphone and converts it to the respective text. This kind of interfaces helps in building the voice triggered apps like chat bots, etc.

There are two approaches of implementing speech recognition on SharePoint.

• First using the Speech Recognition interfaces.
• Other way is using the Azure Bing Speech API. It is built on top of WebSockets API. The Speech SDK is available as extensions, which can be leveraged for development.

Let us go with the first approach in this post. We are going to see how the speech recognition interfaces are integrated on to SharePoint applications. In my future articles, I will detail out the integration of Bing Speech API (second approach).

Text Translation on Office 365 SharePoint

Let us look how the text added to Office 365 SharePoint list are translated and saved back to SharePoint using Microsoft Flow and Translator.

Consider a business scenario of having multi-lingual content to be available in SharePoint list and you need write the text in all the required languages. Instead, Microsoft Flow can be leveraged for translating the content on Office 365 SharePoint for the required languages with the help of Translator service.

There are several ways available for implementing the above scenario. The best possible solution is explained in this post.

Note: This article only focuses on translating content present on SharePoint lists. For enabling multilingual feature for whole portal, you can refer to the Microsoft article here.

Building SharePoint Webhook Services with Serverless Architecture

Before looking out at the implementation of webhook models in SharePoint, here let us look at how Serverless web hooks concept can be feasible.

What is Serverless architecture? It allows us to build the required services, without the need of servers. Some Azure services like Azure functions helps building such Serverless models.

The components like webhook service, storage queues, web jobs and SQL database are hosted on Azure. The following picture depicts the Serverless webhooks process, with the help of Serverless components. The components present on the Azure box, depicts serverless architecture model.

Serverless Architecture for SharePoint Webhook Services