Analyze and Classify Images on Office 365 SharePoint using MS Flow and Azure Service

Let us see how the Office 365 SharePoint library images are analyzed using Microsoft Flow and Azure Cognitive Service. By analyzing the images, we can classify the images. Also, we can extract the image description, tags or taxonomy data of image, locations present on images, or even the image categories.

Azure Cognitive service provides Computer Vision API, which helps providing tools to understand the content of any images. Computer vision API helps in classifying the image, identifying captions of image and even image categorizations. Further API helps in recognizing celebrities and landmarks, reading out text from images, analyzing video in real time and generating thumbnails for the videos.

Computer vision API can be leveraged on multiple platforms. Microsoft Flow is one such powerful platform, where we integrate computer vision API for analyzing the images uploaded to SharePoint.

Use Case: Let us see how the images uploaded can be analyzed classified on SharePoint images library. At the end of article, you will know how the below image uploaded can be updated classification and description data.

Making Office 365 SharePoint REST API calls on Microsoft Flow

In this post, let us look how we can work with the SharePoint data using the REST APIs explicitly in the Microsoft Flow. We will be using this approach when no action/connector are available OOB for processing the required SharePoint data on the MS Flow. Person who have knowledge on the APIs used will prefer to use this approach.

In my previous post, we have built the connectors/actions using the REST APIs. The objective of my previous post, was to show how any connector/action can be created and published on the portal. Such actions can be shared with other users and it is readily available as OOB actions for other/end users. Connectors/Actions will be created by developers and the business users will use such connectors on their flows.

At the end, both the approaches yield the same results. Both the approaches will be used only when no connector is available for users to work with the required data. Hope the above explanation differentiates two approaches available for working with the SharePoint data.

Let us look how we can retrieve the required SharePoint data using the REST APIs available. We can use the OOB action “Send an HTTP request to SharePoint”. This action supports any type (GET, PUT, POST, PATCH or DELETE) of service calls. In the sample below, let us only look at the GET operation.

Use Case: Retrieve the use profile properties of item author, whenever an item created on SharePoint list.

Configure Microsoft Flow for Sending Office365 SharePoint File Attachments in the Mails

Here, let us look how to configure Microsoft Flow for sending mails with Office 365 SharePoint library files as attachments.

Imagine there is a requirement of sending mails with SharePoint library files as attachments, if a file/item is created/uploaded. Previously we were creating tools for such requirements. Unfortunately, SharePoint designer doesn’t provide option for attaching files in the mails, in the configurations.

Microsoft Flow provides a solution for the above requirement. The solution just includes few configuration steps, which might take one or two minutes of your effort J

Use Case: Whenever a file is created in the folder, send out a mail notification with the uploaded file as attachment to a particular mail box.

Steps Involved:

The following steps shows the detailed configuration steps.

• Create a Microsoft Flow with a blank template as base.
• Select SharePoint connector, search and select “When a file is created in a folder” trigger. In the trigger, Provide the site address URL and select the necessary library/folder path.

Creating Office 365 SharePoint Custom Connectors on Microsoft Flow

Here, let us look how the custom connectors can be created for accessing SharePoint data on Microsoft Flow.

Microsoft Flow provides multiple connectors from various services including SharePoint to work with the data. The connectors contain multiple triggers and actions. For SharePoint connector by Out of the Box, Microsoft provides 8 triggers and 28 actions on Microsoft flow.

UseCase: Imagine you want to retrieve the SharePoint user profile data of some user. Currently there is no action available for MS Flow developers to retrieve SharePoint user profile data. Such triggers and actions can be created by developers on the Microsoft Flow platform. In this post, let us look how one such custom action can be created and used on the Microsoft Flow platform.

The configuration involves the following steps.

1. Configuring Azure AD Application, which provides necessary permissions and helps in authenticating the calls made from Microsoft Flow.
2. Generating the collection file (Swagger) using postman tool, which will be used as base file while building the custom connector.
3. Configuring the custom connector, which will make call to SharePoint to get the required data with necessary inputs.
4. Testing the custom connector created above.
5. Creating/Configuring the flow, which will also use the custom connector we have created above.

Configure Azure AD Application for Flow Authentications:

• Create a new app on the Azure Active directory.

Content Classification on Office 365 SharePoint using MS Flow and LUIS

In this post, let us look how the Office 365 SharePoint list content can be classified using Microsoft Flow with the help of LUIS text prediction techniques. Text added to Office 365 SharePoint list are classified and saved back to SharePoint using Microsoft Flow and LUIS.

Consider a scenario of having queries list, and admin wants the queries to be auto classified before routing the queries for solution. Microsoft Flow provides a LUIS connector, which helps in predicting and classifying the text being saved to SharePoint.

Note: In a layman scenario on LUIS portal, categories will be intents and queries will be utterances.

Connecting SharePoint and LUIS using Flow

The solution consists of three steps.

• Query is created on SharePoint: Flow is triggered, whenever a new item is created in the SharePoint list.
• Query category is predicted using LUIS: Flow uses LUIS Get Prediction action for classifying the item created. The action predicts the data using the trained app created on the LUIS portal. The output of this step will be matching intents (predicted scores) with the relevant scores.
• Query is updated with the category predicted: After classification, the top scoring intent from the above step is saved to the respective SharePoint list item’s category field.