Building SharePoint Webhook Services with Serverless Architecture

Before looking out at the implementation of webhook models in SharePoint, here let us look at how Serverless web hooks concept can be feasible.

What is Serverless architecture? It allows us to build the required services, without the need of servers. Some Azure services like Azure functions helps building such Serverless models.

The components like webhook service, storage queues, web jobs and SQL database are hosted on Azure. The following picture depicts the Serverless webhooks process, with the help of Serverless components. The components present on the Azure box, depicts serverless architecture model.

Serverless Architecture for SharePoint Webhook Services

Processing Events from SharePoint Webhook Services

Let us theoretically look how the event changes are retrieved from the storage queues, processed and stored on to the logs like SharePoint history lists.

In the previous post, you have seen how to subscribe to the event changes on SharePoint online and how the events are stored on the storage queue. Here, you could see how the events stored can be processed.

Process Data from the Queue - Web Jobs

If we closely look in the previous post, SharePoint service has sent few details to the notification service when the event change had happened on SharePoint. In the details, you can only see what has changed at high level. That is, which list has changes. It doesn’t define, what are the changes.

Event Change Details

Working with SharePoint Events using Webhooks

Introduction

In this post, let us look in detail on webhooks functionality. At a high level, there are two independent steps to work with Webhooks.

• Subscribing to the SharePoint events (remote events)
• Processing Events.

Note: In the recent SharePoint Saturday Event at Chennai, I have explored on using Serverless Webhooks. For the benefits of larger audience, I am writing down as a technical post in detail with explanations.

Subscribing to the SharePoint Events

SharePoint by default provides the subscription mechanism for event changes. For now, only item level changes can be tracked, that on SharePoint online platform.

Introduction to SharePoint Webhooks

What is Webhook?

Web hook is a general concept for http based events, which will provide remote event mechanisms.

Say for example, you have a built a custom application separately which has some specific product related content. Whenever there is a data change on the product repository, your data on the custom application should also be updated. For this scenario, will you be keep on checking the product repository for the changes? The answer should be No. Instead of your application keep on poll the product for changes, product should notify or push the changes to your application.

What is SharePoint Webhook?

Previously for working with event based processes, we were using the server side event receivers and remote events receivers. Such processes can be replaced with the webhooks.

Upload And Set Custom Master Page To Site Using CSOM PowerShell On SharePoint

In this article post, you will learn how to upload custom master page to master page gallery and apply custom master pages to the site using CSOM Powershell on SharePoint 2013 / SharePoint 2016 / SharePoint online sites.

Steps Involved:

The following prerequisites need to be executed before going for any operations using CSOM PowerShell on SharePoint sites.

1. Add the references using the Add-Type command with the necessary reference paths. The necessary references are Microsoft.SharePoint.Client.dll, Microsoft.SharePoint.Client.Runtime.dll.
   
   1. Add-Type -Path "C:\Program Files\Common Files\microsoft shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.dll"  
   2. Add-Type -Path "C:\Program Files\Common Files\microsoft shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.Runtime.dll"  
2. Initialize client context object with the site URL.
   
   1. $siteURL = ""  
   2. $ctx = New-Object Microsoft.SharePoint.Client.ClientContext($siteURL)  
3. If you are trying to access SharePoint Online site then you need to setup the site credentials with credentials parameter and load it to the client context. 
   
   1. #Not required for on premise site - Start  
   2. $userId = "abc@abc.onmicrosoft.com"  
   3. $pwd = Read-Host -Prompt "Enter password" -AsSecureString  
   4. $creds = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($userId, $pwd)  
   5. $ctx.credentials = $creds  
   6. #Not required for on premise site - End  
4. If you are trying to access the SharePoint on premise site then the credentials parameter is not required to be set to the context but you need to run the code on the respective SharePoint Server or you need to pass the network credentials and set the context.
   
   1. #Credentials for on premise site - Start  
   2. $pwd = Read-Host -Prompt "Enter password" -AsSecureString  
   3. $creds = New-Object System.Net.NetworkCredential("domain\userid", $pwd)  
   4. $ctx.Credentials = $creds  
   5. #Credentials for on premise site - End  

Upload and Apply Master Page:

We will see how we can upload the master page file from local system to master page gallery and apply the uploaded master page to the site.

• Get the master page from local.
• Get the master page file local system URL and access the properties of master page file using Get-ChildItem method.
• Then read the master page content using ReadAllBytes method and initialize the content to variable ($fileBytes).
  
  1. $pageFilePath = "D:\custom.master"  
  2. $pageFilePath = Get-ChildItem $pageFilePath  
  3. $fileBytes = [System.IO.File]::ReadAllBytes($pageFilePath.FullName)  
• Set the master page file details.
• Initialize file creation information object to set the master page file properties to be uploaded. 
• To the object created, set the necessary parameters like URL, content. Overwrite parameter can be used to force write the file to the server.
  
  1.   
  2. $newFileInfo = New-Object Microsoft.SharePoint.Client.FileCreationInformation  
  3. $newFileInfo.Url = $pageFilePath.Name  
  4. $newFileInfo.Content = $fileBytes  
  5. $newFileInfo.Overwrite = $true  
• Access the folder and add the master page.
• Using the context, access the web. It should be root web since master page gallery is present at the top level site.
• Then access the master page gallery folder using web and GetByTitle method.
• From the web, get the source list and destination list using the list names. Then access the appropriate folder and the files method to add our new master page. Add new file creation information object created above. Load and execute the query.
  
  1. $web = $ctx.Site.RootWeb  
  2. $folder = $web.Lists.GetByTitle("Master Page Gallery")  
  3. $masterPage = $folder.RootFolder.Files.Add($newFileInfo)  
  4. $ctx.Load($web)  
  5. $ctx.Load($masterPage)  
  6. $ctx.ExecuteQuery()  
• Publish as major version.
• Then to publish the master page, we need to check out the file. Check out is required since the master page is neither checked out nor checked in as major version when the master page is uploaded. Load and execute the query for check out operation.
•  Then check in the master page with comments and check in type. Check in type should be major version.
  1. #Check Out  
  2. $masterPage.CheckOut()  
  3. $ctx.Load($masterPage)  
  4. $ctx.ExecuteQuery()  
  5. #Check In and Publish  
  6. $masterPage.CheckIn("Majorversion checkin",[Microsoft.SharePoint.Client.CheckinType]::MajorCheckIn)  
  7. $ctx.Load($masterPage)  
  8. $ctx.ExecuteQuery()  
• We have successfully uploaded the master page. No we will apply the master page to the site. 
• From the web object, access the custom master url property and set the master page relative url. 
• To set master page for custom pages, use CustomMasterUrl property. 
• To set master page for system master pages, use MasterUrl property. 
• After setting the properties, update the web object. Then load and execute the web object with the context.
  
  1. $web.CustomMasterUrl = $masterPage.ServerRelativeUrl  
  2. $web.MasterUrl = $masterPage.ServerRelativeUrl  
  3. $web.Update()  
  4. $ctx.Load($web)  
  5. $ctx.ExecuteQuery()  

The above steps will help us in uploading and setting the master page. The article focused on setting master pages for root level sites. If the master page needs to be set for subsite, then corresponding web object needs to be created before applying master page.

Retrieve File Version Properties And Version Author Details On SharePoint Using CSOM PowerShell

In this article, you will learn how to retrieve version properties and version author details of files present on SharePoint folders programmatically using CSOM with PowerShell on SharePoint 2013 / SharePoint 2016 / SharePoint online.

Steps Involved:

The following prerequisites need to be executed before going for any operations using CSOM PowerShell on SharePoint sites.

1. Add the references using the Add-Type command with the necessary reference paths. The necessary references are Microsoft.SharePoint.Client.dll, Microsoft.SharePoint.Client.Runtime.dll.
   
   Add-Type –
   
   Path "C:\Program Files\SharePoint Online Management Shell\Microsoft.Online.SharePoint.PowerShell\Microsoft.SharePoint.Client.dll" 
   
   Add-Type –
   
   Path "C:\Program Files\SharePoint Online Management Shell\Microsoft.Online.SharePoint.PowerShell\Microsoft.SharePoint.Client.Runtime.dll" 
2. Initialize client context object with the site URL.
   
   1. $siteURL = ""  
   2. $ctx = New-Object Microsoft.SharePoint.Client.ClientContext($siteURL)  
3. If you are trying to access a SharePoint Online site then you need to set up the site credentials with the credentials parameter and load it to the client context. 
   
   1. #Not required for on premise site - Start  
   2. $userId = ""  
   3. $pwd = Read-Host -Prompt "Enter password" -AsSecureString  
   4. $creds = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($userId, $pwd)  
   5. $ctx.credentials = $creds  
   6. #Not required for on premise site - End  
4. If you are trying to access the SharePoint on premise site then the credentials parameter is not required to be set to the context but you need to run the code on the respective SharePoint Server or you need to pass the network credentials and set the context.
   
   1. #Credentials for on premise site - Start    
   2. $pwd = Read-Host -Prompt "Enter password" -AsSecureString    
   3. $creds = New-Object System.Net.NetworkCredential("domain\userid", $pwd)    
   4. $ctx.Credentials = $creds    
   5. #Credentials for on premise site - End  

Retrieve File Versions:

• Initialize the web object using the context and url.
• Retrieve the folder using the web object and folder title.
• Create caml query object and assign query xml to get the necessary file from the folder.
• Get items with GetItems method using the above query.
  
  1. web = $ctx.Site.OpenWeb("/")  
  2. $folder = $web.Lists.GetByTitle("Documents")  
  3. $query = "<View><Query><Where><Eq><FieldRef Name='Title'/><Value Type='Text'>filename</Value></Eq></Where></Query></View>"  
  4. $camlQuery = New-Object Microsoft.SharePoint.Client.CamlQuery  
  5. $camlQuery.ViewXml = $query  
  6. $items = $folder.GetItems($camlQuery)  
  7. #$items = $folder.GetItems([Microsoft.SharePoint.Client.CamlQuery]::CreateAllItemsQuery())  
  8. $ctx.Load($items)  
  9. $ctx.ExecuteQuery()  
• From the items collections, take the first item and get the necessary details. You can also use foreach loop to see information about multiple files.
• Access the file, and then file versions. Load the file and version objects, and execute the query.
  
  1. $item = $items[0]  
  2. $file = $item.File  
  3. $versions = $file.Versions  
  4. $ctx.Load($file)  
  5. $ctx.Load($versions)  
  6. $ctx.ExecuteQuery()  
  7.   
  8. Write-Host $item["Title"] "Version Details"  
• Then using foreach loop, get each and every version information. The properties which can be accessed are version label, file URL, created time, version id, current version flag, version file size, version comment, and author details.
• To get the version author details, load the author object and execute with context to get information about author. The author details which can be accessed are name, email id, and login id.
  
  1. foreach($version in $versions){  
  2.     $ctx.Load($version)  
  3.     $ctx.ExecuteQuery()  
  4.     Write-Host "--------------------------------------------------------"              
  5.     Write-Host "Version Label      : " $version.VersionLabel              
  6.     Write-Host "Version File URL   : " $version.Url  
  7.     Write-Host "Version Created On : " $version.Created       
  8.       
  9.     Write-Host "Version ID         : " $version.ID  
  10.     Write-Host "Current Version?   : " $version.IsCurrentVersion  
  11.     Write-Host "Version File Size  : " $version.Size  
  12.     Write-Host "Version Comment    : " $version.CheckInComment  
  13.     $modifiedBy = $version.CreatedBy  
  14.     $ctx.Load($modifiedBy)  
  15.     $ctx.ExecuteQuery()  
  16.     Write-Host "File Modified By : "   
  17.     Write-Host $modifiedBy.Title  
  18.     Write-Host $modifiedBy.Email  
  19.     Write-Host $modifiedBy.LoginName  
  20.     Write-Host "--------------------------------------------------------"  
  21. }  

Copy List Items From One SharePoint List To Another Using CSOM PowerShell

In this article, you will learn how to copy list items from one list to another list using CSOM Powershell on SharePoint. This is mainly focused on using PowerShell scripts for any SharePoint 2013 / SharePoint 2016 / SharePoint online sites.

Steps Involved:

The following prerequisites need to be executed before going for any operations using CSOM PowerShell on SharePoint sites.

1. Add the references using the Add-Type command with the necessary reference paths. The necessary references are Microsoft.SharePoint.Client.dll, Microsoft.SharePoint.Client.Runtime.dll.
   
   1. Add-Type –Path "C:\Program Files\SharePoint Online Management Shell\Microsoft.Online.SharePoint.PowerShell\Microsoft.SharePoint.Client.dll"  
   2. Add-Type –Path "C:\Program Files\SharePoint Online Management Shell\Microsoft.Online.SharePoint.PowerShell\Microsoft.SharePoint.Client.Runtime.dll"  
2. Initialize client context object with the site URL.
   
   1. $siteURL = ""  
   2. $ctx = New-Object Microsoft.SharePoint.Client.ClientContext($siteURL)  
3. If you are trying to access SharePoint Online site then you need to setup the site credentials with credentials parameter and load it to the client context. 
   
   1. #Not required for on premise site - Start  
   2. $userId = ""  
   3. $pwd = Read-Host -Prompt "Enter password" -AsSecureString  
   4. $creds = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($userId, $pwd)  
   5. $ctx.credentials = $creds  
   6. #Not required for on premise site - End  
4. If you are trying to access the SharePoint on premise site then the credentials parameter is not required to be set to the context but you need to run the code on the respective SharePoint Server or you need to pass the network credentials and set the context.
   
   1. #Credentials for on premise site - Start    
   2. $pwd = Read-Host -Prompt "Enter password" -AsSecureString    
   3. $creds = New-Object System.Net.NetworkCredential("domain\userid", $pwd)    
   4. $ctx.Credentials = $creds    
   5. #Credentials for on premise site - End  

Copy Operation:

We will see how we can copy the list items from one list to another list. Here, the list should have identical SharePoint list fields. The values will be copied only if the corresponding fields are present on both source and destination lists.

• Using the context, access the web.
• From the web, get the source list and destination list using the list names.
• From the source list, get the items using the query object. The query object should be initiated to access the necessary source list items. In this case, I have used All Items query to get all the items.
• Load and execute the query.
  
  1. $list1 = $ctx.Web.Lists.GetByTitle("List1")  
  2. $list2 = $ctx.Web.Lists.GetByTitle("List2")  
  3. $list1Items = $list1.GetItems([Microsoft.SharePoint.Client.CamlQuery]::CreateAllItemsQuery())  
  4. $fields = $list1.Fields  
  5. $ctx.Load($list1Items)  
  6. $ctx.Load($list1)  
  7. $ctx.Load($list2)  
  8. $ctx.Load($fields)  
  9. $ctx.ExecuteQuery()  

After executing the above steps, we can access the source list, destination list, source list items and source list columns (fields). Now we will see how we can copy items. For each source list item available, execute the below operations.

• For destination list, create list item creation information object and add the object to the destination list.
• Access all the fields using for each loop from the source list.
• For every field, check whether field is not read only field. If it is read only field, then we will not be able to write it to the destination list.
• Also the fields should not be hidden, attachment or content type, since the write operations will not be possible on these fields.
• For other custom fields, execute the copy operations and update the item.
  
  1. foreach($item in $list1Items){  
  2.     Write-Host $item.ID  
  3.     $listItemInfo = New-Object Microsoft.SharePoint.Client.ListItemCreationInformation  
  4.     $list2Item = $list2.AddItem($listItemInfo)  
  5.       
  6.     foreach($field in $fields){  
  7.         #Write-Host $field.InternalName " - " $field.ReadOnlyField   
  8.         if((-Not ($field.ReadOnlyField)) -and (-Not ($field.Hidden)) -and ($field.InternalName -ne  "Attachments") -and ($field.InternalName -ne  "ContentType"))  
  9.         {  
  10.             Write-Host $field.InternalName " - " $item[$field.InternalName]  
  11.             $list2Item[$field.InternalName] = $item[$field.InternalName]  
  12.             $list2Item.update()  
  13.         }  
  14.     }  
  15. }  
  16. $ctx.ExecuteQuery()