SharePoint 2016: Problems crawling content and the “The crawler could not communicate with the server.” error message

Hello,

Recently, after installing a SharePoint 2016 farm (to learn more on how to install a SharePoint 2016 farm in an automated way using PowerShell click here and here), I was configuring the Search Service Application in a customer’s SharePoint 2016 farm and got into a crawling error while trying to crawl the content of one of the web applications that is configured to use a FQDN URL and https, for example https://intranet.contoso.com. This web application is using a Content Database that was migrated from SharePoint 2010 to SharePoint 2016.

The Problem

The error that was being thrown was during the Full Crawl of the Content Source was:

“The crawler could not communicate with the server. Check that the server is available and that the firewall access is configured correctly. If the repository was temporarily unavailable, an incremental crawl will fix this error.”

Failed Attempts

When I saw this message, the first thought that crossed my mind was lack of permissions for the Default Content Access Account. The account was properly configured to have Full Read permissions on the web application and this was not the problem.

My next thought was that this could be a firewall problem and some ports necessary by the Search Service Application to properly crawl the web application content were not opened in the firewall (for a complete list of the necessary ports in a SharePoint 2016 installation, click here or here). However, after the firewall was properly configured, the error remained.

After some searching in the Internet, I tried without any success the following approaches:

  • Disabling the loopback check on the server
  • Disabling https for the web application
  • Creating a new empty web application with a FQDN URL and tried to crawl its contents
  • Resetting the search index
  • Deleting and recreating the content source

The Solution

After all the failed attempts, I was able to solve the problem by extending the web application to a non-FQDN URL (for example http://intranet) and configuring this URL as the start address in the Content Source in the Search Service Application. This URL is internal (the host is only configured in the server’s hosts file and is not configured in the DNS) and is being used for crawling purposes only.

Important detail: For the search results to be correctly presented, the newly added URL was configured in the Default Zone in Alternate Access Mappings (AAM) in Central Administration.

The goal here is to allow users to use the FQDN URL (ex: https://intranet.contoso.com) to access the SharePoint Portal and the search results to include this same URL.

If I had configured the non-FQDN URL in a non-Default zone in AAM, the search results would always include the non-FQDN URL instead of the FQDN URL that users use to access the SharePoint Portal. To learn all about using the Default Zone to crawl content and AAM, please read the following two fantastic articles by Brian Pendergrass:

I was able to configure a FQDN URL as the start address of a Content Source in the Search Service Application in one of my development environments without this problem, so I guess this behavior must be due to some infrastructure configuration in the customer’s environment which I have not yet been able to discover.
If I find a solution that allows the usage of a FQDN URL to be configured as the start address in the Content Source in the Search Service Application, I will update this post.

Hope this helps!

PS: New search capabilities have been introduced in SharePoint 2019. To learn more about them and all the upcoming features in SharePoint 2019, click here.

About the Author

Miguel Isidoro
SharePoint & Office 365 Senior Consultant | Create IT

Miguel Isidoro is a tech enthusiast, focused on Microsoft technologies, specially everything that is related to SharePoint and Office 365 and works with Microsoft and SharePoint technologies for more than 16 years, from SharePoint 2001 up until SharePoint 2016 and Office 365.

LinkedIn Profile: https://www.linkedin.com/in/miguelisidoro/

SharePoint Server gets modern: SharePoint 2019 is here!

SharePoint 2019 is finally here!

It is still not the RTM version, but we can already get a sneak peek of what the new version will bring since SharePoint 2019 Preview was recently announced by Microsoft after the Microsoft Inspire event that took place at Las Vegas from July 15th to July 19th.

Introduction

SharePoint has come a long way and it is nothing but amazing to see the constant innovation that Microsoft has been introducing to the platform over the years!

With this release, SharePoint Server finally gets modern and includes a major set of the features that are already available in SharePoint Online. After the release of SharePoint Online back in 2013 and with the release of SharePoint 2016, Microsoft has adopted the “Cloud-first, mobile first” vision where all new features are first released to SharePoint Online and then gradually released to SharePoint On-Premises through new major versions that occur approximately every 3 years and through Feature Packs that were introduced in SharePoint 2016 and allow the latest Cloud innovations to be brought to the On-Premises world. SharePoint 2016 was the first major SharePoint On-Premises release that took advantage of the extensive Microsoft experience managing SharePoint Online and the second major version was just recently announced: SharePoint 2019.

SharePoint 2016 has brought us several major improvements over its predecessor. Here are some of its main features:

  • New Hybrid Experiences like Cloud Hybrid Search, Hybrid Team Sites and the ability to use Delve as the User Profile Experience
  • Major limit increases (ex: Search Index Size, Number of Site Collections, List View Threshold Increase with Automatic Index Column Creation)
  • MinRoles
  • Feature Packs
  • Durable Links
  • Zero Downtime Patching
  • Integration of the new SharePoint Framework development model (available with SharePoint 2016 Feature Pack 2)

Three years have passed with lots of new features released to SharePoint Online and it is time to take a look at what Microsoft is bringing us with the new SharePoint 2019 Preview release.

What’s New In SharePoint 2019

Lets’ take a look at the major features that the new version will bring!

The main focus of the new release is no doubt the inclusion of major parts of the Modern Experiences already available in SharePoint Online, bringing a responsive user experience and allowing access to SharePoint 2019 from any device (PC and mobile devices). Here are the major new features that were announced and that will be included  in SharePoint 2019:

  • Modern Team Sites
  • Modern Communication Sites
  • Modern Lists
  • Modern Libraries
  • Modern Pages
  • Modern SharePoint Development Platform
  • Support to the latest One Drive Sync Client
  • Improved Hybrid Support
  • Improved support for PowerApps and Flow
  • Improvements on Large File Support, Character Restrictions and File/Folder Names

The following features are not going to be available, possibly in future Feature Packs:

  • Hub Sites
  • Site Designs and Site Scripts
  • Site collection app catalog
  • Power BI web part
  • Most of the modern web parts available in SharePoint Online

As expected, some of major features in Office 365 will not be available and will remain as cloud only offerings:

  • Office 365 Groups
  • Planner
  • Microsoft Teams
  • Microsoft Forms
  • Delve (possible to integrate with SharePoint 2019 in a hybrid scenario)
  • PowerBI (possible to integrate with SharePoint 2019 in a hybrid scenario)

Modern Team Sites

The major difference between SharePoint 2019 and its predecessor is that it will include some of the major UI improvements already available in SharePoint 2019. One of its core components are the Modern Team Sites. Team Sites have always been the major collaboration tool in SharePoint and will continue to be so but now with a modern and responsive user experience, allowing users to easily collaborate and share information within Teams and the whole organization. This is how a new Modern Team site will look like:

One major difference between Team Sites in SharePoint 2019 and their counterparts in SharePoint Online is that they will not be supported in Office 365 Groups and thus not taking advantage of some of the coolest collaboration features in Office 365 such as:

  • Planner
  • Office 365 Group features including a shared calendar and integration with Outlook
  • Ability to connect a Team Site with a Team in Microsoft Teams

Modern Communication Sites

Another major piece of the modern experience that will be included in SharePoint 2019 are the Modern Communication Sites. While Team Sites are all about collaboration and mainly people working on SharePoint Lists and Libraries and sharing information between them, Communication Sites are all about user engagement and having a way to easily share relevant information in an engaging way with a whole department or the entire organization. While in Team Sites, most users are active contributors, in Communication Sites most users will be consumers and there will be a restricted set of users that will publish information for a department or the entire organization to consume. This is how a new Modern Communication site will look like:

Modern Lists

Lists are one of the core collaboration features in SharePoint, allowing users to work on and share information in a structured way, also allowing information from external systems to be presented in SharePoint. Modern Lists will bring the SharePoint Online experience to the On-Premises world, allowing users to interact with the information with a richer user experience, available across multiple devices and browsers. This is how a Modern List will look like:

One of the major advantages is the possibility to easily format column data using the conditional formatting features already available in SharePoint Online. Other major improvements that will be available later this year or in 2019:

  • Row Formatting: Ability to create new formatting experiences for any List or Library with scripting
  • Realtime List updates: Ability to see updates to Lists and Libraries instantly without having to refresh the page
  • New Ways to create Lists: Ability to create Lists based on Excel, templates or other Lists

Modern Libraries

Libraries are the other main collaboration feature in SharePoint, allowing users to work on files (and add metadata) and easily share them with other colleagues in the organization. Modern Libraries will bring the same modern experience already available in SharePoint Online and will allow users to easily sync their files with their computers using the latest version of OneDrive for Business, thus allowing users to work offline and sync files to SharePoint once they get back online. This is how a Modern Library will look like:

Modern Pages

In Classic SharePoint experience, pages are typically rendered server side and are composed by a set of web parts (native SharePoint web parts or custom developed web parts implemented using server side code in the language of your preference, typically C# or VB.NET).  Modern pages are rendered client side and bring the power of client side web parts to give users a responsive user experience, available across multiple devices and browsers. Publishing modern pages is also greatly simplified and users can now create new pages with rich content quickly and easily, This is how a Modern Page will look like:

One of the best parts of Modern pages is the possibility to extend SharePoint using the new SharePoint Framework development model, allowing developers to build modern client side web parts and extensions.

Modern SharePoint Development Platform

With the release of SharePoint 2019, support to SharePoint Framework web parts and extensions will be enhanced.  SharePoint 2019 will include SharePoint Framework 1.4 and will also include support to Webhooks that are already available in SharePoint Online.  Here are the supported development options in SharePoint 2019 (this list may change until RTM version is released):

  • SharePoint Framework client-side web parts
  • SharePoint Framework extensions
  • Webhooks
  • Asset packaging and automatic JavaScript file hosting from app catalog
  • Column formatting

For more details on the new development options, click here.

Support to the latest One Drive Sync Client

Until now, OneDrive support was missing on the On-Premises world, while it is already present for OneDrive and SharePoint Online for quite some time. Fortunately,  this will change since SharePoint 2019 will add OneDrive Sync Client support for both Personal and Team Sites, which will also include files on demand!

Improved Hybrid Support

As part of Microsoft’s strategy to gradually move costumers to the Cloud and SharePoint Online, started with the first stages of hybrid support in SharePoint 2013 SP1 and further enhanced in SharePoint 2016, configuring an hybrid environment will get even easier in SharePoint 2019 with the new features belowallowing organizations to leverage existing investments on On-Premises solutions while also taking advantage of all cloud innovations of SharePoint Online and Office 365:

  • Hybrid configured during deployment: It will be possible to configure a hybrid SharePoint environment during SharePoint deployment
  • New SharePoint Hybrid status bar: A new hybrid status bar will be presented in Central Administration once the SharePoint farm meets the minimum system requirements necessary to enable a hybrid scenario. Also, it will give you direct access to launch the SharePoint Hybrid Configuration Wizard
  • OneDrive in Office 365 by Default: Setting up OneDrive for Business Redirection to Office 365 was already possible in SharePoint 2013 with SP1 and SharePoint 2016.  With SharePoint 2019, when a hybrid environment is configured, users will be using OneDrive For Business in Office 365 by default
  • Modern Search: Although in SharePoint 2016 we already had an integrated Search experience with Cloud Hybrid Search that allows SharePoint Online to store the search index for both Online and On-Premises results, the search experience was not totally consistent in a hybrid environment. SharePoint 2019 will take advantage of the improvements made in Office 365 search and will bring a better search experience for hybrid environments. Some of the major improvements in the search experience are:
    • Users see results as they start typing and the results update as they type (Search as you type)
    • The search results page shows an overview of search results, grouped by type and it will be easy to expand to see all the results of each type, for example people or sites
    • On a people result, users can click the email address to quickly get started on an email to that person

Improved support for PowerApps and Flow

It isn’t quite clear all the improvements that will be included in SharePoint 2019 but there is going to be some deeper integrations with Flow and PowerApps to On-Premises data which will facilitate the creation of Apps and Flows that integrate with Line Of Business systems. The connection to On-Premises systems will be done through the On-Premises Data Gateway (that supports integration with several On-Premises data sources) and with the release of SharePoint 2019 several performance improvements will be included and there will be fewer restrictions than we had before.

Unfortunately, there is still not going to be native PowerApps and Flow buttons directly from On-Premises SharePoint Lists and Libraries.

Improvements on Large File Support, Character Restrictions and File/Folder Names

With SharePoint 2019, there will some important improvements on Large File Support, Character Restrictions and File/Folder Names:

  • It will be possible to store files up to 15 GB in SharePoint 2019 document libraries. This is 5 GB up from the 10 GB supported in SharePoint 2016
  • Support for # and % characters in file and folder names across document libraries in SharePoint and OneDrive for Business
  • SharePoint 2019 will increase the URL path length limit from 260 characters to 400

Conclusion

While Microsoft continues to push customers to the Cloud and to Office 365, Microsoft is now releasing another major release of SharePoint Server that brings many of the major improvements made in the last three years in SharePoint Online to the On-Premises world. Meanwhile, the path in the Online world doesn’t stop and many exciting features are being announced, including the arrival of AI to SharePoint through SharePoint Spaces.

If you want to know more about what recent updates in the SharePoint world, I invite you to read:

To get a broader vision of SharePoint evolution over the years, don’t miss Jeff Teper “Hitting Refresh on SharePoint” articles:

About the Author

Miguel Isidoro
SharePoint & Office 365 Senior Consultant | Create IT

Miguel Isidoro is a tech enthusiast, focused on Microsoft technologies, specially everything that is related to SharePoint and Office 365 and works with Microsoft and SharePoint technologies for more than 16 years, from SharePoint 2001 up until SharePoint 2016 and Office 365.

LinkedIn Profile: https://www.linkedin.com/in/miguelisidoro/

How to install a SharePoint 2016 farm using PowerShell and AutoSPInstaller (Part 2)

Hello,

This post is the second part of a two part series where I will talk about a great way to install SharePoint in a fully automated way using PowerShell.

To read the first part, click here.

Introduction

Installing a SharePoint farm can be a complex task and the installation process is composed by a series of steps:

  • Downloading and installing SharePoint prerequisites
  • Downloading and installing the SharePoint binaries
  • Downloading and installing the lastest SharePoint updates
  • Downloading and installing the required language packs (optional, necessary if you want the SharePoint interface to be presented in a language other than English)
  • Configuring the SharePoint farm using SharePoint Products and Configuration Wizard

All the above steps are time consuming and can be error prone if, for example, you miss to download any of the prerequisites.

Fortunately, it is possible to fully automate a SharePoint farm installation using PowerShell by using a script called AutoSPInstaller. The installation process can be divided in two major steps:

  • Downloading the SharePoint installation package, click here for more details
  • Installing SharePoint and configuring the SharePoint farm

In this post, I will talk about the second part of the process: installing SharePoint and configuring the SharePoint farm.

AutoSPInstaller can be used to install a SharePoint farm in the following versions of SharePoint:

  • SharePoint 2010
  • SharePoint 2013
  • SharePoint 2016
  • SharePoint 2019 (still in Preview – to learn more about all the upcoming features in SharePoint 2019, click here)

The example in this post is based on SharePoint 2016.

Managed Accounts and Pre-Installation Requisites

This is a critical step to ensure a successful SharePoint installation. Before the AutoSPInstaller script is executed, all the necessary managed accounts must be created and the appropriate permissions must be properly granted to each account. If any of these permissions is not granted, you will probably encounter problems during the installation. From my experience doing SharePoint installations, creating these accounts with all the required permissions ensures a fast and successful SharePoint installation. Here is the list of accounts and required permissions:

Name Description Local Rights Domain Rights
sp_farm The server farm account is used to perform the following tasks:
-Configure and manage the server farm.
-Act as the application pool identity for the SharePoint Central Administration Web site.
-Run the Microsoft SharePoint Foundation Workflow Timer Service.
SecurityAdmin and DB_Creator rights on the SQL Instance Domain User
sp_admin The server farm account is used to perform the following tasks:
-Setup
-SharePoint Products Configuration Wizard
Local Administrator on all the SharePoint Servers. SysAdmin, SecurityAdmin and DB_Creator rights on the SQL Instance.
Must be na account with permissions to add rules to the local firewall.
Domain User
sp_pool The Pool account is used to run the Web Application Pools None Domain User
sp_services The Services Account is used to run the Service Application Pool None Domain User
sp_crawl The Default Content Access Account for the Search Service Application None Domain User
sp_search Service Account to run the SharePoint Search “Windows Service” Local Administrator and SysAdmin rights on the SQL instance Domain User
sp_mysitepool Used for the My Sites Web Application None Domain User
sp_userprofilesync The User Profile Synchronization Account None Replicate Directory Changes permission on the domain.
wf_service WorkFlow Manager Service Account Local Administrator and SysAdmin rights on the SQL instance. Domain User
sp_portalsuperuser Object Cache Service Account. The goals of the object cache are to reduce the load on the computer on which SQL Server is running, and to improve request latency and throughput. These user account must be properly configured to ensure that the object cache works correctly. None.
SharePoint: Must be an account that has Full Control access to the Web application.
Domain User
sp_portalsuperreader Object Cache Service Account. The goals of the object cache are to reduce the load on the computer on which SQL Server is running, and to improve request latency and throughput. These user account must be properly configured to ensure that the object cache works correctly. None.
SharePoint: Must be an account that has Full Read access to the Web application.
Domain User


Important Notes: 

  • To avoid problems executing the AutoSPInstaller script, ensure that the passwords of the above accounts do not contain any special characters (ex: $,%,&,/. etc)
  • The above table contains a proposal to name the accounts for the SharePoint installation. If you choose to use different names, ensure that the account names do not exceed 20 characters since NETBIOS names have a limit of 20 characters

Download AutoSPInstaller PowerShell script

To download the latest version of AutoSPInstaller, click here.

The files should be extracted to C:\SP\AutoSPInstaller, assuming that the SharePoint 2016 installation files downloaded with AutoSPSourceBuilder were saved to C:\SP\2016.

This is the folder structure that AutoSPInstaler uses:

\SP\AutoSPInstaller\AutoSPInstallerLaunch.bat

\SP\AutoSPInstaller\AutoSPInstallerInput.xml

\SP\AutoSPInstaller\AutoSPInstallerMain.ps1

\SP\AutoSPInstaller\AutoSPInstallerFunctions.ps1

\SP\AutoSPInstaller\AutoSPInstallerFunctionsCustom.ps1

\SP\AutoSPInstaller\AutoSPInstallerConfigureRemoteTarget.ps1

\SP\AutoSPInstaller\config.xml

\SP\201x\SharePoint\<installation files & folders>

\SP\201x\SharePoint\PreRequisiteInstallerFiles\

\SP\201x\SharePoint\Updates\

\SP\201x\LanguagePacks\xx-xx\ 

\SP\201x\LanguagePacks\xx-xx\Updates\ 

\SP\201x\Updates\


In this example, the root folder of the installation is C:\SP.

Creating the AutoSPInstaller configuration file

The next step, is to configure the AutoSPInstaller input configuration file that can be done in the AutoSPInstaller web site.  The end result of the configurations set in the AutoSPInstaller web site is the download of an XML file (AutoSPInstallerInput.xml) that contains all the settings required to configure the SharePoint farm. The AutoSPInstallerInput.xml file contains among other configurations:

  • SharePoint version to install
  • The SharePoint product key
  • The Farm Account and all Managed Accounts and their credentials
  • The database instance that will support the SharePoint environment
  • The web applications we want to create (ex: Intranet. My Site, etc)
  • The service applications we want to enable and their configurations
  • Outgoing email settings for the farm
  • Database names: this is extremely useful since it allows us to have complete control over the names of all the databases that will be created during the SharePoint farm configuration and avoid having databases with unclear names containing GUIDs which is the default behavior if we configure the farm using SharePoint Products Configuration Wizard (PSConfig)

The AutoSPInstallerInput.xml file should be placed in C:\SP\AutoSPInstaller.

Installing SharePoint

After the AutoSPInstallerInput.xml file is configured, it’s time to install SharePoint!

Follow the following steps:

1. Run AutoSPInstallerLaunch.bat from the command line (run as administrator)

AutoSPInstallerLaunch.bat will launch a PowerShell window that will install SharePoint and configure the farm using as an input the AutoSPInstaller XML configuration file (AutoSPInstallerInput.xml) previously configured using the AutoSPInstaller web site.

2. After the PowerShell window is launched, the following will happen:

  • Managed accounts credentials will be validated. If any of the credentials is incorrect, the script will stop
  • SharePoint prerequisites will be installed
  • SharePoint will be installed
  • SharePoint language packs will be installed
  • SharePoint updates will be installed

During the script execution. a message may be presented saying that the script needs to be re-launched to workaround an known issue with SharePoint 2016 when language packs are being installed.

After the script is re-launched, the installation will resume until all SharePoint binaries are installed.

Configuring the SharePoint Farm

After SharePoint is installed, the script will ask if we want to proceed with farm configuration. To proceed, select “y” and click ENTER. The script will, among others, perform the following actions:

  • Create all managed accounts
  • Configure IIS and SharePoint ULS log file paths
  • Create the configured web applications (ex: Intranet, My Site)
  • Configure the service applications
  • Configure outgoing email settings for the farm

After the farm configuration is completed, the Central Administration will be opened and SharePoint is installed!

 

AutoSPInstaller is also already available for the recently announced new version SharePoint, SharePoint 2019 (still in Preview). To learn more about all the upcoming features in SharePoint 2019, click here.

Happy SharePointing!

About the Author

Miguel Isidoro
SharePoint & Office 365 Senior Consultant | Create IT

Miguel Isidoro is a tech enthusiast, focused on Microsoft technologies, specially everything that is related to SharePoint and Office 365 and works with Microsoft and SharePoint technologies for more than 16 years, from SharePoint 2001 up until SharePoint 2016 and Office 365.

LinkedIn Profile: https://www.linkedin.com/in/miguelisidoro/

How to install a SharePoint 2016 farm using PowerShell and AutoSPInstaller (Part 1)

Hello!

This post is the first part of a two part series where I will talk about a great way to install SharePoint in a fully automated way using PowerShell.

Introduction

Installing a SharePoint farm can be a complex task and the installation process is composed by a series of steps:

  • Downloading and installing SharePoint prerequisites
  • Downloading and installing the SharePoint binaries
  • Downloading and installing the lastest SharePoint updates
  • Downloading and installing the required language packs (optional, necessary if you want the SharePoint interface to be presented in a language other than English)
  • Configuring the SharePoint farm using SharePoint Products and Configuration Wizard

All the above steps are time consuming and can be error prone if, for example, you miss to download any of the prerequisites.

Fortunately, it is possible to fully automate a SharePoint farm installation using PowerShell by using a script called AutoSPInstaller. The installation process can be divided in two major steps:

  • Downloading the SharePoint installation package
  • Installing SharePoint and configuring the SharePoint farm

In this post, I will talk about the first part of the process: downloading the SharePoint installation package.

Downloading the SharePoint installation package

This process can be used for the following versions of SharePoint:

  • SharePoint 2010
  • SharePoint 2013
  • SharePoint 2016
  • SharePoint 2019 (still in Preview – to learn more about all the upcoming features in SharePoint 2019, click here)

In this post, I will use SharePoint 2016 as an example.

The download process will be managed by a script called AutoSPSourceBuilder that will download the SharePoint installation package and all its dependencies and will create a folder structure ready to be used in AutoSPInstaller. The AutoSPSourceBuilder script will download:

  • All SharePoint prerequisites
  • The latest SharePoint updates
  • If specified. the desired language pack

To download the SharePoint 2016 installation package, follow the steps below:

  1. Download SharePoint 2016.

2. Download AutoSPSourceBuilder. After download, the folder should contain the following structure.

3. Mount the .iso image of SharePoint 2016.

After mount, the mounted drive will have SharePoint installation files.

4. Run the following script in a PowerShell window (run as administrator).

.\AutoSPSourceBuilder.ps1 -GetPrerequisites $true -SharePointVersion “2016” -Languages “pt-pt”

Ths script will download SharePoint 2016 with all prerequisites and the Portuguese language pack. Change the “Languages” parameter to the Language Pack you want to include in your installation.

The script will start by detecting the SharePoint image that was previously downloaded. If successfully detected, it will extract the SharePoint installation files from the mounted image to the C:\SP\2016 folder.

5. Once the SharePoint installation files extraction is completed, you will be asked to select the SharePoint update that you want to include on your installation package. Select the latest available package (all SharePoint 2016 updates are cumulative and include all updates contained in the previous updates).

6. After selecting the SharePoint update, the script will resume and start downloading the SharePoint prerequisites. the selected language pack and the selected SharePoint update.

7. After completion, the PowerShell window should look like the image below.

The end result will be a structure of folders and files that will contain all the necessary files to install SharePoint with AutoSPInstaller.

Root folder:

Language Packs folder:

SharePoint bits folder (slipstreamed installation structure):

Updates folder:

To read the second part, click here.

AutoSPInstaller is also already available for the recently announced new version SharePoint, SharePoint 2019 (still in Preview). To learn more about all the upcoming features in SharePoint 2019, click here.

Happy SharePointing!

About the Author

Miguel Isidoro
SharePoint & Office 365 Senior Consultant | Create IT

Miguel Isidoro is a tech enthusiast, focused on Microsoft technologies, specially everything that is related to SharePoint and Office 365 and works with Microsoft and SharePoint technologies for more than 16 years, from SharePoint 2001 up until SharePoint 2016 and Office 365.

LinkedIn Profile: https://www.linkedin.com/in/miguelisidoro/

Intelligent Security in Office 365

One of the main concerns of companies in choosing a Cloud collaboration platform, in which the Office 365 platform is an example, is the security of their information. One of the biggest compliance, security, and privacy challenges in Office 365 is related to the demystification that having the information in the Cloud is less secure than if it resides on the premises of the organizations themselves. To address these same concerns, Microsoft has made a very strong investment in the areas of security, compliance and privacy in the Office 365 platform.

In fact, the Office 365 platform provides users and system administrators with several features to address needs and apply a set of best practices in the areas of security, privacy and compliance. Below we present some of the main features natively offered by the platform.

Regarding Security, the Office 365 platform ensures the encryption of data in transit and at rest. At rest, Bitlocker technology is used to encrypt all information on the servers’ hard drives. In addition, all files are segmented (in small pieces called chunks) and each segment is individually encrypted and encryption keys are securely stored in a different physical location.
In transit, all files are encrypted with TLS using 2048-bit keys.

Regarding Privacy, it is possible to define differentiated access policies based on 4 vectors: user, device, location and sensitivity of the information. Some examples include defining a time bound window when sharing information or only allow sharing with certain domains.

In what regards to Compliance, the main concern is to define a set of rules that allow sensitive information to be protected and to prevent the leakage of sensitive information outside of the organization. The Office 365 platform offers the following features to address this need:

  • Data Loss Prevention: allows organizations to create policies to protect their most sensitive information. Example: prevent documents with credit card information or citizen card numbers from being shared outside the organization.
  • Information Rights Management: allows organizations to create policies that protect the content of documents stored in the Office 365 platform. Examples: prevent documents from being printed and prevent “Copy & Paste” from document contents. It is important to note that IRM policies continue to apply after documents are downloaded and viewed offline.

Using DLP together with IRM provides a very powerful combination in order for companies to protect their most sensitive documents.

  • Mobile Device Management: allows organizations to create policies to manage security when accessing corporate information using mobile devices. Examples include defining PINs to access corporate information, prevent sensitive content from being copied from an corporate document to personal applications (prevent for example a user from copying credit card numbers to the body of an email and sending it to someone outside of the organization) or prevent Screen Capture of information within sensitive documents.

These are just a few of the examples of the vast set of features available to users and administrators to protect information residing on the Office 365 platform.

A very important note is that Microsoft does not have access to the organization’s data and the only occasions where this may be necessary is in resolving support incidents that require access to the data. In these cases, and using a feature called Customer Lockbox, the customer can approve or reject access requests, and access is only granted in case the request is approved. In addition, all accesses are audited to ensure the transparency of the process.

Security is therefore and increasingly not only a central concern for customers who evaluate a migration to the Cloud but rather a concrete reality for those who already enjoy it.

In you are interested in learning more about this subject, you can check out:

About the Author

Miguel Isidoro
SharePoint & Office 365 Senior Consultant | Create IT

Miguel Isidoro is a tech enthusiast, focused on Microsoft technologies, specially everything that is related to SharePoint and Office 365 and works with Microsoft and SharePoint technologies for more than 16 years, from SharePoint 2001 up until SharePoint 2016 and Office 365.

LinkedIn Profile: https://www.linkedin.com/in/miguelisidoro/

Intelligent Security, Compliance and Privacy in Office 365 session at SharePoint Saturday Lisbon

Last Saturday, I delivered a session at the SharePoint Saturday event, that was for the first time held in Lisbon.

My session was entitled “Intelligent Security, Compliance and Privacy in Office 365”,  focused on security, compliance and privacy around the Office 365 platform. The main topics of the session were:

  • Office 365 Platform Security
  • Privacy (Differentiated Access Policies, External Sharing, Granular Access Controls)
  • Compliance (Data Loss Prevention, Information Rights Management, Mobile Device Management)
  • Transparency (Customer Lockbox, SharePoint Insights)
  • Advanced Threat Protection

You can find the slide deck here.

Related Links:

SharePoint Saturday Lisbon 2016 web site
My SlideShare web site
|create|it| web site

About the Author

Miguel Isidoro
SharePoint & Office 365 Senior Consultant | Create IT

Miguel Isidoro is a tech enthusiast, focused on Microsoft technologies, specially everything that is related to SharePoint and Office 365 and works with Microsoft and SharePoint technologies for more than 16 years, from SharePoint 2001 up until SharePoint 2016 and Office 365.

LinkedIn Profile: https://www.linkedin.com/in/miguelisidoro/

SharePoint Tunning Session

Last week I delivered a session at SharePoint Portuguese Community 36th event.

My session was entitled SharePoint Tunning, focused on techniques used to improve the performance of a SharePoint farm. The main topics of the session were:

  • SQL Server performance tips (most of the presentation, since performance in SharePoint is guaranteed mainly at SQL Server level)
  • SharePoint farm performance tips
  • SharePoint development tips

You can find the slide deck here. I hope you understand Portuguese since the slide deck is in my mother language :).

Related Links:

About the Author

Miguel Isidoro
SharePoint & Office 365 Senior Consultant | Create IT

Miguel Isidoro is a tech enthusiast, focused on Microsoft technologies, specially everything that is related to SharePoint and Office 365 and works with Microsoft and SharePoint technologies for more than 16 years, from SharePoint 2001 up until SharePoint 2016 and Office 365.

LinkedIn Profile: https://www.linkedin.com/in/miguelisidoro/

Error “Value does not fall within the expected range” using SPFile.MoveTo in SharePoint

Problem:

Recently, I caught the following error in a piece of code that was trying to move a document between two SharePoint document libraries inside the same SharePoint site (SPWeb):

“Value does not fall within the expected range”

Cause:

The error was being throwned because an absolute URL was being specified as the destination URL in the SPFile.MoveTo method.

Solution:

According to the MSDN article at http://msdn.microsoft.com/en-us/library/ms468280.aspx, we have:

public void MoveTo(
	string newUrl
)

The newUrl parameter in the SPFile.MoveTo method specifies the destination URL. As explained in the article, the SPFile.Moveto can only be used to move files inside the same SharePoint site (SPWeb) and so relative URLs should be used. Example:

About the Author

Miguel Isidoro
SharePoint & Office 365 Senior Consultant | Create IT

Miguel Isidoro is a tech enthusiast, focused on Microsoft technologies, specially everything that is related to SharePoint and Office 365 and works with Microsoft and SharePoint technologies for more than 16 years, from SharePoint 2001 up until SharePoint 2016 and Office 365.

LinkedIn Profile: https://www.linkedin.com/in/miguelisidoro/

SharePoint and “The form cannot be displayed in the browser because the use of session cookies” error

Problem

Recently, a customer has created a new SharePoint environment and added an alternate access mapping (AAM) with the URL containing a “_” character (ex: http://intranet_QA). When opening an InfoPath form in an IE browser (from IE8 up to IE11), the following error was being throwned:

“The form cannot be displayed in the browser because the use of session cookies has been disabled in the current browser settings. In order to load the form, session cookies must be allowed.”

In other browsers, the InfoPath form opened successfully, without throwing any error.

Cause

This behavior is due to the fact that Internet Explorer doesn’t accept cookies with host names that contain the “_” character.

Solution

To solve this problem, the “_” character was removed from the URL. In this example:

Hope this helps!

About the Author

Miguel Isidoro
SharePoint & Office 365 Senior Consultant | Create IT

Miguel Isidoro is a tech enthusiast, focused on Microsoft technologies, specially everything that is related to SharePoint and Office 365 and works with Microsoft and SharePoint technologies for more than 16 years, from SharePoint 2001 up until SharePoint 2016 and Office 365.

LinkedIn Profile: https://www.linkedin.com/in/miguelisidoro/

SharePoint 2007 Workflow – Using the OnWorkflowItemDeleted activity

Introduction

One of the most exciting features included in SharePoint 2007 is workflow support and the possibility of developing our own custom workflows (for more information about workflow development in the SharePoint 2007 platform please click here).

While it is possible to develop workflows to automate a series of activities without any human intervention, the full power of workflows in the SharePoint platform can only be achieved with human-oriented workflows. These workflows are characterized by creating and assigning tasks to users that must complete them in order to fulfill the purpose of the workflow. Some examples of out of the box human-oriented workflows that are included with MOSS 2007 are Approval, Collect Feedback and Issue Tracking.

In the SharePoint world, the most common usage of human-oriented workflows is in the context of document libraries, to perform some work when a new document is uploaded into a document library. Typically, in such scenario, tasks are created and assigned to users throughout the workflow lifetime that will complete them until the workflow purpose has been fulfilled. Having the workflow goal to be achieved is the desired and most common scenario, but unexpected actions may occur before the end of the workflow such as a user to delete the document. The following example will show you how to handle this situation in a custom workflow by using the OnWorkflowItemDeleted activity.

Example

The OnWorkflowItemDeleted activity can be used within a custom workflow to handle an item deletion in any SharePoint list. In this example, this activity will be used to delete all uncompleted tasks associated with the deleted item. After dragging and dropping the OnWorkflowItemDeleted activity into the workflow designer, the Invoke property must be set to the event handler that will perform the work.

For the purpose of this example, please consider the simple approval workflow in the following image:

In this workflow, a parallel activity with two branches is used. One contains the main workflow logic while the other contains a OnWorkflowItemDeleted activity that will handle the item deletion. The Invoke event property was set to be handled by an event handler method called OnWorkflowItemDeleted_Invoked. Let’s take a look at the code:

public sealed partial class ApprovalWorkflow : SequentialWorkflowActivity
{
    /// <summary>
    /// Handles the workflow item deleted event.
    /// </summary>
    /// <param name="sender"></param>
    /// <param name="e"></param>
    private void OnWorkflowItemDeleted_Invoked(object sender, ExternalDataEventArgs e)
    {
        //delete uncompleted tasks when 
        //an item is deleted
        SPWorkflow workflowInstance = 
            workflowProperties.Workflow;
        SPWorkflowTaskCollection taskCollection = 
            GetWorkflowTasks(workflowInstance);
        for (int i = taskCollection.Count; i > 0; i--)
        {
            SPWorkflowTask task = 
                taskCollection[i - 1];
            using (SPWeb web = 
                workflowProperties.Web)
            {
                if (task[SPBuiltInFieldId.TaskStatus]
                    .ToString() != SPResource.GetString
                    (new CultureInfo((int)web.Language, false),
                    "WorkflowTaskStatusComplete", new object[0]))
                {
                    task.Delete();
                }
            }
        }
    }

    /// <summary>
    /// Reads the workflow tasks. This method 
    /// is implemented because the Tasks property
    /// of the SPWorkflow instance takes a 
    /// while to be populated.
    /// </summary>
    public static SPWorkflowTaskCollection 
        GetWorkflowTasks(SPWorkflow workflowInstance)
    {
        SPWorkflowTaskCollection taskCollection = null;
        bool tasksPopulated = false;
        while (!tasksPopulated)
        {
            try
            {
                taskCollection = workflowInstance.Tasks;
                tasksPopulated = true;
            }
            catch { }
        }

        return taskCollection;
    }
}

In the previous example, the following actions are being performed:

  • In the OnWorkflowItemDeleted_Invoked event handler, the workflow instance task collection is obtained by calling the GetWorkflowTasks method;
  • The task collection is iterated and the each task status is checked to verify if it is still uncompleted;
  • If so, the task is deleted.