My today’s Sitecore discovery was that the Sitecore Content Editor have code that is dependent on what’s sent in the Referer header. (Yes, it’s incorrectly spelled in the HTTP standard.)

TL;DR: Set “Referrer-Policy” to “same-origin”. Most other sensible configurations will break Sitecore.

I’ve been working on a project where I found that the content search functionality didn’t work properly in the Content Editor. I found that search results were not limited to the scope of the context item, i.e. results from the entire database was always returned. I also found that no matter how I configured my custom facets, only the global facet where used.

I was just about to file a support ticket for this, and decided to reproduce this in a clean solution and found that it worked just as expected. So obviously we had done something in our solution that broke it. So I started to remove all our own config files, starting with content search related files, but I couldn’t get it working unless I did a full reset of my local development environment. So something fishy was going on here.

So I hocked up the symbol server in dotPeek and started debugging. Soon I found that the FacetSearcher.GetFacets() always returned just the global facet regardless if I had local facets. This was due to the startLocationItem variable always being the Sitecore root item. Strange…

Digging deeper, I eventually found that the SearchHttpTaskAsyncHandler have a LocationFilter property that reads an id parameter from the query string of the referrer header. If the id parameter wasn’t present, it would just use the Sitecore root item id.

So now I knew the probable cause of this. And yes, we’ve tried to follow most web security best practices in this project, so we had opted for setting the Referrer-Policy http header to strict-origin. This basically means that just the domain name would be sent in the Referer header. Not path nor query string.

So the takeaway from this is to set the Referrer-Policy header to same-origin on the CM role. Never set it to anything that disallows sending the query string to the origin site, i.e. don’t use no-referrer, origin or strict-origin.

But if I removed all we’ve followed most security best practices, such as using proper CSP headers etc. So in this solution we had the Referrer-Policy header set to strict-origin. I.e. the visiting browser would basically just send the domain name and not the path or query string.

Looking a bit deeper in the Sitecore assemblies, it turns out there’s quite many usages of request.UrlReferrer.

Sitecore has been using the unversioned standard field “__Display name” along with the built-in “Name” for as long as I can remember. Basically, the item Name is similar to a shared field and is limited to small set of characters. An item name can basically be A-Z, a-z, 9-0, space and a few special characters, such as underscore (_), dash (-), dollar-sign ($). It may also be a single star (*).

Since the Name is very limited, the __Display name fields allows users give item more user friendly names. Since it’s persisted in an unversioned field, it also allows for translation. The Sitecore.Data.Items.Item class has a convenience read-only DisplayName property that returns the content of the __Display name field, if set, otherwise it fallbacks on the item Name. This principle is used basically everywhere in Sitecore. It’s used when rendering the item tree, as link text on rendered links and so on.

In Sitecore 9.2 (I believe), Sitecore made a small, but very important breaking change on how the Display Name works. I can’t recall seeing this change been noted in any release notes either, but I might have missed it.

Update: I realize I’ll need to keep updating to this post as I receive more input. I’ll keep them the bottom of the post.

I recently ran into a strange issue where I noticed TDS wanted to sync a lot of item versions that didn’t exist. It took me a while before I nailed what was actually causing this. In the example below, the “Home” item only have a “en” language version. There is no version on the other languages. Still, TDS wants to serialize many more languages.

If I serialize this item into disk, it indeed adds all these versions as well.

Note: This is not a problem with TDS itself. It’s a consequence of a faulty database.

Sitecore has tons of “Switchers” that are pretty useful in many scenarios. You’re probably familiar with a few of them, such as EditContext, SecurityDisabler and LanguageSwitcher. Others are less known but very valuable when needed.

Switchers temporary puts Sitecore in a different mode for a piece of code and then restores it to its previous mode when done. A typical usage would look something like this:

using (new LanguageSwitcher(lang)) {
  // perform custom operations on "lang" instead of the context language
}

I made a list of common, and not so common switchers. Hope you find some of them useful.

I have a somewhat love and hate relationship to the Sitecore Links Database. It’s really useful in some scenarios, especially in scenarios such as integrations, batch operations etc., where a Solr index or similar might not accurate enough. The Links Database has also caused me many performance headaches over the years due to some issues with its implementation.

The main problem with the default Links Database Provider is that it performs multiple small synchronous SQL “insert” and “delete” operations in the database. As the Sitecore databases grows, so does the Links table. With many insert/delete operations, the table indexes becomes very fragmented and it slows down the system. The application performance may also be quite heavily effected if latencies to SQL Server is increased.

I’ve had on my radar for a few years to write a new one. Finally I made some time to improve the implementation a bit and put it on GitHub.

Sitecore ID’s, or Guids, are many times great as content can be created in different places and be moved between databases without facing the risk of colliding id’s. When integrating stuff into Sitecore, you may need to represent some data as Items. When you author new content in Sitecore that references such imported data, it suddenly becomes a bit trickier to move data between instances etc. You’d basically have to transfer all the integrated data as well to ensure data consistency. It also removes the ability to just remove integrated data and re-run the process, as it would generate new IDs.

So what if we could integrate data from an external source and represent it as items with a predictive Sitecore ID? I.e. give each item a unique ID that won’t collide with anything else, but will always become the same every time the process is run in every system.

When working with Content Search and Solr in Sitecore, it’s quite common that a field needs to be managed in both Sitecore and Solr. It can for example be a computed field or a basic field where you want specific field processing etc. where you need to slightly change the Solr schema. Having the configuration for such scenario split in two places can be quite annoying.

I while ago, I wrote a small module that’ll let you put your Solr Managed Schema configuration as part of the standard Sitecore configuration. This gives a better overview and configuration that plays together, stays together.

The module adds a new <solrManagedSchema> section in the contentSearch/indexConfiguration-section. The content of this section is almost identical to what you’d put in the Solr schema. The only difference is that fields, field types, dynamic fields and copy-fields are grouped separately with a Sitecore “hint” attribute.

Let’s describe the benefits of this with an example. Let’s say you need a computed field returning a set decimal numbers, like a set of variant specs etc. You’d obviously need the computed field class itself, but you’d also need to do a type match (because list of floats are not within the default mapping) and you’d need to modify the schema to support fields with multiple floats. The configuration could end up something like this:

Caching of renderings is a vital part of good performing Sitecore websites. Getting all the settings right aren’t trivial and it’s really only the developers of components that knows the details about how each component can be cached. Therefore caching options are typically managed on the rendering items and kept serialized together with solution source.

However, rendering cache options can also be set on a rendering in the layouts field when it’s used on a page. This is done in the “Control Properties” dialog. Personally I’ve never found a use case where this is needed, but the option is there.

Whether a user has access to the rendering parameters in the “Control Properties” dialog is controlled by “write” access to the Standard Rendering parameters template’s standard value. Sigh!

I guess most of us Sitecore developers are familiar with the JetBrains ReSharper plugin for Visual Studio. The tool actually made me accept moving from the Java/IntelliJ IDEA world to the .Net/Visual Studio world many many years ago and I’m still on the Idea keyboard shortcut scheme.

Besides all the nice refactoring tools, code hints etc that comes with ReSharper, it also comes with a framework for annotating code with attributes. One can argue if this should be used or not in your own code, but it opens for a really nice way for improving code quality when working with external libraries.

As Sitecore has grown over the years, the API becomes larger and larger and there are sometimes multiple ways of achieving the same thing. Sometimes the API is a bit ambiguous to new developers and some operations should be avoided from a performance perspective etc. With ReSharper External Annotations we can give developers code hints and feedback directly in Visual Studio when using the Sitecore API in a way that may not be intended.

PDF documents in Sitecore media library can be indexed using IFilters, but it has faced its limitations regarding Azure support etc and isn’t very effective from a performance point of view. The way the extracted content is indexed also makes it harder to use in multi-language solutions.

I’ve taken a different approach on indexing PDF documents, making it more accurate and increase the performance at the same time. The IFilter approach is a generic approach, supporting multiple file formats. I’ve focused on PDF documents in this post, as it’s a common format. Similar principles can be applied to other file formats as well.

In this post:

• Avoiding heavy computation during index time
• Extracting document content through PDF libraries
• OCR scanning of image/non-text based PDF documents
• Indexing documents with language stemming

When an item is cloned in Sitecore, the clones inherits its values from the source item. This is represented by a null value in each field, meaning that it inherits its value from the clone source item. When a value is written to a field in a clone, that value is used instead, hence breaking the inheritance. This works great in most cases.

In some scenarios you might not want to inherit all the fields. You might want to exclude some of them, enforcing a local value in each field for such clones. By default a few fields are not inherited. Those are __Created, __Created by,  __Updated, __Updated by, __Revision, __Source, __Source item, __Workflow, __Workflow state and __Lock. It’s quite natural that those fields are not inherited to clones, since each item, the source and the clone, should keep their own values of those fields.

You can add your own fields to this list by modifying the ItemCloning.NonInheritedFields setting. It’s a string setting where you can provide a pipe (|) separated list list of field ID’s or field keys. The drawback of the setting being a pipe separated list, is that it’s hard to add additional fields through config patch files. I hope Sitecore will change this in the future.

I discovered that the heap gets very fragmented in SQL Server in some of our solutions. Large tables, such as Items, Shared-, Versioned- and Unversioned-fields, Blobs, Descendants and Links tables, that easily occupies a few GB on disk, also suffered from great fragmentation. More than 90% fragmentation was common.

From what I’ve found, the only way to fix SQL Server Heap fragmentation (the heap is where all the table data is stored), is to have a clustered index on each table.

However, I noticed that no tables in the Sitecore databases have any clustered indexes. All indexes are non-clustered in the common master/core/web databases. Sitecore used to have clustered indexes back in 5.2, but over the course of multiple Sitecore versions, the database schema has changed to non-clustered indexes.

A clustered index means that the table rows as stored in the index order physically on disk. That’s also why there can be only one clustered index per table. With a non-clustered index, there is a second list that has pointers to the physical rows. It’s generally faster to read from a clustered index, but it may be slower to write to it as there may be a need to rearrange the table data.

As you probably know, all fields in Sitecore can have one of three field scopes: Versioned (aka Normal), Unversioned and Shared. Versioned fields have individual version numbers for each language. Unversioned fields have individual values for each language in the same way as versioned fields, but there can be only one value per language. Shared fields are just a single value regardless of language and item version. There are no such thing as a “versioned shared” field type.

This is configured using two check boxes on a field level: Shared and Unversioned. If none are checked, the field becomes a versioned field. As you see, there’s an ambiguous “invalid” state where both check boxes are checked. In this case, Shared has precedence.

Sorting search results are rather straight forward at first glance, but there are some pitfalls to be aware of. When using Sitecore Content Search, the Linq provider supports the OrderBy method and it get serialized into a sort statement in a Solr query. Example:

var result = searchContext.GetQueryable<MyModel>()
   .Where(...)
   .OrderBy(x => x.DisplayName)
   .GetResults()

will be serialized into a Solr query like

?q=...&fq=...&sort=_displayname ASC

This usually works quite well, but consider the following list of item display names: