Sitecore pipeline groups

September 30, 2014 by boro

Recently we’ve (mvp’s) have been lucky enough to get hold of a copy of the technical preview of Sitecore 8. It’s worth noting that anything within this version is subject to change!

One thing that a few people noticed and asked about during the Symposium was some additional config options around pipelines e.g.:

<group groupName="FXM" name="FXM">
    <pipelines>
        <tracking.matchpages>
            ...
        </tracking.matchpages>

<tracking.matchpages>

...

</tracking.matchpages>

This group syntax was something I’d not seen before – read on to find out how it works.

If you want to then run the pipeline you simply need to include the “domain” in the call. This example is from the TrackingManager within the new FXM dll’s:

this.pipeline.Run("tracking.matchpages", matchPagesArgs, "FXM");

1	this.pipeline.Run("tracking.matchpages", matchPagesArgs, "FXM");

where this.pipeline is:

Sitecore.Abstractions.ICorePipeline

1	Sitecore.Abstractions.ICorePipeline

One of the cool additions to the new Abstractions dll!

public interface ICorePipeline
{
	void Run(string pipelineName, PipelineArgs args);
	void Run(string pipelineName, PipelineArgs args, bool failIfNotExists);
	void Run(string pipelineName, PipelineArgs args, string pipelineDomain);
	void Run(string pipelineName, PipelineArgs args, string pipelineDomain, bool failIfNotExists);
}

public interface ICorePipeline

{

void Run(string pipelineName, PipelineArgs args);

void Run(string pipelineName, PipelineArgs args, bool failIfNotExists);

void Run(string pipelineName, PipelineArgs args, string pipelineDomain);

void Run(string pipelineName, PipelineArgs args, string pipelineDomain, bool failIfNotExists);

}

Another addition in the Sitecore 8 otb setup is more structure to the app_config folder. It’s all too easy for this to become rather unruly so this form of segmentation is a nice addition.

Office decor

August 12, 2014 by boro

This is a bit of a change from normal posts! We’ve recently had the office decorated including some very cool graffiti from http://spzero76.com

It’s now a game of spot the clients!

Large number of Sitecore items per folder

July 23, 2014 by boro

Sitecore always recommend a limit of around 100 items per folder. After running some tests on 2 versions of a tree I can see why! Below are a few stats to back that up.

Before:

And after:

Then running some queries over all the descendant items revealed:

Nested folders (after)
Total: 341
Time to read: 639ms

Flat items (before)
Total: 336
Time to read: 1786ms

Its clearly much faster to access all the items even with just some simple nesting (roughly 80 items per folder) rather than 300+ in one folder.

Sitecore – lots of versions or lots of children?

July 23, 2014 by boro

The performance of your site can very quickly deteriorate if you are using a non bucket content structure and either the number of items in a folder bloats or the number of versions of an item bloats.

For a bit more reading on the number of items per folder see: Large number of Sitecore items per folder

These sql scripts allow you to quickly find the number of these offenders:
Children in a folder

SELECT TOP 1000 
	ParentID,
	COUNT(*) AS Total,
    (SELECT Name FROM [dbo].[Items] J WHERE J.ID = I.ParentID) AS Name
FROM [dbo].[Items] I
GROUP BY ParentID
ORDER BY Total DESC

SELECT TOP 1000

ParentID,

COUNT(*) AS Total,

(SELECT Name FROM [dbo].[Items] J WHERE J.ID = I.ParentID) AS Name

FROM [dbo].[Items] I

GROUP BY ParentID

ORDER BY Total DESC

Versions of an item

SELECT 
	COUNT(DISTINCT Version) AS VersionCount, 
	Language ,
	ItemID,
	(SELECT Name FROM [dbo].[Items] WHERE ID = VF.ItemId) AS Name
FROM VersionedFields VF
GROUP BY ItemID, Language
ORDER BY VersionCount DESC

SELECT

COUNT(DISTINCT Version) AS VersionCount,

Language ,

ItemID,

(SELECT Name FROM [dbo].[Items] WHERE ID = VF.ItemId) AS Name

FROM VersionedFields VF

GROUP BY ItemID, Language

ORDER BY VersionCount DESC

Jasmine tests for jquery post

June 27, 2014 by boro

Jasmine is a js framework that allows behaviour driven testing of Javascript components. One challenge I ran into was how to setup the tests in order to mock jQuery.post(…) calls.

An example of the code for making the call is:

$.post(url, JSON.stringify({ "value": "1", "another_value": true })
).done(function (data) {
    handleApiResponse(data);
}).fail(function (data) {
    handleFailedApiResponse(data);
});

$.post(url, JSON.stringify({ "value": "1", "another_value": true })

).done(function (data) {

handleApiResponse(data);

}).fail(function (data) {

handleFailedApiResponse(data);

});

So, how do you test this and validate the correct url & data was used?

First, you need to spy on the ajax call:

spyOn($, "ajax").andCallFake(function (options) {
    var d = $.Deferred();
    d.resolve({});
    return d.promise();
});

spyOn($, "ajax").andCallFake(function (options) {

var d = $.Deferred();

d.resolve({});

return d.promise();

});

The interesting thing to note, even though you are calling $.post, you need to spy on $.ajax
If you want to force the .fail() state of the code then you can use d.reject(). See http://api.jquery.com/category/deferred-object/ for more alternatives.

When you then verify the valid behaviour, you can check eg:

var data = { "value": "1", "another_value": true };
...
expect($.ajax.mostRecentCall.args[0]["type"].toUpperCase()).toEqual("POST");
expect($.ajax.mostRecentCall.args[0]["url"]).toEqual(url);
if (data) {
    expect($.ajax.mostRecentCall.args[0]["data"]).toEqual(data);
}

var data = { "value": "1", "another_value": true };

...

expect($.ajax.mostRecentCall.args[0]["type"].toUpperCase()).toEqual("POST");

expect($.ajax.mostRecentCall.args[0]["url"]).toEqual(url);

if (data) {

expect($.ajax.mostRecentCall.args[0]["data"]).toEqual(data);

}

Sitecore – export all users that are in a role

June 19, 2014 by boro

If you’ve ever dug into the security tables of Sitecore you’ll discover its based on the aspnet membership tables which are typically stored in core.

Recently we needed to export all the users that are in a certain role – it turned out it was easier than expected.

The outcome is that when you view the members of a certain role you can now export to a csv.

There are a few things you need to update to enable the additional ‘Export’ button.

Update the xaml view for the members popup
Patch in the new command
Create a new sitecore command for generating the export

1. Update the xaml view for the members popup

If you navigate to ‘\sitecore\shell\Applications\Security\RoleManager’ and edit ‘ViewMembers.xaml.xml’ you should be able to see 2 buttons – the new one added below:

<Border runat="server">
    <Button Header="Add" ID="Add" Click="rolemanager:addmember" runat="server" />
    <Button Header="Remove" ID="Remove" Click="rolemanager:removemember" runat="server" />

    <Button Header="Export" ID="Export" Click="rolemanager:exportusers" runat="server" />
</Border>

</Border>

A tip for tracking down the xaml file in question – if you fire up either fiddler/chrome developer tools/firebug and open up the members panel and have a look at the pages being loaded you should see something like:

After the ~ you can see /xaml/Sitecore.Shell.Applications.Security.RoleManager.ViewMembers.aspx – this mimics the folder structure with fullstops replaced by folders.

2. Patch in the new command

Using sitecore patch files, create the new command reference

<commands>
      <command name="rolemanager:exportusers" type="###.Shell.Framework.Commands.RoleManager.ExportUsers, ###.Sitecore"/>

    </commands>

</commands>

3. Create a new sitecore command for generating the export

Now you just need the code. The example below allows for profile information to be included in the export, simply update the ProfileFields array to include the properties you want. ComputedFields allows you to amalgamate or process other fields to gather more information as needed.

using System;
using System.Linq;
using Sitecore.Diagnostics;
using Sitecore.Security;
using Sitecore.Security.Accounts;
using Sitecore.Shell.Framework.Commands;
using Sitecore.Shell.sitecore.shell.Applications.MarketingAutomation.Utils;
using Sitecore.Web;
using Sitecore.Web.UI.Sheer;
using System.Collections.Generic;

namespace ###.Shell.Framework.Commands.RoleManager
{
    public class ExportUsers : Command
	{
        protected class UserInRole
        {
            public UserInRole(string email, UserProfile userProfile)
            {
                Email = email;
                UserProfile = userProfile;
            }

            public string Email { get; set; }
            public UserProfile UserProfile { get; set; }
            
            public static string[] ProfileFields
            {
                get
                {
                    return new[]
                        {
                            "CountryId", "CountryName","DateSubscribed", "Firstname", "Lastname"
                        };
                }
            }

            internal static List<ComputedField> ComputedFields
            {
                get
                {
                    return new List<ComputedField>
                        {
                            new ComputedField("FullName", profile => String.Concat(profile["Firstname"], " ", profile["Lastname"]))
                        };
                }
            }

            public static string CsvHeader()
            {
                return string.Concat("Email", ",",
                                     string.Join(",", ProfileFields.Concat(ComputedFields.Select(a => a.Header))));
            }

            public string ToStringCsv()
            {
                return string.Concat(Email, ",",
                                     string.Join(",",
                                                 ProfileFields.Select(key => Clean(UserProfile[key]))
                                                              .Concat(
                                                                  ComputedFields.Select(a => Clean(a.Action(UserProfile))))));
            }

            private static string Clean(string value)
            {
                return value.Replace(",", "");
            }

            internal class ComputedField
            {
                public ComputedField(string header, Func<UserProfile, string> action)
                {
                    Header = header;
                    Action = action;
                }

                public string Header { get; private set; }
                public Func<UserProfile, string> Action { get; private set; }
            }
        }

		public override void Execute(CommandContext context)
		{
			Assert.ArgumentNotNull(context, "context");

		    string queryString = WebUtil.GetQueryString("ro");

		    IEnumerable<UserInRole> users = LoadUsers(queryString);

            string filename = "users_in_role_" + Clean(queryString) + DateTime.Now.ToString("_yyyyMMdd_HHmmss") + ".csv";

            string file = CsvExporter.ExportToFile(users, filename, UserInRole.CsvHeader, (s) => s.ToStringCsv());

            SheerResponse.Download(file);
		}

        private string Clean(string queryString)
        {
            return queryString.Replace("\\", "-").Replace("/", "-").Replace(" ", "-");
        }

        private IEnumerable<UserInRole> LoadUsers(string roleName)
        {
            List<UserInRole> users = new List<UserInRole>();

            Role role = Role.FromName(roleName);

            foreach (
                Account account in
                    Sitecore.Context.User.Domain.LocallyManaged
                        ? RolesInRolesManager.GetRoleMembers(role, false,
                                                             Sitecore.Context.User.Delegation.GetManagedDomains())
                        : RolesInRolesManager.GetRoleMembers(role, false))
            {
                User user = User.FromName(account.Name, false);

                if (user != null)
                {
                    users.Add(new UserInRole(user.Profile.Email, user.Profile));
                }
            }

            return users;
        }
    }
}

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

using System;

using System.Linq;

using Sitecore.Diagnostics;

using Sitecore.Security;

using Sitecore.Security.Accounts;

using Sitecore.Shell.Framework.Commands;

using Sitecore.Shell.sitecore.shell.Applications.MarketingAutomation.Utils;

using Sitecore.Web;

using Sitecore.Web.UI.Sheer;

using System.Collections.Generic;

namespace ###.Shell.Framework.Commands.RoleManager

{

public class ExportUsers : Command

{

protected class UserInRole

{

public UserInRole(string email, UserProfile userProfile)

{

Email = email;

UserProfile = userProfile;

}

public string Email { get; set; }

public UserProfile UserProfile { get; set; }

public static string[] ProfileFields

{

get

{

return new[]

{

"CountryId", "CountryName","DateSubscribed", "Firstname", "Lastname"

};

}

internal static List<ComputedField> ComputedFields

{

get

{

return new List<ComputedField>

{

new ComputedField("FullName", profile => String.Concat(profile["Firstname"], " ", profile["Lastname"]))

};

}

public static string CsvHeader()

{

return string.Concat("Email", ",",

string.Join(",", ProfileFields.Concat(ComputedFields.Select(a => a.Header))));

}

public string ToStringCsv()

{

return string.Concat(Email, ",",

string.Join(",",

ProfileFields.Select(key => Clean(UserProfile[key]))

.Concat(

ComputedFields.Select(a => Clean(a.Action(UserProfile))))));

}

private static string Clean(string value)

{

return value.Replace(",", "");

}

internal class ComputedField

{

public ComputedField(string header, Func<UserProfile, string> action)

{

Header = header;

Action = action;

}

public string Header { get; private set; }

public Func<UserProfile, string> Action { get; private set; }

}

public override void Execute(CommandContext context)

{

Assert.ArgumentNotNull(context, "context");

string queryString = WebUtil.GetQueryString("ro");

IEnumerable<UserInRole> users = LoadUsers(queryString);

string filename = "users_in_role_" + Clean(queryString) + DateTime.Now.ToString("_yyyyMMdd_HHmmss") + ".csv";

string file = CsvExporter.ExportToFile(users, filename, UserInRole.CsvHeader, (s) => s.ToStringCsv());

SheerResponse.Download(file);

}

private string Clean(string queryString)

{

return queryString.Replace("\\", "-").Replace("/", "-").Replace(" ", "-");

}

private IEnumerable<UserInRole> LoadUsers(string roleName)

{

List<UserInRole> users = new List<UserInRole>();

Role role = Role.FromName(roleName);

foreach (

Account account in

Sitecore.Context.User.Domain.LocallyManaged

? RolesInRolesManager.GetRoleMembers(role, false,

Sitecore.Context.User.Delegation.GetManagedDomains())

: RolesInRolesManager.GetRoleMembers(role, false))

{

User user = User.FromName(account.Name, false);

if (user != null)

{

users.Add(new UserInRole(user.Profile.Email, user.Profile));

}

return users;

}

Have you ever edited in the Sitecore web db by mistake?

May 22, 2014 by boro

For version 8, see v8.1 post

I think you’ll agree its rather easy to do!

From the content editor you can select which database you are using to view content:

If you select ‘core’ the tree contains quite a different set of items however the difference between master and web can be very subtle. I know I’ve run into the problem where I’ve edited web by mistake!

In order to build a bit more feedback to the user the sample script below shows how to give a bit more feedback to users:

You need to edit ‘\sitecore\shell\Applications\Content Manager\default.aspx’ and add the following javascript (added after <a id=”SystemMenu” .. />):

	<script type="text/javascript">
		function getParameterByName(name) {
			name = name.replace(/[\[]/, "\\[").replace(/[\]]/, "\\]");
			var regex = new RegExp("[\\?&]" + name + "=([^&#]*)"),
				results = regex.exec(parent.location.search);
			return results == null ? "" : decodeURIComponent(results[1].replace(/\+/g, " "));
		}
		window.onload=function(){ 
			if (getParameterByName('sc_content') == 'web') {
				var captions = document.getElementsByClassName('scCaption');
				for (var i = 0; i < captions .length; ++i) {
					var item = captions [i];  
					item.style.background="#761819 url(/sitecore/shell/themes/standard/gradients/red1.gif) 50% 0 repeat-y";
				}
			}		
		};
	</script>

function getParameterByName(name) {

name = name.replace(/[\[]/, "\\[").replace(/[\]]/, "\\]");

var regex = new RegExp("[\\?&]" + name + "=([^&#]*)"),

results = regex.exec(parent.location.search);

return results == null ? "" : decodeURIComponent(results[1].replace(/\+/g, " "));

}

window.onload=function(){

if (getParameterByName('sc_content') == 'web') {

var captions = document.getElementsByClassName('scCaption');

for (var i = 0; i < captions .length; ++i) {

var item = captions [i];

item.style.background="#761819 url(/sitecore/shell/themes/standard/gradients/red1.gif) 50% 0 repeat-y";

}

};

</script>

Vary Sitecore layout by language

March 28, 2014 by boro

One challenge we faced recently was how to vary sitecore page’s layouts across multiple languages. Out the box the layout field is shared, essentially removing the ability for its content to vary by language. In single language sites this isn’t really an issue but once you introduce several languages things get a bit trickier.

In the past we experimented with simply making the field unshared – often with mixed success. All the variations of layouts language by language became rather messy rather quickly – deltas would help this but you still can end up with some languages missing out on key components. Things like language fallback can help this but the solution still didn’t feel sustainable.

With the introduction of MVC there are a heap more pipelines available, especially around the rendering of the page. The solution we arrived at was to flip the problem around a bit and selectively apply layout changes.

Let me introduce the MOLE (hmm tenuous acronym?? 🙂 – better known as ‘MVC Layout Override Extensions’.

The pipeline this taps into is:

<mvc.getXmlBasedLayoutDefinition>
        ...
</mvc.getXmlBasedLayoutDefinition>

<mvc.getXmlBasedLayoutDefinition>

...

</mvc.getXmlBasedLayoutDefinition>

Out the box there is one processor which simply reads xml from an item and returns the data parsed as an XElement.

Updating this logic slightly, we hijack which item is used as the source for the data, Have a look in reflector at ‘Sitecore.Mvc.Pipelines.Response.GetXmlBasedLayoutDefinition.GetFromLayoutField’:

Item item = PageContext.Current.Item;

if (item == null)
{
	return null;
}
//MOLE the item
Item overrideItem = ItemQuery.GetRelatedLayoutOverrideItem(item);

if (overrideItem != null)
{
	item = overrideItem;

	//it can be useful to know elsewhere during the request if the current page is being mole'd - Items is an easy place to store this info.
	HttpContext.Current.Items[HttpContextItemsKeys.Presentation.Layout] = item.ID.Guid.ToString();
}

Field field = item.Fields[FieldIDs.LayoutField];

if (field == null)
{
	return null;
}

Item item = PageContext.Current.Item;

if (item == null)

{

return null;

}

//MOLE the item

Item overrideItem = ItemQuery.GetRelatedLayoutOverrideItem(item);

if (overrideItem != null)

{

item = overrideItem;

//it can be useful to know elsewhere during the request if the current page is being mole'd - Items is an easy place to store this info.

HttpContext.Current.Items[HttpContextItemsKeys.Presentation.Layout] = item.ID.Guid.ToString();

}

Field field = item.Fields[FieldIDs.LayoutField];

if (field == null)

{

return null;

}

It’s worth noting ‘ItemQuery.GetRelatedLayoutOverrideItem(item);’ is custom code which in this implementation makes use of a new link’ed field to select when an item is being ‘Mole’d.

public static Item GetRelatedLayoutOverrideItem(Item currentItem)
{
	Field layoutOverrideField = currentItem.Fields[FieldNames.LayoutOverride];

	if (layoutOverrideField != null)
	{
		string layoutOverrideFieldValue = layoutOverrideField.Value;

		if (!String.IsNullOrEmpty(layoutOverrideFieldValue))
		{
			Item overrideItem = currentItem.Database.GetItem(new ID(layoutOverrideFieldValue), currentItem.Language);

			if (overrideItem != null && overrideItem.Versions.Count > 0)
			{
				return overrideItem;
			}
		}
	}

	return null;
}

public static Item GetRelatedLayoutOverrideItem(Item currentItem)

{

Field layoutOverrideField = currentItem.Fields[FieldNames.LayoutOverride];

if (layoutOverrideField != null)

{

string layoutOverrideFieldValue = layoutOverrideField.Value;

if (!String.IsNullOrEmpty(layoutOverrideFieldValue))

{

Item overrideItem = currentItem.Database.GetItem(new ID(layoutOverrideFieldValue), currentItem.Language);

if (overrideItem != null && overrideItem.Versions.Count > 0)

{

return overrideItem;

}

return null;

}

This is probably the simplest way of selecting the override item. You could easily introduce the rules engine to give even more flexibility when choosing to override the source items layout.

What I think is really neat about this solution is editors can page edit or preview variants and then assign to any language at any point in time. If you only wanted to override one language variant, you simply only build the relationship to the variant on that given language.

As an add-on we built some ui notifications to indicate if an item was being overwritten, or was a variant in use elsewhere. Because a link field was used to create the relationship, this data is all available in the links db.

Happy mole’ing

Sitecore Related Item Publishing

March 17, 2014 by boro

In the latest release from Sitecore (7.2) a load of neat new features have been added relating to publishing. One of the major issues we’ve seen in the past is that content items are updated but then related items aren’t published correctly. The information in this post explains the updates to sitecore related item publishing.

There is a whole load of information about the updates on the sitecore blog (http://www.sitecore.net/Community/Technical-Blogs/Reinnovations/Archive.aspx) and specifically around the related items at http://www.sitecore.net/Community/Technical-Blogs/Reinnovations/Posts/2014/03/Related-Item-Publishing.aspx

During some tests, this first image shows the master db with several field types relating content. Here you can see things like RTE fields, Rendering Datasources, image fields and multilist fields.

Following a publish of the single item with the check box for ‘Publish related items’ selected, the following related items were then pushed to web:

Pretty neat! All the related items are gathered through a new pipeline so if the out the box rules don’t catch your required scenario, simply add your own steps.

Sitecore Incremental publishes

January 28, 2014 by boro

Its worth noting that during further testing we’ve uncovered some subtle nuances of having the setting Publishing.PublishEmptyItems as false. See the end of the post for more details.

When you are dealing with large amounts of content across multiple languages it can often take a long time for publishes to complete, even if the amount of changes are relatively small.

Out the box you are presented with 3 options for publishing: republish, smart and incremental. This post will detail some nuances of the sitecore incremental publishes when triggered via an agent.

Why incremental?
If you are dealing with a small amount of changes but to a large db, why bother processing or publishing the whole tree. The crux of incremental publishes is that you only process the items that have changed since the last publish. This is achieved using timestamps stored in the Properties table which indicates the last point in time a publish was completed.

The Publish agent
In an out the box installation there is an agent available to trigger publishes over time: Sitecore.Tasks.PublishAgent. This is configured in the <scheduling> section of the config and by default is set to run never. Our implementation set this up as follows:

interval=”00:15:00″
<param desc=”mode (full or smart or incremental)”>incremental</param>
<param desc=”languages”>en-GB, fr-fr</param>

Behind the scenes the agent creates a new Publisher, passes in the parameters and then and sets Deep=true.

What’s the problem then?
Deep introduces an odd problem to incremental publishes – in theory they should be as light as possible but there are certain circumstances where changing one node (eg adding a version to a parent with lots of children) would lead to log entries indicating thousands of items had been published:

INFO Job ended: Publish to ‘web’ (units processed: 11569)

Normal log entries would indicate that at most a hundred or so items should be published each run of the agent.

Can we stop incremental publishes triggering thousands of items?
Surely we can simply set Deep= false? If you make a custom version of the agent its simple enough to parametrize the value passed into the agent however this seemed to introduce other issues.

With the help of support we got to the bottom of a strange issue. After 2 runs of the agent (with 2 languages being published) we would see the child item would be missing the valid version.

Master db	Web db after 1 run of the agent	Web db after ‘parent’ is made publishable
parent (publishable in the future, en-gb only) -child (publishable now, en-gb only)	parent (no versions) -child (no versions)	parent (version in en-gb) -child (no versions)

This problem was only visible once the second language was added to the publish agent settings.

The solution
Simply add the setting: <setting name=”Publishing.PublishEmptyItems” value=”false” />.

And you should end up with:

Master db	Web db after 1 run of the agent	Web db after ‘parent’ is made publishable
parent (publishable in the future, en-gb only) -child (publishable now, en-gb only)	no change	parent (version in en-gb) -child (version in en-gb)

Does this introduce any problems?
Initial testing suggested these changes to the settings would solve our issues. In a multi language solution it was found that subtle problems around the publishing of children items actually caused more problems than were solved by the changes. In the end we avoided the ‘deep’ child publishing by creating a custom version of Sitecore.Publishing.Pipelines.Publish.ProcessQueue, Sitecore.Kernel. In that we then ignored the deep parameter for specific template types.

blog.boro2g .co.uk

Some ideas about coding, dev and all things online.

Author Archives: boro

Sitecore pipeline groups

Office decor

Large number of Sitecore items per folder

Sitecore – lots of versions or lots of children?

Jasmine tests for jquery post

Sitecore – export all users that are in a role

Have you ever edited in the Sitecore web db by mistake?

Vary Sitecore layout by language

Sitecore Related Item Publishing

Sitecore Incremental publishes