Sitecore Azure web-role disk space

by

If you need to deploy your app to Azure cloud-services you’ll notice the installation drive will yo-yo between E: and F: each time you deploy. These drives are always created with rather limited capacity – roughly 1.5GB.

Certain Sitecore operations push a lot of data to disk, things like: logging, media caches, icons, installation history and lots more. As part of your deployment I’d suggest you setup the following changes:

  • Move the data folder to another location
    • This can be patched in with the sc.variable dataFolder
  • Move the media cache
    • This can be patched in with setting Media.CacheFolder
    • Remember to also update the cleanup agents which look to delete old file based media entries. These are referenced in the CleanupAgent.
  • Move the temp folder
    • Now, this isn’t quite so simple. The reason for noting this item is installing update files creates large __UpgradeHistory folders. Also, things like icons are cached in the /temp folder. For this reason /temp needs to live as /temp under the site root. With some help from support they concluded that rather than a virtual directory, a symlink was the best approach. For details on setting one up via powershell, see http://stackoverflow.com/questions/894430/powershell-hard-and-soft-links. As part of our startup scripts we included calling the script to shift things around.
    • The clue a folder is a symlink is the icon will change ala:
      symlink

Grunt and gulp tasks in visual studio 2015 – libsass error

by

One of the neat features baked into the new Visual Studio is the ability to run grunt and gulp tasks. This can be done on demand via the Task Runner Explorer or tied into build events.

Some simple steps for getting started with Grunt are: http://www.iambacon.co.uk/blog/getting-started-with-grunt-sass-and-task-runner-explorer-visual-studio

All pretty cool, if it works! If you’ve got a version of node installed be aware that Visual Studio also ships with its own. It took me a while to track down so thought it worth sharing. If you receive errors when trying to run these tasks eg:

Then try swapping the options so that $(PATH) is higher priority than the MS Web tools option:

path

 

This was found with some help from:

http://blogs.msdn.com/b/webdev/archive/2015/03/19/customize-external-web-tools-in-visual-studio-2015.aspx

Azure WAD Diagnostics – Sitecore counters

by

Azure allows you to pipe loads of stats and diagnostic information to blob and table storage. Google for “WAD Azure” and you’ll find a lot more information.
Its great for logging things like the event queue, performance counters and loads more over time.

If you want to push Sitecore counters through to this you are in luck. Make sure the counters are installed on the box – for web-roles this can be done via a startup task.

Then update your ‘diagnostics.wadcfg’ config to include the counters you want – make sure you include the (*)!

This caught me out for a while, I was missing the (*).

There is a handy cmd you can run to see all the counters available:

No Sitecore logs, no Event log entries and no working site?

by

It’s not a nice place to be I’m sure you’ll agree!

Before we go on, I can’t guarantee one size will fix all!! This worked (well, fixed) our setup – it’s by no means the only way to solve the scary place of no working site.

In our setup we make use of the IIS URL Rewrite module. Under the hood this maps some xml within the web.config to the rules it applies. If for any reason you’ve goofed up this config, easily done with rogue config transforms or general typo’s, then you may have broken this IIS feature. The giveaway, when you double click the feature in IIS you receive an error message.

The reason for posting – this took me a while to track down the first time, now it’s my go-to verification if we don’t get any Sitecore logs or Event log entries.

The issue here isn’t really related to Sitecore, the lack of Sitecore logs is simply another symptom of the problem.

Migrating a sitecore DB to PAAS SQL Azure

by

Moving to the cloud offers many new opportunities for how you store and handle your data. One challenge we ran into was how to migrate a large Sitecore web db into a PAAS sql azure db.

There are quite a few tools for doing this, even SQL management studio can help. The problem we faced was the source db had several filegroups setup, SQL Azure doesn’t support this.

To get around it we first created the DB through the portal, then scripted in the db structure – follow the steps in this link: http://blog.sqlauthority.com/2010/06/04/sql-server-generate-database-script-for-sql-azure/

The second step was then the data. We used the Azure MW tool (https://sqlazuremw.codeplex.com/) . When selecting what to migrate, select ‘data only’. This can take a while to run, several hours in our case, but at least got the data upto the cloud.

Slow Sitecore publishing within Azure

by

We’ve recently been migrating a large Sitecore 6.6 app to Azure. It’s not using the module, instead custom powershell to spin up web-roles and PAAS Dbs. As part of the migration we’ve been benchmarking how long publishes to take when you compare things like: on-premise -> on-premise; on-premise -> azure and azure -> azure.

The stats were taken by republishing the same set of items N times then analysing the logs to look for key entries relating to the publishes starting and finishing. If anyone is interested in how this is done let me know, I have a console app for extracting data from the logs and writing to summaries or CSV’s.

During the initial testing we found the performance into Azure was noticeably slower than on-prem -> on-prem. After things had been tuned up the performance was pretty similar (on-prem to on-prem was roughly 10s benchmark, azure to azure roughly 15s benchmark).

What stumped me for a while was which components to scale up?

You’ve got a few options:

  • Core db
  • Master db
  • Web db
  • Webbox

It turns out a combination of all the above plays some part however the item that had the most noticeable effect was actually the core db!

Thinking about this more it kind of makes sense. The core db plays a bigger role in the application than you might realize: all the user details are held in core; the links db is held in core & the eventqueue is used within core. All these factors contribute to ‘chatty’ behavior outside of the master and web dbs.

Azure diagnostics not writing to blob storage

by

Azure offers some very useful out the box tools for pooling diagnostic information into blob and table storage. http://azure.microsoft.com/en-gb/documentation/articles/cloud-services-dotnet-diagnostics/ contains lots of information on this.

In your solution you should end up with a file: diagnostics.wadcfg under the roles folder. One issue we ran into occurred when we changed the name of the counterpart project, the diagnostics file then sat in the wrong place. Ensure the structure is as follows:

You can verify this by checking the Azure project ccproj file. You should see:

When the site fires up you should then see blob storage contain a folder called: wad-control-container. If you have a look in there it should contain a file that mimics the content of your diagnostics.wadcfg file.

I found it was useful to clear out this folder before testing new deployments.

How to write a basic powershell module that includes multiple scripts

by

As part of a recent project I’ve needed to work closely with some of the Azure api’s. You can achieve some really cool things pretty quickly once you’ve learnt a few basics.

A good tip I picked up was that if you can implement something in .net/c# chance are you can replicate in PowerShell. If there isn’t a specific PowerShell module/api available you can call into the .net dll. This can be loaded via e.g.:

In order to structure your various functions the recommended approach is to use modules. A quick google will bring up more details. What I couldn’t find was how to setup a basic module that includes multiple scripts.

When you create new modules they can live in a couple places, $env:PSModulePath will show you this.

A module structure is pretty basic, you need a psm1 file (and psd1 – see https://www.simple-talk.com/sysadmin/powershell/an-introduction-to-powershell-modules/)
ps structure
Script.ps1 is basic, it contains a simple function:

Custom.psm1 then looks to load all the scripts you have. Note, make sure the folder name matches the psm1 name:

These files need to live in one of your module folders e.g. %UserProfile%\Documents\WindowsPowerShell\Modules

It’s then a case of installing your module:
Check what modules are available to install via: Get-Module -ListAvailable
And install the one you want via: Import-Module Custom. Note, Custom is the name of the folders and psm1 file.

Provided it’s all gone to plan you should now be able to call HiThere from a ps prompt

Debugging Azure web-role startup tasks

by

One feature Azure offers for getting your  boxes configured is the notion of startup tasks – I won’t go into too much detail here as there is lots available online e.g. https://msdn.microsoft.com/en-us/library/azure/hh180155.aspx

As part of setting these up I thought I’d share a few tips / gotchas that caught me out when running powershell from the cmd tasks.

My solution setup was:

  • Site root
    • StartupScripts
      • Script.cmd
      • Script.ps1

So then I’d reference in the task:

 <Task commandLine=”StartupScripts\script.cmd” executionContext=”elevated” taskType=”simple” />

Nothing rocket science so far! So, why didn’t the script work? I could jump on the box and run the cmd and it would be fine.

How to debug the process?
I found the most useful way was to add markers from the cmd and the ps1. The cmd file looked like:

Note, the .\startupScripts part of the ps1 path is v important!

Then the powershell:

Note, if you try to write to log.txt you will get process locked exceptions as the cmd holds locks on the file.
There are all sorts of techniques for writing to a file, this example uses a StreamWriter. Hit up google for different examples.