Part II - Integrating Automated Reverse Azure Database Migration PowerShell Script into Azure DevOps

In my last post, we wrote a handy PowerShell script that takes the latest Master and Web SQL Databases from a Production-level Azure Resource Group and imports them into a Staging/UAT/Dev Azure Resource Group for a seamless reverse database promotion process.  

The original script, however, relies on a developer to run the script manually on a local machine and authenticate their credentials in order to utilize the AzureRm commands:

We can take this script a step further and integrate it as a new stage in the existing Azure DevOps Release Pipeline, or as a new dedicated Release Pipeline that can be executed independently.

In this example, we will create a new Azure DevOps Release Pipeline.  We'll assume a Service Principle connection already exists (which is likely if you're deploying to your App Services using Azure DevOps already) and you have the proper administrator permissions to create pipelines in Azure DevOps.   We'll also be working with an Inline Azure PowerShell script job instead of including a script file from an artifact.  Steps will slightly differ if you want to go that route, but the concept would remain the same. 

Release Pipeline Setup

Head over to the Pipelines > Release dashboard, click the New dropdown and select New release pipeline.

In the 'Select a template' menu, click 'Empty job'.

Modify the Pipeline name, then click on Stage 1 and click the plus sign on Agent job to add a new agent.  Search for 'powershell', find Azure PowerShell task and click the Add button

Set the Azure Subscription to the appropriate service principle, set the Script Type to Inline Script, and set the Azure PowerShell Version to Latest installed version

Save the pipeline and navigate to the Variables section

Variable Setup

Here, we'll add all the variables that we'll consume in the script - allowing for future modification without touching the script code itself.  

In our case, our script calls for the following variables: 

• - sourceResourceGroupName
• - sourceSqlServerName
• - sourceMasterDbName
• - sourceWebDbName
  
  
• - targetResourceGroupName
• - targetSqlServerName
• - targetSqlServerAdminUserName
• - targetSqlServerAdminUserPassword
• - targetMasterDbName
• - targetMasterSqlUserPassword
• - targetWebDbName
• - targetWebSqlUserPassword
• - targetCdServerName
• - targetCmServerName

Script Modifications

Luckily, our original script doesn't need too much tinkering! Just a bit 😉 

First, we'll want to remove the Login-AzureRmAccount command altogether since the Azure PowerShell task in the pipeline will authenticate off of the service principle.

 

We'll then replace any hardcoded variables with their new corresponding variables we previously configured throughout the script using the $env:someVariableName format:

We'll finish this off by placing the modified script in the Inline Script field of our Azure PowerShell task.

Read the full post →

Azure App Service Deployment Error: There is not enough space on the disk.

I recently came across an error during a deployment to one of our Content Delivery App Service instances - stopping the deployment process dead in its tracks. 

The error occurred when deploying to the CD Slot:

This occurred each time we attempted to deploy this step.

The error in full does indicate troubleshooting codes and links (oddly enough, the Microsoft link did not have any trace of the error code):

Failed to deploy web package to App Service. Error Code: ERROR_NOT_ENOUGH_DISK_SPACE More Information: Web Deploy detected insufficient space on disk. Learn more at: http://go.microsoft.com/fwlink/?LinkId=221672#ERROR_NOT_ENOUGH_DISK_SPACE. Error: The error code was 0x80070070.

I found it especially strange given that we saw no indication that we've hit any limit on storage space in Azure Portal.  Initially, I thought it may have something to do with storage on the build server, but was able to quickly rule out this theory by verifying that there was plenty of disk space remaining on that machine.

The error was also evident when I attempted to upload any file directly via FTP:

I began a look for some giant file(s) that may be preventing any additional files from being uploaded (since that's what made the most sense given the context of the error message itself).

By logging into the App Service via FTP, I was able to identify two IIS memory dumps located in the /LogFiles directory.  These both appear to have been taken the month prior - and simply never removed. 

After deleting both memory dump directories, I was able to restart the deployment step - which completed without errors. Direct FTP uploads were also restored.

What's up with that?

Well, your Azure App Services is tied to a particular pricing tier that dictates storage, memory, ACU, etc.  In our case, this particular App Services is configured to use the Standard standard tier - which has a 50GB limit.

By leaving the remnants of these IIS memory dumps on the App Service's storage, we must have surpassed that storage limit.

This brings up some interesting questions:

First, based on the configured pricing tier, is there a limit on how much Sitecore can store in App_Data/MediaCache or /temp folders before hitting that cap?

I assume the answer to that is yes - if your application is not setup to periodically clean stale files (which it should by default), it's possible to reach that storage limit and cause this error to surface.  In that case, the quick fix to get your deployment out would be to remove some or all temp files in the application.

Second, how can we monitor this storage limit of App Services in Azure Portal?

I'll have to circle back on this one as I don't quite have the answer to it.  It may even already exist, and I just haven't spotted it yet.   I'll update this post if I do figure that one out, but please let me know if you have the answer in the comments!

Read the full post →

Sitecore Website Backup Powershell Script

Depending on the environment, infrastructure, and budget - each Sitecore project has their own backup and deployment process.  More recent solutions often include a file backup step during an automated build process - while others take on a more manual process.

Whatever your specific scenario, my rule of thumb for changing anything is: ALWAYS BACKUP BEFORE MAKING CHANGES.

With solutions using less complex deploy processes - I used to select, copy, and paste specific files and directories to a separate location in case something went completely awry.  To save time - and maintain peace of mind before making file changes to a staging or production environment - I now utilize a handy backup script using Powershell that backs up all necessary files (Robocopy) by simply running it.

To use it:

1. Modify and save the Settings.xml file with your own source and destination paths.
   
   
2. Double-click on RunBackup.bat file.  Once you accept the prompt to run the script as an administrator, you'll see the output of each file being transferred to the destination folder.
   
   
3. Watch it run!
   

The script explicitly excludes the following Sitecore specific paths that often aren't changed from build to build - and don't need to take up more space than needed.  To add/remove an excluded entries,  open and modify the _sitecorebackupscript.ps1 file's '$exclude' variable.

"_DEV", "App_Data", "Content", "temp", "upload", "sitecore", "sitecore modules" 

Settings.xml - Configuration settings

 <?xml version="1.0" encoding="UTF-8"?>  
 <settings>  
      <!-- Date format for Destination folder -->  
      <DateFormat>MM-dd-yyyy hhmmss</DateFormat>  
      <!-- Source Website directory to backup (exclude trailing back-slash) -->  
      <SourceDirectory>C:\inetpub\wwwroot\Sitecore8\Website</SourceDirectory>  
      <!-- Destination directory where backup made (exclude trailing back-slash) -->  
      <BackupDirectory>C:\inetpub\wwwroot-backup\Sitecore8</BackupDirectory>  
 </settings>  

_sitecorebackupscript.ps1: The Powershell script

 <# Sitecore Backup Script - Gabriel Streza (sitecoregabe.com)   
 **************************************************************************  
 Powershell script that backups your Sitecore website while excluding unessesary files.  
   
 Modify Settings.xml to control Source and Directory paths.   
   
 **************************************************************************  
 #>  
   
 # Directory definitions  
 $scriptpath = $MyInvocation.MyCommand.Path  
 $directory = Split-Path $scriptpath  
 [xml]$ConfigFile = Get-Content "$directory\Settings.xml"  
   
 # Script Variable Defintions  
 $DateFormat = Get-Date -format $ConfigFile.Settings.DateFormat  
 $SourceDirectory = $ConfigFile.Settings.SourceDirectory  
 $BackupDirectory = $ConfigFile.Settings.BackupDirectory  
 $dest = $BackupDirectory + '_' + $DateFormat  
   
 # Folders to exclude  
 $exclude = "_DEV", "App_Data", "Content", "temp", "upload", "sitecore", "sitecore modules"  
   
 # Run as Administrator  
 if (!([Security.Principal.WindowsPrincipal][Security.Principal.WindowsIdentity]::GetCurrent()).IsInRole([Security.Principal.WindowsBuiltInRole] "Administrator")) { Start-Process powershell.exe "-NoProfile -ExecutionPolicy Bypass -File `"$PSCommandPath`"" -Verb RunAs; exit }  
   
 # File copy  
 robocopy $SourceDirectory $dest /s /xj /xd $exclude  
   
 # Completed  
 Write-Host -NoNewLine 'DONE! Press any key to continue...';  
 $null = $Host.UI.RawUI.ReadKey('NoEcho,IncludeKeyDown');  

RunBackup.bat: Execute the Powershell script

 Powershell.exe -executionpolicy remotesigned -File "_sitecorebackupscript.ps1"  

You can download, modify, and utilize this by heading to this GitHub repo and grabbing a copy:
https://github.com/strezag/Sitecore-Website-Backup-Script/


Happy backup...ing!

Read the full post →