Showing posts with label kudu. Show all posts
Showing posts with label kudu. Show all posts

Monday, May 9, 2022

Latest Azure PaaS Sitecore Logs using a single line of PowerShell

If you’re anything like me, you probably don’t have a passion for manually digging through the series of hundreds of randomly dated folders that look like this in search of the latest Sitecore logs:


Hello darkness, my old friend

Although several tools and approaches are available (including this nifty tool credited to fellow Sitecore MVP Kiran Patil - as well as some of my own previous posts from 2018 and 2019 covering this topic), I've recently adopted a different strategy that's proved to be successful across several Sitecore PaaS clients for quickly obtaining the latest physical Sitecore log for a given server. 

The post-worthy kicker?  It's one line of PowerShell:

$kuduHost = "https://yourazuresitename-xp2-cd.scm.azurewebsites.net"; Write-Output "`n[ LATEST SITECORE LOGS ]`n"; $array = @(); Get-ChildItem "C:\home\site\wwwroot\app_data\logs" -File -Recurse | Where-Object { $_.FullName -match "azure.*.txt" -and $_.LastWriteTime -gt (Get-Date).AddHours(-12) } | ForEach-Object { $path = $_.FullName.replace("C:\home\site\wwwroot\app_data\logs\", "$kuduHost/api/vfs/site/wwwroot/App_Data/logs/"); $array += "`n[$($_.LastWriteTime)]`n$path`n"}; $array | Sort-Object $_.LastWriteTime | Select-Object -Last 3

😬

Okay, it's...kind of a long one-liner...but one line nevertheless 

The above example outputs direct links to the latest three physical Sitecore log files, which match the pattern 'azure.*.

In practice, the desired file can be highlighted from the console and at which point you can copy the URL or open it in a new tab.



Let's break it down

The first line defines a variable for the KUDU host you're using:

$kuduHost = "https://yourazuresitename-xp2-cd.scm.azurewebsites.net"

The second line outputs a (wholly arbitrary and unnecessary) title:

Write-Output "`n`[ LATEST SITECORE LOGS ]`n"

The third line represents an array variable aptly named `$array` (because I'm clever):

$array = @();

This is where it gets exciting. This Get-ChildItem cmdlet gets all files recursively under the site's `\App_Data\logs` location:

Get-ChildItem "C:\home\site\wwwroot\app_data\logs" -File -Recurse
Neat!

We can pipe in a Where-Object cmdlet to filter only file names that match 'azure.*.txt' (or if you want all log types - Publishing, Crawling, Dianoga, SPE, etc. - *.txt) and provide a 12-hour threshold against the `LastWriteTime` property:

Get-ChildItem "C:\home\site\wwwroot\app_data\logs" -File -Recurse |
Where-Object {$_.FullName -match "azure.*.txt" -and $_.LastWriteTime -gt (Get-Date).AddHours(-12)}

We can then pipe in a ForEach-Object cmdlet to iterate through each file:

Get-ChildItem "C:\home\site\wwwroot\app_data\logs" -File -Recurse | Where-Object { $_.FullName -match "azure.*.txt" -and $_.LastWriteTime -gt (Get-Date).AddHours(-12) } | ForEach-Object { $path = $_.FullName.replace("C:\home\site\wwwroot\app_data\logs\", "$kuduHost/api/vfs/site/wwwroot/App_Data/logs/"); $array += "[$($_.LastWriteTime)]`n$path`n"}

Notice that in the ForEach-Object cmdlet, we create a variable called `$path` and set it to a string that takes the file's FullName and replaces the 'system path' portion, and replace it with our `$kuduHost` variable concatenated to `/api/vfs/site/wwwroot/App_Data/logs/`.

$path = $_.FullName.replace("C:\home\site\wwwroot\app_data\logs\", "$kuduHost/api/vfs/site/wwwroot/App_Data/logs/")

Without this string replacement, we'd only get the system path for the files in the dataset, which would still require navigating to the file manually:

Also, within the ForEach-Object cmdlet, a formatted string containing the LastWriteTime and the `$path` variable is added to the `$array` variable:

$array += "[$($_.LastWriteTime)]`n$path`n"

The `n used above allows for line breaks.

After the files have been processed, the `$array` variable is called and sorted by LastWriteTime.

A Select-Object cmdlet is piped in to limit the number of results to 3:

$array | Sort-Object $_.LastWriteTime | Select-Object -Last 3


By combining these together, eliminating spaces, and adding semi-colons to separate commands, we've got our one-liner! 🕺

Bonus: IIS HTTP Request Logs

Using the same approach with a few modifications, the application's raw IIS HTTP Request Logs can also be obtained (differences bolded below):

$kuduHost = "https://yourazuresitename-xp2-cm.scm.azurewebsites.net"; Write-Output "`n[ LATEST IIS LOGS ]`n"; $array = @(); Get-ChildItem "C:\home\LogFiles\http\RawLogs" -Recurse | Where-Object { $_.FullName -match ".log" -and $_.LastWriteTime -gt (Get-Date).AddHours(-12) } | Sort-Object $_.LastWriteTime | ForEach-Object { $path = $_.FullName.replace("C:\home\LogFiles\http\RawLogs\", "$kuduHost/api/vfs/LogFiles/http/RawLogs/"); $array += "[$($_.LastWriteTime)]`n$path`n"}; $array | Sort-Object $_.LastWriteTime | Select-Object -Last 3


Final Thoughts

You can generate variations of this one-liner by changing the various variables, which can be shared with the rest of your development/troubleshooting team and readily ready to copy from an internal Wiki:

Feel free to use and modify the script as you see fit. 🚀

Wednesday, June 12, 2019

Azure App Service Deployment Error: There is not enough space on the disk.

I recently came across an error during a deployment to one of our Content Delivery App Service instances - stopping the deployment process dead in its tracks. 

The error occurred when deploying to the CD Slot:


This occurred each time we attempted to deploy this step.

The error in full does indicate troubleshooting codes and links (oddly enough, the Microsoft link did not have any trace of the error code):
Failed to deploy web package to App Service. Error Code: ERROR_NOT_ENOUGH_DISK_SPACE More Information: Web Deploy detected insufficient space on disk. Learn more at: http://go.microsoft.com/fwlink/?LinkId=221672#ERROR_NOT_ENOUGH_DISK_SPACE. Error: The error code was 0x80070070.

I found it especially strange given that we saw no indication that we've hit any limit on storage space in Azure Portal.  Initially, I thought it may have something to do with storage on the build server, but was able to quickly rule out this theory by verifying that there was plenty of disk space remaining on that machine.

The error was also evident when I attempted to upload any file directly via FTP:

I began a look for some giant file(s) that may be preventing any additional files from being uploaded (since that's what made the most sense given the context of the error message itself).

By logging into the App Service via FTP, I was able to identify two IIS memory dumps located in the /LogFiles directory.  These both appear to have been taken the month prior - and simply never removed. 

After deleting both memory dump directories, I was able to restart the deployment step - which completed without errors. Direct FTP uploads were also restored.

What's up with that?

Well, your Azure App Services is tied to a particular pricing tier that dictates storage, memory, ACU, etc.  In our case, this particular App Services is configured to use the Standard standard tier - which has a 50GB limit.




By leaving the remnants of these IIS memory dumps on the App Service's storage, we must have surpassed that storage limit.

This brings up some interesting questions:

First, based on the configured pricing tier, is there a limit on how much Sitecore can store in App_Data/MediaCache or /temp folders before hitting that cap?

I assume the answer to that is yes - if your application is not setup to periodically clean stale files (which it should by default), it's possible to reach that storage limit and cause this error to surface.  In that case, the quick fix to get your deployment out would be to remove some or all temp files in the application.

Second, how can we monitor this storage limit of App Services in Azure Portal?

I'll have to circle back on this one as I don't quite have the answer to it.  It may even already exist, and I just haven't spotted it yet.   I'll update this post if I do figure that one out, but please let me know if you have the answer in the comments!


Monday, April 29, 2019

Sitecore Azure Kudu Tools PowerShell Module



I've managed to start several blog post drafts with the intention of sharing a few of my PowerShell scripts - but haven't got around to finishing/posting any of them.

This actually led to an epiphany: the scripts I wanted to share all had an underlying theme: obtaining files using the Kudu Rest API using PowerShell.


Huh? Kudu?

Aw, a baby Kudu!

If you don't already know, Kudu is the "engine behind git deployments" in Azure App Service - but can also be used as a built-in diagnostics tool within Azure.   Any time you have an Azure website, you automatically get a 'companion' Kudu (aka SCM) site accessible via the following URL format:  https://{ResourceName}.scm.azurewebsites.net.  

For example, if your CM App Service name is
'mysitecoresite-xp2-small-prd1-cm',

the corresponding Kudu site would be:
https://mysitecoresite-xp2-small-prd1-cm.scm.azurewebsites.net/ 

If you've ever had to Rebuild the xDB index in Azure Search, Sitecore's documentation walks you through how to do so using the Kudu Debug console.

Kudu REST API

One of my favorite feature of Kudu is the Kudu REST API since any files you can access via FTP are also accessible via the Kudu REST API.   You can download files, upload new ones, etc.

Typically, all you need are:
  1. Subscription ID
  2. Resource Group Name
  3. Resource Name
In Azure Portal - you can copy these values from a resource in the overview panel:


Those values are the key to interact with the API using PowerShell's Invoke-RestMethod.
There are a couple prerequisites:
  1. Azure RM module installed and ConnectAccount has been executed.
  2. Valid Azure credentials (same ones used to log into Azure Portal) to invoke Login-AzureRmAccount (converted to Base64)



Sitecore Azure Kudu Tools

I'd never written a PowerShell Module, but I figured this would be a great segway to learning how. I got to reading -- and tinkering -- and refactoring.  

Before long I had written my first PowerShell Module: Sitecore Azure Kudu Tools.

Sitecore Azure Kudu Tools is a collection of functions (three at the time of this post) built to help you get information/files from Sitecore instances on hosted on Azure PaaS using the Kudu Rest API.

It's available on the PowerShell Gallery or check out the GitHub repository.


Get-SitecoreSupportPackage

Allows you to remotely generate a Sitecore Support Package.  The function will download files defined for Sitecore Support Packages into a specified path, obfuscate sensitive data from ConnectionStrings.config, and compress the contents:

\App_Config\* Global.asax
\Logs\* license.xml
eventlog.xml sitecore.version.xml
Web.config


Why?

While there's a built-in way to obtain Sitecore Support Packages in Sitecore's admin page, being able to remotely obtain this information is a nice alternative.  This is useful not only for providing the required information for Sitecore Support tickets - but also for your own diagnostics.

Usage

Invoke this function using the following syntax:


Output


Alternatively, if you just run Get-SitecoreSupportPackage without any parameters, you'll be prompted to provide each required parameter.



Invoke-SitecoreThumbprintValidation

Provides a way to verify that certificate thumbprints match across Sitecore Azure PaaS Resource Group.  The function will download ConnectionStrings.config and AppSettings.config files from all App Services in a given Resource Group, then display any certificate thumbprints discrepancies.

Why?

If you've ever had to replace thumbprint values across an Azure PaaS Sitecore topology, you know that it can be a bit of a pain identifying all the configs where the old thumbprint needs to be replaced.

Bram Stoop wrote a great post identifying all the places you'll need to modify those thumbprints.  If you're looking to semi-automate this process, you can utilize the Kudu REST API to get all the configurations, then identify any mismatched thumbprint values using PowerShell.

Usage

Invoke this function using the following syntax:


Output




Get-SitecoreFileBackup

Allows you to a full copy of an App Service's website file contents.  This function will download all files from in a given ResourceName.

The following folders are excluded:
_DEV sitecore modules
App_Data sitecore_files
App_Browsers temp
sitecore upload
xsl

Why?

Sometimes I just want everything. If I need to determine discrepancies between environments for any reason, this helps.

Usage

Invoke this function using the following syntax:

Output




What's Next?

Now that the module is established, I've created a few GitHub Issues which I hope to get through quickly.  I also plan to add more functions in the future and am 100% open to contributions. 

Is it perfect? Far from it! It's a work in progress. Thrilled to share anyway.

If you're interested in contributing, please check out the contribute section on the Github repo.

via GIPHY

😊