Showing posts with label report. Show all posts
Showing posts with label report. Show all posts

Tuesday, July 15, 2025

SPExAI GPT: SPE Script Generation, No Module Required

Earlier this year, I built SPExAI Report Builder during the 2025 Sitecore Hackathon, a natural language interface for generating Sitecore PowerShell Extensions (SPE) reports. The concept was simple: use plain English to get real, working scripts. The result? A winning module that turned heads and saved hours. 🙌

But what if your team loves the idea but doesn’t have the time, access, or appetite to install a custom module? Perhaps you just need a script now and then. Or maybe you're working in an environment with limited Sitecore customization privileges.

It would be great if you didn't need to rely on installing a module into your Sitecore instance to benefit from this technology. 


✨ Introducing SPExAI GPT

Introducing a new flavor of SPExAI; not as a Sitecore module, but as a custom GPT. It works entirely outside of Sitecore. No installs. No packages. No patch configs. Just a prompt box and your imagination.

Your prompt:

"Report of all templates (ID, Name, Path) and their usage count."

And SPExAI (the GPT version) responds with a fully working script.  Same quality, same intelligence, now universally accessible.



Simply copy the output, paste it into the Sitecore PowerShell ISE, and run it:


Results!



One benefit you get with the SPExAI GPT over the SPExAI module is that you can continue your chat, which isn't available in v1 of the SPExAI (it's all one-shot), allowing you to prompt beyond your initial prompt.  This is great for getting help with fixing errors you may encounter when using the initially generated script, or even for enhancing an existing script.

🧠 Under the Hood

The GPT version builds on the same core ideas as the Hackathon module:

  • 🔍 A system prompt trained to interpret natural-language Sitecore requests and produce accurate PowerShell reports.

  • 📚 A curated knowledgebase of common SPE scripting patterns tailored to tasks like item audits, rendering usage, field value searches, and more.

  • ⚙️ Designed to speak "Sitecore" fluently, not just ChatGPT syntax, but the nuances of $SitecoreContext, $item.Paths.FullPath, Get-Item, and all the other familiar constructs.

I've adapted the system prompt to better align with ChatGPT’s structure, ensuring cleaner completions, improved formatting, and fewer hallucinations.


🚀 Try It Yourself

If you're looking to generate SPE scripts without touching your Sitecore instance, this GPT-based version might be the way to go. Whether you're prototyping, documenting, or training a team, it removes friction and delivers fast results.

You can find the SPExAI GPT on the Chat GPT store by selecting GPTs in the left-hand panel of ChatGPT, then searching for `spexai`:



You can also access SPExAI GPT directly using this link:

If you use it, please take a moment to leave a rating. 
If you have suggestions for improving the GPT, I'm open to them! 😀

Tuesday, April 22, 2025

SPExAI Report Builder: A Winning Sitecore Hackathon Module

The results of the 2025 Sitecore Hackathon are in...

Look what I got! 😭👇

It's a really decent piece of hardware!

I'm super proud of this accomplishment, given that I was pushing into delirium territory near the end of the event, having been up for 32 hours straight and all...running on mostly caffeine and adrenaline to get it done. 

Here's a snapshot of my real-time X updates throughout the event:



Sitecore Hackathon?

It's a virtual community-driven event where teams worldwide (52 teams across 13 countries this year) compete to build the most impactful Sitecore module given a set of categories (e.g., "Best use of AI" or "Best tool for XM Cloud") within a strict 24-hour timeframe.

Typically, the event is held in late February / early March, with teams registering roughly 4-6 weeks in advance. Submissions are then judged by a panel of long-time Sitecore MVPs and community members who review each completed entry, test the functionality, and collectively pick a winner.  

Submission Requirements are clearly laid out in the GitHub repository to which each team is assigned:

Winner Benefits

  • 🗣 Name recognition across Sitecore's official channels; the winning team is announced officially at SUGCON Europe and highlighted in the MVP community.
  • 🛒 $150 Amazon Gift Card (per team member)
  • 🏆 A customized Hackathon trophy

My Past Hackathons

Wasn't my first rodeo! 🤠
Here's a rundown of my past Sitecore Hackathon participations:

The name "Sitecorepunk 2077" is a not-so-subtle reference to
the 2020 video game Cyberpunk 2077 (which, admittedly,
I've barely played, but I liked the play-on-words when I came up with it).

I kinda love that I'm a 2x Sitecore Hackathon winner now 😅


The Idea

This year, instead of multiple categories (and multiple winners), the organizer's idea prompt was simply:

"Free for all — you can create your own idea for the Hackathon solution. Show us what you got!" 

It was a huge opportunity to build without barriers, with one winning team to take it all. I've been waiting for an opportunity to bring this idea I've been mulling over in my head for months to life, and this was it. 

Problem Statement

The biggest hurdle for analyzing content and creating reports in Sitecore PowerShell Extensions (SPE) has always been the technical skill needed (PowerShell scripting/syntax + SPE-specific commandlets).  

You'd need to train up; learn how to query items using Get-Item and Get-ChildItem commands, declare an array object to store results, utilize for loops and if conditions, etc.  

And if PowerShell scripting isn't your thing, well...


In the age of generative AI though, this technical skills barrier can be dramatically lifted for non-technical Sitecore authors and admin, and/or drastically reduce the turnaround time for developers tasked with writing custom PowerShell reports.



Enter: SPExAI Report Builder

What is it?

SPExAI Report Builder is an installable Sitecore PowerShell Extensions module that allows users to describe their Sitecore report in natural language, which in turn generates a complete and reusable SPE script:

  • 🧠💻 Type your prompt
  • 📜💾 Generate a PowerShell script and save it
  • 🛠️🚀 Run it or modify it

"SPExAI" stands for Sitecore PowerShell Extension x Artificial Intelligence, which combines the power of SPE with modern LLM tech. 

Compatibility

SPExAI Report Builder works with Sitecore 10.x or later.  I tested on Sitecore 10.0, 10.3, and 10.4 during the event, but I'm pretty confident that it would also work with other versions, too.

I didn't get a chance to test this on XM Cloud, but given SPE's flexibility, it is likely compatible.


How about a quick demo?

Say you need to audit template usage across the content tree.

When activating SPExAI from the ribbon, a dialog appears where you set a title, select the root context for the report, and provide a description.

"Report of all templates (ID, Name, Path) and their usage count."



SPExAI generates this clean, complete, and reusable script, which is stored in a dedicated part of the content tree:

Running the generated script without any modifications (which, on its own, included an option to select a root context, making it easily reusable against different parts of the tree) provides an accurate result set!



Another demo!

"Report of all renderings (ID, Name, Path) and their usage count under a selected content root."

SPExAI again generates a clean, complete, and reusable script, stored again using the name provided:

Running the generated script confirms that the script has been correctly generated and provides expected results.  



What's truly amazing is that we can generate 75-100 lines of working PowerShell code in seconds.

The code is appropriately structured, cohesive, error-free, and ready to be run immediately—no developer needed!



How SPExAI Works (Under the Hood)

API Settings

Before anything runs, the module looks for a specific Sitecore item:
/sitecore/system/Modules/PowerShell/Script Library/SPExAI Report Generator/API Settings:

There are four required fields:
  • API Keyyour OpenAI secret key

  • Model – the ID of the OpenAI model to use (e.g. o3-mini-2025-01-31)

  • Knowledgebase – a markdown-formatted reference block full of Sitecore PowerShell examples, documentation, best practices, etc.  

  • System Prompt – the instruction template that tells the model exactly how to behave, respond, etc. 


The module will abort early if any of these are missing.

Model Selection

During development, I tried a few different OpenAI models. The one that gave me the most consistent, one-shot responses was o3-mini-2025-01-31.

If you want to try a different OpenAI model later, simply update the Model field with the name; no code changes are required. (Expanding beyond OpenAI to Anthropic Claude or Google Gemini is also possible as part of a potential future v2.)

The Knowledgebase

This field contains raw reference material to guide the AI's responses. Think of it as an internal code cookbook, mostly pulled from the official SPE documentation and a compiled generic collection of snippets from my private repository of PowerShell scripts.

It includes sample report formats, SPE-specific syntax, and usage patterns that the model should stick to when replicating and generating new reports.

Looks like this:

The System Prompt

This is the master instruction set. Essentially, "You are a Sitecore PowerShell assistant...you do this, this, and that..." with additional specific constraints and formatting rules.

It includes a {0} token that the Knowledgebase content replaces.

Check it out:


A good chunk of the hackathon effort was spent refining the directives that the model should abide by. With every test run, I found myself adding to the list of rules. 

Getting the model to stick to the directives was...challenging to say the least (one-shot prompting definitely has its limitations depending on the model).  

Finding the right combination of rules for the model to consider was tricky, and I'm sure both the base system prompt and the knowledge base content could use even further refinement beyond what I could get done before the deadline.  Either way, I feel like I struck a solid balance for v1.

The good news is that the module was built to easily modify the system prompt in the configuration item without touching the underlying code, hypothetically allowing you to continuously improve the final output. 

UX Flow

SPExAI provides a new button in the Sitecore Ribbon. When clicked, this button surfaces a dialog window for the user's input.  



Users fill out the Report Name, set the Report Scope (tree selector), and the Describe your Report fields.  

It takes only a few seconds for the script to be generated.  

Users are then presented with the following options:



SPExAI Code Breakdown

Here's how the pieces come together behind the scenes:

1. Load the API Settings

The script set the four field values into variables.


2. Present a dialog for user input

The user's inputs from the dialog (report name, scope, description) are stored as global variables.


3. Variable validation

Validate that variables, like the script name, are valid and don't already exist in the saved script location.

4. Invoke the custom `Invoke-OpenAIChat` function

Invoke-OpenAIChat sends a custom one-shot prompt (including merging system instructions, knowledgebase, and user input) to OpenAI’s Chat Completion API and returns the generated response.

5. Save the script to the tree

Upon successful script generation, the module saves all generated scripts under a dedicated folder:  /sitecore/system/Modules/PowerShell/Script Library/SPExAI Report Generator/Content Reports/Reports/SPExAI Generated

6. Open, Run, or Close Dialog
After saving the script item, the module presents a modal dialog with its three choices:

1. Open Script Item – jumps to the new item in the Content Editor
2. Run Report – immediately executes the report using Invoke-Script
3. Close – exits with no action



Video Demo

As part of the entry, a video demo is required. You can check it out here:



Some Final Thoughts

If you haven’t seen it, fellow long-time MVP Rodrigo Peplau compiled a list of this year's submissions - all worth checking out. 

The quality of entries this year made it hard to predict how things would shake out. Winning was unexpected, but also an absolute honor.

Bummed I couldn't be at SUGCON EU to accept the award in person, but luckily the announcement was recorded. 😀  I will cherish this screenshot for all time:


Each year I've participated (whether on a team or solo), I've come away with valuable hackathon experience and a solid module, or at least the beginnings of one, that I could share and expand on further. I've always enjoyed the satisfaction of shipping something interesting and useful to others under competitive pressure. 

If you’re considering participating next year, I highly encourage it.  About 20% of the teams are solo, but it's not for everyone.  I recommend grouping up with others, especially if it's your first Hackathon. 

It’s a great way to push yourself, learn something new, make connections, and contribute to the spirit of the Sitecore community.

Keep on hackin'! 👨‍💻

Friday, September 8, 2023

Building Better Reports: 5 Sitecore PowerShell Extensions Functions for Your Toolbelt


When navigating the Sitecore ecosystem effectively, PowerShell extensions aren't just a helpful tool; they're practically a prerequisite. As someone who has spent a substantial amount of time in this space, I have distilled a set of functions that I've found myself using time and time again. 

Join me as I unpack a curated list of five functions that have become staples in my Sitecore toolkit.

Get-IsPublished

The "Get-IsPublished" function checks if a specific item, identified by its ID, is published on a "web" database. It takes an item as input and queries for it against the "web" database using its ID. If it finds the item in the database, it returns "TRUE," indicating that it is published. If it doesn't find the item (i.e., if the item is null), it returns "FALSE", indicating that it is not published.

This is particularly useful for displaying a column in your SPE report to denote if the item is published to the web database:


Get-ItemSitecoreCELink

The "Get-ItemSitecoreLink" function generates a URL to open a specific item in the Sitecore Content Editor. It takes a Sitecore item as input and uses various properties of that item (like its ID, version number, and language) to construct a URL. This URL, when accessed, will open the Sitecore Content Editor with that specific item loaded, allowing for easy navigation directly to the editing interface for that item. 

Please note that the base of the URL ("https://yourcmurl.com/") in this function is a placeholder that you'd replace with your actual Sitecore Content Management URL.

This is useful if the report is exported to Excel or CSV as it provides the direct link to reach the item without manually traversing the tree or searching for the item by GUID.


Get-LinkFieldUrl

The "Get-LinkFieldUrl" function retrieves the URL from a Sitecore item's link field. It takes a Sitecore item as its input and utilizes a regular expression to extract and return the URL stored in a "My Link Field" field. The regular expression is designed to find and capture the URL stored as a value in a link HTML element's URL attribute.  

If the "My Link Field" is not empty and matches the pattern specified by the regular expression, the URL is retrieved and returned. If the field is empty or doesn't match the pattern, the function returns nothing, essentially returning a null value.

This function couples well for reports if you need to extract the URL out of a Sitecore Link Field value to display in your report:

Assert-HasLayout

The "Assert-HasLayout" function checks if a given Sitecore item has a "final layout" defined. It accepts a Sitecore item as its input and uses the built-in "Get-Layout" function to retrieve the final layout details of the item.  If the item has a final layout (meaning the $layout variable is not null or empty), the function returns a "TRUE" string, indicating that a layout is present.

If no layout is found for the item (meaning the $layout variable is null or empty), it returns a "FALSE" string.

Usage example


Get-FormattedDate

The "Get-FormattedDate" function takes a raw date string as its input and attempts to turn it into a more user-friendly date format.

The raw date string is expected to follow a particular "yyyyMMddTHHmmssZ" pattern ("20230908T123456Z" representing September 8, 2023, 12:34:56 PM in Coordinated Universal Time, for example) - which is precisely how Sitecore typically stored DateTime fields in the database.

The function reads this string and converts it to a date format that is more commonly used, which includes the month, day, and year (like "09/08/2023"). If, for any reason, it can't convert the input into a date (maybe because the input doesn't follow the expected pattern), it will simply return an empty string. This way, even if it receives unexpected inputs, it won't crash and will still produce a result, even if that result is just an empty string.

By passing in the raw DateTime field value into the Get-FormattedDate function, the report will convert it to a readable string value when exporting the report to Excel, CSV, or JSON.




Whether you're a seasoned Sitecore developer or just starting out, these 5 functions can become essential tools in your developer toolkit, helping you navigate the complexities of Sitecore with greater ease and efficiency ✌.

Tuesday, February 28, 2023

Sitecore xDB Contact Lookup Utility

My client's lead flow includes capturing form data into a custom database. Each record contains an xDB Contact ID to correlate the data with the data captured in xDB directly.  We had a scenario where we had hundreds of records in the custom database where the xDB contact ID was missing, likely due to a networking issue between the CD and xConnect.

I needed a way to match users captured in the custom database against the data in xDB to obtain an existing xDB contact ID.  Luckily, Sitecore's documentation provides some sample code to help get me started: xConnect Client API (C#) (sitecore.com)

Thus was born, the...

🧰 xDB Contact Lookup Utility


The utility is your standard .aspx file that can be dropped into the/sitecore/admin folder or at the root of CM instance. It combines a series of xConnect API calls to search for contact information in the xDB.  Applicable parameters include email address, first name, last name, or xDB contact ID.

It can be used in two ways:

First, through the UI (as shown above), you have the ability to populate text boxes to obtain parameters and view the result list on the page.

Secondly as a simple rest endpoint, with the ability to pass parameters into the URL and return a JSON response containing a matching xDB contact ID.

For example: 
/xDB-Contact-Lookup.aspx?email=test@test.com&firstName=test&lastName=test




👨‍💻 Code Breakdown

Page_Load

The Page_Load event checks if a query parameter named "email," "lastName," or "firstName" are present in the URL. 

If at least the "email" parameter is present, the method calls the SearchForContactIdByEmail .If the lastName and firstName parameters accompany the email parameter, the SearchForContactIdByName method is also used to search for a contact. The search results are returned as a JSON response which contains an xDB contact ID of the first match or "NoMatch" if no match was found. The JSON string is written to the HTTP response, ending the response. If the "email" query parameter does not exist, the method does nothing and loads the UI.

btnLookup_Click

This button click handler retrieves user input from four text boxes for known contact ID, email, last name, and first name. If the user input is for a known contact ID, it calls the SearchForContactById method to retrieve the contact's information and displays it in an HTML table. 

If there is no matching contact, it displays a message saying that no contact was found. 

If the user input is for email, last name, and/or first name, it calls SearchForContactIdByEmail and/or SearchForContactIdByName methods to retrieve contact information and displays it in an HTML table. 

If there are no matching contacts, it displays a message saying that no contact was found.

There are three methods implemented for using the xConnect API:

SearchForContactById

This method searches for a Contact object in xDB using a known contact ID. The method SearchForContactById takes a string parameter knownContactId and returns a Contact object. The method first converts the knownContactId string to a Guid, creates an instance of Sitecore.XConnect.Client.XConnectClient, and uses the Get method of the client object to retrieve a Contact object using a Sitecore.XConnect.ContactReference object created from the Guid

It also uses a Sitecore.XConnect.ContactExpandOptions object to specify which facets of the Contact object to retrieve. 

If the Get method throws an XdbExecutionException, the method catches the exception and returns null.

SearchForContactIdByName

This method searches for a list of Contact objects by their LastName and FirstName facets of PersonalInformation

SearchForContactIdByName takes two string parameters, lastName and firstName, and returns a list of Contact objects. 

Inside the Sitecore.XConnect.Client.XConnectClient using block, the method creates an IAsyncQueryable object named queryableLastName that represents a xConnect API query to xDB. The query retrieves all contacts where the LastName and FirstName facets of PersonalInformation match the values of the lastName and firstName parameters. 

The query then sorts the results in descending order of LastModified. The WithExpandOptions method is used to specify which facets of the Contact object to retrieve. The method then retrieves the query results using a batch enumerator, which is added to a List<Contact> object named idsList. 

If the query throws an XdbExecutionException, the method catches the exception and returns null.

SearchForContactIdByEmail

This method searches for a list of Contact objects by their EmailAddress and optionally their LastName and FirstName facets of PersonalInformation

SearchForContactIdByEmail takes three string parameters, email, lastName, and firstName, and returns a list of Contact objects. 

Inside the using block, the method creates an IAsyncQueryable object named queryable that represents a xConnect API query to the xDB. The query retrieves all contacts where the PreferredEmail facet of EmailAddressList matches the email parameter value. 

The query then sorts the results in descending order of LastModified. The WithExpandOptions method is used to specify which facets of the Contact object to retrieve. The method then retrieves the query results using a batch enumerator, which is added to a List<Contact> object named idsList

If the firstName and lastName parameters are not empty, the method checks if the FirstName facet of PersonalInformation and LastName facet of PersonalInformation match the values of the firstName and lastName parameters respectively for each Contact object retrieved. 

If both match, the Contact object is added to idsList. If the firstName and lastName parameters are empty, all Contact objects retrieved are added to the idsList

If the query throws an XdbExecutionException, the method catches the exception and returns null.

Final Result

The full code can be copied here. Add the content to a .aspx file and place it where you need it.


As always, feel free to modify or build on top of this to satisfy your own requirements.

🚀


Friday, July 31, 2020

Generate Google Lighthouse Reports with Docker using PowerShell



While browsing Docker Hub, I came across this nifty Google Lighthouse Docker image (by Jay Moulin) which allows you to execute a Lighthouse audit against a given URL in a containerized application - made possible by the Google Chrome Headless Docker base image.  From a practical standpoint, this feels more reliable than running Lighthouse in the Chrome browser where extensions and other variables can easily interfere with the results of the audit. 

You can check out the Dockerfile for this image here: 

Consuming it is pretty straightforward.  With Docker installed and running while switched to Linux containers, two commands are all you need:


Additional options for the Lighthouse audit, like controlling the emulated device form factor (mobile vs. desktop), controlling the throttling method (devtools, provided, simulate), or defining specific categories (Accessibility, Best Practices, Performance, PWA, or SEO) can be included after the URL.
 
However, that's quite a bit of text to remember, and memorizing a bunch of Lighthouse CLI options are not something I see myself doing.  😋 

BUT - we can make this tool more approachable by wrapping it in a PowerShell script. 😍

The name of the game is simplicity: execute .\LighthouseReport.ps1 from a PowerShell terminal, pass in a URL/standard Lighthouse options, and let it run. 


👨‍💻 A Little PowerShell

In a new PowerShell file, we'll add a mandatory string parameter called $Url
We'll also include non-mandatory string parameters:
  • $FormFactor
    • Valid options for the '--emulated-form-factor=' flag are 'none', 'desktop', or 'mobile'. 

    • Default value when no parameter is provided will be 'desktop'

  • $Throttling
    • Valid options for the '--throttling-method=' flag are 'devtools', 'provided', or 'simulate'. 

    • Default value when no parameter is provided will be 'provided.'

  • $Categories (array of strings)
    • Valid options for the '--only-categories=' flag are 'accessibility', 'best-practices', 'performance', 'pwa', 'seo'. 

    • Default value when no parameter is provided will a comma-delimited string of all applicable categories 'accessibility,best-practices,performance,pwa,seo'

  • $DestinationPath
    • The local path to where the report will be 'dropped.' (used as a volume mapping to the container's '/home/chrome/reports' directory)

    • Default value when no parameter is provided will be "C:/lighthouse"

We'll add the docker pull command for femtopixel/google-lighthouse first.  During the initial execution of the script,  all required images will be downloaded from Docker Hub.  If your image becomes stale or a newer version is available, this will automatically update the image. 

Then add the docker run command with the -v flag to mount a volume to map the local $DestinationPath to the /home/chrome/reports directory on the container. Include the $URL parameter at the end, and all options following:


When the docker run command is executed, Docker will take over, and Lighthouse will begin to execute on the container. Once completed, a .html file will be available in the $DestinationPath

To take it a step further, we can open the $DestinationPath in Windows Explorer by using an Invoke-Item command:


If we want to open the .html report, we can set the PowerShell location to the $DestinationPath, followed by an Invoke-Item where we pass in Get-ChildItem latest .html file.


Simple - yet effective!

🏁 Final Script


⌨ Example Usage

Desktop form factor auditing all categories:


Desktop form factor auditing Best Practices, Performance and SEO only:


 
Mobile form factor auditing Performance only:


 
  

💡 TIP: When setting a parameter (-FormFactor, -Throttling, -Categories), you can use Ctrl+Space to display valid options and hit enter to select it.



👍 Result



🙌 Feel free to grab a copy and modify it to your liking.

Thursday, September 5, 2019

Azure Application Insights: Logs & Requests Viewer using Sitecore PowerShell Extensions

Last September, I wrote about accessing Sitecore Logs from Azure PaaS instances using the (now deprecated) AzureAILogs.html file provided by Sitecore. The knowledgebase article was updated in mid-January , 2019 – and the AzureAILogs.html file had been replaced with a new /sitecore/admin page dubbed AzureTools.aspx.

This updated admin page contains all the same functionality found in the AzureAILogs.html, with the addition of being able to pull log traces and requests from Application Insights.



Installation is easy: download the AzureTools.zip files, drop in the /sitecore/admin/AzureAILogs.aspx into the you’re your site’s root.

Admittedly, this admin page is great - but I could also see several aspects SPE being particularly useful (like the OOB SPE ListView - which would easily allow us to filter/sort/search through a series of log entries). An additional option to see raw color-coded logs would also be cool. 😊

Using the existing AzureTools.aspx as a general guide, we can re-create the GUI with general ease.

We’ll need:
1) Option to get Requests or Logs
2) Option to selected a Role (values pulled from API)
3) Option to control recency.
4) Option to control the severity.


The end result will consume the Application Insights REST API endpoints and allow a user to pull logs from Application Insights inside the CMS.


API Access

To start, we'll need to make sure we can work with the API by obtaining an Application Insights App ID and a corresponding App Insights API key  Sitecore's documentation already lists the.

Sitecore's documentation covers this but it's as simple as logging into Azure Portal and navigating to your Application Insights service. 

Under Configure, select 'API Access':


The Application Insights App ID will be displayed the following screen:

Copy this value and store it temporarily.

Click the 'Create API Key' button.
Give it a name and check the 'Read telemetry' checkbox.

After clicking 'Generate key', you'll have one opportunity to copy the 'App Insights API key'.  Copy and store this value temporarily.


Initial Communication with the API

Our script will utilize the two values to interact with the API.

Before building our UI in SPE, we'll need to confirm API communication by obtaining the server roles from Application Insights.  We can set a variable to call a function that will grab an ArrayList of roles:

Our function will build the URL, include the property URL authorization header containing the API key, and return an array list.



User Interface

Now that we have confirmed communication to the API and obtained our list of roles, we can pass the variable into a new function that will be responsible for building and displaying the UI:

The dialog should contain a series of radio buttons and checkbox lists, all of which will be used to provide options to build out another API call to obtain the traces or requests from AppInsights.

The output displays as follows:

Notice line 51 in the above snippet calls a Get-LogsOrRequests function which accepts a series of parameters from the dialog options upon selecting the OK button.

This function builds out the proper API URL and query parameters based on the passed the selected values passed in. Invoke-WebRequest is used to make the call to the API, which will return a JSON object of log entries from AppInsights based on those parameters.

Line 119 contains a final call to a function called 'Set-PostDialog' which provides options for displaying the results.

The output here is a ModalDialog with three buttons:


Selecting 'Script View' will display the results of the API in a color-coded Show-Result window:


Selecting 'List View' will process the results to an acceptable format for a standard SPE ListView result window (filtering, exporting, etc is obviously all included here):


Finally, selecting the 'Download' button will download a .txt file of the contents retrieved from the API.


Final Script





Installation

Manual

  1. Create a new Sitecore item based on the SPE PowerShell Script template and copy the final script above into the Script Body field.
  2. Replace the default "XXXXXXXXXXXXXXXXXXXXXXXXX" placeholder values in the $aiAppID and $apiKey variables with your own.  

Sitecore Package

  1. Download the Sitecore package and install from GitHub.
  2. Navigate to the PowerShell script located here:
    /sitecore/system/Modules/PowerShell/Script Library/Azure Application Insights Logs/Toolbox/Azure Application Insights Logs
  3. Replace the default "XXXXXXXXXXXXXXXXXXXXXXXXX" placeholder values in the $aiAppID and $apiKey variables with your own.  
The script will be available to run from the PowerShell Toolbox in the Start Menu.


Source Code

The full script can also be found on GitHub.
Feel free to grab a copy and modify it how you see fit.  



Friday, August 17, 2018

Basic Sitecore Audit Trail with Powershell


Update: This report has been included in the latest version SPE v5.0: 
https://github.com/SitecorePowerShell/Console/issues/1033


Here's a question posted to our internal Sitecore Slack channel yesterday:

The answer was 'no'.

This isn't the first time our clients have requested 'Audit Trail' functionality before, either. And we're talking about the basics here:

  • • When did a user log in?
  • • When did a user log out?
  • • Who published what, and when?
  • • When was this item's workflow executed and by whom?

Unfortunately, there isn't an 'out of the box' solution to easily obtain this data.
The most common "solution" has always been to utilize the Sitecore Log Analyzer's Audit tab - which is great for developers, but not for CMS users.

This tool has saved my sanity so many times. 

Others in the community have shared promising solutions in the past, most of which have become unsupported over time. The Sitecore Audit Trail Marketplace module, for example, was last updated in 2015 and only supports up to version 7.5 (fun fact: mainstream support for this version ended in December 2017!).  Additionally, the setup was heavy - requiring a custom database connection. It also, unfortunately,  had it some known issues associated with it.
While the feature set was fairly extensive, a simpler solution still wasn't available.

The Advanced System Reporter also has an audit feature, but again the last time this module was updated was in 2015 and supports up to version 8.0.

And then it hit me.
POWERSHELL!!!!

You've got to know this by now...my default reaction to any problem is to ask myself: "can this be done using Powershell?".

The idea was to create a Powershell report that consumes the accessible Sitecore log files which already contain the data we need.  The report should include all lines marked as 'AUDIT'' and split into columns.  The user should be able to provide a date range to narrow down the audit.  

Powershell would automatically provide the rest: keyword filtering, sorting, exporting, etc. 

Let's script it.

In order to allow a user to configure a date range, we need to create an interactive dialog

The result:



If the user clicks cancel, we'll want to abort the whole operation.
Otherwise, we'll proceed and create the properties needed for our final ListView.
We'll call a function named Get-Audit where our logic will process.


Within the Get-Audit function, we first obtain the location of the Sitecore log folder (UPDATE: We'll use $SitecoreLogFolder instead. SPE creates a variable $SitecoreLogFolder and resolves the path for you: https://doc.sitecorepowershell.com/working-with-items/variables).  We'll the pull the log files from the resolved log path and filter out everything but the standard log. files:

We'll make sure the user has selected a date range and filter the list of files using those inputs. If a user didn't set a date range, we'll simply use the original file set - which will display the most recent entries. While we're at it, we'll define a 'regex' string used to filter only AUDIT items, and an array object to hold our line objects:


We'll start a loop based on our file count. From there, we can get the contents of the file.


We need a way to include the date for each line object (specifically for our final sorting), so we'll build out a simple date string which we'll use later.


We'll now loop through each line in the file and check if it's marked as 'AUDIT'. If the condition matches, we'll append the simple date string we created previously to the beginning of the line string and sanitize the string by removing double-spaces. In some cases, the audit line will contain ManagedPoolThread #XX instead of an ID. We'll sanitize this as well.

The line now looks like this:
8/17/2018 8324 00:13:47 INFO AUDIT (sitecore\Anonymous): Logout

We'll use the space between each data point to our advantage and split each property into individual objects. The username requires some general sanitation, but more importantly, we'll build out one more DateTime object which we'll use to sort the lines before returning our array. 


Finally, we sort and return our array for the table to process.


Final Script

Putting it all together, our script looks like this:


When the script runs, we get a pretty clear view of who's been doing what:

As always, feel free to use, modify, and build on it as you see fit. This has only been tested on a handful of environments so far, so bugs are still possible.
If you do spot any issues, feel free to report them in the comments - or make the necessary changes and submit a pull request to https://github.com/strezag/sitecore-audit-trail-powershell.

Happy SitecorePowershelling!

Wednesday, January 3, 2018

Sitecore Powershell: Valid Page URLs Report

Preface: I love the Sitecore Powershell Extensions Module, and I opt to use it every chance I get.


One of my clients had a simple request:
"Please provide a list in Excel of every single valid URL on the live global site, please." 

After running ScreamingFrog and obtaining a report with missing URLs (the final list returned was faulty – likely due to the software’s inability to hit specific links only available via AJAX rendered components) - we had a couple options on the table:

  1. Similar to functionality found in a Sitemap component, we need a simple ASPX page that loops through the content that filters out everything but the global English version, generate the URLs, trigger a web request to determine the URL's web status, and display it on the page (or create a Download button to get the list).  Code this, deploy it, etc.
  2. Do all of the above - but with Powershell - which happened to already be installed on the CMS

GUESS which I opted for? :)
Yeah!...you guessed it!

Let's get right into it.

Using the Get-ChildItem command and targetting a specific part of the content tree (explicitly using the Web DB), we get the initial list of English versioned items.
 $itemsWithMatchingCondition = Get-ChildItem 
          -Path web:'/sitecore/content/WebsiteName/Home' 
          -Language 'en' 
          -Version * 
          -Recurse 

With this specific implementation, I was lucky enough to have a stable template naming convention where all items using a template that ended with "Page" were always going to be...well...pages.
(Without this luck, I may have had to check if the item contained at least a main layout within the renderings).

To filter this, we'll use a simple IF statement with a LIKE operator against the initial item list's item:
 iif ($item.Template.Name -like $script:pageString)

Now that we have a list of page items we want to process, we need to generate the item's URL.

This handy function that sets the site context, configures the UrlOptions, and gets the URL via the LinkManager does just that:
function Get-ItemUrl($itemToProcess){
     [Sitecore.Context]::SetActiveSite("website")
     $urlop = New-Object ([Sitecore.Links.UrlOptions]::DefaultOptions)
     $urlop.AddAspxExtension = $false
     $urlop.AlwaysIncludeServerUrl = $true
     $linkUrl = [Sitecore.Links.LinkManager]::GetItemUrl($itemToProcess,$urlop)
     $linkUrl
}

Here's the fun part!

Per the requirement, we'll need to validate that the URLs Sitecore was generating were actually functioning.  Any non-functioning URLs (if any) shouldn't be included in the final report (only status code 200).

Powershell lets us make web requests - which we could then check the status of.
All we need to do here is pass in the URL we generated and expect a true or false value in return:

function IsValidPageStatus($urStr){
    $return = $false;
    $HTTP_Request = [System.Net.WebRequest]::Create($urStr)
    $HTTP_Response = $HTTP_Request.GetResponse()
    $HTTP_Status = [int]$HTTP_Response.StatusCode
    if ($HTTP_Status -eq 200) {
        $return = $true
    }
    else {
        Write-Host $urStr
        Write-Host "Response: " $HTTP_Status
        $return = $false
    }
    $HTTP_Response.Close()
    return $return
}

(Note: Any page URL that fails will be listed in the console after the script completes.)

After every URL goes through this check, we add the item to the array list:

if($isValidUrl){
      $script:itemIDsWithPassedCriteria.Add($item) > $null 
}

Finally, build out the report - which can then be exported via the Powershell ISE in CSV/Excel format:

if ($script:itemIDsWithPassedCriteria.Count -eq 0)
{
    Write-Warning "No page items found."
}else{
$props = @{
 InfoTitle = "Live Page Urls"
 InfoDescription = "Provides a list of all valid page URLs "
 PageSize = 100
}
    $script:itemIDsWithPassedCriteria|Show-ListView @props -Property 
       @{ Label = "Url"; Expression = { Get-ItemUrl ($_) } }
    Close-Window 
}


Here's the full script:

<#
.SYNOPSIS
  Provides a list report of all valid page URLs  
.AUTHOR
Written by Gabe Streza
#>
# Variables
$script:pageString = "* Page" #page string
function GetItemsWhichUsePageTemplate()
{
    $itemsWithMatchingCondition = Get-ChildItem -Path web:'/sitecore/content/WebsiteName/Home' 
                                                        -Language 'en' -Version * -Recurse 
    { 
        if ($item.Template.Name -like $script:pageString)
        {
            $linkUrl = Get-ItemUrl($item)
            $isValidUrl = IsValidPageStatus($linkUrl)
            if($isValidUrl){
                $script:itemIDsWithPassedCriteria.Add($item) > $null # The output of the Add is ignored
            }
        }
    }
}
function Get-ItemUrl($itemToProcess){
     [Sitecore.Context]::SetActiveSite("website")
     $urlop = New-Object ([Sitecore.Links.UrlOptions]::DefaultOptions)
     $urlop.AddAspxExtension = $false
     $urlop.AlwaysIncludeServerUrl = $true
     $linkUrl = [Sitecore.Links.LinkManager]::GetItemUrl($itemToProcess,$urlop)
     $linkUrl
}
function IsValidPageStatus($urStr){
    $return = $false;
    $HTTP_Request = [System.Net.WebRequest]::Create($urStr)
    $HTTP_Response = $HTTP_Request.GetResponse()
    $HTTP_Status = [int]$HTTP_Response.StatusCode
    if ($HTTP_Status -eq 200) {
        $return = $true
    }
    else {
        Write-Host $urStr
        Write-Host "Response: " $HTTP_Status
        $return = $false
    }
    $HTTP_Response.Close()
    return $return
}

$script:itemIDsWithPassedCriteria = New-Object System.Collections.ArrayList
GetItemsWhichUsePageTemplate

if ($script:itemIDsWithPassedCriteria.Count -eq 0)
{
    Write-Warning "No page items found."
}else{
$props = @{
 InfoTitle = "Live Page Urls"
 InfoDescription = "Provides a list of all valid page URLs "
 PageSize = 100
}
    $script:itemIDsWithPassedCriteria|Show-ListView @props 
                                                            -Property @{ Label = "Url"; Expression = { Get-ItemUrl ($_) } }
    Close-Window 
}
Write-Host "Done."

This took about 8 minutes to process a 2000 page site - which is good for a one-time run - but there are certainly some optimizations we should make if this was a report the client would use repeatedly in order to make it a bit snappier.  For this purpose, we're all set!

Feel free to grab this, tinker with it, and make it your own!

Let me know in the comments if this has helped - or if you have any additional recommendations.