Thursday, January 23, 2020

Reviving the Screenshots Ribbon Button in Sitecore using ScreenshotLayer API and PowerShell


Have you noticed that the Screenshots button has been removed in 9.3?

I guess it's not that surprising since I don't remember any point in time where it actually functioned. 

It's always gone something like this;

Click the Screenshots button:
The Screenshots button.  So familiar yet so foreign to me

Get a message stating you need to purchase the "relevant service".

Click Open

I never pass up a chance to open the Sitecore App Center
App Center Opens.  No one profits. Ever.
Great.


You may be asking, why don't the content authors just use an extension or some other tool?
Or, you know, the built-in capabilities of any Chromium-based browser.

🤷‍♂️

The request called for consistent, full-page screenshot capabilities in Sitecore.
I thought that seemed pretty do-able.

Ever hear of ScreenshotLayer?


What's ScreenshotLayer?

It's a highly reliable REST API service that produces screenshots.
Screenshotlayer is a lightweight REST API built to deliver high quality PNG, JPEG & GIF website screenshots at unparalleled speeds and through a simple interface. 
While I was researching and proving this out, I found that using open source libraries like Freezer often returned inconsistent results.  I landed on ScreenshotsLayer given the ease of integration. Basically; feed it a URL and some parameters, and you get a high-quality screenshot back.

The service is free up to 100 screenshots per month, with a reasonably priced subscription model otherwise. https://screenshotlayer.com/product


Limitations

- CM needs to be accessible on the web without IP restrictions for API to consume.  This won't work locally.

- Added cost: Depending on the number of screenshots taken per month, you'll hit the free 100 screenshots quick.  A Basic Plan subscription - which accounts for 10,000 screenshots per month should suffice in most cases. This doesn't seem out of the ordinary considering there used to be a service you'd have to sign-up for any way.



Goal

Our goal is to add a new Content Editor Ribbon button called 'Screenshots' in the same location that the old button once sat.  If you're running Sitecore 9.2 or below, you'll want to manually remove or deny read access to the existing Screenshots button in the Core database.



The Script

The script should initially get the selected item, assert that the item has a layout, and get the item's URL.  (The Get-ItemUrl function may need to be customized further for your own needs)




User Input

Let's assume a content author should make a simple selection for their screenshot; Mobile or Desktop.  We can show a simple dialog with two buttons; Mobile or Desktop.
Simple, yet effective.

Upon selecting Mobile, we'll simply set a variable.  If we have a Page URL, we'll call our Get-Screenshot function and pass in the URL and screenshot size as parameters:



Preparing the API Query

The API expects some common parameters defined in the URL query parameters, specifically, the API key, page URL, and viewport. We can concatenate all of our options and append it to the API URL endpoint.




Consuming the API

We can now execute a WebRequest to the API and cast our result to an image stream.



Saving and Downloading the Image

With an image in the memory stream, we'll need to temporarily store the image as a physical file on the file system.  In our approach, we'll utilize the $SitecoreDataFolder variable and create a 'screenshots' folder within it.  We'll build some conditions to check for the presence of this location.  Once the image has been saved, we'll invoke the Download-File function 


Putting it All Together


SPE Integration

Creating a New Module

Right-click on the Script Library folder and select Module Wizard


We'll name our module Screenshots and select Content Editor - Ribbon as our integration point:

You'll end up with a new module folder and several items:

We'll want to delete all folders in this new module except for the Content Editor > Ribbon > Presentation > Preview.  We can add a new PowerShell Script item under the Preview folder:

First thing's first....set an icon in the appearance section (you know about sitecoreicons.com,  right? 😉)

Now, let's make sure from an integration standpoint, the Screenshots button is only displayed for items with a Layout present.  We can easily accommodate this by editing the Show Rule in the Interactive section:
The Rules Engine makes this a no-brainer
Finally, we can add our script to the Script body field of our item.  Now would be a good time to make sure the $apiKey variable has been customized before saving. 



Activating the New Module

In order for the system to pick up our new module and its corresponding integration points, we need to Sync the library.  This can be achieved by opening a new ISE session and selecting the Sync Library Content Editor Ribbon sub-option of the Rebuild All button under the Settings tab:

We should now have a Screenshots button that displays in the Presentation > Preview chunk of the ribbon whenever an item with a layout is activated. 
wo0t!

Final Result




Starter Package

If you're looking to use or expand this functionality to fit your own requirements, you can feel free to download and install the Sitecore package from: https://github.com/strezag/Sitecore.SPE.Screenshots

Happy capturing! 📸



Wednesday, October 23, 2019

Sitecore Docker Setup; "curl: (6) Could not resolve host"


Given the general Docker hype and the clear signals that as an industry containerization adoption will continue becoming more widespread, I decided to take Docker for another spin after several years ignoring it completely and begin fiddling with Sitecore Docker Images.

Sticking closely to the quick start guide on the community-driven Sitecore Docker Images repository - I ran into one snag:


curl: (6) Could not resolve host: dist.nuget.org
curl: (6) Could not resolve host: download.microsoft.com
curl: (6) Could not resolve host: aka.ms
curl: (6) Could not resolve host: www.7-zip.org

The \docker-images\windows\9.0.2\sitecore-assets\Dockerfile itself only confirmed that a simple curl command initializes the download process from several hard-coded sources:

After some Googling, I found a post - Fix Docker's networking DNS config -by @nottrobin which outlined a similar error.

The key takeaway: this happens "usually because DNS lookups are failing in Docker images".

Their recommendation is to "change the DNS settings of the Docker daemon" by adding a new configuration file to /etc/docker/daemon.json - which I couldn't find.  Given that I'm using a Windows 10 machine and @nottrobin was using Linux, my guess is that there are path differences here. 

I ended up modifying the following file:
C:\ProgramData\Docker\config\daemon.json

And adding a new "dns" parameter (pointing to Google's public 8.8.8.8 DNS)

After saving and restarting Docker, the 'Could not resolve host' error no longer killed the build process. 🥳



Thursday, September 5, 2019

Azure Application Insights: Logs & Requests Viewer using Sitecore PowerShell Extensions

Last September, I wrote about accessing Sitecore Logs from Azure PaaS instances using the (now deprecated) AzureAILogs.html file provided by Sitecore. The knowledgebase article was updated in mid-January , 2019 – and the AzureAILogs.html file had been replaced with a new /sitecore/admin page dubbed AzureTools.aspx.

This updated admin page contains all the same functionality found in the AzureAILogs.html, with the addition of being able to pull log traces and requests from Application Insights.



Installation is easy: download the AzureTools.zip files, drop in the /sitecore/admin/AzureAILogs.aspx into the you’re your site’s root.

Admittedly, this admin page is great - but I could also see several aspects SPE being particularly useful (like the OOB SPE ListView - which would easily allow us to filter/sort/search through a series of log entries). An additional option to see raw color-coded logs would also be cool. 😊

Using the existing AzureTools.aspx as a general guide, we can re-create the GUI with general ease.

We’ll need:
1) Option to get Requests or Logs
2) Option to selected a Role (values pulled from API)
3) Option to control recency.
4) Option to control the severity.


The end result will consume the Application Insights REST API endpoints and allow a user to pull logs from Application Insights inside the CMS.


API Access

To start, we'll need to make sure we can work with the API by obtaining an Application Insights App ID and a corresponding App Insights API key  Sitecore's documentation already lists the.

Sitecore's documentation covers this but it's as simple as logging into Azure Portal and navigating to your Application Insights service. 

Under Configure, select 'API Access':


The Application Insights App ID will be displayed the following screen:

Copy this value and store it temporarily.

Click the 'Create API Key' button.
Give it a name and check the 'Read telemetry' checkbox.

After clicking 'Generate key', you'll have one opportunity to copy the 'App Insights API key'.  Copy and store this value temporarily.


Initial Communication with the API

Our script will utilize the two values to interact with the API.

Before building our UI in SPE, we'll need to confirm API communication by obtaining the server roles from Application Insights.  We can set a variable to call a function that will grab an ArrayList of roles:

Our function will build the URL, include the property URL authorization header containing the API key, and return an array list.



User Interface

Now that we have confirmed communication to the API and obtained our list of roles, we can pass the variable into a new function that will be responsible for building and displaying the UI:

The dialog should contain a series of radio buttons and checkbox lists, all of which will be used to provide options to build out another API call to obtain the traces or requests from AppInsights.

The output displays as follows:

Notice line 51 in the above snippet calls a Get-LogsOrRequests function which accepts a series of parameters from the dialog options upon selecting the OK button.

This function builds out the proper API URL and query parameters based on the passed the selected values passed in. Invoke-WebRequest is used to make the call to the API, which will return a JSON object of log entries from AppInsights based on those parameters.

Line 119 contains a final call to a function called 'Set-PostDialog' which provides options for displaying the results.

The output here is a ModalDialog with three buttons:


Selecting 'Script View' will display the results of the API in a color-coded Show-Result window:


Selecting 'List View' will process the results to an acceptable format for a standard SPE ListView result window (filtering, exporting, etc is obviously all included here):


Finally, selecting the 'Download' button will download a .txt file of the contents retrieved from the API.


Final Script





Installation

Manual

  1. Create a new Sitecore item based on the SPE PowerShell Script template and copy the final script above into the Script Body field.
  2. Replace the default "XXXXXXXXXXXXXXXXXXXXXXXXX" placeholder values in the $aiAppID and $apiKey variables with your own.  

Sitecore Package

  1. Download the Sitecore package and install from GitHub.
  2. Navigate to the PowerShell script located here:
    /sitecore/system/Modules/PowerShell/Script Library/Azure Application Insights Logs/Toolbox/Azure Application Insights Logs
  3. Replace the default "XXXXXXXXXXXXXXXXXXXXXXXXX" placeholder values in the $aiAppID and $apiKey variables with your own.  
The script will be available to run from the PowerShell Toolbox in the Start Menu.


Source Code

The full script can also be found on GitHub.
Feel free to grab a copy and modify it how you see fit.  



Thursday, July 11, 2019

Azure Search 1,000 Field Limit: Generating Values for AddIncludedField using PowerShell

If Azure Search is set up as your search provider, you're probably already aware of the limitations that come with it. One common issue you may discover (even well after your initial launch) is the limit on the number of fields Azure Search allows - a maximum of 1,000 fields.

In our case, after a major feature implementation and the introduction of a new site into our solution, the following log errors surfaced during the index rebuild process:

Exception: Sitecore.ContentSearch.Azure.Http.Exceptions.AzureSearchServiceRESTCallException Message: {"error":{"code":"","message":"The request is invalid. Details: definition : Invalid index: The index contains 1001 leaf fields (fields of a non-complex type). An index can have at most 1000 leaf fields.\r\n"}} 
The rebuild would get stuck and not budge.  This has been documented.

In our case, the <indexAllFields> property was set to true (which is the default value) causing the index field count to exceed the limit.

For the web and master index, there are two ways to work around this limitation within the Sitecore.ContentSearch.Azure.DefaultIndexConfiguration.config or overriding patch file:
  1. Set <indexAllFields> to false - then includethe fields which are necessary in the <include hint="list:AddIncludedField"> section.
  2.  Set <indexAllFields> to true and then exclude the fields which are unnecessary in the <exclude hint="list:AddExcludedField"> section. 

I like the first option.

We initially attempted to manually create all fields we believed were needed for our custom search services. This turned out to be rather tedious - and left a lot of room for error had we missed a field.

To quickly and easily identify all custom fields to include in the AddIncludedField section, I put together this super simple Sitecore PowerShell Extensions snippet:

Notice the path to the User Defined folder.  If you run this to include all custom fields with the query above as-is, you may run into the same 1,000 field limit error.  You can simply change the path and run this multiple times (or build a script out of it) to generate sections in bulk for templates that make sense for your specific solution.

The output can be simply copied and pasted into the AddIncludedField section of the Sitecore.ContentSearch.Azure.DefaultIndexConfiguration.config file:



The index should be able to rebuild successfully with all the custom fields present.
If you need to trim the index further, you should be able to determine your exclusions with a bit more granularity knowing that you haven't missed anything.

Happy Indexing! 🚀


Wednesday, June 12, 2019

Azure App Service Deployment Error: There is not enough space on the disk.

I recently came across an error during a deployment to one of our Content Delivery App Service instances - stopping the deployment process dead in its tracks. 

The error occurred when deploying to the CD Slot:


This occurred each time we attempted to deploy this step.

The error in full does indicate troubleshooting codes and links (oddly enough, the Microsoft link did not have any trace of the error code):
Failed to deploy web package to App Service. Error Code: ERROR_NOT_ENOUGH_DISK_SPACE More Information: Web Deploy detected insufficient space on disk. Learn more at: http://go.microsoft.com/fwlink/?LinkId=221672#ERROR_NOT_ENOUGH_DISK_SPACE. Error: The error code was 0x80070070.

I found it especially strange given that we saw no indication that we've hit any limit on storage space in Azure Portal.  Initially, I thought it may have something to do with storage on the build server, but was able to quickly rule out this theory by verifying that there was plenty of disk space remaining on that machine.

The error was also evident when I attempted to upload any file directly via FTP:

I began a look for some giant file(s) that may be preventing any additional files from being uploaded (since that's what made the most sense given the context of the error message itself).

By logging into the App Service via FTP, I was able to identify two IIS memory dumps located in the /LogFiles directory.  These both appear to have been taken the month prior - and simply never removed. 

After deleting both memory dump directories, I was able to restart the deployment step - which completed without errors. Direct FTP uploads were also restored.

What's up with that?

Well, your Azure App Services is tied to a particular pricing tier that dictates storage, memory, ACU, etc.  In our case, this particular App Services is configured to use the Standard standard tier - which has a 50GB limit.




By leaving the remnants of these IIS memory dumps on the App Service's storage, we must have surpassed that storage limit.

This brings up some interesting questions:

First, based on the configured pricing tier, is there a limit on how much Sitecore can store in App_Data/MediaCache or /temp folders before hitting that cap?

I assume the answer to that is yes - if your application is not setup to periodically clean stale files (which it should by default), it's possible to reach that storage limit and cause this error to surface.  In that case, the quick fix to get your deployment out would be to remove some or all temp files in the application.

Second, how can we monitor this storage limit of App Services in Azure Portal?

I'll have to circle back on this one as I don't quite have the answer to it.  It may even already exist, and I just haven't spotted it yet.   I'll update this post if I do figure that one out, but please let me know if you have the answer in the comments!


Monday, April 29, 2019

Sitecore Azure Kudu Tools PowerShell Module



I've managed to start several blog post drafts with the intention of sharing a few of my PowerShell scripts - but haven't got around to finishing/posting any of them.

This actually led to an epiphany: the scripts I wanted to share all had an underlying theme: obtaining files using the Kudu Rest API using PowerShell.


Huh? Kudu?

Aw, a baby Kudu!

If you don't already know, Kudu is the "engine behind git deployments" in Azure App Service - but can also be used as a built-in diagnostics tool within Azure.   Any time you have an Azure website, you automatically get a 'companion' Kudu (aka SCM) site accessible via the following URL format:  https://{ResourceName}.scm.azurewebsites.net.  

For example, if your CM App Service name is
'mysitecoresite-xp2-small-prd1-cm',

the corresponding Kudu site would be:
https://mysitecoresite-xp2-small-prd1-cm.scm.azurewebsites.net/ 

If you've ever had to Rebuild the xDB index in Azure Search, Sitecore's documentation walks you through how to do so using the Kudu Debug console.

Kudu REST API

One of my favorite feature of Kudu is the Kudu REST API since any files you can access via FTP are also accessible via the Kudu REST API.   You can download files, upload new ones, etc.

Typically, all you need are:
  1. Subscription ID
  2. Resource Group Name
  3. Resource Name
In Azure Portal - you can copy these values from a resource in the overview panel:


Those values are the key to interact with the API using PowerShell's Invoke-RestMethod.
There are a couple prerequisites:
  1. Azure RM module installed and ConnectAccount has been executed.
  2. Valid Azure credentials (same ones used to log into Azure Portal) to invoke Login-AzureRmAccount (converted to Base64)



Sitecore Azure Kudu Tools

I'd never written a PowerShell Module, but I figured this would be a great segway to learning how. I got to reading -- and tinkering -- and refactoring.  

Before long I had written my first PowerShell Module: Sitecore Azure Kudu Tools.

Sitecore Azure Kudu Tools is a collection of functions (three at the time of this post) built to help you get information/files from Sitecore instances on hosted on Azure PaaS using the Kudu Rest API.

It's available on the PowerShell Gallery or check out the GitHub repository.


Get-SitecoreSupportPackage

Allows you to remotely generate a Sitecore Support Package.  The function will download files defined for Sitecore Support Packages into a specified path, obfuscate sensitive data from ConnectionStrings.config, and compress the contents:

\App_Config\* Global.asax
\Logs\* license.xml
eventlog.xml sitecore.version.xml
Web.config


Why?

While there's a built-in way to obtain Sitecore Support Packages in Sitecore's admin page, being able to remotely obtain this information is a nice alternative.  This is useful not only for providing the required information for Sitecore Support tickets - but also for your own diagnostics.

Usage

Invoke this function using the following syntax:


Output


Alternatively, if you just run Get-SitecoreSupportPackage without any parameters, you'll be prompted to provide each required parameter.



Invoke-SitecoreThumbprintValidation

Provides a way to verify that certificate thumbprints match across Sitecore Azure PaaS Resource Group.  The function will download ConnectionStrings.config and AppSettings.config files from all App Services in a given Resource Group, then display any certificate thumbprints discrepancies.

Why?

If you've ever had to replace thumbprint values across an Azure PaaS Sitecore topology, you know that it can be a bit of a pain identifying all the configs where the old thumbprint needs to be replaced.

Bram Stoop wrote a great post identifying all the places you'll need to modify those thumbprints.  If you're looking to semi-automate this process, you can utilize the Kudu REST API to get all the configurations, then identify any mismatched thumbprint values using PowerShell.

Usage

Invoke this function using the following syntax:


Output




Get-SitecoreFileBackup

Allows you to a full copy of an App Service's website file contents.  This function will download all files from in a given ResourceName.

The following folders are excluded:
_DEV sitecore modules
App_Data sitecore_files
App_Browsers temp
sitecore upload
xsl

Why?

Sometimes I just want everything. If I need to determine discrepancies between environments for any reason, this helps.

Usage

Invoke this function using the following syntax:

Output




What's Next?

Now that the module is established, I've created a few GitHub Issues which I hope to get through quickly.  I also plan to add more functions in the future and am 100% open to contributions. 

Is it perfect? Far from it! It's a work in progress. Thrilled to share anyway.

If you're interested in contributing, please check out the contribute section on the Github repo.

via GIPHY

😊


Friday, March 29, 2019

Sitecore Twitter Hashflags


One of my favorite mediums into the Sitecore community is Twitter.  It's a place where you can interact with other Sitecore enthusiasts, promote your blog content / events, and have meaningful discussions about our favorite CMS.   It reveals how tight-knit our community really is.  If you're not already part of it, it's never too late to join!  😎

If you are active on Twitter, you've probably seen them.  Those little images that are embedded after specific hashtags are used.  Those are 'hashflags' and they've been around for almost a decade.

According to Wikipedia:
In 2010, Twitter introduced "hashflags" during the 2010 World Cup in South Africa. They reintroduced the feature on June 10, 2014, in time for the 2014 World Cup in Brazil, and then again on April 10, 2015, with UK political party logos for the 2015 UK General Election.
Since the first hashflag in 2010, 1,300+ active hasflags are available for use (1,309 at the time of this post - an unofficial running list can be found here)

Given that none of the Twitter-provided hashflags have anything available for Sitecore-related hastags (#Sitecore for example), I figured there is certainly a way we can include.

And thus, the Sitecore Twitter Hashflag Google Chrome Extension was born!


The extension runs only on specified Twitter domains (https://twitter.com, https://mobile.twitter.com, and https://tweetdeck.twitter.com).

A loop is configured to discover hashtags in a defined array of strings and replaces some HTML to include embedded images.  If you have the extension installed, it will inject any applicable hashtag to also include a hashflag.

The extension currently supports the following:
- #Sitecore
- #SitecoreMVP
- #SitecoreJSS
- #SCHackathon
- #WomenOfSitecore
- #SPE
- #Coveo
- #SUGCON
- #SitecoreSYM
- #SXA

The repository is open source and open to contributions here.


This project is just for fun with a hope to encourage the use of Sitecore hashtags.
It's okay to have fun!  😁

Enjoy!