Blog posts by Darren S

Episerver Developer Meetup Melbourne October 10th 2019

2019-09-16T04:11:05.0000000Z

Announcing I'll be presenting at an Episerver Developer Meetup again... but this time in my home town of Melbourne!!!

Luminary will be hosting Melbourne's first Episerver Developer Meetup from 6pm on Thursday 10th October 2019.

Speakers will include;

Marcus Babajews (Episerver APAC)

Damien Dias (Episerver APAC)

Darren Stahlhut (Luminary)

Matthew Boniface (Empired)

As Melbourne's first Episerver Developer Meetup, we will be discussing a wide range of topics and current trends in Episerver, as well as an open Q&A session.

The overall goal of this meetup is to kick start the local Episerver Development Community in Melbourne and have some fun.

https://www.meetup.com/Episerver-Melbourne/events/263574319/

Code - Blazor + Episerver blog series

2019-07-27T02:29:09.0000000Z

In my last blog, I announced the start of a blog series on Episerver Blazor as a follow up to a presentation I did at the Episerver Sydney Meetup https://world.episerver.com/blogs/darren-s/dates/2019/7/blazor--episerver-blog-series/

Today, I'm happy to continue the series with a massive code drop 🎉

Here is the Github repo 👇
https://github.com/DarrenStahlhut/Episerver.Blazor

This is a work in progress, I'd love your contributions so please feel free to fork the repo and submit a PR.

The next blog posts in this series will cover the following topics:

Getting started with Episerver Blazor, a step-by-step installation guide (for anyone that needs it).
Concepts of Blazor, understanding the fundamental questions of why and how to use Blazor.
Getting Episerver Content and Routing running in Blazor.Serverside
How to call Javascript Library's from C# using Blazor Javascript Interop.

Please post any comments or issues here or github.

Have an awesome weekend!

Introduction - Blazor + Episerver blog series

2019-07-13T03:46:41.0000000Z

Following my presentation and demo of Blazor + Episerver Content Delivery API in Sydney at the Episerver Developer Meetup - July 2019. I've decided to start a series of blog posts to share the content with the global community.

For those that attended the Sydney meetup, thank you, I had a great time building the proof of concept and I'm keen to expand on it over the coming weeks.

Here is the Blazor slide deck from the meetup: http://bit.ly/2LmbP3N

This first post in the series is mostly just to announce the series, share the slide deck, and get some general interest going.

In the next post, I'll share some code (via github) including Client-side and Server-side Blazor running against Episerver Content Delivery API

I'm particularly interested in expanding the solution to add Episerver Routing (to the Server-side app) and demonstrate how Javascript interop works, to keep the front-end JS developers happy 😁.

Anyone eager to get a headstart can have a look at these references (included in the slide deck);

Blazor Official Site: https://dotnet.microsoft.com/apps/aspnet/web-apps/client
Blazor Official Documentation: https://docs.microsoft.com/en-us/aspnet/core/blazor/?view=aspnetcore-3.0
Blazor Video Steven Sanderson: https://www.youtube.com/watch?v=0RfUPr0KrSM
Blazor Official Blog announcing release: https://devblogs.microsoft.com/aspnet/blazor-now-in-official-preview/
Episerver Content Delivery API Developer Guide: https://world.episerver.com/documentation/developer-guides/content-delivery-api/
Episerver Content Delivery API Reference: https://sdk.episerver.com/ContentDeliveryAPI/2.x/Index.html

Blog Series Updates:

2019-07-27 - Code available on GitHub - https://world.episerver.com/blogs/darren-s/dates/2019/7/code---blazor--episerver-blog-series/

Episerver Developer Meetup Sydney July 10th 2019

2019-06-26T04:09:17.0000000Z

I'm excited to announce I'll be speaking at the Episerver Developer Meetup in Sydney on Wednesday, July 10th 2019.

I'm going to be talking about;

Blazor - https://dotnet.microsoft.com/apps/aspnet/web-apps/client
Azure Cognitive Services / Computer Vision Add-On - https://world.episerver.com/blogs/darren-s/dates/2019/3/using-ai-to-analyze-images-and-generate-metadata/

Damian Dias will talk about Commerce and CMS Certifications best practices.

Nicola Ayan will talk about Episerver Find and share lessons learned from recent projects.

Marcus Babajews will give a short update on Episerver Campaign.

https://www.eventbrite.com/e/episerver-developer-meetup-sydney-tickets-62842623980

GitFlow is a natural fit for Episerver DXC

2019-04-01T01:01:50.0000000Z

In this blog, I'll explain a little about GitFlow and why it aligns so well with Episerver DXC development.

What is GitFlow?

If you're not familiar with GitFlow, it's just another branching and merging strategy. It was made popular by Vincent Driessen at nvie and has since been widely adopted by the development community.

It's also built into the most popular Git Clients including SourceTree and GitKraken and is available via commands in Git console.

What I like most about GitFlow, is not the merging strategy itself, but its things that having a standard strategy offers;

We can more easily collaborate with external developers and Agencies. We can simply by say "we use GitFlow" and they'll know what we mean.
Onboarding Developers is faster, there are a lot of good learning resources around.

GitFlow merging strategy (in a nutshell)

There are two main branches in GitFlow;

master - the code you have or are going to release.
develop branch - the latest development, changes for the next release.

Besides the main branches, we also have;

feature branches - this is where you actually do your development and commits.
release branches - a branch to get code ready to release to master.
hotfixes - for emergency fixes for master branch to be immediately released.

Why does this work well with Episerver DXC?

Because of Episerver DXC's linear deployment path, code must be deployed from Integration > Preproduction > Production.

This means we can think of our Master branch as the code ready to be deployed to Integration.

Deployments above Integration are managed via the Paas Portal or via Episerver Support.

If you also want to have an internal CI/CR (build and release) environment for internal QA, you can run it off the Develop branch

This all means we can use the standard implementation of GitFlow.

Wrapping it up

I'm not saying GitFlow is perfect, but it is a decent merging strategy. It can become complicated when you aren't certain about the order that Features will be released and I have some very large scale projects that GitFlow struggles to handle so we have a customized strategy.

But generally speaking, for Episerver DXC projects where the deployment chain is linear, I've found it's a great fit.

This might change in the future if we are allowed to use our own DevOps tools to deploy directly to Preproduction (or even Production) which is something I've heard is on the technical roadmap.

Resources

GitFlow by Vincent Driessen
https://nvie.com/posts/a-successful-git-branching-model/
GitFlow Atlassian
https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow

Using AI to analyze Images in Episerver

2019-03-26T01:45:46.0000000Z

Inspired by Artificial Intelligence? Me too!

In this blog, I'll show you how to configure Azure Cognitive Services, add the Computer Vision C# SDK to Episerver, and work with Azure Computer Vision's APIs to analyze Images.

What can we do with AI in the CMS

After reading the official documentation I had several cool ideas;

By using the Describing images feature we can generate a human-readable description of the image (eg alt text) to improve Accessibility and SEO.
We could detect and prevent inappropriate content by using the Adult and Racy images feature. Imagine validating User Generated Content, or auditing an entire Media Library with a scheduled task.
In line with the inappropriate visual content, we can use the Recognize Text feature to read embedded text, then validate it against a list of banned words.
We could automatically set the focal point and crop images by using the Thumbnails and Area of Interest feature. I think this one is really cool. You've likely seen tools that allow CMS Editors to select a focal point for an image, we can now use AI to set it.
We can use the Color schemes feature to allow us to automatically select a theme to complement the Image's colors (eg a light or dark theme of a hero Banner).

For the remainder of this blog, I will demonstrate the first idea, generating descriptive text for an image.

Configuring Azure Cognitive Services Computer Vision

You will need an Azure account to perform this step.

In your Azure Portal, click Create a resource > AI + Machine Learning > Computer Vision (or search for Computer Vision).

Then complete the form

Enter a Name (eg ComputerVision)
Select your Subscription
Select a Location (remember it, you'll need it later)
Select a Pricing Tier (the F0 free tier is fine)
Select or Create a Resource Group (eg Episerver)

Finally, after Azure has provisioned the service, you'll need to copy Key 1 (actually either key will work) to use in your code (yes I have reset my keys, sorry!).

A quick test using Postman

At this point the AI Service is running in Azure, it's a good time to quickly confirm things are working. Using Postman I made a POST using the API. This is the Image URL I sent to the service, and this the JSON result.

{
  "categories": [
    {
      "name": "outdoor_oceanbeach",
      "score": 0.99609375,
      "detail": {
        "landmarks": []
      }
    }
  ],
  "adult": {
    "isAdultContent": false,
    "isRacyContent": false,
    "adultScore": 0.011389658786356449,
    "racyScore": 0.030652720481157303
  },
  "color": {
    "dominantColorForeground": "White",
    "dominantColorBackground": "Blue",
    "dominantColors": [
      "Blue",
      "White"
    ],
    "accentColor": "0750C4",
    "isBwImg": false,
    "isBWImg": false
  },
  "imageType": {
    "clipArtType": 0,
    "lineDrawingType": 0
  },
  "tags": [
    {
      "name": "water",
      "confidence": 0.9997772574424744
    },
    {
      "name": "outdoor",
      "confidence": 0.9996986389160156
    },
    {
      "name": "sky",
      "confidence": 0.9993903636932373
    },
    {
      "name": "beach",
      "confidence": 0.9672492146492004
    },
    {
      "name": "shore",
      "confidence": 0.963735044002533
    },
    {
      "name": "ocean",
      "confidence": 0.9035242199897766
    },
    {
      "name": "nature",
      "confidence": 0.896171510219574
    },
    {
      "name": "wave",
      "confidence": 0.4871264696121216
    },
    {
      "name": "sandy",
      "confidence": 0.14046534895896912
    },
    {
      "name": "voyage",
      "confidence": 0.055084559276120444
    },
    {
      "name": "landscape",
      "confidence": 0.02963506257691794
    },
    {
      "name": "sand",
      "confidence": 0.02806604807717514
    }
  ],
  "description": {
    "tags": [
      "water",
      "outdoor",
      "beach",
      "ocean",
      "nature",
      "man",
      "wave",
      "board",
      "shore",
      "surfing",
      "sand",
      "body",
      "standing",
      "umbrella",
      "sandy",
      "walking",
      "riding",
      "snow",
      "white",
      "blue"
    ],
    "captions": [
      {
        "text": "a sandy beach next to the ocean",
        "confidence": 0.9815995674986626
      }
    ]
  },
  "requestId": "26b96fb1-8b67-4bd6-b6ac-4e9621769a12",
  "metadata": {
    "width": 1200,
    "height": 800,
    "format": "Jpeg"
  }
}

The next step is to move from a raw web request and a JSON result to the C# SDK.

Adding Azure Computer Vision C# SDK to Episerver

This step is easy, it's just adding a Nuget Package to your Episerver CMS Project.

Search for Azure Cognitive > select Microsoft.Azure.CognitiveServices.Vision.ComputerVision

After the install, Build your solution to make sure everything is ok, then move on.

Working with the Computer Vision APIs

My example will use the Describing Images feature of the API. I will register an OnSavingContent event, pass the image stream to the API, and save the data returned from the Computer Vision API into read-only properties for later use.

Firstly, I need a new Class, which I called ImageFile, that inherits from ImageData to add some extra properties.

Description - To be populated by a Human
UseAiDescription - An editable boolean, initially set by AI, can be later set by a Human
AiDescription - Read-only property to hold the image description text returned by the AI analysis
AiDescriptionConfidence - Read-only property to hold the AI's confidence in its description

using EPiServer.Core;
using EPiServer.DataAbstraction;
using EPiServer.DataAnnotations;
using EPiServer.Framework.DataAnnotations;
using EPiServer.Web;
using System.ComponentModel.DataAnnotations;

namespace EpiserverDemo.Models.Media
{
    [ContentType(GUID = "0A89E464-56D4-449F-AEA8-2BF774AB8731")]
    [MediaDescriptor(ExtensionString = "jpg,jpeg,gif,bmp,png")]
    public class ImageFile : ImageData
    {
        [Display(
        Name = "Description",
            Description = "Manually populated Description of the Image",
            GroupName = SystemTabNames.Content,
            Order = 10)]
        [UIHint(UIHint.Textarea)]
        public virtual string Description { get; set; }

        [Display(
        Name = "Fallback to AI Description",
            Description = "If true, the AI Description will be used when Description is empty",
            GroupName = SystemTabNames.Content,
            Order = 10)]
        public virtual bool UseAiDescription { get; set; }

        [Display(
            Name = "AI Description",
            Description = "AI generated Description of the Image",
            GroupName = SystemTabNames.Content,
            Order = 20)]
        [Editable(false)]
        [UIHint(UIHint.Textarea)]
        public virtual string AiDescription { get; set; }

        [Display(
            Name = "AI Description Confidence",
            Description = "AI confidence rating for the generated Description",
            GroupName = SystemTabNames.Content,
            Order = 30)]
        [Editable(false)]
        public virtual double AiDescriptionConfidence { get; set; }
    }
}

Next, I needed to register the OnSavingContent event. You'll see for brevity I've hard-coded the Subscription Key (Key 1 copied from Azure) and the Computer Vision Endpoint (you can find a list of Endpoint URLs here based on the Location you selected in Azure).

The Computer Vision API can accept either a URL or Stream. I opted for the stream so that I didn't have to persist the Image to the database to get a URL.

You will also see a bit of logic to set the UseAiDescription property to true when Confidence is greater than or equal to 90%. I found the AI Description was often good when Confidence was above 85%.

using System.Collections.Generic;
using EpiserverDemo.Models.Media;
using EPiServer;
using EPiServer.Core;
using EPiServer.Framework;
using EPiServer.Framework.Initialization;
using EPiServer.ServiceLocation;
using Microsoft.Azure.CognitiveServices.Vision.ComputerVision;
using Microsoft.Azure.CognitiveServices.Vision.ComputerVision.Models;

namespace EpiserverDemo.Infrastructure
{
    [InitializableModule]
    [ModuleDependency(typeof(EPiServer.Web.InitializationModule))]
    public class ImageAiMetaDataInitialization : IInitializableModule
    {
        public void Initialize(InitializationEngine context)
        {
            var eventRegistry =
            ServiceLocator.Current.GetInstance<IContentEvents>();
            eventRegistry.SavingContent += OnSavingContent;
        }

        public void Preload(string[] parameters)
        {
        }

        private void OnSavingContent(object sender, ContentEventArgs e)
        {
            if (e.Content is ImageFile)
            {
                SetAiMetaData(e.Content as ImageFile);
            }
        }

        public void Uninitialize(InitializationEngine context)
        {
            var eventRegistry = ServiceLocator.Current.GetInstance<IContentEvents>();
            eventRegistry.SavingContent -= OnSavingContent;
        }

        public static void SetAiMetaData(ImageFile imageFile)
        {
            // get a stream of the image from binary data
            var stream = imageFile.BinaryData.OpenRead();

            // subscription key to use with Azure Computer Vision (found in Azure Portal)
            var subscriptionKey = "1774efe94d4e48ec96ad9d89d2ed4fc3";

            // specify the features to return, there's some cool stuff in here
            List<VisualFeatureTypes> features =
                new List<VisualFeatureTypes>()
                {
                VisualFeatureTypes.Categories, VisualFeatureTypes.Description,
                VisualFeatureTypes.Faces, VisualFeatureTypes.ImageType,
                VisualFeatureTypes.Tags, VisualFeatureTypes.Color,
                VisualFeatureTypes.Adult, VisualFeatureTypes.Objects
                };

            // make a Computer Vision client
            ComputerVisionClient computerVision = new ComputerVisionClient(
                new ApiKeyServiceClientCredentials(subscriptionKey),
                new System.Net.Http.DelegatingHandler[] { });

            // specify the Azure region endpoint (based on Computer Vision service location in Azure Portal)            
            computerVision.Endpoint = "https://australiaeast.api.cognitive.microsoft.com";

            // analyse the image, return ImageAnalysis (sync for now, not await async)
            ImageAnalysis imageAnalysis = computerVision.AnalyzeImageInStreamAsync(stream, features).Result;

            // set Image AI Description
            var caption = imageAnalysis.Description.Captions[0];
            if (caption != null)
            {
                // set confidence first
                imageFile.AiDescriptionConfidence = caption.Confidence;

                // set the boolean based on Confidence. Only do this the first time AI Description populated
                if (string.IsNullOrEmpty(imageFile.AiDescription))
                {              
                    // anything less that 90% confidence is often an inaccurate description
                    imageFile.UseAiDescription = (caption.Confidence >= 0.90);
                }

                // set the AI Description
                imageFile.AiDescription = caption.Text;
            }
        }
    }
}

The end result

When your code is ready, run the CMS and drag in an Image, I used this one.

This will trigger the OnSavingContent event and send your Image to the Computer Vision API for analysis, and finally populate your properties.

Here's a screencast (gif) of this whole process to give you a sense of the speed of the API (it's pretty fast considering I'm using the Free Pricing Tier);
https://i.imgur.com/sgBBFLg.gifv

Rendering the image

The last thing left to do is to render the image, the idea here is to;

Use the Description property as alt text when populated (by a Human). Humans still describe Images better than Machines.
or
Use the AIDescription property as alt text if the Description property is empty and the UseAiDescription is true.

I won't add the View for this because it's pretty straight forward (and this blog is long enough) but let me know if you need it.

<img src="/globalassets/beach.jpg" alt="a sandy beach next to the ocean" >

Wrapping it up

I hope I have inspired you to consider how can AI can be used to improve your CMS Editors experience and the quality of content.

I'm pretty impressed with the AI generated Description when its Confidence rating is high. But after using the APIs, I can admit that Humans are still far better at describing Images than Machines.

That said I do see a lot of potential, not just to assist the CMS Editor when they upload individual images, but especially by processing large amounts of images with a Scheduled Job.

Official Documentation References

Azure Cognitive Services Computer Vision
https://docs.microsoft.com/en-us/azure/cognitive-services/computer-vision/home
Computer Vision API
https://westus.dev.cognitive.microsoft.com/docs/services/5adf991815e1060e6355ad44/operations/56f91f2e778daf14a499e1fa
Computer Vision C# SDK
https://docs.microsoft.com/en-us/azure/cognitive-services/computer-vision/quickstarts-sdk/csharp-analyze-sdk

Episerver Personalization with persistent anonymous profiles

2018-10-12T02:00:00.0000000Z

Episerver Personalization is a powerful feature to ensure your website displays relevant content to your visitors. At Luminary we needed a way to persist an anonymous visitor's personalisation profile across sessions without using Episerver Insight or custom cookies. Here's how we did it...

Generating image metadata and SVG thumbnails with Episerver DXC Azure Blob storage

2018-07-30T02:00:00.0000000Z

Episerver has an ImageData base class that natively supports images. But we needed to extend it to provide metadata fields and display SVG thumbnails. Here's how we did it...