Wednesday, 18 July 2018

How to invoke Cortana channel actions in node.js



Ugh.  It took me a while to figure it out, and I had to reverse engineer some code to do it. There were breadcrumbs, but of course being a newb it wasn't immediately obvious.

Botframework allows you to connect to channels. You are likely invoked from a channel. That means you can send meta-data back to that channel. In the previous version of the framework, you called a method off the Message called channelData to set the JSON response.  But channelData is deprecated in favor of the new sourceEvent method.  So forget the "Launching apps or websites from a Cortana skill" example (that is in C# and the old way)...

The next hint is in the documentation. sourceEvent takes a map.  The old version just took some action in a JSON with a type : LaunchUri and uri : "http://whatever.com/"

What is this map? Well, you can have your bot connected to one or more channels (or "*" for all.) The source code implementing sourceEvent was the final piece. I found examples setting the facebook channel, or using directline... But what is the channel name for Cortana?  Well, it is cortana.

Wrapping the JSON up underneath the channel name and magically Cortana will launch the app.

msg.sourceEvent( 
{ cortana : {
    action : { 
      type : "LaunchUri",
      uri : "https://www.octavianit.com"  
      }
    } 
  } )

But be careful as Cortana will close the channel after invoking the app (that the protocol maps to).  And you should consider that speakers (screenless devices) will not support this behavior and should have a UX alternative.



Tuesday, 17 July 2018

Building your own Cortana Music Player

Ever since Amazon killed the cloud Music Storage subscription, I've been annoyed. I uploaded my 2000+ CD collection to the cloud to preserve it and allow me to access it from anywhere.

I've been looking for an alternative, but didn't find anything as cost effective as Amazon's dead program ($25 for the year? a steal). Google had a Google Music Play plan that included YouTube Red for $15/mo. But that cost made me uncomfortable (that, and being on a monthly plan.)

But I believe I found a solution!  OneDrive... you get 1TB of storage with an Office 365 account for $99/yr. That is value, and you get Office too.

But here is what I find more exciting.

I got my Invoke today and had to (like really) see if I could build my own music player... Out of the box Cortana will only let you hook up to streaming services. But you can't play music from your PC or OneDrive (unless you BlueTooth to the device).

How hard can it be to build a skill to play your music from OneDrive? As it turns out - not hard at all.

Amazon Music continually griefed me because it never kept my songs together in their albums when imported (as were imported via iTunes). But when I synced my library to OneDrive - the directory structure is intact. And as it happens, the OneDrive REST API will let you retrieve your directories and walk the files... and you can use your MSA authentication to keep it all personal or share those files...

So, how hard is it to get Cortana and botframework to play an MP3 you have stored on OneDrive?  This easy.

var audioCard = new builder.AudioCard(session)
        .autoloop(true)
        .autostart(true)
        .title('Nephatiti by 808 state')
        .media([
            { url : 'https://onedrive.live.com/download?cid=00E75C36F57E8A5B&resid=E75C36F57E8A5B%216254&authkey=AEAEHi1WUjheHj4' }]);  
var msg = new builder.Message(session)
 .addAttachment(audioCard)
 .text('Now playing Nephatiti by 808 State)
 .speak('<speak version="1.0" xmlns="http://www.w3.org/2001/10/synthesis" xml:lang="en-US">Playing Nephatiti by <say-as interpret-as="number_digit">808</say-as> State.</speak>');
session.send(msg);

What will be even more fun will be using the language recognition services to solve another Alexa pet peeve... search for my classic and latin music titles!

Monday, 16 July 2018

End to End Developer example video of basic Cortana skill






For those wanting a visual walk through of the process to create a Cortana skill with botframework!

Uses node.js.

Friday, 13 July 2018

Get your Azure QnA Bot to Speak with Cortana

Building a QnA bot and hooking it up to Cortana is ridiculously simple.

If you have a FAQ that is in Q: A: format you can import it via the Azure QnA Maker tools and auto-create a "knowledge base".  No coding required. You can do this here https://www.qnamaker.ai.

The next step is creating your bot. Microsoft has standardized on their botframework to do this. https://dev.botframework.com is your gateway.

If you are like me and like the simplicity of node.js then pick the QnA template.

Go back to qnamaker and view the code to extract the QnA keys and host.
Then go back to the Azure portal and update your bot for the Application Settings blade sections shown here.

The test web app will now be successfully linked to your QnA bot! But don't forget the last step...

Go to your Channels blade and set up Cortana. Then, go to the build blade and open the online editor. In the app.js code, you will see that the template uses the standard QnA dialog builder - that does not say the resulting answers back with the Cortana speech channel. Add an override like this.

There you have it. Now "Hey Cortana, ask Bernie Question Bot Test what is a dwarf planet?"





Getting the most out of your free Azure subscription for development.

I discovered why my Azure free trial was eating cash. It seems that the free trial defaults to reasonable paid service tiers, and not the free development tiers. So when you set up your Microsoft account (MSA) for development purposes, make sure you double and tripple check that the plan you pick starts with an F (for free) and not an S.

When you use bot builder or QnA builder, you might need to go back after you create a project from template and change the "App Service Plan" to something free. If you see your $ decreasing and you're not doing anything - you've configured something incorrectly.


Thursday, 12 July 2018

Building Bots in Azure

I am starting to build Cortana bots. So far its been interesting.

I had trouble building the Azure function bots from template. The node.js example for the simple "echo" bot... First crack, the Azure "Test in Web Chat" didn't work. Errors implied there was a permission issue.

I tried again, and had a deployment error on the bot function template.

I tried again, and on the third try it deployed. To my knowledge, I did nothing differently.

The advantage of a function bot over a web app bot is supposedly pay per invocation (that should be cheaper, right)? The issue with the example code for node.js on the function bot is that every potential used library is embedded in the index.js code where the two line "echo" functionality is embedded in the middle!

WHY?  Why? Well, javascript as a language doesn't have a '#include' statement. Client side, you do the includes on the document for your browser to take care of.

In node, we have requires... that allows us to load modules. But why is this not used in the function bot example? I figure its a work around. But the down side is this: every time I edit the function in the portal for this example, I am touching a 200K line file! 




Tuesday, 21 March 2017

Pi Zero W

When I installed the SD the Pi Zero W came with from CanaKit, I had a heck of a time getting it working. First, the micro USB hub I had from Amazon basics didn't work.
Then I had to manually load up the wifi config.
I installed libSDL, advmame (thanks Silverman) and fceux and ... performance was crappy.
But I couldn't see anyone else saying Asteroids was broken out the box.
So I tried to install my Pi2 SD...
I learned that older versions of raspbian don't work with the Zero. Not even if you
  • sudo apt-get update
  • sudo apt-get install rpi-update
  • sudo apt-get dist-upgrade
  • sudo rpi-update
    Its because PiZeroW runs best with the latest raspbian build, Jessie. It doesn't like the old build, Wheezy as the wifi is flakey.
    So after a couple flips of build, it looks like my original video settings we mangled on the SD card that shipped with the Pi Zero W. My latest attempt work - and Pacman is playable with latest advMame - but asteroid is not. That said, Super Mario works like a charm...
    Now I need to discover how to tune the Pi Zero.
    And my Orange Pi arrived...