AutoMate


Welcome to AutoMate...


AutoMate is a tool that allows you to make shortcuts within Cortana for deep linking.  What is deep linking? Modern apps, in particular the new UWP apps, support being "driven" by URL. This is great - because it means you can automate those applications.  This is what AutoMate does. It has a simple "query" a user can set to call a "protocol handler" to tell an app to do something. It can be simple (for example; open the app) to complex (for example; open the app with credentials and perform actions on behalf of the user).  A query and a protocol URI combine into what AutoMate calls a shortcut.

You can create voice invocations for not-so-simple bookmarks on the web, open a recurring meeting, or anything you want as long as the app you're calling supports linking.

For example, you can say "Hey Cortana, ask AutoMate maps find restaurants in Redmond" and AutoMate uses Cortana to open the Maps app with that search.

There are some baked in shortcuts.
You can add your own personal shortcuts, or edit the ones provided if you are my Patreon Patron.

AutoMate uses @# to denote what the intents are in the query (invocation phrase). For each @#, you provide an example (for validation and readability). Then you can use the @# to denote where the intents are mapped into the URL (protocol handler). Finally, you can automatically generate the Regular Expression from the query by having the "Generate Regex" check box on, OR you can create your own expression (if you are a regex guru! You can build very complicated and powerful expressions with a little training.)

This skill is presently being demonstrated to friends; let me know if you have any protocols you'd like included on release!




References

URI Schemes and Protocol Handlers



  • https://docs.microsoft.com/en-us/windows/uwp/launch-resume/launch-default-app
  • https://www.iana.org/assignments/uri-schemes/uri-schemes.xhtml
  • https://www.w3.org/wiki/UriSchemes
  • https://en.wikipedia.org/wiki/Uniform_Resource_Identifier

  • Regular Expressions



  • https://en.wikipedia.org/wiki/Regular_expression
  • https://regexr.com/
  • https://regexone.com/
  • 5 comments:

    1. I just found automate and see that it can bring back some of the missing features Cortana looses on android vs a UWP that calls Cortana for example like Netflix on android vs Netflix on windows.

      How though could I implement this on android or for a web app through the edge for android browser? I have been playing with It but I do not do any coding and it seems really complex to me/

      ReplyDelete
    2. If you can see the URL, you can automate it! For example,
      https://www.netflix.com/search?q=bones
      If I am logged in, it will search for bones via the default browser.
      However, if you want to call the app, you need to discover what if any protocol handler has been installed. I found this for iOS
      https://ios.gadgethacks.com/news/always-updated-list-ios-app-url-scheme-names-0184033/
      That implies on iOS, the protocol handler is nflx://. It might be the same on Android - I haven't tried. If the protocol is undocumented, you can set up a tool like Fiddler to watch watch possible interactions to get the URL.

      ReplyDelete
      Replies
      1. It works for web client - and you can event tag a favorite show. http://www.netflix.com/watch/70302157?t=300 ... but it looks like the scheme on android is not obvious. Just a matter of time to find it.

        Delete
      2. found this on stack - doesn't work on android
        https://stackoverflow.com/questions/34730105/how-do-i-deep-link-to-netflix-on-tvos

        Delete
      3. Android settings told me that www.netflix.com should be opened by netflix. And sure enough... it works... try this:
        query: netflix @1
        sample: bones
        protocol: http://www.netflix.com/search?q=@1

        On Cortana V2 on Android, it launches netflix with search (after log in that seems to cache my credentials)

        Delete