Youyeetoo R1: First Impressions

Unboxing and Language

In my search for multiple core ARM processors in SBCs (I was drooling over AmpereOne's 192 cores), I found a new board called youyeetoo R1. It appears to be an Orange Pi 5 competitor. Its got a Rockchip RK3588s 8 core, big little (A76/A55) 4 and 4 core and the big cores burst at 2.4GHz. Also like the OPi5, it has a Mali 610 GPU - and more features that seemed attractive like an NPU (neural processing unit, can do simultaneous operations like you'd want in a neural net firing your activation / objective functions and back propagation weighting adjustments), and a NFC (Near Field Communication) sensor (that could be used to read you Yubi key, phone, or allow for lil' robot communication if they are in proximity to each other). It also has support for two monitors via MIPI-DSI and HDMI ports. I ordered an 8/64 EMMC with Debian pre-installed so I could play... 

It arrived well packaged.

Double bubble wrap!

And the box wasn't squished in any way.

The box confirmed what I bought - though the Wifi module was misleading (it was not included). The power supply was typical 12V 4A barrel tip.

Inside we had a spacer and two anti-static bags.

The board was on top of the heat sink.

Board looked clean.

The heat sink looks like it blanket covers the CPU and other chips. It came with some thermal double sided tape.

The board is bigger than the Orange Pi.

I booted it up.  To my surprise, it did NOT launch a frame buffer (fb) console. It has U-Boot installed, and apparently dumps its console via serial tty (so if I had a debug serial cable that fit and a FTDI USB adapter, I might be able to watch it boot, and or interact with U-Boot.) See the picture below for the UART debug plug on the bottom of the board (and its #1, so pins are not shared with GPIO block.)

So it has a Debian/Linaro build with XFCE4.  I like XFCE4. But, I don't read simplified Chinese (though I have about 100 Mandarin/pinyin words in my vocabulary.)

So the first step, let's get a language we know.  Open a console, and install English.

sudo dpkg-reconfigure locales

sudo update-locale LC_ALL=en_US.UTF-8

The first command brings up a CURSES dialog in Chinese, but it is easy to find and select en_US and create the resources. The second command sets the default language to English. You need to `sudo reboot` after this.

Installing Software

So we are running a 5.10 kernel.  My Orange Pi is pegged at 5.15.  My Odroid and other machines are on 6.1.

There are a few tools I can't live without.  Most important for me are benchmarks, partition managers, developer tools, process monitors, and graphics drivers...

sudo apt install xfce4-terminal

sudo apt install gnome-system-monitor

sudo apt install gparted

sudo apt install build-essential

I hit my first glitch.  The system is pegged (more on this later).  Turns out I needed to force a dpkg and reinstall to get current build-essential.

sudo apt install --reinstall dpkg-dev libdpkg-perl

So now I can re-install build essentials.  Great. I try chromium browser, and get errors about bad dates and certs. Ok, lets fix the time ISO style.

sudo date -s "2024-01-25 17:15"

But I really want to autoset the time on boot - not just for the duration of whatever capacitor is on the board.

linaro@linaro-alip:~$ sudo timedatectl set-ntp true

Failed to set ntp: NTP not supported

Huh. No NTP?  Not good. And I can't  install timesyncd as its pegged. So how many packages are locked?

linaro@linaro-alip:~$ apt-mark showhold | wc -l 

1340

No way. Update only told me 96 packages were locked. Now I am worried. Where are the Debian packages actually coming from?

linaro@linaro-alip:/etc/apt$ cat sources.list

deb http://mirrors.tuna.tsinghua.edu.cn/debian/ bullseye main contrib non-free

deb-src http://mirrors.tuna.tsinghua.edu.cn/debian/ bullseye main contrib non-free

deb http://mirrors.tuna.tsinghua.edu.cn/debian/ bullseye-updates main contrib non-free

deb-src http://mirrors.tuna.tsinghua.edu.cn/debian/ bullseye-updates main contrib non-free

deb http://mirrors.tuna.tsinghua.edu.cn/debian/ bullseye-backports main contrib non-freedeb-src http://mirrors.tuna.tsinghua.edu.cn/debian/ bullseye-backports main contrib non-free

deb http://mirrors.tuna.tsinghua.edu.cn/debian-security bullseye-security main contrib non-free

deb-src http://mirrors.tuna.tsinghua.edu.cn/debian-security bullseye-security main contrib non-free

Now I am worried. I am locked into an unofficial, Chinese repo. This is not uncommon.  The Orange Pi is also tied to Huawei. I know we also have Hon Hai / Foxconn SBC. I have heard horror stories (usually about Mini PCs) of all sorts of spyware and malware, usually on "Windows" distributions.  I install stock Ubuntu intel from Live USB on all my Minis.  It suffices to say, `netstat -epa` and process monitor's network traffic made me feel much better that information was not being sent somewhere I didn't want it to go. However, I could not redirect to official repos (that I can with Korean Odroid.)  This makes me unhappy.

Quick Orange Pi Comparison

The form factor is slightly bigger. But we have more ports. And we favor one side, that actually isn't bad (as it is more rack friendly.)

I am irritated by both that we don't have an EMMC socket (like Odroid). This means to flash the EMMC, you need to treat the board like an ADB/USB micro storage device (will cover in another blog).

On the Orange Pi 5 (that does have a fb console), we only have a power button.

But on Youyeetoo R1, we have 4 buttons.  From left to right: Reset, Power, Recover, and Boot.

I have not figured out exactly what the async behind the buttons actually does aside from the obvious (but I will talk about it in another blog post).

Quick GL Performance

XFCE4 performs well.  There is no cursor flicker (like on Le Potatoe). `glmark2-es2` runs okay, but oddly, benched about 760 compared to the Orange Pi 980 at same resolution (it should be the same, right, as it has the same hardware.) There were some odd artifacts with GL, so I suspect the Mali (Panfrost?) driver isn't current. Also, libmali.so, not there, so glmark2 will run with the LLVM pipe (effectively unaccelerated software rendering), that means any game emulators will need to support GLES. 

Other Quirks

When I tried to get to a fb console via `alt Fx`, the device hung.

I managed to completely corrupt OS and eventually the root partition... fixing that is for another blog.

Next Steps

My next step is to try build and install Ubuntu, as unfortunately there is not an image. I understand Youyeetoo plans to provide one eventually, but I believe this device is currently not for a casual hobbyist.  To do that, the following would need to happen

• Baked in GRUB support
• USB boot
• fb console
• Use of official repositories
• Not locking down packages 

Note that vendors lock packages to have a stable OS. But this is at the expense of developers. So... 

I do look forward to continuing to play with the device, in particular its NPU and NFC, however I won't be sticking it in a game cabinet any time soon.  It will likely get added to my home-lab rack.

90 day new user estimates

Looking at how many users on Google or Microsoft are using my skills.

	Microsoft	Google
Art of War	23	540
Safety Moments	54	101
Meditations	12	67
Ambient Sounds	65
AutoMate	5

Art of War wins on Google. Ambient Sounds wins on Microsoft. And hey, for all this effort, at least Google is sending me a T-Shirt. I suspect Colossal Cave will beat them all...

Google tells me on average 15 new users a day try Art of War.

Actions On Google

Well, after some frustration, I have a chat bot working on Google Actions. But now that the frustration is over, the flood gate is open!

The first challenge; using my existing code. Safety Moments, my first skill, uses json files. I needed to upload them to a Google bucket. And then, for whatever reason, the promise loading these files took > 5s to execute (a download of a file <1K), so my conversation was over before my content loaded. There was no synchronous hack I found. Then, I figured I'd load them on invocation of the cloud function. It worked for a while, and then it just stopped working. No idea why. I received connection resets downloading the files. So second solution? Embed all the json in the webhook itself.  (What is a webhook?  It is the node code that fulfills the request to do something after LU digestion.)

The second challenge; competing SDKs. It looks like Dialogflow and Actions on Google were inseparable. But they are. And if you follow the code examples for Dialogflow (that I did, because it looked like a really cool tool compared to LUIS), you will discover mismatches between how Actions On Google (for assistants) implements bot behavior. So... I couldn't immediately figure out what was Dialogflow SDK (agent.add) and what was Actions SDK (conversation.ask). It manifested itself in my conversation data not persisting and me scratching my head.

The third challenge; moving targets.  The Google tech is migrating from V1 to V2, and Dialogflow seems not to have caught up to Actions. Part of my confusion was out-of-the-box webhooks were V1.  Dialogflow examples integrating with Actions was V1. But V2 is pretty much were all Actions documentation and examples sit.  Ugh. Now I know what to look for.

What I Like

• International availability.
• Excellent TTS for female and male voice.
• Dialogflow (the tool) and how it builds intents (utterances) and actions (entities).
• You can build dev without charge.
• Events get fired when media stops playing.

What I Dislike

• You can't have a Card before a SimpleResponse.  I use cards as banners.
• Cards can have one button, though the JSON supports multiple buttons.  Why?
• You can't segment long conversations naturally. You have to make them turn based and granular. Not so good for my experiments.That will, in my opinion, seriously cripple enterprise capability.
• Your action will not pass certification if it does not end the dialog on a question. This forces dialogs to be very transactional.

References

Actions on Google and Dialogflow.

• https://developers.google.com/actions/
• https://dialogflow.com/
• https://firebase.google.com/

Useful documentation

• https://developers.google.com/actions/assistant/responses
• https://developers.google.com/actions/reference/nodejs/lib-v1-migration

Building your own Cortana Music Player

Ever since Amazon killed the cloud Music Storage subscription, I've been annoyed. I uploaded my 2000+ CD collection to the cloud to preserve it and allow me to access it from anywhere.

I've been looking for an alternative, but didn't find anything as cost effective as Amazon's dead program ($25 for the year? a steal). Google had a Google Music Play plan that included YouTube Red for $15/mo. But that cost made me uncomfortable (that, and being on a monthly plan.)

But I believe I found a solution!  OneDrive... you get 1TB of storage with an Office 365 account for $99/yr. That is value, and you get Office too.

But here is what I find more exciting.

I got my Invoke today and had to (like really) see if I could build my own music player... Out of the box Cortana will only let you hook up to streaming services. But you can't play music from your PC or OneDrive (unless you BlueTooth to the device).

How hard can it be to build a skill to play your music from OneDrive? As it turns out - not hard at all.

Amazon Music continually griefed me because it never kept my songs together in their albums when imported (as were imported via iTunes). But when I synced my library to OneDrive - the directory structure is intact. And as it happens, the OneDrive REST API will let you retrieve your directories and walk the files... and you can use your MSA authentication to keep it all personal or share those files...

So, how hard is it to get Cortana and botframework to play an MP3 you have stored on OneDrive?  This easy.

var audioCard = new builder.AudioCard(session)
        .media([
            { url : 'https://onedrive.live.com/download?cid=00E75C36F57E8A5B&resid=E75C36F57E8A5B%216254&authkey=AEAEHi1WUjheHj4' }]);  
var msg = new builder.Message(session)
 .addAttachment(audioCard)
 .text('Now playing Nephatiti by 808 State)
 .speak('<speak version="1.0" xmlns="http://www.w3.org/2001/10/synthesis" xml:lang="en-US">Playing Nephatiti by <say-as interpret-as="number_digit">808</say-as> State.</speak>');
session.send(msg).endConversation();

Edit: It is extremely important to end the conversation after sending an audio card on Windows because if Cortana has a dialog going, regardless of any input hint, the volume will be set low. Ending the conversation keeps the volume at 100% at the expense of disconnecting Cortana from the bot.  Also, Cortana will ignore every field in the audio card (like title).

What will be even more fun will be using the language recognition services to solve another Alexa pet peeve... search for my classic and latin music titles!

End to End Developer example video of basic Cortana skill

For those wanting a visual walk through of the process to create a Cortana skill with botframework!

Uses node.js.

Get your Azure QnA Bot to Speak with Cortana

Building a QnA bot and hooking it up to Cortana is simple in node.js under BotFramework V3.

If you have a FAQ that is in Q: A: format you can import it via the Azure QnA Maker tools and auto-create a "knowledge base".  No coding required. You can do this here https://www.qnamaker.ai.

The next step is creating your bot. Microsoft has standardized on their botframework to do this. https://dev.botframework.com is your gateway.

If you are like me and like the simplicity of node.js then pick the QnA template.

Go back to qnamaker and view the code to extract the QnA keys and host.

Then go back to the Azure portal and update your bot for the Application Settings blade sections shown here.

The test web app will now be successfully linked to your QnA bot! But don't forget the last step...

Go to your Channels blade and set up Cortana. Then, go to the build blade and open the online editor. In the app.js code, you will see that the template uses the standard QnA dialog builder - that does not say the resulting answers back with the Cortana speech channel. Add an override like this.

(See GitHub QnAMaker patch for a V4 node example.)

There you have it. Now "Hey Cortana, ask Bernie Question Bot Test what is a dwarf planet?"

For C#, it is slightly more complicated. You need to subclass the the BasicQnAMakerDialog with something that sends a message with speak attached. This is done as an inner class of RootDialog (from the V3 C# template that comes with Azure Web App Bots.

    // Dialog for QnAMaker GA service
    [Serializable]
    public class BasicQnAMakerDialog : QnAMakerDialog
    {
        // Go to https://qnamaker.ai and feed data, train & publish your QnA Knowledgebase.
        // Parameters to QnAMakerService are:
        // Required: qnaAuthKey, knowledgebaseId, endpointHostName
        // Optional: defaultMessage, scoreThreshold[Range 0.0 – 1.0]
        public BasicQnAMakerDialog() : base(new QnAMakerService(new QnAMakerAttribute(RootDialog.qnaAuthKey, RootDialog.qnaKBId, "No good match in FAQ.", 0.5, 1, RootDialog.endpointHostName)))
        { }

        // Override to also include the knowledgebase question with the answer on confident matches
        protected override async Task RespondFromQnAMakerResultAsync(IDialogContext context, IMessageActivity message, QnAMakerResults results)
        {
            if (results.Answers.Count > 0)
            {
                IMessageActivity response = context.MakeMessage();
                response.Text = "Here is the match from FAQ:  \r\n  Q: " + results.Answers[0].Questions[0] + "  \r\n A: " + results.Answers[0].Answer;
                response.Speak = response.Text;
                response.InputHint = "acceptingInput";
                await context.PostAsync(response);
            }
        }
    }

(See GitHub QnAMaker patch for a V4 C# example.)

Building Bots in Azure

I am starting to build Cortana bots. So far its been interesting.

I had trouble building the Azure function bots from template. The node.js example for the simple "echo" bot... First crack, the Azure "Test in Web Chat" didn't work. Errors implied there was a permission issue.

I tried again, and had a deployment error on the bot function template.

I tried again, and on the third try it deployed. To my knowledge, I did nothing differently.

The advantage of a function bot over a web app bot is supposedly pay per invocation (that should be cheaper, right)? The issue with the example code for node.js on the function bot is that every potential used library is embedded in the index.js code where the two line "echo" functionality is embedded in the middle!

WHY?  Why? Well, javascript as a language doesn't have a '#include' statement. Client side, you do the includes on the document for your browser to take care of.

In node, we have requires... that allows us to load modules. But why is this not used in the function bot example? I figure its a work around. But the down side is this: every time I edit the function in the portal for this example, I am touching a 200K line file!