Developing for Google Home: Getting Started

I recently blogged about the new Google Home I received for Christmas. In this blog, I'll talk about how developers can extend Google Home and teach it new conversational skills. Please note I am only 3 days in here, so this is just an introduction.

I'm a career software developer, so naturally I'm interested in extending Home further by developing new abilities for it. Amazon Echo is known to have an extensive developer and partner program with over 5,000 skills available; so what about Home? Google Home also has a developer program, which you can find out about at Actions on Google. Note, when you are developing for Home you are really developing for the Google Assistant, something that potentially can be used in additional ways (such as on Google Pixel phones).

There are several ways to develop conversation actions for Google Home and its Conversation API. One is with the Actions SDK, which includes a NodeJS client library and a web simulator. The other is API.AI, a web-based interface. API.AI is where I'm starting, and that's what I'm writing about today.

I've just started on learning, but I've already gone through the first tutorial and added a new conversation action that I can test in the web simulator or with my actual Google Home device. I can already tell this is going to be a great deal of fun.

Getting Started Sample: Silly Name Maker

Defining an Agent in API.AI

The API.AI site has a walk-through that gives you the steps to build a simple app named Silly Name Maker - and it's really simple - all it does is ask you for a number and a color, and constructs a silly name from that information. But, it does the job of getting you on your way. If you faithfully go to the API.AI site and enter the instructions, you will soon have an agent that looks like this:

Silly Name Maker tutorial sample in API.AI

and interacts like this:

User: Hey Google, Talk to Silly Name Maker.
Silly Name Maker: Hi! Welcome to Silly Name Maker!
Let's get started. What is your lucky number?
User: 22
Silly Name Maker: What is your favorite color?
User: Blue
Silly Name Maker: Alright, your silly name is Blue 22.
I hope you like it! See you next time.

Yes, it's not terribly useful but it does show you the ropes of defining an agent and running it. 

Intents

So what goes into making this agent? It consists of a number of intents. The first intent is named start_name_maker, and it's the first thing that happens after you invoke Silly Name Maker by asking Home to "Talk to Silly Name Maker". That's because it's tagged as a Welcome intent. All this intent does is welcome you to the agent, and prompt the user to select a number (and when the user does that, they'll be triggering the second intent). It's important for Google Home agents to clearly identify themselves to the user so they understand they're now interacting with an agent (app) and not Google.

start_name_maker intent

The second intent is called make_name, and it gets triggered when the user says a number or says "My lucky number is number". The intent can recognize several patterns of input, any of which will trigger it. Note how easy it is in API.AI to define user input - you simply provide samples of what someone would say, and then you can mark parts of that as parameter values. Parameter values have types; for example, @sys.color is a color and @sys.number is a number. 

Some of those "user says" patterns may provide parameter data - in this case, both patterns define the parameter $number. The intent has another parameter, named $color. Since $color is marked as a required parameter, and none of the defined "user says" patterns provide $color, Home will automatically prompt for it when this intent is triggered. That's why the example dialog above asks "What is your favorite color?" The make_name intent incorporates the parameters in its response: "Alright, your silly name is $color $number. I hope you really like it! See you next time!". Although it's not done in this case, it's possible to list multiple text responses for an intent (if more than one is present, one will be selected at random for the response).

name_maker intent

Now let's revisit the earlier sample dialog, with an understanding of what is happening behind the scenes:

User: Hey Google, Talk to Silly Name Maker.
[Silly Name Maker agent is launched. start_make_name intent runs,
and provides its response]
Silly Name Maker: Hi! Welcome to Silly Name Maker!
Let's get started. What is your lucky number?
[start_make_name intent ends]
User: 22
[make_name intent matches user input of number, and executes]
[$number parameter is from the user input]
[$color parameter value has not been provided yet, 
so API.AI prompts for it automatically]
Silly Name Maker: What is your favorite color?
User: Blue
[make_name intent provides its response]
Silly Name Maker: Alright, your silly name is Blue 22.
I hope you like it! See you next time.
[make_name intent ends]

Testing and Deploying your Agent

You can run the agent in a couple of ways. There's a web simulator on the right-hand side of the API.AI site you can interact with: just click the microphone icon and say something. You'll see how that was interpreted and what the response is. Click the Play link to have the response spoken to you.

Test Console built into API.API

Even better, you can test this on your own Google Home device. If you're logged into API.AI with the same Google account that your Google Home device uses, you can deploy it and go over and talk to the Home. Click Integrations on the left-side menu. On the list of One-click Integrations, click Actions on Google.

Integrations

On the Actions on Google dialog that appears, review the options (you can for example change the voice Home will use), then click Authorize at bottom right.

Actions on Google dialog with Authorize option

Now there is a Preview option available at bottom right. Click it. Preview is a temporary deployment (to do a permanent deployment, you'll need to first formally register your agent).

Actions on Google dialog with Preview option

Once you click Preview, you'll soon see confirmation that your agent has been published. Now you can walk over to your Home device and try it out. Begin with "Hey Google, Talk to Silly Name Maker."

Preview published confirmation

In my next post, I'll develop my first solo agent for Google Home.

At Home with Google Home

I just became the proud owner of a Google Home. This review will cover the initial experience of setting up and using Google Home.

Why Google Home?

First off, how did I end up with a Google Home, and not an Amazon Echo? The simple answer is, I received it for Christmas as a gift from my family. This year we celebrated Christmas a couple of days early so that our college-aged daughter Susan could join us. I'd overheard a few whispers here and there that "we got Dad something cool," so I knew there was some kind of special gift awaiting me - could it possibly be one of these home assistants? It turns out my family had purchased a Google Home for me. I've now had the Google Home for 36 hours, and in this post, I'll share what the initial experience has been like.

In November 2014, Amazon released Amazon Echo, their home smart speaker. It has been enormously successful for Amazon. Several former colleagues of mine who now work for Amazon have been regularly tweeting about the ever-expanding skills available for Alexa, the Amazon Echo assistant. Much more recently, in November 2016 Google released their competitor, Google Home. If I'd made the purchase myself, I admit I would have been torn: you can tell from reading reviews that both products are compelling with plenty of potential to do even more.

Reviews seem to suggest that both products are very good additions to the home and of course each maker has some advantages over the other. Both can play music or send things to your TV. You can order a zillion products from Amazon using Echo, and there's a vast network of partners providing support for just about every kind of smart home device there is. From Home you get amazing search and voice recognition plus the ability to ask general knowledge questions. The Home also lets you direct video from Netflix and other sources to your TV via Chromecast and control some home devices, including the Nest thermostat.

I've been aware of Amazon Echo and Google Home for the last year but hadn't had the opportunity to play with either before now. To be honest, I wasn't too keen to jump into anything voice-driven because I haven't had very good experiences with voice recognition in the past--this is my own fault, as I tend to talk fast or speak too low. But I know the technology has been getting better and better. From reading reviews, both Amazon Echo and Google Home are good useful products. Well, I was now the owner of Google Home so it was time to find out what all the excitement was about.

Unpacking

Home comes nicely packaged in an attractive white box which is very reminiscent of the experience you get with Apple products. Inside is the Google Home unit, a wall adapter, and some minimalist start-up instructions.

Home resembles an air freshener. It's white with an angled round top. The bottom colored band (gray on mine) covers the speaker. You can buy bands in other colors if you want to match your home decor. 

Google Home on my fireplace mantle

The Get Started card has just 3 steps: plug the power cable into the home, plug the other end in to a wall socket, and download the Google Home app on your phone.

The phone app detects your Home device and performs a quick test which causes your Home to make a sound--confirming you are connected to the right device.

You also configure access to your wireless network for the Home, choose a default music service, and tell it what address it's located at.

  

There are more things you can configure to get the most out of the device--see other sections below--but at this point you can already start using Home for music and search queries. From here on, you use the phone app to do further setup on Google Home such adding home devices and service accounts (see later below).

Basic Usage and Interaction

To get Google Home's attention you say a wake-up phrase of "OK Google" or "Hey Google," followed by your command or question.  I find "Hey Google" to be easiest to say.

As I mentioned earlier, I wasn't sure that *I* would get much out of a voice-driven product as I tend to speak too rapidly and softly. And how well would it do with a whole family of people talking at it? This turns out not to be a worry with Google Home: from the moment we plugged it in, everyone in our family has had no problems at all being understood. The voice recognition is really good, and it can pick up voice from quite far away.

The round top appears to be featureless white, but as you interact with Home you'll find it can do a variety of fun things with lights. As soon as Home recognizes the wake-up phrase, four lights appear to show it is active.

Google Home when active

The top of Home also responds to touch. You can tap the top to pause (or resume) playing music or video, or to interrupt a response. You can dial your finger around the top in a clockwise motion to increase volume or in the other direction to decrease volume. Of course, you can also do all these things by speaking to it.

From the reference card included in the box, here are some examples of commands you can give Home (in each case, preceded by the wake-up phrase):

"What does circumlocution mean?"
"What's the latest news from NPR?"
"Who won the Giants game?"
"How is the S&P 500 doing?"

Google Home also has great contextual memory. Once you are discussing something, it knows what "it", "he", or "she" means. For example:

"Who is president of the united states?"
"When was he born?"
"What is his favorite food?"

Since the phone app configuration includes telling the Home the address where it is located, you can ask it lots of locale-based questions:

"What's the temperature outside?"
"Where's the nearest supermarket?"
"What time is 'Passengers' playing?"

And there are also some things it can't do--yet:

"Give me directions to Panda Express" responds with an apology that Home can't give directions yet. This is kind of odd, since Google has fantastic direction-giving ability when you use it from a computer or phone. This is clearly coming, though. I have to remind myself that this product was only released last month.

Controlling the Thermostat
Home is clearly behind Amazon when it comes to partnerships that allow you to control a vast array of home devices, but they do have support right now for Nest thermostats, Philips Hue lights, Samsung Smart Things, plus the IFTTT online service.

No doubt this list will expand over time. In the meantime, I do have a Nest thermostat in my home so I configured that. In the Google Home app, I added the Nest device and provided my account information, and that was that.

Now I can give commands and queries like these:

"What's the inside temperature?"
"What's the thermostat set to?"
"Set the heat to 72 degrees"
"Make it warmer"
"Make it cooler"

I can see the day when I have more smart home devices and control them via Google Home, such as lighting and alarms.

Music

One of the most useful aspects of Home is playing music: that's where having this device in the home with a good speaker (as opposed to using your phone) really shines. The Home's speaker is pretty good and music sounds great coming from it.

In configuring Home with the phone app, I selected Google Play Music as my default music service, which automatically makes my Google Music library available to Home (alternatively, I could have selected YouTube Red, Spotify or Pandora). Now I can play songs from my library like this:

"Play my Christmas playlist"
"Skip this song"

Oddly, though, it doesn't seem I can play an individual song or album by name: "Play song" doesn't work. After some research, it appears you currently can only play playlists, not individual songs or albums from your Google Music Library. That's a bit of a downer, and hopefully will be addressed soon. I believe this may just be an issue with Google Play Music and not the other music services.

On the other hand, we've had great success just asking Home to play things that we don't even own in our music library. My wife has been getting hours of Christmas atmosphere with requests like these:

"Play some Christmas music"
"Play some instrumental Christmas music"

It's not really clear to me yet where this music is coming from--since I haven't paid for it. It makes me wonder if there's some kind of premium music trial going on that will eventually need to be paid for.

But wait, there's more. Radio is also available through Google Home, via the TuneIn service:

"Play KOST FM"

Video

I have a Chromecast device connected to my TV, and that is also something Google Home understands. After using the phone app to tell Home our Netflix account, I can now give commands like the following:

"Play Zootopia from Netflix on TV"
"Fast forward 10 minutes"
"Play 'What Does the Fox Say' from YouTube on TV"

I only have one TV and one Chromecast, in the same room as Google Home--but if I had a more elaborate setup I could tell Home where to play things:

"Play Zootopia from Netflix in Living Room"

Although video worked right away on my Chromecast, audio did not. It took me about a day of trial and error to get audio working (I had go to my LG TV Settings and turn speakers on instead of letting it default to integration with my home theater). This is an extra step I'd prefer to avoid but its not a bad reflection on the Google Home itself. When I next upgrade my TV and home theater, I'll be paying attention to Consumer Electronics Control (CEC). As I understand it, with the right set up playing video from Home can automatically turn on the TV and switch input to Chromecast.

Photos

Home can also play photos (from my Google Photos collection) on my TV via Chromecast. You can reference photos by date, by special occasion, or even by subject. I'm not sure how Google manages to find photos by subject, but it does. If (for example) I say "Play car photos on TV", it finds all my photos that contain an automobile. There's some kind of impressive analytics going on with Google Photos.

"Play photos from Christmas on TV"
"Play photos from December 22 on TV"
"Play car photos on TV"
"Play dog photos on TV"

Fun

The Home also has plenty of fun built into it, including some personality in the Google Assistant that powers Home. My son has been building up his comedic material by constantly asking Home "Tell me a joke" and getting a different joke each time.

Home can also play games. For example,

"Play mad libs"
"Play lucky trivia"
"Play crystal ball"
"Entertain me"

Google Home has the concept of applications, known as conversation actions. When you're interacting with an action such as Lucky Trivia (which is multi-player by the way), you're executing a conversation action. You don't have to say "OK Google" before each response, which is nice.

How Useful Is it?

I admit it, I expected Google Home to be an interesting geek toy but didn't really expect it to become an integral part of my home--but I was wrong. After just a few days, it's regularly used by my entire family.

Everything the Home does can also be done with my phone, so how useful is it, really? Well, very! Think about the number of times you pick up your phone or pull it out of a pocket--it's a lot. Just being able to call out voice commands is extremely convenient. When I first installed the unit, I wondered if it would be necessary to be right in front of it in order for it to recognize speech, but it turns out it does really well even when we call out commands from another room.

I also wondered how strict the Home would be about the format of commands and queries: would we have to memorize exact patterns of words in order for it to understand us? Many voice-driven devices in the past have ended up "training the user" to speak the way the device can understand. Fortunately, that isn't necessary here. The Home has excellent natural language speech recognition and is very tolerant of variations in phrasing. All of my family members have been able to interact with it right from the start.

One limitation of the Home is that it is linked to a single account--in our case, my music library, my Netflix account, my calendar, etc. There's no support as yet for switching profiles for different users. I presume this will come in time.

One other limitation that I don't expect to get resolved is there's no support for competing services like Apple iTunes. That's a shame, because the rest of my family is on Apple devices. I think that's a shortcoming of any home assistant, whether from Google, Amazon, or (coming soon) Apple: you won't necessarily be able to access every service you want to.

In my next blog post on Google Home, I'll talk about how developers can teach Google Home new actions.

The Geek's Guide to Dieting, Part 3: Diet-friendly Fast-food Options

This is part 3 in a series on dieting for geeks. Today I'll discuss diet-friendly lunch options when eating out.

Buffalo Wild Wings

Buffalo Wild Wings most popular items are the boneless wings with one of the many sauces. You're essentially looking at 100 calories per wing here, so choose a smaller-size portion such as the snack size (5 wings, 1 sauce) and you'll be okay.

Buffalo Wild Wings Snack-size Boneless Wings (500)

Chili's

A lot of the delicious food at Chili's is out of reach: we're talking calorie counts in the thousands. However, they do have some low-calorie options that are quite good. Although they don't generally list calories on the menu, they do have a Lighter Choices section which does. Options here are Spicy Chile Chicken (420), Ancho Salmon (590), Classic Sirloin with Grilled Avocado (420), Mango-Chile Chicken (510), Mango-Chile Talapia (520), Grilled Chicken Salad (440), and Margarita Grilled Chicken (590). I'm personally fond of the Mango-Chile Chicken.

Chili's Mango-Chile Chicken (510)

In-n-Out Burger

In-n-Out Burger can be done, but you'll probably be getting a single item rather than a full meal. A cheeseburger is 480, but protein-style (no bun) is 330. A Double-Double is 670, or protein style is 520. French fries are 395.

In-n-Out Protein Cheeseburger (330)

Jersey Mike's

Jersey Mike's is hands-down my favorite sandwich chain, but like most sub places the majority of the food is way up there in calories. However, there are some lower-calorie options. Start by getting a mini-size sandwich on whole wheat. A Turkey (#7) mini on whole wheat, prepared Mike's Way, is 410.

Jersey Mike's #7 Turkey Mini on Wheat, Mike's Way (410)

Kentucky Fried Chicken

KFC might seem like something to avoid on a diet, but you can make it work. Original-recipe chicken is lower than you might think, at Breast (320), Thigh (270), Drumstick (160), Wing (120). Individual sides really vary, so choose carefully. On the higher side we have Cole Slaw (170), Mac & Cheese (170), and Mashed Potatoes & Gravy (120). On the low side is Corn on the Cob (70) and Green Beans (25) which are quite doable.

KFC Original Chicken Breast (320)

Panda Express

Panda Express nicely lists the calories on their menu. In particular, take note that the popular sides (white rice, fried rice, chow mein) are 380-520 calories alone! That means you're in trouble before you even select an entree. Fortunately, there is the mixed veggies side option which is only 80 calories. If you go with mixed veggies, you can enjoy an entree or two without killing your calorie count. The entries with lowest calorie counts are broccoli beef (150), string bean chicken (190) and mushroom chicken (170).

Panda Express Broccoli Beef (150)

Panera

Panera is one of my favorite places to eat, but some of their yummy food is off-the-charts high in calories (mac and cheese is 450-900). Be careful of sides: a baguette or soft dinner role on the side will add 180/190 calories to your meal. But you can make Panera work if you focus on the lower-calorie choices, and they helpfully have the calories listed on the menu. The Green Goddess Cobb Salad with Chicken makes a filling lunch at 520 calories. If you get a Pick 2, good choices for soup are Turkey Chili (170) or Broccoli Cheddar soup (230). For Pick 2 sandwiches, best bets are Steak and Arugula (250) or Turkey Breast on Whole Grain (280).

Panera Pick 2 Steak & Arugula Sandwich (250)+Turkey Chili (170)

Rubio's Coastal Grill

Rubio's is a popular Mexican food chain headquartered in Southern California. Famous for their fish tacos, Rubio's offers delicious fresh food and prominently displays calories on their menu; unfortunately, much of this yummy food is off-the-charts high in calories. Good options for the calorie-conscious dieter are individual tacos such as the Wild Mahi Mahi Taco (240), Classic Steak Taco (200), Classic Chicken Taco (250), Grilled Gourmet Taco with Chicken (330). The Grilled Street Tacos entree (three snack-size tacos) is also diet-friendly, at Chicken (100) or Steak (120). The dish that I always get, however, is the Balsamic & Roasted Veggie Salad with Chicken (310): it's a large bowl of food and is really good.

Rubio's Balsamic and Roasted Veggie Salad (310)

Sonic

Sonic is another tasty place where most of the menu is out of reach (don't go near a milkshake), but there are options for you. For burgers, your best bet is the Sonic Jr. Deluxe Cheeseburger (420) or Sonic Jr. Double Cheeseburger (520). For chicken, the Classic Grilled Chicken Sandwich (450), Chicken Strip Sandwich (440), or Crispy Chicken Sandwich (580).

Sonic Classic Grilled Chicken Sandwich (450)

Subway

Subway isn't my favorite sandwich place, but given its size and popularity I thought I should list it here. Start with 6" sandwiches on wheat, and beware sauces that can add hundreds of additional calories. Sandwiches on Subway's "Fresh Fit Choices" menu include Turkey Breast (280), Black Forest Ham (290), Subway Club (310), Oven Roasted Chicken (320), Roast Beef (320), Rotisserie-style Chicken (350), and Sweet Onion Chicken Teriyaki (370).

Subway 6" Turkey Breast on Whole Wheat (280)

Well, there you have it--fast food on a diet can be done, if you look for the low-calorie options. I'll be expanding this list over time.

In Part 4, we'll look at surviving the holidays.