BikeSharing360

During our Connect(); event this year we presented 15 demos in Scott Guthrie’s and Scott Hanselman’s keynotes. If you missed the keynotes, you can watch the recording in Channel 9.

This year, we built the technology stack for a fictional company named BikeSharing360, which allows users to rent bikes from one location to another.

BikeSharing360 is a fictitious example of a smart bike sharing system with 10,000 bikes distributed in 650 stations located throughout New York City and Seattle. Their vision is to provide a modern and personalized experience to riders and to run their business with intelligence.

In this demo scenario, we built several apps for both the enterprise and the consumer (bike riders). You can find all other BikeSharing360 repos in the following locations:

• Mobile Apps
• Backend Services
• Websites
• Single Container Apps
• Multi Container Apps
• Cognitive Services Kiosk App
• Azure Bot App

BikeSharing360 Modern Kiosk with Cognitive Services

During Connect(); 2016 we showcased many technologies available to you as a developer across Azure, Office, Windows, Visual Studio and Visual Studio Team Services. We’ve also heard from you that you love to have real-world applications through which you can directly experience what’s possible using those technologies. This year, then, we built out a full bikerider scenario for our Connect(); 2016 demos and are delighted to share all the source code with you.

Note: This document is about the Kiosk app only.

This kiosk app leveraged Cognitive Services to enable customers to interact with the kiosk through face detection, face recognition, voice verification, text-to-speech, speech-to-text, language understanding, and emotion detection, which allowed them to complete a transaction without the need for traditional input or touch or even pulling out your wallet.

Requirements

• Windows 10
• Visual Studio 2015 Update 3 (14.0 or higher) to compile C# 6 language features (or Visual Studio MacOS)
• Microsoft Azure subscription

Screens

Setup

Download or clone the repository.

1. Create an Azure account if you don't already have one using the steps in the next section of this README.

2. Create a Cognitive Service Key for the Face SDK:

   1. In Azure portal, click + New button
   2. Type "Cognitive Service APIs" in the search box
   3. Click Cognitive Service APIs (preview) in search result
   4. Click Create button
   5. Give a unique name to the Account Name field (for example, "FaceApiAccount")
   6. Click Api type to configure required settings
   7. Pick Face API (preview) from the list of services
   8. Click Pricing tier to select a proper pricing tier
   9. Set proper Resource Group option
   10. Click Legal terms to review and agree on the terms
   11. Click Create button
   12. Shortly you should receive a notification if the deployment succeeds. Click on the notification should bring you to the service account you just created
   13. Click on Keys and take a note of either one of the two keys.

3. Create a Cognitive Service Key for the Voice Verification SDK

   1. Similar to how you create the Face Api service, but this time, pick Speacker Recognition APIs (preview) from the list of services.
   2. Take note of either one of the two keys. We will need it in the later steps.

4. Set up a LUIS model using the instructions on setting up the LUIS model from the BOT sample

5. Create a face verification profile for yourself

   1. Clone the repository https://github.com/Microsoft/Cognitive-Face-Windows

   2. Open Sample-WPF\Controls\FaceIdentificationPage.xaml.cs, change Line 67 to have a more user-friendly name, instead of a GUID.

       public static readonly string SampleGroupName = Guid.NewGuid().ToString();
      

   3. Follow the instruction in README.md to build the sample

   4. Run the sample

   5. Click on Subscription Key Management tab, paste in the Face API key you saved earlier

   6. The sample comes with a set of training data under Data/PersonGroup folder in the repository, create a new folder (with a name of your choice), and copy one or more of your profile images into that folder. Frontal or near-frontal face works the best. Delete other folders in the example data set to reduce the api calls to your Face Api service

   7. Click on Face Identification tab

   8. Click Load PersonGroup button

   9. Select the Data/PersonGroup folder on your local disk

   10. If any of your profile images contains a valid face, the image will show up. It means your Face profile is registered successfully.

   11. The status pane should also contain the face profile id for each person, take note of your face profile id.

       Example log:

       [12:22:42.547589]: Response: Success. Person "Family1-Dad" (PersonID:9acfe7e1-6196-4230-aed8-a0b172ee2298) created
       [12:22:43.913174]: Request: Creating person "Family1-Daughter"
       [12:22:44.228009]: Response: Success. Person "Family1-Daughter" (PersonID:c32d0abe-ef03-40fc-b50d-66c989a9957e) created
       

   12. You need the group name, and the Face profile Id

6. Create a voice verification profile for yourself

   1. Clone the repository https://github.com/Microsoft/Cognitive-SpeakerRecognition-Windows

   2. Follow the instruction in README.md to build Verification/SPIDVerficationAPI_WPF_Sample.sln

   3. Run the sample

   4. Paste your Speacker Recognition account key in the Subscription Key Management tab

   5. Click on Scenario 1: Make a new Enrollment tab

   6. Pick one phrases from the ten available phrases

   7. Click Record button then start speaking your chosen phrase via microphone. Click Stop Recording after wards, the status pane should show your phrase if everything works as expected:

      [12:02:22.651469]: Your phrase: <XXX XXX X XXX>
      

   8. The sample code doesn't show you the Speaker profile id in the status pane. To work around it, you can click Reset Profile button as soon as it's enabled, the application will show a message similar to the below in the status pane. Take note of this id.

      [12:07:03.877365]: Resetting profile: 54aa9c1d-a815-44b6-9696-26be765dd840
      

   9. Repeat the recoding/stop recording using your chosen phrase until Remaining Enrollments reaches 0.

   10. You need the Speaker profile Id.

7. Open the BikeSharing.Clients.CogServicesKiosk.sln solution a. Update the Cognitive Services keys in the App.xaml.cs b. Add a row representing yourself in the constructor of the Data\UserLookupServices.cs class Build and run the application

Running the demo

You can find the steps to run through the demo script found in the Documents folder of this repo.

Enjoy!

How to sign up for Microsoft Azure

You need an Azure account to work with this demo code. You can:

• Open an Azure account for free Azure subscription. You get credits that can be used to try out paid Azure services. Even after the credits are used up, you can keep the account and use free Azure services and features, such as the Web Apps feature in Azure App Service.
• Activate Visual Studio subscriber benefits. Your Visual Studio subscription gives you credits every month that you can use for paid Azure services.
• Not a Visual Studio subscriber? Get a $25 monthly Azure credit by joining Visual Studio Dev Essentials.

Blogs posts

Here's links to blog posts related to this project:

• Xamarin Blog: Microsoft Connect(); 2016 Recap
• The Visual Studio Blog: Announcing the new Visual Studio for Mac
• The Visual Studio Blog: Introducing Visual Studio Mobile Center (Preview)
• The Visual Studio Blog: Visual Studio 2017 Release Candidate

Clean and Rebuild

If you see build issues when pulling updates from the repo, try cleaning and rebuilding the solution.

Copyright and license

• Code and documentation copyright 2016 Microsoft Corp. Code released under the MIT license.

Code of Conduct

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.