Hi everyone, This blog is about integrating Voice recognition in SAP Screen Personas Flavor, so that the end user can provide input through Voice command instead of keyboard press or mouse click.
I have used Web kit speech recognition which is a API available for Google chrome 25 and above. This can make the Personas Flavor voice driven.
I have named the flavor as "Persona". She can also reply you back using the speech synthesis. For example when you say "Good Morning" she can reply you back with a suitable answer after checking current time in your location using Javascript.
The user can give voice commands for Persona to close any popup or enter data in text field or even for launching a particular flavor and transaction.
Voice Interaction in SAP Personas Flavor
Sorry guys forgot to add the part for voice command being wrong. When the voice command turns to be wrong or not understandable Persona apologizes for not understanding the command, repeats the command which she has understood and asks to repeat the command again by the user. It would be something like "Sorry I cannot understand *your_command*.Can you repeat it again please."
1.Create a Javascript file
Create a JavaScript file and write the function for selecting the transaction and return the transaction code.Save the file in ".js" format.
I have attached the file "check_Transaction_IDs" rename the file to ".js" format.
The JS file contains code like below. I have given only the logic below but attached the completed file
transaction_check = function(str1,str2){
var t;
switch(str1){
case 'CREATE':
switch(str2){
case 'INQUIRY':
t="VA11";
break;
}
break;
case 'CHANGE':
switch(str2){
case 'INQUIRY':
t="VA12";
break;
}
break;
case 'DISPLAY':
switch(str2){
case 'INQUIRY':
t="VA13";
break;
}
break;
}
return t;
}
After this upload the file in the system using the transaction "/n/personas/admin".
![Create resource.JPG]()
Then after uploading note the GUID for using the script in Personas Flavor
![guid.JPG]()
2.Welcome message on Flavor Load
The next step is bringing welcome message when the flavor gets loaded for this purpose we check weather the flavor is loaded the first time or not and then proceed further.
The user name is stored in a variable called name using the method session.info.user.
var name = session.info.user;
To check weather the flavor is loaded the first time we use the session variable. We check weather the session variable is empty if its empty then the flavor is loaded the first time and so perform the necessary actions and set the session variable value.
if(session.utils.get("check") != "first"){
//write the code be to perform when the flavor is loaded the first time in the session
}
Next we have to produce the voice for Persona to welcome the user for this purpose we use SpeechSynthesisUtterance class available in chrome. This speech synthesis does not need any external permission need. It works fine without any permission with the help of speechSynthesis object.
we should create an object for the SpeechSynthesisUtterance class and set the language.
su = new SpeechSynthesisUtterance();
su.lang = "en";
The next step is we should get the JSON array with all supporting voices and assign a voice to Persona. Even though the voice is not assigned to persona a default voice is set to it.
The next important thing is we should build the string for making Persona speak that string, we use the username which is stored in the variable called "name" and welcome the user with nice greetings.
su.text= "Hi "+name+". I am Persona, your Personal Voice guide";
speechsynthesis.speak(su);
Make sure you set the session variable after completing all these steps so that Persona does not welcome the user each time a refresh occurs or change of flavor occurs, you can the value of the variable "check" to any string.
session.utils.put("check","first");
3.Performing the interaction with the flavor:
First get the time of the client location as in his system using the Javascript and update the greet word according to the current time as Good morning or Noon or Evening.
var greet ='';
var time = new Date();
var hour = time.getHours();
if(hour < 12 && hour >= 6)greet = "GOOD MORNING";
else if (hour >= 12 && hour <= 16)greet = "GOOD NOON";
else greet = "GOOD EVENING";
Create an object for the class webkitSpeechRecognition and specidy the variable "lang" of the class as "en-US" so as to recognize the the language spoken as US English.
var recognizer = new webkitSpeechRecognition();
recognizer.lang = "en-US";
In the video which i have shown above the connection is not an secured i.e Http connection so for time the user needs to click the allow button, but if the connect is secured i.e Https the script will not ask for authorization each time, it is a one time process if it is a secured( Https) connection and the user need not press on allow each time.
Also create the object for Speech Synthesis as specified above during the welcoming the user.
su = new SpeechsynthesisUtterance();
su.lang = "en";
If needed specify the slang voice which needs to be Spoken in the variable called "voice" when Persona speaks.I have use two voices one on US slang the female and another is German slang the Male.
var voice = speechsynthesis.getVoices();
su.voice=voice[2];
The next step is to get the result as the user speaks on the microphone and store in a object when the user stops speaking the result has to be processed and actions have to be performed accordingly.For this purpose we use the even onresult and write the function.
recognizer.onresult = function(event) {
//Get the voice and convert into object as soon as the user stops speaking
if (event.results.length > 0) {
var result = event.results[event.results.length-1];
}}
Process the result and perform the action once the user has stopped speaking.
if(result.isFinal) {
//Perform actions
}
4. Colloquial Interaction with Persona
Get the string from the result object and convert the string to Uppercase so that it can be used to perform actions.
var check = result[0].transcript;
check = check.toUpperCase();
If the user says Good Morning or Good Evening or Goon Noon Check with the variable given above and reply the user accordingly. If its a Hi, Hello or How are you process it accordingly using suitable cases.
We can change the voice by checking the session variable and updating the variable call voice in the class using the object "su".
if(check.indexOf("CHANGE VOICE") != (-1)||check.indexOf("CHANGE YOUR VOICE") != (-1) || check.indexOf("I DONT LIKE YOUR VOICE") != (-1) ||check.indexOf("SPEAK IN MALE VOICE") != (-1) ||check.indexOf("SPEAK IN FEMALE VOICE") != (-1)){
if(session.utils.get("g_voice") == null)
{session.utils.put("g_voice","4");
su.voice = voice[4];
su.text = "Yup, Do I sound good now?";}
else{
var num = parseInt(session.utils.get("g_voice"));
if(num==4){
su.voice = voice[2];
session.utils.put("g_voice","2");
su.text = "Do I sound sweet now ";
}
else{
su.voice = voice[4];
session.utils.put("g_voice","4");
su.text = "Is it fine now";
}
}
speechsynthesis.speak(su);
}
To Play the music define the variable in form of audio as
var snd = new Audio(`data:audio/wav;base64,//uQZAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAWGluZwAAAA8AAEv8Aj....`);
As from my organisation i do not have authorization to access the audio links i gave it in the form of base64 format or else you can give the link inside the audio as
var snd = new Audio(*link to mp3 file*);
To play and pause the audio use the methods snd.play() and snd.pause().
5. Performing actions on Personas elements using voice commands:
The main objective is to perform actions on the personas objects and also execute scripts using the voice commands, for this we get the result in the type of the string and use various validations to know what action needs to be performed.
Start a transaction using the voice command:
We have to include the JavaScript library which we have uploaded in the system the purpose of uploading it is that the function in the script can be used globally on any of the flavor we create.
session.utils.include("0021F60F04171EE68CF23D6564B061FE",false);
The second parameter is specified as false because it needs to be executed only when it called and not before. Write the function to to execute the function on the above file and get the transaction code accordingly.
var num = check.indexOf(" TRANSACTION ");
check = (check.slice((num+12),check.length)).trim();
var value = check.split(" ");
var t_e = transaction_check(value[0],value[1]);
session.callTransaction(t_e);
And in the end we need to refresh the screen so that the changes in the screen gets updated.
sap.personas.scripting.executeScriptInternal({src: ''});
Execute a Script and press a personas button:
To execute a personas script we need to follow the same above given logic with the personas function to perform the required function.
su.text = "Hang on, Running the Sales and Distribution process with default values";
speechsynthesis.speak(su);
session.utils.executeScript("wnd[0]/scrptPersonas_4");
sap.personas.scripting.executeScriptInternal({src: ''});
Translate the flavor:
To perform the translation the flavor has to be already translated using the /n/admin/personas translation feature. The trick is to change the url with the desired language in the parameter sap-language.
First get the determine which language to be changed to and update or construct the url. After that change the url so the flavor gets rendered in the desired url.I have the flavor available in four different languages German, French, Spanish, English.
var translate = function(){
if(check.indexOf("TRANSLATE") != (-1) || check.indexOf("CHANGE THE LANGUAGE") != (-1)|| check.indexOf("CHANGE LANGUAGE") != (-1)){
var url = "http://hostname:port/sap/bc/personas?sap-client=600&sap-language=";
var value = check.split(" ");
for(var i=0;i<value.length;i++)
{console.log(value[i]);
switch(value[i])
{
case "GERMAN":
url+="DE";
break;
case "FRENCH":
url+="FR";
break;
case "SPANISH":
url+="ES";
break;
case "ENGLISH":
url+="EN";
break;
default:
url+="EN";
break;
}
}
window.top.location=url;
}};
Enter Data using Voice Command:
To enter the data check the index of the label name and replace the unnecessary sub-string and paste the data in the relevant input fields.
var data = function(check){
if(check.indexOf("SALES ORGANISATION") != -1){
var num = check.replace( /^\D+/g, '');
session.findById("wnd[0]/usr/ctxtPersonas_1457421438811").text = num;
sap.personas.scripting.executeScriptInternal({src: ''});
su.text = "Okay. Whats Next?";
speechsynthesis.speak(su);
setTimeout(function(){recognizer.start();},2000);
}
If the input field has to be deleted or cleared it be like below
else if(check == "CLEAR ALL" || check == "CLEAR"){
session.findById("wnd[0]/usr/ctxtPersonas_1457421438811").text="";
//Clear rest of the input fields
sap.personas.scripting.executeScriptInternal({src: ''});
su.text = "Okay, Clean as a new slate now";
speechsynthesis.speak(su);
}
And that's it everything is set now and the user can interact with Persona and she will perform what ever action you ask her to perform.
SAP Personas is evolving in a steady manner I hope so sooner or later it will be enabled on mobile device and tablets, during that time also this feature work flawlessly as it requires only chrome to run.
Thank You for reading the blog. Hope I have shared something interesting and useful.
