Natural Language Understanding: Setup

This is video 02_02. Here we are on the IBM Cloud dashboard. The first Watson machine learning offering we're going to work with is one for natural language recognition. They have a couple of possible services. Let's take a look at what there is. So I'm going to hit Create Resource. And when that loads. We'll look at AI / Machine Learning. And one option is the watsonx Assistant. This includes natural language and is also a framework to create a complete bot. Another option is Natural Language Understanding. And here it is. This allows us to just do natural language models. Since we only want a natural language understanding model, his is what we will select. The location where it will run can be left as is or you can select one closer to you. I'm going to leave it as is. We can also use the free, otherwise known as the Lite plan. Finally, we're going to change the service name. So if we scroll down we'll see the service name and we're going to make it. "LinkedInNaturalLanguage". And we'll need to just accept the terms here and press Create. The easiest way to train the service is to use a CSV file. And we've got one created for us as part of the collateral for this chapter. So let's go to the desktop. So on the desktop I've got our code for this chapter. So I'll go into it and there's a new folder called "IBMWatsonCollateral". And inside that is a CSV file for our natural language trainer. And let's take a look at it. So I'll open with and go to TextEdit. And here we can see that we have rows with two pieces of data a sample phrase and a category. We give it about ten samples for each category, and the classification process will do the rest. Here we see we have categories for greetings, help, visual recognition, price prediction and none. The "none" category is for things we don't understand the intent of. We can use the curl command to send our CSV file to the service to be trained inside a terminal window. And for this I have a command that I'm just going to paste straight in here. And this is the curl command to send up our CSV file into our service and get trained for a model. Under the free service we can only train one model. So this is the one we're going to train. And the first piece of information that we need is that API key. And the API key can be found back in the browser. So I'm going to switch back to the browser real quick and we're going to grab that for the service. So if we go back to the browser we can go to Resource List. And here's our list of resources, we can see under AI / Machine Learning we've got our new "LinkedInNaturalLanguage". And we'll open that up. And here's our API key. So I'm going to copy that. Go back into the terminal window and we replace this little placeholder with the API key. And we can see the content type that we're sending to this post is "multipart/form-data". And here we've got the form data that we actually are sending for the training data. And that's pointing to the CSV file on the disk. And we're giving a type of "text/csv". Also part of the form data is we're passing in the language being English, and also giving a name to the model and the name we're giving the model is "LinkedInClassifier". And the final piece of information we need is the URL for the service that we're training that matches up with that API key. And we can go back into the browser to get that as well. So I'll go back into Google Chrome. And right underneath the API key is that URL. So I'll copy it. Go back into the terminal window. And we'll paste this in. And it'll send up that CSV file. And very quickly it's going to come back with return that has the model ID. And the model ID uniquely identifies this model. And it's how we're going to call it in the future for this particular train model. Now this model isn't trained quite yet and we can see what the status is, is that the status rate now is starting. Right. So this is basically where it is in the training process. So it's not a trained model yet. We can't actually use it. So in order to find out if it's finally been trained there is a command that we can use for that as well. So this is going to be a curl command. And I'm going to paste this right in here. And we need the API key. And so I'll just copy it right from above. You could go back to the browser to find this if you didn't remember it. But we've got it right here inside the terminal window. So I will replace out the API key just like that. And also we need the URL, and we can go get that above. And it'll start with this "https" and go all the way to right before the "v1". So we'll grab that and replace out this placeholder for the URL. And the final piece of information we need is the model ID. And the model ID, as we just discussed, came back from that initial return. So we can see it right here. We've got the model ID and we'll grab that just like this and replace out this placeholder model ID. And here it is. We can see the status is "training". So here it is right here. Status being "training"... We can see the type of the model is a classification. And this might take several minutes for it to happen. And you can actually run this command several times and just wait for the status to be done. Now eventually you're going to refresh the model and you'll see the status will move out of training to "deploying" and finally "standby". And when you get that standby status as we got here, that's when you know that the model has been completely trained and is ready for us to use. And in order to test it, we've also got a command for that. So I've got a curl command to test the model. And I'll just paste this in just like this. And we can see we've basically got those same three parameters that we've been using. So the first being the API key. Once again I'm just going to grab it from the browser above. And here's our API key. Just like that we will copy it and replace this out. And also we've got our URL. I'll take that just like before. Up to the "v1" and replace out the URL placeholder. And the final piece of information that we need is the model ID. And we'll just grab that from above as well and put this into the model ID. And there's a final little bit of text that we need here, which is actually what the classification messages were going to send to it. And after the model ID we'll put that in. And what we're looking for is what's in this picture. So we're trying to classify that piece of text. So we're saying we're doing a classification on this model. And we're looking for what class is the "what's in this picture?" phrase a part of. And here we get a result almost immediately. And it's giving a "confidence" on each one of our classes. And right now this 0.90 is saying that it's 90% positive that what that particular text was looking for was what we called a class of visual recognition, which is absolutely correct. And with that, we have our trained model for natural language understanding, and we could potentially deploy it and use it in our app.