Earlier this month at BUILD 2017, we announced the public preview of Video Indexer as part of Microsoft Cognitive Services. Video Indexer enables customers with digital and audio content to automatically extract metadata and use it to build intelligent innovative applications. You can quickly sign up for Video Indexer from https://vi.microsoft.com/ and try the service out for free during our preview period.
On top of using the portal, developers can easily build custom apps using the Video Indexer API. In this blog, I will walk you through an example of using the Video Indexer API to do a search on a keyword, phrase, or detected person’s name across all public videos in your account as well as sample videos and then to get the deep insights from one of the videos in the search results.
Getting Access to the Video Indexer API
To get started with the Video Indexer API, you must sign in using a Microsoft, Google, or Azure Active Directory account. Once signed in with your preferred account, you can easily subscribe to our free preview of the Video Indexer API. The following steps will walk you through the process of registering for access.
To subscribe to the API, go to the Products tab and click Free Preview. On the next page, click the Subscribe button. You should now have access to the API. If you find that you do not have access, contact visupport@microsoft.com.
After getting access, you can then return to the Products tab, and go to the Video Indexer APIs – Production link.
You should now see the Video Indexer API documentation page. On the left side of the page, you will see a list of several action options. Each action page contains information about that request including which parameters are optional and which ones are required. You can test any of these by clicking Try it, setting the appropriate parameters, and then clicking Send.
To use an external tool like Postman to test the API, you will need to download the Video Indexer APIs – Production Swagger .json file. You can do this by selecting the API definition download button on the top right of the page and choosing Open API to get the Swagger .json file. Save the file somewhere locally on your machine to use in the next section.
Here, I will demonstrate how to use Postman to test the API. To follow along, you can download and install Postman here. Launch Postman and click Import in the top left.
Navigate to and choose the Video Indexer APIs – Production Swagger .json file that you previously downloaded and saved locally.
You should now see the API actions under Collections.
To submit calls to any of the actions, you need a key that is specific to your subscription. This can be found by going back to Video Indexer APIs – Production page and clicking Try it. Once it gets to the next page, you can scroll down to the Headers section and find where it says Ocp-Apim-Subscription-Key. You can see your key by clicking on the eye icon.
Copy both the name of the key (Ocp-Apim-Subscription-Key) and the key itself because you will need both for Postman.
Running a Search Call Across Videos
Going back to Postman, go to the action call that you want to test out. In this case, I will start with search, which is going to be a typical user interaction with the API. In particular, I’m showing a search on a keyword across all public videos in your account as well as sample videos. Search is an HTTP GET call with the request URL https://videobreakdown.azure-api.net/Breakdowns/Api/Partner/Breakdowns/Search
Go to Headers and enter in the name of the key (Ocp-Apim-Subscription-Key) where it says key and the key itself where it says value. You can set a Header Preset on Postman using the key and value to prevent having to type them in every time. It saves time and is easy to set up, so it is definitely worth doing!
To set the parameters of the action, click Params and set the values for the parameters you wish to set. Remove the unchanged parameters by hovering to the right corner of the parameter and selecting the x.
In this search example, I’m setting the privacy to “Public”, language to “English”, textScope to “Transcript”, and searchInPublicAccount to “true”. I am also clearing out all the parameters that I have not changed. For query, enter in the word that you would like to search for across the videos. In this example, let’s search for the keyword “Azure”.
Upon selecting Send, you will get a JSON response with the results of the search.
The JSON response of the search contains a results section that gives back the videos that contain your query term and the relevant time ranges from each resulting video. The section is an array in which each element is a resulting video along with its basic information, social likes, number of views, and search matches with start times.
Below is an outline of some of what to expect in the JSON response for search.
JSON Response for Search
Results (Array – each element has the following info)
Basic video and user information
accountId
id
name
description
userName
createTime
privacyMode
state
social
likes
views
searchMatches (Array – each element has the following info)
startTime
type (tells the user whether match is from audio based transcript or OCR)
text
You should also test this out by uploading and processing a few of your own videos in your account. You can do this on the Video Indexer Preview site if you are logged in, or you can use the Upload HTTP POST call from the Video Indexer API in Postman. For your search request, set searchInPublicAccount to “false” to only search through the videos on your account. Set the query to a keyword that is more relevant to your videos and privacy to either “Public” or “Private” based on the settings of your video.
Next, I will show how to take the results of a search and get the expanded insights of the video.
Running a Breakdown Call on a Video
Take the id of the first result of your search.
Now go to the breakdown action. Breakdown is the HTTP GET call with the request URL https://videobreakdown.azure-api.net/Breakdowns/Api/Partner/Breakdowns/:id
You will need to put in your subscription key name and key again. If you have a preset set up with the key, you will just need to select it.
Click the Params button and enter in the id from the search result for the id parameter. Set the language of the breakdown to “English” and click Send.
You should now see the JSON response for the breakdown request.
The JSON response of the breakdown contains general information on the video and the account that uploaded it in addition to three sections called summarizedInsights, breakdowns, and social.
The summarizedInsights section holds information on the distinct faces, topics, and audio effects in the video as well as the different time ranges in which each appear. In addition, the section provides information on positive, negative, and neutral sentiments throughout the video as well as the time ranges for each.
The breakdowns section serves as a more expansive version of the summarizedInsights. Here, you will find transcript blocks, categories of audio effects, and information to allow for content moderation. The breakdowns section also provides more details on topics, faces, and voice participants of the video.
The trasncriptBlocks section within breakdowns serves as a timeline of the video. You will find information on lines, OCRs, faces, etc. for each time block. The social section provides data on likes and number of views.
Below is an outline of some of what to expect in the JSON response for breakdown.
JSON Response for Breakdown
Basic video and user information
This section has extensive information on the name, owner, id, etc. of the video.
summarizedInsights
faces
topics
sentiments
audio effects
breakdowns
general information on video
accountID
id
state
processingProgress
externalURL
insights
trancsriptBlocks
topics
faces
contentModeration
audioEffectsCategories
social
likes
viewsYou can get a more information on the JSON response for breakdown here.You have the data and are now well on your way to having more insights on your videos and the opportunity to further innovate. Try a few more examples with your own content!
For more details, please take a look at the Video Indexer Documentation. Follow us on Twitter @Video_Indexer to get the latest news on the Video Indexer.
If you have any questions or need help, contact us at visupport@microsoft.com.
Quelle: Azure
Published by