[Day 7] Google Video Intelligence AI - 2

Published:

今天又是美好的開始，大家吃中秋烤肉了嗎？
這是這系列的第二篇文章，要來入門Video Intelligence API，這隻可以透過上傳影片，但實際上可以幹嘛呢？實際上包含下列幾個項目的偵測：

  • Label Detection: 偵測狗、花、人物之類的物件
  • Shot Change Detection: 可以偵測場景轉換
  • Explicit Content Detection: 偵測是否包含成人資訊
  • Speech Transcription: 將影片裡的聲音轉成文字
  • Object Tracking: 物件追蹤並回報物件在影片裡的位置

現在就來開始測試看看。

前情提要：記得先Enable API，放置環境變數的教學可以看這系列第三天的文章
語言一樣使用Golang，然後跑在docker裡，之後也會放上github

為了一致性，我就開一個module video，然後專門放video Intelligence API的code。來看看我video.go

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
package video

import (
  "context"
  "fmt"
  "io"
  "log"

  "github.com/golang/protobuf/ptypes"

  video "cloud.google.com/go/videointelligence/apiv1"
  videopb "google.golang.org/genproto/googleapis/cloud/videointelligence/v1"
)

func DemoCode(w io.Writer, file string) error {
  ctx := context.Background()

  // Creates a client.
  client, err := video.NewClient(ctx)
  if err != nil {
    log.Fatalf("Failed to create client: %v", err)
    return nil
  }

  op, err := client.AnnotateVideo(ctx, &videopb.AnnotateVideoRequest{
    InputUri: file,
    Features: []videopb.Feature{
      videopb.Feature_LABEL_DETECTION, // Feature_LABEL_DETECTION
    },
  })
  if err != nil {
    log.Fatalf("Failed to start annotation job: %v", err)
    return nil
  }

  resp, err := op.Wait(ctx)
  if err != nil {
    log.Fatalf("Failed to annotate: %v", err)
    return nil
  }

  // Only one video was processed, so get the first result.
  result := resp.GetAnnotationResults()[0]

  for _, annotation := range result.SegmentLabelAnnotations {
    fmt.Fprintf(w, "Description: %s\n", annotation.Entity.Description)


    for _, category := range annotation.CategoryEntities {
      fmt.Fprintf(w, "\tCategory: %s\n", category.Description)
    }

    for _, segment := range annotation.Segments {
      start, _ := ptypes.Duration(segment.Segment.StartTimeOffset)
      end, _ := ptypes.Duration(segment.Segment.EndTimeOffset)
      fmt.Fprintf(w, "\tSegment: %s to %s\n", start, end)
      fmt.Fprintf(w, "\tConfidence: %v\n", segment.Confidence)
    }
  }
  return nil
}

主要是照著Quickstart: Using Client Libraries的demo code來改，我主程式的地方import module以後就呼叫video.DemoCode(os.Stdout, "gs://cloud-samples-data/video/cat.mp4")，仔細觀察code裡面會看到有個Feature_LABEL_DETECTION這就是這邊主要要偵測的資訊。

好，來看看output吧。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
Description: pet
        Category: animal
        Segment: 0s to 14.84s
        Confidence: 0.83893955
Description: tabby cat
        Category: cat
        Segment: 0s to 14.84s
        Confidence: 0.30844954
Description: maine coon
        Category: cat
        Segment: 0s to 14.84s
        Confidence: 0.32171762
Description: whiskers
        Segment: 0s to 14.84s
        Confidence: 0.30017045
Description: animal
        Segment: 0s to 14.84s
        Confidence: 0.9441419
Description: kitten
        Category: cat
        Segment: 0s to 14.84s
        Confidence: 0.36152288
Description: small to medium sized cats
        Category: mammal
        Segment: 0s to 14.84s
        Confidence: 0.7987513
Description: cat
        Category: pet
        Segment: 0s to 14.84s
        Confidence: 0.99747396

針對緬因貓 maine coon的判別不是很確定，但很確定這裡面都是貓，不完全是小貓，屬於偏中型尺寸的貓。想追根究柢到底正不正確嗎？
下載cat.mp4: gsutil -m cp gs://cloud-samples-data/video/cat.mp4 .

這只是一個14秒的一隻貓的影片，算是給我們的小故事大啟示！好，今天的文章就先到這邊，這是今天的code的github，明天再來玩點其他API吧。