點燈坊

失くすものさえない今が強くなるチャンスよ

VAD 整合 Vue 與 Webpack

Sam Xiao's Avatar 2024-06-11

VAD 亦可搭配 Vue 與 Webpack 使用,唯 VAD 除了 JavaScript 外,還需要其他 WASM 檔案,需在 Webpack 做額外的設定。

Version

Voice Activity Detection for JavaScript 0.7
Vue CLI 5.0.8
Vue 2.6
Webpack 5.91
copy-webpack-plugin 12.0.2

Install Package

$ npm i @ricky0123/vad-web
$ npm i -D copy-webpack-plugin
  • @ricky0123/vad-web:安裝 VAD for Web 版本
  • copy-webpack-plugin:使用 Webpack 將 VAD 所需檔案複製到 public/js 目錄下

Webpack

vue.config.js

let { defineConfig } = require('@vue/cli-service')
let CopyPlugin = require('copy-webpack-plugin')

module.exports = defineConfig({
  transpileDependencies: true,
  configureWebpack: {
    plugins: [
      new CopyPlugin({
        patterns: [
          {
            from: __dirname + "/node_modules/@ricky0123/vad-web/dist/vad.worklet.bundle.min.js",
            to: __dirname + "/public/js"
          },
          {
            from: __dirname + "/node_modules/@ricky0123/vad-web/dist/silero_vad.onnx",
            to: __dirname + "/public/js/[name].onnx"
          },
          {
            from: __dirname + "/node_modules/onnxruntime-web/dist/*.wasm",
            to: __dirname + "/public/js/[name].wasm"
          },
        ]
      })
    ]
  },
  devServer: {
    devMiddleware: {
      writeToDisk: true,
    },
    headers: {
      "Cross-Origin-Opener-Policy": "same-origin",
      "Cross-Origin-Embedder-Policy": "require-corp",
    }
  }
})
  • 定義 Webpack 與 DevServer 設定

Line 2

let CopyPlugin = require('copy-webpack-plugin')
  • 引用 copy-webpack-plugin

Line 6

configureWebpack: {
  plugins: [
    new CopyPlugin({      
    })
  ]
}
  • 新增 configureWebpack 定義 Webpack 設定
  • 新增 plugins 定義 copy-webpack-plugin Webpack plugin

Line 10

{
  from: __dirname + "/node_modules/@ricky0123/vad-web/dist/vad.worklet.bundle.min.js",
  to: __dirname + "/public/js"
},
  • vad.worklet.bundle.min.js 複製到 public/js 目錄下

Line 14

{
  from: __dirname + "/node_modules/@ricky0123/vad-web/dist/silero_vad.onnx",
  to: __dirname + "/public/js/[name].onnx"
},
  • silero_vad_onnx 複製到 public/js 目錄下

Line 18

{
  from: __dirname + "/node_modules/onnxruntime-web/dist/*.wasm",
  to: __dirname + "/public/js/[name].wasm"
},
  • 將所有 wasm 複製到 public/js 目錄下

Line 27

devMiddleware: {
  writeToDisk: true,
},
  • 使用 npm run serve 時,仍會使用 CopyPlugin 將檔案複製到 public/js 目錄下

Line 30

headers: {
  "Cross-Origin-Opener-Policy": "same-origin",
  "Cross-Origin-Embedder-Policy": "require-corp",
}
  • 避免出現 env.wasm.numThreads is set to 4, but this will not work unless you enable crossOrigin|solated mode. 錯誤訊息
public/js/ort-training-wasm-simd.wasm
public/js/ort-wasm-simd-threaded.jsep.wasm
public/js/ort-wasm-simd-threaded.wasm
public/js/ort-wasm-simd.jsep.wasm
public/js/ort-wasm-simd.wasm
public/js/ort-wasm-threaded.wasm
public/js/ort-wasm.wasm
public/js/silero_vad.onnx
public/js/vad.worklet.bundle.min.js
  • 透過 copy-webpack-plugin 將在 public/js 目錄下增加以上檔案供 VAD 使用

Template

App.vue

<template>
  <div>
    <main class="main">
      <div>VAD Test</div>
    </main>
  </div>
</template>

<script>
import { MicVAD } from "@ricky0123/vad-web"

export default {
  name: 'App',
  data: () => ({
    myVad: null
  }),
  async mounted() {
    this.myVad = await MicVAD.new({
      onSpeechStart: () => {
        console.log("onSpeechStart")
      },
      onSpeechEnd: () => {
        console.warn('onSpeechEnd')
      },
    })
    this.myVad.start()
  }
}
</script>

Line 10

import { MicVAD } from "@ricky0123/vad-web"
  • vad-web 引用 MicVAD

Line 18

this.myVad = await MicVAD.new({
  onSpeechStart: () => {
    console.log("onSpeechStart")
  },
  onSpeechEnd: () => {
    console.warn('onSpeechEnd')
  },
})
this.myVad.start()
  • 使用 MicVAD.new() 建立 myVad object
  • SpeechStart event:當偵測到講話開始時觸發
  • SpeechEnd event: 當偵測到講話結束時觸發

Conclusion

  • Webpack 的 DevServer 預設是將檔案複製到記憶體而已,需搭配 writeToDisk: true 才會將檔案複製到 public/js 目錄下/

Reference

Ricky Samore, Voice Activity Detection for JavaScript