Model: MediaPipe FaceMesh

A person making various face gestures, with a wireframe mesh laid on top of the face
handsfree.update({facemesh: true})
  • 468 2D face landmarks
  • Track up to 4 faces at once
  • Extra helpers and plugins coming soon

This model doesn’t come with any bonuses or plugins yet but they’ll come soon. The API will remain exactly the same, so feel free to started with this model today!

Usage

With defaults

const handsfree = new Handsfree({facemesh: true})
handsfree.start()

With config

const handsfree = new Handsfree({
  facemesh: {
    enabled: true,
    // The maximum number of faces to detect [1 - 4]
    maxNumFaces: 1,

    // Minimum confidence [0 - 1] for a face to be considered detected
    minDetectionConfidence: 0.5,
    
    // Minimum confidence [0 - 1] for the landmark tracker to be considered detected
    // Higher values are more robust at the expense of higher latency
    minTrackingConfidence: 0.5
  }
})

handsfree.start()

Data

// faceIndex [0 - 3] An array of landmark points for each detected face
handsfree.data.facemesh.multiFaceLandmarks[faceIndex] == [
  // Landmark 0
  {x, y},
  // Landmark 1
  {x, y},
  // ...
  // Landmark 467
  {x, y}
]

// face 0, landmark 0
handsfree.data.facemesh.multiFaceLandmarks[0][0].x
handsfree.data.facemesh.multiFaceLandmarks[0][0].y

Examples of accessing the data

handsfree = new Handsfree({facemesh: true})
handsfree.start()

// From anywhere
handsfree.data.facemesh.multiFaceLandmarks[0]

// From inside a plugin
handsfree.use('logger', data => {
  if (!data.facemesh) return

  console.log(data.facemesh.multiFaceLandmarks[0][0])
})

// From an event
document.addEventListener('handsfree-data', event => {
  const data = event.detail
  if (!data.facemesh) return

  console.log(data.facemesh.multiFaceLandmarks)
})

Projects

The following projects all use MediaPipe Facemesh, however, they weren’t all necessarily done with Handsfree.js:

No results found.

Menu