Model: MediaPipe Pose

A person flailing their arms and legs, with a skeleton overlaid above their arms, torso, legs, and face
handsfree.update({pose: true})
  • Full body mode with 33 2D pose landmarks
  • Upper body mode with 25 2D upper pose landmarks
  • Extra helpers and plugins coming soon

This model doesn’t come with any bonuses or plugins yet but they’ll come soon. The API will remain exactly the same, so feel free to started with this model today!

Usage

With defaults

const handsfree = new Handsfree({pose: true})
handsfree.start()

With config

const handsfree = new Handsfree({
  pose: {
    enabled: false,
    
    // Outputs only the top 25 pose landmarks if true,
    // otherwise shows all 33 full body pose landmarks
    // - Note: Setting this to true may result in better accuracy 
    upperBodyOnly: false,

    // Helps reduce jitter over multiple frames if true
    smoothLandmarks: true,

    // Minimum confidence [0 - 1] for a person detection to be considered detected
    minDetectionConfidence: 0.5,

    // Minimum confidence [0 - 1] for the pose tracker to be considered detected
    // Higher values are more robust at the expense of higher latency
    minTrackingConfidence: 0.5
  }
})

handsfree.start()

Data

Pose Landmarks

// An array of landmark points for the face
handsfree.data.pose.poseLandmarks == [
  // Landmark 0
  {x, y, visibility},
  // Landmark 1
  {x, y, visibility},
  // ...
  // Landmark 32
  {x, y, visibility}
]

// landmark 0
handsfree.data.pose.poseLandmarks[0].x
handsfree.data.pose.poseLandmarks[0].y
// The confidence in this pose landmark
handsfree.data.pose.poseLandmarks[0].visibility

Examples of accessing the data

handsfree = new Handsfree({pose: true})
handsfree.start()

// From anywhere
handsfree.data.pose.poseLandmarks

// From inside a plugin
handsfree.use('logger', data => {
  if (!data.pose) return

  console.log(data.pose.poseLandmarks)
})

// From an event
document.addEventListener('handsfree-data', event => {
  const data = event.detail
  if (!data.pose) return

  console.log(data.pose.poseLandmarks)
})

Projects

The following projects all use MediaPipe Pose or TensorFlow PoseNet, however, they weren’t all necessarily done with Handsfree.js:

Menu