Hey, Scratchers!

We have some exciting news to share with you: Face Sensing is here! Face Sensing blocks allow you to make games and costumes that interact with your face using your device’s camera. Give yourself a silly moustache, draw a picture with your nose, or catch virtual donuts with your mouth: the sky’s the limit.



If you’ve ever used a face filter, used your nose to guide a ball through a virtual maze, or used an app to see what glasses or a new hairstyle would look like on your face, you’ve used the same type of technology that powers Face Sensing.

Face Sensing blocks have been trained to recognize faces, so they can see where features like your nose, ears, and eyes are on your head. You can use those parts of your face to perform actions (make a sprite disappear when it hits my mouth!), guide sprites or costumes (make this hat go to the top of my head!), or anything else you can imagine.



This extension keeps your personal information private: images never leave your device. Just like Video Sensing blocks, the only person who can see the images from your webcam is you. (Want to learn more about this? Check out the FAQ below!)

We can’t wait to see what you create with Face Sensing! Check out these resources to help you create with these new blocks:
Face Sensing Starter Project & Tutorial
Face Sensing Coding Cards

Scratch On!
The Scratch Team

- - - -

FAQ

Is Face Sensing safe and private?
Yes – the images from your webcam are never transmitted outside of your device. Face Sensing blocks detect faces using a technology called “machine learning” (a subset of AI that enables machines to learn from data), but it is not training on your data while you use it. It has already been trained to see faces using a large number of photos of faces.

Was Face Sensing tested in Scratch Lab?
Yes! You may remember Face Sensing blocks from Scratch Lab, the space for the Scratch Team to test experimental blocks. Your feedback in Scratch Lab helped us improve the way these blocks work before this new feature was ready for release—thank you!

If I use the “Set Thumbnail” button while my face is on-screen, will it capture an image of my face?
When using the Face Sensing extensions, only your computer can sense your face or video motion. None of your data is stored or sent to Scratch or any other site, and your camera view is not captured when setting your project thumbnail, so your data remains private.

Does Face Sensing work on all devices?
Our goal is to make Face Sensing blocks work on all devices and web browsers supported by Scratch. Currently, Face Sensing blocks may not work on some Android and iOS devices. They may run slowly on some Chromebooks or older computers.

I changed my mind about using Face Sensing blocks in my project, but the video feed is still on my screen. How can I get rid of it?
No problem! If you no longer want to use the Face Sensing extension blocks in your project, remove all of the extension's blocks from the script area (drag blocks to the block palette to delete). Make sure to turn off any stage monitors by ensuring all checkboxes are unchecked in the blocks palette (next to blocks like “face tilt“), too. Save your project. Then, reload it by refreshing the browser or closing and reopening the project. If you want to remove the video feed without removing the Face Sensing blocks, use the ”turn video on/off" block from the Video Sensing extension.

What can I do if Face Sensing feels slow or laggy?
Face Sensing may run slowly on some devices. You can try closing other applications and browser tabs. If you are using your webcam in another app while Face Sensing is running—for example, a video call—try turning off your camera in that app.

What if my camera is not working?
When you load Face Sensing blocks, you should see video from your webcam on the Scratch stage. If it doesn’t appear, you may need to give Scratch permission to use your camera. Click “allow” when Scratch prompts for permission to use the camera, or check your web browser’s settings.

Can I use Face Sensing blocks without a camera?
While you can see and edit the code inside Face Sensing projects without a camera, you’ll need to use your device’s camera in order to use Face Sensing functions.

Can my action figure/stick figure drawing/plush toy use Face Sensing blocks, too?
Face Sensing blocks can detect faces in photos, and even in drawings. Try holding different things with faces up to your camera and see what happens!

Why don’t Face Sensing blocks see my face?
The technology behind Face Sensing is not perfect. It is best at seeing faces that are brightly lit and close to the camera. It will often recognize faces even if they are partially hidden by a mask, glasses, hat, headscarf, or other covering. It does not do well at seeing faces that are far away, in the dark, or hidden.

I made a Face Sensing project in Scratch Lab. Can I share it on Scratch?
Yes! You can upload and share any Scratch Lab projects you’ve created using Face Sensing blocks. As soon as these blocks are live, your Scratch project will work the same as it does in Scratch Lab. Just choose File>Load from My Computer in the project editor.

When will Face Sensing be available in languages other than English?
Scratch’s volunteer translators are working on translating Face Sensing blocks into multiple languages. This work is still in progress, and we hope to share more with you soon.

When will Face Sensing be available offline?
Our team is hard at work making Face Sensing blocks available for downloadable Scratch Apps. We’ll be sure to share an update with you as soon as we have it.