生成由 AI 驱动的注视追踪脸部图像,并创建可交互操作的 React 组件,以跟随 Cursor。
https://github.com/user-attachments/assets/c8684ae4-3009-47cb-8ad4-0e7af65a608f
Generate a grid of face images with different gaze directions using AI, then use them to create an interactive face that follows the user's cursor in real-time.
This project has two parts:
Perfect for creating engaging portfolio headers, interactive avatars, or fun UI elements!
You have two options to generate the face images:
The easiest way to get started is using the dedicated Replicate model created in collaboration with fofr. Simply upload your face image and it will automatically generate:
To use it:
The model automatically uses the Expression Editor model to generate all the images you need. See the model page for more details.
If you prefer more control over the generation process or want to customize the parameters, you can use the Python script described below.
# Clone or download this repository
cd face_looker
# Create a virtual environment (recommended)
python -m venv .venv
source .venv/bin/activate # On Windows: .venv\Scripts\activate
# Install dependencies
pip install -r requirements.txtexport REPLICATE_API_TOKEN=your_token_hereOr add it to your .bashrc/.zshrc for permanent use:
echo 'export REPLICATE_API_TOKEN=your_token_here' >> ~/.zshrc
source ~/.zshrcImage Requirements:
Where to put your image:
Any filename works! You can use any name like me.jpg, portrait.png, selfie.jpeg, etc.
Any location works! You can put it in the project folder or anywhere on your computer
Example file structure:
face_looker/
├── main.py
├── my_face.jpg ← Your image can go here
├── requirements.txt
└── README.mdOr anywhere else:
/Users/you/Pictures/portrait.png
/path/to/any/folder/selfie.jpgBefore running the script, make sure you have:
REPLICATE_API_TOKEN environment variablepip install -r requirements.txt** How many images will be created?**
The default settings create 121 images (11×11 grid), but you can customize this:
| Step Size | Grid Size | Total Images | Generation Time | Smoothness |
|---|---|---|---|---|
--step 5 |
7×7 | 49 images | ~2-3 minutes | Basic |
--step 3 (default) |
11×11 | 121 images | ~5-8 minutes | Good |
--step 2.5 |
13×13 | 169 images | ~8-12 minutes | Smooth |
--step 2 |
16×16 | 256 images | ~12-18 minutes | Very smooth |
Basic usage:
# Any filename works!
python main.py --image ./my_face.jpg --out ./out
python main.py --image ./portrait.png --out ./out
python main.py --image /Users/you/Pictures/selfie.jpg --out ./outCustom grid density:
# Smoother transitions (more images, longer generation time)
python main.py --image ./my_face.jpg --out ./out --step 2.5
# Faster generation (fewer images, less smooth)
python main.py --image ./my_face.jpg --out ./out --step 5Full options:
python main.py \
--image ./my_face.jpg \ # Input image path or URL
--out ./out \ # Output directory
--min -15 \ # Minimum gaze value
--max 15 \ # Maximum gaze value
--step 3 \ # Step size (smaller = more images)
--size 256 \ # Output image size (256x256)
--skip-existing # Skip already generated images| Parameter | Default | Description |
|---|---|---|
--min |
-15 | Minimum pupil position (left/up) |
--max |
15 | Maximum pupil position (right/down) |
--step |
3 | Grid spacing (smaller = smoother, more images) |
--size |
256 | Output image dimensions (256×256) |
Image count formula: ((max - min) / step + 1)²
Examples:
step=3 (default): 121 images (11×11 grid)step=2.5: 169 images (13×13 grid)step=5: 49 images (7×7 grid)The script generates:
gaze_px{X}_py{Y}_256.webp filesgaze_px0_py0_256.webp (looking at center)gaze_px15_py0_256.webp (looking right)gaze_px0_pym15_256.webp (looking up)index.csv mapping filenames to coordinatesout/
├── gaze_px-15_py-15_256.webp
├── gaze_px-15_py-12_256.webp
├── ...
├── gaze_px15_py15_256.webp
└── index.csv# Copy all face images to your React public folder
cp -r ./out/faces /path/to/your-react-app/public/faces# Copy the hook and component
cp useGazeTracking.js /path/to/your-react-app/src/hooks/
cp FaceTracker.jsx /path/to/your-react-app/src/components/
cp FaceTracker.css /path/to/your-react-app/src/components/import FaceTracker from './components/FaceTracker';
function App() {
return (
);
}import FaceTracker from './components/FaceTracker';
function Header() {
return (
);
}For more control, use the useGazeTracking hook directly:
import { useRef, useEffect } from 'react';
import { useGazeTracking } from './hooks/useGazeTracking';
function CustomFaceComponent() {
const containerRef = useRef(null);
const { currentImage, isLoading, error } = useGazeTracking(
containerRef,
'/faces/'
);
return (
);
}If you change the generation parameters, update these constants in useGazeTracking.js:
// Must match your generation parameters!
const P_MIN = -15; // Same as --min
const P_MAX = 15; // Same as --max
const STEP = 3; // Same as --step
const SIZE = 256; // Same as --size/* FaceTracker.css */
.face-tracker {
width: 100%;
height: 100%;
position: relative;
overflow: hidden;
background: #f0f0f0;
border-radius: 50%; /* Circular face */
box-shadow: 0 10px 30px rgba(0,0,0,0.2);
}
.face-image {
user-select: none;
pointer-events: none;
}
.face-debug {
position: absolute;
top: 10px;
left: 10px;
background: rgba(0, 0, 0, 0.7);
color: white;
padding: 10px;
border-radius: 5px;
font-family: monospace;
font-size: 12px;
}useEffect(() => {
// Preload all face images
const images = [];
for (let py = -15; py <= 15; py += 3) {
for (let px = -15; px <= 15; px += 3) {
const img = new Image();
img.src = `/faces/gaze_px${px}_py${py}_256.webp`;
images.push(img);
}
}
}, []);--step value (e.g., --step 5)The component automatically supports touch events! The face will follow finger movement on mobile devices.
Check your API token:
echo $REPLICATE_API_TOKENVerify Replicate credits:
Resume interrupted generation:
python main.py --image ./my_face.jpg --out ./out --skip-existingVerify images are in the correct location:
public/faces/ directory existsCheck browser console for errors:
Verify configuration matches:
P_MIN, P_MAX, STEP in useGazeTracking.js must match generation parameters--step 5 or higher)--size 128)Replicate charges per second of GPU time, it should only be a couple of cents.
Check current pricing at replicate.com/pricing
See the examples folder for:
Contributions welcome! Feel free to:
MIT License - feel free to use in personal and commercial projects!
Questions? Open an issue or contact @kylancodes on X
Like this project? Give it a ⭐ on GitHub!
暂无开放 Issues,或尚未同步最近议题。