sesame-robot

# The Sesame Robot Project ___

___ **Greetings, from your new best friend.** Sesame is an accessible Open-Source robotics project based on the ESP32 microcontroller system, with an emphasis on expression and movement. This project is designed for makers and engineers of all skill levels! Sesame offers a dynamic platform designed to start working with walking robots. To build a sesame robot, you will need basic soldering skills, $50-60 in hardware components, access to a 3D printer, and a basic understanding of Arduino IDE. This repository contains the CAD design files, STL files, build and wiring guides, and the base/expanded firmware for the ESP32-based controller. There is also some included debugging firmware that may be helpful in getting your Sesame up and running. ## Features * **Quadruped Design:** Uses 8 servo motors (2 per leg) to achieve roughly 8 total degrees of freedom. * **Emotive Display:** Features a 128x64 OLED screen acting as a reactive face that syncs with movement. * **Fully Printable:** Designed entirely for 3D printing in PLA with minimal supports. * **Network Connectivity:** Connect to your WiFi network for remote control and API access. * **JSON API:** RESTful API for programmatic control from Python, JavaScript, and more. * **Conversational Faces:** Expressive emotion library with talk variants for voice assistant projects. * **Sesame Studio:** New animation composer software to easily create custom movements. * **Sesame Companion App:** Python application for voice control and advanced interactions. * **Serial CLI:** Control the robot and trigger animations via a Serial Command Line Interface or the web UI. * **Pre-programmed Emotes:** Includes animations for Walking, Waving, Dancing, Pointing, Resting, and more. ## Watch the launch video on YouTube ___ ## Getting Started Follow these steps to build your own Sesame Robot: ### 1. Gather Parts Check the **[Bill of Materials (BOM)](hardware/bom/README.md)** for a complete list of required electronics and hardware. * Microcontroller: Lolin S2 Mini (recommended for DIY builds), Sesame Distro Board V3 (Current, pre-flashed, supports Bambu Lab battery), V2 (legacy, USB-only), or ESP32-DevKitC-32E with Distro Board V1 (legacy) * Actuators: 8x 180 Degree MG90 Servos * Power: 5V 3A source (USB-C PD for S2 Mini and V2 Distro Board, or battery + buck converter; see BOM for the Bambu Lab 14500 7.4V 800mAh Li-ion Battery option) ### 2. Print Parts Download the STLs and follow the **[Printing Guide](hardware/printing/README.md)**. * Designed for PLA * Minimal supports required ### 3. Build & Wire Follow the **[Build Guide](docs/build-guide/README.md)** and **[Wiring Guide](docs/wiring-guide/README.md)** to assemble the frame and connect the electronics. ### 4. Flash Firmware Upload the code from the **[Firmware Directory](firmware/README.md)**. * Requires Arduino IDE * Configure WiFi AP settings ### 5. Create Animations Use **[Sesame Studio](software/sesame-studio/README.md)** to visually design poses and sequences for your robot. --- ## Software & Firmware ### Sesame Studio Sesame Studio is a standalone desktop application included in `software/sesame-studio/`. It allows you to: * Visually pose the robot using a schematic interface. * Generate C++ code for servo angles automatically. * Sequence frames into complex animations. [**> Go to Sesame Studio**](software/sesame-studio/README.md) ### Sesame Simulator The Sesame Simulator, created by Jay Li, is a Rust-based 3D simulation environment for testing Sesame's movements and kinematics in a virtual space. It features: * **Physics-based Simulation:** Test walking and balance without hardware. * **Web-based Interface:** Run the simulator directly in your browser. * **URDF Integration:** Accurate modeling of Sesame's physical properties. [**> Go to Sesame Simulator**](https://one-for-all.github.io/sesame-robot-sim/) ### Sesame ML Tools ML tools Created by Luke Hollis for Sesame, world model for online planning, policy eval, and RL finetuning. * **Robot description:** Self-contained printable STLs, CAD-derived MJCF, validated URDF, camera/IMU/foot frames, primitive contact geometry, masses and actuator limits * **Tasks:** Stand, challenged recovery, commanded locomotion, and goal/obstacle navigation * **Policy evaluation:** Multi-seed JSON/CSV reports, strict task success, domain parameters, tracking/fall/collision/energy/latency metrics, MP4/GIF and onboard/split views [**> Go to Sesame ML Repo**](https://github.com/lukehollis/sesame-ml) ### Sesame Companion App The Sesame Companion App is a Python-based application that enables advanced control and interaction with your robot over your local network. It leverages the new JSON API and network mode features to provide: * **Voice Assistant Integration:** Control Sesame with voice commands and see real-time emotional expressions. * **Remote Control:** Command your robot from anywhere on your local network. * **Face Control:** Change expressions dynamically based on conversation or context. * **API Examples:** Reference implementation for building your own integrations. The Companion App works with robots running the latest firmware with network mode enabled. [**> Go to Sesame Companion App Repository**](https://github.com/dorianborian/sesame-companion-app) ### Firmware The ESP32 firmware (`sesame-firmware-main.ino`) handles the kinematics, face display, and WiFi control interface. * **Web UI:** Control the robot from your phone via the built-in Access Point. * **Custom Faces:** Add your own bitmaps (guide in firmware docs). [**> Go to Firmware Docs**](firmware/README.md) --- ## Contributing This robot is a platform for building new features, cosmetics, tools, and ideas. Since the current firmware is a basic implementation, pull requests are very welcome for: * Kinematics improvements * New animations * Improved Web UI/UX * Sensor integration (Ultrasonic, Gyro, etc.) I would also love to see forks of this project with new hardware, software, faces, etc. Be sure to send me a message if you end up building one, and I might feature you on my website or channel! --- *Created by [Dorian Todd](https://www.doriantodd.com/). Need help with your Sesame Robot? Send me a message on Discord, my username is "starphee"*