The music video and audio used in this project are from the iconic Bad Apple!! Touhou fan video:

Bad Apple!! feat.nomico — Alstroemeria Records
MV uploaded by kasidid2

All rights to the original music and video belong to their respective owners. This project is for personal/educational use only.

The SSD1306 is a 1-bit display — every pixel is either on or off. Each frame of the video is stored as raw binary: 128×64 pixels at 1 bit per pixel = exactly 1024 bytes per frame. All frames are packed back-to-back into a single bad_apple.bin file with no headers or delimiters.

To read frame N, the ESP32 seeks to byte N × 1024 in the file and reads 1024 bytes straight into a buffer, then pushes it to the display. No parsing needed.

The MP3 file is stored separately on the SD card and decoded in real time using the ESP8266Audio library, outputting to the MAX98357A I2S DAC.

Both audio and video start from the same millis() reference point. Instead of incrementing frames one by one, the ESP32 calculates the expected frame based on elapsed time — so if rendering falls behind, frames are skipped to catch up, keeping video locked to wall time and in sync with audio.

Audio decoding runs on Core 0 as a FreeRTOS task, while video rendering runs on Core 1 (the default Arduino loop core). This ensures the audio buffer stays fed even when the display is busy pushing a frame over I2C.

⚠️ The SD_MODE pin on the MAX98357A should be pulled to 3.3V through a 100kΩ resistor for normal operation. Most breakout boards (e.g. Adafruit's) already do this onboard.

⚠️ GPIO 34 is input-only and has no internal pullup. Connect a 10kΩ resistor between SW and 3.3V externally.

In PlatformIO or Arduino IDE, install:

• Adafruit SSD1306
• Adafruit GFX Library
• earlephilhower/ESP8266Audio

bad_apple.bin is already provided in this repo — no conversion needed. Just use it as-is.

If you want to regenerate it yourself for any reason, you'll need Python, Pillow, and ffmpeg. Extract frames with ffmpeg -i bad_apple.mp4 -vf "fps=20,scale=128:64" frames/frame_%04d.png then run python pack_frames.py.

Download the MP3 from the original video using a YouTube to MP3 converter like HC-YT Downloader:

【東方】Bad Apple!! ＰＶ【影絵】 — https://youtu.be/FtutLA63Cp8

Rename the downloaded file to exactly bad_apple.mp3.

Format your SD card as FAT32, then copy both files to the root:

bad_apple.bin
bad_apple.mp3

Open bad-apple-esp32.ino in Arduino IDE or PlatformIO and flash to your ESP32. Open the serial monitor at 115200 baud — it will wait for the SD card to be inserted, then start playback automatically.

bad-apple-esp32/
├── bad-apple-esp32.ino   — Main firmware: display, audio, sync, volume control
├── pack_frames.py        — Python script to convert video frames into bad_apple.bin
├── bad_apple.bin         — Packed binary of all video frames (1024 bytes each)
├── .gitignore
├── LICENSE
└── README.md

pack_frames.py — Takes the PNG frames extracted by ffmpeg, converts each one to 1-bit (black and white), and writes them back-to-back into a single binary file. Run this once on your PC before copying to the SD card.

bad-apple-esp32.ino — The ESP32 firmware. Handles SD card init, OLED display, I2S audio decoding, frame-skip sync logic, FreeRTOS task setup, and rotary encoder volume control.

bad_apple.bin — The output of pack_frames.py. Not stored in the repo (too large) — you generate it yourself. Lives on the SD card, not in the project folder.

Distributed under the MIT License. See LICENSE for more information.