Synesthesia

Music from what you see: photos, videos and your camera. Colours set the key, shapes and movement play the melody, the look of the picture picks the genre.

Drop a photo, a few photos or a video here, or press Choose a photo or video (pasting a picture works too)
  • 🖼A photo becomes a music loop you can play, edit and save as audio, MIDI, a music card or a short video.
  • 🎬Several photos or a video get a whole soundtrack that changes with every scene, saved as an MP4 for Reels, TikTok or YouTube.
  • 📷Your camera plays live: the music follows the picture or your dance moves. Take photos or record videos with the music.
🔒 Everything happens in this browser tab: your photos, videos and camera are never uploaded.

Questions

How do I turn a photo into music?

Open Synesthesia, press “Choose a photo or video” (or drop a photo on the page) and press Play. The app reads the photo’s colours, its most striking points and its overall look, picks a matching genre and plays a music loop made for that picture. You can change the genre, tempo and mood, or edit the points and chords.

Is it free?

Yes, for personal and other noncommercial use: hobby projects, study, teaching, and sharing what you make on social media. Commercial use (in a company, a product, client work or ads) needs a paid licence from the author, Victoria Lisitsyna: lisitsynavictoria@gmail.com.

Are my photos or videos uploaded anywhere?

No. Everything runs inside your browser tab: your photos, videos and camera picture never leave your device, and the page is technically blocked from sending them anywhere. There is no account and nothing is stored.

Can it add music to a video automatically?

Yes. Drop a video (up to 3 minutes) or several photos. The app finds the scene changes, gives every scene its own key and chords in one continuous soundtrack, and saves an MP4 in 16:9, 9:16 (Reels, TikTok, Shorts), 1:1 or the original shape. With “Follow the dancer” the music also takes the tempo of the person in the video and gets calmer or more energetic with their moves.

Can I make music with my webcam?

Yes. Press “Use camera”: choose whether the music follows the picture (colours held up to the camera change the key) or your moves (body and hand tracking: dance and the bass and drums come in, stand still and the music holds); every change lands on the beat. You can take photos or record videos with the music.

Which music styles are there?

Seventeen, played with real recorded instruments: Classical Piano, Cinematic, Ambient, Shoegaze, Dream Pop, Indie Rock, Hard Rock, Pop, Synthwave, House, Jazz, Lo-fi Hip-Hop, Boom Bap, Trap, Electronic, Brazilian Phonk and Folk / Acoustic. The app suggests the style that fits the picture best and shows how well each one matches.

How does it decide what the music sounds like?

With simple, explainable rules instead of a black box: the biggest colourful area picks the key (the colour wheel is laid out on the circle of fifths), brightness picks the mood, each main colour becomes a chord, and the points of interest are read left to right like notes on a staff. The “Why it sounds like this” panel explains every choice for your picture.

Can I use the music in a music app?

Yes. Save the music as a WAV file, the notes as MIDI (for GarageBand, Ableton, FL Studio or MuseScore), or each instrument group as a separate track in a ZIP.

More: Photo to music · Soundtracks for videos · Music from your webcam

How it works

Synesthesia reads pictures the way a musician might read a painting: a single photo, every scene of a video or a slideshow, or the live picture from your camera. Everything is calculated with simple, deterministic rules, so the same picture always gives the same music, and every step can be explained. Below is how one picture becomes music; videos and the camera repeat it scene by scene or bar by bar.

  1. Look at the photo. The image is shrunk to about 190 px and analysed in a background thread. Three things are measured:
    • Colour palette: the 6 main colours and how much space each takes (k-means clustering).
    • Points of interest: up to 7 spots with strong edges and distinctive colour.
    • Photo fingerprint: eight values from 0 to 1: brightness, contrast, saturation, colourfulness, warmth, sharpness, texture and complexity.
  2. Pick a style. Each of the 17 styles has a target fingerprint (Shoegaze likes blurry, warm, glowing photos; Trap likes very dark, cold ones…). The style whose target is closest to your photo wins. You can always pick another one by hand.
  3. Find the key. The hue of the biggest colourful area picks the root note. The colour wheel is laid out on the circle of fifths, so similar colours give related keys. Brightness picks the mood (Lydian, major … minor, Phrygian), and the style can push it brighter or darker.
  4. Build the chords. Each main colour becomes a chord: its hue angle from the key colour picks the scale degree, its share of the frame sets how long it lasts. The style decides the chord type: triads, sevenths, add9, power chords… Tightly clustered points add an extra colour tone.
  5. Lead the voices. Every chord is voiced as close as possible to the previous one, and two neighbouring voices are never allowed to clash a semitone apart.
  6. Play the points. Points of interest are read left to right like notes on a staff. The points are read relative to each other, so a small cluster still becomes a full phrase. As a melody: higher in the photo (and, a little, brighter) = higher note, and the distance between points picks the interval (close points sound consonant, far apart ones add gentle tension; harsh intervals are never used). If the points form a flat line of similar colours, they play a simple motif instead of a monotone tune. As drums (the default in rock and jazz): the same rhythm is played by a drummer, higher points on higher drums, the strongest point on a cymbal. You can switch with “Points play”.
  7. Arrange and play. The style supplies the tempo range, rhythm patterns, swing, instruments and effects. Instruments are real recordings (piano, guitars, bass, strings, saxophone, flute, horn, harp, drum kits) played note by note with Tone.js; synthesizers are used only where the genre itself is synthetic (synthwave, house bass, 808). Each style was mixed by measuring every instrument group and balancing them.

Saving and editing

🔒 Privacy: the photo never leaves your device. The page is technically blocked from making network requests, and nothing is stored.

The style library

What each style sounds like, and which photos it is chosen for.

Credits

Synesthesia is free for personal and other noncommercial use under the PolyForm Noncommercial License 1.0.0 (see LICENSE and NOTICE on GitHub), including sharing what you make on social media. Commercial use (in a company, a product, client work, ads or anything sold) needs a paid licence: lisitsynavictoria@gmail.com. Videos and music cards saved from the app carry a small “Made with Synesthesia” line.

Instrument recordings are used under the Creative Commons Attribution 3.0 licence (CC BY 3.0); audio engine: Tone.js (MIT); MP4 packing: mp4-muxer (MIT).