1Getting started in three moves
No device, no electrode, nothing to install. A phone with a camera, a living plant, and five free minutes to judge for yourself.
Set the phone down
On a tripod, or propped against something. Held in the hand, the tremor of your fingers completely drowns the plant's movement.
Frame the foliage
Close up, filling the frame. Tap Activate camera and allow access when the browser asks.
Let it run
Thirty seconds for the filters to settle. The music starts as soon as both measurements agree.
2Framing well
This is the only step that takes a little care, and it decides everything else. The app looks for movements on the order of a tenth of a pixel: anything else moving in the frame is noise.
Close, and filling the frame
The foliage should occupy most of the frame. A plant seen from across a room mostly gives you the room.
Steady light
Avoid backlighting or changing light. A passing shadow is measured as movement.
Contrast
Sharp edges, veins, outlines. A zone with no detail can reveal no movement, however long you watch.
Nothing else moving
A curtain, a screen, someone walking past: all of it enters the measurement and blurs what the plant is doing.
3What the app measures
Two independent measurements run in parallel, and they do not do the same job. It is their crossing that tells a real plant apart from a plastic one, a photograph or a screen.
Movement
The camera divides the image into 64 zones and tracks their micro-movements 30 times a second. The common mode is removed: whatever moves everywhere at once, such as a change in light, is set aside. What remains is the foliage's own movement, measured by its spatial coherence : a leaf covers several neighbouring zones that move together, noise does not.
Recognition
An on-board neural network compares the image with 61 descriptions: 41 of plants across 14 families, and 20 counter-examples (plastic, silk, paper, screen, wall, fabric, dried flowers). It returns the family and a confidence. The model runs on your device, no image is sent anywhere.
Two rules, and they differ
| When | Condition | Why |
|---|---|---|
| To start | Plant movement and plant recognised | This double lock is what stops a green curtain from singing. |
| To continue | Recognition is enough | A still plant is still a plant. The music only stops when the model no longer sees a plant. |
4The screen, explained
Everything is read on the image itself, with nothing to unfold.
The drawing on the image
While measuring, Plantsound draws the plant over the image, live: its glowing outline, each leaf with its own colour and number (F1, F2…), its veins, and the main axis of each leaf as a dotted line. A leaf keeps its colour and number from one frame to the next, and its glow follows the movement measured beneath it: the leaf lights up when it moves.
The drawing relies on the colour of the foliage, not on the recognition model. Purple or variegated foliage is followed too, through its saturation, as soon as the model sees a plant in it. When the model recognises nothing, the drawing fades: you see what colour alone has found, in the background. A vein does not split a leaf in two, and two overlapping leaves stay apart.
| Setting | Choices | Where |
|---|---|---|
| Drawing on the image | Outline, leaves and veins · Outline and leaves · Outline only · No drawing | Camera settings |
| Measurement grid | Visible · Faded (default) · Subtle · Hidden |
The measurement grid is the set of green rectangles: the zones where the movement measurement finds a signal. Useful for framing, it can be faded or hidden once the plant is in place; the measurement itself does not change. Both choices are kept from one visit to the next.
5The fourteen families
Each recognised family has its musical profile: a scale, a tonic, a timbre, a speed, a way of placing notes. This is not decorative: two plants from different families do not sound alike, and the result is reproducible.
| Family | Musical character |
|---|---|
| Succulent | Minor, deep, very slow. A plant that stores and waits. |
| Cactus | Minor pentatonic, marimba, staccato. Dry and cut. |
| Fern | Dorian, glass, ornamented. The cut of the frond. |
| Climber | Dorian, piano, legato. The continuous movement of the vine. |
| Palm | Major, pad, broad. |
| Bamboo and grasses | Japanese in sen, singing bowls. The rustle. |
| Large leaf | Lydian, pad, wide and luminous. |
| Flowering plant | G major, bells, unhurried. Blossom calls. |
| Cut flowers | Damanhur scale, organic timbre, tied notes. Picked, gathered, and already ending. |
| Orchid | Lydian, glass, arpeggios. The only one allowed virtuosity. |
| Indoor tree | Harmonic minor, deep, sparse. |
| Herb | Pentatonic, marimba, quick and slight. |
| Moss | Chromatic, deep, very slow and very low. |
| Foliage | Neutral profile, handpan. Chosen when the form cannot be read: better a plain character than one drawn at random. |
6The four settings
Scale, reference note, sound type and speed. The plant chooses them, and you can take over each one independently.
Two ways to change them
- Tap the box on the image. A list opens, with every value and its colour. That is the shortest route.
- Or unfold « Impose settings » below the image, where the same lists live, plus the language of the labels.
Automatic, or imposed
Each list keeps Automatic at the top, and that entry says what the plant chose. A setting you impose is marked by a violet outline on its box. Setting it back to « Automatic » returns the decision to the plant.
Each box is coloured by its value: it lets you see at a glance that the plant has changed scale or instrument, without reading.
7Scales and frequencies
Nineteen scales are available. None can produce a harsh dissonance: that is what makes any signal listenable, however agitated the plant.
Western
Major, natural minor, harmonic minor, chromatic.
Modes
Dorian, Phrygian, Lydian.
Pentatonic
Major, minor, blues, in sen, hirajoshi, Chinese.
Traditions
Native American, Indian (bhairav), Arabic (hijaz), Persian.
Wellbeing
Healing, Damanhur.
The reference frequency
Four tunings, adjustable from the Reference notebox, which shows both pieces of information together.
| Frequency | Name | What it changes |
|---|---|---|
| 440 Hz | Standard | The international reference. |
| 432 Hz | « Emotional » | A retuning of everything, by −32, −41 and −53 hundredths of a semitone. The same notes are played, simply lower. The tuning retunes, it does not transpose the composition. |
| 429.62 Hz | « Water » | |
| 426.7 Hz | « Earth » |
The tuning is a listening setting: it belongs to no plant profile, and it is remembered from one visit to the next.
How it plays
The figure shown under Speed counts beats, not notes. At each one the plant may sound or stay silent, and four rules decide: the profile's density, the activity measured at that instant, the family's articulation, and the refusal to play the same pitch twice in a row. A foliage plant set to 96 therefore sounds about thirty times a minute.
That is deliberate, and it is what Observation mode preserves: the silences say the plant did nothing during that time. Dynamic mode loosens those four rules and nothing else: the displayed tempo becomes roughly the number of notes heard, as on the boxes where two clips are attached to a leaf. The plant still chooses every pitch; it no longer chooses the silences.
| Family | Speed | Observation | Dynamic |
|---|---|---|---|
| Foliage | 96 | 32 notes/min | 87 notes/min |
| Cactus | 190 | 80 | 206 |
| Indoor tree | 36 | 19 | 37 |
| Aromatic | 176 | 113 | 192 |
Measured over one minute of play, with the same signal on both sides. The cactus exceeds its own tempo because its staccato articulation doubles one note in six, and the aromatic for the same reason. Like the tuning, the mode is a listening setting, remembered from one visit to the next.
8The eleven timbres
Every sound is a recorded sample, not an oscillator. They are built by physical synthesis: the natural modes of a resonating body for the percussion, additive synthesis for the pads.
| Timbre | Character |
|---|---|
| Handpan | The default timbre. A tuned steel idiophone: its partials fall exactly on the octave and the twelfth, which sets it apart from a bell and lets notes overlap without beating. Finger attack, 20 ms. |
| Organic | A handpan attack with a very discreet New Age pad behind it, both triggered by the same note, in a 65 to 35 ratio. The attack keeps the crystalline impact, the pad provides the tail: the notes melt into one another. |
| Steel drum | Same family as the handpan, but struck with a mallet. Brighter, decays twice as fast, long reverb. |
| New Age pad | The softest in the General MIDI set. Made to sit behind something else. |
| Piano | Plain, the widest range. |
| Marimba | Wooden bar, fast decay. |
| Bells | Full bell spectrum, untuned: hum, prime, tierce, quint. |
| Bowls | Tibetan singing bowl, with its characteristic beating. |
| Pad | Sustained, warm, breathing. |
| Glass | Crystalline, inharmonic, high. |
| Deep | The lowest register, very gently saturated. |
9Slow activity
A small houseplant does not stir: it drifts towards the light, a leaf relaxes, turgor changes. All of it happens over minutes, far below what the fast channel measures. A second measurement channel goes to find it.
Two minutes of listening
The time the measurement needs to settle. The « slow activity » box shows the countdown, then the value.
45 s of calm
The mode only engages after three quarters of a minute without clear movement. An outdoor plant never enters it.
Immediate exit
At the first clear movement, the fast channel takes over at once, with no delay.
In calm mode, notes land every five to fifteen seconds, at the pitch the plant has actually reached. This is not a decorative slowdown: it is the rhythm of what it is doing.
10The environment
The app reads eight weather values around you, after the camera opens. Only two act on the sound. The other six are there to be read, and they carry a different colour so there is no mistaking them.
| Value | Acts? | On what |
|---|---|---|
| Atmospheric pressure | Yes | Reverb, filter opening, air breath |
| UV index | Yes | The same three, in the opposite direction |
| Conditions, temperature, humidity, wind, cloud, precipitation | No | Display only |
Your position is rounded to the kilometre before being sent to the weather service, and you can type a location by hand rather than allowing geolocation.
11What it would say
A plant recognised, the music running, and a pink button appears on the image, bottom left: What it would say. Press it, and a photo goes to a vision model that reads it. Ten seconds later, the plant speaks.
It does not really speak, and the app does not claim otherwise. The first person is a device of writing: what is read are visible signs on a plant, crossed with what the app already measures around it.
Pale foliage has two opposite explanations: too little light, or too much. Drooping foliage too: thirst, or drowned roots. No photograph settles that on its own. The UV index does: bleached patches with an index of 8 mean sunburn; the same patches with an index of 1 and stems stretching out mean the opposite. Temperature, air humidity, the measured agitation of the foliage and the recognised family travel with the image.
The ten needs
Every sentence carries a need and a direction. The need chooses the icon, the direction the colour: blue for “more”, orange for “less”, green for “nothing to change”. Two opposite needs must never look alike.
| Need | More | Less |
|---|---|---|
| Water | I am thirsty | Too much water, I am drowning |
| Light | I need more light | Too much direct sun |
| Air humidity | The air is too dry | The air is too still |
| Temperature | I am cold | I am too hot |
| Place | I would like to move | · |
| Soil and pot | I am cramped | The soil is too compacted |
| Nutrients | I am hungry | Too much feed, my tips are burning |
| Pruning | Remove my dead leaves | · |
| Worth watching | Look at me more closely | · |
| Nothing to change | · | I am fine here |
Three sentences at most, the surest first. None at all if there is nothing certain to say.
How to read a sentence
What it says
The request, in one line. That is what you read from a distance, the phone set down beside the plant.
The sign
In quotation marks, what justifies the sentence: “my lower leaves are yellowing while my young shoots are green”. Look up and check.
The action
Preceded by an arrow, the only line addressed to you: what to do today, in one sentence.
To the right of the sentence, the plant states its confidence in words: I am sure of it, I think, perhaps. Below that threshold it stays silent rather than offer a fragile verdict.
When it says nothing
That is an answer, and often the right one. If your soil is not in frame, the plant will not rule on watering. If the shot is too blurred, too dark or too distant to make out the leaves, it returns no sentence at all and asks to be taken again, closer. A tool that always answers something is a tool whose answers are worth nothing.
One photo, reduced, leaves the moment you press, and is kept nowhere. No address, no position, no identifier travels with it: the model receives a picture of a plant and some numbers. The rest of Plantsound (the measurement, the recognition and the music) goes on running entirely on your device, offline included. When offline, the button says why it does not answer.
Experimental feature. It will sometimes be wrong: that is why every sentence displays the sign it rests on. You keep the last word.
12Recording
The Recordbutton, below the music settings, produces two files at once. They do not contain the same thing, and that is deliberate.
The audio
What you heard, captured just after the limiter: exactly the final sound. WebM/Opus or MP4 depending on the browser.
The MIDI
The score, written at source. It reopens in any music software, with the tuning written in as pitch bend. Sustained notes on channel 2, struck notes on channel 1.
13MIDI, OSC and controller mode
The MIDI settings card, below the camera settings, takes the plant out of the app. Two uses, which can be combined: doubling Plantsound's music to an external instrument, or sending the signal itself, before any scale, instrument or botanical profile, so that a synthesiser or a piece of software makes its own music from it. Every section of the card is folded: you only open the one you need.
Connecting an instrument
- Tick Enable MIDI output, then choose the device. Plantsound sees whatever the system sees: an instrument over USB, over Bluetooth (paired beforehand in the system settings), or the virtual ports of music software.
- The Stop everything (panic) button turns off every note on all sixteen channels and resets pitch bend to the centre. Each note is sent with its note-off already scheduled anyway: none can get stuck if you close the page.
- Controller mode: the app's sound falls silent, while the plant is still listened to, recognised and turned into notes. Your synthesiser becomes the only voice.
Notes
By default, short notes go out on the base channel and held notes on the next one. Each voice can have its own channel, or not be sent at all:
| Setting | What it does |
|---|---|
| Melody, ornaments, pads, drone | One channel per voice: the melody on one module, the drone on another. |
| Slow channel | Notes born from slow movement, when the plant moves over the minute, go out on their own channel. |
| Transposition, lowest and highest note | Bring the notes into your instrument's range, by whole octaves. |
| Velocity | The plant's (it follows the movement intensity) or a fixed value. |
| Program changes | Can be turned off, so that Plantsound does not change the sound selected on your synthesiser. |
| Tuning as pitch bend | Sends the reference frequency (432 Hz, for instance); set the same bend range on the instrument. |
| Channel rotation | Gives each note its own channel while it sounds, so that the pitch bend of a microtonal note does not detune its neighbour. |
Continuous signals
A route reads a source, shapes it and sends it on a channel. The Modular synthesis preset sets up eleven of them at once: movement to pitch as pitch bend (for a MIDI to CV module), coherence to the filter, the active leaf's position to panning, the weather to the reverb.
| Element | Choices |
|---|---|
| Sources | Raw luminance of the tracked leaf (no filter), fast and slow waveforms, movement intensity and coherence, deformation, position of the active leaf, foliage area and number of leaves, recognition confidence, botanical family, plant presence, pressure, UV, temperature, humidity, wind. |
| Destinations | 7-bit CC, 14-bit CC (16,384 steps), pitch bend, channel aftertouch, or note: the source quantised over a range and a scale, without going through the plant's music. |
| Shaping | Output range, curve (linear, exponential, logarithmic, S-curve), smoothing from 20 ms to 5 s, inversion. |
| Rate | From 15 to 100 messages per second. A value is only sent again when it changes. |
Clock
When sent, the MIDI clock follows the plant's steps, 24 ticks per quarter note, with Start and Stop: a sequencer or a clock module locks onto it. When received from a MIDI input, it imposes its tempo on the music and brings it back into phase at every step; after one second without a clock, the plant goes back to its own tempo. The One plant step equals setting picks the correspondence: a quarter, an eighth or a sixteenth note.
Your own scales and tunings
A scale can be written in semitones (0 2 3 7 9), in cents (0 204 386 702 884) or as ratios (9/8 5/4 3/2), as in a Scala file. It joins the list of scales and can be microtonal: the internal sound plays the offsets, the MIDI output sends them as pitch bend. The same section sets the melody's range, its starting octave and a free tuning between 400 and 480 Hz.
OSC and WebSocket
Every source also streams continuously over WebSocket, as binary OSC or JSON: normalised under /plantsound/<source>, raw under /plantsound/brut/<source>, with /plantsound/note for each note and /plantsound/famille. TouchDesigner, Max or Open Stage Control read them directly. For classic OSC over UDP (Pure Data, SuperCollider, VCV Rack…), the OSC bridge downloadable from the card runs on the receiving computer, with Node.js.
Artist mode and latency
The last panel shows every source, normalised and raw, live, along with the actual latency. In short: the raw signal lags by one frame (33 ms at 30 frames per second); the fast waveform adds about 0.4 s, the time the filter needs to see a plant movement; notes follow the step grid and are timestamped to leave exactly with the sound.
14Offline
Plantsound computes everything on your device. The network only serves to deliver the model and check your access. Store those files once and the app then opens with no connection.
Install the app
From your browser menu: Install or Add to Home Screen.
Store the files
In the settings, the Use offlinepanel. About 67 MB to download once, ideally over Wi-Fi.
Seven days of autonomy
After that a reconnection is required, and it reopens another seven days.
15Trial and access
Five minutes free
No account, no card. Enough to see the curve react to your breath and hear the first music.
€94 once
One-off purchase, no subscription. For all your plants, on all your devices, updates included.
Recovering your access
On a new device, tap I already have access and enter the email address used for the purchase. A password was sent to you at the time; the Forgotten password button sends a new one.
16If something goes wrong
| What you see | What to do |
|---|---|
| Inert, when the plant is alive | Wait two minutes: the slow channel is settling and the box shows its countdown. Otherwise move the frame closer and check the phone is properly supported. |
| Too few edges in the image | Light it better or move closer. Without sharp edges, no movement can be measured. |
| Not a plant | Frame the foliage more closely. The model also refuses, deliberately, plastic, silk, paper, a photograph and a screen. |
| No image is coming from the camera | Another app is already using it, or permission was refused. Close the other tabs, then reload. |
| The module did not start | The diagnostic card names the file at fault. The button opens verif.php, which continues the check on the server side. |
| Sound unavailable | The message appears below the music settings. Reload the page; on iPhone, check that silent mode is off. |
| Incomplete package | Remove the offline package and install it again, over Wi-Fi. |
The diagnostic card at the bottom of the app shows the version of the loaded files. If it differs from the one reported by verif.php, your browser is serving an old page: reload with the cache cleared.
17Your data
- Your email address is kept, so you can recover your access.
- Your location, if you allow it, is rounded to the kilometre before the weather is fetched. You can also type a location by hand.
- Your settings (tuning, language, offline mode) stay in your browser's local storage.
The full detail is set out in the data protection notice.