a tangible audio puzzle game, by LudicVoice

Jarbles is a sound-puzzle game that uses tangible objects from the traditional children’s games of Jacks and Marbles as inspiration for its player interactions. The player selects a puzzle and then the line-art Jacks are ‘thrown’ and scattered across a white screen. Touching a Jack activates its inherent sound and brings out its colour. The goal of this first phase of the game is to find the pairs of Jacks that best blend together according to the unique logic of any given audio puzzle. The puzzles may be melodic, rhythmic, semantic, or may work according to more abstract arrangements of audio frequencies. When two Jacks are properly combined, they transform into a multi-coloured Marble. That is to say, they are JARBLED. Once all of the the Jacks have been merged into Marbles, the player must complete the puzzle by fusing the marbles in proper order. When a puzzle is completed the player is rewarded by hearing the entire audio composition from start to finish. Completing one puzzle earns the player a point and unlocks the next puzzle on the puzzle branch. Players may also replay a completed puzzle and improve their scores in the categories of Speed, Accuracy, Listening and Decisiveness. Achieving a perfect score of 100 results in a special visual treat.








Ian Arwajo design, programming

FX Dupas sound design

Taylan Ulger concept art and color palettes

Jason Camlot design, direction

Audio Puzzles were contributed by the following individuals: FX Dupas, Mathieu Lavoie, Maxime Jacquet, Maxime Hervé, Jason Camlot

 The Jarbles team and LudicVoice gratefully acknowledge the support and advice received from the GRAND NCE research and commercialization network, TAG and AmpLab at Concordia University, Michel Fortin, Pierre-Alexandre Fortin, and the games of Jacks and Marbles.


Below are selected images and documentary materials pertaining to the conceptualization and design of JARBLES.

Poster for JARBLES presented at the GRAND NCE in Toronto, Ontario, 2013.










Black and White version of JARBLES splash screen. Not used in final app.











Update November 2nd, 2012: Color! Here’s a concept for you:

Perhaps we should explore multiple styles of display for each puzzle, to vary the visual framing of what is, essentially, the same “gameplay” for each puzzle.

Colors can also “shimmer” or change slightly.

I also started sketching title screen menu icons and puzzle icons, a transition for the JARBLES title (ray-tracing), and a question of whether single player should exist (it should). There’s also the “Import” icon for trying out your own sound game, and yes, it is possible to open .zip files from the internet to load into the game.



conducted by Ian and FX on Wednesday, October 24, 2012.

Ian Arwajo’s report on the playtest session:

Here is a rundown of what we’ve learned (for testing I used FX’s
original puzzle, the one of his wife singing):

Testers were naturally inclined to hold on a selected jack to replay
the sound. They were frustrated when this didn’t happen.

While the two-touch merge made sense to the testers, they complained
that the force needed to merge the jacks was too high, and the visual
feedback for merging the wrong jacks too small / ambiguous.

Many testers selected more that two jacks at once, which was
problematic to merging. Selecting a new jack should swap out the
previously picked-up jack when two are selected?

The solution of the sound-puzzle should be obvious and clear. For
instance, in the case of a melody, there cannot be more than one
“correct-sounding” ordering of marbles. FX agreed that the train
puzzle might be too difficult in light of the ordering. There was a
great discussion between FX and one of Will’s friends about this.

Many testers didn’t connect the marbles that they created in the first
phase to the marbles in the second phase.

Testers wanted to touch the marbles to play their sounds.

Having a mute hit a marble should *not* play the marble sound, since
the game gets cacophonous (and thus annoying and unplayable) very

When a mute hits a marble, there should be auditory feedback (snapping?).
here should be no lag in the second phase. — (On this, Bronson came
later and said he was having problems with Pure Data causing lag over
time in Skipping stones and previously in Ethereal. He hasn’t fixed
the problem. I’m going to have to fix the lag last, I think. It might
take some delving into Pure Data’s source code and collaborating with

Feedback for flicking a shooter marble (mute) should be greater in
some way (auditory?), and the mute should deteriorate slower.
In the second phase, the number-based ordering in the player’s
“gutter” made players confused, because they believed if the order was
1 2 3 4, as it always is, that that should mean the correct order.
(Get rid of numbering system.)

Testers complained about the lack of a visual distinction between
marbles and between jacks. (For marbles, I have yet to connect the
waveform data to the marble, but that will be done.)

Particularly, Lynn’s comment was that when you set a jack down, pick
another up to listen, and realize this jack and the previous jack go
together, oftentimes you can’t find the previous jack on the board
because they all look the same. Obviously we don’t want the visual
distinction between jacks to give away which ones are paired, but that
doesn’t mean every jack can’t be visually distinct. I think in order
to solve this problem, we can bring back your idea from the dusty
original design: the jacks start out looking the same, but when you
play a jack for the first time, it morphs a bit. I can try to do that
by deteriorating the jack’s mesh based on the waveform… or I should
be able to, haha.

The noise in the room while people were playing made it hard to
listen. Salvadore played with headphones, but even then it was too
loud. One player got frustrated very quickly and remarked that all the
jacks “sound the same.” But I don’t think this is a valid complaint.
However, it may point out to us that the difficult curve in this game
needs to be graduated much more than we thought. We should start with
very, very easy, well-known ordering of sounds, two marbles only. A
good puzzle might be “Jar” and “Buls.” The first puzzle should be
well-thought out and extremely simple. The jacks could even contain a
bit of the tutorial in sound form.

Lynn liked the art style, and didn’t want color to be involved in the
main game, but agreed with me to provide color as a reward during the
“victory event” of each puzzle.

Many people eventually “got” which sounds go together, and everyone
reached the second phase of the game. I think the time it took them
was on average 3 minutes. (This is good news!)

There is inevitably something I’ve missed. 🙂




Screen shots of the the black and white line art style presently being used for Jarbles:




The concept for this game was developed by Jason Camlot and Ian Arawjo, the main goal being to come up with a very basic tangible interface by which a player would manipulate sound in order to solve sound puzzles.  We first played with a few basic kid’s games and toys to get a feel for the possibilities of adapting these games to our application.  We played with actual marbles (on the sloped 11th floor of the EV building).  We pinched and shaped play doh, thinking that sound dough would be a fun thing to play with, but probably more fun in reality than in a digital version.  And eventually, I was able to find Jacks at a toy store, and I introduced Ian to this game.  We decided to work with Jacks and Marbles as our starting point, and then thought about how we could associate these play objects (jacks and marbles) with sound manipulation.  Our first concept doc had a bouncing pink rubber ball (a Jacks ball) bouncing along one side of the iPad screen, keeping tempo.  Players would have to ‘pick up’ a jack in time to the bouncing ball, which would trigger a sample embedded in it.  Players would take turns sampling the jacks, and then would have to figure out which two jacks to pick up simultaneously, in order to create a marble (if the two samples went together).  The working title of the game, at this stage, was Jacks Into Marbles.  Ian went ahead and developed jacks for the interface.  The jacks remain in the game, but the bouncing ball is gone.  Now the player must select two jacks by pressing on them to sample them and then try to combine the jacks according to a sonic logic of combination.  When two jacks that go together are identified and dragged together, the transform in to a marble.


The basic idea, that there is an incoherent (incohate) set of sounds or messages that must be recombined through interacting with the jacks and then the marbles in order to make them intelligible made me think that a better name for the game is JARBLES.  Jarbles in the sense of jarbled samples, but also to evoke the action of merging jacks and marbles to solve our puzzles.


Marbles and Mutes

Marbles and Mutes

Marbles in order of solution

Judgment: Marbles in Proper Order

Once the jacks have been converted into marbles we arrive at something that feels like an air hockey table.  The eight jacks that have been combined to form four marbles (the numbers can vary depending on difficulty of the puzzle) are on the table, and the two players are on opposite sides with a shooter (mute, or blackened marble) waiting to be aimed at the marbles.  When a mute hits a marble it makes it play its now consolidated sample.  The goal of this section of the game is to determine the proper order of samples contained in the four marbles.  The first player to hit the four marbles in the right order, thus aligning them in a grid at one side of the table top, completes the puzzle and wins the game.  When all four (or more or less) marbles are aligned in the correct order, the completed sound puzzle will play.

Jason Camlot and Ian Arawjo first brainstorm meeting about Jacks and Marbles (to become JARBLES)

Leave a Reply