I’m proud to announce the Radiolab Hyper Audio Player. Go check it out! Comments, bugs, fixes and random ideas are most welcome in the comments, or here.

The Radiolab Hyper Audio Player v1, brought to you by WNYC Radiolab, SoundCloud and Mozilla Drumbeat

What is it?

In brief, this experimental HTML5-player is a the happy marriage of

1: Radiolab, IMO the best radio show in the world. Radiolab let us experiment with “Desperately Seeking Symmetry“, a show which was particularly suited for an audio-visual experience, and for building a custom HTML5 player using:
2: Popcorn.js, a Mozilla-backed JavaScript framework for integrating the web into audio and video productions.
3: SoundCloud, an innovative and successful music sharing platform, known for its characteristic waveform player and commenting functionality.

What does it do?

First of all: the player is a fully open experience – that means no flash, no proprietary codecs; only pristine HTML5, CSS and JavaScript. The demo works beautifully (well, mostly) both on a Linux computer and on the iPad. So, what does it do?

It displays images and comments from SoundCloud player

When Radiolab puts a link to an image in a Soundcloud comment, Popcorn.js makes it display in the main window.  These comments are visible on SoundCloud, but not in the player. Other comments made by Radiolab are. This allows for Radiolab to experiment with listener engagement in new ways. This was one of the main points that came out of my meetings with Radiolab – displaying additional information (links to research etc that did not make it into the final cut, for example) was not an attractive feature as exploring and filtering the creativity and ideas coming from listeners. In this demo, Creative Commons licensed images from  listener who attended the shows appear. In future versions, we’d like to integrate YouTube and other videos.

The hyperlinked transcript is used to visualize the audio, helps navigate the audio (and facilitates search).

The hyperlinked transcript feature was explored in our first demo, Hyperdisken. In this demo we used it in two ways:

- If you click the transcript button in the right side, you get a transcript. Clicking any word in the transcript (done by 3Play) sends you to that point in the timeline / player. The transcript scrolls and always shows you where in the text the show is currently.

- the “word river” (by Paul Rouget) visualizes the transcript when there are no images shown. It doesn’t serve any function other than an aesthetic one, and only works in newer versions of Firefox and Chrome because it uses new CSS3 and was never intended to work on older browsers. Our feeling is that the experience degrades gracefully when on an older browser – and hey, it’s about time to upgrade anyway. (Get Firefox 4, or Chrome!)

In conclusion

I look forward to getting more feedback from WNYC / Radiolab and their listeners. We have taken a first, pioneering step into building a player that does something new with Radio, but this is only the beginning. Leading this project was a challenge for everyone involved. For me, it meant hearding and pushing (not always successfully) a bunch of very talented cats with no time to spare, and get an advanced, working prototype built in little more than a week. Mark Boas and his Happy Worm team deserve wild praise for throwing everything in their hands down and maintaining focus and architectural overview during a rather chaotic week. Mark will post his account of hos the demo was built, with documentation and source code on GitHub shortly.  Popcorn developer Steven Weerdenburg did an excellent job on the Popcorn plugin for Soundcloud, with Support from his team.  I owe huge thanks to Ben, Henrik, Johan and Johannes from SoundCloud,, and not least Music & Tech Ninja Lee Martin for taking time out to provide an awesome design. Thanks Paul Rouget for helping and being absolutely brilliant. Finally, I am proud to work with the Radiolab team and look forward to taking this further.

I’m excited to share the first tangible results of the Hyper Audio project, which I now lead at Mozilla. Before you read any further, please spend a few minutes with the demo and leave your feedback – however mundane – in the comments:

What does it do?

The HTML5 player plays 2 files (.ogg files with .mp3 fallback): the original, danish-language radio show, and an English version. The links on the right spawn as the player progresses, as do the footnotes. Each link is timestamped. This means you can use them as chapters – click the small audio symbol, which will send you to the exact point in the timeline. But the killer feature of the demo is the linked transcript at the bottom of the page. In the words of radio futurist James Cridland:

This offers disaggregated audio and a transcript. It offers translation into English from the slightly less accessible Danish. Using HTML5, it also shows you the translation alongside the original language as it plays. Highlight a part of the transcript, and you can tweet that part automatically. It’s a very impressive and strong example of what’s possible with new technologies.

How did we do it?

Mark Boas  worked tirelessly on the demo, and has written up a great post that outlines the technology and procedure behind the demo. My workflow was this:

- Anders shared the session file, source interviews, in a stripped-down version, with me. He also shared the document containing his script, notes and research.

- I submitted the source interview to Castingwords. The transcript arrived 6 days later, and cost 34 USD.

- Ran the clear-text script through Universal Subtitles and asked friends to translate, while cross-referencing with the original script (in danish).

- Edited up the session in Hindenburg and downloaded the podcast version to compare with, and, after some work, got everything in sync.

- created a stub based on a previous popcorn demo, with the help of Ben Moscowitz. Added Popcorn based on the notes and audio. This was a labour-intensive task, and needed many corrections, but also a great learning experience for me. With time, the Butter App will make this a lot easier for audio. Also, during the procedure the Hindenburg team wrote a parser for the editing app which may, in the future, allow journalists to add Popcorn as part of their workflow. See this slideshow for more on Hindenburg Popcorn integration.

- Ordered a transcript from 3play. This formed the basis for the linked transcript.  See Mark Boas post as to how we generated the danish version.

- Once the demo was taking shape, I asked Christian Valentiner at DR to help design the demo to look and feel closer to a DR page thank our stub. He did an amazing job in very little time.

- Polishing, 15 iterations, and we were (almost) done – we still fix bugs and listen to feedback!

Why?

The goal of this demo was to “open up” Harddisken, a brilliant danish tech radio show, and to show how opting for “open” can help reach an international audience and engage with listeners in new ways. In short, the intention was to create a first, awesome demonstration of the Hyper Audio concept.

The first tenet of The Cathedral and the Bazaar is

“Every good work of software starts by scratching a developer’s personal itch”

I outlined my itches with radio in a previous post on Hyper Audio. I got the idea for this demo from conversations with former colleagues at Harddisken and in particular an email exchange between my (new) colleague Tristan Nitot  and host and reporter Anders Høeg Nissen:

“Hey Tristan

You may remember that we did a pretty long interview back in December – and now I’ve finally gotten around to actually broadcasting it (..) most of it’s in Danish, but should you want to listen to the bits where you talk (or listen to the whole thing and hone your language skillz – you can download it here (the Mozilla stuff is in the last 20 minutes or so).”‘

If we break this down, there are a number of itches to scratch here:

1: A long interview of which only parts are used.This is frustrating both for the interviewee and the interviewer.

2: Tristan would probably never listen to this as; and if he did, he would not be able to appreciate his own wisdom because the show is in Danish

3: Navigating in the standard flash player is painful – chances are Tristan would give up (especially since he uses a Mac, which to this day works badly with DRs player).

4: A google search for “Tristan Nitot Harddisken” will not find the audio for Tristan (or you) in case he loses the link. Try for yourself!

I’m pleased with the final result and thankful for the amazing support we received from Mozilla, DR and Harddisken, and not least the Popcorn team. Special thanks to Barry Threw for providing support on all stages of the project, and to all my other friends who helped along the way.

Nick at Hindenburg just sent me this mockup showing how Popcorn integration and authoring could work from inside their software. This is part of a series of demos I am currently working on, showing how popcorn and HTML5 Audio could augment the radio listening experience. More on that and Harddisken soon. Also, see this post about Hindenburg.

This is the full-length edit – great work, Gabriel!

GML / Graffiti Analysis Workshop – Mozilla Drumbeat Festival 2010 from Gabriel Shalom on Vimeo.

Ogg: GML / Graffiti Analysis Workshop – Mozilla Drumbeat Festival 2010

Today Gabriel and I brainstormed on how a popcorn-enabled version of the GML Drumbeat Festival Video could look. We came up with the below mockup in about 30 minutes. Main features: timeline displays events and triggers text-based resources and links which scroll in sync with the timeline. Gabriel showed me this documentary as an example. I really like the text-approach as a way to add context without removing attention from the main video. The GML viewer displays gml files referenced visually in the video and links back to 000000book.com, the database containing GML tags. Departing from this great popcorn mashup, our priority was to reduce visual flutter and assist the video without removing focus unneccesarily. The GML player will require some tweaking, as will the timeline and text-based ressources.

Another idea was to display the b-roll in a secondary window (Paul Rouget thought this could be interesting). We could still use the GML viewer window for this so that GML tags show when relevant. They will also be referenced and linked to in the ressource window. I for one have no idea how much of this is easy to pull off technically – for b-roll to be referenced in a single video file, HTML <video> would need to be able to handle tracks. We looked into this, but weren’t sure this is in the spec at this point. Anyone interested in working on this demo, please get in touch – there is a high likelihood we hack it on the march 26th Buttercamp in NYC. Gabriel is currently finishing editing the full-length video which will be done by that time. Very excited!

I mounted a bunch of cameras during Evan Roths Graffiti Markup Language talk at the Mozilla Drumbeat Barcelona festival in November. Gabriel Shalom and I are now working on this video + a longer edit, also multicam. We’d love your ideas to take it further, particularly input before we start to experiment with popcorn.js and Butter. This is a first draft.

GML / Graffiti Analysis Workshop – Mozilla Drumbeat Festival 2010 from Gabriel Shalom on Vimeo.

WebM version here, .ogv here

Jesse Scott (Graffiti Research Labs), center, accepts the Open Web Award on behalf of Evan Roth. Right: Mark Surman, Director, Mozilla Foundation, Left: Henrik Moltke (European Project Lead, Mozilla Drumbeat).

Here’s the official Mozilla post. Congratulations to Evan Roth, the GML team and community, and to the open web!

Mozilla is moving beyond software, looking for ways to bring the technologies and culture that underlie Firefox into new areas like art, media and education. The Open Web Award was the first dip of our toes into combining digital art and the open web, and we’re excited to dive further in. With our new Mozilla Drumbeat initiative, we are engaging creators globally to shape the future of the web. This is not done by what some of you might call ‘nerds’ and what we call developers. You are the future of the web, and we need your help to keep it open, avesome and available for everyone to play with.

We were excited to see more than 100 submissions for the Transmediale Open Web Award. The jury narrowed it down to three great contributions that all show potential, and could be seeds for incredibly exciting developments for art and the web in the coming years.
The winner – Graffiti Markup Language – enables the capture of graffiti motion data and turns it into something that can be shared and manipulated indefinitely. One can imagine not only great and exciting graffiti art but all kinds of motion expression. Just like the web itself, the potential of GML is totally open ended. Anyone can – like LEGO – built whatever they want with it. For Mozilla, GML also has great educational value because it explains open data and standards in a surprising and fresh way to new people. We congratulate Evan Roth and the GML community on winning the award.

We  also want to award a distinction to the other two finalists, Booki and Thimbl. Like GML, they represent exciting opportunities for creative expression on the web. Next week we launch the new Mozilla Drumbeat website. We will feature all three of these projects. We look forward to working together with everyone involved.

[Updated: Transcript now available]

Nick is an old friend and radio geek. He was the technician when I did my first radio programme, and taught me the basics I still work by. Since then much has happened. I work for Mozilla, and have recently started thinking about how audio and radio and the open web can merge. Nick works on Hindenburg, an excellent piece of audio software for journalists,  which could be part of the kind of radio experience, I now work on creating a demo for.  A seamless experience that supports the most important element in journalism: the story. One in which technology simplifies and enhances the experience while the power of the open web – adding depth,  detail, and infinite time/space. Nothing less!

Nick and I met in Copenhagen to discuss where we’d like to see radio go in the future. Here’s the interview – if you make it to the end, it ends with a question, which I’d love to hear your comments on!

I recorded using the iphone field recorder app and edited using the Hindenburg desktop app. Download the raw interview and session files here,. You can grab an Ogg Vorbis version of the edited interview here, mp3 here.

Not surprisingly, Nick and I share a bunch of itches about the present way things work in radio production.   I’ll elaborate on mine in a separate blog post, but Hindenburg scratches a few of those I had when I was a freelance radio journalist. That’s a good start. Of course, Hindenburg is not free software (at least in the freedom sense – there is a free version with limited functionality), but it orients itself towards standards, and Nick and his team have open minds.

Why is Hindenburg interesting for hyper audio?

I won’t elaborate much here, but basically, the term hyper audio combines the hyperlink with audio. Nothing original here, I’m basically riffing off this blog post by Tristan Nitot, and drawing inspiration from technologies brewing at Seneca College – most notably popcorn.js. Everything that can be done with <video> can be done with <audio>. But that’s a whole different story.

The strength of Hindenburg is it’s ease of use and it’s ability to let you focus on the story. It allows you to add markers while you record, and it would be fairly simple to add images and other audio from your recording device (in my case an iPhone 4). These transfer with the file to your editing software, and Hindenburg outputs XML- based session files that could be used for sharing session data, merging your audio with semantic data seems seams a realistic feat. As a journalist, why not not add the sources used, list the Wikipedia entries you drew from, and link to the people in your story? Comments. Geo data. Copyright information – whatever helps strengthen the conversation that audio could be part of. As Nick and I discuss, simply combining time coordinates with transcripts and making these searchable would be a huge improvement over the flat audio we mostly encounter on the web. Imagine a Soundcloud player that shows your markers and links and integrates with comments on your blog -  a player that build solely on open standards. I could go on.

Nick: “You can do anything. It’s just code. But really, want you want to do is something useful.

Henrik: And what would be useful for the listener?

Nick: That’s a really good question. You should put it out to your listeners.

Henrik: We’ll do that! Thanks.

Hindenburg – quick hands-on impressions

- Hindenburg is easy to use, lightweight, intuitive and does 99% of what I need. It just .. feels right and is extremely user-friendly; and it has a very lean workflow from start to end. This is so different from any other audio editing application that I’ve used in the past.

- Auto levels rock. Period. This, on it’s own, is a killler app in itself, and also the main difference between the free and the paid app. It doesn’t replace using your ears (or experience) but it helps a lot.

- No hardware. This is the most sucky feature about Pro Tools – I bought their hardware/software soluation 3 times (but never paid for upgrades – and never felt I got my money’s worth from Digidesign’s Support) and always had to carry around a clunky, external sound card, that I did not need 90% of the time (unless recording into pro tools); the “micro” sound card dongle ruined my USB port, and of course, if I needed to make a quick change in an earlier session and did not have the hardware with me I was out of luck. The Hindenburg app is slim, powerful and fast, as opposed to Pro Tools.

- The iPhone integration is brilliant. Soundcloud integration is coming, hopefully. This means you can record a quick and dirty interview, edit on the iphone, and publish, on the fly.

- It’s WAY better than Audacity or any other freeware apps I used, and very stable. I have not yet worked on advanced projects, and of course automations and filters are still the domain of the Pro Tools of the world. And that’s fine.

- It’s FAST! Putting out this 25:27 interview in .WAV format took 6 seconds (!). Mp3 took 40. Pro Tools would have to “bounce” in real time, which means outputting  happens at 1:1 speed. Ridiculous for radio journalism, because you always have to fix something after you’ve finalized your project.

- Of course, knowing the developers makes the customer experience so much more personal – they actually listen to feedback!

This is my first post about Hyper Audio. Expect more – maybe a podcast; I like this conversation-based approach.