Hyperaudio for the TikTok Generation
I only really started paying attention to TikTok when a friend of mine began posting TikToks in a common Slack group. (I'm assuming they are called TikToks, at the risk of having a George-Bush-like "I use The Google" moment).
I didn’t really have much of a clue what TikTok was until that point: something something social media, something something launched in China. I’m not even sure about the last bit.
Realising my ignorance I looked it up on Wikipedia :
"TikTok is an iOS and Android social media video app for creating and sharing short lip-sync, comedy, and talent videos. The app was launched in 2017 by ByteDance, for markets outside of China. ByteDance has previously launched Douyin for the China market (Chinese: 抖音) in September 2016. TikTok and Douyin are the same, but run on different servers to comply with Chinese censorship restrictions."
TikToks appear to be small looping video clips - a large portion of them with audio created and added separately and usually to great effect. With most TikToks the audio is as important as the video, so something that you really should consume with your earbuds firmly in.
Turns out that TikTok can be quite amusing, I expect it can be informative too, usable for social commentary even protest, as Peter Yeung points out in "Gun Violence and Global Warming: Teens are Using TikTok to Fight Back". Even mainstream news organisations are getting hip with the kids.
I’m fascinated by TikTok for a number of reasons - my interest was initially piqued as it appeared to be video based social media and I work a lot with video on the web, but as I delved deeper I realised that audio was a big part of it – and deeper – there were a number of editing options and effects.
It reminded me of a tool I’ve recently gone back to working on, one I started thinking about back in 2012 and formed a team to build in the following years. We managed to create what you could call a working prototype, before having to move on to other things for a while. The tool I’m talking about is Hyperaudio.
Hyperaudio makes it very simple to create videos too - the way you do it is worlds apart from the TikTok approach - but let’s say that there are strong similarities in that Hyperaudio also gives people an intuitive way to add background music and effects.
I took TikTok’s success to be encouraging and when I read A. Khaled’s piece "The Kids are Learning Video Editing from TikTok" I became further enthused.
Khaled makes an important point - apps like TikTok don’t require the full functionality and complexity of video-editing software such as Premiere or Final Cut in order to provide a valid editing experience. Admirably TikTok makes video-editing easy, even a pleasure - which is great news, as audiovisual media seems to be quickly becoming a popular way for kids to express themselves.
Hyperaudio is a different beast to TikTok - it deals for the most part with the spoken-word - this is both its strength and perhaps its weakness. If audio or indeed video is to be used on the Hyperaudio platform, it must be transcribed.
It’s a happy coincidence that speech-to-text algorithms can produce some very interesting meta-data. As a side-effect of detecting the word, it also knows at which point in time the word started to be said and in most cases stopped. These start and stop times are key - by knowing where any given passage of spoken word begins and ends we can do a number of things, one of those important things is edit.
Most people are used to editing text - Hyperaudio uses the text editing paradigm, it provides a source and target window - to start editing video all you need to do is select and drag text from one to another. We know when the first word of the selection started and when the last word ended and so we know which part of the video to play. This provides a fairly straightforward content-driven way of editing.
Like the makers of TikTok we also recognise that music is a powerful ingredient in telling stories, so we give mixers a choice of audio tracks to add over the top of their creations.
It’s also possible to add transitions between clips and titles before or after them. In the next version we want to take things further and provide the ability to add images and perhaps special effects.
So that’s really it - but from these limited building blocks, great things can be created and all of this (similar to Hypertext) with a kind of ‘view source’ built in - because of the loose referencing Hyperaudio employs to represent a mix (just a series of start and stop times) every clip can be traced back to its original context and remixes can be extended as well as reduced. Want to remix something to show more context? Hyperaudio allows that.
We want to encourage the deconstruction as well as the construction of media. Since we want to promote critical thinking and evaluation of media, being able to trace the origin of content easily is a key objective, as is evaluating the context. Much of the media that is spread on social networks lacks context or a way of establishing it. We’ve got that baked in.
So what’s next? Well as a working prototype the current version of Hyperaud.io is great for running trials and testing assumptions, but we want to do so much more. We want to make things better organised, put back a service to align text to videos, allow a number of different video sources, refine the remixer, represent non spoken-word audio including music, include better navigation and the ability to easily create subdomains and smaller communities.
This year we’ve taken Hyperaudio into the classroom and received a good response from both teachers and students. We’ve run a session at the Creative Commons Summit and gained a jury recommendation as finalists of Cultural Innovation International Prize.
Currently collaborating with WFMT and the Studs Terkel Radio Archive (for whom we’ve created a custom remixer) to run community events in Chicago.
And we’re looking for partners to collaborate with to gain enough funding to create the next version of Hyperaudio. TikTok demonstrates that there is an appetite for creating videos for fun and even education and that coupled with our positive feedback makes us want to ride that wave.
I’ll leave you with one of my favourite Hyperaudio Mixes which was made by a group of kids at Chattanooga Public Library it gets a bit TikTok near the end.
Please contact us at [email protected] if you wish to collaborate or perhaps run events in your area.
By Mark Boas