Rethinking Remixing
Much of the current web is powered by advertising. It’s a model that means many services can be given away (seemingly) for free – but really we are paying with our attention, and often (through tracking) with our personal data. This is not a new model – for many years television and radio has for the main part been subsidised by advertising. However we’re starting to see a desire for advertising not to pollute our viewing, listening and reading habits, with the advent of subscription based audio and video services. Although personally, I find myself wondering if I use some of these services enough to warrant paying that fixed monthly fee.
The Web Monetization API is a proposed web standard which could encourage new models to flourish – alternatives to the ad-based models and the tracking that goes with it. Tracking is so prevalent because advertising is significantly more effective when targeted. The more advertisers know about you, the more efficient their algorithms.
So what type of models does the Web Monetization API make possible? This is a question that I’ve been mulling over since the Grant for the Web program was launched last year – an initiative led by Coil in collaboration with Mozilla and Creative Commons.
To explore new models – I’d first like to explain a little about a passion of mine – Hyperaudio.
Encouraging Reuse and Sharing
Hyperaudio is what happens when you combine hypertext with audiovisual content. The result of this mix is an interactive transcript resulting in highly accessible media – making it more intuitively navigated, shared, searched and remixed.
With other similarly enthusiastic supporters I’ve been working on the technology behind Hyperaudio for the best part of a decade now. We’ve created a number of demos, a working prototype (Hyperaud.io), worked with news organizations and archivists to build out new platforms (Studs Terkel Radio Archive and Palestine Remix) and even applied the technology to build the core of a successful product (Trint.com).
The power that comes from being able to edit and remix media from transcripts occupies a lot of our thinking time and it’s great to see companies like Descript validating our ideas by applying similar thinking to editing podcasts. Further encouragement is supplied by the Hyperaudio community and more broadly the Text AV community which brings together people and orgs from various corners of the globe to discuss and collaborate on text-based audio and video projects.
One of the questions on our minds as we discuss taking Hyperaud.io from working prototype to sustainable product is how do we encourage people to use it? Specifically – how do we reward people for creating remixable content for our platform?
"Hyperaudio Remix Sketch" by Mark Boas is licensed under CC BY-SA 4.0
Web Monetization
The promise of Web Monetization is paying for the services and content we consume proportionally to how we consume them. In that respect it resembles the ad-based model, but that’s where the similarity ends. Unlike the ad-based model there us no requirement to track people. Essentially the way the Web Monetization API works is that by including a tag in your web-page together with your payment code (provided by a number of digital payment services) you can receive payment from anyone with a web monetization enabled browser and a digital wallet. Currently compatible browsers include Puma Browser (natively) and Firefox Chrome and Edge with an extension installed.
We really want to put text-based editing and remixing at the heart of the next version of Hyperaudio (hyper.audio) and encourage a dynamic that recognises and rewards both the creator of source content and the remixer. Remixing is what Hyperaudio is all about, and what the technology underpinning it makes easy – it’s what differentiates us from other transcript-based-media solutions. Elements of media literacy can be taught through remixes – through both the construction and deconstruction of them. It can be hugely beneficial to the creator of source material to have their content exposed within a well made remix – whether that be a digest of podcasts (audio newsletters?) or a video essay.
So the scene is set for the implementation of a new system that will reward content generators, remixers and sharers of that content. How do we achieve this?
A Different Model
Well with the Web Monetization API. One of the simplest ways to apportion monies is by using probabilistic revenue sharing – imagine the remix as a pie, say we give half the pie to the remixer and split the other half among the people who created the sources of the remix.* Basically we’d roll virtual dice and numbers 1, 2 and 3 would mean we add the remixers payment code to the web monetization tag – 4, 5 or 6 and one of the content creators codes would be used. We can refine things further as we know the proportion of content produced by each creator, we can assign probability accordingly. The law of averages dictates that the more a certain piece of content is accessed the more accurate the split, but then micropayments kind of rely on that anyway.
*Note – the split between remixer and source content creator is yet to be decided.
OK, so that’s interesting in itself – we are recognising the remixer as doing valid creative work which has the added benefit of exposing source content creators work.
But maybe you’re tasking yourself – who is going to go to the trouble of remixing before the critical mass of consumers have web payments activated?
A couple of things to bear in mind here:
Remixing content with Hyperaudio is fun in itself.
We'll keep track of how many people view remixes and source material and expose that info publicly, which we hope people find valuable.
But what about people who try and game the system? Say somebody just takes a remix and makes very few or no changes to it? Well, this is where our fascination for the “anatomy” of a remix comes in handy. Not only do we know what makes up a remix, we also keep track of the history of that remix. When somebody remixes a remix, the previous remixers are also factored into the reward equation along with source creators – proportionally – depending on how much of the remix is used.
This may sound a little complicated but it’s all maths and probability – something that computers are really good at, and for humans we’ll take care to create good visual guides to how compensation is split.
So we have source content creators and remixers being rewarded – what about people who share remixes and content? This is perhaps the most radical proposal we’re making – rewarding the sharers.
Some of you may be familiar with how Medium.com implements sharing. To explain briefly – readers can highlight text and share on social media, the social media post this generates contains the text highlighted, together with a link back to the text in the page. We’ll do something similar and (where the license permits) take this further to create rich social media embeds with clips of the actual media shared with overlaid captions – perfect for social media consumption. We’ll also allow sharing of the entire piece, whether source content or remix.
Since we create the shareable link we can encode the sharer’s identity within it, so that if somebody is signed up to hyper.audio we can track who has shared and assign a portion of the web payment to them.
But just because somebody clicks on a link it doesn’t mean that they consumed the media, so we’re going to be keeping track of which parts are actually viewed or listened to with a view to providing the most accurate split possible. This data will also be exposed to remixers and source content creators alike. We think stats of how people’s media is being used and consumed are fascinating and we’re hoping creators will think the same.
Content Federation
In the current version of hyperaud.io we’ve experimented with having a number of subdomains – such as schools.hyperaud.io and chattanooga.hyperaud.io. This makes it easier for groups of similar interests to keep a subset of content. All content currently flows into hyperaud.io under folders named after the subdomain, so everyone can benefit. With the new version we may let the subdomains choose, but would encourage widespread sharing as possible.
Having subdomains also allows us to create safe areas – for schools (say) where we want to keep close tabs on people registered and content added. Another interesting dynamic would be to encourage schools to share content and with web monetization even be compensated for that content.
View Source for Media
We’ve drawn inspiration from many places with Hyperaudio – we’re big fans of the web and we see Hyperaudio as a very webby technology. When remixes are created we do not by default create new files, preferring to take start and stop times of each clip and string them together – by using this kind of loose referencing we provide both immediacy and flexibility. Another advantage of this approach is that the source content of each remix is close at hand as clips are never really removed from context. This is great for teaching media literacy when we want to emphasise the evaluation of content and the establishment of context.
Screenshot of a Hyperaudio Remix authored by School Children in Italy is licensed under CC BY-SA 4.0
These concepts are not new. It feels like a lifetime ago when I was lucky enough to meet up with the inventor of Hypertext – Ted Nelson. Ted showed me Xanadu, I demoed Hyperaud.io and it turns out that these systems shared some common principles. I was forward enough to ask for a quote, which was kindly supplied:
"It's delightful to see what you're doing with http://hyperaud.io/pad – very much the same kind of referential editing that Xanadu is based on." – Ted Nelson
Screenshot of Xanadu by Ted Nelson is licensed under CC BY-SA 4.0
Thankfully, Hyperaudio is not quite as ambitious.
More Than Words
Turns out transcribing audio into text works well for podcasts and vlogs where the spoken word is prominent. But we wanted to cover all facets of audio and so we came up with a system to represent non spoken-word. To achive this we used a waveform font, because it’s just a font we don’t need to change the way Hyperaudio works and so for the next version of Hyperaudio we’ll also make music and other sounds available for adding to the mix.
In our prototype we allow a single background music track to be added – we’ll take this further in the next version allowing any number of music tracks to be added to a remix – we see this as important source content to be factored into the payment equation and are curious to see if people make music available under the Creative Commons licenses as we have seen on other sites.
Screenshot from Wavefont Demo is licensed under CC BY-SA 4.0
Transparent Licencing and Automatic Attribution
It’s nice to be transparent, and when it comes to licencing permissions it’s important to be so, which is why we’ll bake in Creative Common licensing right into the remixing platform – where licensing permits, we automatically generate credits for the remix that will give the appropriate attribution. The idea is to make licensing clear and frictionless. As our transcripts are rendered to semantic HTML we can even express these credits as both human and machine readable markup following the Creative Commons Rights Expression Language (ccREL) proposed standard.
As with the prototype – initially at least we will not host any media on the hyper.audio platform, instead opting to allow users to upload media via hyper.audio to a number of other platforms, we’ve experimented with YouTube, SoundCloud and Internet Archive but we want to add many more – making it as straightforward as possible for users to publish to multiple platforms and share any metadata created on hyper.audio - such as transcripts, captions and licensing as required.
Funding
After much time working on the underlying technology and many conversations on how the next version of Hyperaudio should work, we’re very excited to be embarking on the next stage in our journey. We’ll be applying for funding from Grant for the Web for the web monetization side of the project. Maybe you have an idea on how you might weave web monetization into a product or service, or perhaps fancy working on the monetization technology itself – if that’s the case maybe you should apply too.
We’re always trying to broaden the Hyperaudio community and all our code is free and open source – if you want to be part of this or collaborate in any way my DMs are open.
I’m maboa on Twitter - if you're interested in Hyperaudio and related technology check out my a newsletter https://tinyletter.com/maboa
Title image "The Day I Accepted Everything and Wrote it Down. On the Bus. Next to a Sandwich." by shift one is licensed under CC BY-ND 2.0