Friday, December 30, 2016

Making Better Museum Video with YouTube Data

I presented evaluation methods for YouTube as a part of a collaborative panel presented by the Media Production and Branding Special Interest Group and the Data & Insights Special Interest Group at MCN 2016 in New Orleans. The panel featured evaluation methods research by Trilce Navarrete and Elena Villaespesa, and a case study of SFMOMA video strategies by Emily Robbins. I am finally posting an unpacking of my portion of the presentation and outline some of the next steps for my YouTube video research that was inspired by a presentation by Emily at Museums and the Web 2015.

The tool

I can't take credit for this brilliant tool created by Digital Methods, but it is an awesome resource and organization. They have a collection of wonderful tools, the one I am using is the YouTube data scraper:
Screenshot of digital methods interface.

https://tools.digitalmethods.net/netvizz/youtube/mod_videos_list.php

Using the channel id approach I can scrape the full list of every publicly posted video on a single channel. Then I use chrome developer tools to grab the channel id from the code side by selecting the channel name and locating the  "data-ytid" field. Below is a screenshot that shows this.

YouTube screenshot demonstrating where to place developer tool to get channel ID code.Screenshot of developer tools window showing the field highlighted.


So what channels do I pull data from? In other words, who do I benchmark against? I look at institutions I consider peers to my own. So not only a similar range of content being presented but also a similar budget. In addition to contemporary art museums of a similar size I like to also include what I call aspirational channels. For me this is the Louisiana Channel whose rate of production is beyond anything we could reasonably take on at the MCA, but I love their productions and their channel's high performance reflects that quality.

The data

Once the data is scraped I pulled it into Tableau taking the entire data set by video title and pulling that into the graph and looking at measurable elements such as Duration versus view count with the channel titles sorted by color so that I could identify the densities. Hooray for data visualization!

Tableau screenshot showing views mapped against duration for Hammer Museum, Louisiana Channel, MOCA, Museum of Contemporary Art, Stedelijk, Walker, and Yerba Buena

This map looks at the view count against the date published so that you can see when channels are born and who is doing burst publishing, many videos at time in little bursts as opposed to with regularity over time. In this you can see exactly when MOCA TV was in full swing, particularly the launch, where you see the highest density of publishing.

Tableau screenshot showing views mapped against release date for Hammer Museum, Louisiana Channel, MOCA, Museum of Contemporary Art, Stedelijk, Walker, and Yerba Buena. We see clumps of when popular channels came into existence, like MOCATV during 2013-late 2014.

For kicks I switched view count to like count, and as I presumed likes directly reflect views but with lower numbers so let's just ignore that and look at views where we get numbers that make us feel a little more relevant. Because, as you'll note even in the above graph I capped it at 25k because let's get real, museum video is only so popular as a thing.

Tableau screenshot showing likes mapped against release date for Hammer Museum, Louisiana Channel, MOCA, Museum of Contemporary Art, Stedelijk, Walker, and Yerba Buena. We see clumps of when popular channels came into existence, like MOCATV during 2013-late 2014.

A question that came up from the group when I started sharing my graphs was how often are people publishing content? So This stacked bar graph became a really interesting tool for examining bursts of activity and who is creating at regular patterns, and here we see that the Louisiana Channel's consistency is incredibly impressive. And something happened the week of May 31 at the Walker in 2015, and yes I remembered exactly what it was, the Superscript conference had just happened it was the mass upload of all the symposium content so we'll see a similar burst of activity like this for last week as they just posted their Avant Museology symposium documentation.
Tableau screenshot showing number of releases mapped against release date for Hammer Museum, Louisiana Channel, MOCA, Museum of Contemporary Art, Stedelijk, Walker, and Yerba Buena.

Learnings

Looking at the data I am getting some ideas about good practices, not necessarily best, about what makes video work well. Frequency of posts and short durations certainly seem like important aspects of the content. The data also demonstrates that there are exceptions to every rule. And finally, an easy win if popularity is your goal–Celebrities! 

So how am I applying these learnings if my goal is to get more views and followers?
First what are we making and how is it doing? Our production methods to date have quite simply been to capture more program content and create better and more produced content, the edited narratives that accompany exhibitions.

New practices

Post video clips (outtakes)
Timely release of content
Prototype informal content capture that would be long term sustainable for the scale of our production team.

Wait, is the goal reasonable?

This goal directly conflicts with our distribution method. We embed from Vimeo on our website which is our primary traffic destination from social, so I am not getting any of the Vimeo stats incorporated into our performance. I'll never be the Louisiana channel with that kind of distribution method. So how do I prepare our channel for a future where it is better integrated into a new digital strategy? Or do I let go of my YouTube obsession and embrace alternative distribution strategies and instead focus on the aggregation of the analytics across these media. Perhaps, but I am a huge YouTube fan, and I love the idea of our museum videos being integrated into the fabric of the diverse world of YouTube content, MOCA TV did get that right for a little while. So can we be players in this game and end up a part of someone's YouTube K-hole? Maybe someone discovers contemporary art one long night of clicking.

Next steps


Fill out this survey to participate in the conversation (even if it is months later, the data is still useful). The answers to these questions help the team that worked on the presentation all of this was a part of continue to develop tools to benefit our work.

Create a museum video data exchange. The scraper tool only gets the surface level information, number of views, duration, etc., not the level of detail we can get on our backend analytics. But we can start exchanging CSV files from our channels to compare analytics. Maybe even create a museum video data repository so that we can start being a smart as Netflix and redesigning our content based on how people are engaging. Or maybe not since we can barely keep up with content production, but you get the idea.

In context of the insights we gain, look again at how the mainstream is functioning– there is plenty of information out there, for example– http://tubularinsights.com/youtube-statistics-competition/

Figure out a free way to aggregate data across our platforms, e.g. YouTube and Vimeo. Surely there are services that can do it, but let's roll up our sleeves and figure it out!

Continue to use the MCN network to develop a toolkit for understanding engagement with our video content across platforms.

Start gathering more detailed evaluative data on our video. We do user testing for websites and interactives, why not the video itself? Can we get as clever as Netflix in the creation of our content, where we shape the methods and styles to improve our approach to sustainable content creation?

What makes a good video?