lies, damn lies, and metrics

In his 2014 Code|Words essay, Michael Peter Edson waxes prophetic about the “dark matter of the Internet,” heralding the beginning of a new age, where everything is interactive, collaborative, creative. In the list of factoids he uses to support his theory, he cites number after number: 140 billion Facebook images; 185.4 million Tumblr accounts with 83.1 billion posts; a billion TED talk videos “served”. (Having been on tumblr and Facebook for some time now, I can tell you that it’s likely only a miniscule percent of those posts and images are original, but copyright in this “new” web community is a topic for another time.)

Even while touting the measurement these huge digital impacts, Edson seems to look down on traditional GLAM institutions for measuring their non-digital impacts:

“Some of the disconnect between what institutions could do online and what they do do online can be attributed to the clothesline paradox, a term environmental pioneer Steve Baer coined to describe the phenomenon in which activity that can be measured easily (e.g., running a clothes dryer) is valued over equally important activity that eludes measurement (e.g., drying clothes outside.) The same can be said for the way in which institutions habitually value activity such as visits to museums or journal articles published by their scholars over equally meaningful but more difficult to measure activity such as the sharing of museum-related materials on social media sites or the creation of Wikipedia pages.”[1]

The core concept is this: everything that matters is measurable, and old-timey institutions like museums and libraries are just measuring the wrong thing.

A recent case study of an impressive suite of digital offerings from the Cleveland Museum of Art seems to have taken this idea to heart. The authors are clearly and justly proud of the work the team at CMA have done, and their responsive, dynamic Collection Wall sounds like something I’d like to play with. But in the enumeration of all the digital tools they use and all the data they gather, the story seemed to turn slightly from “gather data to help find new ways reach our patrons with our collection” to “gather data… to gather data.” As they put it:

The visitor-experience backbone captures “user” moments (i.e. transactions) that document a person’s relationship with a museum – from parking garage entrances and special exhibition entrances, to attendance at museum cocktail nights, children’s art-class registrations, donor-circle renewals, and volunteer shift check-ins.[2]

I can’t be the only person that finds this a little creepy.

Counting things – patrons, tickets, blog posts, images, whatever – makes it easy for us to tell a story about “progress” :  “One year we had x widget uses logged, and the following year we had x + 100 widget uses logged. We’re aiming for an even bigger number this year – maybe as high as x + 250!” These stories make it easier to justify our continued existence, help us find (or keep) funding, and make great bullet points in marketing materials. But what are they really measuring? At the risk of sounding incredibly trite, can you measure inspiration? Is there a metric for Eureka moments?

A few days ago I read[3] an entertainingly foulmouthed essay by instructional designer Sean Michael Morris on critical pedagogy in the age of digital learning. Morris warns that in the age of student metrics and constant tracking, it is too often the case that “Learners […] are very much the objects of the learning process, and not the subjects. Learning is being done to them while they’re observed to see how well the technology is working, or the content… to see, to put it frankly, how well learners are responding to the treatment.”[4]

While the digital initiatives and user-data-analytics world of libraries and museums is more ethnographic (what are they doing?) and less experimental (what do they do if I do this?), it feels as though there is still a distinct separation between the content holding institution and the user or patron. Are patrons learners? Are museum staff and librarians teachers? We should be. And if Edson’s utopian internet dark matter is to exert its force, the patrons will need to be teachers and the staff and librarians learners – which is difficult if not impossible to build analytics for.

Just as it is important to ask ourselves why we want to use this digital tool or that interactive design, it is important to ask – and to justify to our patrons – why we gather the data we do, and how we plan to use it. What will our patrons walk away with? Can we and should we measure that?

 

 

[1] Michael Peter Edson, Dark Matter: The Dark Matter of the Internet is Open, Social, Peer-to-Peer and Read/Write—And It’s the Future of Museums, 2014, https://medium.com/tedx-experience/dark-matter-a6c7430d84d1#.n7rbitvn9

[2] Jane Alexander and Elizabeth Bolander, “A Digital Road Map: Developing and Evaluating Museum-wide Digital Strategy,” in Technology and Digital Initiatives, Juilee Decker, ed. Lanham, Maryland: Rowland & Littlefield, 2015.

[3] On Twitter, where I have 755 followers as of today – how’s that for a countable thing

[4] Sean Michael Morris, “Critical Pedagogy in the Age of Learning Management.” January 24, 2016. http://www.seanmichaelmorris.com/blog/2016/1/24/critical-pedagogy-learning-management 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s