>> DEANNA MARCUM: Welcome to those of you on the webcast. I hope you are following along
and getting a lot from the program. It's very exciting here in person, largely because we
have so many great comments from all of you, and we want to continue that with this panel
as well. We've talked a lot about what is going on right now, what are the gaps, we've
had lots of comments about successful projects, but in this panel, we're going to try to go
a little deeper into those successful projects and think about what would be needed to bring
these successful efforts to a level of scale that begins to feel like part of this national
platform. I'm very happy to introduce the panelists we have talking about this, and,
let's see, in the order in which they are speaking, Jon Voss, who is the Director of
Strategic Partnerships at History Pin. The second speaker will be Anne Wootton, who is
the CEO of the pop up archive. The third speaker is Mary Flanagan, who is a professor at Dartmouth
College, and the final speaker will be Trevor Munoz, who is the assistant dean at the University
of Maryland libraries. When we've talked about gaps earlier today, we've heard indications
of some of the things that are needed, crowd sourcing solutions, computational approaches,
linked data. These are some of the things that our panelists will describe today, and
I hope that as they talk about their projects and how they think they might be expanded
to become something bigger, I'd be very interested in having them comment on what are the emerging
technologies that make these solutions possible, what kinds of criteria can we apply to other
projects to see if they fit into this national landscape that we've been describing, and
what needs to happen to move projects from individual research projects or individually
funded projects to something bigger, and then I have another question I'd like all of you
to think about. We have a number of national organizations that have a mission to serve
our needs, and there are lots of them, but, you know, just for sake of argument, there's
the Library of Congress, the National Archives, OCLC. Internet Archive falls into this category,
although with a different funding model, I will have to say. We have organizations with
national missions. What do we need from them? And I'd really love to hear the panelists
talk about that too. What can we do to make sure that all of our efforts are leading in
at least a parallel direction and pointing to something bigger that we can all be very
proud of? So, with that, I am going to turn to the panelists for your comments.
>> JON VOSS: I guess I will start. I'm Jon. I wear two hats usually, at least, history
pen which is about building community around local history , and on the other side, I co
founded the linked open data libraries, museums and archives summit, and it's the latter part
that I'm going to talk a little bit about, with linked open data, and I have to say,
it's amazing to be in a room with so many of you here and on the Twitter feed as well.
This panel is some of my favorite people that I'm still getting to know, and knowing that
I came completely from outside of the world, somebody passionate about stealing all of
this stuff, and it turns out that people wanted to make it more available, so I'm excited
to be a part of that, and I also say right now I'm a bit sad with Nepal and Baltimore
and extreme poverty, racism, systemic rather, that it's hard to think that what we're doing
is so important and takes such precedent, but at the same time, libraries are a lifeblood
to so many people, and I think it really heartens me to know that particularly public libraries
are there when all other government intervention has failed and to see that again and again,
so it's an honor to be here. Linked open data is something that gets thrown around quite
a bit, so for a second, what is it? And I'll say without getting into any technical detail,
it's ultimately a combination of technology, policy and law that has the potential to build
a more useful, discoverable and interoperable ecosystem, but more than anything, it's part
of a fundamental cultural shift. So, linked open data uses the technology of the world
wide web to share resources in a machine readable and predictable way. Those are the two key
pieces to it, and technically, it's a combination of standards, tools and protocols, but also,
licensing and data normalized to scale. So, this is hopefully moving us from what started
as a web of interlinked documents, remember, 1991, 92, to a web of interlinked data, and
that is ultimately the potential. The environment and the culture of linked open data sees the
web less as a traditional publishing platform, and I think this is part of a cultural shift,
and more as a living interoperable network that provides new and entirely different ways
of engaging and collaborating with communities to enrich collections, connect them to the
universities of knowledge, and build new things from them, which you're going to hear from
the other panelists here and the exciting opportunities around that.
What is it not then? One, it's not magic. You can't write a grant and then say it's
going to be linked open data too. That would be awesome. There actually aren't computer
wizards that can turn you content data into something that it's not. It just doesn't work
that way. There are standards, and there are systems and protocols that allow us to do
things and make things more easily accessible, but it's not going to happen just like that.
You also can't say this is going to solve everything. You can't load lamb something
and expect it to solve all these problems. Finally, it's not something that you can just
buy. You can't say to a vendor, throw the load lamb on it, that'll make it work, it's
got to be linked open data. We can push our vendors to build tools to make things more
interoperable, and that's something we really have to do, but that's more about the policy
bit of it. The potential is huge here. Obviously, unlocking the world's cultural heritage data
for good is a big deal, one big library. You know, this is ultimately where we can start
to bring things together, use them together in ways that we're not just going to one place,
but that we have at our fingertips all the world's knowledge. Well, it's a dreamy big
idea, but crazy enough, the technology is already here. The shifts for us here in cultural
heritage are really more about licensing and policy. On the enterprise data, linked data
is being used in major, massive corporations around the world to link different data sets
around their regions, etc. One minute? Geez. But for us, I think it can be something bigger.
So, let me dig into, you've heard mention of, obviously today Digital Public Library
of America, Europiana, Digital Enzed, these are different massive international aggregates
that are starting to make things useful in more ways. So, what I expect us to see in
my last minute and hope to see is more specifically, and we can dig into this in the Q & A probably,
more multi institutional collaborations focused on specific domains. So, there are more and
more projects that are taking, A, a discreet domain, two, identifiable resources, and hopefully
for multi institutions and building applications and visualizations around them. It doesn't
have to be huge, it doesn't have to be major resource suck. It's experimental. It's 1993
in the world of the web, basically, when we think about linked data. So, I'm hoping to
see more of this community approach, and finally, really, the dream about linked open data is
a community approach in which we can take this technology, use the culture of the world
wide web and let it change everything. >> ANNE WOOTTON: Hi, everyone. I'm Anne, and
I'm one of the co founders of Pop Up Archive, and I think if there's one take away you get
from my remarks today, it should be that I would like to offer all of your moms an API
tutorial, and I think that any of these computational, scalable projects that are worth funding should
do the same, even if they're not able to give you or your mom the tutorial themselves, they
should have extensive and interactive documentation so you can teach yourselves. Some quick background
on pop up archive, and I think the reason I'm sharing my experience right now is because
of the work that we're doing specifically through a hybrid business model to help scale
both preservation of digital audio, we work with disparate sources from individuals to
very large institutions and partner with the Internet archive in doing that, and then also,
accessibility, in part through computational creation of meta data, specifically automatically
generated transcripts and semantic analysis of audio and speech. So, audio/visual collections
have received a lot less attention to date than text and image based ones, understandably,
and fundamental improvements in computational capacity are definitely changing that, but
it's really just the tip of the iceberg. I could talk all afternoon, and I will touch
briefly on some of the recent significant improvements in speech detect software, for
example, but I don't think that my role here today is to share all of those details. I'm
happy to talk about it more, you know, offline, or online.
So, you know, significant improvements in speech detect software that we see already
being applied commercially, whether it's through voice recognition and command, like Siri,
or medical transcription, which Nuance is a major commercial provider of. There are
trade offs, of course, between accuracy and search. One of our engineers wrote a blog
post on the trade offs between accuracy and search for public media, if you're curious
about that, and then the potential for this technology when we apply it to large AV collections
that have previously only been searchable at best via limited meta data and cataloging.
On that note, I would like to mention the importance of combining computational approaches,
whether speech to text or other wave form analysis, with crowd sourcing efforts like
meta data games and tilt factory, which I'm sure Mary will tell you more about with her
work at Dartmouth, so that is automation plus human intelligence, which is critical, and
then BBC's work on the world service archive is a really great example of there are open
source components that they're slowly releasing as a part of that work, but more fundamentally,
we built pop up archive because everyone at small and medium sized organizations was asking
for drag and drop. They wanted something that was as easy to use as Facebook, they wanted
it to integrate with the commercial cloud storage they were already using, and they
wanted to be able to tweet it with the click of a button. So, we had already spent a year
at the UC Berkeley School of Information, my co founder Bailey and I, doing user research
and surveying the broad ecosystem of archival audio. At the same time, bigger organizations
want to handle this computational work themselves, and they have more complex access policies
and rights considerations and should absolutely still be able to benefit from taxpayer funded
projects. So, it requires a very thoughtful balance of outputs, which in my experience
so far, is frankly very hard to strike. So, for this reason, in terms of criteria
and what needs to happen, I would highlight, as many others have today, the importance
of interoperability, accessibility, reusability and educational resources, and this is generally
where API's come into play. More specifically, successful computational projects need to
take into account international standards for AV collections, whether meta data schemers
on the AV side, which are also leveraging linked data to talk to each other, integrate
with national databases, like the DPLA, linked data efforts like what Jon just talked about,
and then arguably one of the most important roles we have played is as a straight forward
web interface instead of educational resources that help producers and archivists ultimately
benefit from research that would sit dormant indefinitely in an open source re-po hub.
The Internet archives TV NEWS archives also comes to mind as a good example of that type
of project. We talked a lot about the various merits and drawbacks as software as a service
versus downloadable open source packages. We work with a variety of these technologies,
and I'll pass on the detail there because we don't have a lot of time, but our experience
is that bringing these research ideas to broad implementation requires a combination of these
technologies, and ultimately, what we're up against, not surprisingly, is that regardless
of size, the organizations that stand to benefit from this type of scaling are limited in terms
of their resources and their aversion to risk. So, I think it's pretty standard for government
funded projects to be open data projects, but there's a big spectrum in terms of open
data that museums and libraries and archives can use and open data that they don't even
have the resources to handle. Small and medium sized organizations don't even have, you know,
they can barely setup their own my sequel database to run Omecca, but then bigger institutions
have all kinds of rules about firewalls and access. The DPLA is doing some really great
work to review copyright policies and publish findings, and we need that work desperately,
because no matter how much we're capable of computational, it hardly matters if no one
is ever going to hear it or see it. So, I'll just close by saying to get these grant funded
projects from research to implementation, it requires resources and vision and will
to act, and in our experience with media organizations in particular, that often means connecting
the potential and vitality of the archive with the immediate pressing revenue concerns
of the leadership, and that is really is a question of technology.
>> MARY FLANAGAN: It's great to follow up with Anne. It's like we're having this very
intense conversation also in our own minds. I'm really pleased to be here representing
the Meta Data Games project which is how I ended up as a designer. Also, the work that
we're doing on a national level building a crowd sourcing consortium, a conversation
about what we can actually do to implement, to come up with best practices and somehow
bridge some of these challenges together. So, folks on the panel, in one way or another,
are really deeply engaged with creating these practical solutions, for engaging with the
public, for making systems that are smart, for making lightweight systems that can appeal
to a variety of institutions, and that was one of the big motivators behind making meta
data games, which is a software platform that actually de couples game interfaces from the
core software architecture so that we can actually rescan front end games and put games
out into the public that solve a variety of challenging issues, such as collecting meta
data tags, verifying or correcting in the case of our work with the bio heritage diversity
library, correcting transcriptions that conflict with one another or that are run through OCR
and have errors, so you can use game interfaces to engage with people in a variety of ways,
and this is all, of course, inspired by Luis von Ahn and his great work with recapture
stuff and Google image search, but we're really trying to keep using these things for an open
source platform. So, the platform is designed to kind of have people make their own game
interfaces as well and own their own data. To date, the project served 45 different collections,
so instead of serving a silo, it's moving across institutions, including work with the
British library, the Holocaust museum, so there's this way we're trying to work across
institutions, and trying to meet the demand of scale is really interesting. Dartmouth
gave us some funding to do some Amazon cloud testing, and we found some really interesting
stuff. Most of the organizations we're working with want us to host their data.
The system is designed so that the data can live anywhere, right, in content hubs, but
actually, people really didn't want their content data to actually merge with, you know,
they want to keep their data kind of pristine and separate, so when it comes to what needs
to happen to make bigger impact, I think in some ways, we need to buy in and trust some
of the techniques to make really strong and accurate information, and also, inclusive
information. It's been touched on earlier today about the way that archives can be democratized
through participation and community engagement, and I think crowd sourcing is a really amazing
way to do this, but we have to let it happen, and we have to figure out how that goes. So,
because of that project and other conversations with Bob Horton, I kind of took it on to start
escalating this conversation, and, so, creating a, in summer of 2014, began to work on using
an IMLS forum award to create the crowd sourcing consortium of libraries and archives, the
SCCLA. So, we've done a number of things in this space. We conducted two national environmental
scans. You may have, some people in the room have probably filled out one of our surveys,
aimed at libraries and archives to understand the field.
So, the first survey with 350 participants found that over 70 percent of the respondents
thought that a lack of technical expertise was a barrier to try crowd sourcing in institutions.
All this stuff is on crowdconsortium.org. It's really interesting information about
an environmental scan. The second survey uncovered perceived barriers to crowd sourcing more
deeply, and that was about 160 participants. In that survey, less than 10 percent of the
participants reported that their institution had incorporated linked open data, so this
is, and the numbers were low about projecting into the future what was going to happen with
that. So, we also did some webinars with crowd sourcing 101, and did some webinars,
and they were quite well attended. People are interested. They want to know about how
to do this stuff, what it would take to scale, and, so, next week, we have planned a culminating
meeting of this project where we have cross institutional support as Brett had mentioned.
The University of Maryland is hosting, and Dartmouth will host engaging the public. So,
there are probably about 50 people, but we also, and we'll be webcasting that too, so
it'll be available, same kind of thing, and so it's really to uncover best practices,
and we're doing a facilitative conversation with a design firm who will really help the
communities pull out and actually get some work done in a practical way. There are a
lot of makers, there are people who really want to try a new way of thinking, so we're
working on that way to figure out how we can think about engaging the public and consolidate
our national knowledge. So, I like the idea that was brought up earlier of what are our
top 10 points. Maybe we can also incorporate that into this conversation since these are
happening so closely. So, I think that's all I need to say right now. We'll talk more about
things in the Q & A. >> TREVOR MUNOZ: Well, thank you to IMLS for
inviting me. It's a really almost impossible task to try to follow these three great people
on this panel, and when we were planning this panel and our phone conversation about it,
we were kind of trying to figure out, you know, kind of the relationship of what I was
going to talk about to the three things we've heard about already, because part of why I
ended up on this panel was to talk a little bit about data curation as a practice that
we might scale up, and I think through that lens, data curation is really interesting,
because I think it's less yet another set of protocols or tools or methodologies and
more something that sits in relationship or that's the strategy that helps us think about
how we deploy all the things we've already heard about, computational techniques and
crowd sourcing and linked open data, and then I think in this regard, it's most useful to
think of data curation as kind of our strategic guide for how we, as cultural heritage organizations
specifically, you know, deploy and interact with some of these, the things that we've
already heard about, because I think the interesting role that a sort of data curation focus plays
in this conversation around the national digital platform is that it asks us to think both
about repeatable units in that kind of business and technology sense of scale that we heard
a little bit about this morning, right? If part of curation is preservation, then we
need to get lean and fast and efficient when we're collecting material from the web or
we're collecting audio/visual material and making sure that that lasts, but I think what's
interesting is preservation being a component of curation, right? We can never actually
get to curation by just collecting more stuff, right?
That preservation is an important part of our mission, but that curation asks us to
do that and to think differently about the materials that we're bringing in, and I think
that's what makes it really interesting and perhaps what might shape how we think about
data curation projects as part of the national digital platform funding priority. In other
words, a lot of the things that we're seeing so far are about helping institutions participate
in the platform to bring stuff to the national digital aggregation or aggregations, and I
would also like us to see proposals and funded projects that are focusing on, you know, in
parallel with that important task, working with communities of users, you know, the absence
of our sort of end users has come up today already, working with communities of those
users to be making sort of active reuse of those materials in parallel with all the sort
of coordinated institutional work that we're doing to bring it in and preserve it. So,
just a couple quick examples of what that might look like, I have a colleague at University
of Maryland who is really interested in virtual reunification of image archives, and, you
know, to see work like that of going, all right, we're bringing materials into these
large national collections, can we also be working at the same time with people who want
to begin to tend those collections at the national level, at that large scale, to think
about the shapes of those collections, about their gaps, so that not only the small and
excellent team at DPLA or at Haughty Trust is charged with enhancing the meta data or
the findability of those materials, but that we also have outside people partnering with
them who have their own investment of the material to also be adding meta data or adding
description, adding value. You know, how do we have projects that bring these meta data
mobs to DPLA and Haughty Trust sooner rather than later, because I think it would be a
mistake to think about staging these sequentially, right?
To get all the stuff together and then worry about, you know, how people are going to be
making use of it, but I think curation urges us, as we think about our active management,
to think about inviting those people in who are saying they want to work on materials
that are already in our national digital platform and begin to enrich them in perhaps more idiosyncratic
ways once they're already there. So, the image reunification is one project. I think seeing
more work around round tripping data back to something like the national digital platform,
right, it's there and I get it and I enhance it, how do I give it back? That's a huge challenge
that we really haven't graveled with. I think, you know, this has come up, you know, getting
serious about, you know, the usability and collection development and sessioning around
our web archives, that that would be another area we could be working on right away. So,
those are some near term priorities that I think we might bring into the conversation.
I think I'll stop there and we can go to discussion. >> DEANNA MARCUM: Great. Thank you all for
really interesting comments. We'll take questions in just a minute. How hard has it been for
you in your individual projects to work with libraries, archives, museums?
>> TREVOR MUNOZ: It's awesome. I guess I'll just say this, and this is about, data curation
is a strategy, right? I work in a library, so, you know, it's great to work with libraries
and archives and museums, but I think, you know, a sense of changing the priorities of
what we spend more of our time on relative to other things has been a challenging conversation,
you know, and having leadership in these organizations, sort of helping shape this difficult conversation
of maybe there are things we need to stop doing so that we can do other things, that's
something I guess I'd like to see more of that would make my life easier.
>> DEANNA MARCUM: Good. >> ANNE WOOTTON: I can say, and I may be digging
my grave even deeper here, but you can add public media organizations. What I'm going
to say, you know, it really depends. Can it be tough and frustrating? Absolutely, but
I was going to mention this actually, there are internal silos, so it, you know, it really
depends which group you're working with, which person you're working with, and part of the
take away's that I've had over the last few years are that the outputs of these projects
need to be understandable and parsible by the internal advocates at those organizations,
or even the people who may be, like, on the verge, tippable, but they need to see, like,
just what they can leverage and benefit from and what that implementation will entail,
and then it can really, you know, open all kinds of doors, because everybody's learning
and it is a question of what the learning curve is and how much you can expect to educate.
>> JON VOSS: I'd say just it varies so greatly by size and the facilities that they have
and the tech resources that they have, person resources, as a lot of people have called
out. At Historypin, we have very different strategies with small and medium institutions.
They can be historical societies, but they have a great deal of access to the social
good, because they're so connected in the local community, but on the technical side,
obviously, we're dealing with big challenges. On the other side of it, on the national/international
aggregators, we can start talking about the linked data. It's very different.
>>MARY FLANAGAN: Yeah, I guess I would say, as a, you know, working outside of library
and institutional archives, the meta data games actually emerged out of collaboration
with an archivist at Dartmouth, and originally, I hope he's not in trouble if I say this,
he said just stay away from the library for awhile, stay away from the archive, think
outside of it. It's by being an outsider in some ways that his argument was you're going
to see this stuff differently, and I would agree with that. I think it's, you know, again,
I'm speaking as an outsider, but I would agree that it's been really helpful, and I've had
nothing but really strong interest in the project, but most institutions do seem to
feel overwhelmed about trying something new when they feel like there are many, many things
to catch up, and they can't necessarily decide which is going to get them ahead faster.
>> DEANNA MARCUM: Let's open it up to the audience for your questions, comments. Did
lunch put you to sleep? >>QUESTION FROM AUDIENCE: I guess I'll say
something. Hi. Ben Virshbaum, New York Public Library and Web PL Labs. Been working with
most of you folks for years on all this stuff, and, yeah, so many great points have been
raised. I think that, yeah, we find ourself in an interesting point, with crowd sourcing
especially. Where, I come from an institution where we did have an appetite to try it and
see if it could be done and there was traction with users, and we now have produced some
significant corps of data that are messy but interesting or not even that messy in some
cases but in some ways curatorial interest has been reprioritized or moved on but public
interest has not so I think and there is this connotention of going after the trend and
going after the long distance run that this kind of data curation, data stewardship implies
and requires for it to really have the impact we all want it to have. Trevor has been extending
that chain of custody with a colleague at University of Pennsylvania, Katy R. that of
curating the menus data from what is in the menu transcription project, and now we are
trying to think how do we get that round trip, how do we get that back, how do we build a
new data infrastructure where we can attach these new forms of data to core records, that
integration question is so important, so this is really just comments. I think Clifford
Lynch spoke about the need to be able to message the public more effectively in a more unified
way about these efforts I think there is a bigger narrative about this kind of work that
we need to work on developing. There is a call to action involved but also what can
we do. This is a great data migration project we are all involved in, I have a lot of metaphors
in mind. Community gardening, but that participation in the act of stewardship of the forward migration
of the data of these collections develops incredible bonds with the public and raises
awareness of these materials, so it does a lot of good with these missions. Trevor has
written eloquently on this before. So it think that that messaging and the narrative of that
work of moving data. It sounds geeky and inaccessible at first but it is translated into very grand
public service narratives that we can develop. And to call back to the mix of human and automated
approaches is really powerful, and that is well established with OCR, with text recognition
and the cleanup. we are seeing very interesting developments in the audio visual space, going
from messy transcripts to with text recognition, to targeted extracted entities, and the cleanup,
with cleaner meta data. We are doing some work with historical maps with the same premise
where we are doing a good first pass on show pass of maps, urban scale maps getting kind
of building shapes and footprints and then breaking the quality control and validation
into smaller pieces. So, I think these are all really important elements, the human computer
collaboration, the narrative of this work, and that figuring out the long distance kind
of stamina required to really see these things through, because I think we're well past now
the question of do people want to do it or will they even participate. They will, and
I think more could. It could scale further. So, those are just a couple of reflections.
Wonderful presentations. Thank you. >> DEANNA MARCUM: Thanks. Comments?
>>TREVOR MUNOZ: Well, just one follow up comment on that. I think the insight of people who
have worked deeply in crowd sourcing is the importance of having the community of people
who want to do something, you know, with the things that are produced from crowd sourcing,
right? They have a research project, whether we're talking about the galaxy folks, where
they're trying to identify galaxies, or as we work with the menus data, part of cleaning
it up is actually figuring out, you know, what matters to the people who are trying
to do some kind of scholarly, or sort of who are hobbyists or enthusiasts about it, what
matters to them, so that community of interest, being there and working with that community
and sort of cultivating them is a real key part of that, and that goes beyond just the
kind of mechanics of, you know, sharing data out and data in.
>>MARY FLANAGAN: I think we can also raise the profile and help with this narrative that
Ben speaks of if we're also, you know, focusing some time on the interoperability and the
round tripping and making sure that organizations can trust, oh, wow, I've seen the map, I understand
the loop, and it's working, and it can work for other groups, and once we can start to
unite the projects, I think that's another way that that can become visible.
>>DEANNA MARCUM: And it certainly helps with the issue of scale, because we'll never be
able to do all of these things if we try to do them ourselves, right? We have to open
it up to others. >>JON VOSS: Ben, I really liked your point
about the timeframes. I think that's a huge bit. We started to consider knowledge communities,
right? So, these were enthusiasts or people who know a location really well, etc. It takes
time for them to trust you, for starters, and then there's also this idea of co creating,
so kind of adding this design theory to not only how we do things but how the community
does things with us, and that's a matter of trust, it's a matter of building their confidence,
but it is definitely a longer term timeframe than a two year grant. So if you get these
grants and you are about to write the reports, now you are ready to get busy with the community.
>>DEANNA MARCUM: Thanks. >>QUESTION FROM AUDIENCE: I want to see what
this group has to say about some other issues than what you've talked about. I think it's
interesting that our panel is primarily addressing things that I would say most are not completely
core yet. We are interested in both those things, trying very hard to work in that space,
but it is still somewhat less core than other things like collective research volumes or
circulating, so one question I have is you have thought about how to scale these, perhaps
becoming core library work, what lessons have you learned that you think might apply to
also thinking about approaches to scaling and sharing other perhaps, you know, even
more fundamental or essential functions of things that, to me, scream out, things like
account management, to have, you know, best in class user accounts and best in class privacy
around that, things like shared discovery layers or tools for creating your own discovery
layer. Are there lessons you've learned from your work to scale these projects that could
be applied in those other problems? >> JON VOSS: Well, a lot of this, for Historypin
particularly, is looking at, you know, increasingly, just design thinking around users, and I think
that's been huge. I don't have a great concept of how things work behind the scenes at libraries.
You know, is there a way you can tell us what you have, tell us how
we can use it, and I think, in the last five years of working on this, it has been, it
started with this is obviously what you need and this is how you use it, and now we're
really going back and redesigning everything and thinking, okay, how is it that you work
now and understand these work flows. A lot of the work that you guys are doing, you think
of partnerships with D schools and design, kind of really looking at that element is
a huge part of how we should approach these problems with the user in mind.
>> ANNE WOOTTON: Successful complex systems are essentially much more simple, and to start
building something complex from scratch, it's almost doomed to fail, and the reason pop
up art exists is because Bailey and I spent a year plus in graduate school at a school
of information which was very focused on design thinking and iterative software development,
and it was totally new to both of us. We were humanities people, former journalists and
didn't even realize just how steeped in it we were until we left, but having that time
to go out and do user research, and, I mean, I'm finding surveys that we did in 2011 that
I don't even remember that are still relevant and informative to the work that we're doing
now, and I think, you know, in the day to day, at most museums, libraries and archives,
there just isn't time to do that kind of work, and when you're applying for grant funding,
it's often the component that ends up kind of hitting the cutting room floor. I know
there was some conversation about this earlier today too. I think it's really important,
not just for documentation and maintenance to be a part of these projects, but also for
user research and education to be a part of it too.
>> TREVOR MUNOZ: I guess I want to pick at the question a little bit, because I feel
like there are a lot of things that libraries are doing where they are trying to share some
of these core things, a number of these activities, but I guess what I find that might be translatable
is a lot of those activities, whether it's, you know, the tools around integrated library
systems or shared print repositories or something, they feel, at the moment, like they're being
done in a kind of back of the house, let's sort of keep the bother out of the site and
our communities, and that, I think, is a different spirit than a lot of these things that aren't
yet core but we want to become core. I mean, I think a lot of the case that many of us
are making are about, you know, the excitement or, you know, the engagement that we can get
out of doing this new thing, and that's a lot of the argument that we're making to our
leadership, and I don't see that same kind of argument being made around some of these
other really key important things, like, you know, what should the library discovery layers
be like or our e book activities or shared print, so I wonder if that's the thing that
needs to come across and that sort of thinking about, you know, how is this not a back of
the office task but a part of our sort of engagement with our community of users and
people that care passionately about all this stuff that we have. I don't know, that's a
sort of half formed >>QUESTION FROM AUDIENCE: (Off mic.)
>>TREVOR MUNOZ: Well, I mean, just, I think it's, to me, it resonates more with actually
the point you were making about e books, right? Like, in a lot of sense, I think, you know,
it's to not be dependent on vendors, but we haven't gone all the way through to saying,
like, what would it mean for us to have sort of self determination in this area, right?
Like, the ILS could become a powerful and exciting thing, that could be a bit much,
but could be kind of a positive thing if we thought about it in terms of taking back ownership
of our relationship with our communities of use and having that be sort of the central
narrative of the project instead of just being, like, let's be another entrance into a marketplace.
>>ANNE WOOTTON: I'm just going to qualify what I said earlier. When Pop Up Archive started,
we never thought that machine transcription would even be a part of what we did, so we
were taking people back to square one and saying fundamentally, for a lot of producers
who found themselves archivists, what is your ideal relationship with this material that
you're creating and becoming stewards of, whether you like it or not, that you recognize
the value in, but fundamentally, you know, what relationship to this would work for you,
how can we, we talk about making archiving part of the production work flow, right, and
fundamentally, what should that look like, let's talk about that, let's ask a lot of
people to draw their visions of what that might look like and then try to boil it down,
and for us, the computational piece was, like, icing on the cake that has become something
I'm happy to talk about but wasn't even why we started doing this.
>>MARY FLANAGAN: Just really quickly, you know, in thinking about crowd sourcing and
tagging, it seems like we're actually gathering data, and the original emphasis was to do
that about archival material we didn't know much about, and that is still a really helpful
thing, but I think what we're hearing over and over again as we iterate these projects
is that we're really, software developers are really engineering the social and that
maybe libraries, museums and archives are really far more social designed than they
are artifact designed, and if that's the case, has anyone played trivia crack? Okay, thank
you. Thank you. So, I was, all my students play trivia crack, it's an app, and I was
just, this is an anecdote, I'm just going to go there though, so I was diagramming for
my students in my game design class, okay, well, how does it work? It's a trivia game
online. You think, oh, the majority of the task is about gathering questions, but users
can submit trivia questions, and if you actually look at the ecosystem of trivia crack, about
10 percent is really time spent with the trivia question, and that's not what you would think
if you just saw the words, well, I don't know what you would think, but if you said I'm
going to go play a trivia app, you would not think of it as 90 percent social engineering,
right? And I think that this is something that just emerges more and more in the work
that I've been doing. It becomes far more of a design challenge in the social sphere
than the technological sphere, although the technology is, of course, in there, iteratively
so. >>DEANNA MARCUM: Question down here?
>>QUESTION FROM AUDIENCE: Michael again. Sorry. I want to throw the gauntlet down a little
bit about scale. I wonder throughout the day and throughout this project if we're talking
about an incremental change model or, I don't want to use the word radical, the way Google
talks about a 10 times improvement model, and I have the feeling that we're thinking
about scale collectively in a very small way. A lot of us were really profoundly struck
when Ted served its billionth video in 2013. When we look at the traction, the profound
user engagement you get on Reddit. When our curators appear on Reddit. It is many orders
of magnitude, some of you have seen what I've been talking about with the vlog brothers,
Hank and John Green, just tens of millions of people raising millions and millions of
dollars, and I wonder if we're subconsciously designing a project that's just going to be
a little bit better, sort of a digital version of the levels of scale that we get with traditional
libraries, or if we're really thinking about, you know, within a decade, we're going to
have seven or eight billion people online, the kinds of scale that could and should be
designed with them in mind. Most of the next five billion are going to be from kind of
marginal governments, most of them are going to be poor, most of them are going to be hungry
for the kinds of resources that these organizations stand for. What are we going to do when they
arrive? What are we going to do now so that when they arrive, there's something for them
that's good? >> JON VOSS: I'll start. Thanks for that,
Michael. Yeah, man, let's do it this way. I mean, the things, and I've sworn to myself
I'm going to make it through without swearing once, so the stuff that Seb is doing that
is so exciting to me is that if you close your building down for three years and you
reinvent everything, what would you come up with, and that's the kind of stuff that people
are coming up with, and what would happen if we closed our libraries down completely
for three years? I mean, obviously, you wouldn't want to do this, but that's the kind of thinking
where we could get it to scale, where we could start to say how do we build everything either
around the content, around the people, what are the assets that we have, what does that
look like. That's a very exciting conversation to have. Let's do it right now.
>>DEANNA MARCUM: What are those features? >> JON VOSS: I don't know. I think to me,
I would start by building something in the environment of the web. I mean, that's what
I love about what Seb and Aaron and the whole team has done over there, is that the database
was the key to everything. You could surface things, you could change things around, and
I would say it wouldn't be just what I have in my own institution, but what everyone else
has. So, if we all started collectively to go at it that way, now you've got something
very different, and I would also say these institutions have huge social value, and it's
been completely underestimated, and that's something that I would want to focus on as
well, you know, what do they really do to strengthen the communities. Nobody really
talks about libraries in that way, but that's a huge element so the evaluation part of that
is a factor too. >>DEANNA MARCUM: Mary, are you ready to jump
in? >>MARY FLANAGAN: Well, I think it's a great
question, and it's always in a planning meeting, you know, how far can we go, right? The futurists
have different visions of what 2030 will look like. So, I think, regardless of what kind
of radical change we have, we know that there are certain pain points anyway, and that may
be the place to focus, so that we can actually allow for, like, talking, having data be able
to be free and talk to each other. We're going to need that no matter what happens, so being
able to incorporate the public in a new way has to be a part of this conversation. So,
I think that we have some really interesting seeds for that kind of radical conversation.
>>ANNE WOOTTON: I want to be at the table for that conversation. I don't want to repeat
myself, but I think we, you know, I like to frame our work often as, you know, what we
can do to make audio findable by Google, right, and so I come at it, especially working in
public media and, like, RSS feeds, this open protocol where we can go out and identify
all of this media that is, you know, digital radio of today, and, you know, assess its
value independently and incorporate it into the work that we do for that reason, and I
think, yeah, you know, I'd love to be a part of that conversation about making it happen
more systemically. >>TREVOR MUNOZ: I was reflecting on something
that came up in the conversation earlier this morning about, you know, if we look at the,
you know, the communities that many of the institutions that we work at consider their
constituency, it's actually not a national constituency, it's something much more local
to them, and there's a great, you know, sort of, there's a great inertia to stay, you know,
to preserve the institution which serves that local constituency that provides our little
local resources, and I don't know, maybe part of the way to break that open, you know, because
I think it would be difficult for me, you know, to tell the state of Maryland, for instance,
you know, like, you know, the state of Maryland will get some good stuff, but we're off on
this national thing, you know, this is part of the problem that we all have in many of
our institutions, and that part of perhaps what some of us are struggling toward here
is the way in which something like the national digital platform, though I think the portfolio
or some other term might work better here, is, you know, the narrative that allows us
to sort of make that argument that helps us kind of break open our local institution a
little bit more, and I wonder if that helps us toward some of the challenge, Mike, that
you're pointing us to. On a technical level, right, I think it is about, like, that the
web is the way to go, you know, that that's kind of a proven direction, but, I mean, there's
so many other dimensions to this about sort of, you know, the institutions of realities,
and we have to break those open too in order to sort of really attack your problem of scale.
>>MARAY FLANAGAN: I love the idea of, like, thinking about, okay, why isn't, the ask me
anything, why isn't that a reference library. To really engage with the public and interface
where people are at and the tools are at is really exciting.
>>DEANNA MARCUM: I think you're the next person. >> QUESTION FROM AUDIENCE: Blain Dessey, Library
of Congress, and I represent one of those institutions that Deanna mentioned, but I
have a very practical question, and coming at this from a national institution, or even
if I were from a funding agency, when we talk about scaling to practice, what does that
mean tangibly? I'm not a technologist, I'm just a bureaucrat.(Laughing.) That's what
I am. It's not that funny. But, you know, if you were to come to me and say we got a
scale to practice, this is critical, what does that mean? What would I do? Do I start
convening meetings tomorrow? Do I buy people software? Do I write reports? What are we
talking about in terms of people in positions of influence being able to move this forward?
>>JON VOSS: It might be to get out of the way, honestly. That might be part of it. I
do think that the working on the web and the way of the web is a totally different cultural
construct, and, I mean, it goes along with Michael's thing here too, if we're going to
talk about this kind of a scale, we're making, we are making a lot of small incremental changes,
and even if you look at the meta data that we're dealing with, you know, we've gone from
card catalog records to something very different now when we're starting to put things on the
web, that's a whole different thing, and we don't want to make those little adjustments,
we want to do something quite a bit differently, and I think, for me, from somebody who's a
policy maker or somebody in a major institution, it's how do you enable the people on your
teams to do this quite a bit differently and experiment with things quite a bit, you know,
and not necessarily radical, but entirely different ways. That is how people are communicating
differently. Without also alienating, or keeping in mind your end user or ultimate goals, but
it's not to be an institution. >>TREVOR MUNOZ: I mean, I would suggest that,
like, if we wanted to talk about scaling up practice, I mean, maybe you would start off
with a meeting in which you'd bring together everyone in the organization and say here's
where we're trying to get, and we're going to change everybody's job, we're not just
going to add a new person to do this new thing and the other nine of you are going to keep
doing what you were doing before. I think that would be a really critical thing, if
you were going to start talking about scaling up practice. You know, it's got to be everybody's
job. >>ANNE WOOTTEN: We are all pretty wary of
not reinventing the wheel, and when it comes to scale, that's, like, paramount, so to give
your team, to have that meeting and say let's go out and see what's been done, and this
is why I talked about the importance of any of these projects, especially when they're
federally funded, having outputs that are digestible by the people at organizations
who are going to be able to take that information and figure out how they work it into what
they're trying to scale at their organization and how they can leverage that or latch on
to that and then in turn, you know, provide back to their work more information, more
tidbits that others can add to sort of the collective knowledge.
>>MARY FLANAGAN: I want to take this to the next week meeting. This is a great question.
>>DEANNA: Tom. >> QUESTION FROM AUDIENCE: Tom Cramer from
Stanford University. There was something that happened, Jon, while you were speaking. I
just wanted to talk about briefly the value of verbs or the value of tools, which is the
way, I think, you scale practice, and the way you make things accessible to a greater
group of people is by giving them a simple tool that does a job well and that people
understand the value, and so you can pop up archive, you can Google things, you can Reddit
things, you can even do data curation, I think we know what that is. I think from someone
who is working on link data, It's still too abstract. Crowd sourcing is kind of a mindless
panacea. I think from a national digital platform perspective, one of the things that this group
might do is identify what those little tools are and really drive the identification and
adoption of those tools and really get more people playing in the milieu.
>>ANNE WOOTTEN: Great. >>DEANNA MARCUM: Jim
>>QUESTION FROM AUDIENCE: Yeah, I wanted to talk a little bit about Blain's question,
what should the national organizations do, and I'd like to, I think there are four areas
where they can be very helpful. One is the area of advocacy, talk this up, making it
clear that this is a national priority. Two, information policy, helping us get the right
legal and legislative framework so we can better serve our users through content and
technology. Third is professional development. We'll hear more about that, but I think these
organizations have the ability to build the type of leadership and professional development
capacities for the wider community, and fourth is global leadership, positioning us around
the world in terms of how we're going to knit together these various initiatives and to
help us address the question of Europeana is easy, but how do we deal with the developing
worlds of Africa, Asia, South America, and really begin to address those questions, because
these organizations have important connections in those areas.
>>DEANNA MARCUM: Great. Thanks, Jim. Any response from the panel? Just gratitude, I think.
>>QUESTION FROM AUDIENCE: I'm Jeff Resnic. I am with the National Library of Medicine.
I direct the history of medicine division. Particularly, as many know, I was talking
with Brewster about it earlier today, about NLM and the national institute of health leading
the way in open access, and this is something we continue to do despite challenges, and
we hope that those efforts and the resources can be of use to this community. Looking forward,
you know, if there are resources, the national library of medicine is one of America's three
national libraries that could be useful, I'd be happy to connect with you and have a conversation
and invite you out to our institution. Just wanted to put in a plug there. This has been
an incredibly engaging conversation, and from that vantage point of one of the other national
libraries, I think that the idea of knowledge communities as they are defined in the medical
or scientific context, certainly the national science foundation plays a role, but there's
no other place in the United States that generates so much medical knowledge, and the national
library of medicine is a storehouse of that through our databases and various other resources,
so just to throw that out there, to think about, you know, as we grow these communities
of knowledge, as we find their interconnectivity, what about the personalization of that knowledge
as it relates to health, as it relates to well being, I think that's a dimension that
I'll call out. Finally, one of the premier advances that's unfolding now at the national
institute of health is the precision medicine initiative that President Obama announced
in his State of the Union earlier this year, and the NIH director Francis Collins is firmly
behind this, and I think that defines, in many ways, the personalized nature of data
of information vis a vis health in this world of exploding data, so how do we navigate that,
and I think all the questions that you're raising, you know, overlaying them on top
of personal data, individual data, in a world of, you know, a pool of data, it puts a particular
spin on this. >> DEANNA MARUM: Extremely good point. Thank
you. Lucy? >> QUESTION FROM AUDIENCE: Hi. I'll just quickly
add a fifth one to what Jim added as something that I think we could be working on, even
as other tools and issues need to be being developed for this portfolio, I'm using the
new word, is that we need to be actually working with the users of the future on educational,
on their digital literacy in order to use these resources and make of them what they
will so that they can make videos or reuse videos to make new projects or do what Trevor
was talking about, I'm sorry, Rich does, or encourages people to do with the products
in archives, to make new, reuse digital materials and make new things. So, I know we sponsor
a new small grant to encourage literacy and engagement with historical materials, but
obviously, there's a need to encourage that kind of work with data materials, all types
of, you know, for reading, I mean, education, the department grant, education does it too,
but, I mean, that type of development of younger people needs to carry along, or else, in a
sense, we're building a product that you're going to say, well, they're just going to
grow up in it, you know, they're the digital generation, they'll know how to use it, but
I think that there's been, you know, enough work to show that they don't just know how
to use it, but to keep working on that side, it also teaches us what we don't know about
how they use things, which is equally important, the case user studies and all that part of
it. So, that's my fifth part. >>DEANNA MARCUM: Are there other people ready
to speak? >>JON VOSS: In the meantime, I just want to
get back to the sciences part there, because I think we're pretty humanities heavy in this
meeting, so I really like the science point, and when you talk about, particularly linked
data, scaling things across connective archives, databases, huge advances are happening there.
You also, you have so much more funding in those areas. I mean, when we talk about doing
linked open data projects within humanities, the scale is so much smaller, and I think
there's a lot to be learned from what's happening in that space that can be transferrable, and
I love this idea of intersection, the data exploration, you know, how you visualize this
data. It could be pretty intense. >>DEANNA MARCUM: I think we all have so much,
we're so much in the debt of the national library of medicine for doing so much good
work in this area, so I think, it's really great to see you here so that the humanities
and the sciences can talk together a bit more. >>MARY FLANAGAN: Thank you, Deanna. That sounds
like a wonderful note for us to wrap up on. Jim just asked me if there was chocolate,
so. (Laughing.)
>>DEANNA MARCUM: Please join me in thanking the panel for their very good comments.
(Applause.)