Digitization 101

Between 1998-2000, I worked on a digitization demonstration project in Fairport, NY on women's suffrage.  The web site for Winning the Vote has changed since then, but it does still exist. What also still exists are the lessons I learned, and there is one that I want to talk about in a new context.

Then...

For each suffragist profiled, we had a biography written.  As the project manager, I reviewed everything before it went online, including the biographies. As an African American, the biography of Frederick Douglass caused me to ask questions. The acceptable words used to describe an African American have changed drastically since the 1800s. While it is possible to write about Douglass without using any of them, what words should be used to describe his second wife, who was not a woman of color?  We all knew that detail needed to be stated, but what would be the correct words? I spent a long time asking people of their opinion.  I even asked the question on a couple email discussion lists. I didn't feel as if I received good answers and my choice - European American - was not yet a term that was widely used (and it still isn't).  Thankfully, our project historian located text of Douglass talking about his second wife and that text is how we talked about her.  He said:

No man, perhaps, had ever more offended popular prejudice than I had then lately done. I had married a wife. People who had remained silent over the unlawful relations of white slave masters with their colored slave women loudly condemned me for marrying a wife a few shades lighter than myself. They would have had no objection to my marrying a person much darker in complexion than myself, but to marry one much lighter, and of the complexion of my father rather than of that of my mother, was, in the popular eye, a shocking offense, and one for which I was to be ostracized by white and black alike. (Douglass, Life and Times… p. 534.)

Now...

Many weeks ago, I participated in a webinar where the first two speakers started their presentations by placing themselves in a theoretical or cultural context.  When it became my turn, I quickly did the same, although I had not planned on doing so. At this point, I don't remember what I said about myself, but it likely included that I come from a corporate background and that I'm originally from south-central Pennsylvania (and yes that does matter).  I believe the other two women included in their descriptions the theories they use for their mental models.

A few weeks ago, I was at a training session where we were asked to provide our preferred pronouns when we introduced ourselves.  This was not my first encounter with the need to do this, but the first time that one person's preferred pronouns (they/them/their) caused a bit of angst among a couple of the participants.

When we look at historical figures - those who are no longer living - we often have to put them in context, because they did not do that for themselves. Most did not publicly state what words they wanted used when describing them.  Nor did they state the framework they used when thinking about an issue.  We use whatever information we can find to try to build that context, knowing that it could be quite flawed.  A good example of this is Eleanor Roosevelt's relationship with two of her female friends.  People guess and speculate, but Roosevelt left nothing behind to put those friendships in a context, which answers the questions we have.

While we were fortunate with Winning the Vote to find text of Douglass talking about his second wife, we really don't know what words he or she (Helen Pitts Douglass) used to describe her ethnicity.  Now, however, we have an opportunity to build the context for a living individual whom we are adding to a repository (e.g., The History Makers).  What might we capture in text or metadata?  The first thoughts that come to my mind are:

• How the person prefers to be addressed. This would include pronouns as well as  honorifics.  I think of Mrs. Medgar Evers (Myrlie Evers-Williams) who has spoken publicly on what it means to her when someone she does not know calls her first name.
• Better information on the person's ethnicity.  With more people having their DNA tested, we should capture more than the category the person fits into for the Census.  Personally, my ethnicity is more complicated that I thought, based on my DNA results, yet I identify out of habit as being African American.
• The words and phrases the person uses to describe himself/herself/themselves.  This might be how the person describes their work or personal life.  For example - and thinking of a family member - is the person an architect, artist, professor, or all three?
• The person's gender, gender identity, and sexual orientation. While a person might not want to have this shared publicly, I would hope that the person would understand its usefulness, in terms of context, in the future.  I think of David Bowie, how he lived his life, and then the speculations which occurred after his death.
• Information on what influenced the person.  This could be where the person grew up, what tradition the person was trained in, or something else.

Yes, that would be work and, yes, that would be helpful.  In 100 years, when the words we use to describe people have changed again, knowing how someone described themselves would solve a headache that I know will occur.

By the way, perhaps we each should get this started by writing this information for ourselves and placing it somewhere were it can be found (online or offline).

---

The photo is of a Carte de visite showing Frederick Douglass. This work is the collection of St. John Fisher College, Lavery Library.

The book title 1984 was written during World War II and has had an impact on generations since then.  It has become the book title to note when talking about surveillance, a topic that is now frequently in the news.  And it came to my mind as I watched Cybercrimes with Ben Hammersley: Surveillance, a BBC news program.  The half-hour program - like many others - talked about Eric Snowden, but what stood out to me was what it said about metadata.  No, this wasn't the first time that I'd heard the terms "surveillance" and "metadata" linked together, but this time I recognized an opportunity.

All librarians understand that how an item is described is important.  Library users make many decisions based on those descriptions, which are now in the form of digital records.  In college, I worked for a librarian (Mrs. Martinez), who enjoyed describing a new book and creating the best catalogue record for it.  I can still see the smile on her face when she had succeeded!  However, we see the catalogue description as leading to an item.  In surveillance that description - the metadata - can be more important than the item itself. 

In surveillance, the metadata of a thousand items (people, phone calls, emails, etc.) can be mined for what could be useful information.  The mining is done by computers and the work can find a "needle in a haystack", or in other words a connection that might not have been found otherwise.  It is humans that launch the computers on these tasks and humans who then look at and analyze the results. 

Like everyday, today I generated a fair amount of metadata.  I used my computer on the Internet, as well as my mobile devices.  All of the apps and web sites likely generated data as I used them, and some of that data I willingly shared.  Likely there is other data that was captured without my permission or knowledge.  My car does not have all of the up-to-date technology in it, but it does have some sort of a computer (and every computer generates data).  And since I was out driving around, it could be that a police officer captured my license plate with an automatic license plate reader.  If I happen to use my credit card today, that will also generate data.  And, of course, I watched cable TV and the cable service tracks some information about my usage (or how would it know that it can raise its rates and people will not complain?).  Given all this data that I'm creating, my privacy is an illusion.

However, all of this data has created an opportunity for us (librarians).  We are the people who understand how to describe data elements, so that the correct information is captured.  We're the ones that understand how to map (crosswalk) data elements across systems (and imagine how many systems are being involved with this worldwide).  We have the skills to help extract data - broadly or narrowly - and then analyze the results.  And...we understand the ethical use of information.  (And I will argue that people who understand the ethical use of information should work for organizations that do not use information ethically.  We need to be in those organizations and part of their internal conversations, which is how we might help them to change.)

If working with this type of data and metadata is of interest to you (or someone you know), consider what you need to do to get the job you want.  Perhaps you need to take university courses.  Perhaps you need a degree or an advanced certificate.  Maybe you need to do other professional development.  Likely you need to do some research on the jobs available and the organizations that have those positions.  You likely need to do some networking, both as a way of learning more and as a way of getting yourself known (e.g., SLA's new Data Caucus).  How long will it take you to get a job in this area?  That's up to you and may be dependent on what skills you need to develop.

It has been said that Eric Snowden's revelations didn't change anything at all.  It could be that the "change" needs us.  We're the missing ingredient.  Who among us will get involved?


Resources:

• 1984 book summary
• 1984 Wikipedia page
• Cybercrimes with Ben Hammersley: Surveillance - The web-version of this program is not available within the U.S.
• Can Snowden finally kill the 'harmless metadata' myth?
• NSA stores metadata of millions of web users for up to a year, secret files show 
• Metadata: Surveillance state of confusion 
•  Stanford students show that phone record surveillance can yield vast amounts of information

The second edition of ‘Preservation Metadata,’ written by Brian Lavoie and Richard Gartner, focuses on new developments in preservation metadata, since the last report, made possible by the emergence of PREMIS as a de facto international standard.

Specialists in the field of electronic information provision for digital preservation at OCLC Research and the Centre for e-Research at Kings College London, Brian and Richard pick up from the first edition of the report, reminding us ‘it is no exaggeration to assert that preservation metadata, and the PREMIS Data Dictionary in particular, have become part of best practice underpinning responsible long-term stewardship of digital materials.’

The report goes on to outline key implementation topics that have emerged since the publication of the PREMIS Data Dictionary, including community outreach, packaging, tools, PREMIS implementations in digital preservation systems and implementation resources.

Neil Beagrie, Director of Charles Beagrie Ltd and Managing Editor of the DPC Technology Watch Reports praises the new edition, noting that it “is a deservedly popular report first published in 2005 …extensively updated to reflect developments over the past eight years in preservation metadata practice.”

Adrian Brown, Director of the Parliamentary Archives concurs, calling it ‘an excellent report, clearly and accessibly written, neutral, thorough, and fulfilling the brief. It is likely to be of interest to the DPC membership, and also to a much wider audience.’

The report will be well received by digital preservation practitioners interested in learning about the key developments in preservation metadata, especially as these developments concern the PREMIS Data Dictionary; and will appeal to anyone seeking to learn more about the general topic of preservation metadata.

The not-for-profit DPC is an advocate and catalyst for digital preservation. The coalition ensures its members can continue to deliver resilient long-term access to digital content and services through knowledge exchange, capacity building, assurance, advocacy and partnership. Its primary objective is raising awareness of the importance of the preservation of digital material and the attendant strategic, cultural and technological issues. The DPC Technology Watch Reports support this objective through an advanced introduction to topics that have a major bearing on its vision to ‘make our digital memory accessible today.

The second edition of ‘Preservation Metadata’ is the latest in the state of the art Technology Watch Reports that give an advanced introduction to ensuring that high-value and vulnerable digital resources can be managed beyond the limits of technological obsolescence.

Read Brian Lavoie and Richard Gartner’s report ‘Preservation Metadata’ by downloading from the DPC website now: http://dx.doi.org/10.7207/twr13-03

If you know anyone who is a professional photographer, you need to tell them about this blog post.  Why?  Because the metadata that person has "in" his photos, that could connect the photo to the photographer, is being removed.  According to the Embedded Metadata Manifesto:

...a number of the more popular services strip this embedded information from images when the images are uploaded to the services or processed on their servers.

This created quite a discussion in one of my classes last week! We recognize that this stripping of metadata is affecting all of us, but we know that it is affecting professional photographers even more.  Embedding metadata in a photo is done by the camera and editing software.  Metadata - including information on who took the photo - can be quite helpful when searching across photos or even demonstrating ownership.  Strip that information out of the photo and that photo could have been taken by anyone at any time.

Why would a social media site remove the metadata?  My guess is it is part of the file resizing that occurs, since sites like Facebook really don't want to be storing large files.  They would prefer to resize the files to something more manageable.  That does mean that the file you have on Facebook is really not the same as the photo that you took.

The report does include information on social media sites that do not strip out the metadata.  I'm saddened to see that Flickr, which is use heavily, didn't do well in the tests.

Today, Henrik de Gyor spoke via Skype to my class that is studying digitization (Creating, Managing & Preserving Digital Assets).  One of the questions asked was about the skills digital asset management professional need and he responded by pointing us toward this blog post on the topic.  Later a student asked which coding language I thought he might learn and my response was XML.

There are other markup languages, like HTML, which means that learning one (like XML) will help a person learn others.  I think anyone who is working with web sites needs to understand markup languages. (A point that was proven later in the class, when one of the students led us in a short Drupal workshop.)

Henrik mentions the LAMP stack in his blog post, which includes Linux, Apache, MySQL, and a scripting language (e.g., PHP, Perl, or Python).  I know from conversations that organizations need people who understand MySQL as well as a scripting language.  I also learned from my student this afternoon that originally you needed to know LAMP in order to install Drupal, so these are indeed worth knowing...as are the other coding languages and technologies that Henrik mentions.

How do you learn these technologies?  Yes, you may be able to find a class to take.  For those that want to learn on their own (and can be motivated to do so), there are sites like Codecademy that can help.  No matter how you learn, be willing to learn through trial and error.  Honestly, you will learn more from your errors than from your successes.

Our conversation with Henrik covered many related topics in an hour, including metadata. Once we had ended the Skype session, I told the class my latest metadata story.

On Monday evening, I was walking to my car when my cell phone rang.  Imagine my surprise when the person on the other end said she was from NBC News!  She had been trying to contact me to ask in person to use a photo of mine on the evening news.  The photo taken at Florida Gulf Coast University, that carries a Creative Commons license, had already been used by the Atlantic Wire, which is how she found out.  But how did the Atlantic Wire find it?  Metadata.  (I'm sure of it!) [If you do not watch NCAA basketball, the you don't know that FGCU has unexpectedly advanced in a tournament and gained national attention.]

I must admit that this was not the first photo that has caught the attention of a company (NMAI, New Orleans, Lucille Ball's grave)  I have a good eye, a decent camera, and a drive to add metadata to my photos in Flickr.  In fact, when I'm looking for a photo to use, I turn to my Flickr account first because I know that I've done a relatively good job organizing and tagging my photos there.

Is metadata important?  Yes...vitally important to our digital libraries and our DAM (digital asset management) systems.  But there is nothing better to demonstrate the power of metadata than being able to say that metadata led to a photo from Flickr being on the evening news!

I tweeted that (above) from a session at the ALISE 2013 conference that was about the Tea Party and Occupy movements.  What does it mean, really?

We've all heard or said things that weren't exactly correct, but whose sentiment was correct.  It could be a misquote or even a made-up quote from someone.  It might be totally contrived like the image to the right.  The question is...if the sentiment is correct, does the "exactness" matter?

While this question was raised in talking about content produced by social movements, I find it interesting from the point of view of our digital archives.  We want our archives to be accurate, but does that mean that we want the content to be verifiable or true?  What if the sentiment was accurate, while the words were not?  Would we place these things in our archives for future access?  If yes, how would we label them?  Personally, I think the descriptions could be tricky, since it could mean understanding who create the information and why.

As I write this, I am also thinking of images that have been Photoshopped liked the one, several years ago, of the British royal family where they altered Prince William's smile.  Was the image accurate?  No. Yet is was an official photograph and has been archived as such, I'm sure.  I wonder if the metadata includes a note saying that William's smile is a fake?

The 67-minute video of this CNI Project Briefing will be of interest to librarians, those involved in digitization, and Dead Heads alike.  Very interesting to hear them talk about how they acquired content, how it was processed, etc., as well as how they are crowdsourcing some of the information.

Description: The Grateful Dead Archive (GDA) at UC Santa Cruz represents one of the most significant popular culture collections of the 20th Century and documents the band’s activity and influence in contemporary music from 1965 to 1995. At CNI’s spring 2012 membership meeting, UC Santa Cruz University Librarian Virginia Steel and Project Manager Robin Chandler discussed the particular challenges of merging a traditional archive with a socially constructed one.

In 2011, Debra Schiff did a series of video interviews with Peter Hirtle.  Peter talks about EAD, partnerships, risk (copyright) mitigation, and more.  Since he is always someone worth listening to, here is a link to Schiff blog and the videos.

On April 4, the Google[x] team unveiled "Project Glass", which makes eyeglasses a computer interface device for both input and output.  A video introduction to the concept is below.



There is obviously recognition software involved in this (voice and image) as well as fine-tuned geographic information systems, among other things. The video does not demonstrate anything that would require intensive amounts of metadata, however, I could see that descriptions (metadata) of things and places would be necessary.  That would allow for logical connections to be made between places, events, objects, etc.  The question is, who will create that metadata?  Can it be automatically (of semi-automatically) generated?

By the way, for me, this is reminiscent of SixthSense, which was demoed at TED in 2009.

I had not considered the ethics of metadata until a student mentioned it in a literature review.  The American Bar Association has a web page on the topic, Metadata Ethics Opinions Around the U.S.  The page states:

While metadata is often harmless, it can potentially include sensitive, confidential, or privileged information. As such, it presents a serious concern for attorneys charged with maintaining confidentiality -- both their own and their clients. Professional responsibility committees at several bar associations around the country have weighed in on attorneys' ethical responsibilities regarding metadata, but there is no clear consensus on the major metadata issues.

The page then contains information for specific jurisdictions on:

• What is the Sender's Duty When Transmitting Metadata?
• May the Recipient Review or "Mine" Metadata?
• Must the Recipient Notify Sender if Metadata is Found?

While this page deals specifically with attorney, I wonder if libraries, museums and archives have found that they need to be careful about the information that they put into metadata fields?  Do you have concerns about privacy or confidentiality when creating metadata?

There is still time to register for this....

---

• "Excellent - I would recommend DF to anyone anticipating a digitization program"
• "I was very pleased. The team was exceptionally knowledgeable, friendly and personable."
• "Thanks, it has been an invaluable experience."
• "A really useful course and great fun too!"

• Planning and management
• Fund raising
• Understanding the audience
• Social media and its impact
• Metadata - introduction and implementation
• Copyright and intellectual property
• Sustainability, value and impact
• Financial issues
• Implementing digital resources
• Digital preservation

---

• Electronic records and their management: key concepts and terminology; unique properties of electronic records; implications of these properties for their management; understanding and coping in hybrid paper and electronic environments
• Incorporating electronic records management into the information fabric of your organisation: drivers for electronic records and information management; key elements of a successful electronic records program; mandates and policies; technical and social infrastructure; establishing collaborations with key personnel
• Getting started: simple first steps to begin managing your organisation’s information; understanding your organisation’s information landscape; sources of information to help you manage information; gaining practical experience; learning from the experience of others

• Computer science basics for recordkeeping and preservation
• Digital archiving and preservation processes: comparing digital archiving and analogue archiving technologies
• Systems to support digital recordkeeping and preservation: current status and limitations; the Open Archival Information System (OAIS) standard
• Metadata standards and metadata maintenance
• Organisational and business considerations
• The future agenda: how might the management and preservation of digital records evolve in future?

---

Suzanne R. Graham, Cataloging Services Librarian, University of Georgia School of Law Library
 
Can our users get where they need to go?  Community tagging helps to solve this.

The things Suzanne wants to look at for tagging were:

• things that were unindexed - Moving to new software, information had been lost  How could tagging help recover this?  (e.g., genre headings)
• users had unsuccessful searches - She can get a report of searches with no direct hits.  Niche searches.
• things that were unintuitive (unintuitive cataloguing)  - e.g., adding "NAFTA" to records about that treaty.
• places where user had tagged - look where broader tags might help.  She runs reports in her system to find places like this need to be enhanced.

New tags are placed in the 655 field.  That field is shown in the software's tag cloud.

Focus on access.
Be efficient, effective  and reversible.

Gretchen Caserotti, Head of Children and Teen Services, Darien Library

They are using SOPAC2 for their catalogue.

Kids do everything in a keyword approach / Amazon mentality

For example - Eight grade students are required to read a memoir, but how do they find the right memoir?  The catalogue returns 760+ hits, so how do they get to something more appropriate?  She had staff go into the catalogue and tag two books each with the tag "middlesex memoir" to create a list of pre-selected titles.

Kids are using tagging to keep track of their summer reading - what they read and/or what they want to read.  Having unique tags for each kid isn't a problem and it makes it fun for the students. 

Young people are willing to jump in and get their hands dirty.  We need to let them.

Question - How do you get staff to participate in tagging?
Answer - Limit how much you are asking staff to do.  Can the do one each?  Keep it bit size.  Over time, they will realize that it is not difficult.
Answer - Tags can create a cheat-sheet that the reference librarians can use when the desk is busy. 

Question - How do you prevent people from creating inappropriate tags?
Answer - You can't, but you'll find those tags  You can delete if you need to...and they haven't needed to.  User reviews seem to be fun and honest in terms of whether people are pro or con.

Question - Are adults tagging at Darien?
Answer - Yes, some users are engaging with the materials, e.g., writing reviews.  She thinks that they are also tagging.  Her focus, though, is working with kids.

Question - Can they identify individual patrons from their tags?
Answer - They could track back to figure out who the registered user is, but they don't (Suzanne).

DAM LA 2010 highlights all the important issues - from the fundamentals of how to get started with a DAM solution to the latest and best practices in the management of digital media. Attendance at DAM LA 2010 ensures that everyone involved in the capture, storage and application of digital media assets is fully briefed on the latest developments and best practices.

Since I don't describe myself as a metadata librarian, metadata can make my eyes cross. However, I have discussed metadata in this blog (quite amazing!.  So this Wednesday night, I want to curl up with cup of coffee and highlight several Digitization 101 blog posts on it.)

• Metadata (definition)
• The "M" word
• Taxonomies
• Talking across time and space 
• Metadata is still not understood
• IL2007: Searching, Metadata, and my final post about the conference -- Read the notes about Tom Reamy's presentation

Want to dig into the archives yourself?  Use the "popular labels" on the right side of the blog OR use your favorite Internet search engine to search this site (e.g, site:hurstassociates.blogspot.com plus whatever terms are relevant to you).