Digitization 101

2017 has been quite a year.  I even I blogged more this year than I had since 2013!  Looking back at those posts, and what was happening elsewhere, this is what stands out to me:

• It seemed as if the Library of Congress Copyright Office would go through an upheaval this year, but that did not happen. The Copyright Office still reports to the LOC and I've not heard any recent public discussions about moving it elsewhere.  Given how politics is infusing everything, the fact that the Copyright Office is staying as is, may be a good thing.  We don't need that office becoming part of a political jousting match.
• Nothing happened in terms of updating the U.S. Copyright Law (Title 17).  Yes, there are sections that need updating and that updating requires deep thought, not just quick action. When the Congress does consider changes to the law, I hope they will do so without a political agenda and without undue influence from their major donors.
• A phrase we are hearing every day is "fake news." One way of combating fake news is to providing reliable and verified information resources.  Thanks to libraries and other information providers who have placed reliable and verifiable information online for others to use.  Thank you for providing not just one side of story, but providing many sides.  Thank you for digitizing older information, which helps us put into context what is happening today.
• Funding for many government agencies is in flux. That means that either funding has shifted away from them or there are rumors that they will lose funding.  Among those agencies is the Institute of Museum and Library Services (IMLS).  It is important that the IMLS survive and thrive. Why?  Quoting the IMLS:

  The Institute of Museum and Library Services is the primary source of federal support for the nation's approximately 120,000 libraries and 35,000 museums and related organizations. Our mission is to inspire libraries and museums to advance innovation, lifelong learning, and cultural and civic engagement. Our grant making, policy development, and research help libraries and museums deliver valuable services that make it possible for communities and individuals to thrive.

• One of the ways IMLS helps all of us is by hosting events, where we can talk to each other about specific issues. One such event was held in October on the National Digital Platform.  We all should be grateful for those institutions who are willing to develop a digital platform which will help all of us.
• This year Apple released its iPhone X, marking 10 years of increasingly sophisticated smartphones.  Smartphones and other digital devices are all around us. There are a growing number of wifi and bluetooth enabled devices.  An interesting activity is to count the number of wifi devices in your home.  (One friend counted 30!)  If you are surrounded by smart devices, then it may be hard to remember that is not normal for everyone.  Yes, there are people who are still using very basic flip phones.  And there are people who need to borrow wifi hotspots from their local libraries, so they can wifi at home.  There is still a digital divide in 2017 and there is still a need for digital literacy training.

Personally:

• I had never marched in a protest before, but in 2017 I marched in four events held in Syracuse: The Women's March, the March for Science, the People's Climate March, and the Procession of Neighbors. The latter was in support of the immigrant and refugee communities in Syracuse. 
• Out of my blog post about the March for Science came an invitation to be part of the book Science not Silence: Voices from the March for Science Movement, which will be released by MIT Press in March 2018.
• I wrote a series of blog posts on Upping You Library Intelligence, which were well received. 
• In collaboration with Copyright Clearance Center's Beyond the Book podcast, I released a blog post on library deserts which was paired with a Beyond the Book interview on the topic.  That blog post was  read over 3200 times making it my most read blog post of 2017.

Lastly, a one negative that is not attached to any one news event:

• The feeling of not belonging.  That feeling swept over immigrants, people of color, women, and many others.  One good aspect of social media is that we have each been able to find a tribe online were we do belong, and were we can be supported.

I'm sure there is much more that I should be noting and likely your list would be quite different., and that is okay.  What will 2018 have in store for us?  Let's hope that it provides lots of positives!

As the web site says:

Recommended Format Specifications are hierarchies of the physical and technical characteristics of creative formats, both analog and digital, which will best meet the needs of all concerned, maximizing the chances for survival and continued accessibility of creative content well into the future. 

And:

The specifications seek to provide a framework within which creative works should have the flexibility to grow and develop, and also help ensure that these creative works be accessible and authentic into the future.

The Library of Congress identified six basic areas of creative output: Textual Works and Musical Compositions; Still Image Works; Audio Works; Moving Image Works; Software and Electronic Gaming and Learning; and Datasets/Databases. Technical teams worked to identify recommended formats for each category and to establish hierarchies of format preferences.  The Library is committed to reviewing these specifications on an annual basis.

The Library of Congress noted that:

The specifications which the Library is now publishing do not replace or supersede the Best Edition Statement, which provides guidance to publishers and creators in fulfilling their obligations with regard to the registration or deposit of their works under the terms of the Copyright Law. Instead, it seeks to complement that work, building upon the knowledge gained from working with the Best Edition Statement and providing a broader set of recommendations, aimed at providing guidance and clarity in a creative world, which is both rich with potential and rife with pitfalls, and afforded numerous competing options for information format or container.

The specifications are available at:

• Recommended Format Specifications (HTML)
• Recommended Format Specifications [PDF: 660 KB, 17 pp.]

This is an excellent resources, which I hope many organizations will rely on.  The layout of the document and the level of detail included makes it easy for anyone to use, from students to digitization managers.  I encourage you to spread the word about its existence.

On May 21, I spoke at the annual conference for the Society of Ohio Archivists. Thanks to the event organizers and especially Ron, Glenn and Eric for their help in getting me there and back, and ensuring that all went well for my presentation.

The event was held at the Ohio Historical Center in Columbus, OH, which has nice space for events of this size. The event included sessions on digitization and other topics of interest to archivists. Participants came from across Ohio and even from as far away as Chicago!

I gave the plenary session entitled “What We Are Learning From Google & Flickr About Digitization and Partnership”. An excerpt of my slides is below. What I wanted to do was to get people to step back and think about what Google and Flickr are, and to think about how we are (or could) interact with them. Reactions to my presentation indicated that I truly did open people's eyes to the breadth of what Google is doing and introduce people to a side of Flickr that some were unaware of. In fact, even though it seems like Flickr is heavily used in the library and archives community, there were people in the audience who did not know about the service.

As I reflect on the questions that were asked and the conversations that occurred later in the day, these thoughts come to mind:

• Collaboration is the way to go when thinking about digitization.
• Hooking up with a for-profit organization has its benefits, but we need to keep in mind that they are focused on profit, while we're focused on service.
• We all need to be aware of those for-profit organizations (e.g., Google and Flickr) that are working with non-for-profit organizations and what they are doing. This stuff does get reported in the news, on various web sites and in a number of blogs. We need to be seeing/skimming/reading the announcements.
• We need to be mindful of all that Google is doing. We blind ourselves to the company's power when we think of them as only a search engine.
• While Google in not the only search engine in the world, Flickr is also not the only photo-sharing service. Each service has its pros and cons.
• Google is challenging how we think about digitization (e.g., access vs. preservation, quantity vs. quality). It is important to know what is being sacrificed and what benefits are being received.

Again thanks to the SOA for allowing me to speak to them on this topic. I'm glad that I was able to present compelling information and provide a different perspective. (An excerpt of my slides is below.) I also appreciated and learned from the sessions I was able to attend, and truly enjoyed touring the digitization facility at Ohio Historical Center.

---

Technorati tags: Digitization, Google, Flickr, Archives, Library of Congress

Time flies and it has been nearly a month since the FLICC Forum in Washington, D.C. The 25th Annual FLICC (Federal Library and Information Center Committee) Forum was held the day after the 7th anniversary of 9-11 at the Library of Congress. (I flew to D.C. on Sept. 11.) Last week, Natalie Gozzard posted notes from the Forum. Let me supplement those with some other thoughts. (BTW there were 10 speakers in total.)

The Librarian of Congress, James Billington was on hand in the morning and gave introductory remarks. He also helped to hand out several awards, including one to Jill Golden, an SU student, who is working in Germany. During her acceptance speech, Jill said:

I love the joyful moment when people are connected to the knowledge they seek.

I was on a panel about "Web 3D and Virtual Worlds" with Stefan Geens and moderated by Cindy Hill. Stefan Geens oversaw the building on the Swedish Embassy in Second Life. While I enjoyed hearing Stefan talk about the virtual embassy, I enjoyed more hearing about his life as a telecommuter. Stefan is from Belgium, currently lives in Egypt and works for Sweden. Now that is a 21st century worker!

In the afternoon, Aneesh Chopra, the Secretary of Technology for the Commonwealth of Virginia spoke on how they are making government information more accessible. WOW! He said that the public web site promotes transparency in agency outcomes. Virginia has won awards for what they have done in making information accessible and building a great online presence. Chopra was a very good speaker and should be heard by other's in government who are doing more thinking than doing.

Bonnie Klein also spoke in the afternoon about "Marking of Government Information." The vision is to more clearly mark information produced (or disseminated) by the government so that its copyright status is clearly known. Right now there are no standard markings. As users of government information, having documents clearly marked would be a great benefit.

The last speaker of the day was Paulette Robinson whose role was to be the "weaver." The weaver listens to all of the presentations and pulls out the highlights as well as those topics that seems to be present across presentations. I had never been to a conference where someone was assigned this role AND had time on the agenda to present her results.

Finally, I should say something about the security in that part of Washington, which is very close to the U.S. Capitol. Yes, some streets are blocked off and you need proper ID to drive through, but walking was okay. Yes, there was security as we entered the Library of Congress, but not as drastic as what we go through at the airport. Yes, everywhere in that area, the security was noticeable. It does bother me that we have to be this cautious, but I'm glad that it is not too, too onerous.

---

Technorati tags: Copyright, Second Life, Library of Congress

Okay, what makes this article so interesting for me is that it describes the Scribe book-scanning system that has been developed by the Internet Archive. Here are the details:

• Combination of hardware and free software
• "takes high-quality images of books and then does a set of manipulations, gets them in optical character recognition and compressed, so you can get beautiful, printable versions of the book that are also searchable," according to Brewster Kahle
• Works with Linux (Ubuntu); support for Windows has been dropped
• Books are held in a V-shaped cradle
• Uses two digital cameras
• "Free software is used almost every step of the way"
• Books are scanned at the Library of Congress and the files are processed at the Internet Archive (ah, the Internet) using a cluster of 1,000 machines
• "Image processing for an average book takes about 10 hours on the cluster, and while the project still uses proprietary optical character recognition (OCR) software, Kahle says that many open source applications come into play, including the netpbm utilities and ImageMagick, and the software performs 'a lot of image manipulation, cropping, deskewing, correcting color to normalize it -- [it] does compression, optical character recognition, and packaging into a searchable, downloadable PDF; searchable, downloadable DjVu files; and an on-screen representation we call the Flip Book.'"

The Open Content Alliance has 40 members and is currently digitizing 12,000 books per months across five locations.

And what is being digitized at the Library of Congress? Well, it includes "Civil War and genealogical documents, technical and artistic works concerning photography, scores of books, and the 850 titles written, printed, edited, or published by Benjamin Franklin." Very cool!

---

Thanks to the digitizationblog for finding this article. Mark, you truly made my day!

---

Technorati tags: Brewster Kahle, Open Content Alliance, digitization

On March 20, the Librarian of Congress -- Dr. James H. Billington -- gave testimony before the
U.S. House of Representatives Subcommittee on the Legislative Branch. Here are some highlights from his testimony (which is 7 pages in length): [my emphasis added]

• It took two centuries for the Library of Congress to acquire today's analog collection—32 million printed volumes, 12.5 million photographs, 59.5 million manuscripts and other materials – a total of more than 134 million physical items. By contrast, with the explosion of digital information, it now takes only about 15 minutes for the world to produce an equivalent amount of information. Researchers at Cal-Berkeley produced estimates of the amount of information produced and circulated on the Internet in 2003 – it was equivalent to 37,000 times the content of one Library of Congress. Most of this information exists only in digital form: so-called born-digital items, many of which are already irretrievably lost.
• The average life of a Web site has been estimated to be 44 to 75 days, and information not actively preserved today could literally be gone tomorrow.
• To cite some examples: of a sample set of 56 primary sources identified by the Congressional Research Service to support research on Hurricane Katrina in 2005 and 2006, 21% were no longer available on the Web in 2007. Web sites relating to the national elections of 1994-the first time the Web played a role in such elections—have also vanished. It was not until 2000 that the Library began preserving election Web sites. Political scholars wishing to write the history of how the Web has influenced politics will have to do so without important pieces of the puzzle.
• In late 1994, we launched a program to digitize 5 million items of American history and culture for educational purposes—the National Digital Library. The budget was $60 million with a 3-to-1 private match for every dollar of congressional support. By the end of the 1990s, the Library had well over 5 million items of American history on-line. We have continued this process and now have more than 11 million items on our American Memory Web site for educational use by teachers and librarians. The Library has benefited from the support of the Ad Council to promote the Library's educational and literacy programs. Our overall Web usage climbs continuously and now stands at more than 5 billion electronic hits each year.
• Through NDIIPP [National Digital Information Infrastructure and Preservation Program], the Library has built a national network of 67 partners to collect, save and provide access to a body of high-quality research and educational content in digital form. We have been working closely with content providers, technology innovators, libraries, archives, and end-users to advance the science and practice of preserving important at risk materials that are perishable and often exist only in digital form... The Library now manages a total of 295 terabytes of digital content, including 66 terabytes of digital material preserved by our partners across the nation.
• Just as the Library has acquired, preserved and made accessible more than 134 million traditional analog items (books, manuscripts, maps, music and movies), we are now applying the skills and values of traditional librarianship to the digital world. I have been told by members of Congress and their staff that if they want information, they simply find it on Google, and you can indeed find a flood of information on Google – sometimes hundreds, and even thousands, of sources for a single query. Our goal is to integrate the best available electronic information into the knowledge, judgment and wisdom contained in books and in the minds of our curators so that Congress and the American people continue getting the same authentic, reliable information and knowledge that have been the hallmark of the Library since its inception in 1800. [Comment: Even with the Library of Congress nearby, members of Congress are relying on Google and other search engines for their information needs!]
  
• The scope of our digital strategy encompasses every aspect of the Library and envisions our playing a central role for the nation in three ways: (1) digitizing and distributing online for educational purposes primary materials from the Library of Congress and other repositories, (2) gathering and preserving in the Library and other cooperating institutions important digital material produced elsewhere and in danger of disappearing for use by Congress and the nation, and (3) converting as many of the Library's processes and products into electronic and digital forms as possible.

There is also impressive information in his testimony about the new National Audiovisual Conservation Center (NAVCC) located in Culpeper, VA. This indeed will be a state-of-the-art facility.

At the local level, we are not always cognizant of what the Library of Congress is doing. Therefore it is good that Dr. Billington's testimony is available online. Hopefully many people will at least skim it.

I am always hoping that those who are learning (those who are running ahead of the pack and blazing new or different trails) will give presentations are conferences and write articles. So...those of you at the Library of Congress...please make your rounds and visit the library conferences. Talk about what you are doing. Show us pictures. Bring us up to speed. We're all ready to learn from you.

---

Thanks to M.R.Weaver, one of my students, for pointing out this testimony. I noted in our class discussion that those who are job seekers should note this quote from the testimony:

We have shared with Congress some of our ongoing efforts to ensure the professional development of our staff, training, mentoring, and performance planning and evaluation. We have a large number of staff who are retirement-eligible, and we will have to hire many new staff with specialized skills that are often hard to find.

---

Technorati tags: Digitization, Library of Congress