Digitization 101

As received in email.

---

Five members of the International Imaging Industry Association have created this educational web site geared toward end users.  The site is divided into five categories:

• Learn
• Prepare
• Protect
• Recover
• Resources

While you may not have use for this site, you may not of a user (perhaps a friend or family member) who would benefit from seeing it.  Go ahead...pass it along!

These guidelines (four pages) are not specifically for a digitization program, but they still may be useful.  At least worth taking a look at.

Bill LeFurgy, digital initiatives librarian at the Library of Congress, wrote:

I often hear from students and others with questions about the skills they need to compete in the job market. What programming languages should I learn? How much do I need to know about specific digital formats? Which standards should I study in detail?

 “No, no–those are the wrong questions!”

That’s what I want to shout when I hear this line of inquiry. But I don’t yell because it’s rude and because I know the impulse flows from my own bias toward broader, more adaptive skills.

Read his complete answer, especially if you're a student wondering about the skills you really need!

As received in email...

---

The Metropolitan NY Library Council has a libguide (bibliography) on digitization.  The guide include links on planning, image manipulation, copyright and more.  Missing is information on software (collection management systems).  Currently, METRO is using CONTENTdm and Omeka, although some if its members may be using other software.

I believe that this guide was recently updated, which is always good to see.  Often resource lists are placed online and then forgotten which leads to broken links and outdated information.

METRO has been very active in digitization for a number of years.  It recently released its book Digitization in the Real World, which contains case studies of small and medium-size organizations that have embarked on digitization program.

Recently, Andrew Stawowczyk Long from the National Library of Australia posted a message to the IMAGELIB discussion list about this free image to PDF batch converter that he had written. (He noted that is seems to work well, but has not been tested extensively.)  That makes me wonder if there is other free or open source software that people are developing for digitization programs and where it is all housed.  Is there a repository for this stuff somewhere?  If you were trying to find free or open source software for your program where would you look?

Received via email.

---

Organised jointly by the IOP Printing and Graphics Science Group and the University of the Arts London (Materials and the Arts Research Centre - MATAR), in association with the Society for Imaging Science and Technology.

Programme

The two-day international conference aims to inform those responsible for the preservation of digitally printed materials about developments in digital photography and printing technologies and progress in research on inks and substrates and their significance for conservation and preservation issues and techniques. We also hope to develop links between related industries and the conservation/preservation world.

The event is aimed at an international audience of photographers, conservators, preservation personnel, conservation scientists and those working in the digital printing, ink and paper industries.

Further information about the conference is available at http://ppp10.iopconfs.org/index.html.

Registration Open

To register for the conference, please visit the website at http://ppp10.iopconfs.org/Registration/page_37015.html.

Enquiries:

Dawn Stewart
The Institute of Physics,
76 Portland Place,
London W1B 1NT, UK.

Tel: +44(0)20 7470 4800
Fax: +44(0)20 7470 4900
E-mail: dawn.stewart@iop.org

The following arrived in email...

---

---

Technorati tag: JPEG2000

Michael Bennett & David Lowe released two emails this year about a JPEG 2000 survey that they did. The first email said:

First, thanks very much to those who responded this past fall to our survey of digital project staff regarding JPEG 2000 implementation at your institutions. We have made the results available via our institutional repository at: http://digitalcommons.uconn.edu/libr_pubs/16/

You may choose to download the survey results as a standalone .xls spreadsheet file or, if you prefer a somewhat smoother viewing experience, download and extract the .html version contained in the zip file at the same URL.

From our abstract:

The survey results reveal several key areas that JPEG 2000's user community will need to have addressed in order to further enhance adoption of the standard, including perspectives from cultural institutions that have adopted it already, as well as insights from institutions that do not currently have it in their workflows. Current users are concerned about limited compatible software capabilities with an eye toward needed enhancements. They realize also that there is much room for improvement in the area of educating and informing the cultural heritage community about the advantages of JPEG 2000. A small set of users, in addition, alerts us to serious problems of cross-codec consistency and they relate file validation issues that would likely be easily resolved given a modicum of collaborative attention toward standardization. Responses from non-users disclose that there are lingering questions surrounding the format and its stability and permanence, stoked largely by a dearth of currently available software functionality, from the point of initial capture and manipulation on through to delivery to online users.

And then (second email) they announced that they had presented their findings at IS&T Archiving 2009 Conference. The final report is available at http://digitalcommons.uconn.edu/libr_pubs/19/ and the presentation is online at http://digitalcommons.uconn.edu/libr_pres/21/

JPEG 2000 is being adopted by more and more organizations. Surveys and reports like this will help us understand who is using it and why, as well as the issues or concerns that they run into. One day, JPEG 2000 may be the norm. For now, it's still a decision that people must make. I hope more continue to say "yes" to it.

---

Technorati tag: JPEG2000

As has become my tradition at year-end in recent years, I'm stopping to take a look back and a look forward. What has stood out to me in 2008?

• Google: (posts on Google)
  
• continued to grow its books digitization program by adding more collections/libraries to it.
  
• branched into digitizing magazines.
  
• joined an effort to digitize newspapers.
  
• added photos from Life magazine to its image collection.
• settled issues related to copyright and its book digitization efforts.
  
• Microsoft ended its book digitization efforts (post).
  
• A side-by-side demonstration of four automated book scanners was done in Germany (post, post). Since most organizations could never do that, it was good to see a group take on the task. Also good to see that a report to be written on it.
• JPEG2000 has gained wider acceptance. (related posts)
• "Digital preservation" discussed as not being the correct term. Should we instead use words that describe the outcome not the process? (post)

Personally, I need to mention a few things, too:

• My work on the New York State Regents Advisory Council on Libraries. This has taught me things about libraries, library support agencies, and state budgets that I would have never learned otherwise. (posts)
• Writing Federated Search Report and Tool Kit for Free Pint. This report took much effort and thankfully has been well-received. (post)
• The changes coming to my work life in 2009. Actually, when you teach at the college level, there is a lot of work that must be done before classes begin, so I've already been working at my new job, but officially begin in a few days. I am definitely looking forward to the balance and interplay of teaching and consulting. (post)
  

And what do I see for 2009?

• Google will continue to grow. I do wonder if/when Google will find itself with the same fate as DEC, IBM and others (fallen and no longer relevant).
  
• Organizations and government entities will start to ask questions (more loudly) about the wisdom of relying on major corporations for creating, managing and preserving digital assets.
  
• More organizations will cooperate to build sustainable digitization programs. Not only will libraries, museums and archives cooperate, but for-profit organizations will also find it useful to collaborate (on digitization, equipment and software development, and digital archives).
• Shifting budgets will cause organizations to be more creative. An unusual benefit of a down economy is that organizations turn to more creative solutions; solutions that they would ignore in good economic times.
  
• Unfortunately, many small organizations will still be unable to digitize materials that deserve broader access. I keep waiting for something that will help all small organizations jump on the digitization bandwagon...and waiting...and waiting.
  

Thank you for continuing to read this blog, for the comments and emails that you send, and for recommending this blog to other people. As you can see from the ClusterMap on the left side of this blog, this blog is read by people around the world. Your comments and questions, as well as the number of people who read this blog (and their locations) help me stay dedicated to writing nearly daily on digitization and its related topics.

If you are reading this blog and would like to receive updates in email or through your RSS reader, it is easy to subscribe to this blog to that can happen. Having updates delivered to you can make reading any blog easier for you.

Finally, may you have a Happy New Year! I know that 2008 has been a tough year for many people and organization because of the economy. At the moment, 2009 looks like it will throw us more challenges. However, may you and your organization find ways of getting through 2009 with your sanity, budget, etc. intact.

Related blog posts:

• Thank you (and some statistics) (2007)
• Digitization 101 Year in Review: Words to Take into 2007 (2006)
• Digitization 101 Year in Review: News (2006)
• Digitization 101 Year in Review: Influences (2006)

---

Technorati tags: JPEG2000,Copyright,Digitization,Digital Preservation,Regents,RAC,Book,Federated Search

Several people blooged iPRES 2008, which was held in England. Below are e excerpts from their posts. Much more information is available if you follow the links back to the complete posts.

Trends in archives and libraries are toward JPEG2000 as an image storage format, even though it has to be converted before browsers can use it. Oya Rieger, talking about digitization of books, cited the smaller file size compared to JPEG, and the support of archival features such as incorporation of metadata into the file.

PREMIS is gaining traction as a format for preservation metadata. For instance, the British Library is using PREMIS 1.1, along with METS and MODS, for an eJournal archiving project. The article is available on the D-Lib site.

Stephen Abrams gave a presentation on JHOVE2, which is still in the design and prototyping stages. It's getting obvious from various discussions that the problems in JHOVE are becoming more of a concern. -- File Format Blog

Oya Rieger has been speaking about their large scale book digitisation processes. They first entered an agreement with Microsoft, and later with Google; they were naturally very disappointed when Microsoft pulled out, although this did give them unrestricted access to the books digitised under that programme. On the down-side, they suddenly found they need 40 TB of storage to manage these resources, and it took a year or so before they could achieve this. Oya related their work to the OAIS preservation reference model, and it was interesting to see not only that infamous diagram, but also a mapping of actual tools to the elements of the process model. It’s worth looking at her paper to see this; I noted that they were using ADORe for the archival storage layer, but there were several other tools that I did not manage to note down. -- Digital Curation Blog

Alex Ball is talking about the problems in curating engineering and CAD data. In what appears to be a lose-lose strategy for all of us, engineering is an area with extremely long time requirements for preserving the data, but increasing problems in doing so given the multiple strangleholds that IPR has: on the data themselves, on the encodings and formats tied up in specific tightly controlled versions of high cost CAD software, coupled with “engineering as a service” approaches, which might encourage organisations to continue to tightly hold this IPR. An approach here is looking for light-weight formats (he didn’t say desiccated but I will) that data can be reduced to. They have a solution called LiMMA for this. Another approach is linking preservation planning approaches with Product Lifecycle Management. In this area they are developing a Registry/Repository of Representation Information for Engineering (RRoRIfE). Interesting comment that for marketing purposes the significant properties would include approximate geometry and no tolerances, but for manufacturing you would want exact geometry and detailed tolerances. -- Digital Curation Blog

Richard Wright talking about storage and the “cost of risk”. In early days dropping a storage device meant losing a few kilobytes, now it could be GBytes and years of work. Storage costs declining and capacity increasing exponentially roughly related to Moore’s law (doubling every 18 months). Usage is going up, too, and risk is proportionate to usage, so risk is going up too. Risk proportional to no of devices and to size and to use… plus the more commonly discussed format obsolescence, IT infrastructure obsolescence etc. So if storage gets really cheap, it gets really risky! -- Digital Curation Blog

Question from Steve Knight about how we move to a position where there is a market for digital preservation solution? -- Digital Curation Blog

Sarah Jones on developing a toolkit for a Data Audit Framework: to help institutions to determine what data are held, where they are located and how they are being managed. 4 pilot sites including Edinburgh and Imperial already under way, UCL and KCL in planning. Detailed workflow has been developed as a self-audit tool. Four phases in the audit process, the second being identifying and classifying the assets; looks like major work. Turns out the pilots are related to department level rather than institutions, which makes sense knowing academic attitudes to “the Centre”! I did hear from one institution that it was difficult getting responses in some cases. Simple online tool provided. DAF to be launched tomorrow (1 October) at the British Academy, together with DRAMBORA toolkit. -- Digital Curation Blog

In discussion, we felt that the point of David’s remarks was that we should understand that perfection was not achievable in the digital world, as it never was in the analogue world. We have never managed to keep all we wanted to keep (or to have kept) for as long as we wanted to keep it, without any damage. -- Digital Curation Blog

One recurring them, picked up at the outset by Lynne Brindley and in Steve Knight’s closing remarks, was that ‘digital preservation’ is not the term to be using in discussions with our institutions and the world, echoing remarks on the DCC blog which Brian later picked up on here. Steve prefers the phrase ‘permanent access’. which is indeed outcome-focussed. However, we’ve also said in PoWR that preservation isn’t always forever, so I would prefer something a little more all-embracing - ‘long-lived access’ might fit. -- JISC PoWR

---

Technorati tags: Copyright, Digitization, Digital Preservation, iPRES2008

David Lowe, Preservation Librarian at the University of Connecticut Libraries, is circulating a survey regarding JPEG2000. In the last year, I've learned more about JPEG2000 and blogged a bit about it. I quickly learned that some people perceived it to be an inferior format, but that indeed the standard is quite good (a testament to those who helped develop it). JPEG2000, however, may not be the first format that many programs consider. Lowe's survey hopes to help us understand if programs are considering JPEG2000 and what they are using it for. Having this solid information will be very helpful. In fact, it may help to build more trust in the format.

David Lowe's survey announcement said:

I am writing to solicit your help with a survey of library-related digital project staff regarding the implementation of the JPEG 2000 standard for digital images (specifically still images and not motion). We estimate that this task will take approximately 15 minutes of your time. It is available now at [this link].

The survey will remain open until October 31, 2008. We will post survey results via a report uploaded to our institutional repository, digitalcommons.uconn.edu. Please note that in our report, personal information from the survey will not be revealed, and any comments used will remain unattributed.

---

Technorati tag: JPEG2000

For those of you interested in JPEG2000, you may want to read this article. Abstract:

Digital preservation requires a strategy for the storage of large quantities of data, which increases dramatically when dealing with high resolution images. Typically, decision-makers must choose whether to keep terabytes of images in their original TIFF format or compress them. This can be a very difficult decision: to lose visual information though compression could be a waste of the money expended in the creation of the digital assets; however, by choosing to compress, the costs of storage will be reduced. Wavelet compression of JPEG 2000 produces a high quality image: it is an acceptable alternative to TIFF and a good strategy for the storage of large image assets.

Moreover, JPEG 2000 may be considered a format that can guarantee an efficient robustness to bit errors and offers a valid quality with transmission or physical errors: this point of view is confirmed by the case study results that we report in this article, concerning image quality after occurrence of random errors by a comparison among different file formats. Easy tools and freeware software can be used to improve format robustness by duplicating file headers inside or outside the image file format, enhancing the role of JPEG 2000 as a new archival format for high quality images.

---

Technorati tag: JPEG2000

Gary McGath wrote a blog post on BiggTIFF early in July. For those who know nothing about BigTIFF, the post and the comments will be useful. And the post includes links to additional information.

What is BigIFF? It's a technical standard. Quoting McGath:

BigTIFF changes the TIFF format by using 64-bit offsets and otherwise keeping everything else as much the same as possible. This means it can describe really large images. It also means that software that handles TIFF can be modified to handle BigTIFF with relatively little programming effort.

---

Technorati tags: Standards, Digital Image

The slides for this presentation - JPEG2000 for Archives and Libraries - are online. Both Dr. Robert Buckley and Justin Dávila delivered information in this session. It's easy to flip through the slides, so I'll not excerpt anything here. Be sure to look at the definitions of JPEG2000.

---

Technorati tag: JPEG2000

Thanks to Peter Murray for posting this info to a discussion list.

The presentation slides are now online from a session hosted by the JPEG2000 in Archives and Libraries Interest Group of the LITA division of ALA. The session was Archiving in Practice with JPEG2000.

---

Technorati tag: JPEG2000

From the BCR web site:

BCR is inviting public comment on the draft version of BCR CDP Imaging Best Practices. It is intended to serve as a guide for practitioners in cultural heritage institutions that seek to create digital images from physical objects. Originally published by the Colorado Digitization Program in 2003 as the Western States Digital Imaging Best Practices, this updated version reflects the many changes that have occurred in the digitization field during the last five years.

We will accept comments on the document until close of business, Friday, June 13, 2008. Thank you for your contributions.

---

Technorati tag: Digitization

Found on a NYS Document Heritage Program discussion list.

---

• George Eastman House
• Rochester Institute of Technology
  
• Image Permanence Institute

I've been keeping my eyes open for JPEG2000 info and this crossed my desk this morning.

---

The Digital Preservation Coalition has examined JPEG 2000 in a report published today. The report concludes that JPEG 2000 represents a great stride forward for the archival community. The format now allows for greater compression rates and a recompression rate that is visually lossless.

The findings come as the Digital Preservation Coalition launch its latest Technology Watch Report written by Dr. Robert Buckley, a Research Fellow with Xerox, ‘JPEG 2000 – a practical digital preservation standard?’. The report looks in-depth at the new format and the challenges it has to cope with. JPEG 2000 is widely used to collect and distribute a variety of images from geospatial, medical imaging, digital cinema, and image repositories to networked images. Interest in JPEG 2000 is now growing in the archival and library sectors, as institutions look for more efficient formats to store the results of major digitisation programmes.

The report is aimed at organisations involved in the management and storage of digital information. The in-depth report will help archives, libraries and other institutions make informed decisions about JPEG 2000 format and their future storage needs.

JPEG 2000 can reduce storage requirements by an order of magnitude compared to an uncompressed TIFF file. Dr. Buckley says, “This new format has come at a time of heightened awareness about the access to digital documents. Any format that can assist archives and libraries to do this is welcome.”

The format will also enable users to open as much of the file as they need at that time. This means a viewer, for example, could open a gigapixel image almost instantly. This is achieved by retrieving a decompressed low–resolution display sized image from the JPEG 2000 codestream. Coupled with this, the users’ ability to zoom, pan and rotate an image have been enhanced.

Adrian Brown, head of digital preservation, The National Archives said: “This is a very timely addition to the DPC's Technology Watch Report series as many organisations are themselves reviewing the JPEG2000 format. This concise, comprehensive and clear guide will be of interest to practitioners across the digital preservation community.”

The report concludes that JPEG 2000 offers much more flexibility and features than JPEG, but at the cost of greater complexity. It is however a great stride forward, and of major significance for the information management community.

To download a pdf of the report please go to: www.dpconline.org/graphics/reports/index.html#twr0801

For further information please contact, Tim Matthews, tim.matthews@nationalarchives.gov.uk, or 020 8392 5277.

For further information on the DPC please contact, Frances Boyle, fb@dpconline.org or 01904 435320.

About The Digital Preservation Coalition (DPC)

The Digital Preservation Coalition (DPC) is a cross sectoral member organisation established in 2001 to foster joint action to address the urgent challenges of securing the preservation of digital resources in the UK and to work with others internationally.

---

Technorati tag: JPEG2000