MAC 2008 Cleaning Cobwebs: Studies in Archiving Web-Based Records

Presenters:
Rosemary Pleva Flynn, Energy and Environmental Research Center, University of North Dakota
Philip C. Bantin, Indiana University Bloomington
Mark J. Myers, Kentucky Department for Libraries and Archives

Internet Archiving Tools include:

Archive-It
Web-at-Risk: Collaboration between LOC, California Digital Library, New York University and University of North Carolina
Echo Depository Project (WAW): Collaboration between UIUC and LOC
Web Curator Toolset (WCT): Collaboration between Library of New Zealand and the British Library

Some tools let you select specific content, others select domains or directories. You can set quotas and block hosts or domains. The best strategy is to identify series that have long term value and harvest those. Bulk harvest is too large and costly, item by item too slow.

The way your institution uses your website will determine how often and what to capture. Static pages can be captured less frequently. Most crawlers can be set to capture on a specific schedule.

Generally good to capture one link off the domain name, but this can get costly in terms of size and sites can be blocked if too large.

Items that aren't captured include:

• Java Script
• Streaming Video
• Dynamic Database
• Protected sites
• Form driven sites

The tool must document authenticity by capturing header information and other metadata. Most use a Dublin Core basis which is not designed for records, but more for photographs. Archive-It does not capture metadate as well as WAW and WAS.

The tool must be able to restrict access to copyrighted information. Again Archive-It doesn't do this well, WCT and WAS have better authorization tools.

The search engine must be able to search metadata and full text to be effective.

For preservation purposes the final format must be non-proprietary. The Web Archive file format (WARC) has been proposed as a standard to combine multiple digital resources into an aggregate archival file with metadata. Wayback Machine uses this type of file format.

Whatever service used must be able to migrate information preserved.

Institutions have the option to build a service using existing tool sets or to join a service. Build your own is harder and takes more resources in terms of expertise and server space, but is more flexible to fit your needs.

Mark Myers of KDLA uses pdf as primary file format. Other formats are reformatted to pdf. Pages are captured based on the state retention schedule.

Harvesters available are:

• Wayback Machine
• Grab-a-Site

They did manual harvesting and consultation with other agency offices. Most harvesters have problems with things like drop down menus. Just guessing, but a Flash menu would probably also cause major problems with harvesting layers.

Other problems arising include boundaries of the harvested site, functionality loss because of loss of structure and content management systems.

Rosemary and Mark are co-chairs of the ARMA Task Force on Website Management which will be developing guidelines and best practices for identifying and archiving websites.

The question remains "Are websites records?" and "to what extent?" In my experience there are a lot of records on websites that are permanent, but most of these are also printed. This will continue to change as we see more and more documents born digital and never in print. And even if they are in print the electronic version is much more versatile due to searchability.

MAC 2008 More Than One way to Meet the Challenge

Systematic Approaches to the Capture and Preservation of Complex Digital Artifacts

Steve Burbeck, Rockefeller Archive Center, Collaborative Electronic Records Project (CERP)
Glen McAninch for Kelly Eubank, NC State Archives, Electronic Mail Capture and Preservation Tool (EMCAP)
Mike Smorul, University of Maryland, Archival Prototypes and Lessons Learned

All three panelists were involved in developing tools for archiving email and websites. Often dynamic and a challenge to capture and migrate because of attachments, forwards and replies. Web 2.0 will pose even more challenges.

OAIS Open Archival Information System Reference Model:

• Submission to archives
• Packaged for archiving
• Packaged for dissemination

A system designed to delete electronic documents when retention is met won't work for historic preservation because of the chance for accidental deletion.

The system must also be able to handle a variety of formats partly because there are no email standards and many systems. Also attachments. Security of email in regard to viruses and the volume of email also a challenge.

It was interesting that all three projects came up with applications that worked differently at the archives end, but all started with an XML schema to prepare email for storage.

EMCAP's end result was an hmailserver. Users drag email identified as permanent records to an archive folder on their list. These emails can be accessed, but cannot be deleted or modified after archiving.

MAC 2008 The Useful Ten Words of the Ten Thousand: Describing Photographs in Words

Cynthia R. Miller, Curator of Photography and Prints, The Henry Ford

This session was very basic information. The most interesting thing was the contrast between the MPLP talk the day before and the very time consuming process of researching and identifying individual photographs today. Somewhere there has to be a balance between describing collections enough to provide good access and agonizing over descriptions for every image in out collections.

Main Topic should include details: Who, What, Where, When and Why?

Subject terms will depend on the needs of researchers and the mission and resources of your repository.

She recommended using a controlled vocabulary for consistent search terms. The Library of Congress has subject headings for prints and photographs. The Getty also has an online thesaurus which is better for objects.

The most important thing to keep in mind is what terms the researcher will be searching for. This is the reason that I'm not sure controlled vocabularies are the best method for cataloging. The search term one person might use could be completely different than the search term another researcher might use.

Earthquake

It's really strange to wake up in a strange bed to find it shaking and hear the door rattling. The first time I thought I was dreaming, but the second time Debbie and Nancy felt it too. Come to find out it was an earthquake.

Window Seat Finished

Can you tell I'm sitting in a hotel with nothing to do? I just looked at Flickr and saw my latest photo uploads, so I decided to blog them.

When I started this blog I meant to chronicle the building of my house, but it seems so far away that it just doesn't seem relevant. But I have lots of little projects that I want to do and I can blog them as I finish them.

The house has two dormer windows in the front. One is just open to the living room, but the other is open to the loft. The area is so small it really isn't useful, so I decided to build a window seat there that will double as a storage chest.

Before the project began this is what I had. This is actually the window open to the living room, but I didn't take before photos. I haven't decided yet what to do with this window, but it does need something.

This is the completed window seat. The cushion hides a handle and a zipper hinge which provides access to the storage space underneath.

When I went to buy foam to make a cushion, I found out that the only width I could get was 24 inches. That wasn't enough to cover the handle and hinges, so I decided to build the cubbyholes behind the cushion. They hold books, quilt blocks in process, notepads and pencils.

The face and cubbyholes are cedar to match the handrails and newel posts on the staircase. The only change I need to make is to take off the baseboard and replace it with a step. You have to jump to get on the seat.

My youngest daughter uses this space to curl up and read. I usually use it to make to-do lists or to draw out my next project and make materials lists.

The next project will be to add book cases on either side of the window seat. Maybe then I can get all my books out of boxes.

MAC 2008 Plenary Session-The Future Belongs to Archives

Jennifer Schaffner of OCLC

MPLP - More Product Less Process.

Talked about scanning for access versus scanning for preservation. Quantity not quality to improve access. Funders are more likely to support proposals for large scale access.

To add lots of records easily:

• Look at converting preservation microfilm because it is cheap to digitize.
• Use a digital camera to give a taste of collections when processed.
• Digitize things frequently requested.

This session generated a lot of discussion afterwards for several reasons. First a photograph online with no metadata is useless. In most cases it will never even be found. And collections with limited descriptions are also hard to use and find relevant material. How useful is a collection with several boxes of correspondence with nothing but dates on each folder? We have collections like this and they get no use. But collections with more in-depth descriptions do get used.

Also many institutions depend on use fees and copy fees as a revenue stream. If everything is digitized and online a major revenue source dries up.

MAC 2008 Electronic Document Discovery

These are my notes from the above session at the Midwest Archives Conference in Louisville, KY this week. Most of it is a repetition of the same things I've heard before, but still important.

Robert Webb - attorney perspective
Paul Engel - vendor perspective
Jim Cundy - records manager perspective

Webb
Electronic records are most important and hardest to manage, especially email because of the number of copies on different networks and servers.

The institution holding records does have the right to push back against requests that would cause undue burden or cost.

Every records manager should sit down with IT and legal departments to see if the mechanisms are in place to freeze routine destruction of electronic records in the event of litigation. The freezing of destruction can be narrowed down by scope and time period.

ESI is Electronically Stored Information

It is helpful to work with IT and prepare a data map tracking the flow of data in, out and through the system.

In the event of a request for records you should keep a record of the efforts made to locate requested records.

Cundy
ESI is addressed early in the legal process and included any record in any location. ESI includes: email and attachments, documents, audio (voice mail), video (teleconferences, webcam), photos from any source, databases, web pages, chat, IM, text messaging, and any other new format that may come up.

It isn't enough to have a retention schedule. Implement it and check up on offices to make sure they are following procedures.

File management is more important for electronic records than for paper, because it is so easy to make duplicates.

Communicate with IT and legal and educate management about value of records management. Work with people outside records management to gain an understanding of other aspects of electronic records.

Engel
Media does not determine importance of record.

The average ediscovery event costs $1.5M and usually targets email.

Be aware of compression issues, compressed files greatly increase cost of discovery.

petrification - changing the electronic document to an unalterable image

Key issues are:
spoliation - changing metadata by improper handling
significant resources
Must know outcome of litigation support software before use to protect metadata