Nothing relevant to SEO

New issue of IJDC

dccbi — Wed, 21 Oct 2009 16:59:00 +0000

The latest issue (volume 4, issue 2) of the International Journal of Digital Curation is now available. It’s a bumper issue, with two letters to the editor (a whiff of controversy there!), 8 peer-reviewed papers (originating from last year’s International Digital Curation Conference), and 6 general articles (two of which came from last year’s iPres08 conference). I’m really pleased with this issue, which as always is extremely interesting.

This is the last issue to be produced by Richard Waller as Managing Editor, and I’d like to pay tribute to his dedication in making IJDC what it is today. He has sourced most of the general articles himself, and those who have worked with him as authors will know the courteous detail with which he has edited their work. They may not know the sheer blood, sweat and tears that have been involved, nor the extraordinarily long hours that Richard has put in to make IJDC what it is, alongside his “day job” of editing Ariadne. Thank you so much, Richard.

We will have a new Production Editor for the next issue, whom I will introduce when that comes out (we hope at about the same time as this year’s International Digital Curation Conference in London… have you registered yet?). We have some interesting plans to develop IJDC in volume 5, next year.

Update: I thought I should have said a bit more about the contents, so the following is abridged from the Editorial.

Two papers are linked by their association with data on the environment. Baker and Yarmey develop their viewpoint with environmental data as background, but their emphasis is more on arrangements for data stewardship. Jacobs and Worley report on experiences in NCAR in managing its “small” Research Data Archive (only around 250 TB!).

Halbert also looks at elements of sustainability, in distributed approaches that are cooperatively maintained by small cultural memory organizations. Naumann, Keitel and Lang report on work developing and establishing a well-thought out preservation repository dedicated to a state archive. Sefton, Barnes, Ward and Downing address metadata, plus embedded semantics; their viewpoint is that of document author. Gerber and Hunter similarly address metadata and semantics, this time from the viewpoint of compound document objects

Finally, we have two papers loosely linked through standards, though from different points on the spectrum of the general to the particular, as it were. At the particular end, Todd describes XAM, a standard API for storing fixed content; while from the more general end, Higgins provides an overview of continuing efforts to develop standards frameworks.

Moving on to general articles, in this case I would like to mention first my colleagues Pryor and Donnelly, who present a white (or possibly green?) paper on developing curation skills in the community.

Next, I would highlight two very interesting articles that originated from iPres 2008. These are Dappert and Farquahar who look at how explicitly modelling organisational goals can held define the preservation agenda. Woods and Brown describe how they have created a prototype virtual collection of 100 or so of the thousands of CD-ROMs published from many sources, including the US Government Printing Office. Shah presents the second part of his interesting independently-submitted work on preserving ephemeral digital videos. Finally, Knight reports from a Planets workshop on its preservation approach, while Guy, Ball and Day report from a UK web archiving workshop.

SUN PASIG: October 2009

dccbi — Mon, 19 Oct 2009 19:25:00 +0000

As readers of this blog may have guessed, I was in San Francisco for the iPres 2009 Conference (17 blog posts in 2 days is something of a personal record!). This conference was followed by several others, including the Sun Preservation & Archiving SIG (Sun-PASIG), from Wednesday to Friday. I didn’t feel quite so moved to blog the presentations as at iPres (and I was also knackered, not to put too fine a point on it). But I did not want to pass it b y completely unremarked, particularly as I really like the event. This is the second Sun-PASIG meeting I’ve attended, following one in Malta in June of this year (see two previous blog posts).

It’s a very different kind of meeting from iPres. The agenda is constructed by a small group, forcefully led by Art Pasquinelli of Sun and Michael Keller of Stanford. The presentations are just that; not papers. This let’s them be more playful and pragmatic, also more up-to-date. Of course, there’s a price to pay for a vendor-sponsored conference, although I won’t reveal here what it is!

Tom Cramer has put up the slides at Stanford, so you can explore things I was less interested in. In the first session, the presentation that really grabbed me was Mark Leggott from Prince Edward Island (I confess, guiltily, I don’t really know where this is) talking about Islandora. This is a munge of Fedora and Drupal, with a few added bits and bobs. It looked like a fantastic example of what a small, committed group with ideas and some technical capability can do. Nothing else on day 1 caught my imagination quite so strongly, although I enjoyed Neil Jeffries’ update on activities in Oxford Libraries, and Tom Cramer’s own newly pragmatic take on a revised version of the Stanford Digital Repository.

On day 2 there were lots of interesting presentations. Of particular interest perhaps was the description of the University of California Curation Center’s new micro-services approach to digital curation infrastructure. I’m not quite sure I get all of this, mainly perhaps as so much was introduced so quickly; however as I read more about each puzzling micro-service, it seems to make more sense. BTW I congratulate the ex-CDL Preservation Group on their new UC3 moniker! ‘Tis pity it came the same week as the New York Times moan about overloading the curation word (here if you are a registered NYT reader)…

I also very much liked the extraordinary presentation by Dave Tarrant of Southampton and Ben O-Steen of Oxford on their ideas for creating a collaborative Cloud. Just shows what can be done if you don’t believe you can’t! The slides are here but don’t give the flavour; you just had to be there.

In a presentation particularly marked by dry style and humour, Keith Webster of UQ talked about Fez, and shortly after Robin Stanton of ANU talked about ANDS; both very interesting. The day ended with a particularly provocative talk by Mike Lesk, once at NSF for the Digital Library Initiatives, now at Rutgers. Mike’s aim was to provoke us with increasingly outrageous remarks until we reacted; if he failed to get a pronounced reaction, it was more to do with the time of day and the earlier agenda. But this is a great talk, and mostly accessible from the slides.

On the 3rd day, we had a summing up from Cliff Lynch, interesting as ever, followed by breakouts. I went to the Data Curation group (surprise!), to find a half dozen folk, apparently mostly from IT providers, very concerned about dealing with data at extreme scale. It’s a big problem (sorry), but not quite what I’d have put on the agenda. But in a way it typifies Sun-PASIG: never quite what you thought, always challenging and interesting.

Shortly thereafter I had to leave, but in the middle of a fascinating discussion about the future of Sun-PASIG, particularly with the shadow of the Oracle acquisition looming. I certainly believe that the group would be useful to the new organisation, and very much hope that it survives. Next year in Europe?

iPres 2009: van Horik on MIXED framework for curation of file formats

dccbi — Tue, 06 Oct 2009 23:35:00 +0000

Scholars in the Netherlands can deposit or search information in a repository system called DANS EASY, containing about 500,000 files, with a wide diversity of formats. How do I deal with a file called cars.DBF, now an obsolete format. There system can read such formats and convert them to the XML-based MIXED format, which identifies the data type and contains information on structure and content. So this was a smart conversion from the binary, obsolete dbase file to an XML reusable file. In the future it can be converted from this format to a current format of choice. This process (allegedly) does not require multiple migrations…

They have a SDFP community model for spreadsheet and tabular data. Have created some code for DBF and DataPerfect formats that they had to reverse engineer, in SourceForge; this a very labour-intensive activity, and really should be a community effort.

Question: does reverse engineering expose to risk? Don’t know…

iPres 2009: Brown on font problems

dccbi — Tue, 06 Oct 2009 23:16:00 +0000

They have a very large collection of documents, some of which had Texas Instrument calculator fonts, which had maths symbols in them, but didn’t always render properly with font substitutions. Several other examples, including barcode fonts (where font substitution can give the numeric value, not losing information but losing functionality).

The top 10 fonts in a collection tend to be the same; it’s the long tail of up to 3,000 or so that might be the problem. Font names help a bit but there are huge variations in font names, eg 50+ for Arial alone! In fact, it’s quite difficult to get useful matches from font names with fonts in font tables, some of which have very weak information content. Times new Roman satisfies about 38% of documents in their collection; Windows XP + Word satisfies about 80% of the documents in the collection; the large collection of fonts they assembled would satisfy about 95% of the collection, many more would be needed to build that up higher.

Worst example was a Cyrillic font, called Glasnost-light but rendered as ASCII; the problem was related to the pre-Unicode code space in some way I didn’t understand. A font substitution looked hopeful; it produced Cyrillic, but unfortunately not Russian, as the encoding was different.

Comment: this is a difficult problem much dealt with in the commercial community, who have secret tables. But even Adobe only deals with a couple of thousand fonts.

iPres 2009: Tarrant, the P2 Registry, Where the Semantic Web and Web 2.0 meet format risk management

dccbi — Tue, 06 Oct 2009 22:56:00 +0000

P2-registry is a demo of what we can do if we publish in a web 2 fashion. The mainstream here is the web, for the community

Linked data: every slide has links to where the stuff comes from. See the graph on linked data, let’s get in that graph. Using linked data reduces redundancy, facilitates re-use and maximises discovery. The community is not just consumers, also publishers. Because of links to namespaces, this contributes to building trust.

The main node is DBpedia, which is in fact Wikipedia marked up as RDF. Lots of people reference it and link to it. Give URIs to things: Tarrant has a URI; his home page is not him; has a URL that’s not the same (but relates).

4 rules of linked data: use URIs as the names of things; use HTTP URIs so they can be looked up; when someone looks them up, provide useful information, include links to other useful things.

Here, data are facts, facts are represented as triples, in RDF. OWL & RDFS provide means to represent your RDF model. It’s machine readable and validatable. Importing data from multiple domains, you can use OWL to say a thing in one domain is the same as another thing in different domain.. Used PRONOM and Wikipedia to build a small ontology that describes what can be done by different software. The underlying registry is a triple store, it understands RDF, so 19 possible answers are turned into 70 with some data alignment. Then used these data to perform a basic risk analysis on PDF.

Take home message: data hidden in registries is not easily discoverable so is little used, so publish it on the web and it can be much more widely used.

Trust seems an issue in so many name spaces, but hopefully it all works out….

iPres 2009: Kirschenbaum & Farr on digital materiality: access to the computers

dccbi — Tue, 06 Oct 2009 21:58:00 +0000

This seems to be about the digital equivalent of literary personal papers; an urgency based on the recent deaths of authors like John Updike & others. Based on planning grant funding from NEH, resulting in a deliverable as a White Paper.

Digital objects in this case are artefacts, not just records; both the physical and the virtual require materiality. Some of this is regarding the computers as important parts of the creative context.

Recommendations: keep the hardware and storage media. You can tell things from hand-writing on diskette labels, etc.

Recommendation: Image disks (both pictorial images, but also forensic imaging), see Jeremy Leighton John.

Recommendation: computer forensics (see forthcoming CLIR/Mellon report on Computer Forensics in Cultural Heritage, expected to be available next fall).

Recommendation: document the original environment, eg 360 degree views.

Recommendations: value from interviewing the donors themselves.

Recommendation: since they are balancing lots of needs, they need to put careful thought for interface development.

Recommendation: Scholarly Communication Needs, have to have new tools and methodologies on citation (eg of a tracked change in a Word document), reproduction, copyright and IP issues. White paper available at http://www.neh.gov/ODH …

There is a time window open now that may not stay open for long, for computers from the early 1908s!

iPres 2009: Guttenbrunner on Digital Archaeology, recovering digital objects from audio waveforms

dccbi — Tue, 06 Oct 2009 21:31:00 +0000

Early home computers often used audio cassettes as data media. Quite a bit of such data still exist in audio tapes in various archives, getting in worse and worse condition. Can they migrate the data without the original system in the future?

The system they used is the Philips Videopac+ G7400, basically a video game system released in 1983… and another one (!).

Data are encoded in bitstreams, which in turn are encoded in analogue waveforms (via a microphone/headphone socket pair and an audio cassette system!). They worked out how the waveforms responded to changes in the data (basically reverse-engineering the data encodings; would not have been so easy without a working computer). As a result, they were able to write a migration tool from the audio streams to non-obsolete formats.

It turned out there was already a solution that worked where there was a good signal from the tape, but these were often very old tapes in poor condition, so they implemented a different approach, which worked better.

Using old tapes, the other system recognised found no files. The actual system recovered 6 out of 23. Their new implementation recovered 22 out of 23 files, in some cases with errors. They checked by re-encoding the recovered files (on new tapes) and reloading to the actual system; most had minor errors that could be fixed if you knew what you were doing.

They think their findings are valid for all systems that use audio encodings, although there will be wide variations in encodings and file types, but it’s not extensible to other media types.

iPres 2009: Pennock on ArchivePress

dccbi — Tue, 06 Oct 2009 21:27:00 +0000

Blogs are a new medium but an old genre, witness Samuel Pepys’ diaries for instance (now also a blog!). But since they are web based, aren’t they already archived through web archiving? However, simple web archiving treats blogs simply as web pages; pages that change but in a sense stay the same. Web archiving also can’t easily respond to triggers, like RSS feeds relating to new postings. Web archiving approaches are fine, but don’t treat the blogs as first class objects.

New possibilities can help build new corpora for aggregating blogs to create a preserved set for institutional records and other purposes. ArchivePress is a JISC Rapid Innovation (JISCRI) project, which once completed will be released as open source. The project started with a small 10-question survey, for which the key question was: which parts of blogs should archiving capture. In descending order the answers were posts, comments, tag & category names, embedded objects, and the blog name & URLs. These findings were broadly in agreement with an earlier survey 9see paper for reference).

Set out to find the significant properties of blogs. Significant properties, they see as in the eye of the stakeholder. First round this includes content (posts, comments, embedded objects), context (including authors & profiles), structure, rendering and behaviour.

To achieve this, they build on the Feed plugin for WordPress, which gathers the content as long as a RSS or Atom feed is available. WordPress is arguably the most widely used, it’s open source, it’s GPL and it has publicly available schemas.

Maureen showed the AP1 demonstrator based on the DCC blogs [disclosure: I’m from the DCC!], including blog posts written today that had already been archived. The AP2 demonstrator (the UKOLN collection) will harvest comments, and resolving some rendering and configuration issues from AP1; and will allow administrators to add new categories (tags?).

It seems to work; there turned out to be more variations in feed content than expected. Configuration is tricky, so must make it easier.

iPres 2009: Collaboration

dccbi — Tue, 06 Oct 2009 19:28:00 +0000

iPres 2009: Martha Anderson on Enabling Collaboration for Digital Preservation

Collaboration is what you do when you can’t solve a problem by yourself. Digital Preservation is such a problem. That was Martha’s summary of her very interesting presentation recapping the NDIPP so far, and giving some excellent guidelines relating to modes of collaboration. She spoke also about an upcoming National Digital Stewardship collaboration, which if I understood it is based round organisations (government?) taking some shared responsibility for the future of their data.

iPres 2009: Panel on challenges on distributed digital preservation

All the speakers participate in Private LOCKSS Networks (PLNs), although there are others eg Chronopolis. Meta Archive Cooperative is growing slowly, recent new members include Hull in the UK, but has a list of up to 40 potential associates. Alabama Digital Preservation Network (ADPnet?) focuses particularly on being simple and cheap. Canadian library consortium (COPL?) has a PLN with 8 members out of 12 in the consortium.

Organisational challenges on starting up, eg creation of Meta Archive West as new startup versus extension of existing. Issues are the same: organisational, technology and sustainability, of which the first and last are the parts of the iceberg under the water! Some very interesting points made about many aspects of these networks.

iPres 2009: Micah Altman Keynote on Open Data

dccbi — Tue, 06 Oct 2009 17:22:00 +0000

Open Data is at the intersection of scientific practice, technology, and library/archival practice. Claims that data are at the nucleus of scientific collaboration, and data are needed for scientific replication. Science is not just scientific; it becomes science after community acceptance. Without the data, the community can’t work.

Open data also support new forms of science & education: data intensive science, which also promoted inter-disciplinarity. Open data also democratise science: crowd-sourcing, citizen science, developing country re-use, etc. Mentions Open Lab Notebook (Jean-Claude Bradley), Galaxy Zoo etc.

Open data can be scientific insurance; that little extra bit of explanation makes your own data more re-usable, and can give your project extended life after initial funding ends.

Data access is key to understanding social policy. Governments attempt to control data access “to evade accountability”.

Why do we need infrastructure? [Huh?] While many large data sets are in public archives, many datasets are hard to find. Even problems in professional data archives: links, identifiers, access control, etc. So, core requirements…

Stakeholder incentives
Dissemination inc metadata & documentation
Access control
Provenance: chain of control, verification of metadata & the bits
Persistence
Legal protection
Usability
Business model…

Institutional barriers: no-one (yet?) gets tenure for producing large datasets [CR: not sure that’s right, in some fields eg genomics etc data papers amongst highest cited]. Discipline versus institutional loyalties for deposit. Funding is always an issue, and potential legal issues raise their heads: copyright, database rights, privacy/confidentiality etc.

Social Science was amongst the first disciplines to establish shared data archives (eg ICPSR, UKDA etc), in the 1960s [CR: I believe as an access mechanism originally: to share decks of cards!]. Mostly traditional data, not far beyond quantitative data. More recently community data collections have been established, eg Genbank etc; success varies greatly from field to field. Institutional repositories mostly preserve outputs rather than data, and most only have comparatively small collections. They provide so far only bit-level preservation, mostly not designed to capture tacit knowledge, and have limited support for data. More recently still, virtual hosted archives are happening: institutionally supported but depositor-branded (?), eg Dataverse Network at Harvard; Data360, Swivel. Some of these have already gone out of business; what does that do to trust re persistence of service & data; can you self-insure through replication?

Cloud computing models are interesting, but mostly Beta, and often dead on arrival or soon after. What about storing data in social networks (which are often in/on the cloud). Mostly they don’t really support data (yet), but they do “leverage” that allegiance to a scientific community.

Altman illustrated a wide range of legal issues affecting data; not just intellectual property, but also open access, confidentiality, privacy, defamation, contract. Traditional ways of handling some of this was de-identification of data; unfortunately this is working less and less well, with several cases of re-identification published recently (eg Netflix problem, Narayan et al). [CR; refreshing to hear a discussion that is realistic about the impossibility of complete openness!]

So instead of de-identifying at the end, we’re going to have to build in confidentiality (of access) from the beginning! Current Open Access licences don’t cover all IP rights (as they vary so widely), don’t protect 3^rd party liability, and often mutually incompatible.

Altman ending on issues at intersections, starting with data citation: “a real mess”. At least should be some form of persistent identifier. UNF as a robust, coded data integrity check (approximation, normalisation, fingerprinting, representation). Technology can facilitate persistent identifier [CR: not a technology issue!], deep citation (subsets), versioning. Scientific practices evolve: replications standards, scientific publications standards.

There is a virtuous circle here: publish data, get data cited, encourages more data publication and citation!

Next BKN, which sounds like a Mendeley/Zotero/Delicious-like system, transforming treatment of bibliographies & structured lists of information.

The Dataverse network: an open source, federated web 2.0 data network, a gateway to >35,000 social science studies. Now being extended towards network data. Has endowed hosting.

DataPASS, a broad-based collaboration for preservation.

Syndicated Storage Project: replication ameliorates institutional risk to preservation. Virtual organisations need policy-based, auditable, asymmetric replication commitments. Formalise these commitments, and layer on top of LOCKSS. Just funded by IMLS to take the prototype, make it easier to configure, open source etc.

Prognostication: archiving workflow must extend backwards to research data collection [CR: Yeah!!!]. Data dissemination & preservation increasingly hybrid approach. Strengthening links from publication to data, makes science more accountable. Effective preservation & dissemination is a co-evolutionary process: technology, institution & practice all change in reaction to each other!

Question: what do you mean by extending backwards? Archiving often captured when the research is done; becomes another chore, lose opportunity to capture context. So if the archive can tap into the research grid, the workflow can be captured in the archive.

Question (CR): depositor/re-user asymmetry? It does exist; data citation can help this!