Skip to Main Content

Blog Category: Organization

April 1, 2004

Artstor Testing: Preliminary Findings

During the 2003-2004 academic year, over thirty colleges, universities and museums have been participating in Artstor’s “test” phase. The goal of Artstor testing was to assess our progress in building the Artstor image collections and software tools. Our institutional test partners – fourteen in the fall semester of 2003, and over thirty-five in the spring of 2004 – were invited to participate on the basis of a range of considerations, but mostly because of their ability and willingness to bring diverse perspectives to bear on key aspects of Artstor’s development. Users at these test institutions have had continuous access to Artstor for use in teaching and scholarship. Institutional staff — including librarians, visual resources professionals, and instructional and information technology staff — have been working closely with Artstor to assess the testing experience. We are grateful to all these individuals for their willingness to help understand their varied interests and needs, and to provide useful guidance for improving all aspects of Artstor. A full list of our test partners is at the bottom of this page.

What follows is a brief interim report of the results of this testing. When the test phase is completed, we will update these findings and post them on the website.

Technology
Technology is an important aspect of any digital library delivery system, and the test findings led us to improve the functionality, usability, and performance of ARTstor’s technology and software. We experienced problems launching the software in some Macintosh environments. As of April, the system works on the OS10 operating system, and we anticipate that the difficulties we experienced with the earlier OS9 system will be resolved by June. We have also developed several approaches to “interoperating” with institutional digital collections and software (e.g., image management systems, learning management courseware, and other digital resources). To facilitate access for our participating institutions, we are now pursuing several approaches to authentication and authorization of end users. Most importantly, testing confirmed our belief that it is essential that ARTstor develop software that is tailored to the considerable variety of needs of its different users. Because we have chosen to build our own software rather than work only with existing software products, we believe that we have been better able to improve and adjust that software to reflect user comments and criticisms in a timely manner.

Feedback during testing also suggested that we build additional user-friendly tools to expand the possible uses of ARTstor’s content. In one specific example, our testers assisted us in designing an off-line viewer that allows the use of ARTstor content in the classroom, with no reliance on Internet connectivity. This off-line viewer was created in response to tester concerns about relying on a network connection to present images in class and by the widespread need to make presentations (in a classroom or elsewhere) without connecting to the Internet. We are very grateful to James Madison University for its generosity in allowing ARTstor to build from their code in their widely adopted MDID viewer to create the initial ARTstor Offline Viewer.

Finally, because ARTstor is hosted, launched and distributed in a way that enables users to work in an active software environment (i.e., dragging and dropping images from one saved group to another), testing allowed us to see how we needed to enhance our technological infrastructure. We improved server capacity and the load-balancing system, and also reconfigured the application server to enhance speed and performance.

Usability Testing
This fall, the Digital Knowledge Center (DKC) at Johns Hopkins University worked closely with ARTstor on usability testing, including conducting one-on-one tests with users at many of the test institutions. Because of the forty-two ARTstor usability tests, and the detailed findings that the DKC reported as a result, we learned about specific barriers, which we were able to remove or ameliorate. The majority of this work will be completed prior to ARTstor’s launch this summer.

Users
During the testing phase, we learned a great deal about the various needs of potential ARTstor users – especially about the range of potential users. While we believe it is clear that ARTstor provides value to individuals studying Art History and related fields, it also hopes to meet the broader need for a campus-wide image resource that is accessible to scholars, teachers and students across the humanities and beyond. We are also encouraged by the fact that our test phase indicated that ARTstor’s software provides the right tools for most users of digital images and is sufficiently user-friendly to enable individuals who are relatively new to digital technologies to feel comfortable using ARTstor.

Content
Feedback also indicated that the ARTstor Charter Collection is broad enough to engage a reasonably wide audience, and to support a good range of teaching needs, as well as deep enough in some areas to support aspects of scholarly research. ARTstor continues to seek new collections to provide greater breadth as well as the depth that scholars will require. One of ARTstor’s greatest challenges in building collections is balancing the need to add a great many more images with the desire to maintain appropriately high standards of image and data quality. Various users at different test institutions championed both “sides” of this equation – some calling for more images even at lower quality and others preferring that we include only the highest quality images in ARTstor. We do not believe there is one “right” path in this area; meanwhile, we are encouraged by the fact that so many users have urged us to strike a sensible balance, and to retain our core commitment to enhancing the quality of images and data over time.

ARTstor Licensing Agreement
Licensing digital content — especially images of art — is extremely complicated. Balancing intellectual property requirements with the educational and research needs of the non-profit educational community is a challenge. Based on concerns expressed by some of the fourteen initial test institutions, we made some significant revisions to our user agreements. We held a productive half-day meeting in December 2003 with user agreement representatives from our Fall test institutions in an effort to solve common issues. The final outcome, we believe, is an agreement that balances the needs and interests of ARTstor users, of their respective institutions, and of the many institutions that help ARTstor to build its collections. The vast majority of our new test institutions signed the revised agreement with scarcely any comments or questions. Given ARTstor’s strong commitment to work with all non-profit educational institutions in order to foster the use of digital media in teaching and learning, we regard the general acceptability of this new user agreement as a very important outcome of ARTstor testing.

What ARTstor Offers
We have deliberately designed ARTstor in a way that differs in many ways from other online information resources. ARTstor is relatively unusual in providing three separate but integrated components. First, it consists of a very large and constantly growing online database of searchable images and accompanying conformation. Second, it has embedded the database in a software “tool-kit” that enables the user to search and make active use of the collections. Finally, ARTstor also provides a restricted environment through its own network that is accessible only to members of non-profit institutions. By providing a restricted network (not accessible publicly through the open internet) and by seeking to balance the interests of content providers and users, we believe that ARTstor can play an important role in working through these complex intellectual property issues in a way that individual institutions – on their own- might find extremely difficult to accomplish.

These three inter-related components and services make ARTstor very different from a straightforward online digital database purchased by a library. During the testing period, we have begun to learn about ARTstor’s “spillover” effects, including how the availability of such a resource encourages certain forms of inter-disciplinary work and begins to help with a very powerful integration of the critical related process of teaching, learning, and research.

While we have only begun to observe and assess the impact that ARTstor will have for users, we also should note how the testing period has been instructive with regard to the collaborations that have been explored and have begun to flourish across organizational boundaries; librarians, visual resources professionals, and instructional and information technology staff have come together to support the use of ARTstor. While ARTstor “lands” in the library, we will be interested to see if – or how – the broad applicability of ARTstor will be a catalyst for new organizational collaborations.

Conclusion
The preliminary findings from this test phase have taught us many important lessons – about almost every aspect of ARTstor. Most of the improvements we have made would not have been possible without the commitment of our testing partners, to whom we are very grateful. We believe that the contributions of all these partners – as well as the professional relationships we have been created or built upon through this process – will make ARTstor a much stronger and more useful resource for all its users.

ARTstor’s Test Partners

  • Art Institute of Chicago
  • Bowling Green State University
  • Bryn Mawr College
  • Carleton College
  • Columbia University
  • Connecticut College
  • Dallas Museum of Art
  • Emory University
  • Getty Research Institute
  • Harvard University
  • Hunter College
  • James Madison University
  • Johns Hopkins University
  • Los Angeles County Museum of Art
  • Metropolitan Museum of Art
  • Minneapolis College of Art and Design
  • National Gallery of Art
  • New York University
  • Northwestern University
  • Ohio State University – Main Campus
  • Pennsylvania State University
  • Perpich Center for Arts Education
  • Princeton University
  • Roger Williams University
  • Sarah Lawrence College
  • Smith College
  • Smithsonian American Art Museum
  • Trinity University
  • University of California – San Diego
  • University of Miami
  • University of Michigan – Ann Arbor
  • University of Oregon
  • University of Notre Dame
  • University of Virginia
  • Walters Art Museum
  • Wesleyan University
  • William Paterson University of New Jersey
  • Williams College, Sterling & Francine Clark Art Institute

Continue Reading »

Posted in
December 1, 2003

Letter from The Chairman & Executive Director

Welcome to ARTstor, an initiative of The Andrew W. Mellon Foundation.
ARTstor’s purpose is to create a large – and indefinitely growing – database of digital images and accompanying scholarly information for use in art history and other humanistic fields of learning, including the related social sciences.

ARTstor will be a not-for-profit organization, and its materials will only be made available for use by not-for-profit educational institutions, such as colleges and universities, museums, libraries, research institutes and similar organizations. The goal is to enhance teaching, scholarship and learning in fields of knowledge that use images and associated scholarly materials for study and research, as well as in lectures, classrooms, conferences and similar settings.

ARTstor’s objective in creating its database is to carefully select “collections” that are intrinsically significant, and that have sufficient breadth, depth and coherence to make them genuinely useful to faculty, curators, students and others.

Over time, ARTstor hopes to build – in collaboration with other institutions – a database that will consist of millions of images and related data. It will include collections from a wide variety of civilizations, time-periods, and media, as well as from different sources, such as museums, archaeological sites, photo-archives, slide collections, and published materials that promise to be unusually helpful as scholarly tools. Users will be able to search across an individual “collection” in the database or across multiple collections, as a single large “library” of materials.

Participation in ARTstor will be through institutional site licenses. Fees will be set according to a sliding-scale based on a number of institutional characteristics. The object is to make participation as broad as possible across a great range of educational institutions, while generating some revenue to offset a share of ARTstor’s considerable operating costs. The ARTstor database will be able to be accessed directly by any individual who is an authenticated member of a participating institution.

ARTstor began as an organization in the early fall of 2001. During the past eighteen months, it has been creating its initial digital collections, addressing technology issues, consulting with members of the museum and academic communities, and preparing for the time – during the academic year 2003-2004 – when materials could be made available for use at educational institutions.

While we hope that these initial collections will be useful from the very start, we also want to underscore a number of important points:

First, even the initial collections will not be complete at the time of release, simply because the process of creating a coherent group of images and data is highly labor-intensive and time-consuming. The entire process – from choosing a project; reaching institutional collaborative agreements; undertaking photography (or digitizing already-existing images); updating catalog information; and guiding the entire production process carefully to ensure quality-control – is complex as well as costly, and it simply cannot be rushed. Consequently, the content and size of the initial database will inevitably be illustrative of what can be achieved over time as new material is added. We hope that, within the next eighteen months, we will have something in the range of 400,000 images and data online. But even that – measured against the infinite universe of art-objects – is obviously only the barest of beginnings.

Second, while ARTstor can do a considerable amount in creating an inter-institutional network, as well as building online collections, it is clear that no single organization can possibly do more than a small fraction of what needs to be accomplished if the national and international community of educational institutions is to be well-served. The hope, therefore, is that the ARTstor database and network can soon begin to function as a public utility that would eventually become a very broad-based co-operative enterprise, with participating institutions contributing digital materials while simultaneously benefiting from the growing database. ARTstor will exercise responsibility for maintaining – and adding significantly to – this database, just as it will maintain the complex systems (and staff) essential to this initiative. But we fully expect that there will come a time when the not-for-profit educational community of museums, colleges, universities and others will essentially “own” and operate the system.

Third, while ARTstor considers its primary purpose to be the creation and provision of digital images and related materials for scholarly and instructional use, it also hopes to do more than “deliver a product.” In fact, because so little is known about the most effective ways to build and use digital collections of this kind, we will need advice, criticism, suggestions – and even some patience! – from participating institutions, so that we can all learn together about users needs, software adaptations, image quality standards, metadata standards, and collection-building. As with any new technology, we expect that any number of mistakes will inevitably be made along the way, and that only a community-wide effort – sharing expertise, experience, and new ideas – can lead to genuinely useful and enriching results.

In the meanwhile, please read and ponder, and check back as we update the site over the coming months.

With best regards –

Neil Rudenstine and James Shulman

Continue Reading »

Posted in
March 19, 2003

Artstor’s Fall 2003 Testing

Artstor (in collaboration with 14 colleges, universities, and museums) will be conducting a test during the Fall of 2003. During the test period, users at these institutions will have access to the Artstor content and tools for their educational and scholarly work; institutional staff will work closely with Artstor staff on assessment of how well Artstor serves various user needs. During the testing period, Artstor will also be engaged in expanding its operational capacity so that it will be prepared to work with a larger number of institutions in 2004.

Key areas Artstor will explore with its test partners include:

Technology Issues

  • Evaluating the functionality and usability of the Artstor interface(s)
  • Exploring the various ways in which Artstor can help institutions to support the creation and management of image groups for use on course websites or electronic course reserves
  • Examining solutions for “interoperating” with institutional image management systems, learning management courseware, and other digital resources
  • Identifying different institutional contexts for authorizing and authenticating users
  • Understanding and managing network performance issues

Users and Uses

  • Understanding how Artstor will be used by different segments of the community
  • Exploring how Artstor can partner with institutions to promote broad use of this new resource, in pedagogy as well as in individual research
  • Identifying potential barriers to adoption of the Artstor service
  • Evaluating the training and support needs of different types of users, and understanding how institutions are likely to address these needs
  • Assessing image and metadata quality standards for various users and uses
  • Assessing and improving user awareness and understanding of the terms and conditions of use

Collections
It should be noted that any feedback gathered in this area will influence longer term planning, since content development is a complex, ongoing process.

  • Gathering feedback on the content and presentation of the charter collections
  • Identifying promising areas for collection development that will address the needs of a broad range of users
  • Understanding how Artstor collections complement local image collections, both analog and digital

This test represents only one aspect of Artstor’s dialogue with the community. In order to become a truly community-wide resource, Artstor staff will continue to engage in conversations with a range of individuals and institutions in an ongoing basis to learn more about the issues outlined above, as well as other issues surrounding the building, disseminating, and usage of image collections. Indeed, through this test Artstor hopes to learn how best to gather this sort of feedback routinely, in order to assure that such assessment and learning develops as a core element of the Artstor service.

Test Participants

The following institutions will be participating in the fall 2003 test. These institutions were selected based on their ability to provide a diverse range of perspectives on the key issues identified above.

  • Art Institute of Chicago
  • Harvard University
  • Hunter College (City University of New York)
  • James Madison University
  • Johns Hopkins University
  • Metropolitan Museum of Art
  • National Gallery of Art
  • New York University
  • Pennsylvania State University
  • Princeton University
  • Sarah Lawrence College
  • Smith College
  • University of California, San Diego
  • Williams College/Sterling and Francine Clark Art Institute*

*The staff and faculty at Williams College and The Clark Art Institute deserve special thanks for their participation in an early pilot Artstor project in the Fall of 2002.

Continue Reading »

Posted in