Collection Assessment
Purpose:
To determine the qualitative level of collection support for a specific academic program, to identify the collection’s maintenance and development needs, and to create a budget plan.
Situation:
An evaluation of a library collection emerges through an academic program context. The academic program may have historical traditions which date to the institution’s founding; it may be newly formed from several older programs; it may have a long tradition of preparing students at the baccalaureate level and only recently added a research program. In any case, academic programs evolve and the collection evaluation provides an opportunity to review and reconcile academic program objectives to collection development and management objectives.
Library collections are accumulations of published and unpublished materials. A "good" collection for students working on the undergraduate or master's level may be a "bad" collection for students or faculty working at advanced research levels. In all cases the collection has to be judged relative to the academic program. A "good" library collection means that a student or faculty member can locate and gain access to needed resources in a reasonable period of time.
The library collection manager is responsible for molding the collection. Collections will usually be developed first at the "instructional" level, which may be defined as an organized group of publications containing commonly accepted important titles published in a particular field. A research level collection will be more through; it will contain not only the instructional level titles but the "edges" in focused areas. Through a research collection, the faculty and students will be able to review the historical development of the field deeply as well as have access to a broad range of current materials. Building collections at the research level involves a concerted effort to sustain a wide array of current purchases, including materials in "grey" publication areas which are not available through normal trade channels, to locate retrospective materials in the out of print market, to build more comprehensive serials collections, including publications of relatively obscure associations or learned societies, to purchase foreign language materials, to develop access to a broad selection of electronic and remote sources of information, to create vigorous gift-exchange programs, special collections, and private/public sector development programs. Developmental efforts will take a large number of directions, all of which have a significant budgetary impact.
What a collection evaluation statement will contain:
An effective collection evaluation statement will contain the following parts: 1) a clear description of the current and anticipated directions in the academic program, both at the instructional and research level; 2) a description of the current collection resources, both in terms of ownership and access; 3) an evaluation of the collection, in terms of ownership, access, and condition relative to the current academic program; 4) recommendations to improve access to needed resources; and 5) a proposed estimate of the budget required to maintain the collection and implement recommended improvements.
Collection evaluation is a lengthy and labor intensive activity. Consequently, the collection manager generally begins a collection review prior to the faculty academic program review initiative. This approach requires a general knowledge on the part of the collection manager of the academic program under review and collaboration with selected faculty attached to the program prior to the completion of the self-study. The draft collection evaluation statements will need to be adapted to the specific academic program as the academic program self-study reviews become available. Collection evaluations cannot be considered finished until the collection manager and faculty reached agreement on collection objectives and relate the evaluation statement to the "List of Narrative Information," which are the data contained on Program Review forms 2, 4, 7, and 11 as required by the Program Review Process. The collection manager must be part of the Academic Program Review Team for the collection evaluations to be successfully concluded.
Relating the academic program to the current collection:
Library collections are primarily organized at the University of Florida in two classification systems: most materials received since 1977 are in Library of Congress classifications (LC) and most materials received prior to 1977 are classified in Dewey Decimal classifications (DDC). Continuations in non-science areas begun prior to 1977 have not generally been reclassified from Dewey to LC.
The collection manager in charge of the evaluation should associate the programs under review to LC and DDC classifications and produce a list of the classifications. The list should represent the "core" collection; i.e., the primary group of research resources supporting the academic program at all levels. For example: English/American Language and Literature: LC classes, PR, PS and sections of PN, Z; Dewey classes, 810-810.9 and 820.8-820.9.
In several circumstances library collections are not organized by either LC or DDC classifications; for example, the Federal Documents collection, the P. K. Yonge Library of Florida History; the Baldwin Library, and so forth. The collection manager should determine if unclassified or uniquely classified library collections should be considered part of the "core" collection and involve the various collection coordinators in the evaluation process.
Each core collection will have "edges" or boundaries which need to be defined. These collection overlaps will be called "indirectly related collections." Pertinent classifications should be defined for the indirectly related collections at a broad level. For instance, a review of the American/English Language and Literature collection will have an important indirect relationship to History classifications for the United Kingdom, Ireland, United States, and other British Commonwealth countries.
The collection manager should asked one or several faculty members involved in the program to review the list of classifications/subject headings for both the core and indirectly related collections as soon as the list is completed.
Since the collection statement contains inventory statistics and scope statements outlining the strengths of both the core and the indirectly related collections, the collection manager should develop a list of the collection coordinators who ought to be involved in the evaluation and request their appointment to the "team." A list should be developed requesting specific technical services support and sent to the ADTS, with a coopy to the ADCM.
Inventory and description of the current collection:
Collection Inventory
As libraries become more interdependent and shift development toward purchase of access to information/materials in addition to the materials themselves, the inventory process becomes more complex. While counting local bibliographic records remains a fundamental step, the inventory process must consider not only what is owned by the library but the resources to which the library buys access or secures access to through inter/intra institutional cooperative agreements. The inventory indicates only the size of the collection accessible but it supplies data which forms a basis of comparison with other classified research collections.
Prior to initiating a collection inventory, the bibliographer should consider the following points:
-
Users often initially approach specific discipline collections through general interdisciplinary serials, proceedings of societies, and so forth. These general collections which report research in broad areas may not be included in the indirectly related collections list but the pertinent series and serials held in these collections should be listed.
-
Collections should be inventoried by format.
-
The evaluation should ascertain the accessibility of specific resources at the University but not owned by the library, other research library collections with whom we have special borrowing privileges (CRL, RLG, etc.), or collections/tools for which access has been purchased; e.g., ICPSR, Dow-Jones, Lexis/Nexis, and so forth. The inclusion of this data should be based on the "reasonable" time of access; in other words, can the students and faculty here at Florida access this information in a reasonable time frame and, therefore, access comes close to or the equivalent of ownership.
Step 1: Count the core collection bibliographic units online in each of the designated classes and sub-classes by MARC formats to determine the number of titles held by the Library. Until this count can be completed by machine it will be necessary to count by call number searches.
Step 2: Request a shelf-list count by the appropriate classifications for unconverted materials. If your core collection inventory includes materials housed in the library but not classified, contact the appropriate curator for data.
Step 3: Inventory the total number of serial titles in LUIS by broad designated classes not by subclass. Machine counts for serial records may be requested by classifications. Summary holdings information should be supplied, so that the run of a particular serial title can be determined.
Step 4: Inventory indirectly related collections owned by the library. General counts by call number may be available. Make inquiries to see if counts are available before you do call number counts through LUIS.
Step 5: List the collections/access tools not owned by the library but available at the University. Do not include collections which are not generally available to the public. Request a description of specific collection strengths from appropriate curators.
Step 6: List accessible but remote collections. Quite often these materials will be available through electronic means or special ILL agreements through RLG, CRL, and so forth. Provide a scope note for these accessible collections.
Collection Evaluation
Following the inventory, which assembles the entire universe available to the faculty and students in the academic program, the collection manager analyzes the data and reaches a judgement as to how well the collection supports the academic program.
Step 1: The collection manager should consider the completeness and quality of the collection—all items owned by the libraries or accessible to the faculty and students in the program. Some of the criteria are:
- Do the holdings reflect areas of specializations currently important
or anticipated to become important in the discipline?
Do selection guidelines and current collecting intensities cover the core materials?
Do ILL requests include core materials?
Do ILL requests exceed copyright guidelines?
Do retrospective holdings need to provide an historical perspective for the field?
Are foreign languages materials collected to the necessary extent?
Are important changes in publication patterns or in publication formats (especially electronic access) adequately considered?
Are there gaps in the collection?
Is there a large want list for the collection?
Are there current collecting areas which should be de-emphasized? -
Is the reference collection adequate? Are important tools missing?
-
Is the serials collection adequate? Are important titles or parts of runs missing?
-
What are the collection’s strengths and weaknesses?
-
What significant additions or new types of access have been made to the collection during the past five years?
-
What additions or new types of access should have been made during the past five years but were not?
-
Overall, does the collection adequately support the instructional and research program? More specifically, a statement should be made as to whether the collection is poor, adequate, good, or excellent at each program level: baccalaureate, masters, Ph.D. This statement is a preliminary judgement and need not be shared.
Step 2: Comparing the collection to an external standard.
The collection manager’s preliminary judgement should be tested against an external standard. The external standard may take several forms: a published bibliography, an especially prepared bibliography, a sample of the holdings of another library known to support comparable research level academic program through RLIN or OCLC, various indexing and/or abstracting services, conspectus verification studies, and so forth. Whatever external standard is used, it should be up to date, selective, and one which lends itself to a statistical approach. The collection manager should define the external standard, or combination of standards, why it has been chosen, and how it is used.
Step 3: Collection Condition.
The physical condition of the collection should be reviewed and a statement should be jointly written by the collection manager and the Preservation Officer stating the current collection condition. The statement should address the various formats in the collection—paper, film, computer files—and deal with both physical and housing/storage conditions. Are the materials brittle, worn, damaged? Are substantial numbers of materials missing? In regard to those parts of the collection for which access is purchased, the vendor’s preservation policy should be determined.
Definition of what needs to be done to improve the collection:
Developing collection objectives should be a collaborative effort between the collection manager and the faculty. Since it is most likely that the program self study will not be completed at this time, the objectives of necessity will deal with general approaches rather than specific approaches. In the final review, specific approaches may be included in the statement but will have to wait for the publication of the "List of Narrative Information" in the Program Review, specifically the data provided on Program Review forms 2, 4, 7, and 11. Maintenance objectives and objectives can be organized around the following questions, and the collection manager should not hesitate to add others.
- What need to be done to make the collection an adequate support for the instructional and research program?
- What holdings need to be enhanced to better reflect areas of specializations currently important or anticipated to become important in the discipline?
- What changes need to be made in the current collecting intensity?
- Which ILL requests exceed CONTU restrictions?
- Which ILL requests represent core materials?
- Which retrospective holdings should be made accessible?
- What languages should be collected?
- What foreign publications ought to be represented?
- Which important changes in publication patterns or in publication formats (especially electronic access) need to be considered?
- Which gaps in the collection ought to be filled?
- Are there large want lists for the collection?
- Are there current collecting areas which should be de-emphasized during the next five years?
The collection manager should list the collection management objectives, leaving space to list the estimated budget required to achieve the objective and a five year timeline.
Budget review:
A budget review will provide information which reflects the costs associated with creating the current collection. The budget review should generally span the past five years and form the basis for projecting a five year budget plan. The following data should be included and most of it will be supplied to the bibliographer upon request by the ADCM
- Funds allocated specifically to support the collection (regular and special)
- Expenditures by code
- Estimate of funds allocated to support the indirectly related collections
- Average cost changes for monographs and serials over the five year period
- Changes in the ratio of expenditures for monographs and serials during the past five years
- Average number of volumes added each year during the past five years
- Cost of emerging information technologies
- EDP expenditures
- Effect of currency fluctuations
- Cost of equipment to access new information technology
- Projected cost of maintaining current collecting intensities over the next five years
- Savings anticipated by altering current collecting intensities
- Collection objectives and estimates of the cost of implementing these objectives
Services provided by the Library in support of research/study for discipline:
A narrative statement should be made concerning the following:
- FTE allocated for the subject specialist and other library staff directly involved in developing, managing, and providing reference services for the collection.
- Specialized reference services for electronic format and specialized BI.
- Whether adequate access to the materials is provided by the Catalog Department.
- Whether the library is adequately preserving the collection
- Whether there is adequate access to ILL services.
Conclusion:
A summary concluding statement should be written which concisely draws together the various data into an overall assessment regarding the collection's adequacy to support the academic program. The conclusion will function as an executive summary.
The preliminary draft statement is now complete and should be distributed to library collection managers for comment.
Description of the current and anticipated academic program:
Following the general collection evaluation and, probably, toward the end of the academic program review, the collection manager should obtain the draft narrative statements required under the "List of Narrative Information" in Program Review, specifically the data provided on Program Review forms 2, 4, 7, and 11, and tailor the collection evaluation statement to the information contained in these forms. If the forms are not available, the collection manager should work directly with the faculty to determine as closely as possible what information the forms most likely will contain.
This part of the evaluation will most likely take place over a short period of time. When the collection manager has adapted and edited the statement to reflect the "List of Narrative Program information," he or she should turn the statement over to the appropriate faculty as a draft. Following comment by the faculty, the draft should be put into final form, and the collection manager should complete form 14. The collection evaluation should be included in the Academic Program Review study, with a copy to the ADCM