A Sharable Distributed Library System for Multisite Software Development

environment || control-files || library-commands || structure || control-files || templates || remote-access || heterogeneous-environments

Software libraries for medical research require additional features not found in typical implementations. Development may occur in geographically distributed locations which increasingly employ security firewalls. Personnel resources are limited so reports and notices must be automatically generated. Development proceeds rapidly, so the library must be unobtrusive to be accepted.

We have implemented a library system based on RCS primitives with a hierarchical design. Proxy servers, operating via e-mail protocols, allow remote copies of libraries to be employed, while enforcing sequential updates through checkout locks. The overarching design goal of our library system has been a public library, with a searchable catalogs, overdue notices and inter-library transfers.

Environmental Variables

Environmental variables control which libraries you which to "subscribe" to and the order in which you want to search libraries. This creates an hierarchical library system and allows more specific, personal libraries to be searched before more general, system-wide libraries. Libraries are specified by a colon delimited list of absolute paths to the root of the library directory tree.

name use default
LIB_PATH which libraries to search ~/lib
LIB_HOME central library directory /usr/local/library

Control Files

Sytem-wide Library Control Files

System-wide library control files are located in the central library directory (${LIB_HOME}). They serve to document all libraries which have been created (for system-wide searches), list library personnel entrusted to perform critical operations and establish architecture specific parameters for compiling and loading binary executable programs.

name use
${LIB_HOME}/LIBRARIES master library list
${LIB_HOME}/LIBRARIANS authorized librarians
${LIB_HOME}/Template.* architecture specific templates

Library Specific Control Files (in ../mylib/etc)

Library specific control files contain the library "card catalog", recent changes to the library, who has currently checked-out modules and who is interested in library reports.

name use
Catalog list of all modules in library
Makefile.template file to create Makefile
Template.* architecture specific templates
New list of new modules added
Changes list of recent changes
CheckedOut list of modules currently checked out
Notify list of who to send reports to
.last_report time of last libreport run
Reports/ directory with hardcopy of periodic library report

Commands

Library commands may be broken down into five groups: creation, module access, module information, library maintenance and utilities. Creation commands create either libraries or modules. Module access commands control module placement within libraries and control module locks. They allow the user to: check in and out a module, remove a module lock and transfer a module to a new library. Module information commands print documentation, print a plain-text version, help locate a relevant module and search for patterns within modules. Librarian commands summarize library status, report recent library changes, remove a module and maintain Makefile's for compiled executables. Utility commands are used by other library commands. They provide the functionality to: report the current hardware/software environment, locate a module and aid in remote operations.

type command use
creation libcreate create a new library
  libtmpl create a module template
access libci check in a piece of software
  libco check out and lock a piece of software
  libunlock unlock a module
  libtransfer inter-library transfer
information libdoc print module documentation
  libcat look at a module
  libsearch keyword search of library catalogs
  libgrep pattern search w/in modules
librarian libstat library status report
  libreport report on recent changes to a library
  libdelete remove a module from a library
  libmkmf make library Makefile for current architecture
  libmake make binary executables for current architecture
utility libarch return current hardware/software environment
  libinfo report library information for module
  libfind locate a module in the library system
  libproxy act as proxy for remote check in/out

Library Directory Structure (setup by libcreate)

name use
../mylib/LIB source files with revision history
../mylib/pro IDL/Wave procedures (plain text copies)
../mylib/src source files (compiler and script, plain text copies)
../mylib/h include files (plain text copies)
../mylib/lib object libraries
../mylib/bin executable programs (binaries in sub-directories)
../mylib/doc document files
../mylib/etc miscellaneous files

Documentation Templates (created by libtmpl)

The documentation templates are patterned after UNIX man pages. They contain the following major sections:
NAME
SYNOPSIS
PARAMETERS
DESCRIPTION
HISTORY

Remote Library Access

Library proxy software at each site will manage remote check-in/check-out of software modules. Although each site has working copies of libraries, the proxy design ensures that software updates are first made to locked modules ("originals") in the master libraries and then propagated to the working copies. This approach prevents conflicting modifications.

Heterogeneous Environments

To support heterogeneous hardware and/or software environments, environment specific subdirectories are created in the src, bin and lib directories for object code, linked executables and archive libraries respectively. These subdirectories are named according to the architecture as reported by 'libarch', a library program which determines the current hardware/software configuration.

The following csh code snippet will correctly set the path variable to enable searching the appropriate subdirectories (if they exist). The code snippet assumes that the LIB_PATH environmental variable has been previously set.

set ARCH = `libarch`
if (($ARCH != '') && $?LIB_PATH) then
foreach l (~ `echo $LIB_PATH | tr : \ `)
if ((-d $l/bin/$ARCH) && (`echo $path | grep $l/bin | wc -l`)) then
set path = (`echo $path | sed s%$l/bin\ %$l/bin/$ARCH\ $l/bin\ %`)
endif
end
endif
unset ARCH