Data Migration

How data are migrated to Arctos depends on the state of your digitization project as well as the size and scope of your collection. There are two main ways for a new collection to add data to Arctos, and we will work with you to decide on the best migration workflow for your data:

Data Entry Interface

If you are just getting started in digitizing your collection, you may want to use the single-record data entry interface. This form is customizable to a certain degree, and uses existing data authorities and code tables to populate values for geography, taxonomy, collectors, attribute types and (for some attributes, e.g., sex) the possible values, and part names. The advantage of entering data using this form is that it requires minimal knowledge about how data are structured in Arctos. The disadvantage is that data entry is slower because the records are entered one at a time; however, the process can be sped up by customizing the form to carry over values from one record to the next.
Read the “How-to” guide for entering data for a single record

Bulkloading Data

The most common way that collections migrate their data to Arctos is by bulkloading. Data are formatted as a comma-delimited (CSV) file, and column headings must match Arctos field names (see the “Bulkloader Builder”). In addition, values for data controlled by authorities and code tables must match the values in Arctos exactly, or the data won’t load properly (records are flagged and will need to be fixed before trying again). Arctos has developed a number of pre-bulkloading tools to help with checking and cleaning data. The advantage of bulkloading data is that a large number of records can be loaded at once. The disadvantage is that there is more of a learning curve, and it requires some level of knowledge about how Arctos data are structured; however, the Arctos Working Group and Technical Support (see contacts) are available to help.
Read the “How-to” guide for bulkloading data
Download a Powerpoint presentation on getting data from Excel to Arctos

Arctos also collaborates with VertNet to clean data prior to migration into Arctos. As part of its workflow, VertNet has developed data migration tools that are customized for each collection. These tools map data to the DarwinCore standard and look for data incompleteness and inconsistencies. Collections may run their data through the VertNet migrators as a first pass in data cleaning, before the data come to Arctos for bulkloading.

Costs of Data Migration

Data migration can be a simple or complicated task, depending on the collection size, complexity of data types, and state of the data. Collections may contract with the Arctos lead programmer at $70 per hour to assist with the migration process. Collections also may include funding for data migration in grant budgets. We can help to estimate the cost of data migration based on an initial questionnaire (see Join Arctos) and subsequent discussions. Collections who are just starting to digitize their collections, or who have simple, ‘clean’ data, can typically enter their data into Arctos at minimal or no cost.

Elizabeth Wommack and Joshua Penalba cataloging Steller's Jay specimens into Arctos

Elizabeth Wommack and Joshua Penalba cataloging Steller’s Jay specimens into Arctos