Christmas Bird Count Database specification

John W. Shipman, john@nmt.edu
Zoological Data Processing
507 Fitch Avenue NW
Socorro, NM 87801
(505) 835-0235
Homepage: http://www.nmt.edu/~shipman

Contents

• 1. Introduction
  • 1.1 Scope
  • 1.2 Definitions
  • 1.3 Typographic conventions
• 2. Nomenclatural base
• 3. General structure of the database
  • 3.1 The countId field
  • 3.2 How the database is linked
• 4. File layouts
  • 4.1 Circle file, cir
  • 4.2 Standard linkage file, std
  • 4.3 Effort file, eff
  • 4.4 Weather file, wea
  • 4.5 Census file, cen
  • 4.6 As-published circle data, aspub

1. Introduction

The Christmas Bird Count (CBC) censuses, published in the periodicals Audubon Field Notes and American Birds, have been taken since 1900. As a source of long-term population data, it has few peers; however, its utility in printed form is limited.

This document describes a system for representing the data from these censuses in a general computer database form.

Most years' data (1st-62nd and 73rd-90th) were entered by Zoological Data Processing. Data for the 63rd--72nd CBCs were obtained from Carl Bock from old tapes produced by his project at the University of Colorado. Data for the 91st and succeeding CBCs were provided by the National Audubon Society from their publishing operation. There are some minor differences between these sources in the way data were captured; the record layout narratives below notes the differences between the years.

1.1 Scope

This version is limited to the data from the U.S.A. and Canada (including the French islands of St.-Pierre and Miquelon). Inclusion of the data from other countries would require significant modifications to the method of encoding bird names.

1.2 Definitions

The word circle generally refers to aspects of one count that are generally unchanged from year to year: the count area's latitude and longitude, its geographic name, and so forth. The term count will generally refer to one year's census of that circle.

The term standard refers to those names and coordinates that are used in the computer database. The term as-published refers to the names and coordinates that appear in the published counts.

1.3 Typographic conventions

Actual computer file names, field names, and field contents (character strings as entered and stored) are shown in typewriter type.

The symbol ``_'' is used to represent a space character.

2. Nomenclatural base

For proper interpretation of the six-letter codes used to describe types of birds, the CBC database depends on an infrastructure of files that describe all the possible bird types from the census file. This system is described in a separate document, `A system for representing taxonomic nomenclature'.

3. General structure of the database

Before we discuss the layout of the database, it is relevant to bring up a sticky problem of organization that affects the design.

One of the most persistent problems in the design of this database is how to treat circles whose centers have shifted slightly one or more times. When the center moves only a minute or two of latitude or longitude, for the purposes of most researchers the censuses from the different centers might as well be treated as from the same location.

However, sometimes even a tiny shift can make a huge difference in the habitat coverage. For example, if a circle containing no open water is shifted so that it includes a reservoir, any conclusions based on sudden increases in waterfowl populations may be unwarranted.

For this reason, I think it would be best if the user of the database has a choice as to whether to lump overlapping circles or not.

An earlier version of this database used the concatenation of the latitude and longitude fields as the link that connected census data to circle and effort data. However, in practice this meant that a number of common operations---such as correcting center coordinates, or deciding whether or not to lump overlapping circles---required a lengthy pass through many megabytes of census data to change the link values.

The solution is to use a single identifier called the countId field to identify each circle.

3.1 The countId field

We need to identify each count---that is, each set of data for a single circle counted in a particular year. We call this identifier the countId.

The published counts have used two different schemes for identifying the counts within a year:

• The older counts are numbered within each year. Because of counts arriving late and being added to the sequence after these numbers were assigned, you will occasionally find counts with numbers like 127A or even 127B.
• More recent counts have used a two-letter code for each circle within a state. Although counts can include two or even three states, a count is always associated with the first state listed. So, for example, in state NM, code ZU refers to the Zuni circle.

These identifiers alone are not sufficient to identify a single count. Count #1225 might be Modoc, CA in one year and Malibu, CA in another year.

So, we define an aggregate called the countId field which consists of the CBC number, left-zero-padded to three digits, followed by the circle's identifier.

In the older counts entered by Shipman (and the data from the 63rd--72nd CBCs converted from the Bock project), the countId has the form

    yyynnnna

In data derived from the publisher of the modern counts (Clinchy Associates), the countId has the form

    yyyssii_

Here are some examples of countId fields:

  0760034_  76th CBC, count #34.
  0830947A  83rd CBC, count #947A.
  104NMZU_  104th CBC, circle ZU in state NM.

3.2 How the database is linked

The countId field, then, uniquely identifies one circle counted in one year. So this field is the key that relates the various files of the database.

The std file serves as the pivot, relating census and effort records to a particular circle name and center. Since the census records are by far the biggest part of the database, omitting the lat-long of the circle center from that database means that we can change the effective location of census records by using a different std file.

We can, for example, have one version of the std file that does not lump circles, no matter how tiny the shift. This version would be good for waterfowl studies, where a tiny shift might drastically change the inclusion of waterfowl habitat. We could have another version that lumps all circles that overlap at least 10%, and another version that lumps all circles that overlap at least 50%. We can even set up special std files that lump all the counts in a particular bioregion.

So, here are the six principal files of the CBC database and their relationships:

1. The cir table describes each center, and the name that goes with it, along with other permanent features like codes for the bioregion. This table is keyed by lat-long (that is, the latitude concatenated with the longitude).
2. The std table relates each countId key with a lat-long key, so it connects the cir table with all the other tables.
3. Each record of the eff table describes one year's count of a single circle. It gives the number of observers, effort data, and other fields such as date. This table is keyed by countId.
4. The wea table gives weather data, where available. It too is keyed by countId.
5. Each record of the aspub table describes the name and location of one year's count as published. This file tracks changes in name and nominal coordinates, including even errors, so long as it was published that way. This table is also keyed by countId.
6. For each form of bird printed in the body of a circle, there is one census (cen) record, keyed by countId.

4. File layouts

All the files produced for this project are ``flat files,'' meaning that each field has a fixed size. This form was chosen because it is easy to import into most any database system.

4.1 Circle file, cir

Every area that has ever been counted will have at least one corresponding circle record in file cir, defining its location. In those cases where the center has been moved one or more times for a distance of more than 1 minute of latitude or longitude, there may be multiple circle records for a count of a given name.

Only the ``standard'' coordinates and names are defined in this file. We have tried to use the most recent names and the most accurate map-checked coordinates whenever possible, but in many cases the coordinates are an estimate or an outright guess. We welcome any additional information that may help us in establishing the true locations of the counts; please forward any such information in writing to the author.

Here are the field sizes, field names, and the descriptions of their contents.

• 9, latLong: Latitude and longitude of the count center. This field has the form ddmmDDDMM, where dd is degrees and mm is minutes of north latitude, and DDD is degrees and MM is minutes of west longitude. Counts in east longitude are treated as west longitude beyond 180°; for example, Amchitka Island, at 179° 18' E. Long. is treated as 180° 42' W. Long. Examples of the latitude and longitude fields:

        400909036    40° 9' N. Lat., 90° 36' W. Long.
                     (Rushville, IL)
        512518042    51° 25' N. Lat., 179° 18' E. Long.
                     (Amchitka, AK)

• 4, phys: Up to two standard physiographic region codes, defined only in the ``lower 48'' and Canada. The first or only code is for the major region, and the second is a minor region code when the circle is on the border between two physiographic regions, or blank for circles contained within a single region. These codes are defined at http://www.mbr-pwrc.usgs.gov/bbs/physio.html.
• 1, water: This is a code for major water body types. It may be any of the following:

        _   No salt water (blank)
        o   Open ocean included in circle
        e   Ocean estuary included in circle (no open ocean)
        p   Pelagic

• 1, odd: This field is a code to identify counts taken in any area other than standard-sized circles.

        _   Ordinary circle, 15-mile diameter (blank)
        p   Pelagic-only transect
        x   Odd-shaped and not pelagic-only

• 6, regs: Region codes. This field has the form rrsstt, where rr, ss, and tt are codes for up to three regions (states or provinces). The encoding of the region codes group depends on what country the region is in. In the U.S., standard two-letter postal abbreviations are used (e.g. ny, tn). The one count in France (St. Pierre et Miquelon Islands) uses the special region code fr. In Canada the regions are:

        ab    Alberta
        bc    British Columbia
        mb    Manitoba
        nb    New Brunswick
        nf    Newfoundland (including Labrador)
        nt    North West Territories
        ns    Nova Scotia
        on    Ontario
        pe    Prince Edward Island
        pq    Province Québec
        sk    Saskatchewan
        yt    Yukon Territory

  The codes are used from left to right as necessary, and the unused fields are all blanks. For example, if the count is entirely within one region, rr is used and ss and tt are blank. If a circle falls within two states, rr is the primary state code (this determines which state's section of the listing includes that count), ss is the secondary state code, and tt is blank. Here are some examples of encoding the regs field:

        mb____  Entirely within Manitoba.
        ny____  Entirely within New York.
        onny__  Listed with Ontario, also in New York.
        iailmo  Iowa, Illinois, Missouri (Keokuk).
        fr____  St. Pierre et Miquelon Islands (France).

• (varies), cirname: Circle name. This field is variable length (trailing blanks are trimmed) but will not exceed 50 characters. Certain common abbreviations are used in this field:

        M.A.     Management Area
        N.M.     National Monument
        N.P.     National Park
        N.W.R.   National Wildlife Refuge
        P.P.     Provincial Park
        S.P.     State Park
        W.M.A.   Wildlife Management Area

4.2 Standard linkage file, std

The std file stands between the circle file and the effort and census data, representing our current best guesses about the exact association between bird sightings and locality names and coordinates.

Fields are:

9, latLong: Standard latitude and longitude of the count center, as in the cir file.
• 8, countId: Links to a particular circle's effort, weather and census files.

4.3 Effort file, eff

Each effort record corresponds to one year's counting of a circle. Fields:

• 8, countId: Identifies a specific count.
• 8, yyyymmdd: Date, containing the last two digits of the year, followed by the month number and day number, e.g., 19871225 for Christmas 1987. In the Bock years, and where omitted from the original, this date is the date of Christmas (arbitrarily).
• 3, nobs: Number of observers.
• 5, phtot: The total party-hours are represented here as a scaled number of tenths, that is, there is an assumed decimal after the first four digits. For example, 43 party-hours would be represented as __430, and 43.25 party-hours as __433. Unknown values are represented as the value -1; data analysis programs that use these fields should omit from their data sets any record containing such a value.
• 5, phf: Party-hours by foot. In this and the following four fields, the times are scaled in exactly the same way as the total party-hours data. Blank fields indicate no effort, as opposed to fields containing -1, which means unknown effort.
• 5, phc: Party-hours by car.
• 5, pho: Party-hours by all other means.
• 5, hfd: Hours at feeders.
• 5, howl: Hours owling.
• 5, pmtot: Total party-miles. In this and the following four fields, these four five-digit fields, party-miles data (total, by foot, by car, and by other means) are represented as a number of tenths, i.e. there is an assumed decimal after the fourth digit. For example, 4.3 miles would be represented as ___43. Again, the value -1 is used to indicate unknown data.
• 5, pmf: Party-miles by foot.
• 5, pmc: Party-miles by car.
• 5, pmo: Party-miles by all other means.
• 5, mowl: Miles owling. This datum was taken starting with the 82nd CBC.

4.4 Weather file, wea

Weather data is separated from effort data because it is recorded for relatively few years---just the Bock data (63rd--72nd CBCs).

• 8, countId: Identifies a specific count.
• 2, amcld: AM cloud cover. All the weather data (cloud, precipitation, temperature, wind, snow, ice) are valid only in data for the Bock years. Codes for cloud cover are:

        bl   Blue
        pc   Partly Cloudy
        mc   Moderately Cloudy
        tc   Totally Cloudy
        pf   Partly Foggy
        mf   Moderately Foggy
        tf   Totally Foggy

• 2, amprec: AM precipitation, which uses these codes:

      no   No precipitation
      ir   Intermittent rain
      lr   Light rain
      mr   Moderate rain
      hr   Heavy rain
      is   Intermittent snow
      ls   Light snow
      ms   Moderate snow
      hs   Heavy snow
      ic   Intermittent combination (sleet, freezing rain, hail,
            snow/rain)
      lc   Light combination
      mc   Moderate combination
      hc   Heavy combination

• 2, pmcld: PM clouds, encoded in the same way as amcld.
• 2, pmprec: PM precipitation, encoded in the same way as amprec.
• 3, lotemp: Low temperature in Fahrenheit, signed.
• 3, hitemp: High temperature in Fahrenheit, signed.
• 3, winddir: Wind direction, one of the 16 principal compass points: N, NNE, NE, ENE, E, and so forth. This field should be left justified, filled with blanks. For example, NE would be stored as ne_.
• 2, lowind: Low wind speed, in miles per hour.
• 3, hiwind: High wind speed.
• 3, snow: Snow cover, in inches.
• 2, ice: Freshwater ice cover. Uses these codes:

      o    Open
      mo   Mostly open
      po   Partly open
      f    Frozen over

4.5 Census file, cen

Each form of bird (species or not) mentioned in the body of a count results in one body (census) record. Furthermore, for records where multiple genders or age classes are mentioned, each different combination of gender and age is encoded in a separate record.

Warning! One implication of the above paragraph is that applications programs cannot expect there to be only one record for a given form within a given count. For example, there might be three Bald Eagle records within one count: one record each for adults, immatures, and birds of unknown age. Given all the different age and sex and other codes, there may be many! Therefore, applications programs that want to extract species totals must find all matching records, and then total them.

Fields of the census record are:

• 8, countId: Identifies the specific count.
• 3, formseq: A sequence number which can be used to sort body records into the order they were entered. Since the checklist order varies considerably over the years, being able to sort body records into a phylogenetic order may not put them into the published order.
• 6, form: A code describing the type of bird seen. The formation of this code from the common name is described in a companion document, An encoding system for North American Bird Names. Forms shorter than full length are flush left with right blank padding.
• 1, rel: Relation code, describing the relationship between the form and altform fields. In most cases, where a single form code is involved, both rel and altform fields are blank. For hybrid forms, and forms given as a pair of alternatives (e.g. ``Hammond's/Dusky Flycatcher''), the relation code is:

        x   Hybrid of form and altform, e.g.,
            Mallard x Common Pintail
        /   Either form or altform,  e.g.,
            Hammond's/Dusky flycatcher.

• 6, altform: Alternate form code. This is normally blank, but when rel is nonblank, altform contains the second form code. Here are some examples of the way the primary form code, relation code, and alternate form code are used together.

        yebloo         Yellow-billed Loon (standard species)
        Gavia          Gavia sp. (form not identified to species)
        raptor         raptor sp. (form not identified to species)
        amewigxmallar  Mallard x American Wigeon (hybrid)
        dowwoo/haiwoo  Downy / Hairy Woodpecker (two alternatives)
        blugoo         Blue Goose (subspecific identification)
        resfli         Red-shafted Flicker (subspecific identification)

  For hybrids and pairs of alternatives, the convention is to place in the form field the code that is lower in the alphabet, and place the other code in the altform field. This is so a given hybrid will always have the same code structure; without this rule, someone looking for Mallard x American Wigeon hybrids would have to search for two code groups, amewigxmallar and mallarxamewig.
• 8, census: The number of individuals seen. The value -1 is used to signify that no number was given. This is usually the case for birds ``in count area count week but not seen count day;'' the number is occasionally omitted in the main body of the count as well. If more than 99,999,999 blackbirds ever turn up on a count (and the Pine Prairie circle in Lousiana has had over ten million), this field will have to be expanded. Census counts for the Bock years are truncated to only two significant digits, so for example a count of 329,000 would appear as 320,000.
• 1, age: Age code, encoded like this:

        _     Unknown (blank)
        a     Adult
        i     Immature
        p     Female/immature (symbolized by Greek phi)

• 1, sex: One of the following sex codes:

        _     Unknown
        m     Male
        f     Female
        p     Female/immature (phi)

• 1, plus: Normally blank, but contains the symbol ``+'' for birds listed as ``in count area count week but not seen count day.'' Bock years do not contain such records.
• 1, q: Normally blank, but may contain the symbol ``q'' to denote a questionable record. Deciding exactly which remarks by the regional editor translate into a question mark is a judgement call; generally remarks about the numbers are not flagged, but any remarks about the identification are. The idea here is to put a ``q'' when there is some question as to whether the bird was there at all, but not in cases where the bird was probably there but the numbers are suspiciously high.

4.6 As-published circle data, aspub

In order to aid in tracking name and center changes (and typographical errors, some of which persist for many years), aspub tracks as closely as possible the actual published center coordinates and circle names used in each year. This file should be proofed carefully against the published circle accounts.

Exception: in cases where the published latitude and longitude have obviously been transposed, the aspub file need not track this typo. In many cases, the coordinates would not physically fit in the fields anyway.

Fields are:

• 8, countId: Identifies a specific count.
• 9, latLong: Latitude and longitude, encoded in the same way as in the cir file.
• 6, regs: Region codes, encoded as in the cir file.
• (variable), cirname: Circle name, as in cir. For historical reasons, a different and larger set of abbreviations is used in this field:

        AFB    Air Force Base
        Co.    County
        Cos.   Counties
        Ft.    Fort
        GMA    Game Management Area
        I.     Island
        Jct.   Junction
        L.     Lake
        MA     Management Area
        MBR    Migratory Bird Refuge
        Mt.    Mount
        Mtn.   Mountain
        Mts.   Mountains
        NF     National Forest
        NFRA   National Forest Recreation Area
        NG     National Grasslands
        NGP    National Game Preserve
        NHP    National Historical Park
        NL     National Lakeshore
        NM     National Monument
        NP     National Park
        NR     National River
        NRA    National Recreational Area
        NS     National Seashore
        NWR    National Wildlife Refuge
        NWSC   Naval Weapons Support Center
        PP     Provincial Park
        Pt.    Point
        R.     River
        RA     Recreational Area
        SF     State Forest
        SFWA   State Fish & Wildlife Area
        SGA    State Game Area
        SGP    State Game Preserve
        SGR    State Game Reserve/Refuge
        SP     State Park
        SRA    State Recreational Area
        SWR    State Wildlife Refuge
        Twp.   Township
        WA     Wildlife Area
        WMA    Wildlife Management Area
        WR     Wildlife Refuge
        WS     Wildlife Sanctuary

  In order that circle records will sort consistently by locality name, abbreviations should not be used at the beginning of the name. For example, ``Point Pelee,'' not ``Pt. Pelee;'' ``Mount Olive,'' not ``Mt. Olive.'' The exception to this exception is ``St.'' for ``Saint:'' ``St. Louis,'' not ``Saint Louis.'' Also, words that are a significant part of a name should be not abbreviated, for example ``Salt Lake City'' instead of ``Salt L. City,'' and ``Rocky Mountain NP,'' not ``Rocky Mtn. NP.''

Last updated: 2010/03/10 19:55:50
URL: http://www.nmt.edu/~shipman/z/cbc/db_spec.html