PBDB Data Service: Change Log

• The summary operation didn't actually work the way it was supposed to; these problems have been fixed, and colls/summary now does everything that it did under data1.1 and much more.
• Many tests have been added to the source code, in order to make sure that the data service actually does what the documentation says it is supposed to.

• A new parameter lithology has been added for selecting fossil occurrences, specimens, and collections. If this parameter is specified, only occurrences from the given lithologies will be selected.
• If paleocoordinates are returned (output block paleoloc), any records whose coordinates are blank will now have an explanatory message included in the geoplate field. The message is put there becase it is an unstructured text field and the coordinate fields should always be either empty or contain a valid coordinate.

• If the output format is text (csv or tsv), the summary lines giving the number of occurrences that were analyzed and the number that were excluded will only appear if the parameter datainfo is included in the request. Otherwise, the list of field names will be the first line of the output, consistent with the output of the other data service operations.

• A new operation has been added: combined/auto. This is designed for use by Navigator or other web application clients, and provides a single prefix-search operation for use in auto-completion. This can be configured to search any or all of the following: geologic time intervals, geologic strata, taxa, and database contributor names.

• The parameter private specifies that embargoed data to which you have access should be included in query results. Any URL that contains this parameter will only work if you are simultaneously logged in to the PBDB Classic environment, and will return a 401 Not Authenticated error otherwise. No parameter value is needed.
• You can now specify explicitly whether object identifiers in the results are reported as numbers or using the extended identifier syntax, using the parameter extids with the value yes or no. The defaults are still the same: extended identifiers with the compact vocabulary, numeric identifiers with the PBDB vocabulary.

The following documentation pages have been added:

• Record identifiers and record numbers
• Special parameters

• The following changes have been made in the output of fossil occurrence records:
• New output block coll is available. It reports the name of the collection in which the occurrence was found, along with any remarks about the collection.
• The flags field in the basic occurrence record now records whether the occurrence is identified to an ichnotaxon (I) or form taxon (F).
• The new operation occs/byref selects occurrences using all of the usual parameters including the parameters for selecting references. It sorts the results by default according to the reference from which they were entered, although this can be overridden.
• The operation occs/list no longer takes parameters for selecting occurrences by the reference from which they were entered. Use occs/byref instead.
• The operations occs/refs and occs/byref now accept the following new parameters: pub_type, ref_doi, ref_id.
• The new operation occs/geosum summarizes occurrences according to their present-day geographical location. This is essentially the same operation as colls/summary, but it really belongs here instead of in the Fossil collections section.
• The parameter taxon_reso has been renamed to idreso. The old name is still accepted as an alias.
• The parameter ident_type has been renamed to idtype. The old name is still accepted as an alias. It also takes a new value: reid, which selects all identifications of occurrences that have more than one.
• The occurrence operations now take the following new parameters: idqual, idmod, idgenmod, and idspcmod. These select or exclude occurrences based on the taxonomic name modifiers associated with the occurrence (e.g. cf., ?, n. sp.).
• The occurrence operations now take the new parameters coll_match and coll_re. You can use either one if you want to retrieve occurrences only from collections which have a specific word or phrase in their name. The first alternative takes the % and _ wildcards, while the second takes a regular expression. In either case, the pattern is matched against the collection_name and collection_aka fields. Occurrences will only be returned if they are found in a matching collection.
• The operation /data1.2/occs/single_doc.html now accepts an extended identifier either of type occ or rei, and also takes the parameter idtype. These two changes together allow the retrieval of any identification of the requested occurrence.

• The following changes have been made to the output of fossil collection records:
• The output block rem is obsolete. The information it had displayed (collection_aka) is now part of the basic output block for collections.
• The new operation colls/byref selects collections using all of the usual parameters including the parameters for selecting references. It sorts the results by default according to the primary reference from which they were entered, although this can be overridden.
• The operation colls/list no longer takes parameters for selecting collections by the reference from which they were entered. Use colls/byref instead.
• The operations colls/refs and colls/byref now accept the following new parameters: pub_type, ref_doi, ref_id.
• The new parameters listed for occurrence operations above can also be used to select collections that contain matching occurrences. These parameters include: idreso, idtype, idqual, idmod, idgenmod, idspcmod.
• The collection operations now take the new parameters coll_match and coll_re. You can use either one if you want to retrieve collections which have a specific word or phrase in their name. The first alternative takes the % and _ wildcards, while the second takes a regular expression. In either case, the pattern is matched against the collection_name and collection_aka fields, and collections will be returned if either field matches.

• You can now query for specimen records, and for measurement records associated with them.
• The operations specs/single and specs/list allow you to retrieve information about fossil specimens entered in the database, using the same parameters that are available for selecting occurrences.
• The operations specs/byref and specs/refs allow you to retrieve information about fossil specimens using both parameters that select specimens based on their properties and parameters that select them based on the properties of the associated references.
• The operation specs/measurements allows you to retrieve infomration about measurements of specimens.

• The following changes have been made to the output of taxonomic name records:
• The output block pres is now obsolete. The information it contained is now reported in the field flags. The flag F indicates a form taxon, while I indicates an ichnotaxon.

• The following changes have been made to the output of taxonomic opinion records:
• The output block seq is now available. Just as with the identically named block available for taxonomic names, this block reports the sequence number of the taxon which is the subject of the opinion.
• The operations taxa/refs and taxa/byref now take the parameters idtype and idqual. If you are requesting occurrence, specimen, and/or collection references for a particular taxon or set of taxa, you can use these parameters to fine-tune which particular occurrences will have their references selected.
• The operations taxa/refs and taxa/byref now accept the following new parameters: pub_type, ref_doi.

• No changes except for the general ones listed above.

• No changes except for the general ones listed above.

• The operation refs/list now takes the new parameters pub_type and ref_doi.

• No changes except for the general ones listed above.

• No changes.

• The default limit on the number of results has been removed. You can still specify a limit using the parameter limit, but if you do not explicitly include this parameter then all matching records will be returned.

We have taken precautions to reduce the likelihood of carelessly chosen URLs returning enormous result sets. All of the operations that list data records will still return a parameter error if no parameters are given. For example: /data1.2/taxa/list.json. You must provide at least one parameter to limit the result set, or explicitly include the parameter all_records which will return all records of the indicated type known to the database. For example:

/data1.2/taxa/list.csv?all_records

Please be careful when doing this, because the result may exceed 100 MB in size. If you are experimenting with different URLs just to see what results you get, it might be a good idea to use something like the following pattern:

/data1.2/taxa/list.csv?rowcount&limit=100& ... other parameters

You will then get back at most 100 records for each query, and the header will tell you how many records the full query found. Once you are sure you have the correct URL, you can then remove the limit parameter.

• There are new options available for specifying taxonomic names. There is a new documentation page specifically discussing the specification of taxonomic names for this version of the data service.
• Object identifiers are now strings instead of numbers. In the text formats (txt, csv, tsv), object identifiers are still reported as numbers for easy comparison with previous PBDB downloads. Otherwise, they have a form like the following examples:

col:1003	This represents the collection with collection_no = 1003.
txn:69296	This represents the taxon with taxon_no = 69296.
ref:5041	This represents the reference with reference_no = 5041. And so on.

When specifying object identifiers in parameter values, you can use either the numeric or the text form, or a form consistent with the LSID specification. For example, each of the following URLs returns the same result:

• /data1.2/taxa/single.json?id=69296
• /data1.2/taxa/single.json?id=txn:69296
• /data1.2/taxa/single.json?id=urn:lsid:paleobiodb.org:txn:69296

We don't expect the LSID form to be used much now, but it lays the foundation for future integration with other databases that understand LSIDs.

• Except as noted below, all operations that take the parameter interval or interval_id will now accept multiple intervals. In general, the time range used for selecting records will span from the beginning of the earliest specified interval to the end of the latest. In other words, you can specify any continuous range but not a range with gaps.
• The operations that return the basic record types (occurrences and collections) now take the additional value full for the parameter show. This will include all output blocks relevant to the main record, but not ent and crmod and other auxiliary blocks (you can include these explicitly if you wish). Including show=full in a URL will guarantee that, now or in the future, all of the information from the main data record will be returned. If we subsequently add new data blocks, that information will be added to the set returned by full as well. Thus, this will be a good idea to include in URLs that you publish.
• The special parameter count has been renamed to rowcount. This was done because count is too useful a word and was needed as a parameter for the diversity operations.
• The special parameter showsource has been renamed to datainfo. This seemed like a more descriptive word to use.

The following additional documentation pages have been added:

• Specifying taxonomic names
• Basis and precision of coordinates

• The following changes have been made in the output of fossil occurrence records:
• The old output block phylo has been split into two new blocks, class and classext.
• New output block subgenus is available, reporting the subgenus if any into which the occurrence falls.
• New output blocks ecospace, taphonomy, and etbasis are available. These report the ecospace and taphonomy information of the taxon corresponding to each occurrence.
• New output block plant is available. This reports the plant organ, if any, identified for each occurrence.
• New output block methods is available, reporting the collection methods, museum information, and related fields.
• New output block resgroup is available, reporting the research group with which this collection is associated.
• New output block full is available. This includes all of the information in the blocks class, subgenus, plant, abund, coords, loc, paleoloc, prot, stratext, lithext, geo, methods, rem.
• The old fields taxon_name, taxon_rank, taxon_no, matched_name, matched_rank, and matched_no have been replaced by the following new fields: identified_name, identified_rank, identified_no, accepted_name, accepted_rank, accepted_no, taxonomic_reason.
• The old fields early_age and late_age are now renamed max_ma and min_ma. They are now part of the basic output block.
• The old field superceded has been removed. The new field flags will contain the letter R for identifications that have been superceded by a reidentification. Note that you will only see these if you specify ident=orig or ident=all.
• The old field llp has been split into latlng_basis and latlng_precision and has been moved to the output block loc. But in the compact vocabulary, it remains the single field prc.
• In the output block loc, the field state is now called stp in the compact vocabulary. The field geogcomments has been added.
• In the output block ident, the old fields genus_name and genus_reso have been renamed to primary_name and primary_reso.
• The old output blocks attr and time been removed. The field pubyr has been added to the basic output block. The time fields were there to support functionality which was never added to Navigator, and so are now removed.
• occs/single has no changes to the parameters. The parameter id is still an alias for occ_id.
• occs/list has the following parameter changes:
• New parameters all_records, match_name, immediate. The parameter id is now an alias for occ_id.
• The parameter timerule now takes the additional value major. The parameter earlybuffer has been replaced by timebuffer, which now provides a default value for latebuffer if the latter is not specified.
• The parameters created_after, authorized_by, etc. are now renamed occ_created_after, occ_authorized_by, etc.
• occs/taxa now takes the same parameters as occs/list, with the exception of all_records, occ_id, and order. It also has the following parameter changes:
• The old parameter rank has been replaced by the new parameters reso and count.

Its output is the same as the output of taxa/list (see below), with the following changes:

• New block occapp reports the age of first and last appearance according to the set of occurrences being analyzed rather than from the full set of occurrences in the database.
• New block subcounts reports subtaxon counts.
• New operation occs/diversity which tabulates fossil diversity over time for any selection of occurrences.
• New operation occs/quickdiv which does a quicker but less precise computation of fossil diversity over time. This operation is intended primarily to support the Navigator application.
• New operation occs/prevalence which reports the most prevalent major taxa from any selection of occurrences. At the moment you can only specify a single time interval with this operation, but that limitation will be removed in a subsequent version of the data service.
• occs/refs has the same parameter changes as occs/list, plus the following:
• The parameters author and primary have been renamed ref_author and ref_primary.
• The parameter year has now been replaced by ref_published, ref_published_after, and ref_published_before.
• The parameters reftitle and pubtitle have been renamed ref_title and pub_title.

Its output has the following changes:

• New output block counts is available, reporting the number of taxa, occurrences, etc. which were entered from each reference. The old field associated_records has been removed.
• The values for fields ref_type and record_type have changed slightly.
• The field comments is now called rem in the compact voacbulary, in order to be consistent with other comments/remarks fields.
• The field language is now called lan in the compact vocabulary, in order to free up lng for longitude.

• The following changes have been made in the output of fossil collection records:
• New output block methods is available, reporting the collection methods, museum information, and related fields.
• New output block resgroup is available, reporting the research group with which this collection is associated.
• The old fields early_age and late_age are now renamed max_ma and min_ma. They are now part of the basic output block.
• The field llp has been split into latlng_basis and latlng_precision and has been moved to the output block loc. But in the compact vocabulary, it remains the single field prc.
• In the output block loc, the field state is now called stp in the compact vocabulary. The field geogcomments has been added.
• The fields cx_int_no, early_int_no, and late_int_no have been removed. The output block time is now empty and deprecated.
• colls/single has no changes to the parameters. The parameter id is still an alias for coll_id.
• colls/list has the following parameter changes:
• New parameters all_records, match_name, immediate. The parameter id is now an alias for coll_id.
• The parameter timerule now takes the additional value major. The parameter earlybuffer has been replaced by timebuffer, which now provides a default value for latebuffer if the latter is not specified.
• The parameters created_after, authorized_by, etc. are now renamed coll_created_after, coll_authorized_by, etc.
• colls/summary has the same parameter changes as colls/list. Its output has the following changes:
• The fields lng_min, lng_max, lat_min and lat_max are now called lx1, lx2, ly1 and ly2 in the compact vocabulary.
• The fields early_age and late_age are now renamed max_ma and min_max. They are now part of the basic output block.
• colls/refs has the same parameter changes as colls/list, plus the following. The parameters author and primary have been renamed ref_author and ref_primary. The parameter year has now been replaced by ref_published, ref_published_after, and ref_published_before. The parameters reftitle and pubtitle have been renamed ref_title and pub_title.
• New output block counts is available, reporting the number of taxa, occurrences, etc. entered from this reference.
• See Bibliographic references below for a list of other changes to the output.

• The following changes have been made to the output of taxonomic name records:
• In the compact vocabulary, the field orig_no is now labeled oid and the field taxon_no is now labeled vid.
• In the pbdb vocabulary, the field rank has been renamed taxon_rank.
• The new field flags indicates special information about certain records.
• The field associated_records has been removed, and the field n_occs has been added to the output blocks size and subcounts.
• The field common has been moved from the basic output to the output block common.
• The field senior_no has been replaced by accepted_no, and the field accepted_name has been added to the basic output.
• The field status has been replaced by difference.
• The field parent_no now lists the senior synonym of the parent taxon. If you want to know the immediate parent taxon, include the output block immparent to get the field immpar_no.
• In the output block app, the fields firstapp_ea, firstapp_la, lastapp_ea, and lastapp_la have been renamed to firstapp_max_ma, firstapp_min_ma, lastapp_max_ma, lastapp_min_ma. New fields early_interval and late_interval have been added.
• The old output block phylo has been renamed to class.
• The following new output blocks are available: common, parent, immparent, subcounts, ecospace, taphonomy, etbasis.
• taxa/single has the following changes to the parameters:
• New parameter exact indicates that information about the exact name specified should be returned, rather than about the currently accepted variant.
• taxa/list has the following changes to the parameters:
• New parameters all_records, match_name, exclude_id, immediate, max_ma, min_ma, interval, interval_id, taxon_created_before, etc., taxon_authorized_by, etc.
• The parameter rel now accepts the new values accepted, variants, and immparent. The values self, parents, and common_ancestor have been renamed to current, parent, and common respectively.
• The parameter status has been renamed to taxon_status. This was necessary in order to distinguish it from op_status.
• New operation taxa/opinions returns information about the taxonomic opinions associated with any set of taxa. This takes the same parameters as taxa/list, plus additional ones for selecting the desired opinions.
• The operation occs/taxa is described above in the section on Fossil occurrences.
• taxa/refs has the same parameter changes as taxa/list, plus the following:
• New parameter select which selects one or more reference modes. You can use this parameter to select authority references, classification references, occurrence references, etc.
• The parameters author and primary have been renamed ref_author and ref_primary.
• The parameter year has now been replaced by ref_published, ref_published_after, and ref_published_before.
• The parameters reftitle and pubtitle have been renamed ref_title and pub_title.

Its output has the following changes:

• New output block counts is available, reporting the number of taxa, occurrences, etc. which were entered from each reference. The old field associated_records has been removed.
• The values for fields ref_type and record_type have changed slightly.
• The field comments is now called rem in the compact voacbulary, in order to be consistent with other comments/remarks fields.
• The field language is now called lan in the compact vocabulary, in order to free up lng for longitude.
• New operation taxa/byref which groups a selected set of taxa by reference and by reference mode. It takes the same parameters as taxa/refs, so that you can make both calls and match up the reference_no values.
• No changes to the operations taxa/auto, taxa/thumb, taxa/icon.

• New operations taxa/opinions, opinions/list, opinions/single provide access to the taxonomic opinions in the database.

• The operation intervals/list has a new parameter scale_level.
• The output field level has been renamed to scale_level.

• No changes.

• The following changes have been made to the output of bibliographic reference records:
• New output block counts is available, reporting the number of taxa, occurrences, etc. which were entered from each reference. The old field associated_records has been removed.
• The values for fields ref_type and record_type have changed slightly.
• The field comments is now called rem in the compact voacbulary, in order to be consistent with other comments/remarks fields.
• The field language is now called lan in the compact vocabulary, in order to free up lng for longitude.
• The operation refs/single has no changes to the parameters.
• The operation refs/list has the following changes to the parameters:
• The parameters author and primary have been renamed ref_author and ref_primary.
• The parameter year has now been replaced by ref_published, ref_published_after, and ref_published_before.
• The parameters reftitle and pubtitle have been renamed ref_title and pub_title.
• The parameters created_after, authorized_by, etc. have been renamed to ref_created_after, ref_authorized_by, etc.
• The operations occs/refs, colls/refs, and taxa/refs are discussed above.

• New output block countries returns a list of the ISO-3166-1 country codes. The output includes the continent on which each country is located.

• No changes except for those indicated above.