
Frequently Asked Questions About netCDF

This document contains answers to some of the most frequently asked
questions on the netcdfgroup mailing list and in the email sent to
support@unidata.ucar.edu.

----------------------------------------------------------------------------

General

   * What Is netCDF?
   * How do I get the netCDF software package?
   * Is there any access to netCDF information on the World Wide Web?
   * What is new in the netCDF 3.3 release?
   * What is the best way to represent [some particular data] using netCDF?
   * Is there a mailing list for netCDF discussions and questions?
   * What is the best way to handle time using netCDF?
   * Who else uses netCDF?
   * What are some references to netCDF?
   * Is there a document describing the actual physical format for a Unidata
     netCDF file?

Platforms and Portability

   * What does netCDF run on?
   * Is netCDF available for Windows 95 or Windows NT?
   * Is netCDF available for the Apple Macintosh?
   * Is netCDF available for MSDOS/Borland platforms?

netCDF and Other Software

   * What other software is available for accessing, displaying, and
     manipulating netCDF data?
   * What other data access interfaces and formats are available for
     scientific data?
   * What is the connection between netCDF and CDF?
   * What is the connection between netCDF and HDF?
   * Has anyone implemented client-server access for netCDF data?

Problems and Bugs

   * Is there a list of reported problems and workarounds?
   * How do I make a bug report?
   * How do I search through past problem reports?

Language Interfaces

   * How does the C++ interface differ from the C interface?
   * How does the Fortran interface differ from the C interface?
   * How does the perl interface differ from the C interface?

Plans

   * Are there plans to add facilities for data compression to netCDF?
   * What other future work on netCDF is planned?

----------------------------------------------------------------------------

General

What Is netCDF?

NetCDF (network Common Data Form) is an interface for array-oriented data
access and a freely-distributed collection of software libraries for C,
Fortran, C++, and perl that provide implementations of the interface. The
netCDF software was developed by Glenn Davis, Russ Rew, Steve Emmerson and
Harvey Davies at the Unidata Program Center in Boulder, Colorado, and
augmented by contributions from other netCDF users. The netCDF libraries
define a machine-independent format for representing scientific data.
Together, the interface, libraries, and format support the creation, access,
and sharing of scientific data.

NetCDF data is:

   * Self-Describing. A netCDF file includes information about the data it
     contains.

   * Network-transparent. A netCDF file is represented in a form that can be
     accessed by computers with different ways of storing integers,
     characters, and floating-point numbers.

   * Direct-access. A small subset of a large dataset may be accessed
     efficiently, without first reading through all the preceding data.

   * Appendable. Data can be appended to a netCDF dataset along one
     dimension without copying the dataset or redefining its structure. The
     structure of a netCDF dataset can be changed, though this sometimes
     causes the dataset to be copied.

   * Sharable. One writer and multiple readers may simultaneously access the
     same netCDF file.

                                  --------

How do I get the netCDF software package?

Source distributions are available via anonymous FTP from the directory
ftp://ftp.unidata.ucar.edu/pub/netcdf/ or from other mirror sites.

Files in that directory include:

netcdf.tar.Z
     compressed tar file of source code for the latest general release,
     version 3.3.1, June 1997
guidec.ps.Z
     NetCDF User's Guide for C (compressed PostScript)
guidec.html.tar.Z
     NetCDF User's Guide for C (compressed tar HTML files)
guidec.ps.Z
     NetCDF User's Guide for Fortran (compressed PostScript)
guidef.html.tar.Z
     NetCDF User's User's Guide for Fortran (compressed tar HTML files)
guide-2.4.3.ps.Z
     NetCDF (Version 2) User's Guide (compressed PostScript)

Binary distributions for some platforms are available from the directory
ftp://ftp.unidata.ucar.edu/pub/netcdf/pub/binary.

Source for the perl interface is available as a separate package,
netCDFPerl, via anonymous FTP from the
ftp://ftp.unidata.ucar.edu/pub/netcdf-perl/ directory.

                                  --------

Is there any access to netCDF information on the World Wide Web?

Yes, the latest version of this FAQ document as well as a hypertext version
of the NetCDF User's Guide and other information about netCDF are available
from http://www.unidata.ucar.edu/packages/netcdf.

                                  --------

What is new in the netCDF 3.3 release?

Version 3.3 is a major update to netCDF that includes a complete rewrite of
the netCDF library. It significantly faster than the previous version and
provides updated interfaces. The netCDF file format is unchanged, so files
written with version 3 can be read with version 2 code and vice versa. For
more details, see the version 3.3 release notes.

                                  --------

What is the best way to represent [some particular data] using netCDF?

There are many ways to represent the same information in any general-purpose
data model. Choices left up to the user in the case of netCDF include which
information to represent as variables or as variable attributes; what names
to choose for variables, dimensions, and attributes; what order to use for
the dimensions of multidimensional variables; what variables to include in
the same netCDF file; and how to use variable attributes to capture the
structure and meaning of data. We provide some guidelines in the NetCDF
User's Guide (e.g., the section on ``Differences between Attributes and
Variables''), but we've found that a little experience helps. Occasionally
we have decided it was useful to change the structure of netCDF files after
experience with how the data is used.

                                  --------

Is there a mailing list for netCDF discussions and questions?

Yes. For information about the mailing list and how to subscribe or
unsubscribe, send a message to majordomo@unidata.ucar.edu with no subject
and with the following line in the body of the message:

     info netcdfgroup


                                  --------

What is the best way to handle time using netCDF?

Discussions of conventions for representing time and handling time-dependent
data is a fairly frequent topic of discussion on the netcdfgroup mailing
list. When the subject comes up, interesting discussions often result, so
we've archived past discussions on this subject at
http://www.unidata.ucar.edu/packages/netcdf/time/.

A summary of Unidata's recommendations is available from
http://www.unidata.ucar.edu/packages/netcdf/time/recs.html. Briefly, we
recommend use of the units conventions supported by the udunits library for
time and other units attributes.

Other groups have established more specific conventions that include the
representation of time in netCDF files. For more information on such
conventions, see the NetCDF Conventions Page at
http://www.unidata.ucar.edu/packages/netcdf/conventions.html.

                                  --------

Who else uses netCDF?

The netCDF mailing list has over 500 addresses (some of which are aliases to
more addresses) in thirty countries. Several groups have adopted netCDF as a
standard way to represent some forms of scientific data.

   * During March and April 1996:

        o over 900 distinct hosts downloaded netCDF software via FTP
        o over 1600 distinct hosts accessed netCDF WWW information via a web
          browser
        o hosts from over 50 countries accessed netCDF software or
          information

A description of some of the projects and groups that have used netCDF is
available from http://www.unidata.ucar.edu/packages/netcdf/usage.html.

                                  --------

What are some references to netCDF?

A primary reference is the User's Guide:

Rew, R. K., G. P. Davis, S. Emmerson, and H. Davies, NetCDF User's Guide for
C, An Interface for Data Access, Version 3, April 1997. (Available from
Unidata or in PostScript form by anonymous FTP from
ftp://ftp.unidata.ucar.edu/pub/netcdf/guidec.ps.Z). There is also a
hypertext version of the User's Guide available at
http://www.unidata.ucar.edu/packages/netcdf/guidec. A similar User's Guide
for Fortran is also available.

Other references include:

Brown, S. A, M. Folk, G. Goucher, and R. Rew, "Software for Portable
Scientific Data Management," Computers in Physics, American Institute of
Physics, Vol. 7, No. 3, May/June 1993, pp. 304-308.

Fulker, D. W., "Unidata Strawman for Storing Earth-Referencing Data,"
Seventh International Conference on Interactive Information and Processing
Systems for Meteorology, Oceanography, and Hydrology, New Orleans, La.,
American Meteorology Society, January 1991.

Jenter, H. L. and R. P. Signell, 1992. "NetCDF: A Freely-Available
Software-Solution to Data-Access Problems for Numerical Modelers".
Proceedings of the American Society of Civil Engineers Conference on
Estuarine and Coastal Modeling. Tampa, Florida. (Also available as a
PostScript file from
http://www.unidata.ucar.edu/packages/netcdf/netcdf.asce.ps).

Kuehn, J.A., "Faster Libraries for Creating Network-Portable Self-Describing
Datasets", Proceedings of the 37th Cray User Group Meeting, (Barcelona,
Spain, March 1996), Cray User Group, Inc.

Rew, R. K. and G. P. Davis, "NetCDF: An Interface for Scientific Data
Access," IEEE Computer Graphics and Applications, Vol. 10, No. 4, pp. 76-82,
July 1990.

Rew, R. K. and G. P. Davis, "The Unidata netCDF: Software for Scientific
Data Access," Sixth International Conference on Interactive Information and
Processing Systems for Meteorology, Oceanography, and Hydrology, Anaheim,
California, American Meteorology Society, pp. 33-40, February 1990.

Rew, R. K. and G. P. Davis, " Unidata's netCDF Interface for Data Access:
Status and Plans," Thirteenth International Conference on Interactive
Information and Processing Systems for Meteorology, Oceanography, and
Hydrology, Anaheim, California, American Meteorology Society, February 1997.

                                  --------

Is there a document describing the actual physical format for a Unidata
netCDF file?

Yes, the NetCDF User's Guide contains an appendix that specifies the netCDF
file format.

Also, the chapter on "NetCDF File Structure and Performance" explains the
physical structure of netCDF data at a high enough level to make clear the
performance implications of different data organizations.

If users only go through the documented interfaces to access netCDF data,
any future changes to the format will be transparent.

----------------------------------------------------------------------------

Platforms and Portability

What does netCDF run on?

The 3.3 release of netCDF is supported on the following platforms:

   * AIX-4.2
   * BSD/OS 2.1, 3.0
   * HPUX-9.05
   * IRIX-5.3, IRIX-6.2, IRIX64-6.2
   * Linux 2.0.27
   * OSF1-4.0
   * SunOS-4.1.3, 4.1.4, 5.5
   * ULTRIX-4.5
   * UNICOS-8.0.4 (Y-MP), 9.0 (C90, T3D), 9.1 (T90), 9.2 (J90),

The INSTALL file in the distribution contains more details about what
compilers and compiler flags were used for building netCDF on various
platforms.

Additionally, here is a Web page of reports of successfull builds in other
environments that we did not or could not try.

                                  --------

Is netCDF available for Windows 95 or Windows NT?

Yes, several users have contributed portions of the port, and we have
provided Makefiles as well as Fortran support. If you wish to build from
source on a Win32 platform, obtain
ftp://ftp.unidata.ucar.edu/dist/ftp/pub/netcdf/contrib/win32/maks.zip,
unpack it, and read WIN32_INSTALL. See the README for more information.

                                  --------

Is netCDF available for the Apple Macintosh?

Unidata doesn't test or maintain a version of netCDF for Macintoshes, but
Chuck Denham (USGS) has ported and made available a version of the netCDF
libraries for Macs.

You can fetch cdflib.o and xdrlib.o for PowerMac via anonymous FTP from the
directory ftp://crusty.er.usgs.gov/incoming/netcdf-mac/. The format is
MacBinary II.

If you are a Matlab user, try using the mexcdf function developed at USGS,
which you can pick up from http://crusty.er.usgs.gov/mexcdf.html. A compiled
netCDF library for PowerMacs is available from
http://crusty.er.usgs.gov/~cdenham/MexCDF/nc4ml5.html as well.

                                  --------

Is netCDF available for MSDOS/Borland platforms?

Unidata doesn't test or maintain a version of netCDF for Borland compilers,
but see the files in the directory
ftp://ftp.unidata.ucar.edu/pub/netcdf/contrib/borland/ for compiled netcdf
2.3.2 libraries for MSDOS/Borland platforms, including XDR and netcdf
libraries for all four memory models.

----------------------------------------------------------------------------

NetCDF and Other Software

What other software is available for accessing, displaying, and manipulating
netCDF data?

Utilities available in the current netCDF distribution from Unidata are
ncdump, for converting netCDF files to an ASCII human-readable form, and
ncgen for converting from the ASCII human-readable form back to a binary
netCDF file or a C or FORTRAN program for generating the netCDF file. A list
of other software that can be used to manipulate or display netCDF data is
available from http://www.unidata.ucar.edu/packages/netcdf/software.html.

                                  --------

What other data access interfaces and formats are available for scientific
data?

The Scientific Data Format Information FAQ provides a good description of
other access interfaces and formats for scientific data, including CDF and
HDF. A brief comparison of CDF, netCDF, and HDF is available in the CDF FAQ.

                                  --------

What is the connection between netCDF and CDF?

CDF was developed at the NASA Space Science Data Center at Goddard, and is
freely available. It was originally a VMS FORTRAN interface for scientific
data access. Unidata reimplemented the library from scratch to use XDR for a
machine-independent representation, designed the CDL text representation for
netCDF data, wrote a User's Guide and made other additions including
aggregate data access, single-file implementation, named dimensions, and
variable-specific attributes.

NetCDF and CDF have evolved independently. CDF now supports many of the same
features as netCDF (aggregate data access, XDR representation, single-file
representation, variable-specific attributes), but some differences remain
(netCDF doesn't support native-mode representation, CDF doesn't support
named dimensions). There is no compatibility between data in CDF and netCDF
form, and as yet no translation software exists to convert data in one form
to data in the other form. For a more detailed description of differences
between CDF and netCDF, see the CDF FAQ.

                                  --------

What is the connection between netCDF and HDF?

The National Center for Supercomputing Applications (NCSA) developed the HDF
software and makes it freely available. HDF is an extensible data format for
self-describing files that was developed independently of netCDF.
Applications and utilities based on HDF are available that support
raster-image manipulation and display and browsing through multidimensional
scientific data. The HDF software includes a package of routines for
accessing each HDF data type, as well as a lower-level interface for
building packages to support new types. HDF supports both C and Fortran
interfaces, and it has been successfully ported to a wide variety of machine
architectures and operating systems. HDF emphasizes a single common format
for data, on which many interfaces can be built.

NCSA has implemented software that provides a netCDF interface to HDF. With
this software, it is possible to use the netCDF calling interface to place
data into an HDF file. The netCDF calling interface has not changed and
netCDF files stored in XDR format are readable, so existing programs and
data will still be usable (although programs will need to be relinked to the
new library). There is currently no support for the mixing of HDF and netCDF
structures. For example, a raster image can exist in the same file as a
netCDF object, but you have to use the Raster Image interface to read the
image and the netCDF interface to read the netCDF object. The other HDF
interfaces are currently being modified to allow multi-file access, closer
integration with the netCDF interface will probably be delayed until the end
of that project.

Eventually, it may be possible to integrate netCDF objects with the rest of
the HDF tool suite. Such an integration will then allow tools written for
netCDF and tools written for HDF to both interact intelligently with the new
data files.

                                  --------

Has anyone implemented client-server access for netCDF data?

Yes, as part of the Distributed Ocean Data System (DODS), developers at the
University of Rhode Island and Massachusetts Institute of Technology have
implemented a client-server based distributed system for access to
oceanographic data over the Internet that supports use of the netCDF
interface for clients. The source code is available in source and binary
forms from ftp://dods.gso.uri.edu/pub/dods/ for the server, sample clients,
and libraries. You do not need to get the source code to try to re-link your
netCDF applications. The server supports use of URL notation for accessing
netCDF data from remote sites running the DODS server.

----------------------------------------------------------------------------

Problems and Bugs

Is there a list of reported problems and workarounds?

Yes, the document Known problems with the netCDF Distribution describes
reported problems and workarounds in version 3.3.1.

                                  --------

How do I make a bug report?

If you find a bug, send a description to support@unidata.ucar.edu. This is
also the address to use for questions or discussions about netCDF that are
not appropriate for the entire netcdfgroup mailing list.

                                  --------

How do I search through past problem reports?

A search form is available at the bottom of the netCDF homepage, providing a
full-text search of the support questions and answers about netCDF provided
by Unidata support staff.

----------------------------------------------------------------------------

Language Interfaces

How does the C++ interface differ from the C interface?

It provides all the functionality of the C interface (except for the
generalized hyperslab access of ncvarputg() and ncvargetg()) and is somewhat
simpler to use than the C interface. With the C++ interface, no IDs are
needed for netCDF components, there is no need to specify types when
creating attributes, and less indirection is required for dealing with
dimensions. However, the C++ interface is less mature and less-widely used
than the C interface, and the documentation for the C++ interface is less
extensive, assuming a familiarity with the netCDF data model and the C
interface.

                                  --------

How does the Fortran interface differ from the C interface?

It provides all the functionality of the C interface. The Fortran interface
uses Fortran conventions for array indices, subscript order, and strings.
There is no difference in the on-disk format for data written from the
different language interfaces. Data written by a C language program may be
read from a Fortran program and vice-versa.

                                  --------

How does the Perl interface differ from the C interface?

It provides all the functionality of the C interface. The Perl interface
uses Perl conventions for arrays and strings. There is no difference in the
on-disk format for data written from the different language interfaces. Data
written by a C language program may be read from a Perl program and
vice-versa.

----------------------------------------------------------------------------

Plans

Are there plans to add facilities for data compression to netCDF?

We have no plans to add data compression to netCDF, although we do plan to
add a form of transparent data packing on write and unpacking on read in
netCDF 4.

Arbitrary array cross-section access and direct access to individual array
values conflict with most simple compression schemes. With netCDF, the
elements of an array variable can be filled in any order or as
cross-sections in any direction. NetCDF permits writing elements in one
order and reading them later in different orders.

Some compression methods require that all the data to be compressed are
known before starting the compression. Techniques like run-length encoding
or anything that depends on exploiting similarities in nearby values can't
be used if nearby values aren't all known at the time some of the data are
to be written.

An alternative that can be implemented above the netCDF library is to adopt
a convention for compressed data that uses a "compression" attribute to
encode the method of compression, e.g.

        x:compression = "rle" ;

for run-length encoding of the data in a variable x. Then when you write the
data, compress them into a bland array of bytes and write all the bytes.
Note that it would be difficult to define the size of such a variable in
advance, since its compressed size depends on its values. You would also
have to give up on cross-section access for such variables, but instead read
the compressed array in all at once and uncompress it before using it.

                                  --------

What other future work on netCDF is planned?

A Java package for accessing netCDF files will be available soon.

Current plans for the next major release, netCDF-4, include the addition of
new types for packed data and the elimination of the 2 Gbyte limit on file
size. A paper is available describing current netCDF status and plans.

Other desirable netCDF enhancements that may be implemented, depending on
available resources include:

   * Link variables pointing to virtual data or data in other files, using
     Davies' FAN notation
   * Multiple unlimited dimensions per dataset
   * Access by key for generalized coordinate variables
   * Support for nested arrays (as in J), ragged arrays, and structures.
   * Support for new types: 64-bit integers, unsigned types
   * Georeferencing library, like udunits
   * NetCDF client-server interfaces
   * Support for wide characters
   * Fortran-90 interface
   * Thread-safety (modulo underlying system calls)

----------------------------------------------------------------------------
