*************************************************************************
*          Build and testing instructions for HDF5's ROS3 VFD           *
*************************************************************************

NOTE: HDF5 2.0 and later requires CMake for building.
      See INSTALL_CMake.txt for complete build instructions using CMake Presets.

                     Table of Contents

Section I:   Preconditions
Section II:  Obtaining the aws-c-s3 library
Section III: Building the aws-c-s3 library from source
Section IV:  Building HDF5 with the ROS3 VFD enabled
Section V:   Testing the ROS3 VFD

************************************************************************

========================================================================
I. Preconditions
========================================================================

- Refer to release_docs/INSTALL for preconditions and instructions for
  building HDF5

========================================================================
II. Obtaining the aws-c-s3 library
========================================================================

As of the HDF5 2.0.0 release (or HDF5 commit 75fff3ed0663d3617e3c5cc3c57c1f0c83b38be4),
HDF5's ROS3 VFD requires the aws-c-s3 library (https://github.com/awslabs/aws-c-s3)
in order to be built. This library is distributed by several package
managers, usually as 'aws-c-s3', but consult your particular package
manager for the name of this package if it's available. HDF5 installs
this library in the following ways in CI testing:

Linux (Ubuntu):

 - from source, as Ubuntu doesn't have a package for aws-c-s3 yet

Windows:

 - vcpkg install aws-c-s3 (make sure to set CMAKE_TOOLCHAIN_FILE to
   point to vcpkg.cmake when configuring HDF5)

MacOS:

 - brew install aws-c-s3

========================================================================
III. Building the aws-c-s3 library from source
========================================================================

If you are unable to obtain the aws-c-s3 library from a package manager
or other distribution method, the library may also be built from source
code. The following instructions have been reproduced and slightly
modified from the aws-c-s3 repository and may be out of date; be sure to
check https://github.com/awslabs/aws-c-s3 for the latest build
instructions.

  1. Obtain the source code for aws-c-s3 and its dependencies

     Note that this build process intends for aws-c-s3 and all its
     dependencies to be cloned into the same top-level directory.

     Only if building on Linux:
       - git clone https://github.com/aws/aws-lc.git
       - git clone https://github.com/aws/s2n-tls.git

     - git clone https://github.com/awslabs/aws-c-common.git
     - git clone https://github.com/awslabs/aws-checksums.git
     - git clone https://github.com/awslabs/aws-c-cal.git
     - git clone https://github.com/awslabs/aws-c-io.git
     - git clone https://github.com/awslabs/aws-c-compression.git
     - git clone https://github.com/awslabs/aws-c-http.git
     - git clone https://github.com/awslabs/aws-c-sdkutils.git
     - git clone https://github.com/awslabs/aws-c-auth.git
     - git clone https://github.com/awslabs/aws-c-s3.git

  2. Build aws-c-s3 and its dependencies

     Note that this build process intends for <install-path> to be the
     same directory for all commands. '--parallel <num-jobs>' is used
     to speed up the build process by building with <num-jobs> concurrent
     processes, and can be omitted if desired.

     Only if building on Linux:
       Build aws-lc:
         - cmake -S aws-lc -B aws-lc/build -DCMAKE_INSTALL_PREFIX=<install-path> -DBUILD_SHARED_LIBS=1
         - cmake --build aws-lc/build --target install --parallel <num-jobs>

       Build s2n-tls:
         - cmake -S s2n-tls -B s2n-tls/build -DCMAKE_INSTALL_PREFIX=<install-path> -DCMAKE_PREFIX_PATH=<install-path> -DBUILD_SHARED_LIBS=1
         - cmake --build s2n-tls/build --target install --parallel <num-jobs>

     Build aws-c-common:
       - cmake -S aws-c-common -B aws-c-common/build -DCMAKE_INSTALL_PREFIX=<install-path> -DBUILD_SHARED_LIBS=1
       - cmake --build aws-c-common/build --target install --parallel <num-jobs>

     Build aws-checksums:
       - cmake -S aws-checksums -B aws-checksums/build -DCMAKE_INSTALL_PREFIX=<install-path> -DCMAKE_PREFIX_PATH=<install-path> -DBUILD_SHARED_LIBS=1
       - cmake --build aws-checksums/build --target install --parallel <num-jobs>

     Build aws-c-cal:
       - cmake -S aws-c-cal -B aws-c-cal/build -DCMAKE_INSTALL_PREFIX=<install-path> -DCMAKE_PREFIX_PATH=<install-path> -DBUILD_SHARED_LIBS=1
       - cmake --build aws-c-cal/build --target install --parallel <num-jobs>

     Build aws-c-io:
       - cmake -S aws-c-io -B aws-c-io/build -DCMAKE_INSTALL_PREFIX=<install-path> -DCMAKE_PREFIX_PATH=<install-path> -DBUILD_SHARED_LIBS=1
       - cmake --build aws-c-io/build --target install --parallel <num-jobs>

     Build aws-c-compression:
       - cmake -S aws-c-compression -B aws-c-compression/build -DCMAKE_INSTALL_PREFIX=<install-path> -DCMAKE_PREFIX_PATH=<install-path> -DBUILD_SHARED_LIBS=1
       - cmake --build aws-c-compression/build --target install --parallel <num-jobs>

     Build aws-c-http:
       - cmake -S aws-c-http -B aws-c-http/build -DCMAKE_INSTALL_PREFIX=<install-path> -DCMAKE_PREFIX_PATH=<install-path> -DBUILD_SHARED_LIBS=1
       - cmake --build aws-c-http/build --target install --parallel <num-jobs>

     Build aws-c-sdkutils:
       - cmake -S aws-c-sdkutils -B aws-c-sdkutils/build -DCMAKE_INSTALL_PREFIX=<install-path> -DCMAKE_PREFIX_PATH=<install-path> -DBUILD_SHARED_LIBS=1
       - cmake --build aws-c-sdkutils/build --target install --parallel <num-jobs>

     Build aws-c-auth:
       - cmake -S aws-c-auth -B aws-c-auth/build -DCMAKE_INSTALL_PREFIX=<install-path> -DCMAKE_PREFIX_PATH=<install-path> -DBUILD_SHARED_LIBS=1
       - cmake --build aws-c-auth/build --target install --parallel <num-jobs>

     Build aws-c-s3:
       - cmake -S aws-c-s3 -B aws-c-s3/build -DCMAKE_INSTALL_PREFIX=<install-path> -DCMAKE_PREFIX_PATH=<install-path> -DBUILD_SHARED_LIBS=1
       - cmake --build aws-c-s3/build --target install --parallel <num-jobs>

Note:
  As shown in the above commands, it is suggested to pass the CMake
  option BUILD_SHARED_LIBS=1 when building aws-c-s3 and each of its
  dependencies if shared libraries will be enabled when building HDF5.
  Otherwise, HDF5's CMake configuration logic might find and use a
  libcrypto library from the system, which could result in linking errors
  depending on the version of that library. If only static libraries will
  be enabled when building HDF5, this option can be omitted.

Note:
  The instructions above install aws-c-s3 and all of its dependencies
  into the same directory with the use of CMAKE_INSTALL_PREFIX. This is
  the suggested way of building aws-c-s3 as it simplifies the process of
  locating aws-c-s3 and its dependencies with CMAKE_PREFIX_PATH, both
  when building aws-c-s3 and HDF5, especially if installing aws-c-s3 to
  a non-standard location.

========================================================================
IV. Building HDF5 with the ROS3 VFD enabled
========================================================================

HDF5 uses CMake's find_package() mechanism to locate the aws-c-library,
so as long as it is installed to a standard location, the CMake option
HDF5_ENABLE_ROS3_VFD=ON simply needs to be passed when configuring HDF5
in order to enable the ROS3 VFD, as in:

cmake -DHDF5_ENABLE_ROS3_VFD=ON ..

If the aws-c-s3 library is installed to a non-standard location, the
environment variable CMAKE_PREFIX_PATH should be set to that path when
configuring HDF5, as in:

CMAKE_PREFIX_PATH=<install-path> cmake -DHDF5_ENABLE_ROS3_VFD=ON ..

Refer to release_docs/INSTALL_CMake.txt for more general instructions on
building HDF5 with CMake.

========================================================================
V. Testing the ROS3 VFD
========================================================================

The ROS3 VFD is tested using a combination of docker, s3proxy and the
AWS CLI. If the 'docker' and 'aws' executables are available in standard
locations which CMake can find and the docker daemon is running, the
ROS3 VFD tests can be enabled by passing the CMake option
HDF5_ENABLE_ROS3_VFD_DOCKER_PROXY=ON, as in:

cmake -DHDF5_ENABLE_ROS3_VFD=ON -DHDF5_ENABLE_ROS3_VFD_DOCKER_PROXY=ON ..

If everything configured correctly, the ROS3 VFD tests can be run with:

ctest -R "S3TEST" -VV .

These tests are separated into different groups based on the particular
functionality being tested. For each group of tests, the CMake logic
will bring up s3proxy, create a testing bucket and populate that bucket
with any testing files needed, run all of the tests for the testing
group and then bring down s3proxy.
