org.eclipse.january.dataset

## Class LazyDataset

- java.lang.Object
- - org.eclipse.january.dataset.LazyDatasetBase
  - - org.eclipse.january.dataset.LazyDataset

- All Implemented Interfaces:
    Serializable, Cloneable, ILazyDataset, IMetadataProvider, INameable
  Direct Known Subclasses:
    LazyDynamicDataset

      public class LazyDataset
      extends LazyDatasetBase
      implements Serializable, Cloneable

  See Also:
    Serialized Form

- - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `protected int[]` | `begSlice`
    `protected int[]` | `delSlice`
    `protected int` | `isize`
    `protected ILazyLoader` | `loader`
    `protected int[]` | `map`
    `protected Map<Class<? extends MetadataType>,List<MetadataType>>` | `oMetadata`
    `protected int[]` | `oShape`
    `protected int[]` | `padding`
    `protected long` | `size`
    `protected int[]` | `sShape`

    - ### Fields inherited from class org.eclipse.january.dataset.LazyDatasetBase

      `metadata, name, shape`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `LazyDataset(ILazyLoader loader, String name, Class<? extends Dataset> clazz, int... shape)` Create a lazy dataset
    `LazyDataset(ILazyLoader loader, String name, int elements, Class<? extends Dataset> clazz, int... shape)` Create a lazy dataset
    `LazyDataset(String name, int dtype, int[] shape, ILazyLoader loader)` Deprecated. Use `LazyDataset(ILazyLoader, String, int, Class, int[])`
    `LazyDataset(String name, int dtype, int elements, int[] shape, ILazyLoader loader)` Deprecated. Use `LazyDataset(ILazyLoader, String, int, Class, int[])`

  - ### Method Summary

    All Methods Static Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `protected SliceND` | `calcTrueSlice(SliceND slice)` Calculate absolute slice
    `LazyDataset` | `clone()` Clone dataset
    `static LazyDataset` | `createLazyDataset(Dataset dataset)` Create a lazy dataset based on in-memory data (handy for testing)
    `protected SliceND` | `createSlice(int[] nstart, int[] nstop, int[] nstep)`
    `boolean` | `equals(Object obj)`
    `int` | `getDType()` Can return -1 when dataset interface is not defined
    `Class<?>` | `getElementClass()`
    `int` | `getElementsPerItem()` Can return -1 for unknown
    `Class<? extends Dataset>` | `getInterface()`
    `static int` | `getMaxSliceLength(ILazyDataset lazySet, int dimension)` Gets the maximum size of a slice of a dataset in a given dimension which should normally fit in memory.
    `int` | `getSize()` The size of the dataset is the number of items in the array
    `Dataset` | `getSlice(IMonitor monitor, int[] start, int[] stop, int[] step)` Get a slice of the dataset.
    `Dataset` | `getSlice(IMonitor monitor, Slice... slice)` Get a slice of the dataset.
    `Dataset` | `getSlice(IMonitor monitor, SliceND slice)` Get a slice of the dataset.
    `Dataset` | `getSlice(int[] start, int[] stop, int[] step)` Get a slice of the dataset.
    `Dataset` | `getSlice(Slice... slice)` Get a slice of the dataset.
    `Dataset` | `getSlice(SliceND slice)` Get a slice of the dataset.
    `LazyDataset` | `getSliceView(int[] start, int[] stop, int[] step)` Get a slice of the dataset.
    `LazyDataset` | `getSliceView(Slice... slice)` Get a slice of the dataset.
    `LazyDataset` | `getSliceView(SliceND slice)` Get a slice of the dataset.
    `LazyDataset` | `getTransposedView(int... axes)` Permute copy of dataset's axes so that given order is old order:
    `int` | `hashCode()`
    `protected Dataset` | `internalGetSlice(IMonitor monitor, SliceND slice)`
    `protected LazyDataset` | `internalGetSliceView(SliceND slice)`
    `void` | `setInterface(Class<? extends Dataset> clazz)` Set interface
    `void` | `setShape(int... shape)` Set a compatible shape for dataset.
    `LazyDataset` | `squeezeEnds()` Remove dimensions of 1 from ends of shape of the dataset
    `String` | `toString()`

    - ### Methods inherited from class org.eclipse.january.dataset.LazyDatasetBase

      `addMetadata, checkPermutatedAxes, checkSliceND, clearMetadata, copyMetadata, copyMetadata, createFromSerializable, dirtyMetadata, findMetadataTypeSubInterfaces, getErrorMetadata, getErrors, getFirstMetadata, getMetadata, getMetadata, getMetadataMap, getName, getRank, getShape, hasErrors, reshapeMetadata, restoreMetadata, setDirty, setErrors, setMetadata, setName, sliceMetadata, transposeMetadata`

    - ### Methods inherited from class java.lang.Object

      `finalize, getClass, notify, notifyAll, wait, wait, wait`

- - ### Field Detail

    - #### oMetadata

          protected Map<Class<? extends MetadataType>,List<MetadataType>> oMetadata

    - #### oShape

          protected int[] oShape

    - #### size

          protected long size

    - #### isize

          protected int isize

    - #### loader

          protected ILazyLoader loader

    - #### begSlice

          protected int[] begSlice

    - #### delSlice

          protected int[] delSlice

    - #### sShape

          protected int[] sShape

      Since:
        2.2

    - #### padding

          protected int[] padding

      Since:
        2.2

    - #### map

          protected int[] map

  - ### Constructor Detail

    - #### LazyDataset

          public LazyDataset(ILazyLoader loader,
                             String name,
                             int elements,
                             Class<? extends Dataset> clazz,
                             int... shape)

      Create a lazy dataset
      Parameters:
        `loader` - lazy loader
        `name` - of dataset
        `elements` - item size
        `clazz` - dataset sub-interface
        `shape` - dataset shape
      Since:
        2.3

    - #### LazyDataset

          public LazyDataset(ILazyLoader loader,
                             String name,
                             Class<? extends Dataset> clazz,
                             int... shape)

      Create a lazy dataset
      Parameters:
        `loader` - lazy loader
        `name` - of dataset
        `clazz` - dataset sub-interface
        `shape` - dataset shape
      Since:
        2.3

    - #### LazyDataset

          @Deprecated
          public LazyDataset(String name,
                                         int dtype,
                                         int elements,
                                         int[] shape,
                                         ILazyLoader loader)

      Deprecated. Use `LazyDataset(ILazyLoader, String, int, Class, int[])`
      Create a lazy dataset
      Parameters:
        `name` - of dataset
        `dtype` - dataset type
        `elements` - item size
        `shape` - dataset shape
        `loader` - lazy loader

    - #### LazyDataset

          @Deprecated
          public LazyDataset(String name,
                                         int dtype,
                                         int[] shape,
                                         ILazyLoader loader)

      Deprecated. Use `LazyDataset(ILazyLoader, String, int, Class, int[])`
      Create a lazy dataset
      Parameters:
        `name` - of dataset
        `dtype` - dataset type
        `shape` - dataset shape
        `loader` - lazy loader

  - ### Method Detail

    - #### createLazyDataset

          public static LazyDataset createLazyDataset(Dataset dataset)

      Create a lazy dataset based on in-memory data (handy for testing)
      Parameters:
        `dataset` - input
      Returns:
        lazy dataset

    - #### getElementClass

          public Class<?> getElementClass()

      Specified by:
        `getElementClass` in interface `ILazyDataset`
      Returns:
        Boxed class of element. Can be null when interface is not defined

    - #### getDType

          public int getDType()

      Can return -1 when dataset interface is not defined
      Specified by:
        `getDType` in class `LazyDatasetBase`
      Returns:
        type of dataset item

    - #### getInterface

          public Class<? extends Dataset> getInterface()

      Returns:
        dataset interface that supports element class and number of elements. Can be null when undefined
      Since:
        2.3

    - #### setInterface

          public void setInterface(Class<? extends Dataset> clazz)

      Set interface
      Parameters:
        `clazz` - dataset sub-interface
      Since:
        2.3

    - #### getElementsPerItem

          public int getElementsPerItem()

      Can return -1 for unknown
      Specified by:
        `getElementsPerItem` in interface `ILazyDataset`
      Returns:
        Number of elements per item

    - #### getSize

          public int getSize()

      Description copied from interface: `ILazyDataset`
      The size of the dataset is the number of items in the array
      Specified by:
        `getSize` in interface `ILazyDataset`
      Returns:
        number of data items

    - #### toString

          public String toString()

      Overrides:
        `toString` in class `Object`

    - #### hashCode

          public int hashCode()

      Overrides:
        `hashCode` in class `LazyDatasetBase`

    - #### equals

          public boolean equals(Object obj)

      Overrides:
        `equals` in class `LazyDatasetBase`

    - #### clone

          public LazyDataset clone()

      Description copied from interface: `ILazyDataset`
      Clone dataset
      Specified by:
        `clone` in interface `ILazyDataset`
      Overrides:
        `clone` in class `LazyDatasetBase`
      Returns:
        a (shallow) copy of dataset

    - #### setShape

          public void setShape(int... shape)

      Description copied from interface: `ILazyDataset`
      Set a compatible shape for dataset. A shape is compatible if it has the capacity to contain the same number of items
      Specified by:
        `setShape` in interface `ILazyDataset`
      Parameters:
        `shape` - to set

    - #### squeezeEnds

          public LazyDataset squeezeEnds()

      Description copied from interface: `ILazyDataset`
      Remove dimensions of 1 from ends of shape of the dataset
      Specified by:
        `squeezeEnds` in interface `ILazyDataset`
      Returns:
        this

    - #### getSlice

          public Dataset getSlice(int[] start,
                                  int[] stop,
                                  int[] step)
                           throws DatasetException

      Description copied from interface: `ILazyDataset`
      Get a slice of the dataset. The returned dataset is a copied selection of items
      Specified by:
        `getSlice` in interface `ILazyDataset`
      Parameters:
        `start` - specifies the starting indexes (can be null for origin)
        `stop` - specifies the stopping indexes (can be null for end)
        `step` - specifies the steps in the slice (can be null for unit steps)
      Returns:
        The dataset of the sliced data
      Throws:
        `DatasetException` - when cannot retrieve slice of lazy dataset

    - #### getSlice

          public Dataset getSlice(Slice... slice)
                           throws DatasetException

      Description copied from interface: `ILazyDataset`
      Get a slice of the dataset. The returned dataset is a copied selection of items
      Specified by:
        `getSlice` in interface `ILazyDataset`
      Parameters:
        `slice` - an array of slice objects (the array can be null or contain nulls)
      Returns:
        The dataset of the sliced data
      Throws:
        `DatasetException` - when cannot retrieve slice of lazy dataset

    - #### getSlice

          public Dataset getSlice(SliceND slice)
                           throws DatasetException

      Description copied from interface: `ILazyDataset`
      Get a slice of the dataset. The returned dataset is a copied selection of items
      Specified by:
        `getSlice` in interface `ILazyDataset`
      Parameters:
        `slice` - an n-D slice
      Returns:
        The dataset of the sliced data
      Throws:
        `DatasetException` - when cannot retrieve slice of lazy dataset

    - #### getSlice

          public Dataset getSlice(IMonitor monitor,
                                  Slice... slice)
                           throws DatasetException

      Description copied from interface: `ILazyDataset`
      Get a slice of the dataset. The returned dataset is a copied selection of items
      Specified by:
        `getSlice` in interface `ILazyDataset`
      Parameters:
        `monitor` - can be null
        `slice` - an array of slice objects (the array can be null or contain nulls)
      Returns:
        The dataset of the sliced data
      Throws:
        `DatasetException` - when cannot retrieve slice of lazy dataset

    - #### getSliceView

          public LazyDataset getSliceView(Slice... slice)

      Description copied from interface: `ILazyDataset`
      Get a slice of the dataset. The returned lazy dataset is a view on a selection of items
      Specified by:
        `getSliceView` in interface `ILazyDataset`
      Parameters:
        `slice` - an array of slice objects (the array can be null or contain nulls)
      Returns:
        The sliced view of a lazy dataset

    - #### getSliceView

          public LazyDataset getSliceView(int[] start,
                                          int[] stop,
                                          int[] step)

      Description copied from interface: `ILazyDataset`
      Get a slice of the dataset. The returned lazy dataset is a view on a selection of items
      Specified by:
        `getSliceView` in interface `ILazyDataset`
      Parameters:
        `start` - specifies the starting indexes (can be null for origin)
        `stop` - specifies the stopping indexes (can be null for end)
        `step` - specifies the steps in the slice (can be null for unit steps)
      Returns:
        The sliced view of a lazy dataset

    - #### getSliceView

          public LazyDataset getSliceView(SliceND slice)

      Description copied from interface: `ILazyDataset`
      Get a slice of the dataset. The returned lazy dataset is a view on a selection of items
      Specified by:
        `getSliceView` in interface `ILazyDataset`
      Parameters:
        `slice` - an n-D slice
      Returns:
        The sliced view of a lazy dataset

    - #### internalGetSliceView

          protected LazyDataset internalGetSliceView(SliceND slice)

    - #### getSlice

          public Dataset getSlice(IMonitor monitor,
                                  int[] start,
                                  int[] stop,
                                  int[] step)
                           throws DatasetException

      Description copied from interface: `ILazyDataset`
      Get a slice of the dataset. The returned dataset is a copied selection of items
      Specified by:
        `getSlice` in interface `ILazyDataset`
      Parameters:
        `monitor` - can be null
        `start` - specifies the starting indexes (can be null for origin)
        `stop` - specifies the stopping indexes (can be null for end)
        `step` - specifies the steps in the slice (can be null for unit steps)
      Returns:
        The dataset of the sliced data
      Throws:
        `DatasetException` - when cannot retrieve slice of lazy dataset

    - #### getSlice

          public Dataset getSlice(IMonitor monitor,
                                  SliceND slice)
                           throws DatasetException

      Description copied from interface: `ILazyDataset`
      Get a slice of the dataset. The returned dataset is a copied selection of items
      Specified by:
        `getSlice` in interface `ILazyDataset`
      Parameters:
        `monitor` - can be null
        `slice` - an n-D slice
      Returns:
        The dataset of the sliced data
      Throws:
        `DatasetException` - when cannot retrieve slice of lazy dataset

    - #### internalGetSlice

          protected Dataset internalGetSlice(IMonitor monitor,
                                             SliceND slice)
                                      throws DatasetException

      Throws:
        `DatasetException`

    - #### getTransposedView

          public LazyDataset getTransposedView(int... axes)

      Description copied from interface: `ILazyDataset`
      Permute copy of dataset's axes so that given order is old order:

            axisPerm = (p(0), p(1),...) => newdata(n(0), n(1),...) = olddata(o(0), o(1), ...)
            such that n(i) = o(p(i)) for all i

      I.e. for a 3D dataset (1,0,2) implies the new dataset has its 1st dimension running along the old dataset's 2nd dimension and the new 2nd is the old 1st. The 3rd dimension is left unchanged.
      Specified by:
        `getTransposedView` in interface `ILazyDataset`
      Parameters:
        `axes` - if zero length then axes order reversed
      Returns:
        remapped view of data

    - #### calcTrueSlice

          protected final SliceND calcTrueSlice(SliceND slice)

      Calculate absolute slice
      Parameters:
        `slice` - an n-D slice
      Returns:
        true slice or null if zero-sized

    - #### createSlice

          protected SliceND createSlice(int[] nstart,
                                        int[] nstop,
                                        int[] nstep)

    - #### getMaxSliceLength

          public static int getMaxSliceLength(ILazyDataset lazySet,
                                              int dimension)

      Gets the maximum size of a slice of a dataset in a given dimension which should normally fit in memory. Note that it might be possible to get more in memory, this is a conservative estimate and seems to almost always work at the size returned; providing Xmx is less than the physical memory. To get more in memory increase -Xmx setting or use an expression which calls a rolling function (like rmean) instead of slicing directly to memory.
      Parameters:
        `lazySet` - lazy dataset
        `dimension` - to slice along
      Returns:
        maximum size of dimension that can be sliced.

Copyright © 2014–2022 Eclipse Foundation. All rights reserved.