org.eclipse.january.dataset

## Class LazyDataset

- java.lang.Object
- - org.eclipse.january.dataset.LazyDatasetBase
  - - org.eclipse.january.dataset.LazyDataset

- All Implemented Interfaces:
    Serializable, Cloneable, ILazyDataset, IMetadataProvider, INameable
  Direct Known Subclasses:
    LazyDynamicDataset

      public class LazyDataset
      extends LazyDatasetBase
      implements Serializable, Cloneable

  See Also:
    Serialized Form

- - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `protected int[]` | `begSlice`
    `protected int[]` | `delSlice`
    `protected int` | `dtype`
    `protected int` | `isize`
    `protected ILazyLoader` | `loader`
    `protected int[]` | `map`
    `protected Map<Class<? extends MetadataType>,List<MetadataType>>` | `oMetadata`
    `protected int[]` | `oShape`
    `protected int[]` | `padding`
    `protected long` | `size`
    `protected int[]` | `sShape`

    - ### Fields inherited from class org.eclipse.january.dataset.LazyDatasetBase

      `catchExceptions, logger, metadata, name, shape`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `LazyDataset(String name, int dtype, int[] shape, ILazyLoader loader)` Create a lazy dataset
    `LazyDataset(String name, int dtype, int elements, int[] shape, ILazyLoader loader)` Create a lazy dataset

  - ### Method Summary

    Methods
    Modifier and Type | Method and Description
    --- | ---
    `protected SliceND` | `calcTrueSlice(SliceND slice)` Calculate absolute slice
    `LazyDataset` | `clone()` Clone dataset
    `static LazyDataset` | `createLazyDataset(Dataset dataset)` Create a lazy dataset based on in-memory data (handy for testing)
    `protected SliceND` | `createSlice(int[] nstart, int[] nstop, int[] nstep)`
    `boolean` | `equals(Object obj)`
    `int` | `getDType()` Can return -1 for unknown
    `int` | `getElementsPerItem()` Can return -1 for unknown
    `static int` | `getMaxSliceLength(ILazyDataset lazySet, int dimension)` Gets the maximum size of a slice of a dataset in a given dimension which should normally fit in memory.
    `int` | `getSize()` The size of the dataset is the number of items in the array
    `Dataset` | `getSlice(IMonitor monitor, int[] start, int[] stop, int[] step)` Get a slice of the dataset.
    `Dataset` | `getSlice(IMonitor monitor, Slice... slice)` Get a slice of the dataset.
    `Dataset` | `getSlice(IMonitor monitor, SliceND slice)` Get a slice of the dataset.
    `Dataset` | `getSlice(int[] start, int[] stop, int[] step)` Get a slice of the dataset.
    `Dataset` | `getSlice(Slice... slice)` Get a slice of the dataset.
    `Dataset` | `getSlice(SliceND slice)` Get a slice of the dataset.
    `LazyDataset` | `getSliceView(int[] start, int[] stop, int[] step)` Get a slice of the dataset.
    `LazyDataset` | `getSliceView(Slice... slice)` Get a slice of the dataset.
    `LazyDataset` | `getSliceView(SliceND slice)` Get a slice of the dataset.
    `LazyDataset` | `getTransposedView(int... axes)` Permute copy of dataset's axes so that given order is old order:
    `int` | `hashCode()`
    `void` | `setShape(int... shape)` Set a compatible shape for dataset.
    `LazyDataset` | `squeezeEnds()` Remove dimensions of 1 from ends of shape of the dataset
    `String` | `toString()`

    - ### Methods inherited from class org.eclipse.january.dataset.LazyDatasetBase

      `addMetadata, checkPermutatedAxes, clearMetadata, copyMetadata, copyMetadata, createFromSerializable, dirtyMetadata, findMetadataTypeSubInterfaces, getElementClass, getErrorMetadata, getErrors, getFirstMetadata, getMetadata, getMetadata, getMetadataMap, getName, getRank, getShape, hasErrors, reshapeMetadata, restoreMetadata, setDirty, setErrors, setMetadata, setName, sliceMetadata, transposeMetadata`

    - ### Methods inherited from class java.lang.Object

      `finalize, getClass, notify, notifyAll, wait, wait, wait`

- - ### Field Detail

    - #### oMetadata

          protected Map<Class<? extends MetadataType>,List<MetadataType>> oMetadata

    - #### oShape

          protected int[] oShape

    - #### size

          protected long size

    - #### dtype

          protected int dtype

    - #### isize

          protected int isize

    - #### loader

          protected ILazyLoader loader

    - #### begSlice

          protected int[] begSlice

    - #### delSlice

          protected int[] delSlice

    - #### sShape

          protected int[] sShape

      Since:
        2.2

    - #### padding

          protected int[] padding

      Since:
        2.2

    - #### map

          protected int[] map

  - ### Constructor Detail

    - #### LazyDataset

          public LazyDataset(String name,
                     int dtype,
                     int elements,
                     int[] shape,
                     ILazyLoader loader)

      Create a lazy dataset
      Parameters:
        `name` -
        `dtype` - dataset type
        `elements` -
        `shape` -
        `loader` -

    - #### LazyDataset

          public LazyDataset(String name,
                     int dtype,
                     int[] shape,
                     ILazyLoader loader)

      Create a lazy dataset
      Parameters:
        `name` -
        `dtype` - dataset type
        `shape` -
        `loader` -

  - ### Method Detail

    - #### createLazyDataset

          public static LazyDataset createLazyDataset(Dataset dataset)

      Create a lazy dataset based on in-memory data (handy for testing)
      Parameters:
        `dataset` -

    - #### getDType

          public int getDType()

      Can return -1 for unknown
      Specified by:
        `getDType` in class `LazyDatasetBase`
      Returns:
        type of dataset item

    - #### getElementsPerItem

          public int getElementsPerItem()

      Can return -1 for unknown
      Specified by:
        `getElementsPerItem` in interface `ILazyDataset`
      Returns:
        Number of elements per item

    - #### getSize

          public int getSize()

      Description copied from interface: `ILazyDataset`
      The size of the dataset is the number of items in the array
      Specified by:
        `getSize` in interface `ILazyDataset`
      Returns:
        number of data items

    - #### toString

          public String toString()

      Overrides:
        `toString` in class `Object`

    - #### hashCode

          public int hashCode()

      Overrides:
        `hashCode` in class `LazyDatasetBase`

    - #### equals

          public boolean equals(Object obj)

      Overrides:
        `equals` in class `LazyDatasetBase`

    - #### clone

          public LazyDataset clone()

      Description copied from interface: `ILazyDataset`
      Clone dataset
      Specified by:
        `clone` in interface `ILazyDataset`
      Overrides:
        `clone` in class `LazyDatasetBase`
      Returns:
        a (shallow) copy of dataset

    - #### setShape

          public void setShape(int... shape)

      Description copied from interface: `ILazyDataset`
      Set a compatible shape for dataset. A shape is compatible if it has the capacity to contain the same number of items
      Specified by:
        `setShape` in interface `ILazyDataset`

    - #### squeezeEnds

          public LazyDataset squeezeEnds()

      Description copied from interface: `ILazyDataset`
      Remove dimensions of 1 from ends of shape of the dataset
      Specified by:
        `squeezeEnds` in interface `ILazyDataset`

    - #### getSlice

          public Dataset getSlice(int[] start,
                         int[] stop,
                         int[] step)
                           throws DatasetException

      Description copied from interface: `ILazyDataset`
      Get a slice of the dataset. The returned dataset is a copied selection of items
      Specified by:
        `getSlice` in interface `ILazyDataset`
      Parameters:
        `start` - specifies the starting indexes (can be null for origin)
        `stop` - specifies the stopping indexes (can be null for end)
        `step` - specifies the steps in the slice (can be null for unit steps)
      Returns:
        The dataset of the sliced data
      Throws:
        `DatasetException`

    - #### getSlice

          public Dataset getSlice(Slice... slice)
                           throws DatasetException

      Description copied from interface: `ILazyDataset`
      Get a slice of the dataset. The returned dataset is a copied selection of items
      Specified by:
        `getSlice` in interface `ILazyDataset`
      Parameters:
        `slice` - an array of slice objects (the array can be null or contain nulls)
      Returns:
        The dataset of the sliced data
      Throws:
        `DatasetException`

    - #### getSlice

          public Dataset getSlice(SliceND slice)
                           throws DatasetException

      Description copied from interface: `ILazyDataset`
      Get a slice of the dataset. The returned dataset is a copied selection of items
      Specified by:
        `getSlice` in interface `ILazyDataset`
      Parameters:
        `slice` - an n-D slice
      Returns:
        The dataset of the sliced data
      Throws:
        `DatasetException`

    - #### getSlice

          public Dataset getSlice(IMonitor monitor,
                         Slice... slice)
                           throws DatasetException

      Description copied from interface: `ILazyDataset`
      Get a slice of the dataset. The returned dataset is a copied selection of items
      Specified by:
        `getSlice` in interface `ILazyDataset`
        `slice` - an array of slice objects (the array can be null or contain nulls)
      Returns:
        The dataset of the sliced data
      Throws:
        `DatasetException`

    - #### getSliceView

          public LazyDataset getSliceView(Slice... slice)

      Description copied from interface: `ILazyDataset`
      Get a slice of the dataset. The returned lazy dataset is a view on a selection of items
      Specified by:
        `getSliceView` in interface `ILazyDataset`
      Parameters:
        `slice` - an array of slice objects (the array can be null or contain nulls)
      Returns:
        The sliced view of a lazy dataset

    - #### getSliceView

          public LazyDataset getSliceView(int[] start,
                                 int[] stop,
                                 int[] step)

      Description copied from interface: `ILazyDataset`
      Get a slice of the dataset. The returned lazy dataset is a view on a selection of items
      Specified by:
        `getSliceView` in interface `ILazyDataset`
      Parameters:
        `start` - specifies the starting indexes (can be null for origin)
        `stop` - specifies the stopping indexes (can be null for end)
        `step` - specifies the steps in the slice (can be null for unit steps)
      Returns:
        The sliced view of a lazy dataset

    - #### getSliceView

          public LazyDataset getSliceView(SliceND slice)

      Description copied from interface: `ILazyDataset`
      Get a slice of the dataset. The returned lazy dataset is a view on a selection of items
      Specified by:
        `getSliceView` in interface `ILazyDataset`
      Parameters:
        `slice` - an n-D slice
      Returns:
        The sliced view of a lazy dataset

    - #### getSlice

          public Dataset getSlice(IMonitor monitor,
                         int[] start,
                         int[] stop,
                         int[] step)
                           throws DatasetException

      Description copied from interface: `ILazyDataset`
      Get a slice of the dataset. The returned dataset is a copied selection of items
      Specified by:
        `getSlice` in interface `ILazyDataset`
        `start` - specifies the starting indexes (can be null for origin)
        `stop` - specifies the stopping indexes (can be null for end)
        `step` - specifies the steps in the slice (can be null for unit steps)
      Returns:
        The dataset of the sliced data
      Throws:
        `DatasetException`

    - #### getSlice

          public Dataset getSlice(IMonitor monitor,
                         SliceND slice)
                           throws DatasetException

      Description copied from interface: `ILazyDataset`
      Get a slice of the dataset. The returned dataset is a copied selection of items
      Specified by:
        `getSlice` in interface `ILazyDataset`
        `slice` - an n-D slice
      Returns:
        The dataset of the sliced data
      Throws:
        `DatasetException`

    - #### getTransposedView

          public LazyDataset getTransposedView(int... axes)

      Description copied from interface: `ILazyDataset`
      Permute copy of dataset's axes so that given order is old order:

          axisPerm = (p(0), p(1),...) => newdata(n(0), n(1),...) = olddata(o(0), o(1), ...)
            such that n(i) = o(p(i)) for all i

      I.e. for a 3D dataset (1,0,2) implies the new dataset has its 1st dimension running along the old dataset's 2nd dimension and the new 2nd is the old 1st. The 3rd dimension is left unchanged.
      Specified by:
        `getTransposedView` in interface `ILazyDataset`
      Parameters:
        `axes` - if zero length then axes order reversed
      Returns:
        remapped view of data

    - #### calcTrueSlice

          protected final SliceND calcTrueSlice(SliceND slice)

      Calculate absolute slice
      Parameters:
        `slice` -
      Returns:
        true slice or null if zero-sized

    - #### createSlice

          protected SliceND createSlice(int[] nstart,
                            int[] nstop,
                            int[] nstep)

    - #### getMaxSliceLength

          public static int getMaxSliceLength(ILazyDataset lazySet,
                              int dimension)

      Gets the maximum size of a slice of a dataset in a given dimension which should normally fit in memory. Note that it might be possible to get more in memory, this is a conservative estimate and seems to almost always work at the size returned; providing Xmx is less than the physical memory. To get more in memory increase -Xmx setting or use an expression which calls a rolling function (like rmean) instead of slicing directly to memory.
      Parameters:
        `lazySet` -
        `dimension` -
      Returns:
        maximum size of dimension that can be sliced.

Copyright © 2014–2019 Eclipse Foundation. All rights reserved.