IPython Widgets Example¶

Purpose¶

Demonstrate common uses of the ipysliderule module

Load necessary packages¶

In [1]:
import warnings
warnings.filterwarnings('ignore') # turn off warnings for demo

from sliderule import icesat2, ipysliderule, io, sliderule
import ipywidgets as widgets
import geopandas
import logging

%load_ext autoreload
%autoreload 2

Initiate SlideRule API¶

  • Sets the URL for accessing the SlideRule service
  • Builds a table of servers available for processing data
In [2]:
# set the url for the sliderule service
# set the logging level
icesat2.init("slideruleearth.io", organization="bathy", loglevel=logging.WARNING)

Set options for making science data processing requests to SlideRule¶

SlideRule follows a streamlined version of the ATL06 land ice height algorithm.

SlideRule also can use different sources for photon classification before calculating the average segment height.
This is useful for example, in cases where there may be a vegetated canopy affecting the spread of the photon returns.

  • ATL03 photon confidence values, based on algorithm-specific classification types for land, ocean, sea-ice, land-ice, or inland water
  • ATL08 Land and Vegetation Height product photon classification
  • Experimental YAPC (Yet Another Photon Classification) photon-density-based classification
In [3]:
# display widgets for setting SlideRule parameters
SRwidgets = ipysliderule.widgets()
widgets.VBox([
    SRwidgets.classification,
    SRwidgets.surface_type,
    SRwidgets.confidence,
    SRwidgets.quality,
    SRwidgets.land_class,
    SRwidgets.yapc_knn,
    SRwidgets.yapc_win_h,
    SRwidgets.yapc_win_x,
    SRwidgets.yapc_min_ph,
    SRwidgets.yapc_weight,
    SRwidgets.length,
    SRwidgets.step,
    SRwidgets.iteration,
    SRwidgets.spread,
    SRwidgets.count,
    SRwidgets.window,
    SRwidgets.sigma,
])
Out[3]:
VBox(children=(SelectMultiple(description='Classification:', index=(0, 2), options=('atl03', 'quality', 'atl08…

Interactive Mapping with Leaflet¶

Interactive maps within the SlideRule python API are built upon ipyleaflet.

Leaflet Basemaps and Layers¶

There are 3 projections available within SlideRule for mapping (Global, North and South). There are also contextual layers available for each projection.

Global (Web Mercator, EPSG:3857) North (Alaska Polar Stereographic, EPSG:5936) South (Antarctic Polar Stereographic, EPSG:3031)
  • USGS 3DEP Hillshade
  • ASTER GDEM Hillshade
  • ESRI Imagery
  • Global Land Ice Measurements from Space (GLIMS)
  • Randolph Glacier Inventory (RGI)
  • ESRI Imagery
  • ArcticDEM
  • Landsat Image Mosaic of Antarctica (LIMA)
  • MODIS Mosaic of Antarctica (MOA)
  • Radarsat Antarctic Mapping Project (RAMP)
  • Reference Elevation Model of Antarctica (REMA)

In addition, most xyzservice providers can be added as contextual layers to the global Web Mercator maps

In [4]:
widgets.VBox([
    SRwidgets.projection,
    SRwidgets.layers,
    SRwidgets.raster_functions
])
Out[4]:
VBox(children=(Dropdown(description='Projection:', options=('Global', 'North', 'South'), tooltip='Projection: …

Select regions of interest for submitting to SlideRule¶

Here, we create polygons or bounding boxes for our regions of interest.
This map is also our viewer for inspecting our SlideRule ICESat-2 data returns.

In [5]:
# create ipyleaflet map in specified projection
m = ipysliderule.leaflet(SRwidgets.projection.value)
# install click handler callback
m.add_selected_callback(SRwidgets.atl06_click_handler)
m.map
Out[5]:
Map(center=[39, -108], controls=(ZoomControl(options=['position', 'zoom_in_text', 'zoom_in_title', 'zoom_out_t…
In [6]:
m.add_layer(
    layers=SRwidgets.layers.value,
    rendering_rule=SRwidgets.rendering_rule
)

Build and transmit requests to SlideRule¶

  • SlideRule will query the NASA Common Metadata Repository (CMR) for ATL03 data within our region of interest
  • When using the icesat2 asset, the ICESat-2 ATL03 data are then accessed from the NSIDC AWS s3 bucket in us-west-2
  • The ATL03 granules is spatially subset within SlideRule to our exact region of interest
  • SlideRule then uses our specified parameters to calculate average height segments from the ATL03 data in parallel
  • The completed data is streamed concurrently back and combined into a geopandas GeoDataFrame within the Python client
In [7]:
%%time
# build sliderule parameters using latest values from widget
parms = SRwidgets.build_atl06()

# clear existing geodataframe results
elevations = [sliderule.emptyframe()]

# for each region of interest
for poly in m.regions:
    # add polygon from map to sliderule parameters
    parms["poly"] = poly 
    # make the request to the SlideRule (ATL06-SR) endpoint
    # and pass it the request parameters to request ATL06 Data
    elevations.append(icesat2.atl06p(parms))
gdf = geopandas.pd.concat(elevations)

CPU times: user 1.57 s, sys: 15.8 ms, total: 1.59 s
Wall time: 12.8 s

Review GeoDataFrame output¶

Can inspect the columns, number of returns and returns at the top of the GeoDataFrame.

See the ICESat-2 documentation for descriptions of each column

In [8]:
print(f'Returned {gdf.shape[0]} records')
gdf.head()

Returned 36865 records
Out[8]:
geometry segment_id n_fit_photons h_mean region dh_fit_dx spot w_surface_window_final gt y_atc rms_misfit h_sigma pflags x_atc cycle rgt
2018-10-20 10:41:00.438582528 POINT (-109.37008 39.26758) 783942.0 27.0 1951.971253 6.0 0.267235 2.0 11.062516 50.0 -2410.635498 1.662260 0.326687 0.0 15702057.0 1.0 333.0
2018-10-20 10:41:00.441383680 POINT (-109.37010 39.26740) 783943.0 34.0 1958.359868 6.0 0.309262 2.0 16.191092 50.0 -2410.580811 1.754874 0.301020 0.0 15702077.0 1.0 333.0
2018-10-20 10:41:00.444191232 POINT (-109.37012 39.26722) 783944.0 27.0 1962.070003 6.0 -0.010137 2.0 3.000000 50.0 -2410.538086 0.421994 0.082993 0.0 15702097.0 1.0 333.0
2018-10-20 10:41:00.447001088 POINT (-109.37015 39.26704) 783945.0 24.0 1962.295167 6.0 -0.012544 2.0 12.143318 50.0 -2410.515625 1.047392 0.215051 0.0 15702117.0 1.0 333.0
2018-10-20 10:41:00.449813248 POINT (-109.37017 39.26686) 783946.0 24.0 1962.453194 6.0 0.009096 2.0 9.107489 50.0 -2410.500000 1.270026 0.260924 0.0 15702137.0 1.0 333.0

Add GeoDataFrame to map¶

For stability of the leaflet map, SlideRule will as a default limit the plot to have up to 10000 points from the GeoDataFrame

GeoDataFrames can be plotted in any available matplotlib colormap

In [9]:
widgets.VBox([
    SRwidgets.variable,
    SRwidgets.cmap,
    SRwidgets.reverse,
])
Out[9]:
VBox(children=(Dropdown(description='Variable:', options=('h_mean', 'h_sigma', 'dh_fit_dx', 'dh_fit_dy', 'rms_…
In [10]:
%matplotlib inline
# ATL06-SR fields for hover tooltip
fields = m.default_atl06_fields()
m.GeoData(gdf, column_name=SRwidgets.variable.value, cmap=SRwidgets.colormap,
    max_plot_points=10000, tooltip=True, colorbar=True, fields=fields)

Deprecated. Will be removed in a future release

Create plots for a single track¶

  • cycles: Will plot all available cycles of data returned by SlideRule for a single RGT and ground track
  • scatter: Will plot data returned by SlideRule for a single RGT, ground track and cycle
  • (to select a track from the leaflet plot above, click on one of the plotted elevations and the RGT and Cycle will automatically get populated below)

The cycles plots should only be used in regions with repeat Reference Ground Track (RGT) pointing

In [11]:
widgets.VBox([
    SRwidgets.plot_kind,
    SRwidgets.rgt,
    SRwidgets.ground_track,
    SRwidgets.cycle,
])
Out[11]:
VBox(children=(Dropdown(description='Plot Kind:', index=1, options=('cycles', 'scatter'), tooltip='Plot Kind: …
In [13]:
%matplotlib widget
# default is to skip cycles with significant off-pointing
SRwidgets.plot(gdf, kind=SRwidgets.plot_kind.value, cycle_start=3,
    legend=True, legend_frameon=False)
Figure
No description has been provided for this image

Save GeoDataFrame to output file¶

  • pytables HDF5: easily read back as a Geopandas GeoDataFrame
  • netCDF: interoperable with other programs
In [14]:
display(SRwidgets.filesaver)

Text(value='ATL06-SR_20240404200050_006.h5')
In [15]:
# append sliderule api version to attributes
version = sliderule.get_version()
parms['version'] = version['icesat2']['version']
parms['commit'] = version['icesat2']['commit']
# save to file in format (HDF5 or netCDF)
io.to_file(gdf, SRwidgets.file,
    format=SRwidgets.format,
    driver='pytables',
    parameters=parms,
    regions=m.regions,
    verbose=True)

Read GeoDataFrame from input file¶

  • pytables HDF5
  • netCDF
In [16]:
display(SRwidgets.fileloader)

Text(value='')
In [17]:
# read from file in format (HDF5 or netCDF)
gdf,parms,regions = io.from_file(SRwidgets.file,
    format=SRwidgets.format,
    driver='pytables',
    return_parameters=True,
    return_regions=True)

Review GeoDataFrame input from file¶

In [18]:
gdf.head()
Out[18]:
segment_id n_fit_photons h_mean region dh_fit_dx spot w_surface_window_final gt y_atc rms_misfit h_sigma pflags x_atc cycle rgt geometry
2018-10-20 10:41:00.438582528 783942.0 27.0 1951.971253 6.0 0.267235 2.0 11.062516 50.0 -2410.635498 1.662260 0.326687 0.0 15702057.0 1.0 333.0 POINT (-109.37008 39.26758)
2018-10-20 10:41:00.441383680 783943.0 34.0 1958.359868 6.0 0.309262 2.0 16.191092 50.0 -2410.580811 1.754874 0.301020 0.0 15702077.0 1.0 333.0 POINT (-109.37010 39.26740)
2018-10-20 10:41:00.444191232 783944.0 27.0 1962.070003 6.0 -0.010137 2.0 3.000000 50.0 -2410.538086 0.421994 0.082993 0.0 15702097.0 1.0 333.0 POINT (-109.37012 39.26722)
2018-10-20 10:41:00.447001088 783945.0 24.0 1962.295167 6.0 -0.012544 2.0 12.143318 50.0 -2410.515625 1.047392 0.215051 0.0 15702117.0 1.0 333.0 POINT (-109.37015 39.26704)
2018-10-20 10:41:00.449813248 783946.0 24.0 1962.453194 6.0 0.009096 2.0 9.107489 50.0 -2410.500000 1.270026 0.260924 0.0 15702137.0 1.0 333.0 POINT (-109.37017 39.26686)

Set parameters and add saved regions to map¶

In [19]:
SRwidgets.set_values(parms)
m.add_region(regions)
Out[19]:
<sliderule.ipysliderule.leaflet at 0xffffabaa50a0>
In [ ]: