sliderule
The SlideRule Python API sliderule.py
is used to access the services provided by the base SlideRule server. From Python, the module can be imported via:
import sliderule
init
- sliderule.init(url='slideruleearth.io', verbose=False, loglevel=20, organization=0, desired_nodes=None, time_to_live=60, bypass_dns=False, plugins=[])[source]
Initializes the Python client for use with SlideRule, and should be called before other ICESat-2 API calls. This function is a wrapper for a handful of sliderule functions that would otherwise all have to be called in order to initialize the client.
- Parameters
url (str) – the IP address or hostname of the SlideRule service (slidereearth.io by default)
verbose (bool) – whether or not user level log messages received from SlideRule generate a Python log message
loglevel (int) – minimum severity of log message to output
organization (str) – SlideRule provisioning system organization the user belongs to (see sliderule.authenticate for details)
desired_nodes (int) – requested number of processing nodes in the cluster
time_to_live (int) – minimum number of minutes the desired number of nodes should be present in the cluster
bypass_dns (bool) – if true then the ip address for the cluster is retrieved from the provisioning system and used directly
plugins (list) – names of the plugins that need to be available on the server
- Returns
Status of version check
- Return type
bool
Examples
>>> import sliderule >>> sliderule.init()
source
- sliderule.source(api, parm={}, stream=False, callbacks={}, path='/source', silence=False)[source]
Perform API call to SlideRule service
- Parameters
api (str) – name of the SlideRule endpoint
parm (dict) – dictionary of request parameters
stream (bool) – whether the request is a normal service or a stream service (see De-serialization for more details)
callbacks (dict) – record type callbacks (advanced use)
path (str) – path to api being requested
silence (bool) – whether error log messages should be generated
- Returns
response data
- Return type
dictionary
Examples
>>> import sliderule >>> sliderule.set_url("slideruleearth.io") >>> rqst = { ... "time": "NOW", ... "input": "NOW", ... "output": "GPS" ... } >>> rsps = sliderule.source("time", rqst) >>> print(rsps) {'time': 1300556199523.0, 'format': 'GPS'}
set_url
- sliderule.set_url(url)[source]
Configure sliderule package with URL of service
- Parameters
urls (str) – IP address or hostname of SlideRule service (note, there is a special case where the url is provided as a list of strings instead of just a string; when a list is provided, the client hardcodes the set of servers that are used to process requests to the exact set provided; this is used for testing and for local installations and can be ignored by most users)
Examples
>>> import sliderule >>> sliderule.set_url("service.my-sliderule-server.org")
set_verbose
- sliderule.set_verbose(enable)[source]
Configure sliderule package for verbose logging
- Parameters
enable (bool) – whether or not user level log messages received from SlideRule generate a Python log message
Examples
>>> import sliderule >>> sliderule.set_verbose(True)
The default behavior of Python log messages is for them to be displayed to standard output. If you want more control over the behavior of the log messages being display, create and configure a Python log handler as shown below:
>>> # import packages >>> import logging >>> from sliderule import sliderule >>> # Configure Logging >>> sliderule_logger = logging.getLogger("sliderule.sliderule") >>> sliderule_logger.setLevel(logging.INFO) >>> # Create Console Output >>> ch = logging.StreamHandler() >>> ch.setLevel(logging.INFO) >>> sliderule_logger.addHandler(ch)
set_rqst_timeout
- sliderule.set_rqst_timeout(timeout)[source]
Sets the TCP/IP connection and reading timeouts for future requests made to sliderule servers. Setting it lower means the client will failover more quickly, but may generate false positives if a processing request stalls or takes a long time returning data. Setting it higher means the client will wait longer before designating it a failed request which in the presence of a persistent failure means it will take longer for the client to remove the node from its available servers list.
- Parameters
timeout (tuple) – (<connection timeout in seconds>, <read timeout in seconds>)
Examples
>>> import sliderule >>> sliderule.set_rqst_timeout((10, 60))
update_available_servers
- sliderule.update_available_servers(desired_nodes=None, time_to_live=None)[source]
Manages the number of servers in the cluster. If the desired_nodes parameter is set, then a request is made to change the number of servers in the cluster to the number specified. In all cases, the number of nodes currently running in the cluster are returned - even if desired_nodes is set; subsequent calls to this function is needed to check when the current number of nodes matches the desired_nodes.
- Parameters
desired_nodes (int) – the desired number of nodes in the cluster
time_to_live (int) – number of minutes for the desired nodes to run
- Returns
int – number of nodes currently in the cluster
int – number of nodes available for work in the cluster
Examples
>>> import sliderule >>> num_servers, max_workers = sliderule.update_available_servers(10)
scaleout
- sliderule.scaleout(desired_nodes, time_to_live, bypass_dns)[source]
Scale the cluster and wait for cluster to reach desired state
- Parameters
desired_nodes (int) – the desired number of nodes in the cluster
time_to_live (int) – number of minutes for the desired nodes to run
bypass_dns (bool) – the cluster ip address is retrieved from the provisioning system and used directly
Examples
>>> import sliderule >>> sliderule.scaleout(4, 300, False)
authenticate
- sliderule.authenticate(ps_organization, ps_username=None, ps_password=None, github_token=None)[source]
Authenticate to SlideRule Provisioning System The username and password can be provided the following way in order of priority: (1) The passed in arguments github_token or ps_username and ps_password; (2) The O.S. environment variables PS_GITHUB_TOKEN or PS_USERNAME and PS_PASSWORD; (3) The ps.<url> entry in the .netrc file in your home directory
- Parameters
ps_organization (str) – name of the SlideRule organization the user belongs to
ps_username (str) – SlideRule provisioning system account name
ps_password (str) – SlideRule provisioning system account password
github_token (str) – GitHub access token (minimum scope/permissions require)
- Returns
True of successful, False if unsuccessful
- Return type
status
Examples
>>> import sliderule >>> sliderule.authenticate("myorg") True
gps2utc
- sliderule.gps2utc(gps_time, as_str=True)[source]
Convert a GPS based time returned from SlideRule into a UTC time.
- Parameters
gps_time (float) – number of seconds since GPS epoch (January 6, 1980)
as_str (bool) – if True, returns the time as a string; if False, returns the time as datatime object
- Returns
UTC time (i.e. GMT, or Zulu time)
- Return type
datetime
Examples
>>> import sliderule >>> sliderule.gps2utc(1235331234) '2019-02-27 19:34:03'
get_definition
- sliderule.get_definition(rectype, fieldname)[source]
Get the underlying format specification of a field in a return record.
- Parameters
rectype (str) – the name of the type of the record (i.e. “atl03rec”)
fieldname (str) – the name of the record field (i.e. “cycle”)
- Returns
description of each field; see the sliderule.basictypes variable for different field types
- Return type
dict
Examples
>>> import sliderule >>> sliderule.set_url("slideruleearth.io") >>> sliderule.get_definition("atl03rec", "cycle") {'fmt': 'H', 'size': 2, 'nptype': <class 'numpy.uint16'>}
get_version
check_version
- sliderule.check_version(plugins=[])[source]
Check that the version of the client matches the version of the server and any additionally requested plugins
- Parameters
plugins (list) – list of package names (as strings) to check the version on
- Returns
True if at least minor version matches; False if major or minor version doesn’t match
- Return type
bool
toregion
- sliderule.toregion(source, tolerance=0.0, cellsize=0.01, n_clusters=1)[source]
Convert a GeoJSON/Shapefile/GeoDataFrame/list representation of a set of geospatial regions into a list of lat,lon coordinates and raster image recognized by SlideRule
- Parameters
source (str) – file name of GeoJSON formatted regions of interest, file must have name with the .geojson suffix file name of ESRI Shapefile formatted regions of interest, file must have name with the .shp suffix GeoDataFrame of region of interest list of longitude,latitude pairs forming a polygon (e.g. [lat1, lon1, lat2, lon2, lat3, lon3, lat1, lon1]) list of longitude,latitude pairs forming a bounding box (e.g. [lat1, lon1, lat2, lon2])
tolerance (float) – tolerance used to simplify complex shapes so that the number of points is less than the limit (a tolerance of 0.001 typically works for most complex shapes)
cellsize (float) – size of pixel in degrees used to create the raster image of the polygon
n_clusters (int) – number of clusters of polygons to create when breaking up the request to CMR
- Returns
a list of longitudes and latitudes containing the region of interest that can be used for the poly and raster parameters in a processing request to SlideRule.
region = {
“gdf”: <GeoDataFrame of region>
”poly”: [{“lat”: <lat1>, “lon”: <lon1> }, …],
”raster”: {“data”: <geojson file as string>,
”clusters”: [[{“lat”: <lat1>, “lon”: <lon1>}, …], [{“lat”: <lat1>, “lon”: <lon1>}, …]] }
- Return type
dict
Examples
>>> import sliderule, json >>> region = sliderule.toregion("tests/data/grandmesa.geojson") >>> print(json.dumps(region["poly"], indent=4)) [ { "lon": -108.20772968780051, "lat": 38.8232055291981 }, { "lon": -108.07460164311031, "lat": 38.8475137825863 }, { "lon": -107.72839858755752, "lat": 39.01510930230633 }, { "lon": -107.78724142490994, "lat": 39.195630349659986 }, { "lon": -108.17287000970857, "lat": 39.15920066396116 }, { "lon": -108.31168256553767, "lat": 39.13757646212944 }, { "lon": -108.34115668325224, "lat": 39.03758987613325 }, { "lon": -108.2878686387796, "lat": 38.89051431295789 }, { "lon": -108.20772968780051, "lat": 38.8232055291981 } ]