71 aws s3

Loading geospatial datasets from an AWS S3 bucket

Uncomment the following line to install leafmap if needed.

In [1]:

# %pip install -U leafmap

# %pip install -U leafmap

In [2]:

# %pip install -U boto3

# %pip install -U boto3

To Be able to run this notebook you'll need to have AWS credential available as environment variables. Uncomment the following lines to set the environment variables.

In [3]:

# import os
# os.environ["AWS_ACCESS_KEY_ID"] = "YOUR AWS ACCESS ID HERE"
# os.environ["AWS_SECRET_ACCESS_KEY"] = "YOUR AWS ACCESS KEY HERE"

# import os # os.environ["AWS_ACCESS_KEY_ID"] = "YOUR AWS ACCESS ID HERE" # os.environ["AWS_SECRET_ACCESS_KEY"] = "YOUR AWS ACCESS KEY HERE"

In [4]:

import leafmap

import leafmap

In this example, we will use datasets from the Maxar Open Data Program on AWS.

In [5]:

BUCKET = "maxar-opendata"
FOLDER = "events/Kahramanmaras-turkey-earthquake-23/"

BUCKET = "maxar-opendata" FOLDER = "events/Kahramanmaras-turkey-earthquake-23/"

List all the datasets in the bucket. Specify a file extension to filter the results if needed.

In [6]:

items = leafmap.s3_list_objects(BUCKET, FOLDER, ext=".tif")
items[:10]

items = leafmap.s3_list_objects(BUCKET, FOLDER, ext=".tif") items[:10]

Out[6]:

['s3://maxar-opendata/events/Kahramanmaras-turkey-earthquake-23/ard/36/120022103023/2023-02-21/104001008314FC00-ms.tif',
 's3://maxar-opendata/events/Kahramanmaras-turkey-earthquake-23/ard/36/120022103023/2023-02-21/104001008314FC00-pan.tif',
 's3://maxar-opendata/events/Kahramanmaras-turkey-earthquake-23/ard/36/120022103023/2023-02-21/104001008314FC00-visual.tif',
 's3://maxar-opendata/events/Kahramanmaras-turkey-earthquake-23/ard/36/120022103032/2023-02-21/104001008314FC00-ms.tif',
 's3://maxar-opendata/events/Kahramanmaras-turkey-earthquake-23/ard/36/120022103032/2023-02-21/104001008314FC00-pan.tif',
 's3://maxar-opendata/events/Kahramanmaras-turkey-earthquake-23/ard/36/120022103032/2023-02-21/104001008314FC00-visual.tif',
 's3://maxar-opendata/events/Kahramanmaras-turkey-earthquake-23/ard/36/120022103033/2023-02-21/104001008314FC00-ms.tif',
 's3://maxar-opendata/events/Kahramanmaras-turkey-earthquake-23/ard/36/120022103033/2023-02-21/104001008314FC00-pan.tif',
 's3://maxar-opendata/events/Kahramanmaras-turkey-earthquake-23/ard/36/120022103033/2023-02-21/104001008314FC00-visual.tif',
 's3://maxar-opendata/events/Kahramanmaras-turkey-earthquake-23/ard/36/120022103122/2023-02-21/104001008314FC00-ms.tif']

Visualize raster datasets from the bucket.

In [7]:

m = leafmap.Map()
m.add_cog_layer(items[2], name="Maxar")
m

m = leafmap.Map() m.add_cog_layer(items[2], name="Maxar") m

Out[7]:

Download a raster dataset from the bucket.

In [8]:

leafmap.s3_download_file(items[0], outfile="maxar.tif")

leafmap.s3_download_file(items[0], outfile="maxar.tif")

Download a list of raster datasets from the bucket.

In [9]:

leafmap.s3_download_files(items[:2], outdir="maxar")

leafmap.s3_download_files(items[:2], outdir="maxar")

Downloading 1 of 2: maxar/104001008314FC00-ms.tif
Downloading 2 of 2: maxar/104001008314FC00-pan.tif

---

Last update: 2023-03-17
Created: 2021-03-10