Delivery Configs
This article describes how to manage configurations that enable the delivery of Umbra data products directly to customer storage. Configurations are managed in the Organization view in Canopy as the DeliveryConfig
model. Once a DeliveryConfig
has been created, verified, and activated, it can be referenced in tasking requests to direct delivery to the configured storage.
Downloading Data Products
Tasking orders that are delivered to customer storage are still available for download in Canopy.
Delivery Types
Canopy currently supports the DeliveryConfig storage types in the table below.
Type | Description |
---|---|
S3_UMBRA_ROLE | Delivery to an AWS S3 Bucket using an Umbra IAM role to grant write access |
Configuring Storage for Delivery
This section describes the steps to configuring storage for each supported Canopy delivery type.
S3_UMBRA_ROLE - AWS S3
This section describes how to use the S3_UMBRA_ROLE delivery type to deliver tasking orders directly to Amazon Web Services S3 buckets. This configuration requires customers to apply an S3 Bucket Policy to their bucket that grants an Umbra-owned IAM role write access to the bucket. For details on how to create an S3 bucket, see Create a Bucket in the AWS documentation.
Apply S3 Bucket Policy
- In the AWS Console, go to the S3 service.
- Click the name of the bucket you wish to configure.
- Click the Permissions tab.
- Click the Bucket Policy button.
- Copy the Bucket Policy JSON from the section below and modify it as appropriate, replacing
<s3-bucket-name>
with the name of your bucket to grant the rolearn:aws:iam::922714215458:role/prod-prod-s3-external-delivery
access.
If you are looking to test deliveries using the Sandbox, you'll also need to allow the roles described here.
Is your AWS Bucket in GovCloud?
If so, you'll need to replace all instances of the role
arn:aws:iam::922714215458:role/prod-prod-s3-external-delivery
in this Guide with the rolearn:aws-us-gov:iam::537355544329:role/prod-gov-prod-sar-data-delivery-cross-account-role
.You'll also need to set the parameter
isGovcloud
totrue
when creating an S3_UMBRA_ROLE DeliveryConfig via either the API or web app.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Deny",
"Principal": {
"AWS": "*"
},
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::<s3-bucket-name>",
"arn:aws:s3:::<s3-bucket-name>/*"
],
"Condition": {
"Bool": {
"aws:SecureTransport": "false"
}
}
},
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::922714215458:role/prod-prod-s3-external-delivery"
},
"Action": [
"s3:ListBucket",
"s3:PutObject",
"s3:PutObjectAcl"
],
"Resource": [
"arn:aws:s3:::<s3-bucket-name>",
"arn:aws:s3:::<s3-bucket-name>/*"
]
}
]
}
Apply KMS Policy for Encrypted Buckets
If your bucket uses a KMS Key to encrypt data at rest, you'll also need to add the following statement to the policy for your KMS Key:
{
"Sid": "UmbraKMSKeyAccess",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::922714215458:role/prod-prod-s3-external-delivery"
]
},
"Action": [
"kms:GenerateDataKey",
"kms:Decrypt"
],
"Resource": "*"
}
Permissions for S3 Multipart Copy
The kms:Decrypt
permission allows Umbra to perform S3 Multipart Copy of data directly from one bucket to another for best performance. This is required for all users because some files may be larger than 5GB.
For more information see https://repost.aws/knowledge-center/s3-large-file-encryption-kms-key
Debugging S3_UMBRA_ROLE Configs
If you're unable to verify your DeliveryConfigs after following the above steps, contact [email protected]. Before doing so, please also add the following IAM Role to your Bucket Policy:
- Commercial Bucket:
arn:aws:iam::661908204374:role/dev-test-s3-external-delivery
- GovCloud Bucket:
arn:aws-us-gov:iam::134305833187:role/dev-gov-test-sar-data-delivery-cross-account-role
This role allows us to assist in debugging by giving our test environment access. Once debugging is completed, this role may be removed from the Bucket Policy.
Creating a Delivery Configuration
The following section describes the workflow for creating a new DeliveryConfig from the Organization settings in Canopy. For the API reference, see DeliveryConfig.
- Click Add Delivery Method.
- Select the delivery type you wish to configure from the options, complete the Delivery Config form, and click Save For information on prefix templating, see Path Prefix Template.
t3. From the newly added DeliveryConfig in the list, click Validate. If verification is successful, the configuration state will become ACTIVE
DeliveryConfig Verification
When you Verify a DeliveryConfig, Umbra will attempt to push a text file that contains the text "Verified at {ISO8601 timestamp} to the root of the path you configured. If this succeeds, the DeliveryConfig is marked ACTIVE.
Changes to bucket policies after Verification may invalidate the DeliveryConfig and cause deliveries to that bucket to fail.
Delivery Path
This section describes the delivery path and file naming scheme.
The default file naming scheme is shown in the example below:
# Example - AWS S3
s3://{DeliveryConfig.options.bucket}/{DeliveryConfig.options.path}/{Task.id}/{YYYY-MM-DD-hh-mm-ss_{vehicle_name}/YYYY-MM-DD-hh-mm-ss_{vehicle_name}-{product_type}.{ext}
# DeliveryConfig.options.bucket = `yourco-umbra-delivery`
# DeliveryConfig.options.path = `path/to/data`
# Task.id = "4a270ee4-bbfc-4d54-97ae-e30af6950053"
s3://yourco-umbra-delivery/path/to/data/4a270ee4-bbfc-4d54-97ae-e30af6950053/2022-11-10-20-19-51_UMBRA-03/2022-11-10-20-19-51_UMBRA-03_GEC.tif
For a list of possible values for {product_type}
and {ext}
, click here.
The value for {vehicle_name}
should correspond to a satellite id shown in this table but in all upper case (eg. UMBRA-03
).
Path Prefix Template
When defining the Path field for a DeliveryConfig, some Task properties can be accessed as template variables to customize the path prefix of the deliveries. The Task properties available for templating are listed below.
deliveryConfigId, taskName, organizationName, userOrderId
To use a property as a template, enclose the Task property name in curly braces in the DeliveryConfig.options.path
parameter when creating a new DeliveryConfig. For example: path = "{userOrderId}"
.
# Example
# DeliveryConfig.options.bucket = `yourco-umbra-delivery`
# DeliveryConfig.options.path = `{userOrderId}`
# Task.id = "4a270ee4-bbfc-4d54-97ae-e30af6950053"
# Task.userOrderId = `ABC-12345`
s3://yourco-umbra-delivery/ABC-12345/4a270ee4-bbfc-4d54-97ae-e30af6950053/2022-11-10-20-19-51_UMBRA-03/2022-11-10-20-19-51_UMBRA-03_GEC.tif
Troubleshooting
For support with delivery configurations, please contact [email protected].
Updated 3 days ago