Environment

mwaa.services.k8s.aws/v1alpha1

TypeLink
GoDocmwaa-controller/apis/v1alpha1#Environment

Metadata

PropertyValue
ScopeNamespaced
KindEnvironment
ListKindEnvironmentList
Pluralenvironments
Singularenvironment

Describes an Amazon Managed Workflows for Apache Airflow (MWAA) environment.

Spec

airflowConfigurationOptions: {}
airflowVersion: string
dagS3Path: string
endpointManagement: string
environmentClass: string
executionRoleARN: string
executionRoleRef: 
  from: 
    name: string
    namespace: string
kmsKey: string
kmsKeyRef: 
  from: 
    name: string
    namespace: string
loggingConfiguration: 
  dagProcessingLogs: 
    enabled: boolean
    logLevel: string
  schedulerLogs: 
    enabled: boolean
    logLevel: string
  taskLogs: 
    enabled: boolean
    logLevel: string
  webserverLogs: 
    enabled: boolean
    logLevel: string
  workerLogs: 
    enabled: boolean
    logLevel: string
maxWebservers: integer
maxWorkers: integer
minWebservers: integer
minWorkers: integer
name: string
networkConfiguration: 
  securityGroupIDs:
  - string
  securityGroupRefs:
    from: 
      name: string
      namespace: string
  subnetIDs:
  - string
  subnetRefs:
    from: 
      name: string
      namespace: string
pluginsS3ObjectVersion: string
pluginsS3Path: string
requirementsS3ObjectVersion: string
requirementsS3Path: string
schedulers: integer
sourceBucketARN: string
sourceBucketRef: 
  from: 
    name: string
    namespace: string
startupScriptS3ObjectVersion: string
startupScriptS3Path: string
tags: {}
webserverAccessMode: string
weeklyMaintenanceWindowStart: string
FieldDescription
airflowConfigurationOptions
Optional
object
A list of key-value pairs containing the Apache Airflow configuration options
you want to attach to your environment. For more information, see Apache
Airflow configuration options (https://docs.aws.amazon.com/mwaa/latest/userguide/configuring-env-variables.html).
airflowVersion
Optional
string
The Apache Airflow version for your environment. If no value is specified,
it defaults to the latest version. For more information, see Apache Airflow
versions on Amazon Managed Workflows for Apache Airflow (Amazon MWAA) (https://docs.aws.amazon.com/mwaa/latest/userguide/airflow-versions.html).

Valid values: 1.10.12, 2.0.2, 2.2.2, 2.4.3, 2.5.1, 2.6.3, 2.7.2, 2.8.1, 2.9.2,
2.10.1, and 2.10.3.

Regex Pattern: ^[0-9a-z.]+$
dagS3Path
Required
string
The relative path to the DAGs folder on your Amazon S3 bucket. For example,
dags. For more information, see Adding or updating DAGs (https://docs.aws.amazon.com/mwaa/latest/userguide/configuring-dag-folder.html).

Regex Pattern: .*
endpointManagement
Optional
string
Defines whether the VPC endpoints configured for the environment are created,
and managed, by the customer or by Amazon MWAA. If set to SERVICE, Amazon
MWAA will create and manage the required VPC endpoints in your VPC. If set
to CUSTOMER, you must create, and manage, the VPC endpoints for your VPC.
If you choose to create an environment in a shared VPC, you must set this
value to CUSTOMER. In a shared VPC deployment, the environment will remain
in PENDING status until you create the VPC endpoints. If you do not take
action to create the endpoints within 72 hours, the status will change to
CREATE_FAILED. You can delete the failed environment and create a new one.
environmentClass
Optional
string
The environment class type. Valid values: mw1.micro, mw1.small, mw1.medium,
mw1.large, mw1.xlarge, and mw1.2xlarge. For more information, see Amazon
MWAA environment class (https://docs.aws.amazon.com/mwaa/latest/userguide/environment-class.html).
executionRoleARN
Optional
string
The Amazon Resource Name (ARN) of the execution role for your environment.
An execution role is an Amazon Web Services Identity and Access Management
(IAM) role that grants MWAA permission to access Amazon Web Services services
and resources used by your environment. For example, arn:aws:iam::123456789:role/my-execution-role.
For more information, see Amazon MWAA Execution role (https://docs.aws.amazon.com/mwaa/latest/userguide/mwaa-create-role.html).

Regex Pattern: ^arn:aws(-[a-z]+)?:iam::\d{12}:role/?[a-zA-Z_0-9+=,.@\-_/]+$
executionRoleRef
Optional
object
AWSResourceReferenceWrapper provides a wrapper around *AWSResourceReference
type to provide more user friendly syntax for references using ‘from’ field
Ex:
APIIDRef:

from:
name: my-api
executionRoleRef.from
Optional
object
AWSResourceReference provides all the values necessary to reference another
k8s resource for finding the identifier(Id/ARN/Name)
executionRoleRef.from.name
Optional
string
executionRoleRef.from.namespace
Optional
string
kmsKey
Optional
string
The Amazon Web Services Key Management Service (KMS) key to encrypt the data
in your environment. You can use an Amazon Web Services owned CMK, or a Customer
managed CMK (advanced). For more information, see Create an Amazon MWAA environment
(https://docs.aws.amazon.com/mwaa/latest/userguide/create-environment.html).

Regex Pattern: `^(((arn:aws(-[a-z]+)?:kms:[a-z]{2}-[a-z]+-\d:\d+:)?key/)?[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}
kmsKeyRef
Optional
object
AWSResourceReferenceWrapper provides a wrapper around *AWSResourceReference
type to provide more user friendly syntax for references using ‘from’ field
Ex:
APIIDRef:

from:
name: my-api
kmsKeyRef.from
Optional
object
AWSResourceReference provides all the values necessary to reference another
k8s resource for finding the identifier(Id/ARN/Name)
kmsKeyRef.from.name
Optional
string
kmsKeyRef.from.namespace
Optional
string
loggingConfiguration
Optional
object
Defines the Apache Airflow logs to send to CloudWatch Logs.
loggingConfiguration.dagProcessingLogs
Optional
object
Enables the Apache Airflow log type (e.g. DagProcessingLogs) and defines
the log level to send to CloudWatch Logs (e.g. INFO).
loggingConfiguration.dagProcessingLogs.enabled
Optional
boolean
loggingConfiguration.dagProcessingLogs.logLevel
Optional
string
loggingConfiguration.schedulerLogs
Optional
object
Enables the Apache Airflow log type (e.g. DagProcessingLogs) and defines
the log level to send to CloudWatch Logs (e.g. INFO).
loggingConfiguration.schedulerLogs.enabled
Optional
boolean
loggingConfiguration.schedulerLogs.logLevel
Optional
string
loggingConfiguration.taskLogs
Optional
object
Enables the Apache Airflow log type (e.g. DagProcessingLogs) and defines
the log level to send to CloudWatch Logs (e.g. INFO).
loggingConfiguration.taskLogs.enabled
Optional
boolean
loggingConfiguration.taskLogs.logLevel
Optional
string
loggingConfiguration.webserverLogs
Optional
object
Enables the Apache Airflow log type (e.g. DagProcessingLogs) and defines
the log level to send to CloudWatch Logs (e.g. INFO).
loggingConfiguration.webserverLogs.enabled
Optional
boolean
loggingConfiguration.webserverLogs.logLevel
Optional
string
loggingConfiguration.workerLogs
Optional
object
Enables the Apache Airflow log type (e.g. DagProcessingLogs) and defines
the log level to send to CloudWatch Logs (e.g. INFO).
loggingConfiguration.workerLogs.enabled
Optional
boolean
loggingConfiguration.workerLogs.logLevel
Optional
string
maxWebservers
Optional
integer
The maximum number of web servers that you want to run in your environment.
Amazon MWAA scales the number of Apache Airflow web servers up to the number
you specify for MaxWebservers when you interact with your Apache Airflow
environment using Apache Airflow REST API, or the Apache Airflow CLI. For
example, in scenarios where your workload requires network calls to the Apache
Airflow REST API with a high transaction-per-second (TPS) rate, Amazon MWAA
will increase the number of web servers up to the number set in MaxWebserers.
As TPS rates decrease Amazon MWAA disposes of the additional web servers,
and scales down to the number set in MinxWebserers.

Valid values: For environments larger than mw1.micro, accepts values from
2 to 5. Defaults to 2 for all environment sizes except mw1.micro, which defaults
to 1.
maxWorkers
Optional
integer
The maximum number of workers that you want to run in your environment. MWAA
scales the number of Apache Airflow workers up to the number you specify
in the MaxWorkers field. For example, 20. When there are no more tasks running,
and no more in the queue, MWAA disposes of the extra workers leaving the
one worker that is included with your environment, or the number you specify
in MinWorkers.
minWebservers
Optional
integer
The minimum number of web servers that you want to run in your environment.
Amazon MWAA scales the number of Apache Airflow web servers up to the number
you specify for MaxWebservers when you interact with your Apache Airflow
environment using Apache Airflow REST API, or the Apache Airflow CLI. As
the transaction-per-second rate, and the network load, decrease, Amazon MWAA
disposes of the additional web servers, and scales down to the number set
in MinxWebserers.

Valid values: For environments larger than mw1.micro, accepts values from
2 to 5. Defaults to 2 for all environment sizes except mw1.micro, which defaults
to 1.
minWorkers
Optional
integer
The minimum number of workers that you want to run in your environment. MWAA
scales the number of Apache Airflow workers up to the number you specify
in the MaxWorkers field. When there are no more tasks running, and no more
in the queue, MWAA disposes of the extra workers leaving the worker count
you specify in the MinWorkers field. For example, 2.
name
Required
string
The name of the Amazon MWAA environment. For example, MyMWAAEnvironment.

Regex Pattern: ^[a-zA-Z][0-9a-zA-Z-_]*$
networkConfiguration
Required
object
The VPC networking components used to secure and enable network traffic between
the Amazon Web Services resources for your environment. For more information,
see About networking on Amazon MWAA (https://docs.aws.amazon.com/mwaa/latest/userguide/networking-about.html).
networkConfiguration.securityGroupIDs
Optional
array
networkConfiguration.securityGroupIDs.[]
Required
string
networkConfiguration.securityGroupRefs.[]
Required
object
AWSResourceReferenceWrapper provides a wrapper around *AWSResourceReference
type to provide more user friendly syntax for references using ‘from’ field
Ex:
APIIDRef:
from:
  name: my-api || **networkConfiguration.securityGroupRefs.[].from**<br/>Optional | **object**<br/>AWSResourceReference provides all the values necessary to reference another<br/>k8s resource for finding the identifier(Id/ARN/Name) |

| networkConfiguration.securityGroupRefs.[].from.name
Optional | string
| | networkConfiguration.securityGroupRefs.[].from.namespace
Optional | string
| | networkConfiguration.subnetIDs
Optional | array
| | networkConfiguration.subnetIDs.[]
Required | string
|| networkConfiguration.subnetRefs
Optional | array
Reference field for SubnetIDs | | networkConfiguration.subnetRefs.[]
Required | object
AWSResourceReferenceWrapper provides a wrapper around *AWSResourceReference type to provide more user friendly syntax for references using ‘from’ field Ex: APIIDRef:

from:
  name: my-api || **networkConfiguration.subnetRefs.[].from**<br/>Optional | **object**<br/>AWSResourceReference provides all the values necessary to reference another<br/>k8s resource for finding the identifier(Id/ARN/Name) |

| networkConfiguration.subnetRefs.[].from.name
Optional | string
| | networkConfiguration.subnetRefs.[].from.namespace
Optional | string
| | pluginsS3ObjectVersion
Optional | string
The version of the plugins.zip file on your Amazon S3 bucket. You must specify
a version each time a plugins.zip file is updated. For more information,
see How S3 Versioning works (https://docs.aws.amazon.com/AmazonS3/latest/userguide/versioning-workflows.html). | | pluginsS3Path
Optional | string
The relative path to the plugins.zip file on your Amazon S3 bucket. For example,
plugins.zip. If specified, then the plugins.zip version is required. For
more information, see Installing custom plugins (https://docs.aws.amazon.com/mwaa/latest/userguide/configuring-dag-import-plugins.html).

Regex Pattern: .* | | requirementsS3ObjectVersion
Optional | string
The version of the requirements.txt file on your Amazon S3 bucket. You must
specify a version each time a requirements.txt file is updated. For more
information, see How S3 Versioning works (https://docs.aws.amazon.com/AmazonS3/latest/userguide/versioning-workflows.html). | | requirementsS3Path
Optional | string
The relative path to the requirements.txt file on your Amazon S3 bucket.
For example, requirements.txt. If specified, then a version is required.
For more information, see Installing Python dependencies (https://docs.aws.amazon.com/mwaa/latest/userguide/working-dags-dependencies.html).

Regex Pattern: .* | | schedulers
Optional | integer
The number of Apache Airflow schedulers to run in your environment. Valid
values:

* v2 - For environments larger than mw1.micro, accepts values from 2 to
5. Defaults to 2 for all environment sizes except mw1.micro, which defaults
to 1.

* v1 - Accepts 1. | | sourceBucketARN
Optional | string
The Amazon Resource Name (ARN) of the Amazon S3 bucket where your DAG code
and supporting files are stored. For example, arn:aws:s3:::my-airflow-bucket-unique-name.
For more information, see Create an Amazon S3 bucket for Amazon MWAA (https://docs.aws.amazon.com/mwaa/latest/userguide/mwaa-s3-bucket.html).

Regex Pattern: ^arn:aws(-[a-z]+)?:s3:::[a-z0-9.\-]+$ | | sourceBucketRef
Optional | object
AWSResourceReferenceWrapper provides a wrapper around *AWSResourceReference
type to provide more user friendly syntax for references using ‘from’ field
Ex:
APIIDRef:

from:
name: my-api | | sourceBucketRef.from
Optional | object
AWSResourceReference provides all the values necessary to reference another
k8s resource for finding the identifier(Id/ARN/Name) | | sourceBucketRef.from.name
Optional | string
| | sourceBucketRef.from.namespace
Optional | string
| | startupScriptS3ObjectVersion
Optional | string
The version of the startup shell script in your Amazon S3 bucket. You must
specify the version ID (https://docs.aws.amazon.com/AmazonS3/latest/userguide/versioning-workflows.html)
that Amazon S3 assigns to the file every time you update the script.

Version IDs are Unicode, UTF-8 encoded, URL-ready, opaque strings that are
no more than 1,024 bytes long. The following is an example:

3sL4kqtJlcpXroDTDmJ+rmSpXd3dIbrHY+MTRCxf3vjVBH40Nr8X8gdRQBpUMLUo

For more information, see Using a startup script (https://docs.aws.amazon.com/mwaa/latest/userguide/using-startup-script.html). | | startupScriptS3Path
Optional | string
The relative path to the startup shell script in your Amazon S3 bucket. For
example, s3://mwaa-environment/startup.sh.

Amazon MWAA runs the script as your environment starts, and before running
the Apache Airflow process. You can use this script to install dependencies,
modify Apache Airflow configuration options, and set environment variables.
For more information, see Using a startup script (https://docs.aws.amazon.com/mwaa/latest/userguide/using-startup-script.html).

Regex Pattern: .* | | tags
Optional | object
The key-value tag pairs you want to associate to your environment. For example,
“Environment”: “Staging”. For more information, see Tagging Amazon Web Services
resources (https://docs.aws.amazon.com/general/latest/gr/aws_tagging.html). | | webserverAccessMode
Optional | string
Defines the access mode for the Apache Airflow web server. For more information,
see Apache Airflow access modes (https://docs.aws.amazon.com/mwaa/latest/userguide/configuring-networking.html). | | weeklyMaintenanceWindowStart
Optional | string
The day and time of the week in Coordinated Universal Time (UTC) 24-hour
standard time to start weekly maintenance updates of your environment in
the following format: DAY:HH:MM. For example: TUE:03:30. You can specify
a start time in 30 minute increments only.

Regex Pattern: (MON|TUE|WED|THU|FRI|SAT|SUN):([01]\d|2[0-3]):(00|30) |

Status

ackResourceMetadata: 
  arn: string
  ownerAccountID: string
  partition: string
  region: string
conditions:
- lastTransitionTime: string
  message: string
  reason: string
  status: string
  type: string
createdAt: string
lastUpdate: 
  createdAt: string
  error: 
    errorCode: string
    errorMessage: string
  source: string
  status: string
  workerReplacementStrategy: string
status: string
webserverURL: string
FieldDescription
ackResourceMetadata
Optional
object
All CRs managed by ACK have a common Status.ACKResourceMetadata member
that is used to contain resource sync state, account ownership,
constructed ARN for the resource
ackResourceMetadata.arn
Optional
string
ARN is the Amazon Resource Name for the resource. This is a
globally-unique identifier and is set only by the ACK service controller
once the controller has orchestrated the creation of the resource OR
when it has verified that an “adopted” resource (a resource where the
ARN annotation was set by the Kubernetes user on the CR) exists and
matches the supplied CR’s Spec field values.
https://github.com/aws/aws-controllers-k8s/issues/270
ackResourceMetadata.ownerAccountID
Required
string
OwnerAccountID is the AWS Account ID of the account that owns the
backend AWS service API resource.
ackResourceMetadata.partition
Optional
string
Partition is the AWS partition in which the resource exists or will exist
ackResourceMetadata.region
Required
string
Region is the AWS region in which the resource exists or will exist.
conditions
Optional
array
All CRs managed by ACK have a common Status.Conditions member that
contains a collection of ackv1alpha1.Condition objects that describe
the various terminal states of the CR and its backend AWS service API
resource
conditions.[]
Required
object
Condition is the common struct used by all CRDs managed by ACK service
controllers to indicate terminal states of the CR and its backend AWS
service API resource
conditions.[].message
Optional
string
A human readable message indicating details about the transition.
conditions.[].reason
Optional
string
The reason for the condition’s last transition.
conditions.[].status
Optional
string
Status of the condition, one of True, False, Unknown.
conditions.[].type
Optional
string
Type is the type of the Condition
createdAt
Optional
string
The day and time the environment was created.
lastUpdate
Optional
object
The status of the last update on the environment.
lastUpdate.createdAt
Optional
string
lastUpdate.error
Optional
object
Describes the error(s) encountered with the last update of the environment.
lastUpdate.error.errorCode
Optional
string
lastUpdate.error.errorMessage
Optional
string
lastUpdate.source
Optional
string
lastUpdate.status
Optional
string
lastUpdate.workerReplacementStrategy
Optional
string
status
Optional
string
The status of the Amazon MWAA environment.

Valid values:

* CREATING - Indicates the request to create the environment is in progress.

* CREATING_SNAPSHOT - Indicates the request to update environment details,
or upgrade the environment version, is in progress and Amazon MWAA is
creating a storage volume snapshot of the Amazon RDS database cluster
associated with the environment. A database snapshot is a backup created
at a specific point in time. Amazon MWAA uses snapshots to recover environment
metadata if the process to update or upgrade an environment fails.

* CREATE_FAILED - Indicates the request to create the environment failed,
and the environment could not be created.

* AVAILABLE - Indicates the request was successful and the environment
is ready to use.

* PENDING - Indicates the request was successful, but the process to create
the environment is paused until you create the required VPC endpoints
in your VPC. After you create the VPC endpoints, the process resumes.

* UPDATING - Indicates the request to update the environment is in progress.

* ROLLING_BACK - Indicates the request to update environment details,
or upgrade the environment version, failed and Amazon MWAA is restoring
the environment using the latest storage volume snapshot.

* DELETING - Indicates the request to delete the environment is in progress.

* DELETED - Indicates the request to delete the environment is complete,
and the environment has been deleted.

* UNAVAILABLE - Indicates the request failed, but the environment did
not return to its previous state and is not stable.

* UPDATE_FAILED - Indicates the request to update the environment failed,
and the environment was restored to its previous state successfully and
is ready to use.

* MAINTENANCE - Indicates that the environment is undergoing maintenance.
Depending on the type of work Amazon MWAA is performing, your environment
might become unavailable during this process. After all operations are
done, your environment will return to its status prior to mainteneace
operations.

We recommend reviewing our troubleshooting guide for a list of common errors
and their solutions. For more information, see Amazon MWAA troubleshooting
(https://docs.aws.amazon.com/mwaa/latest/userguide/troubleshooting.html).
webserverURL
Optional
string
The Apache Airflow web server host name for the Amazon MWAA environment.
For more information, see Accessing the Apache Airflow UI (https://docs.aws.amazon.com/mwaa/latest/userguide/access-airflow-ui.html).

Regex Pattern: ^https://.+$