Environment
mwaa.services.k8s.aws/v1alpha1
| Type | Link |
|---|---|
| GoDoc | mwaa-controller/apis/v1alpha1#Environment |
Metadata
| Property | Value |
|---|---|
| Scope | Namespaced |
| Kind | Environment |
| ListKind | EnvironmentList |
| Plural | environments |
| Singular | environment |
Describes an Amazon Managed Workflows for Apache Airflow (MWAA) environment.
Spec
airflowConfigurationOptions: {}
airflowVersion: string
dagS3Path: string
endpointManagement: string
environmentClass: string
executionRoleARN: string
executionRoleRef:
from:
name: string
namespace: string
kmsKey: string
kmsKeyRef:
from:
name: string
namespace: string
loggingConfiguration:
dagProcessingLogs:
enabled: boolean
logLevel: string
schedulerLogs:
enabled: boolean
logLevel: string
taskLogs:
enabled: boolean
logLevel: string
webserverLogs:
enabled: boolean
logLevel: string
workerLogs:
enabled: boolean
logLevel: string
maxWebservers: integer
maxWorkers: integer
minWebservers: integer
minWorkers: integer
name: string
networkConfiguration:
securityGroupIDs:
- string
securityGroupRefs:
from:
name: string
namespace: string
subnetIDs:
- string
subnetRefs:
from:
name: string
namespace: string
pluginsS3ObjectVersion: string
pluginsS3Path: string
requirementsS3ObjectVersion: string
requirementsS3Path: string
schedulers: integer
sourceBucketARN: string
sourceBucketRef:
from:
name: string
namespace: string
startupScriptS3ObjectVersion: string
startupScriptS3Path: string
tags: {}
webserverAccessMode: string
weeklyMaintenanceWindowStart: string
| Field | Description |
|---|---|
| airflowConfigurationOptions Optional | object A list of key-value pairs containing the Apache Airflow configuration options you want to attach to your environment. For more information, see Apache Airflow configuration options (https://docs.aws.amazon.com/mwaa/latest/userguide/configuring-env-variables.html). |
| airflowVersion Optional | string The Apache Airflow version for your environment. If no value is specified, it defaults to the latest version. For more information, see Apache Airflow versions on Amazon Managed Workflows for Apache Airflow (Amazon MWAA) (https://docs.aws.amazon.com/mwaa/latest/userguide/airflow-versions.html). Valid values: 1.10.12, 2.0.2, 2.2.2, 2.4.3, 2.5.1, 2.6.3, 2.7.2, 2.8.1, 2.9.2, 2.10.1, and 2.10.3. Regex Pattern: ^[0-9a-z.]+$ |
| dagS3Path Required | string The relative path to the DAGs folder on your Amazon S3 bucket. For example, dags. For more information, see Adding or updating DAGs (https://docs.aws.amazon.com/mwaa/latest/userguide/configuring-dag-folder.html). Regex Pattern: .* |
| endpointManagement Optional | string Defines whether the VPC endpoints configured for the environment are created, and managed, by the customer or by Amazon MWAA. If set to SERVICE, Amazon MWAA will create and manage the required VPC endpoints in your VPC. If set to CUSTOMER, you must create, and manage, the VPC endpoints for your VPC. If you choose to create an environment in a shared VPC, you must set this value to CUSTOMER. In a shared VPC deployment, the environment will remain in PENDING status until you create the VPC endpoints. If you do not take action to create the endpoints within 72 hours, the status will change to CREATE_FAILED. You can delete the failed environment and create a new one. |
| environmentClass Optional | string The environment class type. Valid values: mw1.micro, mw1.small, mw1.medium, mw1.large, mw1.xlarge, and mw1.2xlarge. For more information, see Amazon MWAA environment class (https://docs.aws.amazon.com/mwaa/latest/userguide/environment-class.html). |
| executionRoleARN Optional | string The Amazon Resource Name (ARN) of the execution role for your environment. An execution role is an Amazon Web Services Identity and Access Management (IAM) role that grants MWAA permission to access Amazon Web Services services and resources used by your environment. For example, arn:aws:iam::123456789:role/my-execution-role. For more information, see Amazon MWAA Execution role (https://docs.aws.amazon.com/mwaa/latest/userguide/mwaa-create-role.html). Regex Pattern: ^arn:aws(-[a-z]+)?:iam::\d{12}:role/?[a-zA-Z_0-9+=,.@\-_/]+$ |
| executionRoleRef Optional | object AWSResourceReferenceWrapper provides a wrapper around *AWSResourceReference type to provide more user friendly syntax for references using ‘from’ field Ex: APIIDRef: from: name: my-api |
| executionRoleRef.from Optional | object AWSResourceReference provides all the values necessary to reference another k8s resource for finding the identifier(Id/ARN/Name) |
| executionRoleRef.from.name Optional | string |
| executionRoleRef.from.namespace Optional | string |
| kmsKey Optional | string The Amazon Web Services Key Management Service (KMS) key to encrypt the data in your environment. You can use an Amazon Web Services owned CMK, or a Customer managed CMK (advanced). For more information, see Create an Amazon MWAA environment (https://docs.aws.amazon.com/mwaa/latest/userguide/create-environment.html). Regex Pattern: `^(((arn:aws(-[a-z]+)?:kms:[a-z]{2}-[a-z]+-\d:\d+:)?key/)?[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12} |
| kmsKeyRef Optional | object AWSResourceReferenceWrapper provides a wrapper around *AWSResourceReference type to provide more user friendly syntax for references using ‘from’ field Ex: APIIDRef: from: name: my-api |
| kmsKeyRef.from Optional | object AWSResourceReference provides all the values necessary to reference another k8s resource for finding the identifier(Id/ARN/Name) |
| kmsKeyRef.from.name Optional | string |
| kmsKeyRef.from.namespace Optional | string |
| loggingConfiguration Optional | object Defines the Apache Airflow logs to send to CloudWatch Logs. |
| loggingConfiguration.dagProcessingLogs Optional | object Enables the Apache Airflow log type (e.g. DagProcessingLogs) and defines the log level to send to CloudWatch Logs (e.g. INFO). |
| loggingConfiguration.dagProcessingLogs.enabled Optional | boolean |
| loggingConfiguration.dagProcessingLogs.logLevel Optional | string |
| loggingConfiguration.schedulerLogs Optional | object Enables the Apache Airflow log type (e.g. DagProcessingLogs) and defines the log level to send to CloudWatch Logs (e.g. INFO). |
| loggingConfiguration.schedulerLogs.enabled Optional | boolean |
| loggingConfiguration.schedulerLogs.logLevel Optional | string |
| loggingConfiguration.taskLogs Optional | object Enables the Apache Airflow log type (e.g. DagProcessingLogs) and defines the log level to send to CloudWatch Logs (e.g. INFO). |
| loggingConfiguration.taskLogs.enabled Optional | boolean |
| loggingConfiguration.taskLogs.logLevel Optional | string |
| loggingConfiguration.webserverLogs Optional | object Enables the Apache Airflow log type (e.g. DagProcessingLogs) and defines the log level to send to CloudWatch Logs (e.g. INFO). |
| loggingConfiguration.webserverLogs.enabled Optional | boolean |
| loggingConfiguration.webserverLogs.logLevel Optional | string |
| loggingConfiguration.workerLogs Optional | object Enables the Apache Airflow log type (e.g. DagProcessingLogs) and defines the log level to send to CloudWatch Logs (e.g. INFO). |
| loggingConfiguration.workerLogs.enabled Optional | boolean |
| loggingConfiguration.workerLogs.logLevel Optional | string |
| maxWebservers Optional | integer The maximum number of web servers that you want to run in your environment. Amazon MWAA scales the number of Apache Airflow web servers up to the number you specify for MaxWebservers when you interact with your Apache Airflow environment using Apache Airflow REST API, or the Apache Airflow CLI. For example, in scenarios where your workload requires network calls to the Apache Airflow REST API with a high transaction-per-second (TPS) rate, Amazon MWAA will increase the number of web servers up to the number set in MaxWebserers. As TPS rates decrease Amazon MWAA disposes of the additional web servers, and scales down to the number set in MinxWebserers. Valid values: For environments larger than mw1.micro, accepts values from 2 to 5. Defaults to 2 for all environment sizes except mw1.micro, which defaults to 1. |
| maxWorkers Optional | integer The maximum number of workers that you want to run in your environment. MWAA scales the number of Apache Airflow workers up to the number you specify in the MaxWorkers field. For example, 20. When there are no more tasks running, and no more in the queue, MWAA disposes of the extra workers leaving the one worker that is included with your environment, or the number you specify in MinWorkers. |
| minWebservers Optional | integer The minimum number of web servers that you want to run in your environment. Amazon MWAA scales the number of Apache Airflow web servers up to the number you specify for MaxWebservers when you interact with your Apache Airflow environment using Apache Airflow REST API, or the Apache Airflow CLI. As the transaction-per-second rate, and the network load, decrease, Amazon MWAA disposes of the additional web servers, and scales down to the number set in MinxWebserers. Valid values: For environments larger than mw1.micro, accepts values from 2 to 5. Defaults to 2 for all environment sizes except mw1.micro, which defaults to 1. |
| minWorkers Optional | integer The minimum number of workers that you want to run in your environment. MWAA scales the number of Apache Airflow workers up to the number you specify in the MaxWorkers field. When there are no more tasks running, and no more in the queue, MWAA disposes of the extra workers leaving the worker count you specify in the MinWorkers field. For example, 2. |
| name Required | string The name of the Amazon MWAA environment. For example, MyMWAAEnvironment. Regex Pattern: ^[a-zA-Z][0-9a-zA-Z-_]*$ |
| networkConfiguration Required | object The VPC networking components used to secure and enable network traffic between the Amazon Web Services resources for your environment. For more information, see About networking on Amazon MWAA (https://docs.aws.amazon.com/mwaa/latest/userguide/networking-about.html). |
| networkConfiguration.securityGroupIDs Optional | array |
| networkConfiguration.securityGroupIDs.[] Required | string |
| networkConfiguration.securityGroupRefs.[] Required | object AWSResourceReferenceWrapper provides a wrapper around *AWSResourceReference |
| type to provide more user friendly syntax for references using ‘from’ field | |
| Ex: | |
| APIIDRef: |
from:
name: my-api || **networkConfiguration.securityGroupRefs.[].from**<br/>Optional | **object**<br/>AWSResourceReference provides all the values necessary to reference another<br/>k8s resource for finding the identifier(Id/ARN/Name) |
| networkConfiguration.securityGroupRefs.[].from.name
Optional | string
|
| networkConfiguration.securityGroupRefs.[].from.namespace
Optional | string
|
| networkConfiguration.subnetIDs
Optional | array
|
| networkConfiguration.subnetIDs.[]
Required | string
|| networkConfiguration.subnetRefs
Optional | array
Reference field for SubnetIDs |
| networkConfiguration.subnetRefs.[]
Required | object
AWSResourceReferenceWrapper provides a wrapper around *AWSResourceReference
type to provide more user friendly syntax for references using ‘from’ field
Ex:
APIIDRef:
from:
name: my-api || **networkConfiguration.subnetRefs.[].from**<br/>Optional | **object**<br/>AWSResourceReference provides all the values necessary to reference another<br/>k8s resource for finding the identifier(Id/ARN/Name) |
| networkConfiguration.subnetRefs.[].from.name
Optional | string
|
| networkConfiguration.subnetRefs.[].from.namespace
Optional | string
|
| pluginsS3ObjectVersion
Optional | string
The version of the plugins.zip file on your Amazon S3 bucket. You must specify
a version each time a plugins.zip file is updated. For more information,
see How S3 Versioning works (https://docs.aws.amazon.com/AmazonS3/latest/userguide/versioning-workflows.html). |
| pluginsS3Path
Optional | string
The relative path to the plugins.zip file on your Amazon S3 bucket. For example,
plugins.zip. If specified, then the plugins.zip version is required. For
more information, see Installing custom plugins (https://docs.aws.amazon.com/mwaa/latest/userguide/configuring-dag-import-plugins.html).
Regex Pattern: .* |
| requirementsS3ObjectVersion
Optional | string
The version of the requirements.txt file on your Amazon S3 bucket. You must
specify a version each time a requirements.txt file is updated. For more
information, see How S3 Versioning works (https://docs.aws.amazon.com/AmazonS3/latest/userguide/versioning-workflows.html). |
| requirementsS3Path
Optional | string
The relative path to the requirements.txt file on your Amazon S3 bucket.
For example, requirements.txt. If specified, then a version is required.
For more information, see Installing Python dependencies (https://docs.aws.amazon.com/mwaa/latest/userguide/working-dags-dependencies.html).
Regex Pattern: .* |
| schedulers
Optional | integer
The number of Apache Airflow schedulers to run in your environment. Valid
values:
* v2 - For environments larger than mw1.micro, accepts values from 2 to
5. Defaults to 2 for all environment sizes except mw1.micro, which defaults
to 1.
* v1 - Accepts 1. |
| sourceBucketARN
Optional | string
The Amazon Resource Name (ARN) of the Amazon S3 bucket where your DAG code
and supporting files are stored. For example, arn:aws:s3:::my-airflow-bucket-unique-name.
For more information, see Create an Amazon S3 bucket for Amazon MWAA (https://docs.aws.amazon.com/mwaa/latest/userguide/mwaa-s3-bucket.html).
Regex Pattern: ^arn:aws(-[a-z]+)?:s3:::[a-z0-9.\-]+$ |
| sourceBucketRef
Optional | object
AWSResourceReferenceWrapper provides a wrapper around *AWSResourceReference
type to provide more user friendly syntax for references using ‘from’ field
Ex:
APIIDRef:
from:
name: my-api |
| sourceBucketRef.from
Optional | object
AWSResourceReference provides all the values necessary to reference another
k8s resource for finding the identifier(Id/ARN/Name) |
| sourceBucketRef.from.name
Optional | string
|
| sourceBucketRef.from.namespace
Optional | string
|
| startupScriptS3ObjectVersion
Optional | string
The version of the startup shell script in your Amazon S3 bucket. You must
specify the version ID (https://docs.aws.amazon.com/AmazonS3/latest/userguide/versioning-workflows.html)
that Amazon S3 assigns to the file every time you update the script.
Version IDs are Unicode, UTF-8 encoded, URL-ready, opaque strings that are
no more than 1,024 bytes long. The following is an example:
3sL4kqtJlcpXroDTDmJ+rmSpXd3dIbrHY+MTRCxf3vjVBH40Nr8X8gdRQBpUMLUo
For more information, see Using a startup script (https://docs.aws.amazon.com/mwaa/latest/userguide/using-startup-script.html). |
| startupScriptS3Path
Optional | string
The relative path to the startup shell script in your Amazon S3 bucket. For
example, s3://mwaa-environment/startup.sh.
Amazon MWAA runs the script as your environment starts, and before running
the Apache Airflow process. You can use this script to install dependencies,
modify Apache Airflow configuration options, and set environment variables.
For more information, see Using a startup script (https://docs.aws.amazon.com/mwaa/latest/userguide/using-startup-script.html).
Regex Pattern: .* |
| tags
Optional | object
The key-value tag pairs you want to associate to your environment. For example,
“Environment”: “Staging”. For more information, see Tagging Amazon Web Services
resources (https://docs.aws.amazon.com/general/latest/gr/aws_tagging.html). |
| webserverAccessMode
Optional | string
Defines the access mode for the Apache Airflow web server. For more information,
see Apache Airflow access modes (https://docs.aws.amazon.com/mwaa/latest/userguide/configuring-networking.html). |
| weeklyMaintenanceWindowStart
Optional | string
The day and time of the week in Coordinated Universal Time (UTC) 24-hour
standard time to start weekly maintenance updates of your environment in
the following format: DAY:HH:MM. For example: TUE:03:30. You can specify
a start time in 30 minute increments only.
Regex Pattern: (MON|TUE|WED|THU|FRI|SAT|SUN):([01]\d|2[0-3]):(00|30) |
Status
ackResourceMetadata:
arn: string
ownerAccountID: string
partition: string
region: string
conditions:
- lastTransitionTime: string
message: string
reason: string
status: string
type: string
createdAt: string
lastUpdate:
createdAt: string
error:
errorCode: string
errorMessage: string
source: string
status: string
workerReplacementStrategy: string
status: string
webserverURL: string
| Field | Description |
|---|---|
| ackResourceMetadata Optional | object All CRs managed by ACK have a common Status.ACKResourceMetadata memberthat is used to contain resource sync state, account ownership, constructed ARN for the resource |
| ackResourceMetadata.arn Optional | string ARN is the Amazon Resource Name for the resource. This is a globally-unique identifier and is set only by the ACK service controller once the controller has orchestrated the creation of the resource OR when it has verified that an “adopted” resource (a resource where the ARN annotation was set by the Kubernetes user on the CR) exists and matches the supplied CR’s Spec field values. https://github.com/aws/aws-controllers-k8s/issues/270 |
| ackResourceMetadata.ownerAccountID Required | string OwnerAccountID is the AWS Account ID of the account that owns the backend AWS service API resource. |
| ackResourceMetadata.partition Optional | string Partition is the AWS partition in which the resource exists or will exist |
| ackResourceMetadata.region Required | string Region is the AWS region in which the resource exists or will exist. |
| conditions Optional | array All CRs managed by ACK have a common Status.Conditions member thatcontains a collection of ackv1alpha1.Condition objects that describethe various terminal states of the CR and its backend AWS service API resource |
| conditions.[] Required | object Condition is the common struct used by all CRDs managed by ACK service |
| controllers to indicate terminal states of the CR and its backend AWS | |
| service API resource | |
| conditions.[].message Optional | string A human readable message indicating details about the transition. |
| conditions.[].reason Optional | string The reason for the condition’s last transition. |
| conditions.[].status Optional | string Status of the condition, one of True, False, Unknown. |
| conditions.[].type Optional | string Type is the type of the Condition |
| createdAt Optional | string The day and time the environment was created. |
| lastUpdate Optional | object The status of the last update on the environment. |
| lastUpdate.createdAt Optional | string |
| lastUpdate.error Optional | object Describes the error(s) encountered with the last update of the environment. |
| lastUpdate.error.errorCode Optional | string |
| lastUpdate.error.errorMessage Optional | string |
| lastUpdate.source Optional | string |
| lastUpdate.status Optional | string |
| lastUpdate.workerReplacementStrategy Optional | string |
| status Optional | string The status of the Amazon MWAA environment. Valid values: * CREATING - Indicates the request to create the environment is in progress. * CREATING_SNAPSHOT - Indicates the request to update environment details, or upgrade the environment version, is in progress and Amazon MWAA is creating a storage volume snapshot of the Amazon RDS database cluster associated with the environment. A database snapshot is a backup created at a specific point in time. Amazon MWAA uses snapshots to recover environment metadata if the process to update or upgrade an environment fails. * CREATE_FAILED - Indicates the request to create the environment failed, and the environment could not be created. * AVAILABLE - Indicates the request was successful and the environment is ready to use. * PENDING - Indicates the request was successful, but the process to create the environment is paused until you create the required VPC endpoints in your VPC. After you create the VPC endpoints, the process resumes. * UPDATING - Indicates the request to update the environment is in progress. * ROLLING_BACK - Indicates the request to update environment details, or upgrade the environment version, failed and Amazon MWAA is restoring the environment using the latest storage volume snapshot. * DELETING - Indicates the request to delete the environment is in progress. * DELETED - Indicates the request to delete the environment is complete, and the environment has been deleted. * UNAVAILABLE - Indicates the request failed, but the environment did not return to its previous state and is not stable. * UPDATE_FAILED - Indicates the request to update the environment failed, and the environment was restored to its previous state successfully and is ready to use. * MAINTENANCE - Indicates that the environment is undergoing maintenance. Depending on the type of work Amazon MWAA is performing, your environment might become unavailable during this process. After all operations are done, your environment will return to its status prior to mainteneace operations. We recommend reviewing our troubleshooting guide for a list of common errors and their solutions. For more information, see Amazon MWAA troubleshooting (https://docs.aws.amazon.com/mwaa/latest/userguide/troubleshooting.html). |
| webserverURL Optional | string The Apache Airflow web server host name for the Amazon MWAA environment. For more information, see Accessing the Apache Airflow UI (https://docs.aws.amazon.com/mwaa/latest/userguide/access-airflow-ui.html). Regex Pattern: ^https://.+$ |