Skip to content

Commit 3547adf

Browse files
authored
Rebuild with airflow 2.4.2 (apache#51)
1 parent dcd32b7 commit 3547adf

File tree

287 files changed

+7441
-315
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

287 files changed

+7441
-315
lines changed

README.md

Lines changed: 11 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -19,8 +19,8 @@
1919

2020
# Apache Airflow Python Client
2121

22-
> **_NOTE:_** The Apache Airflow Client is still under active development and some methods
23-
> or APIs might be broken. Please raise an issue in github if you encounter any such issues.
22+
> **_NOTE:_** The Apache Airflow Client is still under active development and some methods
23+
> or APIs might be broken. Please raise an issue in github if you encounter any such issues.
2424
2525

2626

@@ -93,7 +93,10 @@ See [README](./airflow_client/README.md#documentation-for-api-endpoints) for ful
9393

9494
## Release Process
9595

96-
The Python client is generated using Airflow's [openapi spec](https://github.com/apache/airflow/blob/master/clients/gen/python.sh).
96+
Typically releases are done coinciding with major and minor releases to Airflow. Therefore a release of (for example) 2.3.0 of this client
97+
would correspond with 2.3.X of Airflow.
98+
99+
The Python client is generated using Airflow's [openapi spec](https://github.com/apache/airflow/blob/master/clients/gen/python.sh).
97100
To update the client for new APIs do the following steps:
98101
99102
```bash
@@ -106,18 +109,20 @@ git clone git@github.com:apache/airflow.git
106109
Edit the file `airflow/airflow/api_connexion/openapi/v1.yaml`
107110
Make sure it has the following `securitySchema`s listed under security `section`
108111
```yaml
109-
security:
112+
security:
110113
- Basic: []
111114
- GoogleOpenId: []
112115
- Kerberos: []
113116
```
114117
If your deployment of Airflow uses any different authentication mechanism than the three listed above, you might need to make further changes to the `v1.yaml` and generate your own client, see [OpenAPI Schema specification](https://swagger.io/docs/specification/authentication/) for details.
115118
(*These changes should not be commited to the upstream `v1.yaml` [as it will generate misleading openapi documentaion](https://github.com/apache/airflow/pull/17174)*)
116119
117-
```bash
120+
Update the VERSION string in setup.py with the corresponding new version string.
121+
122+
```bash
118123
cd airflow
119124
120-
# bump up the version in python.sh & run the following command
125+
# bump up the version in python.sh & run the following command
121126
./clients/gen/python.sh airflow/api_connexion/openapi/v1.yaml ../airflow-client-python/airflow_client
122127
123128
# raise a PR in github for both the repos (airflow & airflow-client-python)

airflow_client/.openapi-generator/FILES

Lines changed: 48 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,8 @@ client/api/config_api.py
66
client/api/connection_api.py
77
client/api/dag_api.py
88
client/api/dag_run_api.py
9+
client/api/dag_warning_api.py
10+
client/api/dataset_api.py
911
client/api/event_log_api.py
1012
client/api/import_error_api.py
1113
client/api/monitoring_api.py
@@ -27,8 +29,10 @@ client/model/action.py
2729
client/model/action_collection.py
2830
client/model/action_collection_all_of.py
2931
client/model/action_resource.py
32+
client/model/basic_dag_run.py
3033
client/model/class_reference.py
31-
client/model/clear_task_instance.py
34+
client/model/clear_dag_run.py
35+
client/model/clear_task_instances.py
3236
client/model/collection_info.py
3337
client/model/color.py
3438
client/model/config.py
@@ -49,7 +53,17 @@ client/model/dag_detail_all_of.py
4953
client/model/dag_run.py
5054
client/model/dag_run_collection.py
5155
client/model/dag_run_collection_all_of.py
56+
client/model/dag_schedule_dataset_reference.py
5257
client/model/dag_state.py
58+
client/model/dag_warning.py
59+
client/model/dag_warning_collection.py
60+
client/model/dag_warning_collection_all_of.py
61+
client/model/dataset.py
62+
client/model/dataset_collection.py
63+
client/model/dataset_collection_all_of.py
64+
client/model/dataset_event.py
65+
client/model/dataset_event_collection.py
66+
client/model/dataset_event_collection_all_of.py
5367
client/model/error.py
5468
client/model/event_log.py
5569
client/model/event_log_collection.py
@@ -91,6 +105,7 @@ client/model/task_instance_collection.py
91105
client/model/task_instance_collection_all_of.py
92106
client/model/task_instance_reference.py
93107
client/model/task_instance_reference_collection.py
108+
client/model/task_outlet_dataset_reference.py
94109
client/model/task_state.py
95110
client/model/time_delta.py
96111
client/model/trigger_rule.py
@@ -121,8 +136,10 @@ docs/Action.md
121136
docs/ActionCollection.md
122137
docs/ActionCollectionAllOf.md
123138
docs/ActionResource.md
139+
docs/BasicDAGRun.md
124140
docs/ClassReference.md
125-
docs/ClearTaskInstance.md
141+
docs/ClearDagRun.md
142+
docs/ClearTaskInstances.md
126143
docs/CollectionInfo.md
127144
docs/Color.md
128145
docs/Config.md
@@ -147,7 +164,19 @@ docs/DAGRun.md
147164
docs/DAGRunApi.md
148165
docs/DAGRunCollection.md
149166
docs/DAGRunCollectionAllOf.md
167+
docs/DagScheduleDatasetReference.md
150168
docs/DagState.md
169+
docs/DagWarning.md
170+
docs/DagWarningApi.md
171+
docs/DagWarningCollection.md
172+
docs/DagWarningCollectionAllOf.md
173+
docs/Dataset.md
174+
docs/DatasetApi.md
175+
docs/DatasetCollection.md
176+
docs/DatasetCollectionAllOf.md
177+
docs/DatasetEvent.md
178+
docs/DatasetEventCollection.md
179+
docs/DatasetEventCollectionAllOf.md
151180
docs/Error.md
152181
docs/EventLog.md
153182
docs/EventLogApi.md
@@ -198,6 +227,7 @@ docs/TaskInstanceCollection.md
198227
docs/TaskInstanceCollectionAllOf.md
199228
docs/TaskInstanceReference.md
200229
docs/TaskInstanceReferenceCollection.md
230+
docs/TaskOutletDatasetReference.md
201231
docs/TaskState.md
202232
docs/TimeDelta.md
203233
docs/TriggerRule.md
@@ -229,8 +259,10 @@ test/test_action.py
229259
test/test_action_collection.py
230260
test/test_action_collection_all_of.py
231261
test/test_action_resource.py
262+
test/test_basic_dag_run.py
232263
test/test_class_reference.py
233-
test/test_clear_task_instance.py
264+
test/test_clear_dag_run.py
265+
test/test_clear_task_instances.py
234266
test/test_collection_info.py
235267
test/test_color.py
236268
test/test_config.py
@@ -255,7 +287,19 @@ test/test_dag_run.py
255287
test/test_dag_run_api.py
256288
test/test_dag_run_collection.py
257289
test/test_dag_run_collection_all_of.py
290+
test/test_dag_schedule_dataset_reference.py
258291
test/test_dag_state.py
292+
test/test_dag_warning.py
293+
test/test_dag_warning_api.py
294+
test/test_dag_warning_collection.py
295+
test/test_dag_warning_collection_all_of.py
296+
test/test_dataset.py
297+
test/test_dataset_api.py
298+
test/test_dataset_collection.py
299+
test/test_dataset_collection_all_of.py
300+
test/test_dataset_event.py
301+
test/test_dataset_event_collection.py
302+
test/test_dataset_event_collection_all_of.py
259303
test/test_error.py
260304
test/test_event_log.py
261305
test/test_event_log_api.py
@@ -306,6 +350,7 @@ test/test_task_instance_collection.py
306350
test/test_task_instance_collection_all_of.py
307351
test/test_task_instance_reference.py
308352
test/test_task_instance_reference_collection.py
353+
test/test_task_outlet_dataset_reference.py
309354
test/test_task_state.py
310355
test/test_time_delta.py
311356
test/test_trigger_rule.py

airflow_client/README.md

Lines changed: 22 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -230,7 +230,7 @@ fulfilling the request.
230230

231231
This Python package is automatically generated by the [OpenAPI Generator](https://openapi-generator.tech) project:
232232

233-
- API version: 1.0.0
233+
- API version: 2.3.0
234234
- Package version: 2.3.0
235235
- Build package: org.openapitools.codegen.languages.PythonClientCodegen
236236
For more information, please visit [https://airflow.apache.org](https://airflow.apache.org)
@@ -335,12 +335,19 @@ Class | Method | HTTP request | Description
335335
*DAGApi* | [**patch_dags**](docs/DAGApi.md#patch_dags) | **PATCH** /dags | Update DAGs
336336
*DAGApi* | [**post_clear_task_instances**](docs/DAGApi.md#post_clear_task_instances) | **POST** /dags/{dag_id}/clearTaskInstances | Clear a set of task instances
337337
*DAGApi* | [**post_set_task_instances_state**](docs/DAGApi.md#post_set_task_instances_state) | **POST** /dags/{dag_id}/updateTaskInstancesState | Set a state of task instances
338+
*DAGRunApi* | [**clear_dag_run**](docs/DAGRunApi.md#clear_dag_run) | **POST** /dags/{dag_id}/dagRuns/{dag_run_id}/clear | Clear a DAG run
338339
*DAGRunApi* | [**delete_dag_run**](docs/DAGRunApi.md#delete_dag_run) | **DELETE** /dags/{dag_id}/dagRuns/{dag_run_id} | Delete a DAG run
339340
*DAGRunApi* | [**get_dag_run**](docs/DAGRunApi.md#get_dag_run) | **GET** /dags/{dag_id}/dagRuns/{dag_run_id} | Get a DAG run
340341
*DAGRunApi* | [**get_dag_runs**](docs/DAGRunApi.md#get_dag_runs) | **GET** /dags/{dag_id}/dagRuns | List DAG runs
341342
*DAGRunApi* | [**get_dag_runs_batch**](docs/DAGRunApi.md#get_dag_runs_batch) | **POST** /dags/~/dagRuns/list | List DAG runs (batch)
343+
*DAGRunApi* | [**get_upstream_dataset_events**](docs/DAGRunApi.md#get_upstream_dataset_events) | **GET** /dags/{dag_id}/dagRuns/{dag_run_id}/upstreamDatasetEvents | Get dataset events for a DAG run
342344
*DAGRunApi* | [**post_dag_run**](docs/DAGRunApi.md#post_dag_run) | **POST** /dags/{dag_id}/dagRuns | Trigger a new DAG run
343345
*DAGRunApi* | [**update_dag_run_state**](docs/DAGRunApi.md#update_dag_run_state) | **PATCH** /dags/{dag_id}/dagRuns/{dag_run_id} | Modify a DAG run
346+
*DagWarningApi* | [**get_dag_warnings**](docs/DagWarningApi.md#get_dag_warnings) | **GET** /dagWarnings | List dag warnings
347+
*DatasetApi* | [**get_dataset**](docs/DatasetApi.md#get_dataset) | **GET** /datasets/{uri} | Get a dataset
348+
*DatasetApi* | [**get_dataset_events**](docs/DatasetApi.md#get_dataset_events) | **GET** /datasets/events | Get dataset events
349+
*DatasetApi* | [**get_datasets**](docs/DatasetApi.md#get_datasets) | **GET** /datasets | List datasets
350+
*DatasetApi* | [**get_upstream_dataset_events**](docs/DatasetApi.md#get_upstream_dataset_events) | **GET** /dags/{dag_id}/dagRuns/{dag_run_id}/upstreamDatasetEvents | Get dataset events for a DAG run
344351
*EventLogApi* | [**get_event_log**](docs/EventLogApi.md#get_event_log) | **GET** /eventLogs/{event_log_id} | Get a log entry
345352
*EventLogApi* | [**get_event_logs**](docs/EventLogApi.md#get_event_logs) | **GET** /eventLogs | List log entries
346353
*ImportErrorApi* | [**get_import_error**](docs/ImportErrorApi.md#get_import_error) | **GET** /importErrors/{import_error_id} | Get an import error
@@ -387,8 +394,10 @@ Class | Method | HTTP request | Description
387394
- [ActionCollection](docs/ActionCollection.md)
388395
- [ActionCollectionAllOf](docs/ActionCollectionAllOf.md)
389396
- [ActionResource](docs/ActionResource.md)
397+
- [BasicDAGRun](docs/BasicDAGRun.md)
390398
- [ClassReference](docs/ClassReference.md)
391-
- [ClearTaskInstance](docs/ClearTaskInstance.md)
399+
- [ClearDagRun](docs/ClearDagRun.md)
400+
- [ClearTaskInstances](docs/ClearTaskInstances.md)
392401
- [CollectionInfo](docs/CollectionInfo.md)
393402
- [Color](docs/Color.md)
394403
- [Config](docs/Config.md)
@@ -409,7 +418,17 @@ Class | Method | HTTP request | Description
409418
- [DAGRun](docs/DAGRun.md)
410419
- [DAGRunCollection](docs/DAGRunCollection.md)
411420
- [DAGRunCollectionAllOf](docs/DAGRunCollectionAllOf.md)
421+
- [DagScheduleDatasetReference](docs/DagScheduleDatasetReference.md)
412422
- [DagState](docs/DagState.md)
423+
- [DagWarning](docs/DagWarning.md)
424+
- [DagWarningCollection](docs/DagWarningCollection.md)
425+
- [DagWarningCollectionAllOf](docs/DagWarningCollectionAllOf.md)
426+
- [Dataset](docs/Dataset.md)
427+
- [DatasetCollection](docs/DatasetCollection.md)
428+
- [DatasetCollectionAllOf](docs/DatasetCollectionAllOf.md)
429+
- [DatasetEvent](docs/DatasetEvent.md)
430+
- [DatasetEventCollection](docs/DatasetEventCollection.md)
431+
- [DatasetEventCollectionAllOf](docs/DatasetEventCollectionAllOf.md)
413432
- [Error](docs/Error.md)
414433
- [EventLog](docs/EventLog.md)
415434
- [EventLogCollection](docs/EventLogCollection.md)
@@ -451,6 +470,7 @@ Class | Method | HTTP request | Description
451470
- [TaskInstanceCollectionAllOf](docs/TaskInstanceCollectionAllOf.md)
452471
- [TaskInstanceReference](docs/TaskInstanceReference.md)
453472
- [TaskInstanceReferenceCollection](docs/TaskInstanceReferenceCollection.md)
473+
- [TaskOutletDatasetReference](docs/TaskOutletDatasetReference.md)
454474
- [TaskState](docs/TaskState.md)
455475
- [TimeDelta](docs/TimeDelta.md)
456476
- [TriggerRule](docs/TriggerRule.md)

airflow_client/client/__init__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@
2222
2323
# Overview To facilitate management, Apache Airflow supports a range of REST API endpoints across its objects. This section provides an overview of the API design, methods, and supported use cases. Most of the endpoints accept `JSON` as input and return `JSON` responses. This means that you must usually add the following headers to your request: ``` Content-type: application/json Accept: application/json ``` ## Resources The term `resource` refers to a single type of object in the Airflow metadata. An API is broken up by its endpoint's corresponding resource. The name of a resource is typically plural and expressed in camelCase. Example: `dagRuns`. Resource names are used as part of endpoint URLs, as well as in API parameters and responses. ## CRUD Operations The platform supports **C**reate, **R**ead, **U**pdate, and **D**elete operations on most resources. You can review the standards for these operations and their standard parameters below. Some endpoints have special behavior as exceptions. ### Create To create a resource, you typically submit an HTTP `POST` request with the resource's required metadata in the request body. The response returns a `201 Created` response code upon success with the resource's metadata, including its internal `id`, in the response body. ### Read The HTTP `GET` request can be used to read a resource or to list a number of resources. A resource's `id` can be submitted in the request parameters to read a specific resource. The response usually returns a `200 OK` response code upon success, with the resource's metadata in the response body. If a `GET` request does not include a specific resource `id`, it is treated as a list request. The response usually returns a `200 OK` response code upon success, with an object containing a list of resources' metadata in the response body. When reading resources, some common query parameters are usually available. e.g.: ``` v1/connections?limit=25&offset=25 ``` |Query Parameter|Type|Description| |---------------|----|-----------| |limit|integer|Maximum number of objects to fetch. Usually 25 by default| |offset|integer|Offset after which to start returning objects. For use with limit query parameter.| ### Update Updating a resource requires the resource `id`, and is typically done using an HTTP `PATCH` request, with the fields to modify in the request body. The response usually returns a `200 OK` response code upon success, with information about the modified resource in the response body. ### Delete Deleting a resource requires the resource `id` and is typically executing via an HTTP `DELETE` request. The response usually returns a `204 No Content` response code upon success. ## Conventions - Resource names are plural and expressed in camelCase. - Names are consistent between URL parameter name and field name. - Field names are in snake_case. ```json { \"name\": \"string\", \"slots\": 0, \"occupied_slots\": 0, \"used_slots\": 0, \"queued_slots\": 0, \"open_slots\": 0 } ``` ### Update Mask Update mask is available as a query parameter in patch endpoints. It is used to notify the API which fields you want to update. Using `update_mask` makes it easier to update objects by helping the server know which fields to update in an object instead of updating all fields. The update request ignores any fields that aren't specified in the field mask, leaving them with their current values. Example: ``` resource = request.get('/resource/my-id').json() resource['my_field'] = 'new-value' request.patch('/resource/my-id?update_mask=my_field', data=json.dumps(resource)) ``` ## Versioning and Endpoint Lifecycle - API versioning is not synchronized to specific releases of the Apache Airflow. - APIs are designed to be backward compatible. - Any changes to the API will first go through a deprecation phase. # Trying the API You can use a third party client, such as [curl](https://curl.haxx.se/), [HTTPie](https://httpie.org/), [Postman](https://www.postman.com/) or [the Insomnia rest client](https://insomnia.rest/) to test the Apache Airflow API. Note that you will need to pass credentials data. For e.g., here is how to pause a DAG with [curl](https://curl.haxx.se/), when basic authorization is used: ```bash curl -X PATCH 'https://example.com/api/v1/dags/{dag_id}?update_mask=is_paused' \\ -H 'Content-Type: application/json' \\ --user \"username:password\" \\ -d '{ \"is_paused\": true }' ``` Using a graphical tool such as [Postman](https://www.postman.com/) or [Insomnia](https://insomnia.rest/), it is possible to import the API specifications directly: 1. Download the API specification by clicking the **Download** button at top of this document 2. Import the JSON specification in the graphical tool of your choice. - In *Postman*, you can click the **import** button at the top - With *Insomnia*, you can just drag-and-drop the file on the UI Note that with *Postman*, you can also generate code snippets by selecting a request and clicking on the **Code** button. ## Enabling CORS [Cross-origin resource sharing (CORS)](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS) is a browser security feature that restricts HTTP requests that are initiated from scripts running in the browser. For details on enabling/configuring CORS, see [Enabling CORS](https://airflow.apache.org/docs/apache-airflow/stable/security/api.html). # Authentication To be able to meet the requirements of many organizations, Airflow supports many authentication methods, and it is even possible to add your own method. If you want to check which auth backend is currently set, you can use `airflow config get-value api auth_backends` command as in the example below. ```bash $ airflow config get-value api auth_backends airflow.api.auth.backend.basic_auth ``` The default is to deny all requests. For details on configuring the authentication, see [API Authorization](https://airflow.apache.org/docs/apache-airflow/stable/security/api.html). # Errors We follow the error response format proposed in [RFC 7807](https://tools.ietf.org/html/rfc7807) also known as Problem Details for HTTP APIs. As with our normal API responses, your client must be prepared to gracefully handle additional members of the response. ## Unauthenticated This indicates that the request has not been applied because it lacks valid authentication credentials for the target resource. Please check that you have valid credentials. ## PermissionDenied This response means that the server understood the request but refuses to authorize it because it lacks sufficient rights to the resource. It happens when you do not have the necessary permission to execute the action you performed. You need to get the appropriate permissions in other to resolve this error. ## BadRequest This response means that the server cannot or will not process the request due to something that is perceived to be a client error (e.g., malformed request syntax, invalid request message framing, or deceptive request routing). To resolve this, please ensure that your syntax is correct. ## NotFound This client error response indicates that the server cannot find the requested resource. ## MethodNotAllowed Indicates that the request method is known by the server but is not supported by the target resource. ## NotAcceptable The target resource does not have a current representation that would be acceptable to the user agent, according to the proactive negotiation header fields received in the request, and the server is unwilling to supply a default representation. ## AlreadyExists The request could not be completed due to a conflict with the current state of the target resource, e.g. the resource it tries to create already exists. ## Unknown This means that the server encountered an unexpected condition that prevented it from fulfilling the request. # noqa: E501
2424
25-
The version of the OpenAPI document: 1.0.0
25+
The version of the OpenAPI document: 2.3.0
2626
Contact: dev@airflow.apache.org
2727
Generated by: https://openapi-generator.tech
2828
"""

0 commit comments

Comments
 (0)
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy