Skip to content

Update Python Client to 3.0.1 #128

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 19, 2025
Merged

Update Python Client to 3.0.1 #128

merged 1 commit into from
May 19, 2025

Conversation

kaxil
Copy link
Member

@kaxil kaxil commented May 16, 2025

This brings in the new Airflow API v2 to the Python client

As part of this change the following breaking changes have occurred:

  • The API returns 422 status code instead of 400 for validation errors.

    For instance when the request payload, path params, or query params are invalid.

  • When listing a resource for instance on GET /dags, fields parameter is not supported anymore to obtain a partial response.

    The full objects will be returned by the endpoint. This feature might be added back in upcoming 3.x versions.

  • Passing list in query parameters switched from form, non exploded to form, exploded i.e before ?my_list=item1,item2 now ?my_list=item1&my_list=item2

  • execution_date was deprecated and has been removed. Any payload or parameter mentioning this field has been removed.

  • Datetime format are RFC3339-compliant in FastAPI, more permissive than ISO8601, meaning that the API returns zulu datetime for responses, more info here Why do the generated docs use Z-terminated ISO strings when python doesn't? fastapi/fastapi#7693 (comment). Both Z and 00+xx are supported for payload and params.

    This is due FastAPI and pydantic v2 default behavior.

  • PATCH on DagRun and TaskInstance are more generic and allow in addition to update the resource state and the note content.

    Therefore the two legacy dedicated endpoints to update a DagRun note and TaskInstance note have been removed.

    Same for the set task instance state, it is now handled by the broader PATCH on task instances.

  • assets/queuedEvent endpoints have moved to assets/queuedEvents for consistency.

  • dag_parsing endpoint now returns a 409 when the DagPriorityParsingRequest already exists. It was returning 201 before.

  • clearTaskInstances endpoint default value for reset_dag_runs field has been updated from False to True.

  • Pool name can't be modified in the PATCH pool endpoint anymore. Pool name shouldn't be updated via pool PATCH API call.

  • Logical date is now a nullable. In addition it is a nullable required payload field for Triggering a DagRun endpoint.

@potiuk
Copy link
Member

potiuk commented May 16, 2025

Just one comment @kaxil -> You know we are using this repo just to keep archive of generated code and releasing is done directly from airflow repo? Just want to make sure that you have not missed that part while you were not doing releases

Copy link
Member

@potiuk potiuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to increase Python version to 3.9 - the error is due to Python 3.8 still used.

@kaxil
Copy link
Member Author

kaxil commented May 16, 2025

Just one comment @kaxil -> You know we are using this repo just to keep archive of generated code and releasing is done directly from airflow repo? Just want to make sure that you have not missed that part while you were not doing releases

Yup :)

You need to increase Python version to 3.9 - the error is due to Python 3.8 still used.

ah, thanks

Copy link
Member

@pierrejeambrun pierrejeambrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice thanks, just one question.

This brings in the new Airflow API v2 to the Python client

As part of this change the following breaking changes have occurred:

- The API returns 422 status code instead of 400 for validation errors.

  For instance when the request payload, path params, or query params are invalid.

- When listing a resource for instance on GET ``/dags``, ``fields`` parameter is not supported anymore to obtain a partial response.

  The full objects will be returned by the endpoint. This feature might be added back in upcoming 3.x versions.

- Passing list in query parameters switched from ``form, non exploded`` to ``form, exploded``
  i.e before ``?my_list=item1,item2`` now ``?my_list=item1&my_list=item2``

- ``execution_date`` was deprecated and has been removed. Any payload or parameter mentioning this field has been removed.

- Datetime format are RFC3339-compliant in FastAPI, more permissive than ISO8601,
  meaning that the API returns zulu datetime for responses, more info here fastapi/fastapi#7693 (comment).
  Both ``Z`` and ``00+xx`` are supported for payload and params.

  This is due FastAPI and pydantic v2 default behavior.

- PATCH on ``DagRun`` and ``TaskInstance`` are more generic and allow in addition to update the resource state and the note content.

  Therefore the two legacy dedicated endpoints to update a ``DagRun`` note and ``TaskInstance`` note have been removed.

  Same for the set task instance state, it is now handled by the broader PATCH on task instances.

- ``assets/queuedEvent`` endpoints have moved to ``assets/queuedEvents`` for consistency.

- dag_parsing endpoint now returns a 409 when the DagPriorityParsingRequest already exists. It was returning 201 before.

- ``clearTaskInstances`` endpoint default value for ``reset_dag_runs`` field has been updated from ``False`` to ``True``.

- Pool name can't be modified in the PATCH pool endpoint anymore. Pool name shouldn't be updated via pool PATCH API call.

- Logical date is now a nullable. In addition it is a nullable required payload field for Triggering a DagRun endpoint.
@kaxil kaxil merged commit 56c1242 into main May 19, 2025
1 check passed
Copy link
Member

@pierrejeambrun pierrejeambrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, I was able to successfully run test_python_client.py against my breeze API.

At some point we might want to rewrite it because it's not runnable in this current state. (pytest is missing from deps, generate_access_token is actually coming from airflow core, so you have to vendor this in to be able to actually run it, which shouldn't be needed.)

@kaxil
Copy link
Member Author

kaxil commented May 19, 2025

Nice, I was able to successfully run test_python_client.py against my breeze API.

At some point we might want to rewrite it because it's not runnable in this current state. (pytest is missing from deps, generate_access_token is actually coming from airflow core, so you have to vendor this in to be able to actually run it, which shouldn't be needed.)

Agreed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy