-
Notifications
You must be signed in to change notification settings - Fork 319
feat: adds time_zone to external config and load job #2229
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
tests/unit/test_external_config.py
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In addition to testing to_api_repr(), I think we also need to test against the class property itself, and also from_api_repr().
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the original author of this test file (specifically test_external_config.py
) designed it to perform the tests you are discussing, even if it is not immediately apparent and would argue the current state is satisfactory:
- the new attribute (
time_zone
) is tested intest_to_api_repr_base()
- the new attribute is tested in
test_from_api_repr_base()
- during the above tests, we also explicitly test the getter and setter for the new attribute
If we decide that this pattern/philosophy is not satisfactory, then it seems that we will need to completely rewrite this file from top to bottom.
See below for a walkthrough:
TestExternalConfig.test_to_api_repr_base()
- We create an
ExternalConfig
object (ec
) from theExternalConfig
class - We then set each class attribute individually (
ec.autodetect=True
,ec.time_zone=self.TIME_ZONE
, etc)- by default, this tests the setter for each attribute covered by
BASE_RESOURCE
includingtime_zone
- by default, this tests the setter for each attribute covered by
- We use
ec.to_api_repr()
to creategot_resource
. This tests the ability to useto_api_repr()
- We compare
got_resource
toexp_resource
(which includes all the attributes we just set) ensuring thatto_api_repr()
performed as expected
TestExternalConfig.test_from_api_repr_base()
- Also relies on
BASE_RESOURCE
to facilitate testing - we use
from_api_repr()
to create anExternalConfig
object (ec
) based onBASE_RESOURCE
#L43 - we use
_verify_base()
to ensure that theec
object matches the expected values fromBASE_RESOURCE
#L44 & #L129- by default, this tests the getter for each attribute covered by
BASE_RESOURCE
includingtime_zone
- it is intended to check every single item in
BASE_RESOURCE
(as long as it andBASE_RESOURCE
are in sync)
- by default, this tests the getter for each attribute covered by
- we then update the
api_repr
with more elements (schema
andfields
, etc) #L51 - we use
_verify_base()
again to ensure that the expected values fromBASE_RESOURCE
are present along with the additional elements #L67
If we look at the remainder of the file, we see this pattern repeated several times. Looking at:
test_from_api_repr_csv
test_to_api_repr_csv
test_from_api_repr_bigtable
test_to_api_repr_bigtable
we see this philosophy and pattern repeated almost exactly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Regarding adding a to_api_repr
and from_api_repr
pair, there may be some merit to creating a single test that includes all the attributes as part of a BASE_RESOURCE
much as we see in test_external_config.py
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Similar to https://github.com/googleapis/python-bigquery/pull/2229/files#r2191183829, I think in general we need to test against to/from_api_repr and the property itself. The same goes for LoadJobConfig
and LoadJob
. Could you add these changes? Otherwise the PR looks good in general.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The pattern/philosophy for test_load_config.py
is different than the approach used in test_external_config.py
Here, the author chose to test each attribute in more of a stand alone fashion using a trifecta of tests:
x_missing()
x_hit()
x_setter()
But:
the author did not test to_api_repr()
or from_api_repr()
directly (I can add such a pair of tests).
Examples:
def test_write_disposition_missing()
def test_write_disposition_hit()
def test_write_disposition_setter()
def test_time_zone_missing()
def test_time_zone_hit()
def test_time_zone_setter()
missing
Confirms that when the target class is created, the attribute is not populated
self.assertIsNone(config.write_disposition)`
hit
Sets (behind the scenes using ._properties
) the value of the attribute
Confirms that the getter functions correctly and that the expected/result values match
config._properties["load"]["writeDisposition"] = write_disposition
self.assertEqual(config.write_disposition, write_disposition)
setter
Sets the value of the attribute directly using the setter function
Confirms that the setter functions correctly by examining the _properties
object directly
config.write_disposition = write_disposition
self.assertEqual(config._properties["load"]["writeDisposition"], write_disposition)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added a test_to_api_repr()
and a test_from_api_repr()
test because the original file did not include them.
This commit introduces new configuration options for BigQuery load jobs and external table definitions, aligning with recent updates to the underlying protos.
New options added:
time_zone
: Time zone used when parsing timestamp values that do not have specific time zone information. (Applies toLoadJobConfig
,LoadJob
, andExternalConfig
)Changes include:
Added corresponding properties (getters/setters) to
LoadJobConfig
,LoadJob
, andExternalConfig
.Updated docstrings and type hints for all new attributes.
Updated unit tests to cover the new options, ensuring they are correctly handled during object initialization, serialization to API representation, and deserialization from API responses.