Skip to content

Commit 27615fd

Browse files
Suppress FileNotFoundError when deleting keys in the obstore adapter (#3140)
* Suppress FileNotFoundError when deleting keys in the obstore adapter * Add unit test * Document bugfix changes * Run delete nonexistent key test across all stores * Suppress only builtins.FileNotFoundError in obstore.delete adapter --------- Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
1 parent 5731c6c commit 27615fd

File tree

3 files changed

+20
-1
lines changed

3 files changed

+20
-1
lines changed

changes/3140.bugfix.rst

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
Suppress `FileNotFoundError` when deleting non-existent keys in the `obstore` adapter.
2+
3+
When writing empty chunks (i.e. chunks where all values are equal to the array's fill value) to a zarr array, zarr
4+
will delete those chunks from the underlying store. For zarr arrays backed by the `obstore` adapter, this will potentially
5+
raise a `FileNotFoundError` if the chunk doesn't already exist.
6+
Since whether or not a delete of a non-existing object raises an error depends on the behavior of the underlying store,
7+
suppressing the error in all cases results in consistent behavior across stores, and is also what `zarr` seems to expect
8+
from the store.

src/zarr/storage/_obstore.py

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -188,7 +188,13 @@ async def delete(self, key: str) -> None:
188188
import obstore as obs
189189

190190
self._check_writable()
191-
await obs.delete_async(self.store, key)
191+
192+
# Some obstore stores such as local filesystems, GCP and Azure raise an error
193+
# when deleting a non-existent key, while others such as S3 and in-memory do
194+
# not. We suppress the error to make the behavior consistent across all obstore
195+
# stores. This is also in line with the behavior of the other Zarr store adapters.
196+
with contextlib.suppress(FileNotFoundError):
197+
await obs.delete_async(self.store, key)
192198

193199
@property
194200
def supports_partial_writes(self) -> bool:

src/zarr/testing/store.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -401,6 +401,11 @@ async def test_delete_dir(self, store: S) -> None:
401401
assert not await store.exists("foo/zarr.json")
402402
assert not await store.exists("foo/c/0")
403403

404+
async def test_delete_nonexistent_key_does_not_raise(self, store: S) -> None:
405+
if not store.supports_deletes:
406+
pytest.skip("store does not support deletes")
407+
await store.delete("nonexistent_key")
408+
404409
async def test_is_empty(self, store: S) -> None:
405410
assert await store.is_empty("")
406411
await self.set(

0 commit comments

Comments
 (0)
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy