-
Notifications
You must be signed in to change notification settings - Fork 28
Description
Describe the bug
Creating an item where a property is an array of objects (with many objects inside) fails with a 500 Internal Server Error illegal_argument_exception
when the Elasticsearch index does not already exist.
If a smaller version of the item (containing only a few objects in that same array) is submitted first, the index is created successfully and the larger item can then be ingested without issues.
This suggests that Elasticsearch dynamically maps the field incorrectly (e.g., as a string or keyword) when the array is large, leading to a type conflict on subsequent inserts.
To Reproduce
- Start STAC API backed by Elasticsearch (with no existing index).
- Submit an item with a property like:
"properties": { "tags": [ { "key": "landcover", "value": "forest" }, { "key": "confidence", "value": "high" }, { "key": "source", "value": "satellite" } // ... many more objects ] }
- Observe a 500 error due to type conflict:
BadRequestError(400, 'illegal_argument_exception', "can't merge a non object mapping [properties.tags] with an object mapping")
- Submit a similar item, but with only one or two
tags
objects. - Retry the original item and it will work now.
Expected behavior
The API should support valid arrays of objects regardless of their size or whether the index exists beforehand.
Logs
HTTPError('500 Server Error: Internal Server Error')
{
"code": "BadRequestError",
"description": "BadRequestError(400, 'illegal_argument_exception', \"can't merge a non object mapping [properties.tags] with an object mapping\")"
}
Environment
- STAC API version:
ghcr.io/stac-utils/stac-fastapi-es:v6.0.0
- Elasticsearch version:
docker.elastic.co/elasticsearch/elasticsearch:8.11.0
- Deployment method: docker-compose on Linux
24.04.2 LTS (Noble Numbat)
Additional context
This seems to be caused by Elasticsearch's dynamic mapping feature incorrectly inferring the field type when a complex array is first encountered. A workaround is to predefine the mapping or submit a smaller example item first but this behavior is non-obvious and error-prone.