Skip to content

Support vertical sharding for parquet queryable #6879

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jul 28, 2025

Conversation

yeya24
Copy link
Contributor

@yeya24 yeya24 commented Jul 14, 2025

What this PR does:

This PR implements vertical sharding for parquet queryable by leveraging a series label filtering hook in parquet-common. See prometheus-community/parquet-common#83. Since that PR is still not merged so mark this PR as draft.

Integration test for parquet querier has been updated to enable vertical sharding.

Which issue(s) this PR fixes:
Fixes #

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

@yeya24 yeya24 force-pushed the parquet-vertical-sharding branch from 1c79456 to 0c64f1b Compare July 14, 2025 04:59
@yeya24 yeya24 marked this pull request as ready for review July 15, 2025 17:44
@dosubot dosubot bot added component/querier go Pull requests that update Go code labels Jul 15, 2025
Copy link
Member

@alanprot alanprot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing! thanks

@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Jul 15, 2025
@yeya24
Copy link
Contributor Author

yeya24 commented Jul 16, 2025

The fuzz failure seems related. I will spend more time investigating it

@yeya24 yeya24 mentioned this pull request Jul 21, 2025
3 tasks
@yeya24 yeya24 force-pushed the parquet-vertical-sharding branch 4 times, most recently from efb09e0 to bc6e293 Compare July 27, 2025 19:14
yeya24 added 3 commits July 27, 2025 12:36
Signed-off-by: yeya24 <benye@amazon.com>
Signed-off-by: yeya24 <benye@amazon.com>
@yeya24 yeya24 force-pushed the parquet-vertical-sharding branch from bc6e293 to a3bc1f0 Compare July 27, 2025 19:44
yeya24 added 4 commits July 27, 2025 19:27
Signed-off-by: yeya24 <benye@amazon.com>
Signed-off-by: yeya24 <benye@amazon.com>
Signed-off-by: yeya24 <benye@amazon.com>
Signed-off-by: yeya24 <benye@amazon.com>
@yeya24
Copy link
Contributor Author

yeya24 commented Jul 28, 2025

Merging as correctness test is able to pass if disabling vertical sharding for binary expression without type of queries if parquet is enabled.

@yeya24 yeya24 merged commit f1b6e20 into cortexproject:master Jul 28, 2025
30 of 32 checks passed
@yeya24 yeya24 deleted the parquet-vertical-sharding branch July 28, 2025 06:52
rubywtl pushed a commit to rubywtl/cortex that referenced this pull request Jul 28, 2025
rubywtl pushed a commit to rubywtl/cortex that referenced this pull request Jul 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/querier go Pull requests that update Go code lgtm This PR has been approved by a maintainer size/L
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy