-
Notifications
You must be signed in to change notification settings - Fork 1
feat: Adds Oracle OCI Tracer #497
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
+1,104
−53
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Introduced a new module `oci_tracer.py` that provides methods to trace Oracle OCI Generative AI LLMs. - Implemented tracing for both streaming and non-streaming chat completions, capturing metrics such as latency, token usage, and model parameters. - Added detailed logging for error handling and tracing steps to enhance observability. - Included comprehensive type annotations and Google-style docstrings for all functions to ensure clarity and maintainability.
- Introduced a comprehensive Jupyter notebook `oci_genai_tracing.ipynb` demonstrating the integration of Oracle OCI Generative AI with Openlayer tracing, covering non-streaming and streaming chat completions, advanced parameter configurations, and error handling. - Added a simple Python script `simple_oci_example.py` for quick testing of the OCI Generative AI tracer with Openlayer integration. - Created a README file to provide an overview, prerequisites, usage instructions, and supported models for the OCI tracing examples. - Enhanced the `__init__.py` file to include the new `trace_oci_genai` function for easier access to the OCI tracing functionality. - Ensured all new files adhere to coding standards with comprehensive type annotations and Google-style docstrings for clarity and maintainability.
…tion - Updated the `oci_genai_tracing.ipynb` notebook to include new prerequisites for Openlayer setup, emphasizing the need for an Openlayer account and API key. - Improved the configuration section with detailed instructions for setting up Openlayer environment variables. - Refined the tracing logic in the `oci_tracer.py` module to handle streaming and non-streaming chat completions more effectively, including enhanced error handling and metadata extraction. - Added comprehensive logging for better observability of token usage and response metadata. - Ensured all changes adhere to coding standards with thorough type annotations and Google-style docstrings for maintainability.
- Added timing measurements around the OCI client chat method to capture latency for both streaming and non-streaming chat completions. - Introduced a new function `estimate_prompt_tokens_from_chat_details` to estimate prompt tokens when usage information is not provided by OCI. - Updated `handle_streaming_chat`, `handle_non_streaming_chat`, and `stream_chunks` functions to utilize the new timing parameters for improved performance tracking. - Ensured all changes are compliant with coding standards, including comprehensive type annotations and Google-style docstrings for maintainability.
…cer.py - Enhanced code readability by standardizing spacing and formatting throughout the `oci_tracer.py` module. - Ensured consistent use of double quotes for string literals and improved alignment of code blocks. - Updated comments and docstrings for clarity and adherence to Google-style guidelines. - Maintained comprehensive type annotations and logging practices to support maintainability and observability.
… oci_tracer.py - Simplified the streaming statistics tracking by reducing the number of metrics and focusing on essential timing information. - Enhanced performance by introducing a new `_extract_chunk_content` function for fast content extraction from OCI chunks, minimizing overhead during processing. - Removed redundant code related to raw output handling and chunk sampling, streamlining the overall logic for better readability and maintainability. - Updated comments and docstrings to reflect the changes and ensure compliance with Google-style guidelines. - Maintained comprehensive type annotations and logging practices to support ongoing maintainability and observability.
- Added support for the new `oci_tracer` in the `INTEGRATION_DEPENDENCIES` dictionary to ensure comprehensive testing of all integration modules. - Improved code formatting for better readability, including consistent use of double quotes and alignment of code blocks. - Streamlined the `run_integration_test` function by consolidating command construction for executing test scripts. - Updated print statements for clarity in test output, ensuring a more informative summary of test results. - Ensured compliance with Google-style docstrings and maintained comprehensive type annotations throughout the test suite.
…xamples - Refactored the `oci_genai_tracing.ipynb` notebook to enhance clarity and organization, including a new setup section for Openlayer API key and inference pipeline ID. - Removed the `README.md` and `simple_oci_example.py` files as they are no longer needed, consolidating documentation within the notebook. - Improved the structure of the notebook by replacing raw cells with markdown cells for better readability and user experience. - Ensured all changes comply with coding standards, including comprehensive type annotations and Google-style docstrings for maintainability.
…n options - Updated the `trace_oci_genai` function to include an optional `estimate_tokens` parameter, allowing users to control token estimation behavior when not provided by OCI responses. - Enhanced the `oci_genai_tracing.ipynb` notebook to document the new parameter and its implications for token estimation, improving user understanding and experience. - Modified the `extract_tokens_info` function to handle token estimation more robustly, returning None for token fields when estimation is disabled. - Ensured all changes comply with coding standards, including comprehensive type annotations and Google-style docstrings for maintainability.
- Updated the `extract_inputs_from_chat_details` function to convert message roles to lowercase for consistency with OpenAI format. - Removed commented-out code related to system message extraction to enhance code clarity and maintainability.
gustavocidornelas
approved these changes
Aug 6, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
feat: Add Oracle OCI Generative AI tracing integration with enhanced token handling
Overview
This PR introduces comprehensive tracing support for Oracle Cloud Infrastructure (OCI) Generative AI services with advanced token estimation capabilities, enabling automatic monitoring and observability for OCI LLM interactions through Openlayer.
Files Added/Modified
Core Integration
src/openlayer/lib/integrations/oci_tracer.py
- Main tracing integration with token estimationsrc/openlayer/lib/integrations/__init__.py
- Updated exportsExamples & Documentation
examples/tracing/oci/oci_genai_tracing.ipynb
- Interactive notebook with token estimation examplesexamples/tracing/oci/openlayer_oci_example.py
- Advanced usage with token configurationTesting
tests/test_integration_conditional_imports.py
for OCI tracer validationUsage
Basic Integration with Token Estimation
Token Handling Options
Environment Setup
Streaming Support with Token Tracking
Technical Deep Dive
Token Extraction Architecture
response.data.usage
response.data.chat_response.usage
prompt_tokens
,completion_tokens
,total_tokens
cached_tokens
fromPromptTokensDetails
Estimation Logic
Performance Optimizations
_extract_chunk_content()
functionError Resilience Patterns
Testing & Validation
Comprehensive Coverage
Quality Assurance
Backward Compatibility
trace_<provider>()
conventionsestimate_tokens=True
maintains existing functionalityBenefits
For Developers
For Operations
Oracle OCI Generative AI users can now leverage Openlayer's powerful tracing capabilities with precision token control, zero application changes, and production-grade performance.