diff --git a/blog/fluxninja-acquisition-2024-03-17/blog.md b/blog/fluxninja-acquisition-2024-03-17/blog.md new file mode 100644 index 00000000..16e87cb4 --- /dev/null +++ b/blog/fluxninja-acquisition-2024-03-17/blog.md @@ -0,0 +1,120 @@ +--- +slug: fluxninja-joins-coderabbit +title: FluxNinja joins CodeRabbit +description: + CodeRabbit has acquired FluxNinja, a startup that provides a platform for + building scalable generative AI applications. +authors: [gur, vishu] +tags: + [ + "AI", + "Acquisition", + "Generative AI", + "Code Review", + "Load Management", + "Rate Limiting", + "Caching", + "Scheduler", + ] +image: ./preview.png +--- + + + +We are excited to announce that CodeRabbit has acquired +[FluxNinja](https://fluxninja.com), a startup that provides a platform for +building scalable generative AI applications. This acquisition will allow us to +ship new use cases at an industrial-pace while sustaining our rapidly growing +user base. FluxNinja's Aperture product provides advanced rate & concurrency +limiting, caching, and request prioritization capabilities that are essential +for reliable and cost-effective AI workflows. + + + +Since our launch, +[Aperture's open-source](https://github.com/fluxninja/aperture) core engine has +been critical to our infrastructure. Our initial use case centered around +[mitigating aggressive rate limits](../openai-rate-limits-2023-10-23/blog.md) +imposed by OpenAI, allowing us to prioritize paid and real-time chat users +during peak load hours while queuing requests from the free users. Further, we +used Aperture's +[caching and rate-limiting capabilities](../how-we-built-cost-effective-generative-ai-application-2023-12-23/blog.md) +to manage costs that in turn allowed us to offer open-source developers a fully +featured free tier by minimizing abuse. These capabilities allowed us to scale +our user base without ever putting up a waitlist and at a price point that is +sustainable for us. With Aperture's help, CodeRabbit has scaled to over 100K +repositories and several thousand organizations under its review in a short +period. + +We started CodeRabbit with a vision to build an AI-first developer tooling +company from the ground up. Building enterprise-ready applied AI tech is unlike +any other software engineering challenge of the past. Based on our learnings +while building complex workflows, it became apparent that we need to invest in a +platform that can solve the following problems: + +- Prompt rendering: Prompt design and rendering is akin to responsive web + design. Web servers render pages based on the screen size and other + parameters, for example, on a mobile device, navigation bars are usually + rendered as hamburger menus, making it easier for human consumption. + Similarly, we need a prompt server that can render prompts based on the + context windows of underlying models and prioritize the packing of context + based on business attributes, making it easier for AI consumption. It's not + feasible to include the entire repository, past conversations, documentation, + learnings, etc. in a single code review prompt because of the context window + size limitations. Even if it was possible, AI models exhibit poor recall when + doing an inference on a completely packed context window. While tight packing + may be acceptable for use cases like chat, it’s not for use cases like code + reviews that require accurate inferences. Therefore, it's critical to render + prompts in such a way that the quality of inference is high for each use-case, + while being cost-effective and fast. In addition to packing logic, basic + guardrails are also needed, especially when rendering prompts based on inputs + from end-users. Since we provide a free service to public repositories, we + have to ensure that our product is not misused beyond its intended purpose or + tricked into divulging sensitive information, which could include our base + prompts. + +- Validation & quality checks: Generative AI models consume text and output + text. On the other hand, traditional code and APIs required structured data. + Therefore, the prompt service needs to expose a RESTful or gRPC API that can + be consumed by the other services in the workflow. We touched upon the + rendering of prompts based on structured requests in the previous point, but + the prompt service also needs to parse, validate responses into structured + data and measure the quality of the inference. This is a non-trivial problem, + and multiple tries are often required to ensure that the response is thorough + and meets the quality bar. For instance, we found that when we pack multiple + files in a single code review prompt, AI models often miss hunks within a file + or miss files altogether, leading to incomplete reviews. + +- Observability: One key challenge with generative AI and prompting is that it's + inherently non-deterministic. The same prompt can result in vastly different + outputs, which can be frustrating, but this is precisely what makes AI systems + powerful in the first place. Even slight variations in the prompt can result + in vastly inferior or noisy outputs, leading to a decline in user engagement. + At the same time, the underlying AI models are ever-evolving, and the + established prompts drift over time as the models get regular updates. + Traditional observability is of little use here, and we need to rethink how we + classify and track generated output and measure quality. Again, this is a + problem that we have to solve in-house. + +While FluxNinja's Aperture project was limited to solving a different problem +around load management and reliability, we found that the underlying technology +and the team's expertise were a perfect foundation for building the AI platform. +Prompt engineering is in its nascent stage but is emerging as a joystick for +controlling AI behavior. Packing the context window with relevant documents +(retrieval augmented generation, aka RAG) is also emerging as the preferred way +of providing proprietary data compared to fine-tuning the model. Most AI labs +focus on increasing the context window rather than making fine-tuning easier or +cheaper. Despite the emergence of these clear trends, applied AI systems are +still in their infancy. None of the recent AI vendors seem to be building the +"right" platform, as most of their focus has been on background/durable +execution frameworks, model routing proxies/gateways, composable RAG pipelines, +and so on. Most of these approaches fall short of what a real-world AI workflow +requires. The right abstractions and best practices will still have to appear, +and the practitioners themselves will have to build them. AI platforms will be a +differentiator for AI-first companies, and we are excited to tackle this problem +head-on with a systems engineering mindset. + +We are excited to have the FluxNinja team on board and to bring our users the +best-in-class AI workflows. We are also happy to welcome +[Harjot Gill](https://www.linkedin.com/in/harjotsgill/), the founder of +FluxNinja, and the rest of the team to CodeRabbit. diff --git a/blog/fluxninja-acquisition-2024-03-17/preview.png b/blog/fluxninja-acquisition-2024-03-17/preview.png new file mode 100644 index 00000000..727049d8 Binary files /dev/null and b/blog/fluxninja-acquisition-2024-03-17/preview.png differ diff --git a/docs/guides/prompt-customization.md b/docs/guides/prompt-customization.md index b96563c9..f270bd23 100644 --- a/docs/guides/prompt-customization.md +++ b/docs/guides/prompt-customization.md @@ -142,7 +142,7 @@ syntax node matches the rule or not. There are three kinds of atomic rule: #### Relational rule -Relational rule defines the relationship between two syntax nodes. There are +A relational rule defines the relationship between two syntax nodes. There are four kinds of relational rule: `inside`, `has`, `follows` and `precedes`. All four relational rules accept a sub-rule object as their value. The sub-rule
Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.
Alternative Proxies: