|
| 1 | +--- |
| 2 | +slug: fluxninja-joins-coderabbit |
| 3 | +title: FluxNinja joins CodeRabbit |
| 4 | +description: |
| 5 | + CodeRabbit has acquired FluxNinja, a startup that provides a platform for |
| 6 | + building scalable generative AI applications. |
| 7 | +authors: [gur, vishu] |
| 8 | +tags: |
| 9 | + [ |
| 10 | + "AI", |
| 11 | + "Acquisition", |
| 12 | + "Generative AI", |
| 13 | + "Code Review", |
| 14 | + "Load Management", |
| 15 | + "Rate Limiting", |
| 16 | + "Caching", |
| 17 | + "Scheduler", |
| 18 | + ] |
| 19 | +image: ./preview.png |
| 20 | +--- |
| 21 | + |
| 22 | + |
| 23 | + |
| 24 | +We are excited to announce that CodeRabbit has acquired FluxNinja, a startup |
| 25 | +that provides a platform for building scalable generative AI applications. This |
| 26 | +acquisition will allow us to ship new use cases at an industrial scale while |
| 27 | +sustaining our rapidly growing user base. FluxNinja's Aperture product provides |
| 28 | +advanced rate-limiting, caching, and request prioritization capabilities for |
| 29 | +building reliable and cost-effective AI workflows. |
| 30 | + |
| 31 | +<!--truncate--> |
| 32 | + |
| 33 | +Since our launch, Aperture's open-source core engine has been critical to our |
| 34 | +infrastructure. Our initial use case centered around mitigating aggressive rate |
| 35 | +limits imposed by OpenAI, allowing us to prioritize paid and real-time chat |
| 36 | +users during peak load hours while queuing requests from the free users. |
| 37 | +Further, we used Aperture's caching and rate-limiting capabilities to offer |
| 38 | +open-source developers a fully featured free tier while minimizing abuse. These |
| 39 | +capabilities allowed us to scale our user base without ever putting up a |
| 40 | +waitlist and at a price point that is sustainable for us. With Aperture's help, |
| 41 | +CodeRabbit has scaled to over 100K repositories and several thousand |
| 42 | +organizations under its review in a short period. |
| 43 | + |
| 44 | +We started CodeRabbit with a vision to build an AI-first developer tooling |
| 45 | +company from the ground up. Building enterprise-ready applied AI tech is unlike |
| 46 | +any other software engineering challenge of the past. Based on our learnings |
| 47 | +while building complex workflows, it became apparent that we need to invest in a |
| 48 | +platform that can solve the following problems: |
| 49 | + |
| 50 | +- Prompt server: Prompt design and runtime rendering is akin to responsive web |
| 51 | + design, in which a page has to be rendered differently based on the screen |
| 52 | + size and other parameters. We need a platform that can render prompts based on |
| 53 | + the context windows of underlying models and prioritize the context packing |
| 54 | + based on business attributes. For instance, it's impossible to include the |
| 55 | + entire repository and past conversations in a single prompt for code review. |
| 56 | + Even if it were possible, LLM models exhibit poor recall when doing an |
| 57 | + inference on a large context window. While it may be acceptable for use cases |
| 58 | + like chat, it’s not for use cases like code reviews that require accurate and |
| 59 | + precise outputs. |
| 60 | + |
| 61 | +- Observability into LLM outputs: One key challenge with prompting is that it's |
| 62 | + inherently non-deterministic. The same prompt can result in vastly different |
| 63 | + outputs, which can be frustrating, but this is precisely what makes AI systems |
| 64 | + powerful in the first place. Even slight variations in the prompt can result |
| 65 | + in vastly inferior or noisy outputs, leading to a decline in user conversion. |
| 66 | + At the same time, the underlying AI models are ever-evolving, and the same |
| 67 | + prompts drift over time as the models get regular updates. Traditional |
| 68 | + observability is of little use here, and we need to rethink how we classify |
| 69 | + and track different outputs and their quality. Again, this is a problem that |
| 70 | + we have to solve in-house. |
| 71 | + |
| 72 | +- Guardrails: Since we provide a free service to public repositories, we must |
| 73 | + ensure that our product is not misused beyond its intended purpose or tricked |
| 74 | + into divulging sensitive information, which could include our base prompts. |
| 75 | + |
| 76 | +While FluxNinja's Aperture project was limited to solving a different problem |
| 77 | +around load management and reliability, we found that the underlying technology |
| 78 | +and the team's expertise were a perfect foundation for building the AI platform. |
| 79 | +Prompt engineering is in its nascent stage but is emerging as a joystick for |
| 80 | +controlling AI behavior. Packing the context window with relevant documents |
| 81 | +(retrieval augmented generation, aka RAG) is also emerging as the preferred way |
| 82 | +of providing proprietary data compared to fine-tuning the model. Most AI labs |
| 83 | +focus on increasing the context window rather than making fine-tuning easier or |
| 84 | +cheaper. Despite the emergence of these clear trends, applied AI systems are |
| 85 | +still in their infancy. None of the recent AI vendors are building the "right" |
| 86 | +platform, as most of their focus has been on background/durable execution |
| 87 | +platforms, model routing proxies/gateways, chaining RAG pipelines using reusable |
| 88 | +components, and so on. Most of these approaches fall short of what a real-world |
| 89 | +AI workflow requires. The right abstractions and best practices will still have |
| 90 | +to appear, and the practitioners themselves will have to build them. Creating |
| 91 | +the “right” AI platform will be a differentiator for AI-first companies, and we |
| 92 | +are excited to tackle this problem head-on with a systems engineering mindset. |
| 93 | + |
| 94 | +We are excited to have the FluxNinja team on board and to bring our users the |
| 95 | +best-in-class AI workflows. We are also happy to welcome Harjot Gill, the |
| 96 | +founder of FluxNinja, and the rest of the team to CodeRabbit. |
0 commit comments