Skip to content

Mostly Revert 'GITBOOK-31: Announcing the 1.0 SDK' #1351

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 4, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@ description: >-
GPTQ & GGML allow PostgresML to fit larger models in less RAM. These
algorithms perform inference significantly faster on NVIDIA, Apple and Intel
hardware.
featured: false
tags: [engineering]
image: ".gitbook/assets/image (14).png"
---

# Announcing GPTQ & GGML Quantized LLM support for Huggingface Transformers
Expand Down
7 changes: 7 additions & 0 deletions pgml-cms/blog/announcing-support-for-aws-us-east-1-region.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,10 @@
---
description: >-
We added aws us east 1 to our list of support aws regions.
featured: false
tags: [product]
---

# Announcing Support for AWS us-east-1 Region

<div align="left">
Expand Down
2 changes: 2 additions & 0 deletions pgml-cms/blog/data-is-living-and-relational.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ description: >-
A common problem with data science and machine learning tutorials is the
published and studied datasets are often nothing like what you’ll find in
industry.
featured: false
tags: [engineering]
---

# Data is Living and Relational
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
description: >-
How to use the pgml.embed(...) function to generate embeddings with free and
open source models in your own database.
image: ".gitbook/assets/blog_image_generating_llm_embeddings.png"
features: true
---

# Generating LLM embeddings with open source models in PostgresML
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@ description: >-
PostgresML makes it easy to use machine learning on your data and scale
workloads horizontally in our cloud. One of the most common use cases is to
improve search results.
featured: true
image: ".gitbook/assets/image (2) (2).png"
tags: ["Engineering"]
---

# How-to Improve Search Results with Machine Learning
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,11 @@
---
featured: true
tags: [engineering, product]
image: https://postgresml.org/dashboard/static/images/open_source_ai_social_share.png
description: >-
Quickly and easily transition from the confines of the OpenAI APIs to higher
quality embeddings and unrestricted text generation models.
image: ".gitbook/assets/blog_image_switch_kit.png"
---

# Introducing the OpenAI Switch Kit: Move from closed to open-source AI in minutes
Expand Down
1 change: 1 addition & 0 deletions pgml-cms/blog/postgres-full-text-search-is-awesome.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
description: >-
If you want to improve your search results, don't rely on expensive O(n*m)
word frequency statistics. Get new sources of data instead.
image: ".gitbook/assets/image (53).png"
---

# Postgres Full Text Search is Awesome!
Expand Down
11 changes: 11 additions & 0 deletions pgml-cms/blog/postgresml-is-going-multicloud.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,16 @@
# PostgresML is going multicloud

<div align="left">

<figure><img src=".gitbook/assets/lev.jpg" alt="Author" width="100"><figcaption></figcaption></figure>

</div>

Lev Kokotov

Jan 18, 2024


We started PostgresML two years ago with the goal of making machine learning and AI accessible and easy for everyone. To make this a reality, we needed to deploy PostgresML as closely as possible to our end users. With that goal mind, today we're proud to announce support for a new cloud provider: Azure.

### How we got here
Expand Down
3 changes: 3 additions & 0 deletions pgml-cms/blog/speeding-up-vector-recall-5x-with-hnsw.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@ description: >-
HNSW indexing is the latest upgrade in vector recall performance. In this post
we announce our updated SDK that utilizes HNSW indexing to give world class
performance in vector search.
tags: [engineering]
featured: true
image: ".gitbook/assets/blog_image_hnsw.png"
---

# Speeding up vector recall 5x with HNSW
Expand Down
30 changes: 21 additions & 9 deletions pgml-cms/blog/using-postgresml-with-django-and-embedding-search.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,21 @@
---
tags:
- engineering
description: >-
An example application using PostgresML and Django to build embedding based search.
tags: [engineering]
---

# Using PostgresML with Django and embedding search

<div align="left">

<figure><img src=".gitbook/assets/lev.jpg" alt="Author" width="100"><figcaption></figcaption></figure>

</div>

Lev Kokotov

Feb 15, 2024

Building web apps on top of PostgresML allows anyone to integrate advanced machine learning and AI features into their products without much work or needing to understand how it really works. In this blog post, we'll talk about building a classic to-do Django app, with the spicy addition of semantic search powered by embedding models running inside your PostgreSQL database.

### Getting the code
Expand Down Expand Up @@ -56,13 +67,14 @@ And that's it! In just a few lines of code, we're generating and storing high qu

Djago Rest Framework provides the bulk of the implementation. We just added a `ModelViewSet` for the `TodoItem` model, with just one addition: a search endpoint. The search endpoint required us to write a bit of SQL to embed the search query and accept a few filters, but the core of it can be summarized in a single annotation on the query set:

<pre class="language-python"><code class="lang-python"><strong>results = TodoItem.objects.annotate(
</strong> similarity=RawSQL(
```python
results = TodoItem.objects.annotate(
similarity=RawSQL(
"pgml.embed('intfloat/e5-small', %s)::vector(384) &#x3C;=> embedding",
[query],
)
).order_by("similarity")
</code></pre>
```

This single line of SQL does quite a bit:

Expand All @@ -76,7 +88,7 @@ All of this happens inside PostgresML. Our Django app doesn't need to implement

Before going forward, make sure you have the app running either locally or in a cloud provider of your choice. If hosting it somewhere, replace `localhost:8000` with the URL and port of your service.

The simplest way to interact with it is to use cURL or an HTTP client of your preference. If running in debug mode locally, the Rest Framework provides a nice web UI which you can access on [http://localhost:8000/api/todos/](http://localhost:8000/api/todo/) using a browser.
The simplest way to interact with it is to use cURL or your preferred HTTP client. If running in debug mode locally, the Rest Framework provides a nice web UI which you can access on [http://localhost:8000/api/todo/](http://localhost:8000/api/todo/) using a browser.

To create a to-do item with cURL, you can just run this:

Expand All @@ -103,13 +115,13 @@ In return, you'll get your to-do item alongside the embedding of the `descriptio

The embedding contains 384 floating point numbers; we removed most of them in this blog post to make sure it fits on the page.

You can try creating multiple to-do items for fun and profit. If the description is changed, so will the embedding, demonstrating how the `intfloat/e5-small` model is understanding the semantic meaning of your text.
You can try creating multiple to-do items for fun and profit. If the description is changed, so will the embedding, demonstrating how the `intfloat/e5-small` model understands the semantic meaning of your text.

### Searching

Once you have a few embeddings and to-dos stored in your database, the fun part of searching can begin. In a typical search example with PostgreSQL, you'd now be using `tsvector` to keyword match your to-dos to the search query with term frequency. That's a good technique, but semantic search is better.

We've created a simple search endpoint that accepts a query, a completed to-do filter, and a limit. To use it, you can just do this:
Our search endpoint accepts a query, a completed to-do filter, and a limit. To use it, you can just run this:

```bash
curl \
Expand All @@ -122,7 +134,7 @@ curl \
If you've created a bunch of different to-do items, you should get only one search result back, and exactly the one you were expecting:

```json
"Make a New Year resolution list"
"Make a New Year resolution"
```

You can increase the `limit` to something larger and you should get more documents, in decreasing order of relevance.
Expand Down
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy