Skip to content

Commit 7698509

Browse files
montanalowmontanalow
authored
Support instructor embeddings (#596)
1 parent 41de0aa commit 7698509

36 files changed

+910
-217
lines changed

‎.github/workflows/ci.yml

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ jobs:
3434
path: |
3535
~/.cargo
3636
pgml-extension/target
37-
~/.pgx
37+
~/.pgrx
3838
key: ${{ runner.os }}-rust-${{ hashFiles('pgml-extension/Cargo.lock') }}
3939
- name: Submodules
4040
run: |
@@ -43,20 +43,20 @@ jobs:
4343
run: |
4444
curl https://sh.rustup.rs -sSf | sh -s -- -y
4545
source ~/.cargo/env
46-
cargo install cargo-pgx --version "0.7.1"
46+
cargo install cargo-pgrx --version "0.7.1"
4747
48-
if [[ ! -d ~/.pgx ]]; then
49-
cargo pgx init
48+
if [[ ! -d ~/.pgrx ]]; then
49+
cargo pgrx init
5050
fi
5151
52-
cargo pgx test
52+
cargo pgrx test
5353
54-
cargo pgx stop
55-
cargo pgx start
54+
cargo pgrx stop
55+
cargo pgrx start
5656
5757
# psql -p 28813 -h 127.0.0.1 -d pgml -P pager -f tests/test.sql
5858
59-
cargo pgx stop
59+
cargo pgrx stop
6060
6161
6262

‎.github/workflows/package-extension.yml

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -79,47 +79,47 @@ jobs:
7979
curl -sLO https://github.com/deb-s3/deb-s3/releases/download/0.11.4/deb-s3-0.11.4.gem
8080
sudo gem install deb-s3-0.11.4.gem
8181
dpkg-deb --version
82-
- name: Install pgx
82+
- name: Install pgrx
8383
uses: postgresml/gh-actions-cargo@master
8484
with:
8585
working-directory: pgml-extension
8686
command: install
87-
args: cargo-pgx --version "0.7.1"
88-
- name: pgx init
87+
args: cargo-pgrx --version "0.7.1"
88+
- name: pgrx init
8989
uses: postgresml/gh-actions-cargo@master
9090
with:
9191
working-directory: pgml-extension
92-
command: pgx
92+
command: pgrx
9393
args: init --pg11=/usr/lib/postgresql/11/bin/pg_config --pg12=/usr/lib/postgresql/12/bin/pg_config --pg13=/usr/lib/postgresql/13/bin/pg_config --pg14=/usr/lib/postgresql/14/bin/pg_config --pg15=/usr/lib/postgresql/15/bin/pg_config
9494
- name: Build Postgres 11
9595
uses: postgresml/gh-actions-cargo@master
9696
with:
9797
working-directory: pgml-extension
98-
command: pgx
98+
command: pgrx
9999
args: package --pg-config /usr/lib/postgresql/11/bin/pg_config
100100
- name: Build Postgres 12
101101
uses: postgresml/gh-actions-cargo@master
102102
with:
103103
working-directory: pgml-extension
104-
command: pgx
104+
command: pgrx
105105
args: package --pg-config /usr/lib/postgresql/12/bin/pg_config
106106
- name: Build Postgres 13
107107
uses: postgresml/gh-actions-cargo@master
108108
with:
109109
working-directory: pgml-extension
110-
command: pgx
110+
command: pgrx
111111
args: package --pg-config /usr/lib/postgresql/13/bin/pg_config
112112
- name: Build Postgres 14
113113
uses: postgresml/gh-actions-cargo@master
114114
with:
115115
working-directory: pgml-extension
116-
command: pgx
116+
command: pgrx
117117
args: package --pg-config /usr/lib/postgresql/14/bin/pg_config
118118
- name: Build Postgres 15
119119
uses: postgresml/gh-actions-cargo@master
120120
with:
121121
working-directory: pgml-extension
122-
command: pgx
122+
command: pgrx
123123
args: package --pg-config /usr/lib/postgresql/15/bin/pg_config
124124
- name: Build debs
125125
env:

‎pgml-docs/docs/blog/benchmarks/python_microservices_vs_postgresml/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
## PostgresML
44

55
```
6-
cargo pgx run --release
6+
cargo pgrx run --release
77
```
88

99
### Schema

‎pgml-docs/docs/blog/postgresml-is-moving-to-rust-for-our-2.0-release.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -170,7 +170,7 @@ Spoiler alert: idiomatic Rust is about 10x faster than native SQL, embedded PL/p
170170
LIMIT 1;
171171
```
172172

173-
We're building with the Rust [pgx](https://github.com/tcdi/pgx/tree/master/pgx) crate that makes our development cycle even nicer than the one we use to manage Python. It really streamlines creating an extension in Rust, so all we have to worry about is writing our functions. It took about an hour to port all of our vector operations to Rust with BLAS support, and another week to port all the "business logic" for maintaining model training and deployment. We've even gained some new capabilities for caching models across connections (independent processes), now that we have access to Postgres shared memory, without having to worry about Python's GIL and GC. This is the dream of Apache's Arrow project, realized for our applications, without having to change the world, just our implementations. 🤩 Single-copy end-to-end machine learning, with parallel processing and shared data access.
173+
We're building with the Rust [pgrx](https://github.com/tcdi/pgrx/tree/master/pgrx) crate that makes our development cycle even nicer than the one we use to manage Python. It really streamlines creating an extension in Rust, so all we have to worry about is writing our functions. It took about an hour to port all of our vector operations to Rust with BLAS support, and another week to port all the "business logic" for maintaining model training and deployment. We've even gained some new capabilities for caching models across connections (independent processes), now that we have access to Postgres shared memory, without having to worry about Python's GIL and GC. This is the dream of Apache's Arrow project, realized for our applications, without having to change the world, just our implementations. 🤩 Single-copy end-to-end machine learning, with parallel processing and shared data access.
174174

175175
## What about XGBoost and friends?
176176
ML isn't just about basic math and a little bit of business logic. It's about all those complicated algorithms beyond linear regression for gradient boosting and deep learning. The good news is that most of these libraries are implemented in C/C++, and just have Python bindings. There are also bindings for Rust ([lightgbm](https://github.com/vaaaaanquish/lightgbm-rs), [xgboost](https://github.com/davechallis/rust-xgboost), [tensorflow](https://github.com/tensorflow/rust), [torch](https://github.com/LaurentMazare/tch-rs)).

‎pgml-docs/docs/developer_guide/overview.md

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,7 @@ The development environment for each differs slightly, but overall we use Python
6262
6363
## Postgres extension
6464
65-
PostgresML is a Rust extension written with `tcdi/pgx` crate. Local development therefore requires the [latest Rust compiler](https://www.rust-lang.org/learn/get-started) and PostgreSQL development headers and libraries.
65+
PostgresML is a Rust extension written with `tcdi/pgrx` crate. Local development therefore requires the [latest Rust compiler](https://www.rust-lang.org/learn/get-started) and PostgreSQL development headers and libraries.
6666
6767
The extension code is located in:
6868
@@ -72,17 +72,17 @@ cd pgml-extension/
7272

7373
You'll need to install basic dependencies
7474

75-
Once there, you can initialize `pgx` and get going:
75+
Once there, you can initialize `pgrx` and get going:
7676

7777
#### Pgx command line and environments
7878
```commandline
79-
cargo install cargo-pgx --version "0.7.1" && \
80-
cargo pgx init # This will take a few minutes
79+
cargo install cargo-pgrx --version "0.7.4" && \
80+
cargo pgrx init # This will take a few minutes
8181
```
8282

8383
#### Update postgresql.conf
8484

85-
`pgx` uses Postgres 15 by default. Since `pgml` is using shared memory, you need to add it to `shared_preload_libraries` in `postgresql.conf` which, for `pgx`, is located in `~/.pgx/data-15/postgresql.conf`.
85+
`pgrx` uses Postgres 15 by default. Since `pgml` is using shared memory, you need to add it to `shared_preload_libraries` in `postgresql.conf` which, for `pgrx`, is located in `~/.pgrx/data-15/postgresql.conf`.
8686

8787
```
8888
shared_preload_libraries = 'pgml' # (change requires restart)
@@ -91,19 +91,19 @@ shared_preload_libraries = 'pgml' # (change requires restart)
9191
Run the unit tests
9292

9393
```commandline
94-
cargo pgx test
94+
cargo pgrx test
9595
```
9696

9797
Run the integration tests:
9898
```commandline
99-
cargo pgx run --release
99+
cargo pgrx run --release
100100
psql -h localhost -p 28813 -d pgml -f tests/test.sql -P pager
101101
```
102102

103103
Run an interactive psql session
104104

105105
```commandline
106-
cargo pgx run
106+
cargo pgrx run
107107
```
108108

109109
Create the extension in your database:
@@ -147,10 +147,10 @@ By default, the extension is built without CUDA support for XGBoost and LightGBM
147147

148148

149149
```commandline
150-
CUDACXX=/usr/local/cuda/bin/nvcc cargo pgx run --release --features pg15,python,cuda
150+
CUDACXX=/usr/local/cuda/bin/nvcc cargo pgrx run --release --features pg15,python,cuda
151151
```
152152

153-
If you ever want to reset the environment, simply spin up the database with `cargo pgx run` and drop the extension and metadata tables:
153+
If you ever want to reset the environment, simply spin up the database with `cargo pgrx run` and drop the extension and metadata tables:
154154

155155
```postgresql
156156
DROP EXTENSION IF EXISTS pgml CASCADE;
@@ -190,7 +190,7 @@ Basic installation can be achieved with:
190190
cd postgresml/pgml-dashboard
191191
```
192192

193-
2. Set the `DATABASE_URL` environment variable, for example to a running interactive `cargo pgx run` session started previously:
193+
2. Set the `DATABASE_URL` environment variable, for example to a running interactive `cargo pgrx run` session started previously:
194194
```commandline
195195
export DATABASE_URL=postgres://localhost:28815/pgml
196196
```

‎pgml-docs/docs/user_guides/setup/v2/installation.md

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -92,25 +92,25 @@ sudo apt-get install postgresql
9292
cd pgml-extension
9393
```
9494

95-
5. Install [`pgx`](https://github.com/tcdi/pgx) and build the extension (this will take a few minutes):
95+
5. Install [`pgrx`](https://github.com/tcdi/pgrx) and build the extension (this will take a few minutes):
9696

9797
**With Python support:**
9898

9999
```bash
100100
export POSTGRES_VERSION=15
101-
cargo install cargo-pgx --version "0.7.1" && \
102-
cargo pgx init --pg${POSTGRES_VERSION} /usr/bin/pg_config && \
103-
cargo pgx package
101+
cargo install cargo-pgrx --version "0.7.4" && \
102+
cargo pgrx init --pg${POSTGRES_VERSION} /usr/bin/pg_config && \
103+
cargo pgrx package
104104
```
105105

106106
**Without Python support:**
107107

108108
```bash
109109
export POSTGRES_VERSION=15
110110
cp docker/Cargo.toml.no-python Cargo.toml && \
111-
cargo install cargo-pgx --version "0.7.1" && \
112-
cargo pgx init --pg${POSTGRES_VERSION} /usr/bin/pg_config && \
113-
cargo pgx package
111+
cargo install cargo-pgrx --version "0.7.4" && \
112+
cargo pgrx init --pg${POSTGRES_VERSION} /usr/bin/pg_config && \
113+
cargo pgrx package
114114
```
115115

116116
6. Copy the extension binaries into Postgres system folders:
@@ -152,12 +152,12 @@ sudo apt-get install postgresql
152152
For example, `openssl` requires some environment variables set in `~/.zsh` for
153153
the compiler to find the library.
154154

155-
4. Install [`pgx`](https://github.com/tcdi/pgx) and build the extension (this will take a few minutes):
155+
4. Install [`pgrx`](https://github.com/tcdi/pgrx) and build the extension (this will take a few minutes):
156156

157157
```
158-
cargo install cargo-pgx && \
159-
cargo pgx init --pg15 /usr/bin/pg_config && \
160-
cargo pgx install
158+
cargo install cargo-pgrx && \
159+
cargo pgrx init --pg15 /usr/bin/pg_config && \
160+
cargo pgrx install
161161
```
162162

163163

0 commit comments

Comments
 (0)
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy