Skip to content

Commit 2c16cf2

Browse files
levkklevkk
authored
Rust extension - XGBoost (#286)
* Rust XGBoost POC * README * vendor * cleanup * remove submodule * readme * Vendor * readme * Cleaner * version * move files
1 parent 68859f3 commit 2c16cf2

File tree

1,451 files changed

+309176
-2
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

1,451 files changed

+309176
-2
lines changed

‎.editorconfig

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ end_of_line = lf
55
insert_final_newline = true
66
charset = utf-8
77

8-
[*.py]
8+
[*.py, *.rs]
99
indent_style = space
1010
indent_size = 4
1111

‎pgml-extension/.dockerignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -158,3 +158,5 @@ cython_debug/
158158
# and can be added to the global gitignore or merged into this file. For a more nuclear
159159
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
160160
#.idea/
161+
162+
pgml_rust/target/

‎pgml-extension/Dockerfile

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,18 @@
1+
# FROM rust:1-bullseye AS rust_builder
2+
# COPY pgml_rust /pgml_rust
3+
# WORKDIR /pgml_rust
4+
# RUN apt-get update && apt-get install -y postgresql-13 libpq-dev cmake libclang-dev
5+
# RUN cargo install cargo-pgx
6+
# RUN cargo pgx init
7+
# RUN cargo pgx package
8+
19
FROM debian:bullseye-slim
210
MAINTAINER docker@postgresml.com
311

412
RUN apt-get update
513
ARG DEBIAN_FRONTEND=noninteractive
614
ENV TZ=Etc/UTC
7-
RUN apt-get install -y postgresql-plpython3-13 python3 python3-pip postgresql-13 tzdata sudo cmake libpq-dev
15+
RUN apt-get install -y postgresql-plpython3-13 python3 python3-pip postgresql-13 tzdata sudo cmake libpq-dev libclang-dev
816

917
# Cache this, quicker
1018
RUN pip3 install xgboost sklearn diptest torch lightgbm transformers datasets sentencepiece sacremoses sacrebleu rouge

‎pgml-extension/pgml_rust/.cargo/config

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
[build]
2+
# Postgres symbols won't be available until runtime
3+
rustflags = ["-C", "link-args=-Wl,-undefined,dynamic_lookup"]

‎pgml-extension/pgml_rust/.gitignore

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
.DS_Store
2+
.idea/
3+
/target
4+
*.iml
5+
**/*.rs.bk
6+
Cargo.lock

‎pgml-extension/pgml_rust/Cargo.toml

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
[package]
2+
name = "pgml_rust"
3+
version = "0.0.0"
4+
edition = "2021"
5+
6+
[lib]
7+
crate-type = ["cdylib"]
8+
9+
[features]
10+
default = ["pg13"]
11+
pg10 = ["pgx/pg10", "pgx-tests/pg10" ]
12+
pg11 = ["pgx/pg11", "pgx-tests/pg11" ]
13+
pg12 = ["pgx/pg12", "pgx-tests/pg12" ]
14+
pg13 = ["pgx/pg13", "pgx-tests/pg13" ]
15+
pg14 = ["pgx/pg14", "pgx-tests/pg14" ]
16+
pg_test = []
17+
18+
[dependencies]
19+
pgx = "=0.4.5"
20+
xgboost = { path = "rust-xgboost" }
21+
rustlearn = "0.5"
22+
once_cell = "1"
23+
rand = "0.8"
24+
25+
[dev-dependencies]
26+
pgx-tests = "=0.4.5"
27+
28+
[profile.dev]
29+
panic = "unwind"
30+
lto = "thin"
31+
32+
[profile.release]
33+
panic = "unwind"
34+
opt-level = 3
35+
lto = "fat"
36+
codegen-units = 1

‎pgml-extension/pgml_rust/README.md

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
# Rust meet PostgresML
2+
3+
Here we have some POC code to use Rust for PostgresML.
4+
5+
## Dependencies
6+
7+
All dependencies are vendored. I downloaded XGBoost 1.62 and all its submodules. We're also using the `master` branch of `xgboost` Rust crate.
8+
9+
If you haven't already, install:
10+
11+
- `cmake`
12+
- `libclang-dev`
13+
14+
## Local development
15+
16+
1. `cargo install pgx`
17+
2. `cargo pgx run`
18+
3. `DROP EXTENSION IF EXISTS pgml_rust;`
19+
4. `CREATE EXTENSION pgml_rust;`
20+
5. `SELECT pgml_train('pgml.diabetes', ARRAY['age', 'sex'], 'target');`
21+
6. `SELECT * FROM pgml_predict(ARRAY[1, 5.0]);`
22+
23+
Lots of todos, but still a decent PoC.

‎pgml-extension/pgml_rust/pgml_rust.control

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
comment = 'pgml_rust: Created by pgx'
2+
default_version = '@CARGO_VERSION@'
3+
module_pathname = '$libdir/pgml_rust'
4+
relocatable = false
5+
superuser = false

‎pgml-extension/pgml_rust/rust-xgboost/.gitignore

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
# Generated by Cargo
2+
# will have compiled files and executables
3+
/target/
4+
/examples/*/target/
5+
/xgboost-sys/target/
6+
7+
# Remove Cargo.lock from gitignore if creating an executable, leave it for libraries
8+
# More information here https://doc.rust-lang.org/cargo/guide/cargo-toml-vs-cargo-lock.html
9+
Cargo.lock
10+
11+
# These are backup files generated by rustfmt
12+
**/*.rs.bk

‎pgml-extension/pgml_rust/rust-xgboost/.gitmodules

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
[submodule "xgboost-sys/xgboost"]
2+
path = xgboost-sys/xgboost
3+
url = https://github.com/davechallis/xgboost
4+
branch = master

0 commit comments

Comments
 (0)
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy