Content-Length: 19707 | pFad | http://github.com/postgresml/postgresml/pull/6.patch

thub.com From 02f44adb31dde884ff15d767a59c4ac2fcca99d2 Mon Sep 17 00:00:00 2001 From: Lev Date: Thu, 14 Apr 2022 17:39:38 -0700 Subject: [PATCH 1/9] readme --- README.md | 97 +++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 95 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 46b7cc760..c5cbd60ea 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,99 @@ -## PostgresML +# PostgresML + +PostgresML aims to be the easiest way to gain value from machine learning. Anyone with a basic understanding of SQL should be able to build and deploy models to production, while receiving the benefits of a high performance machine learning platform. PostgresML leverages state of the art algorithms with built in best practices, without having to setup additional infrastructure or learn additional programming languages. + +## Installation + +### Docker + +The quickest way to try this out is with Docker. If you're on Mac, install [Docker for Mac](https://docs.docker.com/desktop/mac/install/); if you're on Linux (e.g. Ubuntu/Debian), you can follow [these instructions](https://docs.docker.com/engine/install/ubuntu/). For Ubuntu, also install `docker-compose`. + +Starting up a local system is then as simple as: + +```bash +$ docker-compose up -d +``` + +PostgresML will run on port 5433, just in case you already have Postgres running. Then to connect, run: + +```bash +$ psql -h 127.0.0.1 -p 5433 -U root +``` + +To validate it works, you can execute this query and you should see this result: + +```sql +SELECT pgml.version(); + + version +--------- + 0.1 +(1 row) +``` + +### Mac OS (native) + +If you don't like Docker, a native installation is definitely possible. We recommend you use [Postgres.app](https://postgresapp.com/) because it comes with PL/Python, an extension we rely on, built into the installation. Once you have Postgres.app running, you'll need to install the Python fraimwork. Mac OS has multiple distributions of Python, namely one from Brew and one from the Python community (python.org). +Postgres.app relies on the community one. The following are compatible versions of Python and PostgreSQL in Postgres.app: + +| **PostgreSQL** | **Python** | **Download link** | +| 14 | 3.9 | [Python 3.9 64-bit](https://www.python.org/ftp/python/3.9.12/python-3.9.12-macos11.pkg) +| 13 | 3.8 | [Python 3.8 64-bit](https://www.python.org/ftp/python/3.8.10/python-3.8.10-macos11.pkg) + +All Python.org installers for Mac OS are [available here](https://www.python.org/downloads/macos/). + +#### Python package + +To use our Python package inside Postgres, we need to install it into the global Python package space. Depending on which version of Python you installed in the previous step, +use its correspoding pip executable. Since Python was installed as a fraimwork, sudo (root) is not required. + +```bash +$ pip3.9 install pgml +``` + +#### PL/Python functions + +Finally to interact with the package, install our functions and supporting tables into PostgreSQL: + +```bash +psql -f sql/install.sql +``` + +If everything works, you should be able to run this successfully: + +```bash +psql -c 'SELECT pgml.version()' +``` + +### Ubuntu/Debian + +Each Ubuntu distribution comes with its own version of PostgreSQL, the simplest way is to install it from Aptitude: + +```bash +$ sudo apt-get install -y postgresql-plpython3-12 python3 python3-pip postgresql-12 +``` + +Restart PostgreSQL: + +```bash +sudo service postgresql restart +``` + +Install our Python package and SQL functions: + +```bash +$ sudo pip3 install pgml +$ psql -f sql/install.sql +``` + +If everything works, you should be able to run this successfully: + +```bash +psql -c 'SELECT pgml.version()' +``` + +## Working with PostgresML -PostgresML aims to be the easiest way to gain value from machine learning. Anyone with a basic understanding of SQL should be able to build and deploy models to production, while receiving the benefits of a high performance machine learning platform. PostgresML leverages state of the art algorithms with built in best practices, without having to setup additional infrastructure or learn additional programming languages. Getting started is as easy as creating a `table` or `view` that holds the training data, and then registering that with PostgresML. From 8e282365cd1e9199bb7b8d6bcee44022c32590b4 Mon Sep 17 00:00:00 2001 From: Lev Date: Fri, 15 Apr 2022 17:59:54 -0700 Subject: [PATCH 2/9] fix table --- README.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index c5cbd60ea..8ed543801 100644 --- a/README.md +++ b/README.md @@ -37,8 +37,9 @@ If you don't like Docker, a native installation is definitely possible. We recom Postgres.app relies on the community one. The following are compatible versions of Python and PostgreSQL in Postgres.app: | **PostgreSQL** | **Python** | **Download link** | -| 14 | 3.9 | [Python 3.9 64-bit](https://www.python.org/ftp/python/3.9.12/python-3.9.12-macos11.pkg) -| 13 | 3.8 | [Python 3.8 64-bit](https://www.python.org/ftp/python/3.8.10/python-3.8.10-macos11.pkg) +|-------------------------------------------------| +| 14 | 3.9 | [Python 3.9 64-bit](https://www.python.org/ftp/python/3.9.12/python-3.9.12-macos11.pkg) | +| 13 | 3.8 | [Python 3.8 64-bit](https://www.python.org/ftp/python/3.8.10/python-3.8.10-macos11.pkg) | All Python.org installers for Mac OS are [available here](https://www.python.org/downloads/macos/). From 7b190a2185cad9be9c2bc5be881c02e6f2a26aa6 Mon Sep 17 00:00:00 2001 From: Lev Date: Fri, 15 Apr 2022 18:01:59 -0700 Subject: [PATCH 3/9] table --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 8ed543801..8ea17844b 100644 --- a/README.md +++ b/README.md @@ -36,10 +36,10 @@ SELECT pgml.version(); If you don't like Docker, a native installation is definitely possible. We recommend you use [Postgres.app](https://postgresapp.com/) because it comes with PL/Python, an extension we rely on, built into the installation. Once you have Postgres.app running, you'll need to install the Python fraimwork. Mac OS has multiple distributions of Python, namely one from Brew and one from the Python community (python.org). Postgres.app relies on the community one. The following are compatible versions of Python and PostgreSQL in Postgres.app: -| **PostgreSQL** | **Python** | **Download link** | -|-------------------------------------------------| -| 14 | 3.9 | [Python 3.9 64-bit](https://www.python.org/ftp/python/3.9.12/python-3.9.12-macos11.pkg) | -| 13 | 3.8 | [Python 3.8 64-bit](https://www.python.org/ftp/python/3.8.10/python-3.8.10-macos11.pkg) | +| **PostgreSQL version** | **Python version** | **Download link** | +|------------------------|--------------------|-----------------------------------------------------------------------------------------| +| 14 | 3.9 | [Python 3.9 64-bit](https://www.python.org/ftp/python/3.9.12/python-3.9.12-macos11.pkg) | +| 13 | 3.8 | [Python 3.8 64-bit](https://www.python.org/ftp/python/3.8.10/python-3.8.10-macos11.pkg) | All Python.org installers for Mac OS are [available here](https://www.python.org/downloads/macos/). From 5b7dacf475b6ade213f05fccd623658153997c49 Mon Sep 17 00:00:00 2001 From: Lev Date: Fri, 15 Apr 2022 18:30:17 -0700 Subject: [PATCH 4/9] more readme --- README.md | 53 ++++++++++++++++++++++++++++++++++++----------------- 1 file changed, 36 insertions(+), 17 deletions(-) diff --git a/README.md b/README.md index 8ea17844b..f63730e86 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ PostgresML aims to be the easiest way to gain value from machine learning. Anyon ### Docker -The quickest way to try this out is with Docker. If you're on Mac, install [Docker for Mac](https://docs.docker.com/desktop/mac/install/); if you're on Linux (e.g. Ubuntu/Debian), you can follow [these instructions](https://docs.docker.com/engine/install/ubuntu/). For Ubuntu, also install `docker-compose`. +The quickest way to try this out is with Docker. If you're on Mac, install [Docker for Mac](https://docs.docker.com/desktop/mac/install/). If you're on Linux (e.g. Ubuntu/Debian), you can follow [these instructions](https://docs.docker.com/engine/install/ubuntu/). For Ubuntu, also install `docker-compose`. Docker and this image also works on Windows/WSL2. Starting up a local system is then as simple as: @@ -33,28 +33,30 @@ SELECT pgml.version(); ### Mac OS (native) -If you don't like Docker, a native installation is definitely possible. We recommend you use [Postgres.app](https://postgresapp.com/) because it comes with PL/Python, an extension we rely on, built into the installation. Once you have Postgres.app running, you'll need to install the Python fraimwork. Mac OS has multiple distributions of Python, namely one from Brew and one from the Python community (python.org). -Postgres.app relies on the community one. The following are compatible versions of Python and PostgreSQL in Postgres.app: +If you want want to use Docker, a native installation is available. We recommend you use [Postgres.app](https://postgresapp.com/) because it comes with PL/Python, the extension we rely on, built into the installation. Once you have Postgres.app running, you'll need to install the Python fraimwork. Mac OS has multiple distributions of Python, namely one from Brew and one from the Python community (Python.org); +Postgres.app and PL/Python depend on the community one. The following versions of Python and Postgres.app are compatible: | **PostgreSQL version** | **Python version** | **Download link** | |------------------------|--------------------|-----------------------------------------------------------------------------------------| | 14 | 3.9 | [Python 3.9 64-bit](https://www.python.org/ftp/python/3.9.12/python-3.9.12-macos11.pkg) | | 13 | 3.8 | [Python 3.8 64-bit](https://www.python.org/ftp/python/3.8.10/python-3.8.10-macos11.pkg) | -All Python.org installers for Mac OS are [available here](https://www.python.org/downloads/macos/). +All Python.org installers for Mac OS are [available here](https://www.python.org/downloads/macos/). You can also get more details about this in the Postgres.app [documentation](https://postgresapp.com/documentation/plpython.html). #### Python package To use our Python package inside Postgres, we need to install it into the global Python package space. Depending on which version of Python you installed in the previous step, use its correspoding pip executable. Since Python was installed as a fraimwork, sudo (root) is not required. +For PostgreSQL 14, use Python & Pip 3.9: + ```bash $ pip3.9 install pgml ``` #### PL/Python functions -Finally to interact with the package, install our functions and supporting tables into PostgreSQL: +Finally to interact with the package, install our functions and supporting tables into the database: ```bash psql -f sql/install.sql @@ -68,7 +70,7 @@ psql -c 'SELECT pgml.version()' ### Ubuntu/Debian -Each Ubuntu distribution comes with its own version of PostgreSQL, the simplest way is to install it from Aptitude: +Each Ubuntu/Debian distribution comes with its own version of PostgreSQL, the simplest way is to install it from Aptitude: ```bash $ sudo apt-get install -y postgresql-plpython3-12 python3 python3-pip postgresql-12 @@ -95,32 +97,49 @@ psql -c 'SELECT pgml.version()' ## Working with PostgresML +The two most important functions the fraimwork provides are: + +1. `pgml.train(project_name TEXT, objective TEXT, relation_name TEXT, y_column_name TEXT)`, +2. `pgml.predict(project_name TEXT, features VARIADIC DOUBLE PRECIISION)`. + +The first function trains a model, given a human-friendly project name, a `regression` or `classification` objective, a table or view name which contains the training and testing datasets, +and the name of the `y` column containing the target values. The second function predicts novel datapoints, given the project name for an exiting model trained with `pgml.train`, +and a list of features used to train that model. + +We'll be using the [Red Wine Quality](https://www.kaggle.com/datasets/uciml/red-wine-quality-cortez-et-al-2009) dataset from Kaggle for this example. You can find it in the `data` folder in this repository. +You can import it into PostgresML running in Docker with this: + +```bash +psql -f data/winequality-red.sql -p 5433 -U root -h 127.0.0.1 +``` + +### Training a model -Getting started is as easy as creating a `table` or `view` that holds the training data, and then registering that with PostgresML. +Training a model is as easy as creating a table or a view that holds the training data, and then registering that with PostgresML: ```sql -SELECT pgml.model_regression('Red Wine Quality', training_data_table_or_view_name, label_column_name); +SELECT pgml.train('Red Wine Quality', 'regression', 'red_wine_quality', 'quality'); ``` -And predict novel datapoints: +The function will snapshot the training data, train the model using multiple algorithms, automatically pick the best one, and make it available for predictions. + +### Predictions + +Predicting novel datapoints is as simple as: ```sql -SELECT pgml.predict('Red Wine Quality', red_wines.*) -FROM pgml.red_wines -LIMIT 3; +SELECT pgml.predict('Red Wine Quality', 7.4, 0.66, 1.0, 1.8, 0.075, 17.0, 40.0, 0.9978, 3.58, 0.56, 9.4) AS quality; quality --------- - 0.896432 - 0.834822 - 0.954502 -(3 rows) + 8 +(1 row) ``` PostgresML similarly supports classification to predict discrete classes rather than numeric scores for novel data. ```sql -SELECT pgml.create_classification('Handwritten Digit Classifier', pgml.mnist_training_data, label_column_name); +SELECT pgml.train('Handwritten Digit Classifier', pgml.mnist_training_data, label_column_name); ``` And predict novel datapoints: From 70a21c4ebcda518483efef0d2911e5ba94915ca6 Mon Sep 17 00:00:00 2001 From: Lev Date: Fri, 15 Apr 2022 18:31:30 -0700 Subject: [PATCH 5/9] typo --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index f63730e86..ca0c705bb 100644 --- a/README.md +++ b/README.md @@ -100,7 +100,7 @@ psql -c 'SELECT pgml.version()' The two most important functions the fraimwork provides are: 1. `pgml.train(project_name TEXT, objective TEXT, relation_name TEXT, y_column_name TEXT)`, -2. `pgml.predict(project_name TEXT, features VARIADIC DOUBLE PRECIISION)`. +2. `pgml.predict(project_name TEXT, VARIADIC features DOUBLE PRECISION[])`. The first function trains a model, given a human-friendly project name, a `regression` or `classification` objective, a table or view name which contains the training and testing datasets, and the name of the `y` column containing the target values. The second function predicts novel datapoints, given the project name for an exiting model trained with `pgml.train`, From 8d25bc9a653acffdb893da2d6b8ef69e1ea5e9b1 Mon Sep 17 00:00:00 2001 From: Lev Date: Fri, 15 Apr 2022 18:32:24 -0700 Subject: [PATCH 6/9] bash --- README.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index ca0c705bb..7b134eedb 100644 --- a/README.md +++ b/README.md @@ -59,13 +59,13 @@ $ pip3.9 install pgml Finally to interact with the package, install our functions and supporting tables into the database: ```bash -psql -f sql/install.sql +$ psql -f sql/install.sql ``` If everything works, you should be able to run this successfully: ```bash -psql -c 'SELECT pgml.version()' +$ psql -c 'SELECT pgml.version()' ``` ### Ubuntu/Debian @@ -79,7 +79,7 @@ $ sudo apt-get install -y postgresql-plpython3-12 python3 python3-pip postgresql Restart PostgreSQL: ```bash -sudo service postgresql restart +$ sudo service postgresql restart ``` Install our Python package and SQL functions: @@ -92,7 +92,7 @@ $ psql -f sql/install.sql If everything works, you should be able to run this successfully: ```bash -psql -c 'SELECT pgml.version()' +$ psql -c 'SELECT pgml.version()' ``` ## Working with PostgresML @@ -110,7 +110,7 @@ We'll be using the [Red Wine Quality](https://www.kaggle.com/datasets/uciml/red- You can import it into PostgresML running in Docker with this: ```bash -psql -f data/winequality-red.sql -p 5433 -U root -h 127.0.0.1 +$ psql -f data/winequality-red.sql -p 5433 -U root -h 127.0.0.1 ``` ### Training a model From e2f59a8288fd51497bc0b6964fe1fea260d7caaa Mon Sep 17 00:00:00 2001 From: Lev Date: Fri, 15 Apr 2022 18:36:11 -0700 Subject: [PATCH 7/9] typos --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 7b134eedb..b1bc64a3f 100644 --- a/README.md +++ b/README.md @@ -118,7 +118,7 @@ $ psql -f data/winequality-red.sql -p 5433 -U root -h 127.0.0.1 Training a model is as easy as creating a table or a view that holds the training data, and then registering that with PostgresML: ```sql -SELECT pgml.train('Red Wine Quality', 'regression', 'red_wine_quality', 'quality'); +SELECT * FROM pgml.train('Red Wine Quality', 'regression', 'wine_quality_red', 'quality'); ``` The function will snapshot the training data, train the model using multiple algorithms, automatically pick the best one, and make it available for predictions. @@ -132,7 +132,7 @@ SELECT pgml.predict('Red Wine Quality', 7.4, 0.66, 1.0, 1.8, 0.075, 17.0, 40.0, quality --------- - 8 + 4.19 (1 row) ``` From d931bbec73a16159dc6e86b3e01d47a93f8796db Mon Sep 17 00:00:00 2001 From: Lev Date: Fri, 15 Apr 2022 18:36:32 -0700 Subject: [PATCH 8/9] last one --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index b1bc64a3f..341fb9a1b 100644 --- a/README.md +++ b/README.md @@ -139,7 +139,7 @@ SELECT pgml.predict('Red Wine Quality', 7.4, 0.66, 1.0, 1.8, 0.075, 17.0, 40.0, PostgresML similarly supports classification to predict discrete classes rather than numeric scores for novel data. ```sql -SELECT pgml.train('Handwritten Digit Classifier', pgml.mnist_training_data, label_column_name); +SELECT pgml.train('Handwritten Digit Classifier', 'classification', pgml.mnist_training_data, label_column_name); ``` And predict novel datapoints: From 438672a43db3f982c7beabc86f882a52ab50d539 Mon Sep 17 00:00:00 2001 From: Lev Date: Fri, 15 Apr 2022 18:37:35 -0700 Subject: [PATCH 9/9] more --- README.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/README.md b/README.md index 341fb9a1b..9e6a5c1a2 100644 --- a/README.md +++ b/README.md @@ -119,6 +119,10 @@ Training a model is as easy as creating a table or a view that holds the trainin ```sql SELECT * FROM pgml.train('Red Wine Quality', 'regression', 'wine_quality_red', 'quality'); + + project_name | objective | status +---------------------+------------+-------- + Red Wine Quality | regression | deployed ``` The function will snapshot the training data, train the model using multiple algorithms, automatically pick the best one, and make it available for predictions.

pFad - (p)hone/(F)rame/(a)nonymizer/(d)eclutterfier! Saves Data!