diff --git a/.gitignore b/.gitignore index 5bb4e110325..b091fbc48e0 100644 --- a/.gitignore +++ b/.gitignore @@ -13,6 +13,10 @@ tests/test_orca/images/linux/failed/ doc/python/raw.githubusercontent.com/ +docs/ +docs_tmp/ +pages/examples/ + # Don't ignore dataset files !*.csv.gz !*.geojson.gz diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index ad666ea805c..f4cfaf94447 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -8,7 +8,7 @@ then explains the technical aspects of preparing your contribution. ## Code of Conduct -Please note that all contributos are required to abide by our [Code of Conduct](CODE_OF_CONDUCT.md). +Please note that all contributos are required to abide by our Code of Conduct. ## Different Ways to Contribute @@ -19,7 +19,7 @@ it is important to understand the structure of the code and the repository. - The [`plotly.graph_objects`](https://plotly.com/python/graph-objects/) module (usually imported as `go`) is [generated from the plotly.js schema](https://plotly.com/python/figure-structure/), so changes to be made in this package need to be contributed to [plotly.js](https://github.com/plotly/plotly.js) - or to the code generation system in `./codegen/`. + or to the code generation system in `./bin/codegen/`. Code generation creates traces and layout classes that have a direct correspondence to their JavaScript counterparts, while higher-level methods that work on figures regardless of the current schema (e.g., `BaseFigure.for_each_trace`) are defined in `plotly/basedatatypes.py`. @@ -38,16 +38,17 @@ it is important to understand the structure of the code and the repository. These are organized in subdirectories according to what they test: see the "Setup" section below for more details. -- Documentation is found in `doc/`, and its structure is described in [its README file](doc/README.md). +- Documentation is found in `doc/`, and its structure is described in its README file. The documentation is a great place to start contributing, since you can add or modify examples without setting up a full environment. -Code and documentation are not the only way to contribute: -you can also help by: +Code and documentation are not the only way to contribute. +You can also help by: - Reporting bugs at . Please take a moment to see if your problem has already been reported, and if so, add a comment to the existing issue; - we will try to prioritize those that affect the most people. + we will try to prioritize those that affect the most people + and that are accompanied by small, runnable examples. - Submitting feature requests (also at ). Again, please add a comment to an existing issue if the feature you want has already been requested. @@ -219,11 +220,11 @@ Once you have done that, run the `updateplotlyjs` command: ```bash -python commands.py updateplotlyjs +python bin/updatejs.py ``` This downloads new versions of `plot-schema.json` and `plotly.min.js` from the `plotly/plotly.js` GitHub repository -and places them in `plotly/package_data`. +and places them in `resources` and `plotly/package_data` respectively. It then regenerates all of the `graph_objs` classes based on the new schema. ### Using a Development Branch of Plotly.js @@ -232,7 +233,8 @@ If your development branch is in [the plotly.js repository](https://github.com/p you can update to development versions of `plotly.js` with this command: ```bash -python commands.py updateplotlyjsdev --devrepo reponame --devbranch branchname +# FIXME commands.py didn't provide --devrepo or --devbranch +python bin/updatejs.py --dev --devrepo reponame --devbranch branchname ``` This fetches the `plotly.js` in the CircleCI artifact of the branch `branchname` of the repo `reponame`. @@ -255,5 +257,6 @@ You can then run the following command *in your local plotly.py repository*: ```bash -python commands.py updateplotlyjsdev --local /path/to/your/plotly.js/ +# FIXME: commands.py didn't provide --local +python bin/updatejs.py --dev --local /path/to/your/plotly.js/ ``` diff --git a/Makefile b/Makefile new file mode 100644 index 00000000000..56f5a2895ba --- /dev/null +++ b/Makefile @@ -0,0 +1,78 @@ +# Manage plotly.py project. + +RUN = uv run +PACKAGE_DIRS = _plotly_utils plotly +CODE_DIRS = ${PACKAGE_DIRS} scripts + +ifdef MKDOCS_ALL +EXAMPLE_SRC = $(wildcard doc/python/*.md) +else +EXAMPLE_SRC = doc/python/cone-plot.md doc/python/strip-charts.md +endif + +EXAMPLE_DST = $(patsubst doc/python/%.md,pages/examples/%.md,${EXAMPLE_SRC}) + +## commands: show available commands +commands: + @grep -h -E '^##' ${MAKEFILE_LIST} | sed -e 's/## //g' | column -t -s ':' + +## docs: rebuild documentation +.PHONY: docs +docs: + ${RUN} mkdocs build + +## docs-lint: check documentation +docs-lint: + ${RUN} pydoclint ${PACKAGE_DIRS} + +## docs-tmp: rebuild documentation saving Markdown in ./tmp +docs-tmp: + MKDOCS_TEMP_DIR=./docs_tmp ${RUN} mkdocs build + +## examples-batch: generate Markdown for all doc/python +examples-batch: + ${RUN} bin/run_markdown.py --outdir pages/examples --inline --verbose 1 ${EXAMPLE_SRC} + +## examples: generate Markdown for individual doc/python +examples: ${EXAMPLE_DST} + +pages/examples/%.md: doc/python/%.md + ${RUN} bin/run_markdown.py --outdir pages/examples --inline --verbose 2 $< + +## format: reformat code +format: + ${RUN} ruff format ${CODE_DIRS} + +## generate: generate code +generate: + ${RUN} bin/generate_code.py --codedir plotly + ${RUN} ruff format plotly + +## lint: check the code +lint: + ${RUN} ruff check ${CODE_DIRS} + +## test: run tests +test: + ${RUN} pytest tests + +## updatejs: update JavaScript bundle +updatejs: + ${RUN} bin/updatejs.py --codedir plotly + +## --: -- + +## clean: clean up repository +clean: + @find . -name '*~' -delete + @find . -name '.DS_Store' -delete + @rm -rf .coverage + @rm -rf .pytest_cache + @rm -rf .ruff_cache + @rm -rf dist + @rm -rf docs + @rm -rf pages/examples + +## sync: update Python packages +sync: + uv sync --extra dev diff --git a/README.md b/README.md index 11f117aca59..8e6ef158f46 100644 --- a/README.md +++ b/README.md @@ -76,13 +76,13 @@ Built on top of [plotly.js](https://github.com/plotly/plotly.js), `plotly.py` is ## Installation -plotly.py may be installed using pip +plotly.py may be installed using pip: ``` pip install plotly ``` -or conda. +or conda: ``` conda install -c conda-forge plotly @@ -90,8 +90,7 @@ conda install -c conda-forge plotly ### Jupyter Widget Support -For use as a Jupyter widget, install `jupyter` and `anywidget` -packages using `pip`: +For use as a Jupyter widget, install the `jupyter` and `anywidget` packages using `pip`: ``` pip install jupyter anywidget @@ -112,14 +111,14 @@ command line utility (legacy as of `plotly` version 4.9). #### Kaleido -The [`kaleido`](https://github.com/plotly/Kaleido) package has no dependencies and can be installed -using pip +The [`kaleido`](https://github.com/plotly/Kaleido) package has no dependencies +and can be installed using pip: ``` pip install -U kaleido ``` -or conda +or conda: ``` conda install -c conda-forge python-kaleido @@ -129,13 +128,13 @@ conda install -c conda-forge python-kaleido Some plotly.py features rely on fairly large geographic shape files. The county choropleth figure factory is one such example. These shape files are distributed as a -separate `plotly-geo` package. This package can be installed using pip... +separate `plotly-geo` package. This package can be installed using pip: ``` pip install plotly-geo==1.0.0 ``` -or conda +or conda: ``` conda install -c plotly plotly-geo=1.0.0 @@ -145,7 +144,7 @@ conda install -c plotly plotly-geo=1.0.0 ## Copyright and Licenses -Code and documentation copyright 2019 Plotly, Inc. +Code and documentation copyright Plotly, Inc. Code released under the [MIT license](https://github.com/plotly/plotly.py/blob/main/LICENSE.txt). diff --git a/bin/check-all-md.py b/bin/check-all-md.py new file mode 100644 index 00000000000..c06b00e3393 --- /dev/null +++ b/bin/check-all-md.py @@ -0,0 +1,17 @@ +from pathlib import Path +import os +import sys +from run_markdown import _parse_md + +TMP_FILE = "tmp.py" + +for filename in sys.argv[1:]: + content = Path(filename).read_text() + blocks = _parse_md(content) + for i, block in enumerate(blocks): + Path(TMP_FILE).write_text(block["code"].strip()) + sys.stdout.write(f"\n{'=' * 40}\n{filename}: {i}\n") + sys.stdout.flush() + sys.stdout.write(f"{'-' * 40}\n") + sys.stdout.flush() + os.system(f"python {TMP_FILE} > /dev/null") diff --git a/codegen/__init__.py b/bin/codegen/__init__.py similarity index 85% rename from codegen/__init__.py rename to bin/codegen/__init__.py index b299fa36045..294566f7f06 100644 --- a/codegen/__init__.py +++ b/bin/codegen/__init__.py @@ -1,8 +1,6 @@ import json import os -import os.path as opath import shutil -import subprocess import sys from codegen.datatypes import build_datatype_py, write_datatype_py # noqa: F401 @@ -87,45 +85,30 @@ def preprocess_schema(plotly_schema): items["colorscale"] = items.pop("concentrationscales") -def make_paths(outdir): - """Make various paths needed for formatting and linting.""" +def make_paths(codedir): + """Make various paths needed for code generation.""" - validators_dir = opath.join(outdir, "validators") - graph_objs_dir = opath.join(outdir, "graph_objs") - graph_objects_path = opath.join(outdir, "graph_objects", "__init__.py") + validators_dir = codedir / "validators" + graph_objs_dir = codedir / "graph_objs" + graph_objects_path = codedir / "graph_objects" / "__init__.py" return validators_dir, graph_objs_dir, graph_objects_path -def lint_code(outdir): - """Check Python code using settings in pyproject.toml.""" - - subprocess.call(["ruff", "check", *make_paths(outdir)]) - - -def reformat_code(outdir): - """Reformat Python code using settings in pyproject.toml.""" - - subprocess.call(["ruff", "format", *make_paths(outdir)]) - - -def perform_codegen(outdir, noformat=False): - """Generate code (and possibly reformat).""" +def perform_codegen(codedir, noformat=False): + """Generate code.""" # Get paths - validators_dir, graph_objs_dir, graph_objects_path = make_paths(outdir) + validators_dir, graph_objs_dir, graph_objects_path = make_paths(codedir) # Delete prior codegen output - if opath.exists(validators_dir): + if validators_dir.exists(): shutil.rmtree(validators_dir) - if opath.exists(graph_objs_dir): + if graph_objs_dir.exists(): shutil.rmtree(graph_objs_dir) # Load plotly schema - project_root = opath.dirname(outdir) - plot_schema_path = opath.join( - project_root, "codegen", "resources", "plot-schema.json" - ) - + project_root = codedir.parent + plot_schema_path = project_root / "resources" / "plot-schema.json" with open(plot_schema_path, "r") as f: plotly_schema = json.load(f) @@ -193,18 +176,18 @@ def perform_codegen(outdir, noformat=False): # Write out the JSON data for the validators os.makedirs(validators_dir, exist_ok=True) - write_validator_json(outdir, validator_params) + write_validator_json(codedir, validator_params) # Alls alls = {} # Write out datatypes for node in all_compound_nodes: - write_datatype_py(outdir, node) + write_datatype_py(codedir, node) # Deprecated # These are deprecated legacy datatypes like graph_objs.Marker - write_deprecated_datatypes(outdir) + write_deprecated_datatypes(codedir) # Write figure class to graph_objs data_validator = get_data_validator_instance(base_traces_node) @@ -212,7 +195,7 @@ def perform_codegen(outdir, noformat=False): frame_validator = frame_node.get_validator_instance() write_figure_classes( - outdir, + codedir, base_traces_node, data_validator, layout_validator, @@ -242,7 +225,7 @@ def perform_codegen(outdir, noformat=False): # Write plotly/graph_objs/graph_objs.py # This is for backward compatibility. It just imports everything from # graph_objs/__init__.py - write_graph_objs_graph_objs(outdir) + write_graph_objs_graph_objs(codedir) # Add Figure and FigureWidget root_datatype_imports = datatype_rel_class_imports[()] @@ -287,12 +270,13 @@ def __getattr__(import_name): # __all__ for path_parts, class_names in alls.items(): if path_parts and class_names: - filepath = opath.join(outdir, "graph_objs", *path_parts, "__init__.py") + filepath = codedir / "graph_objs" + filepath = filepath.joinpath(*path_parts) / "__init__.py" with open(filepath, "at") as f: f.write(f"\n__all__ = {class_names}") # Output datatype __init__.py files - graph_objs_pkg = opath.join(outdir, "graph_objs") + graph_objs_pkg = codedir / "graph_objs" for path_parts in datatype_rel_class_imports: rel_classes = sorted(datatype_rel_class_imports[path_parts]) rel_modules = sorted(datatype_rel_module_imports.get(path_parts, [])) @@ -317,18 +301,13 @@ def __getattr__(import_name): graph_objects_rel_classes, init_extra=optional_figure_widget_import, ) - graph_objects_path = opath.join(outdir, "graph_objects", "__init__.py") - os.makedirs(opath.join(outdir, "graph_objects"), exist_ok=True) + graph_objects_path = codedir / "graph_objects" + graph_objects_path.mkdir(parents=True, exist_ok=True) + graph_objects_path /= "__init__.py" with open(graph_objects_path, "wt") as f: f.write("# ruff: noqa: F401\n") f.write(graph_objects_init_source) - # Run code formatter on output directories - if noformat: - print("skipping reformatting") - else: - reformat_code(outdir) - if __name__ == "__main__": if len(sys.argv) != 2: diff --git a/codegen/compatibility.py b/bin/codegen/compatibility.py similarity index 93% rename from codegen/compatibility.py rename to bin/codegen/compatibility.py index 2b57685ff2e..fdb4fe4b576 100644 --- a/codegen/compatibility.py +++ b/bin/codegen/compatibility.py @@ -1,5 +1,4 @@ from io import StringIO -from os import path as opath from codegen.utils import write_source_py @@ -150,15 +149,15 @@ def build_deprecation_message(class_name, base_type, new): """ -def write_deprecated_datatypes(outdir): +def write_deprecated_datatypes(codedir): """ Build source code for deprecated datatype class definitions and write them to a file Parameters ---------- - outdir : - Root outdir in which the graph_objs package should reside + codedir : + Root directory in which the graph_objs package should reside Returns ------- @@ -166,13 +165,13 @@ def write_deprecated_datatypes(outdir): """ # Generate source code datatype_source = build_deprecated_datatypes_py() - filepath = opath.join(outdir, "graph_objs", "_deprecations.py") + filepath = codedir / "graph_objs" / "_deprecations.py" # Write file write_source_py(datatype_source, filepath) -def write_graph_objs_graph_objs(outdir): +def write_graph_objs_graph_objs(codedir): """ Write the plotly/graph_objs/graph_objs.py file @@ -183,14 +182,14 @@ def write_graph_objs_graph_objs(outdir): Parameters ---------- - outdir : str - Root outdir in which the graph_objs package should reside + codedir : str + Root directory in which the graph_objs package should reside Returns ------- None """ - filepath = opath.join(outdir, "graph_objs", "graph_objs.py") + filepath = codedir / "graph_objs" / "graph_objs.py" with open(filepath, "wt") as f: f.write( """\ diff --git a/codegen/datatypes.py b/bin/codegen/datatypes.py similarity index 98% rename from codegen/datatypes.py rename to bin/codegen/datatypes.py index 28b11d1fc59..acabac95da3 100644 --- a/codegen/datatypes.py +++ b/bin/codegen/datatypes.py @@ -1,6 +1,5 @@ -import os.path as opath -import textwrap from io import StringIO +import textwrap from codegen.utils import CAVEAT, write_source_py @@ -219,6 +218,9 @@ def _subplot_re_match(self, prop): else: property_docstring = property_description + # FIXME: replace '][' with ']\[' to avoid confusion with Markdown reference links + # property_docstring = property_docstring.replace("][", "]\\[") + # Write get property buffer.write( f'''\ @@ -595,14 +597,6 @@ def write_datatype_py(outdir, node): None """ - # Build file path - # filepath = opath.join(outdir, "graph_objs", *node.parent_path_parts, "__init__.py") - filepath = opath.join( - outdir, "graph_objs", *node.parent_path_parts, "_" + node.name_undercase + ".py" - ) - - # Generate source code + filepath = (outdir / "graph_objs").joinpath(*node.parent_path_parts) / f"_{node.name_undercase}.py" datatype_source = build_datatype_py(node) - - # Write file write_source_py(datatype_source, filepath, leading_newlines=2) diff --git a/codegen/figure.py b/bin/codegen/figure.py similarity index 99% rename from codegen/figure.py rename to bin/codegen/figure.py index a15d806937c..b0ed1026793 100644 --- a/codegen/figure.py +++ b/bin/codegen/figure.py @@ -1,5 +1,4 @@ from io import StringIO -from os import path as opath from codegen.datatypes import ( reindent_validator_description, @@ -705,7 +704,7 @@ def add_{method_prefix}{singular_name}(self""" def write_figure_classes( - outdir, + codedir, trace_node, data_validator, layout_validator, @@ -720,8 +719,8 @@ def write_figure_classes( Parameters ---------- - outdir : str - Root outdir in which the graph_objs package should reside + codedir : str + Root directory in which the graph_objs package should reside trace_node : PlotlyNode Root trace node (the node that is the parent of all of the individual trace nodes like bar, scatter, etc.) @@ -768,5 +767,5 @@ def write_figure_classes( ) # Format and write to file - filepath = opath.join(outdir, "graph_objs", f"_{fig_classname.lower()}.py") + filepath = codedir / "graph_objs" / f"_{fig_classname.lower()}.py" write_source_py(figure_source, filepath) diff --git a/codegen/utils.py b/bin/codegen/utils.py similarity index 99% rename from codegen/utils.py rename to bin/codegen/utils.py index 3d660328e51..c002574ada7 100644 --- a/codegen/utils.py +++ b/bin/codegen/utils.py @@ -1,11 +1,9 @@ -import os -import os.path as opath -import textwrap from collections import ChainMap from importlib import import_module from io import StringIO -from typing import List import re +import textwrap +from typing import List CAVEAT = """ @@ -35,10 +33,7 @@ def write_source_py(py_source, filepath, leading_newlines=0): """ if py_source: # Make dir if needed - filedir = opath.dirname(filepath) - # The exist_ok kwarg is only supported with Python 3, but that's ok since - # codegen is only supported with Python 3 anyway - os.makedirs(filedir, exist_ok=True) + filepath.parent.mkdir(exist_ok=True) # Write file py_source = "\n" * leading_newlines + py_source @@ -121,7 +116,7 @@ def write_init_py(pkg_root, path_parts, rel_modules=(), rel_classes=(), init_ext init_source = build_from_imports_py(rel_modules, rel_classes, init_extra) # Write file - filepath = opath.join(pkg_root, *path_parts, "__init__.py") + filepath = pkg_root.joinpath(*path_parts) / "__init__.py" write_source_py(init_source, filepath) @@ -168,6 +163,9 @@ def format_description(desc): # replace {2D arrays} with 2D lists desc = desc.replace("{2D arrays}", "2D lists") + # FIXME: replace '][' with ']\[' to avoid confusion with Markdown reference links + # desc = desc.replace("][", r"]\\[") + return desc diff --git a/codegen/validators.py b/bin/codegen/validators.py similarity index 94% rename from codegen/validators.py rename to bin/codegen/validators.py index 4cef19fa29b..04ea65d2f8a 100644 --- a/codegen/validators.py +++ b/bin/codegen/validators.py @@ -1,4 +1,3 @@ -import os.path as opath import json import _plotly_utils.basevalidators @@ -54,7 +53,7 @@ def get_data_validator_params(base_trace_node: TraceNode, store: dict): } -def write_validator_json(outdir, params: dict): +def write_validator_json(codedir, params: dict): """ Write out a JSON serialization of the validator arguments for all validators (keyed by f"{parent_name}.{plotly_name}) @@ -64,8 +63,8 @@ def write_validator_json(outdir, params: dict): Parameters ---------- - outdir : str - Root outdir in which the validators package should reside + codedir : str + Root directory in which the validators package should reside params : dict Dictionary to store the JSON data for the validator Returns @@ -78,7 +77,7 @@ def write_validator_json(outdir, params: dict): raise ValueError("Expected params to be a dictionary") # Write file - filepath = opath.join(outdir, "validators", "_validators.json") + filepath = codedir / "validators" / "_validators.json" with open(filepath, "w") as f: f.write(json.dumps(params, indent=4)) diff --git a/bin/generate_code.py b/bin/generate_code.py new file mode 100644 index 00000000000..94fef3991cf --- /dev/null +++ b/bin/generate_code.py @@ -0,0 +1,27 @@ +"""Generate code.""" + +import argparse +from pathlib import Path + +import utils + + +def main(): + """Main driver.""" + + args = parse_args() + codedir = utils.select_code_directory(args) + utils.perform_codegen(codedir, noformat=args.noformat) + + +def parse_args(): + """Parse command-line arguments.""" + + parser = argparse.ArgumentParser() + parser.add_argument("--noformat", action="store_true", help="prevent reformatting") + parser.add_argument("--codedir", type=Path, help="code directory") + return parser.parse_args() + + +if __name__ == "__main__": + main() diff --git a/bin/generate_reference_pages.py b/bin/generate_reference_pages.py new file mode 100644 index 00000000000..cba3d633276 --- /dev/null +++ b/bin/generate_reference_pages.py @@ -0,0 +1,58 @@ +"""Generate the code reference pages and navigation.""" + +import os +from pathlib import Path + +import mkdocs_gen_files + + +# Saving Markdown files? +temp_dir = os.getenv("MKDOCS_TEMP_DIR", None) +if temp_dir is not None: + temp_dir = Path(temp_dir) + +# Set up the generation engine. +nav = mkdocs_gen_files.Nav() + +# Match each Python file. +for path in sorted(Path("plotly").rglob("*.py")): + # Documentation path. + module_path = path.relative_to(".").with_suffix("") + doc_path = path.relative_to(".").with_suffix(".md") + full_doc_path = Path("reference", doc_path) + + # Handle dunder special cases. + parts = tuple(module_path.parts) + if parts[-1] == "__init__": + parts = parts[:-1] + doc_path = doc_path.with_name("index.md") + full_doc_path = full_doc_path.with_name("index.md") + elif parts[-1] == "__main__": + continue + + # Save constructed data. + nav[parts] = doc_path.as_posix() + mkdocs_gen_files.set_edit_path(full_doc_path, path) + + # Save in-memory file. + with mkdocs_gen_files.open(full_doc_path, "w") as writer: + ident = ".".join(parts) + writer.write(f"# {ident}\n\n") + writer.write(f"::: {ident}") + + # Save to disk if requested. + if temp_dir is not None: + temp_path = temp_dir / doc_path + temp_path.parent.mkdir(exist_ok=True, parents=True) + with open(temp_path, "w") as writer: + ident = ".".join(parts) + writer.write(f"# {ident}\n\n") + writer.write(f"::: {ident}") + +# Generate navigation summary. +with mkdocs_gen_files.open("reference/SUMMARY.md", "w") as writer: + writer.writelines(nav.build_literate_nav()) +if temp_dir is not None: + temp_path = temp_dir / "SUMMARY.md" + with open(temp_path, "w") as writer: + writer.writelines(nav.build_literate_nav()) diff --git a/bin/run_markdown.py b/bin/run_markdown.py new file mode 100644 index 00000000000..d427c03c650 --- /dev/null +++ b/bin/run_markdown.py @@ -0,0 +1,255 @@ +#!/usr/bin/env python3 +""" +Process Markdown files with embedded Python code blocks, saving +the output and images. +""" + +import argparse +from contextlib import redirect_stdout, redirect_stderr +import io +from pathlib import Path +import plotly.graph_objects as go +import sys +import traceback + + +def main(): + args = _parse_args() + for filename in args.inputs: + _do_file(args, Path(filename)) + + +def _do_file(args, input_file): + """Process a single file.""" + + # Validate input file + if not input_file.exists(): + print(f"Error: '{input_file}' not found", file=sys.stderr) + sys.exit(1) + + # Determine output file path etc. + stem = input_file.stem + output_file = args.outdir / f"{input_file.stem}{input_file.suffix}" + if input_file.resolve() == output_file.resolve(): + print(f"Error: output would overwrite input '{input_file}'", file=sys.stderr) + sys.exit(1) + + # Read input + try: + with open(input_file, "r", encoding="utf-8") as f: + content = f.read() + except Exception as e: + print(f"Error reading input file: {e}", file=sys.stderr) + sys.exit(1) + + # Parse markdown and extract code blocks + _report(args.verbose > 0, f"Processing {input_file}...") + code_blocks = _parse_md(content) + _report(args.verbose > 1, f"- Found {len(code_blocks)} code blocks") + + # Execute code blocks and collect results + execution_results = [] + figure_counter = 0 + for i, block in enumerate(code_blocks): + _report(args.verbose > 1, f"- Executing block {i + 1}/{len(code_blocks)}") + figure_counter, result = _run_code(block["code"], args.outdir, stem, figure_counter) + execution_results.append(result) + _report(args.verbose > 0 and bool(result["error"]), f" - Warning: block {i + 1} had an error") + _report(args.verbose > 1 and bool(result["images"]), f" - Generated {len(result['images'])} image(s)") + + # Generate and save output + content = _generate_markdown(args, content, code_blocks, execution_results, args.outdir) + try: + with open(output_file, "w", encoding="utf-8") as f: + f.write(content) + _report(args.verbose > 1, f"- Output written to {output_file}") + _report(args.verbose > 1 and any(result["images"] for result in execution_results), f"- Images saved to {args.outdir}") + except Exception as e: + print(f"Error writing output file: {e}", file=sys.stderr) + sys.exit(1) + + +def _capture_plotly_show(fig, counter, result, output_dir, stem): + """Saves figures instead of displaying them.""" + # Save PNG + png_filename = f"{stem}_{counter}.png" + png_path = output_dir / png_filename + fig.write_image(png_path, width=800, height=600) + result["images"].append(png_filename) + + # Save HTML and get the content for embedding + html_filename = f"{stem}_{counter}.html" + html_path = output_dir / html_filename + fig.write_html(html_path, include_plotlyjs="cdn") + html_content = fig.to_html(include_plotlyjs="cdn", div_id=f"plotly-div-{counter}", full_html=False) + result["html_files"].append(html_filename) + result.setdefault("html_content", []).append(html_content) + + +def _generate_markdown(args, content, code_blocks, execution_results, output_dir): + """Generate the output markdown with embedded results.""" + lines = content.split("\n") + + # Sort code blocks by start line in reverse order for safe insertion + sorted_blocks = sorted( + enumerate(code_blocks), key=lambda x: x[1]["start_line"], reverse=True + ) + + # Process each code block and insert results + for block_idx, block in sorted_blocks: + result = execution_results[block_idx] + insert_lines = [] + + # Add output if there's stdout + if result["stdout"].strip(): + insert_lines.append("") + insert_lines.append("**Output:**") + insert_lines.append("```") + insert_lines.extend(result["stdout"].rstrip().split("\n")) + insert_lines.append("```") + + # Add error if there was one + if result["error"]: + insert_lines.append("") + insert_lines.append("**Error:**") + insert_lines.append("```") + insert_lines.extend(result["error"].rstrip().split("\n")) + insert_lines.append("```") + + # Add stderr if there's content + if result["stderr"].strip(): + insert_lines.append("") + insert_lines.append("**Warnings/Messages:**") + insert_lines.append("```") + insert_lines.extend(result["stderr"].rstrip().split("\n")) + insert_lines.append("```") + + # Add images + for image in result["images"]: + insert_lines.append("") + insert_lines.append(f"![Generated Plot](./{image})") + + # Embed HTML content for plotly figures + if args.inline: + for html_content in result.get("html_content", []): + insert_lines.append("") + insert_lines.append("**Interactive Plot:**") + insert_lines.append("") + insert_lines.extend(html_content.split("\n")) + + # Insert the results after the code block + if insert_lines: + # Insert after the closing ``` of the code block + insertion_point = block["end_line"] + 1 + lines[insertion_point:insertion_point] = insert_lines + + return "\n".join(lines) + + +def _parse_args(): + """Parse command-line arguments.""" + parser = argparse.ArgumentParser(description="Process Markdown files with code blocks") + parser.add_argument("inputs", nargs="+", help="Input .md files") + parser.add_argument("--inline", action="store_true", help="Inline HTML in .md") + parser.add_argument("--outdir", type=Path, help="Output directory") + parser.add_argument("--verbose", type=int, default=0, help="Integer verbosity level") + return parser.parse_args() + + +def _parse_md(content): + """Parse Markdown and extract Python code blocks.""" + lines = content.split("\n") + blocks = [] + current_block = None + in_code_block = False + in_region_block = False + + for i, line in enumerate(lines): + # Check for region start/end markers + if "

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy