Skip to main content

Release notes for v2.37.11

Graphistry Staff
Updated

Highlights

  • Features
    • PyGraphistry: GPU hypergraphs
    • API: Hypergraphs
    • Custom email config
  • Infrastructure
    • RAPIDS 0.19 - Including deprecation warning for CUDA 10.2
    • General dependency upgrades
  • Admin
    • status.graphistry.com: Uptime tracker
    • Architecture docs
  • Fixes & tweaks
    • test-gpu.sh: Support CUDA_SHORT_VERSION tag
    • pivot health check: No longer requires auth
    • pivot fixes
    • dashboard update: Added more introductory resources
    • air gapping: Removed all remaining known calls to public web services
    • air gapping: Added optional AIR_GAPPED=1 environment flag and admin-panel setting IS_AIR_GAPPED
    • parquet uploads: support in no-code file uploader
    • parquet uploads: auto-decompress file.parquet.gz
    • tolerate duplicated nodes: auto-deduplicate
    • xls uploads: handle bigger datasets
    • CSV character encodings
    • File & viz gallery UI

Graphistry Versions

Server 2.37.11
JS React+Vanilla 3.7.6
Python PyGraphistry client 0.17.2 -> 0.18.2

Third-Party Versions

Caddy 2.3.0
CUDA (In-Docker) 11.0 (+ 10.2 deprecated)
Django 3.1
Docker (CE)
19.03.2
Docker Compose 1.24.1
Elasticsearch node driver 14.2.2
Gremlinpython 3.4.8
Jupyter 1.0.0
Neo4j-python-driver 4.1.0
NodeJS 14.15.4 -> 14.6.1
Pandas 1.1 -> 1.2
Postgres 12.5
Python 3.7.7 -> 3.7.10
Neo4j node driver 4.1.1
Notebook 6.2.0
RAPIDS 0.18 -> 0.19
Arrow 1.0.1
Splunk node SDK 1.9.0
Tornado 6.1

 

Features

  • Hypergraph file uploader: The file uploader has been upgraded to support hypergraph capabilities:
    • An inferred visualization will (generally) start appearing as soon as you upload a file, meaning you can focus more on tweaking than on initial authoring 
    • Use multiple columns for nodes, not just a src/dst pair
    • Optionally, connect row values to a row ID node rather than to one another, which is often useful for event data and sample data
    • Control which edges get created for 1 row of data. Ex: ip->domain, ip->user, and user->email
    • Control whether to merge nodes from different columns, e.g., merge values from src_ip and dest_ip columns as part of the same category "ip", and keep them distinct from a product_id column
  • PyGraphistry: The hypergraph method has been optimized:
    • Pandas: Vectorized
    • GPU support: specify `engine="cudf"`, and optionally, pass in a cudf.DataFrame
    • Added support for bigger-than-memory: `engine="dask"`, `engine="dask_cudf"`
  • API: Datasets can now specify a hypergraph transform for turning tabular data into graphs. This was previously only possible with the Python client, which limited where it could be done, and caused performance issues for the client and networking layers
  • XLS/XLSX: Larger XLS/XLSX files are now supported. Most notably, this fixes a bug on the uploader UI, and on the backend, adds 1-CPU-per-sheet parallel processing. The overall experience will still be slower than fast file formats, most notably during initial upload.
  • Custom email config: Self-hosting users can now specify a custom SMTP, Sendgrid, or Mailgun configuration when using email-based workflows. Will be documented in the following release; contact staff for interim guidance.

Infrastructure

  • RAPIDS 0.19: See official blogpost
  • CUDA 10.2 Deprecated: RAPIDS 0.19 is the last version of RAPIDS that supports CUDA < 11.0 . That means this and the next 1-2 releases of Graphistry will work with CUDA 10.2, and around early/mid-June, we expect to require CUDA >= 11.0
  • General system upgrades: See version update list. Addresses potential CVEs in Linux, Python, JavaScript.

Admin

  • status.graphistry.com: Public uptime monitor of our various public endpoints, including Graphistry Hub . Incident status will be communicated in Slack.
  • Architecture docs: See admin site

Fixes & tweaks

  • test-gpu.sh: Script etc/scripts/test-gpu.sh will now use the CUDA_SHORT_VERSION (10.2, 11.0, ...) of your release
  • pivot health checks: /pivot/health no longer requires being logged in
  • pivot fixes: Investigation cloning and template instantiation bugs were fixed; unclear if experienced in practice (impacting versions 2.34.*-2.36.*)
  • dashboard update: Added links to graph-app-kit, pygraphistry, and other resources
  • air gapping: Removed all known external calls to several js/css libraries that makes the UI unresponsive in some environments. Exceptions like ZenDesk remain enabled by default.
  • air gapping: Added the ability to enable explicit air gapping to disable optional external dependencies
    • Effect: This disables optional components like our support integration use the public internet
    • Benefit: Increase privacy, and potentially avoid usage slowdowns from waiting on non-responsive requests
    • Method 1: Before first system start, in your data/config/custom.env, set AIR_GAPPED=1
    • Method 2: User -> Admin portal -> Settings -> Config -> Check IS_AIR_GAPPED and save
  • parquet uploads: support in no-code file uploader
  • parquet uploads: auto-decompress file.parquet.gz into file.parquet
  • tolerate duplicated nodes: auto-deduplicate
  • file & viz gallery UI: Sorted by date by default + digest download links 
  • CSV character codings: The uploader now defaults to UTF-8 and attempts to infer alternate encodings; API users can now specify encoding names when not using UTF-8

Migration

No breaking changes; upgrade as usual

RAPIDS 0.20 requires CUDA >= 11.0. That means while the current release of Graphistry supports 10.2, the upcoming RAPIDS 0.20 might not: if you have not upgraded your host environment to 11.0, you should prepare to. We expect the RAPIDS 0.20 version of Graphistry to be earl/mid-June, and 1-2 additional releases of Graphistry before then.

 

Comments

0 comments

Article is closed for comments.

Powered by Zendesk