Graph based enterprise solution patterns

Let’s define some of the terms that we will use in this discussion.

Graph — A data structure used to model pairwise relationships between objects. (Vertices and edges)

Graph algorithms or analytics — analytic tools used to determine strength and direction of relationships between objects in a graph. The focus of graph analytics is on pairwise relationship between two objects at a time and structural characteristics of the graph as a whole.

Graph database — Technology for storage and query or analytical processing. This is a niche space, targeting real-time graph analysis, geared more towards lighter weight workloads that hit parts of the graph. A full graph analysis is possible but that

• LPG (Labelled Property Graph) — Each vertex and relationship has a set of key-value pairs (properties).
• RDF (Resource Description Framework) — W3C specification for knowledge representation. This is an abstract model where data is stored as triples (subject-predicate-object on relational or triple stores) and with a focus on semantics, discrete logic and inference.

Knowledge Graph — a System that acquires and integrates information into an ontology and applies a reasoner to derive new knowledge. The data is organized as a semantic graph; meaning is encoded with the data

This discussion is centered around the knowledge representation on the modern LPG graph databases, where some of the much coveted enterprise capabilities are becoming a common place.

• Graph traversal performance is directly correlated to how much data is on the RAM, so they are getting increasingly faster on capable infrastructure.
• Can scale both in terms of distributed storage and compute strategy (map-reduce like). Only a few are truly distributed but most can scale pretty well to deal with <1TB of data.
• Some are ACID compliant (!).
• A few promising ones are even OpenSource!

However, this is a fast evolving space where newer and better products are being cranked out every year. The space is a wild-wild-west but we are seeing a slow and gradual shift towards a common standard for a SQL like query language, etc.

Common enterprise solution patterns that benefit from the use of Graphs

Graphs are specialized data structures that are very good at enabling certain types of analysis patterns but the returns diminish quickly if we attempt to turn them to general purpose data lakes or BI platforms or narrow data lookup service or search service or data management platforms.

Some of the areas where graphs have been be successfully utilized in enterprise solutions include the following.

1. Representing complex and highly connected data domains
2. Lot of Joins are involved in answering a question (Segmentation, etc.)
3. Building Journey views (longitudinal connected views)
4. Real time predictions (powered by link analysis, similarity analysis, etc.)
5. Real time surveillance (using a combination of techniques listed above)

I must note that a lot of common analysis techniques where graphs are a natural fit, such as, Inference or Clustering or Path analysis, etc fit within #4 or #5 listed above.