Similarity, embeddings and ML pipelines
|
This section documents the current Neo4j This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production, as Neo4j iterates quickly. This section’s bibliography lists the reference material consulted while preparing these pages. |
Beyond path-finding and centrality, the Graph Data Science (GDS) library scores pairwise similarity between nodes, projects a graph’s structure into fixed-length vectors, and wraps feature engineering plus training into repeatable ML pipelines. All three build on the same in-memory projected graph used by other GDS algorithms.
Node Similarity and KNN
Node Similarity computes an exact pairwise similarity score (Jaccard, overlap, or cosine by default) between nodes that share a relationship type to a common set of neighbors — the classic "customers who bought similar products" or "actors who co-starred with the same people" pattern. It is exhaustive, so it is best on graphs where the compared node set is not huge. KNN (k-nearest neighbors) approximates the same kind of scoring using a randomized nearest-neighbor search, trading a small amount of accuracy for scaling to much larger candidate sets, and it also accepts pre-computed embedding properties as its similarity input instead of relationship overlap.
// Co-starring pattern: Person nodes that acted in overlapping sets of movies,
// found via shared ACTED_IN neighborhoods.
CALL gds.graph.project(
'people-movies',
['Person', 'Movie'],
{ ACTED_IN: { orientation: 'UNDIRECTED' } }
);
CALL gds.nodeSimilarity.stream('people-movies', {
topK: 10,
similarityCutoff: 0.1
})
YIELD node1, node2, similarity
RETURN gds.util.asNode(node1).name AS person1,
gds.util.asNode(node2).name AS person2,
similarity
ORDER BY similarity DESC
LIMIT 20;
// KNN over the same projection -- approximate, scales further, and can run
// directly against an already-written embedding property instead of topology.
CALL gds.knn.stream('people-movies', {
nodeLabels: ['Person'],
nodeProperties: ['embedding'],
topK: 5,
sampleRate: 0.5
})
YIELD node1, node2, similarity
RETURN gds.util.asNode(node1).name AS person1,
gds.util.asNode(node2).name AS person2,
similarity
ORDER BY similarity DESC
LIMIT 20;
Both algorithms are also available as .write and .mutate variants to persist a SIMILAR_TO relationship
or an in-memory relationship for downstream use, and both are covered under
GDS Algorithms, which groups similarity and
embedding algorithms together in the same reference.
Embeddings: FastRP and Node2Vec versus GraphSAGE
FastRP and Node2Vec are transductive: they produce a fixed vector per node in the graph they were run on,
purely from structure (and optionally node properties), and are fast enough to run on large graphs as a
one-off mutate/write step. GraphSAGE is inductive — it trains a model over local neighborhood
aggregation functions, so once trained it can embed nodes it never saw during training (a node added after
training, or a disjoint sub-graph) without retraining. Reach for FastRP or Node2Vec for a quick, cheap
structural embedding to feed clustering, KNN or a pipeline’s feature list; reach for GraphSAGE when new nodes
will need embeddings on an ongoing basis without a full re-run, at the cost of a heavier, model-based training
step.
// FastRP: fast structural embedding, mutated onto the in-memory graph as a
// property so it can feed KNN, clustering, or a pipeline feature step.
CALL gds.fastRP.mutate('people-movies', {
embeddingDimension: 128,
randomSeed: 42,
mutateProperty: 'embedding'
})
YIELD nodeCount, nodePropertiesWritten;
// GraphSAGE: train an inductive model, then embed nodes not seen at train time.
CALL gds.beta.graphSage.train('people-movies', {
modelName: 'actorGraphSage',
featureProperties: ['embedding'],
aggregator: 'mean',
sampleSizes: [25, 10]
});
CALL gds.beta.graphSage.mutate('people-movies', {
modelName: 'actorGraphSage',
mutateProperty: 'sageEmbedding'
})
YIELD nodeCount, nodePropertiesWritten;
GDS ML pipelines
A GDS pipeline packages the repetitive parts of a supervised task — feature engineering, train/test
splitting, model selection and training — into a reusable, named object. The two pipeline types are
node classification (predict a label on each node, e.g. fraud/not-fraud) and link prediction (predict
whether an edge should exist between two nodes, e.g. "will these two accounts transact"). Both follow the
same shape: add one or more feature steps (often embeddings or graph algorithm outputs), configure a
train/test split, add one or more candidate models, train to pick and fit the best one, then run the
trained model with .predict.
// Link prediction pipeline sketch: FastRP-derived features, a logistic
// regression candidate, then training on the projected graph.
CALL gds.beta.pipeline.linkPrediction.create('coStarPipeline');
CALL gds.beta.pipeline.linkPrediction.addFeature('coStarPipeline', 'hadamard', {
nodeProperties: ['embedding']
});
CALL gds.beta.pipeline.linkPrediction.configureSplit('coStarPipeline', {
testFraction: 0.2,
trainFraction: 0.6,
validationFolds: 3
});
CALL gds.beta.pipeline.linkPrediction.addLogisticRegression('coStarPipeline');
CALL gds.beta.pipeline.linkPrediction.train('people-movies', {
pipeline: 'coStarPipeline',
modelName: 'coStarModel',
targetRelationshipType: 'ACTED_IN',
sourceNodeLabel: 'Person',
targetNodeLabel: 'Person'
})
YIELD modelInfo
RETURN modelInfo.bestParameters;
The node classification pipeline mirrors this shape (gds.beta.pipeline.nodeClassification.create,
.addNodeProperty/.selectFeatures, .addLogisticRegression or .addRandomForest, .train) but predicts a
node property instead of an edge. A dedicated treatment of graph algorithms for data science, and of GDS with
Neo4j specifically, exists — see the bibliography.
See GDS Machine Learning Pipelines for the full pipeline catalog, including model configuration and evaluation metrics. Embeddings produced here are also the natural feature source for Vector Search and GenAI, where they get indexed for approximate nearest-neighbor retrieval instead of, or alongside, KNN.