Home

Benchmark Specifications and Results

Home » Benchmarks » Social Network Benchmark

The Social Network Benchmark consists in fact of three distinct benchmarks on a common dataset, since there are three different workloads. Each workload produces a single metric for performance at the given scale and a price/performance metric at the scale.  The full disclosure further breaks down the composition of the metric into its constituent parts, e.g. single query execution times.

  • Interactive Workload.  The Interactive SNB workload is the first one we are releasing, in draft stage. It is defined in plain text, yet we have example implementations in neo4j's Cypher, SPARQL and SQL. The interactive workloads tests a system's throughput with relatively simple queries with concurrent updates.  One could call the Interactive Workload an OLTP workload, but while queries typically touch a small fraction of the database, this can still be up to hundreds of thousands of values (the two-step neighborhood of a person in the social graph, often). 
  • Business Intelligence Workload. There is a first stab at this workload formulated in SPARQL, tested against Openlink Virtuoso. The BI workload consists of complex structured queries for analyzing online behavior of users for marketing purposes.  The workload stresses query execution and optimization. Queries typically touch a large fraction of the data and do not require repeatable read.  The queries will be concurrent with trickle load (not out yet). Unlike the interactive workload, the queries touch more data as the database grows.
  • Graph Analytics Workload. This workload is not yet available. It will test the functionality and scalability of the SUT for graph analytics that typically cannot be expressed in a query language. The workload is still under development, but will consist of algorithms like PageRank, Clustering and Breadth First Search. The analytics is done on most of the data in the graph as a single operation.  The analysis itself produces large intermediate results.  The analysis is not expected to be transactional or to have isolation from possible concurrent updates.

All the SNB scenarios share a common scalable synthetic data set, generated by a state-of-the art data generator. We strongly believe in a single dataset that makes sense for all workloads, that is, the interactive and BI workloads will traverse data that has sensible PageRank outcomes, and graph clustering structure, etc. This is in contrast to LinkBench, released by the team of Facebook that manages the OLTP workload on the Facebook Graph, which closely tunes to the low-level MySQL query patterns Facebook sees, but whose graph has a structure that is unrealistic (no community structures of correlations between values and structure).

The SNB specification contains the description of the benchmark and all information about its software components can be found on the SNB developer page

Social Network Benchmark (SNB) Audited Results

SF

Speed (ops/sec)

Cost
Software
Hardware
Test Sponsor
Date
10 101.20 30,427  ​Sparksee 5.1.1 2*(Xeon 2630v3 8-core 2.4GHz) 256GB RAM Sparsity Technologies SA 2015Apr27
30 1287.17 20,212 Virtuoso 07.50.3213 v7fasttrack 2*(Xeon2630 6-core 2.4GHz) 192GB RAM OpenLink Software 2015Apr27
30 86.50 30,427 ​Sparksee 5.1.1 2*(Xeon 2630v3 8-core 2.4GHz) 256GB RAM Sparsity Technologies SA 2015Apr27
100 1200.00 ​€20,212 Virtuoso 07.50.3213 v7fasttrack 2*(Xeon2630 6-core 2.4GHz) 192GB RAM OpenLink Software 2015Apr27
100 81.70 37,927 Sparksee 5.1.1 2*(Xeon 2630v3 8-core 2.4GHz) 256GB RAM Sparsity Technologies SA 2015Apr27
300 635 €20,212 Virtuoso 07.50.3213 v7fasttrack 2*(Xeon2630 6-core 2.4GHz) 192GB RAM OpenLink Software 2015Apr27