LDBC Semantic Publishing Benchmark (LDBC-SPB)

The Semantic Publishing Benchmark (SPB) is an LDBC benchmark for testing the performance of RDF engines inspired by the Media/Publishing industry. In particular, LDBC worked with British Broadcasting Corporation BBC to define this benchmark, for which BBC donated workloads, ontologies and data. The publishing industry is an area where significant adoption of RDF is taking place.

There have been many academic benchmarks for RDF but none of these are truly industrial-grade. The SPB combines a set of complex queries under inference with continuous updates and special failover tests for systems implementing replication.

SPB performance is measured by producing a workload of CRUD (Create, Read, Update, Delete) operations which are executed simultaneously. The benchmark offers a data generator that uses real reference data to produce datasets of various sizes and tests the scalability aspect of RDF systems. The benchmark workload consists of (a) editorial operations that add new data, alter or delete existing (b) aggregation operations that retrieve content according to various criteria. The benchmark also tests conformance for various rules inside the OWL2-RL rule-set.

The SPB specification contains the description of the benchmark and the data generator and all information about its software components can be found on the SPB developer page.

Semantic Publishing Benchmark (SPB) Audited Results

SF Triples RW Agents Interactive (Q/sec) Updates (ops/sec) Cost Software Hardware Test Sponsor Date FDR
3
256M
16 / 4
335.48
25.66
177,474 USD
GraphDB EE 10.0.1 AWS r6id.8xlarge Ontotext AD FDR, summary
3
256M
24 / 0
413.16
0.00
207,474 USD
GraphDB EE 10.0.1 AWS r6id.8xlarge Ontotext AD FDR, summary
3
256M
64 / 4
1121.76
9.53
652,422 USD
GraphDB EE 10.0.1 3×AWS r6id.8xlarge Ontotext AD FDR, summary
3
256M
64 / 0
985.63
0.00
562,422 USD
GraphDB EE 10.0.1 3×AWS r6id.8xlarge Ontotext AD FDR, summary
5
1B
16 / 4
105.76
10.45
177,474 USD
GraphDB EE 10.0.1 AWS r6id.8xlarge Ontotext AD FDR, summary
5
1B
24 / 0
158.10
0.00
207,474 USD
GraphDB EE 10.0.1 AWS r6id.8xlarge Ontotext AD FDR, summary
5
1B
64 / 4
372.56
4.04
652,422 USD
GraphDB EE 10.0.1 3×AWS r6id.8xlarge Ontotext AD FDR, summary
5
1B
64 / 0
408.68
0.00
562,422 USD
GraphDB EE 10.0.1 3×AWS r6id.8xlarge Ontotext AD FDR, summary
1
64M
8 / 2
100.85
10.19
37,504 EUR
GraphDB EE 6.2 Intel Xeon E5-1650v3 6×3.5Ghz, 96GB RAM Ontotext AD FDR
1
64M
8 / 2
142.76
10.67
35,323 EUR
GraphDB SE 6.3 alpha Intel Xeon E5-1650v3 6×3.5GHz, 64GB RAM Ontotext AD FDR
3
256M
8 / 2
29.90
9.50
37,504 EUR
GraphDB EE 6.2 Intel Xeon E5-1650v3 6×3.5Ghz, 96GB RAM Ontotext AD FDR
3
256M
8 / 2
54.64
9.50
35,323 EUR
GraphDB SE 6.3 alpha Intel Xeon E5-1650v3 6×3.5GHz, 64GB RAM Ontotext AD FDR
1
64M
22 / 2
149.04
156.83
20,213 USD
Virtuoso v7.50.3213 Intel Xeon E5-2630 6×2.30GHz, 192 GB RAM OpenLink Software FDR
3
256M
22 / 2
80.62
92.71
20,213 USD
Virtuoso v7.50.3213 Intel Xeon E5-2630 6×2.30GHz, 192 GB RAM OpenLink Software FDR
3
256M
30 / 3
115.38
109.85
24,528 USD
Virtuoso v7.50.3213 AWS r3.8xlarge OpenLink Software FDR
5
1B
22 / 2
32.28
72.72
20,213 USD
Virtuoso v7.50.3213 Intel Xeon E5-2630 6×2.30GHz, 192 GB RAM OpenLink Software FDR
5
1B
30 / 3
45.81
55.45
24,528 USD
Virtuoso v7.50.3213 AWS r3.8xlarge OpenLink Software FDR

LDBC-certified auditors

SPB audits can be commissioned from the following LDBC-certified auditors:

  • Pjotr Scholtze