Benchmarks

Go

The overhead of policy enforcement is benchmarked in model_b_test.go. The testbed is:

Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz, 2601 Mhz, 4 Core(s), 8 Logical Processor(s)

The benchmarking result of go test -bench=. -benchmem is as follows (op = an Enforce() call, ms = millisecond, KB = kilo bytes):

Test case	Rule size	Time overhead (ms/op)	Memory overhead (KB)
ACL	2 rules (2 users)	0.015493	5.649
RBAC	5 rules (2 users, 1 role)	0.021738	7.522
RBAC (small)	1100 rules (1000 users, 100 roles)	0.164309	80.620
RBAC (medium)	11000 rules (10000 users, 1000 roles)	2.258262	765.152
RBAC (large)	110000 rules (100000 users, 10000 roles)	23.916776	7,606
RBAC with resource roles	6 rules (2 users, 2 roles)	0.021146	7.906
RBAC with domains/tenants	6 rules (2 users, 1 role, 2 domains)	0.032696	10.755
ABAC	0 rule (0 user)	0.007510	2.328
RESTful	5 rules (3 users)	0.045398	91.774
Deny-override	6 rules (2 users, 1 role)	0.023281	8.370
Priority	9 rules (2 users, 2 roles)	0.016389	5.313

C++

The overhead of policy enforcement of Casbin CPP is benchmarked in tests/benchmarks directory with the help of Google's benchmarking tool. The testbed for these benchmarks is:

Intel(R) Core(TM) i5-6300HQ CPU @ 2.30GHz, 4 cores, 4 threads

Here is the benchmarking result of executing casbin_benchmark target built in Release configuration (op = an enforce() call, ms = millisecond):

Test case	Rule size	Time overhead (ms/op)
ACL	2 rules (2 users)	0.0195
RBAC	5 rules (2 users, 1 role)	0.0288
RBAC (small)	1100 rules (1000 users, 100 roles)	0.300
RBAC (medium)	11000 rules (10000 users, 1000 roles)	2.113
RBAC (large)	110000 rules (100000 users, 10000 roles)	21.450
RBAC with resource roles	6 rules (2 users, 2 roles)	0.03
RBAC with domains/tenants	6 rules (2 users, 1 role, 2 domains)	0.041
ABAC	0 rule (0 user)	NA
RESTful	5 rules (3 users)	NA
Deny-override	6 rules (2 users, 1 role)	0.0246
Priority	9 rules (2 users, 2 roles)	0.035

Lua (JIT)

Lua Casbin's overhead of policy enforcement is benchmarked in bench.lua. The testbed was a Ubuntu VM with the CPU:

AMD Ryzen(TM) 5 4600H CPU @ 3.0GHz, 6 Cores, 12 Threads

The benchmarking result of luajit bench.lua is as follows (op = an enforce() call, ms = millisecond):

Test case	Rule size	Time overhead (ms/op)
ACL	2 rules (2 users)	0.0533
RBAC	5 rules (2 users, 1 role)	0.0972
RBAC (small)	1100 rules (1000 users, 100 roles)	0.8598
RBAC (medium)	11000 rules (10000 users, 1000 roles)	8.6848
RBAC (large)	110000 rules (100000 users, 10000 roles)	90.3217
RBAC with resource roles	6 rules (2 users, 2 roles)	0.1124
RBAC with domains/tenants	6 rules (2 users, 1 role, 2 domains)	0.1978
ABAC	0 rule (0 user)	0.0305
RESTful	5 rules (3 users)	0.1085
Deny-override	6 rules (2 users, 1 role)	0.1934
Priority	9 rules (2 users, 2 roles)	0.1437