Greg Heartsfield
53c2a8051c
improvement: reduce logging
2022-12-22 10:29:27 -06:00
Greg Heartsfield
168cf513ac
feat: perform full checkpoints and truncate WAL every 2k events
2022-12-22 10:11:05 -06:00
Greg Heartsfield
ea204761c9
fix: do not show slow queries more than once per sub
2022-12-20 15:41:50 -06:00
Greg Heartsfield
c270ae1434
improvement: reduce event count for db writer pauses
2022-12-20 15:25:24 -06:00
Greg Heartsfield
64bd983cb6
perf: every 5000 persisted events, pause for 500ms for backups
...
I have observed backups running for a very long time under heavy load,
this introduces some artificial delay to give the online backup enough
time to make progress.
2022-12-20 15:05:04 -06:00
Greg Heartsfield
1c153bc784
perf: shed DB query load when queue gets large
2022-12-20 13:23:21 -06:00
Greg Heartsfield
dc11d9a619
improvement: explicitly rollback transaction on duplicate event
2022-12-20 13:23:04 -06:00
Greg Heartsfield
cd1557787b
improvement: log write pool
2022-12-20 13:21:57 -06:00
Greg Heartsfield
86bb7aeb9a
improvement: function to check pool capacity
2022-12-20 10:07:01 -06:00
Greg Heartsfield
ce37fc1a2d
build: bump version to 0.7.12
2022-12-19 14:50:42 -06:00
Greg Heartsfield
2cfd384339
perf: drop db handles that are not quickly read
2022-12-19 00:18:39 -06:00
Greg Heartsfield
8c013107f9
perf: increase upper bound for sqlite mmap
2022-12-18 23:19:43 -06:00
Greg Heartsfield
64a4466d30
perf: backing down on max_blocking_threads
2022-12-18 23:14:41 -06:00
Greg Heartsfield
1596c23eb4
perf: increase blocking threads now that contention is reduced
2022-12-18 22:46:32 -06:00
Greg Heartsfield
129badd4e1
perf: reduce per thread mmap allocation for DB
2022-12-18 22:45:32 -06:00
Greg Heartsfield
6f7c080180
improvement: reduce number of writer blocking threads from 4->2
2022-12-18 22:32:31 -06:00
Greg Heartsfield
af92561ef6
perf: remove shared cache mode (experiment)
2022-12-18 22:15:50 -06:00
Greg Heartsfield
d833a3e40d
perf: reduce logging
2022-12-18 22:11:46 -06:00
Greg Heartsfield
462eb46642
build: bump version to 0.7.11
2022-12-18 20:52:01 -06:00
Greg Heartsfield
cf144d503d
perf: reduce logging for slow queries
2022-12-18 20:47:11 -06:00
Greg Heartsfield
fb8375aef2
build: bump version to 0.7.10
2022-12-18 13:46:18 -06:00
Greg Heartsfield
88ac31b549
perf: increase channel size for DB communication
2022-12-18 13:44:28 -06:00
Greg Heartsfield
677b7d39e9
improvement: log slow requests that return zero results
2022-12-18 13:42:31 -06:00
Greg Heartsfield
b24d2f9aaa
perf: set default blocking threads to lower value
2022-12-18 12:20:57 -06:00
Greg Heartsfield
7a3899d852
build: bump version to 0.7.9
2022-12-18 09:21:07 -06:00
Greg Heartsfield
818108b793
improvement: upgrade multiple dependencies
...
Updating anyhow v1.0.66 -> v1.0.67
Updating async-trait v0.1.59 -> v0.1.60
Updating cxx v1.0.83 -> v1.0.84
Updating cxx-build v1.0.83 -> v1.0.84
Updating cxxbridge-flags v1.0.83 -> v1.0.84
Updating cxxbridge-macro v1.0.83 -> v1.0.84
Updating itoa v1.0.4 -> v1.0.5
Updating link-cplusplus v1.0.7 -> v1.0.8
Updating proc-macro2 v1.0.47 -> v1.0.48
Updating quote v1.0.21 -> v1.0.22
Updating rustversion v1.0.9 -> v1.0.11
Updating ryu v1.0.11 -> v1.0.12
Updating scratch v1.0.2 -> v1.0.3
Updating serde v1.0.150 -> v1.0.151
Updating serde_derive v1.0.150 -> v1.0.151
Updating serde_json v1.0.89 -> v1.0.90
Updating syn v1.0.105 -> v1.0.106
Updating thiserror v1.0.37 -> v1.0.38
Updating thiserror-impl v1.0.37 -> v1.0.38
Updating unicode-ident v1.0.5 -> v1.0.6
2022-12-18 09:16:09 -06:00
Greg Heartsfield
d10348f7e1
feat: configurable blocking threads
2022-12-18 09:14:04 -06:00
Greg Heartsfield
8598e443d8
wip: add configuration for future feature (client concurrent db limits)
2022-12-17 23:19:48 -06:00
Greg Heartsfield
43222d44e5
feat: perform optimization after seeing many events
2022-12-17 23:18:54 -06:00
Greg Heartsfield
7c1516c4fb
perf: add index for tags
2022-12-17 23:17:53 -06:00
Greg Heartsfield
0c72053a49
perf: increase mmap size to 1GB
2022-12-17 23:17:16 -06:00
Greg Heartsfield
3f32ff67ab
improvement: minor logging
2022-12-17 23:11:14 -06:00
Greg Heartsfield
0b9778d6ca
refactor: simplify tracking of subscriptions
2022-12-17 20:46:58 -06:00
Greg Heartsfield
9be04120c7
build: bump version to 0.7.8
2022-12-17 12:01:43 -06:00
Greg Heartsfield
cc06167e06
perf: add composite index for tag table
2022-12-17 12:01:20 -06:00
Greg Heartsfield
b6e33f044f
improvement: limit db connection max lifetime
2022-12-17 10:47:35 -06:00
Greg Heartsfield
1b2c6f9fca
build: bump version to 0.7.7
2022-12-17 10:09:44 -06:00
Greg Heartsfield
0d8d39ad22
feat: add rate limiting setting for subscription creation
2022-12-17 09:27:29 -06:00
Greg Heartsfield
0e851d4f71
build: bump version to 0.7.6
2022-12-17 07:51:57 -06:00
Greg Heartsfield
3c880b2f49
perf: pull distinct to outermost SQL
2022-12-17 07:49:28 -06:00
Greg Heartsfield
7a4c9266ec
improvement: make hexsearch structs sortable
2022-12-17 07:49:05 -06:00
Greg Heartsfield
e8557d421b
build: bump version to 0.7.5
2022-12-16 17:21:00 -06:00
Greg Heartsfield
7ca9c864f2
improvement: DB pool logging shows used connections directly
2022-12-16 17:01:49 -06:00
Greg Heartsfield
838aafd079
improvement: consistent log messages for client/sub ids
2022-12-16 15:22:27 -06:00
Greg Heartsfield
e554b10ac2
improvement: tweak sub/sql logging for slow queries
2022-12-16 14:55:45 -06:00
Greg Heartsfield
b0bfaa48fc
improvement: ignore duplicate REQ messages
2022-12-16 14:37:02 -06:00
Greg Heartsfield
2e9b1b6ba7
docs: comment reason for force_no_match
2022-12-16 14:35:21 -06:00
Greg Heartsfield
4d9012d94c
improvement: upgrade docker builder and base images
2022-12-16 14:33:08 -06:00
Greg Heartsfield
ffe7aac066
improvement: upgrade multiple dependencies
...
Updating aho-corasick v0.7.19 -> v0.7.20
Updating async-trait v0.1.58 -> v0.1.59
Updating axum v0.5.17 -> v0.6.1
Updating axum-core v0.2.9 -> v0.3.0
Updating bytes v1.2.1 -> v1.3.0
Updating cc v1.0.76 -> v1.0.78
Updating crossbeam-utils v0.8.12 -> v0.8.14
Updating cxx v1.0.82 -> v1.0.83
Updating cxx-build v1.0.82 -> v1.0.83
Updating cxxbridge-flags v1.0.82 -> v1.0.83
Updating cxxbridge-macro v1.0.82 -> v1.0.83
Updating flate2 v1.0.24 -> v1.0.25
Updating libc v0.2.137 -> v0.2.138
Updating matchit v0.5.0 -> v0.7.0
Updating miniz_oxide v0.5.4 -> v0.6.2
Updating openssl v0.10.42 -> v0.10.44
Updating openssl-sys v0.9.77 -> v0.9.79
Updating parking_lot_core v0.9.4 -> v0.9.5
Updating pest v2.4.1 -> v2.5.1
Updating pest_derive v2.4.1 -> v2.5.1
Updating pest_generator v2.4.1 -> v2.5.1
Updating pest_meta v2.4.1 -> v2.5.1
Updating prost v0.11.2 -> v0.11.3
Adding rustversion v1.0.9
Updating serde v1.0.147 -> v1.0.150
Updating serde_derive v1.0.147 -> v1.0.150
Updating serde_json v1.0.88 -> v1.0.89
Updating sha-1 v0.10.0 -> v0.10.1
Updating syn v1.0.103 -> v1.0.105
Updating tokio v1.22.0 -> v1.23.0
Updating tokio-macros v1.8.0 -> v1.8.2
Updating toml v0.5.9 -> v0.5.10
Updating tonic v0.8.2 -> v0.8.3
Updating tower-http v0.3.4 -> v0.3.5
Updating typenum v1.15.0 -> v1.16.0
2022-12-16 11:17:05 -06:00
Greg Heartsfield
f9695bd0a9
fix: db schema version updates correctly for v9
2022-12-16 10:01:49 -06:00
