Commit Graph

511 Commits

Author SHA1 Message Date
Jon Chambers 67ed035b36 Retry serializable key transactions. 2021-01-13 17:38:29 -05:00
Jon Chambers ad30786f4a Parallelize message persisters. 2021-01-12 18:50:14 -05:00
Jon Chambers 2e01da5ec1 Add a task to enable/disable accelerated crawling. 2021-01-11 19:29:18 -05:00
Jon Chambers 8fb37a0024 Log when a crawling cycle has wrapped up. 2021-01-11 19:29:18 -05:00
Jon Chambers 9412a7424c Return HTTP/429 whenever somebody tries to get contacts from the old directory system. 2021-01-11 19:29:10 -05:00
Jon Chambers f8cbb4f386 Temporarily suspend client version metrics to reduce load on our metric aggregator. 2021-01-11 14:04:44 -05:00
Ehren Kret 86ccaa52a5
Allow configuration of multiple directory account crawler listeners (#325)
* Allow configuration of multiple directory account crawler listeners

Only one should update the local redis directory. This one is marked
with replicationPrimary true. The others in the list only serve to
issue replication requests over to CDS replication load balancers.

* Update one more metric name
2021-01-10 17:11:02 -06:00
Jon Chambers cc3e5d23e4 Enable Lettuce adaptive topology refreshes. 2021-01-10 16:20:35 -05:00
Jon Chambers cac86d1f77 Standardize toplogy event handling strategy. 2021-01-10 15:14:12 -05:00
Jon Chambers 22f7bb822f Raise log level of toplogy changes. 2021-01-10 15:14:12 -05:00
Jon Chambers 1b53f10091 Reload scripts across the whole cluster if one shard is missing the script. 2021-01-10 15:00:12 -05:00
Jon Chambers bac268a21c Don't send a reply to clients until messages are safely in a non-volatile store. 2021-01-10 13:03:40 -05:00
Jon Chambers 321e6e6679 Don't validate cluster membership (allow new shards to join dynamically). 2021-01-10 12:58:35 -05:00
Moxie Marlinspike 22ef058cb6 Temporarily disable registration abuse system 2021-01-09 15:57:55 -05:00
Jon Chambers 9ee6419bc0 Publish directory updates to multiple SQS queues. 2021-01-08 18:07:18 -05:00
Jon Chambers 3bf0188e7f Turn off alphanumeric sender ID for all countries. 2021-01-08 06:18:53 -05:00
Jon Chambers 91fc0fd623 Revert "Delete data in the storage service when deleting accounts."
This reverts commit ff1a721d5b.
2021-01-08 06:18:39 -05:00
Jon Chambers d2fcf68381 Record the status message when clients reject websocket messages. 2020-12-23 12:29:15 -05:00
Jon Chambers a4d0c17efd Record OS versions for iOS requests. 2020-12-23 11:36:31 -05:00
Jon Chambers ff1a721d5b Delete data in the storage service when deleting accounts. 2020-12-23 11:35:38 -05:00
Jon Chambers c870a1bbd5 Introduce a storage service client. 2020-12-23 11:35:38 -05:00
Ehren Kret ebf332a8c9
Record delivery duration excluding noise from non-primary devices (#311)
* Record delivery duration excluding noise from non-primary devices

* Extract method
2020-12-21 10:28:39 -06:00
Jon Chambers 85d1fff18f Actually increment the Android request counter. 2020-12-11 11:46:07 -05:00
Jon Chambers 6bb106c2cb Drop the Redis command timeout back down to 3 seconds to facilitate debug data collection. 2020-12-11 11:20:10 -05:00
Jon Chambers e551fd2c1b Revert "Pause checks for GV1 migration when checking for capability downgrades."
This reverts commit e7745db36e.
2020-12-10 17:02:41 -05:00
Jon Chambers 34a11c2338 Record OS versions for desktop and SDK versions for Android. 2020-12-10 17:02:05 -05:00
Jon Chambers 0de3a400eb Record unsuccessful server-to-client requests in more detail. 2020-12-10 17:01:46 -05:00
Jon Chambers e524ff965d Add a utility method for getting client platform tags from UA strings for metrics. 2020-12-10 17:01:46 -05:00
Jon Chambers 7ba689aaeb Measure adoption of the `gv1-migration` capability. 2020-12-09 19:08:52 -05:00
Jon Chambers 92fde83b3a Discard oversized messages bound for desktop clients via websockets. 2020-12-07 15:03:35 -05:00
Jon Chambers 3a268aef50 Reduce logging level for Lettuce connection events. 2020-12-07 11:56:41 -05:00
Jon Chambers f673bd8d7b Set device capabilities when linking a new device. 2020-12-02 13:21:08 -05:00
Ehren Kret 299b680013
Always include UUID in UD certificate (#300) 2020-12-01 08:56:55 -06:00
Jon Chambers 81e8352391 Time (and count) SQS "send message" operations. 2020-11-25 15:05:05 -05:00
Jon Chambers 1a627d6a87 Extend Redis command timeout to 3.5 seconds to avoid TCP retransmission "coincidences." 2020-11-25 15:04:06 -05:00
Ehren Kret 00a3e562dc
Force use of UCS-2 instead of GSM-7 for SMS to China (#297) 2020-11-20 14:41:48 -06:00
Jon Chambers 0628c9161c Use named threads for the JsonMetricsReporter executor service. 2020-11-18 15:46:14 -05:00
Jon Chambers 9b28672e19 Honor disabled metric attributes in JsonMetricsReporter. 2020-11-18 15:46:14 -05:00
Jon Chambers d764058a04 Measure contact intersection rate directly. 2020-11-18 14:28:53 -05:00
Jon Chambers 0aafe38496 Stop recording Lettuce latency metrics. 2020-11-17 13:20:37 -05:00
Jon Chambers e7745db36e Pause checks for GV1 migration when checking for capability downgrades. 2020-11-17 09:25:12 -05:00
Jon Chambers 474b879b16 Only notify CDS if an account attribute change actually changes an account's discoverability. 2020-11-16 10:54:12 -05:00
Jon Chambers 0a23b57ff8 Report Dropwizard metrics via the Wavefront proxy. 2020-11-13 17:14:13 -05:00
Jon Chambers 251e1b51c5 Make Micrometer batch size configurable. 2020-11-13 17:13:39 -05:00
Jon Chambers 2cc6c959a5 Revert "Temporarily suspend reporting of Lettuce latency metrics."
This reverts commit 2045153495a823b06334e7cbd86fb89c946c1cea.
2020-11-11 13:05:49 -05:00
Jon Chambers fb9aa672c9 Include the name of the calling thread when a command times out. 2020-11-11 13:05:35 -05:00
Jon Chambers 325e65db7f Expand UA parsing tests to cover OS details in desktop strings. 2020-11-11 13:05:18 -05:00
Jon Chambers 103b49ec45 Record the number of non-success responses from clients when sending messages via websockets. 2020-11-10 11:47:57 -05:00
Jon Chambers 6c78d7544f Capture a thread dump when Redis commands time out. 2020-11-10 11:47:39 -05:00
Jon Chambers 7cf50a15d0 Include client age/UA string when closing due to a spurious keepalive request. 2020-11-10 11:45:12 -05:00
Jon Chambers adbc4e9fec Record the platforms of clients that send a keepalive without a local presence. 2020-11-10 11:45:12 -05:00
Jon Chambers 4815434dd7 Record the platforms of clients that are getting displaced. 2020-11-10 11:45:12 -05:00
Jon Chambers b25e50bdae Drop API keys from Micrometer configuration. 2020-11-09 09:26:56 -05:00
Ehren Kret 604287244f Update copyright statement on all source files
IntelliJ Copyright Profile used to automate this.
2020-11-04 11:55:35 -05:00
Jon Chambers 4a4a721e90 Log timeouts in addition to incrementing a counter to make it easier to get precise timestamps. 2020-10-30 11:35:59 -04:00
Jon Chambers a4062b338e Count timeouts directly. 2020-10-29 10:51:18 -04:00
Ehren Kret 5587b7d469 Expose gv1-migration on profile endpoint 2020-10-28 13:00:57 -04:00
Ehren Kret 26870d134f Set source UUID when delivering envelopes from message cache/db on websocket 2020-10-28 12:38:32 -04:00
Jon Chambers 0431a2abb1 De-dupe connection event logging messages. 2020-10-28 12:29:14 -04:00
Ehren Kret c2db2d3cbd
Add GV1 Migration capability 2020-10-27 16:17:21 -04:00
Jon Chambers 05d9ec673e
Send push notifications if websockets close before all messages are delivered 2020-10-27 16:02:55 -04:00
Jon Chambers 1732cf9243 Add filters/tasks to enable/disable request logging. 2020-10-23 11:35:06 -04:00
Jon Chambers ab62c19de9 Temporarily suspend reporting of Lettuce latency metrics. 2020-10-23 11:30:42 -04:00
Jon Chambers 96d3a69479 Use container-managed executors for APN/GCM senders. 2020-10-23 11:30:03 -04:00
Jon Chambers 8523bb1ad8 Change the "oversized message" threshold from 64kB to 1MB. 2020-10-23 11:13:19 -04:00
Jon Chambers e6da54d9b8 Resolve build error introduced while merging. 2020-10-20 19:04:44 -04:00
Jon Chambers 0a843dc086 Tighten the "prune peers" interval; move from fixed-rate to fixed-delay scheduling. 2020-10-20 19:00:55 -04:00
Jon Chambers 7b3ed2dcbf Catch exceptions thrown while pruning missing peers. 2020-10-20 19:00:55 -04:00
Jon Chambers 42ed6c3ded Add clients to the "cleanup" list before actually setting their presence keys. 2020-10-20 19:00:55 -04:00
Jon Chambers 23ca011ac1 Record account deletion reasons. 2020-10-20 19:00:34 -04:00
Jon Chambers d82b3dc429 Record a count of deleted accounts by country. 2020-10-20 19:00:34 -04:00
Jon Chambers e391793c58 Remove now-redundant Redis execution time metrics. 2020-10-20 19:00:11 -04:00
Jon Chambers 236cef4b56 Report Lettuce command latency via Micrometer. 2020-10-20 19:00:11 -04:00
Jon Chambers 45687513bf Revert "Revert "Share resources between Lettuce clients.""
This reverts commit 334f509be599fa6a501026e900d912ff7187e150.
2020-10-20 19:00:11 -04:00
Jon Chambers 019ffdaf12 Add a command for dumping Redis command stats. 2020-10-20 18:59:44 -04:00
Jon Chambers df847431eb Measure total bytes written to websockets and failed send attempts. 2020-10-20 17:22:30 -04:00
Jon Chambers 99f488d48f Drop websocket connection names (unused for a while now). 2020-10-19 11:24:35 -04:00
Jon Chambers 05929871c9 Rename PushSender to MessageSender and add docs. 2020-10-19 11:24:35 -04:00
Jon Chambers 74b3daa70a Collapse WebsocketSender into PushSender. 2020-10-19 11:24:35 -04:00
Jon Chambers 5e30b0499a Move provisioning message-sending to its own manager class. 2020-10-19 11:24:35 -04:00
Jon Chambers 85c7347899 Add a command for dumping Redis SLOWLOG output. 2020-10-15 12:18:37 -04:00
Jon Chambers 3a84775912 Log cluster topology change events, too. 2020-10-13 16:07:08 -04:00
Jon Chambers 290a82e61c Log when Lettuce connection events happen. 2020-10-13 16:07:08 -04:00
Jon Chambers adac7d7fb2 Estimate the size of message entity lists sent via the REST API. 2020-10-13 15:49:11 -04:00
Jon Chambers 52320ebb91 Revert "Share resources between Lettuce clients."
This reverts commit eab1f503a5.
2020-10-13 12:44:54 -04:00
Jon Chambers eab1f503a5 Share resources between Lettuce clients. 2020-10-11 14:36:28 -04:00
Jon Chambers a9d0aa136d Add OS-reported metrics for cached/buffered memory. 2020-10-11 13:43:15 -04:00
Jon Chambers 691ab3080d Fix some metrics names/types. 2020-10-11 12:37:17 -04:00
Jon Chambers c5147e0c68 Report direct memory metrics. 2020-10-11 11:37:51 -04:00
Jon Chambers e9b0829860 Report the maximum number of file descriptors allowed by the OS. 2020-10-11 11:27:57 -04:00
Jon Chambers 95428ab8b0 Report GC metrics. 2020-10-11 11:08:24 -04:00
Jon Chambers 775d56fe52 Drop the "repair message queue metadata" script. 2020-10-09 18:18:30 -04:00
Jon Chambers ac2ff29288 Make sure to close scheduled reporters. 2020-10-09 18:05:00 -04:00
Jon Chambers 8e1975efe4 Record the number of deletable accounts per crawled chunk. 2020-10-08 10:51:41 -04:00
Curt Brune 39c09733d3
Add /v1/payments/auth endpoint 2020-10-08 10:51:01 -04:00
Jon Chambers e1c397993d
Require Android clients to support the gv2-3 capability 2020-10-06 16:49:49 -04:00
Jon Chambers 58ca4baf71 Time account deletion operations. 2020-10-06 11:04:47 -04:00
Jon Chambers 5245b68689 Remove temporary metrics. 2020-10-06 11:04:47 -04:00
Jon Chambers 2b6811cb1b Really delete old accounts instead of just removing their push channels. 2020-10-06 11:04:47 -04:00
Jon Chambers c82496b972 Remove the "repair queue metadata" script. 2020-10-05 16:57:16 -04:00