Commit Graph

364 Commits

Author SHA1 Message Date
Jon Chambers 61f515670c Add plumbing for deleting accounts and all associated data. 2020-09-25 11:39:17 -04:00
Jon Chambers 789af0f8a6 Add support for deleting keys associated with an account. 2020-09-25 11:39:17 -04:00
Jon Chambers 86fae58c96 Add support for deleting account entities from the database. 2020-09-25 11:39:17 -04:00
Jon Chambers c54d3abe47 Check for the second-gen GV2 capability when linking devices. 2020-09-24 19:04:02 -04:00
Jon Chambers 6fe511eb50 Fix a bad size check when loading stored messages. 2020-09-23 18:02:33 -04:00
Jon Chambers 17d18b22c7 Drop pub/sub sending logic from WebsocketSender. 2020-09-23 14:51:02 -04:00
Jon Chambers 66a04ed730 Don't explicitly notify clients when messages get persisted. 2020-09-23 14:51:02 -04:00
Jon Chambers 7e14a0bc30 Drop pub/sub operations from WebsocketConnection. 2020-09-23 14:51:02 -04:00
Jon Chambers 77de0f86dc Require desktop clients to send the new gv2-2 capability flag. 2020-09-23 12:05:58 -04:00
Jon Chambers 3b4bc9163a Untangle thread pool names, tweak sizes, and add instrumentation. 2020-09-22 10:21:33 -04:00
Jon Chambers e146135bd1 Don't attempt to send more messages if sending failed for any reason. 2020-09-22 10:21:33 -04:00
Jon Chambers e9e18afb4a Add a (failing) integration test demonstrating an infinite loop. 2020-09-22 10:21:33 -04:00
Jon Chambers 62c31eb202 Revert "Revert keyspace delivery for all messages"
This reverts commit 4dc49604b6.
2020-09-22 10:21:33 -04:00
Jon Chambers 1eacee85ae Count how many iOS users set the old GV2 capability flag. 2020-09-21 18:58:07 -04:00
Jon Chambers 5986145282 Add a second-generation GV2 capability and ignore the old capability for iOS devices. 2020-09-21 18:57:53 -04:00
Jon Chambers b134a69a28 Record the number of authentications for users with/without GV2 support. 2020-09-21 15:42:13 -04:00
Jon Chambers 83f9eacac4 Refactor UserAgentTagUtil to parse UA strings with UserAgentUtil. 2020-09-21 12:24:08 -04:00
Jon Chambers baab6b951b Add a general utility class for parsing user-agent strings. 2020-09-21 12:24:08 -04:00
Jon Chambers 903a1bec91 Reject (eventually) oversize messages. 2020-09-17 17:07:20 -04:00
Jon Chambers ebc3a251b7 Drop the UUID addressing capability flag entirely. 2020-09-14 15:36:29 -04:00
Jon Chambers a567f4a6de Don't check UUID capability when blocking capability downgrades. 2020-09-14 15:36:29 -04:00
Jon Chambers 4dc49604b6
Revert keyspace delivery for all messages
* Revert "Send all messages via keyspace notifications when a feature flag is enabled."

This reverts commit fadcf62166.

* Revert "Consolidate semaphore release logic."

This reverts commit c02b255766.

* Revert "Represent stored message state as an enumeration rather than a collection of booleans."

This reverts commit 89788fa665.

* Revert "Refactor: collapse state into semaphores/atomic booleans."

This reverts commit a052e2ee8f.

* Revert "Refactor: move sendNextMessagePage into its own method."

This reverts commit 158e5004b7.

* Revert "Avoid querying the database if we think all new messages are in the cache."

This reverts commit 6f9ff3be37.

* Revert "Query for more stored messages if an update happens while we're already processing a batch."

This reverts commit f766c57743.

* Revert "Only send the "queue cleared" message once per websocket session."

This reverts commit 8f53152c3e.

* Revert "Let processStoredMessages handle requery logic."

This reverts commit 7bbc88d716.

* Revert "Only allow one thread to process stored messages at a time."

This reverts commit 68256d2343.
2020-09-14 15:35:10 -04:00
Jon Chambers fadcf62166 Send all messages via keyspace notifications when a feature flag is enabled. 2020-09-11 13:12:17 -04:00
Jon Chambers c02b255766 Consolidate semaphore release logic. 2020-09-11 13:12:17 -04:00
Jon Chambers 89788fa665 Represent stored message state as an enumeration rather than a collection of booleans. 2020-09-11 13:12:17 -04:00
Jon Chambers a052e2ee8f Refactor: collapse state into semaphores/atomic booleans. 2020-09-11 13:12:17 -04:00
Jon Chambers 158e5004b7 Refactor: move sendNextMessagePage into its own method. 2020-09-11 13:12:17 -04:00
Jon Chambers 6f9ff3be37 Avoid querying the database if we think all new messages are in the cache. 2020-09-11 13:12:17 -04:00
Jon Chambers f766c57743 Query for more stored messages if an update happens while we're already processing a batch. 2020-09-11 13:12:17 -04:00
Jon Chambers 8f53152c3e Only send the "queue cleared" message once per websocket session. 2020-09-11 13:12:17 -04:00
Jon Chambers 7bbc88d716 Let processStoredMessages handle requery logic. 2020-09-11 13:12:17 -04:00
Jon Chambers 68256d2343 Only allow one thread to process stored messages at a time. 2020-09-11 13:12:17 -04:00
Ehren Kret f88c440c48
Automatically retry when Twilio returns unreachable (#190)
* Parse and log the Twilio error code

* Automatically retry without sender ID when Twilio returns unreachable

* Remove attempt count and pass around whether or not sender id was used
2020-09-10 13:58:39 -05:00
Jon Chambers cfa56ba6d4 Remove the "send online messages via keyspace notifications" feature flag. 2020-09-10 10:41:20 -04:00
Jon Chambers 2c6b646d87 Enforce no capability downgrade on device verification 2020-09-09 16:05:00 -04:00
Jon Chambers e7572094b5 Require all enabled devices to support GV2. 2020-09-09 16:05:00 -04:00
Jon Chambers 5e34823a49 Optionally send online-only messages via keyspace notifications. 2020-09-09 14:42:09 -04:00
Jon Chambers fdef21a871 Record and listen for ephemeral messages in a separate queue. 2020-09-09 14:42:09 -04:00
Jon Chambers d40cff8a99 Revert "Add a system for storing, retrieving, and notifying listeners about ephemeral (online) messages."
This reverts commit 06754d6158.
2020-09-08 15:55:09 -04:00
Jon Chambers 8927e45ded Revert "Optionally send online-only messages via keyspace notifications."
This reverts commit 12fe28d8ab.
2020-09-08 15:55:09 -04:00
Jon Chambers 1a93df92d4 Replace DeliveryStatus with a simple boolean. 2020-09-08 11:29:33 -04:00
Jon Chambers 12fe28d8ab Optionally send online-only messages via keyspace notifications. 2020-09-08 11:19:55 -04:00
Jon Chambers 06754d6158 Add a system for storing, retrieving, and notifying listeners about ephemeral (online) messages. 2020-09-08 11:14:42 -04:00
Jon Chambers 1d5087374e Jettison UUID-or-E164 plumbing in favor of UUID-only. 2020-09-08 09:30:47 -04:00
Jon Chambers 8356264fe0 Rename RedisClusterMessagesCache and related classes to just MessagesCache. 2020-09-08 09:30:47 -04:00
Jon Chambers 18ecd748dd Entirely discard the old message cache machinery. 2020-09-08 09:30:47 -04:00
Jon Chambers e324f27655 Stop sending/processing CONNECTED pub/sub messages. 2020-09-03 13:52:43 -04:00
Jon Chambers afd645fb11 Retrieve messages using commands available in Redis 3. 2020-09-03 13:31:55 -04:00
Jon Chambers 5b42593fbb Persist messages one page at a time. 2020-09-03 12:08:46 -04:00
Jon Chambers 5c04f2634a Use a dedicated executor service for dispatching keyspace notifications. 2020-09-03 11:04:48 -04:00
Jon Chambers ad01610d1e Rely on the client presence manager to decide whether to send push notifications. 2020-09-03 11:04:48 -04:00
Jon Chambers 697c380cd1 Close websocket connections when displaced. 2020-09-03 11:04:48 -04:00
Jon Chambers 81e8143a43 Rely solely on the clustered message cache. 2020-09-02 11:57:33 -04:00
Jon Chambers 8409986ef5 Mirror persistence operations from the new persister to the old persister. 2020-09-02 11:02:40 -04:00
Jon Chambers 2b50367d7f Put message persisters behind feature flags. 2020-09-02 11:02:40 -04:00
Jon Chambers 1dcc491fec Move cache-mirroring operations to the calling thread. 2020-09-01 12:34:37 -04:00
Ehren Kret d715f86713 Refactor to constants 2020-09-01 10:55:26 -04:00
Ehren Kret 5221828705 Increase maximum sticker size to 300 kibibytes
In preparation for animated stickers, allow stickers to be up to 300
kibibytes.
2020-09-01 10:55:26 -04:00
Jon Chambers 6aa4acd3db Mirror "clear queue" operations to the clustered cache. 2020-09-01 10:55:07 -04:00
Jon Chambers 15936c29c1 Let Dropwizard manage the lifecycle of the feature flag manager. 2020-09-01 10:50:59 -04:00
Jon Chambers 8b70c69a0d Replace metrics with logging statements. 2020-08-31 15:57:17 -04:00
Jon Chambers dfe80a30dc Make ScourMessageCacheCommand a ConfiguredCommand instead of an EnvironmentCommand. 2020-08-31 15:57:17 -04:00
Jon Chambers ce026e7ad0 Don't send contacts to CDS if they've opted out of discoverability. (SERVER-130) 2020-08-27 15:58:02 -04:00
Jon Chambers 58e3122dab Add a discoverableByPhoneNumber account attribute. (SERVER-129) 2020-08-27 15:58:02 -04:00
Jon Chambers 3b55b2d1b2 Actually make the "scour message cache" available to Dropwizard. Oops. 2020-08-27 15:15:04 -04:00
Jon Chambers 2326e61de5 Clear and re-create gauges to avoid "stuck" feature flag reporting. 2020-08-27 13:18:12 -04:00
Jon Chambers 32b18c9509 Add an endpoint for getting the current state of feature flags. 2020-08-27 13:18:12 -04:00
Jon Chambers acf52ad8a3 Make feature flag manager tests use a real database to avoid over-mocking. 2020-08-27 13:18:12 -04:00
Jon Chambers 08dd493f98 Don't report exceptions as part of traffic metrics. 2020-08-27 13:17:57 -04:00
Jon Chambers 07bbe7dfb2 Return to an async model for push notification latency. 2020-08-27 10:51:44 -04:00
Jon Chambers 0aa1b80e3e Add a command for persisting any detached messages in the old message cache. 2020-08-27 10:51:12 -04:00
Jon Chambers 5ac390281e Add an abstract base class for Redis singleton tests. 2020-08-27 10:51:12 -04:00
Jon Chambers ac465c5a18 Add a Lettuce-based Redis singleton client. 2020-08-27 10:51:12 -04:00
Jon Chambers 1ef3546822
Add support for server-side feature flags 2020-08-26 20:27:33 -04:00
Jon Chambers e74ad2b555 Make RedisClusterMessagesCache a Managed class. 2020-08-25 10:58:01 -04:00
Jon Chambers 71c0056c66 Use lots of specific subscriptions instead of one monster subscription to minimize load. 2020-08-25 10:58:01 -04:00
Jon Chambers 56b27ea785 Record experiment outcomes with timers instead of counters. 2020-08-25 10:57:44 -04:00
Jon Chambers 2d75f59d33 Add support for UUID-only delivery certificates. (SERVER-132) 2020-08-20 17:05:53 -04:00
Jon Chambers a709a3bcc0 Remove a candidate metric provider. 2020-08-20 15:40:56 -04:00
Jon Chambers 34bf5112e0 Drop TimeProvider. 2020-08-20 15:40:24 -04:00
Jon Chambers bfe18d1d28 Re-nerf the clustered message persister. 2020-08-20 15:38:09 -04:00
Jon Chambers 6a76afc20d Add a test to make sure the persister is respecting persist delays. 2020-08-20 15:38:09 -04:00
Jon Chambers 9c469c2f96 Base persister tests on a real Redis cluster. 2020-08-20 15:38:09 -04:00
Jon Chambers 2ab42f3dd6 Refine and expand clustered message cache metrics. 2020-08-19 11:39:05 -04:00
Jon Chambers af34b43a8d Reactivate the message notification experiment. 2020-08-19 11:39:05 -04:00
Jon Chambers 0f71cc7864 Rename metrics associated with cluster circuit breakers for clarity. 2020-08-18 17:59:00 -04:00
Jon Chambers df90de3a5f Change default Lettuce command timeout to 10s. 2020-08-18 16:21:42 -04:00
Jon Chambers 42ea7a9814 Revert Lettuce connection pooling. 2020-08-18 16:21:42 -04:00
Jon Chambers c683cbdb2d Time Redis operations. 2020-08-18 12:20:12 -04:00
Jon Chambers d243b73678 Make Lettuce connection pools configurable. Double the default size. 2020-08-18 12:20:12 -04:00
Jon Chambers dc28d063aa Reactivate the explicit client presence experiment. 2020-08-17 11:34:27 -04:00
Jon Chambers bb6045c1d0 Disarm the client presence manager experiment. 2020-08-15 20:23:05 -04:00
Jon Chambers f1a74b5939 Disarm new message keyspace notifications. 2020-08-15 20:23:05 -04:00
Jon Chambers 6fb9038af1 Move to a synchronous, pooled connection model for Redis clusters. 2020-08-14 17:15:56 -04:00
Jon Chambers 27f721a1f5 Update to resilience4j 1.5.0. 2020-08-14 17:15:56 -04:00
Jon Chambers 5717dc294e Combine the read/write breakers for Redis clusters. 2020-08-14 17:15:56 -04:00
Jon Chambers ae0f8df11b Break out FaultTolerantPubSubConnection as its own thing so different use cases can have their own subscription space. 2020-08-14 17:15:56 -04:00
Jon Chambers 77460ba502 Remove keyspace notification configuration checks because AWS doesn't support `CONFIG GET`. 2020-08-13 15:32:25 -04:00
Jon Chambers f8235da4d8 Fix an issue where the queue for a thread pool was not bounded. 2020-08-13 12:46:11 -04:00
Jon Chambers 8d3316ccd6 Listen for new messages via keyspace notifications. 2020-08-13 12:17:04 -04:00