Commit Graph

1402 Commits

Author SHA1 Message Date
Jon Chambers 225932b4c9
Add emojis/"about" text to profiles 2021-01-20 15:42:47 -05:00
Jon Chambers 6b850b9894 Allow (versioned) profile names up to 380 base64 characters long. 2021-01-20 11:08:10 -05:00
Jon Chambers 943a5d1036
Shard push scheduling cache 2021-01-19 15:50:12 -05:00
Jon Chambers 0a23ce870a Allow message persisters to be disabled by a feature flag. 2021-01-17 11:13:12 -05:00
Jon Chambers a57ce1dd17 Add machinery to allow a percentage of message sends to succeed. 2021-01-15 17:05:16 -05:00
Jon Chambers 81c1ba6eef Respond to all "message send" attempts with HTTP/503. 2021-01-15 15:34:14 -05:00
Jon Chambers 9c53d818f4 Use separate clusters for message cache read/write operations. 2021-01-15 13:51:39 -05:00
Jon Chambers e5a2c1ab10 Always return an empty list of prekeys. 2021-01-15 12:27:10 -05:00
Jon Chambers 67ed035b36 Retry serializable key transactions. 2021-01-13 17:38:29 -05:00
Jon Chambers 2e01da5ec1 Add a task to enable/disable accelerated crawling. 2021-01-11 19:29:18 -05:00
Jon Chambers 9412a7424c Return HTTP/429 whenever somebody tries to get contacts from the old directory system. 2021-01-11 19:29:10 -05:00
Jon Chambers f8cbb4f386 Temporarily suspend client version metrics to reduce load on our metric aggregator. 2021-01-11 14:04:44 -05:00
Ehren Kret 86ccaa52a5
Allow configuration of multiple directory account crawler listeners (#325)
* Allow configuration of multiple directory account crawler listeners

Only one should update the local redis directory. This one is marked
with replicationPrimary true. The others in the list only serve to
issue replication requests over to CDS replication load balancers.

* Update one more metric name
2021-01-10 17:11:02 -06:00
Jon Chambers 1b53f10091 Reload scripts across the whole cluster if one shard is missing the script. 2021-01-10 15:00:12 -05:00
Jon Chambers bac268a21c Don't send a reply to clients until messages are safely in a non-volatile store. 2021-01-10 13:03:40 -05:00
Moxie Marlinspike 22ef058cb6 Temporarily disable registration abuse system 2021-01-09 15:57:55 -05:00
Jon Chambers 9ee6419bc0 Publish directory updates to multiple SQS queues. 2021-01-08 18:07:18 -05:00
Jon Chambers 3bf0188e7f Turn off alphanumeric sender ID for all countries. 2021-01-08 06:18:53 -05:00
Jon Chambers 91fc0fd623 Revert "Delete data in the storage service when deleting accounts."
This reverts commit ff1a721d5b.
2021-01-08 06:18:39 -05:00
Jon Chambers a4d0c17efd Record OS versions for iOS requests. 2020-12-23 11:36:31 -05:00
Jon Chambers ff1a721d5b Delete data in the storage service when deleting accounts. 2020-12-23 11:35:38 -05:00
Jon Chambers c870a1bbd5 Introduce a storage service client. 2020-12-23 11:35:38 -05:00
Jon Chambers e551fd2c1b Revert "Pause checks for GV1 migration when checking for capability downgrades."
This reverts commit e7745db36e.
2020-12-10 17:02:41 -05:00
Jon Chambers 34a11c2338 Record OS versions for desktop and SDK versions for Android. 2020-12-10 17:02:05 -05:00
Jon Chambers e524ff965d Add a utility method for getting client platform tags from UA strings for metrics. 2020-12-10 17:01:46 -05:00
Jon Chambers 92fde83b3a Discard oversized messages bound for desktop clients via websockets. 2020-12-07 15:03:35 -05:00
Ehren Kret 299b680013
Always include UUID in UD certificate (#300) 2020-12-01 08:56:55 -06:00
Ehren Kret 00a3e562dc
Force use of UCS-2 instead of GSM-7 for SMS to China (#297) 2020-11-20 14:41:48 -06:00
Jon Chambers e7745db36e Pause checks for GV1 migration when checking for capability downgrades. 2020-11-17 09:25:12 -05:00
Jon Chambers 474b879b16 Only notify CDS if an account attribute change actually changes an account's discoverability. 2020-11-16 10:54:12 -05:00
Jon Chambers 325e65db7f Expand UA parsing tests to cover OS details in desktop strings. 2020-11-11 13:05:18 -05:00
Ehren Kret 604287244f Update copyright statement on all source files
IntelliJ Copyright Profile used to automate this.
2020-11-04 11:55:35 -05:00
Ehren Kret 5587b7d469 Expose gv1-migration on profile endpoint 2020-10-28 13:00:57 -04:00
Ehren Kret 26870d134f Set source UUID when delivering envelopes from message cache/db on websocket 2020-10-28 12:38:32 -04:00
Ehren Kret c2db2d3cbd
Add GV1 Migration capability 2020-10-27 16:17:21 -04:00
Jon Chambers 05d9ec673e
Send push notifications if websockets close before all messages are delivered 2020-10-27 16:02:55 -04:00
Jon Chambers 96d3a69479 Use container-managed executors for APN/GCM senders. 2020-10-23 11:30:03 -04:00
Jon Chambers 23ca011ac1 Record account deletion reasons. 2020-10-20 19:00:34 -04:00
Jon Chambers 99f488d48f Drop websocket connection names (unused for a while now). 2020-10-19 11:24:35 -04:00
Jon Chambers 05929871c9 Rename PushSender to MessageSender and add docs. 2020-10-19 11:24:35 -04:00
Jon Chambers 74b3daa70a Collapse WebsocketSender into PushSender. 2020-10-19 11:24:35 -04:00
Jon Chambers 3a84775912 Log cluster topology change events, too. 2020-10-13 16:07:08 -04:00
Jon Chambers a9d0aa136d Add OS-reported metrics for cached/buffered memory. 2020-10-11 13:43:15 -04:00
Curt Brune 39c09733d3
Add /v1/payments/auth endpoint 2020-10-08 10:51:01 -04:00
Jon Chambers e1c397993d
Require Android clients to support the gv2-3 capability 2020-10-06 16:49:49 -04:00
Jon Chambers 2b6811cb1b Really delete old accounts instead of just removing their push channels. 2020-10-06 11:04:47 -04:00
Jon Chambers c82496b972 Remove the "repair queue metadata" script. 2020-10-05 16:57:16 -04:00
Jon Chambers bb087caddc Don't panic if a queue exists, but is empty when repairing metadata. 2020-10-04 16:09:56 -04:00
Jon Chambers 5e3f8b9c2e Disallow insertion of duplicate messages. 2020-10-04 15:34:14 -04:00
Jon Chambers 1ccfe928f7 Add a test to make sure that we don't double-insert messages with the same GUID. 2020-10-04 15:34:14 -04:00
Jon Chambers 952cfae4e6 Repair queue metadata before persisting queues. 2020-10-04 15:25:06 -04:00
Jon Chambers 02a2c3224f Discard unused feature flag constants/mocking. 2020-10-04 10:48:42 -04:00
Jon Chambers fc05529574 Let MessagePersister manage its own worker thread. 2020-10-03 11:43:34 -04:00
Jon Chambers 76665dd56e Retry Redis commands that time out. 2020-10-01 17:17:07 -04:00
Jon Chambers 885fa6beae Add tests for Device#isEnabled. 2020-10-01 12:54:35 -04:00
Jon Chambers 65cdd5fcbe Drop the 365-day check when deciding if an account is enabled. 2020-10-01 12:54:35 -04:00
Jon Chambers 0c6f05f34a Add a (failing!) test for sending a sealed-sender message after a non-sealed-sender message. 2020-10-01 11:06:43 -04:00
Jon Chambers 57d2ef8740 Return queues to the "to persist" list if something goes wrong during persistence. 2020-09-30 10:39:56 -04:00
Jon Chambers a97e0982e3 Add an integration test for message persistence. 2020-09-30 10:39:56 -04:00
Jon Chambers eaa2060d84 Fix an incorrect locking key and some previously-suppressed lock contention issues. 2020-09-30 10:39:56 -04:00
Jon Chambers c7230ccbb0 Remove messages from the cache in bulk. 2020-09-29 10:58:02 -04:00
Jon Chambers fc71ced660 Persist messages in batches. 2020-09-29 10:58:02 -04:00
Jon Chambers 599cd766e1 Let Dropwizard manage persister thread lifecycles. 2020-09-29 10:58:02 -04:00
Alan Evans e64c8007c0
Detect GV2 capability in non-gcm Android devices 2020-09-28 15:54:10 -04:00
Jon Chambers e6d4620af1 Only allow linking desktop clients if they support the third-generation GV2 capability. 2020-09-25 17:08:32 -04:00
Jon Chambers 656e6db846 Only consider desktop devices GV2-capable if they send the third-gen GV2 capability. 2020-09-25 17:08:32 -04:00
Jon Chambers 30474e3a2b Add a test for message ordering. 2020-09-25 11:41:58 -04:00
Jon Chambers a553eba574 Add an API endpoint for deleting accounts. 2020-09-25 11:39:17 -04:00
Jon Chambers 61f515670c Add plumbing for deleting accounts and all associated data. 2020-09-25 11:39:17 -04:00
Jon Chambers 789af0f8a6 Add support for deleting keys associated with an account. 2020-09-25 11:39:17 -04:00
Jon Chambers 86fae58c96 Add support for deleting account entities from the database. 2020-09-25 11:39:17 -04:00
Jon Chambers 66a04ed730 Don't explicitly notify clients when messages get persisted. 2020-09-23 14:51:02 -04:00
Jon Chambers 7e14a0bc30 Drop pub/sub operations from WebsocketConnection. 2020-09-23 14:51:02 -04:00
Jon Chambers 77de0f86dc Require desktop clients to send the new gv2-2 capability flag. 2020-09-23 12:05:58 -04:00
Jon Chambers e9e18afb4a Add a (failing) integration test demonstrating an infinite loop. 2020-09-22 10:21:33 -04:00
Jon Chambers 62c31eb202 Revert "Revert keyspace delivery for all messages"
This reverts commit 4dc49604b6.
2020-09-22 10:21:33 -04:00
Jon Chambers 5986145282 Add a second-generation GV2 capability and ignore the old capability for iOS devices. 2020-09-21 18:57:53 -04:00
Jon Chambers 83f9eacac4 Refactor UserAgentTagUtil to parse UA strings with UserAgentUtil. 2020-09-21 12:24:08 -04:00
Jon Chambers baab6b951b Add a general utility class for parsing user-agent strings. 2020-09-21 12:24:08 -04:00
Jon Chambers ebc3a251b7 Drop the UUID addressing capability flag entirely. 2020-09-14 15:36:29 -04:00
Jon Chambers 4dc49604b6
Revert keyspace delivery for all messages
* Revert "Send all messages via keyspace notifications when a feature flag is enabled."

This reverts commit fadcf62166.

* Revert "Consolidate semaphore release logic."

This reverts commit c02b255766.

* Revert "Represent stored message state as an enumeration rather than a collection of booleans."

This reverts commit 89788fa665.

* Revert "Refactor: collapse state into semaphores/atomic booleans."

This reverts commit a052e2ee8f.

* Revert "Refactor: move sendNextMessagePage into its own method."

This reverts commit 158e5004b7.

* Revert "Avoid querying the database if we think all new messages are in the cache."

This reverts commit 6f9ff3be37.

* Revert "Query for more stored messages if an update happens while we're already processing a batch."

This reverts commit f766c57743.

* Revert "Only send the "queue cleared" message once per websocket session."

This reverts commit 8f53152c3e.

* Revert "Let processStoredMessages handle requery logic."

This reverts commit 7bbc88d716.

* Revert "Only allow one thread to process stored messages at a time."

This reverts commit 68256d2343.
2020-09-14 15:35:10 -04:00
Jon Chambers fadcf62166 Send all messages via keyspace notifications when a feature flag is enabled. 2020-09-11 13:12:17 -04:00
Jon Chambers a052e2ee8f Refactor: collapse state into semaphores/atomic booleans. 2020-09-11 13:12:17 -04:00
Jon Chambers 6f9ff3be37 Avoid querying the database if we think all new messages are in the cache. 2020-09-11 13:12:17 -04:00
Jon Chambers f766c57743 Query for more stored messages if an update happens while we're already processing a batch. 2020-09-11 13:12:17 -04:00
Jon Chambers 8f53152c3e Only send the "queue cleared" message once per websocket session. 2020-09-11 13:12:17 -04:00
Jon Chambers 7bbc88d716 Let processStoredMessages handle requery logic. 2020-09-11 13:12:17 -04:00
Jon Chambers 68256d2343 Only allow one thread to process stored messages at a time. 2020-09-11 13:12:17 -04:00
Ehren Kret f88c440c48
Automatically retry when Twilio returns unreachable (#190)
* Parse and log the Twilio error code

* Automatically retry without sender ID when Twilio returns unreachable

* Remove attempt count and pass around whether or not sender id was used
2020-09-10 13:58:39 -05:00
Jon Chambers 2c6b646d87 Enforce no capability downgrade on device verification 2020-09-09 16:05:00 -04:00
Jon Chambers e7572094b5 Require all enabled devices to support GV2. 2020-09-09 16:05:00 -04:00
Jon Chambers fdef21a871 Record and listen for ephemeral messages in a separate queue. 2020-09-09 14:42:09 -04:00
Jon Chambers d40cff8a99 Revert "Add a system for storing, retrieving, and notifying listeners about ephemeral (online) messages."
This reverts commit 06754d6158.
2020-09-08 15:55:09 -04:00
Jon Chambers 06754d6158 Add a system for storing, retrieving, and notifying listeners about ephemeral (online) messages. 2020-09-08 11:14:42 -04:00
Jon Chambers 1d5087374e Jettison UUID-or-E164 plumbing in favor of UUID-only. 2020-09-08 09:30:47 -04:00
Jon Chambers 8356264fe0 Rename RedisClusterMessagesCache and related classes to just MessagesCache. 2020-09-08 09:30:47 -04:00
Jon Chambers 18ecd748dd Entirely discard the old message cache machinery. 2020-09-08 09:30:47 -04:00
Jon Chambers 5c04f2634a Use a dedicated executor service for dispatching keyspace notifications. 2020-09-03 11:04:48 -04:00
Jon Chambers 81e8143a43 Rely solely on the clustered message cache. 2020-09-02 11:57:33 -04:00
Jon Chambers 8409986ef5 Mirror persistence operations from the new persister to the old persister. 2020-09-02 11:02:40 -04:00
Jon Chambers 2b50367d7f Put message persisters behind feature flags. 2020-09-02 11:02:40 -04:00
Ehren Kret 5221828705 Increase maximum sticker size to 300 kibibytes
In preparation for animated stickers, allow stickers to be up to 300
kibibytes.
2020-09-01 10:55:26 -04:00
Jon Chambers ce026e7ad0 Don't send contacts to CDS if they've opted out of discoverability. (SERVER-130) 2020-08-27 15:58:02 -04:00
Jon Chambers 58e3122dab Add a discoverableByPhoneNumber account attribute. (SERVER-129) 2020-08-27 15:58:02 -04:00
Jon Chambers 32b18c9509 Add an endpoint for getting the current state of feature flags. 2020-08-27 13:18:12 -04:00
Jon Chambers acf52ad8a3 Make feature flag manager tests use a real database to avoid over-mocking. 2020-08-27 13:18:12 -04:00
Jon Chambers 08dd493f98 Don't report exceptions as part of traffic metrics. 2020-08-27 13:17:57 -04:00
Jon Chambers 07bbe7dfb2 Return to an async model for push notification latency. 2020-08-27 10:51:44 -04:00
Jon Chambers 0aa1b80e3e Add a command for persisting any detached messages in the old message cache. 2020-08-27 10:51:12 -04:00
Jon Chambers 5ac390281e Add an abstract base class for Redis singleton tests. 2020-08-27 10:51:12 -04:00
Jon Chambers ac465c5a18 Add a Lettuce-based Redis singleton client. 2020-08-27 10:51:12 -04:00
Jon Chambers 1ef3546822
Add support for server-side feature flags 2020-08-26 20:27:33 -04:00
Jon Chambers e74ad2b555 Make RedisClusterMessagesCache a Managed class. 2020-08-25 10:58:01 -04:00
Jon Chambers 56b27ea785 Record experiment outcomes with timers instead of counters. 2020-08-25 10:57:44 -04:00
Jon Chambers 2d75f59d33 Add support for UUID-only delivery certificates. (SERVER-132) 2020-08-20 17:05:53 -04:00
Jon Chambers 34bf5112e0 Drop TimeProvider. 2020-08-20 15:40:24 -04:00
Jon Chambers bfe18d1d28 Re-nerf the clustered message persister. 2020-08-20 15:38:09 -04:00
Jon Chambers 6a76afc20d Add a test to make sure the persister is respecting persist delays. 2020-08-20 15:38:09 -04:00
Jon Chambers 9c469c2f96 Base persister tests on a real Redis cluster. 2020-08-20 15:38:09 -04:00
Jon Chambers af34b43a8d Reactivate the message notification experiment. 2020-08-19 11:39:05 -04:00
Jon Chambers 42ea7a9814 Revert Lettuce connection pooling. 2020-08-18 16:21:42 -04:00
Jon Chambers d243b73678 Make Lettuce connection pools configurable. Double the default size. 2020-08-18 12:20:12 -04:00
Jon Chambers dc28d063aa Reactivate the explicit client presence experiment. 2020-08-17 11:34:27 -04:00
Jon Chambers bb6045c1d0 Disarm the client presence manager experiment. 2020-08-15 20:23:05 -04:00
Jon Chambers f1a74b5939 Disarm new message keyspace notifications. 2020-08-15 20:23:05 -04:00
Jon Chambers 6fb9038af1 Move to a synchronous, pooled connection model for Redis clusters. 2020-08-14 17:15:56 -04:00
Jon Chambers 27f721a1f5 Update to resilience4j 1.5.0. 2020-08-14 17:15:56 -04:00
Jon Chambers 5717dc294e Combine the read/write breakers for Redis clusters. 2020-08-14 17:15:56 -04:00
Jon Chambers ae0f8df11b Break out FaultTolerantPubSubConnection as its own thing so different use cases can have their own subscription space. 2020-08-14 17:15:56 -04:00
Jon Chambers 77460ba502 Remove keyspace notification configuration checks because AWS doesn't support `CONFIG GET`. 2020-08-13 15:32:25 -04:00
Jon Chambers 8d3316ccd6 Listen for new messages via keyspace notifications. 2020-08-13 12:17:04 -04:00
Jon Chambers 2c29f831e8 Add an explicit client presence system. 2020-08-13 10:56:26 -04:00
Jon Chambers 9457325119 Add pub/sub affordances to FaultTolerantRedisCluster. 2020-08-13 10:56:26 -04:00
Jon Chambers 189f8afcc9 Warm up the test cluster before running tests to avoid transient startup jitters. 2020-08-13 10:56:26 -04:00
Jon Chambers 5fad8f74b1 Factor MessagePersister into its own class. 2020-08-11 15:50:22 -04:00
Jon Chambers e35e34d2e0 Move operation-mirroring logic to MessagesManager. 2020-08-11 15:50:22 -04:00
Jon Chambers 31a215d4d6 Use "global." instead of "g." as the prefix for global config options. 2020-08-11 11:55:35 -04:00
Ehren Kret b97158bf7b
Create global remote config controllable in the signal server configuration (#127)
* Add global config controller through file rather than database

* Do no permit attempting to set or delete global config entries
2020-08-10 16:31:15 -05:00
Jon Chambers 3286c5e174 Disable Redis persistence for tests. 2020-08-06 11:22:51 -04:00
Ehren Kret 4fa3a136ad
Remove arbitrary SMS and add a NANPA message service (#123)
* Remove arbitrary SMS code

This code has run its course and is no longer needed for now.

* Add elements to sample config that were left out

* Add a messaging service for NANPA

* Fixup sample config capitalization
2020-08-05 13:35:11 -05:00
Ehren Kret 57e1339230
Further restrict user agent pattern matching (#120)
* Further restrict user agent pattern matching

* Add static qualifier to method
2020-08-04 12:58:16 -05:00
Jon Chambers 0bc5566976 Mirror delete-after-persist operations to the clustered message cache. 2020-08-04 10:58:41 -04:00
Jon Chambers ad97731d46 Reduce the maximum number of versions in play to 1,000. 2020-08-03 15:42:15 -04:00
Jon Chambers 40684a93a2 Restrict user-agent version matching to a more confined space. 2020-08-03 15:42:15 -04:00
Jon Chambers f3b644ceb8 Update the push latency manager to use UUIDs and a Redis cluster. 2020-08-03 15:36:02 -04:00
Jon Chambers 901ba6e87f Added a push latency manager. 2020-08-03 15:36:02 -04:00
Jon Chambers 76389bd584 Clear would-be-persisted messages from the cache cluster, but don't store them to the database. 2020-07-30 19:14:39 -04:00
Jon Chambers 7bf8650d59 Un-manage FaultTolerantRedisCluster so it shuts down at JVM shutdown instead of Jetty shutdown. 2020-07-30 18:37:38 -04:00
Jon Chambers baf563e46d Temporarily disarm the actual persisting part of the message persister. 2020-07-30 17:12:37 -04:00
Jon Chambers beac73b6c8 Add a cluster-capable message persister 2020-07-30 11:39:14 -04:00
Jon Chambers f9f93c77e2 Use UUIDs instead of phone numbers as account identifiers in clustered message cache 2020-07-30 11:39:14 -04:00
Jon Chambers 6fc1b4c6c0 Add a cluster-backed message cache. 2020-07-30 11:39:14 -04:00
Jon Chambers 639898ec07 Expand Experiment to deal with async suppliers and Optionals. 2020-07-30 11:39:14 -04:00
Jon Chambers 3d3790fdbc Add binary execution methods to ClusterLuaScript. 2020-07-30 11:39:14 -04:00
Jon Chambers 69c8968cb0 Add byte-array-based methods to FaultTolerantRedisCluster. 2020-07-30 11:39:14 -04:00
Jon Chambers 7f8f2641f6 Simplify registration lock counting by avoiding inactive accounts. 2020-07-28 11:48:20 -04:00
Jon Chambers 022dbb606f Count registration lock versions when crawling the account database. 2020-07-28 11:48:20 -04:00
Jon Chambers eea073f882 Decommission the old cache. 2020-07-28 10:29:28 -04:00
Jon Chambers fc1d88f5bb Read exclusively from the cache cluster. 2020-07-27 15:11:40 -04:00
Ehren Kret 89bafea61f Move SMS strings to configuration 2020-07-27 11:23:21 -05:00
Jon Chambers 0fbf31ec98 Clear each cluster node individually. 2020-07-22 11:12:21 -04:00
Jon Chambers db9b7ca447 Fix slot assignment when building a cluster for tests. 2020-07-22 11:04:10 -04:00
Jon Chambers eecc71c77f
Revert batch message storage. (#95) 2020-07-20 16:28:32 -04:00
Jon Chambers 215125de26 Update tests. 2020-07-20 10:30:29 -04:00
Jon Chambers 247d869a5c De-randomize message tests to minimize flakiness. 2020-07-14 18:46:39 -04:00
Ehren Kret b7e0e5a356 Create a strategy class to decide which sender id to use
The rules around selecting sender ids can get complicated with some
countries not supporting it and others requiring pre-registration that
may result in having a different sender id for that country than
others. This strategy class handles the logic of dealing with this
expanded configuration and applying the appropriate sender id or none
when it's not appropriate to do so at all.
2020-07-14 10:53:48 -05:00
Jon Chambers c5d0d4acd0 Revert "Move rate limiter logic to Lua scripts"
This reverts commit b585c6676d.
2020-07-09 12:30:25 -04:00
Jon Chambers dcb11f7606 Log errors from experiments. 2020-07-07 15:17:25 -04:00
Jon Chambers 933ce42d5a Test rate limiters against a real cluster. 2020-07-07 15:17:25 -04:00
Ehren Kret e021286eee Add configuration by country for sending from alpha IDs 2020-07-07 10:17:40 -05:00
Ehren Kret 0ee7a66033
Keep trying ports until you get one lower than 55535 (#83)
* Keep trying ports until you get one lower than 55535

* Rename method and change to do...while

* Limit attempts to 11,000 to find an open redis cluster port
2020-07-07 10:12:31 -05:00
Jon Chambers 42c797ee97 Set the default log level for tests to WARN. 2020-07-07 11:05:39 -04:00
Jon Chambers b585c6676d
Move rate limiter logic to Lua scripts 2020-07-06 10:10:13 -04:00
Jon Chambers f5ddb0f1f8 Test ClusterLuaScript against a real Redis cluster. 2020-07-02 18:58:30 -04:00
Ehren Kret 25a5a8db68
Set avatar to null on Account when request is false (#78) 2020-06-29 15:53:31 -05:00
Jon Chambers a68d91b54c Resolve some test flakiness by adding a deterministic "wait" mechanism. (SERVER-86) 2020-06-29 12:24:25 -04:00
Jon Chambers bbf5e1fa78 Use the UA string from websocket upgrade requests if available. 2020-06-17 15:40:18 -04:00
Jon Chambers 6adcebb247 Return to just using counters instead of timers for measuring experiment outcomes. 2020-06-17 15:34:02 -04:00
Jon Chambers a0bebca1e6 Extend Experiment to report more detail when results don't match. 2020-06-16 16:46:41 -04:00
Jon Chambers c17cc07b73 Instrument BlockingThreadPoolExecutor. 2020-06-12 14:43:50 -04:00
Jon Chambers 9afc433db4 Record exceptions associated with server responses. 2020-06-11 22:08:07 -04:00
Jon Chambers f701e3d834 Record distributions of timer values; stop recording error causes. 2020-06-11 11:50:36 -04:00
Jon Chambers 0671f05c05 Introduce experiment comparison methods for suppliers. 2020-06-11 11:50:36 -04:00
Jon Chambers 0713da7393 Record experiment results with a timer instead of a counter. 2020-06-11 11:50:36 -04:00
Jon Chambers 28c765bd9a Add an in-app-context test for websocket metrics. 2020-06-09 15:45:32 -04:00
Jon Chambers ec858b2d4c
Set a timeout for Redis cluster operations and shut down the cluster as part of service shutdown 2020-06-07 18:27:57 -04:00
Jon Chambers 52310b5dd9 Compare results of reads from old and new Redis caches. 2020-06-07 18:27:11 -04:00
Jon Chambers c2a4a2778e Introduce the Experiment class to compare results from parallel systems. 2020-06-07 18:27:11 -04:00
Jon Chambers 1388103919 Mirror writes to the cache cluster. 2020-06-06 20:37:48 -04:00
Jon Chambers fe1054d58a Introduce a Lettuce-based fault-tolerant Redis cluster accessor. 2020-06-06 20:37:48 -04:00
Jon Chambers ba6ac778fc Update to Pushy v0.14.1. 2020-06-05 12:21:56 -04:00
Jon Chambers 228ffcbfce Differentiate between websocket and "boring" HTTP traffic. 2020-05-28 12:52:49 -04:00
Jon Chambers 06c82ee87d Celebrate the diversity of UA strings when generating tags for metrics. 2020-05-27 19:35:42 -04:00
Jon Chambers 9ba5ee8043 Move UA tag extraction into its own utility class. 2020-05-27 19:35:42 -04:00
Ehren Kret eede4e50ca
Use hashed UUID to spread last seen updates over a full day (#40) 2020-05-26 13:38:52 -07:00
Jon Chambers aa10f63d9f Add the timestamp using the `add` method. 2020-05-22 17:39:25 -04:00
Jon Chambers a13c44d81a Capture request-level metrics (path, status, client platform/version). 2020-05-20 17:48:19 -04:00
Ehren Kret 7da9e88c0b Add hashKey to RemoteConfig
This allows the percentages for different entries in remote config to
be aligned so one remote config can be a subset of another.
2020-05-13 11:08:22 -07:00
Jon Chambers 1c73c91133 Report the number of days until the CDS CA cert expires as a metric so we can set an alarm. 2020-05-12 12:57:11 -04:00
Jon Chambers 001a9310c3
Support device transfers (SERVER-41, SERVER-42) (#32)
This change introduces a `transfer` device capability and account creation argument in support of the iOS device transfer effort.
2020-05-12 12:23:18 -04:00
Jon Chambers acfbab5915 Update to Pushy v0.13.11. 2020-05-04 17:50:35 -04:00
Ehren Kret 48c324fe86 Use a static sequence of randomness in tests
The RemoteConfigControllerTest#testMath unit test would occassionally
fail because randomness doesn't necessarily group into expected ranges
over a finite trial count. This changes the test to use a predefined
PRNG sequence instead of one that varies with each test so that the
test will no long randomly fail.
2020-04-29 17:31:43 -07:00
Ehren Kret 0c495e7e72 Workaround lack of internal retry on transaction rollback
The get endpoint for key fetching can fail if the transaction cannot
complete because of simultaneous modification. Clients currently
receive 500 from this and retry if it happens, but this test case runs
into it without retrying and then complains that not all the threads
completed successfully. This workaround adds some retry attempts.
2020-04-29 17:10:13 -07:00
Ehren Kret 50ccfee201 Allow remote config to send non-boolean values
This version of remote config allows non-boolean values to be returned
to clients but unfortunately limits the configuration to only one
value or another. There is no way to configure more than two values
for the same key with this setup.
2020-04-29 10:51:10 -07:00
Moxie Marlinspike 95f0ce1816 Support for advertising payment addresses on profile 2020-04-22 12:32:53 -07:00
Moxie Marlinspike 1408ac77f9 Make storageCapable a boolean result rather than an auth token 2020-04-09 10:19:49 -07:00
Ehren Kret 7e97d10ae1 Fix account dropping new style registration locks 2020-04-06 09:27:23 -07:00
Ehren Kret 41286650cc Create attachments V3 endpoint for CDN2 on GCP
In preparation for resumable uploads, this creates a separate
attachment authorization endpoint that creates a signed URL for
accessing GCP Storage through Signal's CDN2. This should allow Signal
clients to do byte-level resume of media uploads.
2020-04-02 10:20:42 -07:00
Moxie Marlinspike 3c8e7c6c10 Add storage capability and return KBS creds on rereg w/ storage set 2020-03-27 10:45:48 -07:00
Moxie Marlinspike ac1153c7cf Additional limits 2020-03-14 18:10:07 -07:00
Moxie Marlinspike 3b1672a4a7 Update zkgroups to 0.4.0 2020-03-14 16:30:13 -07:00
Moxie Marlinspike e5746c19cf Support for GV2 capability flag 2020-02-07 11:53:28 -08:00
Moxie Marlinspike e399f9e851 Generate external creds for KBS based on UUID 2020-01-22 13:47:33 -08:00
Moxie Marlinspike e4e20c2d25 Add support for UUID buckets in remote config 2020-01-22 11:28:08 -08:00
Moxie Marlinspike 08a70664f4 Support for getting/setting remote config variables 2020-01-21 13:38:58 -08:00
Moxie Marlinspike 75fc35ee4b Parameterize access to zk operations 2020-01-21 11:29:08 -08:00
Moxie Marlinspike ba3102d667 Support for versioned profiles
Includes support for issuing zkgroup auth credentials
2020-01-21 11:04:06 -08:00
Moxie Marlinspike 8a9fed64f2 Support for first/last profile name length 2020-01-13 18:55:04 -08:00
Moxie Marlinspike 71c7e30548 Increase max size for sticker manifest 2019-12-19 10:29:47 -08:00
Moxie Marlinspike 886db1a2c3 Bump max sticker count to 201 2019-12-18 17:08:51 -08:00
Moxie Marlinspike b4c06db031 Make redis failures on write-back retrieve non-fatal 2019-11-20 12:36:22 -08:00
Moxie Marlinspike 2b987e6e93 Usernames can't start with numbers 2019-11-20 12:36:22 -08:00
Moxie Marlinspike 523134f24b Username reservation table 2019-11-20 12:36:22 -08:00
Moxie Marlinspike 99c228dd6d Support for setting and looking up usernames 2019-11-20 12:36:22 -08:00
Moxie Marlinspike c623f70caa Add support for capabilities 2019-11-14 13:36:40 -08:00
Jeffrey Griffin f16b783378 return backup, not storage, credentials for reg lock 2019-11-05 10:36:33 -08:00
Brian Acton be4b75932b since onCrawlChunk() is now protected, we need to invoke timeAndProcessChunk() in our unit tests 2019-10-29 18:20:03 -07:00
Brian Acton eddfacd0f4 add timers to the account crawler listeners 2019-10-25 21:30:48 -07:00
Jeffrey Griffin 69742839c0 uuid-based account crawler 2019-08-27 14:42:14 -07:00
Moxie Marlinspike 20b5f0e681 Reset cache index 2019-08-27 14:08:50 -07:00
Moxie Marlinspike 3803b8f284 Fix for jedis pool deadlock
1) Remove nested pool checkouts

2) Add a max wait so it won't block forever on deadlock
2019-08-27 14:02:42 -07:00
Jeffrey Griffin cf78047830 revert to phone number-based account crawler 2019-08-26 14:00:15 -07:00
Moxie Marlinspike 79f2efdfd9 Make UUID in sealed sender certificate optional for buggy clients 2019-08-26 11:09:54 -07:00
Jeffrey Griffin 07822b371f replicate uuids to contact discovery 2019-08-26 11:09:54 -07:00
Moxie Marlinspike 7a3a385569 Support for UUID based addressing 2019-08-26 11:09:54 -07:00
Moxie Marlinspike 10724fee04 Support for sticker pack uploads 2019-07-24 16:29:56 -07:00
Moxie Marlinspike 11902dec3c Support for v2 registration lock 2019-07-11 18:15:14 -07:00
Moxie Marlinspike 4fdbe9b9ff Support for push preauth 2019-07-11 18:15:10 -07:00
Moxie Marlinspike a6e7e30177 Add requester to recaptcha validation 2019-07-11 17:31:34 -07:00
Moxie Marlinspike 5b69ff7e94 Break out keys database and accounts database 2019-06-19 17:16:37 -07:00
Moxie Marlinspike bc0c6be4c5 We don't need to support disabled accounts for the signed PK API 2019-06-12 12:32:15 -07:00
Moxie Marlinspike 105a38a7db Update gcm-sender-async to use jdk11 httpclient 2019-05-30 10:46:40 -07:00
Moxie Marlinspike a029768d24 Reenable account cleaner 2019-05-10 10:42:42 -07:00
Moxie Marlinspike 4d9c9206cf Delay processing FCM uninstalled feedback
Check to make sure client is not still active before unregistering,
since FCM feedback seems to be often erroneous
2019-05-07 10:04:22 -07:00
Moxie Marlinspike 35116f9229 Clean up concepts of enabled account state
1) Rename "active" methods to be "enabled," since they aren't
   really about "activity."

2) Make authentication fail if a device or account is in dissabled
   state.

3) Let some controllers authenticate accounts that are in a
   disabled state.
2019-05-04 12:31:50 -07:00
Moxie Marlinspike a1f90cd39b Temporarily disable account cleaner 2019-05-03 12:09:01 -07:00
Moxie Marlinspike 45dc7459b8 Temporarily disable GCM unregistered feedback 2019-05-03 11:51:21 -07:00
Jeffrey Griffin 6877b663f1 enable up to 40 account updates per chunk in AccountCleaner 2019-05-03 10:58:57 -07:00
Jeffrey Griffin 3c69f81a10 expire accounts explicitly 2019-05-02 21:14:57 -07:00
Jeffrey Griffin d316d57e5d fix DirectoryController tests 2019-05-02 19:20:23 -07:00
Moxie Marlinspike 0c81556b90 Switch websocket-resources from ListenableFuture to CompletableFuture 2019-05-02 15:05:44 -07:00
Moxie Marlinspike d0d375aeb7 Break out into a multi-module project 2019-05-01 13:19:11 -07:00