Commit Graph

528 Commits

Author SHA1 Message Date
Jon Chambers 426e6923ac Continue to poll for config changes after startup. 2021-02-04 13:46:37 -05:00
Jon Chambers b413f665d8 Make DynamicConfigurationManager not-really-Managed. 2021-02-04 11:33:33 -05:00
Moxie Marlinspike 5e1a572bd8
Track impact of unsealed sender rate limits (#374) 2021-02-03 08:58:47 -08:00
Ehren Kret 0dcb4b645c
Build Dynamo DB backed Message Store (#358)
* Work in progress...

* Finish first pass draft of MessagesDynamoDb

* Use begins_with everywhere for destination device id

* Remove now unused methods

* First basic test built

* Add another test case

* Remove comment

* Verify more of the message contents

* Ensure all methods are tested

* Integrate MessagesDynamoDb into the MessagesManager

This change plugs the MessagesDynamoDb class into the live serving
flow in MessagesManager.

Tests are not yet as comprehensive for this big a change as they
should be, but they now compile and pass so checkpointing here with a
commit.

* Put DynamoDB before RDBS when deleting specific messages

* Extract method

* Make aws sdk version into a property

* Rename clientBuilder

* Discard messages with no GUID

* Unify batching logic into one function

* Comment on the source of the value in this constant

* Inline method

* Variable name swizzle

* Add timers to all public methods

* Add missing return statements

* Reject messages that are too large with response code 413

* Add configuration to control dynamo DB timeouts

* Set server timestamp from the ReceiptSender

* Change to shorter key names to optimize IOPS

* Fix tests broken by changing column names

* Fix broken copyright template output

* Remove copyright template error text

* Add experiments to control use of dynamo and rds in message storage

* Specify instance profile credentials for the dynamic configuration manager

* Use property for aws sdk version

* Switch dynamo to instance profile credentials

* Add metrics to the batch write loop

* Use placeholders in logging
2021-02-03 10:03:19 -06:00
Jon Chambers fc4c8d6054 Update to the latest version of libphonenumber. 2021-02-01 21:25:14 -05:00
Jon Chambers 1a27c7eabc Add a (failing) test for new Ivory Coast phone numbers. 2021-02-01 21:25:14 -05:00
Jon Chambers 408b959441 Require a push challenge when registering (or else require a captcha). 2021-02-01 20:44:21 -05:00
Jon Chambers 35fc98a188 Add an experiment enrollment manager. 2021-02-01 11:08:16 -05:00
Moxie Marlinspike 92f6a79e1f
Add a dynamic configuration manager 2021-02-01 11:01:58 -05:00
Jon Chambers 8f94ed68a3 Ignore expired devices when checking for GV1->GV2 migration capability. 2021-01-30 16:55:05 -05:00
Jon Chambers ce1a4b94cb Actually store emoji/about text in the database. 2021-01-27 10:34:13 -05:00
Jon Chambers 92a0deffcf Add more robust tests for about/emoji fields. 2021-01-27 10:34:13 -05:00
Jon Chambers 97b6f6028b Fix a minor typo in the help text for a feature flag task. 2021-01-25 18:03:38 -05:00
Jon Chambers 611e8c39ee Actually drop feature flag config. 2021-01-25 15:20:06 -05:00
Jon Chambers 01f1c263a6 Add a meter for captcha requests. 2021-01-25 14:58:27 -05:00
Jon Chambers 24ea6a9f1d Revert "Temporarily disable registration abuse system"
This reverts commit 22ef058cb6.
2021-01-25 14:58:27 -05:00
Jon Chambers 46c800b8b7 Smoosh request logging tasks together rather than having one task for each direction. 2021-01-25 14:58:15 -05:00
Jon Chambers f10be893ce Drop the old feature flag controller. 2021-01-25 14:55:57 -05:00
Jon Chambers c606c1664f Add admin tasks for listing, setting, and deleting feature flags. 2021-01-25 14:55:57 -05:00
Jon Chambers 225932b4c9
Add emojis/"about" text to profiles 2021-01-20 15:42:47 -05:00
Jon Chambers 6b850b9894 Allow (versioned) profile names up to 380 base64 characters long. 2021-01-20 11:08:10 -05:00
Jon Chambers 943a5d1036
Shard push scheduling cache 2021-01-19 15:50:12 -05:00
Moxie Marlinspike b25da8ceaa
Don't attempt SMS to iran (#355) 2021-01-19 09:13:37 -08:00
Ehren Kret 10cdb7387d Be consistent with use of DataSize class 2021-01-18 17:01:43 -06:00
Ehren Kret dd436dd1dd Create a Meter for tracking messages larger than 256kib 2021-01-18 17:01:43 -06:00
Jon Chambers 13b84635b5
Drop an unused message database index. (#352) 2021-01-18 10:26:03 -06:00
Moxie Marlinspike 27534d408f
Log when messages cache detects topology change (#354)
Co-authored-by: Moxie Marlinspike <moxie+github@signal.org>
2021-01-17 17:13:23 -08:00
Jon Chambers 0a23ce870a Allow message persisters to be disabled by a feature flag. 2021-01-17 11:13:12 -05:00
Jon Chambers c355ef8d53 Reduce the message cache thread pool size. 2021-01-16 11:15:25 -05:00
Jon Chambers 1feb23ba99 Stop periodic topology refreshes. 2021-01-16 03:35:36 -05:00
Jon Chambers 59a0fd0799 Embiggen message cluster thread pool. 2021-01-16 02:57:04 -05:00
Jon Chambers 00b5cfcf17 Allow the client presence manager to use an entirely separate cluster. 2021-01-16 02:57:04 -05:00
Jon Chambers 9e342f253d Use the same client for inserts and reads in the message cache cluster. 2021-01-16 01:50:40 -05:00
Jon Chambers 20c48b6bb2 Expand message-related thread pools to 1 thread per shard. 2021-01-16 01:50:40 -05:00
Jon Chambers 4f9e7bb572 Separate Lettuce thread pools. 2021-01-16 01:18:05 -05:00
Jon Chambers 0a322d5a9f Add a "doomsday switch." 2021-01-15 18:05:18 -05:00
Jon Chambers 59eb6d10c1 Gate based on destination rather than random. 2021-01-15 18:05:18 -05:00
Jon Chambers a57ce1dd17 Add machinery to allow a percentage of message sends to succeed. 2021-01-15 17:05:16 -05:00
Moxie Marlinspike b100b3c36b Reject traffic without logging exceptions 2021-01-15 16:23:53 -05:00
Jon Chambers 81c1ba6eef Respond to all "message send" attempts with HTTP/503. 2021-01-15 15:34:14 -05:00
Jon Chambers 93ae4d1ee6 Move the client presence manager to its own breaker. 2021-01-15 13:51:39 -05:00
Jon Chambers 9c53d818f4 Use separate clusters for message cache read/write operations. 2021-01-15 13:51:39 -05:00
Jon Chambers e5a2c1ab10 Always return an empty list of prekeys. 2021-01-15 12:27:10 -05:00
Jon Chambers 67ed035b36 Retry serializable key transactions. 2021-01-13 17:38:29 -05:00
Jon Chambers ad30786f4a Parallelize message persisters. 2021-01-12 18:50:14 -05:00
Jon Chambers 2e01da5ec1 Add a task to enable/disable accelerated crawling. 2021-01-11 19:29:18 -05:00
Jon Chambers 8fb37a0024 Log when a crawling cycle has wrapped up. 2021-01-11 19:29:18 -05:00
Jon Chambers 9412a7424c Return HTTP/429 whenever somebody tries to get contacts from the old directory system. 2021-01-11 19:29:10 -05:00
Jon Chambers f8cbb4f386 Temporarily suspend client version metrics to reduce load on our metric aggregator. 2021-01-11 14:04:44 -05:00
Ehren Kret 86ccaa52a5
Allow configuration of multiple directory account crawler listeners (#325)
* Allow configuration of multiple directory account crawler listeners

Only one should update the local redis directory. This one is marked
with replicationPrimary true. The others in the list only serve to
issue replication requests over to CDS replication load balancers.

* Update one more metric name
2021-01-10 17:11:02 -06:00