Commit Graph

538 Commits

Author SHA1 Message Date
Ehren Kret 9936b2967e Don't use signaling key anymore
Signaling key was deprecated over 2 years ago. It's time for it to go.
2021-02-08 09:45:33 -06:00
Ehren Kret 0971613ac0 Remove messages table from account DB
This is quite out of date. Messages has been a separate database for a
long long time.
2021-02-08 09:45:33 -06:00
Jon Chambers f7d855c59e Drop unsealed sender rate limit logging from `info` to `debug`. 2021-02-05 11:43:37 -05:00
Jon Chambers 2fe743649d Extract common UUID operations into a utility class. 2021-02-05 10:40:27 -05:00
Jon Chambers a015237fd2 Don't request data from DynamoDB if we already have it locally. 2021-02-05 10:40:27 -05:00
Jon Chambers e1f4deaacc Add a generalized countItemsMatchingQuery method for DynamoDB stores. 2021-02-05 10:40:27 -05:00
Jon Chambers 1dceee3fa0 Make DynamoDBConfiguration more consistent. 2021-02-05 10:40:27 -05:00
Jon Chambers 3a17a7c98f Explicitly make the dynamic config worker a daemon thread. 2021-02-04 19:05:51 -05:00
Jon Chambers 3298db8683 Split traffic between the Postgres and Dynamo pre-key stores. 2021-02-04 15:18:53 -05:00
Jon Chambers d4d9403829 Add a Dynamo-backed key store. 2021-02-04 15:18:53 -05:00
Jon Chambers 426e6923ac Continue to poll for config changes after startup. 2021-02-04 13:46:37 -05:00
Jon Chambers b413f665d8 Make DynamicConfigurationManager not-really-Managed. 2021-02-04 11:33:33 -05:00
Moxie Marlinspike 5e1a572bd8
Track impact of unsealed sender rate limits (#374) 2021-02-03 08:58:47 -08:00
Ehren Kret 0dcb4b645c
Build Dynamo DB backed Message Store (#358)
* Work in progress...

* Finish first pass draft of MessagesDynamoDb

* Use begins_with everywhere for destination device id

* Remove now unused methods

* First basic test built

* Add another test case

* Remove comment

* Verify more of the message contents

* Ensure all methods are tested

* Integrate MessagesDynamoDb into the MessagesManager

This change plugs the MessagesDynamoDb class into the live serving
flow in MessagesManager.

Tests are not yet as comprehensive for this big a change as they
should be, but they now compile and pass so checkpointing here with a
commit.

* Put DynamoDB before RDBS when deleting specific messages

* Extract method

* Make aws sdk version into a property

* Rename clientBuilder

* Discard messages with no GUID

* Unify batching logic into one function

* Comment on the source of the value in this constant

* Inline method

* Variable name swizzle

* Add timers to all public methods

* Add missing return statements

* Reject messages that are too large with response code 413

* Add configuration to control dynamo DB timeouts

* Set server timestamp from the ReceiptSender

* Change to shorter key names to optimize IOPS

* Fix tests broken by changing column names

* Fix broken copyright template output

* Remove copyright template error text

* Add experiments to control use of dynamo and rds in message storage

* Specify instance profile credentials for the dynamic configuration manager

* Use property for aws sdk version

* Switch dynamo to instance profile credentials

* Add metrics to the batch write loop

* Use placeholders in logging
2021-02-03 10:03:19 -06:00
Jon Chambers fc4c8d6054 Update to the latest version of libphonenumber. 2021-02-01 21:25:14 -05:00
Jon Chambers 1a27c7eabc Add a (failing) test for new Ivory Coast phone numbers. 2021-02-01 21:25:14 -05:00
Jon Chambers 408b959441 Require a push challenge when registering (or else require a captcha). 2021-02-01 20:44:21 -05:00
Jon Chambers 35fc98a188 Add an experiment enrollment manager. 2021-02-01 11:08:16 -05:00
Moxie Marlinspike 92f6a79e1f
Add a dynamic configuration manager 2021-02-01 11:01:58 -05:00
Jon Chambers 8f94ed68a3 Ignore expired devices when checking for GV1->GV2 migration capability. 2021-01-30 16:55:05 -05:00
Jon Chambers ce1a4b94cb Actually store emoji/about text in the database. 2021-01-27 10:34:13 -05:00
Jon Chambers 92a0deffcf Add more robust tests for about/emoji fields. 2021-01-27 10:34:13 -05:00
Jon Chambers 97b6f6028b Fix a minor typo in the help text for a feature flag task. 2021-01-25 18:03:38 -05:00
Jon Chambers 611e8c39ee Actually drop feature flag config. 2021-01-25 15:20:06 -05:00
Jon Chambers 01f1c263a6 Add a meter for captcha requests. 2021-01-25 14:58:27 -05:00
Jon Chambers 24ea6a9f1d Revert "Temporarily disable registration abuse system"
This reverts commit 22ef058cb6.
2021-01-25 14:58:27 -05:00
Jon Chambers 46c800b8b7 Smoosh request logging tasks together rather than having one task for each direction. 2021-01-25 14:58:15 -05:00
Jon Chambers f10be893ce Drop the old feature flag controller. 2021-01-25 14:55:57 -05:00
Jon Chambers c606c1664f Add admin tasks for listing, setting, and deleting feature flags. 2021-01-25 14:55:57 -05:00
Jon Chambers 225932b4c9
Add emojis/"about" text to profiles 2021-01-20 15:42:47 -05:00
Jon Chambers 6b850b9894 Allow (versioned) profile names up to 380 base64 characters long. 2021-01-20 11:08:10 -05:00
Jon Chambers 943a5d1036
Shard push scheduling cache 2021-01-19 15:50:12 -05:00
Moxie Marlinspike b25da8ceaa
Don't attempt SMS to iran (#355) 2021-01-19 09:13:37 -08:00
Ehren Kret 10cdb7387d Be consistent with use of DataSize class 2021-01-18 17:01:43 -06:00
Ehren Kret dd436dd1dd Create a Meter for tracking messages larger than 256kib 2021-01-18 17:01:43 -06:00
Jon Chambers 13b84635b5
Drop an unused message database index. (#352) 2021-01-18 10:26:03 -06:00
Moxie Marlinspike 27534d408f
Log when messages cache detects topology change (#354)
Co-authored-by: Moxie Marlinspike <moxie+github@signal.org>
2021-01-17 17:13:23 -08:00
Jon Chambers 0a23ce870a Allow message persisters to be disabled by a feature flag. 2021-01-17 11:13:12 -05:00
Jon Chambers c355ef8d53 Reduce the message cache thread pool size. 2021-01-16 11:15:25 -05:00
Jon Chambers 1feb23ba99 Stop periodic topology refreshes. 2021-01-16 03:35:36 -05:00
Jon Chambers 59a0fd0799 Embiggen message cluster thread pool. 2021-01-16 02:57:04 -05:00
Jon Chambers 00b5cfcf17 Allow the client presence manager to use an entirely separate cluster. 2021-01-16 02:57:04 -05:00
Jon Chambers 9e342f253d Use the same client for inserts and reads in the message cache cluster. 2021-01-16 01:50:40 -05:00
Jon Chambers 20c48b6bb2 Expand message-related thread pools to 1 thread per shard. 2021-01-16 01:50:40 -05:00
Jon Chambers 4f9e7bb572 Separate Lettuce thread pools. 2021-01-16 01:18:05 -05:00
Jon Chambers 0a322d5a9f Add a "doomsday switch." 2021-01-15 18:05:18 -05:00
Jon Chambers 59eb6d10c1 Gate based on destination rather than random. 2021-01-15 18:05:18 -05:00
Jon Chambers a57ce1dd17 Add machinery to allow a percentage of message sends to succeed. 2021-01-15 17:05:16 -05:00
Moxie Marlinspike b100b3c36b Reject traffic without logging exceptions 2021-01-15 16:23:53 -05:00
Jon Chambers 81c1ba6eef Respond to all "message send" attempts with HTTP/503. 2021-01-15 15:34:14 -05:00