Commit Graph

393 Commits

Author SHA1 Message Date
Jon Chambers 8fdbcbef44 Send directory updates in batches. 2021-05-04 15:18:43 -04:00
Chris Eager fac4538f6f Migrate rate limiters to rate limiter cluster 2021-05-03 10:57:34 -05:00
Chris Eager 4f6b132449 Add secondaryCacheCluster to RateLimiter 2021-04-30 15:26:17 -05:00
Ehren Kret e4dbb8efe7 Fix deserialization for Duration in dynamic config 2021-04-28 15:57:23 -05:00
Jon Chambers 59e401f41e Use a MIME Base64 decoder for attachment signing keys. 2021-04-26 18:53:15 -04:00
Jon Chambers 6196856a7c Use the JDK-provided Base64 encoder/decoder. 2021-04-26 18:17:03 -04:00
Jon Chambers 0e8d4f9a61 Drop Bouncy Castle as a dependency. 2021-04-26 17:58:19 -04:00
Chris Eager 5ee56b022c Add CustomFriendlyName parameter to Twilio Verify requests 2021-04-26 14:31:52 -05:00
Chris Eager 6c37b658ac Migrate VoiceVerificationControllerTest to JUnit 5 2021-04-26 14:31:52 -05:00
Chris Eager 1f53900345 Migrate VoiceVerificationController to Util#findBestLocale 2021-04-26 14:31:52 -05:00
Chris Eager deece33a0d Fix parameterized testWhoAmI 2021-04-26 14:31:52 -05:00
Chris Eager 13053da97f Add Twilio Verify experiment to AccountController 2021-04-26 14:31:52 -05:00
Chris Eager 4c019aef15 Migrate PendingAccountsTest to JUnit 5 2021-04-26 14:31:52 -05:00
Chris Eager bab5e5769b Add TwilioVerifyExperimentEnrollmentManager 2021-04-26 14:31:52 -05:00
Chris Eager 76cbf734ad Add TwilioVerifySender to TwilioSmsSender 2021-04-26 14:31:52 -05:00
Chris Eager 17ba630014 Add TwilioVerifySender 2021-04-26 14:31:52 -05:00
Chris Eager f190462879 Fully implement unsealed sender cardinality rate limiter 2021-04-23 11:45:53 -05:00
Chris Eager 6c6f073bc2 Pass re-migration account in test 2021-04-22 17:11:10 -05:00
Chris Eager abafa2ccac Include ATTR_MIGRATION_VERSION in update() 2021-04-21 14:13:13 -05:00
Chris Eager 5974328d9c Ensure accounts are deleted after batch migration; store migration failures for later processing 2021-04-21 11:13:07 -05:00
Chris Eager a472774734 Add threadpoool to increase Accounts → Dynamo migration throughput 2021-04-21 11:13:07 -05:00
Chris Eager 166d203e8e Don’t PUT unmigrated accounts in update() 2021-04-21 11:13:07 -05:00
Chris Eager f2a1a65a45 Migrate MessageControllerTest to JUnit 5 2021-04-20 13:00:29 -05:00
Jon Chambers 52478e7de0 Test the account crawler against a real redis cluster. 2021-04-20 13:58:27 -04:00
Chris Eager 59bbd0c43c
Add Accounts DynamoDB
* Add additional test cases to AccountsTest
* Migrate AccountsManagerTest to JUnit 5
* Add AccountsDynamoDbConfiguration
* Add Account.dynamoDbMigrationversion
* Add DynamicAccountsDynamoDbMigrationConfiguration
* Add AccountsDynamoDb to AccountsManager
* Add AccountsDynamoDbMigrator
2021-04-16 14:24:24 -05:00
Andrew Bissell 1f24c913a6 add unit tests for SmsSender 2021-04-08 09:17:47 -04:00
Jon Chambers 2cb788ceb7 Delete secure backups when deleting accounts. 2021-04-01 11:30:48 -04:00
Jon Chambers 257fef9734 Add a secure backup service client. 2021-04-01 11:30:48 -04:00
Jon Chambers ab4e94edab Revert "Revert "Delete data in the storage service when deleting accounts.""
This reverts commit 91fc0fd623.
2021-03-30 17:17:32 -04:00
Chris Eager ad6c271f9d Add dynamic configuration for signup captcha by country code 2021-03-30 10:22:02 -05:00
Chris Eager 91bd061110 Migrate deprecated Lettuce method and enum usages 2021-03-29 14:51:26 -05:00
Ehren Kret 3745a0b81d Update from 684 to 776 for payment address length to account for signature 2021-03-27 00:45:47 -05:00
Jon Chambers 7e29ed1cc7 Block attempts to set wallet addresses from unsupported countries. 2021-03-26 12:21:11 -04:00
Jon Chambers 5965f0fd22 Add a dynamically-configured list of allowed country codes for payments. 2021-03-26 12:21:11 -04:00
Jon Chambers a816aa0186 Revert "Add a storage client method for checking wheter a user has a stored manifest."
This reverts commit 8b6012f8a8.
2021-03-22 10:32:38 -04:00
Jon Chambers 089b6b1644 Retry attempts to get messages after a delay; close connections after a finite number of retries. 2021-03-22 10:32:25 -04:00
Chris Eager e5ae0572c5 Add android-2021-03 verification format 2021-03-18 12:49:50 -05:00
Chris Eager 63dac3bd9f Migrate AccountControllerTest to JUnit 5 2021-03-18 12:45:20 -05:00
Jon Chambers 0bc1369e04 Work through the full list of supported locales when choosing a language for voice verification. 2021-03-18 13:29:09 -04:00
Jon Chambers ca2f7d2eed Parse locale strings when sending voice verification codes. 2021-03-18 13:29:09 -04:00
Jon Chambers 8b6012f8a8 Add a storage client method for checking wheter a user has a stored manifest. 2021-03-18 12:43:12 -04:00
Chris Eager ad838b4827 Add isEnrolled(e164, experiment) for pre-registration experiments 2021-03-16 13:09:10 -05:00
Chris Eager 25f603efc9 Add DynamicPreRegistrationExperimentEnrollmentConfiguration 2021-03-16 13:09:10 -05:00
Chris Eager b5bd16c6a9 Migrate DynamicConfigurationTest to JUnit 5 2021-03-16 13:09:10 -05:00
Chris Eager 14bfa83bb8 Migrate ExperimentEnrollmentManagerTest to JUnit 5 2021-03-16 13:09:10 -05:00
Chris Eager 5dc8086968 Migrate Twilio numbers from static to dynamic configuration 2021-03-16 12:53:51 -05:00
Jon Chambers efe7f2e4c1 Remove vestiges of per-country sender ID logic/configuration. 2021-03-10 14:58:46 -05:00
Jon Chambers fb2fc2335a Require messaging service IDs; remove fallback-to-random-number logic. 2021-03-10 14:58:46 -05:00
Jon Chambers 345e116699 Place our trust in a Twilio message service. 2021-03-10 14:58:46 -05:00
Jon Chambers e50a1c0646 Revert "Turn off alphanumeric sender ID for all countries."
This reverts commit 3bf0188e7f.
2021-03-10 14:58:46 -05:00
Jon Chambers 3cdc58200a Copy headers from the initial websocket upgrade request into subsequent resource requests. 2021-03-10 14:54:11 -05:00
Jon Chambers a1434524a4 Allow the storage service client to trust the Signal CA root. 2021-03-10 10:53:33 -05:00
Jon Chambers 07886a9722 Introduce a utility class for working with forwarding chains in HTTP headers. 2021-03-10 10:37:10 -05:00
Jon Chambers fde1b49729 Introduce a set of gauges for various network statistics as reported by `nstat`. 2021-03-09 11:45:53 -05:00
Jon Chambers af2a8548c3 Use Durations everywhere, drop unused constructors, and add tests. 2021-03-05 12:47:36 -05:00
Jon Chambers f57a4171ba Gather IP-based metrics for international, unsealed-sender messages. 2021-03-05 11:54:01 -05:00
Jon Chambers eb6fe11da1
Add tools to decline messages from senders meeting specific conditions 2021-02-24 18:09:26 -05:00
Jon Chambers 5354104128 Only apply unsealed sender rate limits to targeted country codes. 2021-02-23 14:17:16 -05:00
Ehren Kret b5ade5dc12
Only return payment address from latest profile (#408)
* Only return payment address from latest profile

* Rename `currentVersionedProfile` to `currentProfileVersion`

* Change return type to Optional

* Update service/src/main/java/org/whispersystems/textsecuregcm/controllers/ProfileController.java

Co-authored-by: Jon Chambers <63609320+jon-signal@users.noreply.github.com>

* Fix broken test

Co-authored-by: Jon Chambers <63609320+jon-signal@users.noreply.github.com>
2021-02-23 12:29:47 -06:00
Jon Chambers 06ca5f14fc Record the age of accounts that send unsealed-sender messages. 2021-02-23 12:16:45 -05:00
Chris Eager f85c6bf828 Demonstrate JUnit 5 works by migrating an existing test 2021-02-19 13:30:44 -06:00
Ehren Kret 93f845610d Remove payments list from Account 2021-02-19 13:18:30 -06:00
Chris Eager aa8525385a
Temporarily support `IncomingMessage.online` (#404)
iOS versions prior to 5.5.0.7 send `online` on `IncomingMessage`, rather
than on the top-level entity. This adds a temporary server-side adaptation,
to prevent client-side issues, like persistent typing indicators.
2021-02-18 12:21:16 -06:00
Ehren Kret 142376f360 Add payment address to the versioned profile 2021-02-16 16:12:50 -06:00
Moxie Marlinspike 2dbab70c8c Create utility endpoint for currency conversion 2021-02-16 11:57:34 -08:00
Jon Chambers 5f49772ca6 Control enforcement of unsealed sender rate limits via dynamic configuration. 2021-02-12 15:43:05 -05:00
Jon Chambers ff448950ed Collapse the feature flag system into the dynamic config system. 2021-02-12 15:23:34 -05:00
Jon Chambers ad32555cc9 Drop the old push scheduler Redis singleton. 2021-02-11 14:01:11 -05:00
Ehren Kret be8a1acca9
Remove message database from the codebase (#395)
* Remove message database from the codebase

* Remove unused ExperimentEnrollmentManager in test

* Be more stylish
2021-02-11 10:50:03 -06:00
Jon Chambers e0ed8fa0b8
Introduce a hyper-log-log-based cardinality rate limiter 2021-02-11 10:36:26 -05:00
Jon Chambers dcbf285fae Un-ignore message controller tests. 2021-02-11 10:14:48 -05:00
Jon Chambers 71510a8199 Decommission the old directory cache. 2021-02-10 11:02:11 -05:00
Jon Chambers 2f105ed0a4
Add support for remote client deprecation 2021-02-09 12:24:44 -05:00
Jon Chambers 6c0de89de8 Remove scary message-throttling feature flags. 2021-02-08 11:55:01 -05:00
Jon Chambers aa99e202b4
Clarify behavioral contract of the pre-key store 2021-02-08 11:45:57 -05:00
Jon Chambers 04728ea4bc Drop the old Postgres-based pre-key store. 2021-02-08 11:40:43 -05:00
Ehren Kret d09b36b1d5 Fix deprecated imports for Java6Assertions and Matchers 2021-02-08 09:45:33 -06:00
Ehren Kret a5dd4f5fac Remove signaling key from Device and AccountAttributes 2021-02-08 09:45:33 -06:00
Jon Chambers 3298db8683 Split traffic between the Postgres and Dynamo pre-key stores. 2021-02-04 15:18:53 -05:00
Jon Chambers d4d9403829 Add a Dynamo-backed key store. 2021-02-04 15:18:53 -05:00
Moxie Marlinspike 5e1a572bd8
Track impact of unsealed sender rate limits (#374) 2021-02-03 08:58:47 -08:00
Ehren Kret 0dcb4b645c
Build Dynamo DB backed Message Store (#358)
* Work in progress...

* Finish first pass draft of MessagesDynamoDb

* Use begins_with everywhere for destination device id

* Remove now unused methods

* First basic test built

* Add another test case

* Remove comment

* Verify more of the message contents

* Ensure all methods are tested

* Integrate MessagesDynamoDb into the MessagesManager

This change plugs the MessagesDynamoDb class into the live serving
flow in MessagesManager.

Tests are not yet as comprehensive for this big a change as they
should be, but they now compile and pass so checkpointing here with a
commit.

* Put DynamoDB before RDBS when deleting specific messages

* Extract method

* Make aws sdk version into a property

* Rename clientBuilder

* Discard messages with no GUID

* Unify batching logic into one function

* Comment on the source of the value in this constant

* Inline method

* Variable name swizzle

* Add timers to all public methods

* Add missing return statements

* Reject messages that are too large with response code 413

* Add configuration to control dynamo DB timeouts

* Set server timestamp from the ReceiptSender

* Change to shorter key names to optimize IOPS

* Fix tests broken by changing column names

* Fix broken copyright template output

* Remove copyright template error text

* Add experiments to control use of dynamo and rds in message storage

* Specify instance profile credentials for the dynamic configuration manager

* Use property for aws sdk version

* Switch dynamo to instance profile credentials

* Add metrics to the batch write loop

* Use placeholders in logging
2021-02-03 10:03:19 -06:00
Jon Chambers 1a27c7eabc Add a (failing) test for new Ivory Coast phone numbers. 2021-02-01 21:25:14 -05:00
Jon Chambers 408b959441 Require a push challenge when registering (or else require a captcha). 2021-02-01 20:44:21 -05:00
Jon Chambers 35fc98a188 Add an experiment enrollment manager. 2021-02-01 11:08:16 -05:00
Moxie Marlinspike 92f6a79e1f
Add a dynamic configuration manager 2021-02-01 11:01:58 -05:00
Jon Chambers 8f94ed68a3 Ignore expired devices when checking for GV1->GV2 migration capability. 2021-01-30 16:55:05 -05:00
Jon Chambers 92a0deffcf Add more robust tests for about/emoji fields. 2021-01-27 10:34:13 -05:00
Jon Chambers 24ea6a9f1d Revert "Temporarily disable registration abuse system"
This reverts commit 22ef058cb6.
2021-01-25 14:58:27 -05:00
Jon Chambers f10be893ce Drop the old feature flag controller. 2021-01-25 14:55:57 -05:00
Jon Chambers c606c1664f Add admin tasks for listing, setting, and deleting feature flags. 2021-01-25 14:55:57 -05:00
Jon Chambers 225932b4c9
Add emojis/"about" text to profiles 2021-01-20 15:42:47 -05:00
Jon Chambers 6b850b9894 Allow (versioned) profile names up to 380 base64 characters long. 2021-01-20 11:08:10 -05:00
Jon Chambers 943a5d1036
Shard push scheduling cache 2021-01-19 15:50:12 -05:00
Jon Chambers 0a23ce870a Allow message persisters to be disabled by a feature flag. 2021-01-17 11:13:12 -05:00
Jon Chambers a57ce1dd17 Add machinery to allow a percentage of message sends to succeed. 2021-01-15 17:05:16 -05:00
Jon Chambers 81c1ba6eef Respond to all "message send" attempts with HTTP/503. 2021-01-15 15:34:14 -05:00
Jon Chambers 9c53d818f4 Use separate clusters for message cache read/write operations. 2021-01-15 13:51:39 -05:00
Jon Chambers e5a2c1ab10 Always return an empty list of prekeys. 2021-01-15 12:27:10 -05:00
Jon Chambers 67ed035b36 Retry serializable key transactions. 2021-01-13 17:38:29 -05:00