Commit Graph

2637 Commits

Author SHA1 Message Date
Chris Eager ecbef9c6ee Add micrometer metrics to RateLimiter 2023-02-27 16:33:27 -06:00
Chris Eager ef2cc6620e Add `@Produces` annotation for validation error response 2023-02-27 16:33:18 -06:00
ravi-signal b8f363b187
Add documentation to challenge controller 2023-02-24 17:41:15 -06:00
Sergey Skrobotov c3f4956ead OpenAPI support 2023-02-24 13:03:30 -08:00
Chris Eager 047f4a1c00 Update metric name 2023-02-24 13:07:07 -06:00
Sergey Skrobotov 41c0fe9ffa Adding a uniform configuration for all json/yaml mapper use cases: part 2 2023-02-24 09:28:55 -08:00
Sergey Skrobotov 6edb0d49e9 Adding a uniform configuration for all json/yaml mapper use cases: bugfix 2023-02-23 20:01:32 -08:00
Sergey Skrobotov b9b4e3fdd8 Adding a uniform configuration for all json/yaml mapper use cases: part 1 2023-02-23 16:38:48 -08:00
Jon Chambers 6ee9c6ad46 Remove deprecated registration service response fields 2023-02-23 12:41:56 -08:00
Sergey Skrobotov 7529c35013 Rate limiters code refactored 2023-02-23 10:49:06 -08:00
erik-signal 378b32d44d
Add missing token field to OutgoingMessageEntity 2023-02-23 11:18:07 -05:00
Chris Eager e1fcd3e3f6 Remove Lettuce command latency recorder 2023-02-23 10:17:31 -06:00
Chris Eager d7ad8dd448 Add micrometer timer to `FaultTolerantPubSubConnection` 2023-02-23 10:17:24 -06:00
Chris Eager 859f2302a9 Remove unused metrics 2023-02-23 10:17:24 -06:00
Chris Eager a6d11789e9 Add `ClosedChannelException` to expected errors 2023-02-23 10:17:16 -06:00
Chris Eager 43f83076fa Update to reactor 3.5.3 2023-02-23 10:16:57 -06:00
erik-signal 71c0fc8d4a
Improve metrics around spam report tokens. 2023-02-22 15:43:44 -05:00
Chris Eager 1f4f926ce6 Add platform tag to subscription receipt metrics 2023-02-22 14:31:30 -06:00
Chris Eager 35286f838e
Add `/v1/verification` 2023-02-22 14:27:05 -06:00
Jon Chambers e1ea3795bb Reuse registration sessions if possible when requesting pre-auth codes 2023-02-22 12:45:26 -05:00
erik-signal 95237a22a9
Relax validation to allow null reporting tokens. 2023-02-22 11:06:51 -05:00
Katherine Yen 11c93c5f53
Keep username hash during reregistration 2023-02-21 09:07:30 -08:00
Jon Chambers b59b8621c5 Add reporter platform as a reported message dimension 2023-02-17 16:44:13 -05:00
Chris Eager 44c61d9a58 Allow updates if the profile already has a payment address 2023-02-17 16:44:01 -05:00
Jon Chambers 7561622bc8 Log cases where we fall back to a no-op spam-reporting token provider 2023-02-14 12:35:56 -05:00
Jon Chambers b041566aba Simplify construction of spam reporting token providers 2023-02-14 12:35:56 -05:00
Jon Chambers cb72158abc Add the presence of spam reporting tokens as a dimension 2023-02-14 12:35:21 -05:00
Jon Chambers 5c432d094f Fix a typo in a metric name 2023-02-14 12:34:48 -05:00
Chris Eager 24ac48b3b1 Update counter name 2023-02-10 14:54:02 -06:00
Katherine Yen c03060fe3c
Phone number discoverability update endpoint 2023-02-10 11:52:51 -08:00
Chris Eager c16006dc4b Add `PUT /v2/account/number` 2023-02-10 12:09:03 -06:00
Sergey Skrobotov 8fc465b3e8
removing redundant logic in new registration flow 2023-02-09 09:06:48 -08:00
Chris Eager ce689bdff3
Use `DisabledPermittedAuthenticatedAccount` at `DELETE /v1/accounts/me` 2023-02-09 09:05:29 -08:00
Chris Eager e23386ddc7
Remove unused JUnit extension from test 2023-02-09 09:05:11 -08:00
Jon Chambers 0f17d63774
Add tests for `ProvisioningController` 2023-02-09 09:04:52 -08:00
Katherine Yen 4fc3949367
Add zkproof validation in username flow 2023-02-09 09:02:53 -08:00
Sergey Skrobotov 7c3f429c56 Update E164 constraint message 2023-02-08 13:22:00 -08:00
Sergey Skrobotov 7558489ad0 Registration Recovery Password support in `/v1/registration` 2023-02-08 13:20:23 -08:00
Katherine Yen 4a3880b5ae
`usernameHashes` on reserve request can't be null 2023-02-07 08:44:04 -08:00
Chris Eager a4a45de161
Add `/v1/registration` 2023-02-06 16:11:59 -06:00
Chris Eager 358a286523
Use `java.util` Hex and Base64 codecs 2023-02-06 12:16:59 -06:00
Sergey Skrobotov 8afe917a6c Registration recovery passwords store and manager 2023-02-03 16:33:03 -08:00
Katherine Yen 24f515ccb4
Revert "Revert "Stored hashed username"" 2023-02-02 11:20:44 -08:00
Katherine Yen c98b54ff15
Revert "Stored hashed username" 2023-02-01 14:31:44 -08:00
Katherine Yen d93d50d038
Stored hashed username 2023-02-01 12:08:25 -08:00
Jon Chambers 448365c7a0 Preserve legacy registration API error handling 2023-01-31 15:45:23 -05:00
Sergey Skrobotov 8d0e23bde1 AuthenticationCredentials name changed to SaltedTokenHash 2023-01-30 15:45:24 -08:00
Sergey Skrobotov dc8f62a4ad `/v1/backup/auth/check` endpoint added 2023-01-30 15:39:42 -08:00
Jon Chambers cd4a4b1dcf Retire `VoiceVerificationController` 2023-01-30 16:28:14 -05:00
Jon Chambers 38a0737afb Retire `ReportSpamTokenHandler` interface in favor of `ReportedMessageListener` 2023-01-30 16:27:54 -05:00
Jon Chambers 4a2768b81d Add spam report token support to `ReportedMessageListener` 2023-01-30 16:27:54 -05:00
Jon Chambers 00e08b8402 Simplify parsing/validation of spam report tokens 2023-01-30 16:27:54 -05:00
erik-signal a89e30fe75
Clarify naming around spam filtering. 2023-01-27 11:40:33 -05:00
gram-signal a01fcdad28
Add in controller for SVR2 auth. 2023-01-27 09:15:52 -07:00
Chris Eager 2a99529921 Remove old badge strings 2023-01-26 09:23:11 -06:00
Sergey Skrobotov c934405a3e fixing config field names 2023-01-25 17:28:03 -08:00
Sergey Skrobotov eb499833c6 refactoring of ExternalServiceCredentialGenerator 2023-01-25 15:20:28 -08:00
Chris Eager dd98f7f043 Support changing just the currency of an existing subscription 2023-01-25 15:14:17 -06:00
Chris Eager e8978ef91c Add tests for `SubscriptionController#setSubscriptionLevel` 2023-01-25 15:14:17 -06:00
Jon Chambers 4ce85fdb19 Treat "check code" exceptions as `false` for legacy API compatibility 2023-01-25 14:39:29 -05:00
Jon Chambers 035ddc4834 Fix a mistake where we're looking for verification codes in place that hasn't been deployed yet (but will be soon!) 2023-01-25 11:43:06 -05:00
Chris Eager c2f40b8503 Remove duplicate code 2023-01-25 11:09:23 -05:00
Jon Chambers cf738a1c14 Look for registration service errors in response bodies in addition to status responses 2023-01-25 10:49:36 -05:00
erik-signal 52d40c2321
Add metrics for spam report tokens received. 2023-01-24 10:25:40 -05:00
erik-signal ab26a65b6a
Introduce spam report tokens 2023-01-19 11:13:43 -05:00
Jon Chambers 1c1714b2c2 Clarify a counter name 2023-01-17 17:13:06 -05:00
Jon Chambers accb017ec5 Use a longer expiration window for quantile calculation 2023-01-17 17:13:06 -05:00
Chris Eager 304782d583 Use processor from SubscriptionProcessorManager for issued receipts 2023-01-17 16:12:03 -06:00
Chris Eager f361f436d8
Support PayPal for recurring donations 2023-01-17 12:20:17 -06:00
Dimitris Apostolou 9a06c40a28 Fix typos 2023-01-13 16:05:06 -06:00
Chris Eager e6ab97dc5a Update enabled-required authenticator metrics 2023-01-13 14:05:56 -06:00
Chris Eager b8967b75c6 Update dropwizard to 2.0.34 2023-01-13 12:17:39 -06:00
Chris Eager 74d9849472 Update badge strings 2023-01-13 12:08:05 -06:00
Fedor Indutny 96b753cfd0
Add an extra kb to max sticker size 2023-01-13 12:07:45 -06:00
Jon Chambers 5a89e66fc0 Convert `AccountIdentityResponse` to a record 2023-01-13 12:36:17 -05:00
Jon Chambers b4a143b9de Convert `RegistrationLockFailure` to a record 2023-01-13 12:36:02 -05:00
Jon Chambers 050035dd52 Convert `ExternalServiceCredentials` to a record 2023-01-13 12:36:02 -05:00
Jon Chambers 7018062606 Explicitly create registration sessions 2023-01-09 15:27:07 -05:00
Jon Chambers 9e1485de0a Assume stored verification codes will always have a session ID instead of a verification code 2023-01-09 15:27:07 -05:00
Jon Chambers 4e358b891f Retire `StoredVerificationCode#twilioVerificationSid` 2023-01-09 15:27:07 -05:00
Ehren Kret 4044a9df30 stop warning about lack of syntax specification during proto generation 2023-01-09 12:20:07 -06:00
Ehren Kret 5a7b675001 import cleanup on controllers package 2023-01-09 12:20:07 -06:00
Ehren Kret 3be4e4bc57 remove unused exception type 2023-01-09 12:20:07 -06:00
Chris Eager 5de51919bb Remove `Subscriptions.PCI` 2023-01-05 12:02:34 -06:00
Chris Eager b02b00818b Remove `Subscriptions.PCI` attribute 2023-01-04 11:31:46 -06:00
Chris Eager 010f88a2ad Remove `Subscriptions.C` attribute 2023-01-04 11:31:46 -06:00
Jon Chambers 60edf4835f Add a `pni` capability to `UserCapabilities` 2022-12-21 16:26:07 -05:00
Jon Chambers a60450d931 Convert `UserCapabilities` to a record 2022-12-21 16:26:07 -05:00
erik-signal d138fa45df
Handle edge cases of Math.abs on integers. 2022-12-20 12:25:04 -05:00
Katherine Yen 2c2c497c12
Define reregistrationIdleDays DistributionSummary with custom expiry 2022-12-20 09:21:24 -08:00
Katherine Yen cb5d3840d9
Add paymentActivation capability 2022-12-20 09:20:42 -08:00
Fedor Indutny 9aceaa7a4d
Introduce ArtController 2022-12-19 11:58:16 -08:00
Katherine Yen 636c8ba384
Add metric for distribution of account idle time at reregistration 2022-12-16 13:50:29 -08:00
Ravi Khadiwala 65ad3fe623 Add hCaptcha support 2022-12-16 11:28:30 -06:00
Chris Eager 24ac32e6e6 Add `PayPalExperienceProfileInput.userAction` 2022-12-13 10:03:58 -06:00
Katherine Yen 26f5ffdde3
Enable case-sensitive usernames 2022-12-13 07:59:37 -08:00
Jon Chambers a883426402 Simplify account cleaner 2022-12-06 16:21:25 -06:00
Chris Eager 2f21e930e2 Add `minimum` one-time donation amont to validation error map 2022-12-06 16:21:15 -06:00
Chris Eager 5fb158635c Use existing `WebApplicationException` entity, if available 2022-12-06 16:21:15 -06:00
Sergey Skrobotov 9cf2635528 some accounts classes refactorings 2022-12-05 09:30:40 -08:00
Chris Eager d0e7579f13 Revert transaction descriptor 2022-12-01 18:52:45 -06:00
Chris Eager 2ecbb18fe5 Add support for one-time PayPal donations 2022-12-01 18:11:35 -06:00
Chris Eager df8fb5cab7 Move messages cache stale discard to a separate scheduler 2022-12-01 18:09:28 -06:00
katherine-signal 99ad211c01 Enforce minimum amount by currency for one time donations 2022-11-28 11:44:59 -08:00
katherine-signal fb4ed20ff5
Remove groups v2 capability
* wip removing groups v2 capabilities

* comments

* finish removing groups v2 references

* hardcode gv1migration flag on user capability, remove other references
2022-11-21 09:31:47 -08:00
Jon Chambers cb50b44d8f Allow the account cleaner to operate on multiple accounts in parallel 2022-11-18 11:15:00 -05:00
Jon Chambers ae57853ec4 Simplify deletion reason reporting 2022-11-18 11:15:00 -05:00
Jon Chambers 2881c0fd7e Allow the account cleaner to act on all accounts in a crawled chunk 2022-11-18 11:15:00 -05:00
Chris Eager 483fb0968b Use badge name in level configuration for one-time donations 2022-11-18 11:05:23 -05:00
Jon Chambers e8ee4b50ff Retire the legacy "abusive hosts" system in favor of newer tools 2022-11-18 10:54:25 -05:00
Chris Eager 4f8aa2eee2 Mark flaky test `@Disabled` 2022-11-17 13:23:42 -06:00
Chris Eager 397d3cb45a
Add consolidated subscription configuration API 2022-11-16 12:27:00 -06:00
Chris Eager e883d727fb
Note deprecation of localized string 2022-11-16 12:09:00 -06:00
Chris Eager 986545a140
Set `error_if_incomplete` for subscription payment behavior 2022-11-16 12:08:21 -06:00
Sergey Skrobotov 836307b0c7 adding a metric for ipv4/ipv6 requests count 2022-11-15 11:17:01 -08:00
Sergey Skrobotov c32067759c refactoring: use constants for header names 2022-11-15 11:16:49 -08:00
Chris Eager f41bdf1acb Make `MessagesController#getPendingMessages` fully async 2022-11-11 13:19:57 -06:00
Chris Eager 77d691df59 Always use reactived message processing in `WebSocketConnection` 2022-11-11 13:14:39 -06:00
Jon Chambers a01f96e0e4 Temporarily disable account freezing on contention 2022-11-10 18:53:58 -05:00
erik-signal 1d1e3ba79d
Add metric to track newly-locked accounts. 2022-11-10 12:55:08 -05:00
Jon Chambers 2c9c50711f
Avoid reading from a stale `Account` after a contested reglock event 2022-11-10 12:41:50 -05:00
Jon Chambers d3f0ab8c6d
Introduce an alternative exchange rate data provider 2022-11-10 10:25:06 -05:00
erik-signal 80a3a8a43c Lock account when number owner lacks registration lock. 2022-11-09 14:03:09 -05:00
Chris Eager e6e6eb323d Update metric name 2022-11-08 11:15:42 -06:00
Chris Eager 681a5bafb4
Update `MessagesManager#getMessagesForDevice`
- add `subscribeOn()`
- use `CompletableFuture` for consistency
2022-11-08 09:38:52 -06:00
Chris Eager 5bec89ecc8 Measure individual message timeouts 2022-11-08 09:37:37 -06:00
Chris Eager 69ed0edb74
Revert "Add more detailed queue processing rate metrics"
This reverts commit bbbab4b8a4.
2022-11-08 09:35:39 -06:00
Chris Eager ad5925908e Change dispatch queues to `LinkedBlockingQueue`s 2022-11-04 11:08:17 -05:00
Chris Eager d186245c5c Move all receipt sending work to executor 2022-11-04 11:08:06 -05:00
Chris Eager bbbab4b8a4 Add more detailed queue processing rate metrics 2022-11-04 11:06:38 -05:00
Chris Eager f83080eb8d Update metric name 2022-11-03 14:50:20 -05:00
Chris Eager e0178fa0ea Move additional handling of `MessagesManager#delete` to executor 2022-11-03 13:02:25 -05:00
Chris Eager c6a79ca176 Enable metrics on messages fluxes 2022-11-03 13:02:25 -05:00
Chris Eager 6426e6cc49 Enable reactor Schedulers metrics 2022-11-03 13:02:25 -05:00
Chris Eager b13cb098ce lettuce: set `publishOnScheduler` to `true` 2022-11-03 13:02:25 -05:00
Jon Chambers afda5ca98f Add a test for checking push challenge tokens 2022-11-03 11:14:59 -05:00
Chris Eager eb57d87513 Remove message listener key only after successfully unsubscribing 2022-11-03 11:09:11 -05:00
Chris Eager fbf6b9826e tests: only call `SQLite.setLibraryPath` once 2022-11-03 11:08:43 -05:00
Chris Eager a01b29a6bd set off_session=true for subscription updates 2022-11-02 14:34:26 -05:00
Chris Eager 102992b095 Set `off_session=true` when creating subscriptions 2022-11-02 11:30:29 -05:00
Chris Eager bd69905f2e Remove obsolete donation endpoint 2022-11-02 11:29:03 -05:00
Chris Eager aa829af43b Handle expected case of empty flux in message deletion 2022-10-31 12:29:25 -05:00
Chris Eager c10fda8363
Use reactive streams for WebSocket message queue
Initially, uses `ExperimentEnrollmentManager` to do a safe rollout.
2022-10-31 10:35:37 -05:00
Jon Chambers 74d65b37a8 Discard old Twilio machinery and rely entirely on the stand-alone registration service 2022-10-28 10:40:37 -04:00
Jon Chambers 91626dea45
Count accounts rather than devices that are stories-capable 2022-10-25 16:36:05 -04:00
sergey-signal 5868d9969a
minor changes to utility classes (#1127) 2022-10-25 08:48:56 -07:00
erik-signal 90490c9c84
Clean up the TestClock code a bit more. 2022-10-21 15:27:15 -04:00
Chris Eager 8ea794baef
Add additional handling for nullable field in recurring donation record 2022-10-21 12:56:39 -05:00
erik-signal fe60cf003f
Clean up testing with clocks. 2022-10-21 12:39:47 -04:00
erik-signal 0c357bc340
Add metrics tracking story capability adoption. 2022-10-20 12:25:03 -04:00
Chris Eager b711288faa
Run GitHub Action in a container 2022-10-18 16:59:35 -05:00
Jon Chambers e7048aa9cf Allow the reconciliation client to trust multiple CA certificates to facilitate certificate rotation 2022-10-18 11:17:47 -04:00
Jon Chambers 0120a85c39 Allow HTTP clients to trust multiple certificates to support certificate rollover 2022-10-18 11:17:47 -04:00
Jon Chambers a41d047f58 Retire `CertificateExpirationGauge` in favor of other expiration monitoring tools 2022-10-18 11:17:47 -04:00
Jon Chambers 0a64e31625 Check verification codes for changing phone numbers against the stand-alone registration service when possible 2022-10-18 11:17:15 -04:00
Jon Chambers 8088b58b3b Clarify default value for `includeE164` 2022-10-18 11:16:06 -04:00
erik-signal a7d5d51fb4
Improve testing of MultiRecipientMessageProvider 2022-10-17 16:50:39 -04:00
Chris Eager 378d7987a8
device capabilities: prevent `stories` downgrade 2022-10-17 15:25:13 -04:00
erik-signal 3e0baf82a4
Filter unknown UUIDs for /multi_recipient&story=true. 2022-10-13 15:33:51 -04:00
erik-signal 17a3c90286
Add "urgent" query parameter to /v1/messages/multi_recipient endpoint. 2022-10-11 11:10:11 -04:00
Chris Eager 6341770768
Update `SubscriptionManager` to store processor+customerId in a single attribute and a map
- add `type` query parameter to `/v1/subscription/{subscriberId}/create_payment_method`
2022-10-07 14:26:17 -05:00
Jon Chambers d2fa00f0c6
Add experiment to test standalone registration service 2022-10-06 15:42:53 -04:00
erik-signal d6c9652a70
Fix internal server error when sending stories to unknown recipient. 2022-10-06 13:53:57 -04:00
Jon Chambers 3c655cdd5a Migrate to "regionCode" instead of "region" to avoid tag name conflicts 2022-10-05 15:15:46 -04:00
Jon Chambers 83ab926f96 Add a dimension for story messages 2022-10-05 15:15:22 -04:00
erik-signal 544e4fb89a
Adjust routing for stories. 2022-10-05 12:20:42 -04:00
erik-signal 966c3a8f47 Add routing for stories. 2022-10-05 10:44:50 -04:00
Ravi Khadiwala c82c2c0ba4 Add country tag to twilio failures 2022-09-30 12:03:46 -05:00
Ravi Khadiwala 6e595a0959 add an optionals utility and fix push challenge metric 2022-09-30 12:02:47 -05:00
Ravi Khadiwala a79d709039 Return 403 when a push challenge is incorrect 2022-09-30 12:02:47 -05:00
Ravi Khadiwala 07ed765250 Update abusive message filter and filter account creates 2022-09-20 14:52:18 -05:00
Ravi Khadiwala 2e497b5834 Fix operator order in metric calculation 2022-09-15 14:04:18 -05:00
Ravi Khadiwala 61b3cecd17 Fix missing increment on recaptcha counter 2022-09-14 17:07:26 -05:00
Ravi Khadiwala a4a666bb80 Add metrics for recaptcha reasons 2022-09-14 16:00:11 -05:00
Ravi Khadiwala c14621a09f Add metrics for captcha scores 2022-09-14 16:00:11 -05:00
Ravi Khadiwala d0a8899daf Change discriminator seperator and default width 2022-09-14 15:53:15 -05:00
Chris Eager e25252dc69 Remove unused exception 2022-09-12 11:19:15 -05:00
Chris Eager 8b65c11e1e Update batch check entities from two optional fields to a single field 2022-09-12 11:19:01 -05:00
Chris Eager 320c5eac53 Add support for PNIs at `v1/profile/identity_check/batch` 2022-09-09 10:55:34 -05:00
Ehren Kret 8199e0d2d5 Set resource field on log entry 2022-09-07 19:37:26 -05:00
Ehren Kret 7d171a79d7 Remove redundant @NotNull annotation 2022-09-07 19:37:26 -05:00
Ehren Kret 3b99bb9e78 Log remote config delete events 2022-09-07 19:37:26 -05:00
Ehren Kret abd0f9630c Create GCP Logging implementation of AdminEventLogger 2022-09-07 19:37:26 -05:00
Ehren Kret a4508ec84f Add new event logging module 2022-09-07 19:37:26 -05:00
Ravi Khadiwala 4032ddd4fd Add reserve/confirm for usernames 2022-09-07 11:49:49 -05:00
Chris Eager 4c677ec2da Remove deprecated /v1/attachments 2022-09-07 11:48:16 -05:00
Chris Eager c05692e417 Update deprecated `CircuitBreakerConfig` usage 2022-09-07 11:47:15 -05:00
gram-signal ae1edf3c5c
Remove experiment associated with auth1->auth2 rollout. 2022-08-31 12:10:46 -06:00
gram-signal b17f41c3e8
Check if dashes work in dynamic configuration keys. 2022-08-29 15:51:37 -06:00
gram-signal 08db4ba54b
Update authentication to use HKDF_SHA256. 2022-08-29 14:20:47 -06:00
gram-signal cb6cc39679
Ignore null identity key. 2022-08-29 13:26:49 -06:00
Jon Chambers b6bf6c994c Remove a spurious `@Nullable` annotation 2022-08-26 15:22:23 -04:00
Jon Chambers 3bb4709563 Add CLDR region as a dimension 2022-08-26 12:41:51 -04:00
Jon Chambers b280c768a4 Allow signup captchas to target CLDR two-letter region codes 2022-08-26 12:41:51 -04:00
Chris Eager 616513edaf Remove unused jdbi dependency 2022-08-25 13:40:46 -07:00
Chris Eager 38b77bb550 Update libphonenumber to 8.12.54 2022-08-25 13:40:32 -07:00
Jon Chambers e72d1d0b6f Stop reading attribute-based messages from the messages table 2022-08-22 13:37:39 -07:00
Ravi Khadiwala 1891622e69 Zero-pad discriminators less than initial width 2022-08-22 13:36:38 -07:00
Chris Eager 628a112b38 Include country code for verify failure 2022-08-19 12:21:05 -07:00
Jon Chambers 50f5d760c9 Use existing tagging tools for keepalive counters 2022-08-16 13:16:19 -07:00
Jon Chambers 7292a88ea3 Record table performance metrics around reported messages 2022-08-16 13:15:30 -07:00
Jon Chambers 07cb3ab576 Add a "sealed sender" dimension to the sent message counter 2022-08-16 13:11:12 -07:00
Chris Eager 27b749abbd Filter expired items from Dynamo 2022-08-16 13:09:47 -07:00
Chris Eager 27f67a077c Add metrics for report-verification-succeeded response 2022-08-16 13:08:16 -07:00
Ravi Khadiwala 393e15815b Rename secondary account key namespace for usernames 2022-08-15 10:51:52 -05:00
Ravi Khadiwala a7f1cd25b9 Remove UAK normalization code
All accounts now have UAKs in top-level attributes
2022-08-15 10:47:52 -05:00
Ravi Khadiwala 953cd2ae0c Revert "Delete any leftover usernames in the accounts db"
This reverts commit a44c18e9b7.

Old username cleanup is finished.
2022-08-15 10:45:38 -05:00
ravi-signal a84a7dbc3d
Add support for generating discriminators
- adds `PUT accounts/username` endpoint
- adds `GET accounts/username/{username}` to lookup aci by username
- deletes `PUT accounts/username/{username}`, `GET profile/username/{username}`
- adds randomized discriminator generation
2022-08-15 10:44:36 -05:00
Chris Eager 24d01f1ab2 Revert "device capabilities: prevent `stories` downgrade"
This reverts commit 1c67233eb0.
2022-08-12 14:21:27 -05:00
Chris Eager 06eb890761 Improve e164 normalization check by re-parsing without country code 2022-08-12 10:52:55 -07:00
Chris Eager 6d0345d327 Clean up Util 2022-08-12 10:52:55 -07:00
Chris Eager 1c67233eb0 device capabilities: prevent `stories` downgrade 2022-08-12 10:51:16 -07:00
Jon Chambers b4281c5a70
Send non-urgent push notifications with lower priority 2022-08-12 11:06:31 -04:00
Jon Chambers 5f6b66dad6 Add support for scheduling background push notifications 2022-08-12 10:57:59 -04:00
Jon Chambers c2be0af9d9 Refactor `ApnPushNotificationSchedulerTest` to use a `Clock` 2022-08-12 10:57:59 -04:00
Jon Chambers a53a85d788
Refactor scheduled APNs notifications in preparation for future development 2022-08-12 10:47:49 -04:00
Ravi Khadiwala a44c18e9b7 Delete any leftover usernames in the accounts db
The account username field should not currently be populated
2022-08-11 16:23:51 -05:00
Jon Chambers 4d78437fe4 Add a country code dimension to the non-normalized number counter 2022-08-10 15:03:01 -04:00
Jon Chambers 2bfe2c8ff8 Add an "urgent" dimension to the "sent messages" counter 2022-08-10 15:00:46 -04:00
Chris Eager 65da844d70 Small test cleanup 2022-08-09 15:32:44 -05:00
Chris Eager 5275c27ee1 Fix incorrect test Javadoc 2022-08-09 13:06:15 -07:00
Chris Eager 390580a19d Count cases when the a message’s destination UUID doesn’t match the account’s PNI 2022-08-09 13:06:15 -07:00
Jon Chambers 147917454f Measure the depth of the queue for the FCM executor 2022-08-04 15:53:26 -04:00
Jon Chambers 39562775d9 Use a fixed-size thread pool for sending FCM notifications 2022-08-04 15:37:22 -04:00
Jon Chambers 4a0ef1f834 Measure the time taken to send APNs push notifications 2022-08-04 10:43:07 -04:00
Jon Chambers 85b16b674d Measure the time taken to send FCM push notifications 2022-08-04 10:43:07 -04:00
Jon Chambers ab5d8ba120 Use `ApiFutures#addCallback` for FCM futures 2022-08-04 10:43:07 -04:00
Jon Chambers 28076335e0 Generate APNs payloads using a payload builder 2022-08-04 10:37:30 -04:00
Jon Chambers 9e9333424f Retire `RetryingApnsClient` 2022-08-04 09:59:18 -04:00
Jon Chambers 6f0faae4ce
Introduce common push notification interfaces/pathways 2022-08-03 10:07:53 -04:00
Jon Chambers 0a6d724f2c Remove `GCMSender` 2022-08-02 17:31:35 -04:00
Jon Chambers 8956e1e0cf Check for null FCM error codes 2022-08-02 17:29:31 -04:00
Jon Chambers c9ae991aa3 Add an experiment to allow a phased transition from the old GCM API to the current FCM API 2022-08-02 15:34:09 -04:00
Jon Chambers 421d594507 Introduce an `FcmSender` 2022-08-02 15:34:09 -04:00
Jon Chambers 9c03f2e468
Add support for receiving, storing, and returning `urgent` flags on messages 2022-08-02 12:05:23 -04:00
Chris Eager 1175ff5867 Log cause when queue processing hits max retries 2022-08-02 08:36:16 -07:00
Chris Eager 55df593561 Clean up `MessageAvailabilityListener` if the websocket client is closed 2022-08-02 08:35:16 -07:00
Chris Eager a06a663b94 Use the envelope’s destination UUID for receipt source UUID 2022-08-02 08:34:20 -07:00
Chris Eager 3d2f7e731f Remove `Envelope.source` 2022-08-02 08:34:20 -07:00
Chris Eager 2575372639 Add missing `increment()` to displacement counter 2022-08-02 08:30:54 -07:00
Jon Chambers faa6e8324a Fix a test issue where we were ignoring some test parameters 2022-08-01 11:02:33 -04:00
Jon Chambers d0e3fb1901 Unconditionally write messages to the messages table as envelopes 2022-08-01 10:44:27 -04:00
Jon Chambers 04287c5073 Optionally write messages as envelopes to the messages table 2022-07-29 11:10:50 -04:00
Jon Chambers 0c76fdd36c Read bare envelopes from the messages table if possible 2022-07-29 11:10:50 -04:00
Jon Chambers 3636626e09 Make `Envelope` the main unit of currency when working with stored messages 2022-07-29 10:59:02 -04:00
Jon Chambers 3e0919106d Add a method to build an `OutgoingMessageEntity` from an `Envelope` 2022-07-29 10:59:02 -04:00
Jon Chambers d385838dc1 Add methods to convert `IncomingMessage`/`OutgoingMessageEntity` instances into `Envelope` entities 2022-07-29 10:59:02 -04:00
Jon Chambers e28f1e8ceb Remove the unused `destination` property from `IncomingMessage` 2022-07-29 10:59:02 -04:00
Jon Chambers 3d875f1ce5 Convert incoming/outgoing message entities to records 2022-07-29 10:59:02 -04:00
Jon Chambers c4c5397b44 Convert group credential redemption time to a `long` 2022-07-28 10:08:25 -04:00
Jon Chambers 6b6f9b2405 Add PNI to `GroupCredentials` responses that contain `AuthCredentialWithPni` instances 2022-07-28 10:08:25 -04:00
Jon Chambers 7d4a8d03a4 Mark old group credential getter as `@Deprecated` 2022-07-28 10:08:25 -04:00
Jon Chambers e9119da040 Retire the (unused!) binary message format 2022-07-28 09:59:00 -04:00
Chris Eager aa36dc95ef Add `MicrometerCommandLatencyRecorder` to Redis clusters 2022-07-27 14:23:14 -07:00
Chris Eager a6f9409a39 Remove dynamic configuration feature flags; add `DynamicMessagePersisterConfiguration` 2022-07-27 14:19:10 -07:00
Chris Eager 41a113e22c Stop queue persistence attempt if items aren’t removed from cache 2022-07-27 14:19:10 -07:00
Chris Eager 4cfcdb0c96 editorconfig formatting 2022-07-27 14:19:10 -07:00
Ravi Khadiwala 36050f580e Handle duplicate device ids more gracefully 2022-07-27 11:15:32 -05:00
Jon Chambers 98760b631b Don't wrap "change number" arguments in `Optional` 2022-07-27 10:55:37 -04:00
Jon Chambers d00aa1e77a Fix an inconsistent check for the presence/absence of "change number" arguments 2022-07-27 10:55:37 -04:00
Jon Chambers dce391a248
Add support for setting PNI-associated registration IDs and identity keys when changing numbers 2022-07-26 15:19:27 -04:00
Antonin Tritz c252118cfc External links in comments from http to https 2022-07-26 15:17:58 -04:00
Chris Eager e9fd32de79 Only attempt to unsubscribe if the queue name is not null 2022-07-26 10:20:10 -07:00
Chris Eager bc02fe3831 Only unsubscribe from keyspace notifications if the node has the slot 2022-07-26 09:21:50 -07:00
Jon Chambers d290aad27b Associate source/destination country codes for reported messages 2022-07-14 09:31:12 -04:00
Jon Chambers 6754ec5e10 Pass disconnection reason (if known) to clients 2022-07-13 15:30:03 -04:00
Jon Chambers 1ba00a66eb Pass the reason for displacement to presence displacement listeners 2022-07-13 15:30:03 -04:00
Jon Chambers 1dd7d33e23
Simplify `Device` entity 2022-07-13 13:55:20 -04:00
Jon Chambers e200548e35 Introduce an account change validator 2022-07-13 12:24:39 -04:00
Jon Chambers fdf7b69996 Remove a temporary workaround for incorrect envelope types from iOS clients 2022-07-13 11:30:52 -04:00
Jon Chambers 92d36b725f Allow presence keys to expire if not periodically renewed 2022-07-13 11:28:55 -04:00
Jon Chambers 4e131858ca Generalize scope of and expand size of websocket scheduled executor service 2022-07-13 11:28:55 -04:00
Ravi Khadiwala a45d95905e Be permissive in account-create accept-language
Currently, if we fail to parse a user's accept-language in account
creation, creation will fail. While it's a suboptimal experience to get
a verify code in the wrong language, it might be better than not being
able to sign up at all.
2022-07-13 11:22:31 -04:00
Ehren Kret 0fdfdabf2a merge GroupController into CertificateController 2022-07-01 13:04:24 -05:00
Jon Chambers a25e967978 Remove spurious mocking 2022-07-01 12:26:22 -05:00
Jon Chambers 38e30c7513 Allow callers to get an expiring profile key credential 2022-07-01 12:26:22 -05:00
Jon Chambers e38e5fa17d Allow callers to request a combined group auth credential 2022-07-01 12:26:22 -05:00
Ehren Kret 42fd29d38b Update badge description text to remove boost & sustainer language 2022-06-21 13:36:12 -05:00
Ehren Kret bf6d3aa324 Fix batching issue that was causing duplication in large queries 2022-06-20 12:41:10 -05:00
Ehren Kret 023ccc6563 Add a unit test for the batch identity check endpoint 2022-06-20 12:32:31 -05:00
Ehren Kret da49db5b9e Move batch identity checks off the common fork join pool 2022-06-20 11:07:33 -05:00
Ehren Kret cc8dda28cc Simplify logic for batching bulk identity check request 2022-06-20 10:28:20 -05:00
Ehren Kret d31550d444 Only wrap Base64 decode with the exception catch 2022-06-20 09:42:33 -05:00
Ehren Kret 51f37350eb Because one shouldn't take the size of null things 2022-06-17 15:03:25 -05:00
Ehren Kret ecfa161da8 Validate the request 2022-06-17 14:32:38 -05:00
Ehren Kret e3778c17ea Use POST not GET for request requiring body 2022-06-17 14:31:15 -05:00
Ehren Kret cbc95415b7 Add endpoint to batch check identity keys 2022-06-17 12:20:30 -05:00
Ravi Khadiwala 327eb0219d Bypass account registration captcha on test devices 2022-06-13 15:01:46 -07:00
gram-signal a853748303
Revert "Update ChangeNumber to allow reset of registration IDs."
This reverts commit 7001ad1445.
2022-06-09 11:51:50 -06:00
gram-signal 7001ad1445
Update ChangeNumber to allow reset of registration IDs. 2022-06-02 16:37:32 -06:00
Ravi Khadiwala 5cfb133f79 Use redis for abusive hosts autoblock
Also delete postgres dependencies that we no longer need
2022-05-31 10:08:10 -05:00
Ehren Kret 5df24edebf Remove all unused imports remaining in project 2022-05-25 17:15:20 -05:00
Ehren Kret 95d0293a96 Remove unused counter 2022-05-25 16:57:01 -05:00
Ehren Kret f5a2efb57c Clarify hidden static method call 2022-05-25 16:51:12 -05:00
Ehren Kret e4b9ae4eee Clear up warnings 2022-05-25 16:50:58 -05:00
Ehren Kret bc1ac5a37f Remove unused fields 2022-05-25 16:50:43 -05:00
Ehren Kret 96ac56faac Remove unused import 2022-05-25 16:38:11 -05:00
Ehren Kret f0bc444388 Remove unused timer 2022-05-25 16:37:51 -05:00
Ravi Khadiwala 8584f47d95 Add more metrics to createAccount captcha 2022-05-25 15:50:11 -04:00
Jon Chambers f6235b8c08 Check for newly-expired accounts before previously-expired accounts for metric accuracy 2022-05-25 15:49:54 -04:00
Jon Chambers d452e90470 Move `AccountCleaner` into its own crawler 2022-05-25 15:46:18 -04:00
Jon Chambers 418a869451 Increase max deletions per chunk to 256 2022-05-25 14:19:04 -04:00
Jon Chambers cf89e2215c Fully delete already-expired accounts 2022-05-25 14:19:04 -04:00
Jon Chambers a4ca1ef1a8 Move `AccountCleanerTest` out of the `test` package; reduce visibility of test-only fields 2022-05-25 14:19:04 -04:00
Ehren Kret c38572307d Remove unused imports 2022-05-20 10:20:45 -07:00
Fedor Indutny 20902df122
Fix ZRANGE arguments in get_items.lua 2022-05-19 12:14:02 -07:00
Fedor Indutny d31ddb72f3
Optimize message deletion by skipping lookup 2022-05-18 13:02:21 -07:00
Jon Chambers 75c92eaa93 Drop high-cardinality distribution summaries 2022-05-11 14:26:38 -04:00
Chris Eager 0445adcac3 Conclude `ReportMessageManager` ACI migration 2022-05-11 11:26:24 -07:00
gram-signal 06dd4c5026
Derive username from ACI for CDS{H,I} (#989)
* Derive username from ACI for CDS{H,I}

* Update sample YAML.
2022-05-02 08:41:38 -06:00
Ehren Kret 058caadf4f Use BigDecimal#compareTo for numeric equality testing 2022-04-29 14:20:09 -05:00
Ehren Kret 7b7d309105 Apply stripe conversion factor to gift badge amount check 2022-04-29 13:48:10 -05:00
Ehren Kret 63be7b93ce Record level on boost payment intent 2022-04-29 12:06:15 -05:00
Ehren Kret 578ea12b59 Add gift badges to user capabilities 2022-04-29 11:08:36 -05:00
Ehren Kret 364e59be57 Add shape to duration to ensure it's render as an integer 2022-04-29 10:14:39 -05:00
Ehren Kret fece4dac9e Add duration to boost badges
Lets clients know how long the badge will last for after purchase.
2022-04-29 10:14:39 -05:00
Ehren Kret ce85c1aabc Add name and description for gift badge 2022-04-29 10:14:39 -05:00
Ehren Kret 0ac2ce5e72 Add gift badge to the level output for boost badges 2022-04-29 10:14:39 -05:00
Ehren Kret 391c800bf5 Add gift configuration to subscription controller 2022-04-29 10:14:39 -05:00
Ehren Kret f6471cf8f9 Add gift configuration source 2022-04-29 10:14:39 -05:00
Jon Chambers 1eaff753a6 Count "forbidden identity key change" events 2022-04-22 15:53:43 -04:00
Ehren Kret c29d5de1eb Refactor two more switch statements to new switch style 2022-04-19 13:39:38 -05:00
Ehren Kret d94c171d63 Use new style switch statement 2022-04-19 13:35:26 -05:00
Ehren Kret 53203dbcef Refactor common pattern for checking Account capabilities 2022-04-19 10:33:54 -05:00
Ehren Kret 9e66f8ac11 Add gift badges device capability 2022-04-19 10:33:54 -05:00
Ehren Kret 796fb3b4cd Refactor Device#equals method 2022-04-19 10:33:54 -05:00
gram-signal 473ecbdf2d
Allow primary to set and provide new signed prekeys for linked devices (#950) 2022-04-15 12:39:47 -06:00
Chris Eager 7b3703506b Remove number from `ReportMessageManager#store` 2022-04-13 16:41:32 -04:00
Jon Chambers 5816f76bbe Add support for getting (limited) profiles by phone number identifier 2022-04-13 16:27:57 -04:00
Ehren Kret 355996bafc Add outcome type to subscription information endpoint 2022-04-08 12:31:31 -05:00
Ehren Kret c2bb46f41d Add outcome network status and outcome reason to subscription info 2022-04-08 12:31:31 -05:00
Ehren Kret 12f76c24b1 Add failure information for the latest charge on a subscription 2022-04-08 12:31:31 -05:00
Jon Chambers 3dc0d0bb92 Revert "Revert "Replace curve25519-java with libsignal-server""
This reverts commit c06a5ac96c.
2022-03-29 12:24:45 -04:00
Jon Chambers 2bb8f92af1 Revert "Revert "Replace zkgroup with libsignal-server""
This reverts commit fa3a9570d6.
2022-03-29 12:24:45 -04:00
Jon Chambers 5b7d5d2b93 Reduce "unrecoverable error" messages when sending GCM notifications from WARN to DEBUG 2022-03-25 14:34:44 -07:00
Jon Chambers 2b27db18d8 Count GCM failures by error code 2022-03-25 14:34:44 -07:00
Jon Chambers f3c811cc03 Move "no local subscription" warnings from WARN to DEBUG 2022-03-25 14:34:44 -07:00
Chris Eager 77fd01bd9f Accept source ACI at `/v1/messages/report` 2022-03-25 14:27:09 -07:00
Jon Chambers fa3a9570d6 Revert "Replace zkgroup with libsignal-server"
This reverts commit 86a09b16ff.
2022-03-24 12:11:46 -04:00
Jon Chambers c06a5ac96c Revert "Replace curve25519-java with libsignal-server"
This reverts commit 06a57ef811.
2022-03-24 12:11:46 -04:00
Jon Chambers 33467b42da Remove a deprecated/unused field 2022-03-24 10:50:49 -04:00
Jon Chambers 13fb641113 Make field name casing consistent 2022-03-24 10:50:49 -04:00
Jon Chambers 53f17c2baa Drop the legacy message and relay fields from message entities 2022-03-24 10:50:49 -04:00
Jordan Rose 06a57ef811 Replace curve25519-java with libsignal-server
These APIs stemmed from a common source long ago, so there's not much
to change!
2022-03-24 10:50:18 -04:00
Jordan Rose 86a09b16ff Replace zkgroup with libsignal-server 2022-03-24 10:50:18 -04:00
Ravi Khadiwala c70d7535b9 Make TURN configuration dynamic
Also enables conditionally including more TURN servers for gradual
rollouts
2022-03-23 14:38:02 -05:00
Jon Chambers 2a832d36d7 Remove AcceptNumericOnlineFlagRequestFilter 2022-03-23 14:31:01 -05:00
gram-signal 1578c89475
Only allow primary device to update identity key. 2022-03-22 14:39:04 -06:00
Jon Chambers bab6b36e4d Count reported messages by destination country code in addition to source country code 2022-03-22 09:56:27 -04:00
Jon Chambers f75e616397 Introduce a listener pattern for reported messages 2022-03-22 09:56:27 -04:00
Jon Chambers 7ba0f604e6 Tag the push challenge request counter by country 2022-03-17 16:07:48 -04:00
Chris Eager cf8a4cc939 Decrease receipt sender executor thread pool 2022-03-17 13:07:03 -07:00
Jon Chambers 2f6b0b1a55 Tag push challenge attempt metrics by country code 2022-03-16 15:00:04 -04:00
Jon Chambers c048074c31 Tag captcha attempt metrics with UA platform 2022-03-16 15:00:04 -04:00
Ravi Khadiwala 5a88ff0811 Use the async dynamo client to batch uak updates 2022-03-14 14:02:16 -05:00
Chris Eager de68c251f8 Instrument the receipt sender executor 2022-03-11 17:20:52 -08:00
Chris Eager 7c9ae3561d Send delivery receipts asynchronously 2022-03-11 16:34:22 -08:00
Chris Eager b608ece57e Remove `supportsAnnouncementGroups` metric 2022-03-11 16:18:00 -08:00
Chris Eager 8dfffebaf1 Remove unnecessary check for destination UUID 2022-03-11 16:17:54 -08:00
Chris Eager fef37f739b Remove unused classes and methods 2022-03-10 11:08:38 -05:00
Ravi Khadiwala 1571f14815 Add a feature flag to disable account normalization 2022-03-09 14:03:21 -06:00
Ravi Khadiwala 9cb098ad8a Add a top-level uak to existing items
Items wirtten before we started storing the uak at
the top level only store the uak in the
account blob. The will be updated on account
crawl
2022-03-09 14:03:21 -06:00
Jon Chambers 9b9edbae0e Drop `DeadLetterHandler` (which is functionally unused) 2022-03-08 10:03:06 -05:00
Chris Eager 491155d1cf Remove `@Consumes` from GET/DELETE in `/v1/subscriptions` 2022-03-04 15:36:02 -08:00
Chris Eager 54207254f1 Remove unused configuration 2022-03-04 11:44:17 -08:00
Chris Eager 1395dcc0be Make the enterprise client canonical 2022-03-04 11:44:17 -08:00
Chris Eager 2a68d9095d Remove transitional and legacy client 2022-03-04 11:44:17 -08:00
Chris Eager a984b3640e Further refine score distribution summary 2022-03-04 11:44:17 -08:00
Jon Chambers f6cbc32ee7 Align "link device" and "fetch pre-keys" rate limits 2022-03-04 10:45:06 -05:00
Chris Eager 602614acf6 Refine assessment metrics 2022-03-03 16:09:49 -08:00
Chris Eager 3854b7d472 Remove `@Consumes` from `GET /v1/config` 2022-03-03 16:08:55 -08:00
Jon Chambers 5e25481088 Fix a trivial typo in a constant's name 2022-03-03 10:59:19 -05:00
Jon Chambers fe86e15d80 Remove PNI repair code 2022-03-03 10:25:11 -05:00
Jon Chambers 179b4a69eb Clear one-time pre-keys when PNIs are assigned 2022-03-03 10:24:54 -05:00
Chris Eager eee6307789
Move score floor to dynamic configuration, add distribution summary 2022-03-02 15:18:33 -08:00
Chris Eager 9fc5002619
Add stories capability 2022-03-02 15:16:21 -08:00
Ravi Khadiwala faa6ae284a Add uak as a top level attribute 2022-03-02 10:41:09 -06:00
Jon Chambers 8b4355b21d Add a "challenge issued" counter tagged by country and platform 2022-03-02 10:40:47 -06:00
Jon Chambers e8835da740 Tag captcha success rate counter by country 2022-03-02 10:40:47 -06:00
Dimitris Apostolou 93d06e3f4d Fix typos 2022-03-01 14:14:52 -08:00
Chris Eager 935e268dec Parameterize sitekey 2022-03-01 10:40:42 -08:00
Jon Chambers 3a1c716c73 Remove an unused rate limiter 2022-02-25 13:50:17 -08:00
Chris Eager f3457502a6 Support different v2 captcha actions 2022-02-25 13:49:47 -08:00
Ravi Khadiwala d3cd1d1b15 Use GetLatestConfiguration in config manager
Use StartConfigurationSession/GetLatestConfiguration instead of
GetConfiguration since the latter has been deprecated
2022-02-23 15:36:33 -06:00
Ravi Khadiwala f5a75c6319 Simplify RateLimitExceeded with no retry-duration
- Avoid passing negative durations in error cases
- Drop unused message
- Return a duration for a bad forwarded-for
2022-02-23 15:25:24 -06:00
Ravi Khadiwala ae3a5c5f5e Return a Retry-After on rate-limited responses
Previously, only endpoints throwing a RetryLaterException would include
a Retry-After header in the 413 response. Now, by default, all
RateLimitExceededExceptions will be marshalled into a 413 with a
Retry-After included if possible.
2022-02-23 15:25:24 -06:00
Jordan Rose c367a71223 APNS: include a collapse-id for non-VOIP notifications
This has two benefits:

- The APNS server should only send an iOS client a single push
  notification for any missed messages while the device is offline
  (server-side coalescing). Note that the client can still turn that
  into multiple "user notifications" as it pulls from its queue.

- If multiple notifications get delivered but iOS is unable to process
  them (say, because the phone just restarted and hasn't been unlocked
  yet), the user should only get one "You may have received messages"
  notification (client-side coalescing).
2022-02-18 10:41:10 -05:00
Chris Eager d259ef0348 Update rate limit exceeded counter 2022-02-17 13:23:48 -08:00
Chris Eager 288cbf4a80 Clean up null-ability of incoming message entity fields 2022-02-17 13:23:48 -08:00
Chris Eager ba5e5a780f Throw an exception instead of using Optional<Resposne> 2022-02-17 13:23:48 -08:00
Chris Eager 73fa3c3fe4 Add test for content length validation 2022-02-17 13:23:48 -08:00
Chris Eager 579eb85175 Reject invalid envelope types 2022-02-17 13:23:48 -08:00
Chris Eager b2b20072ae Add `MicrometerRegistryManager` 2022-02-17 13:18:30 -08:00
Chris Eager a2c4d3fe95 Use a strong reference to the application shutdown gauge 2022-02-17 13:18:30 -08:00
Ravi Khadiwala 31e2be2e4d Fixup invalid accept-language counter
- Fix name
- Add platform/version tags to the counter
2022-02-10 11:57:25 -06:00
Ravi Khadiwala 9f5d97e1c6 Silence noisy warnings for invalid Accept-Language 2022-02-10 11:57:25 -06:00
Ravi Khadiwala baaae6cd9f Add `@NotNull` to controller args where appropriate
Notably, `@Valid` doesn't imply `@NotNull`
2022-02-10 11:57:04 -06:00
Chris Eager ed398aa7b9 Add `DeviceCapabilities.pni` 2022-02-09 15:56:01 -08:00
Chris Eager 6e2ae42dab Add platform to metrics 2022-02-09 15:55:07 -08:00
Jordan Rose 41bf2b2c42
Add a binary format for incoming messages
The existing, general incoming message endpoint accepts messages as
JSON strings containing base64 data, along with all the metadata as
other JSON keys. That's not very efficient, and we don't make use of
that full generality anyway. This commit introduces a new binary
format that supports everything we're using from the old format (with
the help of some query parameters like multi-recipient messages).
2022-02-07 16:05:03 -08:00
Chris Eager 639d634426
Restore displaced UUID from deleted accounts table when present 2022-01-31 13:01:36 -08:00
Chris Eager 5358fc4f43 Use `setRegistrationLockFromAttributes` 2022-01-31 12:56:59 -08:00
Jon Chambers 99eda80a78 Measure rate limit challenge responses by platform 2022-01-27 16:05:52 -05:00
Jon Chambers cbbdea1ba4 Impose more stringent requirements on cardinality checks 2022-01-26 16:13:12 -05:00
Jon Chambers 05e7c98620 Return an `AccountIdentityResponse` when changing phone numbers 2022-01-26 14:44:53 -05:00
Jon Chambers 1f1d618dea Rename `AccountCreationResult` to `AccountIdentityResponse` (since it's not just for account creation any more) 2022-01-26 14:44:53 -05:00
Jon Chambers b18117ef89 Add tests for unidentified access when requesting profiles 2022-01-25 14:07:44 -05:00
Jon Chambers 44cb796574 Add more detail to "could not get acceptable languages" logging 2022-01-25 12:21:35 -05:00
Jon Chambers f69db11f42 Drop some unused dynamic configuration properties 2022-01-21 18:46:37 -05:00
Chris Eager 96a680dcf0 Remove displaced account from directory when changing numbers 2022-01-21 18:46:11 -05:00
Jon Chambers c612663490
Handle `null` `AccountAttributes` when verifying linked devices 2022-01-14 14:47:46 -05:00
Jon Chambers de5d967d18 Track metrics for dynamic config load failures 2022-01-14 14:47:12 -05:00
Jon Chambers 7fc63f7847 Allow callers to specify one or more dynamic config classes 2022-01-14 14:45:29 -05:00
Jordan Rose 146655e997 Add a "sameAvatar" flag to CreateProfileRequest
If sameAvatar is set (and "avatar" is also set), the avatar field for
a profile will be copied from the existing profile. This saves S3
churn and client bandwidth.
2022-01-04 13:57:41 -08:00
Chris Eager bb27dd0c3b Migrate from `Object[]` parameters to `Stream<Arguments>` 2022-01-03 15:10:02 -08:00
Chris Eager d7a3c12bbe Drop unused DynamoDB class rules 2022-01-03 15:10:02 -08:00
Chris Eager a1e84f5a88 Migrate service tests to JUnit 5 2022-01-03 15:10:02 -08:00
Chris Eager b758737907
Migrate remaining JUnit 4 Redis cluster tests to `RedisClusterExtension`
* Increase redis cluster initialization wait to 10 seconds
* Move to JUnit 5 `Assumptions`
2022-01-03 14:59:39 -08:00
Chris Eager 3e777df86c Timeout `sendNextMessagePage` after 5 minutes 2021-12-20 11:31:11 -08:00
Chris Eager 278b4e810d Add (failing) test for send message timeouts 2021-12-20 11:31:11 -08:00
Chris Eager 346c7cd743 Remove null-check of destination UUID 2021-12-20 11:31:11 -08:00
Jon Chambers 5baa51d547 Migrate challenge-issuing configuration into the abusive message filter module 2021-12-16 12:22:19 -05:00
Jon Chambers 616db337e1 Remove an old rate limiting feature flag 2021-12-16 12:22:19 -05:00
Jon Chambers 3895871462 Repair missing PNIs in JSON blobs on account load 2021-12-16 11:17:51 -05:00
Jon Chambers a87b84fbe2 Return an empty response if somebody requests a profile key credential with a non-existent version 2021-12-16 10:30:55 -05:00
Jon Chambers 20c95e2606 Register `ApplicationShutdownMonitor` last
This will make it start last and shut down first, which is pretty much what we want for shutdown state monitoring
2021-12-15 19:27:23 -05:00
Jon Chambers 22dccaeddb Count cases where we can find a given account, but not the given profile version 2021-12-15 15:53:47 -05:00
Jon Chambers e611a70ba4 Remove usernames from profile responses 2021-12-15 15:53:47 -05:00
Jon Chambers 66845d7080 Refactor: separate the various types of profile responses 2021-12-15 15:53:47 -05:00
Jon Chambers 4ea7278c6f Remove unversioned profile properties from `Account` entities 2021-12-15 15:53:47 -05:00
Jon Chambers 2b2e26f14b Remove deprecated, unversioned profile setters 2021-12-15 15:53:47 -05:00
Jon Chambers b496ef8d6f Fix an issue where the deleted accounts lock client was trying to talk to the deleted accounts table 2021-12-15 13:16:32 -05:00
Jon Chambers 2d1ca98605 Consolidate DynamoDB clients/configuration 2021-12-15 12:42:41 -05:00
Jon Chambers eaa4c318e3 Add usernames to `whoami` and account creation responses 2021-12-15 11:47:10 -05:00
Jon Chambers 31373fd1ba Add a command for assigning usernames to individual users 2021-12-15 11:46:33 -05:00
Chris Eager 0ce87153e5 Remove second database for `AbusiveHostRules` 2021-12-08 16:03:02 -08:00
Chris Eager dba1711e8d Convert `AbusiveHostRule` to a record 2021-12-08 14:43:21 -08:00
Chris Eager a70b057e1c
Add second (migration) database to `AbusiveHostRules` 2021-12-08 12:46:05 -08:00
Ehren Kret 3df143dd3d Switch to zonky test embedded postgres and use postgres 11.13.0 2021-12-03 16:07:07 -05:00
Jon Chambers d78d7c726e Fix a UUID transposition when requesting profile key credentials 2021-12-03 16:06:26 -05:00
Jon Chambers d0ad580c7d Add (failing!) tests for getting a profile key credentials for somebody else 2021-12-03 16:06:26 -05:00
Ehren Kret 4a8a2a70b5 Return 400 instead of 500 when amount is too small 2021-12-03 12:24:16 -06:00
Jon Chambers 20a71b7df2 Add tests for generating profile key credentials 2021-12-03 12:16:13 -05:00
Jon Chambers 68412b3901 Allow the "get profile" endpoint to include a PNI credential 2021-12-03 12:16:13 -05:00
Chris Eager 31e5058b15 Make temporary iOS envelope type adaptation a little wider 2021-12-03 12:06:31 -05:00
Jon Chambers 14cff958e9 Migrate challenge-issuing rate limiters to the abusive message filter 2021-12-03 11:52:58 -05:00
Jon Chambers 9628f147f1 Separate statically- and dynamically-configured rate limiters 2021-12-03 11:01:15 -05:00
Chris Eager 13e346d4eb
Distinguish local vs remote in `ClientPresenceManager#disconnectPresence` 2021-12-02 14:32:42 -08:00
Fedor Indutny e507ce2f26
Include ACI and PNI in DeviceResponse 2021-12-02 17:21:05 -05:00
Jon Chambers 9c62622733 Handle cases where a message might be missing a destination UUID 2021-12-02 14:06:49 -05:00
Jon Chambers 62aa0cef39 Set destination UUIDs for receipts 2021-12-02 14:06:49 -05:00
Jon Chambers 401953313a Remove all vestiges of the relational account database 2021-12-02 12:46:43 -05:00
Chris Eager 4d2403d619 Use `assertTimeoutPreemptively` instead of `assertTimeout` 2021-12-02 12:17:08 -05:00
Jon Chambers c5f261305d Store destination UUIDs when persisting messages to DynamoDB 2021-12-02 12:17:08 -05:00
Jon Chambers 394f58f6cc Add a (failing!) check for destination UUIDs 2021-12-02 12:17:08 -05:00
Jon Chambers 674bf1b0e0 Drop a myserious empty test 2021-12-02 12:17:08 -05:00
Jon Chambers 606ddd8a9b Populate destination UUID for messages delivered via websocket 2021-12-02 12:17:08 -05:00
Jon Chambers e23a1fac50 Remove old Postgres-backed remote config machinery 2021-12-02 12:16:43 -05:00
Jon Chambers 342323a7e6 Use canonical usernames throughout `AccountsManager` 2021-12-02 12:02:29 -05:00
Jon Chambers efb410444b Introduce a username validator 2021-12-02 12:02:29 -05:00
Jon Chambers 17c9b4c5d3 Transition from Postgres-backed remote config store to Dynamo-backed store 2021-12-02 11:13:07 -05:00
Jon Chambers 706de8e2f1 Add a command to migrate remote configuration entries from Postgres to DynamoDB 2021-12-02 11:13:07 -05:00
Jon Chambers 23bc11f3b6 Introduce a DynamoDB-backed remote config store 2021-12-02 11:13:07 -05:00
Jon Chambers 4eb7dde1c8 Migrate RemoteConfigsTest to Junit 5 2021-12-02 11:13:07 -05:00
Jon Chambers 064861b930 Consolidate profiles store tests and discard `ProfilesStore` interface 2021-12-02 10:49:54 -05:00
Jon Chambers afa910bbd7 Drop relational profiles store 2021-12-02 10:49:54 -05:00
Jon Chambers 6aceb24fd2 Drop profile migration tools 2021-12-02 10:49:54 -05:00
Jon Chambers d94e86781f
Migrate username storage from a relational database to DynamoDB 2021-12-01 16:50:18 -05:00
Jon Chambers acfcb18f29 Allow overwriting of previously-delted profiles 2021-12-01 11:59:18 -05:00
Jon Chambers f7ff8e3837 Add a (failing!) test for deleting, then resetting profiles 2021-12-01 11:59:18 -05:00
Jon Chambers 048e17c62b Use a memoizing supplier instead of a looping thread to cache remote config entries 2021-11-30 16:35:42 -07:00
Jon Chambers d89b4f7e95 Compare versioned profiles synchronously; log a subset of mismatches for further investigation 2021-11-30 16:35:29 -07:00
Chris Eager 795b226b90 Mark methods that update `SignedPreKey`s as `@ChangesDeviceEnabledState` 2021-11-30 10:40:12 -07:00
Jon Chambers e485c380e0 Change the name of the CSV file argument to avoid upstream naming collisions 2021-11-30 11:31:12 -05:00
Jon Chambers bb4f4bc441 Modify the "migrate profiles" command to accept a list of UUIDs/versions from a CSV file 2021-11-30 11:22:51 -05:00
Jon Chambers 65b49b2d9c
Use a "for each" strategy in profile migration methods 2021-11-24 16:54:30 -05:00
Jon Chambers 9e7010f185
Migrate profiles from a relational database to DynamoDB 2021-11-24 14:48:41 -05:00
Jon Chambers 3bb8e5bb00 Set a TTL for `Account` entries in the general cache 2021-11-24 14:47:36 -05:00
Jon Chambers 2a4d1da2ca Delete accounts from Redis after they've been deleted from Dynamo 2021-11-24 13:47:53 -05:00
Jon Chambers 6b71b66bd2 Take no action if badge list is omitted entirely 2021-11-24 13:47:39 -05:00
Jon Chambers ebf24fb125 Add a (failing!) test for clients omitting badges in profile update requests 2021-11-24 13:47:39 -05:00
Jon Chambers 46d64b949e Don't read "soft-deleted" profiles
Nothing is actually "soft-deleting" profiles yet, and this
is a first step toward migrating profiles to a new data
store.
2021-11-24 12:02:07 -05:00
Jon Chambers 6919354520
Fix a counting bug with reported messages 2021-11-23 17:28:39 -05:00
Jon Chambers a42fe9bfb0 Add crawler names to log messages 2021-11-23 16:22:09 -05:00
Jon Chambers ee1f8b34ea Add a command for reserving usernames 2021-11-23 16:21:03 -05:00
Jon Chambers c910fa406d Migrate reserved usernames from a relational database to DynamoDB 2021-11-23 16:21:03 -05:00
Jon Chambers 559205e33f Log cases where accounts are missing or have inconsistent PNIs 2021-11-23 15:40:31 -05:00
Chris Eager c0756e9c60 Attempt an orderly websocket close on displacement before a hard disconnect 2021-11-23 11:36:32 -07:00
Ehren Kret bf1190696e Add badge workaround for old Android builds 2021-11-23 09:58:06 -06:00
Jon Chambers 71dd0890de Restore an accidentally-removed PNI consistency check 2021-11-23 10:53:40 -05:00
Jon Chambers e5acdf1402 Don't update the PNI attribute during general account updates 2021-11-23 10:53:40 -05:00
Jon Chambers 0f08b6bb59 Drop "got successful captcha" messages from `INFO` to `DEBUG` 2021-11-22 17:06:34 -05:00
Jon Chambers 6198a7b69a Remove spurious `@JsonProperty` annotations 2021-11-22 15:43:09 -05:00
Jon Chambers 067aee6664 Remove unused properties from `OutgoingMessageEntity` 2021-11-22 15:43:09 -05:00
Jon Chambers 138a2ebbd0 Drop transactional logic from phone number identifier migration 2021-11-22 15:32:24 -05:00
Jon Chambers 296f6a7a88 Make phone number identifiers non-optional 2021-11-22 15:32:24 -05:00
Jon Chambers 069ffa9921 Drop PNI migration tools 2021-11-22 15:32:24 -05:00
Jon Chambers f42fd8a840 Retire unused diagnostic metrics 2021-11-22 15:31:16 -05:00
Jon Chambers 10f27af6f2 Retire old unsealed-sender meters 2021-11-22 15:31:16 -05:00
Jon Chambers 0bbd34d060 Use text blocks where possible 2021-11-22 15:30:31 -05:00
Jon Chambers 282daeb0dc Add a command to assign PNIs to accounts that don't already have one 2021-11-22 15:03:19 -05:00
Jon Chambers d33b313c11 Break down legacy "get profile" requests by platform 2021-11-19 12:37:39 -05:00
Ehren Kret fb7316c9ae Return subscription status string in GET 2021-11-19 11:36:01 -06:00
Ehren Kret 279b0a51d9 Use latest invoice on subscription to generate receipts 2021-11-19 11:25:38 -06:00
Ehren Kret 6547d5ebf3 More consistent naming of receipt credentials endpoints 2021-11-19 10:14:00 -06:00
Ehren Kret 4f1ef9a039 Add additional http status codes to /v1/subscription/boost/receipt_credentials 2021-11-19 10:11:33 -06:00
Jon Chambers 077ead71a5 Rename legacy profile methods to separate them in metrics 2021-11-18 11:31:15 -05:00
Ehren Kret caba110266 Revert "Revert "Remove transparent SVG for badging""
This reverts commit 0fdb23c1e9.
2021-11-18 10:23:09 -06:00
Ehren Kret 0fdb23c1e9 Revert "Remove transparent SVG for badging"
This reverts commit 13a84f0c72.
2021-11-18 10:09:34 -06:00
Ehren Kret 13a84f0c72 Remove transparent SVG for badging 2021-11-18 10:04:42 -06:00
Jon Chambers 669bd58e33 Drop the unused `Key` utility class 2021-11-17 10:57:29 -05:00
Ehren Kret 7ea43a728d Set boost description from configuration 2021-11-16 17:21:57 -06:00
Chris Eager 71b38356b1 Update to Mockito 4.0.0 2021-11-16 15:56:35 -07:00
Chris Eager 5a99708f56 Update some deprecated usages 2021-11-16 15:56:13 -07:00
Jon Chambers 1dae05651f Add PNIs to account creation and `whoami` responses 2021-11-16 15:08:10 -05:00
Jon Chambers 5164e92538 Shorten metric names 2021-11-16 15:08:10 -05:00
Jon Chambers f89a20dbc7 Allow callers to set/retrieve keys by ACI or PNI 2021-11-16 15:08:10 -05:00
Jon Chambers 3a4c5a2bfb Store and retrieve one-time pre-keys by UUID 2021-11-16 15:08:10 -05:00
Jon Chambers 5e1334e8de s/KeysDynamoDb/Keys/ 2021-11-16 15:08:10 -05:00
Jon Chambers fa6e3d3690 Allow clients to request PNI-based group credentials 2021-11-16 15:08:10 -05:00
Jon Chambers 9383e7716b Resolve `CertificateControllerTest` warnings and recommendations 2021-11-16 15:08:10 -05:00
Jon Chambers cfe34fbf0f Allow unsealed-sender messages to be addressed by PNI 2021-11-16 15:08:10 -05:00
Jon Chambers 9fe110625c Add a `destinationUuid` field to envelopes 2021-11-16 15:08:10 -05:00
Jon Chambers 975f753c2b Add an endpoint for testing whether an account with a given ACI or PNI exists 2021-11-16 15:08:10 -05:00
Jon Chambers e6237480f8 Require that unidentified access keys be exactly 16 bytes 2021-11-16 15:08:10 -05:00
Chris Eager 26f876a2cb Check service configurations in verify phase 2021-11-16 11:43:07 -07:00
Jon Chambers ab9e6ac48a Revert "Replace zkgroup with libsignal-client"
This reverts commit 73ea6e4251.
2021-11-16 11:35:10 -05:00
Jon Chambers c1d6c04ab2 Revert "Replace curve25519-java with libsignal-client"
This reverts commit 0011b8925b.
2021-11-16 11:35:10 -05:00
Jon Chambers 888cec3d56 Introduce a filter for correcting numeric "online" flags 2021-11-16 10:15:14 -05:00
Jon Chambers 1461bcc2c2 Correct envelope types for certain iOS builds 2021-11-16 10:15:14 -05:00
Jon Chambers 11f1cf80bd Move `MessageControllerTest` out of the `tests` sub-package to expose package-private elements from the class under test 2021-11-16 10:15:14 -05:00
Jordan Rose c675cc8b26 Test the response code for invalid serialized zkgroup objects
Test by Jon, making sure this is consistent even without up-front size
checking.
2021-11-16 09:52:38 -05:00
Jordan Rose 0011b8925b Replace curve25519-java with libsignal-client
These APIs stemmed from a common source long ago, so there's not much
to change!
2021-11-16 09:52:38 -05:00
Jordan Rose 73ea6e4251 Replace zkgroup with libsignal-client 2021-11-16 09:52:38 -05:00
Jon Chambers e4441dddbb Consolidate Redis client resources 2021-11-16 09:52:12 -05:00
Jon Chambers 2015ba77ca Switch to a disallowed prefix model instead of a disallowed country code model 2021-11-15 15:44:55 -05:00
Jon Chambers 6ada76da7f Parallelize assignment of phone number identifiers 2021-11-12 11:03:46 -05:00
Jon Chambers cbdec0cb22 Remove legacy push latency measurement pathways 2021-11-11 15:44:07 -05:00
Chris Eager de6e9d31c9 Add dedicated crawler for directory reconciler 2021-11-11 13:38:13 -07:00
Jon Chambers f0a6be32fc Add a crawler to assign PNIs to existing accounts 2021-11-10 11:15:05 -05:00
Ehren Kret 5c4855cca6 Remove trailing space 2021-11-10 10:11:49 -06:00
Ehren Kret 2e1e380418 Also update description text of boost badge 2021-11-10 10:10:53 -06:00
Ehren Kret d07f0b4f71 Update badge description text 2021-11-10 09:51:51 -06:00
Jon Chambers aaa2a6eef1
Break down push latency metrics by VOIP/not-VOIP and optionally by client version 2021-11-10 10:35:41 -05:00
Jon Chambers b1f56c3324 Resolve formatting complaints 2021-11-10 10:15:14 -05:00
Jon Chambers da5c0ae4b6 Enable Payments Beta for more country codes 2021-11-10 10:15:14 -05:00
Jon Chambers 1e1394560d
Check length of cancellation reason list before getting reason codes 2021-11-09 11:42:44 -05:00
Jon Chambers bae0196bcf
Tolerate null UUID attribute values 2021-11-09 11:00:27 -05:00
Jon Chambers 3398955c1a
Add basic support for phone number identifiers 2021-11-09 10:23:08 -05:00
Chris Eager a1b925d1e0 Reduce visiblity on one constructor 2021-11-08 14:30:52 -07:00
Chris Eager 31c0c3275f Use the latest and in config, `@JsonCreator` 2021-11-08 14:30:52 -07:00
Chris Eager 0a4392f700 Streamline `ExternalServiceCredentialsGenerator` construction 2021-11-08 14:30:52 -07:00
Chris Eager eb86986cf4 Add /v2/directory/auth endpoint 2021-11-08 14:30:52 -07:00
Ehren Kret 1053a47e42 Add an exception mapper for CompletionExceptions 2021-11-04 19:12:31 -05:00
Ehren Kret 99b1f48e0e Copy badges from existing account on re-reg 2021-11-04 18:00:47 -05:00
Jon Chambers 6dddf54222 Consolidate rate-limit counters 2021-11-01 14:33:05 -04:00
Jon Chambers c931103712 Remove unused utility classes 2021-11-01 10:51:47 -04:00
Jon Chambers ad1aeea74b Add an abusive message filter interface and submodule 2021-11-01 10:51:47 -04:00
Ehren Kret ae7f8af03e Mark boost and subscription configuration as not null 2021-10-28 16:12:23 -07:00
Ehren Kret a52c91a665 Add names to subscription levels 2021-10-28 14:48:44 -07:00
Ehren Kret 94bf3a3902 Extract logic for created header controlled resource bundles 2021-10-28 14:26:53 -07:00
Ehren Kret f5a539e128 Add subscriptions resource bundle 2021-10-28 14:15:20 -07:00
Ehren Kret 24480b2090 Add endpoint to fetch information on boost badges 2021-10-26 14:05:59 -07:00
Ehren Kret a124b3abe9 Fix encoding of boost amounts 2021-10-26 08:08:09 -07:00
Ehren Kret 090d722b61 Add method to retrieve receipt credentials for a boost payment 2021-10-25 14:54:40 -07:00
Ehren Kret d27ec6fe8d Create boost create endpoint 2021-10-25 12:58:32 -07:00
Jon Chambers 8d34f3447b Drop an unused registration meter 2021-10-25 14:52:21 -04:00
Ehren Kret 72b52965b9 Expand definition of badge SVGs 2021-10-25 10:28:02 -07:00
Jon Chambers ae7077c643
Refresh accounts from storage when checking for device state changes after requests 2021-10-22 14:02:28 -04:00
Jon Chambers 11598e855f Count non-normalized or impossible numbers 2021-10-22 14:01:54 -04:00
Jon Chambers 534c577f59 Enforce phone number normalization when creating accounts or changing numbers 2021-10-22 14:01:54 -04:00
Jon Chambers 7762afc497 Add a method for verifying that numbers are normalized in addition to being dialable 2021-10-22 14:01:54 -04:00
Jon Chambers a3fe4b9980 Update/parameterize ValidNumberTest 2021-10-22 14:01:54 -04:00
Jon Chambers 598599cd14 Use a default `reportMessage` configuration if not specified 2021-10-21 15:34:57 -05:00
Ehren Kret 07cd69ab34 Add endpoint for fetching boost amounts 2021-10-21 13:56:35 -05:00
Jon Chambers 3b764bed7a Make `DynamicConfigurationManager` generic 2021-10-19 11:52:29 -04:00
Jon Chambers c91d5c2fdb Count reported messages per sender 2021-10-19 11:47:54 -04:00
Jon Chambers 40f7e6e994 Remove unused imports 2021-10-19 11:47:54 -04:00
Ehren Kret ee9aa9ce12 Round up by days on receipt expiration 2021-10-18 12:11:27 -05:00
Ehren Kret 08304bf375 Validate GetReceiptCredentialsRequest 2021-10-18 11:57:18 -05:00
Ehren Kret 8b8c6237be Use last subscription created at time as a subscription generation number 2021-10-14 12:06:19 -05:00
Ehren Kret c0837104cd Bring badge configuration into levels information 2021-10-14 11:35:18 -05:00
Jon Chambers fe21d014f7 Remove legacy rate-limiting tools 2021-10-14 11:43:18 -04:00
Ehren Kret 75c5032cd3 Add method to set default payment method 2021-10-14 10:37:12 -05:00
Jon Chambers f84e7aebd0 Count numbers that can't be normalized because another account has the normalized form of the number 2021-10-14 11:10:12 -04:00
Jon Chambers c379a3d297 Remove deprecated counters 2021-10-14 10:52:20 -04:00
Jon Chambers eedeaaecee Update rate-limiting for requests matching specific criteria 2021-10-14 10:42:16 -04:00
Ehren Kret 64eeb1e361 Move to using collections for sprites and svgs for badges 2021-10-13 23:25:39 -05:00
Ehren Kret e07597eba7 Add initial text for donation badges 2021-10-13 23:08:02 -05:00
Ehren Kret 5f2656710c Update badge configuration to new style 2021-10-13 22:58:57 -05:00
Ehren Kret 1af53f2612 Simplify getLevels API 2021-10-13 13:28:31 -05:00
Ehren Kret c89cfa4927 Move property setting above servlet construction 2021-10-13 13:17:07 -05:00
Ehren Kret bbde93a3c7 Enable unwrapping of CompletionStage 2021-10-13 11:46:20 -05:00
Ehren Kret b01b76d78f
First pass at subscriptions API
This is an incomplete first pass at building the subscriptions API. More API endpoints are still to be added along with controller tests.
2021-10-12 21:23:20 -05:00
Ehren Kret 75c22038eb Create empty Badges_en.properties file 2021-10-11 22:27:30 -05:00
Jon Chambers 3c1705994d Count accounts with non-normalized phone numbers 2021-10-11 10:13:08 -04:00
Chris Eager d2bc3c7360 Add dynamic configuration to disable directory reconciler 2021-10-04 14:38:47 -07:00
Jon Chambers 9734433f00 Use the default `SecureRandom` algorithm for tests 2021-10-04 11:37:32 -04:00
Ehren Kret 5bd08800bb Remove public modifier from test class 2021-10-01 14:27:07 -05:00
Ehren Kret 3032415141
Add receipt redemption API to chat server 2021-10-01 12:44:47 -05:00
Jon Chambers ba58a95a0f Add support for changing phone numbers 2021-10-01 10:15:33 -04:00
Jon Chambers aa4bd92fee Lazy-load scripts; fall back to `eval` if `evalsha` returns `NOSCRIPT` 2021-09-29 16:08:17 -04:00
Jon Chambers f37c76dab1 Drop `LuaScript` 2021-09-29 16:08:17 -04:00
Jon Chambers 863969c77c Resolve ClusterLuaScript warnings/suggestions 2021-09-29 16:08:17 -04:00
Jon Chambers 2383aaaa3d Update ClusterLuaScript formatting 2021-09-29 16:08:17 -04:00
Jon Chambers 715d1157ad Reject Redis commands when disconnected 2021-09-29 16:07:26 -04:00
Jon Chambers 4aaae3f445 Use a single configuration URI for Redis clusters 2021-09-29 14:46:09 -04:00
Jon Chambers 8359ef73f4 Cycle all connected websockets on any device or account enabled state change 2021-09-29 14:18:35 -04:00
Jon Chambers c6bb649adb Simplify map-building logic 2021-09-29 14:18:35 -04:00
Jon Chambers e333cbd94d Close websockets on account deletion 2021-09-29 14:11:16 -04:00
Chris Eager 5189cbe5c7 apply editorconfig formatting 2021-09-29 10:31:39 -07:00
Chris Eager d1d6e5c652 Filter stale ephemeral messages from cache 2021-09-29 10:31:39 -07:00
Chris Eager 3e5087e60b Remove obsolete ephemeral queue handling 2021-09-29 10:31:39 -07:00
Chris Eager 93c3cea912 Clean up old "ephemeral" metrics 2021-09-29 10:31:39 -07:00
Chris Eager e824b861d4 Drop `accounts` table 2021-09-29 10:31:20 -07:00
Chris Eager e8dd1e0bf2 Fixup formatting 2021-09-29 10:31:20 -07:00
Ehren Kret 559026933d Add low and high detail svgs to badges 2021-09-27 17:00:09 -05:00
Ehren Kret 7864405efd Remove single URL in favor of density based sprite sheets 2021-09-27 16:50:18 -05:00
Jon Chambers a5575902de Pause if we bump into an exception while trying to persist messages 2021-09-27 13:38:17 -04:00
Ehren Kret 041aed2d72 Remove zkgroup enabled flag
The last remnants of a time before zkgroup have been swept away.
2021-09-23 09:24:06 -05:00
Jon Chambers 02a296e500 Use a MIME Base64 decoder for provisioning messages 2021-09-22 16:30:44 -04:00
Jon Chambers 98e41f9a37
Improve Redis exception handling 2021-09-22 10:31:39 -04:00
Chris Eager 6a71d369e2
More Accounts cleanup
* Remove `AccountStore`
* Clean up `AccountsDynamoDb#delete`
* Rename `AccountsDynamoDb` → `Accounts`
* Remove unused configuration
* Move Accounts scan page size to static configuration
* Remove disabled tests and related methods
2021-09-21 15:25:16 -07:00
Ehren Kret 75661fa800 Add JavaTimeModule 2021-09-20 16:42:38 -05:00
Ehren Kret df5498e1c0 Add test for self badge fetching 2021-09-20 16:42:38 -05:00
Ehren Kret c0af911197 Show invisible badges to query for self 2021-09-20 16:42:38 -05:00
Ehren Kret 44bc90e5ab Return a badge with additional properties when fetching your own profile 2021-09-20 16:42:38 -05:00
Ehren Kret 5c1cde1b28 Add visibility modifying helper method in AccountBadge 2021-09-20 16:41:09 -05:00
Ehren Kret 3172b571c6 Move "testing" string into BadgeConfiguration 2021-09-20 16:41:09 -05:00
Ehren Kret 17e8b77e88 Add unit test for setting badges 2021-09-20 16:41:09 -05:00
Ehren Kret 8011935a3b Fix compilation issues created by constructor changes 2021-09-20 16:41:09 -05:00
Ehren Kret 3f3052c23c Remove isZkEnabled 2021-09-20 16:41:09 -05:00
Ehren Kret 8f17f45339 Reorder creating of testing badges
Need their expiration time to always get refreshed on set so reorder
how they're created to ensure that's the case.
2021-09-20 16:41:09 -05:00
Ehren Kret 009e2eeb97 Enable editing of badges on profiles 2021-09-20 16:41:09 -05:00
Ehren Kret c70fa48835 Fix indentation of ProfileController#setProfile 2021-09-20 16:41:09 -05:00
Ehren Kret bd5f5c407b Add method to merge badge ids list into a profile 2021-09-20 16:41:09 -05:00
Ehren Kret 2bc573a53d Add additional test badges for translation 2021-09-20 16:41:09 -05:00
Ehren Kret 537d61d5bd Add badges to CreateProfileRequest
This will permit users to set the order and visibility of badges on
their profile.
2021-09-20 16:41:09 -05:00
Jon Chambers 09519ae942 Only retry websocket sending if the client is still connected 2021-09-20 14:24:07 -04:00
Chris Eager 2a67b2e610
Remove Accounts Postgres 2021-09-20 11:10:24 -07:00
Chris Eager 8161f55a82 Add dynamic configuration for setting Dynamo as primary 2021-09-17 13:28:45 -07:00
Chris Eager ecee189ad8 Add `AccountDatabaseCrawler.dedicatedDynamoMigrationCrawler` 2021-09-17 11:27:20 -07:00
Fedor Indutny 383d744bd8
Log the error message when retrying queue send 2021-09-16 18:03:42 -04:00
Jon Chambers c2ba8ab562 Identify receipt destinations by UUID instead of e164 2021-09-16 10:47:03 -04:00
Chris Eager cd49ea43c0 Use `queryPaginator` when loading messages 2021-09-16 10:46:37 -04:00
Chris Eager 53aa45a2bb Use `queryPaginator` when deleting messages 2021-09-16 10:46:37 -04:00
Chris Eager 83e0a19561 Migrate `MessagesDynamoDbRule` to `MessagesDynamoDbExtension` 2021-09-16 10:46:37 -04:00
Jon Chambers 6a5d475198 Add a "refresh websocket on number change" provider 2021-09-16 10:37:34 -04:00
Jon Chambers 49ccbba2e3 Generalize the "watch for websockets that need to be refreshed" listener 2021-09-16 10:37:34 -04:00
Fedor Indutnyy 41735ed40e Introduce queueDrainRetry counter metric 2021-09-16 10:30:19 -04:00
Ehren Kret 2d11a433c9 Wrap all calls to getAcceptableLanguages
ContainerRequestContext#getAcceptableLanguages throws a
ProcessingException if the header has invalid values in it. Rather than
error out of the request entirely with the exception handler for that
exception, we just treat it as though no Accept-Languages header was
specified.
2021-09-16 09:28:21 -05:00
Ehren Kret e79ab2521f Rename field in ConfiguredProfileBadgeConverter 2021-09-16 09:28:21 -05:00
Ehren Kret fb1f99da87 Add a method to enable a badge for all accounts 2021-09-16 09:28:21 -05:00
Ehren Kret 08c6a8c2e5 Add category to badges 2021-09-16 09:28:21 -05:00
Ehren Kret ce3835e176 Rename id to name in the configuration 2021-09-16 09:28:21 -05:00
Ehren Kret 39f6eadbb9 Add test for add and remove badges 2021-09-16 09:28:21 -05:00
Ehren Kret 16dba09b61 Handle merging badges when adding to account 2021-09-16 09:28:21 -05:00
Ehren Kret d5ebf2f2ed Rename name to id in Account#removeBadge 2021-09-16 09:28:21 -05:00
Ehren Kret 8a8e6e7b49 Rename name to id in the stored badge information and expose id in the profile endpoint 2021-09-16 09:28:21 -05:00
Ehren Kret 34e21b9f7b Change name to id on AccountBadge
This makes it distinct from the localized name field on the Badge
entity that is returned.
2021-09-16 09:28:21 -05:00
Ehren Kret 98a31d1474 Switch ProfileController to the actual badge converter 2021-09-16 09:28:21 -05:00
Chris Eager 72a0c1be0f Tune mismatch logging 2021-09-15 16:46:10 -07:00
Ehren Kret 5b25e38e41 Ensure badges are in ordered collections 2021-09-15 16:20:15 -05:00
Chris Eager 2fb400280b Remove unused parameter from `deleteMessageByDestinationAndGuid` 2021-09-15 10:14:08 -07:00
Ehren Kret 79ad09524e
Implement the ProfileBadgeConverter interface 2021-09-15 10:32:20 -05:00
Chris Eager 5f8accb492 Add acceptable languages from request to variable 2021-09-14 17:43:39 -07:00
Chris Eager 6fcadc2297 Handle exception reading `Accept-Language` header 2021-09-13 18:07:16 -07:00
Chris Eager 3f4e1522eb Only put accounts that exhaust optimistic lock retries in migration retry table 2021-09-13 15:00:01 -07:00
Graeme Connell 6304c84cdb Add ContactDiscoveryWriterTest based on mock. 2021-09-13 15:20:21 -06:00
Chris Eager 894297efa9 Add dynamic configuration for doing a mismatch post-check 2021-09-13 13:54:19 -07:00
Chris Eager a51a7a0901 Add `MigrationMismatchedAccounts` to `AccountsManager` 2021-09-13 13:54:19 -07:00
Chris Eager 372e131e25 Update `PaymentsControllerTest` 2021-09-13 09:58:42 -07:00
Sophiah Ho cd66a1ceb7
fix merge issue after 2021 Aug 15 commit d1735c7e57 (#137) 2021-09-13 09:39:11 -07:00
Sophiah Ho feb59deb28
Use BigDecimal instead of Double for currency rate calculations (#134)
use BigDecimal instead of double for accuracy
2021-09-10 16:15:57 -05:00
Nicolas Remond 489519a982 Use Map.of() for statically defined map 2021-09-10 14:27:18 -05:00
Chris Eager 6a6555e2d5 Add metrics for AuthEnablementRequestEventListener displacements 2021-09-10 12:01:05 -07:00
Chris Eager 49489a6021 Re-check mismatched accounts after a delay, to avoid false positives from concurrent requests 2021-09-10 11:31:44 -07:00
Chris Eager 8cd93d68e4 Add MetricsUtil 2021-09-10 11:31:44 -07:00
Chris Eager f3b9a8d97f Add account to migration retry table on transient dynamo failure 2021-09-10 11:30:49 -07:00
Chris Eager b91a69d8b3 Add asynchronous chunk pre-read to `AccountDatabaseCrawler` 2021-09-10 11:14:11 -07:00
Chris Eager 624e40e3b7 Add separate `AccountsDatabaseCrawler` for DynamoDB migration 2021-09-10 11:14:11 -07:00
Chris Eager 23a076a204 Update `Account#getNextDeviceId` to not reuse disable device’s IDs 2021-09-10 10:48:48 -07:00
Chris Eager 016141a05d Add `DevicesHelper` 2021-09-10 10:48:48 -07:00
Graeme Connell a064b25a14 Fix CDS writer to use AccountsManager. 2021-09-10 11:36:06 -06:00
Ehren Kret bd40e32f3b Send acceptable languages instead of request into the profile badge converter 2021-09-10 10:53:04 -05:00
Ehren Kret 81a21c0d5f Use @NotNull since @NotEmpty doesn't support URL 2021-09-10 10:49:31 -05:00
Ehren Kret 6478210330 Update configuration for badges to use URL instead of String 2021-09-10 10:49:31 -05:00
Ehren Kret aa1c37fe26 Create configuration for badges 2021-09-10 10:14:16 -05:00
Ehren Kret 6ee23b0186 Create resource bundle for badges 2021-09-10 10:11:56 -05:00
Jon Chambers 40eb445592 Add a command to set a user's discoverability in CDS 2021-09-10 10:34:20 -04:00
Jon Chambers ce7d687205 Add a shutdown monitor that publishes shutdown state as a metric 2021-09-08 16:37:05 -04:00
Chris Eager 758900b7a8 Register AuthEnablementApplicationEventListener 2021-09-08 13:11:09 -07:00
Chris Eager 539b62a829 Add request event listener that handles device.isEnabled changes 2021-09-08 13:11:09 -07:00
Jon Chambers 2866f1b213 Include e164 in account creation (whoami) responses 2021-09-07 16:52:32 -04:00
Ehren Kret fc1465c05d Wire up stored account badges to the profile endpoints 2021-09-07 15:51:29 -05:00
Ehren Kret bc887ec6fa Add visibility flag to badge storage 2021-09-07 15:50:29 -05:00
Ehren Kret 84b3d324bb Creates a storage object for badges 2021-09-07 15:49:41 -05:00
Ehren Kret fc10108788 Make fields final in Badge entity 2021-09-07 15:39:48 -05:00
Ehren Kret fbbc1bec58 Add badge entity to profile 2021-09-07 15:39:48 -05:00
Graeme Connell 2059bb5ef8 Update test to handle read-then-write in ContactDiscoveryWriter. 2021-09-07 13:41:47 -06:00
gram-signal b080a5db4d Get-and-set accounts, since other updates may have made them stale.
Co-authored-by: Chris Eager <79161849+eager-signal@users.noreply.github.com>
2021-09-07 13:41:47 -06:00
Graeme Connell b4aabd799b Canonical discoverability writing. 2021-09-07 13:41:47 -06:00
Jon Chambers 92f035bc2a Add a "change number" device/account capability 2021-09-07 15:07:30 -04:00
Chris Eager 18a6df34bd Add timers to `processChunk` and `deleteRecentlyDeletedUuids` 2021-09-03 14:54:51 -07:00
Chris Eager b1274125c9 Add start/chunk/sleep logging to crawler 2021-09-03 14:54:51 -07:00
Chris Eager dceebc1c8d Consistently use `whenCompleteAsync(…, migrationThreadPool)` 2021-09-03 14:02:51 -07:00
Chris Eager 6aadb4b458 Parameterize registration lock constructor when updating account attributes 2021-09-03 14:02:27 -07:00
Fedor Indutnyy 703405b874 Start WebSocket before registering its presence 2021-08-27 16:41:07 -04:00
Jon Chambers d1735c7e57
Retire `AmbiguousIdentifier` 2021-08-27 13:40:46 -04:00
Jon Chambers 1f815b49dd Measure APNs rejections by reason 2021-08-27 11:52:29 -04:00
Jon Chambers a9339b7037 Update to Pushy 0.15.0 2021-08-27 11:52:29 -04:00
Jon Chambers f2c6ca182d Include the current server version in the tag list for Dropwizard metrics 2021-08-27 11:52:11 -04:00
Jon Chambers b946c27a20 Remove a metric aggregator 2021-08-27 11:52:11 -04:00
Jon Chambers 8a8a848fac Record error metrics from Twilio Verify 2021-08-26 12:22:17 -04:00
Chris Eager aeb9f67266 Migrate MessageSenderTest to JUnit 5 2021-08-25 12:25:10 -05:00
Chris Eager e08c5a412e Insert ephemeral messages in the standard cache queue 2021-08-25 12:25:10 -05:00
Chris Eager a7443a9ece Don’t persist ephemeral messages; clear ephemeral field when sending to clients 2021-08-25 11:17:00 -05:00
Chris Eager 54fe3b9a43 Update TextSecure.proto 2021-08-25 11:17:00 -05:00
Ehren Kret ba522b1691 Clean redis message cache structure 2021-08-24 10:30:52 -05:00
Jon Chambers 739c5bf22c Add a counter to estimate announcement group adoption 2021-08-23 17:31:34 -04:00
Chris Eager 7cdadeb791 Register circuit breaker metrics for FaultTolerantPubSubConnection 2021-08-23 15:49:19 -05:00
Chris Eager dadf43b93e Consolidate directory reconciliation on v3 endpoints 2021-08-19 14:18:38 -05:00
Chris Eager 19f7b207b7 Extract configuration for WebSocket max message sizes 2021-08-19 14:09:57 -05:00
Chris Eager a398e2269c Update `AccountsManager` mismatch comparison 2021-08-19 14:08:48 -05:00
Chris Eager 2e28fb97a4 Delete DynamoDB accounts with invalid UUIDs in AccountsManager#create 2021-08-19 14:05:21 -05:00
Chris Eager 5c68d83a93 Add integration test for re-registration with and without Dynamo DB 2021-08-19 14:05:21 -05:00
Chris Eager 0b7c3ad745 .editorconfig formatting 2021-08-16 16:32:26 -05:00
Chris Eager 0cde06557d Catch and log unexpected exceptions keyspace notification executor service 2021-08-16 16:32:26 -05:00
Chris Eager 27844fe692
Add JUnit 5 RedisClusterExtension 2021-08-13 12:07:04 -05:00
Ehren Kret 779051ef9f Add minThreads(64) to multiRecipientMessageExecutor 2021-08-12 13:03:40 -05:00
Ehren Kret d13741fbd5 Change from using parallel streams to using an ExecutorService 2021-08-12 12:05:01 -05:00
Ehren Kret f7f870fe62 Execute send multi-recipient message loop in parallel 2021-08-12 12:05:01 -05:00
Chris Eager de59aa099d Add uncaught exception handler 2021-08-12 11:10:05 -05:00
Ehren Kret 57a478b898 Remove unused local variable 2021-08-12 10:26:23 -05:00
Ehren Kret 3e8d79e147 Remove unused lua script to delete by sender and timestamp 2021-08-11 17:38:55 -05:00
Ehren Kret a46045d987 Remove unused methods that delete messages by sender and timestamp 2021-08-11 17:30:39 -05:00
Ehren Kret 662c905b80 Remove deprecated delete messages endpoint
DELETE /v1/messages/{source}/{timestamp} has been deprecated a long
time and has minimal usage each day at this point. Dropping support
for this endpoint to improve message cache storage flexibility.
2021-08-11 16:17:44 -05:00
Chris Eager 31022aeb79
Use refreshing `AuthenticatedAccount` for `@Auth` 2021-08-11 14:52:25 -05:00
Chris Eager b3e6a50dee Send 508 status code for legacy clients that produce rate limit challenges 2021-08-11 11:57:30 -05:00
Chris Eager d29764d11f Only process updates for enabled devices in PushFeedbackProcessor 2021-08-11 11:54:42 -05:00
Chris Eager f8e4f6727a Reorder crawler listeners so updates happen after read-only processing 2021-08-11 11:54:20 -05:00
Ehren Kret 63d05df8a3 Fix indentation 2021-08-10 10:02:04 -05:00
Ehren Kret 52d13d1d62 Remove unused lua script 2021-08-10 10:02:04 -05:00
Ehren Kret f58a320223 Remove unused method from MessagesCache 2021-08-10 10:02:04 -05:00
Chris Eager 3e01bc1174 Add metric for content-length header distribution 2021-08-06 14:41:16 -05:00
Jon Chambers d1ada7f998 Revert "Continue to verify rate limiters by e164 during UUID migration period"
This reverts commit ce5edbb7fc.
2021-08-06 14:33:59 -05:00
Jon Chambers 73c368ea86 Use UUIDs instead of e164s to associate accounts with push notifications. 2021-08-04 14:38:28 -04:00
Jon Chambers ce5edbb7fc Continue to verify rate limiters by e164 during UUID migration period 2021-08-04 14:15:21 -04:00
Jon Chambers a680639718 Use UUIDs as rate limiter keys. 2021-08-04 14:15:21 -04:00
Ehren Kret becf6afbdd Block until all UUID bytes are read or EOF 2021-08-03 17:59:48 -05:00
Ehren Kret 1dda015c6a Update multi-recipient message sending to handle unrestricted destinations 2021-08-03 17:31:39 -05:00
Chris Eager cfd31e98ff Move version comparison to after more meaningful checks 2021-08-03 11:03:41 -05:00
Jon Chambers bcb89924b4
Simplify optimistic write logic 2021-08-03 11:54:26 -04:00
Ehren Kret 1f6318a919 Rename constant 2021-08-02 13:14:44 -05:00
Ehren Kret b0667b258b Implement EnterpriseRecaptchaClient 2021-08-02 13:14:44 -05:00
Ehren Kret 4c3a48f5be Use more specific prefix for recaptcha transition 2021-08-02 13:14:44 -05:00
Ehren Kret 33fb7a72de Use RecaptchaClient interface 2021-08-02 13:14:44 -05:00
Ehren Kret 2c808e369c Create a transitional recaptcha client for upgrading 2021-08-02 13:14:44 -05:00
Ehren Kret 906d0be382 Setup recaptcha client interface 2021-08-02 13:14:44 -05:00
Ehren Kret 2aaddd721f Rename existing captcha client 2021-08-02 13:14:44 -05:00
Jon Chambers 4e2284b83f Retire old GV2 adoption metrics. 2021-08-02 12:51:49 -05:00
Chris Eager d5d9978e48 Use non-stale account in mutator when adding a new device 2021-08-02 11:38:03 -05:00
Chris Eager d45659ac76 Reduce contention when updating `device.lastSeen` 2021-08-02 11:26:15 -05:00
Jon Chambers 13a07dc6cd Drop the active user counter. 2021-07-29 15:40:27 -04:00
Chris Eager 51b7a8d868 Add excluded E164s configuration to pre-registration experiment 2021-07-29 14:16:40 -05:00
Chris Eager df9c0051c9 Reconcile inactive and undiscoverable accounts when using v3 endpoints 2021-07-29 10:56:44 -05:00
Jon Chambers 331ff83cd5
Drop legacy PIN-based registration lock plumbing 2021-07-29 11:51:14 -04:00
Jon Chambers 44838d6238 Verify that nobody's addressing API calls by e164 any more. 2021-07-29 11:50:36 -04:00
Chris Eager 5400abb065 Better support unhandled exception logging on websocket requests 2021-07-28 14:06:09 -05:00
Jon Chambers f47fefb73e Lock accounts for the duration of deletion operations. 2021-07-27 13:12:39 -04:00
Jon Chambers cdef745a7a Drop a not-very-helpful metric (logging works better in this case). 2021-07-27 13:12:39 -04:00
Jon Chambers 1a1eab4ec0 Also clear profiles on re-registration. 2021-07-27 13:05:54 -04:00
Jon Chambers 3a966ef345 Reuse account UUIDs when registering an account with a recently-deleted e164. 2021-07-27 13:05:54 -04:00
Jon Chambers be20c04cd8 Identify accounts for which to delete keys by UUID. 2021-07-27 13:05:54 -04:00
Jon Chambers d09dcc90fe Add methods for getting, clearing, locking recently-deleted account records. 2021-07-27 13:05:54 -04:00
Jon Chambers 0117fc12c7 Actually increment the moved "new user" counter. 2021-07-27 12:09:51 -04:00
Jon Chambers ef9a7fda9a Publish outstanding SQS operation count as a gauge. 2021-07-27 11:15:41 -04:00
Chris Eager 13447df1e0 Update validation for NotNull items in IncomingMessagesList 2021-07-27 10:39:30 -04:00
Jon Chambers 3608c5bfb0 Wait for outstanding requests to be resolved before shutting down the directory queue. 2021-07-27 10:36:53 -04:00
Jon Chambers 34dbff6786 Switch to an async SQS client. 2021-07-27 10:36:53 -04:00
Jon Chambers a6066bfc2f Migrate DirectoryQueueTest to JUnit 5. 2021-07-27 10:36:53 -04:00
Jon Chambers 8579190cdf
Consolidate account creation/directory updates into `AccountsManager` 2021-07-27 10:27:47 -04:00
Chris Eager 917f667229 Remove AccountController and KeysController from websocket 2021-07-26 14:27:43 -05:00
Chris Eager 317a551bdb Migrate MetricsRequestEventListenerTest to JUnit 5 2021-07-26 12:06:29 -05:00
Chris Eager 27e9271473 Add request path and user agent to unhandled exception logging 2021-07-26 12:06:29 -05:00
Fedor Indutny 11dff6c546 more controllers 2021-07-26 12:06:17 -05:00
Fedor Indutny e6712937ca fix indent 2021-07-26 12:06:17 -05:00
Fedor Indutny cf8887bb5a Provide more WebSocket endpoints 2021-07-26 12:06:17 -05:00
Chris Eager 696340f780 Migrate DeviceControllerTest to JUnit 5 2021-07-26 11:18:17 -05:00
Chris Eager 86ddcbaa08 Migrate CertificateControllerTest to JUnit 5 2021-07-26 11:18:17 -05:00
Chris Eager 2144d2a8d8 Migrate AttachmentControllerTest to JUnit 5 2021-07-26 11:18:17 -05:00
Chris Eager f7af861b31 Migrate SecureStorageControllerTest to JUnit 5 2021-07-26 11:18:17 -05:00
Chris Eager 208a09b3ae Migrate RemoteConfigControllerTest to JUnit 5 2021-07-26 11:18:17 -05:00
Chris Eager 831023e41d Migrate PaymentsControllerTest to JUnit 5 2021-07-26 11:18:17 -05:00
Chris Eager ff627793d6 Migrate DirectoryControllerTest to JUnit 5 2021-07-26 11:18:17 -05:00
Chris Eager f971c76a99 Migrate StickerControllerTest to JUnit 5 2021-07-26 11:18:17 -05:00
Chris Eager 8f41176c76 Enable "sms" transport for +98 2021-07-26 10:40:05 -05:00
Ehren Kret 31bbbbb5e0 Raise default message TTL to 14 days 2021-07-20 14:08:08 -05:00
Jon Chambers effcd6038d Also record dimensional metrics for circuit breakers and retries. 2021-07-19 16:56:16 -04:00
Jon Chambers 12be7d49c2 Clear one-time pre-keys on re-registration. 2021-07-19 10:05:01 -04:00
Jon Chambers 14863b575e Clear one-time pre-keys when a device is unlinked. 2021-07-19 10:05:01 -04:00
Jon Chambers 32a95f96ff Add a pessimistic locking system for operations on recently-deleted account records 2021-07-16 16:52:58 -04:00
Jon Chambers b757d4b334 Measure how many "send message" requests are still using e164-based addressing. 2021-07-16 16:52:58 -04:00
Chris Eager bd03d910fe Set authenticated device after updating last seen 2021-07-16 16:52:58 -04:00
Chris Eager 01ef855157 Return a non-stale account from base authenticator when last seen is updated 2021-07-16 16:52:58 -04:00
Chris Eager 817866caf3 Use fresh accounts to update in PushFeedbackProcessor 2021-07-16 16:52:58 -04:00
Chris Eager 158d65c6a7 Add optimistic locking to account updates 2021-07-16 16:52:58 -04:00
realturner 62022c7de1 Migrate AppConfig to SDK v2 to detect and use web identify token 2021-07-16 16:48:33 -04:00
Chris Eager a824b5575d
Add dynamic configuration for using DynamoDB in AccountsDatabaseCrawler 2021-07-06 13:01:24 -05:00
Jon Chambers 78819d5382 Remove expiration logic when checking token validity.
The data store will no longer return tokens that have expired, and we no longer need to check for expiration in application space.
2021-07-06 11:03:49 -04:00
Jon Chambers d128bc782a Retire Postgres-backed pending account/device tables. 2021-07-06 11:03:49 -04:00
Chris Eager 530b2a310f Ensure active future is always completed 2021-07-02 15:05:11 -05:00
Chris Eager d5b0d99a54 Remove unused method 2021-07-02 15:05:11 -05:00
Chris Eager 43be72d076 Add test for ManagedPeriodicWork; fix shutdown not awaiting active execution 2021-07-02 15:05:11 -05:00
Chris Eager 0f6c866c8d Update imports 2021-07-02 15:05:11 -05:00
Chris Eager bac78e9291 Switch DeletedAccountsTableCrawler metrics to a basic Metrics#summary 2021-07-02 15:05:11 -05:00
Chris Eager c22ea78672 Add crawler to process migration retry accounts 2021-07-02 15:05:11 -05:00
Chris Eager a85afe827d Avoid NPE by using scheduledFuture as the Gauge state object 2021-07-02 15:05:11 -05:00
Chris Eager abaed821ec Add additional case to unit test 2021-07-02 15:05:11 -05:00
Chris Eager 6fa9dcd954 Refactor to use shared recurringJobExecutor 2021-07-02 15:05:11 -05:00
Chris Eager 819d59cd79 Update reconciliation crawler to use secondary index 2021-07-02 15:05:11 -05:00
Chris Eager 2f88f0eedb Refactor to use single threaded scheduled executor 2021-07-02 15:05:11 -05:00
Chris Eager 74ff491671 Rename ManagedPeriodicWorkCache to ManagedPeriodicWorkLock 2021-07-02 15:05:11 -05:00
Chris Eager eac48a6617 Don’t delete accounts after reconciling 2021-07-02 15:05:11 -05:00
Chris Eager 19617c14f8 Improved logging in ManagedPeriodcWork 2021-07-02 15:05:11 -05:00
Chris Eager fc7291c3e8 Migrate DeletedAccountsTableCrawler to ManagedPeriodicWork 2021-07-02 15:05:11 -05:00
Chris Eager 88db808298 Add abstract ManagedPeriodicWork 2021-07-02 15:05:11 -05:00
Chris Eager 5193abdab3 Add DeletedAccountsTableCrawler 2021-07-02 15:05:11 -05:00
Chris Eager a315c9be92 Add DeletedAccounts DynamoDB table 2021-07-02 15:05:11 -05:00
Chris Eager fc1541591a Add AbstractDynamoDbStore#scan 2021-07-02 15:05:11 -05:00
Chris Eager ae97c4db9f Use editorconfig in AbstractDynamoDbStore 2021-07-02 15:05:11 -05:00
Chris Eager 26bc5973b5 Clear message queue before and after removing a device 2021-07-02 10:48:42 -05:00
Chris Eager e52b8c8423 Implement DatadogConfig in DatadogConfiguration 2021-07-02 10:48:05 -05:00
Jon Chambers 7395489bac Add tests for pending account/device managers. 2021-07-02 11:30:13 -04:00
Jon Chambers b384ed7f5c Add a counter for requests for delivery certificates with/without e164s. 2021-07-01 10:59:10 -04:00
Jon Chambers e3afcae7d3 Gather data to verify safety of retiring legacy reglock system. 2021-07-01 10:58:47 -04:00
Jon Chambers 9faeed7b20 Count E164 authentications versus UUID authentications. 2021-07-01 10:51:34 -04:00
Jon Chambers 49adcca80e Use Optional.isEmpty(). 2021-07-01 10:51:34 -04:00
Jon Chambers 49c43a6816 Simplify distribution summary for "days since last seen." 2021-07-01 10:51:34 -04:00
Jon Chambers 84f85ae098 Collapse various account meters into a single, multi-dimensional counter. 2021-07-01 10:51:34 -04:00
Jon Chambers 3d581941ab Add plumbing and configuration to migrate pending accounts/devices to DynamoDB. 2021-07-01 10:50:52 -04:00
Jon Chambers d2d39baede Add a DynamoDB-backed stored verification code store. 2021-07-01 10:50:52 -04:00
Jon Chambers 111f5ba024 Use java.time classes for stored verification code expiration; add tests. 2021-07-01 10:50:52 -04:00
Jon Chambers ce3fb7fa99 Extract a common base class for verification code store tests. 2021-07-01 10:50:52 -04:00
Jon Chambers fc421d3f21 Introduce a common interface for verification code stores. 2021-07-01 10:50:52 -04:00
Jon Chambers 71bea759c6 Consolidate StoredVerificationCode constructors. 2021-07-01 10:50:52 -04:00
Jon Chambers bf1dd791a5 Drop caching for pending accounts/devices. 2021-07-01 10:50:52 -04:00
Chris Eager 4c99577c08 Add configuration for Datadog batch size 2021-06-30 16:44:25 -05:00
Graeme Connell 5d5c63e6d4 Update profile controller to S3 AWSv2. 2021-06-30 13:09:18 -06:00
Graeme Connell 42ff3f8432 Switch SQS to Amazon SDKv2. 2021-06-30 12:46:12 -06:00
Ehren Kret 7883352b74 Match random capability generation in test 2021-06-21 17:32:31 -05:00
Ehren Kret 982d122d18 Match random capability generation in test 2021-06-21 17:32:31 -05:00
Ehren Kret d8d94407c6 Create announcement group capability 2021-06-21 17:32:31 -05:00
Chris Eager 28cfc54170 Update FunctionCounter builder to use non-null object and method 2021-06-11 11:27:45 -05:00
Jon Chambers 2ee7279743 Pause nstat counters. 2021-06-11 12:26:56 -04:00
Jon Chambers eb1b073385 Add a hostname-aware reporter factory. 2021-06-10 14:23:05 -04:00
Jon Chambers c634185b6f Standardize a utility method for getting local host names. 2021-06-10 14:23:05 -04:00
Jon Chambers 2c33d22a30 Stop recording specific client versions in metrics until we know we need them again. 2021-06-08 12:25:31 -04:00
Jon Chambers 58d3a12eff Set hostname to lowercase to avoid strange case mismatch issues; log hostname failures. 2021-06-07 17:17:46 -04:00
Jon Chambers 88c4b2be97 Correct a misunderstanding about the metrics host tag. 2021-06-07 16:29:44 -04:00
Jon Chambers 6cbd57f19f Include environment/service/version as common metric tags. 2021-06-04 18:17:09 -04:00
Jon Chambers 5522376584 Include a host tag with metrics. 2021-06-04 18:17:09 -04:00
Jon Chambers 5089c37d28 Drop a pair of unused commands. 2021-06-04 12:35:06 -04:00
Jon Chambers 1ccf24e68c Add a command to check dynamic config files. 2021-06-04 12:34:48 -04:00
Jon Chambers 411f7298f2 Enforce validation constraints for dynamic configuration objects. 2021-06-04 12:34:48 -04:00
Jon Chambers 5b0214c6f2
Make pre-key `take` operations more null-safe 2021-06-04 11:18:59 -04:00
Jon Chambers 735573e61b Make reporting intervals configurable. 2021-06-03 17:50:41 -04:00
Graeme Connell c545cff1b3 Switch DynamoDB to AWSv2.
Switch from using com.amazonaws.services.dynamodbv2 to using
software.amazon.awssdk.services.dynamodb for all current DynamoDB uses.
2021-06-03 13:37:10 -06:00
Jon Chambers cbd9681e3e Configure histograms and exclude high-cardinality metrics. 2021-06-03 14:12:02 -04:00
Jon Chambers ca876e40ca Add a second metric aggregator. 2021-06-03 14:12:02 -04:00
Jon Chambers 76f5a71727
Include server version in logging tags 2021-06-03 11:24:25 -04:00
Jon Chambers 117de2382d Verify that API consumers can skip/clear VOIP tokens. 2021-06-02 16:50:49 -05:00
Jon Chambers 25e7036451 Send a payload with mutable content for non-VOIP topics. 2021-06-02 16:50:49 -05:00
Jon Chambers 3131bd3dd9 Allow iOS callers to specify whether they're providing a VOIP token for preauth. 2021-06-02 16:50:49 -05:00
brock-signal c97be15e79
Fix NPE when a null message comes in from a client 2021-06-01 15:00:41 -06:00
Ehren Kret 164fc40990 Rename receipt type and add new client-to-client plaintext type for decryption error receipts 2021-05-28 11:33:44 -05:00
Ehren Kret 6456af6284 Upgrade to latest protobuf
This upgrades to protobuf 3.17 and uses maven to automatically rebuild
the generated code instead of using prefabricated checked in Java
files.
2021-05-28 11:33:44 -05:00
Ehren Kret 6f0750790c Add metric to count number of legacy messages sent 2021-05-27 11:13:42 -05:00
Chris Eager 3e61b5c49d Add call chain and mismatch check for push token timestamp 2021-05-27 11:10:58 -05:00
Chris Eager 5ee093f87c Add mismatch for signed pre-key; remove mismatch for migration version 2021-05-26 10:58:23 -05:00
Chris Eager 67067f1d2d Remove last-seen and registration lock comparisons 2021-05-25 10:47:57 -05:00
Ehren Kret 07f9bb112e Use separate object for multi recipient response
`needsSync` was being sent back from the server in the JSON response
which is an unnecessary and constantly false field in multi-recipient
message sending endpoint as it's always sealed sender.
2021-05-25 10:30:39 -05:00
Ehren Kret 417d48c452 Block downgrading sender key support
Disallow linking an additional device to an account that has already
upgraded to having sender key support where the linked device does not
have sender key support. This should prompt the person attempting to
link the older application to upgrade in order to complete the linking
process.
2021-05-25 10:30:26 -05:00
Chris Eager 215621a9b0 Remove temporary adaptation for nested `IncomingMessage.online` 2021-05-24 11:36:15 -05:00
Graeme Connell c3f53c4dd9 Fix infinite loop in TorExitNodeManager. 2021-05-21 14:50:15 -06:00
Graeme Connell c10b64c367 Simplify S3ObjectMonitor API, try-with-resource. 2021-05-21 14:50:15 -06:00
Graeme Connell 722055c8b5 Switch S3ObjectMonitor to AWSv2 SDK. 2021-05-21 14:50:15 -06:00
Ehren Kret 5290656c3b Fix typo 2021-05-20 15:11:44 -05:00
Ehren Kret cf59d849b0 @Min does not apply to byte[] use @Size instead 2021-05-20 10:15:49 -05:00
Ehren Kret ee3b91e4fb Register MultiRecipientMessageProvider with the websocket interface too 2021-05-20 10:15:49 -05:00
Jon Chambers 94ac3f6cc8 Return Optional.empty() for present-but-not-routed IPs. 2021-05-18 17:43:30 -04:00
Jon Chambers b89de860d3 Add support for getting country codes for ASNs. 2021-05-18 17:43:30 -04:00
Jon Chambers f8c623074b Introduce an ASN-to-IP manager. 2021-05-18 17:43:30 -04:00
Jon Chambers 1160af9522 Add a utility class for associating IP addresses with ASNs. 2021-05-18 17:43:30 -04:00
Jon Chambers 3056ea8cbc More clearly separate concerns for explicitly getting monitored objects. 2021-05-18 17:00:30 -04:00
Jon Chambers 28e3b23e8c Add an "excessively large object" safeguard. 2021-05-18 17:00:30 -04:00
Jon Chambers fbaf4a09e2 Use the S3 object monitor to retrieve Tor exit node lists. 2021-05-18 17:00:30 -04:00
Jon Chambers cfa8cbedc1 Introduce an S3 object monitor. 2021-05-18 17:00:30 -04:00
Chris Eager bacf524ae6 Add optional logging for mismatches 2021-05-18 14:39:30 -05:00
Graeme Connell aa65d34c36 Set min/max threads for backup/storage service.
From https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ThreadPoolExecutor.html:
 When a new task is submitted in method execute(java.lang.Runnable),
 and fewer than corePoolSize threads are running, a new thread is
 created to handle the request, even if other worker threads are idle.
 If there are more than corePoolSize but less than maximumPoolSize
 threads running, a new thread will be created only if the queue is full.

Since we utilize an unbounded queue, we'll never hit the condition that
the queue is full, so the pool will never grow past corePoolSize.  Given
that, explicitly state that our max is 1 thread.  This should be a noop
operationally.

Thanks to https://github.com/dropwizard/dropwizard/pull/834 for building
in warnings to help us find this.
2021-05-18 13:34:59 -06:00
Ehren Kret 0cd3640f13 Add more tests 2021-05-18 13:09:40 -05:00
Ehren Kret c595d9415c Change from quadratic scan to use more memory instead 2021-05-18 13:09:40 -05:00
Ehren Kret 1a604d8c79 Add unit test to readU16 2021-05-18 13:09:40 -05:00
Ehren Kret f76e6705c0 Add handling of registration id in multi recipient send payload 2021-05-18 13:09:40 -05:00
Chris Eager 89470ff536 Add class prefix to counter 2021-05-17 17:17:48 -05:00
Chris Eager d252e579f4 Get more detailzed serialization mismatches 2021-05-17 15:42:54 -05:00
Chris Eager 30b2c2b5ad Fix observed mismatches by swapping in original UUID 2021-05-17 15:42:17 -05:00
Chris Eager 282f39141e Add additional tests for AccountStore#create 2021-05-17 15:42:17 -05:00
Chris Eager 85e4de6933 Switch platform value from `null` to "unrecognized" in metrics 2021-05-17 15:41:11 -05:00
Chris Eager 00c9023e74 Include server GUID when sending messages over websocket 2021-05-14 17:10:15 -05:00
Chris Eager 2a3ea13c9e Classify DynamoDB mismatches in AccountsManager 2021-05-14 15:57:47 -05:00
Ehren Kret 6906336dfb Include the uuid with the list of mismatched devices 2021-05-14 14:46:56 -05:00
Chris Eager df01be2dca Don’t throw exceptions from ReportMessageManager#store() 2021-05-13 18:33:02 -05:00
Chris Eager e320626c6e
Add report message API 2021-05-13 17:19:34 -05:00
Jon Chambers 03dac2bf7e Break down rate limit overruns by country. 2021-05-13 18:16:47 -04:00
Ehren Kret 57ff9f86f5 Refactor repeated use of the UserCapabilities constructor 2021-05-13 12:08:59 -05:00
Ehren Kret bfd2c32d4e Add sender key capability 2021-05-12 18:15:25 -05:00
Ehren Kret e9a3d52d7f Add an optional description to the payment intent request 2021-05-12 18:14:33 -05:00
Jon Chambers d45154f2aa Measure captcha challenge success rates. 2021-05-12 12:20:53 -05:00
Chris Eager 760462f8fb Add configuration for regional SMS verification text 2021-05-12 12:20:46 -05:00
Jon Chambers 46110d4d65
Add client challenges for prekey and message rate limiters 2021-05-11 16:21:32 -05:00
Ehren Kret 02d06af3fc Replace use of MDC with custom fields on the appender 2021-05-11 14:08:20 -05:00
Ehren Kret b100f09205 Actually instantiate the donation controller in the service 2021-05-11 10:03:04 -05:00
Ehren Kret 670b69df24 Add host and service tags 2021-05-11 10:03:04 -05:00
Ehren Kret 13ecbe7e53 Fix missing defaults 2021-05-11 10:03:04 -05:00
Ehren Kret 17047513c3 Create stripe api endpoint for apple pay donations 2021-05-11 10:03:04 -05:00
Chris Eager 4571042814 Add missing increment to counter 2021-05-10 10:45:46 -04:00
Ehren Kret 9cb89b42bf Create a logstash tcp socket appender factory 2021-05-07 16:02:55 -05:00
Jon Chambers bf32b766a5 Don't generate stack traces for rate limit exceptions. 2021-05-07 10:44:31 -04:00
Jon Chambers f0a8b5a54a Allow the environment to manage the Tor exit node manager's lifecycle. 2021-05-06 15:38:24 -04:00
Jon Chambers b81b811400 Actually instantiate the Tor exit node manager. 2021-05-06 12:21:30 -04:00
Jon Chambers b41f97233e Measure source country for all pre-key requests. 2021-05-06 11:58:14 -04:00
Jon Chambers 350de1c759 Add a simple utility class for testing if an IP belongs to a Tor exit node. 2021-05-06 11:57:18 -04:00
Jon Chambers dfb8a419e7 Include message IDs when sending message batches. 2021-05-04 16:15:04 -04:00
Jon Chambers cf495ef7cf Key the message rate limiter to sender e164, not UUID. 2021-05-04 15:18:59 -04:00
Jon Chambers 8fdbcbef44 Send directory updates in batches. 2021-05-04 15:18:43 -04:00
Ehren Kret 1a8c40c02a Patch code to use new Base64 2021-05-04 13:34:27 -05:00
Ehren Kret 20677d4be1 Fix logic error 2021-05-04 13:34:27 -05:00
Ehren Kret c448c37cc9 Add logic to handle sending a common payload to multiple recipients 2021-05-04 13:34:27 -05:00
Chris Eager fac4538f6f Migrate rate limiters to rate limiter cluster 2021-05-03 10:57:34 -05:00
Chris Eager 7e805d1592 Add rate limiters cluster to all RateLimiters 2021-04-30 17:18:56 -05:00
Chris Eager 0e6cfb460d Fix potential NullPointerException in RateLimiter 2021-04-30 16:15:36 -05:00
Chris Eager 4f6b132449 Add secondaryCacheCluster to RateLimiter 2021-04-30 15:26:17 -05:00
Ehren Kret e4dbb8efe7 Fix deserialization for Duration in dynamic config 2021-04-28 15:57:23 -05:00
Jon Chambers 59e401f41e Use a MIME Base64 decoder for attachment signing keys. 2021-04-26 18:53:15 -04:00
Jon Chambers 6196856a7c Use the JDK-provided Base64 encoder/decoder. 2021-04-26 18:17:03 -04:00
Jon Chambers 0e8d4f9a61 Drop Bouncy Castle as a dependency. 2021-04-26 17:58:19 -04:00
Chris Eager 62315f423c Record duration of successful verifications 2021-04-26 14:31:52 -05:00
Chris Eager 5ee56b022c Add CustomFriendlyName parameter to Twilio Verify requests 2021-04-26 14:31:52 -05:00
Chris Eager 6c37b658ac Migrate VoiceVerificationControllerTest to JUnit 5 2021-04-26 14:31:52 -05:00
Chris Eager 1f53900345 Migrate VoiceVerificationController to Util#findBestLocale 2021-04-26 14:31:52 -05:00
Chris Eager deece33a0d Fix parameterized testWhoAmI 2021-04-26 14:31:52 -05:00
Chris Eager 13053da97f Add Twilio Verify experiment to AccountController 2021-04-26 14:31:52 -05:00
Chris Eager 4c019aef15 Migrate PendingAccountsTest to JUnit 5 2021-04-26 14:31:52 -05:00
Chris Eager bab5e5769b Add TwilioVerifyExperimentEnrollmentManager 2021-04-26 14:31:52 -05:00
Chris Eager f68390e96f Add Twilio Verify methods to SmsSender 2021-04-26 14:31:52 -05:00
Chris Eager 76cbf734ad Add TwilioVerifySender to TwilioSmsSender 2021-04-26 14:31:52 -05:00
Chris Eager 17ba630014 Add TwilioVerifySender 2021-04-26 14:31:52 -05:00
Chris Eager 3121867f72 Remove incorrect license header 2021-04-23 13:27:22 -05:00
Chris Eager f190462879 Fully implement unsealed sender cardinality rate limiter 2021-04-23 11:45:53 -05:00
Chris Eager ac72c8b2de Remove log 2021-04-22 18:15:52 -05:00
Chris Eager 6c6f073bc2 Pass re-migration account in test 2021-04-22 17:11:10 -05:00
Chris Eager 0663fe30df Add cancellation reason codes to error logging 2021-04-22 17:11:10 -05:00
Chris Eager b6cb23cbb5 Handle potentially null item from Dynamo delete outcome 2021-04-22 15:24:58 -04:00
Chris Eager b75456acf3 Allow migration thread pool to be scaled up 2021-04-21 15:23:02 -05:00
Chris Eager abafa2ccac Include ATTR_MIGRATION_VERSION in update() 2021-04-21 14:13:13 -05:00
Jon Chambers b75dec40ac Drop syslog4j in favor of Dropwizard's/Logback's syslog appender. 2021-04-21 11:23:52 -05:00
Chris Eager 5974328d9c Ensure accounts are deleted after batch migration; store migration failures for later processing 2021-04-21 11:13:07 -05:00
Chris Eager a472774734 Add threadpoool to increase Accounts → Dynamo migration throughput 2021-04-21 11:13:07 -05:00
Chris Eager 166d203e8e Don’t PUT unmigrated accounts in update() 2021-04-21 11:13:07 -05:00
Chris Eager f2a1a65a45 Migrate MessageControllerTest to JUnit 5 2021-04-20 13:00:29 -05:00
Chris Eager b7c56108ca Rate limit unsealed sender by E164 2021-04-20 13:00:29 -05:00
Jon Chambers 52478e7de0 Test the account crawler against a real redis cluster. 2021-04-20 13:58:27 -04:00
Chris Eager 59bbd0c43c
Add Accounts DynamoDB
* Add additional test cases to AccountsTest
* Migrate AccountsManagerTest to JUnit 5
* Add AccountsDynamoDbConfiguration
* Add Account.dynamoDbMigrationversion
* Add DynamicAccountsDynamoDbMigrationConfiguration
* Add AccountsDynamoDb to AccountsManager
* Add AccountsDynamoDbMigrator
2021-04-16 14:24:24 -05:00
Jon Chambers 4d33ba48cc Discard some tools intended to detect abusive behavior that turned out to not actually be that helpful. 2021-04-14 11:18:19 -04:00
Sanket.Ghenand@tomtom.com 14f5271c20 logger use current class 2021-04-13 11:44:59 -05:00
Sanket.Ghenand@tomtom.com 37bda0b035 remove unused imports 2021-04-13 11:44:59 -05:00
erdinc 0572951c8a Remove temporary variable 2021-04-08 12:38:06 -05:00
Andrew Bissell 1f24c913a6 add unit tests for SmsSender 2021-04-08 09:17:47 -04:00
Chris Eager ffcabe6fc4 Update metric names in RemoteConfigs 2021-04-07 16:53:06 -04:00
Jon Chambers 2cb788ceb7 Delete secure backups when deleting accounts. 2021-04-01 11:30:48 -04:00
Jon Chambers 257fef9734 Add a secure backup service client. 2021-04-01 11:30:48 -04:00
Chris Eager dea359ef91 Add metric tag for prekey target type 2021-03-31 17:59:47 -04:00
Jon Chambers 64c9648dd8 Also trust the GeoTrust Global CA cert for now. We'll remove it soon. 2021-03-31 17:59:32 -04:00
Jon Chambers 6dfd13118d Pin the trusted APNs CA cert. 2021-03-31 17:59:32 -04:00
Jon Chambers 5c23f62cec Record deletion errors in logs/metrics. 2021-03-30 17:17:32 -04:00
Jon Chambers ab4e94edab Revert "Revert "Delete data in the storage service when deleting accounts.""
This reverts commit 91fc0fd623.
2021-03-30 17:17:32 -04:00
Jon Chambers 9589b7758c Disallow generation of certificates with key IDs reserved for testing. 2021-03-30 16:25:36 -04:00
Chris Eager ad6c271f9d Add dynamic configuration for signup captcha by country code 2021-03-30 10:22:02 -05:00
Chris Eager c10d9603ad Add metric for international PreKey requests 2021-03-29 14:52:04 -05:00
Chris Eager 91bd061110 Migrate deprecated Lettuce method and enum usages 2021-03-29 14:51:26 -05:00
Chris Eager 83aa59f4dd Align messages_cluster client name 2021-03-29 14:51:26 -05:00
Ehren Kret 3745a0b81d Update from 684 to 776 for payment address length to account for signature 2021-03-27 00:45:47 -05:00
Jon Chambers 7e29ed1cc7 Block attempts to set wallet addresses from unsupported countries. 2021-03-26 12:21:11 -04:00
Jon Chambers 5965f0fd22 Add a dynamically-configured list of allowed country codes for payments. 2021-03-26 12:21:11 -04:00
Jon Chambers a816aa0186 Revert "Add a storage client method for checking wheter a user has a stored manifest."
This reverts commit 8b6012f8a8.
2021-03-22 10:32:38 -04:00
Jon Chambers a7bad20eae Revert "Add an admin command for printing a subset of account data."
This reverts commit a288b9df8e.
2021-03-22 10:32:38 -04:00
Jon Chambers 089b6b1644 Retry attempts to get messages after a delay; close connections after a finite number of retries. 2021-03-22 10:32:25 -04:00
Jon Chambers 7509520883 Make sure to release the semaphore even if something goes wrong getting messages. 2021-03-22 10:32:25 -04:00
Chris Eager e5ae0572c5 Add android-2021-03 verification format 2021-03-18 12:49:50 -05:00
Chris Eager 63dac3bd9f Migrate AccountControllerTest to JUnit 5 2021-03-18 12:45:20 -05:00
Chris Eager 19295eef46 Add dimensional metrics for account creation and verification 2021-03-18 12:45:20 -05:00
Jon Chambers 0bc1369e04 Work through the full list of supported locales when choosing a language for voice verification. 2021-03-18 13:29:09 -04:00
Jon Chambers ca2f7d2eed Parse locale strings when sending voice verification codes. 2021-03-18 13:29:09 -04:00
Jon Chambers 3ea535a412 Lower logging level for common failures and record failure metrics. 2021-03-18 12:43:31 -04:00
Jon Chambers a288b9df8e Add an admin command for printing a subset of account data. 2021-03-18 12:43:12 -04:00
Jon Chambers 8b6012f8a8 Add a storage client method for checking wheter a user has a stored manifest. 2021-03-18 12:43:12 -04:00
Chris Eager ad838b4827 Add isEnrolled(e164, experiment) for pre-registration experiments 2021-03-16 13:09:10 -05:00
Chris Eager 25f603efc9 Add DynamicPreRegistrationExperimentEnrollmentConfiguration 2021-03-16 13:09:10 -05:00
Chris Eager 152c927929 Reformat to match EditorConig 2021-03-16 13:09:10 -05:00
Chris Eager b5bd16c6a9 Migrate DynamicConfigurationTest to JUnit 5 2021-03-16 13:09:10 -05:00
Chris Eager 14bfa83bb8 Migrate ExperimentEnrollmentManagerTest to JUnit 5 2021-03-16 13:09:10 -05:00
Chris Eager 5dc8086968 Migrate Twilio numbers from static to dynamic configuration 2021-03-16 12:53:51 -05:00
Jon Chambers efe7f2e4c1 Remove vestiges of per-country sender ID logic/configuration. 2021-03-10 14:58:46 -05:00
Jon Chambers fb2fc2335a Require messaging service IDs; remove fallback-to-random-number logic. 2021-03-10 14:58:46 -05:00
Jon Chambers 345e116699 Place our trust in a Twilio message service. 2021-03-10 14:58:46 -05:00
Jon Chambers e50a1c0646 Revert "Turn off alphanumeric sender ID for all countries."
This reverts commit 3bf0188e7f.
2021-03-10 14:58:46 -05:00
Jon Chambers 3cdc58200a Copy headers from the initial websocket upgrade request into subsequent resource requests. 2021-03-10 14:54:11 -05:00
Jon Chambers 933dd81d82 Allow callers to specify a TLS version when constructing a FaultTolerantHttpClient. 2021-03-10 10:53:33 -05:00
Jon Chambers a1434524a4 Allow the storage service client to trust the Signal CA root. 2021-03-10 10:53:33 -05:00
Jon Chambers cdc6afefe2 Add an affordance for providing a custom trust store to FaultTolerantHttpClient. 2021-03-10 10:53:33 -05:00
Jon Chambers 738ec2a38e Use ForwardedIpUtil everywhere we're handling X-Forwarded-For values. 2021-03-10 10:37:10 -05:00
Jon Chambers 07886a9722 Introduce a utility class for working with forwarding chains in HTTP headers. 2021-03-10 10:37:10 -05:00
Jon Chambers fde1b49729 Introduce a set of gauges for various network statistics as reported by `nstat`. 2021-03-09 11:45:53 -05:00
Jon Chambers 58210141f4 Discard unhelpful message count histograms. 2021-03-08 16:57:26 -05:00
Jon Chambers af2a8548c3 Use Durations everywhere, drop unused constructors, and add tests. 2021-03-05 12:47:36 -05:00
brock-signal 1faedd3870 Return Retry-After time to clients when they are rate limited (#421)
* Return Retry-After time to clients when they are rate limited

* Update based on feedback

- New exception type that is mapped differently
- Always report time until allowed on rate limits
- Consume and transform into a differnt exception if we think it will be
  allowed later
2021-03-05 10:23:03 -07:00
Jon Chambers f57a4171ba Gather IP-based metrics for international, unsealed-sender messages. 2021-03-05 11:54:01 -05:00
Jon Chambers df9dc82de5 Record days since last seen when somebody's "last seen" date changes. 2021-03-01 15:31:53 -05:00
Jon Chambers eb6fe11da1
Add tools to decline messages from senders meeting specific conditions 2021-02-24 18:09:26 -05:00
Jon Chambers 0ee3f0a5b5 Fix a goof where the international unsealed sender country meter was incrementing unconditionally. 2021-02-23 16:45:47 -05:00
Jon Chambers 7dabc92447 Actually increment the international unsealed-sender counter instead of just declaring its existence. 2021-02-23 15:36:36 -05:00
Jon Chambers 5354104128 Only apply unsealed sender rate limits to targeted country codes. 2021-02-23 14:17:16 -05:00
Jon Chambers a5118e4daa Record push challenge presence/outcomes by country. 2021-02-23 13:36:37 -05:00
Ehren Kret b5ade5dc12
Only return payment address from latest profile (#408)
* Only return payment address from latest profile

* Rename `currentVersionedProfile` to `currentProfileVersion`

* Change return type to Optional

* Update service/src/main/java/org/whispersystems/textsecuregcm/controllers/ProfileController.java

Co-authored-by: Jon Chambers <63609320+jon-signal@users.noreply.github.com>

* Fix broken test

Co-authored-by: Jon Chambers <63609320+jon-signal@users.noreply.github.com>
2021-02-23 12:29:47 -06:00
Jon Chambers fff8c72f42 Record the rate of unsealed-sender messages from accounts with a non-push-capable master device. 2021-02-23 12:26:23 -05:00
Jon Chambers 06ca5f14fc Record the age of accounts that send unsealed-sender messages. 2021-02-23 12:16:45 -05:00
Jon Chambers 8c9d871268 Log different messages depending on whether rate limits are actually enforced. 2021-02-23 12:16:45 -05:00
Jon Chambers 7266eeee7a Record the rate of international unsealed-sender attempts. 2021-02-23 12:01:09 -05:00
Chris Eager f85c6bf828 Demonstrate JUnit 5 works by migrating an existing test 2021-02-19 13:30:44 -06:00
Ehren Kret 93f845610d Remove payments list from Account 2021-02-19 13:18:30 -06:00
Chris Eager aa8525385a
Temporarily support `IncomingMessage.online` (#404)
iOS versions prior to 5.5.0.7 send `online` on `IncomingMessage`, rather
than on the top-level entity. This adds a temporary server-side adaptation,
to prevent client-side issues, like persistent typing indicators.
2021-02-18 12:21:16 -06:00
Jon Chambers ec783133c1 Close websockets if anything seems fishy at init time; register close handlers early. 2021-02-18 13:20:19 -05:00
Moxie Marlinspike 71f0aab2c6 Actually start the currency managed lifecycle 2021-02-17 14:28:08 -08:00
Ehren Kret 6142998b87 Use stripToNull on payments address 2021-02-16 16:12:50 -06:00
Ehren Kret 142376f360 Add payment address to the versioned profile 2021-02-16 16:12:50 -06:00
Moxie Marlinspike ae329e735f Make quote timestamp based on ftx timestamp 2021-02-16 12:02:39 -08:00
Moxie Marlinspike 2dbab70c8c Create utility endpoint for currency conversion 2021-02-16 11:57:34 -08:00
Jon Chambers 635f669a32 Count slow queue drain events by platform. 2021-02-12 15:43:50 -05:00
Jon Chambers 5f49772ca6 Control enforcement of unsealed sender rate limits via dynamic configuration. 2021-02-12 15:43:05 -05:00
Jon Chambers 6332552346 Record the sender's country when we reject unsealed-sender messages. 2021-02-12 15:23:52 -05:00
Jon Chambers 4fb7afcf7b Drop the feature flags table. 2021-02-12 15:23:34 -05:00
Jon Chambers ff448950ed Collapse the feature flag system into the dynamic config system. 2021-02-12 15:23:34 -05:00
Brian Acton d6319aeb92 just set OWA,OWP,OWI user agents upon token deletion 2021-02-12 15:22:50 -05:00
Brian Acton 8fc6f9c442 when a user unregisters, record his user agent for posterity 2021-02-12 15:22:50 -05:00
Jon Chambers e9ea79cc8e Shorten eviction time to 7 days (to match message retention time). 2021-02-11 14:01:11 -05:00
Jon Chambers ad32555cc9 Drop the old push scheduler Redis singleton. 2021-02-11 14:01:11 -05:00
Ehren Kret be8a1acca9
Remove message database from the codebase (#395)
* Remove message database from the codebase

* Remove unused ExperimentEnrollmentManager in test

* Be more stylish
2021-02-11 10:50:03 -06:00
Jon Chambers e0ed8fa0b8
Introduce a hyper-log-log-based cardinality rate limiter 2021-02-11 10:36:26 -05:00
Jon Chambers dcbf285fae Un-ignore message controller tests. 2021-02-11 10:14:48 -05:00
Ehren Kret ceda459942 Substitute some !Optional.isPresent with Optional.isEmpty 2021-02-10 13:21:44 -06:00
Ehren Kret 28fe44aea4 Remove messages dynamo db experiment 2021-02-10 10:21:49 -06:00
Jon Chambers 71510a8199 Decommission the old directory cache. 2021-02-10 11:02:11 -05:00
Jon Chambers 9cd121c8f6 Record initial queue drain times faceted by client platform. 2021-02-09 19:21:46 -05:00
Jon Chambers 2f105ed0a4
Add support for remote client deprecation 2021-02-09 12:24:44 -05:00
Jon Chambers 704d54dd01 Record a distribution of the number of keys available when checked. 2021-02-08 16:50:27 -05:00
Jon Chambers bee9b61831 Record a distribution of initial queue depths broken down by platform. 2021-02-08 16:06:21 -05:00
Jon Chambers 9c6ce08db0 Break down message send attempts by client platform, message ephemerality, and sealed sender status. 2021-02-08 16:06:04 -05:00
Jon Chambers 6c0de89de8 Remove scary message-throttling feature flags. 2021-02-08 11:55:01 -05:00
Jon Chambers aa99e202b4
Clarify behavioral contract of the pre-key store 2021-02-08 11:45:57 -05:00
Jon Chambers 04728ea4bc Drop the old Postgres-based pre-key store. 2021-02-08 11:40:43 -05:00
Ehren Kret 6865cdfce3 Replace all org.hibernate validators with modern javax.validation 2021-02-08 09:45:33 -06:00
Ehren Kret d09b36b1d5 Fix deprecated imports for Java6Assertions and Matchers 2021-02-08 09:45:33 -06:00
Ehren Kret a5dd4f5fac Remove signaling key from Device and AccountAttributes 2021-02-08 09:45:33 -06:00
Ehren Kret 9936b2967e Don't use signaling key anymore
Signaling key was deprecated over 2 years ago. It's time for it to go.
2021-02-08 09:45:33 -06:00
Ehren Kret 0971613ac0 Remove messages table from account DB
This is quite out of date. Messages has been a separate database for a
long long time.
2021-02-08 09:45:33 -06:00
Jon Chambers f7d855c59e Drop unsealed sender rate limit logging from `info` to `debug`. 2021-02-05 11:43:37 -05:00
Jon Chambers 2fe743649d Extract common UUID operations into a utility class. 2021-02-05 10:40:27 -05:00
Jon Chambers a015237fd2 Don't request data from DynamoDB if we already have it locally. 2021-02-05 10:40:27 -05:00
Jon Chambers e1f4deaacc Add a generalized countItemsMatchingQuery method for DynamoDB stores. 2021-02-05 10:40:27 -05:00
Jon Chambers 1dceee3fa0 Make DynamoDBConfiguration more consistent. 2021-02-05 10:40:27 -05:00
Jon Chambers 3a17a7c98f Explicitly make the dynamic config worker a daemon thread. 2021-02-04 19:05:51 -05:00
Jon Chambers 3298db8683 Split traffic between the Postgres and Dynamo pre-key stores. 2021-02-04 15:18:53 -05:00
Jon Chambers d4d9403829 Add a Dynamo-backed key store. 2021-02-04 15:18:53 -05:00
Jon Chambers 426e6923ac Continue to poll for config changes after startup. 2021-02-04 13:46:37 -05:00
Jon Chambers b413f665d8 Make DynamicConfigurationManager not-really-Managed. 2021-02-04 11:33:33 -05:00
Moxie Marlinspike 5e1a572bd8
Track impact of unsealed sender rate limits (#374) 2021-02-03 08:58:47 -08:00
Ehren Kret 0dcb4b645c
Build Dynamo DB backed Message Store (#358)
* Work in progress...

* Finish first pass draft of MessagesDynamoDb

* Use begins_with everywhere for destination device id

* Remove now unused methods

* First basic test built

* Add another test case

* Remove comment

* Verify more of the message contents

* Ensure all methods are tested

* Integrate MessagesDynamoDb into the MessagesManager

This change plugs the MessagesDynamoDb class into the live serving
flow in MessagesManager.

Tests are not yet as comprehensive for this big a change as they
should be, but they now compile and pass so checkpointing here with a
commit.

* Put DynamoDB before RDBS when deleting specific messages

* Extract method

* Make aws sdk version into a property

* Rename clientBuilder

* Discard messages with no GUID

* Unify batching logic into one function

* Comment on the source of the value in this constant

* Inline method

* Variable name swizzle

* Add timers to all public methods

* Add missing return statements

* Reject messages that are too large with response code 413

* Add configuration to control dynamo DB timeouts

* Set server timestamp from the ReceiptSender

* Change to shorter key names to optimize IOPS

* Fix tests broken by changing column names

* Fix broken copyright template output

* Remove copyright template error text

* Add experiments to control use of dynamo and rds in message storage

* Specify instance profile credentials for the dynamic configuration manager

* Use property for aws sdk version

* Switch dynamo to instance profile credentials

* Add metrics to the batch write loop

* Use placeholders in logging
2021-02-03 10:03:19 -06:00
Jon Chambers 1a27c7eabc Add a (failing) test for new Ivory Coast phone numbers. 2021-02-01 21:25:14 -05:00
Jon Chambers 408b959441 Require a push challenge when registering (or else require a captcha). 2021-02-01 20:44:21 -05:00
Jon Chambers 35fc98a188 Add an experiment enrollment manager. 2021-02-01 11:08:16 -05:00
Moxie Marlinspike 92f6a79e1f
Add a dynamic configuration manager 2021-02-01 11:01:58 -05:00
Jon Chambers 8f94ed68a3 Ignore expired devices when checking for GV1->GV2 migration capability. 2021-01-30 16:55:05 -05:00
Jon Chambers ce1a4b94cb Actually store emoji/about text in the database. 2021-01-27 10:34:13 -05:00
Jon Chambers 92a0deffcf Add more robust tests for about/emoji fields. 2021-01-27 10:34:13 -05:00
Jon Chambers 97b6f6028b Fix a minor typo in the help text for a feature flag task. 2021-01-25 18:03:38 -05:00
Jon Chambers 611e8c39ee Actually drop feature flag config. 2021-01-25 15:20:06 -05:00
Jon Chambers 01f1c263a6 Add a meter for captcha requests. 2021-01-25 14:58:27 -05:00
Jon Chambers 24ea6a9f1d Revert "Temporarily disable registration abuse system"
This reverts commit 22ef058cb6.
2021-01-25 14:58:27 -05:00
Jon Chambers 46c800b8b7 Smoosh request logging tasks together rather than having one task for each direction. 2021-01-25 14:58:15 -05:00