Commit Graph

809 Commits

Author SHA1 Message Date
Andrew Bissell 1f24c913a6 add unit tests for SmsSender 2021-04-08 09:17:47 -04:00
Chris Eager ffcabe6fc4 Update metric names in RemoteConfigs 2021-04-07 16:53:06 -04:00
Jon Chambers 2cb788ceb7 Delete secure backups when deleting accounts. 2021-04-01 11:30:48 -04:00
Jon Chambers 257fef9734 Add a secure backup service client. 2021-04-01 11:30:48 -04:00
Chris Eager dea359ef91 Add metric tag for prekey target type 2021-03-31 17:59:47 -04:00
Jon Chambers 64c9648dd8 Also trust the GeoTrust Global CA cert for now. We'll remove it soon. 2021-03-31 17:59:32 -04:00
Jon Chambers 6dfd13118d Pin the trusted APNs CA cert. 2021-03-31 17:59:32 -04:00
Jon Chambers 5c23f62cec Record deletion errors in logs/metrics. 2021-03-30 17:17:32 -04:00
Jon Chambers ab4e94edab Revert "Revert "Delete data in the storage service when deleting accounts.""
This reverts commit 91fc0fd623.
2021-03-30 17:17:32 -04:00
Jon Chambers 9589b7758c Disallow generation of certificates with key IDs reserved for testing. 2021-03-30 16:25:36 -04:00
Chris Eager ad6c271f9d Add dynamic configuration for signup captcha by country code 2021-03-30 10:22:02 -05:00
Chris Eager c10d9603ad Add metric for international PreKey requests 2021-03-29 14:52:04 -05:00
Chris Eager 91bd061110 Migrate deprecated Lettuce method and enum usages 2021-03-29 14:51:26 -05:00
Chris Eager 83aa59f4dd Align messages_cluster client name 2021-03-29 14:51:26 -05:00
Ehren Kret 3745a0b81d Update from 684 to 776 for payment address length to account for signature 2021-03-27 00:45:47 -05:00
Jon Chambers 7e29ed1cc7 Block attempts to set wallet addresses from unsupported countries. 2021-03-26 12:21:11 -04:00
Jon Chambers 5965f0fd22 Add a dynamically-configured list of allowed country codes for payments. 2021-03-26 12:21:11 -04:00
Jon Chambers a816aa0186 Revert "Add a storage client method for checking wheter a user has a stored manifest."
This reverts commit 8b6012f8a8.
2021-03-22 10:32:38 -04:00
Jon Chambers a7bad20eae Revert "Add an admin command for printing a subset of account data."
This reverts commit a288b9df8e.
2021-03-22 10:32:38 -04:00
Jon Chambers 089b6b1644 Retry attempts to get messages after a delay; close connections after a finite number of retries. 2021-03-22 10:32:25 -04:00
Jon Chambers 7509520883 Make sure to release the semaphore even if something goes wrong getting messages. 2021-03-22 10:32:25 -04:00
Chris Eager e5ae0572c5 Add android-2021-03 verification format 2021-03-18 12:49:50 -05:00
Chris Eager 63dac3bd9f Migrate AccountControllerTest to JUnit 5 2021-03-18 12:45:20 -05:00
Chris Eager 19295eef46 Add dimensional metrics for account creation and verification 2021-03-18 12:45:20 -05:00
Jon Chambers 0bc1369e04 Work through the full list of supported locales when choosing a language for voice verification. 2021-03-18 13:29:09 -04:00
Jon Chambers ca2f7d2eed Parse locale strings when sending voice verification codes. 2021-03-18 13:29:09 -04:00
Jon Chambers 3ea535a412 Lower logging level for common failures and record failure metrics. 2021-03-18 12:43:31 -04:00
Jon Chambers a288b9df8e Add an admin command for printing a subset of account data. 2021-03-18 12:43:12 -04:00
Jon Chambers 8b6012f8a8 Add a storage client method for checking wheter a user has a stored manifest. 2021-03-18 12:43:12 -04:00
Chris Eager ad838b4827 Add isEnrolled(e164, experiment) for pre-registration experiments 2021-03-16 13:09:10 -05:00
Chris Eager 25f603efc9 Add DynamicPreRegistrationExperimentEnrollmentConfiguration 2021-03-16 13:09:10 -05:00
Chris Eager 152c927929 Reformat to match EditorConig 2021-03-16 13:09:10 -05:00
Chris Eager b5bd16c6a9 Migrate DynamicConfigurationTest to JUnit 5 2021-03-16 13:09:10 -05:00
Chris Eager 14bfa83bb8 Migrate ExperimentEnrollmentManagerTest to JUnit 5 2021-03-16 13:09:10 -05:00
Chris Eager 5dc8086968 Migrate Twilio numbers from static to dynamic configuration 2021-03-16 12:53:51 -05:00
Jon Chambers efe7f2e4c1 Remove vestiges of per-country sender ID logic/configuration. 2021-03-10 14:58:46 -05:00
Jon Chambers fb2fc2335a Require messaging service IDs; remove fallback-to-random-number logic. 2021-03-10 14:58:46 -05:00
Jon Chambers 345e116699 Place our trust in a Twilio message service. 2021-03-10 14:58:46 -05:00
Jon Chambers e50a1c0646 Revert "Turn off alphanumeric sender ID for all countries."
This reverts commit 3bf0188e7f.
2021-03-10 14:58:46 -05:00
Jon Chambers 3cdc58200a Copy headers from the initial websocket upgrade request into subsequent resource requests. 2021-03-10 14:54:11 -05:00
Jon Chambers 933dd81d82 Allow callers to specify a TLS version when constructing a FaultTolerantHttpClient. 2021-03-10 10:53:33 -05:00
Jon Chambers a1434524a4 Allow the storage service client to trust the Signal CA root. 2021-03-10 10:53:33 -05:00
Jon Chambers cdc6afefe2 Add an affordance for providing a custom trust store to FaultTolerantHttpClient. 2021-03-10 10:53:33 -05:00
Jon Chambers 738ec2a38e Use ForwardedIpUtil everywhere we're handling X-Forwarded-For values. 2021-03-10 10:37:10 -05:00
Jon Chambers 07886a9722 Introduce a utility class for working with forwarding chains in HTTP headers. 2021-03-10 10:37:10 -05:00
Jon Chambers fde1b49729 Introduce a set of gauges for various network statistics as reported by `nstat`. 2021-03-09 11:45:53 -05:00
Jon Chambers 58210141f4 Discard unhelpful message count histograms. 2021-03-08 16:57:26 -05:00
Jon Chambers af2a8548c3 Use Durations everywhere, drop unused constructors, and add tests. 2021-03-05 12:47:36 -05:00
brock-signal 1faedd3870 Return Retry-After time to clients when they are rate limited (#421)
* Return Retry-After time to clients when they are rate limited

* Update based on feedback

- New exception type that is mapped differently
- Always report time until allowed on rate limits
- Consume and transform into a differnt exception if we think it will be
  allowed later
2021-03-05 10:23:03 -07:00
Jon Chambers f57a4171ba Gather IP-based metrics for international, unsealed-sender messages. 2021-03-05 11:54:01 -05:00
Jon Chambers df9dc82de5 Record days since last seen when somebody's "last seen" date changes. 2021-03-01 15:31:53 -05:00
Jon Chambers eb6fe11da1
Add tools to decline messages from senders meeting specific conditions 2021-02-24 18:09:26 -05:00
Jon Chambers 0ee3f0a5b5 Fix a goof where the international unsealed sender country meter was incrementing unconditionally. 2021-02-23 16:45:47 -05:00
Jon Chambers 7dabc92447 Actually increment the international unsealed-sender counter instead of just declaring its existence. 2021-02-23 15:36:36 -05:00
Jon Chambers 5354104128 Only apply unsealed sender rate limits to targeted country codes. 2021-02-23 14:17:16 -05:00
Jon Chambers a5118e4daa Record push challenge presence/outcomes by country. 2021-02-23 13:36:37 -05:00
Ehren Kret b5ade5dc12
Only return payment address from latest profile (#408)
* Only return payment address from latest profile

* Rename `currentVersionedProfile` to `currentProfileVersion`

* Change return type to Optional

* Update service/src/main/java/org/whispersystems/textsecuregcm/controllers/ProfileController.java

Co-authored-by: Jon Chambers <63609320+jon-signal@users.noreply.github.com>

* Fix broken test

Co-authored-by: Jon Chambers <63609320+jon-signal@users.noreply.github.com>
2021-02-23 12:29:47 -06:00
Jon Chambers fff8c72f42 Record the rate of unsealed-sender messages from accounts with a non-push-capable master device. 2021-02-23 12:26:23 -05:00
Jon Chambers 06ca5f14fc Record the age of accounts that send unsealed-sender messages. 2021-02-23 12:16:45 -05:00
Jon Chambers 8c9d871268 Log different messages depending on whether rate limits are actually enforced. 2021-02-23 12:16:45 -05:00
Jon Chambers 7266eeee7a Record the rate of international unsealed-sender attempts. 2021-02-23 12:01:09 -05:00
Chris Eager f85c6bf828 Demonstrate JUnit 5 works by migrating an existing test 2021-02-19 13:30:44 -06:00
Ehren Kret 93f845610d Remove payments list from Account 2021-02-19 13:18:30 -06:00
Chris Eager aa8525385a
Temporarily support `IncomingMessage.online` (#404)
iOS versions prior to 5.5.0.7 send `online` on `IncomingMessage`, rather
than on the top-level entity. This adds a temporary server-side adaptation,
to prevent client-side issues, like persistent typing indicators.
2021-02-18 12:21:16 -06:00
Jon Chambers ec783133c1 Close websockets if anything seems fishy at init time; register close handlers early. 2021-02-18 13:20:19 -05:00
Moxie Marlinspike 71f0aab2c6 Actually start the currency managed lifecycle 2021-02-17 14:28:08 -08:00
Ehren Kret 6142998b87 Use stripToNull on payments address 2021-02-16 16:12:50 -06:00
Ehren Kret 142376f360 Add payment address to the versioned profile 2021-02-16 16:12:50 -06:00
Moxie Marlinspike ae329e735f Make quote timestamp based on ftx timestamp 2021-02-16 12:02:39 -08:00
Moxie Marlinspike 2dbab70c8c Create utility endpoint for currency conversion 2021-02-16 11:57:34 -08:00
Jon Chambers 635f669a32 Count slow queue drain events by platform. 2021-02-12 15:43:50 -05:00
Jon Chambers 5f49772ca6 Control enforcement of unsealed sender rate limits via dynamic configuration. 2021-02-12 15:43:05 -05:00
Jon Chambers 6332552346 Record the sender's country when we reject unsealed-sender messages. 2021-02-12 15:23:52 -05:00
Jon Chambers 4fb7afcf7b Drop the feature flags table. 2021-02-12 15:23:34 -05:00
Jon Chambers ff448950ed Collapse the feature flag system into the dynamic config system. 2021-02-12 15:23:34 -05:00
Brian Acton d6319aeb92 just set OWA,OWP,OWI user agents upon token deletion 2021-02-12 15:22:50 -05:00
Brian Acton 8fc6f9c442 when a user unregisters, record his user agent for posterity 2021-02-12 15:22:50 -05:00
Jon Chambers e9ea79cc8e Shorten eviction time to 7 days (to match message retention time). 2021-02-11 14:01:11 -05:00
Jon Chambers ad32555cc9 Drop the old push scheduler Redis singleton. 2021-02-11 14:01:11 -05:00
Ehren Kret be8a1acca9
Remove message database from the codebase (#395)
* Remove message database from the codebase

* Remove unused ExperimentEnrollmentManager in test

* Be more stylish
2021-02-11 10:50:03 -06:00
Jon Chambers e0ed8fa0b8
Introduce a hyper-log-log-based cardinality rate limiter 2021-02-11 10:36:26 -05:00
Jon Chambers dcbf285fae Un-ignore message controller tests. 2021-02-11 10:14:48 -05:00
Ehren Kret ceda459942 Substitute some !Optional.isPresent with Optional.isEmpty 2021-02-10 13:21:44 -06:00
Ehren Kret 28fe44aea4 Remove messages dynamo db experiment 2021-02-10 10:21:49 -06:00
Jon Chambers 71510a8199 Decommission the old directory cache. 2021-02-10 11:02:11 -05:00
Jon Chambers 9cd121c8f6 Record initial queue drain times faceted by client platform. 2021-02-09 19:21:46 -05:00
Jon Chambers 2f105ed0a4
Add support for remote client deprecation 2021-02-09 12:24:44 -05:00
Jon Chambers 704d54dd01 Record a distribution of the number of keys available when checked. 2021-02-08 16:50:27 -05:00
Jon Chambers bee9b61831 Record a distribution of initial queue depths broken down by platform. 2021-02-08 16:06:21 -05:00
Jon Chambers 9c6ce08db0 Break down message send attempts by client platform, message ephemerality, and sealed sender status. 2021-02-08 16:06:04 -05:00
Jon Chambers 6c0de89de8 Remove scary message-throttling feature flags. 2021-02-08 11:55:01 -05:00
Jon Chambers aa99e202b4
Clarify behavioral contract of the pre-key store 2021-02-08 11:45:57 -05:00
Jon Chambers 04728ea4bc Drop the old Postgres-based pre-key store. 2021-02-08 11:40:43 -05:00
Ehren Kret 6865cdfce3 Replace all org.hibernate validators with modern javax.validation 2021-02-08 09:45:33 -06:00
Ehren Kret d09b36b1d5 Fix deprecated imports for Java6Assertions and Matchers 2021-02-08 09:45:33 -06:00
Ehren Kret a5dd4f5fac Remove signaling key from Device and AccountAttributes 2021-02-08 09:45:33 -06:00
Ehren Kret 9936b2967e Don't use signaling key anymore
Signaling key was deprecated over 2 years ago. It's time for it to go.
2021-02-08 09:45:33 -06:00
Ehren Kret 0971613ac0 Remove messages table from account DB
This is quite out of date. Messages has been a separate database for a
long long time.
2021-02-08 09:45:33 -06:00
Jon Chambers f7d855c59e Drop unsealed sender rate limit logging from `info` to `debug`. 2021-02-05 11:43:37 -05:00
Jon Chambers 2fe743649d Extract common UUID operations into a utility class. 2021-02-05 10:40:27 -05:00
Jon Chambers a015237fd2 Don't request data from DynamoDB if we already have it locally. 2021-02-05 10:40:27 -05:00
Jon Chambers e1f4deaacc Add a generalized countItemsMatchingQuery method for DynamoDB stores. 2021-02-05 10:40:27 -05:00
Jon Chambers 1dceee3fa0 Make DynamoDBConfiguration more consistent. 2021-02-05 10:40:27 -05:00
Jon Chambers 3a17a7c98f Explicitly make the dynamic config worker a daemon thread. 2021-02-04 19:05:51 -05:00
Jon Chambers 3298db8683 Split traffic between the Postgres and Dynamo pre-key stores. 2021-02-04 15:18:53 -05:00
Jon Chambers d4d9403829 Add a Dynamo-backed key store. 2021-02-04 15:18:53 -05:00
Jon Chambers 426e6923ac Continue to poll for config changes after startup. 2021-02-04 13:46:37 -05:00
Jon Chambers b413f665d8 Make DynamicConfigurationManager not-really-Managed. 2021-02-04 11:33:33 -05:00
Moxie Marlinspike 5e1a572bd8
Track impact of unsealed sender rate limits (#374) 2021-02-03 08:58:47 -08:00
Ehren Kret 0dcb4b645c
Build Dynamo DB backed Message Store (#358)
* Work in progress...

* Finish first pass draft of MessagesDynamoDb

* Use begins_with everywhere for destination device id

* Remove now unused methods

* First basic test built

* Add another test case

* Remove comment

* Verify more of the message contents

* Ensure all methods are tested

* Integrate MessagesDynamoDb into the MessagesManager

This change plugs the MessagesDynamoDb class into the live serving
flow in MessagesManager.

Tests are not yet as comprehensive for this big a change as they
should be, but they now compile and pass so checkpointing here with a
commit.

* Put DynamoDB before RDBS when deleting specific messages

* Extract method

* Make aws sdk version into a property

* Rename clientBuilder

* Discard messages with no GUID

* Unify batching logic into one function

* Comment on the source of the value in this constant

* Inline method

* Variable name swizzle

* Add timers to all public methods

* Add missing return statements

* Reject messages that are too large with response code 413

* Add configuration to control dynamo DB timeouts

* Set server timestamp from the ReceiptSender

* Change to shorter key names to optimize IOPS

* Fix tests broken by changing column names

* Fix broken copyright template output

* Remove copyright template error text

* Add experiments to control use of dynamo and rds in message storage

* Specify instance profile credentials for the dynamic configuration manager

* Use property for aws sdk version

* Switch dynamo to instance profile credentials

* Add metrics to the batch write loop

* Use placeholders in logging
2021-02-03 10:03:19 -06:00
Jon Chambers 1a27c7eabc Add a (failing) test for new Ivory Coast phone numbers. 2021-02-01 21:25:14 -05:00
Jon Chambers 408b959441 Require a push challenge when registering (or else require a captcha). 2021-02-01 20:44:21 -05:00
Jon Chambers 35fc98a188 Add an experiment enrollment manager. 2021-02-01 11:08:16 -05:00
Moxie Marlinspike 92f6a79e1f
Add a dynamic configuration manager 2021-02-01 11:01:58 -05:00
Jon Chambers 8f94ed68a3 Ignore expired devices when checking for GV1->GV2 migration capability. 2021-01-30 16:55:05 -05:00
Jon Chambers ce1a4b94cb Actually store emoji/about text in the database. 2021-01-27 10:34:13 -05:00
Jon Chambers 92a0deffcf Add more robust tests for about/emoji fields. 2021-01-27 10:34:13 -05:00
Jon Chambers 97b6f6028b Fix a minor typo in the help text for a feature flag task. 2021-01-25 18:03:38 -05:00
Jon Chambers 611e8c39ee Actually drop feature flag config. 2021-01-25 15:20:06 -05:00
Jon Chambers 01f1c263a6 Add a meter for captcha requests. 2021-01-25 14:58:27 -05:00
Jon Chambers 24ea6a9f1d Revert "Temporarily disable registration abuse system"
This reverts commit 22ef058cb6.
2021-01-25 14:58:27 -05:00
Jon Chambers 46c800b8b7 Smoosh request logging tasks together rather than having one task for each direction. 2021-01-25 14:58:15 -05:00
Jon Chambers f10be893ce Drop the old feature flag controller. 2021-01-25 14:55:57 -05:00
Jon Chambers c606c1664f Add admin tasks for listing, setting, and deleting feature flags. 2021-01-25 14:55:57 -05:00
Jon Chambers 225932b4c9
Add emojis/"about" text to profiles 2021-01-20 15:42:47 -05:00
Jon Chambers 6b850b9894 Allow (versioned) profile names up to 380 base64 characters long. 2021-01-20 11:08:10 -05:00
Jon Chambers 943a5d1036
Shard push scheduling cache 2021-01-19 15:50:12 -05:00
Moxie Marlinspike b25da8ceaa
Don't attempt SMS to iran (#355) 2021-01-19 09:13:37 -08:00
Ehren Kret 10cdb7387d Be consistent with use of DataSize class 2021-01-18 17:01:43 -06:00
Ehren Kret dd436dd1dd Create a Meter for tracking messages larger than 256kib 2021-01-18 17:01:43 -06:00
Jon Chambers 13b84635b5
Drop an unused message database index. (#352) 2021-01-18 10:26:03 -06:00
Moxie Marlinspike 27534d408f
Log when messages cache detects topology change (#354)
Co-authored-by: Moxie Marlinspike <moxie+github@signal.org>
2021-01-17 17:13:23 -08:00
Jon Chambers 0a23ce870a Allow message persisters to be disabled by a feature flag. 2021-01-17 11:13:12 -05:00
Jon Chambers c355ef8d53 Reduce the message cache thread pool size. 2021-01-16 11:15:25 -05:00
Jon Chambers 1feb23ba99 Stop periodic topology refreshes. 2021-01-16 03:35:36 -05:00
Jon Chambers 59a0fd0799 Embiggen message cluster thread pool. 2021-01-16 02:57:04 -05:00
Jon Chambers 00b5cfcf17 Allow the client presence manager to use an entirely separate cluster. 2021-01-16 02:57:04 -05:00
Jon Chambers 9e342f253d Use the same client for inserts and reads in the message cache cluster. 2021-01-16 01:50:40 -05:00
Jon Chambers 20c48b6bb2 Expand message-related thread pools to 1 thread per shard. 2021-01-16 01:50:40 -05:00
Jon Chambers 4f9e7bb572 Separate Lettuce thread pools. 2021-01-16 01:18:05 -05:00
Jon Chambers 0a322d5a9f Add a "doomsday switch." 2021-01-15 18:05:18 -05:00
Jon Chambers 59eb6d10c1 Gate based on destination rather than random. 2021-01-15 18:05:18 -05:00
Jon Chambers a57ce1dd17 Add machinery to allow a percentage of message sends to succeed. 2021-01-15 17:05:16 -05:00
Moxie Marlinspike b100b3c36b Reject traffic without logging exceptions 2021-01-15 16:23:53 -05:00
Jon Chambers 81c1ba6eef Respond to all "message send" attempts with HTTP/503. 2021-01-15 15:34:14 -05:00
Jon Chambers 93ae4d1ee6 Move the client presence manager to its own breaker. 2021-01-15 13:51:39 -05:00
Jon Chambers 9c53d818f4 Use separate clusters for message cache read/write operations. 2021-01-15 13:51:39 -05:00
Jon Chambers e5a2c1ab10 Always return an empty list of prekeys. 2021-01-15 12:27:10 -05:00
Jon Chambers 67ed035b36 Retry serializable key transactions. 2021-01-13 17:38:29 -05:00
Jon Chambers ad30786f4a Parallelize message persisters. 2021-01-12 18:50:14 -05:00
Jon Chambers 2e01da5ec1 Add a task to enable/disable accelerated crawling. 2021-01-11 19:29:18 -05:00
Jon Chambers 8fb37a0024 Log when a crawling cycle has wrapped up. 2021-01-11 19:29:18 -05:00
Jon Chambers 9412a7424c Return HTTP/429 whenever somebody tries to get contacts from the old directory system. 2021-01-11 19:29:10 -05:00
Jon Chambers f8cbb4f386 Temporarily suspend client version metrics to reduce load on our metric aggregator. 2021-01-11 14:04:44 -05:00
Ehren Kret 86ccaa52a5
Allow configuration of multiple directory account crawler listeners (#325)
* Allow configuration of multiple directory account crawler listeners

Only one should update the local redis directory. This one is marked
with replicationPrimary true. The others in the list only serve to
issue replication requests over to CDS replication load balancers.

* Update one more metric name
2021-01-10 17:11:02 -06:00
Jon Chambers cc3e5d23e4 Enable Lettuce adaptive topology refreshes. 2021-01-10 16:20:35 -05:00
Jon Chambers cac86d1f77 Standardize toplogy event handling strategy. 2021-01-10 15:14:12 -05:00
Jon Chambers 22f7bb822f Raise log level of toplogy changes. 2021-01-10 15:14:12 -05:00
Jon Chambers 1b53f10091 Reload scripts across the whole cluster if one shard is missing the script. 2021-01-10 15:00:12 -05:00
Jon Chambers bac268a21c Don't send a reply to clients until messages are safely in a non-volatile store. 2021-01-10 13:03:40 -05:00
Jon Chambers 321e6e6679 Don't validate cluster membership (allow new shards to join dynamically). 2021-01-10 12:58:35 -05:00
Moxie Marlinspike 22ef058cb6 Temporarily disable registration abuse system 2021-01-09 15:57:55 -05:00
Jon Chambers 9ee6419bc0 Publish directory updates to multiple SQS queues. 2021-01-08 18:07:18 -05:00
Jon Chambers 3bf0188e7f Turn off alphanumeric sender ID for all countries. 2021-01-08 06:18:53 -05:00
Jon Chambers 91fc0fd623 Revert "Delete data in the storage service when deleting accounts."
This reverts commit ff1a721d5b.
2021-01-08 06:18:39 -05:00
Jon Chambers d2fcf68381 Record the status message when clients reject websocket messages. 2020-12-23 12:29:15 -05:00
Jon Chambers a4d0c17efd Record OS versions for iOS requests. 2020-12-23 11:36:31 -05:00
Jon Chambers ff1a721d5b Delete data in the storage service when deleting accounts. 2020-12-23 11:35:38 -05:00
Jon Chambers c870a1bbd5 Introduce a storage service client. 2020-12-23 11:35:38 -05:00
Ehren Kret ebf332a8c9
Record delivery duration excluding noise from non-primary devices (#311)
* Record delivery duration excluding noise from non-primary devices

* Extract method
2020-12-21 10:28:39 -06:00
Jon Chambers 85d1fff18f Actually increment the Android request counter. 2020-12-11 11:46:07 -05:00
Jon Chambers 6bb106c2cb Drop the Redis command timeout back down to 3 seconds to facilitate debug data collection. 2020-12-11 11:20:10 -05:00
Jon Chambers e551fd2c1b Revert "Pause checks for GV1 migration when checking for capability downgrades."
This reverts commit e7745db36e.
2020-12-10 17:02:41 -05:00
Jon Chambers 34a11c2338 Record OS versions for desktop and SDK versions for Android. 2020-12-10 17:02:05 -05:00
Jon Chambers 0de3a400eb Record unsuccessful server-to-client requests in more detail. 2020-12-10 17:01:46 -05:00
Jon Chambers e524ff965d Add a utility method for getting client platform tags from UA strings for metrics. 2020-12-10 17:01:46 -05:00
Jon Chambers 7ba689aaeb Measure adoption of the `gv1-migration` capability. 2020-12-09 19:08:52 -05:00
Jon Chambers 92fde83b3a Discard oversized messages bound for desktop clients via websockets. 2020-12-07 15:03:35 -05:00
Jon Chambers 3a268aef50 Reduce logging level for Lettuce connection events. 2020-12-07 11:56:41 -05:00
Jon Chambers f673bd8d7b Set device capabilities when linking a new device. 2020-12-02 13:21:08 -05:00
Ehren Kret 299b680013
Always include UUID in UD certificate (#300) 2020-12-01 08:56:55 -06:00
Jon Chambers 81e8352391 Time (and count) SQS "send message" operations. 2020-11-25 15:05:05 -05:00
Jon Chambers 1a627d6a87 Extend Redis command timeout to 3.5 seconds to avoid TCP retransmission "coincidences." 2020-11-25 15:04:06 -05:00
Ehren Kret 00a3e562dc
Force use of UCS-2 instead of GSM-7 for SMS to China (#297) 2020-11-20 14:41:48 -06:00
Jon Chambers 0628c9161c Use named threads for the JsonMetricsReporter executor service. 2020-11-18 15:46:14 -05:00
Jon Chambers 9b28672e19 Honor disabled metric attributes in JsonMetricsReporter. 2020-11-18 15:46:14 -05:00
Jon Chambers d764058a04 Measure contact intersection rate directly. 2020-11-18 14:28:53 -05:00
Jon Chambers 0aafe38496 Stop recording Lettuce latency metrics. 2020-11-17 13:20:37 -05:00
Jon Chambers e7745db36e Pause checks for GV1 migration when checking for capability downgrades. 2020-11-17 09:25:12 -05:00
Jon Chambers 474b879b16 Only notify CDS if an account attribute change actually changes an account's discoverability. 2020-11-16 10:54:12 -05:00
Jon Chambers 0a23b57ff8 Report Dropwizard metrics via the Wavefront proxy. 2020-11-13 17:14:13 -05:00
Jon Chambers 251e1b51c5 Make Micrometer batch size configurable. 2020-11-13 17:13:39 -05:00
Jon Chambers 2cc6c959a5 Revert "Temporarily suspend reporting of Lettuce latency metrics."
This reverts commit 2045153495a823b06334e7cbd86fb89c946c1cea.
2020-11-11 13:05:49 -05:00
Jon Chambers fb9aa672c9 Include the name of the calling thread when a command times out. 2020-11-11 13:05:35 -05:00
Jon Chambers 325e65db7f Expand UA parsing tests to cover OS details in desktop strings. 2020-11-11 13:05:18 -05:00
Jon Chambers 103b49ec45 Record the number of non-success responses from clients when sending messages via websockets. 2020-11-10 11:47:57 -05:00
Jon Chambers 6c78d7544f Capture a thread dump when Redis commands time out. 2020-11-10 11:47:39 -05:00
Jon Chambers 7cf50a15d0 Include client age/UA string when closing due to a spurious keepalive request. 2020-11-10 11:45:12 -05:00
Jon Chambers adbc4e9fec Record the platforms of clients that send a keepalive without a local presence. 2020-11-10 11:45:12 -05:00
Jon Chambers 4815434dd7 Record the platforms of clients that are getting displaced. 2020-11-10 11:45:12 -05:00
Jon Chambers b25e50bdae Drop API keys from Micrometer configuration. 2020-11-09 09:26:56 -05:00
Ehren Kret 604287244f Update copyright statement on all source files
IntelliJ Copyright Profile used to automate this.
2020-11-04 11:55:35 -05:00
Jon Chambers 4a4a721e90 Log timeouts in addition to incrementing a counter to make it easier to get precise timestamps. 2020-10-30 11:35:59 -04:00
Jon Chambers a4062b338e Count timeouts directly. 2020-10-29 10:51:18 -04:00
Ehren Kret 5587b7d469 Expose gv1-migration on profile endpoint 2020-10-28 13:00:57 -04:00
Ehren Kret 26870d134f Set source UUID when delivering envelopes from message cache/db on websocket 2020-10-28 12:38:32 -04:00
Jon Chambers 0431a2abb1 De-dupe connection event logging messages. 2020-10-28 12:29:14 -04:00
Ehren Kret c2db2d3cbd
Add GV1 Migration capability 2020-10-27 16:17:21 -04:00
Jon Chambers 05d9ec673e
Send push notifications if websockets close before all messages are delivered 2020-10-27 16:02:55 -04:00
Jon Chambers 1732cf9243 Add filters/tasks to enable/disable request logging. 2020-10-23 11:35:06 -04:00
Jon Chambers ab62c19de9 Temporarily suspend reporting of Lettuce latency metrics. 2020-10-23 11:30:42 -04:00
Jon Chambers 96d3a69479 Use container-managed executors for APN/GCM senders. 2020-10-23 11:30:03 -04:00
Jon Chambers 8523bb1ad8 Change the "oversized message" threshold from 64kB to 1MB. 2020-10-23 11:13:19 -04:00
Jon Chambers e6da54d9b8 Resolve build error introduced while merging. 2020-10-20 19:04:44 -04:00
Jon Chambers 0a843dc086 Tighten the "prune peers" interval; move from fixed-rate to fixed-delay scheduling. 2020-10-20 19:00:55 -04:00
Jon Chambers 7b3ed2dcbf Catch exceptions thrown while pruning missing peers. 2020-10-20 19:00:55 -04:00
Jon Chambers 42ed6c3ded Add clients to the "cleanup" list before actually setting their presence keys. 2020-10-20 19:00:55 -04:00
Jon Chambers 23ca011ac1 Record account deletion reasons. 2020-10-20 19:00:34 -04:00
Jon Chambers d82b3dc429 Record a count of deleted accounts by country. 2020-10-20 19:00:34 -04:00
Jon Chambers e391793c58 Remove now-redundant Redis execution time metrics. 2020-10-20 19:00:11 -04:00
Jon Chambers 236cef4b56 Report Lettuce command latency via Micrometer. 2020-10-20 19:00:11 -04:00
Jon Chambers 45687513bf Revert "Revert "Share resources between Lettuce clients.""
This reverts commit 334f509be599fa6a501026e900d912ff7187e150.
2020-10-20 19:00:11 -04:00
Jon Chambers 019ffdaf12 Add a command for dumping Redis command stats. 2020-10-20 18:59:44 -04:00
Jon Chambers df847431eb Measure total bytes written to websockets and failed send attempts. 2020-10-20 17:22:30 -04:00
Jon Chambers 99f488d48f Drop websocket connection names (unused for a while now). 2020-10-19 11:24:35 -04:00
Jon Chambers 05929871c9 Rename PushSender to MessageSender and add docs. 2020-10-19 11:24:35 -04:00
Jon Chambers 74b3daa70a Collapse WebsocketSender into PushSender. 2020-10-19 11:24:35 -04:00
Jon Chambers 5e30b0499a Move provisioning message-sending to its own manager class. 2020-10-19 11:24:35 -04:00
Jon Chambers 85c7347899 Add a command for dumping Redis SLOWLOG output. 2020-10-15 12:18:37 -04:00
Jon Chambers 3a84775912 Log cluster topology change events, too. 2020-10-13 16:07:08 -04:00
Jon Chambers 290a82e61c Log when Lettuce connection events happen. 2020-10-13 16:07:08 -04:00
Jon Chambers adac7d7fb2 Estimate the size of message entity lists sent via the REST API. 2020-10-13 15:49:11 -04:00
Jon Chambers 52320ebb91 Revert "Share resources between Lettuce clients."
This reverts commit eab1f503a5.
2020-10-13 12:44:54 -04:00
Jon Chambers eab1f503a5 Share resources between Lettuce clients. 2020-10-11 14:36:28 -04:00
Jon Chambers a9d0aa136d Add OS-reported metrics for cached/buffered memory. 2020-10-11 13:43:15 -04:00
Jon Chambers 691ab3080d Fix some metrics names/types. 2020-10-11 12:37:17 -04:00
Jon Chambers c5147e0c68 Report direct memory metrics. 2020-10-11 11:37:51 -04:00
Jon Chambers e9b0829860 Report the maximum number of file descriptors allowed by the OS. 2020-10-11 11:27:57 -04:00
Jon Chambers 95428ab8b0 Report GC metrics. 2020-10-11 11:08:24 -04:00
Jon Chambers 775d56fe52 Drop the "repair message queue metadata" script. 2020-10-09 18:18:30 -04:00
Jon Chambers ac2ff29288 Make sure to close scheduled reporters. 2020-10-09 18:05:00 -04:00
Jon Chambers 8e1975efe4 Record the number of deletable accounts per crawled chunk. 2020-10-08 10:51:41 -04:00
Curt Brune 39c09733d3
Add /v1/payments/auth endpoint 2020-10-08 10:51:01 -04:00
Jon Chambers e1c397993d
Require Android clients to support the gv2-3 capability 2020-10-06 16:49:49 -04:00
Jon Chambers 58ca4baf71 Time account deletion operations. 2020-10-06 11:04:47 -04:00
Jon Chambers 5245b68689 Remove temporary metrics. 2020-10-06 11:04:47 -04:00
Jon Chambers 2b6811cb1b Really delete old accounts instead of just removing their push channels. 2020-10-06 11:04:47 -04:00
Jon Chambers c82496b972 Remove the "repair queue metadata" script. 2020-10-05 16:57:16 -04:00
Jon Chambers c31348ea9a Drop the "insert messages" timeout. 2020-10-05 16:57:01 -04:00
Jon Chambers c885540749 Check that the return of ZRANGEBYSCORE isn't an empty list. 2020-10-05 10:38:40 -04:00