a5118e4daa
Record push challenge presence/outcomes by country.
2021-02-23 13:36:37 -05:00
b5ade5dc12
Only return payment address from latest profile ( #408 )
...
* Only return payment address from latest profile
* Rename `currentVersionedProfile` to `currentProfileVersion`
* Change return type to Optional
* Update service/src/main/java/org/whispersystems/textsecuregcm/controllers/ProfileController.java
Co-authored-by: Jon Chambers <63609320+jon-signal@users.noreply.github.com>
* Fix broken test
Co-authored-by: Jon Chambers <63609320+jon-signal@users.noreply.github.com>
2021-02-23 12:29:47 -06:00
fff8c72f42
Record the rate of unsealed-sender messages from accounts with a non-push-capable master device.
2021-02-23 12:26:23 -05:00
06ca5f14fc
Record the age of accounts that send unsealed-sender messages.
2021-02-23 12:16:45 -05:00
8c9d871268
Log different messages depending on whether rate limits are actually enforced.
2021-02-23 12:16:45 -05:00
7266eeee7a
Record the rate of international unsealed-sender attempts.
2021-02-23 12:01:09 -05:00
f85c6bf828
Demonstrate JUnit 5 works by migrating an existing test
2021-02-19 13:30:44 -06:00
93f845610d
Remove payments list from Account
2021-02-19 13:18:30 -06:00
aa8525385a
Temporarily support `IncomingMessage.online` ( #404 )
...
iOS versions prior to 5.5.0.7 send `online` on `IncomingMessage`, rather
than on the top-level entity. This adds a temporary server-side adaptation,
to prevent client-side issues, like persistent typing indicators.
2021-02-18 12:21:16 -06:00
ec783133c1
Close websockets if anything seems fishy at init time; register close handlers early.
2021-02-18 13:20:19 -05:00
71f0aab2c6
Actually start the currency managed lifecycle
2021-02-17 14:28:08 -08:00
6142998b87
Use stripToNull on payments address
2021-02-16 16:12:50 -06:00
142376f360
Add payment address to the versioned profile
2021-02-16 16:12:50 -06:00
ae329e735f
Make quote timestamp based on ftx timestamp
2021-02-16 12:02:39 -08:00
2dbab70c8c
Create utility endpoint for currency conversion
2021-02-16 11:57:34 -08:00
635f669a32
Count slow queue drain events by platform.
2021-02-12 15:43:50 -05:00
5f49772ca6
Control enforcement of unsealed sender rate limits via dynamic configuration.
2021-02-12 15:43:05 -05:00
6332552346
Record the sender's country when we reject unsealed-sender messages.
2021-02-12 15:23:52 -05:00
4fb7afcf7b
Drop the feature flags table.
2021-02-12 15:23:34 -05:00
ff448950ed
Collapse the feature flag system into the dynamic config system.
2021-02-12 15:23:34 -05:00
d6319aeb92
just set OWA,OWP,OWI user agents upon token deletion
2021-02-12 15:22:50 -05:00
8fc6f9c442
when a user unregisters, record his user agent for posterity
2021-02-12 15:22:50 -05:00
e9ea79cc8e
Shorten eviction time to 7 days (to match message retention time).
2021-02-11 14:01:11 -05:00
ad32555cc9
Drop the old push scheduler Redis singleton.
2021-02-11 14:01:11 -05:00
be8a1acca9
Remove message database from the codebase ( #395 )
...
* Remove message database from the codebase
* Remove unused ExperimentEnrollmentManager in test
* Be more stylish
2021-02-11 10:50:03 -06:00
e0ed8fa0b8
Introduce a hyper-log-log-based cardinality rate limiter
2021-02-11 10:36:26 -05:00
dcbf285fae
Un-ignore message controller tests.
2021-02-11 10:14:48 -05:00
ceda459942
Substitute some !Optional.isPresent with Optional.isEmpty
2021-02-10 13:21:44 -06:00
28fe44aea4
Remove messages dynamo db experiment
2021-02-10 10:21:49 -06:00
71510a8199
Decommission the old directory cache.
2021-02-10 11:02:11 -05:00
9cd121c8f6
Record initial queue drain times faceted by client platform.
2021-02-09 19:21:46 -05:00
2f105ed0a4
Add support for remote client deprecation
2021-02-09 12:24:44 -05:00
704d54dd01
Record a distribution of the number of keys available when checked.
2021-02-08 16:50:27 -05:00
bee9b61831
Record a distribution of initial queue depths broken down by platform.
2021-02-08 16:06:21 -05:00
9c6ce08db0
Break down message send attempts by client platform, message ephemerality, and sealed sender status.
2021-02-08 16:06:04 -05:00
6c0de89de8
Remove scary message-throttling feature flags.
2021-02-08 11:55:01 -05:00
aa99e202b4
Clarify behavioral contract of the pre-key store
2021-02-08 11:45:57 -05:00
04728ea4bc
Drop the old Postgres-based pre-key store.
2021-02-08 11:40:43 -05:00
6865cdfce3
Replace all org.hibernate validators with modern javax.validation
2021-02-08 09:45:33 -06:00
d09b36b1d5
Fix deprecated imports for Java6Assertions and Matchers
2021-02-08 09:45:33 -06:00
a5dd4f5fac
Remove signaling key from Device and AccountAttributes
2021-02-08 09:45:33 -06:00
9936b2967e
Don't use signaling key anymore
...
Signaling key was deprecated over 2 years ago. It's time for it to go.
2021-02-08 09:45:33 -06:00
0971613ac0
Remove messages table from account DB
...
This is quite out of date. Messages has been a separate database for a
long long time.
2021-02-08 09:45:33 -06:00
f7d855c59e
Drop unsealed sender rate limit logging from `info` to `debug`.
2021-02-05 11:43:37 -05:00
2fe743649d
Extract common UUID operations into a utility class.
2021-02-05 10:40:27 -05:00
a015237fd2
Don't request data from DynamoDB if we already have it locally.
2021-02-05 10:40:27 -05:00
e1f4deaacc
Add a generalized countItemsMatchingQuery method for DynamoDB stores.
2021-02-05 10:40:27 -05:00
1dceee3fa0
Make DynamoDBConfiguration more consistent.
2021-02-05 10:40:27 -05:00
3a17a7c98f
Explicitly make the dynamic config worker a daemon thread.
2021-02-04 19:05:51 -05:00
3298db8683
Split traffic between the Postgres and Dynamo pre-key stores.
2021-02-04 15:18:53 -05:00
d4d9403829
Add a Dynamo-backed key store.
2021-02-04 15:18:53 -05:00
426e6923ac
Continue to poll for config changes after startup.
2021-02-04 13:46:37 -05:00
b413f665d8
Make DynamicConfigurationManager not-really-Managed.
2021-02-04 11:33:33 -05:00
5e1a572bd8
Track impact of unsealed sender rate limits ( #374 )
2021-02-03 08:58:47 -08:00
0dcb4b645c
Build Dynamo DB backed Message Store ( #358 )
...
* Work in progress...
* Finish first pass draft of MessagesDynamoDb
* Use begins_with everywhere for destination device id
* Remove now unused methods
* First basic test built
* Add another test case
* Remove comment
* Verify more of the message contents
* Ensure all methods are tested
* Integrate MessagesDynamoDb into the MessagesManager
This change plugs the MessagesDynamoDb class into the live serving
flow in MessagesManager.
Tests are not yet as comprehensive for this big a change as they
should be, but they now compile and pass so checkpointing here with a
commit.
* Put DynamoDB before RDBS when deleting specific messages
* Extract method
* Make aws sdk version into a property
* Rename clientBuilder
* Discard messages with no GUID
* Unify batching logic into one function
* Comment on the source of the value in this constant
* Inline method
* Variable name swizzle
* Add timers to all public methods
* Add missing return statements
* Reject messages that are too large with response code 413
* Add configuration to control dynamo DB timeouts
* Set server timestamp from the ReceiptSender
* Change to shorter key names to optimize IOPS
* Fix tests broken by changing column names
* Fix broken copyright template output
* Remove copyright template error text
* Add experiments to control use of dynamo and rds in message storage
* Specify instance profile credentials for the dynamic configuration manager
* Use property for aws sdk version
* Switch dynamo to instance profile credentials
* Add metrics to the batch write loop
* Use placeholders in logging
2021-02-03 10:03:19 -06:00
1a27c7eabc
Add a (failing) test for new Ivory Coast phone numbers.
2021-02-01 21:25:14 -05:00
408b959441
Require a push challenge when registering (or else require a captcha).
2021-02-01 20:44:21 -05:00
35fc98a188
Add an experiment enrollment manager.
2021-02-01 11:08:16 -05:00
92f6a79e1f
Add a dynamic configuration manager
2021-02-01 11:01:58 -05:00
8f94ed68a3
Ignore expired devices when checking for GV1->GV2 migration capability.
2021-01-30 16:55:05 -05:00
ce1a4b94cb
Actually store emoji/about text in the database.
2021-01-27 10:34:13 -05:00
92a0deffcf
Add more robust tests for about/emoji fields.
2021-01-27 10:34:13 -05:00
97b6f6028b
Fix a minor typo in the help text for a feature flag task.
2021-01-25 18:03:38 -05:00
611e8c39ee
Actually drop feature flag config.
2021-01-25 15:20:06 -05:00
01f1c263a6
Add a meter for captcha requests.
2021-01-25 14:58:27 -05:00
24ea6a9f1d
Revert "Temporarily disable registration abuse system"
...
This reverts commit 22ef058cb6
2021-01-25 14:58:27 -05:00
46c800b8b7
Smoosh request logging tasks together rather than having one task for each direction.
2021-01-25 14:58:15 -05:00
f10be893ce
Drop the old feature flag controller.
2021-01-25 14:55:57 -05:00
c606c1664f
Add admin tasks for listing, setting, and deleting feature flags.
2021-01-25 14:55:57 -05:00
225932b4c9
Add emojis/"about" text to profiles
2021-01-20 15:42:47 -05:00
6b850b9894
Allow (versioned) profile names up to 380 base64 characters long.
2021-01-20 11:08:10 -05:00
943a5d1036
Shard push scheduling cache
2021-01-19 15:50:12 -05:00
b25da8ceaa
Don't attempt SMS to iran ( #355 )
2021-01-19 09:13:37 -08:00
10cdb7387d
Be consistent with use of DataSize class
2021-01-18 17:01:43 -06:00
dd436dd1dd
Create a Meter for tracking messages larger than 256kib
2021-01-18 17:01:43 -06:00
13b84635b5
Drop an unused message database index. ( #352 )
2021-01-18 10:26:03 -06:00
27534d408f
Log when messages cache detects topology change ( #354 )
...
Co-authored-by: Moxie Marlinspike <moxie+github@signal.org>
2021-01-17 17:13:23 -08:00
0a23ce870a
Allow message persisters to be disabled by a feature flag.
2021-01-17 11:13:12 -05:00
c355ef8d53
Reduce the message cache thread pool size.
2021-01-16 11:15:25 -05:00
1feb23ba99
Stop periodic topology refreshes.
2021-01-16 03:35:36 -05:00
59a0fd0799
Embiggen message cluster thread pool.
2021-01-16 02:57:04 -05:00
00b5cfcf17
Allow the client presence manager to use an entirely separate cluster.
2021-01-16 02:57:04 -05:00
9e342f253d
Use the same client for inserts and reads in the message cache cluster.
2021-01-16 01:50:40 -05:00
20c48b6bb2
Expand message-related thread pools to 1 thread per shard.
2021-01-16 01:50:40 -05:00
4f9e7bb572
Separate Lettuce thread pools.
2021-01-16 01:18:05 -05:00
0a322d5a9f
Add a "doomsday switch."
2021-01-15 18:05:18 -05:00
59eb6d10c1
Gate based on destination rather than random.
2021-01-15 18:05:18 -05:00
a57ce1dd17
Add machinery to allow a percentage of message sends to succeed.
2021-01-15 17:05:16 -05:00
b100b3c36b
Reject traffic without logging exceptions
2021-01-15 16:23:53 -05:00
81c1ba6eef
Respond to all "message send" attempts with HTTP/503.
2021-01-15 15:34:14 -05:00
93ae4d1ee6
Move the client presence manager to its own breaker.
2021-01-15 13:51:39 -05:00
9c53d818f4
Use separate clusters for message cache read/write operations.
2021-01-15 13:51:39 -05:00
e5a2c1ab10
Always return an empty list of prekeys.
2021-01-15 12:27:10 -05:00
67ed035b36
Retry serializable key transactions.
2021-01-13 17:38:29 -05:00
ad30786f4a
Parallelize message persisters.
2021-01-12 18:50:14 -05:00
2e01da5ec1
Add a task to enable/disable accelerated crawling.
2021-01-11 19:29:18 -05:00
8fb37a0024
Log when a crawling cycle has wrapped up.
2021-01-11 19:29:18 -05:00
9412a7424c
Return HTTP/429 whenever somebody tries to get contacts from the old directory system.
2021-01-11 19:29:10 -05:00
f8cbb4f386
Temporarily suspend client version metrics to reduce load on our metric aggregator.
2021-01-11 14:04:44 -05:00
86ccaa52a5
Allow configuration of multiple directory account crawler listeners ( #325 )
...
* Allow configuration of multiple directory account crawler listeners
Only one should update the local redis directory. This one is marked
with replicationPrimary true. The others in the list only serve to
issue replication requests over to CDS replication load balancers.
* Update one more metric name
2021-01-10 17:11:02 -06:00
cc3e5d23e4
Enable Lettuce adaptive topology refreshes.
2021-01-10 16:20:35 -05:00
cac86d1f77
Standardize toplogy event handling strategy.
2021-01-10 15:14:12 -05:00
22f7bb822f
Raise log level of toplogy changes.
2021-01-10 15:14:12 -05:00
1b53f10091
Reload scripts across the whole cluster if one shard is missing the script.
2021-01-10 15:00:12 -05:00
bac268a21c
Don't send a reply to clients until messages are safely in a non-volatile store.
2021-01-10 13:03:40 -05:00
321e6e6679
Don't validate cluster membership (allow new shards to join dynamically).
2021-01-10 12:58:35 -05:00
22ef058cb6
Temporarily disable registration abuse system
2021-01-09 15:57:55 -05:00
9ee6419bc0
Publish directory updates to multiple SQS queues.
2021-01-08 18:07:18 -05:00
3bf0188e7f
Turn off alphanumeric sender ID for all countries.
2021-01-08 06:18:53 -05:00
91fc0fd623
Revert "Delete data in the storage service when deleting accounts."
...
This reverts commit ff1a721d5b
2021-01-08 06:18:39 -05:00
d2fcf68381
Record the status message when clients reject websocket messages.
2020-12-23 12:29:15 -05:00
a4d0c17efd
Record OS versions for iOS requests.
2020-12-23 11:36:31 -05:00
ff1a721d5b
Delete data in the storage service when deleting accounts.
2020-12-23 11:35:38 -05:00
c870a1bbd5
Introduce a storage service client.
2020-12-23 11:35:38 -05:00
ebf332a8c9
Record delivery duration excluding noise from non-primary devices ( #311 )
...
* Record delivery duration excluding noise from non-primary devices
* Extract method
2020-12-21 10:28:39 -06:00
85d1fff18f
Actually increment the Android request counter.
2020-12-11 11:46:07 -05:00
6bb106c2cb
Drop the Redis command timeout back down to 3 seconds to facilitate debug data collection.
2020-12-11 11:20:10 -05:00
e551fd2c1b
Revert "Pause checks for GV1 migration when checking for capability downgrades."
...
This reverts commit e7745db36e
2020-12-10 17:02:41 -05:00
34a11c2338
Record OS versions for desktop and SDK versions for Android.
2020-12-10 17:02:05 -05:00
0de3a400eb
Record unsuccessful server-to-client requests in more detail.
2020-12-10 17:01:46 -05:00
e524ff965d
Add a utility method for getting client platform tags from UA strings for metrics.
2020-12-10 17:01:46 -05:00
7ba689aaeb
Measure adoption of the `gv1-migration` capability.
2020-12-09 19:08:52 -05:00
92fde83b3a
Discard oversized messages bound for desktop clients via websockets.
2020-12-07 15:03:35 -05:00
3a268aef50
Reduce logging level for Lettuce connection events.
2020-12-07 11:56:41 -05:00
f673bd8d7b
Set device capabilities when linking a new device.
2020-12-02 13:21:08 -05:00
299b680013
Always include UUID in UD certificate ( #300 )
2020-12-01 08:56:55 -06:00
81e8352391
Time (and count) SQS "send message" operations.
2020-11-25 15:05:05 -05:00
1a627d6a87
Extend Redis command timeout to 3.5 seconds to avoid TCP retransmission "coincidences."
2020-11-25 15:04:06 -05:00
00a3e562dc
Force use of UCS-2 instead of GSM-7 for SMS to China ( #297 )
2020-11-20 14:41:48 -06:00
0628c9161c
Use named threads for the JsonMetricsReporter executor service.
2020-11-18 15:46:14 -05:00
9b28672e19
Honor disabled metric attributes in JsonMetricsReporter.
2020-11-18 15:46:14 -05:00
d764058a04
Measure contact intersection rate directly.
2020-11-18 14:28:53 -05:00
0aafe38496
Stop recording Lettuce latency metrics.
2020-11-17 13:20:37 -05:00
e7745db36e
Pause checks for GV1 migration when checking for capability downgrades.
2020-11-17 09:25:12 -05:00
474b879b16
Only notify CDS if an account attribute change actually changes an account's discoverability.
2020-11-16 10:54:12 -05:00
0a23b57ff8
Report Dropwizard metrics via the Wavefront proxy.
2020-11-13 17:14:13 -05:00
251e1b51c5
Make Micrometer batch size configurable.
2020-11-13 17:13:39 -05:00
2cc6c959a5
Revert "Temporarily suspend reporting of Lettuce latency metrics."
...
This reverts commit 2045153495a823b06334e7cbd86fb89c946c1cea.
2020-11-11 13:05:49 -05:00
fb9aa672c9
Include the name of the calling thread when a command times out.
2020-11-11 13:05:35 -05:00
325e65db7f
Expand UA parsing tests to cover OS details in desktop strings.
2020-11-11 13:05:18 -05:00
103b49ec45
Record the number of non-success responses from clients when sending messages via websockets.
2020-11-10 11:47:57 -05:00
6c78d7544f
Capture a thread dump when Redis commands time out.
2020-11-10 11:47:39 -05:00
7cf50a15d0
Include client age/UA string when closing due to a spurious keepalive request.
2020-11-10 11:45:12 -05:00
adbc4e9fec
Record the platforms of clients that send a keepalive without a local presence.
2020-11-10 11:45:12 -05:00
4815434dd7
Record the platforms of clients that are getting displaced.
2020-11-10 11:45:12 -05:00
b25e50bdae
Drop API keys from Micrometer configuration.
2020-11-09 09:26:56 -05:00
604287244f
Update copyright statement on all source files
...
IntelliJ Copyright Profile used to automate this.
2020-11-04 11:55:35 -05:00
4a4a721e90
Log timeouts in addition to incrementing a counter to make it easier to get precise timestamps.
2020-10-30 11:35:59 -04:00
a4062b338e
Count timeouts directly.
2020-10-29 10:51:18 -04:00
5587b7d469
Expose gv1-migration on profile endpoint
2020-10-28 13:00:57 -04:00
26870d134f
Set source UUID when delivering envelopes from message cache/db on websocket
2020-10-28 12:38:32 -04:00
0431a2abb1
De-dupe connection event logging messages.
2020-10-28 12:29:14 -04:00
c2db2d3cbd
Add GV1 Migration capability
2020-10-27 16:17:21 -04:00
05d9ec673e
Send push notifications if websockets close before all messages are delivered
2020-10-27 16:02:55 -04:00
1732cf9243
Add filters/tasks to enable/disable request logging.
2020-10-23 11:35:06 -04:00
ab62c19de9
Temporarily suspend reporting of Lettuce latency metrics.
2020-10-23 11:30:42 -04:00
96d3a69479
Use container-managed executors for APN/GCM senders.
2020-10-23 11:30:03 -04:00
8523bb1ad8
Change the "oversized message" threshold from 64kB to 1MB.
2020-10-23 11:13:19 -04:00
e6da54d9b8
Resolve build error introduced while merging.
2020-10-20 19:04:44 -04:00
0a843dc086
Tighten the "prune peers" interval; move from fixed-rate to fixed-delay scheduling.
2020-10-20 19:00:55 -04:00
7b3ed2dcbf
Catch exceptions thrown while pruning missing peers.
2020-10-20 19:00:55 -04:00
42ed6c3ded
Add clients to the "cleanup" list before actually setting their presence keys.
2020-10-20 19:00:55 -04:00
23ca011ac1
Record account deletion reasons.
2020-10-20 19:00:34 -04:00
d82b3dc429
Record a count of deleted accounts by country.
2020-10-20 19:00:34 -04:00
e391793c58
Remove now-redundant Redis execution time metrics.
2020-10-20 19:00:11 -04:00
236cef4b56
Report Lettuce command latency via Micrometer.
2020-10-20 19:00:11 -04:00
45687513bf
Revert "Revert "Share resources between Lettuce clients.""
...
This reverts commit 334f509be599fa6a501026e900d912ff7187e150.
2020-10-20 19:00:11 -04:00
019ffdaf12
Add a command for dumping Redis command stats.
2020-10-20 18:59:44 -04:00
df847431eb
Measure total bytes written to websockets and failed send attempts.
2020-10-20 17:22:30 -04:00
99f488d48f
Drop websocket connection names (unused for a while now).
2020-10-19 11:24:35 -04:00
05929871c9
Rename PushSender to MessageSender and add docs.
2020-10-19 11:24:35 -04:00
74b3daa70a
Collapse WebsocketSender into PushSender.
2020-10-19 11:24:35 -04:00
5e30b0499a
Move provisioning message-sending to its own manager class.
2020-10-19 11:24:35 -04:00
85c7347899
Add a command for dumping Redis SLOWLOG output.
2020-10-15 12:18:37 -04:00
3a84775912
Log cluster topology change events, too.
2020-10-13 16:07:08 -04:00
290a82e61c
Log when Lettuce connection events happen.
2020-10-13 16:07:08 -04:00
adac7d7fb2
Estimate the size of message entity lists sent via the REST API.
2020-10-13 15:49:11 -04:00
52320ebb91
Revert "Share resources between Lettuce clients."
...
This reverts commit eab1f503a5
2020-10-13 12:44:54 -04:00
eab1f503a5
Share resources between Lettuce clients.
2020-10-11 14:36:28 -04:00
a9d0aa136d
Add OS-reported metrics for cached/buffered memory.
2020-10-11 13:43:15 -04:00
691ab3080d
Fix some metrics names/types.
2020-10-11 12:37:17 -04:00
c5147e0c68
Report direct memory metrics.
2020-10-11 11:37:51 -04:00
e9b0829860
Report the maximum number of file descriptors allowed by the OS.
2020-10-11 11:27:57 -04:00
95428ab8b0
Report GC metrics.
2020-10-11 11:08:24 -04:00
775d56fe52
Drop the "repair message queue metadata" script.
2020-10-09 18:18:30 -04:00
ac2ff29288
Make sure to close scheduled reporters.
2020-10-09 18:05:00 -04:00
8e1975efe4
Record the number of deletable accounts per crawled chunk.
2020-10-08 10:51:41 -04:00
39c09733d3
Add /v1/payments/auth endpoint
2020-10-08 10:51:01 -04:00
e1c397993d
Require Android clients to support the gv2-3 capability
2020-10-06 16:49:49 -04:00
58ca4baf71
Time account deletion operations.
2020-10-06 11:04:47 -04:00
5245b68689
Remove temporary metrics.
2020-10-06 11:04:47 -04:00
2b6811cb1b
Really delete old accounts instead of just removing their push channels.
2020-10-06 11:04:47 -04:00
c82496b972
Remove the "repair queue metadata" script.
2020-10-05 16:57:16 -04:00
c31348ea9a
Drop the "insert messages" timeout.
2020-10-05 16:57:01 -04:00
c885540749
Check that the return of ZRANGEBYSCORE isn't an empty list.
2020-10-05 10:38:40 -04:00
bb087caddc
Don't panic if a queue exists, but is empty when repairing metadata.
2020-10-04 16:09:56 -04:00
5e3f8b9c2e
Disallow insertion of duplicate messages.
2020-10-04 15:34:14 -04:00
1ccfe928f7
Add a test to make sure that we don't double-insert messages with the same GUID.
2020-10-04 15:34:14 -04:00
3016269268
Revert "Temporarily disable the message persisters entirely."
...
This reverts commit d464721397
2020-10-04 15:25:06 -04:00
952cfae4e6
Repair queue metadata before persisting queues.
2020-10-04 15:25:06 -04:00
df7f209ebc
Revert "Don't insert message batches in transactions."
...
This reverts commit 16eefe333f
2020-10-04 15:12:15 -04:00
d464721397
Temporarily disable the message persisters entirely.
2020-10-04 11:44:35 -04:00
551a85c1e6
Use named variables instead of referring to KEYS/ARGV array indices in message cache scripts.
2020-10-04 11:27:27 -04:00
2686761608
Instrument "get queues to persist" calls and "persist queues" exceptions.
2020-10-04 10:48:42 -04:00
02a2c3224f
Discard unused feature flag constants/mocking.
2020-10-04 10:48:42 -04:00
8ec1dda9ba
Give the persister worker thread a meaningful name.
2020-10-04 10:48:42 -04:00
0308532523
Set a query timeout of 5 seconds when inserting batches of messages.
2020-10-04 10:48:42 -04:00
10b3af2947
Revert "Insert messages individually."
...
This reverts commit 158bfe4816
2020-10-04 10:48:42 -04:00
158bfe4816
Insert messages individually.
2020-10-03 13:13:34 -04:00
16eefe333f
Don't insert message batches in transactions.
2020-10-03 11:43:42 -04:00
65e585e122
Pause only if we're running low on queues to persist.
2020-10-03 11:43:34 -04:00
2ba36ee04c
Add a gauge for worker thread liveness.
2020-10-03 11:43:34 -04:00
fc05529574
Let MessagePersister manage its own worker thread.
2020-10-03 11:43:34 -04:00
07d24f487a
Don't re-register metrics for shared circuit breakers.
2020-10-02 15:05:00 -04:00
811acdb7f5
Use separate namespaces for Redis breaker/retry metrics.
2020-10-02 10:57:05 -04:00
a7266364d1
Refactor peer pruning to be more retry-friendly.
2020-10-01 17:17:07 -04:00
e83b41dc01
Reduce default Redis cluster command timeout to 3 seconds.
2020-10-01 17:17:07 -04:00
76665dd56e
Retry Redis commands that time out.
2020-10-01 17:17:07 -04:00
2d42b478ba
Consolidate cluster and pub/sub circuit breakers.
2020-10-01 17:17:07 -04:00
885fa6beae
Add tests for Device#isEnabled.
2020-10-01 12:54:35 -04:00
65cdd5fcbe
Drop the 365-day check when deciding if an account is enabled.
2020-10-01 12:54:35 -04:00
4302e19aba
Register a UUID argument factory for the messages database.
2020-10-01 11:06:43 -04:00
0c6f05f34a
Add a (failing!) test for sending a sealed-sender message after a non-sealed-sender message.
2020-10-01 11:06:43 -04:00
8040c285cd
Include stack traces when reporting persistence issues.
2020-09-30 11:47:16 -04:00
ada454f56f
Add a meter for persisting individual messages.
2020-09-30 10:39:56 -04:00
57d2ef8740
Return queues to the "to persist" list if something goes wrong during persistence.
2020-09-30 10:39:56 -04:00
a97e0982e3
Add an integration test for message persistence.
2020-09-30 10:39:56 -04:00
eaa2060d84
Fix an incorrect locking key and some previously-suppressed lock contention issues.
2020-09-30 10:39:56 -04:00
3e02c574e7
Log exceptions when persisting messages.
2020-09-30 10:39:56 -04:00
c7230ccbb0
Remove messages from the cache in bulk.
2020-09-29 10:58:02 -04:00
fc71ced660
Persist messages in batches.
2020-09-29 10:58:02 -04:00
6041a9d094
Make exit conditions slightly more conservative.
2020-09-29 10:58:02 -04:00
599cd766e1
Let Dropwizard manage persister thread lifecycles.
2020-09-29 10:58:02 -04:00
e64c8007c0
Detect GV2 capability in non-gcm Android devices
2020-09-28 15:54:10 -04:00
9339823e84
Add temporary metrics to monitor the ratio of enabled/disabled accounts.
2020-09-28 15:33:52 -04:00
e6d4620af1
Only allow linking desktop clients if they support the third-generation GV2 capability.
2020-09-25 17:08:32 -04:00
656e6db846
Only consider desktop devices GV2-capable if they send the third-gen GV2 capability.
2020-09-25 17:08:32 -04:00
30474e3a2b
Add a test for message ordering.
2020-09-25 11:41:58 -04:00
460bd98f1b
Add metrics for messages missing GUIDs.
2020-09-25 11:41:22 -04:00
a553eba574
Add an API endpoint for deleting accounts.
2020-09-25 11:39:17 -04:00
61f515670c
Add plumbing for deleting accounts and all associated data.
2020-09-25 11:39:17 -04:00
789af0f8a6
Add support for deleting keys associated with an account.
2020-09-25 11:39:17 -04:00
86fae58c96
Add support for deleting account entities from the database.
2020-09-25 11:39:17 -04:00
c54d3abe47
Check for the second-gen GV2 capability when linking devices.
2020-09-24 19:04:02 -04:00
6fe511eb50
Fix a bad size check when loading stored messages.
2020-09-23 18:02:33 -04:00
17d18b22c7
Drop pub/sub sending logic from WebsocketSender.
2020-09-23 14:51:02 -04:00
66a04ed730
Don't explicitly notify clients when messages get persisted.
2020-09-23 14:51:02 -04:00
7e14a0bc30
Drop pub/sub operations from WebsocketConnection.
2020-09-23 14:51:02 -04:00
77de0f86dc
Require desktop clients to send the new gv2-2 capability flag.
2020-09-23 12:05:58 -04:00
3b4bc9163a
Untangle thread pool names, tweak sizes, and add instrumentation.
2020-09-22 10:21:33 -04:00
Jon Chambers