ce026e7ad0
Don't send contacts to CDS if they've opted out of discoverability. (SERVER-130)
2020-08-27 15:58:02 -04:00
58e3122dab
Add a discoverableByPhoneNumber account attribute. (SERVER-129)
2020-08-27 15:58:02 -04:00
3b55b2d1b2
Actually make the "scour message cache" available to Dropwizard. Oops.
2020-08-27 15:15:04 -04:00
2326e61de5
Clear and re-create gauges to avoid "stuck" feature flag reporting.
2020-08-27 13:18:12 -04:00
32b18c9509
Add an endpoint for getting the current state of feature flags.
2020-08-27 13:18:12 -04:00
acf52ad8a3
Make feature flag manager tests use a real database to avoid over-mocking.
2020-08-27 13:18:12 -04:00
08dd493f98
Don't report exceptions as part of traffic metrics.
2020-08-27 13:17:57 -04:00
07bbe7dfb2
Return to an async model for push notification latency.
2020-08-27 10:51:44 -04:00
0aa1b80e3e
Add a command for persisting any detached messages in the old message cache.
2020-08-27 10:51:12 -04:00
5ac390281e
Add an abstract base class for Redis singleton tests.
2020-08-27 10:51:12 -04:00
ac465c5a18
Add a Lettuce-based Redis singleton client.
2020-08-27 10:51:12 -04:00
1ef3546822
Add support for server-side feature flags
2020-08-26 20:27:33 -04:00
e74ad2b555
Make RedisClusterMessagesCache a Managed class.
2020-08-25 10:58:01 -04:00
71c0056c66
Use lots of specific subscriptions instead of one monster subscription to minimize load.
2020-08-25 10:58:01 -04:00
56b27ea785
Record experiment outcomes with timers instead of counters.
2020-08-25 10:57:44 -04:00
2d75f59d33
Add support for UUID-only delivery certificates. (SERVER-132)
2020-08-20 17:05:53 -04:00
a709a3bcc0
Remove a candidate metric provider.
2020-08-20 15:40:56 -04:00
34bf5112e0
Drop TimeProvider.
2020-08-20 15:40:24 -04:00
bfe18d1d28
Re-nerf the clustered message persister.
2020-08-20 15:38:09 -04:00
6a76afc20d
Add a test to make sure the persister is respecting persist delays.
2020-08-20 15:38:09 -04:00
9c469c2f96
Base persister tests on a real Redis cluster.
2020-08-20 15:38:09 -04:00
2ab42f3dd6
Refine and expand clustered message cache metrics.
2020-08-19 11:39:05 -04:00
af34b43a8d
Reactivate the message notification experiment.
2020-08-19 11:39:05 -04:00
0f71cc7864
Rename metrics associated with cluster circuit breakers for clarity.
2020-08-18 17:59:00 -04:00
df90de3a5f
Change default Lettuce command timeout to 10s.
2020-08-18 16:21:42 -04:00
42ea7a9814
Revert Lettuce connection pooling.
2020-08-18 16:21:42 -04:00
c683cbdb2d
Time Redis operations.
2020-08-18 12:20:12 -04:00
d243b73678
Make Lettuce connection pools configurable. Double the default size.
2020-08-18 12:20:12 -04:00
dc28d063aa
Reactivate the explicit client presence experiment.
2020-08-17 11:34:27 -04:00
bb6045c1d0
Disarm the client presence manager experiment.
2020-08-15 20:23:05 -04:00
f1a74b5939
Disarm new message keyspace notifications.
2020-08-15 20:23:05 -04:00
6fb9038af1
Move to a synchronous, pooled connection model for Redis clusters.
2020-08-14 17:15:56 -04:00
27f721a1f5
Update to resilience4j 1.5.0.
2020-08-14 17:15:56 -04:00
5717dc294e
Combine the read/write breakers for Redis clusters.
2020-08-14 17:15:56 -04:00
ae0f8df11b
Break out FaultTolerantPubSubConnection as its own thing so different use cases can have their own subscription space.
2020-08-14 17:15:56 -04:00
77460ba502
Remove keyspace notification configuration checks because AWS doesn't support `CONFIG GET`.
2020-08-13 15:32:25 -04:00
f8235da4d8
Fix an issue where the queue for a thread pool was not bounded.
2020-08-13 12:46:11 -04:00
8d3316ccd6
Listen for new messages via keyspace notifications.
2020-08-13 12:17:04 -04:00
2c29f831e8
Add an explicit client presence system.
2020-08-13 10:56:26 -04:00
9457325119
Add pub/sub affordances to FaultTolerantRedisCluster.
2020-08-13 10:56:26 -04:00
189f8afcc9
Warm up the test cluster before running tests to avoid transient startup jitters.
2020-08-13 10:56:26 -04:00
9699b67510
Record the size of outgoing message lists.
2020-08-12 16:57:10 -04:00
d60633a46c
Add a meter for the number of messages we send via websocket connections.
2020-08-12 16:57:10 -04:00
0fcf28e7e7
Use the MessagesManager to actually persist messages.
2020-08-11 15:50:22 -04:00
5fad8f74b1
Factor MessagePersister into its own class.
2020-08-11 15:50:22 -04:00
e35e34d2e0
Move operation-mirroring logic to MessagesManager.
2020-08-11 15:50:22 -04:00
31a215d4d6
Use "global." instead of "g." as the prefix for global config options.
2020-08-11 11:55:35 -04:00
30948de13d
Update a metric provider dependency and remove a workaround for an upstream issue.
2020-08-11 11:02:38 -04:00
b97158bf7b
Create global remote config controllable in the signal server configuration ( #127 )
...
* Add global config controller through file rather than database
* Do no permit attempting to set or delete global config entries
2020-08-10 16:31:15 -05:00
6646be8d94
Make CpuUsageGauge a CachedGauge.
2020-08-10 12:56:37 -04:00
647a2aea64
Cache a reference to the OS management bean to avoid repeated lookups.
2020-08-10 12:56:37 -04:00
58e58ce51c
Remove a candidate metric provider.
2020-08-10 11:03:20 -04:00
4b7e48d3ec
Override default ingestion URI for SignalFx ( #131 )
2020-08-07 15:29:42 -05:00
0e074d3a5a
Copy SignalFxMeterRegistry into a new class to get better logging
2020-08-07 16:01:56 -04:00
ea00224e7f
Add support for reporting metrics to signalfx ( #129 )
2020-08-07 11:10:31 -05:00
38293efe75
Keep a running count of the number of open websockets.
2020-08-06 16:07:34 -04:00
3286c5e174
Disable Redis persistence for tests.
2020-08-06 11:22:51 -04:00
e0f8a28f38
Close connections before closing the whole cluster client.
2020-08-06 11:22:31 -04:00
bf1b00b163
Drop a spurious RedisClusterClient.
2020-08-06 11:22:31 -04:00
4fa3a136ad
Remove arbitrary SMS and add a NANPA message service ( #123 )
...
* Remove arbitrary SMS code
This code has run its course and is no longer needed for now.
* Add elements to sample config that were left out
* Add a messaging service for NANPA
* Fixup sample config capitalization
2020-08-05 13:35:11 -05:00
178a6bd66e
Log the top-level exception name and message when crawling badness happens.
2020-08-05 11:23:16 -04:00
57e1339230
Further restrict user agent pattern matching ( #120 )
...
* Further restrict user agent pattern matching
* Add static qualifier to method
2020-08-04 12:58:16 -05:00
4144423227
Publish percentiles for Micrometer distributions/timers.
2020-08-04 10:58:59 -04:00
4d03514142
Add a command for clearing the messages cache cluster.
2020-08-04 10:58:41 -04:00
0bc5566976
Mirror delete-after-persist operations to the clustered message cache.
2020-08-04 10:58:41 -04:00
925567add5
Actually "plug in" the reglock counter.
2020-08-03 15:43:33 -04:00
ad97731d46
Reduce the maximum number of versions in play to 1,000.
2020-08-03 15:42:15 -04:00
40684a93a2
Restrict user-agent version matching to a more confined space.
2020-08-03 15:42:15 -04:00
f3b644ceb8
Update the push latency manager to use UUIDs and a Redis cluster.
2020-08-03 15:36:02 -04:00
901ba6e87f
Added a push latency manager.
2020-08-03 15:36:02 -04:00
76389bd584
Clear would-be-persisted messages from the cache cluster, but don't store them to the database.
2020-07-30 19:14:39 -04:00
7bf8650d59
Un-manage FaultTolerantRedisCluster so it shuts down at JVM shutdown instead of Jetty shutdown.
2020-07-30 18:37:38 -04:00
7cb24dd96d
Add environment tag to datadog metric reporting
2020-07-30 18:04:16 -04:00
dee040318a
Add the host tag to datadog metric reporting
2020-07-30 18:04:16 -04:00
baf563e46d
Temporarily disarm the actual persisting part of the message persister.
2020-07-30 17:12:37 -04:00
e10246f10b
Use Dropwizard timers/histograms for persister metrics.
2020-07-30 14:27:06 -04:00
a9dfd88671
Start the clustered message persister at application startup.
2020-07-30 12:32:35 -04:00
beac73b6c8
Add a cluster-capable message persister
2020-07-30 11:39:14 -04:00
f9f93c77e2
Use UUIDs instead of phone numbers as account identifiers in clustered message cache
2020-07-30 11:39:14 -04:00
6fc1b4c6c0
Add a cluster-backed message cache.
2020-07-30 11:39:14 -04:00
639898ec07
Expand Experiment to deal with async suppliers and Optionals.
2020-07-30 11:39:14 -04:00
3d3790fdbc
Add binary execution methods to ClusterLuaScript.
2020-07-30 11:39:14 -04:00
69c8968cb0
Add byte-array-based methods to FaultTolerantRedisCluster.
2020-07-30 11:39:14 -04:00
aa25fc7901
Fix UsernamesManager metric/logger names.
2020-07-29 11:00:29 -04:00
4aba493ee2
Fix the key used for database crawler workers.
2020-07-29 10:58:06 -04:00
b9cfac5934
Introduce additional metric aggregators.
2020-07-28 15:11:51 -04:00
f8e97fcc32
revise 12 hour active user fudge to 8 hours for better continuity of data from a month ago
2020-07-28 11:09:41 -07:00
7f8f2641f6
Simplify registration lock counting by avoiding inactive accounts.
2020-07-28 11:48:20 -04:00
022dbb606f
Count registration lock versions when crawling the account database.
2020-07-28 11:48:20 -04:00
fea72b190d
Record message content size as a dimensioned distribution.
2020-07-28 11:47:56 -04:00
eea073f882
Decommission the old cache.
2020-07-28 10:29:28 -04:00
fc1d88f5bb
Read exclusively from the cache cluster.
2020-07-27 15:11:40 -04:00
acbe410e0b
Remove a metric aggregator.
2020-07-27 12:50:49 -04:00
89bafea61f
Move SMS strings to configuration
2020-07-27 11:23:21 -05:00
33a0c4a9ae
Use first party metric aggregator libraries where possible.
2020-07-24 17:21:56 -04:00
4cc5999f05
Configure additional metric aggregators.
2020-07-23 13:31:19 -04:00
0fbf31ec98
Clear each cluster node individually.
2020-07-22 11:12:21 -04:00
db9b7ca447
Fix slot assignment when building a cluster for tests.
2020-07-22 11:04:10 -04:00
eecc71c77f
Revert batch message storage. ( #95 )
2020-07-20 16:28:32 -04:00
5f898a9071
Measure inserted message batch size.
2020-07-20 10:30:29 -04:00
a08f21336a
Be explicit about transaction management.
2020-07-20 10:30:29 -04:00
215125de26
Update tests.
2020-07-20 10:30:29 -04:00
dfa94eac41
Store messages in batches.
2020-07-20 10:30:29 -04:00
247d869a5c
De-randomize message tests to minimize flakiness.
2020-07-14 18:46:39 -04:00
b9b6e1818f
Rename SenderIdSelector to SenderIdSupplier per code review discussion
2020-07-14 10:53:48 -05:00
a7968ccc3c
Address code review comments
2020-07-14 10:53:48 -05:00
b7e0e5a356
Create a strategy class to decide which sender id to use
...
The rules around selecting sender ids can get complicated with some
countries not supporting it and others requiring pre-registration that
may result in having a different sender id for that country than
others. This strategy class handles the logic of dealing with this
expanded configuration and applying the appropriate sender id or none
when it's not appropriate to do so at all.
2020-07-14 10:53:48 -05:00
e3aecb2aa9
apply a 12 hour fudge to daily user counting to account for last seen timestamp fuzzing
2020-07-09 17:43:12 -07:00
116ab83b95
Include a PushType header when sending APNs notifications.
2020-07-09 16:12:20 -04:00
c5d0d4acd0
Revert "Move rate limiter logic to Lua scripts"
...
This reverts commit b585c6676d
2020-07-09 12:30:25 -04:00
06190286ec
Remove temporary circuit breaker suppression.
2020-07-07 16:33:05 -04:00
3bca856e87
Remove a pair of spurious SET calls in the rate limiter script.
2020-07-07 16:33:05 -04:00
b3a778b89a
Temporarily catch and log all script execution exceptions to avoid opening the breaker.
2020-07-07 15:17:25 -04:00
dcb11f7606
Log errors from experiments.
2020-07-07 15:17:25 -04:00
933ce42d5a
Test rate limiters against a real cluster.
2020-07-07 15:17:25 -04:00
6c1ba957bd
Ensure the default alphaId configuration is an empty list rather than null
2020-07-07 10:17:40 -05:00
e021286eee
Add configuration by country for sending from alpha IDs
2020-07-07 10:17:40 -05:00
0ee7a66033
Keep trying ports until you get one lower than 55535 ( #83 )
...
* Keep trying ports until you get one lower than 55535
* Rename method and change to do...while
* Limit attempts to 11,000 to find an open redis cluster port
2020-07-07 10:12:31 -05:00
42c797ee97
Set the default log level for tests to WARN.
2020-07-07 11:05:39 -04:00
b585c6676d
Move rate limiter logic to Lua scripts
2020-07-06 10:10:13 -04:00
f5ddb0f1f8
Test ClusterLuaScript against a real Redis cluster.
2020-07-02 18:58:30 -04:00
ef97f9e738
Revert "Temporarily suspend execution of the "unlock" script."
...
This reverts commit 6aecd8d44a
2020-07-02 18:58:30 -04:00
26a03b55de
Un-reinvent the clustered script execution wheel.
2020-07-02 18:58:30 -04:00
b93a16abae
Honor the step size set in the micrometer config.
2020-07-02 11:40:41 -04:00
ff2783d434
Fixed a goof where we were mirroring a write to the wrong key in the new cache cluster.
2020-07-02 11:40:27 -04:00
25a5a8db68
Set avatar to null on Account when request is false ( #78 )
2020-06-29 15:53:31 -05:00
a68d91b54c
Resolve some test flakiness by adding a deterministic "wait" mechanism. (SERVER-86)
2020-06-29 12:24:25 -04:00
88ec3a5751
Add a counter for dead letter events.
2020-06-26 09:00:11 -04:00
734dc2e37a
Don't block the Redis instance when clearing the cache.
2020-06-19 10:52:18 -04:00
6aecd8d44a
Temporarily suspend execution of the "unlock" script.
2020-06-17 22:27:02 -04:00
bbf5e1fa78
Use the UA string from websocket upgrade requests if available.
2020-06-17 15:40:18 -04:00
7454e55693
Write synchronously to the cache cluster.
2020-06-17 15:38:56 -04:00
c745fe7778
Fix a poorly-mirrored cache delete operation.
2020-06-17 15:35:46 -04:00
6adcebb247
Return to just using counters instead of timers for measuring experiment outcomes.
2020-06-17 15:34:02 -04:00
38f9b8f3dd
Make write operations in `AccountDatabaseCrawlerCache` synchronous.
2020-06-17 10:05:43 -04:00
7faf143a97
Subdivide the account database crawler cache experiment and add logging to track down lingering disagreements.
2020-06-17 09:23:40 -04:00
17cfd4924c
Fixed a poorly-mirrored write operation to the new cluster.
2020-06-16 16:46:41 -04:00
a0bebca1e6
Extend Experiment to report more detail when results don't match.
2020-06-16 16:46:41 -04:00
75cbfa2898
Mirror unlock-via-script calls to the cache cluster.
2020-06-16 16:46:41 -04:00
58a8ed1588
Add a cluster-friendly version of LuaScript.
2020-06-16 16:46:41 -04:00
e032f8df59
Add a command for clearing the cache cluster.
2020-06-16 16:46:41 -04:00
b16e37d80a
Record a histogram of incoming message list sizes.
2020-06-12 14:43:50 -04:00
c17cc07b73
Instrument BlockingThreadPoolExecutor.
2020-06-12 14:43:50 -04:00
6f767a72a7
Add a timer for the private sendMessage method.
2020-06-12 14:43:50 -04:00
11196436e9
Time rate limiter validation calls.
2020-06-12 14:43:50 -04:00
9afc433db4
Record exceptions associated with server responses.
2020-06-11 22:08:07 -04:00
f701e3d834
Record distributions of timer values; stop recording error causes.
2020-06-11 11:50:36 -04:00
4c623ca3c5
Compare Redis reads using Lettuce's synchronous path.
2020-06-11 11:50:36 -04:00
0671f05c05
Introduce experiment comparison methods for suppliers.
2020-06-11 11:50:36 -04:00
0713da7393
Record experiment results with a timer instead of a counter.
2020-06-11 11:50:36 -04:00
Jon Chambers