e021286eee
Add configuration by country for sending from alpha IDs
2020-07-07 10:17:40 -05:00
0ee7a66033
Keep trying ports until you get one lower than 55535 ( #83 )
...
* Keep trying ports until you get one lower than 55535
* Rename method and change to do...while
* Limit attempts to 11,000 to find an open redis cluster port
2020-07-07 10:12:31 -05:00
42c797ee97
Set the default log level for tests to WARN.
2020-07-07 11:05:39 -04:00
b585c6676d
Move rate limiter logic to Lua scripts
2020-07-06 10:10:13 -04:00
f5ddb0f1f8
Test ClusterLuaScript against a real Redis cluster.
2020-07-02 18:58:30 -04:00
ef97f9e738
Revert "Temporarily suspend execution of the "unlock" script."
...
This reverts commit 6aecd8d44a
2020-07-02 18:58:30 -04:00
26a03b55de
Un-reinvent the clustered script execution wheel.
2020-07-02 18:58:30 -04:00
b93a16abae
Honor the step size set in the micrometer config.
2020-07-02 11:40:41 -04:00
ff2783d434
Fixed a goof where we were mirroring a write to the wrong key in the new cache cluster.
2020-07-02 11:40:27 -04:00
25a5a8db68
Set avatar to null on Account when request is false ( #78 )
2020-06-29 15:53:31 -05:00
a68d91b54c
Resolve some test flakiness by adding a deterministic "wait" mechanism. (SERVER-86)
2020-06-29 12:24:25 -04:00
88ec3a5751
Add a counter for dead letter events.
2020-06-26 09:00:11 -04:00
734dc2e37a
Don't block the Redis instance when clearing the cache.
2020-06-19 10:52:18 -04:00
6aecd8d44a
Temporarily suspend execution of the "unlock" script.
2020-06-17 22:27:02 -04:00
bbf5e1fa78
Use the UA string from websocket upgrade requests if available.
2020-06-17 15:40:18 -04:00
7454e55693
Write synchronously to the cache cluster.
2020-06-17 15:38:56 -04:00
c745fe7778
Fix a poorly-mirrored cache delete operation.
2020-06-17 15:35:46 -04:00
6adcebb247
Return to just using counters instead of timers for measuring experiment outcomes.
2020-06-17 15:34:02 -04:00
38f9b8f3dd
Make write operations in `AccountDatabaseCrawlerCache` synchronous.
2020-06-17 10:05:43 -04:00
7faf143a97
Subdivide the account database crawler cache experiment and add logging to track down lingering disagreements.
2020-06-17 09:23:40 -04:00
17cfd4924c
Fixed a poorly-mirrored write operation to the new cluster.
2020-06-16 16:46:41 -04:00
a0bebca1e6
Extend Experiment to report more detail when results don't match.
2020-06-16 16:46:41 -04:00
75cbfa2898
Mirror unlock-via-script calls to the cache cluster.
2020-06-16 16:46:41 -04:00
58a8ed1588
Add a cluster-friendly version of LuaScript.
2020-06-16 16:46:41 -04:00
e032f8df59
Add a command for clearing the cache cluster.
2020-06-16 16:46:41 -04:00
b16e37d80a
Record a histogram of incoming message list sizes.
2020-06-12 14:43:50 -04:00
c17cc07b73
Instrument BlockingThreadPoolExecutor.
2020-06-12 14:43:50 -04:00
6f767a72a7
Add a timer for the private sendMessage method.
2020-06-12 14:43:50 -04:00
11196436e9
Time rate limiter validation calls.
2020-06-12 14:43:50 -04:00
9afc433db4
Record exceptions associated with server responses.
2020-06-11 22:08:07 -04:00
f701e3d834
Record distributions of timer values; stop recording error causes.
2020-06-11 11:50:36 -04:00
4c623ca3c5
Compare Redis reads using Lettuce's synchronous path.
2020-06-11 11:50:36 -04:00
0671f05c05
Introduce experiment comparison methods for suppliers.
2020-06-11 11:50:36 -04:00
0713da7393
Record experiment results with a timer instead of a counter.
2020-06-11 11:50:36 -04:00
05955d0483
Check for null header values before trying to iterate through them.
2020-06-09 15:45:32 -04:00
28c765bd9a
Add an in-app-context test for websocket metrics.
2020-06-09 15:45:32 -04:00
8287317be7
Add account device ID to the prekey rate limiter
...
This limits prekey fetching per device on an account instead of on an
account level.
2020-06-09 10:20:10 -07:00
ec858b2d4c
Set a timeout for Redis cluster operations and shut down the cluster as part of service shutdown
2020-06-07 18:27:57 -04:00
47ece983d2
Added a Redis cluster health check.
2020-06-07 18:27:11 -04:00
52310b5dd9
Compare results of reads from old and new Redis caches.
2020-06-07 18:27:11 -04:00
c2a4a2778e
Introduce the Experiment class to compare results from parallel systems.
2020-06-07 18:27:11 -04:00
1db5977e80
Mirror username deletes unconditionally.
2020-06-07 18:27:11 -04:00
1b5dc0e434
Fixed a potential issue where locks could get out of sync between Redis instances.
2020-06-07 18:27:11 -04:00
f07f02d866
Deliver upgrade link to stale clients
2020-06-06 18:20:55 -07:00
1388103919
Mirror writes to the cache cluster.
2020-06-06 20:37:48 -04:00
fe1054d58a
Introduce a Lettuce-based fault-tolerant Redis cluster accessor.
2020-06-06 20:37:48 -04:00
ba6ac778fc
Update to Pushy v0.14.1.
2020-06-05 12:21:56 -04:00
228ffcbfce
Differentiate between websocket and "boring" HTTP traffic.
2020-05-28 12:52:49 -04:00
f18ab9e5cc
Measure traffic from websockets.
2020-05-28 12:52:49 -04:00
06c82ee87d
Celebrate the diversity of UA strings when generating tags for metrics.
2020-05-27 19:35:42 -04:00
9ba5ee8043
Move UA tag extraction into its own utility class.
2020-05-27 19:35:42 -04:00
eede4e50ca
Use hashed UUID to spread last seen updates over a full day ( #40 )
2020-05-26 13:38:52 -07:00
aa10f63d9f
Add the timestamp using the `add` method.
2020-05-22 17:39:25 -04:00
a25af36e32
Include timestamps in all server-to-client websocket messages.
2020-05-22 15:13:39 -04:00
817f057927
Inject timestamps into responses.
2020-05-22 15:13:39 -04:00
a13c44d81a
Capture request-level metrics (path, status, client platform/version).
2020-05-20 17:48:19 -04:00
45ad8f8ffb
Add the Wavefront/Micrometer reporter as a dependency and configure a registry.
2020-05-20 17:46:07 -04:00
7da9e88c0b
Add hashKey to RemoteConfig
...
This allows the percentages for different entries in remote config to
be aligned so one remote config can be a subset of another.
2020-05-13 11:08:22 -07:00
1c73c91133
Report the number of days until the CDS CA cert expires as a metric so we can set an alarm.
2020-05-12 12:57:11 -04:00
b1d11d4f69
Use APNs signing keys instead of expiring certificates.
2020-05-12 12:48:28 -04:00
001a9310c3
Support device transfers (SERVER-41, SERVER-42) ( #32 )
...
This change introduces a `transfer` device capability and account creation argument in support of the iOS device transfer effort.
2020-05-12 12:23:18 -04:00
8ffadfa1f1
Add payment addresses on account attributes update
2020-05-07 09:52:38 -07:00
50d7929e76
Drop the GCM `RECEIPT` message type (unused).
2020-05-04 17:51:54 -04:00
10840b22c5
Don't let one unregistered device block receipt for others.
2020-05-04 17:51:25 -04:00
acfbab5915
Update to Pushy v0.13.11.
2020-05-04 17:50:35 -04:00
48c324fe86
Use a static sequence of randomness in tests
...
The RemoteConfigControllerTest#testMath unit test would occassionally
fail because randomness doesn't necessarily group into expected ranges
over a finite trial count. This changes the test to use a predefined
PRNG sequence instead of one that varies with each test so that the
test will no long randomly fail.
2020-04-29 17:31:43 -07:00
0c495e7e72
Workaround lack of internal retry on transaction rollback
...
The get endpoint for key fetching can fail if the transaction cannot
complete because of simultaneous modification. Clients currently
receive 500 from this and retry if it happens, but this test case runs
into it without retrying and then complains that not all the threads
completed successfully. This workaround adds some retry attempts.
2020-04-29 17:10:13 -07:00
50ccfee201
Allow remote config to send non-boolean values
...
This version of remote config allows non-boolean values to be returned
to clients but unfortunately limits the configuration to only one
value or another. There is no way to configure more than two values
for the same key with this setup.
2020-04-29 10:51:10 -07:00
fa739c9594
Bump zkgroups to 0.7.0
2020-04-28 08:58:57 -07:00
95f0ce1816
Support for advertising payment addresses on profile
2020-04-22 12:32:53 -07:00
a32c8fabed
Temporarily move GV2 capability from allMatch to anyMatch
2020-04-20 13:42:36 -07:00
6a11501184
Bump zkgroups to 0.6.0
2020-04-20 13:41:54 -07:00
c03fd4645d
Bump zkgroups to 0.5.0
2020-04-09 20:36:34 -07:00
b76c7a4824
Update zkgroups to 0.4.2
2020-04-09 11:21:58 -07:00
1408ac77f9
Make storageCapable a boolean result rather than an auth token
2020-04-09 10:19:49 -07:00
7e97d10ae1
Fix account dropping new style registration locks
2020-04-06 09:27:23 -07:00
56b134facd
Change attachment key from long to base64 of 15 bytes
2020-04-02 10:20:42 -07:00
41286650cc
Create attachments V3 endpoint for CDN2 on GCP
...
In preparation for resumable uploads, this creates a separate
attachment authorization endpoint that creates a signed URL for
accessing GCP Storage through Signal's CDN2. This should allow Signal
clients to do byte-level resume of media uploads.
2020-04-02 10:20:42 -07:00
3c8e7c6c10
Add storage capability and return KBS creds on rereg w/ storage set
2020-03-27 10:45:48 -07:00
4f64513c83
Break out redis pubsub into dedicated cluster
2020-03-16 17:44:42 -07:00
350f5ccb3c
Account for fronted regions
2020-03-14 19:07:42 -07:00
ac1153c7cf
Additional limits
2020-03-14 18:10:07 -07:00
3b1672a4a7
Update zkgroups to 0.4.0
2020-03-14 16:30:13 -07:00
009f81a9a6
Update to dropwizard 2.x
2020-03-14 16:30:13 -07:00
8b10b1dc62
Remove tombstone column from keys table
2020-02-25 12:25:34 -08:00
077c259d5b
Migrate keys to accountsdb
2020-02-23 17:59:30 -08:00
e5746c19cf
Support for GV2 capability flag
2020-02-07 11:53:28 -08:00
e399f9e851
Generate external creds for KBS based on UUID
2020-01-22 13:47:33 -08:00
e4e20c2d25
Add support for UUID buckets in remote config
2020-01-22 11:28:08 -08:00
08a70664f4
Support for getting/setting remote config variables
2020-01-21 13:38:58 -08:00
1d76c644cb
Update version of embedded pg
2020-01-21 13:03:55 -08:00
75fc35ee4b
Parameterize access to zk operations
2020-01-21 11:29:08 -08:00
ba3102d667
Support for versioned profiles
...
Includes support for issuing zkgroup auth credentials
2020-01-21 11:04:06 -08:00
8a9fed64f2
Support for first/last profile name length
2020-01-13 18:55:04 -08:00
71c7e30548
Increase max size for sticker manifest
2019-12-19 10:29:47 -08:00
940bd55079
Update libphonenumber to 8.11.0
2019-12-18 17:32:39 -08:00
886db1a2c3
Bump max sticker count to 201
2019-12-18 17:08:51 -08:00
b4c06db031
Make redis failures on write-back retrieve non-fatal
2019-11-20 12:36:22 -08:00
82486a873a
Delete old username mapping when setting new one
2019-11-20 12:36:22 -08:00
99760ba6a0
Put UUID on server-generated delivery receipt
2019-11-20 12:36:22 -08:00
2b987e6e93
Usernames can't start with numbers
2019-11-20 12:36:22 -08:00
523134f24b
Username reservation table
2019-11-20 12:36:22 -08:00
99c228dd6d
Support for setting and looking up usernames
2019-11-20 12:36:22 -08:00
44d38a00d4
Fix capabilities NPE
2019-11-14 13:36:40 -08:00
c623f70caa
Add support for capabilities
2019-11-14 13:36:40 -08:00
f16b783378
return backup, not storage, credentials for reg lock
2019-11-05 10:36:33 -08:00
a8c932ffe4
Update dropwizard to 1.3.16
2019-10-30 19:32:40 -07:00
be4b75932b
since onCrawlChunk() is now protected, we need to invoke timeAndProcessChunk() in our unit tests
2019-10-29 18:20:03 -07:00
04d7f3a5dc
allow disabled accounts to get KBS auth
2019-10-29 16:50:47 -07:00
eddfacd0f4
add timers to the account crawler listeners
2019-10-25 21:30:48 -07:00
69742839c0
uuid-based account crawler
2019-08-27 14:42:14 -07:00
20b5f0e681
Reset cache index
2019-08-27 14:08:50 -07:00
3803b8f284
Fix for jedis pool deadlock
...
1) Remove nested pool checkouts
2) Add a max wait so it won't block forever on deadlock
2019-08-27 14:02:42 -07:00
e3daf743f2
Fix new account calculation
2019-08-27 11:14:11 -07:00
ae5da74bb1
Update banner
2019-08-26 16:08:30 -07:00
cf78047830
revert to phone number-based account crawler
2019-08-26 14:00:15 -07:00
284428a45a
Support for authentication to KBS
2019-08-26 11:09:54 -07:00
79f2efdfd9
Make UUID in sealed sender certificate optional for buggy clients
2019-08-26 11:09:54 -07:00
07822b371f
replicate uuids to contact discovery
2019-08-26 11:09:54 -07:00
7a3a385569
Support for UUID based addressing
2019-08-26 11:09:54 -07:00
e57f78cf90
Add meter for GCM challenge transmissions
2019-08-01 13:30:49 -07:00
10724fee04
Support for sticker pack uploads
2019-07-24 16:29:56 -07:00
4d09bae09b
Add some logging
2019-07-11 19:57:31 -07:00
11902dec3c
Support for v2 registration lock
2019-07-11 18:15:14 -07:00
4fdbe9b9ff
Support for push preauth
2019-07-11 18:15:10 -07:00
a6e7e30177
Add requester to recaptcha validation
2019-07-11 17:31:34 -07:00
5b69ff7e94
Break out keys database and accounts database
2019-06-19 17:16:37 -07:00
bc0c6be4c5
We don't need to support disabled accounts for the signed PK API
2019-06-12 12:32:15 -07:00
f56d219882
Update dropwizard to 1.3.12
2019-06-11 09:29:30 -07:00
3c6b418ca8
Publish fcm retry metrics
2019-05-30 11:05:05 -07:00
105a38a7db
Update gcm-sender-async to use jdk11 httpclient
2019-05-30 10:46:40 -07:00
e6f25b9c5e
Bring gcm-sender-async in as a module
2019-05-29 11:03:33 -07:00
6e0b956e61
Only set the uninstall feedback timestamp when it's zero
...
Otherwise each send will update the timestamp, preventing it from
aging out to the point where the cleaner will pick it up.
2019-05-26 14:27:30 -07:00
a029768d24
Reenable account cleaner
2019-05-10 10:42:42 -07:00
4d9c9206cf
Delay processing FCM uninstalled feedback
...
Check to make sure client is not still active before unregistering,
since FCM feedback seems to be often erroneous
2019-05-07 10:04:22 -07:00
35116f9229
Clean up concepts of enabled account state
...
1) Rename "active" methods to be "enabled," since they aren't
really about "activity."
2) Make authentication fail if a device or account is in dissabled
state.
3) Let some controllers authenticate accounts that are in a
disabled state.
2019-05-04 12:31:50 -07:00
a1f90cd39b
Temporarily disable account cleaner
2019-05-03 12:09:01 -07:00
45dc7459b8
Temporarily disable GCM unregistered feedback
2019-05-03 11:51:21 -07:00
6877b663f1
enable up to 40 account updates per chunk in AccountCleaner
2019-05-03 10:58:57 -07:00
3c69f81a10
expire accounts explicitly
2019-05-02 21:14:57 -07:00
d316d57e5d
fix DirectoryController tests
2019-05-02 19:20:23 -07:00
92eddf8eb6
Directory feedback v3
2019-05-02 15:49:27 -07:00
0c81556b90
Switch websocket-resources from ListenableFuture to CompletableFuture
2019-05-02 15:05:44 -07:00
d72828b3f4
Fix assembly for multi-module
2019-05-01 14:02:18 -07:00
9220f4d829
Add websocket-resources as a module
2019-05-01 13:19:15 -07:00
66917cd2c0
Add some dependency exclusions
2019-05-01 13:19:15 -07:00
d0d375aeb7
Break out into a multi-module project
2019-05-01 13:19:11 -07:00
Ehren Kret